Nvidia Calls RTX the "Premium" AI PC Platform, NPUs Only Good for "Basic" AI PCs
NVIDIA is creating a new ecosystem of "Premium" AI PCs powered by its RTX hardware which are much faster than AI-capable NPUs that are taking the industry by storm.
NVIDIA's RTX Platform Is The "Premium" Choice of AI PCs & All RTX Owners Can Run GenAI & AI-Powered Workloads Way Faster Than NPUs
NVIDIA is developing a suite of new ecosystems for the consumer market. We have already talked about the new "SFF Enthusiast GeForce GPU" platform which will empower manufacturers to make new compact and small form factor PCs powered by RTX GPUs but that's not all. In a meeting with manufacturers and media, NVIDIA highlighted its contributions to the AI PC segment and how it is planning to take the next leap forward with new ambitious ideas.
Starting with the most basic premise, we have the NVIDIA GeForce RTX GPU. The current GeForce RTX lineup and the ones that came before them have been featuring AI support since 2018, that's almost 6 years ago when the company first introduced its Deep Learning model for gamers known today as DLSS. DLSS is widely in use and the driving force behind DLSS is the Tensor architecture which was adopted first for the data center "Volta" GPUs and launched in the mainstream graphics segment with the arrival of the GeForce RTX 20 "Turing" family.
As of right now, the most basic RTX GPUs, the RTX 2050 and RTX 3050, offer around 15-20 TOPS of AI performance which matches or exceeds the 16 TOPs offered in the fastest NPU, the XDNA 1 from AMD for its Ryzen 8040 "Hawk Point" APUs. These NPUs provide consumers the basic compute capabilities to handle AI tasks locally rather than running them in the cloud which is now considered the "old way" and only useful for large-scale enterprises. NVIDIA has some really fast GPUs such as its recently introduced Blackwell lineup for that too.
This year, multiple PC vendors will be upgrading their AI PC portfolio with faster NPU capabilities, up to 45-50 TOPS (Lunar Lake, Arrow Lake, Strix Point, Snapdragon X). These NPUs are designed to fulfill the requirements of Microsoft CoPilot AI and similar AI-based workloads but while these chipmakers try to hit 50 TOPS, NVIDIA's GPUs are already offering over 100s of TOPS, in some cases, even exceeding 1000 TOPS with the fastest GPUs on the planet.
As such, NVIDIA wants to create a new segment of AI PCs and refer to it as "Premium AI PC". These AI PCs will scale from 100-1300+ TOPS (INT8/FP8), and offer vastly better performance in multiple work loads such as content creation, video capabilities, productivity, gaming, and developer tasks. The NPU can only do so much and even then, chipmakers rely on the total TOPS figure, combining the onboard CPU, GPU, and NPU capabilities while NVIDIA's RTX GPUs can deliver dedicated tensor core performance to users as they will. Furthermore, we recently discussed how NPUs taking priority in these chips while CPU and GPU resources might be nerfed.
The other factor where NVIDIA's bets might pay off with the "Premium AI PC" segment is the AI ecosystem. NVIDIA has a far bigger suite of AI-ready tools that complement their RTX GPUs. The company recently revealed its own Chat With RTX (AI Chatbox) and has included optimizations for a wide range of GenAI and LLM (Large Language Models) in the form of TensorRT which helps boost AI performance by leaps and bounds. We have to mention DLSS again as it is the premiere AI tech for gamers and unlike anything that had been done before.
2 of 9
DLSS 3.5 and DLSS 3.7 are the latest versions of the tech and are not only helping gamers run faster with higher visual quality which can often provide image quality better than the native resolution but also allowing modders to bring new life to older games using RTX Remix, a platform which has become widely popular and has seen several classics being given a modern graphical touch.
There's no doubt that NVIDIA has been ahead of everyone in the AI department, whether that be the enterprise segment or the consumer segment. While others have started to realize the potential of AI just now and fast-forwarding their NPU roadmaps, NVIDIA has silently amassed a huge range of RTX-capable GPUs, a large list of AI-enabled app (500+), and currently has an installed userbase of over 100 million RTX owners who fall in this "Premium AI PC" segment (as of CY2023). This new "Premium AI PC" eco-system will further create a disparity between NPUs and RTX GPUs as there already is between the iGPU and dGPU segments.
So to put everything into perspective, users who are involved in lots of AI tasks on a daily basis will mostly use their GPU and NVIDIA's RTX hardware is equipped with the horsepower to do so and in a much faster and better way than NPUs.
It will be interesting to see how things evolve in the future and we can't wait for more exciting and useful developments in the AI PC segment which not only focus on adding more AI TOPS but also give consumers and gamers a reason to invest in such hardware. NVIDIA's ACE is another technology that I am personally looking forward to and should be a game changer if implemented properly.
What are your thoughts on localized AI tools for the PC platform?
News Source: Benchlife