Push into inference computing
At its annual GTC developer conference in San Jose, California, Nvidia introduced a new central processor and an AI system based on technology from startup Groq. This move forms part of chief executive Jensen Huang’s effort to strengthen the company’s role in inference computing.
Inference involves AI systems answering queries or performing tasks using previously learned data. However, this segment is becoming more competitive. Companies such as Meta are building custom processors, which challenge Nvidia’s dominance in chips used for training AI models.
Huang stated that the “inference inflection has arrived,” signalling a shift in how AI workloads will develop. Meanwhile, the company aims to secure a leading role in this next phase of AI adoption.
Market reaction and growth outlook
The announcement came during a four day conference held in a large arena with capacity exceeding 18,000 attendees. The event has grown into a major showcase for AI innovation.
Following the forecast, Nvidia shares briefly rose before closing up 1.6 percent. Investors reacted to the ambitious $1 trillion opportunity, although Huang did not provide detailed assumptions behind the estimate.
Previously, Nvidia had reiterated a revenue opportunity of around $500 billion for 2026. Therefore, the new projection represents a significant upward revision in expectations for AI driven growth.
New chip roles and system design
Huang explained that inference workloads will split into two distinct steps. First, Nvidia’s Vera Rubin chips will manage the “prefill” stage. In this step, user input converts from human language into tokens that AI systems process.
Next, Groq’s chips will handle the “decode” stage. This phase generates the final response for the user. As a result, the combined system aims to improve both speed and efficiency in real time AI operations.
Groq, a chip startup, specialises in fast and cost effective inference processing. Nvidia licensed its technology for $17 billion in December. This partnership highlights Nvidia’s intent to strengthen its capabilities in inference, where performance and cost remain critical factors.
With inputs from Reuters