Internal documents reveal Amazon’s custom AI chips, Trainium and Inferentia, lag Nvidia’s GPUs in performance and cost, posing a significant challenge to AWS’s cloud strategy. Major AI startups and key partnerships are increasingly opting for Nvidia, threatening AWS’s profit margins and market position in the high-stakes AI arena.
In the high-stakes arena of artificial intelligence, where technological supremacy translates directly into market dominance and staggering profits, Amazon Web Services (AWS) finds itself in an unexpected battle.
For years, the cloud giant built its empire on a strategy of vertical integration, crafting its own data-center chips to optimize costs and performance.
This playbook, a cornerstone of AWS’s legendary profitability, is now being tested in the crucible of generative AI, where Nvidia’s graphics processing units (GPUs) reign supreme.
And according to internal documents, Amazon’s homegrown AI chips are struggling to keep pace.
A confidential Amazon document from July, recently brought to light, paints a revealing picture of the challenges facing AWS’s custom AI chips, Trainium and Inferentia.
Startups, often the bellwethers of technological adoption, have voiced significant concerns.
Cohere, a prominent AI startup, reported that both Trainium 1 and 2 chips were “underperforming” Nvidia’s formidable H100 GPUs.
Beyond performance, Cohere also cited “extremely limited” access to Trainium 2 and persistent service disruptions – issues that Amazon’s chip group, Annapurna Labs, was investigating with “limited” progress.
Similar sentiment echoed from Stability AI, a well-known image-generation firm.
Their assessment found Trainium 2 chips lagging behind Nvidia’s H100s in latency, ultimately making them “less competitive” on both speed and cost.
These are not minor technical glitches; they strike at the very heart of Amazon’s ambition to offer a cost-effective alternative to Nvidia’s increasingly expensive hardware.
The stakes for Amazon could not be higher.
If AWS customers, particularly the burgeoning AI startups, continue to demand Nvidia’s GPUs for their intensive AI workloads, it could significantly erode Amazon’s future cloud profits.
The company’s historical success in cloud infrastructure was predicated on avoiding costly third-party components.
In this new era, paying top dollar for Nvidia’s silicon threatens to undermine that core strategic advantage, leaving AWS stuck with higher operational expenses.
The complaints extend beyond just the latest Trainium models.
Typhoon, another startup, found Nvidia’s older A100 GPUs to be up to three times more cost-efficient than AWS’s Inferentia 2 chips for certain tasks.
AI Singapore similarly concluded that AWS’s G6 servers, equipped with Nvidia GPUs, offered superior cost performance compared to Inferentia 2 across various use cases.
These findings underscore a broader pattern of “challenges adopting” Amazon’s custom AI chips, creating “friction points” and contributing to low usage, as Business Insider previously reported.
The market share reflects this struggle: Nvidia commands over 78% of the AI chip market, while AWS’s chips languish in sixth place with a mere 2%.
Perhaps the most telling sign of Amazon’s uphill battle came with the recent $38 billion partnership between AWS and OpenAI.
Despite Amazon’s deep investment in its own chips, the deal reportedly involves AI cloud servers exclusively containing Nvidia GPUs, with no mention of Trainium processors.
Mizuho analysts deemed this omission “disappointing,” noting it was “logical” for OpenAI to opt for Nvidia, given their superior performance and the widely adopted CUDA platform – a critical ecosystem advantage that provides developers with familiarity and reliability for high-risk AI projects.
Indeed, Amazon employees themselves have acknowledged that technical limitations and comparative issues between their custom chips and Nvidia’s GPUs have become “critical blockers” for potential customers.
Amazon, for its part, acknowledges the feedback, with a spokesperson stating the company is “grateful” for customer insights that help improve its chips.
While dismissing the Cohere case as “not current,” AWS maintains that Trainium and Inferentia have achieved “great results” with customers like Ricoh, Datadog, and Metagenomi.
The company also claims its in-house chips offer 30% to 40% better price performance than current-generation GPUs, and CEO Andy Jassy highlighted that Trainium 2 chips are “fully subscribed” and now represent a “multibillion-dollar” business, primarily with a “small number of very large customers like Anthropic.”
Looking ahead, Amazon expects to accommodate more customers with Trainium 3, previewing later this year.
Anthropic, the AI startup behind the Claude models, stands as Trainium’s most high-profile customer.
AWS has even launched Project Rainier, a massive data center project involving a cluster of half a million Trainium chips dedicated exclusively to training Anthropic’s next-generation AI model, with plans to deploy over 1 million Trainium 2 chips by year-end.
This partnership is a crucial validation point for Amazon.
Yet, even this relationship isn’t without its complexities.
Anthropic recently broadened its partnership with Google, which offers its own in-house AI chips (TPUs), causing a dip in Amazon’s stock.
Anthropic has also publicly detailed outages related to the complexity of using multiple chip architectures.
Jassy’s public stance is one of offering choice, not outright replacement.
He articulated AWS’s strategy of providing “multiple chip options,” a consistent approach across its cloud services.
In the history of AWS, it’s never just one player that over a long period of time has the entire market segment and then it can satisfy everybody’s needs on every dimension,” Jassy noted during a recent earnings call.
While AWS reported a robust 20% revenue growth to $33 billion last quarter, it still lags behind rivals like Microsoft and Google Cloud in percentage growth.
The internal documents and market realities paint a picture of a tech titan grappling with a formidable competitor in a critical emerging market.
Amazon’s engineering prowess is undeniable, and its strategic imperative to control its own hardware is strong.
But overcoming Nvidia’s technological lead, entrenched developer ecosystem, and sheer market momentum will require more than just ambition; it will demand a significant leap in performance, availability, and developer appeal.
The battle for the foundational infrastructure of the AI era is far from decided, but for now, Amazon’s custom chips remain a hopeful challenger in a market dominated by a reigning champion.
The expansion introduces agentic decision flows to bolster investigative capabilities for a national security agency in the EMEA region.
Zoho founder Sridhar Vembu warns that companies must prioritize data sovereignty by building internal AI models to protect sensitive intellectual property.
Cities are deploying artificial intelligence on sanitation vehicles to monitor private property, raising complex questions about the future of urban privacy and municipal oversight.
