Responsible AI in Healthcare

In the buzzing heart of Silicon Valley, the mantra often echoed is “fail fast, move forward.”

It’s a philosophy that fuels rapid innovation, allowing new technologies to stumble, learn, and iterate at breakneck speeds.

But for Archie Mayani, Chief Product Officer at GHX, a global supply chain company specializing in healthcare, this ethos carries a starkly different weight.

Here, the stakes are not merely about a minor glitch or a humorous anecdote; they are, quite literally, life and death.

Imagine an AI-powered dating app that “hallucinates,” perhaps suggesting a wildly inappropriate match.

It might make for an amusing first-date story.

Now, picture an operating room where an AI system fails to anticipate a critical shortage, and the right surgical implant isn’t delivered at the precise moment it’s needed.

The humor evaporates, replaced by a chilling reality that underscores the profound ethical chasm separating AI applications in consumer tech from those embedded in healthcare.

Mayani, whose career spans over two decades in clinical and supply-chain health technologies at giants like Change Healthcare and United Health Group, navigates this delicate terrain daily at GHX.

Her mission is to ensure that the “invisible operating system” of healthcare supply chains—the intricate network that delivers everything from IV fluids to life-saving implants—functions with seamless precision.

GHX leverages data and cloud-based technologies to connect hospitals and providers with their suppliers, striving to make this often-unseen infrastructure as resilient and proactive as possible.

The goal of AI implementation in this critical sector, Mayani explains, is fundamentally patient-centric. It’s about more than just efficiency; it’s about responsible technology use, ensuring patient safety, improving the quality of care, and ultimately, making healthcare more affordable. This isn’t a new frontier for GHX; the company has been harnessing AI and machine learning for a decade and a half. Their work gained particular urgency during the pandemic, when supply chain vulnerabilities were laid bare for the world to see.

The focus then shifted to making disruptions visible and building resilience.

One of the most impactful applications born from this urgency was the anticipation of backorders. Whether the cause was a geopolitical conflict, a natural disaster, or a logistical mishap on a freeway, the ability to foresee a disruption and recommend immediate, local substitutes became a game-changer. An intelligent system that could flag a potential shortage of a vital medication and suggest an available alternative within a hospital’s distributed network moved beyond mere prediction to proactive problem-solving.

Yet, even with these advancements, the journey was far from complete. GHX employs an agile development approach, placing customer feedback at the heart of its innovation cycle. This iterative process led to a pivotal “aha” moment. While hospitals appreciated the early warnings of disruptions, they pointed out a crucial distinction: “The disruption of a Band-Aid is not the same as a disruption of IV fluid.” This feedback illuminated the need for a more nuanced, intelligent system. The response was the development of “clinical sensitivity” and “confidence scores.” This innovation allowed the AI to not only predict a disruption but also to assess its clinical relevance and criticality for a specific organization. What might be a minor inconvenience for one facility could be a catastrophic shortage for another, depending on their patient population and care priorities. This personalization transformed the trajectory of GHX’s AI roadmap, proving that insights, however brilliant, are only useful if they are both predictive and tailored to the unique needs of the end-user. It’s a powerful lesson in the practical application of technology: sometimes, the most sophisticated algorithms need a healthy dose of human context to truly shine.

Looking ahead, Mayani envisions a future where AI automates workflows with increasing sophistication, but always with a “human in the loop.” The aim is to build confidence incrementally, gradually abstracting tasks to AI agents as users become comfortable with their reliability. This cautious approach is a direct acknowledgment of healthcare’s unique demands.

Another area gaining significant traction is the development of “copilot environments.” Mayani cites the example of GHX’s “perfect order dashboard,” which previously presented a wealth of data on supplier performance, order accuracy, and payment timeliness. While informative, extracting actionable insights could be a time-consuming endeavor. Now, with a ChatGPT-like interface, users can simply ask, “Show me the top three defaulting suppliers not delivering supplies on time.” Once identified, the copilot can even draft an email to schedule a quarterly business review, attaching the relevant performance data. What once took hours of data analysis and follow-up now takes mere minutes, freeing up critical human resources for more complex, patient-facing tasks.

Mayani’s advice to aspiring leaders in her field is both simple and profound: learn to say no. In healthcare, everything often feels urgent, and much of it genuinely is. But not everything matters equally. The ability to discern high-value initiatives from those that, while pressing, don’t align with the core mission of patient safety and quality care, is paramount. Unlike a startup that can afford to “fail fast,” healthcare technology operates without that luxury. Innovations must be meticulously planned, responsibly executed, and always underpinned by robust data governance, security, and privacy protocols.

Ultimately, Mayani’s work at GHX is a testament to the profound potential of AI when applied with responsibility, empathy, and a clear understanding of its human impact. It’s not about building the “sexiest” or “coolest” technology, but about focusing on what truly matters: making care more affordable and of the highest possible quality for every patient. In a world increasingly shaped by algorithms, this patient-first philosophy is not just a strategic choice; it’s a moral imperative.