What’s Actually Happening Behind the Scenes

Insurers are increasingly using automated tools for allocating care to predict the length of stays, flag “low value services, and batch-screen claims against diagnoses codes. In practice, this means coverage decisions shift from your doctor’s clinical judgment to an opaque algorithmic score that patients will never see.  

Two high-profile cases show how this scenario usually plays out. In Minnesota, families filed a lawsuit against UnitedHealth over allegations that its nH Predict tool systematically cut off post-acute rehab for Medicare Advantage patients—claims that federal court originally allowed to proceed through February of 2025 before rejecting the company’s bid to limit discovery last week. Meanwhile in California, the court escalated a class action lawsuit against Cigna’s PxDx system for allegedly enabling batch denials without reviewing patient records. The tools may be different, but the pattern is clear: algorithms speed up denials, while patients are left to deal with the same slow, manual appeals process. 

In response to this growing issue, regulators are pushing back. In February 2024, CMS drew a clear line stating that Medicare Advantage plans can use algorithmic tools to assist with coverage decisions, but licensed clinicians will have the final say in determining medical-necessity. Put simply, this means that software advises, but doctors decide. California took this a step further with SB 1120, which kicked in January 1, 2025, mandating human oversight and transparency whenever insurers use AI to influence coverage decisions. Recently, the state even doubled down in May with detailed compliance guidance.  

However, the inclusion of AI in medical decisions is expanding beyond Medicare Advantage. Starting January 1, 2026, CMS will roll out its Wasteful and Inappropriate Service Reduction (WISeR) model in six states, leveraging AI and Machine Learning to “ensure timely and appropriate Medicare payment for select items and services.” While the pilot does include mandatory clinician review, this hybrid approach could go either way—reducing administrative burden or perpetuating the same vague decision-making process that plagued Medicare Advantage. The success of this new iteration is entirely dependent on its implementation. In this case, responsible deployment would involve transparent disclosure when AI is used, detailed audit logs, and genuine recourse when algorithms deny legitimate care. 

As these tools spread from Medicare to Advantage to Original Medicare and across insurance markets, their impact follows a familiar pattern of inequality and there’s a clear trend of where they’re deployed. They cluster in areas with narrow provider networks, among patients whose caregivers lack the bandwidth or language skills to appeal, and in communities cut off by digital divides. Efficiency is the sales pitch of AI, but vulnerability is the target when these algorithms underestimate recovery needs for Medicare seniors with complex conditions and trap Medicaid patients in endless prior authorizations loops. To make matters worse, these systems learn from historically biased data about who received rehab, imaging, or specialist referrals—essentially programming yesterday’s discrimination into tomorrow’s care decisions at massive scale. CMS’s new guardrails are a start, but without transparency requirements and real enforcement mechanisms, they’re little more than suggestions. 

So What Can You Do?

Fighting a faceless algorithm can feel impossible, but here’s practical checklist that can tip the odds in your favor:

• Read the notice for tells: Look for phrasing like “not medically necessary per criteria,” “fails predictive model,” or decisions issued within hours of submission. Ask (in writing) whether an algorithm assisted the decision and which policy or guideline was applied.
• Demand a human review: Under CMS guidance and laws like CA SB 1120, a licensed clinician must be accountable for coverage decisions. Request the physician reviewer’s credentials and a case-specific rationale that addresses your medical facts.
  • Example language to use an appeal:
    • “My clinician attests this service is medically necessary based on [diagnosis/facts]. Please cite the exact Medicare/NCD/LCD or plan policy used and explain how my case was evaluated individually as required.”
    • “If a software tool assisted the decision, identify it and confirm a licensed physician reviewed my file and made the final determination, per CMS.”
    • “I request an expedited review due to the risk of health deterioration.”
• Be strategic about how you escalate: Ask for a peer-to-peer review. Log calls, save portal messages, and for Medicare beneficiaries, contact your State Health Insurance Assistance Program (SHIP) or legal aid. Note if the denial conflicts with Medicare coverage rules.
• Know Your Rights (Medicare Advantage): Plans can use algorithms to assist, but they can’t deny or terminate care without individualized medical-necessity review by clinicians, and must follow Medicare coverage criteria. If you suspect a batch or automated denial, call it out and request documentation.

Individual action helps, but families shouldn’t have to fight case by case. Here’s what policymakers should do next:

1. 1. Mandate audit trails: Every AI-assisted coverage decision should generate a readable log: what tool/criteria were used, what data were considered, performance by subgroup, and which clinician signed off. CA’s SB 1120 and subsequent guidance are starting points; other states should follow.
   2. Publish denial/appeal/reversal rates (by line of business and ZIP code): If specific tools or vendors correlate with high reversal rates (i.e. many denials overturned), regulators and the public should know.
   3. Tie transparency to participation: For federal pilots like WISeR, require model documentation, bias testing, and patient-facing disclosures as conditions of entry. Publish independent evaluations early and often.
   4. Protect continuity of care: When algorithms trigger cuts to rehab days or therapies, require short grace periods while appeals are pending to prevent harm that can’t be undone.

• Invest in navigation, not just negation: If a system can flag “inappropriate” services in milliseconds, it can also surface covered alternatives, community resources, and streamlined approvals for evidence-based care.

What Happens Next

We’re at an inflection point. The woman at the DMV that sparked this piece shouldn’t have had to spend weeks fighting for her mother’s rehab. Right now, while courts are still hearing cases and regulators are writing rules, we have a small window to ensure algorithms serve patients rather than spreadsheets. The question isn’t whether AI will reshape healthcare—it already has. The costs of getting this wrong extends farther than you may know. This is something I’ll explore in my next article, at the epicenter of America’s AI boom, where the true price of algorithmic healthcare is written in power bills and water usage.

Want the bigger picture on how AI could close—or widen—health gaps? Read our explainer: AI in Healthcare: Equity, Disparities & Opportunities