How We Build Clinical Decision Support with GenAI

The fastest way to ruin a clinical decision support tool is to let generative AI freewheel inside the chart. I have seen teams do it. They start with a good intent: surface useful guidance at the point of care, reduce cognitive load, maybe even catch a missed order or contradictory note. Then they give the model too much room, feed it incomplete context, and celebrate a slick demo where everything sounds confident. The first time it sees a borderline case, it invents certainty. That is not support. That is a liability generator.

My view is blunt: CDS with generative AI only works when the model is a controlled reasoning layer, not the source of truth. The source of truth stays in deterministic rules, guideline mappings, medication knowledge, lab thresholds, order context, and patient data that you can inspect. The model helps interpret and present. It does not decide in isolation.

At AST, I have seen this pattern repeatedly across clinical systems: the teams that succeed do not start with the model. They start with the workflow. They map the decision point, identify the action a clinician can actually take, and define the evidence needed to justify the suggestion. Only then do they decide where generative AI adds value. That is also why our best implementations look less like a chatbot and more like a decision layer embedded in the record, with the clinical logic fenced in from the start. When we build this kind of experience, we design it like a production system, not a prototype that hopes to stay polite.

What a real CDS tool has to do

If you are evaluating a clinical decision support platform, I would judge it on four things before I care about the model brand underneath it:

• Timing: Does it appear at the moment the decision is made, not after the chart is already signed?
• Specificity: Does it speak to one action, one guideline, one patient context, or does it dump generic advice?
• Traceability: Can a clinician or auditor see exactly which rule, guideline fragment, or source fact drove the suggestion?
• Failure behavior: When data is missing or conflicting, does it stay quiet, ask for more input, or hallucinate a tidy answer?

That last one matters more than people admit. A model that tries to be helpful with incomplete context is often worse than no model at all. In the field, missing medication history, stale problem lists, duplicate encounters, and partial lab feeds are normal. If your CDS tool does not have a denial path for uncertainty, it will drift into overconfident suggestions. Clinicians notice that immediately. Trust collapses fast.

This is where a lot of teams get surprised. They think the innovation is in synthesis. It is not. The innovation is in constraint. I have watched systems become dramatically safer once we forced the model to work from a bounded retrieval set, a fixed output schema, and a rules engine that can say no. That is the difference between a clinical assistant and an untestable opinion machine.

How I structure the stack

When we build decision support with generative AI, I over-engineer the architecture on purpose. Clinical software should feel boring under pressure. The stack has to look more like a monitored workflow than a prompt experiment.

1. Define the decision point Pick one moment in the workflow where a recommendation matters: ordering, diagnosis refinement, medication selection, follow-up, contraindication review, or documentation quality. If the moment is fuzzy, the tool will be fuzzy.
2. Normalize the clinical inputs Pull structured data from the EHR, not just note text. Problem lists, meds, labs, allergies, vitals, encounter context, and procedure history all need a clean representation.
3. Constrain the evidence set Retrieve only the guideline excerpts, payer rules, internal policies, or care pathways relevant to that patient and that task. Do not let the model browse the whole universe.
4. Force structured output Use a schema that separates recommendation, rationale, evidence, confidence, and escalation. Free-text output is where safety and parsing problems both begin.
5. Gate the action Route anything that changes care through human review, workflow confirmation, or a policy-based allowlist. The tool can suggest. It cannot silently order, sign, or override.
6. Log the chain Store the input snapshot, retrieved evidence, prompt version, model version, and final user action. If you cannot reconstruct the decision later, you do not have CDS, you have a memory problem.

That architecture sounds strict because it is. Clinicians do not need a clever model doing improv inside the chart. They need a system that knows its lane. The lane matters even more when the recommendation is time-sensitive or workflow-sensitive, because a recommendation at the wrong point is functionally noise.

At AST, we have seen this in live integrations: once the CDS layer sits beside the EHR instead of pretending to replace it, the design becomes much more disciplined. We have also seen teams waste weeks because they treated the note as the primary record when the structured facts told a different story. That mismatch is not rare. It is the norm.

Where generative AI actually helps

I am not anti-AI. I am anti-careless AI. Generative models are genuinely useful in CDS when they do jobs that humans hate doing consistently:

• Summarizing the relevant context into a short, clinician-readable brief.
• Explaining why a recommendation appears in plain language tied to the retrieved evidence.
• Reconciling terminology across note text, problem lists, and structured codes.
• Drafting next-step options for review, such as follow-up recommendations or documentation prompts.
• Translating policy into workflow language so clinicians are not forced to read a payer or internal policy document mid-visit.

That is also where the boundary matters. If the model is asked to infer a diagnosis, it will overreach unless the surrounding system is tightly bounded. If it is asked to summarize why a known contraindication matters, it can be excellent. The same model, different control surface.

In our work on clinical AI systems, I keep coming back to this: the user does not need a chatbot that sounds medical. The user needs a decision aid that lands right where the clinical question already exists. That means the tool must understand context like encounter type, specialty, active meds, and whether the recommendation is for an order, a note, a referral, or a problem-oriented assessment. If you ignore that layer, your CDS will feel generic even if the model is powerful.

AST’s practical playbook for implementation

When I advise teams on building a CDS platform, I use the same sequence every time. It keeps the project from turning into a science fair.

1. Choose one narrow use case Start with a decision that is high-value, low-ambiguity, and already has a known policy or guideline backbone.
2. Write the failure cases first Decide what should happen when labs are stale, the med list is incomplete, or the note conflicts with the structured chart.
3. Design the output contract Make the model return a fixed schema with fields clinicians and auditors both understand.
4. Bind the model to evidence Retrieve only approved sources and include snippets or citations the user can inspect.
5. Instrument escalation paths Show when the system is unsure and route those cases to human review or a safer default.
6. Test against ugly charts Use duplicate medications, conflicting allergies, partial histories, and specialty-specific edge cases before you ever touch production.
7. Roll out in shadow mode Let the system suggest without affecting care, then compare its output to real clinician decisions.

Shadow mode is where the truth comes out. The model might look great in a polished demo and still fall apart when real notes are messy. It might also be overly conservative, which is not a success if it misses the moments where CDS actually helps. You do not find that out in a boardroom. You find it out when the system starts seeing the same annoyances clinicians see every day.

That is why I like phased trust models. You do not grant autonomy because the model feels good. You earn it per task, per specialty, and per workflow. In some cases, the right answer is forever assist-only. That is not a failure. That is responsible product design.

If you are comparing platforms, use that table as a filter. I would rather ship a constrained, boring CDS tool that clinicians trust than a flashy one that gets ignored after the first weird recommendation. Adoption is a product of reliability. Reliability is a product of design discipline.

What I would ask a vendor this week

Before you buy a CDS platform, ask the vendor these questions and insist on direct answers:

• Can you show me the exact retrieval sources for a suggestion, not just the final text?
• What happens when the underlying chart data is incomplete or contradictory?
• Can clinicians see why the recommendation fired inside the workflow, without leaving the EHR?
• How do you separate model output from policy logic?
• What is your rollback plan when a model version starts behaving differently?

If the answers are vague, keep walking. A serious platform will have thought about audit logs, versioning, prompt control, source ranking, and fallbacks. A flimsy one will talk about intelligence and personalization while dodging the mechanics.

At AST, when we wire clinical AI into live environments, we assume the chart will be annoying and the workflow will be fragmented. That assumption saves time. It also forces us to test the parts that matter: context selection, response shape, and human approval boundaries. The models change. The operating discipline must not.

If you are building this inside a broader clinical AI stack, the same principles apply across ambient documentation, care-gap support, and task routing. The moment the model starts affecting care, the bar changes. I would rather make that bar explicit on day one than discover it after clinicians stop trusting the system.