Building Trial Recruitment Automation from EHR Data

I have been in enough live care workflows to know the first bad assumption: people think trial matching is a model problem. It is not. It is a data quality problem, a consent problem, a workflow problem, and only then an AI problem.

When teams ask AST to help build recruitment automation, I start with the failure modes we have already seen in production-style healthcare software. A diagnosis buried in a free-text note. Lab values coming through in different units. Medication history split across Epic, scanned PDFs, and an outside feed. A coordinator who trusts the match list half the time and manually rebuilds it the other half. That is the real system. If you do not design for that mess, your AI matching layer becomes decorative.

I learned this the hard way in a workflow prototype that looked great in demo and fell apart the moment we loaded real encounter history. The system was confidently matching patients based on one recent note while missing a contraindication that lived in an imported problem list record. That mistake changed how I design these systems: if the gold standard is “the model read the chart,” you are already in trouble. The gold standard is “the system built a reviewable candidate package from structured data first.”

Start with the data you can defend

Before you talk about embeddings or retrieval, decide what sources are allowed into the recruitment engine. I keep the source layer small on purpose:

• Structured demographics from the EHR.
• Problems, diagnoses, allergies, medications, procedures, encounters, and labs.
• FHIR R4 resources where the vendor supports reliable extraction.
• HL7v2 feeds for event-driven updates when they are already part of the hospital’s interface stack.
• Carefully scoped note text only where the protocol actually needs it.

The reason is simple: every additional source multiplies your ambiguity. If you let the model ingest every possible note, fax, and attachment on day one, you will spend your time auditing exceptions instead of recruiting patients.

At AST, we usually wire this as a clinical data layer that normalizes source feeds into a common patient record before matching starts. That may sound pedestrian, but it is the part that stops the whole thing from wobbling. We have had projects where one vendor sent medication starts as timestamps and another sent date-only values. That looks minor until eligibility depends on whether treatment started before a protocol cutoff. Then it matters a lot.

How I structure the matching engine

I split matching into three distinct passes.

1. Phenotype the cohort deterministically Convert protocol criteria into machine-readable inclusion and exclusion logic. Age, diagnosis, recent lab thresholds, encounter recency, medication exposure, and procedure history belong here. This is the gate that removes obvious non-matches.
2. Use AI for fuzzy interpretation Allow the model to read messy elements such as note text, pathology snippets, or ambiguous clinical phrasing. The model can suggest similarity, presence, or risk of mismatch, but it never overrides hard exclusions.
3. Produce an auditable candidate packet Every match needs a trace: which criteria were satisfied, which were uncertain, what source value was used, and which reviewer approved outreach. If the system cannot explain itself, it is not ready for a real site.

This is where a lot of teams get seduced by “AI matching.” They want a single score. I do not. A single score hides the reason a patient is eligible, and that is exactly what coordinators need when they call a principal investigator or answer site questions.

The practical pattern is to keep the phenotype layer boring and strict. Then build the AI layer to do what humans are bad at: reading loosely structured content, ranking borderline candidates, and summarizing why this patient belongs on the review queue. If you want a service boundary analogy, this is exactly the sort of workflow we build in our clinical AI systems: model output is useful only when the downstream process knows how to distrust it safely.

IRB-safe data flows are the backbone, not paperwork

This part matters more than people want it to. Trial recruitment sits at the intersection of treatment data and research operations, which means your access model needs to be narrow, logged, and purposeful.

I design the flow like this:

• Clinical data is ingested into a secured matching workspace.
• The system computes eligibility inside that restricted environment.
• Recruiters and coordinators see only the minimum necessary candidate summary.
• Any patient outreach request is routed through an approved template and contact workflow.
• Audit trails preserve who viewed what, when, and why.

If the study requires an honest broker, de-identification, or consent checkpoint before outreach, that step belongs in the workflow engine, not in a policy PDF nobody reads. I have seen teams build a technically impressive matcher and then bolt on manual spreadsheet exports for “safety.” That is usually how data leaks and confusion sneak back in.

AST has had to design around this exact tension in live clinical environments: keep the operations team productive without giving them broader access than they need. In practice, that means partitioning the system into a matching service, a review surface, and an outreach service. The matching service can see more than the outreach team. The outreach team can act only after review approval. That separation feels slower at first. It is the reason the system survives contact with compliance.

FHIR phenotyping is useful, but only if you respect the gaps

FHIR R4 is a strong foundation for this work because it gives you clean resource boundaries: Patient, Observation, Condition, MedicationRequest, Procedure, Encounter, and DocumentReference. The trap is assuming every EHR exposes those resources consistently enough for production trial matching. They do not.

When we build around FHIR, we treat it as the canonical access pattern where possible, then backfill edge cases with other interfaces or vendor-native extracts. The matching core should not care whether a lab moved through FHIR Observation or a transformed feed from an interface engine. It should care that the value was normalized, provenance was preserved, and the unit conversion is unambiguous.

That distinction matters because phenotyping fails in quiet ways. A lab cutoff off by one unit. A diagnosis coded in a local terminology variant. A medication exposure that looks current until you notice it is just a med list artifact. The machine does not know which of those is clinically binding unless you teach it.

A practical build sequence I would use this week

1. Choose one protocol Do not build for “all trials.” Pick a study with clear inclusion and exclusion logic and enough volume to test review burden.
2. Write the eligibility map Convert the protocol into structured criteria. Mark each criterion as hard, soft, or notes-only.
3. Inventory your data sources Decide exactly which EHR objects, feeds, and documents are allowed into the matcher.
4. Normalize and timestamp everything Align units, code systems, and temporal rules before scoring.
5. Separate review from outreach The matching output should feed a coordinator queue, not a send button.
6. Log explanation artifacts Keep the source facts, rule hits, and AI rationale together for audit and QA.
7. Run shadow mode Compare system matches with manual coordinator picks before you automate any patient contact.

If you do not run shadow mode, you are betting your workflow on assumptions you have not tested. We have learned that the easiest way to break a recruitment product is to skip the quiet months where the system proves whether it can find patients the way coordinators actually work.

Patient outreach should feel operational, not creepy

Once a candidate is approved, outreach becomes a product design problem. The message has to be accurate, minimal, and approved by the study workflow. It should not reveal sensitive detail in a first pass. It should not overstate eligibility. It should not pretend the system has made a clinical promise.

I always tell teams the same thing: the automation does not replace the coordinator’s judgment, it removes the repetitive searching. That difference matters. Coordinators are not there to babysit a machine. They are there to interpret context, handle exceptions, and make the last-mile judgment that software should never fake.

We have seen better adoption when the system presents the candidate with the reason they were surfaced, the specific protocol criteria they seem to satisfy, and a simple next action. The moment a recruiter has to reverse-engineer the logic, trust falls apart. And once trust goes, the “automation” quietly turns into another tab people ignore.

The most counterintuitive thing I have seen is that the strongest matching systems feel less magical than people expect. They are narrower, more boring, and more explainable than the demo-driven version. That is not a weakness. That is what gets them used in a live clinic and not just admired in a meeting.

At AST, we build these systems as integrated engineering pods, not as a pile of features thrown over the wall. That matters because recruitment automation touches data pipelines, workflow design, and compliance at the same time. If you tackle only one of those layers, the product leaks in the other two.