Machine Learning for Prioritisation in Healthcare Services
Some of the safest, most useful AI in healthcare does not diagnose anything. It helps busy teams decide which request, referral or patient to look at next.
Discussions of AI in healthcare often jump straight to diagnosis: a model reading scans or predicting disease. Those applications are important, and they are also among the most heavily regulated and hardest to validate. A quieter category often delivers value sooner: prioritisation. Healthcare teams face queues everywhere, from referrals and inbound enquiries to messages, follow-ups and care requests. Deciding what to look at first is a real problem that machine learning can help with.
This post describes how we approach prioritisation systems in healthcare settings: what to predict, how to keep clinicians and staff in control, and what to measure.
Prioritisation also fits how healthcare teams already work. Staff sort queues all day using experience and informal rules. A model that makes that sorting faster and more consistent supports their judgement rather than replacing it, which matters for adoption as much as for safety.
Rank, do not decide
The central design choice is that the model orders work; people make decisions. A prioritisation model suggests which items in a queue probably need attention sooner and explains why. A nurse, coordinator or advisor still reviews each item. This keeps the human accountable, makes errors recoverable and is usually a far easier conversation with clinical governance than a system that acts on its own.
Whether a given system counts as a medical device depends on its intended purpose and on local regulation, so involve regulatory advice early. Our post on implementing AI in healthcare safely covers that process in more depth.
Start by observing how the queue is handled today. Watch experienced staff sort their work and ask what they look for. Their implicit rules are often a strong baseline, and some of them can be implemented directly as rules before any model is trained. The model then has to beat that baseline to earn its place.
Choosing what to predict
A model is only as useful as its target. Good targets are concrete, observable and agreed with the people who will use the output:
- Likelihood that a request will need escalation to a clinician.
- Likelihood that a case is time-sensitive, based on how similar cases were handled historically.
- Likelihood that a family or patient will drop out of a process without contact.
- Expected complexity, so cases can be routed to the right level of expertise.
Be careful with historical labels. If past prioritisation was inconsistent or biased, a model trained on it will learn the same patterns. Review the data with clinicians before training anything.
Agree on the cost of each kind of error with clinicians before training. Ranking an urgent case too low is usually far worse than ranking a routine case too high, and the model's thresholds and evaluation should reflect that imbalance rather than treating all mistakes as equal.
Often, simpler models are better
For tabular data such as request type, timings, history and structured answers, gradient-boosted trees or logistic regression frequently perform as well as anything more complex, and they are easier to explain and validate. Language models are useful for turning free-text messages into structured signals first. The best systems often combine the two: an LLM extracts features from text, a simple, well-understood model produces the score.
Validate before anyone relies on it
Before a prioritisation model influences real queues, run it in shadow mode: it scores live items, but staff do not see the scores. Compare its ranking with what actually happened, and with how experienced staff prioritised the same items. This shows how the model performs on current data, without any risk to patients, and gives clinicians concrete evidence to review rather than a promise.
When it does go live, start with a limited group of users and a defined review period. Agree in advance which results would lead you to pause, adjust or stop the rollout.
Designing the interface around the score
- Show the reason for a high priority, such as the phrases or factors that drove it, not just a number.
- Let staff override the order with one action, and record the override as feedback.
- Keep a safety net: rules for clearly urgent signals that bypass the model entirely.
- Never hide low-priority items. Everything stays visible; the model only changes the order.
- Make the system's limits clear in the interface, so nobody treats it as a diagnosis.
Training matters too. Staff who understand what the score means, and what it does not mean, use it better. A short session with real examples of correct and incorrect rankings, and clear guidance on when to override, does more for safe use than any amount of documentation.
Measuring what matters
Accuracy on a test set is necessary but not sufficient. Before launch, evaluate performance across patient groups to detect uneven error rates. After launch, track operational outcomes agreed with the team, such as time to first contact for urgent cases, override rates and missed escalations. A rising override rate is an early sign that the model or the underlying population has drifted.
Data protection runs through all of this. Health data is special-category data under GDPR and protected health information under HIPAA in the United States, which shapes where data is stored, who can see it and how long logs are kept. Getting these decisions right early is far cheaper than retrofitting them after launch.
Plan for drift
Healthcare services change: new referral routes, seasonal demand, updated forms, different patient populations. Each change can shift what the model sees. Monitor input distributions as well as outcomes, schedule periodic revalidation with the clinical team, and keep a documented process for retraining and redeploying the model, including who signs off on a new version.
Context from our healthcare work
Healthcare services operate under heavy, uneven load. On A Place For Mom, a US senior-care platform serving more than 300,000 customers, we delivered the healthcare service side on Next.js and Node.js with event-driven AWS infrastructure built to stay reliable under nationwide demand. Any prioritisation layer in a service like that has to be as dependable as the platform beneath it.
Exploring AI for your healthcare service?
We build healthcare software and AI with privacy and human oversight designed in. Read about our AI in healthcare service, see our healthcare work, or contact us.
