Clinical NLP

Clinical NLP, or clinical natural language processing, is software that reads the written text in health records and turns it into structured data that other software can search and count. The text includes doctors' notes, discharge summaries, scan reports and referral letters. Most of what a patient's record says is written in sentences like these. A record system stores them, but for years no other software could use them. Large language models changed that, because they can read such text at scale.

Here is how it works in general. The software collects the notes and reports for a patient from the record. A language model reads each one. Usually it is built on a foundation model and adapted to medical text. It picks out the facts: each diagnosis, medicine, test result and date. It must also understand what a sentence denies or places in the past. For example, "no chest pain" means the patient does not have chest pain, and "mother had diabetes" is about the mother, not the patient. The software stores each fact in a field, often with a standard code, and links it to the sentence it came from. A person can then check any fact against the original text.

Why the link back to the sentence? A language model can state a fact that the note does not contain, which is called AI hallucination. So when the data is used for care or research, people check a sample of the facts, or every fact that matters.

Take, for example, a drug company that needs 300 patients with one kind of kidney disease for a trial. Clinical trial patient recruitment software reads hospitals' notes and finds the patients who meet the trial's conditions.

Consider a cancer hospital that must report every case to a cancer registry. Clinical data abstraction reads years of notes and pathology reports, and fills in the stage of each cancer and its treatment. Employees in data-abstraction teams once did this reading by hand.

For instance, a real-world evidence platform reads the notes of millions of patients, with their names removed, to find a rare side effect of a medicine.

How is clinical NLP different from an AI medical scribe? The scribe writes notes, from the conversation in the room. Clinical NLP reads notes that are already written. It is also different from intelligent document processing, which reads invoices and forms with a fixed layout. Clinical notes have no fixed layout, and are full of medical short forms.