PhenoPrompt: RAG and Agentic AI meets Clinical Phenotyping

How a new open-source framework from aivancity is making clinical phenotyping accessible to any clinician — just by typing a question.

Introduction: Classroom Innovation That Goes Beyond Four Walls and Course Hours

PhenoPrompt began not in a dedicated research project, but in a classroom during the AI for Health course taught by Prof. Anuradha Kar for 5th year programme grande ecole students at  aivancity.  As part of a course evaluation process students were challenged to identify real-world problems at the intersection of AI and clinical language processing and propose data-driven solutions. It was in this setting that Shafiya Kausar, a current final year Master of Science student on the programme, developed the initial concept: a pipeline that could extract clinical entities from unstructured medical notes using the Inria Medkit library, cluster patients into phenotypic groups, and make those groups queryable without requiring disease-specific algorithms. Recognising both the significance of the idea and its potential to address a genuine gap in computational phenotyping, Prof. Kar encouraged Shafiya and proposed to convert the pipeline into a promptable tool that will benefit clinicians and medical AI researchers for discovering and characterising patient populations and classifying disease phenotypes. Together, they expanded the course prototype into a full research framework, adding the entity-augmented retrieval layer and crucially — an agentic AI component: a tool-calling agent that normalises colloquial or misspelled clinical queries into clean medical concepts before retrieval, making the system robust to the way clinicians actually speak and type. What started as a student assignment became a conference paper accepted at IANLP 2026, the 1st International Conference on AI and Natural Language Processing, and a live, open-source web application deployable on any institution’s clinical corpus. The story of PhenoPrompt is, in that sense, also a story about what a well-posed course project and a timely faculty-student collaboration can produce.

The problem: phenotyping is still a one-disease-at-a-time search method

When a clinical researcher wants to study, say, all patients with type 2 diabetes and early-stage kidney disease at their hospital, they face a surprisingly hard problem. The information exists — scattered across thousands of clinical notes, laboratory records, and prescriptions — but accessing it requires writing a bespoke algorithm: a set of ICD codes to include, lab value thresholds, medication filters, and keyword searches in free text. That algorithm typically takes weeks to build, requires a clinical informatician and a domain expert working together, and must be validated by manually reading hundreds of patient records. And it works for exactly one disease. To study a second disease, the whole cycle starts over.

The Phenotype KnowledgeBase (PheKB), maintained by the eMERGE consortium, is the community’s attempt to share and reuse these algorithms. After years of collective effort across major research institutions, it lists only 46 validated phenotyping algorithms. For the vast complexity of human disease, this small number is a bottleneck that cannot scale.

A cardiologist on a ward round cannot pause to navigate a scatter plot or submit a data request. She needs to ask a question and get an answer — now, in her own clinical language.

This is the problem PhenoPrompt was built to solve. Presented at IANLP 2026 in Casablanca, Morocco, our paper introduces a framework that transforms unstructured clinical notes into a universal, queryable phenotype space — one that any clinician can interrogate in plain English, without writing a single SQL query or ICD code.

What is clinical phenotyping?

A phenotype is the set of observable characteristics of a patient — their symptoms, diagnoses, medication responses, and clinical trajectory — that distinguish them from others. Computational phenotyping is the algorithmic identification of patients who share a particular phenotypic profile from electronic health records (EHRs).

In practice, EHRs contain two very different kinds of data. Structured data — ICD codes, laboratory values, prescription records — is easy to query but captures only a fraction of clinical reality. A substantial proportion of what clinicians actually know about a patient lives in free-text notes: the discharge summary, the consultation report, the nursing note. A patient can have peripheral neuropathy documented in every clinical note and never have it appear in a single ICD code, because it was not the primary reason for admission. Existing phenotyping systems largely miss this.

Why it matters — four downstream uses

• Precision medicine: identify patient subgroups with distinct treatment responses
• Clinical trial cohort discovery: find patients matching eligibility criteria from notes, not just codes
• Disease subtyping: reveal heterogeneity within a single diagnostic category
• Novel phenotype discovery: surface patient clusters that carry no existing diagnostic label

Introducing PhenoPrompt

PhenoPrompt is a three-stage pipeline that connects open-source clinical NLP, unsupervised machine learning, and retrieval-augmented generation into a single, GPU-free, reproducible framework. The key design principle is simple: build the phenotype space once — then let users ask questions about it in plain language, on demand, for any disease.

Stage 1 — Clinical entity extraction with medkit

Each clinical note is processed using the medkit NLP pipeline: sentence and clause tokenization, rule-based extraction of clinical entities, negation and uncertainty detection to retain only affirmed findings, and construction of a TF-IDF feature matrix that captures the most discriminative clinical concepts across the corpus.

Across the full ~28,500-note corpus, the medkit pipeline — after correcting negation handling and normalising entity surface forms to canonical concepts — extracted a vocabulary of 78 distinct affirmed clinical concepts — disorders, medications, findings, and procedures — from raw free-text notes.

Stage 2 — Embedding map and phenotypic clustering

The entity feature matrix is compressed into a dense patient embedding using Latent Semantic Analysis (LSA) (Truncated SVD, 64 dimensions), then projected to two dimensions with UMAP to produce the clinical note embedding map: an interactive scatter plot in which spatial proximity reflects shared phenotypic profiles.

Clustering is performed in the full 64-dimensional embedding using the HDBSCAN algorithm, which automatically detects phenotypic clusters and labels outliers as noise. It outperformed K-Means, Agglomerative, and DBSCAN. Each cluster is characterized by its top enriched clinical entities, providing interpretable phenotype profiles rather than binary labels.

« Inspecting the phenotype profiles, the discovered clusters correspond to recognisable clinical groups — for example, anticoagulation, asthma-on-inhalers, diabetes/metabolic, and anxiety/depression phenotypes. (In our original 500-note evaluation, an LLM judge rated 5 of 6 clusters as coherent, with example labels such as ‘cancer-related phenotype’ and ‘stroke-like neurological syndrome’.) »

Stage 3 — The RAG and Agentic RAG based clinical querying

At the heart of PhenoPrompt’s query interface is a two-layer approach to grounded clinical question answering. In the first layer, standard Retrieval-Augmented Generation (RAG) is applied: rather than relying on a language model’s parametric knowledge, the system first retrieves the most relevant patient notes from the indexed corpus using a weighted entity-matching score that combines exact concept matches, synonym-expanded matches, and partial token matches. The language model then generates its answer exclusively from this retrieved evidence, citing each source note by its identifier — so every clinical claim in the response is directly traceable to a real note in the dataset, eliminating hallucination by design. In the second and optional layer, an agentic RAG mode adds an intelligent pre-processing step before retrieval. A single-step tool-calling agent intercepts the clinician’s free-text query, normalises colloquial or misspelled phrasing, corrects abbreviations, and extracts the salient clinical concepts — so a query like « fetch me diabetic patience with kidney issues » is silently cleaned into well-formed medical terms before the retrieval call is issued. This makes the system robust to the natural variation in how clinicians actually type, without requiring them to learn controlled vocabulary. The agent degrades gracefully to standard retrieval if the tool call is unavailable, ensuring the system remains functional across any deployment environment.

Finally — The prompt interface

This is where PhenoPrompt becomes a clinical decision-support tool rather than a batch analysis pipeline. A clinician types a question — « What medications are documented for patients with diabetes and kidney disease? » — and the system responds with a grounded, cited answer drawn from the actual patient notes.

The retrieval engine processes the query through three complementary mechanisms: exact entity matching, synonym expansion (so « T2DM » finds « diabetes », « breathless » finds « shortness of breath »), and partial token matching. Notes are scored by a weighted TF-IDF formula across all matched entities, the top-K notes are retrieved, and an LLM generates an answer constrained to cite only retrieved evidence — preventing hallucination by design.

The synonym layer was shown to be essential: without it, queries like « breathless » and « fluid overload » returned zero results; with it, all four synonym-dependent queries recovered the full result set.

The system also offers an agentic retrieval mode in which a single-step tool-calling agent first normalises the query into clean clinical concepts — correcting spelling and extracting salient conditions — before issuing the retrieval call. This makes the system robust to colloquial or misspelled questions.

What makes PhenoPrompt novel

Several phenotyping systems exist. What distinguishes PhenoPrompt is a specific combination of properties that, to our knowledge, no prior system achieves together:

Universal scope, not disease-specific

Every existing phenotyping pipeline targets one disease. PhenoPrompt builds a single phenotype space from which any disease can be queried. The index is built once; the query phenotype is specified at query time, not at build time.

Phenotype mix, not binary label

The output is not a case/control label but a ranked entity profile per cluster, showing which conditions, medications, and findings are enriched relative to the full population. This preserves and surfaces the heterogeneity that binary outputs discard.

Entity-augmented retrieval

Rather than dense vector similarity alone, retrieval uses the same medkit entity pipeline that built the index — matching is structural, not just semantic. This makes matches interpretable: you can see exactly which entity triggered a retrieval.

Free-text first

Because entities are extracted from clinical notes rather than billing codes, PhenoPrompt captures conditions that are documented but never coded — a systematic blind spot of all ICD-based systems.

Clinician-accessible

The entire system is deployed as a web application requiring no SQL expertise, no data engineering, and no per-disease algorithm authoring. A clinician types a question and reads an answer with cited source notes.

The live web application

PhenoPrompt is not just a research prototype. It is deployed as a publicly accessible Streamlit web application at https://phenoprompt-skausar-aivancity.streamlit.app/ , built entirely on CPU-only infrastructure and requiring no paid model subscription to use (an open LLM API key unlocks the generative answer layer; without it, the system returns ranked retrieved notes directly).

What you can do in the app:

• Phenotype explorer: view the UMAP embedding map of your corpus, coloured by discovered cluster
• Cluster profiles: see per-cluster cards with the top enrichment-scored entities for each phenotype
• Top concepts view: inspect the most frequent clinical concepts across the corpus
• Map overlay: re-colour the embedding map by discovered cluster or by entity density per note
• Notes table: browse every note with its cluster label and phenotype profile
• PhenoPrompt tab: type clinical questions in plain English and receive grounded, cited answers in standard or agentic retrieval mode

Presenting PhenoPrompt at IANLP 2026

PhenoPrompt was accepted for oral presentation at IANLP 2026 — the 1st International Conference on Artificial Intelligence and Natural Language Processing, held 29–30 June 2026 at the Faculty of Sciences Ben M’Sick (FSBM), Hassan II University of Casablanca, Morocco. The conference brings together researchers and practitioners working at the intersection of AI and NLP, with a particular focus on real-world applications including healthcare, education, and industry.

The paper is presented in Session 8: AI for Healthcare, Biomedical Signals and Clinical Decision Support. The IANLP conference is notable for its strong multilingual and francophone NLP community, and we look forward to discussing how PhenoPrompt’s architecture — built in part on medkit, which was originally developed for French clinical corpora — could be extended to Arabic and French EHR data, opening phenotyping capabilities to clinical data warehouses beyond the Anglophone world.

IANLP 2026 is the first edition of a new conference series dedicated to AI for NLP, organised by AM2I & LTIM & FSBM at Hassan II University of Casablanca. aivancity is represented both in the paper programme and on the scientific committee.

What comes next

Although PhenoPrompt has progressed from a 500-note proof of concept to a 28,500-note corpus, it is an ongoing body of research and development. Future work focuses on validating it with real-world EHRs and clinician annotations, adopting UMLS-grounded biomedical NER, extending support to multilingual clinical notes, and incorporating longitudinal and hierarchical phenotyping to better model disease progression and patient trajectories.

In closing

« PhenoPrompt turns the traditional computational phenotyping workflow from a batch analysis pipeline into an interactive, exploratory interface — accessible to any clinician, for any disease, from any clinical corpus. »

The ambition behind PhenoPrompt is straightforward: clinical knowledge lives in clinical notes, and the people who understand that knowledge best — clinicians — should be able to access it directly, in their own language, without a technical intermediary. We hope that by releasing the framework as open-source software and deploying a live demo, we can invite the community — researchers, clinicians, data scientists — to explore the idea, test it on their own data, and help build it into something clinically useful.

The full paper will be available in the IANLP 2026 proceedings by Springer. The code is at github.com/Shafiya0101/phenoprompt. The live application is at https://phenoprompt-skausar-aivancity.streamlit.app/  . We welcome collaboration, feedback, and questions at kar@aivancity.ai and shafiya.kausar@aivancity.education.

---