PhenoPrompt: RAG and Agent-Based AI Meet Clinical Phenotyping

How a new open-source framework from aivancity is making clinical phenotyping accessible to any clinician—simply by typing a question.

Introduction: Classroom Innovation That Extends Beyond the Four Walls and Class Hours

PhenoPrompt did not originate from a dedicated research project, but rather in a classroom during the “AI for Health” course taught by Prof. Anuradha Kar to fifth-year grande école students at aivancity. As part of the course evaluation process, students were challenged to identify real-world problems at the intersection of AI and clinical language processing and to propose data-driven solutions. It was in this setting that Shafiya Kausar, a current final-year Master of Science student in the program, developed the initial concept: a pipeline capable of extracting clinical entities from unstructured medical notes using the Inria Medkit library, clustering patients into phenotypic groups, and making those groups queryable without requiring disease-specific algorithms. Recognizing both the significance of the idea and its potential to address a genuine gap in computational phenotyping, Prof. Kar encouraged Shafiya and proposed converting the pipeline into a prompt-driven tool that would benefit clinicians and medical AI researchers in discovering and characterizing patient populations and classifying disease phenotypes. Together, they expanded the course prototype into a full research framework, adding the entity-augmented retrieval layer and, crucially, an agentic AI component: a tool-calling agent that normalizes colloquial or misspelled clinical queries into clean medical concepts before retrieval, making the system robust to the way clinicians actually speak and type. What began as a student assignment became a conference paper accepted at IANLP 2026, the 1st International Conference on AI and Natural Language Processing, and a live, open-source web application that can be deployed on any institution’s clinical corpus. In that sense, the story of PhenoPrompt is also a story about what a well-designed course project and a timely faculty-student collaboration can achieve.

The problem: phenotyping is still a method that focuses on one disease at a time

When a clinical researcher wants to study, say, all patients with type 2 diabetes and early-stage kidney disease at their hospital, they face a surprisingly difficult problem. The information exists—scattered across thousands of clinical notes, laboratory records, and prescriptions—but accessing it requires writing a custom algorithm: a set of ICD codes to include, lab value thresholds, medication filters, and keyword searches in free text. That algorithm typically takes weeks to build, requires a clinical informatician and a domain expert to work together, and must be validated by manually reviewing hundreds of patient records. And it works for exactly one disease. To study a second disease, the entire process starts over.

The Phenotype KnowledgeBase (PheKB), maintained by the eMERGE consortium, is the community’s effort to share and reuse these algorithms. After years of collective effort across major research institutions, it lists only 46 validated phenotyping algorithms. Given the immense complexity of human disease, this small number represents a bottleneck that cannot scale.

A cardiologist on a ward round cannot stop to analyze a scatter plot or submit a data request. She needs to ask a question and get an answer—right away, in her own clinical language.

This is the problem PhenoPrompt was built to solve. Presented at IANLP 2026 in Casablanca, Morocco, our paper introduces a framework that transforms unstructured clinical notes into a universal, queryable phenotype space —one that any clinician can query in plain English, without writing a single SQL query or ICD code.

What is clinical phenotyping?

A phenotype is the set of observable characteristics of a patient—their symptoms, diagnoses, responses to medication, and clinical course—that distinguish them from others. Computational phenotyping is the algorithmic identification of patients who share a particular phenotypic profile based on electronic health records (EHRs).

In practice, EHRs contain two very different types of data. Structured data—ICD codes, laboratory values, prescription records—is easy to query but captures only a fraction of clinical reality. A substantial proportion of what clinicians actually know about a patient is contained in free-text notes: the discharge summary, the consultation report, and the nursing notes. A patient may have peripheral neuropathy documented in every clinical note, yet it may never appear in a single ICD code because it was not the primary reason for admission. Existing phenotyping systems largely overlook this.

Why It Matters — Four Downstream Uses

• Precision medicine: identifying patient subgroups with distinct treatment responses
• Clinical trial cohort discovery: Find patients who meet eligibility criteria based on notes, not just codes
• Disease subtyping: revealing heterogeneity within a single diagnostic category
• Discovery of novel phenotypes: identify patient clusters that do not have an existing diagnostic label

Introducing PhenoPrompt

PhenoPrompt is a three-stage pipeline that integrates open-source clinical NLP, unsupervised machine learning, and retrieval-augmented generation into a single, GPU-free, reproducible framework. The key design principle is simple: build the phenotype space once—then let users ask questions about it in plain language, on demand, for any disease.

Stage 1 — Clinical Entity Extraction with MedKit

Each clinical note is processed using the medkit NLP pipeline: sentence and clause tokenization, rule-based extraction of clinical entities, negation and uncertainty detection to retain only confirmed findings, and construction of a TF-IDF feature matrix that captures the most discriminative clinical concepts across the corpus.

Across the entire corpus of approximately 28,500 notes, the medkit pipeline—after correcting negation handling and normalizing entity surface forms to canonical concepts—extracted a vocabulary of 78 distinct affirmed clinical concepts—disorders, medications, findings, and procedures—from raw free-text notes.

Stage 2 — Map Embedding and Phenotypic Clustering

The entity feature matrix is compressed into a dense patient embedding using Latent Semantic Analysis (LSA) (truncated SVD, 64 dimensions), then projected onto two dimensions using UMAP to produce the clinical note embedding map: an interactive scatter plot in which spatial proximity reflects shared phenotypic profiles.

Clustering is performed in the full 64-dimensional embedding using the HDBSCAN algorithm, which automatically detects phenotypic clusters and labels outliers as noise. It outperformed K-Means, Agglomerative, and DBSCAN. Each cluster is characterized by its top enriched clinical entities, providing interpretable phenotype profiles rather than binary labels.

“Upon examining the phenotype profiles, the identified clusters correspond to recognizable clinical groups—for example, anticoagulation, asthma (treated with inhalers), diabetes/metabolic, and anxiety/depression phenotypes. (In our original evaluation of 500 notes, an LLM judge rated 5 out of 6 clusters as coherent, with example labels such as ‘cancer-related phenotype’ and ‘stroke-like neurological syndrome.’)”

Stage 3 — The RAG and Agentic RAG-Based Clinical Querying

At the heart of PhenoPrompt’s query interface is a two-layer approach to grounded clinical question answering. In the first layer, standard Retrieval-Augmented Generation (RAG) is applied: rather than relying on a language model’s parametric knowledge, the system first retrieves the most relevant patient notes from the indexed corpus using a weighted entity-matching score that combines exact concept matches, synonym-expanded matches, and partial token matches. The language model then generates its answer exclusively from this retrieved evidence, citing each source note by its identifier—so every clinical claim in the response is directly traceable to a real note in the dataset, eliminating hallucination by design. In the second and optional layer, an agentic RAG mode adds an intelligent pre-processing step before retrieval. A single-step tool-calling agent intercepts the clinician’s free-text query, normalizes colloquial or misspelled phrasing, corrects abbreviations, and extracts the salient clinical concepts—so a query like “fetch me diabetic patients with kidney issues” is silently cleaned up into well-formed medical terms before the retrieval call is issued. This makes the system robust to the natural variation in how clinicians actually type, without requiring them to learn controlled vocabulary. The agent gracefully falls back to standard retrieval if the tool call is unavailable, ensuring the system remains functional across any deployment environment.

Finally — The prompt interface

This is where PhenoPrompt becomes a clinical decision-support tool rather than a batch analysis pipeline. A clinician types a question— “What medications are documented for patients with diabetes and kidney disease?” —and the system responds with a well-founded, cited answer drawn from the actual patient notes.

The retrieval engine processes the query using three complementary mechanisms: exact entity matching, synonym expansion (so “T2DM” matches “diabetes,” and “breathless” matches “shortness of breath”), and partial token matching. Notes are scored using a weighted TF-IDF formula across all matched entities; the top-K notes are retrieved; and an LLM generates an answer that is constrained to cite only the retrieved evidence —preventing hallucinations by design.

The synonym layer proved to be essential: without it, queries such as “breathless” and “fluid overload” returned zero results; with it, all four synonym-dependent queries returned the full result set.

The system also offers an agent-based retrieval mode in which a single-step tool-calling agent first normalizes the query into clear clinical concepts—correcting spelling and extracting key conditions—before issuing the retrieval request. This makes the system robust against colloquial or misspelled questions.

What Makes PhenoPrompt Unique

There are several phenotyping systems. What sets PhenoPrompt apart is a specific combination of features that, to the best of our knowledge, no previous system has managed to achieve together:

Universal scope, not disease-specific

Every existing phenotyping pipeline targets a single disease. PhenoPrompt constructs a single phenotype space that can be queried for any disease. The index is built once; the query phenotype is specified at query time, not at build time.

Mixed phenotype, not a binary label

The output is not a case/control label but a ranked entity profile for each cluster, showing which conditions, medications, and findings are enriched relative to the entire population. This preserves and highlights the heterogeneity that binary outputs discard.

Entity-Augmented Retrieval

Rather than relying solely on dense vector similarity, retrieval uses the same medkit entity pipeline that built the index—matches are structural, not just semantic. This makes the matches interpretable: you can see exactly which entity triggered a retrieval.

Free-text first

Because entities are extracted from clinical notes rather than billing codes, PhenoPrompt captures conditions that are documented but never coded—a systematic blind spot in all ICD-based systems.

Accessible to clinicians

The entire system is deployed as a web application that requires no SQL expertise, no data engineering, and no algorithm development specific to each disease. A clinician types in a question and reads an answer that includes cited source notes.

The live web application

PhenoPrompt is not just a research prototype. It is deployed as a publicly accessible Streamlit web application at https://phenoprompt-skausar-aivancity.streamlit.app/, built entirely on CPU-only infrastructure and requiring no paid model subscription to use (an open LLM API key unlocks the generative answer layer; without it, the system returns ranked retrieved notes directly).

What you can do in the app:

• Phenotype Explorer: View the UMAP embedding map of your corpus, colored by the clusters identified
• Cluster profiles: View per-cluster cards showing the entities with the highest enrichment scores for each phenotype
• Top Concepts View: Explore the most common clinical concepts across the corpus
• Map overlay: recolor the embedded map based on the identified cluster or on entity density per note
• Table of notes: Browse each note along with its cluster label and phenotype profile
• PhenoPrompt tab: Type clinical questions in plain English and receive well-grounded, cited answers in standard or agentic retrieval mode

Presenting PhenoPrompt at IANLP 2026

PhenoPrompt was accepted for an oral presentation at IANLP 2026—the 1st International Conference on Artificial Intelligence and Natural Language Processing, held June 29–30, 2026, at the Ben M’Sick Faculty of Sciences (FSBM), Hassan II University of Casablanca, Morocco. The conference brings together researchers and practitioners working at the intersection of AI and NLP, with a particular focus on real-world applications, including healthcare, education, and industry.

The paper will be presented in Session 8: AI for Healthcare, Biomedical Signals, and Clinical Decision Support. The IANLP conference is notable for its strong multilingual and Francophone NLP community, and we look forward to discussing how PhenoPrompt’s architecture—built in part on medkit, which was originally developed for French clinical corpora—could be extended to Arabic and French EHR data, opening up phenotyping capabilities to clinical data warehouses beyond the English-speaking world.

IANLP 2026 is the first edition of a new conference series dedicated to AI for NLP, organized by AM2I, LTIM, and FSBM at Hassan II University in Casablanca. aivancity is represented both in the conference program and on the scientific committee.

What's Next

Although PhenoPrompt has evolved from a 500-note proof of concept to a 28,500-note corpus, it remains an ongoing research and development project. Future work will focus on validating it using real-world EHRs and clinician annotations, adopting UMLS-based biomedical NER, extending support to multilingual clinical notes, and incorporating longitudinal and hierarchical phenotyping to better model disease progression and patient trajectories.

In closing

“PhenoPrompt transforms the traditional computational phenotyping workflow from a batch analysis pipeline into an interactive, exploratory interface—accessible to any clinician, for any disease, from any clinical dataset.”

The goal behind PhenoPrompt is straightforward: clinical knowledge is contained in clinical notes, and the people who understand that knowledge best—clinicians—should be able to access it directly, in their own language, without a technical intermediary. We hope that by releasing the framework as open-source software and deploying a live demo, we can invite the community—researchers, clinicians, and data scientists—to explore the idea, test it on their own data, and help develop it into something clinically useful.

The full paper will be available in the IANLP 2026 proceedings published by Springer. The code is available at github.com/Shafiya0101/phenoprompt. The live application is available at https://phenoprompt-skausar-aivancity.streamlit.app/. We welcome collaboration, feedback, and questions at kar@aivancity.ai and shafiya.kausar@aivancity.education.

---