Dr. Anuradha KAR, Associate Professor of AI & Robotics at aivancity
Student research spotlight:
Final-year Master’s students (Year 5 of the Grande École program) at aivancity recently conducted a groundbreaking study on intelligent legal assistants, demonstrating how Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) pipelines can drive transformation in the field of law and justice through AI while upholding ethical standards. The paper, “Building a Legal Chatbot With LLMs and RAG,” authored by aivancity students Jessica Mbounkap, Habiba Djigo, Tasnim Masheh, Aymrick Nzooh, and Anuradha Kar, was published in the book *AI Transformations in Global Legal Systems* by IGI Global in March 2026.
Link to the paper:
The research highlights the significance and practical value of a recent advancement in generative AI known as GraphRAG (Graph Retrieval-Augmented Generation), particularly for applications in the legal field. Unlike conventional retrieval systems, GraphRAG captures and models the relationships among statutes, judicial precedents, and legal doctrines, enabling systems to reason across multiple legal sources, identify relevant analogies, trace the evolution of legal doctrines, and generate transparent, explainable responses for users. Such explainability and traceability are especially critical in legal contexts, where users need to understand not only the outcome but also the reasoning process behind it.
As part of this work, the students developed a modular legal GraphRAG pipeline based on verified legal data sources, including statutes, court judgments, and official legal commentaries. They designed and implemented an autonomous legal document chatbot powered by GraphRAG for advanced document analysis. Unlike traditional RAG systems that rely primarily on vector similarity, the GraphRAG-based system developed in this work constructs knowledge graphs from legal documents, enabling the system to understand complex relationships among entities such as parties, clauses, obligations, and legal concepts. The solution further incorporates an agent-based architecture that goes beyond simple question-answering.
Users can configure the chatbot to run on OpenAI’s GPT-4o, Anthropic’s Claude, or Google’s Gemini, switching providers through the interface without making any changes to the underlying codebase. This flexibility is important in enterprise legal contexts where data governance policies, contractual obligations, or regional compliance requirements may dictate which AI services an organization is permitted to use.
The Knowledge Graph — What You See
The chatbot's indexing pipeline generates an interactive knowledge graph in which each node represents an extracted legal entity (e.g., laws, organizations, people, procedures, or legal concepts). The graph provides a structured representation of legal knowledge, revealing interactions between actors, rules, institutions, and procedures. This structured representation enables the chatbot to answer contextual and relationship-oriented legal queries, going beyond traditional keyword-based retrieval.
- Node size reflects graph degree: highly connected entities appear larger, highlighting structurally important concepts.
- Edges represent relationships between entities, labeled with descriptions such as “grants authority to” or “is enforced by.”
- Relationships are extracted directly by the language model from the source documents,
- Nodes are color-coded by entity type to make them easier to interpret (e.g., organizations, laws, people, concepts).
- Selecting a node displays its description, the number of relationships it has, and its graph community membership.
An overview of our user interface:
Sample answers from our legal chatbot:
To test our chatbot, we uploaded a legal document containing regulations from the United Kingdom (any document can be used, but documents with scanned text or images should be avoided) and asked it some questions. Below are some sample answers provided by ChatGPT (a generic chatbot) and our specialized Agentic GraphRAG chatbot. It is evident that our GraphRAG enabled more sophisticated legal reasoning by retrieving interconnected legal concepts, cases, and provisions through graph traversal, producing contextually richer and constitutionally nuanced answers that go beyond those of standard LLM or keyword-based retrieval systems.
| Question | ChatGPT Result | Our Chatbot's Response |
|---|---|---|
| What is the Royal Prerogative? | The Royal Prerogative consists of powers historically exercised by the monarch that have not been revoked by Parliament. These powers include matters relating to national security, defense, and the deployment of the armed forces. | The Royal Prerogative refers to the discretionary powers historically vested in the Crown but now exercised by government ministers in the name of the monarch. These powers cover a wide range of governmental functions, including national security and defense, which are critical to the sovereignty of the state. While Parliament’s ability to scrutinize actions taken under the Royal Prerogative is limited, these powers are increasingly recognized by courts as subject to review, especially in significant cases such as the UK’s withdrawal from the EU, as demonstrated in the case “R (Miller and Santos) v Secretary of State for Exiting the European Union.” |
| How does the Human Rights Act 1998 protect rights in the UK? | The Human Rights Act 1998 incorporates the European Convention on Human Rights into UK domestic law, allowing individuals to bring human rights claims before UK courts. | The Human Rights Act 1998 (HRA98) is a crucial piece of legislation in the UK that incorporates the rights set forth in the European Convention on Human Rights (ECHR) into domestic law. This allows UK citizens to bring cases before domestic courts for violations of their human rights. However, the Act incorporates these rights in a limited way, as it does not empower courts to invalidate legislation that conflicts with the ECHR. Instead, if a court finds that an Act of Parliament contravenes the ECHR, it can issue a declaration of incompatibility under section 4 of the HRA98. This means that while the courts recognize the conflict, the legislation remains in effect unless Parliament decides to amend it. |
| What is the significance of the Magna Carta of 1215? | The Magna Carta established the principle that the government must act in accordance with the law and with the consent of the people. | The Magna Carta of 1215 is a foundational document that embodies the principle that government must be conducted in accordance with the law and with the consent of the governed. It is significant because it laid the groundwork for the development of constitutional law and the protection of individual rights, influencing legal systems around the world. |
How to use it: Users can try out this chatbot for themselves! Just follow the steps below to get your own GraphRag assistant!
GitHub link: The full source code, documentation, and deployment configuration are available at: https://github.com/Aloys-Aymrick-Nzooh/Legal-Law-Chat-Bot
To run the chatbot locally, follow the steps in the README file provided in the GitHub repository.
We will now examine the practical role of legal chatbots in society and explore how RAG, GraphRAG, and Agentic GraphRAG architectures enable advanced legal reasoning and retrieval capabilities.
Legal Chatbots: A Quiet Transformation in Legal Practice
The legal profession, traditionally conservative and resistant to rapid change, is undergoing a quiet but significant transformation. At the heart of this evolution lies a powerful technology: legal chatbots—AI-powered assistants designed to answer legal questions, guide users through complex procedures, and assist legal professionals with research and documentation. These systems are not intended to replace lawyers; rather, they enhance the delivery of legal services, making it faster, more accessible, and increasingly data-driven.
What makes this development particularly noteworthy is that innovation in this field is not coming solely from technology giants. Students and early-career researchers are actively shaping how these tools should be built and deployed responsibly. This blog explores three interconnected themes: the significance of legal chatbots in the justice system, the critical role of GraphRAG technology, and a spotlight on our student research that demonstrates how to build intelligent legal systems that are both sophisticated and ethically grounded.
Understanding Legal Chatbots: Redefining Access to Justice
Legal chatbots are conversational AI systems trained on legal knowledge, workflows, and structured data. They can assist with a wide range of tasks:
- Client intake and initial consultations
- Explaining legal procedures and requirements
- Generating legal documents and templates
- Answers to frequently asked legal questions
What sets legal chatbots apart from general-purpose AI assistants is their specialized training and design. These systems must operate within carefully defined legal boundaries, incorporating safeguards, structured workflows, and domain-specific datasets to ensure consistency and reliability. Most importantly, legal chatbots hold tremendous potential for democratizing access to justice. For individuals who cannot afford legal representation or live in areas with limited access to legal services, these systems can provide preliminary guidance, help them understand their rights, and prepare them for interactions with legal professionals. While they cannot replace qualified attorneys, they can bridge critical gaps in access to legal information.
What is RAG?
Modern legal chatbots rely on cutting-edge AI technology known as Retrieval-Augmented Generation (RAG). At its core, RAG combines two complementary capabilities:
- Retrieval: Searching through a trusted database of legal documents—including laws, court rulings, legal opinions, and official guidelines.
- Generation: Using a Large Language Model (LLM) to generate contextually relevant answers based on the retrieved documents.
Rather than relying solely on pre-trained knowledge, RAG allows the chatbot to “look up” the most relevant information before generating an answer. This approach significantly improves accuracy and ensures that responses are grounded in authoritative legal sources. The technical process involves converting the input or “ingested” documents into vector embeddings (numerical representations), storing them in specialized vector databases, retrieving relevant document chunks using similarity search, and then feeding this context to the LLM to generate informed answers.
GraphRAG: The Next Evolution
Graph Retrieval-Augmented Generation (Graph RAGs) represents a significant advancement in conversational AI, combining the power of knowledge graphs with retrieval-augmented generation to create more intelligent, context-aware chatbots. Unlike traditional RAG systems, which treat documents as flat text collections, Graph RAGs leverage structured knowledge representations to provide richer, more semantically precise responses.
A typical Graph RAG system consists of four interconnected layers. The Knowledge Graph Layer stores entities, attributes, and relationships extracted from source documents. The Retrieval Layer performs intelligent graph traversal to identify relevant subgraphs based on user queries. The Generation Layer processes the retrieved context using large language models, and the Reasoning Layer facilitates multi-hop inference across the graph structure.
The Role of Meaning in Modern Chatbots
Graph RAGs address critical limitations in current chatbot architectures. By encoding relationships between entities and concepts as graph structures, these systems enable deeper semantic understanding and more accurate information retrieval. They excel at answering complex, multi-hop questions that require reasoning across interconnected concepts—capabilities that are essential for enterprise applications, customer support, and knowledge-intensive domains.
A GraphRAG system is organized around four interconnected components. The Knowledge Graph Layer is responsible for storing the entities, attributes, and relationships extracted from source documents—in the legal context, this includes statutes, judicial precedents, legal principles, and the typed connections between them. The Retrieval Layer performs intelligent graph traversal, identifying relevant subgraphs based on the semantic content of a user’s query rather than simple keyword matching. The Generation Layer takes the retrieved graph context and passes it to a large language model, which synthesizes a coherent, well-grounded response. Finally, the Reasoning Layer enables multi-hop inference—the ability to chain facts across multiple connected nodes in the graph, answering questions that require understanding not just a single entity but the network of relationships in which it is embedded.
Key Advantages: Graph RAGs reduce hallucinations by grounding responses in explicit knowledge relationships, improve retrieval accuracy through semantic graph traversal, enable complex reasoning over interconnected information, and support dynamic knowledge updates without retraining the underlying language model.
Agentic GraphRag: One Step Ahead
Agentic GraphRAG combines retrieval-augmented generation with knowledge graph reasoning and autonomous decision-making capabilities. Unlike traditional RAG systems that retrieve isolated text passages, GraphRAG organizes information as interconnected entities and relationships, enabling contextual and multi-hop reasoning across documents. The agentic component introduces iterative planning and reasoning loops, allowing the system to analyze a query, refine retrieval strategies, evaluate intermediate results, and determine whether additional evidence is required before answering. This architecture is particularly effective in legal domains, where understanding procedural dependencies, institutional relationships, and contextual nuances is essential. By integrating structured graph traversal with adaptive reasoning, Agentic GraphRAG improves explainability, retrieval precision, and response reliability.
Ethical challenges
It must be acknowledged that legal chatbots may face significant ethical challenges due to the sensitive and high-stakes nature of legal services. Ensuring accuracy and accountability is critical, as incorrect guidance can seriously impact users, especially vulnerable individuals. Data privacy and confidentiality must be protected through secure systems and compliance with regulations such as the GDPR. Transparency is also essential; users should clearly understand that they are interacting with an AI system and be informed about its limitations. Another major concern is avoiding the unauthorized practice of law, as chatbots must distinguish between providing general legal information and offering regulated legal advice that requires professional legal qualifications and oversight. However, with appropriate governance, transparency, and human oversight, legal chatbots have the potential to improve access to legal information, enhance efficiency, and support more inclusive and accessible legal services for society.
Conclusion: AI in the Service of Justice
Legal chatbots powered by large language models (LLMs) and RAG pipelines, and enhanced by GraphRAG for complex reasoning, represent a powerful convergence of law and technology. These systems have the potential to transform access to legal information, improve the efficiency and consistency of legal services, and help professionals deliver better outcomes.
However, their success depends not merely on technical sophistication, but fundamentally on ethical integrity and strong governance. The work of aivancity students such as Jessica, Habiba, Tasnim, and Aymrick, supervised by Professor Kar, demonstrates that this integration is possible. They have shown that innovation in legal AI need not compromise on ethics or fairness. Instead, these principles can and should be fundamental to how we build these systems.
As generative AI continues to evolve at an accelerating pace, the real challenge and the genuine opportunity lie in ensuring that technology serves justice rather than undermining it. This requires ongoing collaboration between technologists, legal professionals, ethicists, policymakers, and affected communities. Ultimately, the success of such AI systems depends on a careful assessment of their capabilities and an unwavering commitment to the fundamental principles that make the legal system worthy of trust.
