Dr. Anuradha KAR, Associate Professor of AI & Robotics at aivancity
Student research highlight:
Final year Master (Programme Grande Ecole year 5) students at aivancity recently conducted an impactful research on intelligent legal assistants, demonstrating how Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) pipelines can bring transformation in the field of law and justice with AI while maintaining ethical responsibility. The work, “Building a Legal Chatbot With LLMs and RAG,” authored by aivancity students Jessica Mbounkap, Habiba Djigo, Tasnim Masheh, Aymrick Nzooh, and Anuradha Kar, was published in the book AI Transformations in Global Legal Systems by the IGI Global publication house in March 2026.
Link to the paper:
The research emphasizes the significance and practical value of a recent advancement in generative AI known as GraphRAG or Graph Retrieval-Augmented Generation, particularly for applications in the legal domain. Unlike conventional retrieval systems, GraphRAG captures and models the relationships among statutes, judicial precedents, and legal doctrines, allowing systems to reason across multiple legal sources, identify relevant analogies, trace the evolution of legal doctrines, and generate transparent, explainable responses for users. Such explainability and traceability are especially critical in legal contexts, where users need to understand not only the outcome but also the reasoning process behind it.
As part of this work, the students developed a modular legal GraphRAG pipeline based on verified legal data sources, including statutes, court judgments, and official legal commentaries. They designed and implemented an autonomous legal document chatbot powered by GraphRAG for advanced document analysis. In contrast to traditional RAG systems that depend primarily on vector similarity, the GraphRAG based system developed in this work builds knowledge graphs from legal documents, enabling the system to comprehend complex relationships among entities such as parties, clauses, obligations, and legal concepts. The solution further incorporates an agentic architecture that goes beyond simple question-answering.
Users can configure the chatbot to run on OpenAI’s GPT-4o, Anthropic’s Claude, or Google’s Gemini switching providers through the interface without any changes to the underlying codebase. This flexibility is important in enterprise legal contexts where data governance policies, contractual obligations, or regional compliance requirements may dictate which AI services an organisation is permitted to use.
The Knowledge Graph — What You See
The indexing pipeline of the chatbot produces an interactive knowledge graph where each node represents an extracted legal entity (e.g., laws, organisations, people, procedures, or legal concepts). The graph provides a structured representation of legal knowledge, exposing interactions between actors, rules, institutions, and procedures. This structured representation enables the chatbot to answer contextual and relationship-oriented legal queries, beyond traditional keyword-based retrieval.
- Node size reflects graph degree: highly connected entities appear larger, highlighting structurally important concepts.
- Edges represent relationships between entities, labeled with descriptions such as “grants authority to” or “is enforced by.”
- Relationships are extracted directly by the language model from the source documents,
- Nodes are colour-coded by entity type for easier interpretation (e.g., organisations, laws, people, concepts).
- Selecting a node reveals its description, relationship count, and graph community membership.
An overview of our user Interface :
Sample answers by our legal ChatBot :
For testing our chatbot we uploaded a legal document having legal regulations from the United Kingdom (any document can be used but documents with scanned text or images should be avoided) and asked it some questions. Below you can see some sample answers given by Chatgpt (a generic chatbot) and our specialized Agentic GraphRAG chatbot. It can be seen that our GraphRAG enabled richer legal reasoning by retrieving interconnected legal concepts, cases, and provisions through graph traversal, producing contextually deeper and constitutionally nuanced answers beyond standard LLM or keyword-based retrieval systems.
| Question | ChatGPT Result | Our Chatbot Answer |
|---|---|---|
| What is the Royal Prerogative? | The Royal Prerogative consists of powers historically exercised by the monarch that have not been removed by Parliament. These powers include matters relating to national security, defense, and deployment of the armed forces. | The Royal Prerogative refers to the discretionary powers historically vested in the Crown but now exercised by government ministers in the name of the monarch. These powers cover a wide range of governmental functions, including national security and defense, which are critical to the sovereignty of the state. While Parliament’s ability to scrutinize actions under the Royal Prerogative is limited, these powers are increasingly recognized by courts as reviewable, especially in significant cases like the UK’s withdrawal from the EU, as demonstrated in the case « R (Miller and Santos) v Secretary of State for Exiting the European Union. » |
| How does the Human Rights Act 1998 protect rights in the UK? | The Human Rights Act 1998 incorporates the European Convention on Human Rights into UK domestic law, allowing individuals to bring human rights claims before UK courts. | The Human Rights Act 1998 (HRA98) is a crucial piece of legislation in the UK that integrates the rights from the European Convention on Human Rights (ECHR) into domestic law. This allows UK citizens to bring cases in domestic courts for breaches of their human rights. However, the Act incorporates these rights in a limited way, as it does not empower courts to invalidate legislation that conflicts with the ECHR. Instead, if a court finds that an Act of Parliament contravenes the ECHR, it can issue a declaration of incompatibility under section 4 of the HRA98. This means that while the courts recognize the conflict, the legislation remains in effect unless Parliament decides to amend it. |
| What is the significance of Magna Carta 1215? | Magna Carta established the principle that government must act according to law and with the consent of the governed. | The Magna Carta 1215 is a foundational document that embodies the principle that government must be conducted according to the law and with the consent of the governed. It is significant because it laid the groundwork for the development of constitutional law and the protection of individual rights, influencing legal systems around the world. |
Practical usage: Users can test this chatbot themselves! Just follow the steps below and you can have your own GraphRag assistant!
Github link: The full source code, documentation, and deployment configuration are available at : https://github.com/Aloys-Aymrick-Nzooh/Legal-Law-Chat-Bot
To run the chatbot locally, follow the steps in the readme provided in the github repository.
We now examine the practical role of legal chatbots in society and explore how RAG, GraphRAG, and Agentic GraphRAG architectures enable advanced legal reasoning and retrieval capabilities.
Legal Chatbots: A quiet transformation in legal practice
The legal profession, traditionally conservative and resistant to rapid change, is undergoing a quiet but significant transformation. At the center of this evolution stands a powerful technology: legal chatbots which are AI-powered assistants designed to answer legal questions, guide users through complex procedures, and support legal professionals with research and documentation. These systems are not meant to replace lawyers; rather, they augment legal service delivery, making it faster, more accessible, and increasingly data-driven.
What makes this development particularly noteworthy is that innovation in this space is not coming from technology giants alone. Students and early-career researchers are actively shaping how these tools should be built and deployed responsibly. This blog explores three interconnected themes: the significance of legal chatbots in the justice system, the critical role of GraphRAG technology, and a spotlight on our student research that demonstrates how to build intelligent legal systems that are both sophisticated and ethically grounded.
Understanding legal chatbots: Redefining access to justice
Legal chatbots are conversational AI systems trained on legal knowledge, workflows, and structured data. They can assist with a wide range of tasks:
- Client intake and initial consultations
- Explaining legal procedures and requirements
- Generating legal documents and templates
- Answering frequently asked legal questions
What distinguishes legal chatbots from general-purpose AI assistants is their specialized training and design. These systems must operate within carefully defined legal boundaries, incorporating safeguards, structured workflows, and domain-specific datasets to ensure consistency and reliability. Most importantly, legal chatbots hold tremendous potential for democratizing access to justice. For individuals who cannot afford legal representation or live in areas with limited access to legal services, these systems can provide preliminary guidance, help them understand their rights, and prepare them for interactions with legal professionals. While they cannot replace qualified attorneys, they can bridge critical gaps in access to legal information.
What is RAG ?
Modern legal chatbots rely on a state of the art AI technology known as Retrieval-Augmented Generation (RAG). At its core, RAG combines two complementary capabilities:
- Retrieval: Searching through a trusted database of legal documents—including laws, case judgments, legal opinions, and official guidelines.
- Generation: Using a Large Language Model (LLM) to generate contextually relevant answers based on the retrieved documents.
Rather than relying solely on pre-trained knowledge, RAG allows the chatbot to ‘look up’ the most relevant information before generating an answer. This approach significantly improves accuracy and ensures that responses are grounded in authoritative legal sources. The technical process involves converting the input or ‘ingested’ documents into vector embeddings (numerical representations), storing them in specialized vector databases, retrieving relevant document chunks using similarity search, and then feeding this context to the LLM to generate informed answers.
GraphRAG: The Next Evolution
Graph Retrieval-Augmented Generation (Graph RAGs) represents a significant evolution in conversational AI, combining the power of knowledge graphs with retrieval-augmented generation to create more intelligent, context-aware chatbots. Unlike traditional RAG systems that treat documents as flat text collections, Graph RAGs leverage structured knowledge representations to provide richer, more semantically precise responses.
A typical Graph RAG system comprises four interconnected layers. The Knowledge Graph Layer stores entities, attributes, and relationships extracted from source documents. The Retrieval Layer performs intelligent graph traversal to identify relevant subgraphs based on user queries. The Generation Layer processes retrieved context through large language models, and the Reasoning Layer facilitates multi-hop inference across the graph structure.
Significance in Modern Chatbots
Graph RAGs address critical limitations in current chatbot architectures. By encoding relationships between entities and concepts as graph structures, these systems enable deeper semantic understanding and more accurate information retrieval. They excel at answering complex, multi-hop questions that require reasoning across interconnected concepts—capabilities essential for enterprise applications, customer support, and knowledge-intensive domains.
A GraphRAG system is organised around four interconnected components. The Knowledge Graph Layer is responsible for storing the entities, attributes, and relationships extracted from source documents — in the legal context, this means statutes, judicial precedents, legal principles, and the typed connections between them. The Retrieval Layer performs intelligent graph traversal, identifying relevant subgraphs based on the semantic content of a user’s query rather than simple keyword matching. The Generation Layer takes the retrieved graph context and passes it to a large language model, which synthesises a coherent, grounded response. Finally, the Reasoning Layer enables multi-hop inference — the ability to chain facts across multiple connected nodes in the graph, answering questions that require understanding not just a single entity but the network of relationships in which it sits.
Key Advantages: Graph RAGs reduce hallucinations by grounding responses in explicit knowledge relationships, improve retrieval precision through semantic graph traversal, enable complex reasoning over interconnected information, and support dynamic knowledge updates without retraining the underlying language model.
Agentic GraphRag: One step advanced
Agentic GraphRAG combines retrieval-augmented generation with knowledge graph reasoning and autonomous decision-making capabilities. Unlike traditional RAG systems that retrieve isolated text passages, GraphRAG organises information as interconnected entities and relationships, enabling contextual and multi-hop reasoning across documents. The agentic component introduces iterative planning and reasoning loops, allowing the system to analyse a query, refine retrieval strategies, evaluate intermediate results, and determine whether additional evidence is required before answering. This architecture is particularly effective in legal domains, where understanding procedural dependencies, institutional relationships, and contextual nuances is essential. By integrating structured graph traversal with adaptive reasoning, Agentic GraphRAG improves explainability, retrieval precision, and response reliability.
Ethical challenges
It must be acknowledged that legal chatbots may face significant ethical challenges due to the sensitive and high-stakes nature of legal services. Ensuring accuracy and accountability is critical, as incorrect guidance can seriously impact users, especially vulnerable individuals. Data privacy and confidentiality must be protected through secure systems and compliance with regulations such as GDPR. Transparency is also essential; users should clearly understand they are interacting with an AI system and be informed about its limitations. Another major concern is avoiding the unauthorized practice of law, as chatbots must distinguish between providing general legal information and offering regulated legal advice requiring professional legal qualification and oversight. However, with appropriate governance, transparency, and human oversight, legal chatbots have the potential to improve access to legal information, enhance efficiency, and support more inclusive and accessible legal services for society.
Conclusion: AI in service of justice
Legal chatbots powered by LLMs and RAG pipelines and enhanced by GraphRAG for complex reasoning represent a powerful convergence of law and technology. These systems have the potential to transform access to legal information, improve the efficiency and consistency of legal services, and support professionals in delivering better outcomes.
However, their success depends not merely on technical sophistication, but fundamentally on ethical integrity and strong governance. The work of aivancity students like Jessica, Habiba, Tasnim, Aymrick supervised by Professor Kar demonstrates that this integration is possible. They have shown that innovation in legal AI need not compromise on ethics or fairness. Instead, these principles can be and should be foundational to how we build these systems.
As generative AI continues to evolve at an accelerating pace, the real challenge and the genuine opportunity lies in ensuring that technology serves justice rather than complicating it. That requires ongoing collaboration between technologists, legal professionals, ethicists, policymakers, and affected communities. Ultimately the success of such AI systems requires careful evaluation about the AI capabilities, and unwavering commitment to the fundamental principles that make the legal system worthy of trust.
