AI object, Milord? The jury’s out on chatbots in the legal system

After completing his LLM degree in IP and Technology law this year, Chennai-based Kapil Naresh went all in with another kind of LLM or large language model (a type of artificial intelligence that powers chatbots like ChatGPT, Microsoft’s Bing and Google’s Bard). The 23-year-old, who is also the founder of a legal consultancy firm, has been building an assistive tool called LegalGPT.
“Right now, LegalGPT is only as good as a first-year law student or with better prompts, a rookie lawyer like me. I want to make LegalGPT as advanced as some of the legendary lawyers like Kapil Sibal and Abhishek Manu Singhvi,” says Naresh.
He says he first tested ChatGPT todraft an FIR and a chargesheet. Satisfied with the results, he went on to ask for the template of a power of attorney for a family property-related matter. Emboldened with the respectable draft tossed out by the chatbot, Naresh then created a scenario closer to one of his cases with a jurisdiction and pseudonyms. After some relevant tweaks in the document, it was ready. “I will apply the 80:20 rule to the LegalGPT that I am building. As lawyers we have to do 20% of the interpretative work ourselves,” says Naresh who is a descendant of T Muthuswamy Iyer, the first Indian judge of the Madras High Court in British India.
But isn’t he worried by the recent case of US lawyer Steven A. Schwartz getting into trouble after ChatGPT wrote him a brief citing non-existent cases? Naresh says he is aware of the pitfalls and wants to ensure his own bot is trained on verified legal datasets so it is “credible, efficient and productive”.
Like him, many others believe generative AI will be a gamechanger for the legal profession. A March-April paper from the Harvard Law School listed fourareas where ChatGPT is sure to cause disruption — namely legal research, document generation, legal information, and legal analysis. The paper further adds, “. . . lawyers will soon need to use these new tools if they hope to remain competitive. ”
Instances of limited adoption are already visible in India. In March, Justice Anoop Chitkara of the Punjab and Haryana High Court used ChatGPT “to present a broader picture on bail jurisprudence” while dealing with a case involving assault and murder. This was the first reported instance of a judge seeking the services of a chatbot in India. Just a few days before that, a new version of OpenAI’s GPT-4 passed the American bar exam. In the UK, a top-tier law firm has started using a chatbot called Harvey (named after the hero of ‘Suits’) to help employees draft documents and client memos more quickly.
However, Indian firms have been careful, with only some lawyers and law school students beginning to use the AI chatbot purely in a personal capacity. Niyati Shah, a member at Nishith Desai Associates in their legal innovation and technology division, says it is still early days. “A lot of Indian legal data is protected, confidential and also very disorganised. Even some cases which come up in courts are not published in the public domain. Then there is the question of legal English. When we talk about a word like notice, it will have one meaning in a contractual sense, another meaning in the context of dispute resolution and something else in normal English,” says Shah, illustrating the challenges of building AI software for the legal industry. She says instead, the focus should be on creating generative AI tools for commonpeople to help them understand their legal rights or get advice.
Moving in this direction is Bengaluru-based non-profit Agami India. One of their initiatives is OpenNyAI to create open software and data for the use of AI in justice. Launched under this mission is Jugalbandi which uses generative AI tech to come up with WhatsApp and Telegram bots that help users understand their rights and the application process for over 50 government schemes.
For Karnataka High Court, a pilot called JIVA (Judges’ Intelligent Virtual Assistant) is being developed. It is meant to assist judges in retrieving, through voice or text conversations, the laws and sections they require during court matters. Sachin Malhan, co-founder of Agami India, says there is more on the anvil. “Just like finance or commerce, AI will change the whole terrain of law and justice in the next five to seven years — both in formal systems like the courts and in private, commercial services such as dispute resolution, grassroots legal assistance on property disputes, legal process financing, and such like,” says Malhan. Of course, the challenge is training chatbots in Indian legal languages.
Aman Taneja, principal associate at Ikigai Law, a tech-focused law and public policy firm in Delhi and Bengaluru, stresses the need for guardrails. “There is something known as automation bias where there is a tendency to rely on output just because it’s come out of a computer. So, before anything like this gets deployed in the justice delivery system, it requires extensive testing. One of the other challenges, in case of faulty output, is having clarity on where exactly accountability will lie,” says Taneja.