What are the Risks of using an AI Assistant for Legal Advice?

What are the risks of using an AI assistant for legal advice?

AI assistants like ChatGPT, Claude AI, Gemini and others are becoming increasingly popular as a tool to help make many daily tasks easier. They can organise a schedule, summarise lengthy articles or answer general questions about topics you are researching. However, when it comes to using AI assistants for legal advice, there are some important risks to bear in mind which Victoria Nottage, Director of Knowledge and Learning, summarises below.

LLMs do not understand what they are ‘saying’

AI assistants are tools built on Large Language Models (LLMs), sophisticated pattern-matching systems trained on vast amounts of text from the internet. They predict what words should come next in a sentence based on statistical patterns. This is what allows them to produce fluent and plausible-sounding text.

However, they are not knowledge databases, legal experts or reasoning engines. They don’t “know” whether the cases they cite actually exist or whether the legal principles they explain are accurate. They generate text that sounds authoritative because it mirrors patterns they have learned.

LLMs can make things up ‘hallucinate’

The recent case of Re D (A Child) (Recusal), heard at the Court of Appeal in December 2025, highlights the risks of using AI to help prepare court documents.

In that case, a mother representing herself in a child contact dispute submitted a 60-page document in support of her application.

The document referenced a number of cases to support her arguments. Some cases were properly cited whilst others simply didn’t exist, and other references were correct but irrelevant to her arguments.

While the Court absolved her of any intention to mislead, recognising that litigants in person face enormous challenges, the judge reminded everyone of their duties to the court and warned that such “hallucinations” from AI can mislead, waste time and incur unnecessary costs. The situation also risked damaging the mother’s credibility in the proceedings.

LLMs are not bound by confidentiality

Everyone involved in a family case is required to keep the documents produced in the family court confidential. It is very serious to breach this and can be contempt of court. If you have legal advice, this is protected by secrecy (known as client privilege). This protection can be undermined if you share that information with a third party.

Many AI platforms use inputs to train their models or store conversations on servers that may not be adequately protected. Information shared with a chatbot may also be retained and reused in ways that are not obvious to the user.

Even outside of court, family matters often involve highly sensitive information, from detailed financial data to personal information about a child or circumstances concerning the individuals involved, such as earning capacity or illness. 

The lack of security and confidentiality when using an AI assistant is therefore a significant risk.

LLMs tell you what they think you want to hear

One of the most valuable aspects of legal advice is receiving an objective and strategic assessment of the case. ChatGPT, Claude AI and others are unashamedly sycophantic, leaning towards telling users what they think they want to hear. This can be dangerous when it comes to assessing legal claims and in the context of family separation has the potential to increase conflict at an already sensitive and stressful time.  

By way of illustration, a set of facts were inputted in an LLM. In the same session the LLM was asked to identify the husband’s claims and then in the next input, the wife’s.  First, the husband was told that he could expect not to pay any spousal maintenance and then the wife was informed that she had a claim for being financially supported for the rest of her life. The significant risk here means that separating couples are moved towards polarised positions and away from settlement. We know that increasing conflict for separating families doesn’t help anyone, especially children.  

A good legal advisor will listen carefully, understand the nuanced circumstances and advise you on the appropriate legal bracket for a fair outcome. This approach enables a realistic and calm process towards resolving the financial issues and helps to avoid expending time, cost and emotional energy in unnecessary litigation.

LLMs generate responses based on patterns

Because LLMs generate responses based on patterns, the way you phrase your question will shape the answer you receive.

LLMs won’t tell you the same thing twice

They might tell you what you want to hear, but it will not necessarily be the same thing twice. 

Practical tips for using AI in family law

If you are facing family separation and haven’t yet instructed a lawyer and want to understand how far you can get using an AI chatbot, here are some practical tips:

• Ensure you understand how LLMs work. They predict language, they do not verify legal accuracy.
• Family law can feel like learning a new language. A chatbot can help you understand legal jargon and basic definitions.
• If the output refers to court rules, time limits or cases, double and triple check the sources. If they exist, make sure they still apply to your circumstances.
• Never input your personal information. Always anonymise your questions.
• Never upload case documents. These are confidential and sharing them inappropriately could amount to a breach of confidentiality or, in some circumstances, contempt of court.