.png)
This article is the first in a three-part series exploring AI-driven contract review in the legal industry. Over the coming posts, we’ll take a deep dive into how lawyers can effectively leverage AI for AI-powered contract analysis, examine the growing landscape of AI-powered tools, and discuss the practicalities and limitations of the technology.
At ClauseBase, we’ve been lawyers and legal technologists for decades. In that time, we have never seen technology take off in the legal sector the way Generative AI has. By “taking off”, we don’t necessarily mean the rate of adoption (which is still lower than the hype would suggest). But at least the rate at which lawyers professionals are becoming aware of the technology and the effort and enthusiasm in the exploration of potential use cases are unprecedented.
One of the most obvious applications of Generative AI in legal work is reviewing contracts. The idea makes sense: tools like ChatGPT and Claude are really good at reading and writing material. Contract reviewing can be highly mechanical and repetitive work. Couldn’t they make this easier and faster?
If you’ve ever tried to upload a contract to ChatGPT and ask it to perform a general review for you, you will have found that the answer is “yes, but…”. It typically flags a couple of key issues but completely drops the ball on others. That’s what we also initially found when we did a few reviews of simple NDAs.
Like all tools, Generative AI does some tasks really well and others poorly. It’s important to know the ins and outs of how Generative AI should be used to maximize its performance.
In this article– the first in our broader series on AI-powered contract reviewing – we will provide a general primer for any lawyer looking to leverage this exciting technology in a way that truly fits their needs.
What is AI contract review? A Lawyer’s perspective
Different lawyers have different things in mind when they hear the words “contract review”. This can be either:
- Individual contract redlining: a lawyer receives a document from a counterparty in the context of a negotiation and proceeds to flag unacceptable provisions and mark up the wording where necessary, typically in track changes.
- M&A due diligence: a lawyer receives a data room full of contracts that have already been concluded and must flag provisions that could pose risk for a company acquiring one of the parties to those agreements
In this article, we focus on individual contract redlining. This is mostly because M&A due diligence is a more of a niche application of the technology, and comes with a range of different considerations.
How AI contract reviewing (doesn’t) work
Is AI-powered contract reviewing as simple as just uploading a document to ChatGPT, Claude, or some other preferred Large Language Model (LLM) and asking it to flag risks, suggest edits, and ensure compliance?
In theory: yes. In practice: not quite. There are several problems with this approach. We set them out below.
Challenge 1: AI contract review: what should the LLM look for?
Just asking for a general review is bound to give a couple of useful suggestions, but this leaves far too much up to the LLM’s discretion. As a rule of thumb: The more detailed your prompt, the better your results will be.
Of course, this means clearly defining what you want the AI to actually check for, and writing that down. This is especially important if you find yourself frequently reviewing the same types of contracts for the same types of risk and compliance issues relevant to your organisation. As you likely know, Generative AI can give two completely different answers to the same question. This unpredictability goes down significantly if you can give it a list of structured issues to check for.
Even so, we find that many lawyers and legal teams are reluctant to spend time on this. We often see lawyers interact with the technology in a way that supposes that the LLM can read their mind. This is likely because we have become accustomed to Google-like search quality which does sometimes seem like it can read your mind.
The problem is that your review requirements are wholly dependent on the situation – what industry do you operate in, what is your negotiation power, what is the contract about, what are your main concerns in terms of the liabilities they wish to avoid or the assets they wish to achieve, etc.?
Not being explicit about all this doesn’t mean that the AI has nothing to offer. It just means that you are relegating it to the status of a sparring partner rather than a true assistant. It means settling for a quick-and-easy approach where the AI scans a couple of issues that may or may not be relevant and let you decide if you will do anything with it. This kind of “Did I miss anything” check may still be valuable, but the impact is much lesser for it.
Challenge 2: Large Language Models and Compliance
We cover this topic in detail in a separate post in this series. In a nutshell: most commercial LLMs reuse any data you input into the free version of their tool to train the model further. However, this is generally not the case for the paid version or for API usage (i.e.: when you use the LLM indirectly through a tool that integrates with it, like our own ClauseBuddy).
This is a nuance that is lost on most lawyers. As a result, many lawyers hold off on exploring Generative AI for fear of breaching client confidentiality even though this fear is completely unfounded (if they are using the right version of the tool).
Challenge 3: No follow-up
Suppose the LLM provides an in-depth review of all the clauses you need to check, change, or delete. You then still need to manually make these changes. ChatGPT cannot edit the document for you. Or rather: it cannot make surgical adjustments to the document itself and leave the inherent style, formatting, and layout intact.
Challenge 4: AI in Legal Contract Review: Commercial vs. Legal Analysis
LLMs are currently really good at gauging commercial issues like how much a certain clause favours one party or another. Legal checks, however, are spotty at best. As it stands, the technology simply isn’t robust enough to perform legal analysis accurately.
This is in large part because the legal information that LLMs were trained on primarily consists of theoretical information, such as:
- legislation, which is by definition public
- case law, which is mostly public in most jurisdictions, even though a lot is also behind paywalls of publishers
- limited legal doctrine, mostly in the form of blogs and newsletters from law firms, with a limited amount of publicly available in-depth articles.
What LLMs lack, however, is practical information on how to review contracts. As every legal expert knows, relatively little practical information on this topic is available in written form, let alone publicly available online. While a decent number of tips & tricks are available for common contracts (such as NDAs), most practical information is:
- orally communicated, learned "on the job" and taught by experienced by lawyers
- found in small nuggets of wisdom spread across legal articles and books on specific types of contracts, almost always behind publisher paywalls
- individually acquired through years of experience
LLMs have no access to this information and will therefore have to be explicitly instructed on how to review contracts. There are vendors nowadays that claim to collect this data in order to enrich or fine-tune a standard LLM, but they often underperform.
How purpose-built AI review tools are addressing these problems
Even with these challenges, leveraging LLMs for contract reviewing holds a lot of promise. Many of the above-mentioned problems can be resolved by leveraging the powerful engine of the LLM within a purpose-built tool.
Our own ClauseBuddy is one such tool, but we definitely weren’t the only legal tech vendors who saw the obvious potential of the technology. In fact, Legal Technology Hub, one of the largest directories of legal tech products on the market, reports a whopping 82 solutions that deal with contract reviewing at the time of writing.
In all honesty, it’s never been this easy to build a contract reviewing tool. It’s no wonder then that so many new players are entering the market. How do you make sense of such a fragmented landscape? And are there even that many differences between these tools?
That’s a topic for next time!
Stay tuned if you are interested in a complete overview of what the market for AI contract reviewing tools has to offer and subscribe to our newsletter to make sure you don’t miss any updates!
