Journal DOI 10.68956/journal.legal-engineering-technology

Creative Commons source reading · Legal language models

Enhancing Contract Negotiations with LLM-Based Legal Document Comparison

Savinay Narendra; Kaushal Shetty; Adwait Ratnaparkhi

Originally published in Proceedings of the Natural Legal Language Processing Workshop 2024 · 2024-11

From the article

Selected source text. JournalOS removes markup and shortens excerpts where indicated. The full article provides the complete context.

Abstract

We present a large language model (LLM) based approach for comparing legal contracts with their corresponding template documents. Legal professionals use commonly observed deviations between templates and contracts to help with contract negotiations, and also to refine the template documents. Our comparison approach, based on the well-studied natural language inference (NLI) task, first splits a template into key concepts and then uses LLMs to decide if the concepts are entailed by the contract document. We also repeat this procedure in the opposite direction - contract clauses are tested for entailment against the template clause to see if they contain additional information. The non-entailed concepts are labelled, organized and filtered by frequency, and placed into a clause library, which is used to suggest changes to the template documents. We first show that our LLM-based approach outperforms all previous work on a publicly available dataset designed for NLI in the legal domain. We then apply it to a private real-world legal dataset, achieve an accuracy of 96.46%.

Editorial reading guide

  • Which legal corpus, jurisdiction and evaluation task does the article use?
  • What evidence supports the reported performance, and where could the method fail in practice?