Journal DOI 10.68956/journal.legal-engineering-technology

Creative Commons source reading · Evaluation, bias and explainability

Evaluating LLM-Generated Legal Explanations for Regulatory Compliance in Social Media Influencer Marketing

Haoyang Gui; Thales Bertaglia; Taylor Annabell; Catalina Goanta; Tjomme Dooper; Gerasimos Spanakis

Originally published in Proceedings of the Natural Legal Language Processing Workshop 2025 · 2025-11

From the article

Selected source text. JournalOS removes markup and shortens excerpts where indicated. The full article provides the complete context.

Abstract

The rise of influencer marketing has blurred boundaries between organic content and sponsored content, making the enforcement of legal rules relating to transparency challenging. Effective regulation requires applying legal knowledge with a clear purpose and reason, yet current detection methods of undisclosed sponsored content generally lack legal grounding or operate as opaque “black boxes.” Using 1,143 Instagram posts, we compare gpt-5-nano and gemini-2.5-flash-lite under three prompting strategies with controlled levels of legal knowledge provided. Both models perform strongly in classifying content as sponsored or not (F1 up to 0.93), with Gemini favouring recall (0.93) and GPT favouring precision (0.95), though performance drops by over 10 points on ambiguous cases. We further develop a taxonomy of reasoning errors, showing frequent citation omissions (28.57%), unclear references (20.71%), and hidden ads exhibiting the highest miscue rate (28.57%). While adding regulatory text to the prompt improves explanation quality, it does not consistently improve detection accuracy.

Editorial reading guide

  • Which legal corpus, jurisdiction and evaluation task does the article use?
  • What evidence supports the reported performance, and where could the method fail in practice?