Why a 1B Model Beat a 70B LLM at Following the Rules

👉 Discover how Cambridge Spark helps organisations build the data and AI capabilities needed to turn strategy into measurable impact: [cambridgespark.com](https://www.cambridgespark.com/?utm_campaign=6955829-Podcast%202024%20-%20Onwards&utm_source=Podcast&utm_medium=RSS) An LLM can give well-formed output 99% of the time and still break the domain's rules in most of it. You want to put an LLM into a real decision process, such as invoice approvals, pricing or clinical recommendations. The answers read well. But are they actually allowed, and do they solve the problem? Professor Alessandra Russo is Professor in Applied Computational Logic at Imperial College London and Director of the Thomson Reuters Imperial Frontier AI Research Lab. She co-directs the UKRI Centre for Doctoral Training in Safe and Trusted AI with King's College London. Her neuro-symbolic AI research combines logic with machine learning to build AI systems with functional guarantees. Jeremy Bradley and Alessandra go inside SEM-CTRL, her team's method for semantically controlled decoding in open weight models. You'll hear how a symbolic checker sits in the decoder and masks invalid tokens before they are ever chosen. Monte Carlo tree search then steers generation towards correct answers. No fine-tuning required. This episode is for data scientists, ML engineers and technical leaders responsible for enterprise AI deployment in regulated or high-stakes settings. It assumes some comfort with how LLMs generate tokens. If you want a light overview of AI trends, this one goes further. ## Chapter Markers 00:00 Why AI should be able to say "I don't know" 00:19 Introducing Professor Alessandra Russo 01:32 What organisations worry about beyond hallucination 04:30 Open weight models versus frontier APIs 08:37 The risks that come with open weights 11:16 Inside the SE-CTRL paper 12:39 Validity versus correctness in the blocks world 16:46 A pricing guardrail that needs memory 19:48 Context-sensitive grammars and answer set grammars 24:39 Where the checker sits in the decoder 28:00 Masking tokens before softmax, and temperature 30:20 Monte Carlo tree search for correct solutions 39:01 Where reward functions come from 40:18 Why a 1B model outperformed a 70B model 46:29 Who writes the grammar: ILASP and LLMs 52:47 Limits: fairness, noise and robotics 55:23 What an NHS trust should do first 59:07 Where to find the research ## Useful Links & Resources - SEM-CTRL: Semantically Controlled Decoding: https://doi.org/10.48550/arXiv.2503.01804 How LLMs and agents can be used as robust decision agents in complex decision problems, where hard safeguards can be put in place to prevent unsafe outcomes and reinforcement learning can be used to iterate towards better solutions. - Answer Set Grammars (AAAI 2019) - ILASP: The complexity and generality of learning answer set programs: https://doi.org/10.1016/j.artint.2018.03.005 How operational examples can be used to learn logical specifications for AI tools. - Thomson Reuters Imperial Frontier AI Research Lab - Cambridge Spark: https://www.cambridgespark.com ## Connect With the Show - Jeremy Bradley on LinkedIn: https://www.linkedin.com/in/jeremy-bradley/ - Cambridge Spark on LinkedIn: https://www.linkedin.com/school/cambridge-spark/ Subscribe to Inside the Algorithm so you never miss an episode. Visit cambridgespark.com to find out how Cambridge Spark can upskill your workforce in data and AI. #InsideTheAlgorithm #EnterpriseAI #NeuroSymbolicAI #MachineLearning #AISafety