<p>What does it actually mean for a model to understand audio</p><p><br></p><p>Paper: <a href="https://arxiv.org/abs/2601.19673" target="_blank" rel="noopener noreferer">https://arxiv.org/abs/2601.19673</a></p><p><br></p><p>In this episode, I talk with Iwona Christop, a PhD student at Adam Mickiewicz University, about her recent EACL paper introducing ART (Audio Reasoning Tasks) — a new benchmark designed to evaluate whether multimodal LLMs can truly reason over audio, not just transcribe or classify it.</p><p><br></p><p>Most existing benchmarks test audio skills in isolation (like ASR or classification). But real-world intelligence requires something deeper: combining signals, comparing sounds, tracking context, and making decisions.</p><p><br></p><p>This work takes a different approach:</p><ul><li>No text-only shortcuts — tasks can’t be solved via transcription alone</li><li>Reasoning-first design — models must combine multiple audio cues</li><li>No expert knowledge required — anyone can verify correctness</li></ul><p><br></p><p>We also dive into the diverse task design, including:</p><ul><li>Audio arithmetic (counting and comparing sounds)</li><li>Cross-recording speaker &amp; language identification</li><li>Sound-based reasoning (e.g., inferring properties from audio)</li><li>Speech feature comparison (accents, variations)</li><li>Multimodal reasoning across text and sound</li></ul><p><br></p><p>The dataset includes 9 tasks, 9,000 samples, and 30+ hours of audio — all generated in a scalable way using templates and TTS.</p><p><br></p><p>👉 If you care about multimodal reasoning, evaluation, or the limits of current LLM capabilities, this conversation is for you.</p><p><br></p><p>Iwona Christop:</p><p><a href="https://www.linkedin.com/in/iwona-christop/" target="_blank" rel="noopener noreferer">https://www.linkedin.com/in/iwona-christop/</a></p><p><br></p><p>👍 Like &amp; subscribe for more deep dives into cutting-edge AI research</p><p>🔔 New episodes from EACL 2026 coming soon</p><p><br></p><p>#WiAIR #EACL2026</p>