Beyond Accuracy: Evaluating the learned representations of Generative AI models | Aida Nematzadeh
OCT 23, 202553 MIN
Beyond Accuracy: Evaluating the learned representations of Generative AI models | Aida Nematzadeh
OCT 23, 202553 MIN
Description
<p>Dr. Aida Nematzadeh is a Senior Staff Research Scientist at Google DeepMind where her research focused on multimodal AI models. She works on developing evaluation methods and analyze model’s learning abilities to detect failure modes and guide improvements. Before joining DeepMind, she was a postdoctoral researcher at UC Berkeley and completed her PhD and Masters in Computer Science from the University of Toronto. During her graduate studies she studied how children learn semantic information through computational (cognitive) modeling. Time stamps of the conversation<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0">00:00</a> Highlights<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=80s">01:20</a> Introduction<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=128s">02:08</a> Entry point in AI<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=184s">03:04</a> Background in Cognitive Science & Computer Science <a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=295s">04:55</a> Research at Google DeepMind<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=347s">05:47</a> Importance of language-vision in AI<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=636s">10:36</a> Impact of architecture vs. data on performance <a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=786s">13:06</a> Transformer architecture <a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=870s">14:30</a> Evaluating AI models<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=1142s">19:02</a> Can LLMs understand numerical concepts <a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=1480s">24:40</a> Theory-of-mind in AI<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=1678s">27:58</a> Do LLMs learn theory of mind?<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=1765s">29:25</a> LLMs as judge<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=2156s">35:56</a> Publish vs. perish culture in AI research<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=2400s">40:00</a> Working at Google DeepMind<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=2570s">42:50</a> Doing a Ph.D. vs not in AI (at least in 2025)<a href="https://www.youtube.com/watch?v=gYqr1mGfzE0&t=2900s">48:20</a> Looking back on research careerMore about Aida: <a href="https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbjRLWUh4UWFHZVpBaDlwMWM3b1VrMHAzSFI0UXxBQ3Jtc0tuTHVIWGU3REdBNWJkSGl3dXh6WndyT3g2M2xuUll4enBibVI0SnEzbXltNkxwbFdtY2JSQnoyZXZBR2k3SkhhLWJIUUVzWlVCNnBQc0doaGEyVGNUOGdSWVh3LUpVSW9WNjFLeURoX1ZTR3ctZnpxQQ&q=http%3A%2F%2Fwww.aidanematzadeh.me%2F&v=gYqr1mGfzE0" target="_blank" rel="nofollow">http://www.aidanematzadeh.me/</a>About the Host:Jay is a Machine Learning Engineer at PathAI working on improving AI for medical diagnosis and prognosis. Linkedin: <a href="https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqblMzNnQ1YW9ZcVVDWVFmbXpsVlZ0MWpRRWJiQXxBQ3Jtc0trcjY3aXduQ1Bha0hvVHJJNzZTS2ZxdklWb0pIZG9XbTdaa2tTY1p2bGFlc2plTm1sc09MX0xUTHpCTUhpVGZlZTVZbUVTWk5BSk1Cc25JVmtWQ0FMMHI2UkExOXpNeTBETWhtVnc3LTFsWXJHRktfRQ&q=https%3A%2F%2Fwww.linkedin.com%2Fin%2Fshahjay22%2F&v=gYqr1mGfzE0" target="_blank" rel="nofollow">shahjay22 </a>Twitter: <a href="https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqa1l1MmJwYVZjY2NwX2NOYW1MT21tbWtaVlhuZ3xBQ3Jtc0tuY1ZiU3hUTmQ1QlhyMnlOaE90dEh6M2FtbUZFSmQ2QlRxTlB4MjVJaVdkZVAzWndDSXpVeDZPdzllMlZtUjBvOTVaUHJwQm15VmxvcHFOT19HbVB3MnowYTVMWjlEZ3dleDhQck1vZnZzZGhYYVJ5Yw&q=https%3A%2F%2Ftwitter.com%2Fjaygshah22&v=gYqr1mGfzE0" target="_blank" rel="nofollow"> jaygshah22 </a>Homepage: <a href="https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbWM2WUljMGQwUTZzZmhIYTRybzRlWWlDQ2hGd3xBQ3Jtc0trdWNtM1FkYkxEaE5ZRlIwRzh0WmQ5RFRtS0UyNmwxNkVkcTYxZFV4aHl2bE9wbVRDRUtyNnhac0ppNmZDM2xreDlsTG55WXQyS21BRHNjbmdNZlNKQWlrMmc4c0gxR081b0VaNTU1UGFCSlBrVG12Zw&q=https%3A%2F%2Fjaygshah.github.io%2F&v=gYqr1mGfzE0" target="_blank" rel="nofollow">https://jaygshah.github.io/</a> for any queries.Stay tuned for upcoming webinars!<strong>**Disclaimer: The information in this video represents the views and opinions of the speaker and does not necessarily represent the views or opinions of any institution. It does not constitute an endorsement by any Institution or its affiliates of such video content.**</strong></p>