Support us! https://www.patreon.com/mlst MLST Discord: https://discord.gg/aNPkGUQtc5 Dr. Raphaël Millière is the 2020 Robert A. Burt Presidential Scholar in Society and Neuroscience in the Center for Science and Society, and a Lecturer in the Philosophy Department at Columbia University. His research draws from his expertise in philosophy and cognitive science to explore the implications of recent progress in deep learning for models of human cognition, as well as various issues in ethics and aesthetics. He is also investigating what underlies the capacity to represent oneself as oneself at a fundamental level, in humans and non-human animals; as well as the role that self-representation plays in perception, action, and memory. In a world where technology is rapidly advancing, Dr. Millière is striving to gain a better understanding of how artificial neural networks work, and to establish fair and meaningful comparisons between humans and machines in various domains in order to shed light on the implications of artificial intelligence for our lives. https://www.raphaelmilliere.com/ https://twitter.com/raphaelmilliere TOC: Intro to Raphael - 00:00:00 Raphaël Article - Moving Beyond Mimicry in Artificial Intelligence - 00:01:18 Show Kick off - 00:06:55 LLMs - 00:08:22 Semantic Competence/Understanding - 00:18:13 Forming Analogies/JPG Compression Article - 00:30:02 Compositional Generalisation - 00:37:13 Systematicity - 00:46:53 Language of Thought - 00:51:13 Bigbench (Conceptual Combinations) - 00:57:22 Symbol Grounding - 01:10:58 World Models - 01:26:28 Theory of Mind - 01:30:42 Refs: On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜 (Bender et al) https://dl.acm.org/doi/10.1145/3442188.3445922 The Debate Over Understanding in AI's Large Language Models (Melanie Mitchell) https://arxiv.org/abs/2210.13966 Talking About Large Language Models (Murray Shanahan) https://arxiv.org/abs/2212.03551 Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data (Bender) https://aclanthology.org/2020.acl-main.463/ Winograd SHRLDU https://en.wikipedia.org/wiki/SHRDLU Why the Abstraction and Reasoning Corpus is interesting and important for AI (Mitchell) https://aiguide.substack.com/p/why-the-abstraction-and-reasoning Learning in High Dimension Always Amounts to Extrapolation (Randall Balestriero / LeCun) https://arxiv.org/abs/2110.09485 ChatGPT Is a Blurry JPEG of the Web (Ted Chiang) https://www.newyorker.com/tech/annals-of-technology/chatgpt-is-a-blurry-jpeg-of-the-web Compression for AGI - Jack Rae (OpenAI) | Stanford MLSys #76 https://www.youtube.com/watch?v=dO4TPJkeaaU Shortcut Learning in Deep Neural Networks (Robert Geirhos et al) https://arxiv.org/abs/2004.07780 Stable Diffusion Based Image Compression (Matthias Bühlmann) https://pub.towardsai.net/stable-diffusion-based-image-compresssion-6f1f0a399202 Word meaning in minds and machines (Conceptual combinations) https://arxiv.org/abs/2008.01766 SCAN tasks for compositional learning https://github.com/brendenlake/SCAN Connectionism and Cognitive Architecture: A Critical Analysis (Fodor/Pylyshyn) https://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf BigBench (Conceptual combinations) https://github.com/google/BIG-bench/tree/main/bigbench/benchmark_tasks/conceptual_combinations The symbol grounding problem (Stevan Harnad) https://arxiv.org/html/cs/9906002 Linguistics Meets Philosophy: A Historical Preface (Barbara H. Partee) https://www.cambridge.org/core/books/abs/linguistics-meets-philosophy/linguistics-meets-philosophy-a-historical-preface/90B7E1525F887B280AC7B70F939A5E44 Mapping Language Models to Grounded Conceptual Spaces (Roma Patel, Ellie Pavlick) https://openreview.net/forum?id=gJcEM8sxHK Linguistic relativity (Sapir–Whorf hypothesis) https://en.wikipedia.org/wiki/Linguistic_relativity Do Language Models Know the Way to Rome? (Bastien Liétard et al) https://arxiv.org/abs/2109.07971 Do As I Can, Not As I Say - SayCan (Michael Ahn et al) https://say-can.github.io/ Do Large Language Models learn world models or just surface statistics? [Othello Game] (Kenneth Li) https://thegradient.pub/othello/ Computational Language Acquisition with Theory of Mind (Andy Liu) https://arxiv.org/abs/2303.01502 Language Models as Agent Models (Jacob Andreas) https://arxiv.org/abs/2212.01681 Moving Beyond Mimicry in Artificial Intelligence (Raphael Millière) https://nautil.us/moving-beyond-mimicry-in-artificial-intelligence-238504/ Simulators (Janus) https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators Cooperative principle (Grice's four maxims of conversation - quantity, quality, relation, and manner) https://en.wikipedia.org/wiki/Cooperative_principle Alex (parrot) https://en.wikipedia.org/wiki/Alex_(parrot)