The Jim Rutt Show
@jimruttshow·4.4K subscribers·597 videos
Currents 088: Melanie Mitchell on AI Measurement and Understanding
Posted
March 31, 2023
Views
508
Likes
6
Engagement
1.18%
Search the Record
IndexedEvery word spoken in this episode is indexed. Type any phrase to jump straight to the moment it was said.
Type any word or phrase that may have been spoken. Click a result to seek the player to that exact moment.
Try a name, a topic, or a quoted line
YouTube Description
as posted by the channelClick here for full show notes including episode mentions & recommendations!
Jim talks with Melanie Mitchell about her critique of applying standardized exams to LLMs and the debate over understanding in AI. They discuss ChatGPT and GPT-4's performance on standardized exams, questioning the underlying assumptions, OpenAI's lack of transparency, soon-to-be-released open-source LLMs, prompt engineering, making GPT its own skyhook to reduce hallucinations, the number of parameters in GPT-4, why LLMs should be probed differently than humans, how LLMs lie differently than humans, Stanford's holistic assessment for LLMs, a College Board for LLMs, why the term "understanding" is overstressed today, consciousness vs intelligence, the human drive for compression, working memory limitations as the secret to human intellectual abilities, episodic memory, embodied emotions, the idea that AIs don't care, calling for a new science of intelligence, the effects of differing evolutionary pressures, whether a model of physics could emerge from language learning, how little we understand these systems, and much more.
Links & Promotions
Guests & Subjects Covered
Currents 088Melanie MitchellAI TheyCollege Board
Sentinel Indexing in Progress
Metadata and chapters are available. Claim extraction for this episode is pending.
All video content is delivered via YouTube embedded players in accordance with the YouTube Terms of Service. Sentinel provides research tools that promote discovery and accountability across political media.