(Voiceover) Building on evaluation quicksand
October 16, 2024
16 min
55 of 156
October 16, 2024
(Voiceover) Building on evaluation quicksand
0:00 / 16:36
Excerpt playback isn’t available for this episode — use the full-episode link.
Search the Record
Moment search isn’t open for this episode yet
When it opens, you’ll be able to type any phrase and jump straight to the second it was said.
Nathan Lambert Episodes Around October 16, 2024
Episode 55 of 156 — walk the feed in the order it was published.
16:36This episode(Voiceover) Building on evaluation quicksand
Summary
<p>Read the full post <a href="https://www.interconnects.ai/building-on-evaluation-quicksand" target="_blank">here</a>: https://www.interconnects.ai/p/building-on-evaluation-quicksand</p><p>Chapters</p><p>00:00 Building on evaluation quicksand</p><p>01:26 The causes of closed evaluation silos</p><p>06:35 The challenge facing open evaluation tools</p><p>10:47 Frontiers in evaluation</p><p>11:32 New types of synthetic data contamination</p><p>13:57 Building harder evaluations</p><p>Figures</p><p>Fig 1: <a href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/manual/openai-predictions.webp" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/manual/openai-predictions.webp</a></p> <br /><br />This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit <a href="https://www.interconnects.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2">www.interconnects.ai/subscribe</a>
Episode details and artwork are published by the show’s own feed. Sentinel plays short cited excerpts and links every episode back to its publisher.