OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions
May 13, 2024
14 min
26 of 156
May 13, 2024
OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions
0:00 / 14:05
Excerpt playback isn’t available for this episode — use the full-episode link.
Search the Record
Moment search isn’t open for this episode yet
When it opens, you’ll be able to type any phrase and jump straight to the second it was said.
Nathan Lambert Episodes Around May 13, 2024
Episode 26 of 156 — walk the feed in the order it was published.
14:05This episodeOpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions
Summary
<p>Now we will have some grounding for when weird ChatGPT behaviors are intended or side-effects -- shrinking the Overton window of RLHF bugs.<br />This is AI generated audio with Python and 11Labs.<br />Source code: <a class="linkified" href="https://github.com/natolambert/interconnects-tools" target="_blank">https://github.com/natolambert/interconnects-tools</a><br />Original post: <a class="linkified" href="https://www.interconnects.ai/p/openai-rlhf-model-spec" target="_blank">https://www.interconnects.ai/p/openai-rlhf-model-spec</a></p><p>00:00 OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions<br />02:56 Reviewing the Model Spec<br />08:26 Where RLHF can fail OpenAI<br />12:23 From Model Spec's to personalization</p><p>Fig 1: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_027.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_027.png</a><br />Fig 2: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_029.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_029.png</a><br />Fig 3: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_033.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_033.png</a><br />Fig 4: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_034.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_034.png</a><br />Fig 5: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_041.webp" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_041.webp</a><br />Fig 6: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_046.webp" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-spec/img_046.webp</a></p> <br /><br />This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit <a href="https://www.interconnects.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2">www.interconnects.ai/subscribe</a>
Episode details and artwork are published by the show’s own feed. Sentinel plays short cited excerpts and links every episode back to its publisher.