[Article Voiceover] Reverse engineering OpenAI's o1

<p>What productionizing test-time compute shows us about the future of AI. Exploration has landed in language model training.<br />This is AI generated audio with Python and 11Labs.<br />Source code: <a class="linkified" href="https://github.com/natolambert/interconnects-tools" target="_blank">https://github.com/natolambert/interconnects-tools</a><br />Original post: <a class="linkified" href="https://www.interconnects.ai/p/reverse-engineering-openai-o1" target="_blank">https://www.interconnects.ai/p/reverse-engineering-openai-o1</a></p><p>00:00 Reverse engineering OpenAI's o1<br />01:52 From Q-star to Strawberry to o1<br />05:13 Training o1 with reinforcement learning<br />09:24 What is o1 doing when given a prompt?<br />11:49 Questions to consider to understand o1's structure<br />11:56 1. How does an RL-trained language model act?<br />12:38 2. Is it an online / test-time search?<br />14:20 3. Is it one model at inference?<br />15:29 Open-source o1, the future of o1, and the future of AI</p><p>Fig 1: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_014.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_014.png</a><br />Fig 2: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_016.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_016.png</a><br />Fig 3: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_018.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_018.png</a><br />Fig 4: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_020.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_020.png</a><br />Fig 5: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_024.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_024.png</a><br />Fig 6: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_026.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_026.png</a><br />Fig 7: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_034.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_034.png</a><br />Fig 8: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_048.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_048.png</a></p> <br /><br />This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit <a href="https://www.interconnects.ai/subscribe?utm_medium=podcast&#38;utm_campaign=CTA_2">www.interconnects.ai/subscribe</a>