Podcast · 156 episodes on record

GPT-4o-mini changed ChatBotArena

Published

July 31, 2024

Length

7 min

Position

39 of 156

July 31, 2024

GPT-4o-mini changed ChatBotArena

0:00 / 7:55

0:00Choose a match or chapter to light up its moment7:55

Listen to the full episode

Excerpt playback isn’t available for this episode — use the full-episode link.

Search the Record

Moment search isn’t open for this episode yet

When it opens, you’ll be able to type any phrase and jump straight to the second it was said.

Nathan Lambert Episodes Around July 31, 2024

Episode 39 of 156 — walk the feed in the order it was published.

View all episodes

9:46

Phi 3 and Arctic: Outlier LMs are hints

Apr 30, 2024

13:08

RLHF: A thin line between useful and lobotomized

May 1, 2024

14:05

OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions

May 13, 2024

12:28

OpenAI chases Her

May 16, 2024

9:03

Name, image, and AI's likeness

May 22, 2024

8:29

We aren't running out of training data, we are running out of open training data

May 29, 2024

7:49

RLHF Roundup: Trying to get good at PPO, charting RLHF's impact, RewardBench retrospective, and a reward model competition

Jun 26, 2024

56:30

Interviewing Dean Ball on AI policy: CA SB 1047, upcoming AI disaster response, Llama 3 405B, Chinese open-source AI, and scaling laws

Jun 27, 2024

6:39

Switched to Claude 3.5

Jul 3, 2024

14:19

SB 1047, AI regulation, and unlikely allies for open models

Jul 17, 2024

15:22

Llama 3.1 405b, Meta's AI strategy, and the new open frontier model ecosystem

Jul 23, 2024

7:55This episode

GPT-4o-mini changed ChatBotArena

Jul 31, 2024

1:03:42

Interviewing Sebastian Raschka on the state of open LLMs, Llama 3.1, and AI education

Aug 1, 2024

10:23

A recipe for frontier model post-training

Aug 7, 2024

1:02:22

Interviewing Ross Taylor on LLM reasoning, Llama fine-tuning, Galactica, agents

Aug 8, 2024

8:32

Nous Hermes 3 and exploiting underspecified evaluations

Aug 16, 2024

8:00

On the current definitions of open-source AI and the state of the data commons

Aug 28, 2024

10:31

OLMoE and the hidden simplicity in training better foundation models

Sep 4, 2024

10:40

OpenAI's Strawberry, LM self-talk, inference scaling laws, and spending more on inference

Sep 5, 2024

5:38

A post-training approach to AI regulation with Model Specs

Sep 10, 2024

11:31

Futures of the data foundry business model

Sep 11, 2024

18:51

[Article Voiceover] Reverse engineering OpenAI's o1

Sep 17, 2024

14:04

[Article Voiceover] Llama 3.2 Vision and Molmo: Foundations for the multimodal open-source ecosystem

Sep 27, 2024

1:08:39

Interviewing Riley Goodside on the science of prompting

Sep 30, 2024

10:29

[Article Voiceover] AI Safety's Crux: Culture vs. Capitalism

Oct 2, 2024

11:47

How scaling changes model behavior

Oct 9, 2024

1:00:12

Interviewing Andrew Trask on how language models should store (and access) information

Oct 10, 2024

View all episodes

Summary

And how to understand Llama three point one's results. This is AI generated audio with Python and 11Labs. Source code: <a class="linkified" href="https://github.com/natolambert/interconnects-tools" target="_blank">https://github.com/natolambert/interconnects-tools</a> Original post: <a class="linkified" href="https://www.interconnects.ai/p/gpt-4o-mini-changed-chatbotarena" target="_blank">https://www.interconnects.ai/p/gpt-4o-mini-changed-chatbotarena</a>0:00 GPT-4o-mini changed ChatBotArena 3:23 Llama 3 in the arena 5:13 Partial solutions and next stepsFig 1: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_013.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_013.png</a> Fig 2: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_015.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_015.png</a> Fig 3: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_019.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_019.png</a> Fig 4: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_021.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_021.png</a> Fig 5: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_025.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_025.png</a> Fig 6: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_039.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_039.png</a> Fig 7: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_043.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/new-chatbotarena/img_043.png</a> This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit <a href="https://www.interconnects.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2">www.interconnects.ai/subscribe</a>

GPT-4o-mini changed ChatBotArena

Search the Record

Nathan Lambert Episodes Around July 31, 2024

Phi 3 and Arctic: Outlier LMs are hints

RLHF: A thin line between useful and lobotomized

OpenAI's Model (behavior) Spec, RLHF transparency, and personalization questions

OpenAI chases Her

Name, image, and AI's likeness

We aren't running out of training data, we are running out of open training data

A realistic path to robotic foundation models

AI for the rest of us

Text-to-video AI is already abundant

Frontiers in synthetic data

RLHF Roundup: Trying to get good at PPO, charting RLHF's impact, RewardBench retrospective, and a reward model competition

Interviewing Dean Ball on AI policy: CA SB 1047, upcoming AI disaster response, Llama 3 405B, Chinese open-source AI, and scaling laws

Switched to Claude 3.5

SB 1047, AI regulation, and unlikely allies for open models

Llama 3.1 405b, Meta's AI strategy, and the new open frontier model ecosystem

GPT-4o-mini changed ChatBotArena

Interviewing Sebastian Raschka on the state of open LLMs, Llama 3.1, and AI education

A recipe for frontier model post-training

Interviewing Ross Taylor on LLM reasoning, Llama fine-tuning, Galactica, agents

Nous Hermes 3 and exploiting underspecified evaluations

On the current definitions of open-source AI and the state of the data commons

OLMoE and the hidden simplicity in training better foundation models

OpenAI's Strawberry, LM self-talk, inference scaling laws, and spending more on inference

A post-training approach to AI regulation with Model Specs

Futures of the data foundry business model

[Article Voiceover] Reverse engineering OpenAI's o1

[Article Voiceover] Llama 3.2 Vision and Molmo: Foundations for the multimodal open-source ecosystem

Interviewing Riley Goodside on the science of prompting

[Article Voiceover] AI Safety's Crux: Culture vs. Capitalism

How scaling changes model behavior

Interviewing Andrew Trask on how language models should store (and access) information

Summary