Google ships it: Gemma open LLMs and Gemini backlash
February 22, 2024
17 min
12 of 156
February 22, 2024
Google ships it: Gemma open LLMs and Gemini backlash
0:00 / 17:16
Excerpt playback isn’t available for this episode — use the full-episode link.
Search the Record
Moment search isn’t open for this episode yet
When it opens, you’ll be able to type any phrase and jump straight to the second it was said.
Nathan Lambert Episodes Around February 22, 2024
Episode 12 of 156 — walk the feed in the order it was published.
17:16This episodeGoogle ships it: Gemma open LLMs and Gemini backlash
Summary
<p>Google rejoins the open model party and gets some backlash for a frequent problem for generative AI.<br />This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.<br />Source code: <a class="linkified" href="https://github.com/natolambert/interconnects-tools" target="_blank">https://github.com/natolambert/interconnects-tools</a><br />Original post: <a class="linkified" href="https://www.interconnects.ai/p/gemma-google-ships-it" target="_blank">https://www.interconnects.ai/p/gemma-google-ships-it</a></p><p>00:00 Google ships it: Gemma open LLMs and Gemini backlash<br />03:12 Getting to know Gemma<br />07:11 Alignment details<br />08:55 Aside: What is REINFORCE? Some history of RL<br />11:08 Implementation details and RLHF<br />12:18 Terms of use: RAIL Licenses history repeated<br />14:05 Is Google back on top? Gemini's woes</p><p>Figure 1: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_008.webp" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_008.webp</a><br />Figure 2: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_014.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_014.png</a><br />Figure 3: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_035.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_035.png</a><br />Figure 4: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_051.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_051.png</a><br />Figure 5: <a class="linkified" href="https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_055.png" target="_blank">https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_055.png</a></p> <br /><br />This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit <a href="https://www.interconnects.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2">www.interconnects.ai/subscribe</a>
Episode details and artwork are published by the show’s own feed. Sentinel plays short cited excerpts and links every episode back to its publisher.