Building LLMs from the Ground Up: A 3-hour Coding Workshop
August 31, 2024
166,461
5,130
166
3.18%
Search the Record
IndexedEvery word spoken in this episode is indexed. Type any phrase to jump straight to the moment it was said.
Type any word or phrase that may have been spoken. Click a result to seek the player to that exact moment.
Try a name, a topic, or a quoted line
Sebastian Raschka Episodes Around August 31, 2024
See what was published immediately before and after this episode.
2:45:10Now PlayingBuilding LLMs from the Ground Up: A 3-hour Coding Workshop
Chapters
YouTube Description
as posted by the channelREFERENCES:
1. Build an LLM from Scratch book
2. Build an LLM from Scratch repo
3. GitHub repository with workshop code
4. Lightning Studio for this workshop
DESCRIPTION:
This tutorial is aimed at coders interested in understanding the building blocks of large language models (LLMs), how LLMs work, and how to code them from the ground up in PyTorch. We will kick off this tutorial with an introduction to LLMs, recent milestones, and their use cases. Then, we will code a small GPT-like LLM, including its data input pipeline, core architecture components, and pretraining code ourselves. After understanding how everything fits together and how to pretrain an LLM, we will learn how to load pretrained weights and finetune LLMs using open-source libraries.
---
To support this channel, please consider purchasing a copy of my books
---
---
OUTLINE:
Guests & Subjects Covered
Sentinel Indexing in Progress
Metadata and chapters are available. Claim extraction for this episode is pending.
All video content is delivered via YouTube embedded players in accordance with the YouTube Terms of Service. Sentinel provides research tools that promote discovery and accountability across political media.