Insights from Finetuning LLMs with Low-Rank Adaptation
December 17, 2023
11,336
356
20
3.32%
Search the Record
IndexedEvery word spoken in this episode is indexed. Type any phrase to jump straight to the moment it was said.
Type any word or phrase that may have been spoken. Click a result to seek the player to that exact moment.
Try a name, a topic, or a quoted line
Sebastian Raschka Episodes Around December 17, 2023
See what was published immediately before and after this episode.
13:49Now PlayingInsights from Finetuning LLMs with Low-Rank Adaptation
YouTube Description
as posted by the channelSebastian's books
LoRALowRank Adaptation of Large Language Models,
LitGPT LoRA Tutorial
Low-rank adaptation (LoRA) stands as one of the most popular and effective methods for efficiently training custom Large Language Models (LLMs). As practitioners of open-source LLMs, we regard LoRA as a crucial technique in our toolkit.
In this talk, I will delve into some practical insights gained from running hundreds of experiments with LoRA, addressing questions such as: How much can I save with quantized LoRA? Are Adam optimizers memory-intensive? Should we train for multiple epochs? How do we choose the LoRA rank?
---
To support this channel, please consider purchasing a copy of my books
---
Guests & Subjects Covered
Sentinel Indexing in Progress
Metadata and chapters are available. Claim extraction for this episode is pending.
All video content is delivered via YouTube embedded players in accordance with the YouTube Terms of Service. Sentinel provides research tools that promote discovery and accountability across political media.