Concept · Feb 2024
Finetuning Base Model Methods
- LoRa (Low Rank Adaptation) - a machine learning technique that modifies a retrained model to better suit a specific, usually smaller dataset by adjusting only a small low rank subset of the model’s parameters (usually from the later layers)
- Dora (Weight Decomposed low-rank adaptation) - a recently proposed alternative to LoRa