Published on July 30, 2024, this example applies LoRA fine-tuning to a simple classification MLP and compares performance with adapters enabled and disabled.
Published on July 30, 2024, the example demonstrates LoRA fine-tuning on a 5-million-parameter MLP for classification. The implementation freezes the original weights and trains about 9,000 adapter parameters on a class of labels.
The exercise compares model performance with the adapter enabled and disabled, giving engineers a compact example of applying adapters and evaluating their effect. To check the details and results, consult the original publication and verify the configurations, data, and comparison method it describes.