Meta just open-sourced a Github repo for LLM Training.
Meta Lingua is a minimal and fast LLM training and inference library designed for research
📊 Key features
- Minimal and fast LLM training/inference library for research
- Uses modifiable PyTorch components for experimenting with architectures, losses, data
- Enables end-to-end training, inference, evaluation
- Provides tools for understanding speed and stability
- Structured with core 'lingua' library and 'apps' to showcase usage
🚀 Lingua's performance comparison to other models
- 1B models trained on 60B tokens match DCLM (DataComp-LM) baseline performance on many tasks
- 7B models (Mamba, Llama) show strong results on benchmarks like ARC, MMLU, BBH
- Llama 7B squared ReLU 1T tokens model achieves high scores across tasks
https://github.com/facebookresearch/lingua#machinelearning #deeplearning #artificialintelligence
#AI #LLM #ML #Machine_Learning
🆔
@Ai_Tv