avatar

Lighton AI Research

Welcome to LAIR 🐲
We research massive randomized AI systems 🧠

PAGnol: An Extra-Large French πŸ‡«πŸ‡· Generative Model

Using scaling laws, we efficiently train and publicly release four models, with up to 1.5B parameters for PAGnol-XL. At the time of release, this is the largest non-sparse French language model available. (Not published)

Direct Feedback Alignment scales πŸ“ˆ to modern deep learning tasks and architectures

At variance with previous beliefs, we show that an alternative to backpropagation, devoid of weight transport, can scale to modern deep learning tasks and architectures. (NeurIPS 2020)