Articles and thoughts on AI, Machine Learning, and Software Engineering
Research Engineer at Hugging Face · Scaling RL Environments in the Open
I'm a Research Engineer at Hugging Face, working on scaling reinforcement learning environments in the open: the infrastructure, environments, and training recipes that let open models learn across coding, computer use, simulation, multilingual, and scientific domains. Before this, I founded CognitiveLab, the open-source lab behind Ambari-7B and Nayana (backed by Meta's LLaMA Impact Grant), and built agentic memory and real-time agents at Microsoft Research and knowledge-graph retrieval at Apple. My work has appeared at ICCV, NAACL, AAAI, NeurIPS, and CHI venues, and my open-source projects have 15k+ GitHub stars.
Real-world simulation RL environments for AI agents, from port dock planning to interactive 3D evaluation of every model rollout.
From an idea to a trained 4B model, with the dead ends left in: building a GeoGuesser RL environment end to end.
Train open models with RL inside real agent harnesses, running the same environments across TRL, Harbor, and OpenReward.
RL environments and RL training for LLMs, presented at AMD AI Dev Day.
OpenEnv x Harbor: trainable tokens from any coding agent.
From 'What is an env?' to training your own: the original 30-minute talk on RL environments.
A comprehensive survey and reproduction of modern RL environment frameworks, with 25k+ reads and 150k+ impressions.
A novel multi-agent system that redefines how we generate and refine visual content, presented at the Multi-Agent Workshops at AAAI 2025.
The technical journey of creating a performant bilingual LLM for low-resource languages with limited training data.
Deep dive into the underlying architecture of LLama3 and the structural changes between 7B and 8B parameter models.
A low-code guide to fine-tuning LLMs efficiently using the Axolotl library.
A comprehensive guide to fine-tuning Google's Gemma model with practical examples and best practices.
Unleashing the power of Mixtral: A comprehensive guide to fine-tuning this powerful mixture-of-experts model.
A practical guide and colab notebook to quantize LLMs in GGUF format to run them efficiently on your local machine.
A guide to quantizing LLMs using Activation-Aware Quantization (AWQ) on a Google Colab notebook for optimal performance.
A step-by-step guide to deploying Mistral or Llama models on AWS in just three simple stages.
Fine-tuning the Mistral 7B model for code generation using a single Google Colab notebook.
A journey into fine-tuning LLama2 to create an AI companion with personality and conversational abilities.