BigCode

Team

non-profit

https://www.bigcode-project.org/

BigCodeProject

bigcode-project

Activity Feed

AI & ML interests

None defined yet.

Recent Activity

joelniklaus authored a paper about 1 month ago

Synthetic Pre-pretraining Survives Scale, but Not as a Grammatical Prior

shannons authored a paper about 1 month ago

Context Language Models

infinitylogesh authored a paper about 2 months ago

Ambient @ EgoProactive 2026 : Proactive Egocentric Assistance with Visually Grounded Supervision

View all activity

Papers

BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution

BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions

View all Papers

Articles

BigCodeArena: Judging code generations end to end with code executions

Oct 7, 2025

• 22

sted97

authored a paper 1 day ago

RouterWorld: Scaling Conversational AI with Latency-Aware LLM Routing

Paper • 2610.26511 • Published 16 days ago • 1

loubnabnl

posted an update 3 days ago

Post

4,120

𝐈𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐢𝐧𝐠 🧬 𝐅𝐢𝐧𝐞𝐃𝐍𝐀: 𝐭𝐡𝐞 𝐛𝐞𝐬𝐭 𝐝𝐚𝐭𝐚𝐬𝐞𝐭 𝐭𝐨 𝐭𝐞𝐚𝐜𝐡 𝐚𝐧𝐲 𝐥𝐚𝐧𝐠𝐮𝐚𝐠𝐞 𝐦𝐨𝐝𝐞𝐥 𝐃𝐍𝐀.
Training Qwen3-14B on it produces Carbon-X, the strongest open model on DNA benchmarks, on par with GPT-6.1 Sol.

It contains:
- Midtraining: 300B+ tokens of genomic sequences, papers and web pages mixing DNA with text
- SFT: 200K samples, from variant effect prediction to sequence question answering
- RL: 98K prompts with verifiable rewards

Data, models and the full recipe are open.
HuggingFaceBio/FineDNA

Enjoy this late Halloween treat! 🎃

4 replies

albertvillanova

posted an update 6 days ago

Post

184

Day-0 Mistral Large 4 support in TRL 🚀

- LoRA SFT of the 1T-A52B model on one 8xB300 node, on top of the FP8 checkpoint
- Experts split 8 ways with expert parallelism, everything else sharded with FSDP, all set at load time
- Assistant-only loss out of the box with its training chat template
- ~32.7k tokens per step in ~8 s

Script + Slurm launcher 👇
https://github.com/huggingface/trl/tree/main/examples/sft_mistral_large_4
https://huggingface.co/posts/sergiopaniego/408182860646834