FineDNA: A 300B+ Token Genomic Dataset for Training DNA Language Models

Key Info

FineDNA is introduced as a dataset for teaching any language model DNA, comprising over 300B tokens of genomic sequences, papers and web pages that mix DNA and text, along with 200k SFT and 98k RL samples. Training Qwen3-14B on it is said to produce the strongest open model.

Highlights

• 300B+ tokens of genomic sequences, papers and web pages mixing DNA and text
• Includes 200k supervised fine-tuning (SFT) samples
• Includes 98k reinforcement learning (RL) samples
• Training Qwen3-14B on FineDNA reportedly yields the strongest open model
• Positioned as a dataset for teaching any language model DNA