BixBench3

Benchmarking AI agents on research-study-scale computational biology tasks

BixBench3 evaluates the ability of AI agents to carry out analyses on the scale of complete research studies. Each of the 20 tasks mirrors how scientists often use agents today: the scientist sets the research objective and a general methodological plan, then delegates implementation to the agent. Each task is based on a published computational biology paper. The agent must build a working analysis pipeline that analyzes the paper's raw data and produce structured artifacts supporting its findings. Those 138 artifacts – such as protein abundance matrices or tables of differentially expressed genes – are graded programmatically against reference artifacts from the published study, measuring how well the agent executed the required analyses.