models

OpenAI's MedChemBench Tests AI Models on Drug Discovery Reasoning

Summarized by AI from reporting by OpenAI Blog, published under our editorial policy.

OpenAI's MedChemBench benchmark evaluates AI models on interpreting molecular structures and predicting compound properties in early drug discovery. GPT-6.2 Astra scored 84.5% at the highest reasoning setting, showing promise for accelerating drug development.

A molecular structure diagram with AI-generated predictions, illustrating MedChemBench's evaluation of AI models in drug discovery.

Key takeaways

  • MedChemBench evaluates AI models on reasoning tasks drawn from medicinal chemistry research.
  • GPT-6.2 Astra earned 84.5% of the available credit at the highest reasoning setting.
  • The benchmark tests four components: molecular structures, scientific reasoning, ranking and selection, and property predictions.
  • MedChemBench draws on 117 papers and patents and includes 761 distinct molecular structures.
  • GPT-6 Astra reconstructed an 18-atom ring molecule correctly in all ten responses, while GPT-5.6 Sol and GPT-5.6 Luna failed across all ten responses.

OpenAI introduced MedChemBench, a new benchmark that evaluates how well AI models can interpret molecular structures, reason from experimental evidence, and select compounds in early drug discovery. The benchmark tests whether models can connect a molecule's structure with experimental evidence to explain results, compare compounds, and predict their properties — a crucial capability for developing better medicines.

What MedChemBench Measures

MedChemBench measures these abilities through 99 problems drawn from published research. At the highest reasoning setting (xhigh), GPT-6.2 Astra earned 84.5% of the available credit, compared with 83.3% for GPT-6 Astra, 82.0% for GPT-6.1 Sol, and 58.2% for GPT-5.6 Sol. The benchmark evaluates four components: molecular structures, scientific reasoning, ranking and selection, and property predictions. For most problems, models receive the context available at a chosen point in a research program, including the biological target, the researchers' goals, the compounds studied so far, and selected experimental findings.

The benchmark draws on 117 papers and patents and includes 761 distinct molecular structures. One of the hardest parts of medicinal chemistry is that improving one property of a molecule can make another worse. A compound might act strongly against a disease-related protein but be poorly absorbed or break down too quickly in the body. Medicinal chemists work through these tradeoffs by designing, making, and testing related compounds, then deciding what to try next. For AI to help, it needs to connect changes in molecular structure with experimental evidence and reason about what compounds are worth pursuing.

Model Performance on Molecular Reasoning Tasks

GPT-6 Astra was strongest at reconstructing molecular structures and explaining experimental evidence. However, selecting compounds and predicting their numerical properties proved harder for that model. In one problem, the model had to predict how three related compounds would behave in terms of blocking a protein, crossing a cell barrier, and breaking down in a laboratory assay. The benchmark checks whether the model reconstructs the molecular structures correctly and how closely its predictions match experimental measurements.

In one specific task, models were asked to assemble a molecule from a central structure and two attached groups while preserving an 18-atom ring. At the highest reasoning setting, GPT-6 Astra reconstructed this compound correctly in all ten responses. GPT-5.6 Sol and GPT-5.6 Luna failed across all ten responses — GPT-5.6 Sol produced a 15-atom ring, while GPT-5.6 Luna split the molecule into two disconnected fragments. These errors change the identity of the molecule being analyzed, highlighting the importance of accurate molecular representation in medicinal chemistry reasoning.

Why This Matters for Drug Discovery

This benchmark is a step towards helping scientists develop better medicines more efficiently. By improving AI's ability to interpret molecular structures and predict compound properties, researchers can make more informed decisions about which compounds to pursue. This could lead to faster development of new drugs and treatments. AI models that can accurately predict how a compound will behave in the body can help researchers avoid costly and time-consuming trials, speeding up the drug discovery process.

Across the full benchmark at the shared xhigh setting, GPT-6 Astra earned 83.3% of the available credit, compared with 82.0% for GPT-6.1 Sol, 75.1% for GPT-6.1 Terra, and 58.2% for GPT-5.6 Sol. At max effort, GPT-6 Astra reached 83.8%. Higher reasoning settings improved scores, while large differences between models remained. The component results separate four abilities: reconstructing molecules, explaining findings, selecting compounds, and predicting measurements, showing where models are strongest and where errors remain.

Frequently asked

What is MedChemBench?
MedChemBench is a benchmark from OpenAI that evaluates how well AI models can interpret molecular structures, reason from experimental evidence, and select compounds in early drug discovery.
Which AI model performed best on MedChemBench?
GPT-6.2 Astra earned the highest score of 84.5% at the highest reasoning setting.
How many problems are included in MedChemBench?
MedChemBench includes 99 problems drawn from published research.
What are the four components evaluated by MedChemBench?
The four components are molecular structures, scientific reasoning, ranking and selection, and property predictions.