MedChemBench evaluates AI models on 99 medicinal chemistry problems. GPT‑6.2 Astra gets 84.5% of available credit at maximum reasoning setting. However, these results do not demonstrate an ability to discover drugs.
PRMT5 confronts models with molecular compromises
Inhibiting a protein more effectively is not enough to make a compound preferable. An exercise devoted to PRMT5, a protein studied in anticancer research, requires predicting the behavior of three related compounds based on their structures and previous experimental data. Predictions include inhibition in cells, crossing of a cellular barrier, and rate of degradation by liver enzymes in a laboratory assay.
The results of subsequent experiments are not communicated to the models. They show that compound 23 inhibited PRMT5 at lower concentrations in cells, while compound 24 crossed the barrier better and was more resistant to enzymatic degradation. The exercise therefore verifies the link established between a structure and several behaviors, rather than an isolated property.
These decisions are part of the work of medicinal chemists, who develop, synthesize and then test related molecules before choosing the next course of experiments. An improvement may be accompanied by a degradation elsewhere: a compound that is very active against a protein associated with a disease may, for example, be poorly absorbed or break down too quickly in the body.
GPT‑6 Astra completes challenging molecular assembly
The representation of the molecule constitutes another aspect of the evaluation. The chemical diagrams indicate the bonds between atoms, the cycles as well as the stereochemistry. The repository verifies that this information is preserved when a model reconstructs a compound from a drawing and chemical groups.
The evening letterFree · 7 p.m.
Are you following the race for AI models?
Each output (GPT, Claude, Gemini, Mistral, etc.) decrypted the same evening, in 5 minutes. Free.
Included upon registration: our selection of the best AI guides & comparisons.
Choose your pace
Free · No spam · Unsubscribe in 1 click
An exercise thus requires bringing together a central nucleus and two groups while maintaining a cycle comprising 18 atoms. At xhigh, the maximum setting common to all eleven models in this example, GPT-6 Astra correctly reconstructed the compound in all ten responses. Both GPT‑5.6 Sol and GPT‑5.6 Luna failed each of the ten answers; these errors change the molecule actually represented.
GPT‑6 Astra achieved the best results for molecular reconstruction and explanation of experimental data. Choosing the compounds and anticipating the numerical values of their properties posed more difficulties for him.
Scores are based on previously published research
The corpus brings together 117 articles and patents, with 761 distinct molecular structures. For the majority of exercises, the model receives the information accessible at a stage of the scientific program: biological target, objectives pursued, molecules already examined and certain experimental results. In addition to the reconstruction, the rating concerns the scientific justification, the classification and choice of compounds, as well as the concordance of predictions with categories or values of measured or calculated properties.
At the maximum reasoning setting, GPT-6 Astra earned 83.3% of the available credit, GPT-6.1 Sol 82.0%, and GPT-5.6 Sol 58.2%. Each response corresponds to a single test, without the use of tools, without corrective feedback and without additional experience.
It remains possible that the models have already been exposed to the published sources used. The test evaluates reasoning based on historical data rather than real-world drug discovery: its results do not establish whether the models are capable of discovering new drugs or improving laboratory decisions.

