Translating Chemical Structures

The boring parts of drug discovery

Going from a disease to a new drug is an enormous challenge that optimistically takes 10 years. There are many steps in that process and many may be accelerated by AI. Some steps are scientific challenges like understanding the mechanism of a molecule affects a disease. Others are logistical challenges like placing advertisements to find patients for your clinical trial.

At Edison, we are working on accelerating the individual steps of drug discovery. Rather than treat each as an independent problem, we are automating the process of discovering how to solve the steps. This is known as an emerging topic of autoresearch: the automation of creating and training predictive models. It is our ambition that AI scientists, like Kosmos,¹ can use autoresearch to solve these sub-problems, freeing scientists to work on higher-order questions and ultimately increasing the pace of discovering new medicines.

The model we’re releasing today was developed with autoresearch and tackles a challenging machine-learning task: reading chemical patents. The compromise of drug discovery is that the company that fronts the enormous monetary capital gets an exclusive right to sell and market the drug for a limited amount of time to recoup that money. That “right” is implemented as a patent that shows the chemical structure of the drug. Since the patent must be filed early and with some optionality, the patent actually enumerates a number of specific chemical structures rather than one. Chemical structures are graphs, like a network of roads. The way a patent enumerates these multiple chemical structure graphs is via a drawing that “matches” a family of related graphs. These kinds of drawings are called Markush structures,² named for the first person who proposed and had this idea upheld in a court. They only exist in patents and are one of the most niche technical diagrams imaginable. Despite their importance, parsing these remains quite challenging, with substantial room for improvement.

Here are some examples of Markush structures you can see in recent patents:

How is this done today? Imagine you’ve designed an exciting lead molecule that one day may to be an FDA-approved drug. Maybe you used AI to design the molecule. A key part of the molecular design process is checking the patent landscape. Specially trained experts, typically with a PhD in chemistry and legal training, will find potentially overlapping patents using a set of specialized hand-curated data. Then, they by-hand review your lead molecules against the Markush structures and text explanations listed in the patent. And they return to you a report explaining where you can go in chemical space and what relevant patents you should pay attention to. This is slow and expensive, meaning it can become a bottleneck that adds time to each design-make-test cycle. We wanted to automate part of this process so it can be done rapidly and painlessly, to accelerate the cycle time of designing a lead molecule.

MarkushGlyph

Let us now sharpen up the definitions. Markush structures describe entire families of related compounds through a shared molecular scaffold, variable sites, and accompanying text. The ways in which these are presented can vary strongly across patents (see image above).

Chemical structure recognition itself is not new: OpenAlex indexes more than 200 works matching “optical chemical structure recognition.”³ We were excited by Qian et al.’s MolScribe, whose pioneering image-to-graph approach could recognize a limited subset of Markush backbones.⁴ Morin et al. then introduced MarkushGrapher, which jointly processed the visual and textual parts of a Markush definition,⁵ and Strohmeyer et al. recently extended this line of work with MarkushGrapher-2.⁶ Our work builds on these advances while simplifying the pipeline and improving performance.

Our model that can read Markush structures is called MarkushGlyph and is based on Qwen3.5-2B-Base, a vision-language model released by the Qwen team at Alibaba. In early experiments, we tried simpler architectures (see the OCSR section below) and training from scratch, but found these underperformed - Markush structure recognition is sufficiently challenging and data-scarce that the increased model capacity and heavy pretraining both benefit performance substantially.

Our model accepts a single image as input, containing both the Markush structure diagram and any accompanying text. Since MarkushGlyph will be used in a variety of settings, we do not assume that text annotations will have been OCR’d. We contrast this approach with prior work, which typically uses multi-stage pipelines for separately processing the text and image, and writing the final output. MarkushGlyph outputs CXSMILES-opt, an extension of the standard SMILES molecular representation format, further adapted to be tokenization-friendly.

Metric definition

Markush structures are analogous to regular expressions for chemistry. Determining whether two CXSMILES are equivalent is a non-trivial task - for comparison, regex equivalence determination is known to be PSPACE-complete. Prior literature in the field uses heuristics to approximate true equality. However, we found these prior metrics to have some deficiencies, including not penalizing extra annotations not present in the input. We therefore introduce a stricter metric. While still heuristic-driven, it closes the gap on these issues.

Results

MarkushGlyph achieves SOTA performance across all evaluated benchmarks, under both metric definitions. The improvement is especially large when evaluated with our “strict” metric.

For the purposes of illustration, here are the five Markush structures from above, alongside visualizations of each model’s predictions:

OCSRGlyph

The task of recognizing standard molecular structures in images is typically referred to as optical chemical structure recognition (OCSR). In addition to MarkushGlyph, we release OCSRGlyph: a Transformer encoder/decoder, pre-trained on ImageNet and fine-tuned on OCSR. It accepts an image as input and outputs the corresponding SMILES string.

When analyzing the failures of early training runs and of other models, we found stereochemistry to be a common failure mode. Stereochemistry refers to the 3-D orientation of molecules - certain molecules have right- or left-handed orientations with different biochemical properties, so distinguishing them is important. We ameliorated this by adjusting the training distribution to over-represent chiral molecules.

This resulted in a final model that achieves SOTA accuracy on a benchmark of USPTO-extracted molecules, both when accuracy is defined to require stereochemistry match (93.8%) and when it is not (96.2%).

Release

In the pursuit and spirit of open science, we open-source all artifacts associated with this project: the codebase for training and inference, the datasets, checkpoints for MarkushGlyph and OCSRGlyph, and finally the paper.

To try our models out, ensure you have uv installed and follow these steps:

git clone https://github.com/EdisonScientific/glyph
uv run glyph/scripts/glyph_infer.py --model ocsr image.png
uv run glyph/scripts/glyph_infer.py --model markush image.png

Kosmos for ML research

This project is an example of ML research at Edison, much of which is directed at making Kosmos a better AI scientist. To achieve this goal, we use Kosmos recursively to drive our internal research. In this project, research staff worked with Kosmos to frame the problem, propose hypotheses, and design and execute experiments. Kosmos independently proposed several key ideas, including oversampling chiral molecules for OCSRGlyph and the stricter Markush metric.

If you are intrigued by the work we do to use artificial intelligence to expand the boundaries of scientific knowledge and are interested in joining the effort, please take a look at the open roles on our team.

References