blog›learned cuts
What a learned tokenizer actually learns
On this page
Backpropagation updates every weight in a modern language model except the very first step: chopping raw text into tokens. Whether you split on characters, words, or merge frequent pairs with byte-pair encoding (BPE), the cuts are frozen before training even starts.
Tokenization looks like boring preprocessing until you measure what it does. Swapping the tokenizer alone, keeping the architecture and data identical, shifts both accuracy and compute cost (Bostrom & Durrett; Ali et al.), and the same holds in chemistry (Leon et al.; Wadell et al.). My favourite example is an arithmetic test on GPT-3.5: exact same weights, exact same sums, 76% accuracy with digits grouped one way and 98% with them grouped another.
So why freeze the one piece of the pipeline that never gets a gradient? Feeding raw bytes directly works (ByT5), but long sequences make attention painfully slow. A newer line of architectures pulls tokenization inside the network: read raw bytes, learn where to place boundaries, and compress the sequence on the fly. The cleanest recent design is H-Net, where a tiny router module learns its cuts from the same training loss as every other weight.
What got me hooked on this were Cartesia's two posts on scaling H-Nets (Part I, Part II). One example stuck in my head: they trained a byte-level model on bacterial and archaeal DNA purely to predict the next letter, fed it a human gene, and watched it cut cleanly along codon boundaries. Nobody had told it what a codon was.
There's a catch, though. Cartesia's evidence is five hand-picked gene regions: no genome-wide hit rate, no inventory of the pieces the model actually uses, no random-cut baseline, and no released weights to verify it on.
Still, the idea was too tempting to leave alone, and chemistry felt like a better stress test than English or DNA for two reasons.
First, chemistry comes with an answer key. Nobody describes a molecule atom by atom; a chemist calls it a benzene ring with a carboxylic acid on one side and a sulfonamide on the other. We already agree on what the meaningful pieces are (multi-letter atoms, rings, functional groups, breakable bonds), so for every cut a model learns, I can check whether it respects chemistry or slices straight through an atom.
Second, in H-Net papers the model almost always predicts the next byte. In chemistry, people routinely train models in completely different ways: predicting structural fingerprints, regressing physical properties, or filling in masked spans. If the loss is what trains the router, changing the loss ought to change the tokens. I wanted to see what happens when you hold the model and data fixed and swap out the objective.
To keep things fair, I used SMILES: plain ASCII, one byte per character, and the format almost every molecular dataset already lives in. You can write aspirin in notations that pre-package chemical fragments, like SAFE, but that hands the router the answer before training starts. In SMILES, if the model cuts along chemical lines, it has to figure them out itself.
| notation | aspirin | how it writes a molecule |
|---|---|---|
| SMILES | CC(=O)Oc1ccccc1C(=O)O | Letters are atoms, brackets are branches, a pair of digits closes a ring. |
| SELFIES | [C][C][=Branch1][C][=O][O][C][=C][C][=C][C][=C][Ring1][=Branch1][C][=Branch1][C][=O][O] | Every symbol is a small instruction, so any string of symbols is a valid molecule. |
| InChI | InChI=1S/C9H8O4/c1-6(10)13-8-5-3-2-4-7(8)9(11)12/h2-5H,1H3,(H,11,12) | The formula first, then how the atoms connect, then the hydrogens. Made for looking molecules up in databases. |
| SAFE | c13ccccc14.CC2=O.C4(=O)O.O23 | SMILES rearranged so that each chemical fragment is written in one stretch, with dots between fragments. |
One molecule, four notations. SMILES is the one used in this post.
So I took one byte-level H-Net, trained it on 90 million molecules from PubChem and a few public collections, and varied only the training objective:
- next byte
- fingerprint
- descriptors
- masked fragments
Here is what the four models do to the exact same molecule.
One molecule drawn four times, once for each model, with its text beside the button. Where the dots pull apart, that model decided a new piece begins. Hover or tap any atom to see its piece in all four.
Quick caveat upfront: these are small 20 to 30 million parameter models built from the published 2025 H-Net design, whereas Cartesia's results come from an unreleased router and from models up to more than a hundred times the size. Nothing here contradicts their results, and I list the limits at the end.
How a cut gets a gradient
An H-Net router walks along the byte sequence comparing each position's hidden state to the one right before it. Similar vectors stay glued to the current piece; once two neighbours diverge past a threshold, a new piece starts. Because the comparison runs through learned projections, the network decides for itself what counts as a boundary. The main backbone then runs only over the first vector of each piece (a much shorter sequence) before unpacking the result back out to individual bytes.
Since "cut or don't cut" is a hard binary choice with zero gradient, H-Net sneaks gradients back through a straight-through estimator plus a smoothing step across adjacent pieces. One extra loss term nudges the average piece length toward a target of around three bytes here. That sets how often the router cuts, without telling it where. Everything else comes strictly from the task loss.
All four of my models actually compress twice: bytes into pieces (stage 1, about three bytes each), and pieces into larger stage-2 groups (four to eleven bytes). Most of this post looks at stage 1, since that's the closest equivalent to a tokenizer, though stage 2 turns out to have its own surprises. The figure below shows the whole model, and full details are in the appendix.
My bets going in, and training on one GPU
Before looking at a single cut, I wrote down what each objective asks for and what I expected the router to do:
- Next byte: read left to right and guess the next character. After the codon result, I expected cuts at high-uncertainty spots that would roughly line up with chemical boundaries.
- Fingerprint: predict a 2,048-bit Morgan fingerprint (radius 3) marking which small substructures occur in the molecule. This task literally asks "which fragments are in here?", so it seemed like the strongest candidate to discover clean fragments on its own.
- Descriptors: predict 185 computed physical properties (molecular weight, atom counts, ring counts, lipophilicity). Since many of these are close to sums over atoms, I expected the model to care about atoms and ignore how they group together.
- Masked fragments: hide 25% to 45% of each molecule's bytes (mostly whole chemical fragments plus scattered single bytes, adapted from FACT) and fill them in. Filling in a hidden acid is much easier if the acid sits inside one piece, so I expected fragment cuts here too.
That last objective is admittedly a rigged test. To hide a fragment I first have to decide what a fragment is, and I used chemistry to decide: named functional groups, rings, and the building blocks you get from BRICS. So masked fragments works more as a sanity check: if chemistry is baked into the loss, can the router pick it up? The other three are the blind test.
Everything ran on a single GPU over 90.9 million molecules (89.8M from PubChem, plus COCONUT, ChEBI, LIPID MAPS and public spectral libraries), with 50,000 held out for every measurement in this post. The H-Net paper uses Mamba-2 for its outer layers, but my pure-PyTorch Mamba-2 implementation without optimised kernels was almost three times slower per step (298 ms vs. 106 ms) and took over three times the memory, so I stayed with Transformers.
Training a 20M-parameter model for five epochs took 11 to 17 hours: small by frontier standards, plenty big to watch tokenizers crystallise in real time.
Almost all the drama in the router happens during epoch one. Masked fragments starts off hacking through about a third of multi-character atoms like Cl, then drops to 2% before the first epoch finishes. Next byte barely budges after its very first checkpoint. Descriptors hesitates the longest: after a full epoch its router is still undecided at half of all positions. By epoch two all four have locked in their habits, and the remaining three epochs keep pushing the loss down while leaving the cuts mostly untouched.
One molecule, four vocabularies
Start with a single molecule from the held-out set carrying a sulfonyl group and a carboxylic acid.
Look how wildly the four models disagree. Next byte swallows the entire left half as one big gulp, then chops nervously around ring digits and parentheses. Fingerprint keeps the sulfonyl tidy, then shatters the acid into three bits: C, (=O) and O). Descriptors glues (=O)O together, leaving the acid's carbon stranded on the ring. Only masked fragments hands back both S(=O)(=O) and C(=O)O intact.
Press sample another for a different molecule.
And here is a second molecule against the standard regular-expression tokenizer most SMILES models use (one token per atom or bond symbol):
Anyone can cherry-pick two flattering molecules. To see the real picture, I ran all 50,000 held-out molecules through each model, collected every distinct piece it emitted, and compared against a baseline placing the exact same number of cuts at random.
| pieces | bytes / piece | no atom | whole group | atom cut | |
|---|---|---|---|---|---|
| next byte | 8,620 | 2.75 | 21.7% | 2.9% | 1.1% |
| fingerprint | 4,271 | 2.59 | 17.2% | 2.4% | 0.3% |
| descriptors | 3,810 | 2.82 | 7.7% | 3.9% | 4.1% |
| masked fragments | 14,928 | 4.33 | 3.7% | 14.7% | 0.03% |
Pieces used by each model on 50,000 held-out molecules. The first column counts distinct pieces. The three shares on the right count every use of a piece, so a piece used 1,000 times counts 1,000 times. Hover or tap a column name for what it counts.
Random cutting produces 54,000 to 60,000 distinct pieces, so all four models are learning compact, repeatable vocabularies (between 3,810 and 14,928 pieces). They just don't learn the same one. Across all four models, only 191 pieces are shared, mostly bare basics like C, CC, N and O. Stranger still, more than a fifth (21.7%) of everything the next-byte model emits contains no atom at all, just lone brackets, bond symbols or ring digits.
You can explore every piece below. Each letter is one piece (its first character, sized by how often it appears, coloured by element and placed near similar strings). Click a model name to switch vocabularies, click a category to highlight it, or hover any letter to see how all four models slice a real molecule around that piece. Every piece falls into one of five kinds:
- Symbols only: brackets, ring digits or bond symbols with no atom, such as
),1or=. - Splits an atom: cuts inside a multi-letter atom, like the
lofClor half of[nH]. - Unfinished: whole atoms, but leaves a branch, ring or aromatic ring hanging open (like
ClC(Cl). - Complete: every branch and ring closes cleanly, such as
CCorC(=O). - Known group: a complete piece holding an entire ring or functional group, like
C(=O)O.
Grading the cuts against chemistry
Now for the answer key: three questions, getting progressively stricter.
1. Do they even respect atoms? Some atoms take more than one character in SMILES: Cl, Br, [nH], [N+]. Cutting between C and l slices chlorine in two.
| multi-character atoms split | same, with random cuts | |
|---|---|---|
| next byte | 18% | 47% |
| fingerprint | 5% | 48% |
| descriptors | 65% | 46% |
| masked fragments | 0.4% | 31% |
Share of multi-character atoms split, for each model and for the same number of cuts placed at random. Lower is better.
Three models beat chance, and masked fragments gets down to 0.4%. Then there's descriptors sitting at 65%, worse than random cutting (46%). That was the exact opposite of what I bet on, and it led to the strangest finding in the whole project; hold onto that number for a few sections.
2. Do they break bonds where a chemist would? BRICS defines which single bonds outside rings are sensible places to disconnect a molecule. Subtracting the random-cut baseline gives a clean preference score (zero means no preference).
| preference for breakable bonds | |
|---|---|
| next byte | −0.04 |
| fingerprint | +0.16 |
| descriptors | +0.14 |
| masked fragments | +0.38 |
Above zero: the model cuts at BRICS bonds more often than at other single bonds. Zero is no preference.
Next-byte prediction sits at −0.04: zero preference for chemical bonds. Fingerprint (+0.16) and descriptors (+0.14) both pick up a genuine preference despite never seeing a BRICS label, and masked fragments lands at +0.38.
3. Do functional groups survive intact?
| acid | ester | amide | carbonyl | nitro | sulfonyl | nitrile | CF3 | methoxy | phenyl | |
|---|---|---|---|---|---|---|---|---|---|---|
| next byte | 17% | 27% | 12% | 21% | 100% | 13% | 30% | 3% | 85% | 1% |
| fingerprint | 0% | 0% | 0% | 0% | 97% | 59% | 74% | 0% | 84% | 6% |
| descriptors | 0% | 0% | 0% | 1% | 98% | 96% | 88% | 0% | 74% | 18% |
| masked fragments | 100% | 19% | 31% | 97% | 100% | 98% | 100% | 100% | 100% | 3% |
Masked fragments keeps seven of the ten motifs whole 97% to 100% of the time, as you'd hope.
What surprised me more were fingerprint and descriptors, since nothing told either model what a functional group is. Both keep nitro, sulfonyl, nitrile and methoxy groups intact most of the time. Yet on carboxylic acids, carbonyls, esters, amides and CF3, they score 0% to 1%. They aren't cutting randomly either: they systematically slice right after the carbon and isolate (=O) as its own reusable piece. My bet on the fingerprint model was half right. It really did invent a consistent fragment vocabulary, just an alien one you won't find in an organic chemistry textbook.
Two oddities stand out across the whole table. First, no model keeps a benzene ring whole (1% to 18%). Fair enough: c1ccccc1 is eight bytes, and stage-1 pieces average three to four. Second, even masked fragments keeps only 19% of esters and 31% of amides whole, despite keeping every acid intact. Looking back at my masking code, I did that to myself: my masking recipe pulled from both named functional groups and BRICS blocks, and BRICS breaks the ester C–O bond and the amide C–N bond. I had quietly baked a contradiction into the training loss, and the router reproduced it.
One level up: stage-2 groups
Stage 1 might simply be too short to fit an eight-byte ring. What about stage 2, where the second router merges pieces into groups of four to eleven bytes? Since longer spans trap motifs intact by sheer luck, every number here sits beside a matched random-cut baseline (in brackets in the first table, and in small grey type in the second).
| groups | bytes per group | distinct groups | preference for breakable bonds | 5- and 6-rings kept whole | multi-character atoms split |
|---|---|---|---|---|---|
| next byte | 5.1 | 47,566 | −0.06 | 3% (11%) | 7% (26%) |
| fingerprint | 4.2 | 22,353 | +0.14 | 9% (8%) | 1% (32%) |
| descriptors | 5.3 | 29,509 | +0.21 | 21% (13%) | 17% (26%) |
| masked fragments | 11.0 | 84,273 | +0.08 | 25% (33%) | 0.1% (11%) |
The second level of cuts. In brackets: the same number of cuts placed at random. Underlined: the strongest preference for breakable bonds, and the only ring score clearly above chance.
| acid | ester | amide | carbonyl | nitro | sulfonyl | nitrile | CF3 | methoxy | phenyl | |
|---|---|---|---|---|---|---|---|---|---|---|
| next byte | 57%42% | 64%29% | 63%36% | 76%58% | 100%31% | 38%30% | 88%67% | 33%28% | 98%83% | 4%13% |
| fingerprint | 15%27% | 20%14% | 5%27% | 18%48% | 100%23% | 67%19% | 100%60% | 11%20% | 95%77% | 17%10% |
| descriptors | 40%39% | 33%25% | 27%38% | 56%58% | 98%29% | 100%28% | 99%67% | 76%28% | 99%83% | 41%17% |
| masked fragments | 100%71% | 71%55% | 84%64% | 100%81% | 100%63% | 100%62% | 100%86% | 100%61% | 100%93% | 39%38% |
Move up one level and the leaderboard scrambles completely:
- Descriptors, the model with the ugliest stage-1 atom splits, suddenly becomes the only model that keeps rings intact well above chance (21% vs. 13% random; phenyl rings 41% vs. 17%) and takes first place on breakable bonds (+0.21).
- Masked fragments recovers almost every functional group, including the esters and amides it split at stage 1. Still, at 11 bytes per group its edge over random cuts shrinks, and it keeps rings whole less often than chance (25% vs. 33%).
- Next byte keeps esters and amides whole about twice as often as chance, while still shredding rings (3% vs. 11%).
- Fingerprint stubbornly sticks to its stage-1 habit: sulfonyls and nitriles whole, acids, amides and carbonyls split more often than random cuts would.
Even so, I wouldn't call stage 2 a vocabulary. Across 50,000 molecules, each model generates 22,000 to 84,000 unique stage-2 strings, so most of them are near one-offs.
Why next-byte prediction learns SMILES grammar instead of chemistry
Why did the exact objective that discovered words in English and codons in DNA ignore chemistry here?
Check the per-byte surprise. In a simpler one-stage H-Net I trained as a control, predicting the first byte of a piece costs 0.95 bits, while predicting a byte inside a piece costs 0.65. The router places boundaries wherever the next character is hard to guess (a pattern SOMBRERO also measures in text and code). Compared to its baseline cut rate, it starts a piece 1.85 times as often before a ring-closure digit, 2.0 times before =, and 2.3 times before sulfur, while dropping to 0.12 times before an aromatic c. Once you're walking along a benzene ring, the next character is almost always another c, so the router goes to sleep. Type a SMILES string below to see where those spikes land.
In English, spikes in surprise happen at word boundaries, so cutting on surprise hands you words for free. In SMILES, surprise spikes at ring-closure digits and branch parentheses, which are bookkeeping marks of the notation rather than the molecule (you could number the same ring starting from any atom).
Three quick checks backed this up:
- Chemistry fades as the model improves. Early in training, the one-stage model showed a mild preference for breakable bonds (+0.064). By the end of training, that dropped to +0.008. Getting better at next-byte prediction made the cuts less chemical.
- Plain old BPE is far more chemical. A standard BPE tokenizer matched to the same average piece length scores +0.32 on breakable bonds and keeps rings whole six times as often, simply because frequent substrings in chemistry happen to be common chemical motifs.
- It isn't pure string-index noise, though. Rewriting 5,000 molecules as eight different valid SMILES each gave a bond-cut agreement of 0.48 (against 0.13 random and 1.0 perfect, roughly tied with BPE). The router really is reacting to molecular structure, just not the boundaries a chemist cares about.
And none of this hurt its language modelling: the one-stage H-Net beat a matched BPE Transformer by 1.1% fewer bits per byte on held-out molecules, matching the trend Cartesia report for text.
What the masked-fragment control shows
Since masked fragments cheated by hiding chemical units, I trained a clean control that only masks scattered random bytes, with no fragment spans at all.
That control splits just 1.3% of multi-character atoms, which means masking alone teaches the router what an atom is. Beyond atoms, though, it slices straight through rings and barely cares about breakable bonds. In short: random masking gives you atoms, and masking functional groups gives you functional groups, all the way down to that BRICS ester split. (For scale, at a 35% mask rate the masked-fragment model recovers 93% of hidden bytes and reconstructs 53% of molecules verbatim; whole fragment spans are harder at 84% against 94% for scattered bytes.)
Same loss, same accuracy, opposite cuts
Remember that baffling 65% atom-split rate for the descriptor model? I didn't figure out what was going on until I trained a second round of models.
Because fingerprint and descriptor models output one vector per molecule, you have to collapse the per-byte vectors into a single prediction somehow. In my first round (30M parameters, AdamW, 3 epochs, which produced the vocabularies, atlas and stage-1/2 tables above), I simply averaged across every byte. In the final round (20M parameters, Muon, 5 epochs), I switched to reading out a single CLS token at the start of the string, like BERT.
I treated that change as trivial plumbing. Here is what happened:
| atoms splitaverage every character | atoms splitread the CLS token | accuracyaverage every character | accuracyread the CLS token | |
|---|---|---|---|---|
| fingerprint | 5% | 70% | 0.831 | 0.832 |
| descriptors | 65% | 29% | 0.998 | 0.998 |
Task accuracy didn't budge (0.831 vs. 0.832 on fingerprints, 0.998 vs. 0.998 on descriptors), yet the learned cuts completely inverted in opposite directions:
- Fingerprint jumped from splitting 5% of multi-character atoms to 70%, well above the ~45% random baseline. It actively hunts down chlorine atoms, cutting between
Candl2.7 times more often than its average rate. - Descriptors swung the other way, dropping from 65% down to 29%.
- Stage 2 flipped too: with the CLS head, both models chop acids, esters and amides 99% to 100% of the time (random cuts would split 60% to 83%), while still keeping rings whole above chance.
Since the two rounds also differed in model size and optimizer, I ran a short controlled check at 30M changing only the readout head on the fingerprint model. Same flip: 41% vs. 4%.
My rough guess is that when every byte directly feeds an averaged readout, every position feels pressure to sit inside a sensible piece. Pull the gradient through one CLS position and that local pressure vanishes, leaving the router reacting to raw byte-to-byte contrast (which is largest inside bracketed atoms). That still doesn't explain why descriptors moved in the opposite direction, and I haven't chased it further. Either way, "the objective shapes the cuts" turned out to be only half the story: even a minor architectural choice sitting between the router and the loss can turn the tokenizer upside down without touching task accuracy.
Do cleaner cuts mean a better representation?
After seeing how tidy masked fragments looked, I expected it to have the cleanest internal geometry too. Checking the four final (20M, CLS-head) models on nearest-neighbour similarity search and linear probes for simple properties told a different story.
| atoms split | similarity search | simple properties | drug-property tasks | |
|---|---|---|---|---|
| next byte | 6% | 0.278 | 0.78 | 0.68 |
| fingerprint | 70% | 0.336 | 0.49 | 0.74 |
| descriptors | 29% | 0.277 | 0.85 | 0.55 |
| masked fragments | 2% | 0.282 | 0.77 | 0.55 |
Look at the fingerprint row: the single ugliest tokenizer in the whole experiment (70% of multi-character atoms chopped in half), the worst linear probes (0.49), and by far the best nearest-neighbour similarity search (0.336, where random is 0.103 and the best possible is 0.429). It was trained on fingerprints, similarity is measured in fingerprint space, and it built a strong similarity space without caring how ragged its tokens looked.
You see the same disconnect at the functional-group level. Take about 4,100 molecules from the public Lipophilicity set and ask how often a molecule's nearest neighbour in latent space shares its functional groups:
| acid | ester | amide | sulfonyl | nitrile | CF3 | methoxy | halogen | amine | |
|---|---|---|---|---|---|---|---|---|---|
| next byte | 57% | 27% | 82% | 59% | 45% | 43% | 51% | 66% | 75% |
| fingerprint | 61% | 26% | 85% | 67% | 50% | 48% | 46% | 70% | 76% |
| descriptors | 68% | 24% | 87% | 72% | 78% | 52% | 56% | 71% | 82% |
| masked fragments | 68% | 28% | 85% | 62% | 47% | 48% | 55% | 68% | 77% |
| by chance | 11% | 3% | 51% | 16% | 8% | 7% | 17% | 43% | 37% |
By chance, a molecule with a sulfonyl has a sulfonyl-containing neighbour 16% of the time. In the next-byte model's embedding space that jumps to 59%, even though that exact model chops sulfonyls apart 87% of the time. The network clearly knows a sulfonyl is there; it just doesn't need a sulfonyl token to represent it.
Does any of this matter downstream?
To see how these models hold up on actual chemistry tasks, I ran three comparisons across public benchmarks (details and full tables are in the appendix).
1. Learned cuts vs. fixed BPE. This was the comparison I wanted most, though I only had compute for a matched BPE Transformer control on next-byte prediction (same size, data and step count). Here are both after fine-tuning:
| ESOL | FreeSolv | Lipo | MoleculeACE | QM9 gap (meV) | |
|---|---|---|---|---|---|
| next-byte H-Net | 0.946 | 2.416 | 0.744 | 0.744 | 116 |
| BPE Transformer | 1.098 | 2.629 | 0.765 | 0.714 | 123 |
Error after fine-tuning, lower is better. Underlined: the better of the two.
The H-Net wins four of five benchmarks (trailing only on MoleculeACE) and generates slightly more valid SMILES (98.9% vs. 97.9%). The catch is speed: BPE trains about a quarter faster (0.28 s vs. 0.38 s per step), and its frozen representations probe just as well or better. At this scale, learning the tokenizer is a modest edge rather than a free lunch, and I didn't run fixed-tokenizer controls for the other three objectives.
2. Objective vs. objective. Holding the H-Net fixed and swapping the pretraining loss produces a wide spread. With the weights frozen and a linear probe on top, ESOL solubility RMSE runs from 1.01 (descriptors) to 1.46 (fingerprint). Fine-tuning narrows the gap and reshuffles the order. Here are all four alongside ChemBERTa-2 rerun through the exact same pipeline:
| pretraining | ESOLRMSE | FreeSolvRMSE | LipoRMSE | ACE30 sets | QM9 gapmeV | ADMETrank |
|---|---|---|---|---|---|---|
| next byte | 0.946 | 2.416 | 0.744 | 0.744 | 116 | 0.75 |
| fingerprint | 0.948 | 2.680 | 0.725 | 0.737 | 105 | 0.75 |
| descriptors | 0.895 | 2.483 | 0.671 | 0.710 | 109 | 0.60 |
| masked fragments | 0.902 | 2.300 | 0.670 | 0.703 | 117 | 0.53 |
| ChemBERTa-2 MLM | 0.888 | 2.613 | 0.734 | 0.815 | 204 | 0.76 |
| ChemBERTa-2 MTR | 0.857 | 3.272 | 0.692 | 0.741 | 169 | 0.65 |
Masked fragments takes four of the six columns, with descriptors right behind it. Fingerprint trails on most drug-property tasks yet wins on QM9's quantum-chemistry gap (105 meV). Notably, frozen probe scores were a lousy predictor of who won after fine-tuning.
3. Sanity check against published models. Are these models actually competitive, or am I analysing toy runs? Against ChemBERTa-2 rerun under identical splits, seeds and tuning, both descriptors and masked fragments beat ChemBERTa-2 MLM on all eight MoleculeNet classification sets. Looking at published numbers with a matching split convention (Table 6 of ChemBERTa-3, covering nine models on six tasks), the best of my four beats five of those nine on every set and three more on most. Only MoLFormer-LHPC beats mine overall, winning four of the six sets and pulling far ahead on BBBP (0.908 vs. my 0.726).
Take that strictly as a sanity check rather than a claim of a better method: my models have about nine times as many parameters as ChemBERTa-2, while the published ChemBERTa-3 rows were tuned for far more epochs.
Takeaways
I started this project expecting a clean story about whether an H-Net could discover chemistry from raw bytes. What came out was messier and much more interesting.
Changing the loss rewrites the learned vocabulary: four objectives on the same 90 million molecules ended up sharing just 191 basic pieces. Next-byte prediction, the objective that pulls words out of English and codons out of DNA, mostly learns SMILES syntax because that's where byte-level surprise lives. Fingerprint and descriptor losses do discover regular substructures on their own (including a consistent (=O) rule no chemist would write down), while masked fragments faithfully copies whatever chemical rules and contradictions you put into the masks.
More humbling still: switching the readout head from mean pooling to a CLS token inverted the cuts completely while leaving task accuracy untouched, and the model with the ugliest atom-chopping tokens still won similarity search. It's tempting to inspect a learned router and treat its cuts as a window into what the model understands, like reading codons off a gene. On molecules, that metaphor breaks down fast. A router only cares about lowering the loss, and a network can understand chemistry fine without cutting along human conventions like functional groups, BRICS bonds and canonical SMILES.
What I wouldn't claim
- Scale: 20 to 30 million parameters is tiny next to Cartesia's billion-parameter runs, and they note that biological alignment in DNA appears late in training.
- Router version: this uses the published 2025 router; Cartesia's newer unreleased router may behave differently.
- Backbone: I used Transformer outer layers instead of Mamba-2, which the H-Net paper notes compresses less cleanly.
- Notation: everything above is canonical SMILES. A quick control trained on SAFE kept rings whole far more often at an 11% cost in bits, so the writing system itself is another big lever I barely touched.
Appendix
A. Models and training
- Data. 90.9 million molecules, written as canonical SMILES without stereochemistry and keeping only the largest fragment. Almost all of them come from PubChem (89.8M rows), plus COCONUT (0.44M), ChEBI (0.12M), LIPID MAPS (0.04M) and structures from public spectral libraries (0.19M). A molecule can have more than one source. I held out 50,000 rows, and every cut statistic in the post is computed on those.
- Architecture. A two-stage H-Net that goes from bytes to pieces to groups. The router, smoothing, straight-through estimator and ratio loss follow the H-Net paper (eq. 4–10) and the released code. The ratio loss has a target of 3 and a weight of 0.03. Every stack uses pre-RMSNorm Transformer blocks with RoPE and SwiGLU instead of Mamba-2 for the outer stacks.
- Sizes. I trained two rounds. The first has about 30M parameters and was trained with AdamW for 3 epochs. The final one has about 20M and was trained with Muon for 5 epochs. The first round is behind the vocabulary table, the atlas, both scorecards (pieces and groups), the averaged readout and every benchmark table. The final round is behind the single-position readout, the four-way ranking and the similarity and probe numbers.
- Next byte. Causal stacks, cross-entropy on the next byte.
- Fingerprint. Bidirectional stacks, clean input. The target is a Morgan fingerprint of radius 3 with 2,048 bits, of which about 62 are on per molecule. Trained with binary cross-entropy.
- Descriptors. Bidirectional stacks. The target is 185 normalised RDKit descriptors, trained with mean squared error. The recipe follows ChemBERTa-2's multi-task regression.
- Masked fragments. Bidirectional stacks. For each molecule I draw a mask rate between 25% and 45% and a fragment share between 0% and 100%. Whole fragments are masked first, then random bytes until the rate is reached. A fragment is a functional-group match, a ring or a BRICS block. There are 9.2 per molecule on average, and together they cover 99% of bytes. On average 67% of masked bytes belong to a fragment. Every masked byte becomes a MASK symbol, with no random substitution. The recipe is adapted from FACT.
- Controls. A one-stage next-byte H-Net; a BPE Transformer of the same size on the same molecules (867 merges, about 4.3 bytes per token); random-masking models at several rates; a SAFE-input model.
- Hardware. One GPU. The final next-byte model took 15 hours.
B. Benchmark protocol and full tables
The protocol is the same for every model, including ChemBERTa-2 77M MLM and MTR, which I reran myself instead of quoting published numbers.
- Frozen: I take the mean, sum and final-position states and fit a ridge or logistic regression, with its strength chosen on validation.
- Fine-tuned: the whole network plus a linear head, trained with AdamW. I try four learning rates at seed 0, pick the best on validation, then run two more seeds.
- Splits: scaffold 80/10/10 for MoleculeNet, random for QM9, MoleculeACE's own split, official splits and seeds for TDC.
- Baselines in every frozen table: ECFP4 counts and a bag of bytes.
- Overlap: a benchmark molecule counts as seen when its structure appears in a pretraining row. No benchmark label was ever in pretraining. The seen fractions are ESOL 76%, FreeSolv 82%, Lipophilicity 95%, QM9 2%, MoleculeACE 98%, TDC ADMET 80%.
MoleculeNet classification, fine-tuned. ROC-AUC, higher is better, three seeds.
| model | BBBP | BACE | ClinTox | SIDER | Tox21 | ToxCast | HIV | MUV |
|---|---|---|---|---|---|---|---|---|
| ChemBERTa-2 MLM | .710 | .834 | .824 | .619 | .753 | .647 | .778 | .752 |
| ChemBERTa-2 MTR | .700 | .823 | .838 | .619 | .756 | .669 | .758 | .813 |
| next byte | .684 | .845 | .924 | .653 | .752 | .643 | .782 | .765 |
| fingerprint | .671 | .809 | .928 | .639 | .761 | .645 | .792 | .747 |
| descriptors | .726 | .847 | .876 | .631 | .779 | .673 | .789 | .758 |
| masked fragments | .716 | .842 | .913 | .644 | .773 | .651 | .793 | .763 |
ROC-AUC after fine-tuning, three seeds, higher is better. Underlined: best in column.
Descriptors beats ChemBERTa-2 MLM on all 8 sets and MTR on 7. Masked fragments beats MLM on all 8 and MTR on 6. On MUV, ChemBERTa-2 MTR wins outright.
Against published numbers: ChemBERTa-3's Table 6. This is the only published table I found that uses the same split convention as my runs. The first nine rows are quoted from that paper, and I didn't rerun them. The last four are mine. ROC-AUC, higher is better.
| model | BACE | BBBP | Tox21 | HIV | SIDER | ClinTox |
|---|---|---|---|---|---|---|
| MoLFormer-LHPC | .887 | .908 | .791 | .750 | .622 | .993 |
| Random Forest | .866 | .694 | .674 | .794 | .630 | .689 |
| c3-MoLFormer-1.1B | .819 | .735 | .723 | .762 | .618 | .839 |
| ChemBERTa-MLM-100M | .781 | .700 | .718 | .740 | .611 | .979 |
| Grover-250K | .825 | .674 | .692 | .759 | .619 | .642 |
| GCN | .778 | .642 | .710 | .759 | .613 | .870 |
| DMPNN | .626 | .661 | .706 | .752 | .524 | .642 |
| Infograph-250K | .739 | .639 | .684 | .755 | .627 | .845 |
| Infomax3D-250K | .658 | .624 | .645 | .704 | .588 | .860 |
| mine: next byte | .845 | .684 | .752 | .782 | .653 | .924 |
| mine: fingerprint | .809 | .671 | .761 | .792 | .639 | .928 |
| mine: descriptors | .847 | .726 | .779 | .789 | .631 | .876 |
| mine: masked fragments | .842 | .716 | .773 | .793 | .644 | .913 |
ROC-AUC, higher is better. Grey dots: quoted from ChemBERTa-3, not rerun here. Coloured dots: my four models. Underlined: best in column.
Set by set, the best of my four comes 1st of ten on SIDER, 2nd on HIV and Tox21, and 3rd on BACE, BBBP and ClinTox. Model by model, the best of my four beats Grover, GCN, DMPNN, Infograph and Infomax3D on all six sets, c3-MoLFormer-1.1B and ChemBERTa-MLM-100M on five of six, and the random forest on four of six. Only MoLFormer-LHPC comes out ahead overall (4–2).
Four things to keep in mind when reading it, two that work against my models and two that work in their favour.
- Against me: the published rows were tuned over 18 hyperparameter variations for up to 500 epochs. Mine got four learning rates and a budget of 2,200 steps.
- Against me: c3-MoLFormer-1.1B was pretrained on 1.1 billion molecules and mine on 91 million.
- In my favour: 66 to 96% of these molecules are in my pretraining corpus as structures. Nobody has measured the same thing for the published rows.
- In my favour: "best of my four" picks the best of four models per column.
On BBBP my best fine-tuned number is .726, which is lower than my best frozen linear probe at .745. The probe would be 2nd in the table. BBBP's test fold has only 204 molecules, and fine-tuning overfits it.
Numbers quoted from other papers, under their own splits. These can't go in the same table as mine, because the split convention alone moves scores more than the choice of model does. They're here only as context.
| set | best published | from | mine, and why it is not comparable |
|---|---|---|---|
| QM8 MAE | FARM .0146, Uni-Mol .0156, GEM .0163, MoLFormer .0177 | FARM and MLM-FG, Table 2 | .0249 frozen probe. Theirs are fine-tuned, or frozen with a sequence head. |
| QM7 MAE (kcal/mol) | MLM-FG 64.2, GEM 65.0, MoLFormer 69.1 | MLM-FG, Table 2 | 115.3 frozen probe. Theirs is fine-tuned. |
| QM9 gap | FARM 124 meV, Uni-Mol 127, MolCLR 131, GEM 153, GROVER 196 | FARM, Table 2 | 105 meV at best, but on a random split where theirs is scaffold. Not a win. |
| TDC ADMET | Chemprop, Chemprop-RDKit, MiniMol, Morgan+MLP | the 22 public leaderboards | reported as median leaderboard position, below |
| OGB molhiv | leaderboard top .848; GCN .761, GIN .756 | the OGB leaderboard | .788, a frozen probe against end-to-end models |
Published numbers under other splits, for context only.
FARM and MLM-FG don't even agree with each other on the same baselines. On QM8 they differ by up to 30% for models that neither paper trained. So a single number lifted from either one is weak evidence.
TDC ADMET, 22 sets, fine-tuned. Median leaderboard position, 0 is first.
| objective | 20M final | 30M first round |
|---|---|---|
| masked fragments | 0.55 | 0.53 |
| descriptors | 0.55 | 0.60 |
| next byte | 0.68 | 0.75 |
| fingerprint | 0.74 | 0.75 |
Median leaderboard position on 22 TDC ADMET sets after fine-tuning. 0 is first, so lower is better.
For comparison, ChemBERTa-2 MTR scores 0.65 and MLM 0.76, both over 11 sets. Among the frozen results, the masked-fragment model's linear probe scores 0.927 ROC-AUC on bbb_martins, which is first of 26 on that leaderboard. Its median frozen rank over all 22 sets is still 0.87, though, so this is a single good set and I wouldn't generalise from it.
C. Related work
Models that read bytes or characters. Dropping the tokenizer is an old idea, and most versions of it don't learn where to cut.
- No segmentation. ByT5 and MambaByte run directly on bytes and pay for it in sequence length.
- Fixed-size pieces. CANINE, Hourglass and MEGABYTE downsample or patch at a fixed rate.
- A rule. SpaceByte applies its large blocks after spaces. Hierarchical Autoregressive Transformers wrap a word-level model in byte-level ones, with word boundaries set by rule.
- A separate model. The Byte Latent Transformer starts a patch where a small, separately trained byte model finds the next byte hard to predict. The cuts follow prediction error by construction, and the main loss never reaches them.
- Learned from the loss. Hierarchical Multiscale RNNs learned binary boundaries with a straight-through estimator. Nawrot et al. added a boundary predictor to a Transformer and compared four ways to train it, one of them end to end. Charformer learns a soft mixture over candidate blocks, and MrT5 learns which bytes to delete. H-Net is the version of this idea that trains stably at scale and stacks into several levels.
H-Net follow-ups. H-Net++ applies it to Persian and reports morphological pieces. dnaHNet, from the H-Net authors, scales it to genomes. GeneZip adds gene annotations to the compression target during training. LDARNet trains H-Net-style chunking on DNA with a masked objective, which makes it the closest work to the question in this post, though in a different domain. It reports that the masked model's boundaries align with promoter motifs and splice junctions. It doesn't compare them with the boundaries a next-token model learns on the same data, which is the comparison I make here. DNAChunker also puts a learned tokenizer inside a bidirectional masked DNA model, again without a next-token counterpart. SOMBRERO measures how strongly learned boundaries sit on bytes that are hard to predict, and adds a loss that pushes them there, in next-byte models on text and code.
I couldn't find any earlier work that applies H-Net, or any segmentation trained by the task loss, to molecules. The nearest is SMiBPT, which patches SMILES bytes with a sliding-window entropy rule, the way the Byte Latent Transformer does.
Tokenizers for molecules. The standard is the atom-level regular expression from the Molecular Transformer. SMILES Pair Encoding and Atom Pair Encoding add BPE-style merges on top of it. Smirk compares 34 tokenizers and finds gaps in most of them. SELFIES changes the notation so that every string is a valid molecule.
Fragments in the notation. Group SELFIES, t-SMILES and SAFE write the molecule as a sequence of predefined fragments. In all three, the fragments come from chemistry rules that are fixed before training.
Fragments in the objective. GROVER predicts which functional groups a molecule contains. MLM-FG masks functional-group spans in place of random tokens. FACT masks functional-group spans up to a set coverage and then random tokens, which is the recipe I adapted. MolEncoder shows that mask rates well above the usual 15% help on SMILES. In all of these the tokenizer is fixed, so the objective can change what the model learns but can't change the pieces. With an H-Net it changes both.
References
H-Net
- Hwang, Wang, Gu. Dynamic Chunking for End-to-End Hierarchical Sequence Modeling. ICLR 2026. Code.
- Cartesia Research. From Tokens to Bytes I: Scaling Hierarchical Networks. October 2026.
- Cartesia Research. From Tokens to Bytes II: Learning from Raw Data. October 2026.
- Zakershahrak, Ghodratnama. H-Net++: Hierarchical Dynamic Chunking for Tokenizer-Free Language Modelling in Morphologically-Rich Languages. 2025.
- Shah et al. dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning. 2026.
- Zhao et al. GeneZip: Region-Aware Compression for Long Context DNA Modeling. 2026.
- Ledneva, Kuznetsov. LDARNet: DNA Adaptive Representation Network with Learnable Tokenization for Genomic Modeling. ICML 2026.
- Kim, Shin, Kim, Jung, Lee, Lee, Ahn, Han. DNAChunker: Learnable Tokenization for DNA Language Models. ICML 2026.
- Neitemeier, Serra, Li, Wirges, Balles, Metzen. SOMBRERO: Measuring and Steering Boundary Placement in End-to-End Hierarchical Sequence Models. 2026.
Byte and character models
- Chung, Ahn, Bengio. Hierarchical Multiscale Recurrent Neural Networks. ICLR 2017.
- Clark, Garrette, Turc, Wieting. CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representation. TACL 2022.
- Xue et al. ByT5: Towards a token-free future with pre-trained byte-to-byte models. TACL 2022.
- Tay et al. Charformer: Fast Character Transformers via Gradient-based Subword Tokenization. ICLR 2022.
- Nawrot et al. Hierarchical Transformers Are More Efficient Language Models. Findings of NAACL 2022.
- Nawrot, Chorowski, Łańcucki, Ponti. Efficient Transformers with Dynamic Token Pooling. ACL 2023.
- Yu et al. MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers. NeurIPS 2023.
- Wang, Gangavarapu, Yan, Rush. MambaByte: Token-free Selective State Space Model. COLM 2024.
- Slagle. SpaceByte: Towards Deleting Tokenization from Large Language Modeling. NeurIPS 2024.
- Kallini et al. MrT5: Dynamic Token Merging for Efficient Byte-level Language Models. ICLR 2025.
- Pagnoni et al. Byte Latent Transformer: Patches Scale Better Than Tokens. ACL 2025.
- Neitemeier, Deiseroth, Eichenberg, Balles. Hierarchical Autoregressive Transformers. 2025.
Tokenizers in language models
- Bostrom, Durrett. Byte Pair Encoding is Suboptimal for Language Model Pretraining. Findings of EMNLP 2020.
- Ali et al. Tokenizer Choice For LLM Training: Negligible or Crucial? Findings of NAACL 2024.
- Singh, Strouse. Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs. 2024.
Molecular tokenizers and notations
- Schwaller et al. Molecular Transformer: A Model for Uncertainty-Calibrated Chemical Reaction Prediction. ACS Central Science, 2019.
- Krenn et al. Self-Referencing Embedded Strings (SELFIES). Mach. Learn.: Sci. Technol., 2020.
- Li, Fourches. SMILES Pair Encoding: A Data-Driven Substructure Tokenization Algorithm for Deep Learning. J. Chem. Inf. Model. 61(4), 2021.
- Leon et al. Comparing SMILES and SELFIES tokenization for enhanced chemical language modeling. Scientific Reports 14, 2024.
- Wadell, Bhutani, Viswanathan. Tokenization for Molecular Foundation Models. 2024.
- Cheng et al. Group SELFIES: A Robust Fragment-Based Molecular String Representation. Digital Discovery, 2023.
- Wu et al. t-SMILES: a fragment-based molecular representation framework for de novo ligand design. Nature Communications, 2024.
- Noutahi et al. Gotta be SAFE: A New Framework for Molecular Design. Digital Discovery 3, 2024.
- Mswahili et al. Entropy-based byte patching transformer for self-supervised pretraining of SMILES data. iScience 29(2), 2025.
Chemical language models and objectives
- Rong et al. Self-Supervised Graph Transformer on Large-Scale Molecular Data (GROVER). NeurIPS 2020.
- Chithrananda, Grand, Ramsundar. ChemBERTa: Large-Scale Self-Supervised Pretraining for Molecular Property Prediction. 2020.
- Ahmad et al. ChemBERTa-2: Towards Chemical Foundation Models. 2022. The reference model, rerun here.
- Ross et al. Large-Scale Chemical Language Representations Capture Molecular Structure and Properties (MoLFormer). Nature Machine Intelligence 4, 2022.
- Peng et al. Pre-trained Molecular Language Models with Random Functional Group Masking (MLM-FG). 2024.
- Krüger et al. MolEncoder: towards optimal masked language modeling for molecules. Digital Discovery 4, 2025.
- Singh et al. ChemBERTa-3: an open source training framework for chemical foundation models. Digital Discovery 5(2), 662–685, 2026. Source of the published classification table.
- Nam et al. FACT: Functional Group Alignment and Consistency in Token Space for Structure-aware Molecular Representation Learning. BioNLP 2026. The masking recipe I adapted.
Benchmarks and chemistry rules
- Wu et al. MoleculeNet: a benchmark for molecular machine learning. Chemical Science 9(2), 2018.
- Huang et al. Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development. NeurIPS 2021 Datasets and Benchmarks.
- van Tilborg, Alenicheva, Grisoni. Exposing the Limitations of Molecular Machine Learning with Activity Cliffs (MoleculeACE). J. Chem. Inf. Model. 62(23), 2022.
- Hu et al. Open Graph Benchmark: Datasets for Machine Learning on Graphs. NeurIPS 2020.
- Degen, Wegscheid-Gerlach, Zaliani, Rarey. On the Art of Compiling and Using 'Drug-Like' Chemical Fragment Spaces (BRICS). ChemMedChem 3(10), 2008.