Here is a result that sounds like a prank until you read the numbers. Procedurally generated "alien" coding languages that humans barely parse? Frontier models execute them better than readable English equivalents.
GLOSSOPETRAE is an open-source xenolinguistics engine. Feed it a seed, get a full constructed language. Its CodeSkin module skins real programs (JavaScript in the paper) at four opacity levels from readable keywords to full glyph-swap.
At level L3, human legibility collapses to about 15%. Blind comprehension without the spec sits around 17-40%. But with the grammar spec, Claude Opus and GPT-5.5 score 93-100% on hard programs in alien glyphs.
The uncomfortable punchline: opacity up, model usability up. Human-readable code can actually hurt frontier model performance on some tasks.
The opacity ladder
CodeSkin is a faithful bijection. The unskinned program is identical to the original. All levels execute correctly.
| Level | What changes | Human legibility (approx.) |
|---|---|---|
| L0 | Readable keywords, ASCII identifiers | High |
| L1 | Translated keywords | Medium-high |
| L2 | Translated keywords + identifiers | Medium |
| L3 | Full glyph-swap, alien script tokens | ~15% |
Task accuracy at L0 vs L3 (from the project's reported benchmarks, n=30 seeds per model):
| Model | L0 hard programs | L3 hard programs |
|---|---|---|
| Claude Opus | ~20% | ~97% |
| GPT-5.5 | ~53% | ~100% |
Zero-shot without worked examples still favors glyphs on hard programs: Opus gains +36pp (57% to 93%), GPT-5.5 gains +10pp (87% to 97%).
Why readable code might slow models down
The paper frames this as decoupling: models lean on shallow semantic associations with English keywords (for, while, return). Strip those cues and the model is forced into structural reasoning over the spec.
That matches a separate line of work, Syntax Without Semantics , which shows models can learn novel language syntax through fine-tuning but struggle to transfer algorithmic reasoning into unseen languages. GLOSSOPETRAE flips the question: frontier models already reason structurally in-context when you give them a spec, and English surface form can be noise.
For applied engineers, the takeaway is not "delete English from your codebase." It is: token surface and training priors interact in weird ways. Prompt format, identifier naming, and even language choice can shift reliability more than you expect.
Related projects in the same vein
GLOSSOPETRAE is not alone on the "non-English surface, same token IDs" frontier:
AlienLM reports retaining 81-87% of plaintext-oracle benchmark performance after alien adaptation on models like LLaMA 3 8B and Qwen 2.5. Different goal (privacy and covert channels) but same underlying insight: models are flexible over surface form.
Security implications (do not ignore this)
GLOSSOPETRAE also documents steganography, tokenizer exploitation, and covert agent channels. The engine is a research artifact, but the security lesson is real:
If your monitoring stack assumes "English-looking code in logs," opaque agent languages bypass that assumption.
That matters for anyone deploying autonomous agents with tool access. Guardrails trained on natural language may not generalize to procedurally generated lexicons.
Try it yourself
Install from the repo:
git clone https://github.com/elder-plinius/GLOSSOPETRAE.git cd GLOSSOPETRAE # See PAPER.md and README for API entry points
JavaScript API examples from the docs:
const Glossopetrae = require('glossopetrae'); Glossopetrae.quick(seed); // random language Glossopetrae.forLLM(seed); // optimized for AI learning Glossopetrae.hyperefficient(seed); // maximum token density Glossopetrae.alien(seed); // maximum exoticness
Read PAPER.md for the full factorial design (1,800 trials on key possession, task spec, and opacity).
What I take into client work
- Eval format sensitivity — Before you blame the model, swap surface form (comments, variable names, pseudocode vs code) in your eval harness.
- Agent monitoring — Log tool arguments and outcomes, not just natural-language chat. Opaque encodings are coming.
- Do not chase alien languages for production — The research point is about model behavior, not a recommendation to obfuscate your Next.js API routes.
If you are stress-testing agent guardrails or building code-generation pipelines and want help designing evals that catch format tricks, book a discovery call.