The AlphaSignal digest highlighted a pattern I've seen in production: two small models team up to finish a job neither completes alone.
For document forms, the paper is FormGym (EACL 2026). The helper tool is FieldFinder. Together they explain why your "smart PDF agent" keeps signing the footer as "Your Name."
The task everyone underestimates
End-to-end form filling means: given external data about a user, populate every field in a document-style PDF or scan. Not a web <form>. Not key-value JSON extraction. Placement matters.
FormGym benchmarks 432 fields across 55 documents and 3 task types, requiring knowledge of 236 features per user on the hardest set.
FormGym: Doing Paperwork with Agents (EACL 2026)Baseline vision-language agents (VLAs) score ≤1% in most conditions. Not a typo. GUI agents do better (10.6% to 68.0%) but pay latency and dollar costs per action.
The failure mode is almost always the same: field localization. Models understand the question. They don't know where on the page to write the answer.
FieldFinder: the auxiliary specialist
FieldFinder is a field localization tool. VLAs call it zero-shot to get accurate input regions before typing.
Results from the paper (selected conditions):
| Setup | Before FieldFinder | With FieldFinder |
|---|---|---|
| GPT-4o on FUNSD | 2% | 56% |
| Claude on FUNSD | 32% | Equal or better |
| GPT-4o on Auto Loans | low single digits | up to +16.9 pts |
Maximum swing reported: 2% → 56% on FUNSD for GPT-4o. That's the "two models" story in one number.
Why one big model isn't enough
Monolithic VLAs get semantic understanding cheaply. Spatial precision on scanned paperwork is a different skill. Training one 70B model to excel at both is expensive. Routing localization to a focused tool is cheaper and debuggable.
You can see which step failed:
- Wrong field box → FieldFinder or layout model issue
- Right box, wrong value → retrieval or reasoning issue
That attribution saves weeks in ops automation projects.
GUI agents vs VLA + FieldFinder
Computer-use agents (Operator, Claude Computer Use) navigate UI like humans. They can complete complex flows but:
- Often blow the 5-minute timeout on multi-page loan packets
- Cost more per field than a batched VLA pipeline
- Still make localization errors, usually less absurd than raw VLAs
FormGym's authors position FieldFinder as the middle path: VLA brains, specialist eyes.
FormGym arXiv paperParallel pattern: small models collaborating elsewhere
The same digest cycle surfaced related ideas:
- CORE (Collaborative Reasoning): 3B + 4B model pairs cross-teach on math reasoning, boosting Pass@2 well above single-model training
- SLM-MUX (ICLR 2026): orchestrate multiple small LMs to beat a 72B monolith on GPQA and GSM8K
FormGym is the applied ops version: don't scale one model until you've decomposed the task.
| Framework | Collaboration style |
|---|---|
| FormGym + FieldFinder | Tool-augmented VLA for spatial subtask |
| CORE | Training-time peer hints between SLMs |
| SLM-MUX | Router assigns complementary SLMs per step |
What I'd build for a client
- Classify forms (digital AcroForm vs scanned vs hybrid)
- Run layout + FieldFinder (or Surya-class OCR for block bounds) before any LLM writes text
- Validate with rule checks (SSN format, required fields, signature blocks)
- Human review queue for low-confidence placements only
Skip fully autonomous submit on regulated forms until you measure field-level accuracy, not demo GIF accuracy.
Bottom line
If your agent pipeline handles intake PDFs, insurance apps, or government forms, read FormGym before you buy another generalist VLM license. The win is architectural: pair models by subtask, not parameter count.
Building document automation for ops teams? Book a free call and we'll map localization tools to your form types.