Most of the valuable data in a company still lives in rows and columns. Customer transactions, inventory logs, CRM exports, warehouse tables. For twenty years the winning playbook was gradient-boosted trees, custom ETL, and weeks of feature engineering every time the schema shifted.
Large language models changed text and images. Tabular data mostly stayed on the old treadmill.
I kept seeing the same pattern in client projects: someone drops a CSV into ChatGPT, gets a plausible-sounding answer, and ships it to staging. Then production numbers fall apart because the model tokenized 100.50 as three unrelated pieces and never learned the column-wise distribution that actually matters.
A new class of tabular foundation models (TFMs) is trying to close that gap. They apply in-context learning to structured data: pass labeled rows as context, predict on new rows in one forward pass, no per-dataset training loop. Google shipped TabFM. Nvidia bought Kumo. TabPFN has a Nature paper and millions of downloads. TabICL pushes scale to hundreds of thousands of rows.
This post is the field guide I wish I had before the last three schema-change fire drills.
Why standard LLMs fail on tables
Tables are not documents. Swap two rows or two columns and the underlying problem does not change. Language models read left to right. That sequential bias fights the permutation invariance that tabular ML depends on.
Tokenizers make it worse. A price like 100.50 might become three tokens. Categorical codes get split arbitrarily. The statistical structure that XGBoost exploits column by column gets flattened into a string soup.
Compute cost piles on. Dump a wide enterprise table into a prompt and you burn context window on formatting noise before the model sees enough examples to generalize.
TFMs were built for this geometry from the start. They treat prediction as an in-context learning task: the labeled portion of your table is the prompt, query rows get scored in a single forward pass through frozen weights pretrained on synthetic or relational data.
The tabular foundation model landscape
Four names matter right now. They overlap in goal (zero-shot or few-shot tabular prediction) but differ in data shape, scale, and where they plug in.
| Model | Backer | Sweet spot | Key idea |
|---|---|---|---|
| TabFM | Google Research | Single-table classification and regression | Alternating row/column attention; BigQuery AI.PREDICT integration coming |
| KumoRFM | Nvidia (acquired Kumo, June 2026) | Multi-table relational warehouses | Graph neural networks over linked tables; Predictive Query Language |
| TabPFN | Prior Labs | Small-to-medium single tables | Prior-data fitted network; Nature-backed; strong on ≤10K rows |
| TabICL | Inria / SODA | Large single-table classification | Distribution-aware column embeddings; up to ~500K samples |
TabFM (Google)
Google Research TabFM announcement frames tabular prediction as in-context learning. Training rows plus labels become context; the model predicts on held-out rows without weight updates.
The implementation is scikit-learn compatible. Code is Apache 2.0 on
GitHub
, with PyTorch and JAX weights on
Hugging Face
. Read the license carefully: default pretrained weights ship under a separate non-commercial license, so production use needs a legal pass, not just a pip install.
Google is also pushing TabFM into
BigQuery
via an AI.PREDICT SQL path. That distribution play matters as much as the weights. Analysts who already live in SQL may get zero-shot classification without spinning up a training pipeline.
On benchmarks like TabArena (51 datasets), TabFM in zero-shot mode has been reported to beat heavily tuned gradient-boosted baselines. I have not rerun those evals myself. Treat vendor numbers as a hypothesis to test on your schema, not a purchase order.
KumoRFM (Nvidia)
Enterprise data is rarely one flat CSV. Customers link to orders link to products link to support tickets. Flattening that into a single table for XGBoost is where weeks disappear.
KumoRFM research paper models relational databases as graphs. A Relational Graph Transformer reasons across tables without manual feature joins. Nvidia acquired Kumo in June 2026 (reported around $400M) and is folding the stack into its enterprise AI software line.
The pitch: churn, fraud, and demand forecasts from warehouse-native queries in seconds, not months of pipeline work. Reddit, DoorDash, and Sainsbury's were named as customers before the deal. If you live in Snowflake or Databricks, this is the TFM family aimed at your stack.
TabPFN (Prior Labs)
TabPFN is the model I recommend when someone asks "can deep learning beat XGBoost on a 5,000-row Kaggle set without tuning for four hours?"
The Nature paper reports TabPFN beating strong baselines in 2.8 seconds that took tuned ensembles four hours on classification tasks. Prior Labs ships an open Python package , a managed API, and a no-code UX at TabPFN UX . Modified Apache 2.0 applies to the code; attribution requirements apply if you ship it in a product.
TabPFN v2 handles up to roughly 10K samples and 500 features out of the box. GPU helps. On CPU, keep expectations modest.
TabICL (Inria)
When your table is big, TabPFN's attention costs hurt. TabICL ( arXiv paper , open source ) uses a three-stage pipeline: column-wise embedding, row-wise interaction, then dataset-level in-context learning.
It targets classification on datasets up to hundreds of thousands of rows and hundreds of features, with inference reported up to 10x faster than TabPFN v2 at scale. TabPFN still wins on smaller sets in published comparisons. Pick by row count, not brand loyalty.
The inference trade-off nobody tweets about
TFMs trade training cost for inference cost.
Classic ML: painful upfront fit and hyperparameter search, then dirt-cheap scoring at millions of queries per second.
TFMs: no per-task training, but each prediction may run a forward pass over your entire context table. Latency and GPU dollars add up at high QPS.
| Phase | Best tool | Why |
|---|---|---|
| Schema in flux, many ad hoc questions | TFM | Skip rebuild cycles; test hypotheses in minutes |
| Stable schema, fraud scoring at 50K QPS | XGBoost / LightGBM | Sub-millisecond inference, pennies per million rows |
| Cold start, new product line, no labels yet | TFM zero-shot | Something beats nothing while you collect signal |
| Regulated production with audit trail | Tuned classic ML | Mature tooling, explainability libraries, known ops playbooks |
I call this a dual-stack roadmap, not a rip-and-replace.
Discovery: use TabPFN, TabICL, or TabFM to explore features, validate whether a signal exists, survive schema churn without retraining every week.
Production: once the problem and columns stabilize, pay the one-time cost to train XGBoost or LightGBM, wire proper monitoring, and optimize for inference economics.
Skipping the second step is how you end up with a $40K/month GPU bill to score credit card swipes.
Wiring TFMs into agentic workflows
The pattern I am most excited about is not standalone tabular prediction. It is LLM for reasoning, TFM for numbers.
An agent plans: "Should we restock SKU 4421 given last quarter's sell-through and current lead times?" The LLM should not guess the forecast from vibes. It routes a structured slice of warehouse data to a TFM, gets zero-shot predictions back, and turns that into a recommendation the user can audit.
That split matches how I already design client systems: language model for orchestration and explanation, specialized model for the part that needs calibrated probabilities on structured fields.
MCP tool servers, SQL agents, and warehouse connectors are the glue. The TFM does not need to chat. It needs a clean table in and predictions out.
What I would run this week
If you are evaluating TFMs on real work data, three checks beat reading another hype thread:
- Baseline honestly. Train a tuned XGBoost with the same train/val split. Compare AUC or RMSE, but also wall-clock from raw export to first prediction.
- Schema perturbation. Shuffle column order, add a nullable column, rename a field. See which stack breaks.
- Inference math. Estimate QPS, rows per query, and GPU cost at peak. If the answer scares you, plan the production handoff to trees on day one.
Start local with TabPFN or TabICL if you have a single table under 100K rows. If your truth lives across six normalized tables, read the KumoRFM paper and watch what Nvidia ships into Nvidia AI enterprise tooling. If you are all-in on Google Cloud, track the BigQuery TabFM rollout.
Licensing and ops footnotes
Before any production commit:
- TabFM default weights: non-commercial license separate from Apache code
- TabPFN: attribution requirements in modified Apache terms
- KumoRFM: enterprise / Nvidia SDK path post-acquisition
- All TFMs: GPU memory scales with context size; test on hardware that matches prod
None of this replaces a data contract, PII review, or model monitoring. It just shortens the path from "we have a CSV" to "we have a calibrated guess."
Bottom line
Tabular foundation models do not kill XGBoost. They kill the excuse that every new table needs a six-week science project before you know if the signal is real.
Use TFMs when exploration speed and schema volatility matter. Use classic ML when latency, cost per prediction, and throughput dominate. Wire both behind agents when you need language and numbers in the same answer.
If you are stitching warehouse data into AI agents or trying to pick a scoring stack before a production cutover, book a free discovery call. I am happiest when the demo survives your messiest export, not a sanitized Kaggle file.