PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models

PluRel generates synthetic relational databases through three sequential stages, modeling databases at the schema level (table structure), connectivity level (row-level relationships), and feature level (cell values).

Database schemas are represented as directed acyclic graphs (DAGs), where nodes correspond to tables and edges represent inter-table relationships. A topological ordering of the DAG determines the table generation sequence: initial tables are synthesized independently, while subsequent tables are generated conditionally on their parent tables. Tables are categorized as entity tables (out-degree ≥ 1) or activity tables (remaining nodes). The number of rows, feature columns, and graph topology are all configurable:

from plurel import Config, DatabaseParams, Choices

config = Config(
    database_params=DatabaseParams(
        num_tables_choices=Choices(kind="range", value=[5, 10])
    ),
    schema_file="path/to/schema.sql",  # optional: generate from SQL schema
    cache_dir="~/.cache/plurel",       # optional: cache generated databases
)

Row-level connectivity between table pairs is populated through primary-foreign key relationships. Each table contains feature columns, a primary key column indexing rows, and optional foreign key columns referencing parent table rows. A Hierarchical Stochastic Block Model (HSBM) controls row-level information locality, allowing rows to depend on many parent rows or a small subset, enabling flexible dependency modeling.

Each table is associated with its own SCM defined by a causal graph encoding dependencies between variables. Feature columns correspond to a subset of SCM nodes, supporting numeric, categorical, and boolean types. Tables with foreign keys condition on feature nodes from parent table SCMs.

Temporal correlations are modeled through exogenous inputs combining trend (power-law), cycle (sinusoidal), and fluctuation (random noise) components. Node mechanisms use a projection-reconstruction design: predecessor values and parent table features project into a shared latent space via MLPs, aggregate with exogenous inputs, then reconstruct to the target data type.

Synthesizing a complete relational database requires only a seed and a configuration object. The resulting dataset is fully compatible with RelBench:

from plurel import SyntheticDataset, Config

# create relbench compatible dataset
dataset = SyntheticDataset(seed=0, config=Config())

# create database which can be cached via relbench APIs
db = dataset.make_db()

For large-scale generation, databases can be synthesized in parallel:

pixi run python scripts/synthetic_gen.py \
    --seed_offset 0 \
    --num_dbs 1000 \
    --num_proc 16 \
    --preprocess

We investigate scaling along two axes: the number of synthetic relational databases N (diversity) and the number of pretraining tokens S (size). We find that the validation loss exhibits power-law scaling along both dimensions:

L(N) = A_N · N^−α_N + C_N      L(S) = A_S · S^−α_S + C_S

We conduct a comprehensive grid of experiments across (N, S) ∈ {8, 16, 32, 64, 128, 256, 512, 1024} × {0.5B, 1B, 2B, 4B, 8B, 16B, 32B} tokens. Both N and S must be scaled simultaneously to optimize loss; scaling one dimension alone produces non-monotonic U-shaped curves indicating underfitting or overfitting.

We evaluate whether synthetic pretraining benefits transfer to real-world relational databases from RelBench. We compute the masked token prediction loss on the validation splits of all 18 RelBench tasks using the same synthetic scaling configurations.

Low-diversity settings (8–32 synthetic databases) exhibit poor scaling on RelBench, where larger datasets tend to be suboptimal as loss curves show an upward trend. This undesirable behavior is mitigated as the number of databases increases, with benefits from scaling dataset size becoming evident at higher database counts. Models show consistent zero-shot transfer capability despite the distribution mismatch between synthetic and real data.

Synthetic pretraining creates effective base models for downstream tasks when combined with continued pretraining on real data. Using a leave-one-database-out protocol across six RelBench datasets, we show that synthetic + real pretraining consistently outperforms real-data-only pretraining:

Synthetic data alone is insufficient for robust zero-shot transfer, highlighting that continued pretraining on real data is critical for distribution alignment. The combination of synthetic and real pretraining yields particularly pronounced benefits for behavior-driven and continuous-valued prediction tasks.

Classification (AUROC %)

Regression (R² %)