Spec language

A scenario spec is a versioned JSON document that you write as YAML. It names tables, says how many rows each one gets, and defines every column by exactly one source: a generator, a foreign key, an expression, a lookup or a constant. The engine compiles it into a plan, then generates and checks the data.

The spec's JSON Schema is generated from the engine's Rust types and published as spec.schema.json (dogm schema prints it). The field reference below is built from that file, so it matches the engine you run.

A complete example#

An online shop: customers, their orders, and one payment per order, with calculated totals, a lookup, two rules and a mask block.

spec_version: 1
locale: en-PH
tables:
  customers:
    rows: 1000
    primary_key: { column: customer_id, format: "CUS-{n:06}" }
    columns:
      full_name: { gen: person.full_name, pii: person.name }
      email:     { gen: person.email, from: full_name, unique: true, pii: person.email }
      city:      { gen: ph.city, weights: population }
      joined_on: { gen: date, between: [2024-01-01, 2026-10-01] }
  orders:
    rows_per_parent: { parent: customers, dist: uniform, min: 1, max: 3 }
    primary_key: { column: order_id, format: "ORD-{n:07}" }
    columns:
      customer_id:    { ref: customers.customer_id }
      order_date:     { gen: date, after: parent.joined_on, within_days: [1, 90] }
      item_count:     { gen: int, between: [1, 5] }
      unit_price_php: { gen: choice, values: [249, 399, 589, 1250, 2499], type: money }
      subtotal_php:   { expr: "item_count * unit_price_php" }
      shipping_php:   { expr: "if(subtotal_php >= 1500, 0, 99)", type: money }
      total_php:      { expr: "subtotal_php + shipping_php" }
      status:         { gen: choice, values: [processing, shipped, delivered, cancelled], weights: [1, 1, 2, 1] }
  payments:
    rows_per_parent: { parent: orders, exactly: 1 }
    primary_key: { column: payment_id, format: "PAY-{n:07}" }
    columns:
      order_id:   { ref: orders.order_id }
      method:     { gen: choice, values: [GCash, Card, Bank transfer, Cash on delivery] }
      amount_php: { expr: "parent.total_php" }
      status:     { map: parent.status, cases: { cancelled: refunded, processing: pending }, default: paid }
      paid_on:    { gen: date, after: parent.order_date, within_days: [0, 2] }
rules:
  - "orders.total_php == orders.subtotal_php + orders.shipping_php"
  - "payments.paid_on >= parent.order_date"
mask:
  key_ref: project_key
  columns:
    customers.full_name: { mode: consistent }
    customers.email:     { mode: consistent, from: customers.full_name, on_collision: suffix }

Top level#

The seed isn't part of the spec. It's a run input, like row overrides (--rows customers=48), so one approved spec serves any seed and size.

Tables#

Every generated table has a primary_key and exactly one of rows or rows_per_parent.

rows sizes a root table: an integer, or { per: other_table, ratio, min, max } to size it from another table, for example about one role per six candidates.

rows_per_parent makes a child table and sets how many children each parent row gets. It takes one of four forms:

• { parent, dist, min, max }, where dist is uniform, poisson or geometric (both with mean), or zipf (with exponent s);
• { parent, exactly: n };
• { parent, count: "<expression>" } over the parent's columns, such as count: "parent.reply_count";
• { parent, by: parent.column, cases: { value: <form> }, default: <form> }, which picks a form by a parent column.

A parent row gets at most 100,000 children. A table has at most one owning parent; for many-to-many, add a join table with two ref columns, one to its owning parent and one picked.

primary_key names the key column and its format. {n} is the row's permuted number and {n:06} pads it to six digits; {uuid} is a deterministic version 8 UUID. The numbers are a permutation of the row index, so IDs don't reveal order or volume, and they're unique by construction.

Columns#

Each column has exactly one source:

Any column can also take:

• type, to override the inferred type: int, decimal, money, string, bool, date or datetime;
• nullable: 0.05, the probability of a null;
• unique: true, which the compiler accepts only where uniqueness holds by construction (see below);
• hidden: true, for a value other columns use but the output leaves out;
• pii, the column's personal-data type (person.name, person.email, ph.tin and so on). It labels the column in reports and is the default masking domain.

choice also takes by: column with cases: { value: [values] }, to choose from a list that depends on another column.

What a column can read#

A column reads columns defined before it in the same table, and also:

• prev.column, the previous sibling row (null for the first);
• ordinal, the 0-based position among its siblings, or the row number in a root table;
• parent.column, parent.parent.column and so on up the owning-parent chain.

The compiler orders evaluation and rejects cycles. Here sibling replies alternate between customer and agent, and each one is sent after the previous reply:

spec_version: 1
tables:
  tickets:
    rows: 200
    primary_key: { column: ticket_id, format: "TCK-{n:05}" }
    columns:
      opened_at:   { gen: datetime, between: [2026-01-05T00:00, 2026-03-26T00:00] }
      reply_count: { gen: int, between: [1, 4] }
  replies:
    rows_per_parent: { parent: tickets, count: "parent.reply_count" }
    primary_key: { column: reply_id, format: "RPL-{n:06}" }
    columns:
      ticket_id:   { ref: tickets.ticket_id }
      author_role: { expr: "if(ordinal % 2 == 0, 'customer', 'agent')" }
      sent_at:     { gen: datetime, after: "coalesce(prev.sent_at, parent.opened_at)", within_minutes: [5, 600] }
rules:
  - "replies.sent_at > parent.opened_at"

Unique columns#

Chunks are generated independently, so unique: true has to hold by construction. The compiler accepts it only here:

Anything else is E_UNIQUE_UNSUPPORTED. Names are never unique; real names aren't either.

Rules#

rules are boolean expressions. Each one names exactly one table (payments.paid_on) and runs once per row of that table, where parent. means that table's owning parent. They run on generated values, before masking.

Rules are checks, not constraints. The generators are responsible for compliant data, and validation reports any row that breaks a rule, with the count and the first 20 failing row keys. A rule passes unless it's false, so a null passes, as in a SQL CHECK. A failing rule is a result in the report, never a compile error.

The mask block#

mask.columns maps table.column to a masking mode, and key_ref names a key supplied at run time; a spec never holds key material. A mask spec for real data uses the same format with source: external tables. The masking guide has both, and the modes.

Hash and versions#

• spec_hash is the SHA-256 of the spec's canonical JSON (RFC 8785), so reordering keys doesn't change it. dogm compile prints it.
• engine_version is recorded on every run. Output bytes for one spec hash, seed and set of row overrides are stable within an engine minor version, and any change to generator output, sampling, chunking or a writer bumps the minor version.
• Every released CLI version stays downloadable, so an old run is reproduced with the engine version recorded on it.

Generators#

This list comes from the catalog in engine 0.1.0. Every generator is deterministic and takes the spec's locale.

31 generators, from the engine's catalog (dogm_engine.catalog()).

Generated people are fictional by construction: names come from curated pools, and emails only use RFC 2606 example domains. Government ID formats (TIN, SSS, PhilHealth) are format-valid but not guaranteed nonexistent, so a generated number can match a real one, and reports label those columns as synthetic.

PII types#

These pii types have a fictional replacement pool, so consistent masking can use them as a domain:

• person.name
• person.full_name
• person.first_name
• person.last_name
• person.email
• ph.city
• ph.province
• ph.region
• ph.postal_code
• ph.address
• ph.mobile
• ph.tin
• ph.sss_no
• ph.philhealth_no
• company.name
• job.title

Limits#

In the browser demo, a table holds at most 10 million rows.

Field reference#

Every object in spec.schema.json, in file order. A field marked required must be present; the compiler checks combinations the schema can't express, such as exactly one of rows and rows_per_parent.

From dogm-engine/spec.schema.json, which the engine generates from its Rust types.

Error codes#

Errors are complete (all of them, not only the first), sorted by JSON path, and each has a path, a code, a message and sometimes a hint. The codes are a stable contract that tools can match on. This list is collected from the engine source.

E_SCHEMA_*

The spec's shape: syntax, wrong type, missing or unknown field, unsupported spec_version.

E_SCHEMA_DUPLICATE_COLUMNE_SCHEMA_EXTERNALE_SCHEMA_IDENTE_SCHEMA_MISSING_FIELDE_SCHEMA_ROWSE_SCHEMA_SOURCEE_SCHEMA_SYNTAXE_SCHEMA_TYPEE_SCHEMA_UNKNOWN_FIELDE_SCHEMA_VERSION

E_REF_*

References: unknown table or column, a second owning parent, a parent-chain cycle, a ref with no pick or expr.

E_REF_CYCLEE_REF_NOT_KEYE_REF_NO_PICKE_REF_SECOND_PARENTE_REF_TYPEE_REF_UNKNOWN_COLUMNE_REF_UNKNOWN_TABLE

E_EXPR_*

Expressions: parse errors, unknown column or function, type mismatch, forward reads, prev or ordinal misuse, overflow.

E_EXPR_ARITYE_EXPR_FORWARD_REFE_EXPR_LOOKUPE_EXPR_OVERFLOWE_EXPR_PARSEE_EXPR_PREV_MISUSEE_EXPR_RULE_TABLEE_EXPR_TYPEE_EXPR_UNKNOWN_COLUMNE_EXPR_UNKNOWN_FUNCTION

E_GEN_*

Generators: unknown generator, bad parameter, empty range, weights that don't match values.

E_GEN_BAD_PARAME_GEN_EMPTY_RANGEE_GEN_UNKNOWNE_GEN_WEIGHTS_MISMATCH

E_UNIQUE_*

unique: true on a column with no form that is unique by construction.

E_UNIQUE_UNSUPPORTED

E_MASK_*

The mask block: unknown mode or column, FF1 domain too small, foreign-key mode mismatch, free text not redacted.

E_MASK_BAD_PARAME_MASK_FF1_DOMAINE_MASK_FK_MISMATCHE_MASK_FREE_TEXTE_MASK_NO_DOMAINE_MASK_PII_UNMASKEDE_MASK_UNIQUEE_MASK_UNKNOWN_COLUMNE_MASK_UNKNOWN_MODE

E_LIMIT_*

Over an engine limit: tables, columns, depth, children per parent, rows, expression length.

E_LIMIT_CHILDRENE_LIMIT_COLUMNSE_LIMIT_DEPTHE_LIMIT_EXPR_LENGTHE_LIMIT_ROWSE_LIMIT_TABLES

E_INPUT_*

Run inputs, such as a row override for a table that isn't a root table.

E_INPUT_ROWS

E_AUTHORING_*

The AI authoring format, which the API converts to a spec before compiling.

E_AUTHORING_DUPLICATEE_AUTHORING_FIELDE_AUTHORING_SHAPEE_AUTHORING_VALUE

E_INTERNAL

An engine bug. Report it with the spec that caused it.

E_INTERNAL

Collected from the string literals in dogm-engine/dogm-core/src.