Gold Partitioning

Gold fact tables can grow to hundreds of millions of rows, and reloading the whole table on every run quickly becomes the slowest step in the pipeline. Gold partitioning lets you refresh a fact table one slice at a time — replacing only the partitions present in the incoming batch and leaving the rest of the table untouched.

This is not Tabular partitioning

There are two independent "partition" concepts in EasyFabric, and it is easy to mix them up:

• Gold (lakehouse) partitioning — described on this page. A physical SYSTEMPARTITION column on the Gold Delta table, used by the appendwithdelete load type to overwrite a slice of a fact table. This governs how data is written to the lakehouse.
• Tabular (semantic-model) partitioning — how the Power BI / Fabric semantic model is sliced for refresh, using date-based partition schemes (yyyy, yyyyMM, yyyyMMdd), filters, and history annotations. See Tabular → Partitioning.

They are separate systems: a Gold table can carry a SYSTEMPARTITION column and the semantic model on top of it can additionally be split into import partitions. This page is only about the former.

Gold load types

The write mode is set with load_type on the LoadConfig you pass to modelloader. LoadConfig is a runtime parameter bag — you construct it in the Gold notebook, it is not read from model.yaml.

load_type	Behaviour
full (default)	Overwrites the entire target table with the incoming DataFrame.
append	Appends the incoming rows. Nothing is deleted — removing or replacing rows is the calling notebook's responsibility.
appendwithdelete	Partition-scoped overwrite: deletes the partitions present in the batch, then appends the batch. This is the incremental-refresh primitive for fact tables.

merge exists in the LoadType enum but is not wired up for the Gold layer — requesting it raises Load type ... is not implemented. Use appendwithdelete for incremental fact loads.

The SYSTEMPARTITION column

Gold partitioning is column-based. Each row of a fact table carries a partition value in a column named SYSTEMPARTITION. The value is a discrete string that identifies the slice the row belongs to — for example a load date, a year-month, or a source batch id. Rows that share a SYSTEMPARTITION value form one partition.

Partition handling applies to fact tables only (TableType: Fact). Dimension tables are keyed on their surrogate/business key instead and have no partition column.

Producing the column

The SYSTEMPARTITION column must already be present on the fact source you hand to modelloader. It flows down from ingestion: mark the source attribute(s) that make up the partition with IsPartition in the object configuration (see the IsPartition field). The Gold loader does not synthesise the column for you — a fact source without it cannot be loaded with appendwithdelete. The appendwithdelete delete step keys on the SYSTEMPARTITION name specifically, so keep the default partition column name for incremental fact loads.

How appendwithdelete works

When you load a fact table with load_type="appendwithdelete", the loader performs a partition-scoped swap:

1. It reads the distinct SYSTEMPARTITION values present in the incoming DataFrame.
2. It issues DELETE FROM <fact_table> WHERE SYSTEMPARTITION IN (<those values>), removing every existing row in those partitions.
3. It appends the incoming DataFrame.

The net effect is that every partition present in the batch is fully replaced, while partitions absent from the batch are left exactly as they were. There is no per-row matching — the unit of replacement is the whole partition.

from easyfabric import load_data_gold, Model, LoadConfig

mdl = Model.from_yaml_file("Files/Model/DM/model.yaml")

# df_sales holds only the partitions you want to refresh, e.g. the last 3 load dates.
# Each row carries a SYSTEMPARTITION value identifying its slice.
load_config = LoadConfig.from_dict({
    "model_object_name": "F_Sales",
    "load_type": "appendwithdelete",
})

load_data_gold.modelloader(data_frame=df_sales, load_config=load_config, model_config=mdl)

Worked example

Suppose F_Sales already holds partitions 2026-07-19, 2026-07-20, and 2026-07-21, and your source query returns rows for 2026-07-21 and 2026-07-22:

Partition	Before	Incoming batch	After
2026-07-19	1.2M rows	—	1.2M rows (untouched)
2026-07-20	1.1M rows	—	1.1M rows (untouched)
2026-07-21	0.9M rows	1.0M rows	1.0M rows (replaced)
2026-07-22	—	1.3M rows	1.3M rows (added)

The batch's two partitions are deleted and re-inserted; the two older partitions are never read or rewritten.

Incremental fact loads

There is no "refresh the last N partitions" loop inside the loader — the refresh scope is simply whatever your source query returns. To run an incremental fact load you:

1. Restrict the source query in the Gold notebook to the partitions you want to refresh (for example, the recent load dates or the open period).
2. Ensure each row carries its SYSTEMPARTITION value.
3. Load with appendwithdelete.

Because the delete is scoped to exactly the partitions in the batch, this is safe to re-run: reloading the same partitions produces the same result, and widening the query to include older partitions re-swaps those too. For a first load, or to rebuild the whole table, use full instead.

Relationship to the semantic model

The SYSTEMPARTITION column and the delete-then-append behaviour described here are purely about how the Gold lakehouse Delta table is written. Slicing the Power BI / Fabric semantic model on top of that table into import partitions — by year, month, or filter, with history partitions that are skipped on incremental refreshes — is configured separately via the Partitions: block on the Tabular table. See Tabular → Partitioning for that.