Skip to main content

Configuration config

The general configuration that is used to generate the objects with the requested settings.

Fabric Configuration

The configuration needed to build the model

NameTypeDescription
EnvironmentStringEnvironment (PRD, TST, ACC, DEV) used in Azure Devops pipelineAn automated "conveyor belt" that moves data from one place to another or performs a task automatically.
EnvironmentDescriptionStringLonger name for the environment like Production
ServiceConnectionStringName of the service connection in Azure DevOps
VsSolutionNameStringFilename of the Visual Studio solution
ApiconfigfileStringFilepath to the api configuration file
MetaFolderStringPath on the Meta lakehouseA place where you store both "raw" data (like files) and "organized" data (like tables). It combines the best of a File Cabinet and a Database. in fabric for the meta data files
TenantIdStringThe tenant id where this is deployed
ApplicationClientIdStringThe application id that is used to deploy things that require a service principal for authentication.
ApplicationKeyvaultSecretStringThe secret name in the keyvault for the service principal
ColumnNamesToLowerboolAll columns generated will be lowercase (except for the system columns, those are in UPPER case
ColumnNamesAutoFixboolIf a column has special characters not suitable for delta table those will be replaced
RemoveNewLineFromScriptsbool
BronzeConnectionPrefixboolBronze objects will have a connection prefix
DataplatformFolderStringThe root dataplatform folder
TempFolderStringThe temp folder for local temp in the project.
DatabaseFolderStringFolder where the object yamlA simple way to write configurations. It's basically a list that computers can read easily. files are located
DatabaseOutputFolderStringFolder where the generated table definitions and view definitions will be saved
NotebooksFolderStringFolder where the notebook folders are located
NotebooksOutputFolderStringFolder where the generated notebooks will be saved
RequestFolderStringFolder where the requests will be saved. Useful for debugging purposes
FabricProjectNameStringThe project where the generated objects will be saved into
DimObjectPrefixStringA prefix for a dimension table in Gold
FactObjectPrefixStringA prefix for a fact view in Gold
DimSurrogatekeyPrefixStringA prefix for a surrogate key column in Gold
TransformationObjectPrefixStringA prefix for a transformation table in Gold
WorkspaceStringWorkspace for this environment in Fabric
KeyvaultStringKeyvault for this environment
SchemaHistoryStringHistory schema used for history tables (default='his')
FunctionAppNameStringThe name of the Azure Function used for the triggers
ResourceGroupNameStringThe Azure resource group holding the environment's resources (e.g. the function app)

Fabric Lakehouse

NameTypeDescription
Lakehouse (Required)StringName of the lakehouse for the given layer
Layer (Required)FabricMedallionLayerLayer for the medallion architectureA way of organizing data into three stages: Bronze (raw), Silver (cleaned), and Gold (ready for reports).: Bronze, Silver, Gold, Meta
Workspace (Required)StringFabric Workspace name (workspace has to exist and user needs access)

Tabular Models

NameTypeDescription
Model (Required)StringName of the model (e.g. DM). Used to select the model by name in the loaders
TabularName (Required)StringName of the tabular database/model as deployed to the workspace
CompatibilityLevel (Required)NumberCompatibility level of the tabular model (e.g. 1700)
AddMembersbool
DatabaseSchemaString
WorkspaceString
DefaultPowerBIDataSourceVersionString
LoginTypeString
PartitionDeploymentbool
RoleDeploymentbool
LakehouseString
LakehouseSourceConnectionStringName of the lakehouse connection used as the tabular datasource
BimFolderString
BimFileString
SourceQueryCultureString
ModelFolderString
ModelFileString
DerivedModelFileStringLinks a derived model file, resolved in the same ModelFolder
DataSourceTypeString
DataSourceNameString
DatasetString
TabularSystemFolderString
MeasureTemplateFileString
FormatStringFileString
RoleFileStringFile with the role definitions (e.g. roles.yaml)

Power BI Reports

NameTypeDescription
GroupNameStringA name for the group of reports. Used to reference this setting in Azure DevOps pipeline
FilePathStringRelative filepath to the reports.yaml config file
ReportFolderStringRelative Path to the folder with the PBIX files

Blob triggers

Defined under blobtriggers: — each entry connects an Azure storage container to a notebook that is triggered when a matching file is dropped. See Blob file trigger for the full setup.

Backups

Defined under backups: — each entry defines a backup of a layer to an Azure storage account:

NameTypeDescription
BackupNameStringName of the backup definition
EnabledboolWhether this backup runs (default=true)
StorageAccountStringAzure storage account to back up to
ContainerStringContainer in the storage account
LayerStringLayer to back up (e.g. bronze)
SchemaString[]Schema(s) to back up (e.g. [his])
TableFolderStringFolder with the object YAML files that define the tables to back up
ExceptFoldersString[]Folders to exclude

Bronze, Silver and Gold Table settings

Can be set as bronzetablesettings, silvertablesettings and goldtablesettings

NameTypeDescription
delta.enableChangeDataFeedbool
delta.autoOptimize.optimizeWritebool
delta.autoOptimize.autoCompactbool
delta.logRetentionDurationString
delta.deletedFileRetentionDurationString
spark.sql.parquet.vorder.defaultbool
spark.databricks.delta.optimizeWrite.enabledbool
spark.sql.parquet.vorder.defaultbool
delta.stats.extended.collectbool
delta.stats.extended.injectbool

Those settings are added to the table creation script when generated.