Connection config

The Columns section lists the columns that belong to the table in the model. Each entry specifies the source column name and its data type, which is crucial for defining how data is stored and manipulated within the model.

Connection

Name	Description
ConnectionName	Unique name for the connection. This name is also used in the objects that refer to this. The folder and filename of the yaml should be equalto this name
ConnectionPrefix	Prefix that will be used for every object that is generated for this connection (unless otherwise set in the main configuration file)
ConnectionType**	Type of connection. See list of types. For example azblob for Azure Blobstorage
BronzeFolder	The folder that will be used to save the source files. Those will be save in the bronze lakehouse in the Files section.
FileType	Type of file for this connection. Parquet, CSV, JSON, XML and more
FileExtension	Extension of the file

** = can be set at the object as well

Filetype specific settings

** CSV **

Name	Description
Delimiter	The delimiter used in the CSV. For example: , or ; and so on
Multiline	Set to True if the source file uses multiline
Escape	The escape character that is used
Header	Is the header provided
Dateformat	The dateformat that is used in the source file. For example yyyy-MM-dd. This dateformat will be used to convert to the datetime in resulting delta table in bronze

** JSON **

Name	Description
JsonPath	The root path where to start processing the json. If the items are in an array called rows, supply here: $.rows

** XML **

Name	Description
No extra options	XML is processed by element names

** Parquet **

Name	Description
No extra options	Parquet is read and matched on columnnames

Connectiontype specific settings

azblob

Name	Description
KeyvaultSecretConnectionstring	In case of azblob, the connectionstring is retrieved from the keyvault.
Container	Container on azure blob to connect to
Folder	Folder in the container to look for files

fabricfile

Set to Fabricfiles when files are delivered to fabric directly by a third-party or custom program. In this case the loader will just look into the defined fabric folder in the Files section and process the file accordingly.

Name	Description
BronzeFolder	Place where the files that should be loaded are dropped by the other party

The folder is defined in the object config itself. It will look in the defined bronzefolder for the SourceTable or SourceFilter.

Basic: Example of connection with a csv on azure blob storage

    ConnectionName: exa-azblob
    ConnectionPrefix: exa
    ConnectionType: azblob
    BronzeFolder: Example
    KeyvaultSecretConnectionstring: kvexample
    Container: Example001
    Sourcefolder: ExaFolder
    FileType: csv 
    FileExtension: csv
    Delimiter: ","
    Multiline: True
    Escape: '"'
    Header: true

This example defines a connection to a azure blobstorage, for the authentication to the blob storage, the connectionstring from the keyvault will be used. The settings for the csv are also set here. Those settings will be used when an object is loaded into the bronze layer of fabric.

Basic: Example of connection with a parquet file that is already on Fabric

    ConnectionName: exa-fabricfiles
    ConnectionPrefix: fbf
    ConnectionType: fabricfiles
    BronzeFolder: Example
    FileType: parquet 
    FileExtension: parquet

This example defines a connection to the Fabric Bronze layer. In this case it will look for files in the Files section in the folder Example.