Skip to main content

Intro about the basics

What is EasyFabric?​

EasyFabric automates your entire data platform workflow. As a developer, you only write notebooks and YAML configurations in Visual Studio - EasyFabric handles everything else automatically through Azure DevOps pipelines.

The general idea is this: Create a bunch of yaml files with your wishes and a complete medaillon architecture will be created for you.

Overview of all components involved​

Seems overwhelming at first probably. But you will make it.

EasyFabric Architecture Overview

EasyFabric is an end-to-end data platform automation solution that streamlines the entire data pipeline process, from ingestion to visualization. It combines the power of Azure services with automated DevOps pipelines to deliver a seamless data platform experience.

How Does It Work?​

1. Write Your Code​

Open Visual Studio and:

  • Define configurations in YAML files
  • Create notebooks (.ipynb) for your data transformations
  • Commit your changes to the repository

Fabric workspace

2. EasyFabric Takes Over​

The moment you push your code, EasyFabric deploys through DevOps pipelines the following:

  • Your notebooks
  • Creates and updates lakehouse tables
  • Generates SQL endpoint views
  • Updates semantic models
  • Deploys Power BI reports

At this point all objects are available in the workspace of fabric: Fabric workspace

All objects in the lakehouse will be empty when it's the first time they are deployed. To load data, you need to run a notebook or two. Loading data is a combination of the Wheel package, yaml config files and notebook(s).

3. Built-in Data Processing​

The included wheel package automatically:

  • Loads data from your sources to Bronze layer
  • Transforms Bronze data to Silver with quality checks
  • Converts Silver to Gold with business logic

Key Components​

1. EasyFabric Wheel Package​

The core Python wheel package provides:

  • Data loading from source systems to Bronze layer
  • Built-in data quality and transformation rules for Silver layer
  • Business logic implementation for Gold layer
  • Pre-built connectors and data handling utilities
  • Metadata management capabilities

2. Development Environment​

Developers only need to focus on:

  • Visual Studio for development
  • Creating/modifying YAML configuration files
  • Developing (parts of) notebooks for custom transformations

3. Automated Pipeline Components​

EasyFabric handles the automated deployment of:

  • Lakehouse objects (tables)
  • SQL endpoint views for Gold
  • Metadata files (yaml configuration)
  • Notebooks
  • Semantic models
  • Power BI reports

4. Azure Integration​

Seamlessly integrates with:

  • Azure Storage
  • Azure Key Vault for secrets management
  • Azure EntraID
  • Fabric Capacity management (on the roadmap)

Azure components

In this documentation all configuration files are explained. So take some time to understand the defaults and start deploying a complete fabric environment.