Skip to main content

Hosting the wheel in OneLake

By default every notebook installs the easyfabric wheel from the EasyFabric package index at https://core.easyfabric.net/download/pypi/. A platform whose notebooks may not reach that host can keep its own copy of the wheels in the Meta lakehouseA place where you store both "raw" data (like files) and "organized" data (like tables). It combines the best of a File Cabinet and a Database. and install from there.

Configuration​

In dp.<env>.yaml, replace WheelIndexUrl with WheelFindLinks. Its value is always the same folder in the default lakehouse:

 WheelFindLinks: /lakehouse/default/Files/WheelPackages
WheelPackageSpec: easyfabric

Set exactly one of the two; any other folder is refused at deploymentThe process of "pushing a button" to make your configuration actual, working software in the cloud.. WheelPackageSpec keeps choosing the version, as before: easyfabric follows the newest release, --pre easyfabric also takes previews (development and test only), and easyfabric==<version> pins one. The notebooks then install with pip install --no-index --find-links /lakehouse/default/Files/WheelPackages <spec>.

The path is read through the notebook's default lakehouse, which the deployment sets to the Meta lakehouse. A notebook that installs the wheel but has no default lakehouse fails at deployment, with a message naming the notebook.

--no-index means pip looks nowhere else, so the wheel's own dependencies (pyspark, pandas, pyyaml) must already be present in the Fabric runtime. They are, in both the Spark and the Python runtime. This needs easyfabric 0.9.9 or later: older versions also require deltalake, which the Spark runtime does not have, so the install fails there.

The deployment only rewrites the install line of notebooks it publishes, and deployNotebooks: Changed does not see a change to dp.<env>.yaml alone. When you switch an environment to WheelFindLinks, deploy once with deployNotebooks: All.

Filling the folder: the wheel sync pipeline​

Yml/wheel_sync.yml copies the wheels into the Meta lakehouse of each chosen environment. Create a pipelineAn automated "conveyor belt" that moves data from one place to another or performs a task automatically. on it in Azure DevOps and run it whenever you want a new release available; to run it on a schedule, add a schedules block (an example is in the file).

The file is yours, like the rest of the Yml folder: see Product and example. Adjust the environments and the pool block to the way your agents are selected, for example a named agent or a capability demand. If your platform does not have the pipeline yet, copy it from the platform zip of a release together with the steps it calls (Yml/templates/wheel.sync.yaml) and the scripts under Yml/scripts/Fabric that those steps run.

ParameterDescription
environmentsThe environments to fill, for example acc or [acc, prd] (default acc)
indexUrlThe package index page to mirror (default https://core.easyfabric.net/download/pypi/easyfabric/)

Each run:

  1. lists the wheels already in Files/WheelPackages of the Meta lakehouse;
  2. downloads only the wheels that are missing from it;
  3. checks every download against the sha256 checksum the index publishes, and stops before uploading anything when one does not match;
  4. uploads the new wheels with azcopy, never overwriting a file that is already there.

The pipeline does not look at WheelFindLinks, so you can fill the folder before you switch an environment to it. It runs in PowerShell only and needs no Python on the agent.

A published version never changes, so running the pipeline again is harmless: it uploads nothing when the folder is complete. Previews are mirrored too; which wheel a notebook installs is still decided by WheelPackageSpec.

The pipeline uses the same service connection, Key Vault secret and azcopy route as the deployment pipeline, so it needs no extra permissions.