Skip to main content

load_meta_data

get_objects_by_folder​

def get_objects_by_folder(
        tablefolder: str,
        config_manager: ConfigManager) -> list[TableConfig] | None

Loads metadata configurations from a specified folder or file and returns a list of TableConfig objects. If a single file is specified, metadata will be loaded only from that file. Otherwise, the function traverses the specified folder and loads metadata from all YAML files.

Arguments:

  • tablefolder str - The path to the folder or file containing YAML metadata files. If it starts with "Files/", it will be resolved to the default lakehouse path.
  • config_manager ConfigManager - An instance of ConfigManager used to initialize and validate configurations.

Returns:

  • List[TableConfig] - A list of TableConfig dataclass instances created from the loaded YAML metadata.

Raises:

  • Exception - If config_manager is not initialized or if an error occurs during metadata loading and config_manager.stop_at_error is set to True.

get_model_by_file​

def get_model_by_file(modelfile: str, config_manager: ConfigManager) -> Model

Loads a Model object by reading a given YAML file. Ensures that the configuration manager is initialized and sets up logging for the operation.

If the input file path starts with a specific prefix, it is converted to an absolute path. The function validates the file type to ensure it is a YAML file and parses its content to create a Model instance.

Arguments:

  • modelfile - The path to the metadata YAML file. It can be relative or absolute. If it starts with 'Files/', it is converted to the absolute path in the lakehouse directory.
  • config_manager - An instance of ConfigManager that is responsible for initialization and managing configurations.

Returns:

  • Model - The Model object created from the provided YAML file.

Raises:

  • Exception - If the configuration manager is not initialized.
  • Exception - If the provided file is not a YAML file.
  • Exception - If the stop-at-error policy is enabled in the configuration manager and an error occurs during the process.