Skip to main content

loaders.json

load_json_bronze​

def load_json_bronze(object_info: ObjectInfo, table_config: TableConfig,
                     config_manager: ConfigManager) -> DataFrame

Loads a JSON file into the Bronze layer of a data lakehouse and processes it as per the configuration. Handles column mapping, null column substitution, and schema validation.

Arguments:

  • object_info ObjectInfo - Encapsulates metadata about the object being loaded, including its file path and related properties.
  • table_config TableConfig - Provides the table-level configuration, such as connection configuration and column-mapping details.
  • config_manager ConfigManager - Manages configuration for lakehouse paths, schema details, and metadata for all layers.

Returns:

  • DataFrame - A Spark DataFrame containing the data loaded from the JSON file, mapped and aligned with the schema and configurations.

Raises:

  • Exception - If Bronze lakehouse configuration is not found.
  • Exception - If any error occurs during the JSON file loading or processing.