dlt.extract.hints

table_schema_to_hints

def table_schema_to_hints(table: TTableSchema) -> TResourceHints

[view_source]

Converts table schema into resource hints in place. Does not attempt to extract primary keys, additional table properties etc. into respective TResourceHints fields. Suitable only to be used for a result of utils.new_table.

make_hints

def make_hints(
    table_name: TTableHintTemplate[str] = None,
    parent_table_name: TTableHintTemplate[str] = None,
    write_disposition: TTableHintTemplate[TWriteDispositionConfig] = None,
    columns: TTableHintTemplate[TAnySchemaColumns] = None,
    primary_key: TTableHintTemplate[TColumnNames] = None,
    merge_key: TTableHintTemplate[TColumnNames] = None,
    schema_contract: TTableHintTemplate[TSchemaContract] = None,
    table_format: TTableHintTemplate[TTableFormat] = None,
    file_format: TTableHintTemplate[TFileFormat] = None,
    additional_table_hints: Optional[Dict[str,
                                          TTableHintTemplate[Any]]] = None,
    references: TTableHintTemplate[TTableReferenceParam] = None,
    incremental: TIncrementalConfig = None,
    nested_hints: Optional[TTableHintTemplate[Dict[
        TTableNames, TResourceNestedHints]]] = None
) -> TResourceHints

[view_source]

A convenience function to create resource hints. Accepts both static and dynamic hints based on data.

This method accepts the same table hints arguments as dlt.resource decorator.

DltResourceHints Objects

class DltResourceHints()

[view_source]

table_name

@property
def table_name() -> TTableHintTemplate[str]

[view_source]

Get table name to which resource loads data. May return a callable.

columns

@property
def columns() -> TTableHintTemplate[TTableSchemaColumns]

[view_source]

Gets columns' schema that can be modified in place

compute_table_schema

def compute_table_schema(item: TDataItem = None,
                         meta: Any = None) -> TTableSchema

[view_source]

Computes the table schema based on hints and column definitions passed during resource creation. item parameter is used to resolve table hints based on data. meta parameter is taken from Pipe and may further specify table name if variant is to be used

compute_nested_table_schemas

def compute_nested_table_schemas(root_table_name: str,
                                 naming: NamingConvention,
                                 item: TDataItem = None,
                                 meta: Any = None) -> List[TTableSchema]

[view_source]

Compute the table schema based on the current and all nested hints. Nested hints are resolved recursively.

apply_hints

def apply_hints(
    table_name: TTableHintTemplate[str] = None,
    parent_table_name: TTableHintTemplate[str] = None,
    write_disposition: TTableHintTemplate[TWriteDispositionConfig] = None,
    columns: TTableHintTemplate[TAnySchemaColumns] = None,
    primary_key: TTableHintTemplate[TColumnNames] = None,
    merge_key: TTableHintTemplate[TColumnNames] = None,
    incremental: TIncrementalConfig = None,
    schema_contract: TTableHintTemplate[TSchemaContract] = None,
    additional_table_hints: Optional[Dict[str,
                                          TTableHintTemplate[Any]]] = None,
    table_format: TTableHintTemplate[TTableFormat] = None,
    file_format: TTableHintTemplate[TFileFormat] = None,
    references: TTableHintTemplate[TTableReferenceParam] = None,
    create_table_variant: bool = False,
    nested_hints: TTableHintTemplate[Dict[TTableNames,
                                          TResourceNestedHints]] = None
) -> Self

[view_source]

Creates or modifies existing table schema by setting provided hints. Accepts both static and dynamic hints based on data.

If create_table_variant is specified, the table_name must be a string and hints will be used to create a separate set of hints for a particular table_name. Such hints may be retrieved via compute_table_schema(meta=TableNameMeta(table_name)). Table variant hints may not contain dynamic hints.

This method accepts the same table hints arguments as dlt.resource decorator with the following additions. Skip the argument or pass None to leave the existing hint. Pass empty value (for a particular type i.e. "" for a string) to remove a hint.

Arguments:

• table_name TTableHintTemplate[str] - name of the table which resource will generate

• parent_table_parent str, optional - A name of parent table if you want the resource to generate nested table. Please note that if you use merge, you must define root_key columns explicitly

• incremental Incremental, optional - Enables the incremental loading for a resource.

  Please note that for efficient incremental loading, the resource must be aware of the Incremental by accepting it as one if its arguments and then using are to skip already loaded data. In non-aware resources, dlt will filter out the loaded values, however, the resource will yield all the values again.

• Returns - self for chaining