stratified_packager.processing.dedup

Deduplication of packaged layers sharing one normalized data source (SPEC §12).

Groups packaged vector layers by their provider + normalized decodeUri components, merges each group onto a primary member (whose table hosts the union of every member’s matches and kept fields), and extends warm marking across a shared table. Runs on the algorithm thread during Phase A; all user-facing messaging flows through the passed feedback.

Functions

apply_dedup(material, feedback)

Fold layers sharing one normalized data source into dedup groups (§12).

normalized_source_key(provider_type, source, ...)

Normalize a provider/source pair into the §12 dedup group key.

promote_warm_groups(material, feedback)

Extend warm marking to whole dedup groups (§11/§12).

source_group_key(layer, feedback)

Build the §12 dedup group key: provider + normalized decodeUri components.
stratified_packager.processing.dedup._merge_group(members, feedback)[source]

Merge one dedup group onto its primary member (§12).

Parameters:

  • members (list[_LayerPrep]) – The group, in tree order (the first unfiltered member — else the first — becomes the primary).

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Return type:

None

stratified_packager.processing.dedup._subset_columns(subset_sql, layer)[source]

Name the layer fields referenced by a subset string (§12 kept-field rule).

A pragmatic identifier scan: a field counts as referenced when it appears quoted or as a standalone word inside the subset SQL.

Parameters:

  • subset_sql (str) – The provider-native subset SQL (possibly empty).

  • layer (QgsVectorLayer) – The layer whose field names are scanned for.

Return type:

list[str]

Returns:

The referenced field names, in field order.

stratified_packager.processing.dedup.apply_dedup(material, feedback)[source]

Fold layers sharing one normalized data source into dedup groups (§12).

The first unfiltered member in tree order becomes the primary (the first member when every member is filtered): its table hosts the union of every member’s match set and kept fields (always including the columns each member’s subset references). Runs before staging, so a staged group builds one staging copy through its primary instead of one per member (§8.2). A member whose subset is not portable to the delivered GeoPackage (it references tables or functions absent from it) is excluded from grouping and staged standalone, so its filter is materialized on the source provider rather than re-applied against a shared table it could never evaluate.

Parameters:
Return type:

None

stratified_packager.processing.dedup.normalized_source_key(provider_type, source, label, feedback)[source]

Normalize a provider/source pair into the §12 dedup group key.

Path-typed components are resolved and case-folded (decodeUri does neither) and identifier quoting is stripped.

Parameters:

  • provider_type (str) – The provider key (e.g. ogr, postgres).

  • source (str) – The provider source uri.

  • label (str) – A human label for debug messages (typically the layer name).

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Return type:

tuple[str, frozenset[tuple[str, str]]] | None

Returns:

The hashable key, or None when the source cannot be decoded.

stratified_packager.processing.dedup.promote_warm_groups(material, feedback)[source]

Extend warm marking to whole dedup groups (§11/§12).

Warm marking is effectively per exported table, and a dedup group shares one table: either it rides the warm cache or it does not. The §11 machinery caches only group primaries, so a mark on a non-primary member alone would otherwise vanish silently — promote the whole group instead.

Parameters:
Return type:

None

stratified_packager.processing.dedup.source_group_key(layer, feedback)[source]

Build the §12 dedup group key: provider + normalized decodeUri components.

Parameters:
Return type:

tuple[str, frozenset[tuple[str, str]]] | None

Returns:

The hashable key, or None when the source cannot be decoded.