stratified_packager.processing.dedup

Deduplication of packaged layers sharing one normalized data source (SPEC §12).

Groups packaged vector layers by their provider + normalized decodeUri components, merges each group onto a primary member (whose table hosts the union of every member’s matches and kept fields), and extends warm marking across a shared table. Runs on the algorithm thread during Phase A; all user-facing messaging flows through the passed feedback.

Functions

apply_dedup(material, feedback)

Fold layers sharing one normalized data source into dedup groups (§12).

normalized_source_key(provider_type, source, ...)

Normalize a provider/source pair into the §12 dedup group key.

promote_warm_groups(material, feedback)

Extend warm marking to whole dedup groups (§11/§12).

source_group_key(layer, feedback)

Build the §12 dedup group key: provider + normalized decodeUri components.
stratified_packager.processing.dedup._merge_group(members, feedback)[source]

Merge one dedup group onto its primary member (§12).

Parameters:

  • members (list[_LayerPrep]) – The group, in tree order (the first unfiltered member — else the first — becomes the primary).

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Return type:

None

stratified_packager.processing.dedup._subset_columns(subset_sql, layer)[source]

Name the layer fields referenced by a subset string (§12 kept-field rule).

A pragmatic identifier scan: a field counts as referenced when it appears quoted or as a standalone word inside the subset SQL.

Parameters:

  • subset_sql (str) – The provider-native subset SQL (possibly empty).

  • layer (QgsVectorLayer) – The layer whose field names are scanned for.

Return type:

list[str]

Returns:

The referenced field names, in field order.

stratified_packager.processing.dedup.apply_dedup(material, feedback)[source]

Fold layers sharing one normalized data source into dedup groups (§12).

The first unfiltered member in tree order becomes the primary (the first member when every member is filtered): its table hosts the union of every member’s match set and kept fields (always including the columns each member’s subset references). Runs before staging, so a staged group builds one staging copy through its primary instead of one per member (§8.2).

Parameters:
Return type:

None

stratified_packager.processing.dedup.normalized_source_key(provider_type, source, label, feedback)[source]

Normalize a provider/source pair into the §12 dedup group key.

Path-typed components are resolved and case-folded (decodeUri does neither) and identifier quoting is stripped.

Parameters:

  • provider_type (str) – The provider key (e.g. ogr, postgres).

  • source (str) – The provider source uri.

  • label (str) – A human label for debug messages (typically the layer name).

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Return type:

tuple[str, frozenset[tuple[str, str]]] | None

Returns:

The hashable key, or None when the source cannot be decoded.

stratified_packager.processing.dedup.promote_warm_groups(material, feedback)[source]

Extend warm marking to whole dedup groups (§11/§12).

Warm marking is effectively per exported table, and a dedup group shares one table: either it rides the warm cache or it does not. The §11 machinery caches only group primaries, so a mark on a non-primary member alone would otherwise vanish silently — promote the whole group instead.

Parameters:
Return type:

None

stratified_packager.processing.dedup.source_group_key(layer, feedback)[source]

Build the §12 dedup group key: provider + normalized decodeUri components.

Parameters:
Return type:

tuple[str, frozenset[tuple[str, str]]] | None

Returns:

The hashable key, or None when the source cannot be decoded.