stratified_packager.processing.building

Stratum GeoPackage assembly on the algorithm thread (SPEC §8.3).

The former GDAL worker pipeline is gone: every GeoPackage is now written here, on the algorithm thread, through the QGIS API. QgsVectorFileWriter reads a layer through its provider, so memory, joined, virtual and edited sources need no staging (the QGIS-fid ↔ OGR-FID equivalence problem disappears with it). Per-stratum filtering selects features on a throwaway read layer — a clone of the user’s layer, or a layer over a staging/full-package GeoPackage, never the user’s layer itself — and writes only the selection (onlySelectedFeatures); an empty selection writes an empty table. Layers identical in every stratum (whole_export) ride in a template GeoPackage copied per stratum (SPEC §8.1.5), so their data is written once and copied N times rather than written N times.

Only this module touches the per-stratum gpkg; background zip/move stays off-thread in workers. All user-facing messaging flows through the QgsProcessingFeedback argument; a per-stratum failure is contained (best-effort, SPEC §17) and surfaced as a result, never raised — the caller aggregates failures into one final exception. Cancellation is observed through the feedback.

Functions

discard_gpkg(path)

Best-effort remove a partial stratum gpkg during failure cleanup (SPEC §17).

stage_union(staging_gpkg, read_layer, table, ...)

Stage the union of every stratum's matches into a per-layer GeoPackage (SPEC §8.2).

warm_rejection(warm, expected_tables)

Check the §11 cold-fallback triggers against a warm-cache file.

write_stratum(build, *[, label])

Assemble one stratum gpkg on the algorithm thread (SPEC §8.3); never raises.

write_template(template_path, layers, /, *, ...)

Write the whole_export template gpkg once (SPEC §8.1.5).

write_vector_table(gpkg_path, layer, table, *)

Write a layer (whole or its current selection) into a GeoPackage as one table.

Classes

LayerWrite(layer_id, table, read_layer, ...)

One layer's contribution to a stratum gpkg (algorithm-thread plan; SPEC §8.3).

LayerWriteResult(layer_id, table, ...[, ...])

One layer's outcome inside a stratum gpkg (run-report material, SPEC §9.1).

StratumBuild(name, gpkg_path, layers[, ...])

Everything write_stratum() needs to assemble one stratum gpkg.

StratumWriteResult(name, ok[, error, ...])

A stratum build outcome (success or contained failure).

StyleDoc(name, qml[, sld, default, description])

One named style row destined for layer_styles (SPEC §8.3 step 5).
class stratified_packager.processing.building.LayerWrite(layer_id, table, read_layer, members, kept_field_indexes, whole_export=False, warm_marked=False, keep_if_empty=True, styles=(), metadata_xml='')[source]

One layer’s contribution to a stratum gpkg (algorithm-thread plan; SPEC §8.3).

Parameters:
__init__(layer_id, table, read_layer, members, kept_field_indexes, whole_export=False, warm_marked=False, keep_if_empty=True, styles=(), metadata_xml='')
Parameters:
keep_if_empty: bool = True

KEEP_EMPTY_LAYERS: keep the empty table (with styles) or drop it.

kept_field_indexes: tuple[int, ...]

Indexes into read_layer .fields() to export (excluded fields dropped). An empty tuple exports every field (the writer’s documented all-fields behavior).

layer_id: str

The QGIS layer id (round-trips into results and the embedded project).

members: tuple[LayerMatchPlan, ...]

The match plan(s) whose union defines membership. One element normally; more only for a dedup group sharing this table (SPEC §12) — the union of every member’s match set is selected. A whole_export member makes the whole table unconditional.

metadata_xml: str = ''

QMD layer metadata written after the table lands (empty = none).

read_layer: qgis.core.QgsVectorLayer

The standalone layer features are read and selected from — a clone of the user’s layer, or a layer over the staging/full-package GeoPackage. Its selection is mutated per stratum; it is never a layer owned by QgsProject.

styles: tuple[StyleDoc, ...] = ()

Styles written after the table lands (skipped when it rides in a seed).

table: str

Target table name inside the stratum gpkg.

warm_marked: bool = False

Whether this table belongs to the warm cache (rides in the warm seed, SPEC §11).

whole_export: bool = False

Whether this table is unconditional (rides in the template seed on cold runs).

class stratified_packager.processing.building.LayerWriteResult(layer_id, table, feature_count, status, matched_fids=frozenset({}))[source]

One layer’s outcome inside a stratum gpkg (run-report material, SPEC §9.1).

Parameters:
__init__(layer_id, table, feature_count, status, matched_fids=frozenset({}))
Parameters:
feature_count: int

Rows in the table (0 for an empty/skipped layer).

layer_id: str

The QGIS layer id.

matched_fids: frozenset[int] = frozenset({})

This stratum’s matching feature ids in the read layer’s id space (empty for whole_export/seeded layers). The caller unions these across strata for the §9.1 <unmatched> accounting (read-source total minus the union).

status: str

ok | empty-kept | empty-skipped | warm.

table: str

The target table name.

class stratified_packager.processing.building.StratumBuild(name, gpkg_path, layers, stratum_feature=None, template=None, warm_start=None, expected_warm_tables=(), snapshot_to=None, data_payloads=())[source]

Everything write_stratum() needs to assemble one stratum gpkg.

Parameters:

  • name (str)

  • gpkg_path (Path)

  • layers (tuple[LayerWrite, …])

  • stratum_feature (QgsFeature | None)

  • template (Path | None)

  • warm_start (Path | None)

  • expected_warm_tables (tuple[str, …])

  • snapshot_to (Path | None)

  • data_payloads (tuple[tuple[Path, Path], …])

__init__(name, gpkg_path, layers, stratum_feature=None, template=None, warm_start=None, expected_warm_tables=(), snapshot_to=None, data_payloads=())
Parameters:

  • name (str)

  • gpkg_path (Path)

  • layers (tuple[LayerWrite, …])

  • stratum_feature (QgsFeature | None)

  • template (Path | None)

  • warm_start (Path | None)

  • expected_warm_tables (tuple[str, …])

  • snapshot_to (Path | None)

  • data_payloads (tuple[tuple[Path, Path], …])

data_payloads: tuple[tuple[Path, Path], ...] = ()

(source file, destination) sidecar copies into the build dir (SPEC §14).

expected_warm_tables: tuple[str, ...] = ()

Tables the warm file must hold exactly, or the build falls back cold (SPEC §11).

gpkg_path: Path

Absolute build path of the stratum gpkg.

layers: tuple[LayerWrite, ...]

Layer contributions, already ordered per SPEC §8.3 step 3 (warm-marked first, then whole_export, then the remaining partitioned layers).

name: str

The sanitized stratum name (result/reporting key).

snapshot_to: Path | None = None

Warm-cache destination of the WARM_START_MODE=update warm pass, whose builds hold only warm-marked layers (the snapshot is taken after the last warm-marked layer is written); None = no snapshot.

stratum_feature: QgsFeature | None = None

The stratum feature (membership source); None for the <full> package.

template: Path | None = None

The whole_export template gpkg copied as the cold-run seed (SPEC §8.1.5).

warm_start: Path | None = None

Warm cache file copied as the seed — on WARM_START_MODE=use runs, and on the deliverable pass of WARM_START_MODE=update runs, which seeds from the cache the warm pass just wrote (SPEC §11).

class stratified_packager.processing.building.StratumWriteResult(name, ok, error='', warm_used=False, warm_reason='', layers=<factory>)[source]

A stratum build outcome (success or contained failure).

Parameters:
__init__(name, ok, error='', warm_used=False, warm_reason='', layers=<factory>)
Parameters:
error: str = ''

Failure detail when ok is false.

layers: list[LayerWriteResult]

Per-layer outcomes (empty when the build failed before any layer landed).

name: str

The sanitized stratum name.

ok: bool

Whether the gpkg was assembled completely.

warm_reason: str = ''

The cold-fallback trigger when a warm source existed but was rejected.

warm_used: bool = False

Whether the build started from the warm cache.

class stratified_packager.processing.building.StyleDoc(name, qml, sld='', default=False, description='')[source]

One named style row destined for layer_styles (SPEC §8.3 step 5).

Parameters:
__init__(name, qml, sld='', default=False, description='')
Parameters:
default: bool = False

Whether this is the table’s default style.

description: str = ''

Optional description.

name: str

Style name.

qml: str

Full QML document.

sld: str = ''

SLD document (best-effort; may be empty).

stratified_packager.processing.building._apply_membership(layer_write, stratum_feature, project, strat_layer, stratum_name, feedback)[source]

Select the union of the members’ matching features on the read layer (SPEC §7/§12).

A whole_export member makes the table unconditional (the whole layer is written). Otherwise each member contributes its match set: attribute keys become a selection by key expression, spatial fids a selection by id; selections accumulate (AddToSelection).

Parameters:

  • layer_write (LayerWrite) – The layer contribution (its read layer’s selection is replaced).

  • stratum_feature (QgsFeature | None) – The stratum feature (membership source).

  • project (QgsProject) – The run’s project.

  • strat_layer (QgsVectorLayer | None) – The stratification layer (spatial transforms).

  • stratum_name (str) – The stratum name (feedback only).

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Return type:

tuple[bool, frozenset[int]]

Returns:

(only the selection is written, the selected fids) — the second element feeds the §9.1 orphan accounting and is empty for an unconditional write.

Raises:

QgsProcessingException – On a coordinate-transform failure.

stratified_packager.processing.building._build_layer(build, layer_write, *, warm_used, template_used, project, strat_layer, feedback)[source]

Append (or recognize) one layer’s slice and apply its epilogue (SPEC §8.3).

Layers already present in the seed are reported without rewriting: a warm-marked table on a warm start counts as warm (an empty one follows keep_if_empty); a non-warm-marked whole_export table on a template start keeps its template count and styles (warm-marked ones never ride the template, so they are written like any other layer).

Parameters:

  • build (StratumBuild) – The stratum build.

  • layer_write (LayerWrite) – The layer contribution.

  • warm_used (bool) – Whether the build started from the warm seed.

  • template_used (bool) – Whether the build started from the template seed.

  • project (QgsProject) – The run’s project.

  • strat_layer (QgsVectorLayer | None) – The stratification layer.

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Return type:

LayerWriteResult

Returns:

The layer outcome.

Raises:

QgsProcessingException – On a writer or transform failure (contained upstream).

stratified_packager.processing.building._empty_status(*, kept, count)[source]

Map a table’s presence and row count to its §9.1 status token.

Parameters:

  • kept (bool) – Whether the table exists.

  • count (int) – Its row count.

Return type:

str

Returns:

ok | empty-kept | empty-skipped.

stratified_packager.processing.building._finalize_layer(gpkg_path, layer_write, *, drop_empty, write_styles, matched_fids=frozenset({}))[source]

Apply the per-layer epilogue: empty handling, styles, metadata, outcome.

Parameters:

  • gpkg_path (Path) – The stratum gpkg.

  • layer_write (LayerWrite) – The finished layer contribution.

  • drop_empty (bool) – Whether a zero-feature table is dropped (KEEP_EMPTY_LAYERS=False).

  • write_styles (bool) – Whether to write styles/metadata (skipped for seeded tables).

  • matched_fids (frozenset[int]) – This stratum’s matching fids (read-source id space, §9.1 accounting).

Return type:

LayerWriteResult

Returns:

The layer outcome.

stratified_packager.processing.building._report_seeded(gpkg_path, layer_write)[source]

Report a warm-marked table that already rides in the warm seed (SPEC §11).

The warm cache always retains empty warm-marked tables, so the deliverable’s empty policy is applied here: an empty seeded table is dropped when keep_if_empty is false (empty-skipped), otherwise it stays warm (row count 0 tells the rest).

Parameters:

  • gpkg_path (Path) – The stratum gpkg.

  • layer_write (LayerWrite) – The seeded layer contribution.

Return type:

LayerWriteResult

Returns:

The layer outcome.

stratified_packager.processing.building._seed(build, result, feedback)[source]

Materialize the seed file: warm copy, template copy, or fresh (SPEC §8.3/§11).

A warm_start equal to the build path means the §11 prefetch already copied the cache there — the seed is validated and used in place, no second copy. A rejected warm file (prefetched or not) falls through to the template/fresh path, whose copy overwrites any prefetched bytes.

Parameters:
Return type:

tuple[bool, bool]

Returns:

(warm seed used, template seed used).

stratified_packager.processing.building._select_member(read_layer, member, stratum_feature, *, project, strat_layer, stratum_name, feedback)[source]

Add one member’s matching features to read_layer’s selection (SPEC §7).

Attribute matching selects by key expression (the provider compiles IN to native SQL where it can); spatial matching selects the fids matched against this stratum’s geometry. Selections accumulate, so callers union members — and strata — by not resetting between calls.

Parameters:

  • read_layer (QgsVectorLayer) – The standalone read layer whose selection is extended.

  • member (LayerMatchPlan) – The member’s resolved match plan (never whole_export).

  • stratum_feature (QgsFeature) – The stratum feature (membership source).

  • project (QgsProject) – The run’s project (relation-chain intermediates).

  • strat_layer (QgsVectorLayer | None) – The stratification layer (spatial transforms).

  • stratum_name (str) – The stratum name (feedback only).

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Raises:

QgsProcessingException – On a missing stratification layer or transform failure.

Return type:

None

stratified_packager.processing.building._snapshot_warm(gpkg_path, destination)[source]

Snapshot the gpkg into the warm cache via .part + atomic rename (SPEC §11).

Parameters:

  • gpkg_path (Path) – The stratum gpkg (holding only the warm-marked tables so far).

  • destination (Path) – The warm-cache destination (<warm dir>/<stratum>.gpkg).

Raises:

QgsProcessingException – If the WAL checkpoint is incomplete — copying only the main file would snapshot a stale cache, so this counts as a failed cache write (the §11 machinery then builds the stratum cold and fails the run at the end).

Return type:

None

stratified_packager.processing.building._warm_count(build)[source]

Count the leading warm-marked layers (the snapshot boundary, SPEC §11).

Parameters:

build (StratumBuild) – The stratum build (layers are ordered warm-marked first).

Return type:

int

Returns:

The number of leading warm-marked layer writes.

stratified_packager.processing.building._warm_rejection(build)[source]

Check the §11 cold-fallback triggers against the build’s warm source.

Parameters:

build (StratumBuild) – The stratum build (warm_start is set).

Return type:

str | None

Returns:

The rejection reason, or None when the warm file is usable.

stratified_packager.processing.building.discard_gpkg(path)[source]

Best-effort remove a partial stratum gpkg during failure cleanup (SPEC §17).

Never raises: on Windows a just-failed writeAsVectorFormatV3() (including a writer aborted by cancellation) can leave the file locked, so unlink() would raise PermissionError. Swallowing it keeps a contained per-stratum failure from escalating into a fatal abort; a still-locked file is left for the run’s workdir teardown / the OS.

Parameters:

path (Path) – The partial gpkg to remove.

Return type:

bool

Returns:

Whether the file was removed.

stratified_packager.processing.building.stage_union(staging_gpkg, read_layer, table, members, stratum_features, *, kept_field_indexes=(), project, strat_layer, feedback)[source]

Stage the union of every stratum’s matches into a per-layer GeoPackage (SPEC §8.2).

The staged copy holds only the features some stratum uses, so each stratum later reads its slice from this fast local file instead of re-fetching from a slow source. Matching is computed against the source here; per-stratum reads then re-match against the staged copy (its own fids/keys, so spatial matching survives the FID renumbering).

Parameters:

  • staging_gpkg (Path) – The per-layer staging GeoPackage.

  • read_layer (QgsVectorLayer) – A standalone clone of the source layer (its selection is replaced).

  • table (str) – The staged table name.

  • members (Sequence[LayerMatchPlan]) – The match plan(s) whose union is staged (no whole_export member).

  • stratum_features (Sequence[QgsFeature]) – The real strata features (the <full> pseudo-stratum excluded).

  • kept_field_indexes (Sequence[int]) – Field indexes to stage (empty = every field).

  • project (QgsProject) – The run’s project.

  • strat_layer (QgsVectorLayer | None) – The stratification layer.

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Raises:

QgsProcessingException – On a writer or transform failure.

Return type:

None

stratified_packager.processing.building.warm_rejection(warm, expected_tables)[source]

Check the §11 cold-fallback triggers against a warm-cache file.

Shared by the seed-time check and the Phase-A pre-scan that decides whether staging of warm-covered groups can be skipped on a WARM_START_MODE=use run (§8.2/§11).

Parameters:

  • warm (Path | None) – The warm-cache gpkg path (None counts as missing).

  • expected_tables (Collection[str]) – The warm-marked primaries’ table names.

Return type:

str | None

Returns:

The rejection reason, or None when the warm file is usable.

stratified_packager.processing.building.write_stratum(build, *, label='', project, strat_layer, feedback)[source]

Assemble one stratum gpkg on the algorithm thread (SPEC §8.3); never raises.

Pipeline: seed file (warm copy, template copy, or fresh) → ordered per-layer filtered appends (layers already in the seed are reported, not rewritten) → per-layer empty handling, styles and metadata → data/ payload copies. A failure is contained and returned as ok=False (best-effort, SPEC §17); the partial gpkg is removed.

Parameters:

  • build (StratumBuild) – The stratum build.

  • label (str) – Progress-text prefix naming this build, e.g. Stratum 4/93: urban (falls back to the stratum name when empty).

  • project (QgsProject) – The run’s project (resolves relation-chain intermediates).

  • strat_layer (QgsVectorLayer | None) – The stratification layer (spatial transforms); None only for the <full> package, whose layers are all whole_export.

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Return type:

StratumWriteResult

Returns:

The stratum outcome, with per-layer results.

stratified_packager.processing.building.write_template(template_path, layers, /, *, feedback)[source]

Write the whole_export template gpkg once (SPEC §8.1.5).

Each layer is written in full (no per-stratum selection); a plain file copy of the template then seeds every stratum that shares these layers. Styles and metadata are not written here — they are written per stratum after the seed copy, so the resources/ prefix matches each stratum’s gpkg depth (SPEC §13/§14).

Parameters:

  • template_path (Path) – The template gpkg build path.

  • layers (Sequence[LayerWrite]) – The whole_export layer writes (already deduplicated to primaries).

  • feedback (QgsProcessingFeedback) – Execution feedback channel.

Raises:

QgsProcessingException – If a writer reports an error.

Return type:

None

stratified_packager.processing.building.write_vector_table(gpkg_path, layer, table, *, kept_field_indexes=(), only_selected, feedback)[source]

Write a layer (whole or its current selection) into a GeoPackage as one table.

Shared by stratum assembly, the template and per-layer staging: the table is created on the first write into gpkg_path and appended as a new layer afterwards. The writer reads through the QGIS provider, so joins, virtual fields and unsaved edits are honored without any staging.

Parameters:

  • gpkg_path (Path) – The destination GeoPackage (created on first write).

  • layer (QgsVectorLayer) – The standalone layer to read; its current selection is honored when only_selected is set (never a layer owned by QgsProject).

  • table (str) – The target table name.

  • kept_field_indexes (Sequence[int]) – Field indexes (into layer.fields()) to export; empty exports every field.

  • only_selected (bool) – Whether to write only the current selection.

  • feedback (QgsProcessingFeedback) – Execution feedback channel (carries cancellation into the writer).

Raises:

QgsProcessingException – If the writer reports an error.

Return type:

None