KiloLock Docs

ADR 0031: Native state archive export and import

Context

Kilolock already supports two important escape hatches:

  • plain Terraform/OpenTofu can read and write the current state snapshot

    through the HTTP backend

  • kl import and /v1/admin/state/import can load a raw Terraform v4 state

    snapshot into Kilolock

That is enough for basic current-state migration:

  • local backend -> Kilolock
  • Kilolock -> local backend
  • one Kilolock-backed workflow can export a current snapshot and continue

    elsewhere

However, current-snapshot migration is not the same as native Kilolock preservation.

Kilolock stores additional backend-native value that does not survive a plain snapshot export/import:

  • append-only state_versions history
  • version provenance metadata such as:
    • serial
    • source
    • created_at
    • created_by
  • version tags
  • resource lifecycle spans and queryable historical projections
  • audit-friendly relationships between versions and repair/rollback actions

This matters especially for cloud and hosted usage:

  • customers should not need PostgreSQL access
  • support and operations should not require database dumps for normal migration
  • moving a state between two Kilolock backends should preserve more than only the

    current snapshot

  • offboarding and archival should be possible through backend APIs alone

Today, a database dump can preserve everything, but that is too blunt and too privileged for a state-scoped cloud workflow.

Decision

Kilolock will introduce a native archive export/import capability for one logical state.

This feature is distinct from plain snapshot import/export:

  • snapshot export/import
    • preserves only the current Terraform/OpenTofu state
    • compatible with vanilla tooling
  • native archive export/import
    • preserves the full Kilolock historical record for one state
    • intended for backend-to-backend migration, archival, restore, and

      support-grade portability

The export/import path must be available through Kilolock-owned APIs and CLI workflows so cloud users do not need direct database access.

Decision details

D1. Archive export is a first-class backend feature

Kilolock will treat historical export as a product feature, not as an accidental sequence of low-level API calls.

That means:

  • one explicit export command / endpoint
  • one explicit import command / endpoint
  • a documented archive format version
  • a compatibility contract for future format evolution

D2. The archive unit is one logical state

The initial export/import scope is one state, identified by:

  • workspace / tenant context
  • environment context
  • state name

This ADR does not require environment-wide or tenant-wide archival in the first version.

State-scoped export is the most useful primitive because it matches:

  • targeted migration
  • support handoff
  • point-in-time archival
  • customer offboarding from one active state at a time

D3. Archive contents must preserve full version history

The archive must preserve enough information to reconstruct the backend-native history of a state, not only its current snapshot.

Minimum required contents:

  • state identity metadata
    • logical state name
    • lineage
    • current version id / serial markers
  • every state_versions row for the state
    • id
    • serial
    • raw_state
    • terraform_version
    • source
    • created_at
    • created_by
  • every version tag attached to the state
  • enough import metadata to verify archive integrity and provenance

Optional or derived contents:

  • normalized resource rows
  • resource history projections
  • outputs projections
  • state-engine-specific caches

Those optional/derived contents may be regenerated on import if Kilolock can do so deterministically from the authoritative historical raw states.

D4. Raw historical state remains authoritative

The authoritative historical artifact inside the archive is the raw state stored for each version.

This follows the existing Kilolock model:

  • normalized rows are query accelerators and operational projections
  • raw historical states remain the durable replay and compatibility source

Import should therefore be allowed to:

  • restore historical state_versions
  • regenerate normalized rows from raw state history
  • rebuild caches lazily or eagerly

This keeps the archive durable even if projection schemas evolve.

D5. Import must preserve history semantics, not just bytes

Archive import is not equivalent to calling WriteState repeatedly.

The import path must preserve history semantics such as:

  • serial ordering
  • current-version pointer
  • provenance metadata
  • tags pointing to preserved version IDs

The import path may choose either of these implementation strategies:

  • ID-preserving import
    • restore original historical row IDs where safe and supported
  • ID-remapping import
    • assign new backend-local row IDs while preserving:
      • serial order
      • version metadata
      • tag relationships
      • deterministic mapping in the archive manifest

The external contract is history fidelity, not necessarily byte-identical database primary keys.

D6. Import must be explicit about merge policy

Importing a full historical archive into a destination backend can have conflicts.

The initial import contract should default to a fail-closed policy:

  • refuse import if the destination state name already exists

Future explicit modes may include:

  • import under a new state name
  • restore as archived copy
  • overwrite only with an operator-confirmed destructive flag

But silent merge is out of scope and should be avoided.

D7. Archive export must be cloud-safe

Cloud users should be able to export a state archive without PostgreSQL access.

That implies:

  • export must run through authenticated backend/control APIs
  • access control must be state-scoped or environment-scoped
  • the operation must be auditable

The feature should be usable by:

  • operators
  • delegated workspace owners where product policy allows it
  • support tooling under explicit authorization

D8. Archives must support offline storage and later restore

The archive format must be suitable for:

  • downloading to the operator's machine
  • storing in object storage or support archive systems
  • re-importing later into another Kilolock deployment

This makes native archive export the proper answer to:

  • "How do I move my state with history to another KL backend?"
  • "How do I keep a historical copy without dumping the database?"

D9. Snapshot export remains separate and simpler

This ADR does not replace existing snapshot-oriented escape hatches.

Kilolock will continue to support:

  • plain Terraform/OpenTofu access to the current snapshot
  • terraform state pull
  • raw snapshot import into a state

Those paths remain the right answer for:

  • compatibility
  • simple offboarding
  • migration back to a vanilla backend

The new archive path is for preserving Kilolock-native history, not replacing plain snapshot flows.

Archive format direction

The exact wire format is deferred, but the initial direction should be a single backend-owned archive object with:

  • manifest metadata
  • versioned format identifier
  • state identity block
  • version metadata collection
  • raw state payload collection
  • tag collection
  • optional checksum / digest metadata

Likely implementation options:

  • one JSON document
  • tarball/zip containing:
    • manifest.json
    • versions/<serial>.tfstate.json
    • tags.json

The top-level format must be versioned explicitly, for example:

  • archive_format = "kilolock-state-archive/v1"

API direction

This ADR does not freeze endpoint names, but the product shape should resemble:

  • export:
    • POST /v1/admin/state/export
    • or GET /v1/admin/states/{name}/archive
  • import:
    • POST /v1/admin/state/import-archive

CLI direction should resemble:

kl export state my-state --out my-state.klstate
kl import archive my-state.klstate

The archive workflow should be clearly differentiated from current snapshot import:

kl import snapshot terraform.tfstate
kl import archive my-state.klstate

Consequences

Positive

  • cloud-safe migration path without PostgreSQL access
  • better customer offboarding and portability story
  • support can archive or restore one state precisely
  • backend-to-backend migration preserves Kilolock-native value

Tradeoffs

  • new archive format must be versioned and maintained
  • import logic is more complex than plain WriteState
  • historical restore must handle schema/projection evolution carefully
  • large long-lived states may produce large export bundles

Deferred work

This ADR intentionally stops at architecture and product contract.

Follow-up work can decide:

  • exact archive serialization format
  • checksum and signing strategy
  • compression defaults
  • whether version IDs are preserved or remapped
  • whether resource history projections are exported or regenerated only
  • who is allowed to export/import in cloud UX terms

Non-goals

This ADR does not require:

  • whole-database backup replacement
  • tenant-wide bulk export in v1
  • automatic cross-backend replication
  • preserving every internal cache byte-for-byte

The goal is portable state history fidelity, not a perfect database clone.