KiloLock Docs

ADR 0030: Testing boundaries and coverage reporting

Context

Kilolock's Go test suite currently mixes several valid but different styles:

  • same-package unit tests for internal algorithms and orchestration helpers
  • integration tests behind //go:build integration
  • command tests in cmd/kl that live in package main

This works, but it leaves a few long-term questions open:

  • which tests are intentionally white-box and depend on unexported internals
  • which tests should exercise only public contracts
  • how to interpret coverage when important integration tests are excluded from a

    plain go test -coverprofile=... run

Recent work in internal/apply also showed that local green runs can still leave uncertainty if:

  • critical flows are covered mainly by integration-tagged tests
  • coverage numbers are read without understanding which test classes produced

    them

  • command-layer logic lives in package main, which is awkward to test as a

    true external black-box package

The goal is not to refactor the repo immediately. The goal is to define the future testing direction clearly so later cleanup is intentional rather than ad hoc.

Decision

Kilolock will adopt a three-lane testing model:

  1. black-box package tests for public behavior where the code lives in a

    reusable importable package

  2. same-package white-box tests for internal algorithms, orchestration, and

    low-level helpers where access to unexported internals is justified

  3. explicit integration test lanes for database-backed, Terraform-backed,

    and cross-binary flows

Coverage reporting will also be split by test lane rather than treated as a single universal score.

Decision details

D1. *_test.go already keeps tests out of production binaries

Kilolock will treat normal Go test files as sufficient isolation from release binaries.

This means:

  • tests do not need build tags merely to stay out of production builds
  • build tags should be used to separate test classes such as integration or

    expensive end-to-end flows

D2. Same-package tests are allowed when they provide real value

Kilolock will continue to allow tests in the same package when they are testing internal behavior that would be unnatural or noisy to expose publicly only for testing purposes.

Examples include:

  • merge logic
  • apply tmpdir setup
  • reservation heartbeat behavior
  • orchestration edge cases

The rule is not "same-package tests are bad". The rule is:

  • use same-package tests deliberately for internal mechanics
  • prefer black-box tests where public behavior is the real contract

D3. Public contract tests should prefer black-box shape when practical

When reusable logic lives in an importable package, Kilolock should prefer external-package tests for public behavior.

The intent is to make it easier to verify:

  • exported API shape
  • realistic call paths
  • absence of accidental coupling to unexported helpers

This is a future direction, not an immediate rewrite mandate.

D4. cmd/kl is a special case until more logic moves out of package main

cmd/kl currently contains substantial command behavior directly in package main.

Because of that:

  • same-package command tests are acceptable today
  • simply renaming tests to main_test is not enough to achieve good black-box

    testing

The preferred long-term direction is:

  1. move reusable command logic into importable packages
  2. keep cmd/kl as a thin wiring layer
  3. test the extracted package behavior from an external test package where that

    improves isolation

This ADR does not require that refactor in the current phase.

D5. Coverage must be reported per lane

Kilolock will stop treating one plain coverage percentage as the whole truth.

Instead, coverage should be interpreted in context:

  • unit/default coverage
    • fast signal
    • good for branch and helper blind spots
  • integration coverage
    • slower
    • required for DB-backed / Terraform-backed orchestration paths
  • package-focused coverage
    • useful for high-churn areas such as cmd/kl, internal/apply,

      internal/backend, internal/configscope

This matters because a function may show 0.0% in a default coverage report even though it is exercised by integration-tagged tests.

D6. Coverage is a guide, not a release gate by itself

Kilolock will use coverage reports to find blind spots, but not as a vanity metric or a sole quality gate.

Coverage is useful for:

  • detecting unexercised error paths
  • spotting newly added code with no direct test pressure
  • identifying packages that need test investment

Coverage is not sufficient to prove:

  • behavioral correctness
  • assertion quality
  • product-level confidence

Consequences

Positive

  • clearer intent around white-box vs black-box tests
  • less confusion about whether tests "ship in the binary"
  • more meaningful interpretation of coverage reports
  • a cleaner long-term path for improving cmd/kl test isolation

Tradeoffs

  • more than one coverage report may need to be produced and understood
  • future extraction of command logic out of package main will take work
  • some same-package tests will remain by design, which means full black-box

    purity is not the goal

Deferred work

This ADR deliberately does not require immediate refactoring.

Near-term work remains optional and can be scheduled only when it helps current delivery:

  • add documented coverage commands for default and integration lanes
  • extract selected cmd/kl helpers into importable packages when that creates

    clear testability wins

  • convert specific public-contract tests to external-package form when the code

    structure makes that natural

Suggested coverage commands

Default fast coverage for high-signal packages:

go test -count=1 ./cmd/kl ./internal/apply ./internal/backend ./internal/configscope -coverprofile=coverage.out
go tool cover -func=coverage.out
go tool cover -html=coverage.out

Integration-tagged coverage should be treated as a separate lane and may use a separate profile/output artifact.