Contributing

nix develop            # or: pip install -e '.[dev]'
pytest -q
mypy

Layout

partition.py and resolve.py are pure -- no I/O, no storage imports. Keep them that way: it is what makes the interesting logic testable with hand-written dataclasses in milliseconds, and what keeps the semantics of the manifest format in one place.

storage.py is the only module that imports obstore. Three operations (list, get, conditional put) are all blobmap needs.

Tests

No mocks. MemoryStore and LocalStore are real implementations of the same API as S3Store, so the same suite runs against all three -- MinIO is one extra fixture param, not a second suite:

docker compose up -d
export BLOBMAP_TEST_S3_URL=s3://blobmap-test
export BLOBMAP_TEST_S3_ENDPOINT=http://localhost:9000
export AWS_ACCESS_KEY_ID=minioadmin AWS_SECRET_ACCESS_KEY=minioadmin
export AWS_REGION=us-east-1
pytest -q

tests/zarrgen.py writes synthetic stores as raw objects, with no zarr-python dependency. That is deliberate: blobmap parses this metadata itself, so the fixtures must not be generated by the library whose output they imitate. It also lets us produce layouts zarr-python will not easily write, such as a v3 array using v2-style flat chunk keys.

Diagrams and the call graph

docs/internals.md and docs/operations.md use mermaid, which renders on GitHub and in mkdocs without any build step. tests/test_docs.py checks every diagram is well formed, since a syntax error is otherwise invisible until someone opens the page.

Do not set colours. No fill:, no classDef, no style. Each renderer theming these differently is not a problem worth solving: GitHub light, GitHub dark and Material's palette toggle each override a different part, and mkdocs-material's CSS re-sets .mermaid text with higher specificity than a classDef color:, so a hardcoded fill ends up with the theme's own text colour over it. Letting the theme choose both is the only combination that is legible everywhere. A test enforces this.

Where a box needs emphasis, use structure instead: put it in a labelled subgraph, or mark the label with <b>.

The call graph is generated from the AST:

python tools/callgraph.py --entry partition_store --depth 3
python tools/callgraph.py --module resolve --private

The copy in internals.md is hand-laid-out for readability, so it is not byte-identical to the tool's output, but a test asserts the calls it shows still exist.

Docs

Docstrings are Google style and rendered by mkdocstrings:

pip install -e '.[docs]'
mkdocs serve

Every public function documents its arguments, its return value and anything it raises. Dataclasses use an Attributes: block covering every field.

Examples written as >>> are run by the test suite, since --doctest-modules is on by default. Prefer them over fenced code blocks wherever the example does not need a real backend, so the documentation cannot drift from the code. Use a fenced ```python block only when the example needs S3 or a database.

Storage handles

Anything holding a storage handle is annotated Store, an alias for obstore's ObjectStore defined in storage.py. Other modules import it from there rather than importing obstore, which keeps the dependency at one edge of the package. tests/test_typing.py enforces this, including a mypy probe that checks the annotation actually rejects a non-store.

Store is a closed union of obstore's own backends, so a wrapper around a store, for metrics or a read-only guard, will not satisfy it. If that becomes wanted, the move is to switch storage.py to calling store methods (store.get(key)) instead of module functions (obs.get(store, key)), which makes a structural Protocol possible.

Things that will bite

Changing how a blob id is derived is a breaking change: ids are the join key against blobtier's state table and, through it, against tape addresses. If a change can move an existing id, it belongs behind --force.

Manifest.SCHEMA_VERSION is checked strictly on read. Bump it and provide a migration rather than accepting both shapes.

The JSON Schema in schema.py is the contract, not a derived artifact. Change it and the dataclasses together: tests/test_schema.py cross-checks the hand-written walker against the real jsonschema library, so a keyword the walker does not implement fails the suite rather than being ignored. Every property needs a description; a test enforces that too.