model

blobmap.model

Manifest format and the array metadata that feeds it.

The manifest is pure definition: it changes only when the set of blobs changes. Anything measured, such as tier, last access or current size, lives in blobtier's state table keyed by blob id.

Stdlib only. Nothing in this module knows that storage exists, which is what makes the partitioner testable without a backend.

Attributes:

Name	Type	Description
SCHEMA_VERSION		Manifest format version. Checked strictly on read; a mismatch raises rather than attempting a best-effort parse, because silently misreading a blob definition would misroute chunks.
VERSION		Package version, recorded in each manifest as provenance.
GiB		Convenience constant, 1024**3.
KEY_ENCODINGS	dict[str, str]	Supported chunk key layouts, mapped to an example key. See Array.key_encoding.
METADATA_BASENAMES	tuple[str, ...]	Object basenames that are never archivable. This is what keeps xr.open_zarr from triggering a tape mount.

Array dataclass

Array(
    path: str,
    shape: tuple[int, ...],
    chunks: tuple[int, ...],
    itemsize: int,
    shards: tuple[int, ...] | None = None,
    stored_bytes: int | None = None,
    nobjects_seen: int | None = None,
    is_coordinate: bool = False,
    key_encoding: str = "v3_slash",
)

One zarr array: metadata for structure, LIST for size.

This is the partitioner's input. It is deliberately a plain dataclass with no storage behind it, so the partitioning rules can be tested by writing these out by hand.

Attributes:

Name	Type	Description
path	str	Array path relative to the manifest scope, such as tas or deep/nest/tas. Empty string for an array at the scope root.
shape	tuple[int, ...]	Full array shape, from the zarr metadata.
chunks	tuple[int, ...]	The inner chunk shape. This is not the storage unit when the array is sharded; see object_shape.
itemsize	int	Bytes per element, derived from the dtype.
shards	tuple[int, ...] | None	Shard shape when sharding is in use, else None. When set, this is what becomes one object.
stored_bytes	int | None	Observed compressed total, summed from a LIST. None falls back to the uncompressed estimate, which is correct but pessimistic and makes the width clamp bind hard.
nobjects_seen	int | None	Objects actually present. Differs from nobjects for a sparsely or partially written array.
is_coordinate	bool	True for a dimension coordinate. Coordinates are pinned hot regardless of size, or opening the store hits tape.
key_encoding	str	How to locate the chunk index in an object key. One of v3_slash, v2_slash, v2_flat. Per array rather than per store: in v2 dimension_separator lives in each .zarray, and a v3 array may opt in to v2 style keys, so one store can mix all three.

Example

tas = Array("tas", (400, 4, 4), (10, 4, 4), 4, stored_bytes=2560) tas.nobjects 40 tas.chunk_prefix 'tas/c'

object_shape property

object_shape: tuple[int, ...]

Shape of one stored object: the shard if sharded, else the chunk.

zarr-python inverts the intuitive naming. Array.chunks is the inner chunk of a sharded array and Array.shards is what becomes an object. Using the inner chunk here understates object size by the shard factor, which silently produces blobs many times too large. Raw v3 metadata does not invert: chunk_grid.chunk_shape is already the object, and the inner shape sits in the sharding codec config.

Returns:

Type	Description
tuple[int, ...]	The shape of a single stored object.

Example

Array("tas", (400, 4), (10, 4), 4).object_shape (10, 4) Array("tas", (400, 4), (10, 4), 4, shards=(50, 4)).object_shape (50, 4)

object_grid property

object_grid: tuple[int, ...]

Number of objects along each dimension.

Returns:

Type	Description
int	Object counts per dimension, using
...	object_shape as the unit.

Example

Array("tas", (400, 4, 4), (10, 4, 4), 4).object_grid (40, 1, 1)

nobjects property

nobjects: int

Objects this array would have if fully written.

Deliberately not nobjects_seen: a sparsely written array should be partitioned for the shape it will have, not the shape it has today.

Returns:

Type	Description
int	Total object count across the full grid.

Example

Array("tas", (400, 4, 4), (10, 4, 4), 4, nobjects_seen=3).nobjects 40

uncompressed_object_bytes property

uncompressed_object_bytes: int

Size of one object with no compression at all.

Fixed for the lifetime of the array, since changing it means rewriting every chunk. That is what makes it usable as a hard ceiling in bucket_width: whatever the codec does later, an object cannot exceed this.

Returns:

Type	Description
int	Uncompressed bytes per stored object.

Example

Array("tas", (400, 4, 4), (10, 4, 4), 4).uncompressed_object_bytes 640

total_bytes property

total_bytes: int

Total size of the array, observed if known and estimated otherwise.

Returns:

Type	Description
int	stored_bytes when set, else the uncompressed estimate.

Example

Array("tas", (400, 4, 4), (10, 4, 4), 4).total_bytes 25600 Array("tas", (400, 4, 4), (10, 4, 4), 4, stored_bytes=99).total_bytes 99

avg_object_bytes property

avg_object_bytes: int

Mean size of one stored object.

Divides by the objects actually seen rather than the full grid, so a partially written array is not reported as compressing better than it does.

Returns:

Type	Description
int	Average bytes per object, at least 1.

Example

Array("tas", (400, 4, 4), (10, 4, 4), 4, ... stored_bytes=2560).avg_object_bytes 64

chunk_prefix property

chunk_prefix: str

Where this array's chunk keys begin, relative to the scope.

Returns:

Type	Description
str	The prefix a blob should claim to cover this array's chunks.

Raises:

Type	Description
ValueError	If key_encoding is not one of KEY_ENCODINGS.

Example

Array("tas", (1,), (1,), 4, key_encoding="v3_slash").chunk_prefix 'tas/c' Array("tas", (1,), (1,), 4, key_encoding="v2_flat").chunk_prefix 'tas'

Policy dataclass

Policy(
    t_max_bytes: int = 100 * GiB,
    t_min_bytes: int = 0,
    t_hot_bytes: int = 16 * 1024**2,
    width_clamp: int = 8,
    pow2_floor: int = 64,
)

Thresholds that shape the cut.

Defaults are a starting point, not a recommendation. The right values depend on tape mount and positioning times, so check them against what the HSM actually does before trusting them.

Attributes:

Name	Type	Description
t_max_bytes	int	Target upper bound for one blob. A subtree at or under this becomes a single blob; above it the partitioner descends and eventually buckets. Too large and a restore takes hours.
t_min_bytes	int	Floor below which arrays are coalesced with their neighbours rather than each becoming a blob. Defaults to 0, which disables coalescing: it groups by path adjacency, which is a guess about access correlation, and the cost of guessing wrong is restoring data nobody asked for. Aggregating small objects is the tape layer's job. Raise it only if the HSM cannot bundle recalls.
t_hot_bytes	int	Arrays smaller than this are never archived, regardless of the blobs. This is a "not worth a row" threshold, not a "small variable" one: keeping every sub-gigabyte array hot leaves an unbounded amount of data permanently on disk. Coordinates are pinned by detection, not by size, so this can be small.
width_clamp	int	Bound on the worst case. A blob may not exceed width_clamp * t_max_bytes even if compression degrades to nothing, computed from uncompressed object size.
pow2_floor	int	Widths at or above this are rounded down to a power of two, so a small drift in the observed compression ratio does not renumber every blob. Below it, rounding would throw away too much of the target, which is the common case for sharded arrays.

Example

Policy().t_max_bytes == 100 * GiB True Policy().t_min_bytes # no floor: the tape layer bundles 0

to_json

to_json() -> dict[str, int]

Serialise for embedding in a manifest.

Returns:

Type	Description
dict[str, int]	A plain dict of the five thresholds.

Source code in src/blobmap/model.py

def to_json(self) -> dict[str, int]:
    """Serialise for embedding in a manifest.

    Returns:
        A plain dict of the five thresholds.
    """
    return {
        "t_max_bytes": self.t_max_bytes,
        "t_min_bytes": self.t_min_bytes,
        "t_hot_bytes": self.t_hot_bytes,
        "width_clamp": self.width_clamp,
        "pow2_floor": self.pow2_floor,
    }

from_json classmethod

from_json(d: dict[str, Any]) -> Policy

Rebuild from a manifest, ignoring fields this version does not know.

Parameters:

Name	Type	Description	Default
d	dict[str, Any]	The policy block of a manifest.	required

Returns:

Type	Description
Policy	A Policy, with defaults for anything absent.

Example

Policy.from_json({"t_max_bytes": 5, "added_in_future": 9}) Policy(t_max_bytes=5, t_min_bytes=0, t_hot_bytes=16777216, width_clamp=8, pow2_floor=64)

Source code in src/blobmap/model.py

@classmethod
def from_json(cls, d: dict[str, Any]) -> Policy:
    """Rebuild from a manifest, ignoring fields this version does not know.

    Args:
        d: The `policy` block of a manifest.

    Returns:
        A `Policy`, with defaults for anything absent.

    Example:
        >>> Policy.from_json({"t_max_bytes": 5, "added_in_future": 9})
        Policy(t_max_bytes=5, t_min_bytes=0, t_hot_bytes=16777216, width_clamp=8, pow2_floor=64)
    """
    fields = set(cls.__dataclass_fields__)
    return cls(**{k: int(v) for k, v in d.items() if k in fields})

Pin dataclass

Pin(
    prefix: str,
    reason: str,
    by: str = "",
    at: str = "",
    until: str | None = None,
)

A deliberate instruction to keep a prefix hot.

Distinct from hot_always, which is derived: metadata objects and coordinates are recomputed on every partition because they follow from the store's structure. A pin follows from someone's intent, so it is preserved across repartitions and can only be removed by removing it.

That also means a pin is the one part of a manifest that cannot be reconstructed by re-scanning. Blob definitions can be recomputed; intent cannot. Back the manifest bucket up.

Attributes:

Name	Type	Description
prefix	str	What to keep hot, relative to the manifest scope. Empty string pins the whole scope.
reason	str	Why. Required, because a pin with no stated reason is indistinguishable from one nobody remembers setting.
by	str	Who set it.
at	str	ISO 8601 UTC timestamp when it was set.
until	str | None	ISO 8601 UTC timestamp after which it should be reviewed, or None for an open-ended pin. Expiry is reported, not enforced: silently unpinning a dataset someone is working on would be a worse surprise than a stale pin.

Example

pin = Pin("multiscales/zoom_9", "active ICON analysis", "wilfred", ... "2026-08-19T09:12:00+00:00", "2026-12-01T00:00:00+00:00") pin.covers("multiscales/zoom_9/tas/c/0") True pin.covers("multiscales/zoom_8/tas/c/0") False

to_json

to_json() -> dict[str, Any]

Serialise for the manifest.

Returns:

Type	Description
dict[str, Any]	A dict with prefix, reason, by, at and until.

Source code in src/blobmap/model.py

def to_json(self) -> dict[str, Any]:
    """Serialise for the manifest.

    Returns:
        A dict with `prefix`, `reason`, `by`, `at` and `until`.
    """
    return {
        "prefix": self.prefix,
        "reason": self.reason,
        "by": self.by,
        "at": self.at,
        "until": self.until,
    }

from_json classmethod

from_json(d: dict[str, Any]) -> Pin

Rebuild from a manifest entry.

Parameters:

Name	Type	Description	Default
d	dict[str, Any]	One element of the manifest's pinned list.	required

Returns:

Type	Description
Pin	A Pin.

Source code in src/blobmap/model.py

@classmethod
def from_json(cls, d: dict[str, Any]) -> Pin:
    """Rebuild from a manifest entry.

    Args:
        d: One element of the manifest's `pinned` list.

    Returns:
        A `Pin`.
    """
    return cls(
        str(d["prefix"]),
        str(d["reason"]),
        str(d.get("by", "")),
        str(d.get("at", "")),
        None if d.get("until") is None else str(d["until"]),
    )

covers

covers(key: str) -> bool

Whether this pin applies to a key, relative to the scope.

Parameters:

Name	Type	Description	Default
key	str	Scope-relative object key.	required

Returns:

Type	Description
bool	True if the key is at or below the pinned prefix.

Source code in src/blobmap/model.py

def covers(self, key: str) -> bool:
    """Whether this pin applies to a key, relative to the scope.

    Args:
        key: Scope-relative object key.

    Returns:
        True if the key is at or below the pinned prefix.
    """
    if not self.prefix:
        return True
    return key == self.prefix or key.startswith(self.prefix + "/")

expired

expired(now_iso: str | None = None) -> bool

Whether the review date has passed.

Parameters:

Name	Type	Description	Default
now_iso	str | None	Time to compare against, defaulting to now. ISO 8601.	None

Returns:

Type	Description
bool	False for an open-ended pin, which never expires but also never
bool	gets reviewed -- which is its own problem, so pin show calls
bool	those out separately.

Example

Pin("x", "why", until="2020-01-01T00:00:00+00:00").expired() True Pin("x", "why").expired() False

Source code in src/blobmap/model.py

def expired(self, now_iso: str | None = None) -> bool:
    """Whether the review date has passed.

    Args:
        now_iso: Time to compare against, defaulting to now. ISO 8601.

    Returns:
        False for an open-ended pin, which never expires but also never
        gets reviewed -- which is its own problem, so `pin show` calls
        those out separately.

    Example:
        >>> Pin("x", "why", until="2020-01-01T00:00:00+00:00").expired()
        True
        >>> Pin("x", "why").expired()
        False
    """
    if self.until is None:
        return False
    return (now_iso or now()) > self.until

Bucket dataclass

Bucket(index: int, width: int, key_encoding: str)

The arithmetic that turns a chunk index into a blob number.

This is what makes a manifest a rule rather than a table: chunk 5,000,000 resolves without anyone having enumerated it, so appending to a store needs no manifest change at all.

Attributes:

Name	Type	Description
index	int	Which segment after the blob prefix holds the chunk index. Almost always 0, since time is conventionally the first dimension and is what people slice on.
width	int	How many objects along that dimension go into one blob. Chosen by bucket_width.
key_encoding	str	How to parse the index out of the key. One of v3_slash, v2_slash, v2_flat.

Example

Bucket(0, 2048, "v3_slash").to_json()["width"] 2048

to_json

to_json() -> dict[str, Any]

Serialise for the manifest.

Returns:

Type	Description
dict[str, Any]	A dict with index, width and key_encoding.

Source code in src/blobmap/model.py

def to_json(self) -> dict[str, Any]:
    """Serialise for the manifest.

    Returns:
        A dict with `index`, `width` and `key_encoding`.
    """
    return {
        "index": self.index,
        "width": self.width,
        "key_encoding": self.key_encoding,
    }

from_json classmethod

from_json(d: dict[str, Any]) -> Bucket

Rebuild from a manifest.

Parameters:

Name	Type	Description	Default
d	dict[str, Any]	The bucket block of a blob entry.	required

Returns:

Type	Description
Bucket	A Bucket.

Source code in src/blobmap/model.py

@classmethod
def from_json(cls, d: dict[str, Any]) -> Bucket:
    """Rebuild from a manifest.

    Args:
        d: The `bucket` block of a blob entry.

    Returns:
        A `Bucket`.
    """
    return cls(int(d["index"]), int(d["width"]), str(d["key_encoding"]))

Blob dataclass

Blob(
    id: str,
    prefixes: tuple[str, ...],
    bucket: Bucket | None = None,
)

A set of objects that move to and from tape together.

Every blob has exactly these three fields. bucket is None means one bucket, so the resolved id is always f"{id}_{n}". Unbucketed is the degenerate case of bucketed rather than a different kind of thing, which gives the resolver a single code path and means filling in bucket later does not change existing blob ids.

Attributes:

Name	Type	Description
id	str	Stable identifier, [a-z0-9_]. This is the join key against blobtier's state table and, through it, against tape addresses. Changing how ids are derived is a breaking change.
prefixes	tuple[str, ...]	Key prefixes this blob claims, relative to the manifest scope. More than one when small arrays were coalesced. A bucketed blob must have exactly one.
bucket	Bucket | None	Arithmetic for splitting inside an array, or None for a blob that is a plain prefix.

Example

Blob("b_pr", ("pr/c",)).instance(0) 'b_pr_0' Blob("b_tas", ("tas/c",), Bucket(0, 2048, "v3_slash")).instance(2) 'b_tas_2'

to_json

to_json() -> dict[str, Any]

Serialise for the manifest.

Returns:

Type	Description
dict[str, Any]	A dict with exactly id, prefixes and bucket.

Example

sorted(Blob("b_pr", ("pr/c",)).to_json()) ['bucket', 'id', 'prefixes']

Source code in src/blobmap/model.py

def to_json(self) -> dict[str, Any]:
    """Serialise for the manifest.

    Returns:
        A dict with exactly `id`, `prefixes` and `bucket`.

    Example:
        >>> sorted(Blob("b_pr", ("pr/c",)).to_json())
        ['bucket', 'id', 'prefixes']
    """
    return {
        "id": self.id,
        "prefixes": list(self.prefixes),
        "bucket": self.bucket.to_json() if self.bucket else None,
    }

from_json classmethod

from_json(d: dict[str, Any]) -> Blob

Rebuild from a manifest entry.

Parameters:

Name	Type	Description	Default
d	dict[str, Any]	One element of the manifest's blobs list.	required

Returns:

Type	Description
Blob	A Blob.

Source code in src/blobmap/model.py

@classmethod
def from_json(cls, d: dict[str, Any]) -> Blob:
    """Rebuild from a manifest entry.

    Args:
        d: One element of the manifest's `blobs` list.

    Returns:
        A `Blob`.
    """
    b = d.get("bucket")
    return cls(
        str(d["id"]), tuple(d["prefixes"]), Bucket.from_json(b) if b else None
    )

instance

instance(n: int) -> str

Get the concrete blob id for bucket number n.

Parameters:

Name	Type	Description	Default
n	int	Bucket number, always 0 when bucket is None.	required

Returns:

Type	Description
str	The id blobtier stores tier state against.

Source code in src/blobmap/model.py

def instance(self, n: int) -> str:
    """Get the concrete blob id for bucket number `n`.

    Args:
        n: Bucket number, always 0 when `bucket` is `None`.

    Returns:
        The id blobtier stores tier state against.
    """
    return f"{self.id}_{n}"

Manifest dataclass

Manifest(
    scope: str,
    blobs: tuple[Blob, ...],
    hot_always: tuple[str, ...],
    pinned: tuple[Pin, ...] = (),
    policy: Policy = Policy(),
    epoch: int = 1,
    generated_at: str = "",
    generated_by: str = f"blobmap {VERSION}",
    provenance: dict[str, Any] = dict(),
)

The blob definitions for one scope.

Lives as a single JSON object in a bucket you own, at a path mirroring the data. Never inside the store, because data arrives that must not be altered.

A manifest is pure definition. It changes only when the set of blobs changes: a new variable, a new cut, a forced repartition. Not on reads, writes, tiering or restores.

Attributes:

Name	Type	Description
scope	str	Prefix these definitions apply to, relative to the data store. Usually one zarr store, but may be a parent prefix covering many small stores that should restore together.
blobs	tuple[Blob, ...]	The blob definitions. A list of rules, so its length tracks the number of cut decisions, not the number of blobs and certainly not the number of objects.
hot_always	tuple[str, ...]	Glob patterns that are never archivable, whatever the blobs say. Metadata objects and dimension coordinates. Derived from the store's structure and recomputed on every partition, so this is not where deliberate decisions belong.
pinned	tuple[Pin, ...]	Prefixes someone has deliberately kept hot. Preserved across repartitions, unlike hot_always, and the one part of a manifest that cannot be reconstructed by re-scanning.
policy	Policy	Thresholds used to produce this cut, recorded so a later run can tell whether they changed.
epoch	int	Bumped whenever the definitions change. Lets a resolver decide if it needs to reload without diffing.
generated_at	str	ISO 8601 UTC timestamp of the run that produced this.
generated_by	str	Package and version that produced this.
provenance	dict[str, Any]	Measured numbers per array, for debugging only. These can go stale while the manifest is untouched, so the resolver must never read them.

Example

m = Manifest("cordex/a.zarr", (Blob("b_pr", ("pr/c",)),), ... ("**/zarr.json",)) m.epoch 1 m.bumped().epoch 2

to_json

to_json() -> dict[str, Any]

Serialise, including the schema version.

Returns:

Type	Description
dict[str, Any]	The full manifest as a JSON-compatible dict.

Source code in src/blobmap/model.py

def to_json(self) -> dict[str, Any]:
    """Serialise, including the schema version.

    Returns:
        The full manifest as a JSON-compatible dict.
    """
    return {
        "schema_version": SCHEMA_VERSION,
        "scope": self.scope,
        "epoch": self.epoch,
        "generated_at": self.generated_at,
        "generated_by": self.generated_by,
        "policy": self.policy.to_json(),
        "hot_always": list(self.hot_always),
        "pinned": [p.to_json() for p in self.pinned],
        "blobs": [b.to_json() for b in self.blobs],
        "provenance": self.provenance,
    }

dumps

dumps(indent: int | None = 2) -> str

Serialise to JSON text.

Parameters:

Name	Type	Description	Default
indent	int | None	Passed to json.dumps. Keep it non-None: manifests are read by humans during incidents.	2

Returns:

Type	Description
str	JSON text ready to write to object storage.

Source code in src/blobmap/model.py

def dumps(self, indent: int | None = 2) -> str:
    """Serialise to JSON text.

    Args:
        indent: Passed to `json.dumps`. Keep it non-`None`: manifests are
            read by humans during incidents.

    Returns:
        JSON text ready to write to object storage.
    """
    return json.dumps(self.to_json(), indent=indent)

from_json classmethod

from_json(d: dict[str, Any]) -> Manifest

Rebuild from a parsed manifest.

Parameters:

Name	Type	Description	Default
d	dict[str, Any]	A parsed manifest object.	required

Returns:

Type	Description
Manifest	A Manifest.

Raises:

Type	Description
SchemaError	If the document does not match SCHEMA, including a version this package does not speak. Deliberately strict: silently misreading a blob definition would misroute chunks to the wrong tape unit.

Example

Manifest.from_json({"schema_version": 99, "scope": "s", ... "epoch": 1, "hot_always": [], "blobs": []}) Traceback (most recent call last): ... blobmap.schema.SchemaError: schema_version: expected 3, got 99

Source code in src/blobmap/model.py

@classmethod
def from_json(cls, d: dict[str, Any]) -> Manifest:
    """Rebuild from a parsed manifest.

    Args:
        d: A parsed manifest object.

    Returns:
        A `Manifest`.

    Raises:
        SchemaError: If the document does not match
            `SCHEMA`, including a version this
            package does not speak. Deliberately strict: silently
            misreading a blob definition would misroute chunks to the
            wrong tape unit.

    Example:
        >>> Manifest.from_json({"schema_version": 99, "scope": "s",
        ...                     "epoch": 1, "hot_always": [], "blobs": []})
        Traceback (most recent call last):
            ...
        blobmap.schema.SchemaError: schema_version: expected 3, got 99
    """
    validate_document(d)
    return cls(
        scope=str(d["scope"]),
        blobs=tuple(Blob.from_json(b) for b in d["blobs"]),
        hot_always=tuple(d["hot_always"]),
        pinned=tuple(Pin.from_json(x) for x in d.get("pinned", [])),
        policy=Policy.from_json(d.get("policy", {})),
        epoch=int(d.get("epoch", 1)),
        generated_at=str(d.get("generated_at", "")),
        generated_by=str(d.get("generated_by", "")),
        provenance=dict(d.get("provenance", {})),
    )

loads classmethod

loads(text: str | bytes) -> Manifest

Parse from JSON text.

Parameters:

Name	Type	Description	Default
text	str | bytes	JSON as read from object storage.	required

Returns:

Type	Description
Manifest	A Manifest.

Raises:

Type	Description
SchemaError	If the document does not match the schema.
JSONDecodeError	If the text is not JSON at all.

Example

m = Manifest("s", (Blob("b", ("x",)),), (), ... (Pin("x", "under active analysis"),), ... generated_at="2026-01-01T00:00:00+00:00") Manifest.loads(m.dumps()) == m True

Source code in src/blobmap/model.py

@classmethod
def loads(cls, text: str | bytes) -> Manifest:
    """Parse from JSON text.

    Args:
        text: JSON as read from object storage.

    Returns:
        A `Manifest`.

    Raises:
        SchemaError: If the document does not match the schema.
        json.JSONDecodeError: If the text is not JSON at all.

    Example:
        >>> m = Manifest("s", (Blob("b", ("x",)),), (),
        ...              (Pin("x", "under active analysis"),),
        ...              generated_at="2026-01-01T00:00:00+00:00")
        >>> Manifest.loads(m.dumps()) == m
        True
    """
    return cls.from_json(json.loads(text))

by_id

by_id() -> dict[str, Blob]

Index the blobs by id.

Returns:

Type	Description
dict[str, Blob]	Mapping of blob id to Blob, useful for diffing two manifests.

Source code in src/blobmap/model.py

def by_id(self) -> dict[str, Blob]:
    """Index the blobs by id.

    Returns:
        Mapping of blob id to `Blob`, useful for diffing two manifests.
    """
    return {b.id: b for b in self.blobs}

bumped

bumped(**changes: Any) -> Manifest

Copy with the epoch incremented and the timestamp refreshed.

Parameters:

Name	Type	Description	Default
**changes	Any	Any other fields to replace.	{}

Returns:

Type	Description
Manifest	A new Manifest. The original is unchanged, since manifests are
Manifest	frozen.

Source code in src/blobmap/model.py

def bumped(self, **changes: Any) -> Manifest:
    """Copy with the epoch incremented and the timestamp refreshed.

    Args:
        **changes: Any other fields to replace.

    Returns:
        A new `Manifest`. The original is unchanged, since manifests are
        frozen.
    """
    return replace(self, epoch=self.epoch + 1, generated_at=now(), **changes)

pin_for

pin_for(key: str) -> Pin | None

Find the pin covering a scope-relative key, if any.

Parameters:

Name	Type	Description	Default
key	str	Scope-relative object key.	required

Returns:

Type	Description
Pin | None	The first matching Pin, or None.

Source code in src/blobmap/model.py

def pin_for(self, key: str) -> Pin | None:
    """Find the pin covering a scope-relative key, if any.

    Args:
        key: Scope-relative object key.

    Returns:
        The first matching `Pin`, or `None`.
    """
    for pin in self.pinned:
        if pin.covers(key):
            return pin
    return None

pinned_blob_ids

pinned_blob_ids() -> set[str]

Blob ids whose definition is covered by a pin.

Pinning a prefix that already has a blob cannot remove that blob: repartitioning is additive, and dropping a blob id would orphan whatever tape copy is held against it. So the definition survives, nothing resolves to it any more, and its last-read timestamp goes stale -- at which point an age-based tiering policy would archive exactly the data someone asked to keep on disk.

A consumer must therefore exclude these before archiving. Resolution alone is not enough, because the policy query runs over blob state rather than over keys.

Returns:

Type	Description
set[str]	Ids of blobs covered by at least one pin. Note these are
set[str]	definition ids, not instance ids: a bucketed blob contributes
set[str]	b_tas, which covers every b_tas_n.

Example

m = Manifest("s", (Blob("b_tas", ("tas/c",)),), (), ... (Pin("tas", "under analysis"),)) m.pinned_blob_ids() {'b_tas'}

Source code in src/blobmap/model.py

def pinned_blob_ids(self) -> set[str]:
    """Blob ids whose definition is covered by a pin.

    Pinning a prefix that already has a blob cannot remove that blob:
    repartitioning is additive, and dropping a blob id would orphan
    whatever tape copy is held against it. So the definition survives,
    nothing resolves to it any more, and its last-read timestamp goes
    stale -- at which point an age-based tiering policy would archive
    exactly the data someone asked to keep on disk.

    A consumer must therefore exclude these before archiving. Resolution
    alone is not enough, because the policy query runs over blob state
    rather than over keys.

    Returns:
        Ids of blobs covered by at least one pin. Note these are
        definition ids, not instance ids: a bucketed blob contributes
        `b_tas`, which covers every `b_tas_n`.

    Example:
        >>> m = Manifest("s", (Blob("b_tas", ("tas/c",)),), (),
        ...              (Pin("tas", "under analysis"),))
        >>> m.pinned_blob_ids()
        {'b_tas'}
    """
    return {
        b.id
        for b in self.blobs
        for prefix in b.prefixes
        if any(pin.covers(prefix) for pin in self.pinned)
    }

validate

validate() -> None

Check the invariants that the resolver relies on.

Called automatically on construction. These are the cross-field rules the JSON Schema cannot express: uniqueness of ids, and the constraint that a bucketed blob has exactly one prefix.

Raises:

Type	Description
ValueError	On a duplicate or malformed blob id, a blob with no prefixes, an unknown key encoding, a bucketed blob with more than one prefix, or a non-positive width.

Example

Manifest("s", (Blob("b", ("x",)), Blob("b", ("y",))), ... ()).validate() Traceback (most recent call last): ... ValueError: duplicate blob id 'b'

Source code in src/blobmap/model.py

def validate(self) -> None:
    """Check the invariants that the resolver relies on.

    Called automatically on construction. These are the cross-field rules
    the JSON Schema cannot express: uniqueness of ids, and the constraint
    that a bucketed blob has exactly one prefix.

    Raises:
        ValueError: On a duplicate or malformed blob id, a blob with no
            prefixes, an unknown key encoding, a bucketed blob with more
            than one prefix, or a non-positive width.

    Example:
        >>> Manifest("s", (Blob("b", ("x",)), Blob("b", ("y",))),
        ...          ()).validate()
        Traceback (most recent call last):
            ...
        ValueError: duplicate blob id 'b'
    """
    seen: set[str] = set()
    for b in self.blobs:
        if b.id in seen:
            raise ValueError(f"duplicate blob id {b.id!r}")
        if not b.id or not all(c.isalnum() or c == "_" for c in b.id):
            raise ValueError(f"blob id {b.id!r} is not [a-z0-9_]")
        if not b.prefixes:
            raise ValueError(f"{b.id}: no prefixes")
        seen.add(b.id)
        if b.bucket is None:
            continue
        if b.bucket.key_encoding not in KEY_ENCODINGS:
            raise ValueError(
                f"{b.id}: unknown key_encoding {b.bucket.key_encoding!r}"
            )
        if len(b.prefixes) != 1:
            raise ValueError(
                f"{b.id}: a bucketed blob needs exactly one "
                f"prefix, got {len(b.prefixes)}"
            )
        if b.bucket.width < 1:
            raise ValueError(f"{b.id}: width must be >= 1")
        if b.bucket.index < 0:
            raise ValueError(f"{b.id}: index must be >= 0")

    seen_prefixes: set[str] = set()
    for pin in self.pinned:
        if not pin.reason.strip():
            raise ValueError(
                f"pin on {pin.prefix!r} has no reason; a pin "
                f"nobody can explain is one nobody removes"
            )
        if pin.prefix in seen_prefixes:
            raise ValueError(f"duplicate pin on {pin.prefix!r}")
        seen_prefixes.add(pin.prefix)

now

now() -> str

Get current UTC time as an ISO 8601 string, to second resolution.

Returns:

Type	Description
str	Timestamp such as 2026-08-03T09:12:00+00:00.

Example

stamp = now() stamp.endswith("+00:00") True

Source code in src/blobmap/model.py

def now() -> str:
    """Get current UTC time as an ISO 8601 string, to second resolution.

    Returns:
        Timestamp such as `2026-08-03T09:12:00+00:00`.

    Example:
        >>> stamp = now()
        >>> stamp.endswith("+00:00")
        True
    """
    return datetime.now(timezone.utc).isoformat(timespec="seconds")

default_hot_always

default_hot_always() -> list[str]

Patterns for metadata objects, which are never archivable.

This is the rule that keeps xr.open_zarr from triggering a tape mount, and incidentally why re-reading a store to partition it cannot feed its own event loop: everything the partitioner touches is already ineligible.

Returns:

Type	Description
list[str]	Glob patterns, one per known metadata basename.

Example

"**/zarr.json" in default_hot_always() True

Source code in src/blobmap/model.py

def default_hot_always() -> list[str]:
    """Patterns for metadata objects, which are never archivable.

    This is the rule that keeps `xr.open_zarr` from triggering a tape mount,
    and incidentally why re-reading a store to partition it cannot feed its
    own event loop: everything the partitioner touches is already ineligible.

    Returns:
        Glob patterns, one per known metadata basename.

    Example:
        >>> "**/zarr.json" in default_hot_always()
        True
    """
    return [f"**/{name}" for name in METADATA_BASENAMES]