zarr_metadata.v3.definition

zarr_metadata.v3.definition

The kinds of metadata in a Zarr v3 document, and the scopes this package provides for it.

The kind of metadata a definition defines is its type: CodecDefinition (with the codec's kind, and its size: "static" if the output size is determined by the input size, or "dynamic" if it depends on the values), DataTypeDefinition, ChunkGridDefinition, ChunkKeyEncodingDefinition, StorageTransformerDefinition, and, for the members of a document that the spec does not define, ArrayExtraFieldDefinition and GroupExtraFieldDefinition. What a definition is and how a field is read are in zarr_metadata.definition, which both formats share. ZarrV3Context is the v3 scope, with one slot for each kind: data_types, chunk_grids, chunk_key_encodings, codecs, storage_transformers, array_extra_fields and group_extra_fields. CORE and CORE_AND_EXTENSIONS are the v3 scopes this package provides.

A scope reads whole documents as well as fields. read_array_metadata_v3(document, context=CONTEXT), from zarr_metadata.model, reads each extension point of a v3 array document through the definitions in CONTEXT, and so do the model's from_json and from_key_value. The fill value is checked against the data type it names, by that data type's definition. The chunk grid is checked against the shape, by the grid's definition. The codecs are checked as a pipeline, each by its definition against the chunk it receives.

A member that holds another metadata field is annotated with the field alias of its kind (a shard's codecs: tuple[ZarrV3CodecField, ...]) and is read in the same scope as its parent field. A problem with the nested field belongs to that field and is reported at its location. The parent field is still read, as a document containing the nested field would be. A member that accepts only codecs of static size is annotated ZarrV3StaticCodecField. A shard's index_codecs uses this, because a reader locates the index from a size it knows before reading it. A codec of dynamic size in such a member is a problem at its location, when the field is read in a scope. A name that nothing claims is not checked, and its size is unknown, like everything else about it. ZarrV3MetadataFieldJSON describes the same JSON, but it is checked only as JSON, so a definition rejects a member typed with it. An extension with nothing to configure uses EmptyConfiguration, and is written with its name only.

A data type also declares how its values are stored. storage is a function of the configuration and the nested fields that returns a StorageClass: single bytes, several bytes at a time, or each value in as many bytes as it needs. A struct's storage depends on its fields. storage_of(data_type) calls it for a data type field the scope read. The bytes codec requires an endian for multi-byte numbers, and a struct rejects a field whose values vary in size. A data type that does not declare storage leaves it unknown.

A chunk grid declares which array shapes it fits with shape_rules, a function that yields the spec violations in a grid with its configuration over an array of a given shape (a dimension with no chunk length, or chunks that do not cover the dimension), located in the configuration. A grid that does not declare shape rules fits every shape. It also declares the lengths of its chunks along each axis of an array it fits, with chunk_lengths: a set per axis, because a rectilinear grid's chunks differ in length. A reading stores both results for the grid it read: an entry for each dimension of the shape, with None where the lengths are unknown.

A codec is checked against its input. The array passes its first codec a Chunk: the lengths of the grid's chunks along each of the array's dimensions, and the array's data type field, with None for unknown parts. A codec that receives an array declares the spec violations for a given chunk with chunk_rules, located in its configuration (for example, a transpose whose order has the wrong number of axes). An array -> array codec declares what it passes to the next codec with transition, whatever its chunk rules found. For example, transpose permutes the axes. A reading reads the codec fields as a pipeline. It checks their order (array -> array codecs, one array -> bytes codec, bytes -> bytes codecs), then checks each codec against the chunk it receives, and records a Stage for each codec with that chunk. A codec that contains its own pipelines declares what each one receives with pipelines, keyed by the configuration member that holds each pipeline. For example, a shard's inner codecs receive its inner chunks, and its index codecs receive the shard index. Each pipeline is read the same way, and its stages are stored as the codec's Stage.inner. The reader does not guess. After a codec the scope did not read, or after a codec that does not declare its output, the next codec receives a chunk with nothing known about it, Chunk(), which never causes a problem. A later codec can still declare its own output.

Raw bits are the only data type whose name encodes its configuration. A document writes r followed by the size in bits, and r16 is read as r* (the notation in the specification's table of data types) with {"bits": 16}. A reader that wants to read raw bits its own way defines r*. A data type named r16 is rejected, because that name is read as r*. r* itself is notation, and a document that uses it as a name refers to nothing in any scope.

JSON Schema. zarr_metadata.json_schema writes a v3 field, or a whole zarr.json, as a JSON Schema.

CORE module-attribute

CORE: Final = ZarrV3Context(
    data_types=(
        BOOL_DATA_TYPE,
        INT8_DATA_TYPE,
        INT16_DATA_TYPE,
        INT32_DATA_TYPE,
        INT64_DATA_TYPE,
        UINT8_DATA_TYPE,
        UINT16_DATA_TYPE,
        UINT32_DATA_TYPE,
        UINT64_DATA_TYPE,
        FLOAT16_DATA_TYPE,
        FLOAT32_DATA_TYPE,
        FLOAT64_DATA_TYPE,
        COMPLEX64_DATA_TYPE,
        COMPLEX128_DATA_TYPE,
        RAW_BYTES_DATA_TYPE,
    ),
    chunk_grids=(REGULAR_CHUNK_GRID,),
    chunk_key_encodings=(
        DEFAULT_CHUNK_KEY_ENCODING,
        V2_CHUNK_KEY_ENCODING,
    ),
    codecs=(
        BLOSC_CODEC,
        BYTES_CODEC,
        CRC32C_CODEC,
        GZIP_CODEC,
        SHARDING_INDEXED_CODEC,
        TRANSPOSE_CODEC,
    ),
    group_extra_fields=(CONSOLIDATED_METADATA_EXTRA_FIELD,),
)

A scope with only what the Zarr v3 specification defines, including the envelope of a group's consolidated_metadata, which the spec fixes.

CORE_AND_EXTENSIONS module-attribute

CORE_AND_EXTENSIONS: Final = CORE.update(
    data_types=[
        *CORE.data_types.values(),
        BYTES_DATA_TYPE,
        STRING_DATA_TYPE,
        NUMPY_DATETIME64_DATA_TYPE,
        NUMPY_TIMEDELTA64_DATA_TYPE,
        STRUCT_DATA_TYPE,
    ],
    chunk_grids=[
        *CORE.chunk_grids.values(),
        RECTILINEAR_CHUNK_GRID,
    ],
    codecs=[
        *CORE.codecs.values(),
        CAST_VALUE_CODEC,
        SCALE_OFFSET_CODEC,
        ZSTD_CODEC,
    ],
)

A scope with what the specification defines, plus the extensions that zarr-extensions registers and this package defines.

CodecKind module-attribute

CodecKind = Literal[
    "array_array", "array_bytes", "bytes_bytes"
]

What a codec converts its input to: the three positions a pipeline puts in order.

CodecSize module-attribute

CodecSize = Literal['static', 'dynamic']

Whether the size of a codec's output is determined by the size of its input.

static: it is. For example, bytes writes each element at its width, and crc32c adds four bytes. dynamic: the size depends on the values, as for every compressor.

Lengths module-attribute

Lengths: TypeAlias = tuple[frozenset[int] | None, ...]

For each axis, every chunk length along that axis (a set, because a rectilinear grid's lengths differ), or None where unknown.

StorageClass module-attribute

StorageClass = Literal[
    "single_byte", "multi_byte", "variable_length"
]

How a data type's values are stored: in single bytes, in several bytes at a time, or each value in as many bytes as it needs.

A multi-byte number is stored in a byte order, which the bytes codec's endian sets. A value made of single bytes (a uint8, or a struct of int8 fields) has no byte order. A value whose size varies needs its own codec.

ZarrV3ChunkGridField module-attribute

ZarrV3ChunkGridField = TypeAliasType(
    "ZarrV3ChunkGridField", ZarrV3MetadataFieldJSON
)

A member that holds a chunk grid: a document's chunk_grid.

ZarrV3ChunkKeyEncodingField module-attribute

ZarrV3ChunkKeyEncodingField = TypeAliasType(
    "ZarrV3ChunkKeyEncodingField", ZarrV3MetadataFieldJSON
)

A member that holds a chunk key encoding: a document's chunk_key_encoding.

ZarrV3CodecField module-attribute

ZarrV3CodecField = TypeAliasType(
    "ZarrV3CodecField", ZarrV3MetadataFieldJSON
)

A member that holds a codec. A document's codecs is tuple[ZarrV3CodecField, ...], and so is a shard's.

ZarrV3DataTypeField module-attribute

ZarrV3DataTypeField = TypeAliasType(
    "ZarrV3DataTypeField", ZarrV3MetadataFieldJSON
)

A member that holds a data type: a document's data_type, or a struct field's data type. It is read in the same scope as the object that contains it.

ZarrV3Kind module-attribute

ZarrV3Kind: TypeAlias = (
    DataTypeDefinition[Any]
    | ChunkGridDefinition[Any]
    | ChunkKeyEncodingDefinition[Any]
    | CodecDefinition[Any]
    | StorageTransformerDefinition[Any]
    | ArrayExtraFieldDefinition[Any]
    | GroupExtraFieldDefinition[Any]
)

A definition of a kind of v3 field.

ZarrV3StaticCodecField module-attribute

ZarrV3StaticCodecField = TypeAliasType(
    "ZarrV3StaticCodecField", ZarrV3MetadataFieldJSON
)

A member that holds a codec with a static encoded size. A shard's index_codecs uses this, because a reader locates the index from a size it knows before reading it.

ZarrV3StorageTransformerField module-attribute

ZarrV3StorageTransformerField = TypeAliasType(
    "ZarrV3StorageTransformerField", ZarrV3MetadataFieldJSON
)

A member that holds a storage transformer. A document's storage_transformers is tuple[ZarrV3StorageTransformerField, ...].

ArrayExtraFieldDefinition dataclass

Bases: ExtraFieldDefinition[C]

An extra field of a v3 array document: a member the spec does not define, read by its key.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class ArrayExtraFieldDefinition(ExtraFieldDefinition[C], kind=True, format=3):
    """An extra field of a v3 array document: a member the spec does not define, read by its key."""

    label: ClassVar[str] = "array extra field"
    spec_members: ClassVar[frozenset[str]] = frozenset(
        typeddict_keys(ZarrV3ArrayMetadataJSON).members
    )

label class-attribute

label: str = 'array extra field'

The name of the kind in error messages, for example "codec".

spec_members class-attribute

spec_members: frozenset[str] = frozenset(
    typeddict_keys(ZarrV3ArrayMetadataJSON).members
)

The members the spec defines in a document this kind reads extra fields of, which are never extra fields.

Chunk dataclass

The input a codec receives: chunks with given lengths along each axis, and a data type.

Unknown parts are None. This includes the lengths along an axis that the grid does not declare, and every part of the chunk output by a codec that does not declare its output. A data type field that the scope did not read is kept as written, and it also says nothing about the values. A codec's chunk rules check only what is known, so a chunk with nothing known about it, Chunk(), never causes a problem.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, slots=True)
class Chunk:
    """The input a codec receives: chunks with given lengths along each axis, and a data type.

    Unknown parts are None. This includes the lengths along an axis that
    the grid does not declare, and every part of the chunk output by a
    codec that does not declare its output. A data type field that the
    scope did not read is kept as written, and it also says nothing about
    the values. A codec's chunk rules check only what is known, so a chunk
    with nothing known about it, `Chunk()`, never causes a problem.
    """

    lengths: Lengths | None = None
    """For each axis, the chunk lengths along it; None when even the number of axes is unknown."""
    data_type: ResolvedField[DataTypeDefinition[Any]] | None = None
    """The data type field of the values, as a scope read it. None when no field declares the data type: a document with no data type (its reading holds `UNSET`) passes the pipeline a chunk of unknown type."""

    def __post_init__(self) -> None:
        lengths = cast("object", self.lengths)
        if lengths is not None and not _is_lengths(lengths):
            msg = f"a chunk's lengths are a frozenset of integers or None per axis, got {lengths!r}"
            raise TypeError(msg)
        data_type = cast("object", self.data_type)
        if data_type is not None and not _is_data_type_field(data_type):
            msg = f"a chunk's data type is a data type field a scope read, got {data_type!r}"
            raise TypeError(msg)

    @property
    def rank(self) -> int | None:
        """The number of axes; None when unknown."""
        return None if self.lengths is None else len(self.lengths)

data_type class-attribute instance-attribute

data_type: ResolvedField[DataTypeDefinition[Any]] | None = (
    None
)

The data type field of the values, as a scope read it. None when no field declares the data type: a document with no data type (its reading holds UNSET) passes the pipeline a chunk of unknown type.

lengths class-attribute instance-attribute

lengths: Lengths | None = None

For each axis, the chunk lengths along it; None when even the number of axes is unknown.

rank property

rank: int | None

The number of axes; None when unknown.

ChunkGridDefinition dataclass

Bases: Definition[C]

A chunk grid, and the array shapes it fits.

shape_rules yields the spec violations in a grid with this configuration over an array of a given shape, such as a dimension with no chunk length, or chunks that do not cover the dimension. It receives the configuration, the nested fields as the scope read them, and the shape, and it locates its problems in the configuration. A grid that does not declare shape rules fits every shape.

chunk_lengths determines what the first codec of the array's pipeline receives: the lengths of the grid's chunks along each axis of an array whose shape the grid fits. A regular grid has one length per axis, and a rectilinear grid has every length it lists. It is called only for a grid that its shape rules accept. A grid that does not declare it leaves the lengths along every axis unknown.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class ChunkGridDefinition(Definition[C], kind=True, format=3):
    """A chunk grid, and the array shapes it fits.

    `shape_rules` yields the spec violations in a grid with this
    configuration over an array of a given shape, such as a dimension with
    no chunk length, or chunks that do not cover the dimension. It receives
    the configuration, the nested fields as the scope read them, and the
    shape, and it locates its problems in the configuration. A grid that
    does not declare shape rules fits every shape.

    `chunk_lengths` determines what the first codec of the array's
    pipeline receives: the lengths of the grid's chunks along each axis of
    an array whose shape the grid fits. A regular grid has one length per
    axis, and a rectilinear grid has every length it lists. It is called
    only for a grid that its shape rules accept. A grid that does not
    declare it leaves the lengths along every axis unknown.
    """

    label: ClassVar[str] = "chunk grid"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (ZarrV3ChunkGridField,)

    shape_rules: Callable[[C, Nested, tuple[int, ...]], Iterable[ValidationProblem]] = no_rules
    """The spec violations in this grid over an array of a given shape, located in the configuration."""
    chunk_lengths: Callable[[C, Nested, tuple[int, ...]], Lengths] = unknown_lengths
    """The lengths of the chunks along each axis of an array whose shape the grid fits; None where unknown."""

chunk_lengths class-attribute instance-attribute

chunk_lengths: Callable[
    [C, Nested, tuple[int, ...]], Lengths
] = unknown_lengths

The lengths of the chunks along each axis of an array whose shape the grid fits; None where unknown.

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    ZarrV3ChunkGridField,
)

The field aliases used to annotate a configuration member that holds a field of this kind. For a codec: ZarrV3CodecField and ZarrV3StaticCodecField.

label class-attribute

label: str = 'chunk grid'

The name of the kind in error messages, for example "codec".

shape_rules class-attribute instance-attribute

shape_rules: Callable[
    [C, Nested, tuple[int, ...]],
    Iterable[ValidationProblem],
] = no_rules

The spec violations in this grid over an array of a given shape, located in the configuration.

ChunkKeyEncodingDefinition dataclass

Bases: Definition[C]

A chunk key encoding.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class ChunkKeyEncodingDefinition(Definition[C], kind=True, format=3):
    """A chunk key encoding."""

    label: ClassVar[str] = "chunk key encoding"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (ZarrV3ChunkKeyEncodingField,)

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    ZarrV3ChunkKeyEncodingField,
)

The field aliases used to annotate a configuration member that holds a field of this kind. For a codec: ZarrV3CodecField and ZarrV3StaticCodecField.

label class-attribute

label: str = 'chunk key encoding'

The name of the kind in error messages, for example "codec".

CodecDefinition dataclass

Bases: Definition[C]

A codec: what it converts its input to, and whether the size of its output is static.

A codec that receives an array (array -> array, array -> bytes) declares the spec violations for a given input Chunk with chunk_rules. It receives the configuration, the nested fields as the scope read them, and the chunk, and it locates its problems in the configuration. For example, a bytes codec without endian that receives a multi-byte data type is a problem. An array -> array codec also declares its output with transition: given its input chunk, it returns the chunk the next codec receives. For example, transpose permutes the axes, and cast_value changes the data type. These two functions correspond to the spec: a codec computes its output from the shape and data type of its input, and "If the decoded_representation_type is not supported, this algorithm must fail with an error" (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L987-L994). The transition is called for every input chunk, whatever the chunk rules found, so it must return only what is true in either case. For example, a transpose whose order has the wrong number of axes passes on unknown lengths. A codec that does not declare its output passes the next codec a chunk with nothing known about it.

A codec that contains its own pipelines declares what each one receives with pipelines. Given the codec's input chunk, it returns the chunk that the first codec of each pipeline receives, keyed by the configuration member that holds the pipeline. For example, a shard's inner codecs receive its inner chunks, and its index codecs receive the shard index. Like the transition, it is called whatever the chunk rules found, and must return only what is true in either case. A function that is never called for a codec of its kind is rejected, such as the chunk rules of a bytes -> bytes codec, which receives bytes.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class CodecDefinition(Definition[C], kind=True, format=3):
    """A codec: what it converts its input to, and whether the size of its output is static.

    A codec that receives an array (array -> array, array -> bytes)
    declares the spec violations for a given input `Chunk` with
    `chunk_rules`. It receives the configuration, the nested fields as the
    scope read them, and the chunk, and it locates its problems in the
    configuration. For example, a `bytes` codec without `endian` that
    receives a multi-byte data type is a problem. An array -> array codec
    also declares its output with `transition`: given its input chunk, it
    returns the chunk the next codec receives. For example, `transpose`
    permutes the axes, and `cast_value` changes the data type. These two
    functions correspond to the spec: a codec computes its output from the
    shape and data type of its input, and "If the
    decoded_representation_type is not supported, this algorithm must
    fail with an error"
    (https://github.com/zarr-developers/zarr-specs/blob/fc7dd9c9beb5a50b87f9b08b00bf50fc0048482f/docs/v3/core/index.rst#L987-L994).
    The transition is called for every input chunk, whatever the chunk
    rules found, so it must return only what is true in either case. For
    example, a `transpose` whose `order` has the wrong number of axes
    passes on unknown lengths. A codec that does not declare its output
    passes the next codec a chunk with nothing known about it.

    A codec that contains its own pipelines declares what each one
    receives with `pipelines`. Given the codec's input chunk, it returns
    the chunk that the first codec of each pipeline receives, keyed by the
    configuration member that holds the pipeline. For example, a shard's
    inner codecs receive its inner chunks, and its index codecs receive
    the shard index. Like the transition, it is called whatever the chunk
    rules found, and must return only what is true in either case. A
    function that is never called for a codec of its kind is rejected,
    such as the chunk rules of a bytes -> bytes codec, which receives
    bytes.
    """

    label: ClassVar[str] = "codec"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (ZarrV3CodecField, ZarrV3StaticCodecField)

    kind: CodecKind
    size: CodecSize
    chunk_rules: Callable[[C, Nested, Chunk], Iterable[ValidationProblem]] = no_rules
    """The spec violations for this codec with a given input chunk, located in the configuration."""
    transition: Callable[[C, Nested, Chunk], Chunk] = unknown_chunk
    """The chunk the next codec receives, given the chunk this array -> array codec receives."""
    pipelines: Callable[[C, Nested, Chunk], Mapping[str, Chunk]] = no_pipelines
    """The chunk each contained pipeline receives, keyed by the configuration member that holds the pipeline."""

    def _refusal(self) -> str | None:
        kind: object = self.kind
        if kind not in get_args(CodecKind):
            return f"{self.name!r}: kind is one of {get_args(CodecKind)!r}, got {kind!r}"
        size: object = self.size
        if size not in get_args(CodecSize):
            return f"{self.name!r}: size is one of {get_args(CodecSize)!r}, got {size!r}"
        defaults = {member.name: member.default for member in dataclasses.fields(CodecDefinition)}
        for member in _UNASKED[self.kind]:
            if getattr(self, member) is not defaults[member]:
                return f"{self.name!r}: {member}, which no codec of kind {kind!r} is asked"
        return None

    @classmethod
    def _alias_problems(
        cls, alias: TypeAliasType, definition: Definition[Any], loc: Loc
    ) -> Problems:
        """A codec of dynamic size in a member annotated with `ZarrV3StaticCodecField`, which admits only codecs of static size, is a problem at the field."""
        if alias not in STATIC_SIZE_ALIASES:
            return ()
        if not isinstance(definition, CodecDefinition) or definition.size == "static":
            return ()
        return problem(
            loc,
            f"{definition.name!r} is a codec of dynamic size, and only codecs of static size "
            "may be used here",
            "invalid_value",
        )

chunk_rules class-attribute instance-attribute

chunk_rules: Callable[
    [C, Nested, Chunk], Iterable[ValidationProblem]
] = no_rules

The spec violations for this codec with a given input chunk, located in the configuration.

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    ZarrV3CodecField,
    ZarrV3StaticCodecField,
)

The field aliases used to annotate a configuration member that holds a field of this kind. For a codec: ZarrV3CodecField and ZarrV3StaticCodecField.

label class-attribute

label: str = 'codec'

The name of the kind in error messages, for example "codec".

pipelines class-attribute instance-attribute

pipelines: Callable[
    [C, Nested, Chunk], Mapping[str, Chunk]
] = no_pipelines

The chunk each contained pipeline receives, keyed by the configuration member that holds the pipeline.

transition class-attribute instance-attribute

transition: Callable[[C, Nested, Chunk], Chunk] = (
    unknown_chunk
)

The chunk the next codec receives, given the chunk this array -> array codec receives.

DataTypeDefinition dataclass

Bases: WithFillValue[C]

A data type, and the fill value of an array with that data type.

fill_value is the JSON shape of a fill value, such as Int8FillValue. It is an annotation that the checker reads the same way it reads a configuration's members, including any range. fill_value_rules yields the spec violations in a fill value of that shape that the type cannot express, such as a hex string of the wrong width. The rules receive the configuration, the nested fields as the scope read them (a struct's field types), and the typed fill value. A data type that does not declare its fill value accepts any JSON.

fill_value_canonical writes a fill value without problems (well typed and allowed by the rules) in the single canonical spelling of its value. So two fill values are the same value of the type exactly when their canonical spellings are written the same way: "NaN" and "0x7fc00000" are the same float32, but 0.0 and -0.0 are two different values. It receives the same arguments as the rules. A data type that does not declare it has one spelling per value: the value as written.

storage declares how the values are stored: in single bytes, in several bytes at a time, or each value in as many bytes as it needs. The bytes codec uses this to decide whether the data type it receives needs an endian (it does for multi-byte numbers). A struct's storage depends on its fields, so storage receives the nested fields as the scope read them. A data type that does not declare it leaves its storage unknown.

A data type named the way a document writes raw bits of one size (such as r16) is rejected. That name is read as r*, so this definition would never be used.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class DataTypeDefinition(WithFillValue[C], kind=True, format=3):
    """A data type, and the fill value of an array with that data type.

    `fill_value` is the JSON shape of a fill value, such as
    `Int8FillValue`. It is an annotation that the checker reads the same
    way it reads a configuration's members, including any range.
    `fill_value_rules` yields the spec violations in a fill value of that
    shape that the type cannot express, such as a hex string of the wrong
    width.
    The rules receive the configuration, the nested fields as the scope
    read them (a struct's field types), and the typed fill value. A data
    type that does not declare its fill value accepts any JSON.

    `fill_value_canonical` writes a fill value without problems (well
    typed and allowed by the rules) in the single canonical spelling of
    its value. So two fill values are the same value of the type exactly
    when their canonical spellings are written the same way: `"NaN"` and
    `"0x7fc00000"` are the same `float32`, but `0.0` and `-0.0` are two
    different values. It receives the same arguments as the rules. A data
    type that does not declare it has one spelling per value: the value as
    written.

    `storage` declares how the values are stored: in single bytes, in
    several bytes at a time, or each value in as many bytes as it needs.
    The `bytes` codec uses this to decide whether the data type it
    receives needs an `endian` (it does for multi-byte numbers). A
    struct's storage depends on its fields, so `storage` receives the
    nested fields as the scope read them. A data type that does not
    declare it leaves its storage unknown.

    A data type named the way a document writes raw bits of one size (such
    as `r16`) is rejected. That name is read as `r*`, so this definition
    would never be used.
    """

    label: ClassVar[str] = "data type"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (ZarrV3DataTypeField,)

    @classmethod
    def _filed_under(cls, name: str) -> tuple[str | None, dict[str, JSONValue] | None]:
        if name == RAW_BYTES_NAME:
            return None, None
        written = RAW_BYTES_NAME_PATTERN.fullmatch(name)
        if written is not None:
            return RAW_BYTES_NAME, {"bits": int(written.group(1))}
        return name, None

    def _carrying_name(self, configuration: Mapping[str, JSONValue]) -> str | None:
        if self.name != RAW_BYTES_NAME:
            return None
        return f"r{configuration['bits']}"

    @classmethod
    def _envelope_json(cls, name: str, configuration: Mapping[str, JSONValue]) -> JSONValue:
        # A data type with nothing to configure is written as its bare name,
        # which is how core data types have been written since Zarr v3.0.
        if len(configuration) != 0:
            return {"name": name, "configuration": configuration}
        return name

    storage: Callable[[C, Nested], StorageClass | None] = unknown_storage
    """How the values are stored, given the configuration and the nested fields; None when unknown."""

    def _refusal(self) -> str | None:
        if RAW_BYTES_NAME_PATTERN.fullmatch(self.name) is not None:
            return (
                f"{self.name!r} is how a document writes raw bits of one size, which read as "
                f"{RAW_BYTES_NAME!r}; to read raw bits your own way, define {RAW_BYTES_NAME!r}"
            )
        # Use the explicit form, not bare `super()`, because a dataclass
        # with slots is rebuilt.
        return super(DataTypeDefinition, self)._refusal()

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    ZarrV3DataTypeField,
)

The field aliases used to annotate a configuration member that holds a field of this kind. For a codec: ZarrV3CodecField and ZarrV3StaticCodecField.

label class-attribute

label: str = 'data type'

The name of the kind in error messages, for example "codec".

storage class-attribute instance-attribute

storage: Callable[[C, Nested], StorageClass | None] = (
    unknown_storage
)

How the values are stored, given the configuration and the nested fields; None when unknown.

ExtraFieldDefinition dataclass

Bases: Definition[C]

A member of a v3 document that the spec does not define, an extra field: its name is the member's key, and its configuration is the member's value.

This is not a kind. ArrayExtraFieldDefinition and GroupExtraFieldDefinition are, because one key may mean different things in an array document and in a group document. An extra field is not one of the spec's extension points, and no registry names it, so any key may name one. As a field it is written as the member alone, {key: value}, and it is located at the member's key. A definition reads a member whose value is an object. A member that no definition in scope claims is not read: a reader keeps it as it is written, and lists it in must_understand_fields unless it says "must_understand": false.

finite says whether every number in the member must be finite, as RFC 8259 JSON requires. A member that holds user data sets it to False: consolidated metadata holds node documents, whose attributes may hold NaN, Infinity and -Infinity. A member that nothing claims is not read, so like attributes it may hold any number.

The spec lets every extension member carry "must_understand": false, which tells a reader that does not understand the member that it may ignore it. So the configuration declares must_understand, and a definition whose configuration does not is refused. A member the spec defines, such as an array's shape, is never an extra field, and a definition named after one is refused too.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class ExtraFieldDefinition(Definition[C]):
    """A member of a v3 document that the spec does not define, an extra field: its name is the member's key, and its configuration is the member's value.

    This is not a kind. `ArrayExtraFieldDefinition` and
    `GroupExtraFieldDefinition` are, because one key may mean different
    things in an array document and in a group document. An extra field is
    not one of the spec's extension points, and no registry names it, so
    any key may name one. As a field it is written as the member alone,
    `{key: value}`, and it is located at the member's key. A definition
    reads a member whose value is an object. A member that no definition
    in scope claims is not read: a reader keeps it as it is written, and
    lists it in `must_understand_fields` unless it says
    `"must_understand": false`.

    `finite` says whether every number in the member must be finite, as
    RFC 8259 JSON requires. A member that holds user data sets it to
    False: consolidated metadata holds node documents, whose attributes
    may hold `NaN`, `Infinity` and `-Infinity`. A member that nothing
    claims is not read, so like attributes it may hold any number.

    The spec lets every extension member carry `"must_understand": false`,
    which tells a reader that does not understand the member that it may
    ignore it. So the configuration declares `must_understand`, and a
    definition whose configuration does not is refused. A member the spec
    defines, such as an array's `shape`, is never an extra field, and a
    definition named after one is refused too.
    """

    finite: bool = True
    """Whether every number in the member must be finite."""

    spec_members: ClassVar[frozenset[str]] = frozenset()
    """The members the spec defines in a document this kind reads extra fields of, which are never extra fields."""

    def _finite_numbers(self) -> bool:
        return self.finite

    def _refusal(self) -> str | None:
        finite = cast("object", self.finite)
        if not isinstance(finite, bool):
            return f"{self.name!r}: finite is a boolean, got {finite!r}"
        if not is_typeddict(self.configuration):
            return None
        try:
            takes = _takes_the_waiver(self.configuration)
        except TypeError as error:
            # Name the member, as the checker names a member it cannot read.
            keys = typeddict_keys(self.configuration)
            member = "must_understand" if "must_understand" in keys.members else "extra_items"
            return f"{self.name!r}: {self.configuration.__name__}.{member}: {error}"
        if not takes:
            return (
                f"{self.name!r}: expected a configuration that takes must_understand as a "
                "boolean that may be false, which the spec lets every extension member carry; "
                f"{self.configuration.__name__} does not"
            )
        return None

    @classmethod
    def _name_problem(cls, name: str, at: Loc) -> ValidationProblem | None:
        # No registry names extra fields, so any key the spec does not
        # define may name one.
        if name in cls.spec_members:
            return ValidationProblem(
                at, f"{name!r} is a member the spec defines, not an extra field", "invalid_value"
            )
        return None

    @classmethod
    def _named_configuration(
        cls, value: object
    ) -> tuple[str | None, Mapping[str, object] | None, Problems]:
        # A member whose value is not an object has no configuration. Only
        # a definition that claims the member reads it, and `_refined`
        # refuses such a value then.
        if not is_object(value) or len(value) != 1:
            return None, None, ()
        ((name, member),) = value.items()
        if not isinstance(name, str):
            return None, None, ()
        if not is_object(member):
            return name, None, ()
        return name, {key: item for key, item in member.items() if isinstance(key, str)}, ()

    @classmethod
    def _envelope_problems(cls, value: object) -> Problems:
        name = cls._named_configuration(value)[0]
        if name is None:
            return (
                ValidationProblem(
                    (), "expected a member, as an object with one string key", "invalid_type"
                ),
            )
        bad = cls._name_problem(name, ())
        return () if bad is None else (bad,)

    @classmethod
    def _refined(
        cls, data: object, loc: Loc, definition: Definition[Any] | None
    ) -> tuple[JSONValue | None, Problems]:
        # The member's value is at the member's location, `loc`. A member
        # nothing claims is not read, so like attributes it is user data.
        # A claimed member is user data when its definition says so, and
        # its value must be an object.
        name = cls._named_configuration(data)[0]
        if name is None or not is_object(data):
            return refine_json(data, loc)
        if definition is not None and not is_object(data[name]):
            return None, (ValidationProblem(loc, "expected an object", "invalid_type"),)
        refine = (
            refine_json
            if isinstance(definition, ExtraFieldDefinition) and definition.finite
            else refine_user_data
        )
        value, problems = refine(data[name], loc)
        return (None if len(problems) != 0 else {name: value}), problems

    @classmethod
    def _envelope_json(cls, name: str, configuration: Mapping[str, JSONValue]) -> JSONValue:
        return {name: configuration}

    @classmethod
    def _unclaimed_json(
        cls, json: JSONValue, name: str, configuration: Mapping[str, JSONValue]
    ) -> JSONValue:
        # A member no definition claims is written as it is, whatever its
        # value; only a definition reads it as an object.
        return json

    @classmethod
    def _configuration_loc(cls, loc: Loc) -> Loc:
        return loc

    @classmethod
    def _name_loc(cls, loc: Loc) -> Loc:
        return loc

finite class-attribute instance-attribute

finite: bool = True

Whether every number in the member must be finite.

spec_members class-attribute

spec_members: frozenset[str] = frozenset()

The members the spec defines in a document this kind reads extra fields of, which are never extra fields.

GroupExtraFieldDefinition dataclass

Bases: ExtraFieldDefinition[C]

An extra field of a v3 group document: a member the spec does not define, read by its key. consolidated_metadata is one.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class GroupExtraFieldDefinition(ExtraFieldDefinition[C], kind=True, format=3):
    """An extra field of a v3 group document: a member the spec does not define, read by its key. `consolidated_metadata` is one."""

    label: ClassVar[str] = "group extra field"
    spec_members: ClassVar[frozenset[str]] = frozenset(
        typeddict_keys(ZarrV3GroupMetadataJSON).members
    )

label class-attribute

label: str = 'group extra field'

The name of the kind in error messages, for example "codec".

spec_members class-attribute

spec_members: frozenset[str] = frozenset(
    typeddict_keys(ZarrV3GroupMetadataJSON).members
)

The members the spec defines in a document this kind reads extra fields of, which are never extra fields.

Stage dataclass

One codec of a pipeline, and the chunk it receives.

Source code in src/zarr_metadata/v3/_pipeline.py

@dataclass(frozen=True, slots=True)
class Stage:
    """One codec of a pipeline, and the chunk it receives."""

    codec: ResolvedField[CodecDefinition[Any]]
    """The codec, as the scope read it."""
    incoming: Chunk | None
    """The chunk the codec receives.

    None for a codec that receives bytes (a bytes -> bytes codec, or any
    codec after the array -> bytes codec). Also None for an unclaimed codec
    when its input is not known to be an array.
    """
    inner: Mapping[str, tuple[Stage, ...]] = dataclasses.field(default_factory=_no_stages)
    """The pipelines the codec contains, keyed by the configuration member that holds each one. Each pipeline pairs each codec with the chunk it receives."""

    def __post_init__(self) -> None:
        # Make this read-only, like everything else a reading returns.
        object.__setattr__(self, "inner", MappingProxyType(dict(self.inner)))

    def __reduce__(self) -> tuple[type[Stage], tuple[object, ...]]:
        # A read-only view cannot be pickled, so pickle the stage from its constructor arguments.
        return Stage, (self.codec, self.incoming, dict(self.inner))

codec instance-attribute

codec: ResolvedField[CodecDefinition[Any]]

The codec, as the scope read it.

incoming instance-attribute

incoming: Chunk | None

The chunk the codec receives.

None for a codec that receives bytes (a bytes -> bytes codec, or any codec after the array -> bytes codec). Also None for an unclaimed codec when its input is not known to be an array.

inner class-attribute instance-attribute

inner: Mapping[str, tuple[Stage, ...]] = dataclasses.field(
    default_factory=_no_stages
)

The pipelines the codec contains, keyed by the configuration member that holds each one. Each pipeline pairs each codec with the chunk it receives.

StorageTransformerDefinition dataclass

Bases: Definition[C]

A storage transformer.

Source code in src/zarr_metadata/v3/_definition.py

@dataclass(frozen=True, kw_only=True, slots=True, repr=False)
class StorageTransformerDefinition(Definition[C], kind=True, format=3):
    """A storage transformer."""

    label: ClassVar[str] = "storage transformer"
    field_aliases: ClassVar[tuple[TypeAliasType, ...]] = (ZarrV3StorageTransformerField,)

field_aliases class-attribute

field_aliases: tuple[TypeAliasType, ...] = (
    ZarrV3StorageTransformerField,
)

The field aliases used to annotate a configuration member that holds a field of this kind. For a codec: ZarrV3CodecField and ZarrV3StaticCodecField.

label class-attribute

label: str = 'storage transformer'

The name of the kind in error messages, for example "codec".

ZarrV3Context

A scope that reads Zarr v3 documents. Every v3 reader takes one.

It implements Context, and has one slot for each kind of field a v3 document holds, as ZarrV3ContextSlots declares them. The slots for the spec's extension points are named like the document member that holds such fields: data_types, chunk_grids, chunk_key_encodings, codecs and storage_transformers. array_extra_fields and group_extra_fields hold the definitions of members the spec does not define, by key. Each slot is a read-only mapping from the name each definition is filed under (r* for raw bits) to the definition. The constructor and update take, for each slot, an iterable of definitions or such a mapping, so one scope's slot can be given to another. A definition of another kind than its slot holds, and a key that is not its definition's name, raise TypeError, and two different definitions under one name in a slot raise ScopeConflictError. To replace a definition, or a whole slot, use update. ZarrV3Context() is the scope that claims nothing. Two scopes are equal when they file the same definitions in each slot, and equal scopes have equal hashes.

Source code in src/zarr_metadata/v3/_context.py

@final
class ZarrV3Context:
    """A scope that reads Zarr v3 documents. Every v3 reader takes one.

    It implements `Context`, and has one slot for each kind of field a v3
    document holds, as `ZarrV3ContextSlots` declares them. The slots for
    the spec's extension points are named like the document member that
    holds such fields: `data_types`, `chunk_grids`, `chunk_key_encodings`,
    `codecs` and `storage_transformers`. `array_extra_fields` and
    `group_extra_fields` hold the definitions of members the spec does not
    define, by key. Each slot is a read-only mapping from the name each
    definition is filed under (`r*` for raw bits) to the definition. The
    constructor and `update` take, for each slot, an iterable of
    definitions or such a mapping, so one scope's slot can be given to
    another. A definition of another kind than its slot holds, and a key
    that is not its definition's name, raise `TypeError`, and two
    different definitions under one name in a slot raise
    `ScopeConflictError`. To replace a definition, or a whole slot, use
    `update`.
    `ZarrV3Context()` is the scope that claims nothing. Two scopes are
    equal when they file the same definitions in each slot, and equal
    scopes have equal hashes.
    """

    __slots__ = ("_tables",)

    kinds: ClassVar[Mapping[str, type[ZarrV3Kind]]] = slot_kinds(ZarrV3ContextSlots, is_v3_kind)
    """The kind of definition each slot holds, by slot, as `ZarrV3ContextSlots` declares them."""

    _tables: Tables

    def __init__(self, **slots: Unpack[ZarrV3ContextSlots]) -> None:
        self._tables = filed(self.kinds, slots, 3)

    @classmethod
    def _of(cls, tables: Tables) -> ZarrV3Context:
        """A scope of `tables`, which were filed for this class's slots."""
        context = object.__new__(cls)
        context._tables = tables
        return context

    @classmethod
    def _restored(cls, contents: Mapping[str, Iterable[Definition[Any]]]) -> ZarrV3Context:
        """A scope rebuilt from the definitions in each slot, as `__reduce__` gives them."""
        return cls._of(filed(cls.kinds, contents, 3))

    @property
    def zarr_format(self) -> Literal[3]:
        """The Zarr format of the documents this scope reads: 3."""
        return 3

    @property
    def data_types(self) -> Mapping[str, DataTypeDefinition[Any]]:
        """The data type definitions in scope, by the name each is filed under (`r*` for raw bits)."""
        return self.definitions_of(DataTypeDefinition)

    @property
    def chunk_grids(self) -> Mapping[str, ChunkGridDefinition[Any]]:
        """The chunk grid definitions in scope, by name."""
        return self.definitions_of(ChunkGridDefinition)

    @property
    def chunk_key_encodings(self) -> Mapping[str, ChunkKeyEncodingDefinition[Any]]:
        """The chunk key encoding definitions in scope, by name."""
        return self.definitions_of(ChunkKeyEncodingDefinition)

    @property
    def codecs(self) -> Mapping[str, CodecDefinition[Any]]:
        """The codec definitions in scope, by name."""
        return self.definitions_of(CodecDefinition)

    @property
    def storage_transformers(self) -> Mapping[str, StorageTransformerDefinition[Any]]:
        """The storage transformer definitions in scope, by name."""
        return self.definitions_of(StorageTransformerDefinition)

    @property
    def array_extra_fields(self) -> Mapping[str, ArrayExtraFieldDefinition[Any]]:
        """The definitions of members of an array document that the spec does not define, by key."""
        return self.definitions_of(ArrayExtraFieldDefinition)

    @property
    def group_extra_fields(self) -> Mapping[str, GroupExtraFieldDefinition[Any]]:
        """The definitions of members of a group document that the spec does not define, by key."""
        return self.definitions_of(GroupExtraFieldDefinition)

    def definitions_of(self, kind: type[_K]) -> Mapping[str, _K]:
        """The definitions of `kind` in scope, by the name each is filed under: the slot that holds `kind`. Raises `TypeError` for a kind of the other format."""
        return definitions_in(self._tables, kind)

    def claimant(self, kind: type[D], name: str) -> D | None:
        """The definition of `kind` in scope that reads `name` (a name as a document uses it), or None if there is none. Raises `TypeError` for a kind of the other format.

        This is the definition filed under the name that `filed_under`
        maps `name` to. That is `name` itself, except for raw bits, where
        `r16` is read by the definition of `r*`.
        """
        return claimant_in(self._tables, kind, name)

    def update(self, **slots: Unpack[ZarrV3ContextSlots]) -> ZarrV3Context:
        """This scope, with each slot given here holding only the definitions given for it. The other slots are kept.

        A slot can be given as a mapping from each definition's name to
        it, so a slot of this scope passes back in changed:
        `context.update(codecs={**context.codecs, "gzip": MY_GZIP})`
        replaces the gzip definition and keeps the other codecs, and
        `context.update(codecs=[*context.codecs.values(), MY_CODEC])` adds
        one, raising `ScopeConflictError` if the scope already files
        another definition under its name.
        """
        return self._of(updated(self._tables, self.kinds, slots, 3))

    def union(self, *others: ZarrV3Context) -> ZarrV3Context:
        """The smallest scope that files everything this scope and each of `others` file.

        Raises `ScopeConflictError` when two of them file different
        definitions under one name in a slot. To replace a definition on
        purpose, use `update`. Raises `TypeError` for a scope of another
        format.
        """
        refuse_others(others, ZarrV3Context)
        return self._of(
            joined_tables(self.kinds, [self._tables, *(other._tables for other in others)])
        )

    def __repr__(self) -> str:
        # Keep this short, because it appears in signatures as a default
        # argument. A full repr would list every definition, and `help` on a
        # validator would run to pages.
        count = sum(len(table) for table in self._tables.values())
        return f"ZarrV3Context(<{count} definitions>)"

    def __reduce__(self) -> tuple[object, tuple[dict[str, tuple[Definition[Any], ...]]]]:
        # A scope is defined by the definitions in its slots, so pickle it as
        # them. This lets a scope go to another process along with the
        # documents it will read there.
        return (type(self)._restored, (slot_contents(self._tables, self.kinds),))

    def __copy__(self) -> ZarrV3Context:
        return self

    def __deepcopy__(self, memo: dict[int, object]) -> ZarrV3Context:
        # A scope is immutable, so a copy can be the same object.
        return self

    def __eq__(self, other: object) -> bool:
        if not isinstance(other, ZarrV3Context):
            return NotImplemented
        return self._tables == other._tables

    def __hash__(self) -> int:
        return hash((ZarrV3Context, tables_key(self._tables)))

array_extra_fields property

array_extra_fields: Mapping[
    str, ArrayExtraFieldDefinition[Any]
]

The definitions of members of an array document that the spec does not define, by key.

chunk_grids property

chunk_grids: Mapping[str, ChunkGridDefinition[Any]]

The chunk grid definitions in scope, by name.

chunk_key_encodings property

chunk_key_encodings: Mapping[
    str, ChunkKeyEncodingDefinition[Any]
]

The chunk key encoding definitions in scope, by name.

codecs property

codecs: Mapping[str, CodecDefinition[Any]]

The codec definitions in scope, by name.

data_types property

data_types: Mapping[str, DataTypeDefinition[Any]]

The data type definitions in scope, by the name each is filed under (r* for raw bits).

group_extra_fields property

group_extra_fields: Mapping[
    str, GroupExtraFieldDefinition[Any]
]

The definitions of members of a group document that the spec does not define, by key.

kinds class-attribute

kinds: Mapping[str, type[ZarrV3Kind]] = slot_kinds(
    ZarrV3ContextSlots, is_v3_kind
)

The kind of definition each slot holds, by slot, as ZarrV3ContextSlots declares them.

storage_transformers property

storage_transformers: Mapping[
    str, StorageTransformerDefinition[Any]
]

The storage transformer definitions in scope, by name.

zarr_format property

zarr_format: Literal[3]

The Zarr format of the documents this scope reads: 3.

claimant

claimant(kind: type[D], name: str) -> D | None

The definition of kind in scope that reads name (a name as a document uses it), or None if there is none. Raises TypeError for a kind of the other format.

This is the definition filed under the name that filed_under maps name to. That is name itself, except for raw bits, where r16 is read by the definition of r*.

Source code in src/zarr_metadata/v3/_context.py

def claimant(self, kind: type[D], name: str) -> D | None:
    """The definition of `kind` in scope that reads `name` (a name as a document uses it), or None if there is none. Raises `TypeError` for a kind of the other format.

    This is the definition filed under the name that `filed_under`
    maps `name` to. That is `name` itself, except for raw bits, where
    `r16` is read by the definition of `r*`.
    """
    return claimant_in(self._tables, kind, name)

definitions_of

definitions_of(kind: type[_K]) -> Mapping[str, _K]

The definitions of kind in scope, by the name each is filed under: the slot that holds kind. Raises TypeError for a kind of the other format.

Source code in src/zarr_metadata/v3/_context.py

def definitions_of(self, kind: type[_K]) -> Mapping[str, _K]:
    """The definitions of `kind` in scope, by the name each is filed under: the slot that holds `kind`. Raises `TypeError` for a kind of the other format."""
    return definitions_in(self._tables, kind)

union

union(*others: ZarrV3Context) -> ZarrV3Context

The smallest scope that files everything this scope and each of others file.

Raises ScopeConflictError when two of them file different definitions under one name in a slot. To replace a definition on purpose, use update. Raises TypeError for a scope of another format.

Source code in src/zarr_metadata/v3/_context.py

def union(self, *others: ZarrV3Context) -> ZarrV3Context:
    """The smallest scope that files everything this scope and each of `others` file.

    Raises `ScopeConflictError` when two of them file different
    definitions under one name in a slot. To replace a definition on
    purpose, use `update`. Raises `TypeError` for a scope of another
    format.
    """
    refuse_others(others, ZarrV3Context)
    return self._of(
        joined_tables(self.kinds, [self._tables, *(other._tables for other in others)])
    )

update

update(
    **slots: Unpack[ZarrV3ContextSlots],
) -> ZarrV3Context

This scope, with each slot given here holding only the definitions given for it. The other slots are kept.

A slot can be given as a mapping from each definition's name to it, so a slot of this scope passes back in changed: context.update(codecs={**context.codecs, "gzip": MY_GZIP}) replaces the gzip definition and keeps the other codecs, and context.update(codecs=[*context.codecs.values(), MY_CODEC]) adds one, raising ScopeConflictError if the scope already files another definition under its name.

Source code in src/zarr_metadata/v3/_context.py

def update(self, **slots: Unpack[ZarrV3ContextSlots]) -> ZarrV3Context:
    """This scope, with each slot given here holding only the definitions given for it. The other slots are kept.

    A slot can be given as a mapping from each definition's name to
    it, so a slot of this scope passes back in changed:
    `context.update(codecs={**context.codecs, "gzip": MY_GZIP})`
    replaces the gzip definition and keeps the other codecs, and
    `context.update(codecs=[*context.codecs.values(), MY_CODEC])` adds
    one, raising `ScopeConflictError` if the scope already files
    another definition under its name.
    """
    return self._of(updated(self._tables, self.kinds, slots, 3))

ZarrV3ContextSlots

Bases: TypedDict

The definitions in each slot of a v3 scope, as ZarrV3Context and its update take them.

This is where a v3 scope's slots are declared: each key is a slot, and its value the definitions of the kind that slot holds, as an iterable of them or as a mapping from each one's name to it. ZarrV3Context.kinds is read from these annotations, so a type checker checks every slot a caller names, and every definition given in it.

Source code in src/zarr_metadata/v3/_context.py

class ZarrV3ContextSlots(TypedDict, total=False, closed=True):
    """The definitions in each slot of a v3 scope, as `ZarrV3Context` and its `update` take them.

    This is where a v3 scope's slots are declared: each key is a slot,
    and its value the definitions of the kind that slot holds, as an
    iterable of them or as a mapping from each one's name to it.
    `ZarrV3Context.kinds` is read from these annotations, so a type
    checker checks every slot a caller names, and every definition given
    in it.
    """

    data_types: Iterable[DataTypeDefinition[Any]] | Mapping[str, DataTypeDefinition[Any]]
    """Data types, read from `data_type`."""
    chunk_grids: Iterable[ChunkGridDefinition[Any]] | Mapping[str, ChunkGridDefinition[Any]]
    """Chunk grids, read from `chunk_grid`."""
    chunk_key_encodings: (
        Iterable[ChunkKeyEncodingDefinition[Any]] | Mapping[str, ChunkKeyEncodingDefinition[Any]]
    )
    """Chunk key encodings, read from `chunk_key_encoding`."""
    codecs: Iterable[CodecDefinition[Any]] | Mapping[str, CodecDefinition[Any]]
    """Codecs, read from `codecs` and from the codecs a codec holds."""
    storage_transformers: (
        Iterable[StorageTransformerDefinition[Any]]
        | Mapping[str, StorageTransformerDefinition[Any]]
    )
    """Storage transformers, read from `storage_transformers`."""
    array_extra_fields: (
        Iterable[ArrayExtraFieldDefinition[Any]] | Mapping[str, ArrayExtraFieldDefinition[Any]]
    )
    """Members of an array document that the spec does not define, by key."""
    group_extra_fields: (
        Iterable[GroupExtraFieldDefinition[Any]] | Mapping[str, GroupExtraFieldDefinition[Any]]
    )
    """Members of a group document that the spec does not define, by key."""

array_extra_fields instance-attribute

array_extra_fields: (
    Iterable[ArrayExtraFieldDefinition[Any]]
    | Mapping[str, ArrayExtraFieldDefinition[Any]]
)

Members of an array document that the spec does not define, by key.

chunk_grids instance-attribute

chunk_grids: (
    Iterable[ChunkGridDefinition[Any]]
    | Mapping[str, ChunkGridDefinition[Any]]
)

Chunk grids, read from chunk_grid.

chunk_key_encodings instance-attribute

chunk_key_encodings: (
    Iterable[ChunkKeyEncodingDefinition[Any]]
    | Mapping[str, ChunkKeyEncodingDefinition[Any]]
)

Chunk key encodings, read from chunk_key_encoding.

codecs instance-attribute

codecs: (
    Iterable[CodecDefinition[Any]]
    | Mapping[str, CodecDefinition[Any]]
)

Codecs, read from codecs and from the codecs a codec holds.

data_types instance-attribute

data_types: (
    Iterable[DataTypeDefinition[Any]]
    | Mapping[str, DataTypeDefinition[Any]]
)

Data types, read from data_type.

group_extra_fields instance-attribute

group_extra_fields: (
    Iterable[GroupExtraFieldDefinition[Any]]
    | Mapping[str, GroupExtraFieldDefinition[Any]]
)

Members of a group document that the spec does not define, by key.

storage_transformers instance-attribute

storage_transformers: (
    Iterable[StorageTransformerDefinition[Any]]
    | Mapping[str, StorageTransformerDefinition[Any]]
)

Storage transformers, read from storage_transformers.

storage_of

storage_of(
    data_type: ResolvedField[DataTypeDefinition[Any]],
) -> StorageClass | None

How the values of data_type (a data type field a scope read) are stored; None when unknown.

The storage is unknown when the scope did not read the data type, or its definition does not declare storage. The definition's storage is the extension author's code. Its result is checked to be a storage class. If it raises an error, a note is added that says which data type's storage raised it.

Source code in src/zarr_metadata/v3/_definition.py

def storage_of(data_type: ResolvedField[DataTypeDefinition[Any]]) -> StorageClass | None:
    """How the values of `data_type` (a data type field a scope read) are stored; None when unknown.

    The storage is unknown when the scope did not read the data type, or
    its definition does not declare storage. The definition's `storage` is
    the extension author's code. Its result is checked to be a storage
    class. If it raises an error, a note is added that says which data
    type's storage raised it.
    """
    if not isinstance(data_type, AcceptedField):
        return None
    definition, configuration = data_type.definition, data_type.configuration
    found = asked(
        definition,
        "storage",
        lambda: cast("object", definition.storage(read_only(configuration), data_type.nested)),
    )
    if found is not None and found not in STORAGE_CLASSES:
        msg = (
            f"{definition.name!r}: its storage gives one of {STORAGE_CLASSES!r} or None, "
            f"got {found!r}"
        )
        raise TypeError(msg)
    return found