`zarr_metadata.pydantic`

zarr_metadata.pydantic ¶

Optional pydantic (v2) integration: field types over the core models.

Importing this module requires pydantic; the core package deliberately does not depend on it, so this module is never imported by zarr_metadata itself.

Each exported name is an Annotated field type over the corresponding core model class — the instances ARE the core classes, so values interoperate freely with non-pydantic code (equality, isinstance, nesting). Validation delegates to the library: a raw document routes through from_json (the single source of truth for validation and normalization, so pydantic's field-level coercion can never bypass it). A field type reads the fields of its document in the scope given by pydantic's validation context (pydantic passes this context to every validator). The scope is chosen as follows:

If the context is a Context, every field type of that format reads in it. A field type of the other format raises TypeError.
If the context is a mapping, the v3 field types read in its "zarr_metadata_context" item and the v2 field types read in its "zarr_metadata_context_v2" item. This lets a model with fields of both formats give a scope for each format.
If no scope is given for a format, its field types read in the default scope for that format, CORE_AND_EXTENSIONS or CORE_V2.

A mapping key that starts like these keys but is not one of them raises TypeError, so a misspelled key is not read as no scope. To fix a field's scope whatever the validation context holds, annotate the model class with InScope: Annotated[ZarrV3ArrayMetadata, InScope(CONTEXT)].

For example:

TypeAdapter(zmp.ZarrV3ArrayMetadataField).validate_python(document, context=CONTEXT)
ArrayManifest.model_validate(data, context={"zarr_metadata_context": CONTEXT})
Mixed.model_validate(data, context={"zarr_metadata_context": V3, "zarr_metadata_context_v2": V2})

An existing model instance passes through unchanged when no scope is given. When one is, the instance is read again in it with refined_in, so an instance that scope reads differently is refused. Serialization emits the document as it was written, via to_json. A failed parse raises a pydantic ValidationError with one line error per problem, in the same format pydantic uses for its own errors. Each line error has the problem's kind as its type, the problem's loc under the field's loc, the input found there, and the problem's ctx. If the message contains a ctx placeholder, the message is passed in the ctx as message. Annotate a field with these types, or with a core class and InScope, not with a bare core class, because pydantic cannot build a schema for the core classes.

A node's attributes may hold NaN, Infinity or -Infinity, which the models read and write as zarr-python does. Pydantic writes JSON by its own rules, and by default writes such a number as null. To keep these numbers, a BaseModel with one of these fields needs ser_json_inf_nan="constants" in its model_config. A TypeAdapter over a field type takes no config, so write its value with the model's to_key_value instead.

Pydantic's own JSON code paths limit nesting to less than the 256 levels the readers support: validate_json rejects a document nested about 200 levels deep, and dump_json rejects one nested about 255 levels deep, each with its own error. For a document that deep, call validate_python on the parsed JSON, and write it with the model's to_key_value.

Usage:

import zarr_metadata.pydantic as zmp

class ArrayManifest(BaseModel):
    path: str
    metadata: zmp.ZarrV3ArrayMetadataField

Static type checkers see each field type as its core model class, so manifest.metadata is a zarr_metadata.model.ZarrV3ArrayMetadata.

CONTEXT_KEY `module-attribute` ¶

CONTEXT_KEY: Final = 'zarr_metadata_context'

The key in a mapping validation context that holds the scope for the v3 field types.

CONTEXT_KEY_V2 `module-attribute` ¶

CONTEXT_KEY_V2: Final = 'zarr_metadata_context_v2'

The key in a mapping validation context that holds the scope for the v2 field types.

ZarrV2ArrayMetadataField `module-attribute` ¶

ZarrV2ArrayMetadataField = Annotated[
    InstanceOf[_model.ZarrV2ArrayMetadata],
    BeforeValidator(
        _read_in_context(
            _model.ZarrV2ArrayMetadata,
            _model.ZarrV2ArrayMetadata.from_json,
            _model.ZarrV2ArrayMetadata.refined_in,
            CORE_V2,
            CONTEXT_KEY_V2,
        ),
        json_schema_input_type=_ZarrV2ArrayMetadataSchema,
    ),
    PlainSerializer(
        _model.ZarrV2ArrayMetadata.to_json,
        return_type=_ZarrV2ArrayMetadataSchema,
    ),
]

Field type for a v2 array metadata document (merged .zarray + .zattrs form).

ZarrV2ConsolidatedMetadataField `module-attribute` ¶

ZarrV2ConsolidatedMetadataField = Annotated[
    InstanceOf[_model.ZarrV2ConsolidatedMetadata],
    BeforeValidator(
        _read_in_context(
            _model.ZarrV2ConsolidatedMetadata,
            _model.ZarrV2ConsolidatedMetadata.from_json,
            _model.ZarrV2ConsolidatedMetadata.refined_in,
            CORE_V2,
            CONTEXT_KEY_V2,
        ),
        json_schema_input_type=_ZarrV2ConsolidatedMetadataSchema,
    ),
    PlainSerializer(
        _model.ZarrV2ConsolidatedMetadata.to_json,
        return_type=_ZarrV2ConsolidatedMetadataSchema,
    ),
]

Field type for a v2 .zmetadata document.

ZarrV2GroupMetadataField `module-attribute` ¶

ZarrV2GroupMetadataField = Annotated[
    InstanceOf[_model.ZarrV2GroupMetadata],
    BeforeValidator(
        _read_in_context(
            _model.ZarrV2GroupMetadata,
            _model.ZarrV2GroupMetadata.from_json,
            _model.ZarrV2GroupMetadata.refined_in,
            CORE_V2,
            CONTEXT_KEY_V2,
        ),
        json_schema_input_type=_ZarrV2GroupMetadataSchema,
    ),
    PlainSerializer(
        _model.ZarrV2GroupMetadata.to_json,
        return_type=_ZarrV2GroupMetadataSchema,
    ),
]

Field type for a v2 group metadata document (merged .zgroup + .zattrs form).

ZarrV3ArrayMetadataField `module-attribute` ¶

ZarrV3ArrayMetadataField = Annotated[
    InstanceOf[_model.ZarrV3ArrayMetadata],
    BeforeValidator(
        _read_in_context(
            _model.ZarrV3ArrayMetadata,
            _model.ZarrV3ArrayMetadata.from_json,
            _model.ZarrV3ArrayMetadata.refined_in,
            CORE_AND_EXTENSIONS,
            CONTEXT_KEY,
        ),
        json_schema_input_type=_ZarrV3ArrayMetadataSchema,
    ),
    PlainSerializer(
        _model.ZarrV3ArrayMetadata.to_json,
        return_type=_ZarrV3ArrayMetadataSchema,
    ),
]

Field type for a v3 array metadata document (zarr.json content).

ZarrV3ConsolidatedMetadataField `module-attribute` ¶

ZarrV3ConsolidatedMetadataField = Annotated[
    InstanceOf[_model.ZarrV3ConsolidatedMetadata],
    BeforeValidator(
        _read_in_context(
            _model.ZarrV3ConsolidatedMetadata,
            _model.ZarrV3ConsolidatedMetadata.from_json,
            _model.ZarrV3ConsolidatedMetadata.refined_in,
            CORE_AND_EXTENSIONS,
            CONTEXT_KEY,
        ),
        json_schema_input_type=_ZarrV3ConsolidatedMetadataSchema,
    ),
    PlainSerializer(
        _model.ZarrV3ConsolidatedMetadata.to_json,
        return_type=_ZarrV3ConsolidatedMetadataSchema,
    ),
]

Field type for v3 inline consolidated metadata.

ZarrV3GroupMetadataField `module-attribute` ¶

ZarrV3GroupMetadataField = Annotated[
    InstanceOf[_model.ZarrV3GroupMetadata],
    BeforeValidator(
        _read_in_context(
            _model.ZarrV3GroupMetadata,
            _model.ZarrV3GroupMetadata.from_json,
            _model.ZarrV3GroupMetadata.refined_in,
            CORE_AND_EXTENSIONS,
            CONTEXT_KEY,
        ),
        json_schema_input_type=_ZarrV3GroupMetadataSchema,
    ),
    PlainSerializer(
        _model.ZarrV3GroupMetadata.to_json,
        return_type=_ZarrV3GroupMetadataSchema,
    ),
]

Field type for a v3 group metadata document (zarr.json content).

all `module-attribute` ¶

__all__ = [
    "CONTEXT_KEY",
    "CONTEXT_KEY_V2",
    "InScope",
    "ZarrV2ArrayMetadataField",
    "ZarrV2ConsolidatedMetadataField",
    "ZarrV2GroupMetadataField",
    "ZarrV3ArrayMetadataField",
    "ZarrV3ConsolidatedMetadataField",
    "ZarrV3GroupMetadataField",
]

InScope `dataclass` ¶

A field type's scope, fixed: Annotated[ZarrV3ArrayMetadata, InScope(CONTEXT)] is a field of that model class read in CONTEXT, whatever the validation context holds.

A document is read in context, and a model instance is read again in it with refined_in, so an instance that context reads differently is refused rather than passed through in its own scope. The field writes the document as written, and a type checker sees the model class. A scope of the other format raises TypeError when pydantic builds the schema. InScope builds the field's whole schema, as pydantic's PlainValidator does, so Annotated metadata before it is not applied: put a validator of your own after it.

Source code in src/zarr_metadata/pydantic.py

@dataclass(frozen=True, slots=True)
class InScope:
    """A field type's scope, fixed: `Annotated[ZarrV3ArrayMetadata, InScope(CONTEXT)]` is a field of that model class read in `CONTEXT`, whatever the validation context holds.

    A document is read in `context`, and a model instance is read again
    in it with `refined_in`, so an instance that `context` reads
    differently is refused rather than passed through in its own scope.
    The field writes the document as written, and a type checker sees the
    model class. A scope of the other format raises `TypeError` when
    pydantic builds the schema. `InScope` builds the field's whole schema,
    as pydantic's `PlainValidator` does, so `Annotated` metadata before it
    is not applied: put a validator of your own after it.
    """

    context: ZarrV3Context | ZarrV2Context

    def __post_init__(self) -> None:
        if not _is_context(self.context):
            msg = f"InScope takes a ZarrV3Context or a ZarrV2Context, got {self.context!r}"
            raise TypeError(msg)

    def __get_pydantic_core_schema__(
        self,
        source: Any,  # noqa: ANN401 - pydantic passes the annotated class
        handler: GetCoreSchemaHandler,
    ) -> CoreSchema:
        parts = _FIELD_PARTS.get(source)
        if parts is None:
            msg = f"InScope annotates a model class of zarr_metadata.model, got {source!r}"
            raise TypeError(msg)
        default, schema = parts
        cls: type[Any] = source
        document = handler.generate_schema(schema)
        return core_schema.no_info_before_validator_function(
            _read_in(cls, cls.from_json, cls.refined_in, scoped(self.context, default)),
            core_schema.is_instance_schema(cls),
            json_schema_input_schema=document,
            serialization=core_schema.plain_serializer_function_ser_schema(
                cls.to_json, return_schema=document
            ),
        )

context `instance-attribute` ¶

context: ZarrV3Context | ZarrV2Context

__get_pydantic_core_schema__ ¶

__get_pydantic_core_schema__(
    source: Any, handler: GetCoreSchemaHandler
) -> CoreSchema

Source code in src/zarr_metadata/pydantic.py

def __get_pydantic_core_schema__(
    self,
    source: Any,  # noqa: ANN401 - pydantic passes the annotated class
    handler: GetCoreSchemaHandler,
) -> CoreSchema:
    parts = _FIELD_PARTS.get(source)
    if parts is None:
        msg = f"InScope annotates a model class of zarr_metadata.model, got {source!r}"
        raise TypeError(msg)
    default, schema = parts
    cls: type[Any] = source
    document = handler.generate_schema(schema)
    return core_schema.no_info_before_validator_function(
        _read_in(cls, cls.from_json, cls.refined_in, scoped(self.context, default)),
        core_schema.is_instance_schema(cls),
        json_schema_input_schema=document,
        serialization=core_schema.plain_serializer_function_ser_schema(
            cls.to_json, return_schema=document
        ),
    )

init ¶

__init__(context: ZarrV3Context | ZarrV2Context) -> None

__post_init__ ¶

__post_init__() -> None

Source code in src/zarr_metadata/pydantic.py

def __post_init__(self) -> None:
    if not _is_context(self.context):
        msg = f"InScope takes a ZarrV3Context or a ZarrV2Context, got {self.context!r}"
        raise TypeError(msg)

zarr_metadata.pydantic