zarr_metadata.problems

zarr_metadata.problems

The problems every reader in this package reports, and the error that carries them.

A reader returns problems as values: typed_json.check, definition.resolve and the model readers each return a tuple of ValidationProblem. Each problem has a loc (where it is), a kind (one of the ProblemKind values, for code that handles problems by kind), a message, and the input and ctx its message is made from. A constructor that cannot return problems, such as a model's from_json, raises MetadataValidationError with every problem in .problems.

shown formats a value for a problem's message, as the package's own messages do: as JSON (null, [1, 2], "C"). An extension's rules use it so their messages read like the package's.

ProblemKind module-attribute

ProblemKind = Literal[
    "missing_key",
    "invalid_type",
    "invalid_value",
    "invalid_json",
    "unknown_key",
]

Machine-readable classification of a ValidationProblem.

• missing_key: a required key (document key or store key) is absent.
• invalid_type: a value has the wrong structural type (e.g. a string where a mapping is required, a non-JSON-serializable object).
• invalid_value: a value has an acceptable type but an invalid content (e.g. zarr_format: 2 in a v3 document, order: "Q").
• invalid_json: bytes that do not decode as JSON.
• unknown_key: a key that an object's type does not declare, where the type is closed (as a closed TypedDict is). Zarr configurations rarely state whether they are closed (zarr-developers/zarr-specs#270 has been open since 2023), and many readers reject such a key. It has its own kind so that a caller who tolerates it can distinguish it from a wrong value, and so that it never hides other problems in the same object.

Problems module-attribute

Problems: TypeAlias = tuple[ValidationProblem, ...]

Every problem a reader found, in the order it found them.

__all__ module-attribute

__all__ = [
    "MetadataValidationError",
    "ProblemKind",
    "Problems",
    "ValidationProblem",
    "shown",
]

MetadataValidationError

Bases: ValueError

Raised when a value fails validation, by the entry points that raise instead of returning problems.

Carries every problem found (not just the first) in .problems, as an immutable tuple: a raised error is a finished report, and a caller inspecting it must not be able to edit the record.

Source code in src/zarr_metadata/_json.py

class MetadataValidationError(ValueError):
    """Raised when a value fails validation, by the entry points that raise instead of returning problems.

    Carries every problem found (not just the first) in `.problems`, as an
    immutable tuple: a raised error is a finished report, and a caller
    inspecting it must not be able to edit the record.
    """

    problems: tuple[ValidationProblem, ...]

    def __init__(self, problems: Sequence[ValidationProblem]) -> None:
        self.problems = tuple(problems)
        for entry in cast("tuple[object, ...]", self.problems):
            # Enforce the annotation at runtime. A caller that is not
            # type-checked and passes something else fails here, instead of
            # later, at the code that reads a `loc` from it.
            if not isinstance(entry, ValidationProblem):
                msg = (
                    "MetadataValidationError takes ValidationProblem values, "
                    f"got {type(entry).__name__}"
                )
                raise TypeError(msg)
        super().__init__("\n".join(str(problem) for problem in self.problems))

    def __reduce__(
        self,
    ) -> tuple[
        type[MetadataValidationError], tuple[tuple[ValidationProblem, ...]], dict[str, object]
    ]:
        # By default an exception pickles and copies by calling its class
        # with its `args`, which here hold the message. This one is rebuilt
        # from its problems instead, and the rest of its state (including
        # its notes) is restored after.
        return (type(self), (self.problems,), self.__dict__)

problems instance-attribute

problems: tuple[ValidationProblem, ...] = tuple(problems)

__init__

__init__(problems: Sequence[ValidationProblem]) -> None

Source code in src/zarr_metadata/_json.py

def __init__(self, problems: Sequence[ValidationProblem]) -> None:
    self.problems = tuple(problems)
    for entry in cast("tuple[object, ...]", self.problems):
        # Enforce the annotation at runtime. A caller that is not
        # type-checked and passes something else fails here, instead of
        # later, at the code that reads a `loc` from it.
        if not isinstance(entry, ValidationProblem):
            msg = (
                "MetadataValidationError takes ValidationProblem values, "
                f"got {type(entry).__name__}"
            )
            raise TypeError(msg)
    super().__init__("\n".join(str(problem) for problem in self.problems))

__reduce__

__reduce__() -> tuple[
    type[MetadataValidationError],
    tuple[tuple[ValidationProblem, ...]],
    dict[str, object],
]

Source code in src/zarr_metadata/_json.py

def __reduce__(
    self,
) -> tuple[
    type[MetadataValidationError], tuple[tuple[ValidationProblem, ...]], dict[str, object]
]:
    # By default an exception pickles and copies by calling its class
    # with its `args`, which here hold the message. This one is rebuilt
    # from its problems instead, and the rest of its state (including
    # its notes) is restored after.
    return (type(self), (self.problems,), self.__dict__)

ValidationProblem dataclass

A single problem found in a value: its location, what is wrong, the kind of problem, and the data used in the message.

loc is the path from the root of the checked value to the bad value, e.g. ("codecs", 0, "name") in a document. An empty loc refers to the root. kind classifies the failure for programmatic dispatch; message is the human-readable description.

input and ctx hold the data in the message, in the same way that pydantic's ErrorDetails and zod's issues do. input is the JSON found at loc, e.g. 12 for a gzip level of 12. It is UNSET when nothing is there (zod does the same for a missing key; pydantic gives the object that lacks the key), or when the value there is not JSON, in which case the message shows it. input is the object the caller passed in, not a copy, as in pydantic: if the caller later changes its document, input shows the change. ctx holds what was expected, when that is more than a type:

• gt, ge, lt and le: the bounds from the value's type or from a rule, with the names pydantic uses. For example {"ge": 0, "le": 9} for a gzip level, whose type is Annotated[int, Interval(ge=0, le=9)].
• expected: the values of a closed set, like zod's values, in the order the message lists them. For example the values of a Literal, of node_type, or of zarr_format.

Neither field takes part in equality or the repr: two problems are equal when they have the same location, message and kind. Every function that returns or raises problems fills input from the value its caller passed, so a rule only needs to give the location.

Source code in src/zarr_metadata/_json.py

@dataclass(frozen=True, slots=True)
class ValidationProblem:
    """A single problem found in a value: its location, what is wrong, the kind of problem, and the data used in the message.

    `loc` is the path from the root of the checked value to the bad value,
    e.g. `("codecs", 0, "name")` in a document. An empty `loc` refers to
    the root.
    `kind` classifies the failure for programmatic dispatch; `message`
    is the human-readable description.

    `input` and `ctx` hold the data in the message, in the same way that
    pydantic's `ErrorDetails` and zod's issues do. `input` is the JSON
    found at `loc`, e.g. `12` for a gzip `level` of 12. It is `UNSET` when
    nothing is there (zod does the same for a missing key; pydantic gives
    the object that lacks the key), or when the value there is not JSON,
    in which case the message shows it. `input` is the object the caller
    passed in, not a copy, as in pydantic: if the caller later changes its
    document, `input` shows the change. `ctx` holds what was expected,
    when that is more than a type:

    - `gt`, `ge`, `lt` and `le`: the bounds from the value's type or from
      a rule, with the names pydantic uses. For example `{"ge": 0, "le": 9}`
      for a gzip `level`, whose type is `Annotated[int, Interval(ge=0, le=9)]`.
    - `expected`: the values of a closed set, like zod's `values`, in the
      order the message lists them. For example the values of a `Literal`,
      of `node_type`, or of `zarr_format`.

    Neither field takes part in equality or the repr: two problems are
    equal when they have the same location, message and kind. Every
    function that returns or raises problems fills `input` from the value
    its caller passed, so a rule only needs to give the location.
    """

    loc: tuple[str | int, ...]
    message: str
    kind: ProblemKind
    input: JSONValue | UNSET = dataclasses.field(
        default=UNSET, kw_only=True, compare=False, repr=False
    )
    ctx: Mapping[str, JSONValue] = dataclasses.field(
        default_factory=_no_ctx, kw_only=True, compare=False, repr=False
    )

    def __post_init__(self) -> None:
        # Enforce the annotations at runtime. A rule written without a type
        # checker (an extension's rule may be) then fails where it builds
        # the problem, instead of reporting a problem at an invalid location.
        loc = cast("object", self.loc)
        if not isinstance(loc, tuple) or not all(
            isinstance(part, str) or (isinstance(part, int) and not isinstance(part, bool))
            for part in cast("tuple[object, ...]", loc)
        ):
            msg = f"a ValidationProblem's loc is a tuple of keys and indices, got {loc!r}"
            raise TypeError(msg)
        message = cast("object", self.message)
        if not isinstance(message, str):
            msg = f"a ValidationProblem's message is a string, got {message!r}"
            raise TypeError(msg)
        kind = cast("object", self.kind)
        if not isinstance(kind, str) or kind not in PROBLEM_KINDS:
            msg = f"a ValidationProblem's kind is one of {PROBLEM_KINDS!r}, got {kind!r}"
            raise TypeError(msg)
        ctx = cast("object", self.ctx)
        if isinstance(ctx, _Ctx):
            # Another problem's `ctx`, already checked when that problem was
            # created. `replace` passes it to the copy.
            return
        if (
            not isinstance(ctx, Mapping)
            or not all(isinstance(key, str) for key in cast("Mapping[object, object]", ctx))
            or not is_json(dict(cast("Mapping[str, object]", ctx)))
        ):
            msg = f"a ValidationProblem's ctx is an object of JSON values, got {ctx!r}"
            raise TypeError(msg)
        # Store a view of a private deep copy, with arrays as tuples. Then
        # neither this view nor the caller's original mapping can modify a
        # raised error, which is a finished report.
        held = cast(
            "dict[str, JSONValue]",
            copied(cast("JSONValue", arrays_to_tuples(dict(cast("Mapping[str, object]", ctx))))),
        )
        object.__setattr__(self, "ctx", _Ctx(held))

    def __str__(self) -> str:
        location = ".".join(str(part) for part in self.loc) if self.loc else "<root>"
        return f"{location}: {self.message}"

    def __reduce__(
        self,
    ) -> tuple[Callable[..., ValidationProblem], tuple[object, ...]]:
        # Pickle and copy by calling the constructor again. The view that
        # holds `ctx` does not pickle, but the dict it wraps does.
        return (_problem, (self.loc, self.message, self.kind, self.input, dict(self.ctx)))

ctx class-attribute instance-attribute

ctx: Mapping[str, JSONValue] = dataclasses.field(
    default_factory=_no_ctx,
    kw_only=True,
    compare=False,
    repr=False,
)

input class-attribute instance-attribute

input: JSONValue | UNSET = dataclasses.field(
    default=UNSET, kw_only=True, compare=False, repr=False
)

kind instance-attribute

kind: ProblemKind

loc instance-attribute

loc: tuple[str | int, ...]

message instance-attribute

message: str

__init__

__init__(
    loc: tuple[str | int, ...],
    message: str,
    kind: ProblemKind,
    *,
    input: JSONValue | UNSET = UNSET,
    ctx: Mapping[str, JSONValue] = _no_ctx(),
) -> None

__post_init__

__post_init__() -> None

Source code in src/zarr_metadata/_json.py

def __post_init__(self) -> None:
    # Enforce the annotations at runtime. A rule written without a type
    # checker (an extension's rule may be) then fails where it builds
    # the problem, instead of reporting a problem at an invalid location.
    loc = cast("object", self.loc)
    if not isinstance(loc, tuple) or not all(
        isinstance(part, str) or (isinstance(part, int) and not isinstance(part, bool))
        for part in cast("tuple[object, ...]", loc)
    ):
        msg = f"a ValidationProblem's loc is a tuple of keys and indices, got {loc!r}"
        raise TypeError(msg)
    message = cast("object", self.message)
    if not isinstance(message, str):
        msg = f"a ValidationProblem's message is a string, got {message!r}"
        raise TypeError(msg)
    kind = cast("object", self.kind)
    if not isinstance(kind, str) or kind not in PROBLEM_KINDS:
        msg = f"a ValidationProblem's kind is one of {PROBLEM_KINDS!r}, got {kind!r}"
        raise TypeError(msg)
    ctx = cast("object", self.ctx)
    if isinstance(ctx, _Ctx):
        # Another problem's `ctx`, already checked when that problem was
        # created. `replace` passes it to the copy.
        return
    if (
        not isinstance(ctx, Mapping)
        or not all(isinstance(key, str) for key in cast("Mapping[object, object]", ctx))
        or not is_json(dict(cast("Mapping[str, object]", ctx)))
    ):
        msg = f"a ValidationProblem's ctx is an object of JSON values, got {ctx!r}"
        raise TypeError(msg)
    # Store a view of a private deep copy, with arrays as tuples. Then
    # neither this view nor the caller's original mapping can modify a
    # raised error, which is a finished report.
    held = cast(
        "dict[str, JSONValue]",
        copied(cast("JSONValue", arrays_to_tuples(dict(cast("Mapping[str, object]", ctx))))),
    )
    object.__setattr__(self, "ctx", _Ctx(held))

__reduce__

__reduce__() -> tuple[
    Callable[..., ValidationProblem], tuple[object, ...]
]

Source code in src/zarr_metadata/_json.py

def __reduce__(
    self,
) -> tuple[Callable[..., ValidationProblem], tuple[object, ...]]:
    # Pickle and copy by calling the constructor again. The view that
    # holds `ctx` does not pickle, but the dict it wraps does.
    return (_problem, (self.loc, self.message, self.kind, self.input, dict(self.ctx)))

__str__

__str__() -> str

Source code in src/zarr_metadata/_json.py

def __str__(self) -> str:
    location = ".".join(str(part) for part in self.loc) if self.loc else "<root>"
    return f"{location}: {self.message}"

shown

shown(value: object) -> str

value as a problem's message shows it: as JSON (null, [1, 2]), or as its repr when it is not JSON. A value the interpreter will not write (an integer with too many digits, or a value nested too deep) is described instead.

Source code in src/zarr_metadata/_json.py

def shown(value: object) -> str:
    """`value` as a problem's message shows it: as JSON (`null`, `[1, 2]`), or as its repr when it is not JSON. A value the interpreter will not write (an integer with too many digits, or a value nested too deep) is described instead."""
    if isinstance(value, int) and not isinstance(value, bool) and not _writable(value):
        return f"an integer of {value.bit_length()} bits"
    refined, problems = _refine(value, (), finite=False)
    if any(problem.message == _PAST_THE_LEVELS for problem in problems):
        # Nested deeper than `JSON_DEPTH`. Say so instead of using the repr,
        # which overflows at a depth set by the interpreter and platform.
        return "a value nested too deep to show"
    if len(problems) != 0:
        # Not JSON.
        return shown_by_python(value)
    try:
        return json.dumps(refined, ensure_ascii=False)
    except ValueError:
        # An integer with more digits than the interpreter converts to text.
        # If it is the value itself, show its size. If a container holds it,
        # Python cannot write the container either.
        if isinstance(value, int):
            return f"an integer of {value.bit_length()} bits"
        return shown_by_python(value)