From 9dea6b37cf5e2a51e673f899e5e2803a828738c1 Mon Sep 17 00:00:00 2001 From: Till Varoquaux Date: Sun, 5 Jul 2026 22:58:23 -0400 Subject: [PATCH 1/2] PEP 835: Address Discourse feedback and align better with typing spec Key changes: - Structure & Formatting: Reordered all sections to strictly comply with the PEP 1 template. Added a mandatory "Security Implications" section. - Terminology & Spec Alignment: Consolidated terminology to provide clear definitions for the concepts used throughout the PEP (Type Expression, Type Metadata, Non-Typing Annotation, Symbol Decorator) and align better with the typing spec. - Forward References: Clarified that `Format.TYPE` supports `None @Metadata` structurally, eliminating the need to modify `NoneType.__matmul__`. Consequently, removed the "Supporting None" open issue. - Rejected Ideas: Expanded the section to include "Alternative Syntaxes" (e.g. `<@`, soft keywords, brackets), noting their rejection due to lack of consensus. Moved the `__rmatmul__` rejection out of the Specification section. - Future Work: Added extensive examples of how the syntax might map to future Symbol Decorators, and added a JSR 308-inspired proposal for PEP 746 target-based constraints using intersection types (`&`). --- peps/pep-0835.rst | 869 +++++++++++++++++++++++++++++++--------------- 1 file changed, 580 insertions(+), 289 deletions(-) diff --git a/peps/pep-0835.rst b/peps/pep-0835.rst index 68cfa2b299d..64574ae08fc 100644 --- a/peps/pep-0835.rst +++ b/peps/pep-0835.rst @@ -14,187 +14,221 @@ Post-History: `19-Apr-2026 `__ using the +``@`` operator. This change improves the ergonomics of type constraints and +reduces signature verbosity, benefiting type-directed libraries like +Pydantic, FastAPI, Typer, and SQLModel. Motivation ========== -Since its introduction in :pep:`593`, ``Annotated`` has become the standard -mechanism for attaching context-specific metadata to types. It is widely -embraced by libraries such as Pydantic, FastAPI, and SQLModel. -Historically, these libraries embedded metadata within default values, a pattern -that was concise but problematic for type checkers and runtime defaults: +Metadata as a Semantic Operation +-------------------------------- + +Historically, Python has treated type metadata as an afterthought. +``Annotated[T, M]`` models metadata as a generic container, similar to how +``Union[X, Y]`` once modeled unions. But metadata is not a container; it is a +modifier applied to a type. + +Just as :pep:`604` evolved unions from the ``Union`` container to the ``|`` +operator, the ``@`` syntax evolves metadata into a native language operator. +Elevating metadata to first-class syntax recognizes that modern libraries +increasingly use constraints on types (e.g., bounds, shapes, patterns). + +Previously, applying these constraints forced a frustrating compromise between +type safety and readability. Before :pep:`593` (``Annotated``), frameworks like +Pydantic and FastAPI embedded metadata directly into default values. This was +concise but broke static analysis: .. code-block:: :class: bad - # The older, now discouraged pattern class User(BaseModel): id: int = Field(gt=0) name: str = Field(min_length=3) -The transition to ``Annotated`` cleanly separated types from metadata but -introduced visual noise and cognitive overhead. Library authors often surface -"special" types, like ``PositiveInt`` or ``EmailStr``, to hide this verbosity. -These aliases are discoverable but inherently limited. Users needing unique -constraint combinations must fall back to the full ``Annotated`` syntax:: +Moving to ``Annotated`` fixed the typing semantics but introduced significant +complexity. To hide the visual noise, libraries introduced alias types (e.g., +``PositiveInt``). However, the moment users step outside pre-packaged +constraints, they are forced back into the complex syntax:: from typing import Annotated from pydantic import BaseModel, Field, PositiveInt class User(BaseModel): - # Concise but limited - age: PositiveInt + age: PositiveInt # Concise but limited + id: Annotated[int, Field(gt=0, le=1000)] # Verbose fallback - # Verbose fallback required for specific constraints - id: Annotated[int, Field(gt=0, le=1000)] - name: Annotated[str, Field(min_length=3, max_length=50)] = "Anonymous" +The ``@`` shorthand resolves the ergonomic tradeoff, restoring the conciseness +of early frameworks while preserving the strict semantics of ``Annotated``: -This creates a jarring experience. ``Annotated`` is core to the ecosystem but -remains "hidden" and difficult to use directly. The proposed shorthand bridges -this ergonomic gap. It restores the conciseness of earlier patterns while -adhering to the modern ``Annotated`` standard. By reducing overhead, this -proposal encourages developers to leverage the full power of type metadata: - -.. code-block:: python +.. code-block:: :class: good from pydantic import BaseModel, Field from fastapi import Query class User(BaseModel): - id: int @ Field(gt=0, le=1000) - name: str @ Field(min_length=3, max_length=50) = "Anonymous" - age: int @ Field(gt=0) - email: str @ Field(pattern=r".*@.*") + id: int @Field(gt=0, le=1000) + name: str @Field(min_length=3, max_length=50) = "Anonymous" + age: int @Field(gt=0) + email: str @Field(pattern=r".*@.*") - async def read_items(q: (str | None) @ Query(max_length=50) = None): + async def read_items(q: (str | None) @Query(max_length=50) = None): ... -Rationale -========= - -Developer Ergonomics and Ecosystem Alignment ---------------------------------------------- - -Pydantic and FastAPI now recommend ``Annotated`` over embedding metadata in -default values. However, the resulting code is significantly more verbose. -This proposal restores the earlier pattern's conciseness while adhering to the -modern ``Annotated`` standard. - -Sebastián Ramírez (author of FastAPI, Typer, and SQLModel) noted that a shorter -syntax without extra imports would benefit users. By making the "correct" way -the most ergonomic, we reduce the incentive for discouraged patterns. - -This ergonomic barrier was notably evident in the withdrawal of :pep:`727` -(Documentation Metadata). The extreme verbosity of the syntax in function -signatures was a primary factor in its community pushback. A native shorthand -makes such metadata-heavy standards significantly more viable. - -Conceptual Consistency and Precedent -------------------------------------- - -The ``@`` operator signifies "decoration" or "attachment of metadata" for -functions and classes. Extending this to type expressions leverages that -mental model: just as a decorator attaches behavior to a function, the ``@`` -operator attaches metadata to a type. - -This follows the precedent set by :pep:`604` (``|`` for ``Union``) and -:pep:`585` (generics in built-ins). These PEPs moved common typing constructs -into native operators, making the type system feel like a first-class part of -the language. - -This syntax also draws inspiration from other languages with strong metadata -ecosystems, notably Java. In Java (formalized in `JSR 308 `__) -and other JVM languages, the ``@`` symbol is standard for type annotations: - -.. code-block:: java - - public class Person { - @Column(length = 32) - private String name; - } - -While the exact syntax differs (Python's ``@`` operates inline on the type -expression rather than decorating the declaration), the visual association -between the ``@`` symbol and type-level metadata will be familiar to many -developers. - -Implementation and Performance -------------------------------- - -Making the syntax built-in eases runtime metadata use by removing ``typing`` -module import overhead. This aligns with the trend toward accessible runtime -type introspection. - -The proposed syntax is straightforward to implement. Prototypes for Mypy, -Pyright, and Ruff are compact. Since ``@`` is already a valid expression -operator, these tools do not require parser changes. They handle the new syntax -during semantic analysis. Ruff has already prototyped a ``pyupgrade`` rule -for automated conversion. This enables large codebases to -migrate to the new syntax with minimal manual effort. +Prior Art and Historical Context +-------------------------------- + +The verbosity of ``Annotated`` is not just an aesthetic annoyance; it hinders +adoption. During the 2023 review of :pep:`727`, reviewers warned that forcing +developers to use ``Annotated`` for everyday documentation would introduce +excessive visual noise [1]_. This pushback led to the PEP's withdrawal and +sparked the initial discussions around using ``@`` as a native alternative. + +Today, this friction remains evident across the entire ecosystem. The most +prominent type-directed frameworks in Python (FastAPI, Pydantic, SQLAlchemy, +cattrs, msgspec, Typer, and beartype) all heavily rely on ``Annotated`` [2]_ +[3]_ [4]_ [5]_ [6]_ [7]_ [8]_. Yet, they frequently receive user complaints +about signature bloat [9]_ [10]_ [11]_. In some domains, library authors refuse +to use it entirely: PyTorch [12]_, TorchTyping, and Beartype [13]_ rejected +``Annotated`` for tensor shape typing, deciding instead to wait for a native +language solution. + +When the community debated alternatives in late 2023 [14]_ and 2025 [15]_, +discussion trended toward the ``@`` operator. It visually aligns with existing +Python `decorators `__. +Adopting ``@`` for metadata follows the precedent of repurposing runtime +operators for static typing, as seen with ``[]`` for generics (:pep:`585`) and +``|`` for unions (:pep:`604`) [16]_. + +Precedent in Other Languages +---------------------------- -CPython prototype testing confirms that libraries like ``typer`` and -``pydantic`` work out of the box. +The ``@`` metadata syntax mirrors features in other statically typed languages: + +- **C++:** Uses ``[[attribute]]`` for both symbols and types (e.g., + ``[[nodiscard]] int f();``). +- **C#:** Uses ``[Attribute]`` for reflection-based metadata (e.g., + ``[Required] public string Name;``). +- **Java / Kotlin:** Uses ``@Annotation``. Java's `JSR 308 + `__ introduced ``TYPE_USE`` annotations + to decorate types (e.g., ``List<@NonNull String>``). Kotlin uses ``val x: + @NotNull String``. +- **OCaml:** Uses ``[@attribute]`` postfix syntax to attach metadata to the + preceding AST node (e.g., ``type t = int [@default 0]``). + +Terminology +=========== + +To clarify discussion around annotations and metadata, the following terms +apply: + +- **Type Expression**: e.g., ``x: ``. Expressions evaluating to valid + static types, as specified in the `Typing Specification + `__ + (:pep:`484`). +- **Type Metadata**: e.g., ``int @``. Data attached to a Type Expression + via ``Annotated`` (:pep:`593`, :pep:`746`). +- **Non-Typing Annotation**: e.g., `@no_type_check + `__ + wrapping ``x: ``. + Standard behavior where annotations are evaluated as arbitrary runtime values + rather than strict type hints. +- **Symbol Decorator**: Applying metadata directly to a variable or field + declaration, rather than to its underlying type. While Python does not + currently have a native syntax for variable decorators, this conceptual + distinction is well-established in languages like Java: + + .. code-block:: java + + class Application { + // Field Decorator (Symbol Decorator) + @Inject + private Service s; + + // Type Decorator (Type Metadata) + private @NonNull String name; + } + +- **Symbol Metadata** (or **Field Metadata**): Data attached to a symbol via a + Symbol Decorator. This contrasts with an active runtime descriptor. It is + fundamentally distinct from Type Metadata, as it applies to the specific + instance of the variable rather than the type itself. Specification ============= -The proposed syntax uses the ``@`` (matrix multiplication) operator to attach -metadata to a type:: +The proposed syntax uses the ``@`` (``__matmul__``) operator to attach metadata +to a type:: # Current syntax x: Annotated[int, Range(0, 10)] # Proposed shorthand - x: int @ Range(0, 10) + x: int @Range(0, 10) Operator Precedence ------------------- -The ``@`` operator has higher precedence than the ``|`` operator (bitwise OR, -used for Unions in :pep:`604`). Parentheses are required when attaching -metadata to a Union type: +The ``@`` operator binds tighter than the ``|`` union operator (:pep:`604`). +This aligns with standard Python expression precedence, ensuring predictable +parsing for Type Metadata. + +Because ``@`` binds tightly, developers can attach distinct constraints directly +to specific types within a union. For example, a US zip code might be a 5-digit +integer *or* a 5-character string:: + + zip_code: int @Ge(10000) @Le(99999) | str @Len(5) -- ``int | str @ Metadata`` is equivalent to ``int | Annotated[str, Metadata]`` -- ``(int | str) @ Metadata`` is equivalent to ``Annotated[int | str, Metadata]`` +If you intend to attach metadata to the entire union, you must use parentheses:: -This matches the standard precedence of ``@`` and ``|`` in Python expressions. -The most common union pattern, ``Optional``, works naturally: + # Attaches only to 'str' + int | str @Metadata # equivalent to: int | Annotated[str, Metadata] -- ``int @ Field(gt=0) | None`` is equivalent to - ``Annotated[int, Field(gt=0)] | None`` + # Attaches to the entire union + (int | str) @Metadata # equivalent to: Annotated[int | str, Metadata] -Flattening and Associativity +The "Parentheses Friction" +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Some developers have noted that writing ``(str | None) @Field(...)`` feels +cumbersome for optional fields. + +This awkwardness highlights a missing language feature. Without native Symbol +Decorators, frameworks must co-opt ``Annotated`` to configure fields. When a +developer writes ``address: (str | None) @Field(...)``, they are actually +attempting to configure the ``address`` field, not the ``str | None`` +type. Until native field decorators exist, parentheses remain the structurally +accurate way to express this in the type system. + +Flattening Multiple Metadata ---------------------------- -The ``@`` operator is left-associative. When multiple metadata items are -chained, the resulting ``Annotated`` object is flattened. +When multiple metadata items are chained, the resulting ``Annotated`` object is +flattened. -Specifically, ``T @ m1 @ m2`` is strictly equivalent to -``Annotated[T, m1, m2]``. It must not resolve to a nested structure such as -``Annotated[Annotated[T, m1], m2]``. This mirrors the existing runtime -behavior of ``typing.Annotated``. +Specifically, ``T @m1 @m2`` evaluates to ``Annotated[T, m1, m2]``. It must not +resolve to a nested structure such as ``Annotated[Annotated[T, m1], m2]``. This +mirrors the existing runtime behavior of ``typing.Annotated``. This flattening also applies when the left-hand operand is an existing ``Annotated`` type, regardless of how it was constructed:: - Annotated[int, m1] @ m2 # AnnotatedType(int, m1, m2) — flattened + Annotated[int, m1] @m2 # AnnotatedType(int, m1, m2) (flattened) Runtime Behavior ---------------- The ``@`` operator produces a ``types.AnnotatedType`` instance, a new built-in type implemented in C. The existing ``typing.Annotated`` is unified with this -type: ``typing.Annotated[X, Y]`` returns the same ``types.AnnotatedType`` -object as ``X @ Y``: +type: ``typing.Annotated[X, Y]`` returns the same ``types.AnnotatedType`` object +as ``X @Y``: .. code-block:: pycon - >>> type(int @ Field()) is type(Annotated[int, Field()]) + >>> type(int @Field()) is type(Annotated[int, Field()]) True >>> typing.Annotated is types.AnnotatedType True @@ -205,215 +239,228 @@ An ``AnnotatedType`` object exposes the following attributes: - ``__metadata__``: A tuple of metadata items. - ``__args__``: The tuple ``(origin, *metadata)``, for compatibility with ``typing.get_args()``. -- ``__parameters__``: Lazily computed type variables contained in the type. +- ``__parameters__``: A tuple of unique free type parameters of the type. The ``repr()`` of an ``AnnotatedType`` uses the shorthand syntax: .. code-block:: pycon - >>> int @ Field(gt=0) - int @ Field(gt=0) + >>> int @Field(gt=0) + int @Field(gt=0) ``AnnotatedType`` objects support pickling via ``copyreg``, reconstructing through ``AnnotatedType[origin, *metadata]``. -``None`` on the left-hand side is accepted and uses ``None`` as the -origin: +Handling of ``None`` +-------------------- + +Implementing ``__matmul__`` on ``NoneType`` is explicitly avoided to prevent +masking runtime bugs in non-typing contexts. -.. code-block:: pycon +For example, a developer might forget to validate ``None`` before executing +matrix multiplication on an array: + +.. code-block:: + :class: bad + + def matmul_arrays(a: np.array | None, b: np.array): + return a @ b # oops, forgot to check for None + +If ``__matmul__`` were implemented on ``NoneType``, this would return an +``AnnotatedType`` object instead of raising a ``TypeError``, silently masking +the bug. - >>> None @ Field() - None @ Field() +Because ``Format.TYPE`` is introduced to ``annotationlib``, this limitation is +invisible in valid typing contexts. ``Format.TYPE`` evaluates +structurally and correctly parses ``None @Metadata`` into an ``AnnotatedType`` +without relying on ``NoneType.__matmul__``. Outside of Type Expressions +(e.g., in raw runtime execution), users must use ``Annotated[None, Metadata]``. Supported Left-Hand Operands ----------------------------- The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the -metatype (``type``) and to several typing-related types. The operator is -supported for any left-hand operand that currently supports the ``|`` -operator for making a union. +metatype (``type``) and to exactly the same typing constructs that currently +support the ``|`` operator for unions (``types.GenericAlias``, +``types.UnionType``, ``types.AnnotatedType``, ``typing.TypeVar``, +``typing.ParamSpec``, ``typing.TypeVarTuple``, ``typing.TypeAliasType``, +``typing.ForwardRef``, and ``sentinel`` objects). For all other left-hand operands, the operator returns ``NotImplemented``, allowing normal ``__matmul__`` dispatch to proceed. -Parsing and Grammar -=================== - -This proposal requires no changes to the Python grammar. Because ``@`` is -already a valid operator, it is natively parsed as a binary operation. The -shorthand is resolved during semantic analysis, entirely bypassing the need -to patch grammar files or update the parser. - -How to Teach This -================= - -In Python, the ``@`` symbol already has an established association with -metadata through decorators. The annotation shorthand extends this -intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``, -decorated with ``Field(gt=0)``." +This means ``int @Field()`` produces an ``AnnotatedType``, while ``42 +@something`` is unaffected. Crucially, ``ndarray @Field()`` (using the **class** +as a type annotation) also produces an ``AnnotatedType``, even though ndarray +*instances* define ``__matmul__`` for matrix multiplication. One consequence of +this design is that applying the ``@`` operator to a class object evaluates as +Type Metadata, while applying it to an instance performs arithmetic. -For beginners, the key rule is simple: **in a type annotation, ``@`` means -"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax -remains available and is entirely equivalent for those who find it clearer. - -For experienced developers, the precedence rules follow standard Python -operator precedence (``@`` binds tighter than ``|``), and chaining -``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does. +The only case where the shorthand does not apply is when a class has a +**metaclass** that defines ``__matmul__``. In that case, the metaclass's +operator takes priority via standard Python MRO dispatch. -Documentation and teaching materials should introduce the shorthand alongside -``Annotated``, not as a replacement. The longhand form is still preferred in -contexts where multiple metadata items are passed as a group and chaining -would be unwieldy. -Backwards Compatibility -======================= Forward References and Deferred Evaluation ------------------------------------------- -Under :pep:`749`, annotations are lazily evaluated. The ``annotationlib`` -module provides several formats for retrieving annotations: - -- ``Format.VALUE``: Fully evaluates the annotation. Raises ``NameError`` - if any name is unresolvable. -- ``Format.FORWARDREF``: Wraps unresolvable names in ``ForwardRef`` objects. - However, compound expressions using operators (``@``, ``|``) produce an - opaque ``ForwardRef`` string containing the entire expression. -- ``Format.STRING``: Returns the raw source text with no evaluation. - -For the ``@`` operator, ``Format.FORWARDREF`` is insufficient. Consider:: +Under :pep:`749`'s lazy evaluation, existing ``annotationlib`` formats are +insufficient for ``@``. Specifically, ``Format.FORWARDREF`` fails structurally +on unresolvable names:: class Model: - ref: "NotYetDefined" @ Field(gt=0) + ref: NotYetDefined @Field(gt=0) -Under ``Format.FORWARDREF``, this produces -``ForwardRef('"NotYetDefined" @ Field(gt=0)')``. The metadata ``Field(gt=0)`` -is trapped inside the unresolved string and cannot be inspected until the -forward reference is resolved. This is a blocking issue for libraries like -Pydantic and FastAPI, which inspect metadata at class-definition time. +``Format.FORWARDREF`` produces an opaque ``ForwardRef('"NotYetDefined" +@Field(gt=0)')``. The metadata is trapped inside the unresolved string, blocking +libraries like Pydantic from inspecting it at class-definition time. -This proposal introduces a new format, ``Format.FORWARDREF_STRUCTURAL``. -This format assumes typing semantics and evaluates compound type expressions -**structurally**. It always returns an ``AnnotatedType`` for ``@``, a union -for ``|``, and a ``GenericAlias`` for subscripting. When a name cannot be -resolved, only that name is wrapped in ``ForwardRef``; the surrounding -operators are still evaluated. The example above produces:: +``Format.TYPE`` assumes typing semantics and evaluates Type Expressions +structurally. Unresolvable names are wrapped in a ``ForwardRef`` independently, +leaving operators intact:: AnnotatedType(ForwardRef('NotYetDefined'), Field(gt=0)) -The metadata is immediately accessible. This format also resolves the -pre-existing issue with ``|`` unions, where ``"Foo" | int`` under -``Format.FORWARDREF`` produces ``ForwardRef('Foo | int')`` instead of the -structural ``ForwardRef('Foo') | int``. +The metadata remains immediately accessible. This also resolves the pre-existing +issue where ``"Foo" | int`` incorrectly produced ``ForwardRef('Foo | int')`` +under ``Format.FORWARDREF``. -Interaction with PEP 563 -^^^^^^^^^^^^^^^^^^^^^^^^^ +Importantly, because ``Format.TYPE`` can evaluate structurally when runtime +dunder methods are missing (falling back to ``types.UnionType`` and +``types.AnnotatedType`` for the ``|`` and ``@`` operators), it parses ``None +@Metadata`` without requiring ``NoneType.__matmul__``. -Under :pep:`563` (``from __future__ import annotations``), all annotations -are stored as source-code strings and evaluated via ``eval()`` on access. The -``@`` shorthand works correctly in this context: ``eval("int @ Field(gt=0)")`` -triggers the metatype's ``nb_matrix_multiply`` and produces an -``AnnotatedType``. +Parsing and Grammar +------------------- -However, ``FORWARDREF_STRUCTURAL`` reconstruction from PEP 563 strings is -coarser than from :pep:`749` thunks. When a name is unresolvable, the -``ForwardRef`` may wrap a call expression (e.g., ``ForwardRef('Field(gt=0)')``) -rather than just a name. :pep:`749` provides a strictly better experience and -is the recommended path forward. +No changes to the Python grammar are required. Because ``@`` is already a valid +operator, it is natively parsed as a binary operation. The shorthand is resolved +during semantic analysis, entirely bypassing the need to patch grammar files or +update the parser. -Operator Overloading --------------------- +Rationale +========= -The ``@`` operator is currently used for matrix multiplication -(``__matmul__``). The shorthand is implemented by adding -``nb_matrix_multiply`` to the metatype (``type``), so it applies when a -**type object** (class) appears on the left-hand side — not when an instance -does. +Baking ``Annotated`` into native syntax removes ``typing`` import overhead, +directly aiding runtime type introspection. -This means ``int @ Field()`` produces an ``AnnotatedType``, while -``42 @ something`` is unaffected and follows normal ``__matmul__`` dispatch. -Crucially, ``ndarray @ Field()`` (using the **class** as a type annotation) -also produces an ``AnnotatedType``, even though ndarray *instances* define -``__matmul__`` for matrix multiplication. This is the desired behavior: applying the -``@`` operator to a class object evaluates as type metadata; applying it to -an instance performs arithmetic. +Because ``@`` is already a valid expression operator, type checkers (Mypy, +Pyright) require no parser changes, handling the syntax entirely during semantic +analysis. We have prototyped an automated conversion rule in Ruff, enabling +automated codebase migrations, and CPython prototype testing confirms that +libraries like ``typer`` and ``pydantic`` work natively out of the box. -The only case where the shorthand does not apply is when a class has a -**metaclass** that defines ``__matmul__``. In that case, the metaclass's -operator takes priority via standard Python MRO dispatch. This is an obscure -edge case unlikely to arise in practice. +Why the @ Operator? +------------------- + +Selecting the correct operator for metadata involves balancing three grammatical +considerations: + +1. **Precedence & Spec Restrictions:** The typing specification currently only + allows the ``|`` operator in Type Expressions. Any new operator must bind + tighter than ``|`` (Union) and ``&`` (proposed Intersection) so that + expressions like ``int @Field() | str`` parse correctly without parentheses. + This eliminates operators like ``|``, ``^``, and ``&``. +2. **Backwards Compatibility & Consistency:** Using an existing operator is + preferable to introducing a new keyword (e.g., ``annotated``). It minimizes + parser complexity and mitigates the risk of breaking existing code. It also + aligns with the Python typing specification's precedent of preferring + operators (e.g., ``|``) over keywords within type expressions. +3. **Semantic Clarity:** The chosen syntax should avoid colliding with + established mathematical intuitions for primitive types. + +This leaves the set of overridable binary operators that bind tighter than +``&``: ``**``, ``*``, ``@``, ``/``, ``//``, ``%``, ``+``, ``-``, ``>>``, and +``<<``. + +Standard arithmetic operators like ``+``, ``-``, ``/``, ``//``, ``*``, ``**``, +and ``%`` are misleading in this context. Reading ``int + x`` or ``float / +Field()`` strongly implies mathematical evaluation, not metadata decoration. + +This leaves ``<<``, ``>>``, and ``@``. Of these, ``@`` is the only operator that +possesses an existing association with metadata in Python (via function and +class decorators). Bitwise shift operators (``<<``, ``>>``) lack this +association. + +While ``@`` is used for matrix multiplication in numeric libraries (like NumPy), +it is far less associated with core scalar arithmetic than operators like ``+`` +or ``/``. + +Language Complexity +------------------- + +Adding new syntax always introduces some language complexity. However, while +syntactic complexity increases, the *cognitive* load actually decreases. + +Teaching a beginner ``int @Field(gt=0)`` leverages their existing intuition of +Python decorators. It visually separates the base type from its metadata. +Teaching ``typing.Annotated`` requires referencing obscure parts of the typing +module. By moving metadata from a standard library class to a native grammatical +construct, we reduce the overall cognitive load required to read and write +modern typed Python. + +Backwards Compatibility +======================= typing.Annotated Migration --------------------------- -This proposal replaces the pure-Python ``typing._AnnotatedAlias`` class with -a native C implementation (``types.AnnotatedType``). ``typing.Annotated`` -becomes a reference to this C type rather than a special form with a custom -metaclass. +The pure-Python ``typing._AnnotatedAlias`` class is replaced with a native C +implementation (``types.AnnotatedType``). ``typing.Annotated`` becomes a +reference to this C type rather than a special form with a custom metaclass. The private ``typing._AnnotatedAlias`` class is retained as a deprecated -compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)`` -will continue to work but emit a ``DeprecationWarning``. The shim is -scheduled for removal in Python 3.18. +compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)`` will +continue to work but emit a ``DeprecationWarning``. The shim is scheduled for +removal in Python 3.21 (see `Deprecation Timeline`_). Code that should be updated: -- ``type(ann).__name__ == '_AnnotatedAlias'`` → use - ``isinstance(ann, types.AnnotatedType)`` or - ``typing.get_origin(ann) is Annotated`` -- ``typing._AnnotatedAlias(origin, metadata)`` → use - ``Annotated[origin, *metadata]`` or ``origin @ m1 @ m2`` +- ``type(ann).__name__ == '_AnnotatedAlias'`` → use ``isinstance(ann, + types.AnnotatedType)`` or ``typing.get_origin(ann) is Annotated`` +- ``typing._AnnotatedAlias(origin, metadata)`` → use ``Annotated[origin, + *metadata]`` or ``origin @m1 @m2`` Backporting via typing_extensions ---------------------------------- -Unlike ``X | Y`` (which could be backported by ``typing_extensions`` using -``__or__``), the ``@`` shorthand requires changes to the metatype -(``type.__matmul__``), which cannot be patched from pure Python. The -shorthand is therefore only available on Python 3.16+. The existing -``Annotated[X, Y]`` syntax continues to work on all supported versions and -should be used when backwards compatibility is required. +Like ``X | Y``, the ``@`` shorthand requires changes to the metatype +(``type.__matmul__``), which cannot be patched from pure Python. The shorthand +is only available on Python 3.16+. The existing ``Annotated[X, Y]`` syntax +continues to work on all supported versions and should be used when backwards +compatibility is required. -Rejected Ideas -============== - -Mandatory List Variant ----------------------- +Security Implications +===================== -The syntax ``Type @ [ann1, ann2]`` was considered to group metadata and avoid -chaining ambiguities. While clearer in some contexts, it was deprioritized in -favor of the cleaner ``Type @ ann1 @ ann2``. +There are no direct security implications. -List-based syntax ------------------ - -An alternative syntax using list literals, such as ``[int, Metadata]``, was -rejected due to runtime semantics. In Python, a list literal evaluates to a -mutable ``list`` instance. Allowing lists as type annotations would break the -assumption of runtime checkers (like Pydantic) that annotations evaluate to -valid type constructs or ``GenericAlias`` objects, not arbitrary data -structures. +How to Teach This +================= -Scientific Computing Conflict ------------------------------ +In Python, the ``@`` symbol already has an established association with metadata +through decorators. The annotation shorthand extends this intuition to the type +system: ``int @Field(gt=0)`` reads as "``int``, decorated with ``Field(gt=0)``." -Critics note that ``ndarray @ Metadata`` visually resembles matrix -multiplication on a type whose instances are heavily associated with that -operation. However, the ``@`` -operator distinguishes between **type objects** and **instances**: ``ndarray`` -(the class) appearing in a type annotation is a type object, and ``@`` -produces an ``AnnotatedType``. An ``ndarray`` instance appearing in an -expression still uses NumPy's ``__matmul__`` for matrix multiplication. +For beginners, the key rule is: **in a type annotation, ``@`` means "with this +metadata."** For experienced developers, the mental model maps directly to +standard Python operator precedence (``@`` binds tighter than ``|``). -Since type annotations and arithmetic expressions occupy distinct syntactic -positions, this is a visual concern rather than a runtime conflict. +Documentation and teaching materials should introduce the shorthand as the +primary syntax for applying metadata. The verbose ``typing.Annotated`` form +should be treated as an advanced detail, primarily relevant to library authors +or when dynamically generating types. -Divergence from Type Theory ---------------------------- +Visual Style +------------ -Unlike ``Union`` or ``Generics``, using an operator for metadata is a -Python-specific ergonomic choice rather than a standard type-theoretic -construct. This follows the pragmatic precedent of :pep:`604`. +The shorthand must be strictly formatted as ``type @annot`` (e.g., ``int +@Metadata(...)``), with a space before the ``@`` and no space after it. This +distinguishes it from standard matrix multiplication (``A @ B``) and aligns +visually with function decorators (``@decorator``) and Java annotations. Code +formatters (like Ruff and Black) should enforce this spacing within typing +contexts. Usage Examples ============== @@ -421,15 +468,15 @@ Usage Examples Pydantic Validation ------------------- -The shorthand excels in data validation scenarios:: +The shorthand can be used in data validation scenarios:: from pydantic import BaseModel, Field, HttpUrl from annotated_types import Len class Project(BaseModel): - name: str @ Field(title="Project Name") @ Len(1) - url: HttpUrl @ Field(description="The project homepage") - stars: int @ Field(ge=0) = 0 + name: str @Field(title="Project Name") @Len(1) + url: HttpUrl @Field(description="The project homepage") + stars: int @Field(ge=0) = 0 FastAPI Dependency Injection ---------------------------- @@ -441,7 +488,7 @@ In FastAPI, the shorthand simplifies complex parameter definitions:: app = FastAPI() @app.get("/secure") - async def secure_endpoint(token: str @ Header(description="Authentication token")): + async def secure_endpoint(token: str @Header(description="Auth token")): return {"status": "authorized"} SQLModel and Database Definitions @@ -453,36 +500,280 @@ shorthand syntax makes these definitions significantly cleaner:: from sqlmodel import SQLModel, Field class Hero(SQLModel, table=True): - id: (int | None) @ Field(primary_key=True) = None - name: str @ Field(index=True) + id: (int | None) @Field(primary_key=True) = None + name: str @Field(index=True) secret_name: str - age: (int | None) @ Field(index=True) = None + age: (int | None) @Field(index=True) = None + +Testing and Formal Verification +------------------------------- + +Libraries like Hypothesis (property-based testing) and CrossHair (symbolic +execution) utilize ``annotated-types`` to constrain test generation and +analysis. The shorthand provides a clean syntax for specifying test boundaries:: + + from dataclasses import dataclass + from annotated_types import Ge, Interval + from hypothesis import given + + @dataclass + class InventoryItem: + # A non-negative quantity + quantity: int @Ge(0) + # A price bounded between 1 and 100 + price: float @Interval(gt=0, le=100) + + @given(...) + def test_inventory(item: InventoryItem): + assert item.price * item.quantity >= 0 Reference Implementation ======================== Prototype implementations are available for the following tools: -- **CPython:** `CPython at-type-annot `_ -- **CPython (with annotation-lib structural forward references):** `CPython forward-stringifier `_ -- **Mypy:** `Mypy at-type-annot `_ -- **Mypyc/ast_serialize:** `ast_serialize at-type-annot `_ -- **Pyright:** `Pyright at-type-annot `_ -- **Ruff:** `Ruff at-type-annot `_ +- **CPython:** `CPython at-type-annot + `_ +- **CPython (with annotation-lib structural forward references):** `CPython + forward-stringifier + `_ +- **Mypy:** `Mypy at-type-annot + `_ +- **Mypyc/ast_serialize:** `ast_serialize at-type-annot + `_ +- **Pyright:** `Pyright at-type-annot + `_ +- **Ruff:** `Ruff at-type-annot + `_ + +``builtins.type`` in Typeshed will be updated to include ``def +__matmul__(self, other: Any) -> types.AnnotatedType: ...`` to support type +checkers. + +Rejected Ideas +============== + +Alternative Syntaxes +-------------------- + +Debates around reusing ``@`` yielded several alternatives: + +- A new infix operator: ``int <@ Field(...)`` +- A new soft keyword: ``int annotated Interval(1, 10)`` +- Bracket syntax: ``x: int {Gt(10), Lt(20)}`` + +These were rejected because none garnered wide consensus. The ``@`` symbol also +extends cleanly to Symbol Decorators should the language pursue that route in +the future. + +Reliance on ``__rmatmul__`` +--------------------------- + +We explicitly reject relying on ``__rmatmul__`` on the metadata objects. One +might envision a base class that implements ``__rmatmul__`` to return an +``Annotated`` type:: + + class Metadata[T = object]: + def __rmatmul__(self, typ: TypeForm[T], /) -> TypeForm[T]: + return Annotated[typ, self] + + class Le(Metadata[int]): + ... + +This fails for two reasons. First, Type Expressions require fixed semantics; +relying on arbitrary right-hand objects to implement ``__rmatmul__`` fragments +the grammar and is inconsistent with how ``|`` works. Second, :pep:`593` +explicitly permits any valid Python object as metadata (e.g., strings, dicts). +Modifying the ``type`` metaclass's ``__matmul__`` guarantees consistent behavior +for all metadata without requiring opt-in base classes. + +Type Expressions vs. Non-Typing Annotations +------------------------------------------- + +The boundary between Type Expressions and Non-Typing Annotations is reinforced, +aligning with the formal typing specification. + +Parameter/Field Decorators +-------------------------- + +True parameter/field decorators were rejected for now. Modifying the core +parser to support symbol-level decorators opens a complex design space. This +is strictly scoped to type-level decorations. + +Existing Custom ``@`` Usage +--------------------------- + +Frameworks that currently rely on a custom ``@`` operator in annotations are +violating the typing specification. These are considered Non-Typing +Annotations. To clarify this boundary, ``Format.TYPE`` explicitly does not +handle Non-Typing Annotations. They remain fully supported via ``Format.STRING`` +or ``Format.VALUE`` when wrapped in ``@no_type_check``, and we do not break +metaclasses that implement their own ``__matmul__``. + +Operator Overloading +-------------------- + +Using ``@`` might confuse users expecting matrix multiplication. However, within +a Type Expression, Python intentionally reuses standard operators (like ``|`` +and ``[]``) with typing-specific semantics. + +Format.TYPE Coupling +-------------------- + +The coupling of ``Format.TYPE`` with Type Expression semantics is intentional +and reinforces the boundary between typing evaluation and normal runtime +evaluation. For example, ``NoneType`` explicitly does not implement +``__matmul__`` to prevent masking runtime bugs. Because ``Format.TYPE`` uses +typing semantics, it can safely evaluate ``None @Metadata`` structurally without +altering global ``NoneType`` behavior. + +Future Work +=========== + +While this feature stands on its own, establishing ``@`` as a standard metadata +token opens the door to several speculative extensions. The following ideas +outline one possible direction the ecosystem could take. + +Native Symbol Decorators +------------------------ + +Developers frequently request framework-agnostic field decorators:: + + class User(BaseModel): + @Field(primary_key=True) + id: int + +Reserving ``@`` for metadata provides the grammatical foundation for Symbol +Decorators. Any future proposal must choose between two architectural models: + +- **The Descriptor Model (Active Runtime):** The decorator acts as a runtime + function returning a `descriptor `__, + actively intercepting attribute access (similar to standard Python function + decorators). +- **The Metadata Model (Passive Declaration):** The field configuration is + treated as passive Symbol Metadata, leaving the enclosing class to process it + during creation. + +This PEP anchors Python to the Metadata Model. The modern ecosystem is heavily +driven by type-directed libraries (Pydantic, SQLAlchemy, ``dataclasses``) that +inspect static definitions during class creation rather than relying on +standalone descriptors. + +Because we are establishing the Metadata Model, ``@Field(...) id: int`` will +evaluate identically to ``id: int @Field(...)``. This alignment allows these +frameworks to inspect field configurations and continue working without +requiring extensive library changes. + +.. note:: + This resolves the conceptual tension between value space and type space. + While standard decorators operate in value space (actively modifying runtime + objects), this syntax firmly establishes ``@`` in type space, safely + attaching Symbol Metadata without altering runtime execution. + +Targeted Metadata +----------------- + +Future extensions to :pep:`746` will support annotation targets. By intersecting +a base type with an explicit target constraint, type checkers will validate +*where* metadata is allowed to exist. This prevents misuse (e.g., placing a +``@Column`` on a function parameter rather than a class field): + +*(Note: The following example assumes the ``&`` operator for intersection types +has been added.)* + +:: + + from typing import Target + + class Column: + """Valid only on integers that are fields of a SQLAlchemy Model.""" + __supports_annotated_base__: int & Target.FIELD[SQLAlchemy.Model] + +``Annotated`` has become a foundational building block for modern Python +frameworks. By establishing a clean, native syntax for metadata, this preserves +readability while paving the way for more sophisticated analysis tools and a +richer typing ecosystem. + +Open Issues +=========== + +Deprecation Timeline +-------------------- + +The removal of the ``typing._AnnotatedAlias`` compatibility layer is scheduled +for Python 3.21, following the standard 5-year deprecation policy (:pep:`387`). +However, given the widespread use of ``typing.Annotated``, the exact timeline +for removal (or whether the shim should be retained indefinitely to prevent +churn in legacy code) remains open for community discussion. + +``annotationlib.Format.TYPE`` Extraction +---------------------------------------- + +``Format.TYPE`` enables the structural evaluation of Type Expressions. While +required here to support ``type @annot``, it fundamentally improves the +evaluation of type expressions (such as correctly resolving ``|`` unions that +would otherwise be stringified by ``Format.FORWARDREF``). We invite discussion +on whether this broader semantic addition to ``annotationlib`` warrants +extraction into an independent PEP. + +Acknowledgements +================ + +Thanks to Hugo van Kemenade, Jelle Zijlstra, and Eric Traut for their feedback, +guidance, and assistance in refining this proposal. References ========== -- `Discussion on Python Discourse `_ -- :pep:`563` -- Postponed evaluation of annotations -- :pep:`585` -- Type hinting generics in standard collections -- :pep:`593` -- Flexible function and variable annotations -- :pep:`604` -- Allow writing union types as ``X | Y`` -- :pep:`727` -- Documentation metadata in typing (Withdrawn) -- :pep:`749` -- Implementing PEP 649 +- `Discussion on Python Discourse `_ + +.. [1] *"Functions where the arguments have type annotations can already be + rather long, and Annotated on its own is rather verbose, so I’m generally + glad it’s rare"* — Paul Moore on PEP 727: + https://discuss.python.org/t/32566/17 +.. [2] FastAPI support for ``Annotated``: + https://fastapi.tiangolo.com/tutorial/query-params-str-validations/ +.. [3] Pydantic support for ``Annotated``: + https://docs.pydantic.dev/latest/concepts/types/#annotated-types +.. [4] cattrs support for ``Annotated``: + https://cattrs.readthedocs.io/en/latest/validation.html#annotated +.. [5] msgspec support for ``Annotated``: + https://jcristharif.com/msgspec/supported-types.html#annotated +.. [6] SQLAlchemy 2.0 support for ``Annotated``: + https://docs.sqlalchemy.org/en/20/orm/declarative_tables.html#using-annotated-declarative-table-type-annotated-forms-for-mapped-column +.. [7] Typer support for ``Annotated``: + https://typer.tiangolo.com/tutorial/parameter-types/annotated/ +.. [8] Beartype support for ``Annotated`` (Validators): + https://beartype.readthedocs.io/en/latest/api_vale/ +.. [9] *"Annotated syntax is too long: Introduction of Annotated params made + function params more logical, but on the other hand longer/more verbose"* — + Vitaliy Kucheryaviy, author of Django Ninja: + https://github.com/tiangolo/fastapi/discussions/10055#discussion-5507018 +.. [10] *"I personally find this solution [using Annotated] a bit tedious when + you start having a lot of models/fields"* — g0di, Pydantic user: + https://github.com/pydantic/pydantic/discussions/2419#discussioncomment-7228409 +.. [11] SQLAlchemy 2.0 Migration Guide (advocating Annotated aliases to + mitigate verbosity): + https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#step-five-make-use-of-pep-593-annotated-to-package-common-directives-into-types +.. [12] *"My main concern here is that Annotated[torch.Tensor, dtype] is quite + verbose, and seems to go in the opposite direction to where we'd like to end + up"* — Ralf Gommers, NumPy maintainer: + https://github.com/pytorch/pytorch/issues/98702#issuecomment-1504794519 +.. [13] *"I'm not writing something stupidly verbose like: TensorBatchXChannels + = Annotated[...]"* — Patrick Kidger, author of TorchTyping: + https://github.com/beartype/beartype/discussions/96#discussioncomment-2245014 +.. [14] 2023 Python Discourse discussion proposing ``@`` as an alternative to + Annotated: + https://discuss.python.org/t/40751 +.. [15] 2025 Python Discourse discussion converging on dedicated ``@`` syntax: + https://discuss.python.org/t/103699 +.. [16] *"That by itself doesn’t seem a big objection – type annotations reuse + all kinds of operations, including x[y] and x | y."* — Guido van Rossum, on + repurposing the @ operator for typing: + https://discuss.python.org/t/40751/3 Copyright ========= -This document is placed in the public domain or under the -CC0-1.0-Universal license, whichever is more permissive. +This document is placed in the public domain or under the CC0-1.0-Universal +license, whichever is more permissive. From a4174160aee6f9c4b5e4981acb6001a30cb9ca19 Mon Sep 17 00:00:00 2001 From: Till Varoquaux Date: Fri, 10 Jul 2026 13:56:54 -0400 Subject: [PATCH 2/2] PEP 835: Address reviewer feedback and tighten narrative --- peps/pep-0835.rst | 681 ++++++++++++++++++++-------------------------- 1 file changed, 294 insertions(+), 387 deletions(-) diff --git a/peps/pep-0835.rst b/peps/pep-0835.rst index 64574ae08fc..edd6ce5f013 100644 --- a/peps/pep-0835.rst +++ b/peps/pep-0835.rst @@ -14,110 +14,91 @@ Post-History: `19-Apr-2026 `__ using the -``@`` operator. This change improves the ergonomics of type constraints and -reduces signature verbosity, benefiting type-directed libraries like -Pydantic, FastAPI, Typer, and SQLModel. +This PEP proposes overloading the ``@`` operator on types to allow writing +``Annotated[T, M]`` as ``T @M``. Motivation ========== -Metadata as a Semantic Operation --------------------------------- +:pep:`593` (``Annotated``) has seen widespread adoption. Major frameworks +(FastAPI, Pydantic, SQLAlchemy, msgspec, Typer, beartype) now rely on it as the +standard type metadata mechanism [1]_. -Historically, Python has treated type metadata as an afterthought. -``Annotated[T, M]`` models metadata as a generic container, similar to how -``Union[X, Y]`` once modeled unions. But metadata is not a container; it is a -modifier applied to a type. +However, its verbosity remains a barrier to adoption: -Just as :pep:`604` evolved unions from the ``Union`` container to the ``|`` -operator, the ``@`` syntax evolves metadata into a native language operator. -Elevating metadata to first-class syntax recognizes that modern libraries -increasingly use constraints on types (e.g., bounds, shapes, patterns). +1. **Libraries invent aliases to hide it.** Frameworks routinely create DSLs or + type aliases (e.g., ``PositiveInt``) to shield users from ``Annotated[..., + ...]`` boilerplate [2]_. +2. **Major projects defer adoption.** PyTorch [3]_, TorchTyping, and Beartype + [4]_ deferred adopting ``Annotated`` for tensor shapes, explicitly citing + its verbose syntax and waiting for a native language solution. +3. **It impacts readability.** During the :pep:`727` discussions, participants + warned that relying on ``Annotated`` for everyday documentation would make + code unreadable, contributing to the PEP's withdrawal [5]_. -Previously, applying these constraints forced a frustrating compromise between -type safety and readability. Before :pep:`593` (``Annotated``), frameworks like -Pydantic and FastAPI embedded metadata directly into default values. This was -concise but broke static analysis: +A container-like syntax opposes Python's evolution. Python is actively replacing +generic wrappers like ``typing.Union`` with operators like ``|`` (and the +proposed ``&`` for intersections), evolving type annotations into an arithmetic +of types. -.. code-block:: - :class: bad +The ``@`` shorthand aligns syntax with semantics. By applying type metadata as a +postfix operator, it: + +- **Reduces nesting** (no brackets required). +- **Reduces tokens** (no ``typing.Annotated`` import). +- **Improves locality** (the base type remains front-and-center). +- **Extends the arithmetic of types** (type metadata acts as an operation on an + existing type expression rather than a generic container). + +This resolves the conflict between concise syntax and strict typing:: class User(BaseModel): + # Bad: Early frameworks abused defaults, breaking static analysis id: int = Field(gt=0) - name: str = Field(min_length=3) -Moving to ``Annotated`` fixed the typing semantics but introduced significant -complexity. To hide the visual noise, libraries introduced alias types (e.g., -``PositiveInt``). However, the moment users step outside pre-packaged -constraints, they are forced back into the complex syntax:: + # Verbose: PEP 593 fixed semantics, but hindered adoption + id: Annotated[int, Field(gt=0)] - from typing import Annotated - from pydantic import BaseModel, Field, PositiveInt + # Proposed: Concise while preserving PEP 593 semantics + id: int @Field(gt=0) - class User(BaseModel): - age: PositiveInt # Concise but limited - id: Annotated[int, Field(gt=0, le=1000)] # Verbose fallback +As an operator, ``@`` composes natively:: -The ``@`` shorthand resolves the ergonomic tradeoff, restoring the conciseness -of early frameworks while preserving the strict semantics of ``Annotated``: + # Inside generics + list[Annotated[str, Field(max_length=50)]] + list[str @Field(max_length=50)] -.. code-block:: - :class: good + # Within a union + Annotated[int, Ge(0)] | Annotated[str, Len(5)] + int @Ge(0) | str @Len(5) - from pydantic import BaseModel, Field - from fastapi import Query + # Across an entire union + Annotated[str | None, Field(description="Optional string")] + (str | None) @Field(description="Optional string") - class User(BaseModel): - id: int @Field(gt=0, le=1000) - name: str @Field(min_length=3, max_length=50) = "Anonymous" - age: int @Field(gt=0) - email: str @Field(pattern=r".*@.*") - - async def read_items(q: (str | None) @Query(max_length=50) = None): - ... + # Within callables + Callable[[Annotated[int, Ge(0)]], str] + Callable[[int @Ge(0)], str] -Prior Art and Historical Context +Historical Context and Prior Art -------------------------------- -The verbosity of ``Annotated`` is not just an aesthetic annoyance; it hinders -adoption. During the 2023 review of :pep:`727`, reviewers warned that forcing -developers to use ``Annotated`` for everyday documentation would introduce -excessive visual noise [1]_. This pushback led to the PEP's withdrawal and -sparked the initial discussions around using ``@`` as a native alternative. - -Today, this friction remains evident across the entire ecosystem. The most -prominent type-directed frameworks in Python (FastAPI, Pydantic, SQLAlchemy, -cattrs, msgspec, Typer, and beartype) all heavily rely on ``Annotated`` [2]_ -[3]_ [4]_ [5]_ [6]_ [7]_ [8]_. Yet, they frequently receive user complaints -about signature bloat [9]_ [10]_ [11]_. In some domains, library authors refuse -to use it entirely: PyTorch [12]_, TorchTyping, and Beartype [13]_ rejected -``Annotated`` for tensor shape typing, deciding instead to wait for a native -language solution. - -When the community debated alternatives in late 2023 [14]_ and 2025 [15]_, -discussion trended toward the ``@`` operator. It visually aligns with existing -Python `decorators `__. -Adopting ``@`` for metadata follows the precedent of repurposing runtime -operators for static typing, as seen with ``[]`` for generics (:pep:`585`) and -``|`` for unions (:pep:`604`) [16]_. +When the community debated alternatives in late 2023 [6]_ and 2025 [7]_, +discussion trended toward the ``@`` operator because it visually aligns with +existing Python `decorators +`__. -Precedent in Other Languages ----------------------------- +Adopting ``@`` for type metadata also follows the precedent of repurposing runtime +operators for static typing, as seen with ``[]`` for generics (:pep:`585`) and +``|`` for unions (:pep:`604`) [8]_. -The ``@`` metadata syntax mirrors features in other statically typed languages: +The postfix syntax mirrors features in other statically typed languages: -- **C++:** Uses ``[[attribute]]`` for both symbols and types (e.g., - ``[[nodiscard]] int f();``). -- **C#:** Uses ``[Attribute]`` for reflection-based metadata (e.g., - ``[Required] public string Name;``). -- **Java / Kotlin:** Uses ``@Annotation``. Java's `JSR 308 - `__ introduced ``TYPE_USE`` annotations - to decorate types (e.g., ``List<@NonNull String>``). Kotlin uses ``val x: - @NotNull String``. -- **OCaml:** Uses ``[@attribute]`` postfix syntax to attach metadata to the - preceding AST node (e.g., ``type t = int [@default 0]``). +- **C++:** Uses ``[[attribute]]`` for types (e.g., ``[[nodiscard]] int f();``). +- **Java / Kotlin:** Uses ``@Annotation`` (e.g., ``List<@NonNull String>`` or + ``val x: @NotNull String``). +- **OCaml:** Uses ``[@attribute]`` postfix syntax (e.g., + ``type t = int [@default 0]``). Terminology =========== @@ -125,43 +106,40 @@ Terminology To clarify discussion around annotations and metadata, the following terms apply: -- **Type Expression**: e.g., ``x: ``. Expressions evaluating to valid - static types, as specified in the `Typing Specification +- **Type expression**: e.g., ``x: ``. An expression within a type hint + that evaluates to a valid type, as specified in the `Typing Specification `__ (:pep:`484`). -- **Type Metadata**: e.g., ``int @``. Data attached to a Type Expression +- **Type metadata**: e.g., ``int @``. Data attached to a type expression via ``Annotated`` (:pep:`593`, :pep:`746`). -- **Non-Typing Annotation**: e.g., `@no_type_check +- **Non-typing annotation**: e.g., `@no_type_check `__ - wrapping ``x: ``. - Standard behavior where annotations are evaluated as arbitrary runtime values - rather than strict type hints. -- **Symbol Decorator**: Applying metadata directly to a variable or field - declaration, rather than to its underlying type. While Python does not - currently have a native syntax for variable decorators, this conceptual - distinction is well-established in languages like Java: + wrapping ``x: ``. Annotations that are not intended to be evaluated as + type expressions (e.g.: ignored by type checkers). +- **Symbol decorator**: Applying metadata directly to a variable or field + declaration, rather than to its underlying type. This distinction is + well-established in languages like Java: .. code-block:: java class Application { - // Field Decorator (Symbol Decorator) + // Field Decorator (symbol decorator) @Inject private Service s; - // Type Decorator (Type Metadata) + // Type Decorator (type metadata) private @NonNull String name; } -- **Symbol Metadata** (or **Field Metadata**): Data attached to a symbol via a - Symbol Decorator. This contrasts with an active runtime descriptor. It is - fundamentally distinct from Type Metadata, as it applies to the specific +- **Symbol metadata** (or **field metadata**): Metadata attached to a + symbol. It is distinct from type metadata, as it applies to the specific instance of the variable rather than the type itself. Specification ============= -The proposed syntax uses the ``@`` (``__matmul__``) operator to attach metadata -to a type:: +The proposed syntax uses the ``__matmul__`` operator to attach metadata to a +type:: # Current syntax x: Annotated[int, Range(0, 10)] @@ -172,17 +150,14 @@ to a type:: Operator Precedence ------------------- -The ``@`` operator binds tighter than the ``|`` union operator (:pep:`604`). -This aligns with standard Python expression precedence, ensuring predictable -parsing for Type Metadata. - -Because ``@`` binds tightly, developers can attach distinct constraints directly -to specific types within a union. For example, a US zip code might be a 5-digit -integer *or* a 5-character string:: +Because ``@`` binds tighter than the ``|`` union operator (:pep:`604`), distinct +constraints can be attached directly to specific types within a union without +parentheses. For example, a US zip code might be a 5-digit integer *or* a +5-character string:: zip_code: int @Ge(10000) @Le(99999) | str @Len(5) -If you intend to attach metadata to the entire union, you must use parentheses:: +To attach metadata to the entire union, use parentheses:: # Attaches only to 'str' int | str @Metadata # equivalent to: int | Annotated[str, Metadata] @@ -190,41 +165,27 @@ If you intend to attach metadata to the entire union, you must use parentheses:: # Attaches to the entire union (int | str) @Metadata # equivalent to: Annotated[int | str, Metadata] -The "Parentheses Friction" -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Some developers have noted that writing ``(str | None) @Field(...)`` feels -cumbersome for optional fields. -This awkwardness highlights a missing language feature. Without native Symbol -Decorators, frameworks must co-opt ``Annotated`` to configure fields. When a -developer writes ``address: (str | None) @Field(...)``, they are actually -attempting to configure the ``address`` field, not the ``str | None`` -type. Until native field decorators exist, parentheses remain the structurally -accurate way to express this in the type system. Flattening Multiple Metadata ---------------------------- -When multiple metadata items are chained, the resulting ``Annotated`` object is -flattened. +Chained metadata flattens the resulting ``Annotated`` object. ``T @m1 @m2`` +evaluates to ``Annotated[T, m1, m2]``, never +``Annotated[Annotated[T, m1], m2]``. This mirrors ``typing.Annotated``'s +existing runtime behavior. -Specifically, ``T @m1 @m2`` evaluates to ``Annotated[T, m1, m2]``. It must not -resolve to a nested structure such as ``Annotated[Annotated[T, m1], m2]``. This -mirrors the existing runtime behavior of ``typing.Annotated``. - -This flattening also applies when the left-hand operand is an existing -``Annotated`` type, regardless of how it was constructed:: +This same logic applies when the left-hand operand is an existing ``Annotated`` +type:: Annotated[int, m1] @m2 # AnnotatedType(int, m1, m2) (flattened) Runtime Behavior ---------------- -The ``@`` operator produces a ``types.AnnotatedType`` instance, a new built-in -type implemented in C. The existing ``typing.Annotated`` is unified with this -type: ``typing.Annotated[X, Y]`` returns the same ``types.AnnotatedType`` object -as ``X @Y``: +``@`` produces a ``types.AnnotatedType`` (a new built-in C type). The existing +``typing.Annotated`` unifies with this type. ``typing.Annotated[X, Y]`` and ``X +@Y`` return the exact same object: .. code-block:: pycon @@ -248,17 +209,14 @@ The ``repr()`` of an ``AnnotatedType`` uses the shorthand syntax: >>> int @Field(gt=0) int @Field(gt=0) -``AnnotatedType`` objects support pickling via ``copyreg``, reconstructing -through ``AnnotatedType[origin, *metadata]``. - Handling of ``None`` -------------------- -Implementing ``__matmul__`` on ``NoneType`` is explicitly avoided to prevent -masking runtime bugs in non-typing contexts. +``NoneType`` explicitly avoids implementing ``__matmul__`` to prevent masking +runtime bugs. -For example, a developer might forget to validate ``None`` before executing -matrix multiplication on an array: +For example, a developer might forget to check for ``None`` before matrix +multiplication: .. code-block:: :class: bad @@ -266,154 +224,135 @@ matrix multiplication on an array: def matmul_arrays(a: np.array | None, b: np.array): return a @ b # oops, forgot to check for None -If ``__matmul__`` were implemented on ``NoneType``, this would return an -``AnnotatedType`` object instead of raising a ``TypeError``, silently masking -the bug. +If ``NoneType.__matmul__`` existed, this would silently return an +``AnnotatedType`` instead of raising a ``TypeError``. -Because ``Format.TYPE`` is introduced to ``annotationlib``, this limitation is -invisible in valid typing contexts. ``Format.TYPE`` evaluates -structurally and correctly parses ``None @Metadata`` into an ``AnnotatedType`` -without relying on ``NoneType.__matmul__``. Outside of Type Expressions -(e.g., in raw runtime execution), users must use ``Annotated[None, Metadata]``. +``annotationlib.Format.TYPE`` sidesteps this limitation in typing +expressions. It evaluates structurally, correctly parsing ``None @Metadata`` +into an ``AnnotatedType``. Outside of type expressions, use ``Annotated[None, +Metadata]``. Supported Left-Hand Operands ----------------------------- -The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the -metatype (``type``) and to exactly the same typing constructs that currently -support the ``|`` operator for unions (``types.GenericAlias``, +The ``@`` operator adds ``nb_matrix_multiply`` to ``type`` and to all typing +constructs that support the ``|`` union operator (``types.GenericAlias``, ``types.UnionType``, ``types.AnnotatedType``, ``typing.TypeVar``, ``typing.ParamSpec``, ``typing.TypeVarTuple``, ``typing.TypeAliasType``, ``typing.ForwardRef``, and ``sentinel`` objects). -For all other left-hand operands, the operator returns ``NotImplemented``, -allowing normal ``__matmul__`` dispatch to proceed. +Applying ``@`` to a class evaluates as type metadata; applying it to an instance +performs arithmetic. -This means ``int @Field()`` produces an ``AnnotatedType``, while ``42 -@something`` is unaffected. Crucially, ``ndarray @Field()`` (using the **class** -as a type annotation) also produces an ``AnnotatedType``, even though ndarray -*instances* define ``__matmul__`` for matrix multiplication. One consequence of -this design is that applying the ``@`` operator to a class object evaluates as -Type Metadata, while applying it to an instance performs arithmetic. - -The only case where the shorthand does not apply is when a class has a -**metaclass** that defines ``__matmul__``. In that case, the metaclass's -operator takes priority via standard Python MRO dispatch. +For example, ``int @Field()`` produces an ``AnnotatedType``, while ``42 @ +something`` raises a ``TypeError`` (or delegates to ``__rmatmul__``). +Likewise, ``ndarray @Field()`` produces an ``AnnotatedType``, even though +``ndarray`` instances define ``__matmul__`` for matrix multiplication. +Custom metaclasses can overload ``__matmul__`` provided ``@`` is avoided in type +expressions. Forward References and Deferred Evaluation ------------------------------------------- -Under :pep:`749`'s lazy evaluation, existing ``annotationlib`` formats are -insufficient for ``@``. Specifically, ``Format.FORWARDREF`` fails structurally -on unresolvable names:: +Under :pep:`749`'s lazy evaluation, existing ``annotationlib`` formats do not +preserve the structure of ``@`` expressions when names are unresolvable. +``Format.FORWARDREF`` stringifies unresolvable names:: class Model: ref: NotYetDefined @Field(gt=0) -``Format.FORWARDREF`` produces an opaque ``ForwardRef('"NotYetDefined" -@Field(gt=0)')``. The metadata is trapped inside the unresolved string, blocking -libraries like Pydantic from inspecting it at class-definition time. +This produces an opaque ``ForwardRef('"NotYetDefined" @Field(gt=0)')``. The +metadata is enclosed within the string, preventing libraries from easily +inspecting it. -``Format.TYPE`` assumes typing semantics and evaluates Type Expressions -structurally. Unresolvable names are wrapped in a ``ForwardRef`` independently, -leaving operators intact:: +``Format.TYPE`` assumes typing semantics and evaluates type expressions +structurally: - AnnotatedType(ForwardRef('NotYetDefined'), Field(gt=0)) +* **Unresolvable Names:** Unresolvable names are wrapped in a ``ForwardRef`` + independently, leaving operators intact. +* **Union Types:** The ``|`` operator evaluates to ``types.UnionType``. +* **Annotated Types:** The ``@`` operator evaluates to ``types.AnnotatedType``. +* **Metadata Boundary:** Structural evaluation applies only to the type space. + Expressions within the metadata (the right-hand side of ``@`` or the + subsequent arguments of ``Annotated``) are evaluated as values. Unresolvable + expressions there produce a single opaque ``ForwardRef`` (identical to + ``Format.FORWARDREF``). -The metadata remains immediately accessible. This also resolves the pre-existing -issue where ``"Foo" | int`` incorrectly produced ``ForwardRef('Foo | int')`` -under ``Format.FORWARDREF``. +For example, the ``NotYetDefined @Field(gt=0)`` annotation evaluates into an +``AnnotatedType`` where the metadata remains immediately accessible:: -Importantly, because ``Format.TYPE`` can evaluate structurally when runtime -dunder methods are missing (falling back to ``types.UnionType`` and -``types.AnnotatedType`` for the ``|`` and ``@`` operators), it parses ``None -@Metadata`` without requiring ``NoneType.__matmul__``. - -Parsing and Grammar -------------------- + AnnotatedType(ForwardRef('NotYetDefined'), Field(gt=0)) -No changes to the Python grammar are required. Because ``@`` is already a valid -operator, it is natively parsed as a binary operation. The shorthand is resolved -during semantic analysis, entirely bypassing the need to patch grammar files or -update the parser. +This also resolves the pre-existing limitation where ``"Foo" | int`` produced +``ForwardRef('Foo | int')`` under ``Format.FORWARDREF``. Rationale ========= -Baking ``Annotated`` into native syntax removes ``typing`` import overhead, -directly aiding runtime type introspection. +``Annotated`` forces type metadata to resemble a parameterized class. A postfix +operator provides identical semantics without nesting. + +Reusing the existing ``@`` operator leaves the Python parser unchanged. The +operator's new meaning is isolated to type evaluation, where ``T @M`` lowers to +``Annotated[T, M]``. Type checkers (Mypy, Pyright) also require no parser +changes, handling the syntax directly during semantic analysis. We prototyped a +Ruff conversion rule for automated migrations, and CPython prototype testing +confirms that libraries like ``typer`` and ``pydantic`` continue to work without +modification. + +Parentheses and Verbosity +------------------------- -Because ``@`` is already a valid expression operator, type checkers (Mypy, -Pyright) require no parser changes, handling the syntax entirely during semantic -analysis. We have prototyped an automated conversion rule in Ruff, enabling -automated codebase migrations, and CPython prototype testing confirms that -libraries like ``typer`` and ``pydantic`` work natively out of the box. +Writing ``address: (str | None) @Field(...)`` adds verbosity due to the lack +of native symbol decorators. Frameworks co-opt ``Annotated`` to configure +fields, even though the developer's intent is to configure the ``address`` +field, not the ``str | None`` type. While the ``@`` shorthand cannot fully +eliminate this structural limitation, it significantly reduces the syntactic +weight of the workaround compared to ``Annotated[str | None, Field(...)]``. Why the @ Operator? ------------------- -Selecting the correct operator for metadata involves balancing three grammatical +Selecting the correct operator for metadata involves balancing three considerations: -1. **Precedence & Spec Restrictions:** The typing specification currently only - allows the ``|`` operator in Type Expressions. Any new operator must bind - tighter than ``|`` (Union) and ``&`` (proposed Intersection) so that - expressions like ``int @Field() | str`` parse correctly without parentheses. - This eliminates operators like ``|``, ``^``, and ``&``. -2. **Backwards Compatibility & Consistency:** Using an existing operator is - preferable to introducing a new keyword (e.g., ``annotated``). It minimizes - parser complexity and mitigates the risk of breaking existing code. It also - aligns with the Python typing specification's precedent of preferring - operators (e.g., ``|``) over keywords within type expressions. +1. **Precedence:** Binding tighter than ``|`` (Union) and ``&`` (proposed + Intersection) ensures expressions like ``int @Field() | str`` parse correctly + unparenthesized. This excludes operators like ``|``, ``^``, and ``&``. +2. **Ecosystem Compatibility:** New keywords require ecosystem-wide parser + migrations. Because Python's type system is evolving into an arithmetic of + types (e.g., replacing ``Union`` with ``|``), reusing an operator extends + this pattern without unnecessary churn. 3. **Semantic Clarity:** The chosen syntax should avoid colliding with - established mathematical intuitions for primitive types. + established intuitions for primitive types. This leaves the set of overridable binary operators that bind tighter than ``&``: ``**``, ``*``, ``@``, ``/``, ``//``, ``%``, ``+``, ``-``, ``>>``, and ``<<``. Standard arithmetic operators like ``+``, ``-``, ``/``, ``//``, ``*``, ``**``, -and ``%`` are misleading in this context. Reading ``int + x`` or ``float / -Field()`` strongly implies mathematical evaluation, not metadata decoration. - -This leaves ``<<``, ``>>``, and ``@``. Of these, ``@`` is the only operator that -possesses an existing association with metadata in Python (via function and -class decorators). Bitwise shift operators (``<<``, ``>>``) lack this -association. - -While ``@`` is used for matrix multiplication in numeric libraries (like NumPy), -it is far less associated with core scalar arithmetic than operators like ``+`` -or ``/``. - -Language Complexity -------------------- +and ``%`` are misleading. Reading ``int + x`` or ``float / Field()`` strongly +implies mathematical evaluation, not metadata decoration. -Adding new syntax always introduces some language complexity. However, while -syntactic complexity increases, the *cognitive* load actually decreases. +The remaining candidates are ``<<``, ``>>``, and ``@``. We chose ``@`` because +it already associates with metadata in Python via decorators. While libraries +like NumPy use it for matrix multiplication, it isn't as tied to arithmetic as +operators like ``+`` or ``/``. -Teaching a beginner ``int @Field(gt=0)`` leverages their existing intuition of -Python decorators. It visually separates the base type from its metadata. -Teaching ``typing.Annotated`` requires referencing obscure parts of the typing -module. By moving metadata from a standard library class to a native grammatical -construct, we reduce the overall cognitive load required to read and write -modern typed Python. Backwards Compatibility ======================= -typing.Annotated Migration ---------------------------- - The pure-Python ``typing._AnnotatedAlias`` class is replaced with a native C implementation (``types.AnnotatedType``). ``typing.Annotated`` becomes a reference to this C type rather than a special form with a custom metaclass. -The private ``typing._AnnotatedAlias`` class is retained as a deprecated +To ensure a smooth transition, this legacy class is retained as a deprecated compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)`` will continue to work but emit a ``DeprecationWarning``. The shim is scheduled for -removal in Python 3.21 (see `Deprecation Timeline`_). +removal in Python 3.21 (see `Open Issues`_). Code that should be updated: @@ -422,14 +361,11 @@ Code that should be updated: - ``typing._AnnotatedAlias(origin, metadata)`` → use ``Annotated[origin, *metadata]`` or ``origin @m1 @m2`` -Backporting via typing_extensions ----------------------------------- - -Like ``X | Y``, the ``@`` shorthand requires changes to the metatype -(``type.__matmul__``), which cannot be patched from pure Python. The shorthand -is only available on Python 3.16+. The existing ``Annotated[X, Y]`` syntax -continues to work on all supported versions and should be used when backwards -compatibility is required. +**Backporting via typing_extensions:** Like ``X | Y``, the ``@`` shorthand +requires changes to the metatype (``type.__matmul__``), which cannot be patched +from pure Python. The shorthand is only available on Python 3.16+. The existing +``Annotated[X, Y]`` syntax continues to work on all supported versions and +should be used when backwards compatibility is required. Security Implications ===================== @@ -452,10 +388,7 @@ primary syntax for applying metadata. The verbose ``typing.Annotated`` form should be treated as an advanced detail, primarily relevant to library authors or when dynamically generating types. -Visual Style ------------- - -The shorthand must be strictly formatted as ``type @annot`` (e.g., ``int +**Visual Style:** Format the shorthand as ``type @annot`` (e.g., ``int @Metadata(...)``), with a space before the ``@`` and no space after it. This distinguishes it from standard matrix multiplication (``A @ B``) and aligns visually with function decorators (``@decorator``) and Java annotations. Code @@ -465,10 +398,8 @@ contexts. Usage Examples ============== -Pydantic Validation -------------------- - -The shorthand can be used in data validation scenarios:: +**Pydantic Validation:** The shorthand can be used in data validation +scenarios:: from pydantic import BaseModel, Field, HttpUrl from annotated_types import Len @@ -478,10 +409,8 @@ The shorthand can be used in data validation scenarios:: url: HttpUrl @Field(description="The project homepage") stars: int @Field(ge=0) = 0 -FastAPI Dependency Injection ----------------------------- - -In FastAPI, the shorthand simplifies complex parameter definitions:: +**FastAPI Dependency Injection:** In FastAPI, the shorthand simplifies complex +parameter definitions:: from fastapi import FastAPI, Header, Depends @@ -491,11 +420,8 @@ In FastAPI, the shorthand simplifies complex parameter definitions:: async def secure_endpoint(token: str @Header(description="Auth token")): return {"status": "authorized"} -SQLModel and Database Definitions ---------------------------------- - -SQLModel relies heavily on ``Annotated`` to define column properties. The -shorthand syntax makes these definitions significantly cleaner:: +**SQLModel and Database Definitions:** SQLModel relies on ``Annotated`` to +define column properties. The shorthand syntax cleans up these definitions:: from sqlmodel import SQLModel, Field @@ -505,12 +431,9 @@ shorthand syntax makes these definitions significantly cleaner:: secret_name: str age: (int | None) @Field(index=True) = None -Testing and Formal Verification -------------------------------- - -Libraries like Hypothesis (property-based testing) and CrossHair (symbolic -execution) utilize ``annotated-types`` to constrain test generation and -analysis. The shorthand provides a clean syntax for specifying test boundaries:: +**Testing and Formal Verification:** Libraries like Hypothesis and CrossHair use +``annotated-types`` to constrain test generation. The shorthand provides a clean +syntax for specifying test boundaries:: from dataclasses import dataclass from annotated_types import Ge, Interval @@ -546,8 +469,8 @@ Prototype implementations are available for the following tools: - **Ruff:** `Ruff at-type-annot `_ -``builtins.type`` in Typeshed will be updated to include ``def -__matmul__(self, other: Any) -> types.AnnotatedType: ...`` to support type +``builtins.type`` in Typeshed will be updated to include +``def __matmul__(self, other: Any) -> types.AnnotatedType: ...`` to support type checkers. Rejected Ideas @@ -562,16 +485,14 @@ Debates around reusing ``@`` yielded several alternatives: - A new soft keyword: ``int annotated Interval(1, 10)`` - Bracket syntax: ``x: int {Gt(10), Lt(20)}`` -These were rejected because none garnered wide consensus. The ``@`` symbol also -extends cleanly to Symbol Decorators should the language pursue that route in -the future. +These lacked consensus. The ``@`` symbol also extends cleanly to symbol +decorators if the language pursues that route later. Reliance on ``__rmatmul__`` --------------------------- -We explicitly reject relying on ``__rmatmul__`` on the metadata objects. One -might envision a base class that implements ``__rmatmul__`` to return an -``Annotated`` type:: +We explicitly reject relying on metadata objects implementing ``__rmatmul__`` +(e.g., via a base class) to return an ``Annotated`` type:: class Metadata[T = object]: def __rmatmul__(self, typ: TypeForm[T], /) -> TypeForm[T]: @@ -580,103 +501,83 @@ might envision a base class that implements ``__rmatmul__`` to return an class Le(Metadata[int]): ... -This fails for two reasons. First, Type Expressions require fixed semantics; -relying on arbitrary right-hand objects to implement ``__rmatmul__`` fragments -the grammar and is inconsistent with how ``|`` works. Second, :pep:`593` -explicitly permits any valid Python object as metadata (e.g., strings, dicts). -Modifying the ``type`` metaclass's ``__matmul__`` guarantees consistent behavior -for all metadata without requiring opt-in base classes. - -Type Expressions vs. Non-Typing Annotations -------------------------------------------- - -The boundary between Type Expressions and Non-Typing Annotations is reinforced, -aligning with the formal typing specification. +This approach is rejected for two reasons. First, relying on arbitrary +right-hand objects to implement ``__rmatmul__`` breaks the type expression +model, complicating the assignment of fixed static meanings to operators. This +is inconsistent with how ``|`` works and contradicts the type expression grammar +defined in the specification. Second, :pep:`593` explicitly permits any valid +Python object as metadata (e.g., strings, dicts). Implementing ``__matmul__`` +directly on the ``type`` metaclass avoids opt-in base classes and provides +consistent behavior. Parameter/Field Decorators -------------------------- -True parameter/field decorators were rejected for now. Modifying the core -parser to support symbol-level decorators opens a complex design space. This -is strictly scoped to type-level decorations. +True parameter/field decorators were rejected for now. Modifying the core parser +for symbol-level decorators opens a complex design space; this PEP scopes the +``@`` operator to type expressions. -Existing Custom ``@`` Usage ---------------------------- +Avoiding @ Due to Matrix Multiplication +--------------------------------------- -Frameworks that currently rely on a custom ``@`` operator in annotations are -violating the typing specification. These are considered Non-Typing -Annotations. To clarify this boundary, ``Format.TYPE`` explicitly does not -handle Non-Typing Annotations. They remain fully supported via ``Format.STRING`` -or ``Format.VALUE`` when wrapped in ``@no_type_check``, and we do not break -metaclasses that implement their own ``__matmul__``. +We rejected the argument that ``@`` should be avoided because of its association +with matrix multiplication. Within a type expression, Python already reuses +standard operators (like ``|`` for unions and ``[]`` for generics) with +typing-specific semantics (see `Why the @ Operator?`_). -Operator Overloading --------------------- - -Using ``@`` might confuse users expecting matrix multiplication. However, within -a Type Expression, Python intentionally reuses standard operators (like ``|`` -and ``[]``) with typing-specific semantics. +Evaluating Type Expressions as Runtime Code +------------------------------------------- -Format.TYPE Coupling --------------------- +The design explicitly couples ``Format.TYPE`` to type expression semantics, +reinforcing the boundary between static typing and runtime evaluation: -The coupling of ``Format.TYPE`` with Type Expression semantics is intentional -and reinforces the boundary between typing evaluation and normal runtime -evaluation. For example, ``NoneType`` explicitly does not implement -``__matmul__`` to prevent masking runtime bugs. Because ``Format.TYPE`` uses -typing semantics, it can safely evaluate ``None @Metadata`` structurally without -altering global ``NoneType`` behavior. +* **Non-Typing Annotations:** Frameworks using custom ``@`` operators in + annotations remain supported via ``Format.STRING`` and ``Format.VALUE``. They + are incompatible with ``Format.TYPE``, which enforces typing semantics. +* **Structural Evaluation:** Because ``Format.TYPE`` handles the ``@`` operator + structurally, it can evaluate constructs like ``None @Metadata`` even though + the global ``NoneType`` does not implement ``__matmul__``. Future Work =========== -While this feature stands on its own, establishing ``@`` as a standard metadata -token opens the door to several speculative extensions. The following ideas -outline one possible direction the ecosystem could take. +Although this proposal stands on its own, establishing ``@`` for type metadata +enables several future extensions. Native Symbol Decorators ------------------------ -Developers frequently request framework-agnostic field decorators:: +Developers request framework-agnostic field decorators:: class User(BaseModel): @Field(primary_key=True) id: int -Reserving ``@`` for metadata provides the grammatical foundation for Symbol -Decorators. Any future proposal must choose between two architectural models: +Future proposals will likely need to navigate between two architectural models: -- **The Descriptor Model (Active Runtime):** The decorator acts as a runtime - function returning a `descriptor `__, - actively intercepting attribute access (similar to standard Python function +- **The descriptor model:** The decorator acts as a runtime function returning a + `descriptor `__, actively + intercepting attribute access (similar to standard Python function decorators). -- **The Metadata Model (Passive Declaration):** The field configuration is - treated as passive Symbol Metadata, leaving the enclosing class to process it - during creation. +- **The metadata model:** The field configuration is treated as passive symbol + metadata, leaving the enclosing class to process it during creation. -This PEP anchors Python to the Metadata Model. The modern ecosystem is heavily -driven by type-directed libraries (Pydantic, SQLAlchemy, ``dataclasses``) that +This proposal aligns with the metadata model. Type-directed libraries typically inspect static definitions during class creation rather than relying on standalone descriptors. -Because we are establishing the Metadata Model, ``@Field(...) id: int`` will -evaluate identically to ``id: int @Field(...)``. This alignment allows these -frameworks to inspect field configurations and continue working without -requiring extensive library changes. - -.. note:: - This resolves the conceptual tension between value space and type space. - While standard decorators operate in value space (actively modifying runtime - objects), this syntax firmly establishes ``@`` in type space, safely - attaching Symbol Metadata without altering runtime execution. +``@Field(...) id: int`` would evaluate identically to ``id: int +@Field(...)``. This allows existing frameworks to inspect field configurations +and continue working unmodified. Under this model, value-space decorators modify +objects at runtime, while type-space decorators attach metadata to a type. Targeted Metadata ----------------- -Future extensions to :pep:`746` will support annotation targets. By intersecting -a base type with an explicit target constraint, type checkers will validate -*where* metadata is allowed to exist. This prevents misuse (e.g., placing a -``@Column`` on a function parameter rather than a class field): +Future extensions to :pep:`746` could support annotation targets. By +intersecting a base type with an explicit target constraint, type checkers will +validate *where* metadata is allowed to exist. This mitigates misuse (e.g., +placing a ``@Column`` on a function parameter rather than a class field): *(Note: The following example assumes the ``&`` operator for intersection types has been added.)* @@ -689,32 +590,19 @@ has been added.)* """Valid only on integers that are fields of a SQLAlchemy Model.""" __supports_annotated_base__: int & Target.FIELD[SQLAlchemy.Model] -``Annotated`` has become a foundational building block for modern Python -frameworks. By establishing a clean, native syntax for metadata, this preserves -readability while paving the way for more sophisticated analysis tools and a -richer typing ecosystem. +Establishing a native syntax for metadata provides a structural foundation for +future extensions like annotation targets. Open Issues =========== -Deprecation Timeline --------------------- - -The removal of the ``typing._AnnotatedAlias`` compatibility layer is scheduled -for Python 3.21, following the standard 5-year deprecation policy (:pep:`387`). -However, given the widespread use of ``typing.Annotated``, the exact timeline -for removal (or whether the shim should be retained indefinitely to prevent -churn in legacy code) remains open for community discussion. +**Deprecation Timeline:** As a private class, ``typing._AnnotatedAlias`` could +bypass the standard 5-year deprecation policy (:pep:`387`). Should we +fast-track its removal? -``annotationlib.Format.TYPE`` Extraction ----------------------------------------- - -``Format.TYPE`` enables the structural evaluation of Type Expressions. While -required here to support ``type @annot``, it fundamentally improves the -evaluation of type expressions (such as correctly resolving ``|`` unions that -would otherwise be stringified by ``Format.FORWARDREF``). We invite discussion -on whether this broader semantic addition to ``annotationlib`` warrants -extraction into an independent PEP. +**annotationlib.Format.TYPE Extraction:** ``Format.TYPE`` improves +type expression evaluation (e.g., properly resolving ``|`` unions). Does this +warrant a standalone PEP? Acknowledgements ================ @@ -727,47 +615,66 @@ References - `Discussion on Python Discourse `_ -.. [1] *"Functions where the arguments have type annotations can already be - rather long, and Annotated on its own is rather verbose, so I’m generally - glad it’s rare"* — Paul Moore on PEP 727: - https://discuss.python.org/t/32566/17 -.. [2] FastAPI support for ``Annotated``: - https://fastapi.tiangolo.com/tutorial/query-params-str-validations/ -.. [3] Pydantic support for ``Annotated``: - https://docs.pydantic.dev/latest/concepts/types/#annotated-types -.. [4] cattrs support for ``Annotated``: - https://cattrs.readthedocs.io/en/latest/validation.html#annotated -.. [5] msgspec support for ``Annotated``: - https://jcristharif.com/msgspec/supported-types.html#annotated -.. [6] SQLAlchemy 2.0 support for ``Annotated``: - https://docs.sqlalchemy.org/en/20/orm/declarative_tables.html#using-annotated-declarative-table-type-annotated-forms-for-mapped-column -.. [7] Typer support for ``Annotated``: - https://typer.tiangolo.com/tutorial/parameter-types/annotated/ -.. [8] Beartype support for ``Annotated`` (Validators): - https://beartype.readthedocs.io/en/latest/api_vale/ -.. [9] *"Annotated syntax is too long: Introduction of Annotated params made - function params more logical, but on the other hand longer/more verbose"* — - Vitaliy Kucheryaviy, author of Django Ninja: - https://github.com/tiangolo/fastapi/discussions/10055#discussion-5507018 -.. [10] *"I personally find this solution [using Annotated] a bit tedious when - you start having a lot of models/fields"* — g0di, Pydantic user: - https://github.com/pydantic/pydantic/discussions/2419#discussioncomment-7228409 -.. [11] SQLAlchemy 2.0 Migration Guide (advocating Annotated aliases to - mitigate verbosity): - https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#step-five-make-use-of-pep-593-annotated-to-package-common-directives-into-types -.. [12] *"My main concern here is that Annotated[torch.Tensor, dtype] is quite +.. [1] Major frameworks supporting ``Annotated`` include: + + * FastAPI support for ``Annotated``: + https://fastapi.tiangolo.com/tutorial/query-params-str-validations/ + + * Pydantic support for ``Annotated``: + https://docs.pydantic.dev/latest/concepts/types/#annotated-types + + * cattrs support for ``Annotated``: + https://cattrs.readthedocs.io/en/latest/validation.html#annotated + + * msgspec support for ``Annotated``: + https://jcristharif.com/msgspec/supported-types.html#annotated + + * SQLAlchemy 2.0 support for ``Annotated``: + https://docs.sqlalchemy.org/en/20/orm/declarative_tables.html#using-annotated-declarative-table-type-annotated-forms-for-mapped-column + + * Typer support for ``Annotated``: + https://typer.tiangolo.com/tutorial/parameter-types/annotated/ + + * Beartype support for ``Annotated`` (Validators): + https://beartype.readthedocs.io/en/latest/api_vale/ + +.. [2] Examples of users and framework authors citing ``Annotated`` verbosity: + + * *"Annotated syntax is too long: Introduction of Annotated params made + function params more logical, but on the other hand longer/more verbose"* — + Vitaliy Kucheryaviy, author of Django Ninja: + https://github.com/tiangolo/fastapi/discussions/10055#discussion-5507018 + + * *"I personally find this solution [using Annotated] a bit tedious when + you start having a lot of models/fields"* — g0di, Pydantic user: + https://github.com/pydantic/pydantic/discussions/2419#discussioncomment-7228409 + + * SQLAlchemy 2.0 Migration Guide (advocating Annotated aliases to + mitigate verbosity): + https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#step-five-make-use-of-pep-593-annotated-to-package-common-directives-into-types + +.. [3] *"My main concern here is that Annotated[torch.Tensor, dtype] is quite verbose, and seems to go in the opposite direction to where we'd like to end up"* — Ralf Gommers, NumPy maintainer: https://github.com/pytorch/pytorch/issues/98702#issuecomment-1504794519 -.. [13] *"I'm not writing something stupidly verbose like: TensorBatchXChannels + +.. [4] *"I'm not writing something stupidly verbose like: TensorBatchXChannels = Annotated[...]"* — Patrick Kidger, author of TorchTyping: https://github.com/beartype/beartype/discussions/96#discussioncomment-2245014 -.. [14] 2023 Python Discourse discussion proposing ``@`` as an alternative to + +.. [5] *"Functions where the arguments have type annotations can already be + rather long, and Annotated on its own is rather verbose, so I’m generally + glad it’s rare"* — Paul Moore on PEP 727: + https://discuss.python.org/t/32566/17 + +.. [6] 2023 Python Discourse discussion proposing ``@`` as an alternative to Annotated: https://discuss.python.org/t/40751 -.. [15] 2025 Python Discourse discussion converging on dedicated ``@`` syntax: + +.. [7] 2025 Python Discourse discussion converging on dedicated ``@`` syntax: https://discuss.python.org/t/103699 -.. [16] *"That by itself doesn’t seem a big objection – type annotations reuse + +.. [8] *"That by itself doesn’t seem a big objection – type annotations reuse all kinds of operations, including x[y] and x | y."* — Guido van Rossum, on repurposing the @ operator for typing: https://discuss.python.org/t/40751/3