PEP 764: Inlined typed dictionaries (#4082)

Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Jelle Zijlstra <[email protected]>

Author: 3 people

.github/CODEOWNERS

@@ -642,6 +642,7 @@ peps/pep-0760.rst  @pablogsal @brettcannon
 peps/pep-0761.rst  @sethmlarson @hugovk
 peps/pep-0762.rst  @pablogsal @ambv @lysnikolaou @emilyemorehouse
 peps/pep-0763.rst  @dstufft
+peps/pep-0764.rst  @JelleZijlstra
 peps/pep-0765.rst  @iritkatriel @ncoghlan
 peps/pep-0766.rst  @warsaw
 peps/pep-0767.rst  @carljm

peps/conf.py

@@ -64,17 +64,18 @@
         nitpick_ignore.append(("c:identifier", name))
 del role, name
 
-# Intersphinx configuration
+# Intersphinx configuration (keep this in alphabetical order)
 intersphinx_mapping = {
-    "python": ("https://docs.python.org/3/", None),
-    "packaging": ("https://packaging.python.org/en/latest/", None),
-    "typing": ("https://typing.readthedocs.io/en/latest/", None),
-    "trio": ("https://trio.readthedocs.io/en/latest/", None),
     "devguide": ("https://devguide.python.org/", None),
+    "mypy": ("https://mypy.readthedocs.io/en/latest/", None),
+    "packaging": ("https://packaging.python.org/en/latest/", None),
     "py3.11": ("https://docs.python.org/3.11/", None),
     "py3.12": ("https://docs.python.org/3.12/", None),
     "py3.13": ("https://docs.python.org/3.13/", None),
     "py3.14": ("https://docs.python.org/3.14/", None),
+    "python": ("https://docs.python.org/3/", None),
+    "trio": ("https://trio.readthedocs.io/en/latest/", None),
+    "typing": ("https://typing.readthedocs.io/en/latest/", None),
 }
 intersphinx_disabled_reftypes = []
 

peps/pep-0764.rst

@@ -0,0 +1,334 @@
+PEP: 764
+Title: Inlined typed dictionaries
+Author: Victorien Plot <[email protected]>
+Sponsor: Eric Traut <erictr at microsoft.com>
+Status: Draft
+Type: Standards Track
+Topic: Typing
+Created: 25-Oct-2024
+Python-Version: 3.14
+
+
+Abstract
+========
+
+:pep:`589` defines a :ref:`class-based <typing:typeddict-class-based-syntax>`
+and a :ref:`functional syntax <typing:typeddict-functional-syntax>` to create
+typed dictionaries. In both scenarios, it requires defining a class or
+assigning to a value. In some situations, this can add unnecessary
+boilerplate, especially if the typed dictionary is only used once.
+
+This PEP proposes the addition of a new inlined syntax, by subscripting the
+:class:`~typing.TypedDict` type::
+
+    from typing import TypedDict
+
+    def get_movie() -> TypedDict[{'name': str, 'year': int}]:
+        return {
+            'name': 'Blade Runner',
+            'year': 1982,
+        }
+
+Motivation
+==========
+
+Python dictionaries are an essential data structure of the language. Many
+times, it is used to return or accept structured data in functions. However,
+it can get tedious to define :class:`~typing.TypedDict` classes:
+
+* A typed dictionary requires a name, which might not be relevant.
+* Nested dictionaries require more than one class definition.
+
+Taking a simple function returning some nested structured data as an example::
+
+    from typing import TypedDict
+
+    class ProductionCompany(TypedDict):
+        name: str
+        location: str
+
+    class Movie(TypedDict):
+        name: str
+        year: int
+        production: ProductionCompany
+
+
+    def get_movie() -> Movie:
+        return {
+            'name': 'Blade Runner',
+            'year': 1982,
+            'production': {
+                'name': 'Warner Bros.',
+                'location': 'California',
+            }
+        }
+
+
+Rationale
+=========
+
+The new inlined syntax can be used to resolve these problems::
+
+    def get_movie() -> TypedDict[{'name': str, 'year': int, 'production': TypedDict[{'name': str, 'location': str}]}]:
+        ...
+
+It is recommended to *only* make use of inlined typed dictionaries when the
+structured data isn't too large, as this can quickly become hard to read.
+
+While less useful (as the functional or even the class-based syntax can be
+used), inlined typed dictionaries can be assigned to a variable, as an alias::
+
+    InlinedTD = TypedDict[{'name': str}]
+
+    def get_movie() -> InlinedTD:
+        ...
+
+
+Specification
+=============
+
+The :class:`~typing.TypedDict` class is made subscriptable, and accepts a
+single type argument which must be a :class:`dict`, following the same
+semantics as the :ref:`functional syntax <typing:typeddict-functional-syntax>`
+(the dictionary keys are strings representing the field names, and values are
+valid :ref:`annotation expressions <typing:annotation-expression>`). Only the
+comma-separated list of ``key: value`` pairs within braces constructor
+(``{k: <type>}``) is allowed, and should be specified directly as the type
+argument (i.e. it is not allowed to use a variable which was previously
+assigned a :class:`dict` instance).
+
+Inlined typed dictionaries can be referred to as *anonymous*, meaning they
+don't have a name (see the `runtime behavior <Runtime behavior>`_
+section).
+
+It is possible to define a nested inlined dictionary::
+
+    Movie = TypedDict[{'name': str, 'production': TypedDict[{'location': str}]}]
+
+    # Note that the following is invalid as per the updated `type_expression` grammar:
+    Movie = TypedDict[{'name': str, 'production': {'location': str}}]
+
+Although it is not possible to specify any class arguments such as ``total``,
+any :external+typing:term:`type qualifier` can be used for individual fields::
+
+    Movie = TypedDict[{'name': NotRequired[str], 'year': ReadOnly[int]}]
+
+Inlined typed dictionaries are implicitly *total*, meaning all keys must be
+present. Using the :data:`~typing.Required` type qualifier is thus redundant.
+
+Type variables are allowed in inlined typed dictionaries, provided that they
+are bound to some outer scope::
+
+    class C[T]:
+        inlined_td: TypedDict[{'name': T}]  # OK, `T` is scoped to the class `C`.
+
+    reveal_type(C[int]().inlined_td['name'])  # Revealed type is 'int'
+
+
+    def fn[T](arg: T) -> TypedDict[{'name': T}]: ...  # OK: `T` is scoped to the function `fn`.
+
+    reveal_type(fn('a')['name'])  # Revealed type is 'str'
+
+
+    type InlinedTD[T] = TypedDict[{'name': T}]  # OK, `T` is scoped to the type alias.
+
+
+    T = TypeVar('T')
+
+    InlinedTD = TypedDict[{'name': T}]  # Not OK, `T` refers to a type variable that is not bound to any scope.
+
+Typing specification changes
+----------------------------
+
+The inlined typed dictionary adds a new kind of
+:external+typing:term:`type expression`. As such, the
+:external+typing:token:`~expression-grammar:type_expression` production will
+be updated to include the inlined syntax:
+
+.. productionlist:: inlined-typed-dictionaries-grammar
+    new-type_expression: `~expression-grammar:type_expression`
+                       : | <TypedDict> '[' '{' (string: ':' `~expression-grammar:annotation_expression` ',')* '}' ']'
+                       :       (where string is any string literal)
+
+Runtime behavior
+----------------
+
+Although :class:`~typing.TypedDict` is commonly referred as a class, it is
+implemented as a function at runtime. To be made subscriptable, it will be
+changed to be a class.
+
+Creating an inlined typed dictionary results in a new class, so ``T1`` and
+``T2`` are of the same type::
+
+    from typing import TypedDict
+
+    T1 = TypedDict('T1', {'a': int})
+    T2 = TypedDict[{'a': int}]
+
+As inlined typed dictionaries are are meant to be *anonymous*, their
+:attr:`~type.__name__` attribute will be set to an empty string.
+
+Backwards Compatibility
+=======================
+
+This PEP does not bring any backwards incompatible changes.
+
+
+Security Implications
+=====================
+
+There are no known security consequences arising from this PEP.
+
+
+How to Teach This
+=================
+
+The new inlined syntax will be documented both in the :mod:`typing` module
+documentation and the :ref:`typing specification <typing:typed-dictionaries>`.
+
+As mentioned in the `Rationale`_, it should be mentioned that inlined typed
+dictionaries should be used for small structured data to not hurt readability.
+
+
+Reference Implementation
+========================
+
+Mypy supports a similar syntax as an :option:`experimental feature <mypy:mypy.--enable-incomplete-feature>`::
+
+    def test_values() -> {"int": int, "str": str}:
+        return {"int": 42, "str": "test"}
+
+Pyright added support for the new syntax in version `1.1.387`_.
+
+.. _1.1.387: https://github.com/microsoft/pyright/releases/tag/1.1.387
+
+Runtime implementation
+----------------------
+
+A draft implementation is available `here <https://github.com/Viicos/cpython/commit/49e5a83f>`_.
+
+
+Rejected Ideas
+==============
+
+Using the functional syntax in annotations
+------------------------------------------
+
+The alternative functional syntax could be used as an annotation directly::
+
+    def get_movie() -> TypedDict('Movie', {'title': str}): ...
+
+However, call expressions are currently unsupported in such a context for
+various reasons (expensive to process, evaluating them is not standardized).
+
+This would also require a name which is sometimes not relevant.
+
+Using ``dict`` with a single type argument
+------------------------------------------
+
+We could reuse :class:`dict` with a single type argument to express the same
+concept::
+
+    def get_movie() -> dict[{'title': str}]: ...
+
+While this would avoid having to import :class:`~typing.TypedDict` from
+:mod:`typing`, this solution has several downsides:
+
+* For type checkers, :class:`dict` is a regular class with two type variables.
+  Allowing :class:`dict` to be parametrized with a single type argument would
+  require special casing from type checkers, as there is no way to express
+  parametrization overloads. On the other hand, :class:`~typing.TypedDict` is
+  already a :term:`special form <typing:special form>`.
+
+* If future work extends what inlined typed dictionaries can do, we don't have
+  to worry about impact of sharing the symbol with :class:`dict`.
+
+Using a simple dictionary
+-------------------------
+
+Instead of subscripting the :class:`~typing.TypedDict` class, a plain
+dictionary could be used as an annotation::
+
+    def get_movie() -> {'title': str}: ...
+
+However, :pep:`584` added union operators on dictionaries and :pep:`604`
+introduced :ref:`union types <python:types-union>`. Both features make use of
+the :ref:`bitwise or (|) <python:bitwise>` operator, making the following use
+cases incompatible, especially for runtime introspection::
+
+    # Dictionaries are merged:
+    def fn() -> {'a': int} | {'b': str}: ...
+
+    # Raises a type error at runtime:
+    def fn() -> {'a': int} | int: ...
+
+Open Issues
+===========
+
+Subclassing an inlined typed dictionary
+---------------------------------------
+
+Should we allow the following?::
+
+    from typing import TypedDict
+
+    InlinedTD = TypedDict[{'a': int}]
+
+
+    class SubTD(InlinedTD):
+        pass
+
+What about defining an inlined typed dictionay extending another typed
+dictionary?::
+
+    InlinedBase = TypedDict[{'a': int}]
+
+    Inlined = TypedDict[InlinedBase, {'b': int}]
+
+Using ``typing.Dict`` with a single argument
+--------------------------------------------
+
+While using :class:`dict` isn't ideal, we could make use of
+:class:`typing.Dict` with a single argument::
+
+    def get_movie() -> Dict[{'title': str}]: ...
+
+It is less verbose, doesn't have the baggage of :class:`dict`, and is
+already defined as some kind of special form.
+
+However, it is currently marked as deprecated (although not scheduled for
+removal), so it might be confusing to undeprecate it.
+
+This would also set a precedent on typing constructs being parametrizable
+with a different number of type arguments.
+
+Should inlined typed dictionaries be proper classes?
+----------------------------------------------------
+
+The PEP currently defines inlined typed dictionaries as type objects, to be in
+line with the existing syntaxes. To work around the fact that they don't have
+a name, their :attr:`~type.__name__` attribute is set to an empty string.
+
+This is somewhat arbitrary, and an alternative name could be used as well
+(e.g. ``'<TypedDict>'``).
+
+Alternatively, inlined typed dictionaries could be defined as instances of a
+new (internal) typing class, e.g. :class:`!typing._InlinedTypedDict`. While
+this solves the naming issue, it requires extra logic in the runtime
+implementation to provide the introspection attributes (such as
+:attr:`~typing.TypedDict.__total__`), and tools relying on runtime
+introspection would have to add proper support for this new type.
+
+Inlined typed dictionaries and extra items
+------------------------------------------
+
+:pep:`728` introduces the concept of *closed* type dictionaries. Inlined
+typed dictionaries should probably be implicitly *closed*, but it may be
+better to wait for :pep:`728` to be accepted first.
+
+
+Copyright
+=========
+
+This document is placed in the public domain or under the
+CC0-1.0-Universal license, whichever is more permissive.