From fab9450500ec1af11b6d91db2eda622471a9c0c3 Mon Sep 17 00:00:00 2001 From: Barry Warsaw Date: Mon, 29 Jun 2026 15:05:35 -0700 Subject: [PATCH 1/2] Clarify the atomic publishing requirements * Close some race conditions * Describe conflict resolution and name reservation --- peps/pep-0694.rst | 74 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 73 insertions(+), 1 deletion(-) diff --git a/peps/pep-0694.rst b/peps/pep-0694.rst index fa63fa6b7b9..f1ae18fee9b 100644 --- a/peps/pep-0694.rst +++ b/peps/pep-0694.rst @@ -723,6 +723,66 @@ A publish attempt that fails *synchronously* (i.e. within the publish request it client as an :ref:`error response ` and leaves the session in its current editable state; it does **not** move the session to ``error``. +.. _publishing-session-atomicity: + +Atomic Publication and Conflicts +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Publishing a session is atomic with respect to the release's filename namespace. Because published artifacts +are immutable, the index **MUST** guarantee that it never publishes two files with the same name for the same +release, even in the presence of concurrent uploads arriving through this API or the legacy API. + +The point-in-time conflict check performed when a :ref:`file upload session is created ` +is best-effort: it reflects the published state at the moment of that request and does **not** guarantee the +file will still be conflict-free at publish time, since the published state of the release can change while a +session is open. For example, a file with the same name may be published through the legacy API, or through a +subsequent session for the same, already-published ``name`` and ``version``. The authoritative conflict check +is therefore performed atomically at publish time. + +**Filename reservation.** When a client requests publication, the server **MUST** atomically reserve the +filenames of all files in the session within the target release, and hold that reservation for the duration of +the publish: + +- While the reservation is held, any other attempt to upload a file with one of those filenames to the same + release -- whether through this API or the legacy API -- **MUST** be rejected with a ``409 Conflict``, + exactly as if the file were already published. The index **MUST** enforce this regardless of which upload + path the conflicting request arrives through; this is a requirement on the index's shared published-filename + namespace and does not otherwise modify the legacy API. + +- The reservation governs conflict detection only; the reserved files remain :ref:`staged ` + and **MUST NOT** become publicly visible until the publish succeeds. + +- If the publish succeeds, the reservation converts to permanent publication. If it fails -- synchronously, + leaving the session editable, or during deferred ``processing``, moving it to the ``error`` state -- the + server **MUST** release the reservation, making those filenames available again. + +A still-``open`` session does **not** reserve its filenames; the reservation is acquired only when publication +is requested. For an immediate (``201 Created``) publish the reservation is held only for the duration of the +single request and is effectively unobservable. The requirement is meaningful mainly for deferred (``202 +Accepted`` then ``processing``) publishes, where there is a real window between the index validating one staged +artifact and committing the rest. + +This closes that window at a deliberate, conservative cost: a concurrent upload **MAY** receive a ``409 +Conflict`` during a publish that ultimately fails, and then succeed on retry once the reservation is released. +The index never publishes two files with the same name, at the cost of occasionally rejecting a concurrent +upload that a later retry would allow. + +**Reporting a publish-time conflict.** When the index detects a conflict while publishing -- whether a +filename collision as above, or another precondition that held when the session was created but no longer +holds, such as an exhausted quota or a revoked permission -- it reports the failure according to the +:ref:`publishing session state machine `: + +- If the server is completing the publish synchronously, it **MUST** return an :ref:`error response + ` -- a ``409 Conflict`` for a filename collision -- identifying the conflicting file(s), and + leave the session in its current editable state. + +- If the server accepted the publish for deferred processing and detects the conflict during ``processing``, + it **MUST** move the session to the ``error`` state and report the conflicting file(s) in the session's + ``notices`` (and, where the failure is attributable to a particular file, in that file's ``notices``). + +In either case the client may delete or replace the offending file, or otherwise resolve the conflict, and +publish again, or :ref:`cancel ` the session. + .. _publishing-session-cancellation: Publishing Session Cancellation @@ -884,7 +944,9 @@ A publishing session **MAY** be created for a ``name`` and ``version`` that has example to add wheels for additional platforms to an existing release. However, because published artifacts are immutable, if the ``filename`` in this request matches a file that has already been published for this release, the server **MUST** reject the request with a ``409 Conflict`` and **MUST NOT** overwrite the -published file. +published file. This check is best-effort and reflects the published state at the time of the request; the +authoritative, atomic conflict check is performed at publish time, as described in +:ref:`publishing-session-atomicity`. If the server determines that upload should proceed, it will return a ``202 Accepted`` response, with the response body below. The :ref:`status ` of the publishing session will also @@ -1706,6 +1768,16 @@ as experience is gained operating Upload 2.0. Change History ============== +* TBD + + * Add an **Atomic Publication and Conflicts** section. Specify that publication is atomic with respect to + the release's filename namespace: the server reserves the session's filenames for the duration of a + publish so that concurrent uploads (including through the legacy API) receive a ``409 Conflict``, and + releases the reservation if the publish fails. Clarify that the conflict check at file upload session + creation is best-effort and that the authoritative check happens atomically at publish time, reported + synchronously as a ``409 Conflict`` or, for a deferred publish, by moving the session to the ``error`` + state with the reason in ``notices``. + * `26-Jun-2026 `__ * Session actions now use dedicated endpoint links instead of an ``action`` key in request bodies. From 43052c0ea0446c8fe0ba393d2b2515e54a46007a Mon Sep 17 00:00:00 2001 From: Barry Warsaw Date: Mon, 29 Jun 2026 15:29:07 -0700 Subject: [PATCH 2/2] Session status retention and other clarifications --- peps/pep-0694.rst | 64 ++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 61 insertions(+), 3 deletions(-) diff --git a/peps/pep-0694.rst b/peps/pep-0694.rst index f1ae18fee9b..de05700f085 100644 --- a/peps/pep-0694.rst +++ b/peps/pep-0694.rst @@ -790,9 +790,12 @@ Publishing Session Cancellation To cancel a publishing session, a client issues a ``DELETE`` request to the ``session`` :ref:`link ` given in the :ref:`session creation response body `. -The server then marks the session as ``canceled``, and **SHOULD** purge any data that was uploaded as part of -that session. Future attempts to access that session URL or any of the publishing session URLs **MUST** -return a ``404 Not Found``. +The server then marks the session as ``canceled`` and **SHOULD** purge any data that was uploaded as part of +that session. Once that data is purged, the session's action and data-bearing URLs -- ``links.upload``, +``links.publish``, ``links.extend``, ``links.stage``, and the individual file URLs -- **SHOULD** become +unavailable and **MAY** return ``404 Not Found``. The session status URL (``links.session``) is instead +retained as described in :ref:`publishing-session-retention`: it continues to report the ``canceled`` status +for an index-specific period before it too **MAY** return ``404 Not Found``. Cancellation is only permitted while the session is :ref:`open or in the error state `. If the session is in the ``processing`` state (i.e. because a deferred @@ -820,6 +823,31 @@ The server will respond to this ``GET`` request with the same :ref:`publishing s any changes to ``status``, ``expires-at``, or ``files`` reflected. +.. _publishing-session-retention: + +Session Status Retention +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +A session status URL is not guaranteed to remain valid indefinitely. Once a session reaches a terminal state +-- ``published`` or ``canceled`` -- the server **SHOULD** continue to serve its :ref:`status URL +`, reporting the terminal ``status``, for an index-specific retention period, so +that clients can reliably observe the final outcome of a session they did not watch to completion. After that +retention period elapses, the server **MAY** return ``404 Not Found`` for the status URL. The length of the +retention period is left to the index, but **SHOULD** be long enough to let a client that initiated or +contributed to the session learn its outcome. + +This retention applies only to the session *status*; the data-bearing and action URLs for a terminated session +(for example ``links.stage`` and the individual file URLs) may become unavailable as soon as their underlying +data is no longer needed, as described for :ref:`cancellation `. + +Clients **MUST** be prepared for a session status URL to return ``404 Not Found`` once the session has +terminated and its retention period has elapsed. A ``404`` on a previously valid session status URL is not in +itself an error; clients **SHOULD** treat it as indicating that the session no longer exists. Note that once a +terminated session has been purged, a request to :ref:`create a new session ` for +the same ``name`` and ``version`` will succeed with a ``201 Created`` rather than returning the ``409 +Conflict`` that a still-live session would have produced. + + .. _publishing-session-extension: Publishing Session Extension @@ -1222,6 +1250,15 @@ The client can query status of the file upload session by issuing a ``GET`` requ `. The server responds to this request with the same payload as the file upload session creation response, except with any changes ``status`` and ``expires-at`` reflected. +A file upload session has no existence independent of the publishing session it belongs to, and its status URL +is retained accordingly. While the parent publishing session is in a non-terminal state, the server +**SHOULD** keep each of its file upload session status URLs valid, reporting the file upload session's current +``status`` -- including a terminal ``canceled`` -- so that a client can observe the outcome of any file it +uploaded. Once the parent publishing session itself terminates, its file upload session URLs are retained no +longer than the parent's own :ref:`status URL ` and **MAY** return ``404 Not +Found`` thereafter. As with cancellation, the data-bearing and mechanism portions of these URLs may become +unavailable as soon as the underlying file data is purged, independent of the status URL. + .. _file-upload-session-extension: File Upload Session Extension @@ -1704,6 +1741,16 @@ these constraints upfront is simpler when the project is known at session creati project version. Allowing multiple projects would fundamentally change this model and complicate the definition of what "publish" means for a session. +Even with the single-project restriction, this PEP still improves multi-project releases. A client releasing +several projects at once can fully :ref:`stage ` every project -- creating a publishing +session per project and uploading all of its artifacts -- before a final step runs through and +:ref:`publishes ` each one. This gives the whole set a "stage everything, then +publish" workflow, even though each publish is still per-project and atomic on its own. + +The single-project session also lays a foundation that a future "publish multiple projects" operation could +build on as a separate endpoint, coordinating the publication of several already-staged sessions, without +changing the per-session model defined here. + Why is the version required when creating a publishing session? --------------------------------------------------------------- @@ -1777,6 +1824,17 @@ Change History creation is best-effort and that the authoritative check happens atomically at publish time, reported synchronously as a ``409 Conflict`` or, for a deferred publish, by moving the session to the ``error`` state with the reason in ``notices``. + * Add a **Session Status Retention** section. Specify that after a session reaches a terminal + (``published`` or ``canceled``) state, the server **SHOULD** keep serving its status URL reporting the + terminal status for an index-specific retention period, after which it **MAY** return ``404 Not Found``, + and that clients must be prepared for such a ``404``. Reconcile the cancellation rules accordingly: the + data-bearing and action URLs may become unavailable once purged, while the status URL is retained. + Tie file upload session status URL retention to the parent publishing session: while the parent is + non-terminal the server **SHOULD** keep its file upload session status URLs valid, and once the parent + terminates they are retained no longer than the parent's status URL. + * Expand the "Why is the project name required" FAQ to note that single-project sessions still improve + multi-project releases (all projects can be fully staged before a final step publishes each one) and lay a + foundation for a possible future "publish multiple projects" endpoint. * `26-Jun-2026 `__