Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
ddb7b27
implement store server and node server
d-v-b Feb 28, 2026
38e3b90
Merge branch 'main' of https://github.com/zarr-developers/zarr-python…
d-v-b Mar 1, 2026
8bec48d
update tests and add v2 -> v3 example
d-v-b Mar 1, 2026
a3d0358
add __all__ and clean up tests
d-v-b Mar 1, 2026
71e303f
add docs and changelog
d-v-b Mar 1, 2026
64de16f
add proper server
d-v-b Mar 2, 2026
e40b20b
rework examples (simplify) and make server a context manager
d-v-b Mar 2, 2026
e167ad3
minor tweaks to server
d-v-b Mar 2, 2026
6f983e5
Merge branch 'main' into feat/experimental-server
d-v-b Mar 3, 2026
35f7ad2
Merge branch 'main' into feat/experimental-server
d-v-b Mar 8, 2026
9bfe855
fix: byte-order handling for structured dtypes in the bytes codec (#220)
d-v-b Jul 14, 2026
16b4ff2
Merge branch 'main' of https://github.com/zarr-developers/zarr-python
d-v-b Jul 14, 2026
273dd73
Merge branch 'main' of https://github.com/zarr-developers/zarr-python
d-v-b Jul 17, 2026
df56f7e
Merge branch 'main' of https://github.com/zarr-developers/zarr-python
d-v-b Jul 20, 2026
f73ef8c
Merge branch 'main' of https://github.com/zarr-developers/zarr-python
d-v-b Jul 27, 2026
62756ce
Merge remote-tracking branch 'origin/main' into feat/experimental-server
d-v-b Jul 28, 2026
97aebb0
fix: adapt server branch to current starlette/uvicorn
d-v-b Jul 28, 2026
036156f
Merge branch 'main' into feat/experimental-server
d-v-b Jul 28, 2026
d72b472
feat(zarr-server): scaffold packages/zarr-server
d-v-b Jul 28, 2026
800e6bd
feat(zarr-server): move server code, tests, and example into the package
d-v-b Jul 28, 2026
88c9884
docs(zarr-server): package README; retarget root changelog fragment a…
d-v-b Jul 28, 2026
50bb7b1
refactor!: remove the HTTP server from zarr core
d-v-b Jul 28, 2026
aa33508
ci(zarr-server): add package test and release workflows
d-v-b Jul 28, 2026
3c19122
chore(zarr-server): refresh package lockfile after root server-extra …
d-v-b Jul 28, 2026
2e589b5
fix: reject path-traversal segments in zarr-server request handling
d-v-b Jul 28, 2026
35dd0e1
fix(zarr-server): close absolute-key path-traversal bypass in request…
d-v-b Jul 28, 2026
c4f061c
Merge remote-tracking branch 'origin/feat/experimental-server' into f…
d-v-b Jul 28, 2026
1074284
ci(zarr-server): bump actions/attest to v4.2.0 to match sibling workflow
d-v-b Jul 28, 2026
a31521e
fix(zarr-server): fold backslash separators and reject drive-qualifie…
d-v-b Jul 28, 2026
8ee276b
fix(zarr-server): fix vacuous two-level traversal test
d-v-b Jul 28, 2026
90d15d7
fix(zarr-server): bound BackgroundServer shutdown instead of blocking…
d-v-b Jul 28, 2026
3eb111f
fix(zarr-server)!: only accept HTTP methods the handler implements
d-v-b Jul 28, 2026
a2616a7
docs(zarr-server): document client deps; lint the package's changelog…
d-v-b Jul 28, 2026
760600e
test(zarr-server): add docs dependency group and cover the README rou…
d-v-b Jul 28, 2026
d41e796
refactor(zarr-http-server)!: rename zarr-server to zarr-http-server
d-v-b Jul 28, 2026
f037043
fix(zarr-http-server): reject an empty methods set instead of failing…
d-v-b Jul 28, 2026
6031b7c
fix(zarr-http-server): report the bound port, unblock the loop, 404 u…
d-v-b Jul 28, 2026
3107350
fix(zarr-http-server): close the S5-S13 follow-ups from the adversari…
d-v-b Jul 28, 2026
6f676bc
fix(zarr-http-server): stop reporting I/O failures as misses; enforce…
d-v-b Jul 29, 2026
472ab7b
Merge branch 'main' into feat/experimental-server
d-v-b Aug 11, 2026
964df39
Merge remote-tracking branch 'upstream/main' into claude/fix-pr-3732-…
d-v-b Aug 11, 2026
88093ab
fix(ci): silence BLE001 on two intentional broad excepts
d-v-b Aug 11, 2026
64cf353
chore: drop stale changelog fragment for the chunk key fix
d-v-b Aug 11, 2026
7efd250
ci: check zarr-http-server changelogs and align action pins
d-v-b Aug 11, 2026
c817bf3
docs(http-server): add justfile and Read the Docs scaffold
d-v-b Aug 11, 2026
f0494ed
fix(http-server): correct silent-corruption and HTTP-conformance defects
d-v-b Aug 11, 2026
7ecc919
test(http-server): property tests over a real endpoint
d-v-b Aug 11, 2026
b2c30b3
fix(http-server): stop reading EINVAL from a store as a missing key
d-v-b Aug 11, 2026
76b979e
feat(http-server): stop sealing over CORSMiddleware and uvicorn.Config
d-v-b Aug 11, 2026
a81a063
fix(http-server): lifecycle, HEAD cost, and CORS/route method agreement
d-v-b Aug 11, 2026
596066a
Merge branch 'main' into feat/experimental-server
d-v-b Aug 12, 2026
7582948
ci(http-server): run the justfile's recipes instead of copies of them
d-v-b Aug 12, 2026
5156f49
ci: build the zarr-http-server docs on pre-push
d-v-b Aug 12, 2026
ef87731
feat(http-server): make the read-only guarantee explicit and enforced
d-v-b Aug 12, 2026
a3529d6
feat(http-server): name the read-only and read-write method sets
d-v-b Aug 12, 2026
1a28fe5
feat(http-server): model read-only methods in the type domain
d-v-b Aug 12, 2026
ec30fbc
docs(http-server): add a notebook example, and run both examples in t…
d-v-b Aug 12, 2026
a82ba86
feat(http-server): expose serve() for arbitrary ASGI apps
d-v-b Aug 12, 2026
b1f0c15
refactor(http-server)!: split blocking and background into two functions
d-v-b Aug 12, 2026
d27e557
feat(http-server): default both runners to port="auto"
d-v-b Aug 12, 2026
ad48839
revert: drop changes to files this package does not own
d-v-b Aug 12, 2026
429efc1
Merge branch 'main' into feat/experimental-server
d-v-b Aug 12, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/check_changelogs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,6 @@ jobs:

- name: Check zarr-indexing changelog entries
run: uv run --no-sync python ci/check_changelog_entries.py packages/zarr-indexing/changes

- name: Check zarr-http-server changelog entries
run: uv run --no-sync python ci/check_changelog_entries.py packages/zarr-http-server/changes
117 changes: 117 additions & 0 deletions .github/workflows/zarr-http-server-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
name: zarr-http-server release

on:
workflow_dispatch:
push:
tags:
- 'zarr_http_server-v*'

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: false

jobs:
build:
name: Build wheel and sdist
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: packages/zarr-http-server
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
fetch-depth: 0 # hatch-vcs needs full history + tags

- name: Install Hatch
uses: pypa/hatch@257e27e51a6a5616ed08a39a408a21c35c9931bc
with:
version: '1.16.5'

- name: Build
run: hatch build

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: zarr-http-server-dist
path: packages/zarr-http-server/dist

test_artifacts:
name: Test built artifacts
needs: [build]
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: zarr-http-server-dist
path: dist

- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: false

- name: Set up Python
run: uv python install 3.12

- name: Install built wheel and run import smoke test
run: |
wheel=$(ls dist/*.whl)
uv run --with "${wheel}" --python 3.12 --no-project \
python -c "import zarr_http_server; print('zarr_http_server', zarr_http_server.__version__)"

upload_pypi:
name: Upload to PyPI
needs: [build, test_artifacts]
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/zarr_http_server-v')
runs-on: ubuntu-latest
environment:
name: zarr-http-server-releases
url: https://pypi.org/p/zarr-http-server
permissions:
id-token: write # required for OIDC trusted publishing
attestations: write # required for artifact attestations
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: zarr-http-server-dist
path: dist

- name: Generate artifact attestation
uses: actions/attest@508db95dd578ae2727ebd6217d5ba78e4fbda05d # v4.2.1
with:
subject-path: dist/*

- name: Publish package to PyPI
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2

upload_testpypi:
name: Upload to TestPyPI
needs: [build, test_artifacts]
if: github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
environment:
name: zarr-http-server-releases-test
url: https://test.pypi.org/p/zarr-http-server
permissions:
id-token: write
attestations: write
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: zarr-http-server-dist
path: dist

- name: Generate artifact attestation
uses: actions/attest@508db95dd578ae2727ebd6217d5ba78e4fbda05d # v4.2.1
with:
subject-path: dist/*

- name: Publish package to TestPyPI
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2
with:
repository-url: https://test.pypi.org/legacy/
139 changes: 139 additions & 0 deletions .github/workflows/zarr-http-server.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
name: zarr-http-server

# Job steps delegate to packages/zarr-http-server/justfile, the single source
# of truth for this package's verbs; CI owns only the python matrix and
# caching. Keeping the commands in one place is what makes `just check`
# locally mean the same thing as a green run here.

on:
push:
branches: [main]
paths:
- 'packages/zarr-http-server/**'
- '.github/workflows/zarr-http-server.yml'
# The package resolves zarr from the repo root for its own tests, so a
# core change can break it. Run this suite when core changes too.
- 'src/zarr/**'
pull_request:
paths:
- 'packages/zarr-http-server/**'
- '.github/workflows/zarr-http-server.yml'
# The package resolves zarr from the repo root for its own tests, so a
# core change can break it. Run this suite when core changes too.
- 'src/zarr/**'
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
name: pytest py=${{ matrix.python-version }}
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: packages/zarr-http-server
strategy:
fail-fast: false
matrix:
python-version: ['3.12', '3.13', '3.14']
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true
- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}
- name: Install just
uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
- name: Sync test dependency groups
# The examples group carries the deps the README examples need, so the
# test that reads a served array back with a zarr client runs here
# instead of silently skipping.
run: uv sync --group test --group examples --python ${{ matrix.python-version }}
- name: Run pytest
run: just test

ruff:
name: ruff
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: packages/zarr-http-server
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Install just
uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
- name: Run ruff
run: just lint

mypy:
name: mypy
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: packages/zarr-http-server
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true
- name: Set up Python
run: uv python install 3.12
- name: Install just
uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
- name: Sync test dependency group
run: uv sync --group test --python 3.12
- name: Run mypy
run: just typecheck

docs:
name: docs
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: packages/zarr-http-server
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Install uv
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
with:
enable-cache: true
- name: Install just
uses: extractions/setup-just@53165ef7e734c5c07cb06b3c8e7b647c5aa16db3 # v4
- name: Build docs
run: just docs-check

zarr-http-server-complete:
name: zarr-http-server complete
needs: [test, ruff, mypy, docs]
if: always()
runs-on: ubuntu-latest
steps:
- name: Check failure
if: |
contains(needs.*.result, 'failure') ||
contains(needs.*.result, 'cancelled')
run: exit 1
- name: Success
run: echo Success!
31 changes: 26 additions & 5 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@ ci:
autoupdate_commit_msg: "chore: update pre-commit hooks"
autoupdate_schedule: "monthly"
autofix_prs: false
# mypy runs as a `language: system` hook via `uv run mypy`, which needs `uv`
# and the repo checkout to resolve the dev environment from `uv.lock` —
# unavailable on pre-commit.ci's runners. It is covered instead by the Lint
# GitHub Actions workflow and by local prek runs.
skip: [mypy]
# Both of these are `language: system` hooks that shell out to the local
# toolchain — `uv` for mypy, `uv` and `just` for the docs build — and need
# the repo checkout to resolve their environments from `uv.lock`. Neither is
# available on pre-commit.ci's runners. Each is covered instead by a GitHub
# Actions job (Lint for mypy, zarr-http-server for the docs build) and by
# local prek runs.
skip: [mypy, zarr-http-server-docs]

default_stages: [pre-commit, pre-push]

Expand Down Expand Up @@ -50,6 +52,25 @@ repos:
pass_filenames: false
always_run: true
types_or: [python, pyi]
# Builds the zarr-http-server docs site with warnings as errors, which
# catches a dead cross-reference or a nav entry pointing at a file that
# no longer exists before it reaches CI.
#
# `stages: [pre-push]` overrides the default of running on every commit:
# this is a whole-site build, too slow to pay per commit and only
# actionable before the code leaves the machine. `files:` limits it to
# changes that touch the package, and `pass_filenames: false` because
# mkdocs builds the site, not a list of files. Delegating to the
# justfile keeps one definition of the build shared with CI.
- id: zarr-http-server-docs
name: zarr-http-server docs build
language: system
entry: >-
just --justfile packages/zarr-http-server/justfile
--working-directory packages/zarr-http-server docs-check
pass_filenames: false
files: ^packages/zarr-http-server/
stages: [pre-push]
- repo: https://github.com/scientific-python/cookie
rev: 2026.06.18
hooks:
Expand Down
30 changes: 30 additions & 0 deletions packages/zarr-http-server/.readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Read the Docs configuration for the zarr-http-server docs site, separate from
# the zarr-python site configured by the repo-root .readthedocs.yaml. The RTD
# project for zarr-http-server must set its configuration-file path to
# packages/zarr-http-server/.readthedocs.yaml.
version: 2

build:
os: ubuntu-22.04
tools:
python: "3.12"
jobs:
post_checkout:
# Cancel pull request builds that do not touch this package. Exit code
# 183 cancels the build and reports success to the Git provider. Scoped
# to PR builds ("external" versions) because origin/main is only a
# meaningful diff base there.
- |
if [ "$READTHEDOCS_VERSION_TYPE" = "external" ] && git diff --quiet origin/main -- packages/zarr-http-server;
then
exit 183;
fi
install:
- pip install --upgrade pip
- pip install ./packages/zarr-http-server --group packages/zarr-http-server/pyproject.toml:docs
build:
html:
- mkdocs build --strict -f packages/zarr-http-server/mkdocs.yml --site-dir $READTHEDOCS_OUTPUT/html

mkdocs:
configuration: packages/zarr-http-server/mkdocs.yml
3 changes: 3 additions & 0 deletions packages/zarr-http-server/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Release notes

<!-- towncrier release notes start -->
21 changes: 21 additions & 0 deletions packages/zarr-http-server/LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2015-2025 Zarr Developers <https://github.com/zarr-developers>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Loading
Loading