HTTP server that exposes stores, arrays, groups - #3732
Merged
Conversation
github-actions
Bot
added
the
needs release notes
d-v-b
marked this pull request as ready for review
March 1, 2026 20:10
d-v-b
commented
Mar 1, 2026
| PEP_723_REGEX: Final = r"(?m)^# /// (?P<type>[a-zA-Z0-9-]+)$\s(?P<content>(^#(| .*)$\s)+)^# ///$" | ||
|
|
||
| # This is the absolute path to the local Zarr installation. Moving this test to a different directory will break it. | ||
| ZARR_PROJECT_PATH = Path(".").absolute() |
Contributor
Author
There was a problem hiding this comment.
changes in this file are simplifications to our examples testing infrastructure. Instead of re-writing the script header, we just override the declared zarr dep in the invocation of uv run ...
github-actions
Bot
removed
the
needs release notes
d-v-b
commented
Mar 1, 2026
| @@ -0,0 +1,178 @@ | |||
| """Utilities for determining the set of valid store keys for zarr nodes. | |||
Contributor
Author
There was a problem hiding this comment.
we eventually need to find a more natural place for this code. I'm not sure which module it should live in.
Contributor
Author
Didn't end up doing these things. We can add them later if people are interested. |
Contributor
Author
|
here's a demo of this functionality: |
Contributor
|
Does this really need to belong in the core python library? Is there any advantage to experimenting with it in this repository? |
Contributor
Author
IMO yes.
|
Contributor
Author
|
and putting this in |
Contributor
Author
|
another important use case: in zarr-python today, if you create a custom store, there is currently no way in zarr python to expose that store as a writable endpoint to a zarr-aware client. This PR enables this functionality. |
|
Love this, as someone who regularly uses @manzt https://github.com/manzt/simple-zarr-server |
Contributor
Author
|
thanks @psobolewskiPhD, given your experience with other tools let me know if there are any features missing from this PR and I can add them. |
Contributor
I don't think this isn't a good argument. many zarr datasets are too big to compute on; should we vendor cubed/dask too? That said, i don't plan on helping maintain it so ... no skin off my back hehe |
Contributor
Author
I think visualizing zarr data is far more basic than doing distributed compute on it. It's very common to use data visualization as a basic sanity check when reading or writing data. And the server is less than 500 lines of code. I have not checked but I suspect this is a bit smaller than dask or cubed. |
|
Very cool! Works quite nicely! then this works from my CLI: (napari readers use extensions) What's cool is this worked with tifffile as_zarr! |
Contributor
Author
Good idea, I can add this |
|
I did run into one issues, maybe PBCAK, but when the remote file being served was zarr2, if I didn't specify zarr_version=2 in my local (client) zarr.open things didn't work -- i couldn't get the arrays from a group. I happened to know it was zarr2, but in principle that might not be the case? |
|
Tiled's Zarr integration may also be of interest to folks here. This PR has a more targeted scope, so while there is overlap they aren't doing exactly the same thing. |
Contributor
Author
very cool project! Let us know if there's anything the library here could add that would make your life easier. You might also benefit from some of the pure metadata types defined in zarr-metadata. |
ruff 0.16 selects BLE001 under the root config's `B` prefix, so the package's `uvx ruff check .` job failed on two deliberate blind excepts. Both are intentional and already documented, so they get targeted noqa comments, matching how zarr-developers#4213 handled the same rule in the core tree. Assisted-by: ClaudeCode:claude-opus-5
The DefaultChunkKeyEncoding.decode_chunk_key fix this fragment described was split out into zarr-developers#4219 and has already shipped -- its text is in docs/release-notes.md verbatim. Leaving the fragment here would emit the same paragraph a second time under a zarr-developers#3732 link, for a change this branch no longer contains. Assisted-by: ClaudeCode:claude-opus-5
github-actions
Bot
added
the
needs release notes
check_changelogs.yml validated the root, zarr-metadata and zarr-indexing changes/ directories but not zarr-http-server's, so the new package's fragments were the only ones whose filenames went unchecked. The two new workflows were also written before zarr-developers#4241 bumped the actions group, so they pinned older checkout/setup-uv/attest/pypi-publish SHAs than their siblings. Bump them to the versions main already uses; the release workflow is now identical to zarr-metadata's modulo the package name. Assisted-by: ClaudeCode:claude-opus-5
Brings the package in line with zarr-metadata and zarr-indexing, which each own a justfile and a separate Read the Docs site. The docs content is a scaffold -- an overview page and an API reference over the public namespace -- meant to be filled in later; the point is to get the site wired up and building under --strict now. The `docs` dependency group previously held the runtime deps for the README examples, but Read the Docs and `just docs-check` both expect it to carry the mkdocs toolchain, as it does in the sibling packages. Those example deps move to a new `examples` group and the test job follows. CI gains a `docs` job matching the siblings', so a scaffold that stops building fails the gate. The repo-root .readthedocs.yaml skips PR builds confined to this package now that it has its own site, and the root docs nav links out to it. Creating the Read the Docs project itself is a manual step: point its configuration-file path at packages/zarr-http-server/.readthedocs.yaml. Assisted-by: ClaudeCode:claude-opus-5
Two of these answered a request successfully while leaving the client with data that does not exist. Chunk keys were validated by decoding coordinates and bounds-checking them, never by re-encoding. `int` is lenient in ways a store key is not -- leading zeros, a leading `+`/`-`, surrounding whitespace, underscore separators, non-ASCII decimal digits -- so `c/00/00` decoded to (0, 0) and validated, then went to the store verbatim. A PUT answered 204 and stored the body under a key no reader looks up: success reported, data invisible. Validation now requires the key to equal `metadata.encode_chunk_key(coords)`, which makes the accepted set exactly the set zarr can read. Decoding delegates to the encoding's own decoder rather than reimplementing the default/v2 grammars, so a new or third-party chunk key encoding works without changes here. Resolving a group child caught bare `Exception` and returned False, which turned an unreadable child -- EACCES, EIO, a corrupt metadata document, a missing codec plugin -- into 404. Under the v3 spec an absent chunk is an uninitialized one, so a correct reader answers that 404 by substituting the fill value over data that exists. Only KeyError is caught now; a key that could not be judged surfaces as 5xx rather than being reported absent. The rest are conformance fixes on the same request path: - A Range header the server cannot use is now ignored with a 200 rather than refused with a 416, per RFC 9110 §14.2. This covers an unrecognized unit and a multi-range request, both legal to send. - A suffix range resolves against the object's size, so a 206 always carries the Content-Range that RFC 9110 §15.3.7 requires. Sharding reads a shard index this way, so the header was missing on a hot path. - A last-byte-pos wider than the store can materialize is clamped to the end of the object per §14.1.2 instead of raising out of the store as a 500. - A byte position is parsed as 1*DIGIT rather than by `int`, which accepted `+0`, ` 0` and `0_0`. - PUT to a read-only store answers 403 instead of letting the store's ValueError surface as a 500. Assisted-by: ClaudeCode:claude-opus-5
Adds hypothesis properties that drive a real uvicorn server over a socket, so the assertions cover what only exists on the wire: header parsing, method dispatch, status codes. Each property checks the response *and* the backing store. That pairing is what the previous suite could not do: a PUT to a non-canonical chunk key answered 204 and wrote a key no reader consults, which a response-only assertion cannot see. Refused requests assert the store is byte-for-byte unchanged; accepted writes assert the bytes landed under the key the client named and that a zarr client reads back the values. Keys are generated in two families -- in-band (the node's metadata and the canonical spelling of each chunk key in its grid) and out-of-band (non-canonical spellings, out-of-grid coordinates, traversal probes, a sibling node's keys). Two details worth keeping: - Traversal probes are percent-encoded. An HTTP client resolves dot-segments before sending, so httpx turns "../secret" into "/secret" and a literal probe asserts nothing; encoded, it reaches the server and Starlette decodes it back into a real ".." segment. - The matrix includes LocalStore, not just MemoryStore. MemoryStore slices a `bytes` and accepts any range bound, so it cannot distinguish a clamped over-wide range from a refused one -- mutation-testing the suite showed that property passing against deliberately broken code until a filesystem-backed server was added. Verified by reverting each fix in turn and confirming the corresponding property fails. Assisted-by: ClaudeCode:claude-opus-5
`_names_nothing` reclassified two errnos as absence so a client could not turn a freely chosen key into a 5xx. `ENAMETOOLONG` earns that: it is the store answering about the name -- nothing can be stored under a name it cannot express -- and `encode_chunk_key` never produces a segment near a filesystem's length limit, so it is unreachable for real data. `EINVAL` does not. It is POSIX's catch-all, reachable on a perfectly ordinary short key through a bad seek or an unsupported filesystem feature, and under the v3 spec an absent chunk is an uninitialized one -- so answering 404 has a correct reader write fill values over a chunk that exists but could not be read. That is the same defect already fixed in the group-child lookup and in chunk key validation: reporting "something went wrong" as "it is not there". Nothing exercised EINVAL in practice. A NUL in a key raises ValueError and is rejected before the store anyway, and an over-long key raises ENAMETOOLONG, so this narrows the guard to the case that was doing the work. The traversal, absolute-key and drive-letter guards deliberately stay where they are rather than deferring to the store. LocalStore.get/set are `self.root / key` with no validation -- zarr's normalize_path applies at the StorePath layer, not to raw store keys -- so `../sibling.txt` and an absolute key both write outside the store root. This server is the component that feeds a Store unvalidated strings from the network, so it is the component that has to reject them. Assisted-by: ClaudeCode:claude-opus-5
Wrapping an API means taking responsibility for its parameters, not hiding the ones we did not think to name. Two wrappers were doing the latter. `CorsOptions` carried 2 of `CORSMiddleware`'s 8 parameters, so `allow_headers`, `allow_credentials`, `allow_origin_regex`, `allow_private_network`, `expose_headers` and `max_age` were unreachable without bypassing this package. It now mirrors the full signature, with every key optional. Two of the defaults are ours rather than Starlette's, because the server knows what its caller should not have to. It emits `Content-Range` on every ranged response, which is not a CORS-safelisted response header, so `expose_headers` defaults to `["Content-Range"]` -- without it a browser client could read the bytes but not learn which bytes it got, which made the suffix-range Content-Range fix invisible to exactly the clients CORS exists for. It accepts a `Range` request header, so `allow_headers` defaults to `["Range"]`; Starlette's empty default answered a preflight naming Range with 400. Defaults apply only to absent keys, so an explicit `expose_headers: []` means "expose nothing". `_start_server` passed 4 of `uvicorn.Config`'s 52 parameters, which put TLS, `proxy_headers`/`forwarded_allow_ips`, `root_path`, `log_level`, `limit_concurrency` and unix-socket binds out of reach entirely. A `uvicorn_options: Mapping[str, object] | None` is merged over the three options set here, so a caller key wins. uvicorn ships no TypedDict for Config -- its only TypedDicts are ASGI protocol events -- so the mapping is hand-typed and the cast is confined to the call site. Un-sealing uvicorn makes two BackgroundServer attributes reachable that could previously only be one thing. `url` now reports the scheme actually in use, so configuring TLS yields https, and `host`/`port`/`url` are None for a uds or fd bind rather than naming an address nothing is listening on. Assisted-by: ClaudeCode:claude-opus-5
Clears the findings left open from the review. Lifecycle. `_start_server` raised on its startup timeout without ever signalling the server, so the thread went on to bind the port and serve forever as a daemon with no handle to stop it -- and a retry on the same port then failed with the other error. It now sets should_exit and force_exit and joins before raising. `shutdown()` returned normally when the thread survived both joins, reporting success for a server still bound and still serving; it now raises. The first join also matched uvicorn's own `timeout_graceful_shutdown` exactly, and uvicorn spends ~0.2s tearing down before that wait even begins, so the join always expired first and escalated to force_exit on the orderly path -- which makes uvicorn skip ASGI lifespan shutdown. The join now outlasts the graceful bound by a margin, and reads that bound from the config so it stays correct when a caller sets it through uvicorn_options. HEAD. A HEAD body is discarded at the wire, but HEAD fell through to the GET handler, so answering one transferred the whole value: measured at 10 MB read to report a length. It is now answered from `Store.getsize` -- a stat on a filesystem store, an info call on a remote one -- and reads zero bytes. HEAD is served whenever GET is, which is what Starlette does and what RFC 9110 asks of an origin server; the README and docstrings said otherwise and now say so. CORS. `cors_options["allow_methods"]` was passed through unchecked, so an app could advertise methods its route rejects: a browser caches that preflight and every later cross-origin call fails with 405 after a successful handshake. Advertising an unserved method is now a ValueError at construction, consistent with how unsupported `methods` are already rejected, and `"*"` expands to what is actually served rather than to every verb Starlette knows. An absent `allow_methods` is left alone -- widening it to everything served would newly advertise PUT cross-origin on a write-enabled app that never asked for it. Media type. The JSON content type was keyed off a third hardcoded "zarr.json", so a v2 array's `.zarray` was served as octet-stream. It is now derived from the same tables that decide which keys a node owns. Also documents that `store_app` does not validate keys: it proxies the raw key space and has no array semantics to check against, so a client that misspells a chunk key gets a successful write to a key no reader consults. `node_app` rejects that with 404. Assisted-by: ClaudeCode:claude-opus-5
The workflow repeated the commands the justfile already defines -- `uvx ruff check .` and the mypy invocation were byte-identical copies, and the pytest step differed only by the sync that precedes it. Two definitions of the same verb drift silently: renaming the `docs` dependency group to `examples` required the same edit in both places, and updating only one would have left `just check` and CI testing different things with nothing failing. CI now calls `just test`, `just lint` and `just typecheck` (it already called `just docs-check`), keeping the python matrix and caching, which are genuinely CI's concern. This matches zarr-metadata, whose workflow already states the arrangement; zarr-indexing remains half-converted. Delegating also meant fixing what the shared recipe would otherwise spread: `just lint` ran an unpinned `uvx ruff`, which is precisely how this job broke before -- ruff 0.16 began selecting BLE001 under the root config's `B` prefix and failed on rules the pre-commit-pinned ruff never enforced, with no code change to blame. The recipe now pins the same version .pre-commit-config.yaml does, so the local gate, the pre-commit gate and CI enforce one standard. Assisted-by: ClaudeCode:claude-opus-5
A dead cross-reference or a nav entry pointing at a removed file only fails at `mkdocs build --strict`, which until now happened first in CI. This catches it before the code leaves the machine. Scoped deliberately. `stages: [pre-push]` overrides the repo default of running on every commit: this builds the whole site, which is too slow to pay per commit and is only actionable before pushing. `files:` limits it to changes that touch the package, and `pass_filenames: false` because mkdocs builds a site rather than a list of files. It delegates to `just docs-check` so the build has one definition shared with CI, and is added to `ci.skip` alongside mypy for the same reason that one is skipped: pre-commit.ci's runners have neither `uv` nor the repo checkout needed to resolve the environment. The zarr-http-server workflow covers it there. Note this hook and CI still declare their toolchains separately -- the hook shells out to the local `just`/`uv`, CI installs them itself. That is inherent to pre-commit.ci not being able to run them, and is the same trade already accepted for mypy. Also refreshes packages/zarr-http-server/uv.lock, which references the root project's dependency groups, for the hypothesis and uv bumps that arrived with the main merge. Assisted-by: ClaudeCode:claude-opus-5
Read-only was already the default -- `store_app(store)` answers 405 to PUT, POST, DELETE and PATCH, and POST is unconfigurable because there is no handler behavior for it -- but nothing said so and little pinned it. Only PUT was covered against the default app; POST, DELETE and PATCH were covered only against a fixture built with writes enabled, so "the default app is read-only" was not actually a tested claim. Adds a test class covering both layers the guarantee rests on: `methods`, which decides what the route answers, and the store, which decides whether a write could succeed at all. Each refusal also asserts the value is unchanged, matching the property tests -- a 405 that still wrote would otherwise pass. Serving PUT from a read-only store is now a ValueError at construction. A store's `read_only` is fixed when it is built, so that combination can never succeed; it previously surfaced as a 403 to whichever client tried to write first, long after whoever misconfigured it had moved on. The handler's 403 stays as a backstop for a store whose read_only is not fixed, and the test for it builds the app through the private builder since the public entry points now reject the combination. Documents `store.with_read_only(True)` as the categorical recipe: it is the stronger of the two layers because it holds even if the HTTP layer is misconfigured. Assisted-by: ClaudeCode:claude-opus-5
`READ_ONLY_METHODS` and `READ_WRITE_METHODS` let a call site say which it is, rather than leaving that to the presence or absence of an argument. The read-only one is exactly the default, so passing it changes nothing except that the intent is written down. The value is the other direction: a writable app must name a method set, so `grep -r 'methods='` finds every place that opts into writes -- which is what makes a deployment auditable without a separate read-only entry point. Both are frozensets, so one caller cannot widen the default for every other, and both name HEAD explicitly: Starlette serves it wherever GET goes, and a constant that omitted it would misdescribe the route. `methods` now accepts any `AbstractSet`, which is what lets a frozenset constant be passed where a `set` was previously required. Assisted-by: ClaudeCode:claude-opus-5
Renames the constants to READ_ONLY_HTTP_METHODS / READ_WRITE_HTTP_METHODS so they say what kind of method they hold, matching the HTTPMethod type they are drawn from. Adds `ReadOnlyHTTPMethod = Literal["GET", "HEAD"]`, which moves the distinction from a runtime convention to something a checker enforces: a `frozenset[ReadOnlyHTTPMethod]` cannot contain "PUT", so a read-only interface can be declared rather than merely configured. Verified against mypy --strict -- assigning either a set containing "PUT" or READ_WRITE_HTTP_METHODS to that annotation is an error, while READ_ONLY_HTTP_METHODS is accepted. HTTPMethod is now the union of that and a private `_WriteHTTPMethod` rather than a third hand-written list of the same strings, and both constants plus _SUPPORTED_METHODS are derived from the Literals via get_args. The runtime sets and the static types therefore cannot disagree about what this server serves: widening a Literal is the only edit needed, and a test pins the contents so that widening is deliberate. Assisted-by: ClaudeCode:claude-opus-5
…ests The only example used `with serve_node(...)`, which is the one form that cannot work in a notebook: it shuts the server down when the cell ends, so anyone copying it gets a dead server by the next cell. Nothing in the package mentioned notebooks at all. Adds examples/serve_notebook.ipynb covering the lifecycle a kernel needs -- start with background=True and keep the handle, use it across cells, then shutdown() -- plus metadata and chunk reads, a byte range, and a refused PUT. Two arguments carry it: background=True runs uvicorn in a daemon thread with its own loop so the kernel's loop is untouched, and port=0 means re-running a start cell picks a new port instead of failing with "address already in use". A README section says the same in prose. The notebook is executed by the suite through nbclient, in a real kernel, and asserts its own expectations, so a behavior change fails there rather than in someone's notebook. Verified by mutation: making writes the default breaks the notebook's `assert refused.status_code == 405` and surfaces as a CellExecutionError naming the cell. examples/serve.py is now executed too, which required fixing the same fixed-port footgun the notebook section warns about -- it bound 8000, so it failed if anything else held that port. It runs in-process rather than under `uv run`, because its inline script metadata resolves zarr-http-server from git and would test main instead of the working tree. Assisted-by: ClaudeCode:claude-opus-5
Serving two nodes did not need two servers, but the only way to run several was to reach past this package: `serve_store`/`serve_node` each take exactly one store or node, so a composed app had no route to the background-server ergonomics -- `port=0` into `server.url`, and `shutdown()` -- only a blocking `uvicorn.run`. `_start_server` already did this for any Starlette app; it was just private. `serve(app, ...)` makes it public, with the same background/blocking overloads the shorthands have. `serve_store` and `serve_node` now delegate to it and stay, because they are the common case and are what the docs and examples use; retiring them is still available later. The split the pair muddles is now visible: what an app *serves* (`methods`, `cors_options`, `max_body_size`) is settled when the app is built, and `serve` only decides how it runs. Documents the three ways to serve several nodes -- serve their common parent group, serve the whole store, or mount separate apps and run the result -- with tests covering mounted nodes in *separate* stores, that each mount serves only its own data, and that `serve` runs the composed app in the background. Also derives the bounded-shutdown test's threshold from the timeouts that produce it. It hard-coded 3.0s, which was generous when shutdown could take at most 2x shutdown_timeout and marginal once the join margin was added -- it began failing under load rather than at the moment the constant changed. Assisted-by: ClaudeCode:claude-opus-5
`background: bool` decided whether a call returns immediately with a handle or never returns at all -- the largest difference a call site can have, hidden in a keyword. The return type depended on it too, which is why every runner carried three @overload stanzas: nine in total, all of them working around that one flag, and `background=False` returned None, a value meaningless half the time. `serve(app)` now blocks and `serve_background(app)` returns a BackgroundServer. Neither needs an overload. Splitting forced the shorthand question, since the axes multiply: keeping serve_store/serve_node alongside two modes means six runner functions. The public surface is instead two builders and two runners -- `serve_background(store_app(store))` replaces `serve_store(store, background=True)`. That is one more call, and it puts the two halves where they belong: what an app serves is settled when it is built, and the runner only decides how it runs. _serve.py drops from 1301 to ~1100 lines with the duplicated signatures and docstrings gone. `serve_background` defaults to `port=0` where `serve` defaults to 8000. Deliberate: a background server is reached through `server.url`, and a fixed default makes starting a second one -- or re-running a notebook cell -- fail on a collision, while a blocking server usually wants a port others already know. BREAKING CHANGE: serve_store and serve_node are removed. The package is unreleased, so nothing depends on them yet. Assisted-by: ClaudeCode:claude-opus-5
`serve` defaulted to 8000 and `serve_background` to 0, which read as an arbitrary disagreement between two sibling functions about a shared parameter. Both now default to `"auto"`: prefer 8000, fall back to any free port if it is taken, and report the result through `server.url` and uvicorn's own startup line. What makes the fallback safe is that it applies only to the default. An explicit port still binds exactly that or fails, because a caller who names one usually has a proxy or a container port mapping expecting the server there -- silently moving would break it while looking healthy. `port=0` keeps its OS meaning of "any free port, no preference". The port is bound here and the socket handed to `Server.run(sockets=...)` rather than probing for a free port and passing uvicorn the number: probing releases the port before uvicorn claims it, which is the bind-then-close race that makes "find a free port" helpers flaky. Holding the socket means nothing can take it in between. `Config.port` is set to what was actually bound, so uvicorn's "running on ..." line does not name a port it is not serving. Two cases the mechanism has to respect: a `uds` or `fd` bind in uvicorn_options skips the TCP bind entirely, and the address family comes from `getaddrinfo` rather than a hard-coded AF_INET, which would bind the wrong family for an IPv6 host. Both are covered by tests, as is that an explicit taken port still raises. Assisted-by: ClaudeCode:claude-opus-5
The PR should not reach outside packages/zarr-http-server and .github, and four files did. Two were unrelated churn: docs/api/zarr/experimental.md renamed a heading in the *core* zarr docs about zarr.experimental.cache_store, left over from when this server lived at zarr.experimental.serve, and uv.lock carried an idna bump nothing here asked for. Reverted; `uv lock --check` is clean. Two were premature rather than wrong. mkdocs.yml added a nav link to zarr-http-server.readthedocs.io and .readthedocs.yaml skipped the repo-root docs build for changes confined to this package. Both belong with a Read the Docs project that does not exist yet -- until it does, the nav link 404s and the build skip means neither site builds the package's docs. mkdocs.yml is also what GitHub reported a conflict on, which is what surfaced this. They should land in the follow-up that creates the RTD project. What remains outside the package is two root files that have nowhere else to live: the pre-push docs hook, since a pre-commit hook is necessarily repo-level, and a comment-only change to pyproject.toml noting that the release workflow's `zarr_http_server-v*` tags are among those the `git describe --match v*` filter exists to exclude. Assisted-by: ClaudeCode:claude-opus-5
Contributor
Author
|
this is moved a lot since I opened the PR, so I'm going to post a quick summary of where we landed: What's addedA new What it doesThe core functionality is a Starlette ASGI app that exposes a store, array, or group over HTTP. HTTP methods Public API
|
Contributor
Author
|
this is getting self-merged today when tests are green. thanks all for the input! I'll post an update when the library is live on pypi |
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds an experimental http server in
experimental.serve. This server can expose stores over http. It can also expose arrays and groups over http. Exposing a store means exposing the entire key: value space of the store. Exposing an array means only exposing the metadata + chunks. Exposing a group means only exposing sub-groups and sub-arrays. See #3731 for more on this distinction.The server is an optional dependency implemented via starlette. It handles byte-range reads and other http methods. CORS headers and allowed methods can be configured. I'm considering handling prefix requests like
foo/barby returning a simple HTML document that lists the visible keys underfoo/bar/, for user-friendliness and to aid httpstore readers that use such responses for listing contents. I'd also like to implement convenience functions for kicking off a server from jupyter and a CLI.Opening as a draft while I work on this.
Edit: this has evolved substantially. see the latest update here: #3732 (comment)