feat(7sense): real-time animal sound identification and acoustic visualization

[ruvnet]

Context

The vibecast-7sense example advertises real-time species identification and a manifold visualization. An audit of the code found neither exists, and several documented entry points resolve to nothing:

• No audio capture dependency anywhere; ingestion is ingest_file(&Path) decoding the whole file into a Vec<f32>. The AudioStream::new() in the crate README is not a real type.
• /ws/stream is documented in two READMEs but no such route is registered, and the WebSocket receive task discards every inbound frame.
• No classifier, label set, or logits head exists. Perch 2.0 as integrated emits embeddings only, so POST /api/identify could not have been served.
• No frontend source of any kind. The sevensense-viz crate that ADR-009 specifies was never created, and the API's umap_url points at an unregistered route.
• sevensense-api declares all seven sibling crates as dependencies and imports none of them, reimplementing each stage as a stub that returns zero vectors and empty segments.

This PR closes those gaps. It is being built incrementally; the checklist below tracks progress.

Architecture decisions

Four ADRs record the design before the code lands.

ADR-010 — Real-time streaming ingestion. An AudioSource trait keeps cpal behind a non-default feature so WASM keeps building. A lock-free ring buffer with an explicit lossy-overwrite policy, because blocking a real-time audio callback causes device glitches and stale audio is worth less than current audio. An EMA noise floor with open/close hysteresis replaces global statistics, so a segment can close before the recording ends.

ADR-011 — Acoustic feature extraction. Descriptors are computed from a linear-frequency power spectrum rather than the existing mel path, because log-scaled perceptually-spaced bins do not yield a centroid in Hz.

ADR-012 — Manifold projection. Supersedes ADR-009's technology choices. PCA via randomized SVD is the default rather than UMAP: it is deterministic, so saved viewports stay valid, and it extends to new points in one matrix multiply, which is what streaming needs. UMAP remains opt-in, implemented over the existing HNSW graph rather than via the unmaintained umap-rs.

ADR-013 — Retrieval-based identification. k-NN against a labelled reference index instead of a classifier. Open-set by construction, so unknown species are detectable rather than forced into the nearest class; extensible by adding reference recordings rather than retraining; and every result carries the neighbours that produced it, which is what makes it auditable.

Progress

• ADR-010, ADR-011, ADR-012, ADR-013
• sevensense-audio::features — interpretable acoustic descriptors (ADR-011)
• Streaming ingestion pipeline (ADR-010)
• PCA/UMAP/Poincaré projection (ADR-012)
• Retrieval-based identification (ADR-013)
• Wire sevensense-api to the real crates; add /ws/stream, /projection, /features
• Web UI — 3D manifold explorer with live feature panels
• Benchmarks and optimization pass

Changes so far

sevensense-audio::features computes spectral centroid, spread, skewness, rolloff, flatness, tonality, crest, entropy, slope, and a parabolically-interpolated dominant frequency per frame, plus amplitude and frequency modulation across frames. This populates CallSegment::spectral_centroid — a field whose builder previously had no callers anywhere in the workspace, leaving it permanently None.

Three details worth review:

• Flatness uses exp(mean(ln p)) rather than a direct product. The product of ~1000 bin powers underflows f32 immediately.
• Summary statistics are computed over voiced frames only. Averaging the centroid across silence pulls it toward the noise floor, which is the most common way this kind of summary becomes meaningless.
• Modulation returns None below 32 frames rather than a fabricated rate. Its search band is capped at envelope Nyquist — 50 Hz at the default 100 fps hop, which corrects an overstatement in the first draft of ADR-011 that the implementation disproved.

Testing

23 new tests, each driven by a synthetic signal whose descriptors are known analytically, so a failure points at the maths rather than at a fixture: a 4 kHz tone must yield a 4 kHz centroid; a 5010 Hz tone must resolve to better than half a bin width; a 20 Hz amplitude-modulated carrier must report a 20 Hz rate; a 2→8 kHz sweep must show wide FM extent. Pathological inputs (DC, alternating full-scale, single impulse) are asserted finite across every descriptor.

Full crate suite: 59 passing, 0 failing.

cargo test -p sevensense-audio

---

Generated by Claude Code