From e4d82483b598e8662558d08e3d19994efe9af326 Mon Sep 17 00:00:00 2001
From: jonathan <webmaster@jonathan859.com>
Date: Wed, 10 Jun 2026 16:21:37 +0200
Subject: [PATCH] docs: code map, regression findings, and HTTP API reference

CLAUDE.md gains a function-level code map of web.py/webui.html, a
"verifying changes" section (test scope, no-node UI checks, endpoint
smoke pattern, Windows commit-message gotcha), and findings recorded as
guard rails: why detection must stay adaptive (fixed threshold produced
~600 useless sections/day), why section playback must stay clip-based
(libsndfile FLACs have no SEEKTABLE so browser seeks bisect the file),
and the five places analysis params are coupled.

README gains the adaptive-detection and clip-review features up top, an
HTTP API table for scripting, and a corrected Docker analyses-cache
paragraph (the rw mount is ./recordings/analyses layered over the ro
recordings mount, not a separate ./analyses dir).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---
 CLAUDE.md | 30 ++++++++++++++++++++++++++----
 README.md | 27 ++++++++++++++++++++++++---
 2 files changed, 50 insertions(+), 7 deletions(-)

diff --git a/CLAUDE.md b/CLAUDE.md
index 0e33679..f2dae96 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -26,7 +26,28 @@ python -m pytest tests/           # test suite
 docker compose up -d / down       # web UI mapped to host port 8050
 ```
 
-Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC/waveform analysis — both optional, code degrades gracefully).
+Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC analysis/clips — both optional, code degrades gracefully).
+
+## Code map
+
+`web.py`:
+- Detection: `_compute_rms_windows_wav()` / `analyze_flac()` produce 100 ms RMS windows → `_noise_floor_db()` estimates the rolling floor → `_loud_sections()` emits scored sections → `_package_result()` shapes the `/api/analyze` payload.
+- Clips: `_api_clip()` validates params, `_clip_wav()` / `_clip_flac()` stream the decoded slice, `_wav_header()` builds the 44-byte PCM header.
+- Live headers: `_live_wav_header()`, `_live_flac_header()` (+ `_flac_frame_samples()`, CRC-8 verified).
+- Serving: `_stream()` (Range support), `_copy_to_response()`, `_safe_path()` (path traversal guard).
+
+`webui.html` (one `<script>` block):
+- Clip review: `clipQueue`/`clipCursor` globals, `playClip()`, `playFileSection()`, `hideClipBar()`; markup is the `#clip-bar` div.
+- Day review: `dayHighlights()` builds `dayActiveSections` (chronological); `jumpToDaySection()` arms the queue.
+- J/K: single document-level `keydown` listener — clip queue takes priority, in-player `currentTime` stepping is the fallback when no queue is armed.
+- Analysis: `fetchAnalysis()` (session `analysisCache`), `analyse()` (per-row render), `cachedParamsMatch()` (autoload guard).
+
+## Verifying changes
+
+- `python -m pytest tests/` covers the recorder (`test_isr.py`) and the detector (`test_web.py`).
+- There is no JS toolchain and no `node` on the dev box. After editing `webui.html`, cross-check every `getElementById('x')` against an `id="x"` declaration, and smoke-test endpoints.
+- Endpoint smoke pattern: write a temp WAV/FLAC with a known loud burst, subclass `web._Handler` with `recordings_dir`/`analyses_dir` pointing at the temp dir, serve `web._Server(('127.0.0.1', 0), H)` in a daemon thread, then hit `/api/analyze` and `/api/clip` with urllib — assert section start/score and that `Content-Length == len(body) == 44 + frames × channels × 2`.
+- Dev box is Windows / PowerShell 5.1. Multi-line commit messages: use the Bash tool with `git commit -F - <<'EOF'` — PowerShell here-strings containing quotes get mangled into separate arguments.
 
 ## Non-obvious internals
 
@@ -35,10 +56,11 @@ Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC/
 - **Split timing:** files split at clock-aligned boundaries (`get_next_split_time()`), e.g. `split_minutes = 60` → on the hour.
 - **ALSA:** capture spawns `arecord` as a subprocess, raw PCM read in 100 ms chunks by a thread. Device spec resolution: `default` → exact `hw:X,Y` → partial name → fallback to any literal ALSA PCM name (so `shared_mic` from asound.conf works without appearing in `arecord -l`).
 - **Shutdown:** SIGTERM is converted to KeyboardInterrupt in `main()`; `RecorderManager.stop()` joins all threads against a single shared 25 s deadline to stay inside Docker's `stop_grace_period: 30s`.
-- **Loud-section detection is adaptive:** per-window dB is compared against a rolling noise floor (`NOISE_PERCENTILE`-th percentile per `NOISE_BLOCK_SECONDS` block, min-smoothed over ±2 blocks so events can't raise their own floor; clamped to ≥ `MIN_RMS`). A section needs `margin` dB of prominence and carries a `score` (peak dB above floor) used for ranking. Tests in `tests/test_web.py`.
-- **Analysis cache:** results stored as `<analyses-dir>/<file>.analysis.json` keyed by margin+min_gap; orphans pruned at web startup. In Docker the recordings mount is **read-only** for the web container, so the cache uses a separate `./analyses` bind mount. The `margin` and `min_gap` keys MUST stay first in the cache JSON — `_cached_analysis_params()` reads only the first 256 bytes to avoid parsing the large embedded result. Old `threshold`-keyed caches never match and get overwritten on the next analyse.
+- **Loud-section detection is adaptive — do not regress it to an absolute threshold.** Per-window dB is compared against a rolling noise floor (`NOISE_PERCENTILE`-th percentile per `NOISE_BLOCK_SECONDS` block, min-smoothed over ±2 blocks so events can't raise their own floor; clamped to ≥ `MIN_RMS`). A section needs `margin` dB of prominence and carries a `score` (peak dB above floor) used for ranking. The original fixed RMS threshold flagged every ambience change (passing cars, rain) and produced ~600 useless sections/day — that is why it was replaced. Known limitation: a short (~10 s) swell on a quiet street still flags because the floor blocks are 30 s; the planned fix is an onset/spectral filter or optional Silero VAD, **not** a higher margin. Tests in `tests/test_web.py`.
+- **Analysis params are coupled in five places.** CLI `--margin`/`--min-gap` → `/api/config` → UI inputs `#margin-input`/`#min-gap-input` → `/api/analyze` query params → cache JSON head keys. Renaming or adding a param means touching all five plus `cachedParamsMatch()` (see the threshold→margin change, commit `c84b7d8`).
+- **Analysis cache:** results stored as `<analyses-dir>/<file>.analysis.json` keyed by margin+min_gap; orphans pruned at web startup. In Docker the recordings mount is **read-only** for the web container, so docker-compose layers a read-write `./recordings/analyses` bind mount over it. The `margin` and `min_gap` keys MUST stay first in the cache JSON — `_cached_analysis_params()` reads only the first 256 bytes to avoid parsing the large embedded result. Old `threshold`-keyed caches never match and get overwritten on the next analyse.
 - **Analyze responses:** `/api/analyze` returns `rms_display` (~800 points), never the full per-window RMS list — the UI doesn't use it and it is ~45x larger.
-- **Section playback uses clips, not seeks:** `/api/clip?file&start&end` decodes the slice server-side (wave/soundfile) and returns a standalone 16-bit WAV with exact Content-Length (capped at `CLIP_MAX_SECONDS`). The UI plays chips/J-K through the bottom clip bar (`clipQueue` in webui.html); seeking the full file only happens via "Open in file". Rationale: our FLACs have no SEEKTABLE, so browser seeks bisect the whole file with Range requests.
+- **Section playback uses clips, not seeks:** `/api/clip?file&start&end` decodes the slice server-side (wave/soundfile) and returns a standalone 16-bit WAV with exact Content-Length (capped at `CLIP_MAX_SECONDS`), `Cache-Control: private` so re-listening is free. The UI plays chips/J-K through the bottom clip bar (`clipQueue` in webui.html); seeking the full file only happens via "Open in file". Rationale (finding): libsndfile writes FLAC **without a SEEKTABLE**, so a browser seek bisects the whole multi-hundred-MB file with Range requests — seeking big FLACs in `<audio>` is inherently slow and must not be reintroduced as the primary navigation. Server-side `sf.SoundFile.seek()` on local disk is fast and frame-accurate.
 - **HTTP/1.1 keep-alive:** `_Handler.protocol_version = 'HTTP/1.1'`; every response path must set an accurate `Content-Length`. `_copy_to_response()` force-closes the connection if it under-delivers (file truncated mid-serve).
 - **Live playback:** for files listed in status.json, `/stream/` patches the header on the fly so the browser sees the duration recorded so far and can seek; responses get `Cache-Control: no-store`. WAV: `_live_wav_header` derives sizes from the byte count. FLAC: `_live_flac_header` parses the sample count out of the last frame header in the file tail (CRC-8-verified to reject false sync matches) and rewrites STREAMINFO total_samples — duration is NOT derivable from byte size for FLAC.
 - **Path safety:** every file parameter in `web.py` goes through `_safe_path()`, which resolves and verifies the path stays inside the recordings dir.
diff --git a/README.md b/README.md
index d2a0004..c270156 100644
--- a/README.md
+++ b/README.md
@@ -13,7 +13,9 @@ Records from multiple simultaneous sources — Icecast/HTTP streams and ALSA sou
 - OGG / Opus / FLAC header injection so every split file is independently playable
 - Auto-reconnect on stream drops or device errors
 - WAV and FLAC output for soundcard sources
-- Web UI to browse, download, and analyse recordings
+- Web UI to browse, play, cut, and download recordings — including files still being written
+- **Loud-event detection** — adaptive: events are flagged by how far they rise above the rolling background noise, not by absolute level, and ranked by score
+- **Clip review** — detected sections play as instant server-rendered clips with auto-advance, so a whole day can be reviewed like a highlights reel
 
 ---
 
@@ -21,7 +23,7 @@ Records from multiple simultaneous sources — Icecast/HTTP streams and ALSA sou
 
 ```bash
 pip install requests              # stream recording
-pip install numpy soundfile       # FLAC output + web waveform analysis (optional)
+pip install numpy soundfile       # FLAC output + web analysis/clips for FLAC (optional)
 
 cp config.example.ini config.ini
 # edit config.ini to add your sources
@@ -173,6 +175,25 @@ Shows recordings grouped by day with collapsible sections. Features:
 - **Fast loading** — analysis results are cached server-side on disk and client-side per session; cached waveforms load only for expanded day groups, and collapsed days fetch nothing until opened.
 - **WCAG-compliant** — skip link, `aria-expanded`/`aria-controls` on the player toggle, `aria-live` status, focus management, `role=img` on SVG waveforms.
 
+### HTTP API
+
+Everything the UI does goes through these endpoints, so they can also be scripted:
+
+| Endpoint | Description |
+|----------|-------------|
+| `GET /api/files` | File listing with size, mtime, duration, recording state, cached-analysis params |
+| `GET /api/analyze?file=&margin=&min_gap=` | Loud-section analysis: `rms_display` (~800-point waveform), scored `sections`, `duration` |
+| `GET /api/clip?file=&start=&end=` | Section of a WAV/FLAC decoded server-side, returned as a standalone WAV (max 600 s) |
+| `GET /api/cut?file=&start=&end=` | ffmpeg-trimmed copy of the file as a download |
+| `GET /stream/<name>` | Inline playback with HTTP Range support; live files get an on-the-fly patched header |
+| `GET /download/<name>` | Raw file download |
+| `GET /api/status` | Currently recording files (`status.json` passthrough) |
+| `GET /api/storage` | Disk free/total |
+| `GET /api/config` | Server-side defaults for margin and min-gap (seeds the UI controls) |
+| `DELETE /api/files/<name>` | Delete a recording and its analysis cache |
+
+Analysis, clips, cut, and delete return `409` for files that are still being recorded.
+
 ---
 
 ## How it works
@@ -216,7 +237,7 @@ docker compose down && docker compose up -d --build
 
 **Log file in Docker:** The recorder always logs to stdout, so `docker compose logs -f` shows live output. To persist logs on the host, set `log_file = /app/recordings/recorder.log` in `config.ini` (the `recordings` directory is the bind mount).
 
-**Analysis cache in Docker:** The web container mounts `./recordings` read-only, so analysis cache files are written to a separate `./analyses` bind mount (mapped to `/analyses` inside the container). This directory is created automatically by Docker Compose on first run. Cache files are stored as `analyses/<filename>.analysis.json` on the host.
+**Analysis cache in Docker:** The web container mounts `./recordings` read-only; a separate read-write bind mount of `./recordings/analyses` is layered on top so analysis cache files can still be written. The directory is created automatically by Docker Compose on first run. Cache files are stored as `recordings/analyses/<filename>.analysis.json` on the host.
 
 **File retention:** Individual recordings can be deleted from the web UI. For bulk / automated cleanup, add a cron job on the host:
 ```bash