From 119e631faf4c7fddc0f116f993f6429c02047e5f Mon Sep 17 00:00:00 2001 From: jonathan Date: Wed, 10 Jun 2026 16:13:39 +0200 Subject: [PATCH] feat: instant section playback via server-rendered clips Add /api/clip: decodes a WAV/FLAC slice server-side and returns a small standalone 16-bit WAV with exact Content-Length (capped at 600s, cached client-side since finished recordings are immutable). Active recordings are refused like analyse/cut/delete. Section chips and J/K now play these clips through a bottom player bar instead of seeking the full recording - FLACs have no seek table, so browser seeks bisected hundreds of MB with Range requests and playback lagged or never started. The bar steps through a queue (one file's sections or a whole day's via Highlights), auto-advances to the next section on end for continuous review, and "Open in file" jumps to the same position in the full recording for context. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 1 + README.md | 2 +- web.py | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++ webui.html | 122 ++++++++++++++++++++++++++++++++++++++++++---------- 4 files changed, 224 insertions(+), 24 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 24d039a..0e33679 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -38,6 +38,7 @@ Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC/ - **Loud-section detection is adaptive:** per-window dB is compared against a rolling noise floor (`NOISE_PERCENTILE`-th percentile per `NOISE_BLOCK_SECONDS` block, min-smoothed over ±2 blocks so events can't raise their own floor; clamped to ≥ `MIN_RMS`). A section needs `margin` dB of prominence and carries a `score` (peak dB above floor) used for ranking. Tests in `tests/test_web.py`. - **Analysis cache:** results stored as `/.analysis.json` keyed by margin+min_gap; orphans pruned at web startup. In Docker the recordings mount is **read-only** for the web container, so the cache uses a separate `./analyses` bind mount. The `margin` and `min_gap` keys MUST stay first in the cache JSON — `_cached_analysis_params()` reads only the first 256 bytes to avoid parsing the large embedded result. Old `threshold`-keyed caches never match and get overwritten on the next analyse. - **Analyze responses:** `/api/analyze` returns `rms_display` (~800 points), never the full per-window RMS list — the UI doesn't use it and it is ~45x larger. +- **Section playback uses clips, not seeks:** `/api/clip?file&start&end` decodes the slice server-side (wave/soundfile) and returns a standalone 16-bit WAV with exact Content-Length (capped at `CLIP_MAX_SECONDS`). The UI plays chips/J-K through the bottom clip bar (`clipQueue` in webui.html); seeking the full file only happens via "Open in file". Rationale: our FLACs have no SEEKTABLE, so browser seeks bisect the whole file with Range requests. - **HTTP/1.1 keep-alive:** `_Handler.protocol_version = 'HTTP/1.1'`; every response path must set an accurate `Content-Length`. `_copy_to_response()` force-closes the connection if it under-delivers (file truncated mid-serve). - **Live playback:** for files listed in status.json, `/stream/` patches the header on the fly so the browser sees the duration recorded so far and can seek; responses get `Cache-Control: no-store`. WAV: `_live_wav_header` derives sizes from the byte count. FLAC: `_live_flac_header` parses the sample count out of the last frame header in the file tail (CRC-8-verified to reject false sync matches) and rewrites STREAMINFO total_samples — duration is NOT derivable from byte size for FLAC. - **Path safety:** every file parameter in `web.py` goes through `_safe_path()`, which resolves and verifies the path stays inside the recordings dir. diff --git a/README.md b/README.md index e9914da..d2a0004 100644 --- a/README.md +++ b/README.md @@ -164,7 +164,7 @@ Shows recordings grouped by day with collapsible sections. Features: - **Inline playback** — collapsible `▶ Play` button per row; audio loads lazily via a seekable `/stream/` endpoint with HTTP Range support. Metadata is fetched immediately so the duration is visible without pressing play. - **Waveform analysis** — on demand per file; computes RMS per 100 ms window and marks sections that stand out above the background. Detection is **adaptive**: a rolling noise floor (20th percentile per 30 s block) is estimated across the file, and a section is flagged when the level rises at least *margin* dB (default 12) above that floor. Slow ambience changes — rain setting in, day/night traffic hum — move the floor instead of producing false positives. Each section gets a **score** (its peak dB above the floor) used to rank sections by how much they stand out. Supported for WAV and FLAC (FLAC requires `numpy` + `soundfile`). Pure-Python fallback for WAV when numpy is absent. Results are cached in `recordings/analyses/.analysis.json`; subsequent requests at the same margin and min-gap settings return instantly without re-reading the audio. The cache file is deleted automatically when the audio file is deleted. Orphaned cache files (audio deleted outside the UI) are pruned on startup. - **Grace period** — configurable in the controls bar (default 2 s). Loud sections separated by less than this gap are merged into one. Raise this (e.g. to 15–30 s) when a single event generates many timestamps due to brief quiet gaps within it. -- **Timestamp jump** — after analysis, click any loud-section chip to seek the player to that position and pre-fill the cut panel. Use **J** / **K** keyboard shortcuts to jump to the previous / next section while audio is playing. +- **Clip playback** — clicking a loud-section chip plays a short server-rendered WAV clip (`/api/clip`, pre-roll included) in a player bar at the bottom of the page. Playback starts instantly even for sections deep inside multi-hundred-MB FLACs, because the browser never has to seek the full file. **J** / **K** (or ⏮ / ⏭) step through the queued sections — one file's, or a whole day's after ★ Highlights — and **Auto-advance** plays the next section when one ends, turning a day's detections into a continuous review reel. **⤴ Open in file** switches to the full recording at the same position for context; each chip click also pre-fills the cut panel. - **Cut & download** — `✂ Cut` button opens the player row and reveals a cut panel. Enter start and end times in `m:ss` or `h:mm:ss` format and click **↓ Download cut** to receive an ffmpeg-trimmed copy without re-encoding. Requires ffmpeg (included in the Docker image). - **Filters** — live filename search and from/to date pickers above the table; applied client-side with no additional requests. Shows `N of M shown` when a filter is active. - **Delete** — `✕ Delete` button per row with confirmation prompt; disabled for files currently being recorded; sends `DELETE /api/files/` and re-renders the table. diff --git a/web.py b/web.py index 4fb74c6..f98692e 100644 --- a/web.py +++ b/web.py @@ -55,6 +55,8 @@ NOISE_PERCENTILE = 20 # percentile of windowed dB levels taken as the floor MIN_RMS = 0.002 # ≈ −54 dBFS; the floor never drops below this, so # digital silence does not make every tiny sound loud +CLIP_MAX_SECONDS = 600 # upper bound on /api/clip length + MIME_TYPES = { '.wav': 'audio/wav', '.mp3': 'audio/mpeg', @@ -217,6 +219,17 @@ def _live_flac_header(path: Path, size: int): return None +def _wav_header(n_frames: int, channels: int, framerate: int, sampwidth: int) -> bytes: + """Standard 44-byte PCM WAV header for a clip of known length.""" + data_len = n_frames * channels * sampwidth + byte_rate = framerate * channels * sampwidth + return (b'RIFF' + (36 + data_len).to_bytes(4, 'little') + b'WAVE' + + b'fmt ' + (16).to_bytes(4, 'little') + + struct.pack(' start') + return + end = min(end, start + CLIP_MAX_SECONDS) + + path = self._safe_path(filename) + if path is None: + return + if self._is_active(filename): + self._json_err(409, 'File is currently being recorded — clips unavailable until recording stops') + return + + ext = path.suffix.lower() + try: + if ext == '.wav': + self._clip_wav(path, start, end) + elif ext == '.flac': + if not (NUMPY_AVAILABLE and SOUNDFILE_AVAILABLE): + self._json_err(400, 'FLAC clips require: pip install numpy soundfile') + return + self._clip_flac(path, start, end) + else: + self._json_err(400, f'Clips are not available for {ext} files') + except (ConnectionError, BrokenPipeError): + raise + except Exception as e: + self._json_err(500, f'Clip failed: {e}') + + def _send_clip_headers(self, header: bytes, n_frames: int, channels: int, + sampwidth: int): + self.send_response(200) + self.send_header('Content-Type', 'audio/wav') + self.send_header('Content-Length', str(len(header) + n_frames * channels * sampwidth)) + # Finished recordings are immutable, so stepping back to a section + # replays the clip from the browser cache + self.send_header('Cache-Control', 'private, max-age=86400') + self.end_headers() + self.wfile.write(header) + + def _clip_wav(self, path: Path, start: float, end: float): + with wave.open(str(path), 'rb') as wf: + channels = wf.getnchannels() + sampwidth = wf.getsampwidth() + framerate = wf.getframerate() + n_frames = wf.getnframes() + f0 = min(int(start * framerate), n_frames) + f1 = min(int(end * framerate), n_frames) + if f1 <= f0: + self._json_err(400, 'clip range is beyond the end of the file') + return + + header = _wav_header(f1 - f0, channels, framerate, sampwidth) + self._send_clip_headers(header, f1 - f0, channels, sampwidth) + + wf.setpos(f0) + remaining = f1 - f0 + while remaining > 0: + chunk = wf.readframes(min(32768, remaining)) + if not chunk: + self.close_connection = True # under-delivered + break + self.wfile.write(chunk) + remaining -= len(chunk) // (channels * sampwidth) + + def _clip_flac(self, path: Path, start: float, end: float): + with sf.SoundFile(path) as f: + framerate = f.samplerate + channels = f.channels + n_frames = len(f) + f0 = min(int(start * framerate), n_frames) + f1 = min(int(end * framerate), n_frames) + if f1 <= f0: + self._json_err(400, 'clip range is beyond the end of the file') + return + + header = _wav_header(f1 - f0, channels, framerate, 2) + self._send_clip_headers(header, f1 - f0, channels, 2) + + f.seek(f0) + remaining = f1 - f0 + while remaining > 0: + frames = f.read(min(32768, remaining), dtype='int16', always_2d=True) + if len(frames) == 0: + self.close_connection = True # under-delivered + break + self.wfile.write(frames.tobytes()) + remaining -= len(frames) + def _is_active(self, filename: str) -> bool: """True if isr.py reports this file as currently being recorded.""" try: diff --git a/webui.html b/webui.html index b0eadf9..e0023e4 100644 --- a/webui.html +++ b/webui.html @@ -121,6 +121,16 @@ table.day-table{width:100%;border-collapse:collapse;border:1px solid var(--brd); svg.day-timeline{display:block;width:100%;height:22px} .day-tl-labels{display:flex;justify-content:space-between;font-size:10px; color:var(--muted);font-family:ui-monospace,monospace;margin-top:2px} +/* clip player bar */ +#clip-bar{position:fixed;bottom:0;left:0;right:0;z-index:20;background:var(--surf); + border-top:1px solid var(--brd);padding:8px 28px;display:flex;align-items:center; + gap:10px;flex-wrap:wrap} +#clip-bar audio{flex:1 1 240px;min-width:180px;height:32px} +#clip-label{font-size:12px;color:var(--muted);font-family:ui-monospace,monospace; + white-space:nowrap;overflow:hidden;text-overflow:ellipsis;max-width:38%} +#clip-auto-label{font-size:12px;color:var(--muted);display:flex;align-items:center; + gap:4px;white-space:nowrap;cursor:pointer} +body.clip-open{padding-bottom:70px} @@ -159,6 +169,15 @@ svg.day-timeline{display:block;width:100%;height:22px}
+