feat: instant section playback via server-rendered clips
Add /api/clip: decodes a WAV/FLAC slice server-side and returns a small standalone 16-bit WAV with exact Content-Length (capped at 600s, cached client-side since finished recordings are immutable). Active recordings are refused like analyse/cut/delete. Section chips and J/K now play these clips through a bottom player bar instead of seeking the full recording - FLACs have no seek table, so browser seeks bisected hundreds of MB with Range requests and playback lagged or never started. The bar steps through a queue (one file's sections or a whole day's via Highlights), auto-advances to the next section on end for continuous review, and "Open in file" jumps to the same position in the full recording for context. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -38,6 +38,7 @@ Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC/
|
||||
- **Loud-section detection is adaptive:** per-window dB is compared against a rolling noise floor (`NOISE_PERCENTILE`-th percentile per `NOISE_BLOCK_SECONDS` block, min-smoothed over ±2 blocks so events can't raise their own floor; clamped to ≥ `MIN_RMS`). A section needs `margin` dB of prominence and carries a `score` (peak dB above floor) used for ranking. Tests in `tests/test_web.py`.
|
||||
- **Analysis cache:** results stored as `<analyses-dir>/<file>.analysis.json` keyed by margin+min_gap; orphans pruned at web startup. In Docker the recordings mount is **read-only** for the web container, so the cache uses a separate `./analyses` bind mount. The `margin` and `min_gap` keys MUST stay first in the cache JSON — `_cached_analysis_params()` reads only the first 256 bytes to avoid parsing the large embedded result. Old `threshold`-keyed caches never match and get overwritten on the next analyse.
|
||||
- **Analyze responses:** `/api/analyze` returns `rms_display` (~800 points), never the full per-window RMS list — the UI doesn't use it and it is ~45x larger.
|
||||
- **Section playback uses clips, not seeks:** `/api/clip?file&start&end` decodes the slice server-side (wave/soundfile) and returns a standalone 16-bit WAV with exact Content-Length (capped at `CLIP_MAX_SECONDS`). The UI plays chips/J-K through the bottom clip bar (`clipQueue` in webui.html); seeking the full file only happens via "Open in file". Rationale: our FLACs have no SEEKTABLE, so browser seeks bisect the whole file with Range requests.
|
||||
- **HTTP/1.1 keep-alive:** `_Handler.protocol_version = 'HTTP/1.1'`; every response path must set an accurate `Content-Length`. `_copy_to_response()` force-closes the connection if it under-delivers (file truncated mid-serve).
|
||||
- **Live playback:** for files listed in status.json, `/stream/` patches the header on the fly so the browser sees the duration recorded so far and can seek; responses get `Cache-Control: no-store`. WAV: `_live_wav_header` derives sizes from the byte count. FLAC: `_live_flac_header` parses the sample count out of the last frame header in the file tail (CRC-8-verified to reject false sync matches) and rewrites STREAMINFO total_samples — duration is NOT derivable from byte size for FLAC.
|
||||
- **Path safety:** every file parameter in `web.py` goes through `_safe_path()`, which resolves and verifies the path stays inside the recordings dir.
|
||||
|
||||
@@ -164,7 +164,7 @@ Shows recordings grouped by day with collapsible sections. Features:
|
||||
- **Inline playback** — collapsible `▶ Play` button per row; audio loads lazily via a seekable `/stream/` endpoint with HTTP Range support. Metadata is fetched immediately so the duration is visible without pressing play.
|
||||
- **Waveform analysis** — on demand per file; computes RMS per 100 ms window and marks sections that stand out above the background. Detection is **adaptive**: a rolling noise floor (20th percentile per 30 s block) is estimated across the file, and a section is flagged when the level rises at least *margin* dB (default 12) above that floor. Slow ambience changes — rain setting in, day/night traffic hum — move the floor instead of producing false positives. Each section gets a **score** (its peak dB above the floor) used to rank sections by how much they stand out. Supported for WAV and FLAC (FLAC requires `numpy` + `soundfile`). Pure-Python fallback for WAV when numpy is absent. Results are cached in `recordings/analyses/<filename>.analysis.json`; subsequent requests at the same margin and min-gap settings return instantly without re-reading the audio. The cache file is deleted automatically when the audio file is deleted. Orphaned cache files (audio deleted outside the UI) are pruned on startup.
|
||||
- **Grace period** — configurable in the controls bar (default 2 s). Loud sections separated by less than this gap are merged into one. Raise this (e.g. to 15–30 s) when a single event generates many timestamps due to brief quiet gaps within it.
|
||||
- **Timestamp jump** — after analysis, click any loud-section chip to seek the player to that position and pre-fill the cut panel. Use **J** / **K** keyboard shortcuts to jump to the previous / next section while audio is playing.
|
||||
- **Clip playback** — clicking a loud-section chip plays a short server-rendered WAV clip (`/api/clip`, pre-roll included) in a player bar at the bottom of the page. Playback starts instantly even for sections deep inside multi-hundred-MB FLACs, because the browser never has to seek the full file. **J** / **K** (or ⏮ / ⏭) step through the queued sections — one file's, or a whole day's after ★ Highlights — and **Auto-advance** plays the next section when one ends, turning a day's detections into a continuous review reel. **⤴ Open in file** switches to the full recording at the same position for context; each chip click also pre-fills the cut panel.
|
||||
- **Cut & download** — `✂ Cut` button opens the player row and reveals a cut panel. Enter start and end times in `m:ss` or `h:mm:ss` format and click **↓ Download cut** to receive an ffmpeg-trimmed copy without re-encoding. Requires ffmpeg (included in the Docker image).
|
||||
- **Filters** — live filename search and from/to date pickers above the table; applied client-side with no additional requests. Shows `N of M shown` when a filter is active.
|
||||
- **Delete** — `✕ Delete` button per row with confirmation prompt; disabled for files currently being recorded; sends `DELETE /api/files/<name>` and re-renders the table.
|
||||
|
||||
@@ -55,6 +55,8 @@ NOISE_PERCENTILE = 20 # percentile of windowed dB levels taken as the floor
|
||||
MIN_RMS = 0.002 # ≈ −54 dBFS; the floor never drops below this, so
|
||||
# digital silence does not make every tiny sound loud
|
||||
|
||||
CLIP_MAX_SECONDS = 600 # upper bound on /api/clip length
|
||||
|
||||
MIME_TYPES = {
|
||||
'.wav': 'audio/wav',
|
||||
'.mp3': 'audio/mpeg',
|
||||
@@ -217,6 +219,17 @@ def _live_flac_header(path: Path, size: int):
|
||||
return None
|
||||
|
||||
|
||||
def _wav_header(n_frames: int, channels: int, framerate: int, sampwidth: int) -> bytes:
|
||||
"""Standard 44-byte PCM WAV header for a clip of known length."""
|
||||
data_len = n_frames * channels * sampwidth
|
||||
byte_rate = framerate * channels * sampwidth
|
||||
return (b'RIFF' + (36 + data_len).to_bytes(4, 'little') + b'WAVE'
|
||||
+ b'fmt ' + (16).to_bytes(4, 'little')
|
||||
+ struct.pack('<HHIIHH', 1, channels, framerate, byte_rate,
|
||||
channels * sampwidth, sampwidth * 8)
|
||||
+ b'data' + data_len.to_bytes(4, 'little'))
|
||||
|
||||
|
||||
def _get_audio_duration(path: Path):
|
||||
"""Return duration in seconds for any supported audio file, or None."""
|
||||
ext = path.suffix.lower()
|
||||
@@ -533,6 +546,8 @@ class _Handler(BaseHTTPRequestHandler):
|
||||
self._api_config()
|
||||
elif p == '/api/cut':
|
||||
self._api_cut(qs)
|
||||
elif p == '/api/clip':
|
||||
self._api_clip(qs)
|
||||
elif p.startswith('/download/'):
|
||||
self._download(unquote(p[len('/download/'):]))
|
||||
elif p.startswith('/stream/'):
|
||||
@@ -830,6 +845,114 @@ class _Handler(BaseHTTPRequestHandler):
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def _api_clip(self, qs):
|
||||
"""Serve a section of a WAV/FLAC file as a small standalone WAV.
|
||||
|
||||
Decoding the slice server-side means the browser plays a tiny file
|
||||
instantly instead of seeking inside a multi-hundred-MB recording
|
||||
(FLACs have no seek table, so browser seeks bisect the whole file
|
||||
with Range requests)."""
|
||||
filename = qs.get('file', [None])[0]
|
||||
start_s = qs.get('start', [None])[0]
|
||||
end_s = qs.get('end', [None])[0]
|
||||
|
||||
if not filename or start_s is None or end_s is None:
|
||||
self._json_err(400, 'missing file, start, or end parameter')
|
||||
return
|
||||
try:
|
||||
start = max(0.0, float(start_s))
|
||||
end = float(end_s)
|
||||
except (ValueError, TypeError):
|
||||
self._json_err(400, 'start and end must be numbers (seconds)')
|
||||
return
|
||||
if end <= start:
|
||||
self._json_err(400, 'end must be > start')
|
||||
return
|
||||
end = min(end, start + CLIP_MAX_SECONDS)
|
||||
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
return
|
||||
if self._is_active(filename):
|
||||
self._json_err(409, 'File is currently being recorded — clips unavailable until recording stops')
|
||||
return
|
||||
|
||||
ext = path.suffix.lower()
|
||||
try:
|
||||
if ext == '.wav':
|
||||
self._clip_wav(path, start, end)
|
||||
elif ext == '.flac':
|
||||
if not (NUMPY_AVAILABLE and SOUNDFILE_AVAILABLE):
|
||||
self._json_err(400, 'FLAC clips require: pip install numpy soundfile')
|
||||
return
|
||||
self._clip_flac(path, start, end)
|
||||
else:
|
||||
self._json_err(400, f'Clips are not available for {ext} files')
|
||||
except (ConnectionError, BrokenPipeError):
|
||||
raise
|
||||
except Exception as e:
|
||||
self._json_err(500, f'Clip failed: {e}')
|
||||
|
||||
def _send_clip_headers(self, header: bytes, n_frames: int, channels: int,
|
||||
sampwidth: int):
|
||||
self.send_response(200)
|
||||
self.send_header('Content-Type', 'audio/wav')
|
||||
self.send_header('Content-Length', str(len(header) + n_frames * channels * sampwidth))
|
||||
# Finished recordings are immutable, so stepping back to a section
|
||||
# replays the clip from the browser cache
|
||||
self.send_header('Cache-Control', 'private, max-age=86400')
|
||||
self.end_headers()
|
||||
self.wfile.write(header)
|
||||
|
||||
def _clip_wav(self, path: Path, start: float, end: float):
|
||||
with wave.open(str(path), 'rb') as wf:
|
||||
channels = wf.getnchannels()
|
||||
sampwidth = wf.getsampwidth()
|
||||
framerate = wf.getframerate()
|
||||
n_frames = wf.getnframes()
|
||||
f0 = min(int(start * framerate), n_frames)
|
||||
f1 = min(int(end * framerate), n_frames)
|
||||
if f1 <= f0:
|
||||
self._json_err(400, 'clip range is beyond the end of the file')
|
||||
return
|
||||
|
||||
header = _wav_header(f1 - f0, channels, framerate, sampwidth)
|
||||
self._send_clip_headers(header, f1 - f0, channels, sampwidth)
|
||||
|
||||
wf.setpos(f0)
|
||||
remaining = f1 - f0
|
||||
while remaining > 0:
|
||||
chunk = wf.readframes(min(32768, remaining))
|
||||
if not chunk:
|
||||
self.close_connection = True # under-delivered
|
||||
break
|
||||
self.wfile.write(chunk)
|
||||
remaining -= len(chunk) // (channels * sampwidth)
|
||||
|
||||
def _clip_flac(self, path: Path, start: float, end: float):
|
||||
with sf.SoundFile(path) as f:
|
||||
framerate = f.samplerate
|
||||
channels = f.channels
|
||||
n_frames = len(f)
|
||||
f0 = min(int(start * framerate), n_frames)
|
||||
f1 = min(int(end * framerate), n_frames)
|
||||
if f1 <= f0:
|
||||
self._json_err(400, 'clip range is beyond the end of the file')
|
||||
return
|
||||
|
||||
header = _wav_header(f1 - f0, channels, framerate, 2)
|
||||
self._send_clip_headers(header, f1 - f0, channels, 2)
|
||||
|
||||
f.seek(f0)
|
||||
remaining = f1 - f0
|
||||
while remaining > 0:
|
||||
frames = f.read(min(32768, remaining), dtype='int16', always_2d=True)
|
||||
if len(frames) == 0:
|
||||
self.close_connection = True # under-delivered
|
||||
break
|
||||
self.wfile.write(frames.tobytes())
|
||||
remaining -= len(frames)
|
||||
|
||||
def _is_active(self, filename: str) -> bool:
|
||||
"""True if isr.py reports this file as currently being recorded."""
|
||||
try:
|
||||
|
||||
+99
-23
@@ -121,6 +121,16 @@ table.day-table{width:100%;border-collapse:collapse;border:1px solid var(--brd);
|
||||
svg.day-timeline{display:block;width:100%;height:22px}
|
||||
.day-tl-labels{display:flex;justify-content:space-between;font-size:10px;
|
||||
color:var(--muted);font-family:ui-monospace,monospace;margin-top:2px}
|
||||
/* clip player bar */
|
||||
#clip-bar{position:fixed;bottom:0;left:0;right:0;z-index:20;background:var(--surf);
|
||||
border-top:1px solid var(--brd);padding:8px 28px;display:flex;align-items:center;
|
||||
gap:10px;flex-wrap:wrap}
|
||||
#clip-bar audio{flex:1 1 240px;min-width:180px;height:32px}
|
||||
#clip-label{font-size:12px;color:var(--muted);font-family:ui-monospace,monospace;
|
||||
white-space:nowrap;overflow:hidden;text-overflow:ellipsis;max-width:38%}
|
||||
#clip-auto-label{font-size:12px;color:var(--muted);display:flex;align-items:center;
|
||||
gap:4px;white-space:nowrap;cursor:pointer}
|
||||
body.clip-open{padding-bottom:70px}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
@@ -159,6 +169,15 @@ svg.day-timeline{display:block;width:100%;height:22px}
|
||||
<div id="tbody" role="region" aria-label="Recordings archive"></div>
|
||||
<div id="empty" class="empty" style="display:none" role="status">No recordings found.</div>
|
||||
</div>
|
||||
<div id="clip-bar" hidden role="region" aria-label="Section clip player">
|
||||
<button id="clip-prev" aria-label="Previous section (J)">⏮</button>
|
||||
<button id="clip-next" aria-label="Next section (K)">⏭</button>
|
||||
<span id="clip-label"></span>
|
||||
<audio id="clip-audio" controls preload="auto" aria-label="Section clip playback"></audio>
|
||||
<label id="clip-auto-label"><input type="checkbox" id="clip-auto" checked> Auto-advance</label>
|
||||
<button id="clip-context" title="Open the full recording at this position">⤴ Open in file</button>
|
||||
<button id="clip-close" aria-label="Close clip player">✕</button>
|
||||
</div>
|
||||
<script>
|
||||
const esc = s => String(s)
|
||||
.replace(/&/g,'&').replace(/</g,'<')
|
||||
@@ -203,9 +222,8 @@ let activePlayerIdx = null;
|
||||
let allFiles = [];
|
||||
// dayId -> boolean, persists expanded state across re-renders
|
||||
const dayExpanded = new Map();
|
||||
// cross-file day section navigation (populated by ★ Highlights)
|
||||
// cross-file day section list (populated by ★ Highlights)
|
||||
let dayActiveSections = [];
|
||||
let dayActiveSectionCursor = -1;
|
||||
let dayActiveId = null;
|
||||
|
||||
function groupByDay(files) {
|
||||
@@ -321,6 +339,70 @@ function seekToSection(idx, filename, startSec, endSec, sectionIdx) {
|
||||
}
|
||||
}
|
||||
|
||||
// --- clip player -----------------------------------------------------------
|
||||
// Sections play as small server-rendered WAV clips (/api/clip) in the bottom
|
||||
// bar instead of seeking the full recording, which is slow for big FLACs.
|
||||
// clipQueue holds the active review list (one file's sections, or a whole
|
||||
// day's); J/K and ⏮/⏭ step through it.
|
||||
let clipQueue = [];
|
||||
let clipCursor = -1;
|
||||
|
||||
function playClip(i) {
|
||||
if (i < 0 || i >= clipQueue.length) return;
|
||||
clipCursor = i;
|
||||
const c = clipQueue[i];
|
||||
const cs = Math.max(0, c.start - getPreroll());
|
||||
const ce = c.end + 1.5; // small tail after the section
|
||||
const a = document.getElementById('clip-audio');
|
||||
a.src = '/api/clip?file=' + encodeURIComponent(c.filename)
|
||||
+ '&start=' + cs.toFixed(1) + '&end=' + ce.toFixed(1);
|
||||
a.play().catch(() => {});
|
||||
document.getElementById('clip-label').textContent =
|
||||
`${i + 1}/${clipQueue.length} · ${c.filename} @ ${fmtDur(c.start)}–${fmtDur(c.end)}`
|
||||
+ (c.score != null ? ` · +${Math.round(c.score)} dB` : '');
|
||||
document.getElementById('clip-bar').hidden = false;
|
||||
document.body.classList.add('clip-open');
|
||||
setCutFields(c.fileIdx, c.start, c.end);
|
||||
announce(`Clip ${i + 1} of ${clipQueue.length}: ${fmtDur(c.start)} to ${fmtDur(c.end)} in ${c.filename}`);
|
||||
}
|
||||
|
||||
function hideClipBar() {
|
||||
const a = document.getElementById('clip-audio');
|
||||
a.pause();
|
||||
a.removeAttribute('src');
|
||||
document.getElementById('clip-bar').hidden = true;
|
||||
document.body.classList.remove('clip-open');
|
||||
clipQueue = [];
|
||||
clipCursor = -1;
|
||||
}
|
||||
|
||||
function playFileSection(idx, filename, si) {
|
||||
const secs = sectionMap.get(idx) || [];
|
||||
clipQueue = secs.map(s => ({fileIdx: idx, filename, start: s.start, end: s.end, score: s.score}));
|
||||
playClip(si);
|
||||
}
|
||||
|
||||
document.getElementById('clip-prev').addEventListener('click', () => playClip(clipCursor - 1));
|
||||
document.getElementById('clip-next').addEventListener('click', () => playClip(clipCursor + 1));
|
||||
document.getElementById('clip-close').addEventListener('click', hideClipBar);
|
||||
document.getElementById('clip-audio').addEventListener('ended', () => {
|
||||
if (document.getElementById('clip-auto').checked && clipCursor + 1 < clipQueue.length)
|
||||
playClip(clipCursor + 1);
|
||||
});
|
||||
document.getElementById('clip-context').addEventListener('click', () => {
|
||||
const c = clipQueue[clipCursor];
|
||||
if (!c) return;
|
||||
document.getElementById('clip-audio').pause();
|
||||
const row = document.getElementById('row-' + c.fileIdx);
|
||||
if (row) {
|
||||
const tbl = row.closest('table');
|
||||
if (tbl && tbl.hidden) // file's day is collapsed: expand it
|
||||
document.getElementById('daytgl-' + tbl.id.slice('daytbl-'.length))?.click();
|
||||
row.scrollIntoView({block: 'center'});
|
||||
}
|
||||
seekToSection(c.fileIdx, c.filename, c.start, c.end, null);
|
||||
});
|
||||
|
||||
// filename|margin|gap -> analysis result, so re-renders (filtering,
|
||||
// refresh) never refetch what this session already has
|
||||
const analysisCache = new Map();
|
||||
@@ -369,10 +451,10 @@ async function analyse(idx, filename, cell, btn, force = false) {
|
||||
d.sections.forEach((s, si) => {
|
||||
const c = document.createElement('button');
|
||||
c.className='chip';
|
||||
c.title = 'Jump to this section (or use J/K keys)';
|
||||
c.title = 'Play this section (or use J/K keys)';
|
||||
c.textContent = `${fmtDur(s.start)} – ${fmtDur(s.end)}`
|
||||
+ (s.score != null ? ` · +${Math.round(s.score)} dB` : '');
|
||||
c.addEventListener('click', () => seekToSection(idx, filename, s.start, s.end, si));
|
||||
c.addEventListener('click', () => playFileSection(idx, filename, si));
|
||||
chips.appendChild(c);
|
||||
});
|
||||
} else {
|
||||
@@ -402,21 +484,19 @@ document.addEventListener('keydown', e => {
|
||||
if (e.key !== 'j' && e.key !== 'J' && e.key !== 'k' && e.key !== 'K') return;
|
||||
e.preventDefault();
|
||||
|
||||
// Day-level cross-file navigation when Highlights have been loaded
|
||||
if (dayActiveSections.length) {
|
||||
// Clip queue navigation (a chip was clicked or day highlights are loaded)
|
||||
if (clipQueue.length) {
|
||||
if (e.key === 'j' || e.key === 'J') {
|
||||
const ni = dayActiveSectionCursor > 0 ? dayActiveSectionCursor - 1 : -1;
|
||||
if (ni >= 0) jumpToDaySection(ni);
|
||||
else announce('Beginning of day sections');
|
||||
if (clipCursor > 0) playClip(clipCursor - 1);
|
||||
else announce('Beginning of sections');
|
||||
} else {
|
||||
const ni = dayActiveSectionCursor + 1;
|
||||
if (ni < dayActiveSections.length) jumpToDaySection(ni);
|
||||
else announce('End of day sections');
|
||||
if (clipCursor + 1 < clipQueue.length) playClip(clipCursor + 1);
|
||||
else announce('End of sections');
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Per-file section navigation
|
||||
// Per-file in-player navigation (full-file listening, no clip queue)
|
||||
if (activePlayerIdx === null) return;
|
||||
const sections = sectionMap.get(activePlayerIdx) || [];
|
||||
if (!sections.length) return;
|
||||
@@ -735,8 +815,8 @@ function renderFiles(files) {
|
||||
document.getElementById('dayhl-' + dayId).hidden = true;
|
||||
if (dayActiveId === dayId) {
|
||||
dayActiveSections = [];
|
||||
dayActiveSectionCursor = -1;
|
||||
dayActiveId = null;
|
||||
hideClipBar();
|
||||
}
|
||||
}
|
||||
});
|
||||
@@ -875,8 +955,10 @@ async function dayHighlights(dayId, analyzableFiles) {
|
||||
});
|
||||
});
|
||||
dayActiveSections.sort((a, b) => a.absStart - b.absStart);
|
||||
dayActiveSectionCursor = -1;
|
||||
dayActiveId = dayId;
|
||||
// Arm the clip queue so J/K steps through the day immediately
|
||||
clipQueue = dayActiveSections;
|
||||
clipCursor = -1;
|
||||
|
||||
const box = document.createElement('div');
|
||||
box.className = 'wbox';
|
||||
@@ -936,14 +1018,8 @@ async function dayHighlights(dayId, analyzableFiles) {
|
||||
|
||||
function jumpToDaySection(si) {
|
||||
if (si < 0 || si >= dayActiveSections.length) return;
|
||||
dayActiveSectionCursor = si;
|
||||
const { fileIdx, filename, start, end } = dayActiveSections[si];
|
||||
|
||||
// Close the previous player if switching to a different file
|
||||
if (activePlayerIdx !== null && activePlayerIdx !== fileIdx) closePlayer(activePlayerIdx);
|
||||
|
||||
seekToSection(fileIdx, filename, start, end, null);
|
||||
announce(`Day section ${si + 1} of ${dayActiveSections.length}: ${fmtDur(start)}–${fmtDur(end)} in ${filename}`);
|
||||
clipQueue = dayActiveSections;
|
||||
playClip(si);
|
||||
}
|
||||
|
||||
function applyFilters() {
|
||||
|
||||
Reference in New Issue
Block a user