refactor: one Range-capable file sender; HEAD support; assorted hygiene

web.py
* _download and _stream each had their own copy loop; _download had no
  Range support at all, so an interrupted download of a multi-hundred-MB
  recording had to restart. Both now go through _serve_bytes(), which
  keeps _stream's live-header `prefix` trick (the patched bytes are the
  same length as the header they replace, so Range math is unaffected).
* do_HEAD answers / , /stream/ and /download/ as a body-less do_GET.
  /api/clip and /api/cut return 405 instead — generating a clip or
  shelling out to ffmpeg only to discard the result is worse than not
  answering.
* _api_delete drops the deleted file's _DURATION_CACHE entry, which was
  never evicted.
* WAV durations are rounded to 2dp like FLAC's already were.

isr.py
* _open_output_file flushed the buffer and then called
  close_current_file(), which is overridden to flush again.
* log_level/log_file were merged into the dict copied into every
  source's config; no recorder reads them. Read them where they are used.

webui.html
* scoreOrder() is memoised on the section array's identity — U/I
  otherwise re-sorted a whole day (thousands of sections) per keypress.

Verified with a byte-level harness: full/partial/open-ended/unsatisfiable
ranges byte-identical to disk, download Range, HEAD on all three
supported paths returning headers with an empty body, 405 on /api/clip,
and live-header patching still correct both whole-file and for a range
straddling the patched/disk boundary.

Not done from the audit: BaseRecorder.generate_filename's mkdir per file
open. It runs once per split (once an hour by default), so "once at
startup suffices" was not worth a behaviour change. `nul` stays in
.gitignore — it guards against a Windows shell artifact on a Windows box.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-29 21:29:57 +02:00
co-authored by Claude Opus 5
parent 0e7ea82da6
commit 138f6db0a5
6 changed files with 91 additions and 68 deletions
+1 -1
View File
@@ -35,7 +35,7 @@ Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC
- Clips: `_api_clip()` validates params, `_clip_wav()` / `_clip_flac()` stream the decoded slice, `_wav_header()` builds the 44-byte PCM header. - Clips: `_api_clip()` validates params, `_clip_wav()` / `_clip_flac()` stream the decoded slice, `_wav_header()` builds the 44-byte PCM header.
- Filenames as a clock: `_recording_start()` parses the start time out of a filename stem; `_cut_filename()` turns a (stem, ext, start, end) into a wall-clock-named cut. Both the listing `date` field and `_api_cut()` use them. - Filenames as a clock: `_recording_start()` parses the start time out of a filename stem; `_cut_filename()` turns a (stem, ext, start, end) into a wall-clock-named cut. Both the listing `date` field and `_api_cut()` use them.
- Live headers: `_live_wav_header()`, `_live_flac_header()` (+ `_flac_frame_samples()`, CRC-8 verified). - Live headers: `_live_wav_header()`, `_live_flac_header()` (+ `_flac_frame_samples()`, CRC-8 verified).
- Serving: `_stream()` (Range support), `_copy_to_response()`, `_safe_path()` (path traversal guard). - Serving: `_serve_bytes()` is the single Range-capable file sender — `_stream()` (live header patch via its `prefix` argument, `no_store`) and `_download()` (`Content-Disposition`) both go through it, and it is what makes `do_HEAD` a body-less `do_GET`. Plus `_copy_to_response()`, `_safe_path()` (path traversal guard).
`webui.html` (one `<script>` block): `webui.html` (one `<script>` block):
- Clip review: `clipQueue`/`clipCursor`/`scoreCursor` globals, `playClip()`, `playFileSection()`, `hideClipBar()`; markup is the `#clip-bar` div. The clip label shows wall-clock occurrence time + dB prominence + position (`17:09:56 to 17:09:57 · +30 dB (30 / 426 by time)`): queue entries carry `absStart` (epoch s), derived from `fileStartEpoch(f.date)` — the filename clock — with in-file offsets as fallback for non-standard names; only the filename/in-file offset lives in the tooltip now. **Queue entries carry the filename, never a row index:** `_idx` is reassigned positionally on every `load()`, and `pollStatus()` reloads whenever a recording starts/stops/rolls over, so a stored index goes stale mid-review (a past bug had "Open in file"/cut fields target the row one off after a split rollover). Resolve at use time via `fileIdxByName()`. **`playClip()` builds one `text` string used for both `label.textContent` and the `announce()` aria-live message** — they must never diverge (a past bug had the announcement read `Clip N of M: …` while the label read the wall-clock form). **Two cursors, deliberately independent:** `clipCursor` is the time-order position and follows every play; `scoreCursor` is the loudness walk's own position in `scoreOrder()` and is moved *only* by U/I (and "highlights only" auto-advance) in `stepClip`'s by-score branch — J/K must never touch it, so a J/K detour to review nearby clips leaves the loudness walk's place intact and the next U/I resumes the ranking where it left off. `scoreCursor` resets to `-1` on every explicit jump / queue (re-)arm (`hideClipBar`, `playFileSection`, `jumpToDaySection`, both day-highlights arm points), so the next U/I re-anchors on `clipCursor` (the section just selected) else the loudest. `playClip(i, byScore)`: when `byScore` the count shows `scoreCursor` (rank, suffix "by loudness"), else `i` (time-order index, suffix "by time") — the suffix tells the blind user which dimension the count is in. `stepClip`'s by-score branch passes `byScore=true`; every other caller (chip click, J/K, Prev/Next) leaves it false. Both U/I walks — `stepClip`'s by-score branch and the in-player branch of the keydown handler — share `nextInOrder(order, pos, dir, jump)` for the stepping arithmetic (returns `null` at either end); they differ only in what anchors `pos` and what "play" means, and must not re-inline it. - Clip review: `clipQueue`/`clipCursor`/`scoreCursor` globals, `playClip()`, `playFileSection()`, `hideClipBar()`; markup is the `#clip-bar` div. The clip label shows wall-clock occurrence time + dB prominence + position (`17:09:56 to 17:09:57 · +30 dB (30 / 426 by time)`): queue entries carry `absStart` (epoch s), derived from `fileStartEpoch(f.date)` — the filename clock — with in-file offsets as fallback for non-standard names; only the filename/in-file offset lives in the tooltip now. **Queue entries carry the filename, never a row index:** `_idx` is reassigned positionally on every `load()`, and `pollStatus()` reloads whenever a recording starts/stops/rolls over, so a stored index goes stale mid-review (a past bug had "Open in file"/cut fields target the row one off after a split rollover). Resolve at use time via `fileIdxByName()`. **`playClip()` builds one `text` string used for both `label.textContent` and the `announce()` aria-live message** — they must never diverge (a past bug had the announcement read `Clip N of M: …` while the label read the wall-clock form). **Two cursors, deliberately independent:** `clipCursor` is the time-order position and follows every play; `scoreCursor` is the loudness walk's own position in `scoreOrder()` and is moved *only* by U/I (and "highlights only" auto-advance) in `stepClip`'s by-score branch — J/K must never touch it, so a J/K detour to review nearby clips leaves the loudness walk's place intact and the next U/I resumes the ranking where it left off. `scoreCursor` resets to `-1` on every explicit jump / queue (re-)arm (`hideClipBar`, `playFileSection`, `jumpToDaySection`, both day-highlights arm points), so the next U/I re-anchors on `clipCursor` (the section just selected) else the loudest. `playClip(i, byScore)`: when `byScore` the count shows `scoreCursor` (rank, suffix "by loudness"), else `i` (time-order index, suffix "by time") — the suffix tells the blind user which dimension the count is in. `stepClip`'s by-score branch passes `byScore=true`; every other caller (chip click, J/K, Prev/Next) leaves it false. Both U/I walks — `stepClip`'s by-score branch and the in-player branch of the keydown handler — share `nextInOrder(order, pos, dir, jump)` for the stepping arithmetic (returns `null` at either end); they differ only in what anchors `pos` and what "play" means, and must not re-inline it.
+3 -1
View File
@@ -190,12 +190,14 @@ Everything the UI does goes through these endpoints, so they can also be scripte
| `GET /api/clip?file=&start=&end=` | Section of a WAV/FLAC decoded server-side, returned as a standalone WAV (max 600 s) | | `GET /api/clip?file=&start=&end=` | Section of a WAV/FLAC decoded server-side, returned as a standalone WAV (max 600 s) |
| `GET /api/cut?file=&start=&end=` | ffmpeg-trimmed copy of the file as a download | | `GET /api/cut?file=&start=&end=` | ffmpeg-trimmed copy of the file as a download |
| `GET /stream/<name>` | Inline playback with HTTP Range support; live files get an on-the-fly patched header | | `GET /stream/<name>` | Inline playback with HTTP Range support; live files get an on-the-fly patched header |
| `GET /download/<name>` | Raw file download | | `GET /download/<name>` | Raw file download, with HTTP Range support so an interrupted transfer can resume |
| `GET /api/status` | Currently recording files (`status.json` passthrough) | | `GET /api/status` | Currently recording files (`status.json` passthrough) |
| `GET /api/storage` | Disk free/total | | `GET /api/storage` | Disk free/total |
| `GET /api/config` | Server-side defaults for margin, min-gap, and min-duration (seeds the UI controls) | | `GET /api/config` | Server-side defaults for margin, min-gap, and min-duration (seeds the UI controls) |
| `DELETE /api/files/<name>` | Delete a recording and its analysis cache | | `DELETE /api/files/<name>` | Delete a recording and its analysis cache |
`HEAD` is answered for `/`, `/stream/<name>` and `/download/<name>` (headers only, no body) — the generated endpoints (`/api/clip`, `/api/cut`) return `405` rather than doing the work and discarding it.
Analysis, clips, cut, and delete return `409` for files that are still being recorded. Analysis, clips, cut, and delete return `409` for files that are still being recorded.
--- ---
+6 -5
View File
@@ -641,8 +641,7 @@ class SoundcardRecorder(BaseRecorder):
def _open_output_file(self): def _open_output_file(self):
"""Open a new WAV or FLAC output file for recording.""" """Open a new WAV or FLAC output file for recording."""
self._flush_buffer_to_file() self.close_current_file() # overridden below to flush first
self.close_current_file()
self.current_filename = self.generate_filename(self.file_format) self.current_filename = self.generate_filename(self.file_format)
self.current_file = _AudioFileWriter( self.current_file = _AudioFileWriter(
@@ -747,19 +746,21 @@ class RecorderManager:
config.read(self.config_file) config.read(self.config_file)
# Get general settings (defaults) # Get general settings (defaults)
# Defaults every source inherits. Logging is read separately — it is
# a process-wide setting, not something a recorder ever looks at.
general = { general = {
'output_directory': config.get('general', 'output_directory', fallback='recordings'), 'output_directory': config.get('general', 'output_directory', fallback='recordings'),
'split_minutes': config.getint('general', 'split_minutes', fallback=60), 'split_minutes': config.getint('general', 'split_minutes', fallback=60),
'max_retries': config.getint('general', 'max_retries', fallback=10), 'max_retries': config.getint('general', 'max_retries', fallback=10),
'retry_delay_seconds': config.getint('general', 'retry_delay_seconds', fallback=5), 'retry_delay_seconds': config.getint('general', 'retry_delay_seconds', fallback=5),
'log_level': config.get('general', 'log_level', fallback='INFO').upper(),
'log_file': config.get('general', 'log_file', fallback='recorder.log'),
} }
self.output_dir = general['output_directory'] self.output_dir = general['output_directory']
# Setup logging # Setup logging
self._setup_logging(general['log_level'], general['log_file']) self._setup_logging(
config.get('general', 'log_level', fallback='INFO').upper(),
config.get('general', 'log_file', fallback='recorder.log'))
# Initialize shared audio system for soundcard recorders # Initialize shared audio system for soundcard recorders
self.audio_system = AudioSystem(self.logger) self.audio_system = AudioSystem(self.logger)
+1
View File
@@ -196,6 +196,7 @@ class _OfflineHandler(web._Handler):
def __init__(self, recordings_dir): def __init__(self, recordings_dir):
self.recordings_dir = str(recordings_dir) self.recordings_dir = str(recordings_dir)
self.analyses_dir = str(recordings_dir) self.analyses_dir = str(recordings_dir)
self.command = 'GET' # _send skips the body on HEAD
self.wfile = io.BytesIO() self.wfile = io.BytesIO()
self.close_connection = False self.close_connection = False
self.codes = [] self.codes = []
+72 -60
View File
@@ -281,7 +281,7 @@ def _get_audio_duration(path: Path):
if ext == '.wav': if ext == '.wav':
try: try:
with wave.open(str(path), 'rb') as wf: with wave.open(str(path), 'rb') as wf:
return wf.getnframes() / wf.getframerate() return round(wf.getnframes() / wf.getframerate(), 2)
except Exception: except Exception:
return None return None
if SOUNDFILE_AVAILABLE and ext in ('.flac', '.ogg', '.opus'): if SOUNDFILE_AVAILABLE and ext in ('.flac', '.ogg', '.opus'):
@@ -643,6 +643,16 @@ class _Handler(BaseHTTPRequestHandler):
else: else:
self._send(404, b'Not found', 'text/plain') self._send(404, b'Not found', 'text/plain')
def do_HEAD(self):
"""HEAD for the page and the audio files — some players probe before
fetching. Clips and cuts are generated on demand, so answering HEAD
for those would do all the work and throw the result away."""
p = urlparse(self.path).path
if p == '/' or p.startswith('/download/') or p.startswith('/stream/'):
self.do_GET()
else:
self._send(405, b'Method Not Allowed', 'text/plain')
def do_GET(self): def do_GET(self):
parsed = urlparse(self.path) parsed = urlparse(self.path)
qs = parse_qs(parsed.query) qs = parse_qs(parsed.query)
@@ -757,20 +767,65 @@ class _Handler(BaseHTTPRequestHandler):
pass pass
self._send(200, b'{"active":[]}', 'application/json') self._send(200, b'{"active":[]}', 'application/json')
def _serve_bytes(self, path: Path, size: int, content_type: str,
prefix: bytes = b'', no_store: bool = False,
disposition: str = None):
"""Serve a file with HTTP Range support.
`prefix` replaces the first len(prefix) bytes of the file on the way
out — that is how in-progress recordings get a header patched to the
duration captured so far. It is the same length as the header it
replaces, so all offsets and Range math stay valid.
"""
range_header = self.headers.get('Range', '')
m = re.match(r'bytes=(\d+)-(\d*)', range_header) if range_header else None
if m:
start = int(m.group(1))
end = int(m.group(2)) if m.group(2) else size - 1
end = min(end, size - 1)
if start > end or start >= size:
self._send(416, b'Range Not Satisfiable', 'text/plain')
return
length = end - start + 1
self.send_response(206)
self.send_header('Content-Range', f'bytes {start}-{end}/{size}')
else:
start, length = 0, size
self.send_response(200)
self.send_header('Content-Type', content_type)
self.send_header('Content-Length', str(length))
self.send_header('Accept-Ranges', 'bytes')
if disposition:
self.send_header('Content-Disposition', disposition)
if no_store:
self.send_header('Cache-Control', 'no-store')
self.end_headers()
if self.command == 'HEAD':
return
with open(path, 'rb') as fh:
sent = 0
if start < len(prefix):
head = prefix[start:start + length]
self.wfile.write(head)
sent = len(head)
if sent < length:
fh.seek(start + sent)
# Bound the copy: the file may grow while we serve it, and
# writing more than Content-Length desyncs keep-alive.
self._copy_to_response(fh, length - sent)
def _download(self, filename: str): def _download(self, filename: str):
path = self._safe_path(filename) path = self._safe_path(filename)
if path is None: if path is None:
return return
# Range support matters here too: these files run to hundreds of MB
size = path.stat().st_size # and an interrupted download would otherwise have to restart.
self.send_response(200) self._serve_bytes(path, path.stat().st_size, 'application/octet-stream',
self.send_header('Content-Type', 'application/octet-stream') disposition=f'attachment; filename="{path.name}"')
self.send_header('Content-Disposition', f'attachment; filename="{path.name}"')
self.send_header('Content-Length', str(size))
self.end_headers()
with open(path, 'rb') as fh:
self._copy_to_response(fh, size)
def _stream(self, filename: str): def _stream(self, filename: str):
"""Serve audio for inline playback with HTTP Range support. """Serve audio for inline playback with HTTP Range support.
@@ -795,54 +850,7 @@ class _Handler(BaseHTTPRequestHandler):
elif ext == '.flac': elif ext == '.flac':
prefix = _live_flac_header(path, size) or b'' prefix = _live_flac_header(path, size) or b''
range_header = self.headers.get('Range', '') self._serve_bytes(path, size, content_type, prefix=prefix, no_store=is_active)
m = re.match(r'bytes=(\d+)-(\d*)', range_header) if range_header else None
if m:
start = int(m.group(1))
end = int(m.group(2)) if m.group(2) else size - 1
end = min(end, size - 1)
if start > end or start >= size:
self._send(416, b'Range Not Satisfiable', 'text/plain')
return
length = end - start + 1
self.send_response(206)
self.send_header('Content-Type', content_type)
self.send_header('Content-Range', f'bytes {start}-{end}/{size}')
self.send_header('Content-Length', str(length))
self.send_header('Accept-Ranges', 'bytes')
if is_active:
self.send_header('Cache-Control', 'no-store')
self.end_headers()
with open(path, 'rb') as fh:
sent = 0
if start < len(prefix):
head = prefix[start:start + length]
self.wfile.write(head)
sent = len(head)
if sent < length:
fh.seek(start + sent)
self._copy_to_response(fh, length - sent)
else:
self.send_response(200)
self.send_header('Content-Type', content_type)
self.send_header('Content-Length', str(size))
self.send_header('Accept-Ranges', 'bytes')
if is_active:
self.send_header('Cache-Control', 'no-store')
self.end_headers()
with open(path, 'rb') as fh:
if prefix:
self.wfile.write(prefix)
fh.seek(len(prefix))
self._copy_to_response(fh, size - len(prefix))
else:
# Bound the copy: the file may grow while we serve it, and
# writing more than Content-Length desyncs keep-alive.
self._copy_to_response(fh, size)
def _api_storage(self): def _api_storage(self):
# 'used' is computed client-side from the file list; walking the whole # 'used' is computed client-side from the file list; walking the whole
@@ -885,6 +893,9 @@ class _Handler(BaseHTTPRequestHandler):
except Exception: except Exception:
pass pass
with _DURATION_CACHE_LOCK:
_DURATION_CACHE.pop(filename, None)
self._send(200, json.dumps({'deleted': filename}).encode(), 'application/json') self._send(200, json.dumps({'deleted': filename}).encode(), 'application/json')
def _api_cut(self, qs): def _api_cut(self, qs):
@@ -1119,7 +1130,8 @@ class _Handler(BaseHTTPRequestHandler):
self.send_header('Content-Type', content_type) self.send_header('Content-Type', content_type)
self.send_header('Content-Length', str(len(body))) self.send_header('Content-Length', str(len(body)))
self.end_headers() self.end_headers()
self.wfile.write(body) if self.command != 'HEAD':
self.wfile.write(body)
def _json_err(self, code: int, msg: str): def _json_err(self, code: int, msg: str):
self._send(code, json.dumps({'error': msg}).encode('utf-8'), 'application/json') self._send(code, json.dumps({'error': msg}).encode('utf-8'), 'application/json')
+8 -1
View File
@@ -428,10 +428,17 @@ function playFileSection(idx, filename, si) {
// auto-advance in "highlights only" mode) walk this ranking, so a review // auto-advance in "highlights only" mode) walk this ranking, so a review
// runs from the most to the least interesting event until the user stops — // runs from the most to the least interesting event until the user stops —
// no top-N cutoff. // no top-N cutoff.
// Cached against the section array's identity: a day queue runs to thousands
// of entries and U/I would otherwise re-sort all of them on every keypress.
// Every arming path assigns a fresh array, so identity is a safe key.
let scoreOrderCache = {secs: null, order: null};
function scoreOrder(secs) { function scoreOrder(secs) {
return secs.map((s, i) => ({i, score: s.score || 0})) if (scoreOrderCache.secs === secs) return scoreOrderCache.order;
const order = secs.map((s, i) => ({i, score: s.score || 0}))
.sort((a, b) => b.score - a.score) .sort((a, b) => b.score - a.score)
.map(r => r.i); .map(r => r.i);
scoreOrderCache = {secs, order};
return order;
} }
// Step a position within an ordering: pos + dir, or straight to the extreme in // Step a position within an ordering: pos + dir, or straight to the extreme in