refactor: one Range-capable file sender; HEAD support; assorted hygiene
web.py * _download and _stream each had their own copy loop; _download had no Range support at all, so an interrupted download of a multi-hundred-MB recording had to restart. Both now go through _serve_bytes(), which keeps _stream's live-header `prefix` trick (the patched bytes are the same length as the header they replace, so Range math is unaffected). * do_HEAD answers / , /stream/ and /download/ as a body-less do_GET. /api/clip and /api/cut return 405 instead — generating a clip or shelling out to ffmpeg only to discard the result is worse than not answering. * _api_delete drops the deleted file's _DURATION_CACHE entry, which was never evicted. * WAV durations are rounded to 2dp like FLAC's already were. isr.py * _open_output_file flushed the buffer and then called close_current_file(), which is overridden to flush again. * log_level/log_file were merged into the dict copied into every source's config; no recorder reads them. Read them where they are used. webui.html * scoreOrder() is memoised on the section array's identity — U/I otherwise re-sorted a whole day (thousands of sections) per keypress. Verified with a byte-level harness: full/partial/open-ended/unsatisfiable ranges byte-identical to disk, download Range, HEAD on all three supported paths returning headers with an empty body, 405 on /api/clip, and live-header patching still correct both whole-file and for a range straddling the patched/disk boundary. Not done from the audit: BaseRecorder.generate_filename's mkdir per file open. It runs once per split (once an hour by default), so "once at startup suffices" was not worth a behaviour change. `nul` stays in .gitignore — it guards against a Windows shell artifact on a Windows box. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -35,7 +35,7 @@ Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC
|
||||
- Clips: `_api_clip()` validates params, `_clip_wav()` / `_clip_flac()` stream the decoded slice, `_wav_header()` builds the 44-byte PCM header.
|
||||
- Filenames as a clock: `_recording_start()` parses the start time out of a filename stem; `_cut_filename()` turns a (stem, ext, start, end) into a wall-clock-named cut. Both the listing `date` field and `_api_cut()` use them.
|
||||
- Live headers: `_live_wav_header()`, `_live_flac_header()` (+ `_flac_frame_samples()`, CRC-8 verified).
|
||||
- Serving: `_stream()` (Range support), `_copy_to_response()`, `_safe_path()` (path traversal guard).
|
||||
- Serving: `_serve_bytes()` is the single Range-capable file sender — `_stream()` (live header patch via its `prefix` argument, `no_store`) and `_download()` (`Content-Disposition`) both go through it, and it is what makes `do_HEAD` a body-less `do_GET`. Plus `_copy_to_response()`, `_safe_path()` (path traversal guard).
|
||||
|
||||
`webui.html` (one `<script>` block):
|
||||
- Clip review: `clipQueue`/`clipCursor`/`scoreCursor` globals, `playClip()`, `playFileSection()`, `hideClipBar()`; markup is the `#clip-bar` div. The clip label shows wall-clock occurrence time + dB prominence + position (`17:09:56 to 17:09:57 · +30 dB (30 / 426 by time)`): queue entries carry `absStart` (epoch s), derived from `fileStartEpoch(f.date)` — the filename clock — with in-file offsets as fallback for non-standard names; only the filename/in-file offset lives in the tooltip now. **Queue entries carry the filename, never a row index:** `_idx` is reassigned positionally on every `load()`, and `pollStatus()` reloads whenever a recording starts/stops/rolls over, so a stored index goes stale mid-review (a past bug had "Open in file"/cut fields target the row one off after a split rollover). Resolve at use time via `fileIdxByName()`. **`playClip()` builds one `text` string used for both `label.textContent` and the `announce()` aria-live message** — they must never diverge (a past bug had the announcement read `Clip N of M: …` while the label read the wall-clock form). **Two cursors, deliberately independent:** `clipCursor` is the time-order position and follows every play; `scoreCursor` is the loudness walk's own position in `scoreOrder()` and is moved *only* by U/I (and "highlights only" auto-advance) in `stepClip`'s by-score branch — J/K must never touch it, so a J/K detour to review nearby clips leaves the loudness walk's place intact and the next U/I resumes the ranking where it left off. `scoreCursor` resets to `-1` on every explicit jump / queue (re-)arm (`hideClipBar`, `playFileSection`, `jumpToDaySection`, both day-highlights arm points), so the next U/I re-anchors on `clipCursor` (the section just selected) else the loudest. `playClip(i, byScore)`: when `byScore` the count shows `scoreCursor` (rank, suffix "by loudness"), else `i` (time-order index, suffix "by time") — the suffix tells the blind user which dimension the count is in. `stepClip`'s by-score branch passes `byScore=true`; every other caller (chip click, J/K, Prev/Next) leaves it false. Both U/I walks — `stepClip`'s by-score branch and the in-player branch of the keydown handler — share `nextInOrder(order, pos, dir, jump)` for the stepping arithmetic (returns `null` at either end); they differ only in what anchors `pos` and what "play" means, and must not re-inline it.
|
||||
|
||||
@@ -190,12 +190,14 @@ Everything the UI does goes through these endpoints, so they can also be scripte
|
||||
| `GET /api/clip?file=&start=&end=` | Section of a WAV/FLAC decoded server-side, returned as a standalone WAV (max 600 s) |
|
||||
| `GET /api/cut?file=&start=&end=` | ffmpeg-trimmed copy of the file as a download |
|
||||
| `GET /stream/<name>` | Inline playback with HTTP Range support; live files get an on-the-fly patched header |
|
||||
| `GET /download/<name>` | Raw file download |
|
||||
| `GET /download/<name>` | Raw file download, with HTTP Range support so an interrupted transfer can resume |
|
||||
| `GET /api/status` | Currently recording files (`status.json` passthrough) |
|
||||
| `GET /api/storage` | Disk free/total |
|
||||
| `GET /api/config` | Server-side defaults for margin, min-gap, and min-duration (seeds the UI controls) |
|
||||
| `DELETE /api/files/<name>` | Delete a recording and its analysis cache |
|
||||
|
||||
`HEAD` is answered for `/`, `/stream/<name>` and `/download/<name>` (headers only, no body) — the generated endpoints (`/api/clip`, `/api/cut`) return `405` rather than doing the work and discarding it.
|
||||
|
||||
Analysis, clips, cut, and delete return `409` for files that are still being recorded.
|
||||
|
||||
---
|
||||
|
||||
@@ -641,8 +641,7 @@ class SoundcardRecorder(BaseRecorder):
|
||||
|
||||
def _open_output_file(self):
|
||||
"""Open a new WAV or FLAC output file for recording."""
|
||||
self._flush_buffer_to_file()
|
||||
self.close_current_file()
|
||||
self.close_current_file() # overridden below to flush first
|
||||
|
||||
self.current_filename = self.generate_filename(self.file_format)
|
||||
self.current_file = _AudioFileWriter(
|
||||
@@ -747,19 +746,21 @@ class RecorderManager:
|
||||
config.read(self.config_file)
|
||||
|
||||
# Get general settings (defaults)
|
||||
# Defaults every source inherits. Logging is read separately — it is
|
||||
# a process-wide setting, not something a recorder ever looks at.
|
||||
general = {
|
||||
'output_directory': config.get('general', 'output_directory', fallback='recordings'),
|
||||
'split_minutes': config.getint('general', 'split_minutes', fallback=60),
|
||||
'max_retries': config.getint('general', 'max_retries', fallback=10),
|
||||
'retry_delay_seconds': config.getint('general', 'retry_delay_seconds', fallback=5),
|
||||
'log_level': config.get('general', 'log_level', fallback='INFO').upper(),
|
||||
'log_file': config.get('general', 'log_file', fallback='recorder.log'),
|
||||
}
|
||||
|
||||
self.output_dir = general['output_directory']
|
||||
|
||||
# Setup logging
|
||||
self._setup_logging(general['log_level'], general['log_file'])
|
||||
self._setup_logging(
|
||||
config.get('general', 'log_level', fallback='INFO').upper(),
|
||||
config.get('general', 'log_file', fallback='recorder.log'))
|
||||
|
||||
# Initialize shared audio system for soundcard recorders
|
||||
self.audio_system = AudioSystem(self.logger)
|
||||
|
||||
@@ -196,6 +196,7 @@ class _OfflineHandler(web._Handler):
|
||||
def __init__(self, recordings_dir):
|
||||
self.recordings_dir = str(recordings_dir)
|
||||
self.analyses_dir = str(recordings_dir)
|
||||
self.command = 'GET' # _send skips the body on HEAD
|
||||
self.wfile = io.BytesIO()
|
||||
self.close_connection = False
|
||||
self.codes = []
|
||||
|
||||
@@ -281,7 +281,7 @@ def _get_audio_duration(path: Path):
|
||||
if ext == '.wav':
|
||||
try:
|
||||
with wave.open(str(path), 'rb') as wf:
|
||||
return wf.getnframes() / wf.getframerate()
|
||||
return round(wf.getnframes() / wf.getframerate(), 2)
|
||||
except Exception:
|
||||
return None
|
||||
if SOUNDFILE_AVAILABLE and ext in ('.flac', '.ogg', '.opus'):
|
||||
@@ -643,6 +643,16 @@ class _Handler(BaseHTTPRequestHandler):
|
||||
else:
|
||||
self._send(404, b'Not found', 'text/plain')
|
||||
|
||||
def do_HEAD(self):
|
||||
"""HEAD for the page and the audio files — some players probe before
|
||||
fetching. Clips and cuts are generated on demand, so answering HEAD
|
||||
for those would do all the work and throw the result away."""
|
||||
p = urlparse(self.path).path
|
||||
if p == '/' or p.startswith('/download/') or p.startswith('/stream/'):
|
||||
self.do_GET()
|
||||
else:
|
||||
self._send(405, b'Method Not Allowed', 'text/plain')
|
||||
|
||||
def do_GET(self):
|
||||
parsed = urlparse(self.path)
|
||||
qs = parse_qs(parsed.query)
|
||||
@@ -757,20 +767,65 @@ class _Handler(BaseHTTPRequestHandler):
|
||||
pass
|
||||
self._send(200, b'{"active":[]}', 'application/json')
|
||||
|
||||
def _serve_bytes(self, path: Path, size: int, content_type: str,
|
||||
prefix: bytes = b'', no_store: bool = False,
|
||||
disposition: str = None):
|
||||
"""Serve a file with HTTP Range support.
|
||||
|
||||
`prefix` replaces the first len(prefix) bytes of the file on the way
|
||||
out — that is how in-progress recordings get a header patched to the
|
||||
duration captured so far. It is the same length as the header it
|
||||
replaces, so all offsets and Range math stay valid.
|
||||
"""
|
||||
range_header = self.headers.get('Range', '')
|
||||
m = re.match(r'bytes=(\d+)-(\d*)', range_header) if range_header else None
|
||||
|
||||
if m:
|
||||
start = int(m.group(1))
|
||||
end = int(m.group(2)) if m.group(2) else size - 1
|
||||
end = min(end, size - 1)
|
||||
if start > end or start >= size:
|
||||
self._send(416, b'Range Not Satisfiable', 'text/plain')
|
||||
return
|
||||
length = end - start + 1
|
||||
self.send_response(206)
|
||||
self.send_header('Content-Range', f'bytes {start}-{end}/{size}')
|
||||
else:
|
||||
start, length = 0, size
|
||||
self.send_response(200)
|
||||
|
||||
self.send_header('Content-Type', content_type)
|
||||
self.send_header('Content-Length', str(length))
|
||||
self.send_header('Accept-Ranges', 'bytes')
|
||||
if disposition:
|
||||
self.send_header('Content-Disposition', disposition)
|
||||
if no_store:
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.end_headers()
|
||||
|
||||
if self.command == 'HEAD':
|
||||
return
|
||||
|
||||
with open(path, 'rb') as fh:
|
||||
sent = 0
|
||||
if start < len(prefix):
|
||||
head = prefix[start:start + length]
|
||||
self.wfile.write(head)
|
||||
sent = len(head)
|
||||
if sent < length:
|
||||
fh.seek(start + sent)
|
||||
# Bound the copy: the file may grow while we serve it, and
|
||||
# writing more than Content-Length desyncs keep-alive.
|
||||
self._copy_to_response(fh, length - sent)
|
||||
|
||||
def _download(self, filename: str):
|
||||
path = self._safe_path(filename)
|
||||
if path is None:
|
||||
return
|
||||
|
||||
size = path.stat().st_size
|
||||
self.send_response(200)
|
||||
self.send_header('Content-Type', 'application/octet-stream')
|
||||
self.send_header('Content-Disposition', f'attachment; filename="{path.name}"')
|
||||
self.send_header('Content-Length', str(size))
|
||||
self.end_headers()
|
||||
|
||||
with open(path, 'rb') as fh:
|
||||
self._copy_to_response(fh, size)
|
||||
# Range support matters here too: these files run to hundreds of MB
|
||||
# and an interrupted download would otherwise have to restart.
|
||||
self._serve_bytes(path, path.stat().st_size, 'application/octet-stream',
|
||||
disposition=f'attachment; filename="{path.name}"')
|
||||
|
||||
def _stream(self, filename: str):
|
||||
"""Serve audio for inline playback with HTTP Range support.
|
||||
@@ -795,54 +850,7 @@ class _Handler(BaseHTTPRequestHandler):
|
||||
elif ext == '.flac':
|
||||
prefix = _live_flac_header(path, size) or b''
|
||||
|
||||
range_header = self.headers.get('Range', '')
|
||||
m = re.match(r'bytes=(\d+)-(\d*)', range_header) if range_header else None
|
||||
|
||||
if m:
|
||||
start = int(m.group(1))
|
||||
end = int(m.group(2)) if m.group(2) else size - 1
|
||||
end = min(end, size - 1)
|
||||
if start > end or start >= size:
|
||||
self._send(416, b'Range Not Satisfiable', 'text/plain')
|
||||
return
|
||||
length = end - start + 1
|
||||
|
||||
self.send_response(206)
|
||||
self.send_header('Content-Type', content_type)
|
||||
self.send_header('Content-Range', f'bytes {start}-{end}/{size}')
|
||||
self.send_header('Content-Length', str(length))
|
||||
self.send_header('Accept-Ranges', 'bytes')
|
||||
if is_active:
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.end_headers()
|
||||
|
||||
with open(path, 'rb') as fh:
|
||||
sent = 0
|
||||
if start < len(prefix):
|
||||
head = prefix[start:start + length]
|
||||
self.wfile.write(head)
|
||||
sent = len(head)
|
||||
if sent < length:
|
||||
fh.seek(start + sent)
|
||||
self._copy_to_response(fh, length - sent)
|
||||
else:
|
||||
self.send_response(200)
|
||||
self.send_header('Content-Type', content_type)
|
||||
self.send_header('Content-Length', str(size))
|
||||
self.send_header('Accept-Ranges', 'bytes')
|
||||
if is_active:
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.end_headers()
|
||||
|
||||
with open(path, 'rb') as fh:
|
||||
if prefix:
|
||||
self.wfile.write(prefix)
|
||||
fh.seek(len(prefix))
|
||||
self._copy_to_response(fh, size - len(prefix))
|
||||
else:
|
||||
# Bound the copy: the file may grow while we serve it, and
|
||||
# writing more than Content-Length desyncs keep-alive.
|
||||
self._copy_to_response(fh, size)
|
||||
self._serve_bytes(path, size, content_type, prefix=prefix, no_store=is_active)
|
||||
|
||||
def _api_storage(self):
|
||||
# 'used' is computed client-side from the file list; walking the whole
|
||||
@@ -885,6 +893,9 @@ class _Handler(BaseHTTPRequestHandler):
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
with _DURATION_CACHE_LOCK:
|
||||
_DURATION_CACHE.pop(filename, None)
|
||||
|
||||
self._send(200, json.dumps({'deleted': filename}).encode(), 'application/json')
|
||||
|
||||
def _api_cut(self, qs):
|
||||
@@ -1119,6 +1130,7 @@ class _Handler(BaseHTTPRequestHandler):
|
||||
self.send_header('Content-Type', content_type)
|
||||
self.send_header('Content-Length', str(len(body)))
|
||||
self.end_headers()
|
||||
if self.command != 'HEAD':
|
||||
self.wfile.write(body)
|
||||
|
||||
def _json_err(self, code: int, msg: str):
|
||||
|
||||
+8
-1
@@ -428,10 +428,17 @@ function playFileSection(idx, filename, si) {
|
||||
// auto-advance in "highlights only" mode) walk this ranking, so a review
|
||||
// runs from the most to the least interesting event until the user stops —
|
||||
// no top-N cutoff.
|
||||
// Cached against the section array's identity: a day queue runs to thousands
|
||||
// of entries and U/I would otherwise re-sort all of them on every keypress.
|
||||
// Every arming path assigns a fresh array, so identity is a safe key.
|
||||
let scoreOrderCache = {secs: null, order: null};
|
||||
function scoreOrder(secs) {
|
||||
return secs.map((s, i) => ({i, score: s.score || 0}))
|
||||
if (scoreOrderCache.secs === secs) return scoreOrderCache.order;
|
||||
const order = secs.map((s, i) => ({i, score: s.score || 0}))
|
||||
.sort((a, b) => b.score - a.score)
|
||||
.map(r => r.i);
|
||||
scoreOrderCache = {secs, order};
|
||||
return order;
|
||||
}
|
||||
|
||||
// Step a position within an ordering: pos + dir, or straight to the extreme in
|
||||
|
||||
Reference in New Issue
Block a user