admin
8e496ec2c4
Server (web.py): - /api/analyze no longer returns the full per-window RMS array (~45x larger than the rms_display the UI actually renders); old caches are stripped on read - /api/files reads only the first 256 bytes of each analysis cache to get threshold/min_gap instead of parsing the whole JSON - durations cached by (mtime, size) instead of re-opening every audio header per request; stat() race with deleted files guarded - /api/storage no longer walks the recordings tree (used bytes now computed client-side from the file list) - HTTP/1.1 keep-alive enabled; short writes force-close the connection; client-disconnect tracebacks from aborted seeks silenced - all file copies bounded by the advertised Content-Length so files growing during a response cannot desync the connection Live recording playback: - /stream/ patches in-progress WAV headers to the current file size so browsers show real duration and can seek (on-disk header says 0 frames until the recorder closes the file) - active files served with Cache-Control: no-store - reopening the player for a recording file reloads the source to pick up newly captured audio UI loading: - analyses lazy-load only for expanded day groups; collapsed days defer fetching until opened, and auto-load only when cached parameters match the current controls (no surprise mass recompute) - client-side analysis cache shared by file rows and day highlights, so re-renders and filters never refetch - filename filter debounced (200 ms) - file list auto-refreshes when the active recording set changes, unless audio is playing Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
232 lines
12 KiB
Markdown
232 lines
12 KiB
Markdown
# ISR — Audio Recorder
|
||
|
||
> AI-generated code. Run at your own risk. MIT licence.
|
||
|
||
Records from multiple simultaneous sources — Icecast/HTTP streams and ALSA soundcards — with time-based file splitting.
|
||
|
||
## Features
|
||
|
||
- Multiple sources recorded in parallel (each in its own thread)
|
||
- **Stream recording** — HTTP/Icecast, auto-detects MP3 / OGG / AAC / FLAC / Opus from `Content-Type`
|
||
- **Soundcard recording** — ALSA (`arecord`), works on any Linux / Raspberry Pi
|
||
- Time-aligned file splits (e.g. every hour, on the hour)
|
||
- OGG / Opus / FLAC header injection so every split file is independently playable
|
||
- Auto-reconnect on stream drops or device errors
|
||
- WAV and FLAC output for soundcard sources
|
||
- Web UI to browse, download, and analyse recordings
|
||
|
||
---
|
||
|
||
## Quick start — bare-metal
|
||
|
||
```bash
|
||
pip install requests # stream recording
|
||
pip install numpy soundfile # FLAC output + web waveform analysis (optional)
|
||
|
||
cp config.example.ini config.ini
|
||
# edit config.ini to add your sources
|
||
python isr.py # start recorder (Ctrl+C to stop)
|
||
python web.py # start web UI at http://localhost:8080
|
||
```
|
||
|
||
## Quick start — Docker
|
||
|
||
```bash
|
||
cp config.example.ini config.ini
|
||
# edit config.ini to add your sources (no path changes needed for Docker)
|
||
|
||
docker compose up -d
|
||
# recorder starts immediately; web UI at http://<host>:8050
|
||
docker compose logs -f # tail logs from both services
|
||
docker compose down # graceful stop (waits up to 30 s for files to close)
|
||
```
|
||
|
||
Recordings land in `./recordings/` on the host (bind-mounted into both containers at different internal paths — no config changes required vs. bare-metal).
|
||
|
||
If you only record streams (no soundcard), comment out the `devices` block in `docker-compose.yml`.
|
||
|
||
**Updating a running deployment:**
|
||
```bash
|
||
git pull
|
||
docker compose down
|
||
docker compose up -d --build
|
||
```
|
||
`docker compose down` is required — Docker won't apply changed volume mounts to existing containers without recreating them.
|
||
|
||
---
|
||
|
||
## Configuration
|
||
|
||
`config.ini` uses standard INI format. `[general]` provides defaults; every other section is a recording source. Source sections inherit all general settings and can override any of them.
|
||
|
||
### `[general]`
|
||
|
||
| Key | Default | Description |
|
||
|-----|---------|-------------|
|
||
| `output_directory` | `recordings` | Output path relative to the working directory (or absolute). The Docker setup mounts `./recordings` at `/app/recordings` so this default works unchanged. |
|
||
| `split_minutes` | `60` | Split into a new file every N minutes, aligned to clock boundaries (e.g. 60 → files start at :00, 30 → at :00 and :30). |
|
||
| `filename_pattern` | `%Y%m%d_%H%M%S` | strftime pattern; file extension is appended automatically. |
|
||
| `max_retries` | `10` | Give up after this many consecutive failures per source. |
|
||
| `retry_delay_seconds` | `5` | Wait between retries. |
|
||
| `log_level` | `INFO` | `DEBUG` / `INFO` / `WARNING` / `ERROR` / `CRITICAL` |
|
||
| `log_file` | `recorder.log` | Log file path. In Docker, logs also go to stdout and are visible via `docker compose logs`. |
|
||
|
||
### `type = stream`
|
||
|
||
```ini
|
||
[my_stream]
|
||
type = stream
|
||
url = http://icecast.example.com:8000/live
|
||
username = # leave blank for public streams
|
||
password =
|
||
format = auto # auto | mp3 | ogg | aac | flac | opus
|
||
```
|
||
|
||
`format = auto` detects from the `Content-Type` response header. For OGG/Opus/FLAC the first ~16 KB of each connection is buffered to extract codec headers, which are then prepended to every split file — all files are independently playable.
|
||
|
||
A new file is always opened on (re)connect so gaps between connections are never silently merged.
|
||
|
||
### `type = soundcard`
|
||
|
||
```ini
|
||
[mic_in]
|
||
type = soundcard
|
||
device = default # see device selection below
|
||
sample_rate = 44100
|
||
channels = 2
|
||
format = wav # wav | flac
|
||
```
|
||
|
||
**Device selection:**
|
||
|
||
| Value | Behaviour |
|
||
|-------|-----------|
|
||
| `default` | System default input |
|
||
| `<partial name>` | Case-insensitive substring match against device name |
|
||
| `hw:X,Y` | Exact ALSA hardware ID |
|
||
| `<pcm name>` | Any ALSA PCM defined in `asound.conf` (e.g. the `shared_mic` dsnoop device), even if it doesn't appear in `arecord -l` |
|
||
|
||
Run `python isr.py --list-devices` (or `arecord -l`) to see available devices and their IDs.
|
||
|
||
FLAC output requires `pip install soundfile numpy`.
|
||
|
||
### Multiple sources
|
||
|
||
Every section except `[general]` is a source — they all record simultaneously:
|
||
|
||
```ini
|
||
[general]
|
||
output_directory = recordings
|
||
split_minutes = 60
|
||
|
||
[radio1]
|
||
type = stream
|
||
url = http://radio.example.com:8000/stream1
|
||
filename_pattern = radio1_%Y%m%d_%H%M%S
|
||
|
||
[system_audio]
|
||
type = soundcard
|
||
device = hw:0,0
|
||
filename_pattern = system_%Y%m%d_%H%M%S
|
||
```
|
||
|
||
---
|
||
|
||
## Filename patterns
|
||
|
||
strftime codes are substituted at split time. The file extension is added automatically.
|
||
|
||
| Pattern | Example |
|
||
|---------|---------|
|
||
| `%Y%m%d_%H%M%S` | `20241225_143000.mp3` |
|
||
| `radio_%Y-%m-%d_%H%M` | `radio_2024-12-25_1430.mp3` |
|
||
| `%Y/%m/%d/rec_%H%M%S` | `2024/12/25/rec_143000.mp3` *(subdirs created automatically)* |
|
||
|
||
---
|
||
|
||
## Web UI (`web.py`)
|
||
|
||
```bash
|
||
python web.py # serves ./recordings on port 8080
|
||
python web.py --dir /path/to/audio # custom recordings directory
|
||
python web.py --port 8888 # custom port
|
||
python web.py --threshold 0.03 # loudness threshold 0–1 (default 0.05)
|
||
python web.py --min-gap 15 # grace period in seconds for merging loud sections (default 2)
|
||
python web.py --analyses-dir /path/to/dir # where to store analysis cache files (default: <recordings>/analyses)
|
||
```
|
||
|
||
The browser UI (HTML/CSS/JS) lives in `webui.html`, which `web.py` loads at startup — keep the two files together.
|
||
|
||
Shows recordings grouped by day with collapsible sections. Features:
|
||
|
||
- **Day groups** — recordings are grouped under a collapsible day heading showing date, file count, total duration, and total size. The most recent day is expanded by default; older days start collapsed. Expanded state is preserved across filter changes.
|
||
- **Day highlights** — click **★ Highlights** on any day heading to run loudness analysis across all WAV/FLAC files in that day and display a combined activity timeline SVG. Orange segments show when loud sections occurred relative to the day's time span; blue shows the file extents. Labels show the start, midpoint, and end times.
|
||
- **Inline playback** — collapsible `▶ Play` button per row; audio loads lazily via a seekable `/stream/` endpoint with HTTP Range support. Metadata is fetched immediately so the duration is visible without pressing play.
|
||
- **Waveform analysis** — on demand per file; computes RMS per 100 ms window and highlights loud sections. Supported for WAV and FLAC (FLAC requires `numpy` + `soundfile`). Pure-Python fallback for WAV when numpy is absent. Results are cached in `recordings/analyses/<filename>.analysis.json`; subsequent requests at the same threshold and min-gap settings return instantly without re-reading the audio. The cache file is deleted automatically when the audio file is deleted. Orphaned cache files (audio deleted outside the UI) are pruned on startup.
|
||
- **Grace period** — configurable in the controls bar (default 2 s). Loud sections separated by less than this gap are merged into one. Raise this (e.g. to 15–30 s) when a single event generates many timestamps due to brief quiet gaps within it.
|
||
- **Timestamp jump** — after analysis, click any loud-section chip to seek the player to that position and pre-fill the cut panel. Use **J** / **K** keyboard shortcuts to jump to the previous / next section while audio is playing.
|
||
- **Cut & download** — `✂ Cut` button opens the player row and reveals a cut panel. Enter start and end times in `m:ss` or `h:mm:ss` format and click **↓ Download cut** to receive an ffmpeg-trimmed copy without re-encoding. Requires ffmpeg (included in the Docker image).
|
||
- **Filters** — live filename search and from/to date pickers above the table; applied client-side with no additional requests. Shows `N of M shown` when a filter is active.
|
||
- **Delete** — `✕ Delete` button per row with confirmation prompt; disabled for files currently being recorded; sends `DELETE /api/files/<name>` and re-renders the table.
|
||
- **Live REC badge** — files currently being written by `isr.py` show an animated REC indicator, polled every 5 seconds via `/api/status`. Duration for in-progress files shows `—` in the table (header is unfinalized until recording stops). The file list refreshes automatically when a recording starts, stops, or rolls over to a new split file (unless audio is playing).
|
||
- **Listen while recording** — in-progress files are playable and seekable. For WAV the server patches the (still unfinalized) header on the fly so the browser sees the real duration-so-far; reopening the player reloads the source to pick up newly recorded audio. Live responses are sent with `Cache-Control: no-store`.
|
||
- **Fast loading** — analysis results are cached server-side on disk and client-side per session; cached waveforms load only for expanded day groups, and collapsed days fetch nothing until opened.
|
||
- **WCAG-compliant** — skip link, `aria-expanded`/`aria-controls` on the player toggle, `aria-live` status, focus management, `role=img` on SVG waveforms.
|
||
|
||
---
|
||
|
||
## How it works
|
||
|
||
**Streams:** Connect via HTTP → detect format from `Content-Type` → buffer first ~16 KB to extract OGG/FLAC codec headers → stream raw bytes to disk → at each split boundary open a new file and prepend the saved headers. No transcoding, no decoding — raw bytes in, raw bytes out.
|
||
|
||
**Soundcard:** Spawn `arecord` as a subprocess (raw PCM output) → read 100 ms chunks via a thread → write 16-bit PCM to WAV or FLAC → split at configured boundaries.
|
||
|
||
Both recorder types run in separate threads and retry independently up to `max_retries`.
|
||
|
||
---
|
||
|
||
## Docker notes
|
||
|
||
**ALSA device access:** The `recorder` container needs `/dev/snd` mapped. The container runs as root, so no group configuration is needed — the device mapping alone is sufficient.
|
||
|
||
If ALSA still fails to find the device inside the container, verify the device exists on the host:
|
||
```bash
|
||
arecord -l # list capture hardware
|
||
ls -la /dev/snd # check device nodes
|
||
```
|
||
|
||
**Sharing the soundcard with another app (e.g. darkice):** ALSA `hw:` devices are exclusive — only one process can hold them at a time. `asound.conf` in this repo defines a `dsnoop` virtual device (`shared_mic`) that lets multiple processes capture simultaneously:
|
||
|
||
```bash
|
||
# 1. Deploy the ALSA config to the host (once)
|
||
sudo cp asound.conf /etc/asound.conf
|
||
|
||
# 2. Change darkice (or any other app) to use device "shared_mic" instead of hw:0,0
|
||
|
||
# 3. In config.ini set: device = shared_mic
|
||
# docker-compose.yml already mounts asound.conf and sets ipc: host
|
||
# (ipc: host is required so the container shares the host IPC namespace for dsnoop shared memory)
|
||
|
||
# 4. Restart everything
|
||
sudo systemctl restart darkice
|
||
docker compose down && docker compose up -d --build
|
||
```
|
||
|
||
**Stream-only deployments:** If you don't use soundcard recording, remove the `devices` block and the `asound.conf` volume mount and `ipc: host` line from `docker-compose.yml` — the image works fine without them.
|
||
|
||
**Log file in Docker:** The recorder always logs to stdout, so `docker compose logs -f` shows live output. To persist logs on the host, set `log_file = /app/recordings/recorder.log` in `config.ini` (the `recordings` directory is the bind mount).
|
||
|
||
**Analysis cache in Docker:** The web container mounts `./recordings` read-only, so analysis cache files are written to a separate `./analyses` bind mount (mapped to `/analyses` inside the container). This directory is created automatically by Docker Compose on first run. Cache files are stored as `analyses/<filename>.analysis.json` on the host.
|
||
|
||
**File retention:** Individual recordings can be deleted from the web UI. For bulk / automated cleanup, add a cron job on the host:
|
||
```bash
|
||
# Delete recordings older than 30 days
|
||
find recordings/ -type f -mtime +30 -delete
|
||
```
|
||
|
||
---
|
||
|
||
## Licence
|
||
|
||
MIT
|