ISR/README.md

# ISR — Audio Recorder

> AI-generated code. Run at your own risk. MIT licence.

Records from multiple simultaneous sources — Icecast/HTTP streams and ALSA soundcards — with time-based file splitting.

## Features

- Multiple sources recorded in parallel (each in its own thread)
- **Stream recording** — HTTP/Icecast, auto-detects MP3 / OGG / AAC / FLAC / Opus from `Content-Type`
- **Soundcard recording** — ALSA (`arecord`), works on any Linux / Raspberry Pi
- Time-aligned file splits (e.g. every hour, on the hour)
- OGG / Opus / FLAC header injection so every split file is independently playable
- Auto-reconnect on stream drops or device errors
- WAV and FLAC output for soundcard sources
- Web UI to browse, play, cut, and download recordings — including files still being written
- **Loud-event detection** — adaptive: events are flagged by how far they rise above the rolling background noise, not by absolute level, and ranked by score
- **Clip review** — detected sections play as instant server-rendered clips with auto-advance, so a whole day can be reviewed like a highlights reel

---

## Quick start — bare-metal

```bash
pip install requests              # stream recording
pip install numpy soundfile       # FLAC output + web analysis/clips for FLAC (optional)

cp config.example.ini config.ini
# edit config.ini to add your sources
python isr.py                     # start recorder  (Ctrl+C to stop)
python web.py                     # start web UI at http://localhost:8080
```

## Quick start — Docker

```bash
cp config.example.ini config.ini
# edit config.ini to add your sources (no path changes needed for Docker)

docker compose up -d
# recorder starts immediately; web UI at http://<host>:8050
docker compose logs -f            # tail logs from both services
docker compose down               # graceful stop (waits up to 30 s for files to close)
```

Recordings land in `./recordings/` on the host (bind-mounted into both containers at different internal paths — no config changes required vs. bare-metal).

If you only record streams (no soundcard), comment out the `devices` block in `docker-compose.yml`.

**Updating a running deployment:**
```bash
git pull
docker compose down
docker compose up -d --build
```
`docker compose down` is required — Docker won't apply changed volume mounts to existing containers without recreating them.

---

## Configuration

`config.ini` uses standard INI format. `[general]` provides defaults; every other section is a recording source. Source sections inherit all general settings and can override any of them.

### `[general]`

| Key | Default | Description |
|-----|---------|-------------|
| `output_directory` | `recordings` | Output path relative to the working directory (or absolute). The Docker setup mounts `./recordings` at `/app/recordings` so this default works unchanged. |
| `split_minutes` | `60` | Split into a new file every N minutes, aligned to clock boundaries (e.g. 60 → files start at :00, 30 → at :00 and :30). |
| `filename_pattern` | `%Y%m%d_%H%M%S` | strftime pattern; file extension is appended automatically. |
| `max_retries` | `10` | Give up after this many consecutive failures per source. |
| `retry_delay_seconds` | `5` | Wait between retries. |
| `log_level` | `INFO` | `DEBUG` / `INFO` / `WARNING` / `ERROR` / `CRITICAL` |
| `log_file` | `recorder.log` | Log file path. In Docker, logs also go to stdout and are visible via `docker compose logs`. |

### `type = stream`

```ini
[my_stream]
type     = stream
url      = http://icecast.example.com:8000/live
username =          # leave blank for public streams
password =
format   = auto     # auto | mp3 | ogg | aac | flac | opus
```

`format = auto` detects from the `Content-Type` response header. For OGG/Opus/FLAC the first ~16 KB of each connection is buffered to extract codec headers, which are then prepended to every split file — all files are independently playable.

A new file is always opened on (re)connect so gaps between connections are never silently merged.

### `type = soundcard`

```ini
[mic_in]
type        = soundcard
device      = default   # see device selection below
sample_rate = 44100
channels    = 2
format      = wav       # wav | flac
```

**Device selection:**

| Value | Behaviour |
|-------|-----------|
| `default` | System default input |
| `<partial name>` | Case-insensitive substring match against device name |
| `hw:X,Y` | Exact ALSA hardware ID |
| `<pcm name>` | Any ALSA PCM defined in `asound.conf` (e.g. the `shared_mic` dsnoop device), even if it doesn't appear in `arecord -l` |

Run `python isr.py --list-devices` (or `arecord -l`) to see available devices and their IDs.

FLAC output requires `pip install soundfile numpy`.

### Multiple sources

Every section except `[general]` is a source — they all record simultaneously:

```ini
[general]
output_directory = recordings
split_minutes    = 60

[radio1]
type             = stream
url              = http://radio.example.com:8000/stream1
filename_pattern = radio1_%Y%m%d_%H%M%S

[system_audio]
type             = soundcard
device           = hw:0,0
filename_pattern = system_%Y%m%d_%H%M%S
```

---

## Filename patterns

strftime codes are substituted at split time. The file extension is added automatically.

| Pattern | Example |
|---------|---------|
| `%Y%m%d_%H%M%S` | `20241225_143000.mp3` |
| `radio_%Y-%m-%d_%H%M` | `radio_2024-12-25_1430.mp3` |
| `%Y/%m/%d/rec_%H%M%S` | `2024/12/25/rec_143000.mp3` *(subdirs created automatically)* |

---

## Web UI (`web.py`)

```bash
python web.py                              # serves ./recordings on port 8080
python web.py --dir /path/to/audio         # custom recordings directory
python web.py --port 8888                  # custom port
python web.py --margin 15                  # dB above background noise for a section to count as loud (default 12)
python web.py --min-gap 15                 # grace period in seconds for merging loud sections (default 2)
python web.py --analyses-dir /path/to/dir  # where to store analysis cache files (default: <recordings>/analyses)
```

The browser UI (HTML/CSS/JS) lives in `webui.html`, which `web.py` loads at startup — keep the two files together.

Shows recordings grouped by day with collapsible sections. Features:

- **Day groups** — recordings are grouped under a collapsible day heading showing date, file count, total duration, and total size. The most recent day is expanded by default; older days start collapsed. Expanded state is preserved across filter changes.
- **Day highlights** — click **★ Highlights** on any day heading to run loudness analysis across all WAV/FLAC files in that day and display a combined activity timeline SVG. Orange segments show when loud sections occurred relative to the day's time span; blue shows the file extents. Labels show the start, midpoint, and end times. When a day has more sections than fit as chips, the chips show the top 50 by score (loudest-above-background first) so the most promising events are reviewed first; J/K still steps through all sections in time order.
- **Inline playback** — collapsible `▶ Play` button per row; audio loads lazily via a seekable `/stream/` endpoint with HTTP Range support. Metadata is fetched immediately so the duration is visible without pressing play.
- **Waveform analysis** — on demand per file; computes RMS per 100 ms window and marks sections that stand out above the background. Detection is **adaptive**: a rolling noise floor (20th percentile per 30 s block) is estimated across the file, and a section is flagged when the level rises at least *margin* dB (default 12) above that floor. Slow ambience changes — rain setting in, day/night traffic hum — move the floor instead of producing false positives. Each section gets a **score** (its peak dB above the floor) used to rank sections by how much they stand out. Supported for WAV and FLAC (FLAC requires `numpy` + `soundfile`). Pure-Python fallback for WAV when numpy is absent. Results are cached in `recordings/analyses/<filename>.analysis.json`; subsequent requests at the same margin and min-gap settings return instantly without re-reading the audio. The cache file is deleted automatically when the audio file is deleted. Orphaned cache files (audio deleted outside the UI) are pruned on startup.
- **Grace period** — configurable in the controls bar (default 2 s). Loud sections separated by less than this gap are merged into one. Raise this (e.g. to 15–30 s) when a single event generates many timestamps due to brief quiet gaps within it.
- **Clip playback** — clicking a loud-section chip plays a short server-rendered WAV clip (`/api/clip`, pre-roll included) in a player bar at the bottom of the page. Playback starts instantly even for sections deep inside multi-hundred-MB FLACs, because the browser never has to seek the full file. **J** / **K** (or ⏮ / ⏭) step through the queued sections — one file's, or a whole day's after ★ Highlights — and **Auto-advance** plays the next section when one ends, turning a day's detections into a continuous review reel. **⤴ Open in file** switches to the full recording at the same position for context; each chip click also pre-fills the cut panel.
- **Cut & download** — `✂ Cut` button opens the player row and reveals a cut panel. Enter start and end times in `m:ss` or `h:mm:ss` format and click **↓ Download cut** to receive an ffmpeg-trimmed copy without re-encoding. Requires ffmpeg (included in the Docker image).
- **Filters** — live filename search and from/to date pickers above the table; applied client-side with no additional requests. Shows `N of M shown` when a filter is active.
- **Delete** — `✕ Delete` button per row with confirmation prompt; disabled for files currently being recorded; sends `DELETE /api/files/<name>` and re-renders the table.
- **Live REC badge** — files currently being written by `isr.py` show an animated REC indicator, polled every 5 seconds via `/api/status`. Duration for in-progress files shows `—` in the table (header is unfinalized until recording stops). The file list refreshes automatically when a recording starts, stops, or rolls over to a new split file (unless audio is playing).
- **Listen while recording** — in-progress files are playable and seekable. For WAV and FLAC the server patches the (still unfinalized) header on the fly so the browser sees the real duration-so-far — for FLAC the exact sample count is parsed from the last frame header in the file tail. Reopening the player reloads the source to pick up newly recorded audio. Live responses are sent with `Cache-Control: no-store`.
- **Fast loading** — analysis results are cached server-side on disk and client-side per session; cached waveforms load only for expanded day groups, and collapsed days fetch nothing until opened.
- **WCAG-compliant** — skip link, `aria-expanded`/`aria-controls` on the player toggle, `aria-live` status, focus management, `role=img` on SVG waveforms.

### HTTP API

Everything the UI does goes through these endpoints, so they can also be scripted:

| Endpoint | Description |
|----------|-------------|
| `GET /api/files` | File listing with size, mtime, duration, recording state, cached-analysis params |
| `GET /api/analyze?file=&margin=&min_gap=` | Loud-section analysis: `rms_display` (~800-point waveform), scored `sections`, `duration` |
| `GET /api/clip?file=&start=&end=` | Section of a WAV/FLAC decoded server-side, returned as a standalone WAV (max 600 s) |
| `GET /api/cut?file=&start=&end=` | ffmpeg-trimmed copy of the file as a download |
| `GET /stream/<name>` | Inline playback with HTTP Range support; live files get an on-the-fly patched header |
| `GET /download/<name>` | Raw file download |
| `GET /api/status` | Currently recording files (`status.json` passthrough) |
| `GET /api/storage` | Disk free/total |
| `GET /api/config` | Server-side defaults for margin and min-gap (seeds the UI controls) |
| `DELETE /api/files/<name>` | Delete a recording and its analysis cache |

Analysis, clips, cut, and delete return `409` for files that are still being recorded.

---

## How it works

**Streams:** Connect via HTTP → detect format from `Content-Type` → buffer first ~16 KB to extract OGG/FLAC codec headers → stream raw bytes to disk → at each split boundary open a new file and prepend the saved headers. No transcoding, no decoding — raw bytes in, raw bytes out.

**Soundcard:** Spawn `arecord` as a subprocess (raw PCM output) → read 100 ms chunks via a thread → write 16-bit PCM to WAV or FLAC → split at configured boundaries.

Both recorder types run in separate threads and retry independently up to `max_retries`.

---

## Docker notes

**ALSA device access:** The `recorder` container needs `/dev/snd` mapped. The container runs as root, so no group configuration is needed — the device mapping alone is sufficient.

If ALSA still fails to find the device inside the container, verify the device exists on the host:
```bash
arecord -l           # list capture hardware
ls -la /dev/snd      # check device nodes
```

**Sharing the soundcard with another app (e.g. darkice):** ALSA `hw:` devices are exclusive — only one process can hold them at a time. `asound.conf` in this repo defines a `dsnoop` virtual device (`shared_mic`) that lets multiple processes capture simultaneously:

```bash
# 1. Deploy the ALSA config to the host (once)
sudo cp asound.conf /etc/asound.conf

# 2. Change darkice (or any other app) to use device "shared_mic" instead of hw:0,0

# 3. In config.ini set:  device = shared_mic
#    docker-compose.yml already mounts asound.conf and sets ipc: host
#    (ipc: host is required so the container shares the host IPC namespace for dsnoop shared memory)

# 4. Restart everything
sudo systemctl restart darkice
docker compose down && docker compose up -d --build
```

**Stream-only deployments:** If you don't use soundcard recording, remove the `devices` block and the `asound.conf` volume mount and `ipc: host` line from `docker-compose.yml` — the image works fine without them.

**Log file in Docker:** The recorder always logs to stdout, so `docker compose logs -f` shows live output. To persist logs on the host, set `log_file = /app/recordings/recorder.log` in `config.ini` (the `recordings` directory is the bind mount).

**Analysis cache in Docker:** The web container mounts `./recordings` read-only; a separate read-write bind mount of `./recordings/analyses` is layered on top so analysis cache files can still be written. The directory is created automatically by Docker Compose on first run. Cache files are stored as `recordings/analyses/<filename>.analysis.json` on the host.

**File retention:** Individual recordings can be deleted from the web UI. For bulk / automated cleanup, add a cron job on the host:
```bash
# Delete recordings older than 30 days
find recordings/ -type f -mtime +30 -delete
```

---

## Licence

MIT