refactor: split web.py — detector and audio-format helpers into isrweb/
Pure move, no behaviour change. web.py 1221 -> 807 lines.
The reason is not file size but isolation of change. CLAUDE.md records
that the next step for detection, should flagging ever need improving, is
a spectral filter or an optional Silero VAD over candidate sections —
that work now lands entirely in isrweb/loudness.py and never has to open
the HTTP layer. The byte-level format code (FLAC frame parsing, CRC-8,
live header patching) is a second self-contained cluster, touched only
when a container format misbehaves.
isrweb/loudness.py 229 detector + its tuning constants
isrweb/audiofile.py 250 filename clock, WAV/FLAC headers, MIME types
web.py 807 cache, listing, handler, main()
Import direction is strictly web.py -> {loudness, audiofile}; the two
leaves import neither each other nor web.py, which is what lets the
detector be imported and tested without an HTTP server.
A package rather than flat modules because `audiofile` is a real PyPI
name and `detect` is generic enough to collide; a local file silently
shadows an installed one, which fails confusingly rather than loudly.
web.py stays at the repo root so `python web.py`, the docker-compose
command and the webui.html lookup are all unchanged.
Each module keeps its own optional numpy/soundfile try-import rather than
sharing one. That block cannot meaningfully diverge — if numpy is
installed every copy says True — and sharing it would stop isr.py being a
standalone file you can drop on a Pi and run.
Tests mirror the modules: test_loudness.py, test_audiofile.py, test_web.py
(+ unchanged test_isr.py). New tests/conftest.py owns the sys.path setup
that was previously an inline hack in test_isr.py — which only worked for
the other modules because that file happened to import first — and
provides the write_wav fixture the split would otherwise have duplicated.
Verified:
* 78 passed, same count as before the split; each test file also passes
standalone.
* All 21 moved functions diffed against HEAD:web.py — byte-identical.
* Import graph asserted acyclic; both leaves import with `web` absent
from sys.modules.
* An analysis cache written *before* the split still hits afterwards,
with identical sections — the one way a "pure" move could quietly
invalidate every user's cache.
* Endpoint smoke (analyze bounds, cache hit, cached_analysis in the
listing, clip Content-Length, 8-bit rejection) and the byte-serving
harness (ranges, download Range, HEAD, 405, live-header patching whole
and straddling) both re-run green.
* Dockerfile COPY set verified by building the image's file tree in a
temp dir and importing web.py with nothing else present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -5,14 +5,16 @@ Guidance for Claude Code when working in this repository.
|
||||
## Rules
|
||||
|
||||
- **Always update `README.md`** when user-facing behaviour changes (flags, endpoints, Docker setup, features), and **commit it in the same commit** as the code change. README is the external reference; CLAUDE.md documents internals.
|
||||
- Run `python -m pytest tests/` after changing `isr.py` or `web.py` (tests cover the recorder and the loud-section detector).
|
||||
- Run `python -m pytest tests/` after changing `isr.py`, `web.py` or `isrweb/` (tests cover the recorder, the detector, the filename clock and the HTTP layer). Test files mirror the modules: `test_isr.py`, `test_loudness.py`, `test_audiofile.py`, `test_web.py`; `tests/conftest.py` owns the sys.path setup and the `write_wav` fixture.
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `isr.py` | Recorder: streams (Icecast/HTTP) + ALSA soundcards, time-aligned file splits |
|
||||
| `web.py` | Archive browser: HTTP server, file listing, RMS loudness analysis, cut/delete |
|
||||
| `web.py` | Archive browser: HTTP server, file listing, analysis cache, cut/delete. Entry point — stays at the repo root so `python web.py` and the `webui.html` lookup work |
|
||||
| `isrweb/loudness.py` | The adaptive loud-section detector. Pure computation, imports nothing from the app — future spectral/VAD work lands here and never touches HTTP |
|
||||
| `isrweb/audiofile.py` | What an audio file is and what its name means: the filename clock, FLAC/WAV header parsing, live-header patching. Also a leaf |
|
||||
| `webui.html` | Single-page UI (HTML/CSS/JS), loaded by `web.py` at startup — must sit next to `web.py` and be copied in the Dockerfile |
|
||||
| `config.ini` | Recording sources; copy from `config.example.ini`. `[general]` gives defaults, every other section is a source (`type = stream` or `type = soundcard`) |
|
||||
| `asound.conf` | dsnoop device `shared_mic` so ISR and other ALSA apps can share a soundcard |
|
||||
@@ -30,11 +32,17 @@ Dependencies: `requests` (streams), `numpy` + `soundfile` (FLAC output and FLAC
|
||||
|
||||
## Code map
|
||||
|
||||
`web.py`:
|
||||
**Import direction is strictly `web.py → {isrweb.loudness, isrweb.audiofile}`.** The two leaves never import each other or `web.py`; keep it that way, it is what lets the detector be tested without an HTTP server. Each module carries its own optional numpy/soundfile try-import — that boilerplate cannot meaningfully diverge, and sharing it would stop `isr.py` being a standalone file.
|
||||
|
||||
`isrweb/loudness.py`:
|
||||
- Detection: `_compute_rms_windows_wav()` / `analyze_flac()` produce 100 ms RMS windows (at 48 kHz — `WINDOW_SAMPLES` is a fixed sample count, so windows are wider on lower-rate files; `analyze_wav()` rejects anything that is not 16-bit, which the RMS decoder assumes) → `_noise_floor_db()` estimates the rolling floor → `_loud_sections()` emits scored sections → `_package_result()` shapes the `/api/analyze` payload.
|
||||
- Clips: `_api_clip()` validates params, `_clip_wav()` / `_clip_flac()` stream the decoded slice, `_wav_header()` builds the 44-byte PCM header.
|
||||
|
||||
`isrweb/audiofile.py`:
|
||||
- Filenames as a clock: `_recording_start()` parses the start time out of a filename stem; `_cut_filename()` turns a (stem, ext, start, end) into a wall-clock-named cut. Both the listing `date` field and `_api_cut()` use them.
|
||||
- Live headers: `_live_wav_header()`, `_live_flac_header()` (+ `_flac_frame_samples()`, CRC-8 verified).
|
||||
- Live headers: `_live_wav_header()`, `_live_flac_header()` (+ `_flac_frame_samples()`, CRC-8 verified). Plus `_wav_header()` (44-byte PCM header for clips), `_get_audio_duration()`, `MIME_TYPES`, `AUDIO_EXTENSIONS`.
|
||||
|
||||
`web.py`:
|
||||
- Clips: `_api_clip()` validates params, `_clip_wav()` / `_clip_flac()` stream the decoded slice.
|
||||
- Serving: `_serve_bytes()` is the single Range-capable file sender — `_stream()` (live header patch via its `prefix` argument, `no_store`) and `_download()` (`Content-Disposition`) both go through it, and it is what makes `do_HEAD` a body-less `do_GET`. Plus `_copy_to_response()`, `_safe_path()` (path traversal guard).
|
||||
|
||||
`webui.html` (one `<script>` block):
|
||||
|
||||
@@ -11,6 +11,7 @@ COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY isr.py web.py webui.html ./
|
||||
COPY isrweb/ ./isrweb/
|
||||
|
||||
RUN mkdir -p /recordings
|
||||
|
||||
|
||||
@@ -149,7 +149,7 @@ python web.py --min-duration 1 # discard loud sections shorter than
|
||||
python web.py --analyses-dir /path/to/dir # where to store analysis cache files (default: <recordings>/analyses)
|
||||
```
|
||||
|
||||
The browser UI (HTML/CSS/JS) lives in `webui.html`, which `web.py` loads at startup — keep the two files together.
|
||||
The browser UI (HTML/CSS/JS) lives in `webui.html`, which `web.py` loads at startup. `web.py` also imports the `isrweb/` package (the loud-section detector and the audio-format helpers) — keep `web.py`, `webui.html` and `isrweb/` together.
|
||||
|
||||
Shows recordings grouped by day with collapsible sections. Features:
|
||||
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""Supporting modules for web.py (the archive browser)."""
|
||||
@@ -0,0 +1,250 @@
|
||||
"""What an audio file is, and what its name means.
|
||||
|
||||
Two clusters that share a theme — facts about the files on disk — and share
|
||||
the property of being touched only when a container format misbehaves:
|
||||
|
||||
* the filename clock (recordings are named for their *start* time, and that
|
||||
name is the only reliable time anchor the archive has), and
|
||||
* byte-level format handling: the 44-byte PCM header for clips, duration
|
||||
probing, and the live-header patching that lets a browser show a duration
|
||||
for a file still being recorded.
|
||||
|
||||
Imports nothing from the rest of the app.
|
||||
"""
|
||||
|
||||
import struct
|
||||
import wave
|
||||
from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import soundfile as sf
|
||||
SOUNDFILE_AVAILABLE = True
|
||||
except ImportError:
|
||||
SOUNDFILE_AVAILABLE = False
|
||||
|
||||
AUDIO_EXTENSIONS = {'.wav', '.mp3', '.ogg', '.flac', '.aac', '.opus'}
|
||||
|
||||
# Recording filenames encode the start time as this strftime format (kept in
|
||||
# sync with isr.FILENAME_FORMAT). It is the authoritative recording start and
|
||||
# the only reliable clock anchor — mtime drifts to the last write, so cut clip
|
||||
# names and the displayed date are both derived from this.
|
||||
FILENAME_FORMAT = '%Y%m%d_%H%M%S'
|
||||
|
||||
MIME_TYPES = {
|
||||
'.wav': 'audio/wav',
|
||||
'.mp3': 'audio/mpeg',
|
||||
'.ogg': 'audio/ogg',
|
||||
'.flac': 'audio/flac',
|
||||
'.aac': 'audio/aac',
|
||||
'.opus': 'audio/ogg',
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# The filename clock
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _recording_start(stem: str):
|
||||
"""Parse the recording start time encoded in a filename stem.
|
||||
|
||||
Returns a datetime, or None if the stem is not in FILENAME_FORMAT
|
||||
(e.g. a manually renamed file). strptime ignores any extension because
|
||||
callers pass Path.stem.
|
||||
"""
|
||||
try:
|
||||
return datetime.strptime(stem, FILENAME_FORMAT)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _cut_filename(stem: str, ext: str, start: float, end: float) -> str:
|
||||
"""Name a cut by the real wall-clock span it covers.
|
||||
|
||||
For a recording that started at 22:00:00, a 22:31:30→22:32:30 slice
|
||||
(start=1890, end=1950) becomes ``20260523_22-31-30_22-32-30.flac``.
|
||||
Falls back to the source stem plus second offsets when the filename is
|
||||
not in FILENAME_FORMAT (e.g. a manually renamed recording).
|
||||
"""
|
||||
started = _recording_start(stem)
|
||||
if started is None:
|
||||
return f'{stem}_cut_{int(start)}s-{int(end)}s{ext}'
|
||||
cut_start = started + timedelta(seconds=start)
|
||||
cut_end = started + timedelta(seconds=end)
|
||||
return f'{cut_start:%Y%m%d}_{cut_start:%H-%M-%S}_{cut_end:%H-%M-%S}{ext}'
|
||||
|
||||
|
||||
def _live_wav_header(path: Path, size: int):
|
||||
"""Return the WAV header (through the 'data' chunk header) with RIFF and
|
||||
data sizes rewritten to match the current file size, or None.
|
||||
|
||||
While a WAV file is still being recorded its header claims ~0 frames, so
|
||||
browsers show no duration and refuse to seek. Serving a header patched to
|
||||
the bytes recorded so far fixes both; the patch is the same length as the
|
||||
original header, so all byte offsets and Range math stay valid.
|
||||
"""
|
||||
try:
|
||||
with open(path, 'rb') as fh:
|
||||
hdr = fh.read(512)
|
||||
if len(hdr) < 44 or hdr[:4] != b'RIFF' or hdr[8:12] != b'WAVE':
|
||||
return None
|
||||
pos = 12
|
||||
while pos + 8 <= len(hdr):
|
||||
chunk_id = hdr[pos:pos + 4]
|
||||
chunk_size = int.from_bytes(hdr[pos + 4:pos + 8], 'little')
|
||||
if chunk_id == b'data':
|
||||
data_off = pos + 8
|
||||
patched = bytearray(hdr[:data_off])
|
||||
patched[4:8] = (size - 8).to_bytes(4, 'little')
|
||||
patched[pos + 4:pos + 8] = (size - data_off).to_bytes(4, 'little')
|
||||
return bytes(patched)
|
||||
pos += 8 + chunk_size + (chunk_size & 1)
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
# CRC-8 (poly 0x07) used by FLAC frame headers
|
||||
_CRC8_TABLE = []
|
||||
for _i in range(256):
|
||||
_c = _i
|
||||
for _ in range(8):
|
||||
_c = ((_c << 1) ^ 0x07) & 0xFF if _c & 0x80 else (_c << 1) & 0xFF
|
||||
_CRC8_TABLE.append(_c)
|
||||
|
||||
_FLAC_BLOCKSIZES = {1: 192, 2: 576, 3: 1152, 4: 2304, 5: 4608, 8: 256, 9: 512,
|
||||
10: 1024, 11: 2048, 12: 4096, 13: 8192, 14: 16384, 15: 32768}
|
||||
|
||||
|
||||
def _crc8(data: bytes) -> int:
|
||||
crc = 0
|
||||
for b in data:
|
||||
crc = _CRC8_TABLE[crc ^ b]
|
||||
return crc
|
||||
|
||||
|
||||
def _flac_coded_number(buf: bytes, pos: int):
|
||||
"""Decode the UTF-8-style frame/sample number; returns (value, next_pos)."""
|
||||
b0 = buf[pos]
|
||||
if b0 < 0x80:
|
||||
return b0, pos + 1
|
||||
n, mask = 0, 0x40
|
||||
while b0 & mask:
|
||||
n += 1
|
||||
mask >>= 1
|
||||
if n < 1 or n > 6: # 10xxxxxx is not a valid leading byte
|
||||
return None
|
||||
val = b0 & (mask - 1)
|
||||
for i in range(1, n + 1):
|
||||
c = buf[pos + i]
|
||||
if c & 0xC0 != 0x80:
|
||||
return None
|
||||
val = (val << 6) | (c & 0x3F)
|
||||
return val, pos + 1 + n
|
||||
|
||||
|
||||
def _flac_frame_samples(buf: bytes, pos: int, fixed_bs: int):
|
||||
"""If a valid FLAC frame header starts at pos, return the stream sample
|
||||
count through the end of that frame, else None. Validity is confirmed by
|
||||
the header's CRC-8, so false sync matches in compressed data are rejected."""
|
||||
try:
|
||||
variable = buf[pos + 1] & 0x01
|
||||
bs_code = buf[pos + 2] >> 4
|
||||
sr_code = buf[pos + 2] & 0x0F
|
||||
if bs_code == 0 or sr_code == 15 or buf[pos + 3] & 0x01:
|
||||
return None
|
||||
if (buf[pos + 3] >> 4) > 10: # reserved channel assignment
|
||||
return None
|
||||
coded = _flac_coded_number(buf, pos + 4)
|
||||
if coded is None:
|
||||
return None
|
||||
val, p = coded
|
||||
bs = _FLAC_BLOCKSIZES.get(bs_code)
|
||||
if bs_code == 6:
|
||||
bs = buf[p] + 1
|
||||
p += 1
|
||||
elif bs_code == 7:
|
||||
bs = int.from_bytes(buf[p:p + 2], 'big') + 1
|
||||
p += 2
|
||||
if sr_code == 12:
|
||||
p += 1
|
||||
elif sr_code in (13, 14):
|
||||
p += 2
|
||||
if _crc8(buf[pos:p]) != buf[p]:
|
||||
return None
|
||||
if variable: # val is the frame's starting sample number
|
||||
return val + (bs or 0)
|
||||
return val * (fixed_bs or bs or 4096) + (bs or 0)
|
||||
except IndexError:
|
||||
return None
|
||||
|
||||
|
||||
def _live_flac_header(path: Path, size: int):
|
||||
"""Return the first 26 bytes of a FLAC file with STREAMINFO total_samples
|
||||
patched to the samples recorded so far, or None.
|
||||
|
||||
Like _live_wav_header, but FLAC duration cannot be derived from the byte
|
||||
count (variable compression). Instead the sample count is parsed out of
|
||||
the last frame header in the file tail — each FLAC frame carries its own
|
||||
frame/sample number.
|
||||
"""
|
||||
try:
|
||||
with open(path, 'rb') as fh:
|
||||
head = fh.read(42)
|
||||
if len(head) < 42 or head[:4] != b'fLaC':
|
||||
return None
|
||||
# STREAMINFO must be the first metadata block
|
||||
if head[4] & 0x7F != 0 or int.from_bytes(head[5:8], 'big') != 34:
|
||||
return None
|
||||
fixed_bs = int.from_bytes(head[8:10], 'big')
|
||||
|
||||
tail_len = min(size, 65536)
|
||||
fh.seek(size - tail_len)
|
||||
buf = fh.read(tail_len)
|
||||
|
||||
samples = None
|
||||
for i in range(len(buf) - 20, -1, -1):
|
||||
if buf[i] == 0xFF and (buf[i + 1] & 0xFC) == 0xF8:
|
||||
samples = _flac_frame_samples(buf, i, fixed_bs)
|
||||
if samples:
|
||||
break
|
||||
if not samples:
|
||||
return None
|
||||
|
||||
# Bytes 18-25 hold: sample rate (20 bits) | channels-1 (3) |
|
||||
# bps-1 (5) | total_samples (36). Replace only the low 36 bits.
|
||||
field = int.from_bytes(head[18:26], 'big')
|
||||
field = (field & ~((1 << 36) - 1)) | min(samples, (1 << 36) - 1)
|
||||
patched = bytearray(head[:26])
|
||||
patched[18:26] = field.to_bytes(8, 'big')
|
||||
return bytes(patched)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _wav_header(n_frames: int, channels: int, framerate: int, sampwidth: int) -> bytes:
|
||||
"""Standard 44-byte PCM WAV header for a clip of known length."""
|
||||
data_len = n_frames * channels * sampwidth
|
||||
byte_rate = framerate * channels * sampwidth
|
||||
return (b'RIFF' + (36 + data_len).to_bytes(4, 'little') + b'WAVE'
|
||||
+ b'fmt ' + (16).to_bytes(4, 'little')
|
||||
+ struct.pack('<HHIIHH', 1, channels, framerate, byte_rate,
|
||||
channels * sampwidth, sampwidth * 8)
|
||||
+ b'data' + data_len.to_bytes(4, 'little'))
|
||||
|
||||
|
||||
def _get_audio_duration(path: Path):
|
||||
"""Return duration in seconds for any supported audio file, or None."""
|
||||
ext = path.suffix.lower()
|
||||
if ext == '.wav':
|
||||
try:
|
||||
with wave.open(str(path), 'rb') as wf:
|
||||
return round(wf.getnframes() / wf.getframerate(), 2)
|
||||
except Exception:
|
||||
return None
|
||||
if SOUNDFILE_AVAILABLE and ext in ('.flac', '.ogg', '.opus'):
|
||||
try:
|
||||
with sf.SoundFile(path) as f:
|
||||
return round(len(f) / f.samplerate, 2)
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
@@ -0,0 +1,229 @@
|
||||
"""Adaptive loud-section detection.
|
||||
|
||||
Pure computation: RMS windows -> a rolling noise floor -> scored sections.
|
||||
Nothing here knows about HTTP, caching or the archive on disk, and nothing
|
||||
here imports the rest of the app — so the detector can be worked on, and
|
||||
tested, on its own. That matters because this is the part most likely to
|
||||
change: if flagging (rather than ranking) ever needs improving, the next
|
||||
step is a spectral filter or an optional Silero VAD over candidate sections,
|
||||
and it all lands in this file.
|
||||
|
||||
**Detection is adaptive — do not regress it to an absolute threshold.** A
|
||||
fixed RMS threshold flagged every ambience change (passing cars, rain) and
|
||||
produced ~600 useless sections a day; that is why it was replaced.
|
||||
"""
|
||||
|
||||
import math
|
||||
import struct
|
||||
import wave
|
||||
|
||||
try:
|
||||
import numpy as np
|
||||
NUMPY_AVAILABLE = True
|
||||
except ImportError:
|
||||
NUMPY_AVAILABLE = False
|
||||
|
||||
try:
|
||||
import soundfile as sf
|
||||
SOUNDFILE_AVAILABLE = True
|
||||
except ImportError:
|
||||
SOUNDFILE_AVAILABLE = False
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tuning
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
WINDOW_SAMPLES = 4800 # 100 ms at 48 kHz
|
||||
MARGIN_DB = 12.0 # sections must rise this many dB above the noise floor
|
||||
MIN_GAP_SECONDS = 2.0 # merge loud sections separated by less than this
|
||||
MIN_DURATION_SECONDS = 0.5 # discard loud sections shorter than this
|
||||
|
||||
NOISE_BLOCK_SECONDS = 30.0 # noise floor is estimated per block of this length
|
||||
NOISE_PERCENTILE = 20 # percentile of windowed dB levels taken as the floor
|
||||
MIN_RMS = 0.002 # ≈ −54 dBFS; the floor never drops below this, so
|
||||
# digital silence does not make every tiny sound loud
|
||||
ONSET_SECONDS = 0.5 # a section's score is capped by its sharpest dB rise
|
||||
# within this span, so slow swells rank low
|
||||
|
||||
# Bumped whenever section detection/scoring changes in a way that makes old
|
||||
# cached results wrong (not just differently parameterised). Caches written
|
||||
# with another version never match and are recomputed on the next analyse.
|
||||
DETECTOR_VERSION = 3 # v3: a section open at EOF ends at its last loud
|
||||
# window, not at the file duration
|
||||
|
||||
def _compute_rms_windows_wav(wf, channels: int, sampwidth: int, framerate: int,
|
||||
window_samples: int):
|
||||
"""Yield rms_0_to_1 for every window in the open wave file."""
|
||||
while True:
|
||||
raw = wf.readframes(window_samples)
|
||||
if not raw:
|
||||
break
|
||||
n_samp = len(raw) // (sampwidth * channels)
|
||||
if n_samp == 0:
|
||||
break
|
||||
|
||||
if NUMPY_AVAILABLE:
|
||||
arr = np.frombuffer(raw[:n_samp * sampwidth * channels], dtype='<i2')
|
||||
if channels > 1:
|
||||
arr = arr.reshape(-1, channels).mean(axis=1)
|
||||
rms = float(np.sqrt(np.mean(arr.astype(np.float64) ** 2))) / 32768.0
|
||||
else:
|
||||
fmt = f'<{n_samp * channels}h'
|
||||
samples = struct.unpack(fmt, raw[:n_samp * sampwidth * channels])
|
||||
mono = samples[::channels] if channels > 1 else samples
|
||||
rms = math.sqrt(sum(s * s for s in mono) / len(mono)) / 32768.0
|
||||
|
||||
yield round(rms, 5)
|
||||
|
||||
|
||||
def _noise_floor_db(db_values: list, window_dur: float) -> list:
|
||||
"""Per-window background noise floor in dBFS.
|
||||
|
||||
The floor is the NOISE_PERCENTILE-th percentile of the windowed dB levels
|
||||
in each NOISE_BLOCK_SECONDS block, then min-smoothed over ±2 neighbouring
|
||||
blocks so an event spanning a whole block cannot raise its own floor.
|
||||
Tracks slow ambience changes (day/night, rain, traffic hum) so detection
|
||||
is relative to "how loud it normally is right now"."""
|
||||
n = len(db_values)
|
||||
block = max(1, int(round(NOISE_BLOCK_SECONDS / window_dur)))
|
||||
floors = []
|
||||
for i in range(0, n, block):
|
||||
chunk = sorted(db_values[i:i + block])
|
||||
floors.append(chunk[int(len(chunk) * NOISE_PERCENTILE / 100)])
|
||||
smoothed = [min(floors[max(0, b - 2):b + 3]) for b in range(len(floors))]
|
||||
return [smoothed[min(i // block, len(smoothed) - 1)] for i in range(n)]
|
||||
|
||||
|
||||
def _loud_sections(rms_values: list, window_dur: float, duration: float,
|
||||
margin_db: float, min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> list:
|
||||
"""Sections whose level rises at least margin_db above the local noise
|
||||
floor. Each section carries a 'score': its peak dB above the floor,
|
||||
capped by the sharpest rise observed within ONSET_SECONDS. Real events
|
||||
(voices, impacts, barks) have steep onsets, so their cap equals their
|
||||
peak; a swell that drifts up slower than the noise-floor blocks can track
|
||||
(wind, a distant approaching car) still flags but scores near zero, so
|
||||
score-ranked review (chips, U/I highlights) surfaces events first.
|
||||
|
||||
Sections shorter than min_duration (after min_gap merging) are discarded:
|
||||
without this, every isolated 100 ms window that pops above the floor — a
|
||||
click, a single raindrop — becomes its own section and a day can drown in
|
||||
thousands of sub-second clips."""
|
||||
db = [20 * math.log10(max(r, 1e-6)) for r in rms_values]
|
||||
floor = _noise_floor_db(db, window_dur)
|
||||
min_db = 20 * math.log10(MIN_RMS)
|
||||
onset_win = max(1, int(round(ONSET_SECONDS / window_dur)))
|
||||
|
||||
sections = []
|
||||
start_t = None
|
||||
last_loud_t = None
|
||||
peak = 0.0
|
||||
onset = 0.0
|
||||
|
||||
def _score():
|
||||
return round(max(0.0, min(peak, onset)), 1)
|
||||
|
||||
for i, d in enumerate(db):
|
||||
t = i * window_dur
|
||||
floor_eff = max(floor[i], min_db)
|
||||
if d >= floor_eff + margin_db:
|
||||
if start_t is None:
|
||||
start_t = t
|
||||
peak = 0.0
|
||||
onset = 0.0
|
||||
last_loud_t = t
|
||||
peak = max(peak, d - floor_eff)
|
||||
# Rise within the onset span; a section starting before the file
|
||||
# has history is measured against the floor instead (an event cut
|
||||
# off by a file split must not be punished as a swell).
|
||||
rise = d - db[i - onset_win] if i >= onset_win else d - floor_eff
|
||||
onset = max(onset, rise)
|
||||
else:
|
||||
if start_t is not None and (t - last_loud_t) > min_gap:
|
||||
end_t = last_loud_t + window_dur
|
||||
if end_t - start_t >= min_duration - 1e-9:
|
||||
sections.append({'start': round(start_t, 1),
|
||||
'end': round(end_t, 1),
|
||||
'score': _score()})
|
||||
start_t = None
|
||||
last_loud_t = None
|
||||
|
||||
# A section still open at EOF ends at its last loud window, exactly like
|
||||
# every section closed inside the loop — not at the file duration, which
|
||||
# used to stretch it by up to min_gap of trailing silence and put that
|
||||
# silence into the clip served for it.
|
||||
if start_t is not None and (last_loud_t + window_dur - start_t) >= min_duration - 1e-9:
|
||||
sections.append({'start': round(start_t, 1),
|
||||
'end': round(min(last_loud_t + window_dur, duration), 1),
|
||||
'score': _score()})
|
||||
|
||||
return sections
|
||||
|
||||
|
||||
def _package_result(rms_values: list, framerate: int, n_frames: int,
|
||||
window_samples: int, margin_db: float,
|
||||
min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> dict:
|
||||
window_dur = window_samples / framerate
|
||||
duration = n_frames / framerate
|
||||
|
||||
# Note: no RMS data is returned — the UI is screen-reader oriented and
|
||||
# draws no waveform, so sections + duration is all it needs. The window
|
||||
# length went the same way: it only ever existed to place RMS bars.
|
||||
return {
|
||||
'sections': _loud_sections(rms_values, window_dur, duration, margin_db, min_gap, min_duration),
|
||||
'duration': round(duration, 2),
|
||||
}
|
||||
|
||||
|
||||
def analyze_wav(path: Path, window_samples: int = WINDOW_SAMPLES,
|
||||
margin_db: float = MARGIN_DB,
|
||||
min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> dict:
|
||||
try:
|
||||
with wave.open(str(path), 'rb') as wf:
|
||||
channels = wf.getnchannels()
|
||||
sampwidth = wf.getsampwidth()
|
||||
framerate = wf.getframerate()
|
||||
n_frames = wf.getnframes()
|
||||
# _compute_rms_windows_wav decodes as int16 unconditionally, so an
|
||||
# 8- or 24-bit file would yield garbage RMS and therefore garbage
|
||||
# sections — silently. The recorder only writes 16-bit, but --dir
|
||||
# can point at any directory.
|
||||
if sampwidth != 2:
|
||||
return {'error': f'only 16-bit WAV files can be analysed '
|
||||
f'(this one is {sampwidth * 8}-bit)'}
|
||||
rms_values = list(_compute_rms_windows_wav(
|
||||
wf, channels, sampwidth, framerate, window_samples))
|
||||
except Exception as e:
|
||||
return {'error': str(e)}
|
||||
|
||||
return _package_result(rms_values, framerate, n_frames, window_samples, margin_db, min_gap, min_duration)
|
||||
|
||||
|
||||
def analyze_flac(path: Path, window_samples: int = WINDOW_SAMPLES,
|
||||
margin_db: float = MARGIN_DB,
|
||||
min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> dict:
|
||||
"""Analyse a FLAC file for loudness. Requires numpy and soundfile."""
|
||||
if not NUMPY_AVAILABLE or not SOUNDFILE_AVAILABLE:
|
||||
return {'error': 'FLAC analysis requires: pip install numpy soundfile'}
|
||||
|
||||
try:
|
||||
with sf.SoundFile(path) as f:
|
||||
framerate = f.samplerate
|
||||
channels = f.channels
|
||||
n_frames = len(f)
|
||||
|
||||
rms_values = []
|
||||
while True:
|
||||
frames = f.read(window_samples, dtype='float32', always_2d=True)
|
||||
if len(frames) == 0:
|
||||
break
|
||||
mono = frames.mean(axis=1) if channels > 1 else frames[:, 0]
|
||||
rms = float(np.sqrt(np.mean(mono.astype(np.float64) ** 2)))
|
||||
rms_values.append(round(rms, 5))
|
||||
except Exception as e:
|
||||
return {'error': str(e)}
|
||||
|
||||
return _package_result(rms_values, framerate, n_frames, window_samples, margin_db, min_gap, min_duration)
|
||||
@@ -0,0 +1,30 @@
|
||||
"""Shared test setup.
|
||||
|
||||
Puts the repo root on sys.path so `import isr`, `import web` and
|
||||
`import isrweb.*` work regardless of where pytest is invoked from. This used
|
||||
to be an inline sys.path.insert at the top of test_isr.py, which only
|
||||
happened to work for the other test modules because that file imported first.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import wave
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def write_wav():
|
||||
"""Factory for small 16-bit WAV files. Returns the path written."""
|
||||
def _write(path, seconds=2, framerate=8000, channels=1, sampwidth=2,
|
||||
frames=None):
|
||||
with wave.open(str(path), 'wb') as wf:
|
||||
wf.setnchannels(channels)
|
||||
wf.setsampwidth(sampwidth)
|
||||
wf.setframerate(framerate)
|
||||
wf.writeframes(frames if frames is not None
|
||||
else b'\x00\x00' * (seconds * framerate * channels))
|
||||
return path
|
||||
return _write
|
||||
@@ -0,0 +1,35 @@
|
||||
"""Tests for the filename clock in isrweb/audiofile.py.
|
||||
|
||||
Recordings are named for their *start* time, and that name is the only
|
||||
reliable time anchor the archive has — mtime drifts to the last write.
|
||||
"""
|
||||
|
||||
from isrweb.audiofile import _cut_filename, _recording_start
|
||||
|
||||
|
||||
def test_recording_start_parses_standard_name():
|
||||
from datetime import datetime
|
||||
assert _recording_start("20260523_220000") == datetime(2026, 5, 23, 22, 0, 0)
|
||||
|
||||
|
||||
def test_recording_start_rejects_nonstandard_name():
|
||||
assert _recording_start("radio1_20260523") is None
|
||||
assert _recording_start("notes") is None
|
||||
|
||||
|
||||
def test_cut_filename_uses_wall_clock_span():
|
||||
# Recording started 22:00:00; cut covers 22:31:30 → 22:32:30.
|
||||
name = _cut_filename("20260523_220000", ".flac", 1890, 1950)
|
||||
assert name == "20260523_22-31-30_22-32-30.flac"
|
||||
|
||||
|
||||
def test_cut_filename_rolls_over_the_hour():
|
||||
name = _cut_filename("20260523_220000", ".wav", 3590, 3661)
|
||||
assert name == "20260523_22-59-50_23-01-01.wav"
|
||||
|
||||
|
||||
def test_cut_filename_falls_back_for_nonstandard_name():
|
||||
name = _cut_filename("mixtape", ".mp3", 740, 750.4)
|
||||
assert name == "mixtape_cut_740s-750s.mp3"
|
||||
|
||||
|
||||
+2
-6
@@ -31,14 +31,10 @@ def fixed_clock(dt: datetime):
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Import the module under test
|
||||
# Import the module under test (sys.path is set up in conftest.py)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
import sys
|
||||
import os
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
|
||||
|
||||
import isr # noqa: E402 (must come after sys.path manipulation)
|
||||
import isr
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
|
||||
@@ -0,0 +1,176 @@
|
||||
"""Tests for the adaptive loud-section detector in isrweb/loudness.py."""
|
||||
|
||||
import math
|
||||
import wave
|
||||
|
||||
from isrweb.loudness import _loud_sections, _noise_floor_db, analyze_wav
|
||||
|
||||
WINDOW_DUR = 0.1 # 100 ms windows, as produced by WINDOW_SAMPLES at 48 kHz
|
||||
|
||||
|
||||
def _run(rms, margin_db=12.0, min_gap=2.0, min_duration=0.5):
|
||||
duration = len(rms) * WINDOW_DUR
|
||||
return _loud_sections(rms, WINDOW_DUR, duration, margin_db, min_gap, min_duration)
|
||||
|
||||
|
||||
def test_burst_above_quiet_floor_is_detected():
|
||||
rms = [0.002] * 1200 # 2 min of quiet ambience (−54 dBFS)
|
||||
rms[600:610] = [0.05] * 10 # 1 s burst at −26 dBFS (+28 dB)
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
s = sections[0]
|
||||
assert s['start'] == 60.0
|
||||
assert 60.5 <= s['end'] <= 62.0
|
||||
assert 26.0 <= s['score'] <= 30.0
|
||||
|
||||
|
||||
def test_slow_ambience_swell_is_not_detected():
|
||||
# Level rises 20 dB over 10 minutes and back down — e.g. rain setting in.
|
||||
# The old fixed threshold would have flagged the entire loud half.
|
||||
up = [0.002 * 10 ** (i / 6000 * 1.0) for i in range(6000)] # −54 → −34 dB
|
||||
down = list(reversed(up))
|
||||
assert _run(up + down) == []
|
||||
|
||||
|
||||
def test_burst_still_detected_on_loud_ambience():
|
||||
# Same +28 dB prominence as the quiet test, but on a 20 dB louder floor.
|
||||
rms = [0.02] * 1200
|
||||
rms[600:610] = [0.5] * 10
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 60.0
|
||||
|
||||
|
||||
def test_min_rms_floor_suppresses_blips_in_digital_silence():
|
||||
rms = [0.000001] * 1200
|
||||
rms[600:605] = [0.004] * 5 # −48 dBFS: audible blip, but below MIN_RMS+12dB
|
||||
assert _run(rms) == []
|
||||
rms[600:605] = [0.05] * 5 # −26 dBFS clearly clears the clamped floor
|
||||
assert len(_run(rms)) == 1
|
||||
|
||||
|
||||
def test_min_gap_merges_nearby_bursts():
|
||||
rms = [0.002] * 1200
|
||||
rms[600:605] = [0.05] * 5
|
||||
rms[615:620] = [0.05] * 5 # 1 s gap < 2 s min_gap → one section
|
||||
rms[900:905] = [0.05] * 5 # 28 s away → separate section
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 2
|
||||
assert sections[0]['start'] == 60.0
|
||||
assert sections[0]['end'] >= 61.5
|
||||
assert sections[1]['start'] == 90.0
|
||||
|
||||
|
||||
def test_min_duration_drops_subsecond_blips():
|
||||
# Isolated single-window pops (clicks, single raindrops) spaced wider than
|
||||
# min_gap must not each become their own section — this is what used to
|
||||
# produce thousands of zero-length sections per day.
|
||||
rms = [0.002] * 1200
|
||||
for i in range(600, 660, 30): # 0.1 s blips, 3 s apart (> min_gap)
|
||||
rms[i] = 0.05
|
||||
assert _run(rms) == []
|
||||
# With the filter disabled they are all reported
|
||||
assert len(_run(rms, min_duration=0.0)) == 2
|
||||
|
||||
|
||||
def test_min_duration_keeps_sections_at_or_above_it():
|
||||
rms = [0.002] * 1200
|
||||
rms[600:605] = [0.05] * 5 # exactly 0.5 s
|
||||
sections = _run(rms, min_duration=0.5)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 60.0
|
||||
|
||||
|
||||
def test_min_duration_applies_after_gap_merging():
|
||||
# Two sub-min_duration blips within min_gap merge into one section whose
|
||||
# loud span exceeds min_duration — the merged section must survive.
|
||||
rms = [0.002] * 1200
|
||||
rms[600] = 0.05
|
||||
rms[610] = 0.05 # 1 s apart < 2 s min_gap → merged, 1.1 s span
|
||||
sections = _run(rms, min_duration=1.0)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 60.0
|
||||
assert sections[0]['end'] >= 61.0
|
||||
|
||||
|
||||
def test_slow_swell_scores_near_zero_sharp_burst_scores_high():
|
||||
# A ~10 s swell rises faster than the 30 s floor blocks can track, so it
|
||||
# still flags — but its score must collapse (onset cap) so it ranks at the
|
||||
# bottom, while a sharp burst with the same peak keeps its full score.
|
||||
rms = [0.002] * 1200
|
||||
for k in range(80): # rise −54 → −28 dB over 8 s
|
||||
rms[400 + k] = 0.002 * 10 ** (26 * (k / 80) / 20)
|
||||
rms[480:500] = [0.04] * 20 # hold 2 s at −28 dB
|
||||
for k in range(80): # fall back over 8 s
|
||||
rms[500 + k] = 0.04 * 10 ** (-26 * (k / 80) / 20)
|
||||
rms[900:910] = [0.04] * 10 # sharp 1 s burst, same peak
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 2
|
||||
swell, burst = sections
|
||||
assert swell['start'] < 50.0 < 90.0 <= burst['start']
|
||||
assert burst['score'] >= 24.0 # ≈ full 26 dB prominence
|
||||
assert swell['score'] <= 5.0 # capped by its slow onset
|
||||
assert swell['score'] < burst['score']
|
||||
|
||||
|
||||
def test_burst_at_file_start_keeps_full_score():
|
||||
# No pre-event history to measure a rise against: the onset cap falls back
|
||||
# to prominence above the floor, so events cut off by a file split are not
|
||||
# punished as swells.
|
||||
rms = [0.05] * 10 + [0.002] * 1190
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 0.0
|
||||
assert sections[0]['score'] >= 24.0
|
||||
|
||||
|
||||
def test_section_open_at_eof_ends_at_last_loud_window():
|
||||
# Burst ends 1 s before EOF — less than min_gap, so the section is still
|
||||
# open when the windows run out. It must end where the sound ended, not at
|
||||
# the file duration; otherwise the clip carries a second of trailing
|
||||
# silence and disagrees with every section closed mid-file.
|
||||
rms = [0.002] * 1200
|
||||
rms[1180:1190] = [0.05] * 10 # loud 118.0 → 119.0, file is 120.0 s
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 118.0
|
||||
assert sections[0]['end'] == 119.0
|
||||
|
||||
|
||||
def test_section_running_to_the_very_end_is_capped_at_duration():
|
||||
rms = [0.002] * 1200
|
||||
rms[1190:1200] = [0.05] * 10 # loud right up to EOF
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['end'] == 120.0
|
||||
|
||||
|
||||
def test_noise_floor_tracks_blocks_and_ignores_short_events():
|
||||
quiet_db = 20 * math.log10(0.002)
|
||||
db = [quiet_db] * 1200
|
||||
db[600:650] = [-20.0] * 50 # 5 s event must not raise its own floor
|
||||
floor = _noise_floor_db(db, WINDOW_DUR)
|
||||
assert len(floor) == len(db)
|
||||
assert all(abs(f - quiet_db) < 1.0 for f in floor)
|
||||
|
||||
|
||||
def test_non_16_bit_wav_is_rejected_rather_than_misread(tmp_path, write_wav):
|
||||
# RMS windowing decodes as int16 regardless of the declared sample width,
|
||||
# so an 8-bit file used to produce meaningless sections with no warning.
|
||||
path = tmp_path / 'eight_bit.wav'
|
||||
with wave.open(str(path), 'wb') as wf:
|
||||
wf.setnchannels(1)
|
||||
wf.setsampwidth(1)
|
||||
wf.setframerate(8000)
|
||||
wf.writeframes(b'\x80' * 8000)
|
||||
result = analyze_wav(path)
|
||||
assert 'error' in result
|
||||
assert '16-bit' in result['error']
|
||||
|
||||
|
||||
def test_16_bit_wav_still_analyses(tmp_path, write_wav):
|
||||
path = tmp_path / 'ok.wav'
|
||||
write_wav(path)
|
||||
assert 'error' not in analyze_wav(path)
|
||||
|
||||
|
||||
+15
-224
@@ -1,189 +1,21 @@
|
||||
"""Tests for the adaptive loud-section detector in web.py."""
|
||||
"""Tests for the HTTP layer and file listing in web.py."""
|
||||
|
||||
import io
|
||||
import math
|
||||
import wave
|
||||
|
||||
import web
|
||||
from web import _cut_filename, _loud_sections, _noise_floor_db, _recording_start
|
||||
|
||||
WINDOW_DUR = 0.1 # 100 ms windows, as produced by WINDOW_SAMPLES at 48 kHz
|
||||
|
||||
|
||||
def _run(rms, margin_db=12.0, min_gap=2.0, min_duration=0.5):
|
||||
duration = len(rms) * WINDOW_DUR
|
||||
return _loud_sections(rms, WINDOW_DUR, duration, margin_db, min_gap, min_duration)
|
||||
def test_listing_is_ordered_by_the_filename_clock_not_mtime(tmp_path, write_wav):
|
||||
# mtime is the last write (≈ the end of the recording) and drifts from the
|
||||
# name on any copy or round-trip. Here the older recording is touched last.
|
||||
import os
|
||||
early, late = tmp_path / '20260523_080000.wav', tmp_path / '20260523_090000.wav'
|
||||
write_wav(early)
|
||||
write_wav(late)
|
||||
os.utime(late, (1000, 1000)) # newest recording, oldest mtime
|
||||
|
||||
|
||||
def test_burst_above_quiet_floor_is_detected():
|
||||
rms = [0.002] * 1200 # 2 min of quiet ambience (−54 dBFS)
|
||||
rms[600:610] = [0.05] * 10 # 1 s burst at −26 dBFS (+28 dB)
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
s = sections[0]
|
||||
assert s['start'] == 60.0
|
||||
assert 60.5 <= s['end'] <= 62.0
|
||||
assert 26.0 <= s['score'] <= 30.0
|
||||
|
||||
|
||||
def test_slow_ambience_swell_is_not_detected():
|
||||
# Level rises 20 dB over 10 minutes and back down — e.g. rain setting in.
|
||||
# The old fixed threshold would have flagged the entire loud half.
|
||||
up = [0.002 * 10 ** (i / 6000 * 1.0) for i in range(6000)] # −54 → −34 dB
|
||||
down = list(reversed(up))
|
||||
assert _run(up + down) == []
|
||||
|
||||
|
||||
def test_burst_still_detected_on_loud_ambience():
|
||||
# Same +28 dB prominence as the quiet test, but on a 20 dB louder floor.
|
||||
rms = [0.02] * 1200
|
||||
rms[600:610] = [0.5] * 10
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 60.0
|
||||
|
||||
|
||||
def test_min_rms_floor_suppresses_blips_in_digital_silence():
|
||||
rms = [0.000001] * 1200
|
||||
rms[600:605] = [0.004] * 5 # −48 dBFS: audible blip, but below MIN_RMS+12dB
|
||||
assert _run(rms) == []
|
||||
rms[600:605] = [0.05] * 5 # −26 dBFS clearly clears the clamped floor
|
||||
assert len(_run(rms)) == 1
|
||||
|
||||
|
||||
def test_min_gap_merges_nearby_bursts():
|
||||
rms = [0.002] * 1200
|
||||
rms[600:605] = [0.05] * 5
|
||||
rms[615:620] = [0.05] * 5 # 1 s gap < 2 s min_gap → one section
|
||||
rms[900:905] = [0.05] * 5 # 28 s away → separate section
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 2
|
||||
assert sections[0]['start'] == 60.0
|
||||
assert sections[0]['end'] >= 61.5
|
||||
assert sections[1]['start'] == 90.0
|
||||
|
||||
|
||||
def test_min_duration_drops_subsecond_blips():
|
||||
# Isolated single-window pops (clicks, single raindrops) spaced wider than
|
||||
# min_gap must not each become their own section — this is what used to
|
||||
# produce thousands of zero-length sections per day.
|
||||
rms = [0.002] * 1200
|
||||
for i in range(600, 660, 30): # 0.1 s blips, 3 s apart (> min_gap)
|
||||
rms[i] = 0.05
|
||||
assert _run(rms) == []
|
||||
# With the filter disabled they are all reported
|
||||
assert len(_run(rms, min_duration=0.0)) == 2
|
||||
|
||||
|
||||
def test_min_duration_keeps_sections_at_or_above_it():
|
||||
rms = [0.002] * 1200
|
||||
rms[600:605] = [0.05] * 5 # exactly 0.5 s
|
||||
sections = _run(rms, min_duration=0.5)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 60.0
|
||||
|
||||
|
||||
def test_min_duration_applies_after_gap_merging():
|
||||
# Two sub-min_duration blips within min_gap merge into one section whose
|
||||
# loud span exceeds min_duration — the merged section must survive.
|
||||
rms = [0.002] * 1200
|
||||
rms[600] = 0.05
|
||||
rms[610] = 0.05 # 1 s apart < 2 s min_gap → merged, 1.1 s span
|
||||
sections = _run(rms, min_duration=1.0)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 60.0
|
||||
assert sections[0]['end'] >= 61.0
|
||||
|
||||
|
||||
def test_slow_swell_scores_near_zero_sharp_burst_scores_high():
|
||||
# A ~10 s swell rises faster than the 30 s floor blocks can track, so it
|
||||
# still flags — but its score must collapse (onset cap) so it ranks at the
|
||||
# bottom, while a sharp burst with the same peak keeps its full score.
|
||||
rms = [0.002] * 1200
|
||||
for k in range(80): # rise −54 → −28 dB over 8 s
|
||||
rms[400 + k] = 0.002 * 10 ** (26 * (k / 80) / 20)
|
||||
rms[480:500] = [0.04] * 20 # hold 2 s at −28 dB
|
||||
for k in range(80): # fall back over 8 s
|
||||
rms[500 + k] = 0.04 * 10 ** (-26 * (k / 80) / 20)
|
||||
rms[900:910] = [0.04] * 10 # sharp 1 s burst, same peak
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 2
|
||||
swell, burst = sections
|
||||
assert swell['start'] < 50.0 < 90.0 <= burst['start']
|
||||
assert burst['score'] >= 24.0 # ≈ full 26 dB prominence
|
||||
assert swell['score'] <= 5.0 # capped by its slow onset
|
||||
assert swell['score'] < burst['score']
|
||||
|
||||
|
||||
def test_burst_at_file_start_keeps_full_score():
|
||||
# No pre-event history to measure a rise against: the onset cap falls back
|
||||
# to prominence above the floor, so events cut off by a file split are not
|
||||
# punished as swells.
|
||||
rms = [0.05] * 10 + [0.002] * 1190
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 0.0
|
||||
assert sections[0]['score'] >= 24.0
|
||||
|
||||
|
||||
def test_section_open_at_eof_ends_at_last_loud_window():
|
||||
# Burst ends 1 s before EOF — less than min_gap, so the section is still
|
||||
# open when the windows run out. It must end where the sound ended, not at
|
||||
# the file duration; otherwise the clip carries a second of trailing
|
||||
# silence and disagrees with every section closed mid-file.
|
||||
rms = [0.002] * 1200
|
||||
rms[1180:1190] = [0.05] * 10 # loud 118.0 → 119.0, file is 120.0 s
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['start'] == 118.0
|
||||
assert sections[0]['end'] == 119.0
|
||||
|
||||
|
||||
def test_section_running_to_the_very_end_is_capped_at_duration():
|
||||
rms = [0.002] * 1200
|
||||
rms[1190:1200] = [0.05] * 10 # loud right up to EOF
|
||||
sections = _run(rms)
|
||||
assert len(sections) == 1
|
||||
assert sections[0]['end'] == 120.0
|
||||
|
||||
|
||||
def test_noise_floor_tracks_blocks_and_ignores_short_events():
|
||||
quiet_db = 20 * math.log10(0.002)
|
||||
db = [quiet_db] * 1200
|
||||
db[600:650] = [-20.0] * 50 # 5 s event must not raise its own floor
|
||||
floor = _noise_floor_db(db, WINDOW_DUR)
|
||||
assert len(floor) == len(db)
|
||||
assert all(abs(f - quiet_db) < 1.0 for f in floor)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Filename parsing / cut naming
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_recording_start_parses_standard_name():
|
||||
from datetime import datetime
|
||||
assert _recording_start("20260523_220000") == datetime(2026, 5, 23, 22, 0, 0)
|
||||
|
||||
|
||||
def test_recording_start_rejects_nonstandard_name():
|
||||
assert _recording_start("radio1_20260523") is None
|
||||
assert _recording_start("notes") is None
|
||||
|
||||
|
||||
def test_cut_filename_uses_wall_clock_span():
|
||||
# Recording started 22:00:00; cut covers 22:31:30 → 22:32:30.
|
||||
name = _cut_filename("20260523_220000", ".flac", 1890, 1950)
|
||||
assert name == "20260523_22-31-30_22-32-30.flac"
|
||||
|
||||
|
||||
def test_cut_filename_rolls_over_the_hour():
|
||||
name = _cut_filename("20260523_220000", ".wav", 3590, 3661)
|
||||
assert name == "20260523_22-59-50_23-01-01.wav"
|
||||
|
||||
|
||||
def test_cut_filename_falls_back_for_nonstandard_name():
|
||||
name = _cut_filename("mixtape", ".mp3", 740, 750.4)
|
||||
assert name == "mixtape_cut_740s-750s.mp3"
|
||||
names = [f['name'] for f in web.list_files(str(tmp_path))]
|
||||
assert names == ['20260523_090000.wav', '20260523_080000.wav']
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -211,52 +43,11 @@ class _OfflineHandler(web._Handler):
|
||||
pass
|
||||
|
||||
|
||||
def _write_wav(path, seconds=2, framerate=8000):
|
||||
with wave.open(str(path), 'wb') as wf:
|
||||
wf.setnchannels(1)
|
||||
wf.setsampwidth(2)
|
||||
wf.setframerate(framerate)
|
||||
wf.writeframes(b'\x00\x00' * (seconds * framerate))
|
||||
|
||||
|
||||
def test_listing_is_ordered_by_the_filename_clock_not_mtime(tmp_path):
|
||||
# mtime is the last write (≈ the end of the recording) and drifts from the
|
||||
# name on any copy or round-trip. Here the older recording is touched last.
|
||||
import os
|
||||
early, late = tmp_path / '20260523_080000.wav', tmp_path / '20260523_090000.wav'
|
||||
_write_wav(early)
|
||||
_write_wav(late)
|
||||
os.utime(late, (1000, 1000)) # newest recording, oldest mtime
|
||||
|
||||
names = [f['name'] for f in web.list_files(str(tmp_path))]
|
||||
assert names == ['20260523_090000.wav', '20260523_080000.wav']
|
||||
|
||||
|
||||
def test_non_16_bit_wav_is_rejected_rather_than_misread(tmp_path):
|
||||
# RMS windowing decodes as int16 regardless of the declared sample width,
|
||||
# so an 8-bit file used to produce meaningless sections with no warning.
|
||||
path = tmp_path / 'eight_bit.wav'
|
||||
with wave.open(str(path), 'wb') as wf:
|
||||
wf.setnchannels(1)
|
||||
wf.setsampwidth(1)
|
||||
wf.setframerate(8000)
|
||||
wf.writeframes(b'\x80' * 8000)
|
||||
result = web.analyze_wav(path)
|
||||
assert 'error' in result
|
||||
assert '16-bit' in result['error']
|
||||
|
||||
|
||||
def test_16_bit_wav_still_analyses(tmp_path):
|
||||
path = tmp_path / 'ok.wav'
|
||||
_write_wav(path)
|
||||
assert 'error' not in web.analyze_wav(path)
|
||||
|
||||
|
||||
def test_clip_failure_after_headers_closes_connection_instead_of_writing_json(tmp_path):
|
||||
def test_clip_failure_after_headers_closes_connection_instead_of_writing_json(tmp_path, write_wav):
|
||||
# Once the audio headers are out, an error body would be a second response
|
||||
# on a keep-alive connection that already promised audio/wav plus an exact
|
||||
# Content-Length — desyncing every later request on that connection.
|
||||
_write_wav(tmp_path / 'rec.wav')
|
||||
write_wav(tmp_path / 'rec.wav')
|
||||
h = _OfflineHandler(tmp_path)
|
||||
|
||||
def boom(path, start, end):
|
||||
@@ -271,8 +62,8 @@ def test_clip_failure_after_headers_closes_connection_instead_of_writing_json(tm
|
||||
assert b'error' not in h.wfile.getvalue() # no JSON body
|
||||
|
||||
|
||||
def test_clip_failure_before_headers_still_reports_json(tmp_path):
|
||||
_write_wav(tmp_path / 'rec.wav')
|
||||
def test_clip_failure_before_headers_still_reports_json(tmp_path, write_wav):
|
||||
write_wav(tmp_path / 'rec.wav')
|
||||
h = _OfflineHandler(tmp_path)
|
||||
|
||||
def boom(path, start, end):
|
||||
|
||||
@@ -15,24 +15,22 @@ Usage:
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import wave
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import datetime
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import parse_qs, unquote, urlparse
|
||||
|
||||
try:
|
||||
import numpy as np
|
||||
NUMPY_AVAILABLE = True
|
||||
import numpy # noqa: F401 — only the availability flag is used here,
|
||||
NUMPY_AVAILABLE = True # to refuse FLAC work rather than fail halfway
|
||||
except ImportError:
|
||||
NUMPY_AVAILABLE = False
|
||||
|
||||
@@ -42,434 +40,22 @@ try:
|
||||
except ImportError:
|
||||
SOUNDFILE_AVAILABLE = False
|
||||
|
||||
# The detector and the byte-level format helpers live beside this file. Both
|
||||
# are leaves: they import nothing from here, so `web.py -> {loudness,
|
||||
# audiofile}` is the only import edge and there are no cycles.
|
||||
from isrweb.audiofile import (AUDIO_EXTENSIONS, MIME_TYPES, _cut_filename,
|
||||
_get_audio_duration, _live_flac_header,
|
||||
_live_wav_header, _recording_start, _wav_header)
|
||||
from isrweb.loudness import (DETECTOR_VERSION, MARGIN_DB, MIN_DURATION_SECONDS,
|
||||
MIN_GAP_SECONDS, analyze_flac, analyze_wav)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
AUDIO_EXTENSIONS = {'.wav', '.mp3', '.ogg', '.flac', '.aac', '.opus'}
|
||||
WINDOW_SAMPLES = 4800 # 100 ms at 48 kHz
|
||||
MARGIN_DB = 12.0 # sections must rise this many dB above the noise floor
|
||||
MIN_GAP_SECONDS = 2.0 # merge loud sections separated by less than this
|
||||
MIN_DURATION_SECONDS = 0.5 # discard loud sections shorter than this
|
||||
|
||||
NOISE_BLOCK_SECONDS = 30.0 # noise floor is estimated per block of this length
|
||||
NOISE_PERCENTILE = 20 # percentile of windowed dB levels taken as the floor
|
||||
MIN_RMS = 0.002 # ≈ −54 dBFS; the floor never drops below this, so
|
||||
# digital silence does not make every tiny sound loud
|
||||
ONSET_SECONDS = 0.5 # a section's score is capped by its sharpest dB rise
|
||||
# within this span, so slow swells rank low
|
||||
|
||||
# Bumped whenever section detection/scoring changes in a way that makes old
|
||||
# cached results wrong (not just differently parameterised). Caches written
|
||||
# with another version never match and are recomputed on the next analyse.
|
||||
DETECTOR_VERSION = 3 # v3: a section open at EOF ends at its last loud
|
||||
# window, not at the file duration
|
||||
|
||||
CLIP_MAX_SECONDS = 600 # upper bound on /api/clip length
|
||||
|
||||
# Recording filenames encode the start time as this strftime format (kept in
|
||||
# sync with isr.FILENAME_FORMAT). It is the authoritative recording start and
|
||||
# the only reliable clock anchor — mtime drifts to the last write, so cut clip
|
||||
# names and the displayed date are both derived from this.
|
||||
FILENAME_FORMAT = '%Y%m%d_%H%M%S'
|
||||
|
||||
MIME_TYPES = {
|
||||
'.wav': 'audio/wav',
|
||||
'.mp3': 'audio/mpeg',
|
||||
'.ogg': 'audio/ogg',
|
||||
'.flac': 'audio/flac',
|
||||
'.aac': 'audio/aac',
|
||||
'.opus': 'audio/ogg',
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Audio analysis helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _recording_start(stem: str):
|
||||
"""Parse the recording start time encoded in a filename stem.
|
||||
|
||||
Returns a datetime, or None if the stem is not in FILENAME_FORMAT
|
||||
(e.g. a manually renamed file). strptime ignores any extension because
|
||||
callers pass Path.stem.
|
||||
"""
|
||||
try:
|
||||
return datetime.strptime(stem, FILENAME_FORMAT)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _cut_filename(stem: str, ext: str, start: float, end: float) -> str:
|
||||
"""Name a cut by the real wall-clock span it covers.
|
||||
|
||||
For a recording that started at 22:00:00, a 22:31:30→22:32:30 slice
|
||||
(start=1890, end=1950) becomes ``20260523_22-31-30_22-32-30.flac``.
|
||||
Falls back to the source stem plus second offsets when the filename is
|
||||
not in FILENAME_FORMAT (e.g. a manually renamed recording).
|
||||
"""
|
||||
started = _recording_start(stem)
|
||||
if started is None:
|
||||
return f'{stem}_cut_{int(start)}s-{int(end)}s{ext}'
|
||||
cut_start = started + timedelta(seconds=start)
|
||||
cut_end = started + timedelta(seconds=end)
|
||||
return f'{cut_start:%Y%m%d}_{cut_start:%H-%M-%S}_{cut_end:%H-%M-%S}{ext}'
|
||||
|
||||
|
||||
def _live_wav_header(path: Path, size: int):
|
||||
"""Return the WAV header (through the 'data' chunk header) with RIFF and
|
||||
data sizes rewritten to match the current file size, or None.
|
||||
|
||||
While a WAV file is still being recorded its header claims ~0 frames, so
|
||||
browsers show no duration and refuse to seek. Serving a header patched to
|
||||
the bytes recorded so far fixes both; the patch is the same length as the
|
||||
original header, so all byte offsets and Range math stay valid.
|
||||
"""
|
||||
try:
|
||||
with open(path, 'rb') as fh:
|
||||
hdr = fh.read(512)
|
||||
if len(hdr) < 44 or hdr[:4] != b'RIFF' or hdr[8:12] != b'WAVE':
|
||||
return None
|
||||
pos = 12
|
||||
while pos + 8 <= len(hdr):
|
||||
chunk_id = hdr[pos:pos + 4]
|
||||
chunk_size = int.from_bytes(hdr[pos + 4:pos + 8], 'little')
|
||||
if chunk_id == b'data':
|
||||
data_off = pos + 8
|
||||
patched = bytearray(hdr[:data_off])
|
||||
patched[4:8] = (size - 8).to_bytes(4, 'little')
|
||||
patched[pos + 4:pos + 8] = (size - data_off).to_bytes(4, 'little')
|
||||
return bytes(patched)
|
||||
pos += 8 + chunk_size + (chunk_size & 1)
|
||||
return None
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
# CRC-8 (poly 0x07) used by FLAC frame headers
|
||||
_CRC8_TABLE = []
|
||||
for _i in range(256):
|
||||
_c = _i
|
||||
for _ in range(8):
|
||||
_c = ((_c << 1) ^ 0x07) & 0xFF if _c & 0x80 else (_c << 1) & 0xFF
|
||||
_CRC8_TABLE.append(_c)
|
||||
|
||||
_FLAC_BLOCKSIZES = {1: 192, 2: 576, 3: 1152, 4: 2304, 5: 4608, 8: 256, 9: 512,
|
||||
10: 1024, 11: 2048, 12: 4096, 13: 8192, 14: 16384, 15: 32768}
|
||||
|
||||
|
||||
def _crc8(data: bytes) -> int:
|
||||
crc = 0
|
||||
for b in data:
|
||||
crc = _CRC8_TABLE[crc ^ b]
|
||||
return crc
|
||||
|
||||
|
||||
def _flac_coded_number(buf: bytes, pos: int):
|
||||
"""Decode the UTF-8-style frame/sample number; returns (value, next_pos)."""
|
||||
b0 = buf[pos]
|
||||
if b0 < 0x80:
|
||||
return b0, pos + 1
|
||||
n, mask = 0, 0x40
|
||||
while b0 & mask:
|
||||
n += 1
|
||||
mask >>= 1
|
||||
if n < 1 or n > 6: # 10xxxxxx is not a valid leading byte
|
||||
return None
|
||||
val = b0 & (mask - 1)
|
||||
for i in range(1, n + 1):
|
||||
c = buf[pos + i]
|
||||
if c & 0xC0 != 0x80:
|
||||
return None
|
||||
val = (val << 6) | (c & 0x3F)
|
||||
return val, pos + 1 + n
|
||||
|
||||
|
||||
def _flac_frame_samples(buf: bytes, pos: int, fixed_bs: int):
|
||||
"""If a valid FLAC frame header starts at pos, return the stream sample
|
||||
count through the end of that frame, else None. Validity is confirmed by
|
||||
the header's CRC-8, so false sync matches in compressed data are rejected."""
|
||||
try:
|
||||
variable = buf[pos + 1] & 0x01
|
||||
bs_code = buf[pos + 2] >> 4
|
||||
sr_code = buf[pos + 2] & 0x0F
|
||||
if bs_code == 0 or sr_code == 15 or buf[pos + 3] & 0x01:
|
||||
return None
|
||||
if (buf[pos + 3] >> 4) > 10: # reserved channel assignment
|
||||
return None
|
||||
coded = _flac_coded_number(buf, pos + 4)
|
||||
if coded is None:
|
||||
return None
|
||||
val, p = coded
|
||||
bs = _FLAC_BLOCKSIZES.get(bs_code)
|
||||
if bs_code == 6:
|
||||
bs = buf[p] + 1
|
||||
p += 1
|
||||
elif bs_code == 7:
|
||||
bs = int.from_bytes(buf[p:p + 2], 'big') + 1
|
||||
p += 2
|
||||
if sr_code == 12:
|
||||
p += 1
|
||||
elif sr_code in (13, 14):
|
||||
p += 2
|
||||
if _crc8(buf[pos:p]) != buf[p]:
|
||||
return None
|
||||
if variable: # val is the frame's starting sample number
|
||||
return val + (bs or 0)
|
||||
return val * (fixed_bs or bs or 4096) + (bs or 0)
|
||||
except IndexError:
|
||||
return None
|
||||
|
||||
|
||||
def _live_flac_header(path: Path, size: int):
|
||||
"""Return the first 26 bytes of a FLAC file with STREAMINFO total_samples
|
||||
patched to the samples recorded so far, or None.
|
||||
|
||||
Like _live_wav_header, but FLAC duration cannot be derived from the byte
|
||||
count (variable compression). Instead the sample count is parsed out of
|
||||
the last frame header in the file tail — each FLAC frame carries its own
|
||||
frame/sample number.
|
||||
"""
|
||||
try:
|
||||
with open(path, 'rb') as fh:
|
||||
head = fh.read(42)
|
||||
if len(head) < 42 or head[:4] != b'fLaC':
|
||||
return None
|
||||
# STREAMINFO must be the first metadata block
|
||||
if head[4] & 0x7F != 0 or int.from_bytes(head[5:8], 'big') != 34:
|
||||
return None
|
||||
fixed_bs = int.from_bytes(head[8:10], 'big')
|
||||
|
||||
tail_len = min(size, 65536)
|
||||
fh.seek(size - tail_len)
|
||||
buf = fh.read(tail_len)
|
||||
|
||||
samples = None
|
||||
for i in range(len(buf) - 20, -1, -1):
|
||||
if buf[i] == 0xFF and (buf[i + 1] & 0xFC) == 0xF8:
|
||||
samples = _flac_frame_samples(buf, i, fixed_bs)
|
||||
if samples:
|
||||
break
|
||||
if not samples:
|
||||
return None
|
||||
|
||||
# Bytes 18-25 hold: sample rate (20 bits) | channels-1 (3) |
|
||||
# bps-1 (5) | total_samples (36). Replace only the low 36 bits.
|
||||
field = int.from_bytes(head[18:26], 'big')
|
||||
field = (field & ~((1 << 36) - 1)) | min(samples, (1 << 36) - 1)
|
||||
patched = bytearray(head[:26])
|
||||
patched[18:26] = field.to_bytes(8, 'big')
|
||||
return bytes(patched)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _wav_header(n_frames: int, channels: int, framerate: int, sampwidth: int) -> bytes:
|
||||
"""Standard 44-byte PCM WAV header for a clip of known length."""
|
||||
data_len = n_frames * channels * sampwidth
|
||||
byte_rate = framerate * channels * sampwidth
|
||||
return (b'RIFF' + (36 + data_len).to_bytes(4, 'little') + b'WAVE'
|
||||
+ b'fmt ' + (16).to_bytes(4, 'little')
|
||||
+ struct.pack('<HHIIHH', 1, channels, framerate, byte_rate,
|
||||
channels * sampwidth, sampwidth * 8)
|
||||
+ b'data' + data_len.to_bytes(4, 'little'))
|
||||
|
||||
|
||||
def _get_audio_duration(path: Path):
|
||||
"""Return duration in seconds for any supported audio file, or None."""
|
||||
ext = path.suffix.lower()
|
||||
if ext == '.wav':
|
||||
try:
|
||||
with wave.open(str(path), 'rb') as wf:
|
||||
return round(wf.getnframes() / wf.getframerate(), 2)
|
||||
except Exception:
|
||||
return None
|
||||
if SOUNDFILE_AVAILABLE and ext in ('.flac', '.ogg', '.opus'):
|
||||
try:
|
||||
with sf.SoundFile(path) as f:
|
||||
return round(len(f) / f.samplerate, 2)
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _compute_rms_windows_wav(wf, channels: int, sampwidth: int, framerate: int,
|
||||
window_samples: int):
|
||||
"""Yield rms_0_to_1 for every window in the open wave file."""
|
||||
while True:
|
||||
raw = wf.readframes(window_samples)
|
||||
if not raw:
|
||||
break
|
||||
n_samp = len(raw) // (sampwidth * channels)
|
||||
if n_samp == 0:
|
||||
break
|
||||
|
||||
if NUMPY_AVAILABLE:
|
||||
arr = np.frombuffer(raw[:n_samp * sampwidth * channels], dtype='<i2')
|
||||
if channels > 1:
|
||||
arr = arr.reshape(-1, channels).mean(axis=1)
|
||||
rms = float(np.sqrt(np.mean(arr.astype(np.float64) ** 2))) / 32768.0
|
||||
else:
|
||||
fmt = f'<{n_samp * channels}h'
|
||||
samples = struct.unpack(fmt, raw[:n_samp * sampwidth * channels])
|
||||
mono = samples[::channels] if channels > 1 else samples
|
||||
rms = math.sqrt(sum(s * s for s in mono) / len(mono)) / 32768.0
|
||||
|
||||
yield round(rms, 5)
|
||||
|
||||
|
||||
def _noise_floor_db(db_values: list, window_dur: float) -> list:
|
||||
"""Per-window background noise floor in dBFS.
|
||||
|
||||
The floor is the NOISE_PERCENTILE-th percentile of the windowed dB levels
|
||||
in each NOISE_BLOCK_SECONDS block, then min-smoothed over ±2 neighbouring
|
||||
blocks so an event spanning a whole block cannot raise its own floor.
|
||||
Tracks slow ambience changes (day/night, rain, traffic hum) so detection
|
||||
is relative to "how loud it normally is right now"."""
|
||||
n = len(db_values)
|
||||
block = max(1, int(round(NOISE_BLOCK_SECONDS / window_dur)))
|
||||
floors = []
|
||||
for i in range(0, n, block):
|
||||
chunk = sorted(db_values[i:i + block])
|
||||
floors.append(chunk[int(len(chunk) * NOISE_PERCENTILE / 100)])
|
||||
smoothed = [min(floors[max(0, b - 2):b + 3]) for b in range(len(floors))]
|
||||
return [smoothed[min(i // block, len(smoothed) - 1)] for i in range(n)]
|
||||
|
||||
|
||||
def _loud_sections(rms_values: list, window_dur: float, duration: float,
|
||||
margin_db: float, min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> list:
|
||||
"""Sections whose level rises at least margin_db above the local noise
|
||||
floor. Each section carries a 'score': its peak dB above the floor,
|
||||
capped by the sharpest rise observed within ONSET_SECONDS. Real events
|
||||
(voices, impacts, barks) have steep onsets, so their cap equals their
|
||||
peak; a swell that drifts up slower than the noise-floor blocks can track
|
||||
(wind, a distant approaching car) still flags but scores near zero, so
|
||||
score-ranked review (chips, U/I highlights) surfaces events first.
|
||||
|
||||
Sections shorter than min_duration (after min_gap merging) are discarded:
|
||||
without this, every isolated 100 ms window that pops above the floor — a
|
||||
click, a single raindrop — becomes its own section and a day can drown in
|
||||
thousands of sub-second clips."""
|
||||
db = [20 * math.log10(max(r, 1e-6)) for r in rms_values]
|
||||
floor = _noise_floor_db(db, window_dur)
|
||||
min_db = 20 * math.log10(MIN_RMS)
|
||||
onset_win = max(1, int(round(ONSET_SECONDS / window_dur)))
|
||||
|
||||
sections = []
|
||||
start_t = None
|
||||
last_loud_t = None
|
||||
peak = 0.0
|
||||
onset = 0.0
|
||||
|
||||
def _score():
|
||||
return round(max(0.0, min(peak, onset)), 1)
|
||||
|
||||
for i, d in enumerate(db):
|
||||
t = i * window_dur
|
||||
floor_eff = max(floor[i], min_db)
|
||||
if d >= floor_eff + margin_db:
|
||||
if start_t is None:
|
||||
start_t = t
|
||||
peak = 0.0
|
||||
onset = 0.0
|
||||
last_loud_t = t
|
||||
peak = max(peak, d - floor_eff)
|
||||
# Rise within the onset span; a section starting before the file
|
||||
# has history is measured against the floor instead (an event cut
|
||||
# off by a file split must not be punished as a swell).
|
||||
rise = d - db[i - onset_win] if i >= onset_win else d - floor_eff
|
||||
onset = max(onset, rise)
|
||||
else:
|
||||
if start_t is not None and (t - last_loud_t) > min_gap:
|
||||
end_t = last_loud_t + window_dur
|
||||
if end_t - start_t >= min_duration - 1e-9:
|
||||
sections.append({'start': round(start_t, 1),
|
||||
'end': round(end_t, 1),
|
||||
'score': _score()})
|
||||
start_t = None
|
||||
last_loud_t = None
|
||||
|
||||
# A section still open at EOF ends at its last loud window, exactly like
|
||||
# every section closed inside the loop — not at the file duration, which
|
||||
# used to stretch it by up to min_gap of trailing silence and put that
|
||||
# silence into the clip served for it.
|
||||
if start_t is not None and (last_loud_t + window_dur - start_t) >= min_duration - 1e-9:
|
||||
sections.append({'start': round(start_t, 1),
|
||||
'end': round(min(last_loud_t + window_dur, duration), 1),
|
||||
'score': _score()})
|
||||
|
||||
return sections
|
||||
|
||||
|
||||
def _package_result(rms_values: list, framerate: int, n_frames: int,
|
||||
window_samples: int, margin_db: float,
|
||||
min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> dict:
|
||||
window_dur = window_samples / framerate
|
||||
duration = n_frames / framerate
|
||||
|
||||
# Note: no RMS data is returned — the UI is screen-reader oriented and
|
||||
# draws no waveform, so sections + duration is all it needs. The window
|
||||
# length went the same way: it only ever existed to place RMS bars.
|
||||
return {
|
||||
'sections': _loud_sections(rms_values, window_dur, duration, margin_db, min_gap, min_duration),
|
||||
'duration': round(duration, 2),
|
||||
}
|
||||
|
||||
|
||||
def analyze_wav(path: Path, window_samples: int = WINDOW_SAMPLES,
|
||||
margin_db: float = MARGIN_DB,
|
||||
min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> dict:
|
||||
try:
|
||||
with wave.open(str(path), 'rb') as wf:
|
||||
channels = wf.getnchannels()
|
||||
sampwidth = wf.getsampwidth()
|
||||
framerate = wf.getframerate()
|
||||
n_frames = wf.getnframes()
|
||||
# _compute_rms_windows_wav decodes as int16 unconditionally, so an
|
||||
# 8- or 24-bit file would yield garbage RMS and therefore garbage
|
||||
# sections — silently. The recorder only writes 16-bit, but --dir
|
||||
# can point at any directory.
|
||||
if sampwidth != 2:
|
||||
return {'error': f'only 16-bit WAV files can be analysed '
|
||||
f'(this one is {sampwidth * 8}-bit)'}
|
||||
rms_values = list(_compute_rms_windows_wav(
|
||||
wf, channels, sampwidth, framerate, window_samples))
|
||||
except Exception as e:
|
||||
return {'error': str(e)}
|
||||
|
||||
return _package_result(rms_values, framerate, n_frames, window_samples, margin_db, min_gap, min_duration)
|
||||
|
||||
|
||||
def analyze_flac(path: Path, window_samples: int = WINDOW_SAMPLES,
|
||||
margin_db: float = MARGIN_DB,
|
||||
min_gap: float = MIN_GAP_SECONDS,
|
||||
min_duration: float = MIN_DURATION_SECONDS) -> dict:
|
||||
"""Analyse a FLAC file for loudness. Requires numpy and soundfile."""
|
||||
if not NUMPY_AVAILABLE or not SOUNDFILE_AVAILABLE:
|
||||
return {'error': 'FLAC analysis requires: pip install numpy soundfile'}
|
||||
|
||||
try:
|
||||
with sf.SoundFile(path) as f:
|
||||
framerate = f.samplerate
|
||||
channels = f.channels
|
||||
n_frames = len(f)
|
||||
|
||||
rms_values = []
|
||||
while True:
|
||||
frames = f.read(window_samples, dtype='float32', always_2d=True)
|
||||
if len(frames) == 0:
|
||||
break
|
||||
mono = frames.mean(axis=1) if channels > 1 else frames[:, 0]
|
||||
rms = float(np.sqrt(np.mean(mono.astype(np.float64) ** 2)))
|
||||
rms_values.append(round(rms, 5))
|
||||
except Exception as e:
|
||||
return {'error': str(e)}
|
||||
|
||||
return _package_result(rms_values, framerate, n_frames, window_samples, margin_db, min_gap, min_duration)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Analysis cache helpers
|
||||
|
||||
Reference in New Issue
Block a user