BirdNET-NG Architecture

A distributed bird sound identification system built on the BirdNET deep learning model.

Overview

BirdNET-NG decouples audio capture, processing, and visualization into independently deployable components that communicate over a message broker. This enables community-contributed satellites (microphone nodes), shared inference infrastructure, and multiple UI clients — all with multi-tenant isolation from day one.

graph TD
    subgraph Satellites
        S1["🎙 Satellite 1
(Pi + Mic)"]
        S2["🎙 Satellite 2
(Pi + Mic)"]
        PA["📱 Phone App
(Android)"]
    end

    S1 -->|MQTTS| T
    S2 -->|MQTTS| T
    PA -->|WSS| T

    T["🔀 Traefik
(reverse proxy)"]

    T -->|HTTPS| Web["🌐 Web
(nginx)"]
    T -->|MQTTS| Mosq["📡 Mosquitto
(dynsec)"]
    T -->|HTTPS| Docs["📖 Docs"]

    subgraph Hub Cluster
        Web --> Hub["⚡ Hub API
(Fastify)"]
        Mosq --> Hub
        Hub --> PG["🗄 PostgreSQL"]
        Hub --> Redis["⚙ Redis +
BullMQ"]
        Redis --> W1["🐍 Worker 1
(Python)"]
        Redis --> WN["🐍 Worker N
(Python)"]
    end

    T -->|HTTPS| MinIO["💾 MinIO
(S3 storage)"]
    Hub --> MinIO
    W1 --> MinIO
    WN --> MinIO

    style S1 fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style S2 fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style PA fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style T fill:#4c1d95,stroke:#8b5cf6,color:#e2e8f0
    style Web fill:#065f46,stroke:#22c55e,color:#e2e8f0
    style Hub fill:#065f46,stroke:#22c55e,color:#e2e8f0
    style Mosq fill:#7c2d12,stroke:#f97316,color:#e2e8f0
    style Docs fill:#065f46,stroke:#22c55e,color:#e2e8f0
    style PG fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style Redis fill:#7c2d12,stroke:#f97316,color:#e2e8f0
    style W1 fill:#713f12,stroke:#eab308,color:#e2e8f0
    style WN fill:#713f12,stroke:#eab308,color:#e2e8f0
    style MinIO fill:#4c1d95,stroke:#8b5cf6,color:#e2e8f0

Components

Satellite (packages/satellite)

Lightweight Node.js agent running on a Raspberry Pi with an attached microphone.

Audio Pipeline:

• Captures 3-second WAV chunks at 48kHz mono (BirdNET's native window)
• On-device pre-filtering: adaptive noise floor + spectral peak SNR detection
• Filter settings pushed from hub tenant settings via MQTT config channel
• Local SQLite outbox queue (sql.js) — never blocks on network
• MQTT upload with acknowledgment and exponential backoff

Recording Scheduler:

• Sunrise/sunset calculation from GPS coordinates (NOAA algorithm)
• Profile-based scheduling: dawn_chorus, night_migration, low_power, continuous
• Profiles pushed from hub via MQTT, resolved locally against sun times
• Capture loop pauses outside recording windows

Heartbeat:

• Sends lightweight keepalive at configurable interval (default 30s, MQTT QoS 1)
• Reports state: recording, paused, scheduled_off, error
• Reports noise_floor_rms for adaptive filter calibration
• Reports satellite package version for version tracking
• Heartbeat interval configurable from hub tenant settings
• Decouples online status from audio chunk sending (satellite stays online even when all chunks are filtered)

Capture Modes: alsa (real hardware), simulate (sine wave), replay (pre-recorded WAV files)

Mobile App (packages/mobile)

Android phone satellite built with Capacitor.

• Native AudioRecord plugin (bypasses WebView audio limitations)
• GPS gate: requires location before recording starts (auto-acquire or manual entry)
• GPS auto-update toggle with configurable interval (30s–10min)
• Background mode for continuous recording when app is minimized
• Keep-screen-on toggle
• Settings redesigned as full page with clear GPS/manual location toggle
• Tap-to-record status bar (recording rectangle is the toggle)
• Verbose chunk logging toggle
• Default state: paused (not recording)
• All hub filter settings received and applied via MQTT
• Heartbeat and telemetry (battery, storage via Storage API)
• In-app log viewer for diagnostics

Hub API (packages/hub)

Central Fastify server coordinating the entire system.

Core Services:

• MQTT ingester: receives audio, telemetry, and heartbeats
• Audio storage in MinIO, job enqueueing via BullMQ
• JWT cookie + Bearer token authentication
• Role-based access control with tenant isolation
• Satellite credential provisioning via Mosquitto dynamic security

Background Workers (in-process):

• SatelliteMonitor: checks for offline/degraded satellites every 60s
• DetectionWatcher: rarity checks, WebSocket events, webhook dispatch
• SpeciesImageWorker: downloads bird thumbnails from Wikipedia (max 2 concurrent), stores in MinIO, with Wikimedia rate limiting (500 req/hr unauthenticated, 10,000 req/hr with OAuth 2.0)
• WebhookDispatcher: delivers webhook payloads with HMAC signatures
• AuditService: records login, admin, and user actions to audit_log table

Inference Workers (packages/inference)

Python 3.11 processes running BirdNET TFLite.

• Pull jobs from BullMQ/Redis queue
• Download audio from MinIO, run inference via birdnetlib
• Geo-aware species filtering (GPS + date)
• Enhanced re-processing for uncertain detections (0.4–0.7 confidence):
  • Extended audio window (6s)
  • Frequency isolation (1–10kHz bandpass)
  • Cross-chunk correlation
• Scale with docker compose up --scale worker=N

Web UI (packages/web)

React 19 SPA served by nginx, responsive for mobile.

Pages:

• Dashboard: satellite fleet status, recent detections, active alerts
• Detections: 2-column flex card layout with SpectrogramPlayer (click-to-seek, drag-to-select region, loop playback, signal boost 1x–20x, speed 0.25x–2x, volume, download), vote buttons (Yes/No/Unsure) under confidence bar
• Satellites: sortable table layout (Name, Status, Profile, Version, Last seen) with search and status filter; satellite count (online/total) in header; row click opens a slide-in side panel with full details (General, Hardware, Schedule, Hub Configuration, Actions); version shown with amber warning when satellite version differs from hub version; tenant name shown in All Tenants view
• Verification: review queue (card-based with keyboard shortcuts) + stats/leaderboard
• Analytics: daily trends, hourly activity, top species
• Workers: connected workers with IP-based dedup, queue stats
• Members: tenant member management, invite links
• Species: card catalog of all detected species with stats, sorting, search
• Map: Leaflet.js interactive map with satellite markers and detection data
• Account: profile editing, password change, language preferences, self-delete
• Audit Log: paginated table of all admin and user actions (platform admin)
• Settings: tenant thresholds, watchlists; platform toggles, Wikimedia API, image cache

Features:

• Species thumbnails from Wikipedia (downloaded in background, served from MinIO)
• Multi-language bird names (38 languages from BirdNET label files)
• Species name display: primary bold, secondary in parentheses, Latin italic, clickable thumbnail with lightbox
• Real-time WebSocket notifications for rare species (translated names)
• Responsive layout with hamburger menu on mobile, scrollable sidebar (hidden scrollbar)
• Spectrogram visualization (Cooley-Tukey FFT, Viridis colormap, 0–12kHz)

MQTT Channels

birdnet/{tenant_id}/{satellite_id}/{channel}

Authentication

Two methods, resolved in order:

1. Bearer token — Authorization: Bearer <token> (JWT from mobile login, or API key SHA-256 hashed)
2. JWT cookie — session httpOnly cookie set by login endpoint

Roles (tenant-scoped): viewer < member < admin < owner Platform admin: cross-tenant access, read-only protections, designated via DB flag or PLATFORM_ADMIN_EMAILS env var

Multi-Tenancy

• Row-level isolation via tenant_id on all data tables
• Users are global; membership via tenant_members join table
• Each user can have different roles in different tenants
• MQTT ACLs enforce per-satellite topic scoping
• MinIO paths prefixed by tenant: {tenant_id}/{satellite_id}/{chunk_id}.wav
• Species images are global (not tenant-scoped)

Database

PostgreSQL 17 with 13 migrations (001–013):

Species Images

Bird thumbnails are managed by a background download queue:

1. When a species is first seen, it's queued in species_images table
2. SpeciesImageWorker processes the queue (max 2 concurrent, configurable delay)
3. Fetches original image URL from Wikipedia API, downloads 800px thumbnail
4. Stores in MinIO under species-images/{slug}.jpg
5. Proxy endpoint serves from MinIO with immutable cache headers
6. Frontend shows placeholder (pulsing dots) until download completes
7. Platform admin can clear cache, retry failed downloads, adjust download delay

Rate limiting:

• Sliding 1-hour request window with auto-throttle (slow down at 80%, pause at 95%)
• Respects Retry-After header on 429 responses
• Unauthenticated: 500 req/hr (with proper User-Agent)
• Authenticated (OAuth 2.0): 10,000 req/hr
• Wikimedia OAuth token and contact email configurable in Platform Settings

Technology Stack