BirdNET-NG Architecture

A distributed bird sound identification system built on the BirdNET deep learning model.

Overview

BirdNET-NG decouples audio capture, processing, and visualization into independently deployable components that communicate over a message broker. This enables community-contributed satellites (microphone nodes), shared inference infrastructure, and multiple UI clients — all with multi-tenant isolation from day one.

graph TD
    subgraph Satellites
        S1["🎙 Satellite 1
(Pi + Mic)"]
        S2["🎙 Satellite 2
(Pi + Mic)"]
        PA["📱 Phone App
(Android)"]
    end

    S1 -->|WSS :443| T
    S2 -->|WSS :443| T
    PA -->|WSS :443| T

    T["🔀 Traefik
(reverse proxy)"]

    T -->|HTTPS| Web["🌐 Web
(nginx)"]
    T -->|WSS| Mosq["📡 Mosquitto
(dynsec)"]
    T -->|HTTPS| Docs["📖 Docs"]

    subgraph Hub Cluster
        Web --> Hub["⚡ Hub API
(Fastify)"]
        Mosq --> Hub
        Hub --> PG["🗄 PostgreSQL"]
        Hub --> Redis["⚙ Redis +
BullMQ"]
        Redis --> W1["🐍 Worker 1
(Python)"]
        Redis --> WN["🐍 Worker N
(Python)"]
    end

    T -->|HTTPS| MinIO["💾 MinIO
(S3 storage)"]
    Hub --> MinIO
    W1 --> MinIO
    WN --> MinIO

    style S1 fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style S2 fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style PA fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style T fill:#4c1d95,stroke:#8b5cf6,color:#e2e8f0
    style Web fill:#065f46,stroke:#22c55e,color:#e2e8f0
    style Hub fill:#065f46,stroke:#22c55e,color:#e2e8f0
    style Mosq fill:#7c2d12,stroke:#f97316,color:#e2e8f0
    style Docs fill:#065f46,stroke:#22c55e,color:#e2e8f0
    style PG fill:#1e3a5f,stroke:#3b82f6,color:#e2e8f0
    style Redis fill:#7c2d12,stroke:#f97316,color:#e2e8f0
    style W1 fill:#713f12,stroke:#eab308,color:#e2e8f0
    style WN fill:#713f12,stroke:#eab308,color:#e2e8f0
    style MinIO fill:#4c1d95,stroke:#8b5cf6,color:#e2e8f0

Components

Satellite (packages/satellite)

Lightweight Node.js agent running on a Raspberry Pi with an attached microphone.

Audio Pipeline:

• Captures 3-second WAV chunks at 48kHz mono (BirdNET's native window)
• On-device pre-filtering: adaptive noise floor + spectral peak SNR detection
• Filter settings pushed from hub tenant settings via MQTT config channel
• Local SQLite outbox queue (sql.js) — never blocks on network
• MQTT upload with acknowledgment and exponential backoff

Recording Scheduler:

• Sunrise/sunset calculation from GPS coordinates (NOAA algorithm)
• Profile-based scheduling: dawn_chorus, night_migration, low_power, continuous
• Profiles pushed from hub via MQTT, resolved locally against sun times
• Capture loop pauses outside recording windows

Heartbeat:

• Sends lightweight keepalive at configurable interval (default 30s, MQTT QoS 1)
• Reports state: recording, paused, scheduled_off, error
• Reports noise_floor_rms for adaptive filter calibration
• Reports satellite package version for version tracking
• Heartbeat interval configurable from hub tenant settings
• Decouples online status from audio chunk sending (satellite stays online even when all chunks are filtered)

Capture Modes: alsa (real hardware), simulate (sine wave), replay (pre-recorded WAV files)

Mobile App (packages/mobile)

Android phone satellite built with Capacitor.

• Native AudioRecord plugin (bypasses WebView audio limitations)
• GPS gate: requires location before recording starts (auto-acquire or manual entry)
• GPS auto-update toggle with configurable interval (30s–10min)
• Background mode for continuous recording when app is minimized
• Keep-screen-on toggle
• Settings redesigned as full page with clear GPS/manual location toggle
• Tap-to-record status bar (recording rectangle is the toggle)
• Verbose chunk logging toggle
• Default state: paused (not recording)
• All hub filter settings received and applied via MQTT
• Heartbeat and telemetry (battery, storage via Storage API)
• In-app log viewer for diagnostics

Hub API (packages/hub)

Central Fastify server coordinating the entire system. The hub can run in three modes via the HUB_MODE environment variable:

In full mode, everything runs in one process (the original behavior). In split mode, api and dispatcher run as separate containers. A Redis pub/sub event bridge carries events between them (e.g., detection alerts from the dispatcher are published to Redis and delivered to WebSocket clients by the API). The API process uses a lightweight MqttConfigPusher to push configuration to satellites without running the full MQTT ingester. The image worker writes stats to Redis and polls settings from the database, so it operates correctly in both modes.

Service Registry: All service instances (API, dispatcher, web, workers) register themselves in Redis under birdnet:registry:{service}:{instanceId} with a 30-second TTL. Each registration includes IP address, version, uptime, memory usage, and service-specific metadata (e.g., worker job stats). The System page in the Platform Admin section queries this registry via GET /api/system/status to display a live overview of all running instances, infrastructure health (PostgreSQL, Redis, MQTT, MinIO), and a data summary (detections, queue, images, tenants, users). Detail rows are expandable for per-instance information.

Core Services (API):

• JWT cookie + Bearer token authentication
• Role-based access control with tenant isolation
• Satellite credential provisioning via Mosquitto dynamic security (MqttAdmin)
• WebSocket server for real-time client notifications
• MqttConfigPusher: pushes config to satellites via MQTT (in api mode)

MQTT Connection Resilience:

Both MqttAdmin (credential provisioning) and MqttConfigPusher (satellite config push) maintain persistent MQTT connections with auto-reconnect. When running multiple API instances (--scale api=N), each instance maintains its own MQTT connection. If a connection is temporarily lost (e.g., Mosquitto restart, network blip), API requests that need MQTT will wait up to 10 seconds for reconnection rather than failing immediately. This makes scaled deployments reliable without requiring sticky sessions or a shared MQTT connection.

Background Workers (Dispatcher):

• MQTT Ingester: receives audio, telemetry, and heartbeats; stores audio in MinIO; enqueues inference jobs via BullMQ
• SatelliteMonitor: checks for offline/degraded satellites every 60s
• DetectionWatcher: rarity checks, first-of-season flagging, WebSocket events, webhook dispatch. When a detection is the first observation of a species in the current season for a tenant, it sets is_first_of_season on the detection and generates an alert.
• SpeciesImageWorker: downloads bird thumbnails from Wikipedia (max 2 concurrent), stores in MinIO, with Wikimedia rate limiting (500 req/hr unauthenticated, 10,000 req/hr with OAuth 2.0)
• WebhookDispatcher: delivers webhook payloads with HMAC signatures
• AuditService: comprehensive audit logging of all user, admin, and detection actions to audit_log table (real client IPs via X-Forwarded-For)

Session Grouping: Audio chunks are grouped into recording sessions based on a 5-minute gap boundary. If the time between two consecutive chunks from the same satellite exceeds 5 minutes, a new session begins. The Sessions API exposes these groupings for the Timeline page, including per-session chunk counts and detection summaries.

Inference Workers (packages/inference)

Python 3.11 processes running BirdNET TFLite.

• Pull jobs from BullMQ/Redis queue
• Download audio from MinIO, run inference via birdnetlib
• Geo-aware species filtering (GPS + date)
• Enhanced re-processing for uncertain detections (0.4–0.7 confidence):
  • Extended audio window (6s)
  • Frequency isolation (1–10kHz bandpass)
  • Cross-chunk correlation
• Scale with docker compose up --scale worker=N

Web UI (packages/web)

React 19 SPA served by nginx, responsive for mobile.

Pages:

• Dashboard: satellite fleet status, recent detections, active alerts
• Detections: 2-column flex card layout with SpectrogramPlayer (click-to-seek, drag-to-select region, loop playback, signal boost 1x–20x, speed 0.25x–2x, volume, download), vote buttons (Yes/No/Unsure) under confidence bar
• Satellites: sortable table layout (Name, Status, Profile, Version, Last seen) with search and status filter; satellite count (online/total) in header; row click opens a slide-in side panel with full details (General, Hardware, Schedule, Hub Configuration, Actions); version shown with amber warning when satellite version differs from hub version; tenant name shown in All Tenants view
• Analytics: daily trends, hourly activity, top species, biodiversity indices, weather correlation, migration patterns
• Compare: side-by-side location comparison with Jaccard similarity
• Alerts: custom alert rules (detection/absence/trend triggers) and 6 notification channels
• Workers: connected workers with IP-based dedup, queue stats (platform admin only)
• Members: tenant member management, invite links
• Timeline: chronological view of recording sessions with inline chunks and detections
• Species: card catalog of all detected species with stats, sorting, search
• Map: Leaflet.js interactive map with satellite markers and detection data
• Account: profile editing, password change, language preferences, self-delete
• Audit Log: paginated table of all admin and user actions (platform admin)
• Settings: tenant thresholds, watchlists; platform toggles, Wikimedia API, image cache

Features:

• Species thumbnails from Wikipedia (downloaded in background, served from MinIO)
• Multi-language bird names (38 languages from BirdNET label files)
• Species name display: primary bold, secondary in parentheses, Latin italic, clickable thumbnail with lightbox
• Real-time WebSocket notifications for rare species (translated names)
• Responsive layout with hamburger menu on mobile, scrollable sidebar (hidden scrollbar)
• Spectrogram visualization (Cooley-Tukey FFT, Viridis colormap, 0–12kHz)
• Full i18n support (English + French, ~500+ translation keys, locale-aware formatting)
• Language switcher in sidebar

MQTT Channels

birdnet/{tenant_id}/{satellite_id}/{channel}

Authentication

Two methods, resolved in order:

1. Bearer token — Authorization: Bearer <token> (JWT from mobile login, or API key SHA-256 hashed)
2. JWT cookie — session httpOnly cookie set by login endpoint

Roles (tenant-scoped): viewer < member < admin < owner Platform admin: cross-tenant access, read-only protections, designated via DB flag or PLATFORM_ADMIN_EMAILS env var

Multi-Tenancy

• Row-level isolation via tenant_id on all data tables
• Users are global; membership via tenant_members join table
• Each user can have different roles in different tenants
• MQTT ACLs enforce per-satellite topic scoping
• MinIO paths prefixed by tenant: {tenant_id}/{satellite_id}/{chunk_id}.wav
• Species images are global (not tenant-scoped)

Database

PostgreSQL 17 with 41 migrations (001–041):

Species Images

Bird thumbnails are managed by a background download queue:

1. When a species is first seen, it's queued in species_images table
2. SpeciesImageWorker processes the queue (max 2 concurrent, configurable delay)
3. Fetches original image URL from Wikipedia API, downloads 800px thumbnail
4. Stores in MinIO under species-images/{slug}.jpg
5. Proxy endpoint serves from MinIO with immutable cache headers
6. Frontend shows placeholder (pulsing dots) until download completes
7. Platform admin can clear cache, retry failed downloads, adjust download delay

Rate limiting:

• Sliding 1-hour request window with auto-throttle (slow down at 80%, pause at 95%)
• Respects Retry-After header on 429 responses
• Unauthenticated: 500 req/hr (with proper User-Agent)
• Authenticated (OAuth 2.0): 10,000 req/hr
• Wikimedia OAuth token and contact email configurable in Platform Settings

Technology Stack