BirdNET-NG Architecture

A distributed bird sound identification system built on the BirdNET deep learning model.

Overview

BirdNET-NG decouples audio capture, processing, and visualization into independently deployable components that communicate over a message broker. This enables community-contributed satellites (microphone nodes), shared inference infrastructure, and multiple UI clients — all with multi-tenant isolation from day one.

┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│ Satellite 1  │  │ Satellite 2  │  │ Phone App    │
│ (Pi + Mic)   │  │ (Pi + Mic)   │  │ (Android)    │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │ MQTTS (port 8883, TLS via Traefik) │
       │ WSS (WebSocket, for mobile)        │
       └─────────┬────────┘─────────────────┘
                 ▼
       ┌─── Traefik (reverse proxy) ─────────────────────┐
       │                                                   │
       │  HTTPS ──► Web (nginx)  (birdnet.x.net)          │
       │               └──► Hub API (backend network)     │
       │  MQTTS ──► Mosquitto    (mqtt.birdnet.x.net)     │
       │  HTTPS ──► MinIO GUI    (s3.birdnet.x.net)       │
       └────────────┬─────────────────────────────────────┘
                    │ backend network (internal)
       ┌────────────┴─────────────────────────────────────┐
       │            Hub Cluster                            │
       │                                                   │
       │  ┌──────────┐  ┌───────────┐  ┌──────────────┐   │
       │  │ Web      │  │ Hub API   │  │ Mosquitto    │   │
       │  │ (nginx)  │─►│ (Fastify) │◄─│ (dynsec)     │   │
       │  └──────────┘  └─────┬─────┘  └──────┬───────┘   │
       │                      │                │           │
       │                      ▼                ▼           │
       │  ┌───────────┐  ┌────────────────┐               │
       │  │ PostgreSQL │  │ Redis + BullMQ │               │
       │  └───────────┘  └───────┬────────┘               │
       │                         │                         │
       │                  ┌──────┴───────┐                 │
       │                  ▼              ▼                 │
       │            ┌──────────┐  ┌────────┐               │
       │            │ Worker 1 │  │Worker N│               │
       │            │ (Python) │  │(Python)│               │
       │            └──────────┘  └────────┘               │
       │                                                   │
       │  ┌───────────┐                                    │
       │  │ MinIO      │ (S3: audio + species images)      │
       │  └───────────┘                                    │
       └───────────────────────────────────────────────────┘

Components

Satellite (packages/satellite)

Lightweight Node.js agent running on a Raspberry Pi with an attached microphone.

Audio Pipeline:

• Captures 3-second WAV chunks at 48kHz mono (BirdNET's native window)
• On-device pre-filtering: RMS silence rejection + bird-band frequency check
• Local SQLite outbox queue (sql.js) — never blocks on network
• MQTT upload with acknowledgment and exponential backoff

Recording Scheduler:

• Sunrise/sunset calculation from GPS coordinates (NOAA algorithm)
• Profile-based scheduling: dawn_chorus, night_migration, low_power, continuous
• Profiles pushed from hub via MQTT, resolved locally against sun times
• Capture loop pauses outside recording windows

Heartbeat:

• Sends lightweight keepalive every 30 seconds (MQTT QoS 1)
• Reports state: recording, paused, scheduled_off, error
• Decouples online status from audio chunk sending (satellite stays online even when all chunks are filtered)

Capture Modes: alsa (real hardware), simulate (sine wave), replay (pre-recorded WAV files)

Mobile App (packages/mobile)

Android phone satellite built with Capacitor.

• Native AudioRecord plugin (bypasses WebView audio limitations)
• GPS gate: requires location before recording starts (auto-acquire or manual entry)
• GPS auto-update toggle with configurable interval (30s–10min)
• Background mode for continuous recording when app is minimized
• Keep-screen-on toggle
• Settings panel: rename satellite, manage GPS, unregister with confirmation
• Heartbeat and telemetry (battery, storage via Storage API)
• In-app log viewer for diagnostics

Hub API (packages/hub)

Central Fastify server coordinating the entire system.

Core Services:

• MQTT ingester: receives audio, telemetry, and heartbeats
• Audio storage in MinIO, job enqueueing via BullMQ
• JWT cookie + Bearer token authentication
• Role-based access control with tenant isolation
• Satellite credential provisioning via Mosquitto dynamic security

Background Workers (in-process):

• SatelliteMonitor: checks for offline/degraded satellites every 60s
• DetectionWatcher: rarity checks, WebSocket events, webhook dispatch
• SpeciesImageWorker: downloads bird thumbnails from Wikipedia one at a time, stores in MinIO
• WebhookDispatcher: delivers webhook payloads with HMAC signatures

Inference Workers (packages/inference)

Python 3.11 processes running BirdNET TFLite.

• Pull jobs from BullMQ/Redis queue
• Download audio from MinIO, run inference via birdnetlib
• Geo-aware species filtering (GPS + date)
• Enhanced re-processing for uncertain detections (0.4–0.7 confidence):
  • Extended audio window (6s)
  • Frequency isolation (1–10kHz bandpass)
  • Cross-chunk correlation
• Scale with docker compose up --scale worker=N

Web UI (packages/web)

React 19 SPA served by nginx, responsive for mobile.

Pages:

• Dashboard: satellite fleet status, recent detections, active alerts
• Detections: filterable table, audio playback, spectrogram, verification votes
• Satellites: fleet cards with telemetry, heartbeat state, recording profile push, schedule preview, inline rename
• Verification: review queue (card-based with keyboard shortcuts) + stats/leaderboard
• Analytics: daily trends, hourly activity, top species
• Workers: connected workers, queue stats
• Members: tenant member management, invite links
• Settings: tenant thresholds, watchlists; platform toggles, image cache management
• Preferences: per-user language settings (primary + 2 secondary)

Features:

• Species thumbnails from Wikipedia (downloaded in background, served from MinIO)
• Multi-language bird names (38 languages from BirdNET label files)
• Species name display: primary bold, secondary in parentheses, Latin italic, clickable thumbnail with lightbox
• Real-time WebSocket notifications for rare species (translated names)
• Responsive layout with hamburger menu on mobile
• Spectrogram visualization (Cooley-Tukey FFT, Viridis colormap, 0–12kHz)

MQTT Channels

birdnet/{tenant_id}/{satellite_id}/{channel}

Authentication

Two methods, resolved in order:

1. Bearer token — Authorization: Bearer <token> (JWT from mobile login, or API key SHA-256 hashed)
2. JWT cookie — session httpOnly cookie set by login endpoint

Roles (tenant-scoped): viewer < member < admin < owner Platform admin: cross-tenant access, read-only protections, designated via DB flag or PLATFORM_ADMIN_EMAILS env var

Multi-Tenancy

• Row-level isolation via tenant_id on all data tables
• Users are global; membership via tenant_members join table
• Each user can have different roles in different tenants
• MQTT ACLs enforce per-satellite topic scoping
• MinIO paths prefixed by tenant: {tenant_id}/{satellite_id}/{chunk_id}.wav
• Species images are global (not tenant-scoped)

Database

PostgreSQL 17 with 12 migrations (001–012):

Species Images

Bird thumbnails are managed by a background download queue:

1. When a species is first seen, it's queued in species_images table
2. SpeciesImageWorker processes the queue one at a time (configurable delay, default 5s)
3. Fetches original image URL from Wikipedia API, downloads 800px thumbnail
4. Stores in MinIO under species-images/{slug}.jpg
5. Proxy endpoint serves from MinIO with immutable cache headers
6. Frontend shows placeholder (pulsing dots) until download completes
7. Platform admin can clear cache, retry failed downloads, adjust download delay

Technology Stack