BirdNET-NG Architecture

A distributed bird sound identification system built on the BirdNET deep learning model.

Overview

BirdNET-NG decouples audio capture, processing, and visualization into independently deployable components that communicate over a message broker. This enables community-contributed satellites (microphone nodes), shared inference infrastructure, and multiple UI clients — all with multi-tenant isolation from day one.

┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│ Satellite 1  │  │ Satellite 2  │  │ Phone App    │
│ (Pi + Mic)   │  │ (Pi + Mic)   │  │ (Android)    │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │ MQTTS (port 8883, TLS via Traefik) │
       │ WSS (WebSocket, for mobile)        │
       └─────────┬────────┘─────────────────┘
                 ▼
       ┌─── Traefik (reverse proxy) ─────────────────────┐
       │                                                   │
       │  HTTPS ──► Web (nginx)  (birdnet.x.net)          │
       │               └──► Hub API (backend network)     │
       │  MQTTS ──► Mosquitto    (mqtt.birdnet.x.net)     │
       │  HTTPS ──► MinIO GUI    (s3.birdnet.x.net)       │
       └────────────┬─────────────────────────────────────┘
                    │ backend network (internal)
       ┌────────────┴─────────────────────────────────────┐
       │            Hub Cluster                            │
       │                                                   │
       │  ┌──────────┐  ┌───────────┐  ┌──────────────┐   │
       │  │ Web      │  │ Hub API   │  │ Mosquitto    │   │
       │  │ (nginx)  │─►│ (Fastify) │◄─│ (dynsec)     │   │
       │  └──────────┘  └─────┬─────┘  └──────┬───────┘   │
       │                      │                │           │
       │                      ▼                ▼           │
       │  ┌───────────┐  ┌────────────────┐               │
       │  │ PostgreSQL │  │ Redis + BullMQ │               │
       │  └───────────┘  └───────┬────────┘               │
       │                         │                         │
       │                  ┌──────┴───────┐                 │
       │                  ▼              ▼                 │
       │            ┌──────────┐  ┌────────┐               │
       │            │ Worker 1 │  │Worker N│               │
       │            │ (Python) │  │(Python)│               │
       │            └──────────┘  └────────┘               │
       │                                                   │
       │  ┌───────────┐                                    │
       │  │ MinIO      │ (S3: audio + species images)      │
       │  └───────────┘                                    │
       └───────────────────────────────────────────────────┘

Components

Satellite (packages/satellite)

Lightweight Node.js agent running on a Raspberry Pi with an attached microphone.

Audio Pipeline:

• Captures 3-second WAV chunks at 48kHz mono (BirdNET's native window)
• On-device pre-filtering: RMS silence rejection + bird-band frequency check
• Local SQLite outbox queue (sql.js) — never blocks on network
• MQTT upload with acknowledgment and exponential backoff

Recording Scheduler:

• Sunrise/sunset calculation from GPS coordinates (NOAA algorithm)
• Profile-based scheduling: dawn_chorus, night_migration, low_power, continuous
• Profiles pushed from hub via MQTT, resolved locally against sun times
• Capture loop pauses outside recording windows

Heartbeat:

• Sends lightweight keepalive every 30 seconds (MQTT QoS 1)
• Reports state: recording, paused, scheduled_off, error
• Decouples online status from audio chunk sending (satellite stays online even when all chunks are filtered)

Capture Modes: alsa (real hardware), simulate (sine wave), replay (pre-recorded WAV files)

Mobile App (packages/mobile)

Android phone satellite built with Capacitor.

• Native AudioRecord plugin (bypasses WebView audio limitations)
• GPS gate: requires location before recording starts (auto-acquire or manual entry)
• GPS auto-update toggle with configurable interval (30s–10min)
• Background mode for continuous recording when app is minimized
• Keep-screen-on toggle
• Settings panel: rename satellite, manage GPS, unregister with confirmation
• Heartbeat and telemetry (battery, storage via Storage API)
• In-app log viewer for diagnostics

Hub API (packages/hub)

Central Fastify server coordinating the entire system.

Core Services:

• MQTT ingester: receives audio, telemetry, and heartbeats
• Audio storage in MinIO, job enqueueing via BullMQ
• JWT cookie + Bearer token authentication
• Role-based access control with tenant isolation
• Satellite credential provisioning via Mosquitto dynamic security

Background Workers (in-process):

• SatelliteMonitor: checks for offline/degraded satellites every 60s
• DetectionWatcher: rarity checks, WebSocket events, webhook dispatch
• SpeciesImageWorker: downloads bird thumbnails from Wikipedia one at a time, stores in MinIO
• WebhookDispatcher: delivers webhook payloads with HMAC signatures

Inference Workers (packages/inference)

Python 3.11 processes running BirdNET TFLite.

• Pull jobs from BullMQ/Redis queue
• Download audio from MinIO, run inference via birdnetlib
• Geo-aware species filtering (GPS + date)
• Enhanced re-processing for uncertain detections (0.4–0.7 confidence):
  • Extended audio window (6s)
  • Frequency isolation (1–10kHz bandpass)
  • Cross-chunk correlation
• Scale with docker compose up --scale worker=N

Web UI (packages/web)

React 19 SPA served by nginx, responsive for mobile.

Pages:

• Dashboard: satellite fleet status, recent detections, active alerts
• Detections: filterable table, audio playback, spectrogram, verification votes
• Satellites: fleet cards with telemetry, heartbeat state, recording profile push, schedule preview, inline rename
• Verification: review queue (card-based with keyboard shortcuts) + stats/leaderboard
• Analytics: daily trends, hourly activity, top species
• Workers: connected workers, queue stats
• Members: tenant member management, invite links
• Settings: tenant thresholds, watchlists; platform toggles, image cache management
• Preferences: per-user language settings (primary + 2 secondary)

Features:

• Species thumbnails from Wikipedia (downloaded in background, served from MinIO)
• Multi-language bird names (38 languages from BirdNET label files)
• Species name display: primary bold, secondary in parentheses, Latin italic, clickable thumbnail with lightbox
• Real-time WebSocket notifications for rare species (translated names)
• Responsive layout with hamburger menu on mobile
• Spectrogram visualization (Cooley-Tukey FFT, Viridis colormap, 0–12kHz)

MQTT Channels

birdnet/{tenant_id}/{satellite_id}/{channel}

Channel	Direction	QoS	Purpose
audio	Satellite → Hub	1	Audio chunk (base64 WAV)
telemetry	Satellite → Hub	1	Battery, storage, CPU, GPS
heartbeat	Satellite → Hub	1	Keepalive every 30s with state
config	Hub → Satellite	2	Recording profile push
ack	Hub → Satellite	1	Chunk acknowledgment

Authentication

Two methods, resolved in order:

1. Bearer token — Authorization: Bearer <token> (JWT from mobile login, or API key SHA-256 hashed)
2. JWT cookie — session httpOnly cookie set by login endpoint

Roles (tenant-scoped): viewer < member < admin < owner Platform admin: cross-tenant access, read-only protections, designated via DB flag or PLATFORM_ADMIN_EMAILS env var

Multi-Tenancy

• Row-level isolation via tenant_id on all data tables
• Users are global; membership via tenant_members join table
• Each user can have different roles in different tenants
• MQTT ACLs enforce per-satellite topic scoping
• MinIO paths prefixed by tenant: {tenant_id}/{satellite_id}/{chunk_id}.wav
• Species images are global (not tenant-scoped)

Database

PostgreSQL 17 with 12 migrations (001–012):

Migration	Purpose
001	Core tables: tenants, satellites, audio_chunks, detections, alerts, telemetry
002	User auth: global users, tenant_members, invites, platform/tenant settings
003	User management: blocked flag, updated_at
004	Webhooks table
005	Satellite device_id for dedup
006	Watchlist species in tenant settings
007	Language settings on tenant
008	Per-user language preferences
009	Species images cache table
010	Satellite heartbeat_state and uptime
011	Nullable telemetry fields (mobile compatibility)
012	Species image download queue columns

Species Images

Bird thumbnails are managed by a background download queue:

1. When a species is first seen, it's queued in species_images table
2. SpeciesImageWorker processes the queue one at a time (configurable delay, default 5s)
3. Fetches original image URL from Wikipedia API, downloads 800px thumbnail
4. Stores in MinIO under species-images/{slug}.jpg
5. Proxy endpoint serves from MinIO with immutable cache headers
6. Frontend shows placeholder (pulsing dots) until download completes
7. Platform admin can clear cache, retry failed downloads, adjust download delay

Technology Stack

Component	Technology
Satellite agent	TypeScript / Node.js
Local satellite DB	sql.js (SQLite)
Transport	MQTT (Mosquitto 2.x, dynamic security plugin)
Hub API	TypeScript / Fastify
Authentication	JWT cookies + Bearer API keys (bcrypt, SHA-256)
Job queue	BullMQ + Redis
Database	PostgreSQL 17 (12 migrations)
Object storage	MinIO (audio + species images)
Inference	Python 3.11 / TFLite / birdnetlib
Web UI	TypeScript / React 19 / Vite / nginx
Mobile app	Capacitor (Android) / native AudioRecord plugin
Species translations	38 languages from BirdNET label files
Documentation	VitePress
Reverse proxy	Traefik (HTTPS + MQTTS + WSS)
Containers	Docker Compose (7 containers)
Monorepo	pnpm workspaces