music-assistant-server

1# AirPlay Provider
2
3## Overview
4
5The AirPlay provider enables Music Assistant to stream audio to AirPlay-enabled devices on your local network. It supports both **RAOP (AirPlay 1)** and **AirPlay 2** protocols, providing compatibility with a wide range of devices including Apple HomePods, Apple TVs, Macs, and third-party AirPlay-compatible speakers.
6
7### Key Features
8
9- **Dual Protocol Support**: Automatically selects between RAOP and AirPlay 2 based on device capabilities
10- **Native Pairing**: Supports pairing with Apple devices (Apple TV, HomePod, Mac) using HAP (HomeKit Accessory Protocol) or RAOP pairing
11- **Multi-Room Audio**: Synchronizes playback across multiple AirPlay devices with NTP timestamp precision
12- **DACP Remote Control**: Receives remote control commands (play/pause/volume/next/previous) from devices while streaming
13- **Late Join Support**: Allows adding players to an existing playback session without interrupting other players
14- **Flow Mode Streaming**: Provides gapless playback and crossfade support by streaming the queue as one continuous audio stream
15
16## Architecture
17
18### Component Overview
19
20```
21┌─────────────────────────────────────────────────────────────────┐
22│                      AirPlay Provider                           │
23│  ┌──────────────────────────────────────────────────────────┐  │
24│  │  MDNS Discovery (_airplay._tcp, _raop._tcp)              │  │
25│  └──────────────────────────────────────────────────────────┘  │
26│  ┌──────────────────────────────────────────────────────────┐  │
27│  │  DACP Server (_dacp._tcp) - Remote Control Callbacks     │  │
28│  └──────────────────────────────────────────────────────────┘  │
29└─────────────────────────────────────────────────────────────────┘
30                              │
31        ┌─────────────────────┼─────────────────────┐
32        │                     │                     │
33┌───────▼──────┐     ┌────────▼────────┐    ┌──────▼──────┐
34│ AirPlayPlayer│     │ AirPlayPlayer   │    │AirPlayPlayer│
35│   (Leader)   │     │  (Sync Child)   │    │(Sync Child) │
36└───────┬──────┘     └────────┬────────┘    └──────┬──────┘
37        │                     │                     │
38        └─────────────────────┼─────────────────────┘
39                              │
40                    ┌─────────▼──────────┐
41                    │ AirPlayStreamSession│
42                    │  (manages session)  │
43                    └─────────┬──────────┘
44                              │
45        ┌─────────────────────┼─────────────────────┐
46        │                     │                     │
47┌───────▼──────┐     ┌────────▼────────┐    ┌──────▼──────┐
48│  RaopStream  │     │ AirPlay2Stream  │    │ RaopStream  │
49│ ┌──────────┐ │     │ ┌────────────┐  │    │┌──────────┐ │
50│ │ cliraop  │ │     │ │  cliap2    │  │    ││ cliraop  │ │
51│ └────▲─────┘ │     │ └─────▲──────┘  │    │└────▲─────┘ │
52│      │       │     │       │         │    │     │       │
53│ ┌────┴─────┐ │     │ ┌─────┴──────┐  │    │┌────┴─────┐ │
54│ │  FFmpeg  │ │     │ │  FFmpeg    │  │    ││  FFmpeg  │ │
55│ └──────────┘ │     │ └────────────┘  │    │└──────────┘ │
56└──────────────┘     └─────────────────┘    └─────────────┘
57```
58
59### File Structure
60
61```
62airplay/
63├── provider.py           # Main provider class, MDNS discovery, DACP server
64├── player.py             # AirPlayPlayer implementation
65├── stream_session.py     # Manages streaming sessions for synchronized playback
66├── pairing.py           # HAP and RAOP pairing implementations
67├── helpers.py           # Utility functions (NTP conversion, model detection, etc.)
68├── constants.py         # Constants and enums
69├── protocols/
70│   ├── _protocol.py     # Base protocol class with shared logic
71│   ├── raop.py          # RAOP (AirPlay 1) streaming implementation
72│   └── airplay2.py      # AirPlay 2 streaming implementation
73└── bin/                 # Platform-specific CLI binaries
74    ├── cliraop-*        # RAOP streaming binaries
75    └── cliap2-*         # AirPlay 2 streaming binaries
76```
77
78## Protocol Selection: RAOP vs AirPlay 2
79
80### RAOP (AirPlay 1)
81
82- **Used for**: Older AirPlay devices, some third-party implementations
83- **Features**:
84  - Encryption support (can be disabled for problematic devices)
85  - ALAC compression option to save network bandwidth
86  - Password protection support
87  - Device-reported volume feedback via DACP
88- **Binary**: `cliraop` (based on [libraop](https://github.com/music-assistant/libraop))
89
90### AirPlay 2
91
92- **Used for**: Modern Apple devices, some third-party devices
93- **Features**:
94  - Better compatibility with newer devices
95  - More robust protocol
96  - Required for some devices that don't support RAOP
97- **Binary**: `cliap2` (based on [OwnTone](https://github.com/music-assistant/cliairplay))
98
99### Automatic Selection
100
101When protocol is set to "Automatically select" (default):
102- **Prefers AirPlay 2** for known models (e.g., Ubiquiti devices) that work better with it
103- **Falls back to RAOP** for all other devices
104- Users can manually override via player configuration if needed
105
106## Discovery and Player Setup
107
108### MDNS Service Discovery
109
110The provider discovers AirPlay devices via two MDNS service types:
111
1121. **`_airplay._tcp.local.`** - Primary AirPlay service (preferred)
113   - Contains detailed device information
114   - Announced by most modern devices
115
1162. **`_raop._tcp.local.`** - Legacy RAOP service
117   - Fallback for older devices
118   - If only RAOP service is found, provider attempts to query for AirPlay service
119
120### Player Setup Flow
121
1221. **MDNS service discovered** → `on_mdns_service_state_change()` in [provider.py](provider.py)
1232. **Extract device info** from MDNS properties:
124   - Device ID (from `deviceid` property or service name)
125   - Display name
126   - Manufacturer and model (via `get_model_info()` in [helpers.py](helpers.py))
1273. **Filter checks**:
128   - Skip if player is disabled in config
129   - Skip ShairportSync instances running on the same Music Assistant server (to avoid conflicts with AirPlay Receiver provider)
1304. **Create player** → `AirPlayPlayer` instance
1315. **Register with player controller** → `mass.players.register()`
132
133### Player ID Format
134
135Player IDs follow the format: `ap{mac_address}` (e.g., `ap1a2b3c4d5e6f`)
136
137## Pairing for Apple Devices
138
139Apple TV and Mac devices require pairing before they can be used for streaming.
140
141### Pairing Protocols
142
1431. **HAP (HomeKit Accessory Protocol)** - For AirPlay 2
144   - 6-step SRP authentication with TLV encoding
145   - Ed25519 key exchange
146   - ChaCha20-Poly1305 encryption
147   - Produces 192-character hex credentials
148
1492. **RAOP Pairing** - For AirPlay 1
150   - 3-step SRP authentication with plist encoding
151   - Ed25519 key derivation from auth secret
152   - AES-GCM encryption
153   - Produces `client_id:auth_secret` format credentials
154
155### Pairing Flow
156
1571. **Start pairing** → POST to `/pair-pin-start` (or protocol-specific endpoint)
1582. **Device displays 4-digit PIN** on screen
1593. **User enters PIN** in Music Assistant configuration
1604. **Complete pairing** → SRP authentication and key exchange
1615. **Store credentials** in player config (protocol-specific key: `raop_credentials` or `airplay_credentials`)
162
163**Important**: The DACP ID used during pairing must match the ID used during streaming. The provider uses the first 16 hex characters of `server_id` as a persistent DACP ID to ensure compatibility across restarts.
164
165## Streaming Architecture
166
167### Audio Pipeline
168
169```
170┌─────────────────────────────────────────────────────────────────┐
171│                    Music Assistant Core                          │
172│  ┌──────────────────────────────────────────────────────────┐   │
173│  │  Queue Manager (assembles tracks into continuous stream) │   │
174│  └─────────────────────────┬────────────────────────────────┘   │
175└────────────────────────────┼─────────────────────────────────────┘
176                             │ PCM Audio (44.1kHz, 32-bit float)
177                    ┌────────▼─────────┐
178                    │ StreamSession    │
179                    │ _audio_streamer()│
180                    └────────┬─────────┘
181                             │ Chunks of PCM audio
182        ┌────────────────────┼────────────────────┐
183        │                    │                    │
184┌───────▼──────┐    ┌────────▼────────┐   ┌──────▼──────┐
185│   FFmpeg     │    │    FFmpeg       │   │   FFmpeg    │
186│ (resample,   │    │  (resample,     │   │ (resample,  │
187│  filter,     │    │   filter,       │   │  filter,    │
188│  convert)    │    │   convert)      │   │  convert)   │
189└───────┬──────┘    └────────┬────────┘   └──────┬──────┘
190        │ PCM 44.1kHz 16-bit │                    │
191┌───────▼──────┐    ┌────────▼────────┐   ┌──────▼──────┐
192│  cliraop     │    │    cliap2       │   │  cliraop    │
193│  (RAOP       │    │  (AirPlay 2     │   │  (RAOP      │
194│   protocol)  │    │   protocol)     │   │   protocol) │
195└───────┬──────┘    └────────┬────────┘   └──────┬──────┘
196        │                    │                    │
197        │ Network (RTP)      │ Network (RTP)      │ Network (RTP)
198        │                    │                    │
199┌───────▼──────┐    ┌────────▼────────┐   ┌──────▼──────┐
200│ AirPlay      │    │  AirPlay        │   │  AirPlay    │
201│ Device 1     │    │  Device 2       │   │  Device 3   │
202└──────────────┘    └─────────────────┘   └─────────────┘
203```
204
205### Stream Session Management
206
207The `AirPlayStreamSession` class in [stream_session.py](stream_session.py) manages streaming to one or more synchronized players:
208
2091. **Initialization** (`start()` method)
210   - Calculates start time with connection delay buffer
211   - Converts start time to NTP timestamp for precise synchronization
212
2132. **Client Setup** (per player, `_start_client()` method)
214   - Creates protocol instance (`RaopStream` or `AirPlay2Stream`)
215   - Starts CLI process with NTP start timestamp
216   - Configures FFmpeg for audio format conversion and optional DSP filters
217   - Pipes FFmpeg output to CLI process stdin
218
2193. **Audio Streaming** (`_audio_streamer()` method)
220   - Receives PCM audio chunks from Music Assistant core
221   - Distributes chunks to all players via FFmpeg
222   - Tracks elapsed time based on bytes sent
223   - Handles silence padding if audio source is slow (watchdog mechanism)
224
2254. **Connection Monitoring**
226   - Waits for all devices to connect before starting playback
227   - Monitors CLI stderr for connection status and errors
228   - Removes players that fail to keep up (write timeouts)
229
230### Flow Mode Streaming
231
232AirPlay uses **flow mode** streaming, which means:
233- The entire queue is streamed as one continuous audio stream
234- Enables true gapless playback between tracks
235- Supports crossfade between tracks
236- Once started, the stream continues until explicitly stopped
237
238
239## Multi-Room Synchronization
240
241### Synchronized Playback
242
243The provider supports synchronized multi-room audio by:
244
2451. **Using a single `AirPlayStreamSession`** for the group leader and all sync children
2462. **Coordinating start times** via NTP timestamps
2473. **Distributing identical audio** to all players simultaneously
2484. **Per-player sync adjustment** via `sync_adjust` config option (in milliseconds)
249
250### Group Management
251
252- **Leader**: The primary player that manages the stream session
253- **Members**: Child players synchronized to the leader
254- **Adding members**: Use `set_members()` method in [player.py](player.py)
255- **Removing members**: Stream continues for remaining players
256
257### Late Join Support
258
259When adding a player to an already-playing session (`add_client()` in [stream_session.py](stream_session.py)):
260
2611. **Ring buffer**: Session maintains ~8 seconds of recent audio chunks in memory
2622. **Immediate buffered feed**: Late joiner receives buffered chunks immediately to prime the ffmpeg/CLI pipeline
2633. **Compensated start time**: NTP timestamp accounts for buffer duration: `start_time + (seconds_streamed - buffer_duration)`
2644. **Fast catch-up**: Device processes buffered audio and catches up to real-time position
2655. **Seamless sync**: Joins live stream perfectly synchronized with other players
266
267This approach significantly reduces the delay when adding players to an active session, as the late joiner receives audio data immediately instead of waiting for new chunks.
268
269**Config option**: `enable_late_join` (default: `True`)
270- If disabled: Session restarts with all players when members change
271- If enabled: New players join seamlessly without interrupting others
272
273## DACP (Digital Audio Control Protocol)
274
275### Purpose
276
277DACP allows AirPlay devices to send remote control commands back to Music Assistant while streaming is active. This enables:
278- Using physical buttons on devices (e.g., Apple TV remote)
279- Volume control from the device
280- Play/pause/next/previous commands
281- Shuffle toggle
282- Source switching detection
283
284### DACP Server
285
286The provider registers a MDNS service `_dacp._tcp.local.` (in `handle_async_init()` method in [provider.py](provider.py)) and runs a TCP server to receive HTTP requests from devices.
287
288### Active-Remote ID
289
290Each streaming session generates an `active_remote_id` (via `generate_active_remote_id()` in [helpers.py](helpers.py)) from the player's MAC address. This ID is:
291- Passed to the CLI binary
292- Sent to the device during streaming
293- Used to match incoming DACP requests to the correct player
294
295### Supported DACP Commands
296
297Handled in `_handle_dacp_request()` in [provider.py](provider.py):
298
299| DACP Path | Action |
300|-----------|--------|
301| `/ctrl-int/1/nextitem` | Skip to next track |
302| `/ctrl-int/1/previtem` | Go to previous track |
303| `/ctrl-int/1/play` | Resume playback |
304| `/ctrl-int/1/pause` | Pause playback |
305| `/ctrl-int/1/playpause` | Toggle play/pause |
306| `/ctrl-int/1/stop` | Stop playback |
307| `/ctrl-int/1/volumeup` | Increase volume |
308| `/ctrl-int/1/volumedown` | Decrease volume |
309| `/ctrl-int/1/shuffle_songs` | Toggle shuffle |
310| `dmcp.device-volume=X` | Volume changed by device (RAOP only) |
311| `device-prevent-playback=1` | Device switched to another source or powered off |
312| `device-prevent-playback=0` | Device ready for playback again |
313
314### Volume Feedback
315
316Both **RAOP** and **AirPlay 2** protocols support devices reporting their volume level via DACP.
317
318**Config option**: `ignore_volume` (default: `False`, auto-enabled for Apple devices)
319- Useful when device volume reports are unreliable
320- Apple devices always ignore volume feedback (handled internally)
321
322### Device Source Switching
323
324When `device-prevent-playback=1` is received:
325- User switched the device to another input source
326- Device is powered off
327- Streaming session removes the player from the active session
328
329## External CLI Binaries
330
331### Why External Binaries?
332
333Python is not suitable for real-time audio streaming with precise timing requirements. The AirPlay protocols (especially AirPlay 2) require:
334- Accurate NTP timestamp handling
335- Real-time RTP packet transmission
336- Low-latency audio buffering
337- Precise synchronization across multiple devices
338
339Therefore, the provider uses C-based CLI binaries for the actual streaming.
340
341### Binary Selection
342
343The provider automatically selects the correct binary based on:
344- **Platform**: Linux, macOS
345- **Architecture**: x86_64, arm64, aarch64
346- **Protocol**: RAOP (`cliraop-*`) or AirPlay 2 (`cliap2-*`)
347
348Binaries are located in [bin/](bin/) directory and validated on first use.
349
350### Binary Communication
351
352**Input** (stdin):
353- PCM audio data piped from FFmpeg
354
355**Commands** (named pipe):
356- Interactive commands sent via `AsyncNamedPipeWriter`
357- Examples: `ACTION=PLAY`, `ACTION=PAUSE`, `VOLUME=50`, `TITLE=Song Name`
358
359**Output** (stderr):
360- Status messages and logs
361- Connection state
362- Playback state changes
363- Elapsed time updates
364- Error messages
365
366The provider monitors stderr in a separate task (`_stderr_reader()` in [raop.py](protocols/raop.py) and [airplay2.py](protocols/airplay2.py)) to:
367- Update player state
368- Detect connection completion
369- Handle errors and packet loss
370- Track elapsed time
371
372## NTP Timestamp Synchronization
373
374AirPlay uses **NTP (Network Time Protocol)** timestamps for synchronized playback.
375
376### NTP Format
377
378- **64-bit integer**: Upper 32 bits = seconds, lower 32 bits = fractional seconds
379- **NTP epoch**: January 1, 1900 (not Unix epoch 1970)
380- **Precision**: Nanosecond-level timing
381
382### Key Functions
383
384Available in [helpers.py](helpers.py):
385- `get_ntp_timestamp()`: Get current NTP time
386- `ntp_to_unix_time()`: Convert NTP to Unix timestamp
387- `unix_time_to_ntp()`: Convert Unix to NTP timestamp
388- `add_seconds_to_ntp()`: Add offset to NTP timestamp
389
390### Usage in Streaming
391
3921. Calculate desired start time: `current_time + connection_buffer`
3932. Convert to NTP timestamp
3943. Pass to CLI binary via `-ntpstart` argument
3954. All players start at the exact same NTP time
3965. Per-player `sync_adjust` config allows fine-tuning (+/- milliseconds)
397
398## Player Types
399
400The provider creates players with different types based on whether the device is a native Apple player or a third-party AirPlay receiver.
401
402### PlayerType.PLAYER
403- **Devices**: Apple HomePod, Apple TV, Mac
404- **Reason**: These are standalone music players with native AirPlay support
405- **Behavior**: Exposed as top-level players in Music Assistant UI
406- **Not merged**: These players are NOT combined with other protocols
407
408### PlayerType.PROTOCOL
409- **Devices**: Third-party AirPlay receivers (Sonos, receivers, smart speakers, soundbars)
410- **Reason**: AirPlay is just one output protocol among many for these devices (often supporting Chromecast, DLNA, etc.)
411- **Behavior**: Automatically merged into a **Universal Player** if other protocols are detected for the same device
412- **Example**: A Sonos speaker supporting both AirPlay and Chromecast will appear as a single "Sonos" player with selectable output protocols
413
414**Detection**: Player type is determined in [player.py](player.py) `__init__()` method based on `manufacturer == "Apple"`
415
416**For more details on output protocols and protocol linking**, see the [Player Controller README](../../controllers/players/README.md), which explains:
417- How multiple protocol players for the same physical device are automatically linked
418- The Universal Player concept for devices without native vendor support
419- Protocol selection and device identifier matching
420- Native player linking vs. Universal Player creation
421
422## Configuration Options
423
424### Protocol Selection
425- **`airplay_protocol`**: Choose RAOP, AirPlay 2, or automatic (default: automatic)
426
427### RAOP-Specific
428- **`encryption`**: Enable/disable encryption (default: enabled)
429- **`alac_encode`**: Enable ALAC compression to save bandwidth (default: enabled)
430- **`ignore_volume`**: Ignore device volume reports (default: false)
431
432### General
433- **`password`**: Device password if required
434- **`sync_adjust`**: Per-player timing adjustment in milliseconds (default: 0)
435
436### Pairing (Apple devices only)
437- **`raop_credentials`**: Stored RAOP pairing credentials (hidden)
438- **`airplay_credentials`**: Stored AirPlay 2 pairing credentials (hidden)
439
440## Known Issues
441
442### Broken AirPlay Models
443
444Some devices have known broken AirPlay implementations (see `BROKEN_AIRPLAY_MODELS` in [constants.py](constants.py)):
445- **Samsung devices**: Known issues with both RAOP and AirPlay 2
446- These players are disabled by default
447
448### Limitations
449
4501. **DACP remote control**: Only active while streaming (not when idle)
4512. **Pause while synced**: Not supported; uses stop instead
4523. **Companion protocol**: Not yet implemented for idle state monitoring
453
454## Development Notes
455
456### Testing CLI Binaries
457
458Each binary can be validated with a test command:
459- **cliraop**: `cliraop -check` (should output "cliraop check")
460- **cliap2**: `cliap2 --testrun` (should output "cliap2 check")
461
462### Adding New CLI Commands
463
464To add a new command to the CLI binaries:
4651. Update the CLI binary source code (external repositories)
4662. Update `send_cli_command()` method in [_protocol.py](protocols/_protocol.py)
4673. Send command via named pipe: `await stream.send_cli_command("YOUR_COMMAND=value")`
468
469### Debugging Streaming Issues
470
471Enable verbose logging in Music Assistant to see:
472- CLI binary arguments
473- stderr output from binaries
474- DACP requests
475- Connection state changes
476- Packet loss warnings
477
478## Credits
479
480- **libraop**: RAOP streaming implementation - https://github.com/music-assistant/libraop
481- **OwnTone**: AirPlay 2 implementation - https://github.com/OwnTone
482- **pyatv**: Reference for HAP pairing protocol - https://github.com/postlund/pyatv
483
484## Future Enhancements
485
486- **Companion protocol**: Implement idle state monitoring for Apple devices
487- **AirPlay 2 volume feedback**: Add DACP volume support for AirPlay 2
488