storage

Ansible role that provisions my storage server.

7.2 KB•MD

README.md

7.2 KB • 263 lines • markdown

1# Storage Services Role
2
3Comprehensive media automation and backup services role for homelab environments.
4
5## Overview
6
7This role deploys a complete media automation and backup stack including:
8
9- **Jellyfin**: Media server for streaming movies, TV shows, music
10- **Arr Stack**: Automated media acquisition (Sonarr, Radarr, Prowlarr, Readarr)
11- **Gluetun VPN**: Secure VPN container for Arr stack traffic
12- **Calibre Stack**: Ebook management (Calibre + Calibre-Web)
13- **Restic Server**: Self-hosted backup solution
14
15## Services Included
16
17### Media Server
18- **Jellyfin** (Port 8096): Media streaming server with transcoding support
19
20### Media Automation (Arr Stack)
21- **Sonarr** (Port 8989): TV show automation
22- **Radarr** (Port 7878): Movie automation  
23- **Prowlarr** (Port 9696): Indexer management
24- **Readarr** (Port 8787): Ebook/audiobook automation
25- **Gluetun**: VPN container for secure traffic routing
26
27### Ebook Management
28- **Calibre Server** (Port 8083): Full-featured ebook management
29- **Calibre-Web** (Port 8084): Web interface for library access
30
31### Backup Solution
32- **Restic Server** (Port 8000): REST-based backup repository server
33
34## Requirements
35
36- Docker and Docker Compose installed
37- Sufficient storage space mounted at `/mnt/rstorage` (default)
38- Network access for media downloads (if using Arr stack)
39- VPN credentials if using Gluetun for secure downloads
40
41## Usage
42
43### Basic Deployment
44
45```yaml
46# In your host variables
47storage_enabled: true
48jellyfin_enabled: true
49arr_stack_enabled: true
50calibre_enabled: true
51restic_backup_server_enabled: true
52```
53
54### Deploy to Storage Servers
55
56```bash
57# Deploy all storage services
58ansible-playbook -i inventory/hosts site.yml --limit storage_servers --tags "core,storage"
59
60# Deploy only specific services
61ansible-playbook -i inventory/hosts site.yml --limit storage_servers --tags "jellyfin"
62ansible-playbook -i inventory/hosts site.yml --limit storage_servers --tags "arr-stack"
63```
64
65## Configuration
66
67### Required Vault Variables
68
69For secure operation, configure these encrypted variables:
70
71```yaml
72# VPN Configuration (for Gluetun)
73vpn_service_provider: "nordvpn"  # or "surfshark", "expressvpn", etc.
74wireguard_private_key: "your_wireguard_private_key"
75wireguard_addresses: "10.x.x.x/32"
76
77# Alternative OpenVPN credentials
78openvpn_user: "your_vpn_username"
79openvpn_password: "your_vpn_password"
80
81# Backup Authentication
82restic_backup_username: "backup_user"
83restic_backup_password: "secure_backup_password"
84
85# Optional: Service admin credentials
86jellyfin_admin_username: "admin"
87jellyfin_admin_password: "secure_password"
88calibre_web_admin_username: "admin"
89calibre_web_admin_password: "secure_password"
90```
91
92### Storage Configuration
93
94```yaml
95# Storage paths (customize as needed)
96storage_base_path: "/mnt/rstorage"
97storage_docker_dir: "/docker"
98
99# Media directories will be created automatically:
100# - /mnt/rstorage/media/movies
101# - /mnt/rstorage/media/tv  
102# - /mnt/rstorage/media/books
103# - /mnt/rstorage/downloads
104# - /mnt/rstorage/backups
105```
106
107### Service Customization
108
109```yaml
110# Port configuration
111jellyfin_host_port: 8096
112sonarr_host_port: 8989
113radarr_host_port: 7878
114prowlarr_host_port: 9696
115readarr_host_port: 8787
116calibre_web_host_port: 8084
117restic_backup_host_port: 8000
118
119# Resource limits
120jellyfin_memory_limit: "4g"
121jellyfin_cpu_limit: "4"
122arr_memory_limit: "1g"
123arr_cpu_limit: "1"
124
125# VPN settings
126arr_vpn_enabled: true
127vpn_server_countries: "Netherlands,Switzerland"
128```
129
130## Access URLs
131
132After deployment, services are accessible at:
133
134- Jellyfin: `http://your-server:8096`
135- Sonarr: `http://your-server:8989`
136- Radarr: `http://your-server:7878`  
137- Prowlarr: `http://your-server:9696`
138- Readarr: `http://your-server:8787`
139- Calibre-Web: `http://your-server:8084`
140- Restic Server: `http://your-server:8000`
141
142## Post-Deployment Setup
143
144### 1. Arr Stack Configuration
145
1461. **Prowlarr First**: Configure indexers and connect to other Arr apps
1472. **Quality Profiles**: Set up quality preferences in each Arr app
1483. **Root Folders**: Add media directories (already mounted)
1494. **Download Client**: Configure your preferred download client
150
151### 2. Jellyfin Setup
152
1531. Complete the initial setup wizard
1542. Add media libraries pointing to `/media/movies`, `/media/tv`, etc.
1553. Configure transcoding settings based on your hardware
1564. Set up user accounts and permissions
157
158### 3. Calibre Configuration
159
1601. Access Calibre-Web and complete initial setup
1612. Point to the Calibre library at `/books`
1623. Import existing ebook collection if available
1634. Configure metadata sources and preferences
164
165### 4. Backup Setup
166
1671. Install restic on client machines
1682. Use the generated client script: `/docker/restic-server/client-setup-example.sh`
1693. Initialize repositories and set up automated backups
170
171## Directory Structure
172
173```
174/docker/                          # Docker configurations
175âââ jellyfin/
176â   âââ config/                  # Jellyfin configuration
177â   âââ cache/                   # Transcoding cache
178â   âââ docker-compose.yml
179âââ arr-stack/
180â   âââ sonarr/config/          # Sonarr configuration
181â   âââ radarr/config/          # Radarr configuration
182â   âââ prowlarr/config/        # Prowlarr configuration
183â   âââ readarr/config/         # Readarr configuration
184â   âââ docker-compose.yml
185âââ calibre/
186â   âââ server/config/          # Calibre server config
187â   âââ web/config/             # Calibre-Web config
188â   âââ docker-compose.yml
189âââ restic-server/
190    âââ config/                 # Restic configuration
191    âââ docker-compose.yml
192
193/mnt/rstorage/                   # Media and data storage
194âââ media/
195â   âââ movies/                 # Movie files
196â   âââ tv/                     # TV show files
197â   âââ books/                  # Ebook library
198â   âââ music/                  # Music files
199âââ downloads/
200â   âââ complete/               # Completed downloads
201â   âââ incomplete/             # In-progress downloads
202âââ backups/                    # Restic backup data
203```
204
205## Security Considerations
206
207- All sensitive credentials are stored in encrypted vault variables
208- VPN routing protects download traffic when enabled
209- Backup server requires authentication
210- Services run with non-root user permissions
211- Docker network isolation between services
212
213## Troubleshooting
214
215### Health Check Script
216Run the generated health check script:
217```bash
218/docker/storage-health-check.sh
219```
220
221### Common Issues
222
223**VPN Not Working**: Check Gluetun logs and VPN credentials
224```bash
225docker logs gluetun
226```
227
228**Services Not Accessible**: Verify Docker network and port mappings
229```bash
230docker network ls
231docker compose ps
232```
233
234**Permission Issues**: Check directory ownership
235```bash
236ls -la /docker/
237ls -la /mnt/rstorage/
238```
239
240**Storage Full**: Monitor disk usage
241```bash
242df -h /mnt/rstorage/
243```
244
245## Tags
246
247Use these tags for targeted deployment:
248
249- `storage` - All storage services
250- `jellyfin` - Jellyfin media server only
251- `arr-stack` - All Arr services + VPN
252- `calibre` - Calibre ebook services
253- `restic` - Backup server only
254- `validation` - Health checks and validation
255
256## Dependencies
257
258- `geerlingguy.docker` - Docker installation
259- `community.docker` - Docker Ansible modules
260
261## License
262
263MIT License - See project root for details.

1# Storage Services Role 2 3Comprehensive media automation and backup services role for homelab environments. 4 5## Overview 6 7This role deploys a complete media automation and backup stack including: 8 9- **Jellyfin**: Media server for streaming movies, TV shows, music 10- **Arr Stack**: Automated media acquisition (Sonarr, Radarr, Prowlarr, Readarr) 11- **Gluetun VPN**: Secure VPN container for Arr stack traffic 12- **Calibre Stack**: Ebook management (Calibre + Calibre-Web) 13- **Restic Server**: Self-hosted backup solution 14 15## Services Included 16 17### Media Server 18- **Jellyfin** (Port 8096): Media streaming server with transcoding support 19 20### Media Automation (Arr Stack) 21- **Sonarr** (Port 8989): TV show automation 22- **Radarr** (Port 7878): Movie automation 23- **Prowlarr** (Port 9696): Indexer management 24- **Readarr** (Port 8787): Ebook/audiobook automation 25- **Gluetun**: VPN container for secure traffic routing 26 27### Ebook Management 28- **Calibre Server** (Port 8083): Full-featured ebook management 29- **Calibre-Web** (Port 8084): Web interface for library access 30 31### Backup Solution 32- **Restic Server** (Port 8000): REST-based backup repository server 33 34## Requirements 35 36- Docker and Docker Compose installed 37- Sufficient storage space mounted at `/mnt/rstorage` (default) 38- Network access for media downloads (if using Arr stack) 39- VPN credentials if using Gluetun for secure downloads 40 41## Usage 42 43### Basic Deployment 44 45```yaml 46# In your host variables 47storage_enabled: true 48jellyfin_enabled: true 49arr_stack_enabled: true 50calibre_enabled: true 51restic_backup_server_enabled: true 52``` 53 54### Deploy to Storage Servers 55 56```bash 57# Deploy all storage services 58ansible-playbook -i inventory/hosts site.yml --limit storage_servers --tags "core,storage" 59 60# Deploy only specific services 61ansible-playbook -i inventory/hosts site.yml --limit storage_servers --tags "jellyfin" 62ansible-playbook -i inventory/hosts site.yml --limit storage_servers --tags "arr-stack" 63``` 64 65## Configuration 66 67### Required Vault Variables 68 69For secure operation, configure these encrypted variables: 70 71```yaml 72# VPN Configuration (for Gluetun) 73vpn_service_provider: "nordvpn" # or "surfshark", "expressvpn", etc. 74wireguard_private_key: "your_wireguard_private_key" 75wireguard_addresses: "10.x.x.x/32" 76 77# Alternative OpenVPN credentials 78openvpn_user: "your_vpn_username" 79openvpn_password: "your_vpn_password" 80 81# Backup Authentication 82restic_backup_username: "backup_user" 83restic_backup_password: "secure_backup_password" 84 85# Optional: Service admin credentials 86jellyfin_admin_username: "admin" 87jellyfin_admin_password: "secure_password" 88calibre_web_admin_username: "admin" 89calibre_web_admin_password: "secure_password" 90``` 91 92### Storage Configuration 93 94```yaml 95# Storage paths (customize as needed) 96storage_base_path: "/mnt/rstorage" 97storage_docker_dir: "/docker" 98 99# Media directories will be created automatically: 100# - /mnt/rstorage/media/movies 101# - /mnt/rstorage/media/tv 102# - /mnt/rstorage/media/books 103# - /mnt/rstorage/downloads 104# - /mnt/rstorage/backups 105``` 106 107### Service Customization 108 109```yaml 110# Port configuration 111jellyfin_host_port: 8096 112sonarr_host_port: 8989 113radarr_host_port: 7878 114prowlarr_host_port: 9696 115readarr_host_port: 8787 116calibre_web_host_port: 8084 117restic_backup_host_port: 8000 118 119# Resource limits 120jellyfin_memory_limit: "4g" 121jellyfin_cpu_limit: "4" 122arr_memory_limit: "1g" 123arr_cpu_limit: "1" 124 125# VPN settings 126arr_vpn_enabled: true 127vpn_server_countries: "Netherlands,Switzerland" 128``` 129 130## Access URLs 131 132After deployment, services are accessible at: 133 134- Jellyfin: `http://your-server:8096` 135- Sonarr: `http://your-server:8989` 136- Radarr: `http://your-server:7878` 137- Prowlarr: `http://your-server:9696` 138- Readarr: `http://your-server:8787` 139- Calibre-Web: `http://your-server:8084` 140- Restic Server: `http://your-server:8000` 141 142## Post-Deployment Setup 143 144### 1. Arr Stack Configuration 145 1461. **Prowlarr First**: Configure indexers and connect to other Arr apps 1472. **Quality Profiles**: Set up quality preferences in each Arr app 1483. **Root Folders**: Add media directories (already mounted) 1494. **Download Client**: Configure your preferred download client 150 151### 2. Jellyfin Setup 152 1531. Complete the initial setup wizard 1542. Add media libraries pointing to `/media/movies`, `/media/tv`, etc. 1553. Configure transcoding settings based on your hardware 1564. Set up user accounts and permissions 157 158### 3. Calibre Configuration 159 1601. Access Calibre-Web and complete initial setup 1612. Point to the Calibre library at `/books` 1623. Import existing ebook collection if available 1634. Configure metadata sources and preferences 164 165### 4. Backup Setup 166 1671. Install restic on client machines 1682. Use the generated client script: `/docker/restic-server/client-setup-example.sh` 1693. Initialize repositories and set up automated backups 170 171## Directory Structure 172 173``` 174/docker/ # Docker configurations 175âââ jellyfin/ 176â âââ config/ # Jellyfin configuration 177â âââ cache/ # Transcoding cache 178â âââ docker-compose.yml 179âââ arr-stack/ 180â âââ sonarr/config/ # Sonarr configuration 181â âââ radarr/config/ # Radarr configuration 182â âââ prowlarr/config/ # Prowlarr configuration 183â âââ readarr/config/ # Readarr configuration 184â âââ docker-compose.yml 185âââ calibre/ 186â âââ server/config/ # Calibre server config 187â âââ web/config/ # Calibre-Web config 188â âââ docker-compose.yml 189âââ restic-server/ 190 âââ config/ # Restic configuration 191 âââ docker-compose.yml 192 193/mnt/rstorage/ # Media and data storage 194âââ media/ 195â âââ movies/ # Movie files 196â âââ tv/ # TV show files 197â âââ books/ # Ebook library 198â âââ music/ # Music files 199âââ downloads/ 200â âââ complete/ # Completed downloads 201â âââ incomplete/ # In-progress downloads 202âââ backups/ # Restic backup data 203``` 204 205## Security Considerations 206 207- All sensitive credentials are stored in encrypted vault variables 208- VPN routing protects download traffic when enabled 209- Backup server requires authentication 210- Services run with non-root user permissions 211- Docker network isolation between services 212 213## Troubleshooting 214 215### Health Check Script 216Run the generated health check script: 217```bash 218/docker/storage-health-check.sh 219``` 220 221### Common Issues 222 223**VPN Not Working**: Check Gluetun logs and VPN credentials 224```bash 225docker logs gluetun 226``` 227 228**Services Not Accessible**: Verify Docker network and port mappings 229```bash 230docker network ls 231docker compose ps 232``` 233 234**Permission Issues**: Check directory ownership 235```bash 236ls -la /docker/ 237ls -la /mnt/rstorage/ 238``` 239 240**Storage Full**: Monitor disk usage 241```bash 242df -h /mnt/rstorage/ 243``` 244 245## Tags 246 247Use these tags for targeted deployment: 248 249- `storage` - All storage services 250- `jellyfin` - Jellyfin media server only 251- `arr-stack` - All Arr services + VPN 252- `calibre` - Calibre ebook services 253- `restic` - Backup server only 254- `validation` - Health checks and validation 255 256## Dependencies 257 258- `geerlingguy.docker` - Docker installation 259- `community.docker` - Docker Ansible modules 260 261## License 262 263MIT License - See project root for details.