Server Automation with Ansible

Comprehensive Ansible playbooks for server provisioning and configuration management.

🔐 SECURITY NOTICE

This repository is designed for public sharing while keeping your actual server details private.

What's Public (✅ Safe)

• Ansible playbooks and roles
• Configuration templates
• Documentation
• Role defaults and examples

What's Protected (🔒 Git-ignored)

• Real server IP addresses (inventory/local.yml)
• SSH keys and passwords (.vault_password)
• Encrypted secrets (group_vars/vault.yml)
• Host-specific configurations (host_vars/*.yml)

Quick Setup for New Users

1. Install Python dependencies: pip install -r requirements-python.txt
2. Run the setup script: ./setup-inventory.sh
3. Edit your server details: inventory/local.yml
4. Configure secrets: ansible-vault edit group_vars/vault.yml
5. Test connectivity: ansible all -m ping

See inventory/README.md for detailed instructions.

Playbook Structure

Main Playbooks

• provision.yml - Full server provisioning (orchestrates all roles)
• deploy.yml - Service deployment and management (CI/CD pipeline)
• prod.yml - Production environment provisioning
• staging.yml - Staging environment provisioning
• dev.yml - Development environment provisioning

Role-Specific Playbooks

• user.yml - User management with dynamic host targeting
• homeassistant.yml - Complete Home Assistant Supervised installation
• role-system.yml - System configuration only
• role-backup.yml - Backup system configuration
• role-gui.yml - GNOME desktop environment
• role-snapcast.yml - Snapcast audio client
• role-docker.yml - Docker engine installation (geerlingguy.docker)
• role-security.yml - Comprehensive security hardening (geerlingguy.security)

Security Playbooks

• security-baseline.yml - Fundamental security hardening
• firewall.yml - UFW firewall configuration

Quick Start Examples

User Management with Dynamic Targeting

# Deploy user to specific server
ansible-playbook user.yml -e target_host=192.168.1.100

# Deploy user to inventory group
ansible-playbook user.yml -e target_host=homeassistant

# Custom user name
ansible-playbook user.yml -e target_host=server1 -e user_name=admin

Home Assistant Installation

# Complete HA installation (requires geerlingguy.docker)
ansible-playbook homeassistant.yml

# Specific server from homeassistant group
ansible-playbook homeassistant.yml -l ha-server-01

# Install only specific components
ansible-playbook homeassistant.yml --tags "user,system,docker"

Directory Structure

server_automation/
�� ansible.cfg           # Ansible configuration
�� requirements.yml      # Galaxy role dependencies
�� provision.yml         # Main provisioning playbook
�� prod.yml              # Production environment
�� staging.yml           # Staging environment
�� dev.yml               # Development environment
�� role-*.yml           # Role-specific playbooks
�� security-*.yml       # Security playbooks
�� inventory/
   �� hosts            # Main inventory
   �� production       # Production hosts
   �� staging          # Staging hosts
   �� development      # Development hosts
   �� group_vars/      # Group variables
   �� host_vars/       # Host-specific variables
�� group_vars/
   �� all.yml          # Global variables
   �� prod.yml         # Production variables
   �� staging.yml      # Staging variables
   �� dev.yml          # Development variables
   �� security.yml     # Security variables
�� host_vars/
   �� example-server.yml # Host template
�� roles/
    �� user/            # User management
    �� system/          # System configuration
    �� backup/          # Backup system
    �� gui-gnome/       # GNOME desktop
    �� snapcast-client/ # Audio streaming
    �� security-baseline/ # Security hardening
    �� firewall/        # Firewall configuration

Service Deployment Structure

The services/ directory contains service-specific deployment roles:

services/
   web/                 # Web services (Nginx, Apache)
   api/                 # API services (Node.js, Python)  
   database/            # Database services (PostgreSQL, Redis)
   monitoring/          # Monitoring stack (Prometheus, Grafana)
   cache/               # Caching services (Redis, Memcached)
   templates/           # Service templates and examples
     service-template/  # Template for creating new services

CI/CD Workflows

.gitea/workflows/
   ci.yml              # Validation workflow (lint, test, security)
   deploy.yml          # Deployment workflow (staging, production)

Geerlingguy Roles Integration

This repository integrates two key geerlingguy roles:

geerlingguy.docker (v7.0.0)

• Docker Engine installation and management
• Docker Compose installation
• User group configuration for Docker access

geerlingguy.security (v3.0.0)

• Comprehensive system security hardening
• SSH security configuration
• Fail2Ban installation and configuration
• System auditing with auditd
• Network security hardening
• Automatic security updates

See ROLES-INTEGRATION.md for detailed configuration and usage.

Usage

Installation

# 1. Install Ansible Galaxy roles
ansible-galaxy install -r requirements.yml

# 2. Install collections
ansible-galaxy collection install -r collections.yml

# 3. Set up secure inventory (first time only)
./setup-inventory.sh

# 4. Configure your servers and secrets
nano inventory/local.yml
ansible-vault edit group_vars/vault.yml

Running Playbooks

Full provisioning:

ansible-playbook provision.yml

Environment-specific (if using multiple inventories):

ansible-playbook prod.yml -i inventory/production.yml
ansible-playbook staging.yml -i inventory/staging.yml  
ansible-playbook dev.yml

Role-specific:

ansible-playbook role-system.yml
ansible-playbook role-docker.yml

Security hardening:

ansible-playbook security-baseline.yml -e "security_level=high"
ansible-playbook firewall.yml

Tags

Playbooks support granular execution using tags:

# Run only system tasks
ansible-playbook provision.yml --tags system

# Run everything except GUI
ansible-playbook provision.yml --skip-tags gui

# Run multiple tags
ansible-playbook provision.yml --tags system,user,docker

Available tags: system, user, backup, gui, snapcast, docker, security, firewall

Variables and Configuration

Environment Variables

• env: Environment type (prod, staging, dev)
• security_level: Security strictness (low, medium, high)

Role Configuration

Each role has its own defaults in roles/<role>/defaults/main.yml

Variable Precedence

1. Host variables (host_vars/)
2. Group variables (group_vars/)
3. Role defaults
4. Playbook variables

Security Features

• SSH hardening with fail2ban
• UFW firewall configuration
• Password policy enforcement
• Auditd logging (high security)
• Package security updates
• File permission hardening

Best Practices

• Idempotent playbooks (safe to run multiple times)
• Proper error handling with any_errors_fatal
• Environment-specific configurations
• Tag-based execution for granular control
• Variable precedence following Ansible standards
• Comprehensive logging and reporting

Dependencies

• geerlingguy.docker (Docker engine)
• community.docker (Docker modules)
• ansible.posix (POSIX utilities)
• Various security and monitoring roles

Contributing

1. Follow existing patterns and conventions
2. Maintain idempotency
3. Use appropriate tags
4. Update documentation
5. Test across environments

Ansible Vault Secrets Management

This repository includes comprehensive Ansible Vault integration for secure secrets management:

Vault Structure

vault/
  envs/           # Environment-specific secrets
    dev/
    staging/
    prod/
  secrets/        # Categorized secrets
    api_keys/
    certificates/
    databases/
    ssh_keys/
    cloud_credentials/
  passwords/      # Vault password management
  scripts/        # Helper scripts
  templates/      # Template files

Quick Start

1. Generate vault passwords:

   ./vault/scripts/generate-vault-passwords.sh
   

2. Encrypt vault files:

   ./vault/scripts/encrypt-vault.sh dev
   ./vault/scripts/encrypt-vault.sh staging  
   ./vault/scripts/encrypt-vault.sh prod
   

3. Use in playbooks:

   ANSIBLE_ENV=dev ansible-playbook playbook.yml --vault-password-file=./vault/passwords/.vault-pass.envs
   

See vault/README.md for complete documentation.

Inventory Management

The inventory system supports multiple environments and dynamic cloud integration:

Environments

• Production: High security, strict policies
• Staging: Pre-production testing
• Development: Development and experimentation

Cloud Integration

• AWS EC2 dynamic inventory
• DigitalOcean dynamic inventory
• Custom cloud provider support

See inventory/README.md for detailed usage.

Testing and Validation

Syntax Validation

# Validate playbook syntax
ansible-playbook --syntax-check provision.yml

# Validate inventory
ansible-inventory -i inventory/hosts --list

Integration Testing

# Test vault integration
./vault/scripts/test-vault-integration.sh dev

# Test role integration  
./validate-integration.sh

Dry Runs

# Check what would change
ansible-playbook provision.yml --check --diff

CI/CD Pipeline

This repository includes a comprehensive CI/CD pipeline using Forgejo/Gitea Actions with separate validation and deployment workflows.

Pipeline Overview

Code Push → CI Validation → Merge to Main → Automatic Deployment
     ↓            ↓              ↓                ↓
  Feature    Syntax Check    Staging Deploy   Production Deploy
  Branch     Security Scan   Health Checks    Manual Approval
             Integration     Rollback Ready   Full Monitoring

Workflows

1. CI Workflow (.gitea/workflows/ci.yml)

Triggers: Push to main/develop, Pull Requests Purpose: Comprehensive validation without deployment

Jobs:

• Lint & Syntax Check: YAML validation, Ansible linting, syntax verification
• Integration Tests: Custom validation, dry-run testing, role structure validation
• Documentation Check: README validation, role documentation completeness
• Security Scan: Security best practices, secrets detection, permission checks
• Summary: Consolidated results and status reporting

2. Deploy Workflow (.gitea/workflows/deploy.yml)

Triggers: Push to main (after CI success), Manual workflow dispatch Purpose: Service deployment with rollback capability

Jobs:

• Check CI Status: Ensures CI pipeline completed successfully
• Prepare Deployment: Determines target environments and services
• Deploy Staging: Automatic deployment to staging environment
• Deploy Production: Manual approval required for production
• Emergency Rollback: Automatic rollback on deployment failure

Service Deployment

Deploy Playbook (deploy.yml)

Separate from site.yml, focuses exclusively on service deployment:

# Deploy all services to staging
ansible-playbook -i inventory/staging deploy.yml

# Deploy specific services
ansible-playbook -i inventory/production deploy.yml \
  --extra-vars "target_services=web,api"

# Rollback deployment
ansible-playbook -i inventory/production deploy.yml \
  --tags rollback --extra-vars "rollback_version=v1.2.3"

Service Categories

• Web Services: Frontend applications, reverse proxies (Nginx)
• API Services: Backend APIs, microservices (Node.js, Python)
• Database Services: PostgreSQL, Redis with backup strategies
• Monitoring Services: Prometheus, Grafana, Alertmanager
• Cache Services: Redis, Memcached for performance optimization

Adding New Services

1. Create Service Role:

   cp -r services/templates/service-template services/your-service
   # Edit and customize the service configuration
   

2. Update Deploy Configuration:

   # In group_vars/all.yml or environment-specific files
   your_service_services:
     - name: your-service-app
       image: your-org/your-service
       tag: "v1.0.0"
       ports: ["8080:8080"]
       environment:
         ENV: "production"
   

3. Add Health Checks:

   your_service_health_checks:
     - name: your-service-app
       port: 8080
       path: "/health"
       expected_status: 200
   

Environment Configuration

Staging Environment

• Automatic deployment on main branch merges
• Rolling updates with 50% parallel deployment
• Quick health checks (3 retries, 15s delay)
• Limited retention (7 days backup)

Production Environment

• Manual approval required for deployment
• Blue-green strategy for zero-downtime deployment
• Comprehensive health checks (5 retries, 30s delay)
• Extended retention (30 days backup)
• Email notifications on deployment events

Self-Hosted Runners

The deployment workflow requires self-hosted runners for:

• Server Access: Direct access to deployment targets
• Network Security: Internal network connectivity
• Custom Tools: Ansible, SSH keys, monitoring tools
• Performance: Faster deployment without external dependencies

Runner Setup

# Install Forgejo Runner
sudo apt update
sudo apt install docker.io ansible python3-pip
pip3 install docker

# Configure runner with repository
./setup-runner.sh --token YOUR_RUNNER_TOKEN

Deployment Security

Secrets Management

Required secrets in Forgejo/Gitea repository settings:

• SSH_PRIVATE_KEY: SSH key for server access
• STAGING_HOST: Staging server hostname/IP
• PRODUCTION_HOST: Production server hostname/IP
• VAULT_PASSWORD: Ansible Vault password for secrets

Environment Protection

• Staging: No restrictions, automatic deployment
• Production: Manual approval required, deployment protection rules
• Rollback: Emergency rollback environment with restricted access

Monitoring and Rollback

Health Checks

All deployments include comprehensive health monitoring:

• Container Health: Docker container status and health checks
• Service Endpoints: HTTP health check endpoints
• Database Connectivity: Connection and query validation
• Resource Usage: Memory, CPU, disk space monitoring

Rollback Procedures

# Automatic rollback (triggered on deployment failure)
ansible-playbook -i inventory/production deploy.yml --tags rollback

# Manual rollback to specific version
ansible-playbook -i inventory/production deploy.yml \
  --tags rollback --extra-vars "rollback_version=20231201-123456"

Deployment Logs

• Location: /var/log/deployments/deployment.log
• Retention: Automatic cleanup with configurable retention
• Artifacts: Deployment logs uploaded as workflow artifacts
• Notifications: Email alerts on deployment success/failure

Local Testing

Pre-commit Validation

# Test the same checks that run in CI
./test-ci-locally.sh

# Run specific validation
./validate-integration.sh

# Test deployment playbook
ansible-playbook deploy.yml --check --diff -i inventory/staging

Service Testing

# Test new service locally
docker-compose -f services/your-service/docker-compose.yml up -d

# Test service health
curl -f http://localhost:8080/health

# Test with Ansible
ansible-playbook deploy.yml --tags your-service --check

Troubleshooting

Common Issues

1. Missing roles: Run ansible-galaxy install -r requirements.yml
2. Vault errors: Ensure vault passwords are set and files are encrypted
3. Permission issues: Check SSH keys and sudo privileges
4. Network issues: Verify inventory host connectivity
5. CI failures: Check .gitea/workflows/README.md for troubleshooting

Debugging

# Enable verbose output
ansible-playbook provision.yml -vvv

# Debug specific tasks
ansible-playbook provision.yml --tags system -vvv

# Debug deployment issues
ansible-playbook deploy.yml --check --diff --tags health-check

CI/CD Troubleshooting

# Check CI status
git status && git log --oneline -5

# Test CI locally
./test-ci-locally.sh

# Debug deployment
ansible-playbook deploy.yml --check --diff -vvv

# View deployment logs
tail -f /var/log/deployments/deployment.log

Best Practices

Security

• Use Ansible Vault for all secrets
• Follow principle of least privilege
• Regular security updates and auditing
• Environment-specific security levels

Maintenance

• Regular password rotation
• Backup vault passwords securely
• Test playbooks in staging before production
• Monitor for deprecated modules

Performance

• Use tags for targeted execution
• Leverage facts caching
• Optimize with strategy plugins
• Use async tasks for long-running operations

Support

For issues and questions:

1. Check the troubleshooting section
2. Review role-specific README files
3. Validate syntax and integration
4. Test in development environment first

License

This project is licensed under the MIT License - see the LICENSE file for details.