Devops

Systemd services look simple until they don’t start, restart unexpectedly, or fail silently. Here’s how to write service files that work reliably in production. Basic Structure 1 2 3 4 5 6 7 8 9 10 11 12 # /etc/systemd/system/myapp.service [Unit] Description=My Application After=network.target [Service] Type=simple ExecStart=/usr/bin/myapp Restart=always [Install] WantedBy=multi-user.target Three sections, three purposes: [Unit] - What is this, what does it depend on [Service] - How to run it [Install] - When to start it Service Types Type=simple (Default) 1 2 3 [Service] Type=simple ExecStart=/usr/bin/myapp Systemd considers the service started immediately when ExecStart runs. Use when your process stays in foreground. ...

Local Terraform state works for learning. Production requires remote state—for team collaboration, state locking, and not losing your infrastructure when your laptop dies. Here’s how to set it up properly. Why Remote State? Local state (terraform.tfstate) has problems: No collaboration - Team members overwrite each other’s changes No locking - Concurrent applies corrupt state No backup - Laptop dies, state is gone, orphaned resources everywhere Secrets in plain text - State contains sensitive data Remote backends solve all of these. ...

Docker Compose tutorials show you docker compose up. Production requires health checks, resource limits, proper logging, restart policies, and deployment strategies. Here’s how to bridge that gap. Base Configuration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # docker-compose.yml version: "3.8" services: app: image: myapp:${VERSION:-latest} build: context: . dockerfile: Dockerfile restart: unless-stopped environment: - NODE_ENV=production env_file: - .env ports: - "3000:3000" This is a starting point. Let’s make it production-ready. ...

Ansible is easy to start and hard to master. A simple playbook works great for 5 servers. The same playbook becomes unmaintainable at 50. Here are the patterns that keep Ansible codebases sane as they grow. Project Structure Start with a structure that scales: a ├ ├ │ │ │ │ │ │ │ │ │ ├ │ │ │ ├ │ │ │ └ n ─ ─ ─ ─ ─ s ─ ─ ─ ─ ─ i b a i ├ │ │ │ │ └ p ├ ├ └ r ├ ├ └ g └ l n n ─ ─ l ─ ─ ─ o ─ ─ ─ r ─ e s v ─ ─ a ─ ─ ─ l ─ ─ ─ o ─ / i e y e u b n p ├ └ s ├ └ b s w d s c n p p a ├ └ l t r ─ ─ t ─ ─ o i e a / o g o _ l ─ ─ e o o ─ ─ a ─ ─ o t b t m i s v l ─ ─ . r d g k e s a m n t a / c y u h g ├ └ i h g └ s . e b o x g r v v f / c o r ─ ─ n o r ─ y r a n r s a a g t s o ─ ─ g s o ─ m v s e / r u i t u / t u l e e s s l o s p a w s p a r s / . t n . _ l e . _ l s . y . y v l b y v l . y m y m a . s m a . y m l m l r y e l r y m l l s m r s m l / l v / l e r s . y m l The key insight: separate inventory per environment. Never mix production and staging in the same inventory file. ...

If you SSH into servers, run long processes, or just want a better terminal experience, tmux changes everything. Sessions persist through disconnects, panes let you see multiple things at once, and once you build muscle memory, you’ll wonder how you worked without it. Why tmux? Sessions survive disconnects - SSH drops, laptop sleeps, terminal crashes? Your work continues. Split panes - Editor on the left, tests on the right, logs at the bottom. Multiple windows - Like browser tabs for your terminal. Pair programming - Share a session with someone else in real-time. Scriptable - Automate your development environment setup. Getting Started 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Start new session tmux # Start named session tmux new -s myproject # Detach (inside tmux) Ctrl+b d # List sessions tmux ls # Attach to session tmux attach -t myproject # Kill session tmux kill-session -t myproject The key prefix is Ctrl+b by default. Press it, release, then press the next key. ...

curl is the universal HTTP client. It’s installed everywhere, works with any API, and once mastered, becomes your go-to tool for testing, debugging, and scripting HTTP interactions. Basic Requests 1 2 3 4 5 6 7 8 # GET (default) curl https://api.example.com/users # Explicit methods curl -X POST https://api.example.com/users curl -X PUT https://api.example.com/users/1 curl -X DELETE https://api.example.com/users/1 curl -X PATCH https://api.example.com/users/1 Adding Headers 1 2 3 4 5 6 7 8 # Single header curl -H "Authorization: Bearer token123" https://api.example.com/me # Multiple headers curl -H "Authorization: Bearer token123" \ -H "Content-Type: application/json" \ -H "Accept: application/json" \ https://api.example.com/users Sending Data JSON Body 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Inline JSON curl -X POST https://api.example.com/users \ -H "Content-Type: application/json" \ -d '{"name": "Alice", "email": "alice@example.com"}' # From file curl -X POST https://api.example.com/users \ -H "Content-Type: application/json" \ -d @payload.json # From stdin echo '{"name": "Alice"}' | curl -X POST https://api.example.com/users \ -H "Content-Type: application/json" \ -d @- Form Data 1 2 3 4 5 6 7 8 # URL-encoded (default for -d without Content-Type) curl -X POST https://api.example.com/login \ -d "username=alice&password=secret" # Multipart form (file uploads) curl -X POST https://api.example.com/upload \ -F "file=@document.pdf" \ -F "description=My document" Response Handling Show Headers 1 2 3 4 5 6 7 8 # Response headers only curl -I https://api.example.com/health # Headers + body curl -i https://api.example.com/users # Verbose (request + response headers) curl -v https://api.example.com/users Output Control 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Save to file curl -o response.json https://api.example.com/users # Save with remote filename curl -O https://example.com/file.zip # Silent (no progress bar) curl -s https://api.example.com/users # Silent but show errors curl -sS https://api.example.com/users # Only output body (suppress all else) curl -s https://api.example.com/users | jq '.' Extract Specific Info 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # HTTP status code only curl -s -o /dev/null -w "%{http_code}" https://api.example.com/health # Multiple variables curl -s -o /dev/null -w "Status: %{http_code}\nTime: %{time_total}s\nSize: %{size_download} bytes\n" \ https://api.example.com/users # Available variables # %{http_code} - HTTP status code # %{time_total} - Total time in seconds # %{time_connect} - Time to establish connection # %{time_starttransfer} - Time to first byte # %{size_download} - Downloaded bytes # %{url_effective} - Final URL after redirects Authentication 1 2 3 4 5 6 7 8 9 10 11 # Basic auth curl -u username:password https://api.example.com/secure # Bearer token curl -H "Authorization: Bearer eyJhbG..." https://api.example.com/me # API key in header curl -H "X-API-Key: abc123" https://api.example.com/data # API key in query string curl "https://api.example.com/data?api_key=abc123" Following Redirects 1 2 3 4 5 6 7 8 # Follow redirects (disabled by default) curl -L https://short.url/abc # Limit redirect count curl -L --max-redirs 5 https://example.com # Show redirect chain curl -L -v https://short.url/abc 2>&1 | grep "< location" Timeouts and Retries 1 2 3 4 5 6 7 8 9 10 11 # Connection timeout (seconds) curl --connect-timeout 5 https://api.example.com # Total operation timeout curl --max-time 30 https://api.example.com/slow-endpoint # Retry on failure curl --retry 3 --retry-delay 2 https://api.example.com # Retry on specific HTTP codes curl --retry 3 --retry-all-errors https://api.example.com SSL/TLS Options 1 2 3 4 5 6 7 8 # Skip certificate verification (development only!) curl -k https://self-signed.example.com # Use specific CA certificate curl --cacert /path/to/ca.crt https://api.example.com # Client certificate authentication curl --cert client.crt --key client.key https://api.example.com Cookies 1 2 3 4 5 6 7 8 9 10 11 # Send cookies curl -b "session=abc123; token=xyz" https://api.example.com # Save cookies to file curl -c cookies.txt https://api.example.com/login -d "user=alice&pass=secret" # Load cookies from file curl -b cookies.txt https://api.example.com/dashboard # Both (maintain session) curl -b cookies.txt -c cookies.txt https://api.example.com/action Useful Patterns Health Check Script 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 #!/bin/bash check_health() { local url=$1 local status=$(curl -s -o /dev/null -w "%{http_code}" --max-time 5 "$url") if [ "$status" = "200" ]; then echo "✓ $url" return 0 else echo "✗ $url (HTTP $status)" return 1 fi } check_health "https://api.example.com/health" check_health "https://web.example.com" API Testing 1 2 3 4 5 6 7 8 9 10 # Create resource and capture ID ID=$(curl -s -X POST https://api.example.com/users \ -H "Content-Type: application/json" \ -d '{"name": "Test User"}' | jq -r '.id') # Use captured ID curl -s https://api.example.com/users/$ID | jq '.' # Delete curl -X DELETE https://api.example.com/users/$ID Download with Progress 1 2 3 4 5 # Show progress bar curl -# -O https://example.com/large-file.zip # Resume interrupted download curl -C - -O https://example.com/large-file.zip Parallel Requests 1 2 3 4 5 6 7 8 # Using xargs echo -e "url1\nurl2\nurl3" | xargs -P 4 -I {} curl -s {} -o /dev/null -w "{}: %{http_code}\n" # Using curl's parallel feature (7.68+) curl --parallel --parallel-immediate \ https://api1.example.com \ https://api2.example.com \ https://api3.example.com Debugging Trace All Details 1 2 # Full trace including SSL handshake curl -v --trace-ascii debug.txt https://api.example.com Common Issues 1 2 3 4 5 6 7 8 9 # DNS resolution problems curl -v --resolve api.example.com:443:1.2.3.4 https://api.example.com # Force IPv4 or IPv6 curl -4 https://api.example.com # IPv4 only curl -6 https://api.example.com # IPv6 only # Use specific interface curl --interface eth0 https://api.example.com Config File Save common options in ~/.curlrc: ...

Environment files are deceptively simple. A few KEY=value pairs, what could go wrong? Quite a bit, actually. Here’s how to manage them without shooting yourself in the foot. The Basic Rules 1 2 3 4 # .env DATABASE_URL=postgres://localhost:5432/myapp API_KEY=sk_test_abc123 DEBUG=true Rule 1: Never commit secrets to git. 1 2 3 4 5 6 # .gitignore .env .env.local .env.*.local *.pem *.key Rule 2: Always commit an example file. 1 2 3 4 # .env.example (committed to repo) DATABASE_URL=postgres://localhost:5432/myapp API_KEY=your_api_key_here DEBUG=true New developers copy .env.example to .env and fill in their values. The example documents what’s needed without exposing real credentials. ...

Running AI workloads in containers presents unique challenges that traditional web application patterns don’t address. GPU scheduling, model caching, and bursty inference traffic all require thoughtful architecture. Here’s what actually works in production. The GPU Scheduling Problem Standard Kubernetes scheduling assumes CPU and memory are your primary constraints. When you add GPUs to the mix, everything changes. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 apiVersion: v1 kind: Pod metadata: name: inference-server spec: containers: - name: model image: my-registry/llm-server:v1.2 resources: limits: nvidia.com/gpu: 1 memory: "32Gi" requests: nvidia.com/gpu: 1 memory: "24Gi" The naive approach—one GPU per pod—works until you realize GPUs cost $2-4/hour and sit idle between requests. MIG (Multi-Instance GPU) and time-slicing help, but they introduce complexity: ...

If you work with APIs, logs, or configuration files, you work with JSON. And if you work with JSON from the command line, jq is indispensable. Here are the patterns I use daily. The Basics 1 2 3 4 5 6 7 8 9 10 # Pretty print echo '{"name":"alice","age":30}' | jq '.' # Extract a field echo '{"name":"alice","age":30}' | jq '.name' # "alice" # Raw output (no quotes) echo '{"name":"alice","age":30}' | jq -r '.name' # alice The -r flag is your friend. Use it whenever you want the actual value, not a JSON string. ...

SSH tunnels are one of those tools that seem magical until you understand the three basic patterns. Once you do, you’ll use them constantly. The Three Types Local forwarding (-L): Access a remote service as if it were local Remote forwarding (-R): Expose a local service to a remote network Dynamic forwarding (-D): Create a SOCKS proxy through the SSH connection Let’s break down each one. Local Forwarding: Reach Remote Services Locally Scenario: You need to access a database that’s only available from a server you can SSH into. ...