Posts

grep, awk, and sed are the foundational text processing tools in Unix. They’re old, they’re cryptic, and they’re incredibly powerful once you learn them. grep: Search and Filter grep searches for patterns in text. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # Basic search grep "error" logfile.txt # Case insensitive grep -i "error" logfile.txt # Show line numbers grep -n "error" logfile.txt # Count matches grep -c "error" logfile.txt # Invert (lines NOT matching) grep -v "debug" logfile.txt # Recursive search grep -r "TODO" ./src/ # Only filenames grep -l "password" *.conf Regex with grep 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Extended regex (-E or egrep) grep -E "error|warning|critical" logfile.txt # Word boundary grep -w "fail" logfile.txt # Matches "fail" not "failure" # Line start/end grep "^Error" logfile.txt # Lines starting with Error grep "done$" logfile.txt # Lines ending with done # Any character grep "user.name" logfile.txt # user1name, username, user_name # Character classes grep "[0-9]" logfile.txt # Lines with digits grep "[A-Za-z]" logfile.txt # Lines with letters Context 1 2 3 4 5 6 7 8 # Lines before match grep -B 3 "error" logfile.txt # Lines after match grep -A 3 "error" logfile.txt # Lines before and after grep -C 3 "error" logfile.txt Real Examples 1 2 3 4 5 6 7 8 9 10 11 # Find IP addresses grep -E "\b[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\b" access.log # Find function definitions grep -n "^def \|^function " *.py *.js # Exclude directories grep -r "config" . --exclude-dir={node_modules,.git} # Find files NOT containing pattern grep -L "copyright" *.py sed: Stream Editor sed transforms text line by line. ...

SSH into a server, start a long-running process, lose connection, lose everything. tmux solves this by keeping sessions alive independently of your terminal. Why tmux? Persistence: Sessions survive disconnections Multiplexing: Multiple windows and panes in one terminal Remote pairing: Share sessions with teammates Scriptable: Automate complex layouts Basic Concepts t ├ │ │ │ │ └ m ─ ─ u ─ ─ x S ├ │ │ └ S e ─ ─ e s ─ ─ s s s i W ├ └ W i o i ─ ─ i o n n ─ ─ n n d d ( o P P o 2 n w a a w a n n m 1 e e 2 e d ( 1 2 l c i o k l e l e a c t t i a o b n ) o f w i n d o w s ) Getting Started 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Start new session tmux # Start named session tmux new -s myproject # List sessions tmux ls # Attach to session tmux attach -t myproject # Attach to last session tmux attach The Prefix Key All tmux commands start with a prefix (default: Ctrl+b), then a key. ...

Every developer ends up using curl. It’s installed everywhere, works with any API, and once you learn the patterns, it’s faster than any GUI tool. Basic Requests 1 2 3 4 5 6 7 8 9 10 11 # GET (default) curl https://api.example.com/users # POST with data curl -X POST https://api.example.com/users \ -d '{"name":"Alice"}' # Other methods curl -X PUT ... curl -X DELETE ... curl -X PATCH ... Headers 1 2 3 4 5 6 7 8 9 10 # Set content type curl -H "Content-Type: application/json" ... # Auth header curl -H "Authorization: Bearer TOKEN" ... # Multiple headers curl -H "Content-Type: application/json" \ -H "Authorization: Bearer TOKEN" \ -H "X-Custom-Header: value" ... JSON Data 1 2 3 4 5 6 7 8 9 10 11 12 # Inline JSON curl -X POST https://api.example.com/users \ -H "Content-Type: application/json" \ -d '{"name":"Alice","email":"alice@example.com"}' # From file curl -X POST https://api.example.com/users \ -H "Content-Type: application/json" \ -d @data.json # Pretty output (pipe to jq) curl -s https://api.example.com/users | jq Response Info 1 2 3 4 5 6 7 8 9 10 11 # Show response headers curl -i https://api.example.com/users # Only headers (no body) curl -I https://api.example.com/users # HTTP status code only curl -s -o /dev/null -w "%{http_code}" https://api.example.com/users # Timing info curl -w "Time: %{time_total}s\n" -o /dev/null -s https://api.example.com Authentication 1 2 3 4 5 6 7 8 9 10 11 # Basic auth curl -u username:password https://api.example.com # Bearer token curl -H "Authorization: Bearer eyJ..." https://api.example.com # API key in header curl -H "X-API-Key: abc123" https://api.example.com # API key in query curl "https://api.example.com?api_key=abc123" Form Data 1 2 3 4 5 6 7 8 # URL-encoded form curl -X POST https://example.com/login \ -d "username=alice&password=secret" # Multipart form (file upload) curl -X POST https://example.com/upload \ -F "file=@photo.jpg" \ -F "description=My photo" Following Redirects 1 2 3 4 5 # Follow redirects (3xx) curl -L https://example.com/redirect # Show redirect chain curl -L -v https://example.com/redirect 2>&1 | grep "< HTTP\|< Location" SSL/TLS 1 2 3 4 5 6 7 8 # Ignore SSL errors (dev only!) curl -k https://self-signed.example.com # Use specific cert curl --cacert /path/to/ca.crt https://api.example.com # Client cert authentication curl --cert client.crt --key client.key https://api.example.com Timeouts 1 2 3 4 5 6 7 8 # Connection timeout (seconds) curl --connect-timeout 5 https://api.example.com # Max time for entire operation curl --max-time 30 https://api.example.com # Both curl --connect-timeout 5 --max-time 30 https://api.example.com Debugging 1 2 3 4 5 6 7 8 # Verbose output curl -v https://api.example.com # Even more verbose (includes SSL handshake) curl -vv https://api.example.com # Trace everything to file curl --trace trace.log https://api.example.com Saving Output 1 2 3 4 5 6 7 8 # Save to file curl -o response.json https://api.example.com/data # Save with remote filename curl -O https://example.com/file.zip # Silent mode (no progress) curl -s https://api.example.com Cookies 1 2 3 4 5 6 7 8 # Send cookie curl -b "session=abc123" https://example.com # Save cookies to file curl -c cookies.txt https://example.com/login -d "user=alice" # Send cookies from file curl -b cookies.txt https://example.com/dashboard Retry Logic 1 2 3 4 5 6 7 8 # Retry on failure curl --retry 3 https://api.example.com # Retry with delay curl --retry 3 --retry-delay 2 https://api.example.com # Retry on specific errors curl --retry 3 --retry-connrefused https://api.example.com Useful Aliases Add to your .bashrc: ...

APIs return JSON. Config files use JSON. Logs are often JSON. The jq tool lets you slice, filter, and transform JSON from the command line—no scripting required. Basic Syntax 1 2 3 4 5 jq 'filter' input.json # or cat input.json | jq 'filter' # or curl -s api.example.com | jq 'filter' Pretty Print 1 2 3 4 5 6 7 8 9 10 11 # Ugly JSON echo '{"name":"Alice","age":30}' | jq # Output: # { # "name": "Alice", # "age": 30 # } # Compact (remove whitespace) echo '{"name": "Alice"}' | jq -c # {"name":"Alice"} Extract Fields 1 2 3 4 5 6 echo '{"name":"Alice","age":30}' | jq '.name' # "Alice" # Without quotes echo '{"name":"Alice","age":30}' | jq -r '.name' # Alice Nested Objects 1 2 echo '{"user":{"name":"Alice","email":"alice@example.com"}}' | jq '.user.name' # "Alice" Arrays 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Get element by index echo '[1,2,3,4,5]' | jq '.[0]' # 1 # Get slice echo '[1,2,3,4,5]' | jq '.[1:3]' # [2, 3] # Get all elements echo '[{"name":"Alice"},{"name":"Bob"}]' | jq '.[].name' # "Alice" # "Bob" # Wrap in array echo '[{"name":"Alice"},{"name":"Bob"}]' | jq '[.[].name]' # ["Alice", "Bob"] Filtering 1 2 3 4 5 6 # Select objects matching condition echo '[{"name":"Alice","age":30},{"name":"Bob","age":25}]' | jq '.[] | select(.age > 26)' # {"name": "Alice", "age": 30} # Multiple conditions jq '.[] | select(.status == "active" and .role == "admin")' Transform Data 1 2 3 4 5 6 7 8 9 10 11 # Create new object echo '{"first":"Alice","last":"Smith"}' | jq '{fullName: "\(.first) \(.last)"}' # {"fullName": "Alice Smith"} # Map over array echo '[1,2,3]' | jq 'map(. * 2)' # [2, 4, 6] # Add field echo '{"name":"Alice"}' | jq '. + {role: "admin"}' # {"name": "Alice", "role": "admin"} Real-World Examples Parse API response: ...

Git hooks are scripts that run automatically at specific points in the Git workflow. They’re perfect for enforcing standards, running tests, and automating tedious tasks—all before code leaves your machine. Hook Locations Hooks live in .git/hooks/. Git creates sample files on git init: 1 2 3 4 5 6 ls .git/hooks/ # applypatch-msg.sample pre-commit.sample # commit-msg.sample pre-push.sample # post-update.sample pre-rebase.sample # pre-applypatch.sample prepare-commit-msg.sample # pre-merge-commit.sample update.sample Remove .sample to activate. Hooks must be executable: ...

Nginx powers a significant portion of the internet, yet its configuration syntax trips up even experienced engineers. Here’s a practical guide to the patterns you’ll actually use. Basic Structure 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # /etc/nginx/nginx.conf user nginx; worker_processes auto; error_log /var/log/nginx/error.log; events { worker_connections 1024; } http { include /etc/nginx/mime.types; default_type application/octet-stream; # Include site configs include /etc/nginx/conf.d/*.conf; } Site configs go in /etc/nginx/conf.d/ or /etc/nginx/sites-enabled/. ...

Cron has been scheduling Unix tasks since 1975. It’s simple, reliable, and will silently fail in ways that waste hours of debugging. Here’s how to use it properly. Cron Syntax │ │ │ │ │ └ ─ │ │ │ │ └ ─ ─ ─ │ │ │ └ ─ ─ ─ ─ ─ │ │ └ ─ ─ ─ ─ ─ ─ ─ │ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ c ─ ─ ─ ─ ─ o m D M D H M m a o a o i a y n y u n n t r u d o h o t f f ( e ( 0 w 1 m - ( e - o 2 0 e 1 n 3 - k 2 t ) 5 ) h 9 ( ) 0 ( - 1 7 - , 3 1 0 ) a n d 7 a r e S u n d a y ) Common Schedules 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # Every minute * * * * * /script.sh # Every 5 minutes */5 * * * * /script.sh # Every hour at minute 0 0 * * * * /script.sh # Daily at 3 AM 0 3 * * * /script.sh # Weekly on Sunday at midnight 0 0 * * 0 /script.sh # Monthly on the 1st at 6 AM 0 6 1 * * /script.sh # Every weekday at 9 AM 0 9 * * 1-5 /script.sh # Every 15 minutes during business hours */15 9-17 * * 1-5 /script.sh The Silent Failure Problem Cron’s default behavior: run command, discard output, send errors to email (which you probably haven’t configured). ...

Environment variables seem trivial—just key-value pairs. Then you have 50 of them across 4 environments with secrets mixed in, and suddenly you’re in configuration hell. Here’s how to stay sane. The Hierarchy Configuration should flow from least to most specific: D e f a u l t s ( c o d e ) → C o n f i g f i l e s → E n v v a r s → C o m m a n d - l i n e f l a g s Each layer overrides the previous. Environment variables sit near the top—easy to change per environment without touching code. ...

Make gets overlooked because people think it’s for compiling C code. In reality, it’s a universal task runner with one killer feature: it only runs what needs to run. Basic Structure 1 2 target: dependencies command That’s it. The target is what you’re building, dependencies are what it needs, and commands run to create it. Important: Commands must be indented with a tab, not spaces. Simple Task Runner 1 2 3 4 5 6 7 8 9 10 11 12 13 .PHONY: build test deploy clean build: npm run build test: npm test deploy: build test rsync -avz dist/ server:/var/www/ clean: rm -rf dist/ node_modules/ .PHONY tells Make these aren’t real files—just task names. ...

Every time you type ssh -i ~/.ssh/mykey.pem -p 2222 admin@192.168.1.50, you’re wasting keystrokes. The SSH config file exists to eliminate this friction. Basic Host Aliases Create or edit ~/.ssh/config: H H o o s s t t H U I H U I p o s d s o s d r s e e t s e e o t r n a t r n d N t g N t a d i i a d i m e t n m e t e p y g e p y l F l F 2 o i 2 o i 0 y l 0 y l 3 e 3 e . . 0 ~ 0 ~ . / . / 1 . 1 . 1 s 1 s 3 s 3 s . h . h 5 / 5 / 0 p 1 s r t o a d g _ i k n e g y _ k e y Now just: ...