If you’re running services on Linux, you’re almost certainly using systemd. But there’s a gap between knowing systemctl start nginx and actually writing your own robust service units. Let’s close that gap.

The Anatomy of a Service Unit

A systemd service unit lives in /etc/systemd/system/ and has three main sections:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

[Unit]
Description=My Application Service
After=network.target
Wants=network-online.target

[Service]
Type=simple
User=appuser
Group=appgroup
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/bin/server
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Let’s break down what matters:

[Unit] defines dependencies and metadata:

  • After= ensures services start in order
  • Wants= creates soft dependencies (service still starts if dependency fails)
  • Requires= creates hard dependencies (use sparingly)

[Service] defines the actual service behavior:

  • Type=simple is most common — the process stays in foreground
  • Type=forking for traditional daemons that fork to background
  • Type=oneshot for scripts that run once and exit

[Install] defines how the service integrates with system targets (runlevels).

A Real-World Example: Node.js Application

Here’s a production-ready service for a Node.js app:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

[Unit]
Description=Node.js API Server
Documentation=https://github.com/yourorg/api
After=network-online.target postgresql.service redis.service
Wants=network-online.target
Requires=postgresql.service

[Service]
Type=simple
User=nodeapp
Group=nodeapp
WorkingDirectory=/opt/api

# Environment
Environment=NODE_ENV=production
Environment=PORT=3000
EnvironmentFile=-/opt/api/.env

# Process management
ExecStart=/usr/bin/node /opt/api/dist/server.js
ExecReload=/bin/kill -HUP $MAINPID

# Restart behavior
Restart=always
RestartSec=10
StartLimitInterval=60
StartLimitBurst=3

# Security hardening
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/opt/api/data /var/log/api

# Resource limits
MemoryMax=512M
CPUQuota=80%

# Logging
StandardOutput=journal
StandardError=journal
SyslogIdentifier=api-server

[Install]
WantedBy=multi-user.target

Notable additions:

EnvironmentFile=- — The - prefix means “don’t fail if file doesn’t exist.” Perfect for optional local config.

StartLimitInterval/StartLimitBurst — Prevents rapid restart loops. After 3 failures in 60 seconds, the service stops trying.

Security hardening — ProtectSystem=strict makes the filesystem read-only except for explicitly allowed paths. This is defense in depth.

Resource limits — Prevent runaway processes from taking down the whole server.

The Service Lifecycle

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# Create or edit your service
sudo vim /etc/systemd/system/myapp.service

# Always reload after changes
sudo systemctl daemon-reload

# Enable to start at boot
sudo systemctl enable myapp

# Start now
sudo systemctl start myapp

# Check status
sudo systemctl status myapp

# View logs
sudo journalctl -u myapp -f

The key insight: daemon-reload doesn’t restart services — it just tells systemd to re-read unit files. You still need to restart the service to apply changes.

Debugging Tips

Service won’t start? Check the logs:

1

journalctl -u myapp -n 50 --no-pager

Need more detail? Enable debug output temporarily:

1
2
3

sudo systemctl edit myapp --force
# Add: [Service]
#      Environment=DEBUG=*

Check dependencies:

1
2

systemctl list-dependencies myapp
systemctl list-dependencies myapp --reverse  # What depends on this?

Verify syntax:

1

systemd-analyze verify /etc/systemd/system/myapp.service

Using systemctl edit

Don’t edit unit files directly for customizations — use drop-in overrides:

1
2

# Opens editor for /etc/systemd/system/myapp.service.d/override.conf
sudo systemctl edit myapp

This creates a drop-in that persists across package updates. Much cleaner than editing the main unit file.

Common Patterns

Run a command before start

1
2
3

[Service]
ExecStartPre=/opt/myapp/bin/migrate
ExecStart=/opt/myapp/bin/server

Graceful shutdown

1
2
3
4

[Service]
ExecStop=/opt/myapp/bin/shutdown --graceful
TimeoutStopSec=30
KillMode=mixed

Multiple instances

Use template units (myapp@.service):

1
2

[Service]
ExecStart=/opt/myapp/bin/server --port %i

Then: systemctl start myapp@3000 and systemctl start myapp@3001

Socket activation

Let systemd handle the socket, start your service on-demand:

1
2
3
4
5
6
7

# myapp.socket
[Socket]
ListenStream=3000
Accept=no

[Install]
WantedBy=sockets.target

Security Checklist

Every production service should consider:

  • User= and Group= — never run as root
  • NoNewPrivileges=true — prevent privilege escalation
  • ProtectSystem=strict — read-only filesystem
  • ProtectHome=true — no access to /home
  • PrivateTmp=true — isolated /tmp
  • CapabilityBoundingSet= — drop unnecessary capabilities

For services that need network binding below 1024:

1
2

AmbientCapabilities=CAP_NET_BIND_SERVICE
CapabilityBoundingSet=CAP_NET_BIND_SERVICE

Wrapping Up

systemd service units are deceptively simple. The basics get you running in minutes, but the depth is there when you need it — security hardening, resource limits, complex dependencies, socket activation.

Start simple. Add complexity as needed. And always, always run systemctl daemon-reload after edits.

Computing Arts is where I explore the craft of building with code and infrastructure. Find me at computingarts.com.