Git isn’t just for code anymore. In a GitOps workflow, your entire infrastructure lives in version control, and changes happen through pull requests, not SSH sessions.

The principle is simple: the desired state of your system is declared in Git, and automated processes continuously reconcile actual state with desired state. No more “just SSH in and fix it.” No more tribal knowledge about what’s running where.

The Core Loop

GitOps operates on a continuous reconciliation loop:

┌│││││││││││└─────┌│││└───────────R────Ge────ip─│││└───to───────────────────┐│││┘────◀─────────────▶────┌│││└R─────e─────c────A(A─o────gFr─n────elg─│c│┴───nuo─i────txC─l────/D─e────)────┐│││┘────◀─────────────▶────┌│││└──────────────I─────n(─────fKC─────rul─O────abo─b────seu─s────trd─e────rn)─│r│┘───ue─v───ct─e───te────us────r/────e───────┐│││┘────│││││││││─┐││┘
  1. Declare desired state in Git (manifests, configs, policies)
  2. Detect drift between desired and actual state
  3. Reconcile by applying changes to match Git
  4. Repeat continuously

Repository Structure

A well-organized GitOps repo separates concerns:

i├│││├│││││││││├│││└n────f────rab├├└e├││├││└a├├└c├└sa───n───p───l──ts───v───p───u──reissu/nnrrd├└s├└p├└/auptuecaeboe──t──r──psaesutmtanv──a──o──ieyr--uewcm/gd-rmsewrso/ekpikpukpg-e/aeeprnuanuacuaasnss/aktstgsttsttetttc-stc/tcitcer---ep/ohohoohwvs11smemenmeaie//lisisisycriz/zz//ecaaa/iitttceiiiesooo//nnn...yyyaaammm#lllSharedresources

Base configurations are shared. Environment-specific patches override only what differs. This eliminates copy-paste drift between environments.

Kustomize for Environment Variants

Kustomize lets you layer configurations without templating:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# base/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-service
spec:
  replicas: 1
  template:
    spec:
      containers:
        - name: api
          image: api-service:latest
          resources:
            requests:
              memory: "128Mi"
              cpu: "100m"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

# environments/production/patches/api-service-patch.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-service
spec:
  replicas: 5
  template:
    spec:
      containers:
        - name: api
          resources:
            requests:
              memory: "512Mi"
              cpu: "500m"
            limits:
              memory: "1Gi"
              cpu: "1000m"

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# environments/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
  - ../../base
patches:
  - path: patches/api-service-patch.yaml
images:
  - name: api-service
    newTag: v2.3.1

Production gets 5 replicas with more resources. Dev gets 1 replica with minimal resources. Same base, different overlays.

Flux: The GitOps Operator

Flux watches your Git repo and applies changes automatically:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# clusters/production/flux-system/gotk-sync.yaml
apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: infrastructure
  namespace: flux-system
spec:
  interval: 1m
  url: https://github.com/org/infrastructure
  ref:
    branch: main
  secretRef:
    name: github-credentials
---
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 10m
  sourceRef:
    kind: GitRepository
    name: infrastructure
  path: ./environments/production
  prune: true
  healthChecks:
    - apiVersion: apps/v1
      kind: Deployment
      name: api-service
      namespace: default

Every minute, Flux checks Git. Every 10 minutes, it reconciles state. If someone manually edits a resource in the cluster, Flux reverts it to match Git.

Pull Request Workflow

All changes flow through PRs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

# .github/workflows/validate.yaml
name: Validate Infrastructure
on:
  pull_request:
    paths:
      - 'infrastructure/**'

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Validate Kubernetes manifests
        uses: instrumenta/kubeval-action@master
        with:
          files: infrastructure/
      
      - name: Check Kustomize builds
        run: |
          for env in dev staging production; do
            echo "Building $env..."
            kustomize build infrastructure/environments/$env
          done
      
      - name: Policy check (OPA/Gatekeeper)
        run: |
          conftest test infrastructure/ -p policies/
      
      - name: Dry-run diff
        run: |
          kustomize build infrastructure/environments/staging | \
            kubectl diff -f - --server-side

The PR validates syntax, builds each environment, checks policies, and shows what would change. Reviewers see the exact diff before merge.

Image Automation

Flux can automatically update image tags when new versions are pushed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageRepository
metadata:
  name: api-service
  namespace: flux-system
spec:
  image: ghcr.io/org/api-service
  interval: 1m
---
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImagePolicy
metadata:
  name: api-service
  namespace: flux-system
spec:
  imageRepositoryRef:
    name: api-service
  policy:
    semver:
      range: ">=1.0.0"
---
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageUpdateAutomation
metadata:
  name: api-service
  namespace: flux-system
spec:
  interval: 1m
  sourceRef:
    kind: GitRepository
    name: infrastructure
  git:
    checkout:
      ref:
        branch: main
    commit:
      author:
        email: flux@company.com
        name: Flux
      messageTemplate: 'chore: update {{.AutomatedResource.Name}} to {{.NewImage}}'
    push:
      branch: main
  update:
    path: ./infrastructure/environments
    strategy: Setters

New image pushed → Flux detects it → Flux commits the tag update to Git → Flux applies the change. Full audit trail, zero manual intervention.

Secrets Management

Secrets don’t belong in Git unencrypted. Options:

Sealed Secrets (encryption at rest):

1
2
3
4
5

# Encrypt secret
kubeseal --format yaml < secret.yaml > sealed-secret.yaml

# Commit sealed-secret.yaml to Git
# Controller in cluster decrypts on apply

SOPS (inline encryption):

1
2
3
4
5
6
7

# Encrypted values inline, decrypted by Flux
apiVersion: v1
kind: Secret
metadata:
  name: api-credentials
stringData:
  password: ENC[AES256_GCM,data:abc123...,type:str]

External Secrets Operator (reference external stores):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: api-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault
    kind: ClusterSecretStore
  target:
    name: api-credentials
  data:
    - secretKey: password
      remoteRef:
        key: secret/api
        property: password

The secret reference lives in Git. The actual secret lives in Vault/AWS Secrets Manager/etc.

Rollback Strategy

Git history is your rollback mechanism:

1
2
3
4
5
6
7
8

# Find the last working commit
git log --oneline infrastructure/

# Revert the problematic change
git revert abc123

# Push - Flux applies automatically
git push

Or if you need immediate rollback:

1
2
3
4
5
6
7
8
9

# Suspend Flux reconciliation
flux suspend kustomization apps

# Manually rollback
kubectl rollout undo deployment/api-service

# Fix the issue in Git
# Resume reconciliation
flux resume kustomization apps

The suspension prevents Flux from reverting your manual fix while you prepare the proper Git commit.

Multi-Cluster Patterns

For multiple clusters, structure repos to share configurations while allowing cluster-specific overrides:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

# clusters/us-east-1/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - ../../infrastructure/environments/production
patches:
  - target:
      kind: Deployment
    patch: |
      - op: add
        path: /spec/template/spec/nodeSelector
        value:
          topology.kubernetes.io/region: us-east-1

Each cluster pulls from the same production environment but applies cluster-specific patches (node selectors, ingress configs, regional endpoints).

Anti-Patterns to Avoid

Manual cluster edits: If it’s not in Git, it will be reverted. Train the team: Git is the only way to change production.

Monolithic repos: Separate app configs from platform configs. Different change velocity, different reviewers.

No environment promotion: Don’t push directly to production. PR to dev → test → promote to staging → promote to production.

Ignoring drift alerts: When Flux reports drift, investigate. Someone is bypassing the process.

Getting Started

  1. Start small: One app, one cluster, one environment
  2. Establish PR discipline: No merges without review, no exceptions
  3. Automate validation: Every PR should be validated before human review
  4. Monitor reconciliation: Alert when Flux can’t reconcile (means something is blocking deployment)
  5. Document the process: New team members should be able to deploy on day one

GitOps isn’t about the tools. It’s about the discipline: everything in Git, changes through PRs, automated reconciliation. The tools (Flux, ArgoCD, Kustomize) are enablers. The culture shift is the hard part.

Once you’ve lived with GitOps for a few months, SSH-ing into servers to make changes feels like barbarism. The audit trail, the reviewability, the automatic rollback — you’ll wonder how you ever operated without it.