Back to Writing

GitOps with ArgoCD: Automating Kubernetes Deployments the Right Way

David Childs
•
•
DevOps

Implement GitOps principles using ArgoCD for automated, declarative Kubernetes deployments with built-in rollback and drift detection.

GitOps transformed how we deploy applications to Kubernetes. After implementing ArgoCD across multiple production clusters, I've seen deployment failures drop by 80% and recovery time improve from hours to minutes. Here's how to implement GitOps properly.

Understanding GitOps Principles

GitOps isn't just about storing YAML in Git—it's about:

  • Declarative Infrastructure: Everything defined as code
  • Version Control: Git as the source of truth
  • Automated Synchronization: Continuous deployment from Git
  • Drift Detection: Automatic detection of manual changes
  • Easy Rollbacks: Git history enables instant rollbacks

Setting Up ArgoCD

Installation

# Create namespace
kubectl create namespace argocd

# Install ArgoCD
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Expose ArgoCD Server
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'

# Get initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

Production-Ready Configuration

# argocd-server-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-server-config
  namespace: argocd
data:
  url: "https://argocd.example.com"
  dex.config: |
    connectors:
    - type: github
      id: github
      name: GitHub
      config:
        clientID: $dex.github.clientID
        clientSecret: $dex.github.clientSecret
        orgs:
        - name: your-org
          teams:
          - platform-team
  policy.csv: |
    p, role:admin, applications, *, */*, allow
    p, role:admin, clusters, *, *, allow
    p, role:developer, applications, get, */*, allow
    p, role:developer, applications, sync, */*, allow
    g, your-org:platform-team, role:admin

Repository Structure

Organizing Your GitOps Repository

gitops-repo/
├── apps/
│   ├── production/
│   │   ├── app1/
│   │   │   ├── kustomization.yaml
│   │   │   └── values.yaml
│   │   └── app2/
│   ├── staging/
│   └── development/
├── infrastructure/
│   ├── monitoring/
│   ├── ingress/
│   └── cert-manager/
├── clusters/
│   ├── prod-cluster/
│   │   └── apps.yaml
│   └── staging-cluster/
└── charts/
    └── reusable-charts/

Creating Applications

Application Definition

# apps/production/webapp/application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: webapp
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/gitops-repo
    targetRevision: HEAD
    path: apps/production/webapp
    helm:
      valueFiles:
        - values.yaml
      parameters:
        - name: image.tag
          value: v1.2.3
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - Validate=true
      - CreateNamespace=true
      - PrunePropagationPolicy=foreground
    retry:
      limit: 5
      backoff:
        duration: 5s
        factor: 2
        maxDuration: 3m

Using Kustomize

# apps/production/webapp/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: production

resources:
  - ../../base/webapp

patchesStrategicMerge:
  - deployment-patch.yaml

configMapGenerator:
  - name: app-config
    literals:
      - environment=production
      - log_level=info

images:
  - name: webapp
    newName: registry.example.com/webapp
    newTag: v1.2.3

replicas:
  - name: webapp-deployment
    count: 3

Advanced Deployment Patterns

Blue-Green Deployments

# blue-green-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: webapp-rollout
spec:
  replicas: 5
  strategy:
    blueGreen:
      activeService: webapp-active
      previewService: webapp-preview
      autoPromotionEnabled: false
      scaleDownDelaySeconds: 30
      prePromotionAnalysis:
        templates:
        - templateName: success-rate
        args:
        - name: service-name
          value: webapp-preview
  selector:
    matchLabels:
      app: webapp
  template:
    metadata:
      labels:
        app: webapp
    spec:
      containers:
      - name: webapp
        image: webapp:v1.2.3
        ports:
        - containerPort: 8080

Canary Deployments

# canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: webapp-canary
spec:
  replicas: 10
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: {duration: 5m}
      - analysis:
          templates:
          - templateName: success-rate
      - setWeight: 30
      - pause: {duration: 5m}
      - setWeight: 60
      - pause: {duration: 5m}
      - setWeight: 100
      canaryService: webapp-canary
      stableService: webapp-stable
      trafficRouting:
        istio:
          virtualService:
            name: webapp-vsvc
            routes:
            - primary

Multi-Cluster Management

App of Apps Pattern

# clusters/production/apps.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: production-apps
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/gitops-repo
    targetRevision: HEAD
    path: apps/production
  destination:
    server: https://kubernetes.default.svc
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Cluster Secrets Management

# Using Sealed Secrets
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: database-credentials
  namespace: production
spec:
  encryptedData:
    username: AgBvA8Z1M2G5Lz...
    password: AgCdQ9X3K4H6Ny...

Monitoring and Observability

ArgoCD Metrics

# prometheus-scrape-config.yaml
- job_name: argocd-metrics
  static_configs:
  - targets:
    - argocd-metrics:8082
- job_name: argocd-server-metrics
  static_configs:
  - targets:
    - argocd-server-metrics:8083
- job_name: argocd-repo-server-metrics
  static_configs:
  - targets:
    - argocd-repo-server:8084

Grafana Dashboard

{
  "dashboard": {
    "title": "ArgoCD Operations",
    "panels": [
      {
        "title": "Sync Status",
        "targets": [{
          "expr": "sum by (sync_status) (argocd_app_info)"
        }]
      },
      {
        "title": "Application Health",
        "targets": [{
          "expr": "sum by (health_status) (argocd_app_health_total)"
        }]
      },
      {
        "title": "Sync Operations",
        "targets": [{
          "expr": "rate(argocd_app_sync_total[5m])"
        }]
      }
    ]
  }
}

CI/CD Integration

GitHub Actions Workflow

name: Deploy to Production
on:
  push:
    branches: [main]

jobs:
  update-manifest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          repository: your-org/gitops-repo
          token: ${{ secrets.GITHUB_TOKEN }}
      
      - name: Update image tag
        run: |
          cd apps/production/webapp
          sed -i "s|image: webapp:.*|image: webapp:${{ github.sha }}|" deployment.yaml
          
      - name: Commit and push
        run: |
          git config user.name "GitHub Actions"
          git config user.email "actions@github.com"
          git add .
          git commit -m "Update webapp to ${{ github.sha }}"
          git push

Sync via CLI

# Install ArgoCD CLI
brew install argocd

# Login
argocd login argocd.example.com

# Create application
argocd app create webapp \
  --repo https://github.com/your-org/gitops-repo \
  --path apps/production/webapp \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace production

# Sync application
argocd app sync webapp

# Watch sync status
argocd app wait webapp --health

Handling Secrets

External Secrets Operator

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: app-secrets
spec:
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: app-secrets
  data:
    - secretKey: api-key
      remoteRef:
        key: secret/data/app
        property: api_key
    - secretKey: db-password
      remoteRef:
        key: secret/data/database
        property: password

SOPS Integration

# .sops.yaml
creation_rules:
  - path_regex: .*\.enc\.yaml$
    kms: arn:aws:kms:us-east-1:account:key/id
    
# Encrypted secret
apiVersion: v1
kind: Secret
metadata:
  name: app-secret
data:
  password: ENC[AES256_GCM,data:...,type:str]
sops:
  kms:
    - arn: arn:aws:kms:us-east-1:account:key/id

Disaster Recovery

Backup Strategy

# Backup ArgoCD configuration
kubectl get all -n argocd -o yaml > argocd-backup.yaml
kubectl get applications -n argocd -o yaml > applications-backup.yaml
kubectl get appprojects -n argocd -o yaml > projects-backup.yaml

# Automated backup CronJob
apiVersion: batch/v1
kind: CronJob
metadata:
  name: argocd-backup
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: bitnami/kubectl:latest
            command:
            - /bin/sh
            - -c
            - |
              kubectl get applications -n argocd -o yaml > /backup/apps-$(date +%Y%m%d).yaml
              kubectl get appprojects -n argocd -o yaml > /backup/projects-$(date +%Y%m%d).yaml

Rollback Procedures

# Rollback using Git
git revert <commit-hash>
git push

# ArgoCD will automatically sync

# Or manual rollback
argocd app rollback webapp <revision>

# View history
argocd app history webapp

Best Practices

  1. Separate Config and Code Repos: Keep application code and Kubernetes manifests in different repositories
  2. Use Semantic Versioning: Tag images properly for easy tracking
  3. Implement Progressive Delivery: Use canary or blue-green deployments
  4. Monitor Sync Status: Set up alerts for failed syncs
  5. Regular Backups: Backup ArgoCD configuration and applications
  6. Access Control: Implement RBAC and SSO
  7. Resource Hooks: Use sync waves and hooks for ordered deployments
  8. Prune Carefully: Test prune behavior in staging first

Common Issues and Solutions

Sync Conflicts

# Force sync when manual changes detected
argocd app sync webapp --force

# Ignore specific differences
spec:
  ignoreDifferences:
  - group: apps
    kind: Deployment
    jsonPointers:
    - /spec/replicas

Large Manifests

# Increase timeout and limits
apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cmd-params-cm
data:
  controller.operation.processors: "50"
  controller.status.processors: "50"
  controller.repo.server.timeout.seconds: "180"

Conclusion

GitOps with ArgoCD brings predictability, auditability, and reliability to Kubernetes deployments. Start small with a single application, establish patterns that work for your team, then gradually expand to your entire infrastructure.

The investment in setting up GitOps pays off through reduced deployment failures, faster recovery times, and improved developer experience. With Git as your source of truth and ArgoCD as your automation engine, you can achieve true continuous deployment with confidence.

Share this article

DC

David Childs

Consulting Systems Engineer with over 10 years of experience building scalable infrastructure and helping organizations optimize their technology stack.

Related Articles