GitOps Repository - Flux Operator with ResourceSets

This repository contains a multi-environment GitOps configuration using Flux Operator with ResourceSets and static ResourceSetProviders.

Quick Start

🚀 New to this repository? Start with the Bootstrap Guide for complete setup instructions.

TL;DR - Bootstrap in 3 Steps

# 1. Install Flux Operator
helm repo add flux-operator https://controlplaneio-fluxcd.github.io/charts
helm upgrade -i flux-operator flux-operator/flux-operator \
  --namespace flux-operator-system --create-namespace

# 2. Create Git secret (for private repos)
kubectl create namespace flux-system
kubectl create secret generic flux-system \
  --namespace=flux-system \
  --from-literal=username=git \
  --from-literal=password=<YOUR_GITHUB_TOKEN>

# 3. Bootstrap your environment
kubectl apply -k bootstrap/dev/    # or staging, or production

See BOOTSTRAP.md for detailed instructions.

Validate Before Deploying

Test your configuration before deploying:

python3 scripts/validate.py

This validates YAML syntax, kustomizations, label selectors, and template variables. See TESTING.md for details.

Repository Structure

.
├── bootstrap/
│   ├── dev/
│   │   ├── fluxinstance.yaml        # FluxInstance for dev cluster
│   │   ├── kustomization.yaml       # Bootstrap kustomization
│   │   └── namespace.yaml
│   ├── staging/
│   │   └── ...                      # FluxInstance for staging cluster
│   └── production/
│       └── ...                      # FluxInstance for production cluster
├── apps/
│   ├── cert-manager/
│   │   └── resourceset.yaml         # ResourceSet definition for cert-manager
│   ├── external-dns/
│   │   └── resourceset.yaml         # ResourceSet definition for external-dns
│   └── kube-prometheus-stack/
│       └── resourceset.yaml         # ResourceSet definition for kube-prometheus-stack
├── environments/
│   ├── dev/
│   │   ├── cert-manager-inputs.yaml
│   │   ├── external-dns-inputs.yaml
│   │   ├── kube-prometheus-stack-inputs.yaml
│   │   └── cluster-config.yaml      # ConfigMap for post-build substitution
│   ├── staging/
│   │   └── ...
│   └── production/
│       └── ...
├── scripts/
│   └── validate.py                   # Automated validation script
├── BOOTSTRAP.md                      # Detailed bootstrap guide
├── TESTING.md                        # Testing and validation guide
└── README.md

Architecture

ResourceSets

Each application (cert-manager, external-dns, kube-prometheus-stack) has its own ResourceSet definition in the apps/ directory. ResourceSets define the Helm releases and related resources using templating syntax << inputs.fieldname >>.

ResourceSetInputProviders

Each environment (dev, staging, production) has separate ResourceSetInputProvider files for each application. These providers use the Static type and contain environment-specific values such as:

• Chart versions
• Replica counts
• Resource limits
• Ingress hostnames
• Storage sizes
• Configuration parameters

Post-Build Substitution

Each environment includes a cluster-config.yaml ConfigMap that can be used for post-build substitution via postBuild.substituteFrom. This allows injecting environment metadata without modifying the base templates.

Prerequisites

1. Kubernetes cluster(s) (one per environment or multi-cluster setup)
2. kubectl configured with access to your cluster(s)
3. GitHub Personal Access Token (for private repos) or GitHub App credentials

Installation

For complete installation and bootstrap instructions, see BOOTSTRAP.md.

The bootstrap process will:

1. Install Flux Operator via Helm
2. Create Git authentication secrets
3. Apply the FluxInstance resource for your environment
4. Automatically sync and deploy all applications from this repository

Manual Installation (Alternative)

If you prefer manual installation without FluxInstance, you can apply resources directly:

# Apply all ResourceSet definitions
kubectl apply -k apps/

# Apply environment-specific configurations
kubectl apply -k environments/dev/     # or staging, or production

Note: The FluxInstance bootstrap method (recommended) provides automatic GitOps sync, while manual installation requires you to apply updates manually.

Customization

Adding a New Application

1. Create a new directory under apps/:

   mkdir apps/my-app

2. Create a resourceset.yaml file with your Helm release definition:

   apiVersion: fluxcd.controlplane.io/v1
   kind: ResourceSet
   metadata:
     name: my-app
     namespace: flux-system
   spec:
     inputsFrom:
       - matchLabels:
           app: my-app
     resources:
       # Your Helm release and other resources

3. Create ResourceSetInputProvider files for each environment:

   # environments/dev/my-app-inputs.yaml
   apiVersion: fluxcd.controlplane.io/v1
   kind: ResourceSetInputProvider
   metadata:
     name: my-app-dev
     namespace: flux-system
     labels:
       app: my-app
       environment: dev
   spec:
     type: Static
     defaultValues:
       # Your environment-specific values

Modifying Environment Values

Edit the respective *-inputs.yaml files in the environment directories:

# Edit dev cert-manager configuration
vi environments/dev/cert-manager-inputs.yaml

# Edit production kube-prometheus-stack configuration
vi environments/production/kube-prometheus-stack-inputs.yaml

Using Post-Build Substitution

The cluster-config.yaml ConfigMap in each environment can be used to inject additional variables. Values in this ConfigMap can be referenced in your HelmRelease values using ${variable_name} syntax.

Environment Differences

Setting	Dev	Staging	Production
cert-manager replicas	1	2	3
cert-manager resources	100m/128Mi	200m/256Mi	500m/512Mi
Prometheus replicas	retention	7d	15d
Prometheus storage	10Gi	50Gi	100Gi
Domain	*.dev.example.com	*.staging.example.com	*.example.com

Monitoring Deployment

Check the status of your ResourceSets and HelmReleases:

# Check ResourceSets
kubectl get resourcesets -n flux-system

# Check ResourceSetInputProviders
kubectl get resourcesetinputproviders -n flux-system

# Check HelmReleases across namespaces
kubectl get helmreleases -A

# Check specific application
kubectl get helmrelease -n monitoring kube-prometheus-stack

Troubleshooting

ResourceSet not creating resources

1. Check if ResourceSetInputProvider exists and has correct labels:

   kubectl get resourcesetinputproviders -n flux-system --show-labels

2. Check ResourceSet status:

   kubectl describe resourceset <name> -n flux-system

HelmRelease failing

Check the HelmRelease status and events:

kubectl describe helmrelease <name> -n <namespace>

Template syntax errors

Ensure you're using the correct template syntax: << inputs.fieldname >>

Security Considerations

1. Passwords: Change default passwords in production (especially Grafana admin password)
2. Secrets: Consider using sealed-secrets or external-secrets-operator for sensitive data
3. RBAC: Implement proper RBAC policies for flux-system namespace
4. Network Policies: Add network policies to restrict pod-to-pod communication

Additional Resources

• Flux Operator Documentation
• ResourceSet Definition Guide
• Helm Controller Documentation