GitHub Actions to ArgoCD: a GitOps pipeline for an AI microservice
Deploying a standard web API is straightforward. Deploying an AI microservice that loads a 1.2GB sentence-transformer model into memory, requires CUDA drivers, and demands strict GPU memory allocation is a different beast.
When I first shipped our embedding generation service, I used a traditional push-based deployment model: GitHub Actions would build our Docker image, push it to our registry, and then run kubectl set image deployment/embedding-service directly against our production cluster.
It worked—until it didn’t.
One Tuesday afternoon, a developer changed a Hugging Face cache directory variable in a ConfigMap manually to troubleshoot a model-loading issue. Ten minutes later, a routine CI/CD run executed. The deployment completed successfully from GitHub Actions’ perspective, but the pod fell into a permanent CrashLoopBackOff because the live ConfigMap was out of sync with the environment variables the new Docker image expected. Our deployment pipeline was completely blind to the live cluster state.
That failure drove us to migrate to a declarative GitOps model using GitHub Actions and ArgoCD. Here is the architecture, the code, and the lessons learned from building a production-grade CI/CD pipeline for our AI microservices.
The Architecture
The core philosophy of GitOps is simple: the git repository is the single source of truth for the desired state of your infrastructure. Instead of pushing changes to Kubernetes, we declare our desired state in a GitOps repository, and a controller inside our cluster pulls those changes to reconcile the differences.
flowchart LR dev["Developer Push"] --> gha["GitHub Actions"] gha -->|"Builds Image"| ghcr["GitHub Container Registry"] gha -->|"Updates Yaml"| gitops["GitOps Config Repository"] argocd["ArgoCD Controller"] -->|"Polls Config"| gitops argocd -->|"Applies State"| k8s["Kubernetes Cluster"]
This separation of concerns solved our observability and synchronization problems:
1. The Application Repo contains the Python code, the Dockerfile, and the GitHub Actions CI workflow.
2. The GitOps Config Repo contains the Kubernetes manifests (Kustomize/Helm) representing our environments (staging, production).
3. ArgoCD runs inside our Kubernetes cluster, constantly comparing the active cluster state with the GitOps repository.
The AI Microservice: FastAPI + SentenceTransformers
To understand the pipeline, we must first look at the microservice we are deploying. This FastAPI application exposes a single endpoint to generate text embeddings using all-MiniLM-L6-v2.
Here is the app.py implementation. It includes a custom startup handler to pre-download the model to a local cache folder so that the container does not attempt to download model weights from Hugging Face at runtime—a critical requirement for air-gapped or stable Kubernetes environments.
import os
import logging
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from sentence_transformers import SentenceTransformer
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(“embedding-service”)
# Ensure we use a local cache directory for model weights
MODEL_CACHE_DIR = os.getenv(“HF_HOME”, “/app/model_cache”)
MODEL_NAME = os.getenv(“MODEL_NAME”, “all-MiniLM-L6-v2”)
app = FastAPI(title=“AI Embedding Service”)
# Global variable to hold the warm model
model = None
class EmbedRequest(BaseModel):
text: str
class EmbedResponse(BaseModel):
embedding: list[float]
@app.on_event(“startup”)
def load_model():
global model
logger.info(f”Loading model ‘{MODEL_NAME}’ from cache directory ‘{MODEL_CACHE_DIR}’…”)
try:
# Load model; this will use local files if already downloaded during Docker build
model = SentenceTransformer(MODEL_NAME, cache_folder=MODEL_CACHE_DIR)
logger.info(“Model loaded successfully.”)
except Exception as e:
logger.error(f”Failed to load model on startup: {str(e)}”)
raise e
@app.get(“/healthz”)
def healthz():
if model is None:
raise HTTPException(status_code=503, detail=“Model not initialized”)
return {“status”: “healthy”, “model”: MODEL_NAME}
@app.post(“/embed”, response_model=EmbedResponse)
def get_embedding(payload: EmbedRequest):
if not payload.text.strip():
raise HTTPException(status_code=400, detail=“Text payload cannot be empty”)
try:
embedding = model.encode(payload.text).tolist()
return EmbedResponse(embedding=embedding)
except Exception as e:
logger.error(f”Inference error: {str(e)}”)
raise HTTPException(status_code=500, detail=“Failed to run inference”)
Next, we write the Dockerfile. Notice how we run a build-time python script to download the Hugging Face model directly into the image layers. This avoids high cold-start latencies when our Kubernetes pods scale horizontally.
FROM python:3.10-slim
WORKDIR /app
# Install system dependencies
RUN apt-get update && apt-get install -y –no-install-recommends \
build-essential \
curl \
&& rm -rf /var/lib/apt/lists/*
# Copy and install Python requirements
COPY requirements.txt .
RUN pip install –no-cache-dir -r requirements.txt
# Create cache directory and pre-download model weights during build time
ENV HF_HOME=/app/model_cache
ENV MODEL_NAME=all-MiniLM-L6-v2
RUN python -c “from sentence_transformers import SentenceTransformer; SentenceTransformer(‘$MODEL_NAME‘, cache_folder=’$HF_HOME‘)”
# Ensure directory permissions are correct for non-root execution
RUN chmod -R 755 /app/model_cache
COPY app.py .
EXPOSE 8000
CMD [“uvicorn”, “app:app”, “–host”, “0.0.0.0”, “–port”, “8000”]
The accompanying requirements.txt locks down the dependencies:
uvicorn==0.28.0
sentence-transformers==2.5.1
torch==2.2.1 –extra-index-url https://download.pytorch.org/whl/cpu
pydantic==2.6.4
The GitHub Actions CI Pipeline
Our GitHub Actions workflow handles standard CI checks: linting, testing, building the multi-arch container image, and pushing it to the GitHub Container Registry (GHCR).
The critical step occurs at the end of the workflow: it checks out our separate GitOps repository, updates the image tag inside our Kustomize deployment configuration file using yq, and commits the change back to git. This commit is what triggers ArgoCD.
name: Continuous Integration & Deployment Trigger
on:
push:
branches:
– main
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository_owner }}/embedding-service
GITOPS_REPO: ${{ github.repository_owner }}/gitops-manifests
jobs:
build-and-push:
runs-on: ubuntu-latest
permissions:
contents: write
packages: write
steps:
– name: Checkout Application Code
uses: actions/checkout@v4
– name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
– name: Log in to the Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
– name: Extract Metadata (Tags, Labels)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=sha,format=long
type=ref,event=branch
– name: Build and Push Docker Image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
# Checkout the separate GitOps configuration repository
– name: Checkout GitOps Repository
uses: actions/checkout@v4
with:
repository: ${{ env.GITOPS_REPO }}
token: ${{ secrets.GITOPS_PAT }}
path: gitops-repo
# Modify the container image tag in our deployment manifest
– name: Update Kubernetes Manifest Image Tag
run: |
cd gitops-repo/apps/embedding-service/overlays/production
IMAGE_TAG=”sha-${{ github.sha }}”
FULL_IMAGE_NAME=”${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}”
# Set the new image version in kustomization.yaml
yq eval -i “.images[0].newName = \”$FULL_IMAGE_NAME\”” kustomization.yaml
yq eval -i “.images[0].newTag = \”$IMAGE_TAG\”” kustomization.yaml
cat kustomization.yaml
# Commit and push changes back to the GitOps Repository
– name: Commit and Push Changes to GitOps Repo
run: |
cd gitops-repo
git config –global user.name “github-actions[bot]”
git config –global user.email “github-actions[bot]@users.noreply.github.com”
git add apps/embedding-service/overlays/production/kustomization.yaml
# Only commit and push if changes exist
if ! git diff-index –quiet HEAD; then
git commit -m “chore(release): deploy embedding-service tag sha-${{ github.sha }}”
git push origin main
else
echo “No image tag changes detected. Skipping release commit.”
fi
The GitOps Manifests
In our separate gitops-manifests repository, we structure our deployment using Kustomize to handle different environment overlays cleanly.
Repository Layout
└── apps/
└── embedding-service/
├── base/
│ ├── deployment.yaml
│ ├── service.yaml
│ └── kustomization.yaml
└── overlays/
└── production/
└── kustomization.yaml
Base Manifests
The base/deployment.yaml defines our application structure. It includes resources, port mapping, and our liveness and readiness probes pointing to the FastAPI /healthz endpoint.
apiVersion: apps/v1
kind: Deployment
metadata:
name: embedding-service
labels:
app: embedding-service
spec:
replicas: 2
selector:
matchLabels:
app: embedding-service
template:
metadata:
labels:
app: embedding-service
spec:
containers:
– name: embedding-service
image: ghcr.io/my-org/embedding-service:latest
ports:
– containerPort: 8000
name: http
resources:
requests:
cpu: “1”
memory: “2Gi”
limits:
cpu: “2”
memory: “4Gi”
env:
– name: HF_HOME
value: “/tmp/hf_cache”
– name: MODEL_NAME
value: “all-MiniLM-L6-v2”
livenessProbe:
httpGet:
path: /healthz
port: http
initialDelaySeconds: 15
periodSeconds: 10
readinessProbe:
httpGet:
path: /healthz
port: http
initialDelaySeconds: 20
periodSeconds: 10
The base/service.yaml handles load balancing within the cluster:
apiVersion: v1
kind: Service
metadata:
name: embedding-service
spec:
selector:
app: embedding-service
ports:
– protocol: TCP
port: 80
targetPort: 8000
The base/kustomization.yaml binds them together:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
– deployment.yaml
– service.yaml
Production Overlay
The production overlay overrides parameters and defines where GitHub Actions writes down the updated image tags.
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
– ../../base
# This section is modified dynamically by the GitHub Actions workflow
images:
– name: ghcr.io/my-org/embedding-service
newName: ghcr.io/my-org/embedding-service
newTag: sha-c8d4512b9045b85ee4be775cd440938f3281aa01
ArgoCD Application Definition
The final piece of the pipeline is configuring ArgoCD to watch our GitOps repository and apply the production overlay manifest to our target Kubernetes cluster.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: embedding-service-production
namespace: argocd
spec:
project: default
source:
repoURL: ‘https://github.com/my-org/gitops-manifests.git’
targetRevision: HEAD
path: apps/embedding-service/overlays/production
destination:
server: ‘https://kubernetes.default.svc’
namespace: ai-services
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
– CreateNamespace=true
– ApplyOutOfSyncOnly=true
To apply this setup, run:
Within seconds, the ArgoCD controller will detect the application config, pull the production overlay from the GitOps repo, evaluate the final Kustomize file, and reconcile the target state.
Name: argocd/embedding-service-production
Project: default
Server: https://kubernetes.default.svc
Namespace: ai-services
Status: Synced
Health Status: Healthy
Path: apps/embedding-service/overlays/production
Repo: https://github.com/my-org/gitops-manifests.git
Target: HEAD
RESOURCES:
Group Kind Namespace Name Status
Service ai-services embedding-service Synced
apps Deployment ai-services embedding-service Synced
The Dead Ends I Hit
1. The ArgoCD Image Updater Dynamic Sync Loop
Originally, I avoided the complex GitHub-to-GitOps commit stage by utilizing the popular extension ArgoCD Image Updater. It watches your container registry and applies changes to the cluster when new tags are pushed.
This approach fell apart under real-world usage. Because the image registry tags changed frequently, ArgoCD fell into an infinite state-checking loop. Out-of-sync loops occurred when we updated dynamic variables in our base deployment manifests. The cluster’s live state fluctuated, causing ArgoCD to roll back the container tag continuously to match the git state because the Dynamic Image Updater had not committed the new state back to Git. Writing the state changes directly back to our GitOps repository via GitHub Actions solved this problem definitively.
2. Startup Probes vs Readiness Probes for LLM/Embeddings Models
Initially, I set standard readiness probes on our embedding service pod:
httpGet:
path: /healthz
port: 8000
initialDelaySeconds: 10
periodSeconds: 5
During scaling operations, our microservice took upwards of 45 seconds to download/verify weights and load them onto the CPU/GPU cache. Because the readiness probe kept polling and failing during this warmup window, Kubernetes marked the pod as unhealthy and restarted it. This created an endless restart loop.
To resolve this, we configured a startupProbe that handles the warm-up cycle, allowing the readinessProbe to run strictly when the model initialization completes.
httpGet:
path: /healthz
port: 8000
failureThreshold: 30
periodSeconds: 5
Results and Takeaways
This pipeline brought order to our AI microservice deployments:
- Predictable Deployments: Cluster state drifted only when explicitly changed in Git. Manual edits to staging deployments were instantly reconciled back to the target config state.
- Instant Rollbacks: Rolling back a failed deployment became as simple as executing
git reverton the GitOps repository. ArgoCD processed the state changes and reverted the container image tag and config map changes in less than 5 seconds. - Decoupled Architecture: Our developers don’t need direct cluster access (
kubectl) to ship code. GitHub Actions handles the heavy building and updates the declarative state configuration, while ArgoCD reconciles the cluster.