Private Registry Security

Overview

Container registries are the central distribution point for all images in your Kubernetes clusters. Securing your registry infrastructure is critical because:

• A compromised registry can serve malicious images to every cluster that pulls from it
• Without authentication, anyone can push or pull images
• Without access control, developers may deploy untested or unauthorized images to production
• Without vulnerability scanning at the registry level, known CVEs can proliferate across deployments

CKS Exam Relevance

The CKS exam frequently tests your ability to configure ImagePullSecrets for accessing private registries. You should know how to create secrets, attach them to Pods and ServiceAccounts, and troubleshoot authentication failures. Understanding Harbor and registry architecture is also valuable.

Secure Registry Architecture

Configuring ImagePullSecrets

Creating an ImagePullSecret

ImagePullSecrets allow Kubernetes to authenticate with private registries when pulling images.

# Create a docker-registry secret
kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=myuser \
  --docker-password=mypassword \
  --docker-email=myuser@example.com

# Create in a specific namespace
kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=myuser \
  --docker-password=mypassword \
  --namespace=production

Namespace Scope

ImagePullSecrets are namespace-scoped. If your Pod is in the production namespace, the secret must also be in the production namespace. This is a common source of exam errors.

Examining the Secret

# View the secret
kubectl get secret regcred -o yaml

# Decode the docker config
kubectl get secret regcred -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d | jq .

Decoded output:

{
  "auths": {
    "registry.example.com": {
      "username": "myuser",
      "password": "mypassword",
      "email": "myuser@example.com",
      "auth": "bXl1c2VyOm15cGFzc3dvcmQ="
    }
  }
}

Creating from an Existing Docker Config

If you already have Docker credentials configured:

# Create secret from existing docker config
kubectl create secret generic regcred \
  --from-file=.dockerconfigjson=$HOME/.docker/config.json \
  --type=kubernetes.io/dockerconfigjson

Using ImagePullSecrets in a Pod

apiVersion: v1
kind: Pod
metadata:
  name: private-app
  namespace: production
spec:
  containers:
    - name: app
      image: registry.example.com/myapp:v1.0
  imagePullSecrets:
    - name: regcred

Using ImagePullSecrets with a Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: private-app
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: private-app
  template:
    metadata:
      labels:
        app: private-app
    spec:
      containers:
        - name: app
          image: registry.example.com/myapp:v1.0
      imagePullSecrets:
        - name: regcred

Attaching ImagePullSecrets to a ServiceAccount

Instead of specifying imagePullSecrets on every Pod, attach them to the ServiceAccount. All Pods using that ServiceAccount will automatically use the secret.

# Patch the default service account to include the pull secret
kubectl patch serviceaccount default \
  -n production \
  -p '{"imagePullSecrets": [{"name": "regcred"}]}'

Or declaratively:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: production
imagePullSecrets:
  - name: regcred

Exam Efficiency

Attaching the ImagePullSecret to a ServiceAccount is faster than modifying every Pod spec. On the exam, if multiple workloads need registry access, patch the ServiceAccount.

Verifying ImagePullSecret Configuration

# Check if the secret exists in the correct namespace
kubectl get secrets -n production | grep regcred

# Check if the ServiceAccount has the pull secret
kubectl get serviceaccount default -n production -o yaml

# Test by creating a Pod
kubectl run test --image=registry.example.com/myapp:v1.0 -n production

# Check Pod events for pull errors
kubectl describe pod test -n production | grep -A5 Events

Common error messages:

# Missing secret
Failed to pull image "registry.example.com/myapp:v1.0": 
  rpc error: unauthorized: authentication required

# Wrong credentials
Failed to pull image "registry.example.com/myapp:v1.0": 
  rpc error: unauthorized: incorrect username or password

# Wrong namespace
Error from server (NotFound): secrets "regcred" not found

Multiple Registry Authentication

You can configure authentication for multiple registries in a single secret:

# Create a secret with credentials for multiple registries
kubectl create secret docker-registry multi-regcred \
  --docker-server=registry1.example.com \
  --docker-username=user1 \
  --docker-password=pass1

# Or create from a docker config that has multiple entries
cat <<EOF > /tmp/docker-config.json
{
  "auths": {
    "registry1.example.com": {
      "auth": "$(echo -n 'user1:pass1' | base64)"
    },
    "registry2.example.com": {
      "auth": "$(echo -n 'user2:pass2' | base64)"
    },
    "gcr.io": {
      "auth": "$(echo -n '_json_key:$(cat sa-key.json)' | base64)"
    }
  }
}
EOF

kubectl create secret generic multi-regcred \
  --from-file=.dockerconfigjson=/tmp/docker-config.json \
  --type=kubernetes.io/dockerconfigjson

You can also reference multiple ImagePullSecrets:

spec:
  imagePullSecrets:
    - name: regcred-registry1
    - name: regcred-registry2
    - name: regcred-gcr

Harbor Registry

Harbor is an open-source, enterprise-grade container registry that provides security, identity, and management features on top of the standard OCI registry.

Key Harbor Features

Feature	Description	Security Benefit
Vulnerability Scanning	Built-in Trivy/Clair scanning	Detect CVEs before deployment
Role-Based Access Control	Project-level permissions	Least privilege for registry access
Image Signing	Notary/Cosign integration	Verify image integrity
Image Replication	Cross-registry replication	Disaster recovery, geo-distribution
Audit Logging	Track all registry operations	Forensics and compliance
Garbage Collection	Clean up unused images	Reduce storage and attack surface
Robot Accounts	Automated access with scoped permissions	CI/CD service accounts
Tag Immutability	Prevent tag overwrites	Prevent tag mutation attacks
Content Trust	Require signed images	Only deploy verified images

Harbor Access Control Model

Harbor Vulnerability Scanning Policy

Harbor can block image pulls based on vulnerability scan results:

Project Settings:
  - Vulnerability scanning: Enabled
  - Prevent vulnerable images from running:
    - Severity threshold: High
    - Images with High or Critical vulnerabilities are blocked from pulling
  - Automatically scan on push: Enabled

This provides an additional layer of defense beyond Kubernetes admission controllers.

Harbor Robot Accounts

Robot accounts provide scoped, automated access for CI/CD pipelines and Kubernetes clusters:

# In Harbor UI or API, create a robot account:
# Name: robot$k8s-prod-pull
# Permissions: Pull only on project "production"
# Expiry: 365 days

# Use the robot account credentials for ImagePullSecret
kubectl create secret docker-registry harbor-cred \
  --docker-server=harbor.example.com \
  --docker-username='robot$k8s-prod-pull' \
  --docker-password='<robot-account-token>' \
  --namespace=production

Tag Immutability

Prevent tag mutation attacks by enabling tag immutability:

Harbor Project Settings:
  - Tag Immutability: Enabled
  - Rule: **  (all tags are immutable once pushed)

With tag immutability enabled, once myapp:v1.0 is pushed, it cannot be overwritten. An attacker cannot replace a legitimate image by pushing a malicious image with the same tag.

Registry Best Practices

1. Restrict Registries at the Cluster Level

Use admission controllers to only allow images from trusted registries:

# OPA Gatekeeper constraint (see 02-image-policies.md for full setup)
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sAllowedRegistries
metadata:
  name: allowed-registries
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Pod"]
  parameters:
    registries:
      - "harbor.example.com/"

2. Use Dedicated Service Accounts per Namespace

# production namespace - only pull from production project
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: production
imagePullSecrets:
  - name: harbor-prod-cred

---
# staging namespace - pull from staging project
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: staging
imagePullSecrets:
  - name: harbor-staging-cred

3. Rotate Registry Credentials

# Update the secret with new credentials
kubectl create secret docker-registry regcred \
  --docker-server=registry.example.com \
  --docker-username=myuser \
  --docker-password=new-password \
  --dry-run=client -o yaml | kubectl apply -f -

4. Use TLS for Registry Communication

All registry communication should use TLS. Configure containerd or Docker to trust your registry CA:

# /etc/containerd/config.toml (containerd)
[plugins."io.containerd.grpc.v1.cri".registry.configs]
  [plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.example.com".tls]
    ca_file = "/etc/containerd/certs.d/harbor.example.com/ca.crt"
  [plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.example.com".auth]
    username = "robot$k8s-pull"
    password = "token123"

// /etc/docker/daemon.json (Docker)
{
  "insecure-registries": [],
  "registry-mirrors": ["https://mirror.example.com"]
}

Never Use Insecure Registries

Never add your registry to insecure-registries in production. This disables TLS verification and allows man-in-the-middle attacks. Always configure proper TLS certificates.

Pull-Through Cache

A pull-through cache (registry mirror) reduces external network dependencies and provides a local cache of upstream images.

Security Benefits of Pull-Through Cache

Benefit	Description
Reduced exposure	Nodes do not need direct access to external registries
Availability	Cluster continues to work if upstream registry is down
Scanning gate	Cache can scan images before serving them
Audit trail	All image pulls are logged through the cache
Network isolation	Nodes only communicate with internal cache

Configuring Harbor as a Pull-Through Cache

In Harbor, create a proxy cache project:

Harbor UI:
  Projects > New Project
    - Project Name: docker-hub-cache
    - Access Level: Private
    - Proxy Cache: Enabled
    - Registry Endpoint: https://hub.docker.com (pre-configured)

Kubernetes nodes pull through the cache:

# Instead of: image: nginx:1.25
# Use: image: harbor.example.com/docker-hub-cache/library/nginx:1.25
apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
    - name: nginx
      image: harbor.example.com/docker-hub-cache/library/nginx:1.25

Quick Reference: Registry Commands

Task	Command
Create ImagePullSecret	kubectl create secret docker-registry regcred --docker-server=... --docker-username=... --docker-password=...
Create in namespace	Add --namespace=<ns> to the above
View secret contents	kubectl get secret regcred -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d
Patch ServiceAccount	kubectl patch sa default -p '{"imagePullSecrets":[{"name":"regcred"}]}'
Test image pull	kubectl run test --image=<registry>/<image> --restart=Never
Check pull errors	kubectl describe pod <pod> | grep -A5 Events
Delete pull secret	kubectl delete secret regcred
Update credentials	kubectl create secret docker-registry regcred ... --dry-run=client -o yaml | kubectl apply -f -

Key Takeaways

Summary

1. ImagePullSecrets authenticate Kubernetes with private registries -- know how to create and use them
2. Secrets are namespace-scoped -- ensure the secret is in the same namespace as the Pod
3. Attach secrets to ServiceAccounts for automatic application to all Pods using that account
4. Harbor provides enterprise registry features: RBAC, scanning, signing, replication, and robot accounts
5. Tag immutability prevents attackers from replacing images by overwriting tags
6. Pull-through caches reduce external dependencies and provide scanning/auditing at the network edge
7. Always use TLS for registry communication -- never use insecure registries in production
8. Rotate credentials regularly and use robot accounts with minimal permissions for CI/CD

Common Exam Pitfalls

• Creating the ImagePullSecret in the wrong namespace
• Forgetting to reference the secret in the Pod spec imagePullSecrets field
• Not knowing how to patch a ServiceAccount with pull secrets
• Confusing docker-registry secret type with generic secrets
• Forgetting the --docker-server flag (defaults to Docker Hub if omitted)