It is not always appropriate to push one’s own container images to a public registry. This post shows a quick way to create a private image registry inside a K3s Kubernetes cluster.
Please note that with the following manifest, when the registry resources are removed from the cluster, all images will be removed as well. There is a TODO in the very last line that addresses this.
Also important: the registry is unsecured. Further steps (for example, username/password or certificates ) are required to secure it; that is outside the scope of this tutorial.
Kubernetes Resources
Before this manifest is applied, the domain name under spec:rules:host needs to be changed accordingly.
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: docker-registry-ingress
annotations:
kubernetes.io/ingress.class: "traefik"
spec:
rules:
- host: registry.domain.de
http:
paths:
- path: /
backend:
serviceName: docker-registry-service
servicePort: 5000
---
apiVersion: v1
kind: Service
metadata:
name: docker-registry-service
labels:
run: docker-registry
spec:
selector:
app: docker-registry
ports:
- protocol: TCP
port: 5000
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: docker-registry
labels:
app: docker-registry
spec:
replicas: 1
selector:
matchLabels:
app: docker-registry
template:
metadata:
labels:
app: docker-registry
spec:
containers:
- name: docker-registry
image: registry
ports:
- containerPort: 5000
protocol: TCP
volumeMounts:
- name: storage
mountPath: /var/lib/registry
env:
- name: REGISTRY_HTTP_ADDR
value::5000
- name: REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY
value: /var/lib/registry
volumes:
- name: storage
emptyDir: {} # TODO - make this more permanent later
Apply the manifest with:
kubectl apply -f registry.yaml
Settings
On the K3s nodes, create the file /etc/rancher/k3s/registries.yaml with the following content. The domain name must match the one in the manifest above.
mirrors:
"registry.domain.de":
endpoint:
- "http://registry.domain.de"
I did this on both the server and agent nodes. I am not certain it has to be done on the agents, but I did it there.
After that, restart the server and agents.
On the server run:
systemctl restart k3s
On the agent node run:
systemctl restart k3s-agent
You can check if the changes applied with:
crictl info
There is a section called registry that should list the newly created private registry.
The local workstation also needs to know about the new registry. I am using macOS with Docker Desktop. Under “Preferences → Docker Engine”, extend the settings with the following entry:
{
...
"insecure-registries": [
"registry.domain.de"
]
}
Test the Registry
To test pushing an image to the new registry, I built a small image containing Nginx with a custom HTML file:
<html>
<head><title>Hello World!</title>
<style>
html {
font-size: 500.0%;
}
div {
text-align: center;
}
</style>
</head>
<body>
<div>Hello World!</div>
</body>
</html>
The Dockerfile for the image:
FROM nginx:alpine
COPY index.html /usr/share/nginx/html
Build and tag the image according to your registry domain:
docker build -t registry.domain.de/hello:latest.
Push the image:
docker push registry.domain.de/hello:latest
If that worked, you can further test by removing the image locally and pulling it again from the private registry:
docker rmi registry.domain.de/hello:latest
docker pull registry.domain.de/hello:latest