Kubelet Troubleshooting — CKA Quick Reference

Quick Diagnostic Flow

# 1. Check if kubelet is running
systemctl status kubelet

# 2. If not running, try to start
systemctl start kubelet

# 3. If it fails, check logs
journalctl -u kubelet -n 50 --no-pager
# OR
cat /var/log/syslog | grep kubelet | tail -n 30

# 4. Try running kubelet manually to see errors
/usr/bin/kubelet --help  # check if binary exists

---

Common Failure Scenarios

1) Kubelet Binary Path Wrong

Symptom:

systemd[1]: kubelet.service: Main process exited, code=exited, status=203/EXEC

Diagnose:

# Check where kubelet actually is
whereis kubelet
which kubelet

# Check what path is configured
grep ExecStart /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf

Fix:

vim /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf
# Change: ExecStart=/usr/local/bin/kubelet
# To:     ExecStart=/usr/bin/kubelet

systemctl daemon-reload
systemctl restart kubelet

---

2) Invalid Configuration Argument

Symptom:

Error: unknown flag: --some-invalid-flag

Diagnose:

# Check kubelet logs
journalctl -u kubelet -n 50

# Check config files
cat /var/lib/kubelet/config.yaml
cat /var/lib/kubelet/kubeadm-flags.env
cat /etc/default/kubelet

Fix:

# Remove invalid flag from one of:
vim /var/lib/kubelet/kubeadm-flags.env
# OR
vim /etc/default/kubelet

systemctl restart kubelet

---

3) Wrong CA Certificate Path

Symptom:

failed to load Kubelet config file: unable to read ca certificate

Diagnose:

# Check config
cat /var/lib/kubelet/config.yaml | grep -i ca

# Verify file exists
ls -la /etc/kubernetes/pki/ca.crt

Fix:

vim /var/lib/kubelet/config.yaml
# Fix: clientCAFile: /etc/kubernetes/pki/ca.crt

systemctl restart kubelet

---

4) Port Already in Use

Symptom:

bind: address already in use

Diagnose:

# Check what's using port 10250 (kubelet default)
ss -tlnp | grep 10250
netstat -tlnp | grep 10250

# Check for zombie kubelet
ps aux | grep kubelet

Fix:

# Kill the process
kill -9 <PID>

systemctl restart kubelet

---

5) Missing Kubeconfig

Symptom:

failed to load kubeconfig: stat /etc/kubernetes/kubelet.conf: no such file or directory

Diagnose:

ls -la /etc/kubernetes/kubelet.conf
cat /var/lib/kubelet/config.yaml | grep kubeconfig

Fix:

# Copy from backup or another node
scp other-node:/etc/kubernetes/kubelet.conf /etc/kubernetes/

# Or rejoin the node
kubeadm token create --print-join-command
# Run join command on worker node

---

6) Swap Enabled

Symptom:

failed to run Kubelet: running with swap on is not supported

Diagnose:

swapon --show
free -h

Fix:

# Disable swap
swapoff -a

# Make permanent
sed -i '/swap/d' /etc/fstab

systemctl restart kubelet

---

7) Container Runtime Not Running

Symptom:

failed to get container runtime version: rpc error: connection refused

Diagnose:

# Check containerd/docker
systemctl status containerd
systemctl status docker

# Check socket
ls -la /var/run/containerd/containerd.sock

Fix:

systemctl start containerd
systemctl enable containerd

systemctl restart kubelet

---

8) Wrong API Server Address

Symptom:

failed to get API server URL: unable to load kubeconfig

Diagnose:

cat /etc/kubernetes/kubelet.conf | grep server:

Fix:

vim /etc/kubernetes/kubelet.conf
# Update: server: https://<correct-ip>:6443

systemctl restart kubelet

---

Essential Commands Cheat Sheet

# Status & Logs
systemctl status kubelet
journalctl -u kubelet -f              # follow logs
journalctl -u kubelet -n 100          # last 100 lines
journalctl -u kubelet --since "5 min ago"

# Service Management
systemctl start kubelet
systemctl stop kubelet
systemctl restart kubelet
systemctl daemon-reload               # after config changes
systemctl enable kubelet              # auto-start on boot

# Configuration Locations
/usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf  # service config
/var/lib/kubelet/config.yaml                                # kubelet config
/var/lib/kubelet/kubeadm-flags.env                         # runtime flags
/etc/default/kubelet                                        # extra args
/etc/kubernetes/kubelet.conf                                # kubeconfig

# Validation
kubelet --version
ps aux | grep kubelet
crictl ps                             # check if containers running
kubectl get nodes                     # from control plane

---

CKA Exam Tips

1. Always check status first: systemctl status kubelet
2. Read the full error: Logs usually tell you exactly what's wrong
3. Compare with working nodes: scp, diff, or manual comparison
4. After fixing, wait 30s: Node needs time to register as Ready
5. Don't forget daemon-reload: Required after editing service files
6. Check both service config AND kubelet config: Issues can be in either