How Do You Troubleshoot the Kubernetes Error “cluster unreachable”?

Problem scenario
You get a message "kubernetes cluster unreachable" when running a helm, kubectl, az, or eks command. What should you do?

Possible solution #1
Has a router been reconfigured? Has a new firewall rule been imposed? Has a data center gone down that housed the cluster? Did you receive an email about a maintenance window or a configuration change?

Possible solution #2
What are the permissions of the relevant .yaml file? Can you try changing them? (Idea taken from https://github.com/k3s-io/k3s/issues/1126.)

Possible solution #3
If you were running a kubectl command, can you try to run commands like these?

sudo cp ~/.kube/config ~/.kube/bak.config.bak
kubectl config view --raw >~/.kube/config

(Idea taken from https://github.com/k3s-io/k3s/issues/1126.)

Possible solution #4
Warning: this may install something undesirable. Use this with caution.
If you were using helm, Rancher and K3s, can you try running a command like this?

sudo helm install harbor/harbor --version 1.3.0 --generate-name --kubeconfig /etc/rancher/k3s/k3s.yaml

(Idea taken from https://github.com/k3s-io/k3s/issues/1126.)

Possible solution #5
Are you using Rancher and K3s? Try running this:

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

(Idea taken from https://github.com/k3s-io/k3s/issues/1126 )

Possible solution #6
Can you ping the remote Kubernetes server? Are you not on a server that has access to the remote network? Is the cluster available from the network you are on? Are you running the command from a workstation that is not connected to a VPN?

Possible solution #7
Are you using GKE? If so, read this posting: https://forum.rasa.com/t/rasa-x-kubernetes-cluster-unreachable-error-using-github-actions/37073/8

Possible solution #8
Verify configuration files. If there are two clusters configured, can you try to only use one cluster? Try each cluster one-at-a-time. This may help you pinpoint the problem.

Possible solution #9
Run commands like these:

kubectl config view
kubectl config use-context CONTEXT-CHOSEN-FROM-PREVIOUS-COMMAND-OUTPUT
kubectl config --help

(This was adapted from https://stackoverflow.com/questions/49260135/unable-to-connect-to-the-server-dial-tcp-i-o-time-out.)


If you want assistance troubleshooting a network problem, see this posting.

Leave a comment

Your email address will not be published. Required fields are marked *