How Do You Troubleshoot a PVC That Appears to Still Be in Use?

Problem scenario
You are working with Kubernetes. A PVC cannot be mounted. It seems like the Persistent Volume is still in use. What should you do?

Possible Solution #1
Find out about the volume attachments with these commands:

kubectl get volumeattachment
kubectl get pv

Find the Pods that use it. Is unmounting working? Perhaps the unmounting process has a problem.

Run kubectl describe pod foobar (where "foobar" is the name of the Pod trying to use the PVC).

Investigate if the volume is getting detached before it is attempted to to be reused by a Pod. If your use case allows you to delete the Pod that was using it previously, you may want to do that.

Possible Solution #2
Change the grace period seconds in the YAML that govern access to it.

"By default, Kubernetes will wait 30 seconds to allow processes to handle the TERM signal. " (Quote taken from https://linkerd.io/2.13/tasks/graceful-shutdown/.)

Experiment with a longer or shorter grace period. A shorter one will mean the TERM signal will be sent sooner. Your subsequent utilization of the PV (with a PersistentVolumeClaim) may then work. Alternatively, a longer grace period may make the series of processes complete without the TERM signal.

Possible Solution #3
Examine your monitoring and logs. Are there problems with networking or the hardware involved? See this external posting.