WIUNIX ACADEMY
Kubernetes PVC Pending and CSI: diagnose binding, attach and mount
Page 2 of 3. Review the earlier steps and prerequisites before continuing.
YOUR SYSTEM. YOUR STEPS.
Where are you working?
Choose your OS and version. The prerequisites, commands and troubleshooting follow your environment.
USE YOUR OWN VALUES
Configure this tutorial
Set the example values for your environment; related explanations and copy panels update together. Never enter passwords, private keys or tokens. Values stay in this tab and are not sent to the server or share links. Check actual DNS, certificates and network policy before running commands.
Different IP addresses, domains or ports? Open the settings below, change the values and select “Apply to tutorial”, then copy the updated commands.
Set values for my environment9 editable values
LET’S BUILD
Match the consuming Pod to its actual claim and node
Record the Pod UID and controller reference so a replacement is not mistaken for the original incident. Read the volume-to-claim names and scheduling conditions, then the events for this object. The form’s claim must actually be one of the Pod’s claims; a plausible name in the same namespace is not sufficient.
A Pod can stay Pending before any node has been selected. ContainerCreating with a node name is a later boundary. Event retention is finite and repeating events are aggregated; an empty event list is not evidence that no error occurred. Retain timestamps and the exact UID with central/controller evidence when necessary.
Same authorized Bash shell on the administration host
k get pod 'example-app' -n 'apps' -o jsonpath='{.metadata.uid}{"\n"}{.metadata.ownerReferences}{"\n"}{.status.phase}{"\t"}{.spec.nodeName}{"\n"}{range .spec.volumes[*]}{.name}{"\t"}{.persistentVolumeClaim.claimName}{"\n"}{end}{range .status.conditions[*]}{.type}{"="}{.status}{"\t"}{.reason}{"\t"}{.message}{"\n"}{end}'
k events -n 'apps' --for 'pod/example-app'The selected Pod UID, actual PVC reference, node choice and current failure stage are known.
LET’S BUILD
Read the claim before treating Pending as a fault
Read the claim UID, phase and full spec plus its selected-node annotation. The spec shows storage size, access mode, volume mode, selector, optional data source, class and any explicit volumeName. This is private operational configuration; it contains no need to fetch a Secret for diagnosis.
Distinguish an omitted storageClassName from an explicitly empty string. Omission can use a default class, including defaulting behavior supported by the cluster; an explicit empty class requests no class and does not ask a dynamic provisioner to create storage. A selector or explicit volumeName also changes binding expectations. Do not patch these fields just to clear a Pending label.
Same authorized Bash shell on the administration host
k get pvc 'data-example' -n 'apps' -o jsonpath='{.metadata.uid}{"\n"}{.status.phase}{"\n"}{.spec}{"\n"}{.metadata.annotations.volume\.kubernetes\.io/selected-node}{"\n"}'
k events -n 'apps' --for 'persistentvolumeclaim/data-example'The actual claim request and any selected node are captured, with absent versus explicitly empty class read from the spec.
LET’S BUILD
Separate waiting for a consumer from provisioning failure
After reading the claim, apply its actual class name and inspect the binding mode and provisioner. Immediate can provision/bind before placement. WaitForFirstConsumer intentionally delays topology-sensitive binding/provisioning until a consuming Pod can be scheduled; Pending without a consumer may therefore be normal.
If the provisioner is kubernetes.io/no-provisioner, expect a separately prepared static/local PV rather than a dynamic CSI allocation. For a CSI class, verify that this installed driver supports the chosen mode, topology and access/volume mode. Namespace quota, requested capacity and backend quota/capacity are different checks. The class’s reclaim policy is a deletion consequence, not a repair switch.
Same authorized Bash shell on the administration host
k get storageclass 'example-storage' -o jsonpath='{.metadata.name}{"\n"}{.provisioner}{"\n"}{.volumeBindingMode}{"\n"}{.reclaimPolicy}{"\n"}{.allowedTopologies}{"\n"}'
k get resourcequota -n 'apps'The class’s provisioner/binding design and quota scope explain whether waiting is expected or a provisioning step is failing.
LET’S BUILD
For WaitForFirstConsumer, investigate placement first
Correlate Pod scheduling events with node selectors, affinity, taints/tolerations and resource requests. A claim cannot acquire a suitable zonal volume when no eligible node exists. The selected-node annotation is a coordination hint, not proof a volume has been provisioned or mounted.
Do not set spec.nodeName to force this workflow: bypassing the scheduler can leave a WaitForFirstConsumer claim Pending. Do not remove topology or selected-node annotations blindly. Agree the actual eligible topology with the platform/storage owner; correct the owning workload/class design through its normal reviewed source when that is the identified cause.
Same authorized Bash shell on the administration host
k get pod 'example-app' -n 'apps' -o jsonpath='{.spec.nodeName}{"\n"}{.spec.nodeSelector}{"\n"}{.spec.affinity}{"\n"}{.spec.tolerations}{"\n"}{range .spec.containers[*]}{.name}{"\t"}{.resources}{"\n"}{end}'
k events -n 'apps' --for 'pod/example-app'Scheduling constraints and the storage topology are evaluated together; no node binding or scheduling policy is changed.
LET’S BUILD
When Bound, verify the exact PV relationship
Only after the claim reports a bound volume should the PV branch run. Put the discovered name into the form. Match its claimRef namespace, name and UID to the claim observed earlier, then compare class, capacity, access mode, volume mode and node affinity. A recycled name or an old Released PV is not automatically the same data relationship.
Bound confirms a control-plane relationship, not successful attachment, mounting or readable application data. ReadWriteOnce normally limits writable mounting to one node, not one Pod; multiple Pods on that node can still use it. ReadWriteOncePod is a different CSI capability requiring compatible sidecars. Neither a requested access mode nor a green Bound label establishes application consistency.
Same authorized Bash shell on the administration host
k get pv 'example-volume' -o jsonpath='{.metadata.name}{"\n"}{.status.phase}{"\n"}{.spec.claimRef.namespace}{"/"}{.spec.claimRef.name}{"\t"}{.spec.claimRef.uid}{"\n"}{.spec.storageClassName}{"\n"}{.spec.capacity}{"\n"}{.spec.accessModes}{"\n"}{.spec.volumeMode}{"\n"}{.spec.persistentVolumeReclaimPolicy}{"\n"}{.spec.csi.driver}{"\n"}{.spec.nodeAffinity}{"\n"}'The PV belongs to the exact observed claim UID and its storage semantics/topology are understood.
LET’S BUILD
Check the actual driver and selected-node registration
Run this optional cluster-scoped branch only with delegated permissions and an actual selected node. The driver name is read from the PV rather than invented. Compare the CSIDriver’s attach requirement and lifecycle/mount policy with the node’s registered driver names and topology keys, then the node labels and taints.
Some drivers skip controller attachment. If attachRequired is false, a missing VolumeAttachment can be expected. If the CSIDriver object is absent, Kubernetes defaults to requiring attach; absence is not a proof the driver is healthy or that attach can be skipped. Follow the installed driver’s support documentation. A missing node registration is a node-plugin/registration boundary to take to the storage operator.
Same authorized Bash shell on the administration host
driver=$(k get pv 'example-volume' -o jsonpath='{.spec.csi.driver}') || exit 1
test -n "$driver" || { printf 'STOP: no CSI driver in this PV; use the actual storage owner path.\n' >&2; exit 1; }
k get csidriver "$driver" -o jsonpath='{.metadata.name}{"\n"}{.spec.attachRequired}{"\n"}{.spec.volumeLifecycleModes}{"\n"}{.spec.fsGroupPolicy}{"\n"}'
k get csinode 'worker-a' -o jsonpath='{range .spec.drivers[*]}{.name}{"\t"}{.topologyKeys}{"\n"}{end}'
k get node 'worker-a' -o jsonpath='{.metadata.labels}{"\n"}{.spec.taints}{"\n"}'The real CSI driver, attach behavior and node/topology registration are identified, or their unavailable/forbidden state is recorded.