WIUNIX ACADEMY

Kubernetes DNS failure: trace the Pod, CoreDNS and Service endpoints

Separate NXDOMAIN, timeout and SERVFAIL; compare Pod resolver settings, DNS endpoints and direct Service connectivity without a blind CoreDNS restart.

Advanced35 min readUpdated 2026-10-28

Make this guide yours

YOUR SYSTEM. YOUR STEPS.

Where are you working?

Choose your OS and version. The prerequisites, commands and troubleshooting follow your environment.

Red Hat Enterprise Linux 9Reviewed against official documentation · 2026-10-28

USE YOUR OWN VALUES

Configure this tutorial

Set the example values for your environment; related explanations and copy panels update together. Never enter passwords, private keys or tokens. Values stay in this tab and are not sent to the server or share links. Check actual DNS, certificates and network policy before running commands.

Different IP addresses, domains or ports? Open the settings below, change the values and select “Apply to tutorial”, then copy the updated commands.

Set values for my environment11 editable values

THE OUTCOME

What will you have at the end?

Identify the failing DNS or service layer and make one evidence-backed correction through its owner.

Supported operating systems and versions

Documented read-only kubectl workflow; no cluster resolver, policy or workload was changed during authoring. The current tutorial form accepts simple identifiers of at most32characters. This is a form limitation, not the Kubernetes resource-name limit. Never truncate or rename a live object to fit. For a longer existing resource/context name, follow this same diagnostic sequence in the operator-owned direct CLI/editor workflow with its exact original name; no silent default substitution.

Before you begin

Existing CoreDNS cluster and authorized kubectl access
Existing diagnostic container with dig/curl, or an approved owner-managed debugging procedure
Known workload Service, namespace, port and safe HTTP health endpoint
The current tutorial form accepts simple identifiers of at most32characters. This is a form limitation, not the Kubernetes resource-name limit. Never truncate or rename a live object to fit. For a longer existing resource/context name, follow this same diagnostic sequence in the operator-owned direct CLI/editor workflow with its exact original name; no silent default substitution.

LET’S BUILD

Prepare the selected operator host

These are operator-host packages only. Review the transaction before approving; do not install tools inside application containers. The cluster, server and application binaries must already exist with owner-approved versions. Use Bash for the following panels. Package installation changes this selected host but starts no new application server.

Use an existing Linux kubectl within one minor version of every API-server version; inspect the actual server version in the authorized context before proceeding. Do not install kubelet/kubeadm or overwrite cluster client repositories for this diagnosis.

Selected operator host; approved package transaction

sudo dnf install ca-certificates bash coreutils bind-utils
if ! command -v curl >/dev/null 2>&1; then sudo dnf install curl; fi
cat /etc/os-release
command -v bash curl timeout
command -v kubectl
kubectl version --client --output=yaml
dig -v

How do I know it worked?

The OS/version matches the selection, required tools resolve, and existing application/client versions satisfy the stated compatibility boundary. Stop if a binary is missing or incompatible.

Preserve host security and existing paths

Keep SELinux enforcing and firewalld rules in place. These client-side diagnostics require outbound access to the approved API/TLS/SFTP destination; they need no new inbound host port. A failure in a confined systemd service may differ from an interactive shell. Review AVC denials, file labels and service permissions with the policy owner rather than disabling SELinux, broad allow rules or disabling the firewall. Trust-chain/crypto-policy failures require a compatible endpoint, not weakened system crypto. Existing custom data/config paths need their established labels; never run a recursive relabel blindly.

Selected operator host; read-only security inspection

if command -v getenforce >/dev/null 2>&1; then getenforce; fi
if command -v firewall-cmd >/dev/null 2>&1; then firewall-cmd --state; fi

Start with a falsifiable path

A successful DNS answer only proves name resolution, not an accepting application. A successful TCP connection to a Pod IP does not prove the Service selector, port translation or DNS. Follow source Pod resolver → cluster DNS Service → CoreDNS endpoints → authoritative Kubernetes record; test the workload Service separately.

This read-first workflow assumes CoreDNS and ordinary ClusterIP Services. NodeLocal DNSCache, custom cluster domains, service meshes, hostNetwork Pods and provider DNS need their owning implementation details. Do not replace the shared Corefile or restart all DNS Pods during a diagnosis.

LET’S BUILD

Confirm the source and destination namespaces

Short Service names resolve within the source namespace. The same short name in another namespace may identify a different Service. Use the full configured cluster suffix and a trailing dot for an absolute DNS query; do not assume every cluster uses cluster.local.

Operator terminal; authorized lab context

kubectl config get-contexts 'lab'
kubectl --context='lab' -n 'academy-lab' get pod 'dns-client' -o wide
kubectl --context='lab' -n 'academy-lab' get service 'api' -o wide

How do I know it worked?

The authorized context, source Pod and exact destination Service exist.

LET’S BUILD

Read the resolver inside the failing Pod

Read the actual failing container rather than a node resolver. Inspect nameserver, search suffixes and ndots alongside dnsPolicy, dnsConfig and hostNetwork. With ClusterFirst, cluster names go to cluster DNS; Default inherits node resolver behavior. An application may use a different resolver library or internal cache, so shell success is a control rather than proof about the application.

Operator terminal; authorized lab context

kubectl --context='lab' -n 'academy-lab' exec 'dns-client' -c 'tools' -- cat /etc/resolv.conf
kubectl --context='lab' -n 'academy-lab' get pod 'dns-client' -o jsonpath='{.spec.dnsPolicy}{"\n"}{.spec.dnsConfig}{"\n"}{.spec.hostNetwork}{"\n"}'

How do I know it worked?

Actual DNS server/search path and Pod DNS policy are recorded without modification.

LET’S BUILD

Classify the query failure

These commands require an existing authorized container with dig. Do not install diagnostic packages into an application image or assume nslookup exists. If absent, ask the workload owner to provide an approved debug container or use the official dnsutils manifest after image/security review. Compare UDP and TCP with bounded timeouts from the same network namespace.

Operator terminal; authorized lab context

kubectl --context='lab' -n 'academy-lab' exec 'dns-client' -c 'tools' -- dig +time=2 +tries=1 'api.academy-lab.svc.cluster.local.' A
kubectl --context='lab' -n 'academy-lab' exec 'dns-client' -c 'tools' -- dig +tcp +time=2 +tries=1 'api.academy-lab.svc.cluster.local.' A
kubectl --context='lab' -n 'academy-lab' exec 'dns-client' -c 'tools' -- dig +time=2 +tries=1 kubernetes.default.svc.'cluster.local'. A

How do I know it worked?

Record status, answer, server and timing: NXDOMAIN, SERVFAIL and timeout are distinct outcomes.

Explore all tutorials ↗