Kubernetes Ingress Troubleshooting: Diagnose 404s, TLS Errors, and Routing Failures
KubernetesIngressNetworkingTLSNGINXTroubleshooting

Kubernetes Ingress Troubleshooting: Diagnose 404s, TLS Errors, and Routing Failures

BBinaries Live Editorial Team
2026-11-17
7 min read

A reusable Kubernetes Ingress checklist for isolating 404s, TLS failures, wrong backends, DNS issues, and controller routing problems.

Kubernetes Ingress troubleshooting is easiest when you follow the request from DNS to load balancer, controller, Ingress rule, Service, endpoints, and application. This reusable checklist helps isolate Kubernetes Ingress 404 responses, TLS failures, and incorrect routing without changing several layers at once.

Overview

An Ingress resource is a set of HTTP or HTTPS routing rules. It does not process traffic by itself; an Ingress controller watches the resource and configures a proxy, load balancer, or cloud integration. A routing failure can therefore originate outside Kubernetes, in the controller, or inside the target workload.

Use this request-path model when investigating:

Client → DNS → external load balancer or listener → Ingress controller → Ingress rule → Service → EndpointSlice → Pod → application.

At each layer, ask two questions: did the request arrive, and did this layer select the expected next destination? Avoid editing annotations, Services, DNS records, and certificates simultaneously. A controlled sequence produces clearer evidence and reduces the chance of masking the original fault.

Fast request-path decision tree

  • DNS does not resolve or resolves to the wrong address: investigate the hostname, record type, propagation, and load-balancer address.
  • Connection times out or is refused: check the external listener, firewall or security rules, controller Service, and exposed ports.
  • TLS negotiation fails: inspect the certificate Secret, hostname coverage, certificate validity, listener configuration, and TLS termination point.
  • HTTP 404 is returned: identify which component generated it, then check the Ingress class, host, path, path type, and controller logs.
  • HTTP 502 or 503 is returned: verify the Service port, named target port, EndpointSlices, Pod readiness, and application listener.
  • The wrong application responds: compare the requested host and path with the effective rules, then check for overlapping Ingress resources or multiple controllers.

Checklist by scenario

When Kubernetes Ingress returns 404

  1. Record the exact URL, HTTP method, hostname, path, and response headers. A 404 from an Ingress controller is different from a 404 generated by the application.
  2. Confirm that DNS points to the address managed by the intended controller:
dig +short app.example.com
kubectl get ingress -A
kubectl get svc -A
  1. Inspect the Ingress and confirm its class, host, path, backend Service, and events:
kubectl describe ingress <name> -n <namespace>
kubectl get ingress <name> -n <namespace> -o yaml
kubectl get events -n <namespace> --sort-by=.lastTimestamp
  1. Check that the resource uses the class watched by the controller. Prefer the spec.ingressClassName field where supported. An absent or incorrect class can leave the resource ignored or processed by an unexpected controller.
  2. Compare the request exactly with the rule. Host matching is significant, and a path such as /api is not interchangeable with every variation of /api/. Verify the selected pathType, especially when moving from older manifests.
  3. Check the controller logs while making one test request. The controller namespace and deployment name vary by installation:
kubectl get pods -A | grep -i ingress
kubectl logs -n <controller-namespace> deploy/<controller-deployment> --tail=100

If the controller logs show no matching rule, focus on DNS, the Ingress class, host, path, or controller configuration. If they show a backend selection, test the Service and application separately.

When TLS fails

  1. Determine where TLS terminates: at a cloud load balancer, at the Ingress controller, or at the application. The certificate you inspect must belong to that termination point.
  2. Check the Ingress TLS section and Secret reference:
kubectl get ingress <name> -n <namespace> -o yaml
kubectl get secret <tls-secret> -n <namespace> -o jsonpath='{.type}{"\n"}'
  1. Confirm that the Secret is in the namespace expected by the Ingress and contains the required TLS data. Do not print private key material into a terminal recording or incident channel.
  2. Inspect the certificate presented to the client:
openssl s_client -connect app.example.com:443 -servername app.example.com </dev/null 2>/dev/null | openssl x509 -noout -subject -issuer -dates -ext subjectAltName

Check the hostname in the certificate's subject alternative names, validity dates, intermediate chain, and whether the presented certificate is the one you intended. A valid Secret does not prove that the external listener is serving it.

When traffic reaches the wrong Service

  1. List all Ingress objects for the hostname. Multiple resources may overlap, and controller-specific merge behavior should not be assumed.
  2. Inspect the effective backend references and verify the Service name and port:
kubectl get ingress -A -o wide
kubectl get svc <service> -n <namespace> -o yaml
kubectl get endpointslice -n <namespace> -l kubernetes.io/service-name=<service>
  1. Check Service selectors against Pod labels. A Service can exist and still have no usable endpoints.
  2. Test the Service from inside the cluster to separate Ingress routing from application behavior:
kubectl port-forward -n <namespace> svc/<service> 8080:<service-port>
curl -i http://127.0.0.1:8080/<path>

If the internal request fails, troubleshoot the Service, EndpointSlice, Pod readiness, container port, or application route before changing Ingress rules.

Controller differences to account for

Layer or controllerWhat to verifyWhy it matters
NGINX IngressController class, namespace-scoped annotations, rewrite and SSL settings, and controller logsAnnotations can alter path handling, redirects, timeouts, and backend behavior. Confirm that an annotation is supported by the installed controller version.
TraefikIngressClass, entryPoint, middleware references, and whether the installation also expects IngressRoute resourcesTraefik configuration can combine Kubernetes objects with static or dynamic provider settings. A valid Ingress may not use the intended entry point or middleware.
Cloud load balancerListener ports, health checks, target registration, security rules, and provider-specific annotationsThe cloud listener may terminate TLS or route traffic before it reaches the Kubernetes controller. Provider annotations are not portable between platforms.

For all three, establish the controller's actual Service address and class rather than relying on resource names copied from another cluster. When migrating controllers, remove assumptions about annotations and retest each host and path.

What to double-check

Resource fields and annotations

Field or settingCheck
spec.ingressClassNameMatches the intended controller's IngressClass.
spec.rules[].hostMatches the hostname sent by the client, including the absence or presence of a www subdomain.
spec.rules[].http.paths[]Uses the expected path and pathType, with a valid Service backend and port.
spec.tls[]Lists the same host and references the correct Secret in the Ingress namespace.
Controller annotationsAre documented for the installed controller and do not conflict with the intended rewrite, redirect, body-size, timeout, or TLS behavior.
Controller ServiceExposes the required HTTP and HTTPS ports and has healthy controller endpoints.

A repeatable isolation sequence

  1. DNS: Resolve the hostname from the client network and compare the result with the controller or load-balancer address.
  2. Listener: Test HTTP and HTTPS separately with curl -v. Record redirects, status codes, headers, and the certificate presented.
  3. Ingress: Inspect YAML, class, host, path, backend, events, and overlapping resources.
  4. Controller: Review logs during a single request and confirm that the controller loaded the resource.
  5. Service: Confirm the port mapping, selectors, and EndpointSlices.
  6. Pod: Check readiness, listening ports, network policies, and recent logs.
  7. Application: Test the exact route and host header. Many apparent Ingress failures are application routes or redirects that differ from the external URL.

Common mistakes

  • Assuming every 404 comes from the application. Inspect response headers and controller logs before changing application routes.
  • Using an annotation from another controller. NGINX, Traefik, and cloud integrations have different configuration models. Treat annotations as controller-specific unless documented otherwise.
  • Checking only the Service object. A Service with no matching EndpointSlices cannot send traffic to a healthy backend.
  • Testing by IP without the Host header. Host-based rules require the original hostname. Use curl --resolve when you need to test a specific address while preserving the host:
curl -vk --resolve app.example.com:443:203.0.113.10 https://app.example.com/health
  • Rotating certificates without checking the termination point. Updating a Kubernetes Secret has no effect if TLS terminates at an external listener that uses a different certificate store.
  • Changing several layers at once. Keep a short change log and make one reversible adjustment per test.
  • Ignoring version changes. API versions, controller defaults, annotation support, and cloud-provider integrations can change during cluster or controller upgrades. Validate manifests in a non-production environment where possible.

When to revisit

Return to this checklist whenever you add a hostname or path, renew a certificate, change DNS, migrate Ingress controllers, modify a cloud load balancer, or investigate a production routing incident. It is also useful after a Kubernetes or controller upgrade, especially when a previously accepted manifest begins producing warnings or is no longer selected.

Before seasonal planning cycles, confirm that the runbook still names the correct controller namespace, deployment, IngressClass, listener addresses, and certificate process. When workflows or tools change, update the command examples and annotation table rather than copying old configuration forward.

For the next incident, save the hostname, timestamp, client location, exact command, response status, controller logs, Ingress YAML, Service details, EndpointSlice output, and certificate summary. That evidence turns a one-time fix into a reliable operational record. If the failure reveals broader capacity or readiness issues, pair this workflow with the Kubernetes resource requests and limits guide; for certificate rotation procedures, see the secrets rotation runbook.

Related Topics

#Kubernetes#Ingress#Networking#TLS#NGINX#Troubleshooting
B

Binaries Live Editorial Team

Cloud and Infrastructure Editor

Senior editor and content strategist. Writing about technology, design, and the future of digital media. Follow along for deep dives into the industry's moving parts.