From API Request to Running Application

Objectives

• Explain the responsibilities of the API server, etcd, scheduler, controllers, and kubelet.
• Trace how a declared workload becomes running containers.
• Distinguish Pod replacement from a container restart.

1. Declare an outcome, not a sequence of commands

Imagine asking Kubernetes to keep two copies of a web application running. You declare the desired workload; you do not choose each machine and manually start each process. Kubernetes uses control loops to compare the requested configuration with what exists and take steps toward the requested result. A Deployment expresses this desired state and manages Pods through ReplicaSets. [S1] [S9]

A quick refresher: a container runs an application process using an image. A Pod is the Kubernetes unit that groups one or more containers for execution on a node. A node is a machine that hosts Pods. The control plane makes cluster-wide decisions; node components carry out the work of running containers. These are logical responsibilities: a small practice cluster can place control-plane components and application workloads on the same machine. [S1] [S2]

Keep two ideas separate:

• Specification: what the workload requests, such as two replicas and a particular container image.
• Status: observations about progress, such as whether Pods have been assigned to nodes and whether containers have started. [S2] [S9]

An accepted API request is therefore not proof that the application is running. Workload creation and container startup happen through subsequent work by other components. [S1] [S9]

Check yourself: A Deployment exists, but no application container has started. Is that a contradiction? No. The desired state can exist before the cluster has achieved it.

2. Read the architecture by responsibility

Use the architecture diagram below as a map of responsibility, not as a single chain of direct remote commands. The API server is the shared interface through which the components coordinate cluster objects. [S1]

These responsibilities come from the cluster architecture and workload lifecycle descriptions. [S1] [S2] [S9]

Controllers are active processes, while a Deployment and a ReplicaSet are API objects those processes act on. The distinction matters: storing an object is not the same thing as doing the work it requests. [S1] [S9]

Check yourself: Which component selects a node, and which component asks the runtime to start containers there? The scheduler selects; the kubelet works with the runtime. [S1] [S2]

3. Trace one Deployment from request to execution

The original YAML example in the code examples declares a Deployment named request-path-demo. It is a reading example, not an instruction to apply anything yet: environment and context verification come next in the course. Its two replicas and container image make the desired outcome concrete. The selector matches the Pod-template label so the workload can identify the Pods it manages. [S9]

Follow the numbered sequence diagram:

1. Request and storage: A client such as kubectl sends the workload declaration to the API server. For a successfully accepted write, cluster object data is persisted using the backing store, etcd. The client does not submit the Deployment directly to etcd or to a worker's runtime. [S1]
2. Workload reconciliation: The Deployment controller observes the Deployment and creates or adjusts a ReplicaSet. The ReplicaSet controller creates the missing Pod objects. For this new Deployment, the target is two Pods. The Pod objects can exist before any containers run. [S9]
3. Placement: The scheduler observes each Pod that has no node assignment, selects a suitable node, and records the assignment through the API. Scheduling considers resource requirements and placement constraints; the two replicas need not land on different nodes. [S1]
4. Node execution: The kubelet on an assigned node observes the Pod specification and works with its container runtime to prepare and start the containers. Image retrieval and other startup work can take time or fail. The scheduler itself never launches the application process. [S1] [S2]
5. Feedback: The kubelet reports container and Pod status through the API. Controllers continue reconciling, and clients can inspect the resulting objects and status. This is continuing coordination, not a one-time script that ends after creation. [S1] [S2] [S9]

The sequence diagram compresses repeated observations into one successful path. In practice, the components run independent loops and retry work; creating the Deployment does not synchronously complete every step. Both replicas independently pass through scheduling and startup. [S1] [S9]

Check yourself: If you submit a standalone Pod rather than a Deployment, which part of this trace is absent? The Deployment-to-ReplicaSet creation chain. An ordinary unscheduled Pod still needs placement and node-side execution, but has no Deployment controller maintaining a desired replica count. [S1] [S2] [S9]

4. Use the trace to locate a stalled handoff

Architecture becomes useful when you can ask, 'How far did this workload get?' Start with observations, rather than treating every delay as an application crash.

These are diagnostic starting points, not guarantees that the named component is defective. Scheduling constraints can legitimately make a Pod unschedulable, and a container can wait for startup dependencies after placement. [S1] [S2] Resource requests, rather than current observed utilization alone, determine whether a Pod fits within node allocatable capacity. [S25]

Pending covers more than waiting for a node: it can also include startup work such as downloading images. Running is a coarse Pod phase, not proof that the application can serve users. Readiness is a separate signal. CrashLoopBackOff, when displayed by kubectl, describes container restart backoff rather than a distinct Pod phase. [S2] A failed readiness probe makes a Pod unready without killing its container. [S33]

Check yourself: A Pod is Pending but already has a node assignment. Should you immediately blame the scheduler? No. Inspect its container state and events to find the startup problem. [S2]

5. Distinguish a restarted container from a replacement Pod

A Pod has an identity: its UID. Kubernetes assigns that Pod to a node once. It does not move the same Pod identity to another node. Recovery on another node requires a replacement Pod with a new UID. Even if a replacement reuses the old name, it is a different Pod. [S2]

There are two different recovery paths:

These distinctions follow the Pod lifecycle and Deployment reconciliation models. [S2] [S9]

For ordinary application containers, Pod-level restartPolicy can be Always, OnFailure, or Never. Always restarts after either successful or unsuccessful termination; OnFailure restarts after an error exit; Never does not automatically restart the terminated container. Deployment Pod templates require Always. A restart policy concerns containers within a Pod: it does not recreate a deleted standalone Pod. [S2]

Consider two teaching scenarios, not captured cluster output:

• Scenario A: The Pod UID stays unchanged, while its application container's restart count rises after a crash. This is a container restart, not a new scheduling event for that Pod. [S2]
• Scenario B: A Deployment-managed Pod is deleted while the desired replicas remain two. Its ReplicaSet creates another Pod to restore the count. That new Pod has a different UID and needs its own placement. This is replacement, even if the new Pod's restart count is zero. [S2] [S9]

Restarting does not correct a bad application configuration. Repeated failures can lead to restart backoff; use the failure evidence to investigate instead of assuming repeated restarts mean recovery has succeeded. [S2]

Check yourself: Does a zero restart count prove the workload has never failed? No. You might be looking at a replacement Pod rather than the earlier failed instance.

6. Inspect the evidence and test your mental model

The optional shell example reads workload and Pod details only. Use it later, after verifying your practice cluster and namespace, and only if the example workload exists there. Replace both uppercase values with your verified practice target. No cluster access is required to complete this lesson.

The Deployment and ReplicaSet views reveal the workload chain. The Pod views expose assignment, UID, ownership, phase, and per-container restart information. Compare the UID across observations before calling an event a 'Pod restart'; inspect container details rather than inferring everything from the Pod list's display status. [S2] [S9]

Finish with this paper exercise. For each situation, name the next responsibility to inspect:

1. Two Pod objects exist, neither has a node assignment, and scheduling events mention insufficient memory.
2. A Pod is assigned to a node, but its application container is waiting because its image cannot be retrieved.
3. One application container crashes and starts again; its Pod UID is unchanged.
4. A Deployment-managed Pod disappears, and a different Pod UID appears while desired replicas remain unchanged.

Answers:

1. Inspect scheduling evidence and resource requests against allocatable capacity, not just live memory usage. [S1] [S25]
2. Inspect node-side startup and the waiting reason; placement already happened. [S2]
3. This is container restart handling by the kubelet and runtime, subject to restart policy. [S1] [S2]
4. This is controller-driven Pod replacement through the ReplicaSet, followed by scheduling and node execution for the new Pod. [S1] [S2] [S9]

You are ready for the environment-verification lesson when you can narrate this path without merging the roles: API server and backing store, workload controllers, scheduler, kubelet and runtime, then status feedback.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: request-path-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: request-path-demo
  template:
    metadata:
      labels:
        app: request-path-demo
    spec:
      restartPolicy: Always
      containers:
        - name: web
          image: nginx:stable

Reading-only architecture example, not a production manifest or an instruction to apply. A Deployment requests two Pods via a ReplicaSet. [S9]

# Optional later: replace these values with your verified practice target.
CONTEXT='REPLACE_WITH_VERIFIED_PRACTICE_CONTEXT'
NAMESPACE='REPLACE_WITH_VERIFIED_PRACTICE_NAMESPACE'

# All commands below read existing objects; they do not create this example.
kubectl --context="$CONTEXT" --namespace="$NAMESPACE" get deployment request-path-demo
kubectl --context="$CONTEXT" --namespace="$NAMESPACE" get replicasets -l app=request-path-demo
kubectl --context="$CONTEXT" --namespace="$NAMESPACE" get pods -l app=request-path-demo -o wide
kubectl --context="$CONTEXT" --namespace="$NAMESPACE" get pods -l app=request-path-demo -o yaml
kubectl --context="$CONTEXT" --namespace="$NAMESPACE" describe pods -l app=request-path-demo

Optional read-only inspection after environment verification. Look for metadata.uid, metadata.ownerReferences, spec.nodeName and status.containerStatuses. [S2] [S9]

Key takeaways

• The API server exposes cluster objects; etcd stores cluster data. Neither executes your application containers. [S1]
• A Deployment reconciles through ReplicaSets and Pods; accepting its declaration is not the same as achieving its desired state. [S9]
• The scheduler selects a node; the kubelet on that node works with a CRI-compatible runtime to run containers. [S1]
• A Pending Pod can already be assigned to a node. Inspect container state and events before diagnosing a placement failure. [S2]
• A container restart keeps the Pod UID. A replacement Pod has a new UID and its own placement, even if its name is reused. [S2]
• Readiness and Pod phase answer different questions; a Running phase alone does not prove application availability. [S2] [S33]

Common mistakes

• Treating a successful workload creation response as proof that the application is serving.: Inspect the subsequent controller, scheduling and startup stages, then check readiness rather than stopping at object creation. [S1] [S2] [S9]
• Saying that the scheduler starts containers.: Separate placement from execution: the scheduler assigns a node, while the kubelet works with the container runtime. [S1]
• Calling every Pending Pod unscheduled.: Check node assignment and container waiting reasons. Pending also covers startup work such as pulling images. [S2]
• Saying a Pod moved to another node after a failure.: Compare UIDs. A Pod is assigned once; another node runs a replacement Pod, not the same Pod identity. [S2]
• Expecting restartPolicy to recreate a deleted standalone Pod.: Use a suitable workload controller when replacement is required. Container restart policy operates within the existing Pod. [S2] [S9]

References

1. Cluster Architecture, Kubernetes
2. Pod Lifecycle, Kubernetes
3. Deployments, Kubernetes
4. Resource Management for Pods and Containers, Kubernetes
5. Liveness, Readiness, and Startup Probes, Kubernetes

See the full course: Kubernetes in Practice: From First Pod to Production Troubleshooting