Platform Engineering Knowledge System

Build. Operate. Understand.

Practical engineering knowledge for DevOps, SRE, Platform Engineering, Infrastructure, Cloud and modern production systems.

DEVOPSSREPLATFORMCLOUDKUBERNETESINFRASTRUCTURE
A change travelling from a developer to production Developer to Git to CI and CD to Kubernetes and OpenShift, which fans out to network, storage and compute, then back through observability to SRE and finally production. DEVELOPER GIT CI / CD KUBERNETES / OPENSHIFT NETWORK STORAGE COMPUTE OBSERVABILITY SRE PRODUCTION

The knowledge system

0
Topics
0
Categories
0
Pillars
0
Live
A published page covers it.
0
In pipeline
Next to a series already running.
0
Planned
On the backlog, no date attached.

21 issues published so far. Every figure here is counted from the same file the site is generated from, so it cannot disagree with what is actually published — browse the categories.

From code to production

Eleven stages between an edit and a user. Select any one of them.

Stage 1 of 11

Code

The change starts as an edit on somebody's machine, next to a test that fails until it doesn't.

Commonly
EditorLanguage runtimeUnit testsPre-commit hooks
Read more

READ THE FULL STORY

Who writes Platform Ops, what it covers, why it exists, and how the pieces connect.

Platform Ops is written by Vishal Abhinav and published by Srivan Technologies. The full About page carries the publication promise, author background, coverage model, licensing notes, and the starting points for new readers.

Infrastructure architecture

One request, from the person who made it down to the engineer who gets paged when it fails. Take any layer — by pointer or by tab — and the panel beside it says what that layer is for.

Production doesn't break.
Your observability tells you why.

Five symptoms, and the order to look in. Open one.

Pod pendingSchedulerNodesTaintsAffinityPVC

The scheduler has looked at every node and refused to place it. The pod is fine; the cluster has no seat for it.

Investigation
  1. kubectl describe pod <name>Read the Events at the bottom first. The scheduler says exactly why it declined, once per node.
  2. Insufficient cpu / memoryNo node has room for the requests — not the usage. Compare requests against each node's allocatable, not its capacity.
  3. Untolerated taintA node is marked for something else and this pod has no toleration for it. Common on control-plane and GPU pools.
  4. Didn't match node affinity/selectorA nodeSelector or affinity rule matches nothing. A typo in a label value fails exactly like a full cluster.
  5. Unbound PersistentVolumeClaimThe pod waits for a volume that will never bind — no StorageClass, no capacity, or the wrong access mode.
  6. Nothing in Events at allThe scheduler never saw it. Check the pod is not stuck on an admission webhook or a missing priority class.
CrashLoopBackOffLogsEventsProbesConfigResources

The container starts, exits, and Kubernetes backs off before trying again. The loop is the symptom; the first exit is the cause.

Investigation
  1. kubectl logs <pod> --previousWithout --previous you are reading the instance that has not failed yet. The one that died holds the answer.
  2. Read the exit code1 the application decided to stop. 137 it was killed — usually OOM. 143 it was asked to stop and did.
  3. kubectl describe pod → Last StateReason: OOMKilled means the memory limit, not a leak, ended it. Raise the limit or find what grew.
  4. Liveness probe restarting a healthy appA slow starter fails a liveness probe that began too early. That is what startupProbe is for; raising initialDelaySeconds hides it.
  5. Missing config or secretA mount that is not there, or an env var the app requires and does not find, exits before the first log line.
  6. It only crashes on some nodesThen it is not the image. Look at the node: kernel, runtime version, or a mount that exists in one place.
Node NotReadyKubeletNetworkDiskRuntimeConditions

The kubelet has stopped saying it is healthy. Everything on that node is already being evicted.

Investigation
  1. kubectl describe node <name>The Conditions block names it: MemoryPressure, DiskPressure, PIDPressure, or Ready=Unknown when the kubelet went quiet.
  2. Ready=UnknownThe node stopped reporting rather than reported a problem. That is the network or the kubelet process, not the workload.
  3. systemctl status kubeletThen journalctl -u kubelet -n 200. The kubelet says why it is unhappy in plain text.
  4. DiskPressure/var/lib/kubelet or the image store is full. Image garbage collection has thresholds; a large pulled image can cross them.
  5. Container runtimecrictl info and crictl ps. A kubelet cannot be ready if the runtime under it is not.
  6. CNI not readyNo pod networking means no Ready. Check the network plugin's own pods on that node before blaming the kubelet.
High memoryMetricsWorking setOOMKilledLimitsGC

Usage is climbing. Whether that is a leak, a cache, or a limit set too low is the whole question.

Investigation
  1. kubectl top pod — against the limitA number on its own means nothing. What matters is the ratio to the limit, and the shape over time.
  2. Sawtooth or rampSawtooth is a garbage collector doing its job. A ramp that never comes down is a leak, or a cache with no bound.
  3. Working set, not RSSThe kubelet evicts on working set. Page cache the container touched counts towards it, which surprises people.
  4. OOMKilled with plenty freeThe cgroup limit was reached, not the node's memory. The node being fine is not evidence.
  5. Heap vs off-heapA JVM or Go heap graph can look calm while the process grows — native allocations, thread stacks and mapped files sit outside it.
  6. Requests below limitsBurstable pods are evicted before Guaranteed ones. If it dies under node pressure, the QoS class is the reason.
Connection timeoutDNSNetworkServiceEndpointApplication

The request goes out and nothing comes back. A timeout is the one failure that looks identical at every layer.

Investigation
  1. Resolve it from inside the podnslookup <svc> in the calling pod. Name resolution failing looks exactly like the far end being down.
  2. kubectl get endpoints <svc>A Service with no endpoints is a healthy-looking object that routes nowhere. This is the single most common cause.
  3. Selector mismatchEndpoints are empty because the Service selector matches no pod. Both objects look correct in isolation.
  4. Readiness, not livenessA pod that is running but not ready is removed from endpoints. Traffic stops without a single restart.
  5. NetworkPolicyA default-deny policy in the namespace drops the packet silently. Timeout, never refused.
  6. Listening on 127.0.0.1The app binds loopback instead of 0.0.0.0, so it works inside the container and nowhere else.

What are you building?

Ten jobs that share most of this material and need different parts of it. The number is how many pages are live in that area today.

Cloud Engineer

1 live pages across 5 areas

Kubernetes Engineer

59 live pages across 5 areas

Network Engineer

18 live pages across 4 areas

Security Engineer

36 live pages across 4 areas

DevSecOps Engineer

15 live pages across 5 areas

Observability Engineer

21 live pages across 4 areas

The engineering journey

Or one route through all of it, in the order each layer starts to make sense.

    01Roadmap0 of 18 liveChoose the role you are training for and see the route through the stack.02Foundation7 of 10 liveHow a computer, a process and a filesystem actually behave.03Data Structures0 of 1 liveOrganise data and choose algorithms before scale makes the choice expensive.04Linux10 of 10 liveThe operating system everything above it assumes.05Networking1 of 31 livePackets, routes, DNS and the failures that look like everything else.06Cloud0 of 17 liveSomebody else's computers, with somebody else's failure modes.07Git3 of 8 liveVersion control, and then version control as the source of truth.08CI / CD9 of 21 liveBuild and deliver on every commit, or do it by hand forever.09Containers0 of 12 livePackage the runtime with the app so the target stops mattering.10Kubernetes35 of 35 liveSchedule it, keep it running, reconcile it when it drifts.11OpenShift13 of 13 liveOperate Kubernetes with integrated security, routing and lifecycle controls.12Observability11 of 22 liveMetrics, logs and traces: what the system says about itself.13SRE4 of 25 liveError budgets and incidents — running it, not just shipping it.14Security0 of 31 liveThe layer that is someone else's job right up until it isn't.15Platform0 of 12 liveTurning all of the above into something a team can use.