KateClusters is finished, including the test that attacks Falco

The last KateClusters test was not aimed at an application.

It was aimed at Falco.

If a runtime detector produces an alert when a container does something suspicious, that is useful. If an attacker can stop the detector and the monitoring stack interprets the silence as a quiet day, the system has a large blind spot.

The falco-kill scenario exists for that reason. Preventive controls limit who can interfere with the detection components. The separate heartbeat alert checks whether the expected signal is still arriving.

The test completed, the signal disappeared, and Grafana reported the missing heartbeat. That was the final piece I wanted before calling KateClusters complete.

What the cluster now enforces

The control plane is configured with audit logging and encryption at rest for Secrets. The API server, scheduler, controller manager, and kubelet settings have been reviewed against the selected CIS guidance for this lab. Sensitive files on the host use restricted ownership and permissions.

Namespaces carry Pod Security Admission labels. The default posture for application namespaces is restricted. Workloads run as non-root, disable privilege escalation, drop unnecessary capabilities, and use seccomp. Exceptions are explicit and temporary.

Service Accounts are created per workload. Token automounting is off by default. Workloads that need API access receive projected tokens and narrow RBAC permissions.

Calico enforces NetworkPolicy. Application namespaces begin with denied ingress and egress, then receive specific allowances for DNS, telemetry, and required application traffic.

Falco watches runtime behavior through eBPF. Kubernetes audit events and Falco output are sent into the logging path. Grafana presents the signals and checks for both suspicious events and missing expected events.

That is the configuration. The attacks are what made it a security project.

escape-attempt

The first path tested admission.

A privileged workload with host-oriented capabilities did not pass the restricted namespace policy. The API denied it before the container could start, and the denial appeared in the audit record.

The runtime portion used a separately controlled scenario to exercise behavior that Falco should observe. Shell and namespace-related activity produced the expected events without turning the cluster into a permanently privileged playground.

This distinction matters. If the admission control works, there may be no runtime event because there is no process. The evidence should say "blocked before execution," not "Falco missed it."

cryptominer

The miner simulation did not run mining software. It reproduced selected process and network indicators in a controlled workload.

The workload could start under its permitted pod security context. Its unauthorized egress path could not reach the destination because the namespace policy did not allow it. Falco recorded the relevant process behavior.

The result showed two different controls doing different jobs:

1. NetworkPolicy limited where the workload could communicate.

2. Runtime detection reported what the workload attempted to do.

Blocking the connection did not make the process harmless. Detecting the process did not make the connection impossible. Keeping those claims separate made the evidence clearer.

sa-token-abuse

The default workload had no automatically mounted Service Account token to steal.

The test workload that legitimately required API access received a projected token. It could perform its allowed read. Attempts outside the Role returned 403 responses, and the audit log attributed them to the correct Service Account.

Falco also observed access to the token path during the abuse scenario.

I originally thought the strongest screenshot would be the sequence of 403 responses. It was actually the correlation across layers: the pod identity, the denied authorization decision, the audit entry, and the runtime file access all referred to the same exercise.

falco-kill

This scenario tested both prevention and detection of detection failure.

The normal application identities could not modify the Falco resources. The controlled test used the specific access needed to interrupt the signal. Once the heartbeat stopped, the dead man's switch moved into alerting state.

I also tested recovery. When Falco returned and the expected events resumed, the alert cleared through the normal evaluation path.

An alert that never resolves is not a useful monitor. It becomes wall decoration after the first incident.

The CIS comparison

The final kube-bench run is stored beside the original baseline.

The comparison is not a claim that every check now passes. Some recommendations do not map neatly to the constraints of a single-node learning cluster, and some choices are documented exceptions. The useful result is that each remaining item has a status and a reason rather than disappearing from the summary.

I am keeping the raw outputs in addition to the condensed comparison. A score is convenient. The individual checks explain what the score means.

What did not fit in this lab

KateClusters does not prove high availability. There is one node and one failure domain.

It does not prove safe multi-tenancy between hostile customers. Workloads share a kernel and the control plane shares the node.

It does not prove that the rule tuning will behave at production event volume. The normal workload set is small, and the attacks are controlled.

It also does not solve software supply-chain trust. Most images are still selected by repository and tag. Pod hardening can limit an image after it starts, but it does not prove where the image came from.

That last gap belongs to ProvenancePipeline later in TrustStack.

Before moving on

The project has reproducible setup steps, the before-and-after benchmark data, manifests, attack scripts, cleanup procedures, audit evidence, Falco events, and alert captures.

More importantly, it contains the boring failures: the DNS policy mistake, the overbroad Falco rules, the workloads rejected because I had relied on defaults, and the early alert that could tell me Falco was quiet but not why.

I learned more from those than from the final dashboard.

The next project is PontoAntiCrack, my AWS detection and response system.

AwLZ gave it an organization, central logs, GuardDuty, and a lab account. KateClusters gave me practice keeping prevention, detection, and evidence separate. PontoAntiCrack now has to make automated changes in AWS without treating every suspicious event as permission to destroy something.

I am starting it in dry-run.

After watching NetworkPolicy take out DNS, I am not giving the Lambda write access on its first day.