Skip to project
pedromartins.tech
Applied · 1 item open
Cloud Security / DevSecOps / Detection Engineering2026

TrustStack

Four applied security layers for cloud governance, response, runtime, and supply-chain assurance

A practical reference for platform and security teams: four independently reviewable layers that govern AWS accounts, prove controls work, safely remediate exposure, and prevent unverified workloads from running.

Role
Platform architect, control author, and operator
Environment
Live 5-account AWS organization · real cluster · public repositories
Ownership
Threat models · Terraform · Detection logic · Admission policy · Attack simulations · CI identity · Evidence design
TrustStack architecture connecting AWS governance and automated response to a signed software supply chain and Kubernetes runtime security
truststack-program5 system boundaries
AwLZ
Detections
Response runner
ProvenancePipeline
4 / 4layers applied to live infrastructureOne item open by calendar: a steady-state cost month, earliest September 2026
5.97sattacker API call to revoked ruleMeasured, not estimated - Stratus Red Team detonation
0long-lived cloud or signing credentialsOIDC federation and Sigstore keyless signing
56architecture decision recordsIncluding the two that record being wrong
01 / Interactive system

Move through the system layer by layer

truststack-programLive 5-account AWS organization · real cluster · public repositories · Read-only
Applied · 1 item open
System map

Boundaries and responsibilities

Public or untrustedAuthenticated boundaryInternal-only dependency
READ-ONLYLive 5-account AWS organization · real cluster · public repositoriestool: systemselection: org

AwLZ selected.

02 / Layers

Four repositories, one claim each

Every layer states what it proved, how, and where it stops.

Cloud governance

AwLZ

Can an account do something nobody watches?

A multi-account AWS landing zone written as Terraform. Most "AWS security" portfolios harden a single account; real organizations fail at the boundary - an account nobody governs, a region nobody watches, an access key in a CI runner. This builds the boundary itself as reproducible code, and root credentials are deleted from every member account.

6Terraform stacks, 10 modules
5accounts in two OUs
4service control policies
477checkov checks passed, 0 failed
  • Organizations, two OUs, five accounts, centralized root access
  • Four SCPs, each probed from inside a member account while holding administrator there
  • Organization CloudTrail with a customer-managed key and an Object Lock COMPLIANCE archive in a separate account
  • GuardDuty, Security Hub, Config and Access Analyzer delegated to the security account
  • GitHub OIDC: a read-only plan role, an apply role behind a protected environment, and no static keys anywhere
Proved by
All six stacks applied to a live five-account organization. A denied API call in a forbidden region and a denied StopLogging, captured from an account administrator - the only privilege level at which an SCP test proves anything. Five Config recorders reporting SUCCESS is what proves the cross-account delivery path works end to end.
Stated limit
Cost actuals closed against the first billing window: USD 1.48 of usage in July, entirely covered by credits, and a USD 6.38/month run rate in August once they stopped. Measuring it took two attempts - grouping Cost Explorer by service without filtering the record type nets credits into the same row as usage, and every service reads zero. Still open, by calendar rather than work: GuardDuty and Security Hub sit inside 30-day trials until late August, so September 2026 is the first month that can honestly be called a steady state.
Source
Detection and response

PontoAntiCrack

When something is found, what happens automatically?

Detection-as-code with automated remediation, named after the anti-cheat: the game keeps running, the cheater gets caught and kicked. Anyone can enable GuardDuty. The difference is that the detection is code, has tests, and was proven by attacking the account it protects.

185automated tests
3detections, each a full unit
5.97smeasured time to remediate
17 / 17fixtures agreeing with real EventBridge
  • One runtime, one order: plan, exclusion tag, snapshot, circuit breaker, dry-run gate, apply, audit close, alert
  • Handlers implement plan() and apply() and nothing else, so they cannot get the order wrong
  • Dry run is the default and opting out is explicit
  • Circuit breaker in DynamoDB: five actions per five-minute window, dry runs counted
  • Leaked keys are deactivated and never deleted; root and temporary credentials escalate to a human
Proved by
sg-open detonated with Stratus Red Team: 5.97 seconds from the attacker's API call to the rule being revoked, with the group's unrelated port 443 rule left untouched. Every fixture replayed through aws events test-event-pattern with zero disagreements, and 15 of 17 fixtures captured from real events rather than written from documentation.
Stated limit
Two of the three techniques were never detonated, so no latency is claimed for them. The GuardDuty fixtures are service-generated samples: real type, severity and shape, but no actual compromise produced them. Slack delivery is unproven because no webhook exists - the audit table, not the chat message, is the system of record.
Source
Kubernetes runtime

KateClusters

If something breaks in, does anyone find out?

A kubeadm cluster on a dedicated Debian 13 VM, hardened to the CIS benchmark, with Falco watching syscalls through eBPF and the API server writing an audit log - both shipped into Loki and alerted on in Grafana. A hardened cluster with no detection is a claim; detection with no attack simulation is an unverified claim.

67 → 86CIS PASS
12 → 0CIS FAIL
4attack simulations with raw output
5 / 5CI jobs green on main
  • Pod Security Admission restricted, least-privilege RBAC, no cluster-admin service accounts
  • Calico NetworkPolicy default-deny per namespace, secrets encrypted with SOPS and age
  • Falco with the modern eBPF driver plus custom rules, shipped by promtail into Loki
  • Grafana dashboards, alert rules, and a dead-man's switch that fires when Falco stops reporting
Proved by
The CIS delta is generated from two committed kube-bench runs, each remediation linked to the manifest that fixed it. Four attack write-ups - escape attempt, cryptominer, service-account token abuse, and killing Falco itself - record what was run, what the control did, what fired, and the gap that remained, with raw captured output committed alongside.
Stated limit
One node: hardening and detection, not high availability or multi-tenant isolation. Default-deny covers the five namespaces this repository owns; a newly created namespace is open until it receives policy. One Falco rule cannot fire while egress is already blocked, so prevention and detection are not independent there.
Source
Software supply chain

ProvenancePipeline

Do I know where this container came from?

Build, SBOM, vulnerability scan, keyless signature, SLSA provenance attestation, and admission control that refuses anything unsigned. The deliverable is the rejection, not the pipeline - anyone can show a green CI run, so the artifact is the recorded moment a cluster refuses an image.

4admission cases, one policy
L2SLSA Build level, honestly
0long-lived signing keys
2SBOM formats per image
  • syft SBOMs in SPDX and CycloneDX; grype and trivy failing at CRITICAL with no soft-fail
  • cosign keyless signing through Fulcio, with every entry recorded in the public Rekor transparency log
  • cosign attest for SLSA provenance and the SBOM, bound to the workflow identity that built the image
  • Kyverno verifyImages in Enforce with failurePolicy: Fail, matching signing identity and issuer
Proved by
Four images against one policy: ours by digest, signed by the release workflow, is admitted. An ordinary unsigned image is denied. A genuinely Sigstore-signed image with a genuinely public Rekor entry is denied, because the identity is not ours - a policy that only asked "is this signed" would have admitted it. Ours by tag instead of digest is denied.
Stated limit
SLSA Build L2, not L3, and the gap is real: the provenance predicate is assembled by a step in the same job that builds the image, so anything compromising that job could write its own provenance and have it signed. There is no automated Rekor monitoring, and enforcement currently depends on the repository staying public.
Source
03 / Context

The problem behind the system

Anyone can apply a Terraform module or install a Helm chart. The harder question is whether the control does what its documentation says, and how anybody else would know. TrustStack answers it four times: the deliverable is not working infrastructure, it is reproducible infrastructure plus evidence - including the evidence that came back unflattering.

The four layers are the four places trust is usually asserted instead of proved: an account nobody governs, a container whose origin nobody checked, a runtime nobody watches, and a finding nobody acts on. Each layer is an independent repository with its own threat model, its own gates, and its own docs/evidence/ directory, so it can be reviewed alone.

That rule has a cost, and the cost is the point. Three of the four layers had to publish a result less flattering than the one originally advertised, and those corrections are the most useful thing here.

Read the full project overview

TrustStack is four public repositories that each answer one question about a system's trust: can an account do something nobody watches (AwLZ), do I know where this container came from (ProvenancePipeline), if something breaks in does anyone find out (KateClusters), and when something is found what happens automatically (PontoAntiCrack). All four are applied to real infrastructure - a live five-account AWS organization, a real kubeadm cluster, and a signed release path into GHCR - and all four meet the same definition of finished, agreed before any of the work started: code applied, evidence artifacts committed, a recorded demo, a threat model whose residual risks are stated rather than implied, and a public repository.

THE RULE, AND WHAT IT COSTS - The deliverable is not working infrastructure. Anyone can apply a Terraform module or install a Helm chart. The deliverable is reproducible infrastructure plus evidence that the controls do what the documentation claims, including where they do not. The part worth reading in each repository is docs/evidence/, and the part worth reading there is the entries recording a control failing, a claim being wrong, or a number coming back worse than promised. That rule has a price: three of the four layers had to publish a result less flattering than the one originally advertised.

AWLZ - CLOUD GOVERNANCE - Six independent Terraform stacks and ten modules build a multi-account landing zone: two organizational units, five accounts, four service control policies, an organization CloudTrail whose Object Lock archive lives in a separate log-archive account under its own key, and GuardDuty, Security Hub, Config and Access Analyzer all delegated to a security account. Root credentials are deleted from every member account, so the only interactive way in is IAM Identity Center, and there are no static AWS access keys anywhere - CI plans through GitHub OIDC with a read-only role and applies through a role gated behind a protected environment. Every SCP was probed from inside a member account while holding administrator there, because an SCP is the only mechanism that can deny an account administrator and a test as anything less privileged would prove nothing.

PONTOANTICRACK - DETECTION AND RESPONSE - Detection-as-code with automated remediation, named after the anti-cheat: the game keeps running, the cheater gets caught and kicked. Three detections - public S3 exposure, leaked IAM keys, world-open security groups - each a complete unit of EventBridge pattern, Lambda handler, scoped execution role, fixtures, and tests for both what it catches and what it must not. Every detection runs through one runtime, and the order is the security property: plan read-only, honour the exclusion tag, snapshot to the audit table, check the circuit breaker, pass the dry-run gate, apply, close the audit record, alert. Handlers implement plan() and apply() and nothing else, so they cannot get the order wrong. 185 tests, and a measured 5.97 seconds from an attacker's API call to the offending rule being revoked.

KATECLUSTERS - KUBERNETES RUNTIME - A kubeadm cluster on a dedicated Debian 13 VM, hardened against the CIS benchmark from PASS 67 / FAIL 12 to PASS 86 / FAIL 0, with each remediation linked to the manifest that fixed it. Pod Security Admission at restricted, least-privilege RBAC, Calico default-deny networking, SOPS-encrypted secrets, and Falco watching syscalls through eBPF while the API server writes an audit log - both shipped by promtail into Loki, with Grafana dashboards, alerts, and a dead-man's switch that fires when Falco stops reporting. Then attacked on purpose four times: container escape, cryptominer, service-account token abuse, and killing Falco itself, each with the raw captured output committed alongside the write-up.

PROVENANCEPIPELINE - SOFTWARE SUPPLY CHAIN - Build, syft SBOMs in two formats, grype and trivy failing at CRITICAL with no soft-fail, push to GHCR, cosign keyless signature through Fulcio recorded in the public Rekor transparency log, then SLSA provenance and SBOM attestations - and a Kyverno policy in Enforce with failurePolicy: Fail that refuses anything it cannot verify. The deliverable is the rejection, not the pipeline: four images, one policy. Ours by digest is admitted. An unsigned image is denied. A genuinely Sigstore-signed image with a genuinely public Rekor entry is denied because the identity is not ours - a policy that only asked "is this signed" would have let it in. Ours by tag instead of digest is denied.

WHAT PROVING IT ACTUALLY FOUND - AwLZ's promised CIS before-and-after was not obtainable, so a control experiment ran on the throwaway account instead: measure with the guardrails, detach them from that account only, measure again, reattach. All 35 controls were identical in both states, because no CIS v3.0.0 control reads a service control policy - the original claim was measuring the wrong thing. PontoAntiCrack was fully tested and completely blind twice: a loop guard that matched on the principal ARN could be bypassed by anyone passing --role-session-name pac-anything, and a pattern built entirely from AWS CLI fixtures had never seen the legacy encoding of the same API, so port 22 was opened to the internet and the rule did not match. ProvenancePipeline's design assumption about which attestation the cluster can enforce turned out to be exactly backwards, and the correction is recorded as ADR-009 with the original left visible.

HONESTY, MECHANISED - Rules that are only written down stop being true within a month. Every fixture carries a provenance marker and the command that would capture the real event, and a test fails if the marker is missing or contradicts the detection metadata - that gate fired three separate times in the final session and was right every time. CI fails any pattern test with no negative assertion, because a pattern tested only for what it catches matches everything, forever, silently. Every scanner suppression carries an inline justification. Third-party CI actions are pinned to commit SHAs, not tags. PontoAntiCrack's evidence directory stayed empty for the entire build, with a README saying what would land in it, because a table of realistic-looking latency numbers would have been trivial to write and would have made every other number in the portfolio worthless.

COST - A hard shared ceiling of USD 20/month across the two AWS layers, with budget alerts at 85% and 100% of actual plus 100% of forecast; the cluster and the pipeline cost nothing, running on a local VM and free-tier CI. Security Hub was the lever: CIS v3.0.0 in all five accounts projected USD 24.77/month, over the ceiling, so the five-account configuration was kept alive only long enough to capture the control experiment and then reduced to the delegated security account - and what was lost is stated: there is no longer a live per-account CIS score.

WHAT THE FIRST CLOSED BILLING WINDOW SAID - The projection was USD 13.73/month. The measured run rate is USD 6.38, and the interesting part is not that it came in under. Measuring it correctly took two attempts. Grouping AWS Cost Explorer by service without filtering the record type sums credits into the same row as usage, so every service in the organization reported USD 0 - and a Config recorder that was healthily recording 325 configuration items was indistinguishable from a broken one. Filtered to usage only, July's real cost was USD 1.48 and credits covered all of it. The invoice said zero. Reporting that number alone would have understated the model by the entire amount it existed to measure.

Three lines of the model were wrong, and the reasons are worth more than the figures. KMS matched to the cent. Config was over-projected threefold because the estimate extrapolated a deployment day across a month - Config bills per configuration item recorded and per rule evaluation, and the boundary rules are change-triggered, so a month in which nothing changes bills nothing. Config cost tracks what an organization does, not how long it exists. PontoAntiCrack's nine CloudWatch alarms billed zero because nine falls inside a ten-alarm free tier the estimate never applied - which makes the tenth alarm free and the eleventh a cliff, so the next detection carries a cost the original table could not have predicted. And Secrets Manager was missing from the model entirely at USD 0.40/month: invisible rather than estimated low, which is the worse of the two failures.

One item stays open, and it is a calendar rather than work. GuardDuty and Security Hub are both inside 30-day trials until late August, so no month measured before September 2026 is a steady state, and the repository says so instead of quoting the low number.

04 / Findings

What proving each layer actually found

Every one of these was invisible to a green test suite.

AwLZ

The headline evidence came back negative

What it looked like
The README promised a CIS benchmark score before and after the guardrails - the obvious way to show that service control policies improve an account.
What was actually true
No before existed, so the honest replacement was a control experiment on the throwaway account: measure it with its SCPs, detach them from that account only, measure again, reattach. All 35 controls came back identical in both states. No CIS v3.0.0 control reads a service control policy at all.
What it generalizes to
A benchmark score describes resource configuration; it cannot describe a preventive guardrail. What does show the SCPs working is behavioural - a denied call in a forbidden region, allowed during the experiment window, denied again afterwards. Both probes name resources that do not exist, so nothing was created or destroyed to produce the evidence.
AwLZ

The first cost actual said zero, and zero was wrong

What it looked like
The July billing window closed and Cost Explorer finally returned Estimated: false. Grouped by service, every single line came back at USD 0.00 - CloudTrail, Config, KMS, S3, all of it. A five-account organization with four customer-managed keys and five Config recorders appeared to cost nothing.
What was actually true
Cost Explorer sums the Credit record type into the same row as Usage unless the query filters on it. Filtered to usage only, July cost USD 1.4830 and credits covered exactly that much, then stopped at the month boundary with no signal anywhere. Config alone was USD 0.9970, reconciling to the cent against 325 configuration items and 22 rule evaluations at the published São Paulo rate. The recorder was healthy the whole time - and under the unfiltered grouping, a healthy recorder and a broken one produce the identical number.
What it generalizes to
Publish gross, credit, and net - never only the invoice. The net is what was paid; the gross is the only figure that says anything about the design. The same pass corrected three lines of the model: Config was over-projected threefold because the estimate extrapolated a deployment day across a month, when Config bills what an organization does rather than how long it exists; nine CloudWatch alarms billed nothing because they sit inside a ten-alarm free tier, which makes the eleventh a cliff rather than a cut; and Secrets Manager was absent from the model entirely, which is worse than being estimated low. The superseded projection stays in the repository beside the actual, because the delta is the evidence.
PontoAntiCrack

A loop guard the attacker gets to choose

What it looked like
174 tests green, a coverage gate rejecting any pattern without a negative assertion, every fixture replayed through EventBridge itself, and 14 of 15 fixtures captured from real CloudTrail.
What was actually true
The runtime dropped events whose principal ARN contained pac-, so a remediation could not re-trigger its own detection. But an assumed-role ARN ends in a session name the caller sets freely on every AssumeRole. Stratus happened to run under a session called pac-terraform, so the detection classified a live attack as its own automation and wrote a confident SKIPPED.
What it generalizes to
A security decision made from a field the adversary sets is not a control. Ask it of any predicate: who writes this string? The guard now compares the role-name segment only, and the session name is used for nothing.
PontoAntiCrack

A pattern that knew one of two encodings

What it looked like
176 tests passing, after the first defect was already found and fixed.
What was actually true
AuthorizeSecurityGroupIngress is recorded two ways. The AWS CLI produces nested ipPermissions; a caller using the legacy top-level parameters produces an empty ipPermissions with cidrIp directly on requestParameters. Every fixture had been produced by the CLI, so the pattern had only ever seen one shape. Port 22 was opened to the internet and the rule did not match.
What it generalizes to
Fixtures inherit the bias of whatever produced them - two tools calling the same API are two sources of truth. What kept it from being worse: the handler reads the resource back instead of trusting the event, so a missed encoding was a missed detection rather than a wrong remediation.
KateClusters

The posture claim was broader than the mechanism

What it looked like
The repository described default-deny networking, and the manifests genuinely implement it.
What was actually true
Kubernetes NetworkPolicy is namespaced. The five namespaces this repository owns were covered; a newly created namespace was fully open. Separately, one Falco rule cannot fire while egress is already blocked, because it needs an established socket.
What it generalizes to
The documents now say exactly that, and cluster-wide default-deny is recorded as the closing condition rather than implied as done. Prevention and detection are not independent there, and pretending otherwise would have double-counted a control.
ProvenancePipeline

The assumption was exactly backwards

What it looked like
ADR-005 stated that the cosign attestation is the copy the cluster enforces, and GitHub's is the one it cannot see.
What was actually true
On GHCR it is precisely the reverse. Only building the cluster half of the system revealed it - nothing in the pipeline half could have.
What it generalizes to
The correction is recorded in place as ADR-009, with the original left visible and a warning at the top of the handoff naming the paragraph to distrust in any stale copy. A superseded decision is more useful than a rewritten one.
05 / Security

Threats, controls, and what remains

No control is presented as total risk elimination.

Governance

An account administrator disables logging

Control
The protect-security-services SCP, split so destructive actions have no exemption at all while the actions deployment needs exempt only the deployment principals.
Residual risk
An SCP is the only thing that can deny an account administrator - but no CIS control can see one, so the benchmark score says nothing about it.
Evidence

Log tampering or deletion

Control
A separate log-archive account holding its own key, with the organization trail written into S3 Object Lock in COMPLIANCE mode - not deletable even by the account root.
Residual risk
Retention is time-bound; the objects become deletable again once it expires.
Identity

CI credential theft leads to account takeover

Control
GitHub OIDC everywhere, no static AWS access keys in any repository, role trust pinned to exact sub claims, apply gated behind a protected environment.
Residual risk
A compromised workflow file still runs as that workflow; branch protection and review are the control there.
Response

The automation becomes the incident

Control
Dry run is the default, the snapshot is written before mutation, one scoped role per detection, and a circuit breaker capped at five actions per five-minute window - dry runs counted.
Residual risk
A wrong plan inside allowed bounds is still wrong. Two of the three techniques were never detonated, so no latency is claimed for them.
Detection bypass

A control decides from a field the attacker writes

Control
The loop guard compares the role name only. The session name segment of an assumed-role ARN is chosen freely by the caller and is now used for nothing.
Residual risk
Every predicate over event data needs the same question asked of it: who writes this string?
Supply chain

A signed image that is not ours reaches the cluster

Control
Kyverno verifyImages matching both signing identity and issuer, in Enforce with failurePolicy: Fail, and images pinned by digest rather than tag.
Residual risk
SLSA Build L2, not L3: the provenance predicate is assembled by a step in the same job that builds the image. There is no Rekor monitoring.
Runtime

Container escape or a miner in a compromised workload

Control
Pod Security Admission restricted, no privileged pods or host mounts, default-deny egress in repo-owned namespaces, Falco watching syscalls through eBPF.
Residual risk
NetworkPolicy is namespaced, so a newly created namespace is open until it receives policy - and the mining-port rule cannot fire while egress is already blocked.
06 / Decisions

The trade-offs that shaped the build

Decision

Evidence is the deliverable

A configured control is not a verified control.

Reason
Applying a module proves the provider accepted it, nothing more. The entries worth reading are the ones recording a control failing or a claim being wrong.
Trade-off
Progress looks slower, and an empty evidence directory has to stay empty until something is actually measured.
Revisit when
Never - only the format should evolve.
Decision

SCPs attach to OUs, never the organization root

A bad policy must not be able to lock out the management account.

Reason
A guardrail that can strand its own author is an availability incident waiting for a typo.
Trade-off
A new account is ungoverned until it is placed in an OU, so account creation and OU placement are one step.
Revisit when
Only with a tested break-glass path that does not depend on the policy being edited.
Decision

Coarse pattern, precise handler

The event router matches broadly; the handler reads the resource back.

Reason
An EventBridge pattern cannot evaluate a port range or parse a policy document, so trusting it to be exact builds a wrong remediation.
Trade-off
More Lambda invocations that decide to do nothing.
Revisit when
Only if invocation cost ever outweighs the safety it buys - it does not at this volume.
Decision

Deactivate credentials, never delete them

A leaked access key is set Inactive, and root escalates to a human.

Reason
Deleting a key destroys its last-used record, which is often the only evidence of what the attacker actually reached.
Trade-off
The credential still exists and has to be cleaned up deliberately afterwards.
Revisit when
Only if a retention system captures last-used data independently first.
Decision

Claim SLSA Build L2, not L3

Meet a level honestly instead of implying the next one.

Reason
L2 is met: hosted builder, provenance signed with the workflow's own OIDC identity, public Rekor entries, and a consumer-side check that pins identity and fails closed. L3 is not, and the gap is real.
Trade-off
The lower number is what appears on the page, next to competitors' louder claims.
Revisit when
When provenance generation moves to a context the build job cannot tamper with.
Decision

Keep Security Hub in one account and say what was lost

The benchmark in all five accounts cost more than the ceiling allowed.

Reason
CIS v3.0.0 across five accounts projected USD 24.77/month against a USD 20 ceiling; the delegated security account alone projected USD 13.73. The first closed billing window later measured the reduced configuration at USD 6.38, so the decision was right for a reason the projection got partly wrong - see the cost finding.
Trade-off
There is no longer a live per-account CIS score, and the page says so rather than showing a stale one.
Revisit when
If the budget rises, or if per-account scoring becomes evidence for something a behavioural test cannot cover.
Decision

Report gross cost, credit, and net - never only the invoice

The first closed billing window said USD 0. The architecture cost USD 1.48 and a credit covered it.

Reason
Grouping AWS Cost Explorer by service without filtering the record type sums credits into the same row as usage, so every service reads zero and a healthy Config recorder is indistinguishable from a broken one. The net is what was paid; the gross is the only figure that is evidence about the design.
Trade-off
Three numbers to publish instead of one, and the honest one is the larger.
Revisit when
Never - this is the reporting rule, and the wrong answer came first precisely because it was not being followed.
07 / Implementation

How the decisions appear in the build

remediations/common/runtime.py
01# Condensed from PontoAntiCrack. Handlers implement plan() and apply() and02# nothing else, so a handler cannot get this order wrong.03def execute(detection, raw_event, config, aws, notifier) -> Outcome:04    event = event_parser.parse(raw_event)05    audit = AuditLog(aws.table(config.table_name), config.detection_id)06 07    # 1. Loop prevention - compared against the ROLE NAME only. An earlier08    #    version tested `"pac-" in arn`, and an assumed-role ARN ends in a09    #    session name the CALLER chooses: anyone passing10    #    --role-session-name pac-anything was classified as our own11    #    automation and skipped. Found by detonating a real technique.12    if event.principal.is_pac_automation():13        return _skip(config, "triggered by this system's own remediation role")14 15    # 2. Plan is read-only. None means the resource state is not dangerous.16    plan = detection.plan(event, aws)17    if plan is None:18        return _skip(config, "resource state is not dangerous")19 20    # 3. Snapshot BEFORE any mutation. Everything after this line may fail21    #    without destroying the rollback source or the incident evidence.22    key = audit.open(event, plan, dry_run=config.dry_run)23 24    # 4. Circuit breaker: 5 actions per 5-minute window, dry runs counted.25    state = CircuitBreaker(...).check_and_increment()26    if state.open:27        return _close(audit, key, Status.BLOCKED, plan, state.reason)28 29    # 5. Dry run is the DEFAULT. Opting out is explicit, per detection.30    if config.dry_run:31        return _close(audit, key, Status.DRY_RUN, plan, plan.reason)32 33    # 6. Apply. A failure here still closes the audit record and alerts -34    #    a failed remediation must be visible, not a crashed Lambda.35    actions = detection.apply(plan, aws)36    return _close(audit, key, Status.APPLIED, plan, plan.reason, actions)
Representative / sanitized excerpt

The pipeline every PontoAntiCrack detection runs through. Order is the security property, so it is written once, in one place - and handlers never call the audit log, the breaker, or the notifier themselves.

TerraformAWS Organizations / SCPCloudTrail + Object LockGuardDutyPython / LambdaEventBridgeDynamoDBKubernetes (kubeadm)KyvernoFalco / eBPFLoki + GrafanaCosign / SigstoreSLSAGitHub OIDC
08 / Operations

The unhappy path is part of the design

01

What if the response automation loops on itself?

A remediation's own API calls are CloudTrail events that match its own pattern. The runtime rejects its own remediation role, and the circuit breaker caps everything else at five actions per five-minute window.

Signal
BLOCKED records appear in the DynamoDB audit table with the breaker's reason attached.
Response
The breaker was tripped on purpose during verification: it held after five actions, wrote four BLOCKED records, and left four ports deliberately open rather than acting past its limit.
02

What if a guardrail blocks its own deployment?

It did, twice. The service-protection SCP denied the very apply that manages the services it protects.

Signal
Terraform apply fails with AccessDenied naming the SCP, from a principal that holds administrator.
Response
ADR-014: the policy exempts its own deployment principals for the actions deployment needs, and keeps no exemption at all for destructive actions such as StopLogging or DeleteDetector.
03

What if the runtime detector stops reporting?

Silence from a detector looks identical to a quiet system, so a dead-man's switch in Grafana fires when Falco stops sending.

Signal
Measured at 24 minutes 36 seconds - not the 15 minutes originally estimated, and the document says the measured number.
Response
The cluster runs on a workstation VM, so the alert also fires on every shutdown. That is correct behaviour and is deliberately not tuned away.
04

What if the admission webhook is unreachable?

The Kyverno policy runs with failurePolicy: Fail, so an unavailable policy engine rejects images rather than admitting them.

Signal
Admission denial events on every new pod while the enforcement plane is down.
Response
Restore the policy plane before deploying. Failing open would turn one outage into an unsigned-image window.
05

What if the guardrails lock everyone out?

A documented break-glass role exists, and using it is itself an alertable event.

Signal
A CloudWatch alarm on assumption of the four exact recovery-role ARNs - observed firing, not assumed.
Response
Its SNS topic uses a dedicated customer-managed key, because the AWS-managed alternative accepts no key policy and this alert is precisely what an attacker would want to read or suppress.
09 / Evidence

Proof, source, and inspectable outcomes

10 / Results

What the project demonstrates

Four layers applied to real infrastructure, each with evidence a stranger can read without running anything

5.97 seconds from a real attack technique to the offending security-group rule being revoked

CIS PASS 67 to 86 and FAIL 12 to 0 on a kubeadm cluster, every remediation linked to the manifest that fixed it

A cluster that refuses a genuinely signed image because the signature is not ours

Three advertised claims corrected in public instead of quietly dropped

A cost model measured against its first closed billing window and corrected in three places, with the superseded projection kept beside it

A published program-level threat model whose main output is eight threats no component model could see - and the five of them that still have no control

One item open by calendar rather than work - a month outside two service trials, earliest September 2026

What worked
  • Detonating a control found two defects that a green test suite could not - 174 tests passing when the first surfaced, 176 when the second did.
  • Stating the order of the response pipeline once, in one place, stopped handlers from ever getting it wrong.
  • Writing each limit in the strong form is what makes the remaining numbers believable.
Next iteration
  • Detonate s3-public and iam-key-leak and measure their latency
  • Cluster-wide default-deny with a Calico GlobalNetworkPolicy
  • Move provenance generation to an isolated context for SLSA L3
  • Assert the live Kyverno policy still exists, from the repository that owns the cluster
  • Gate image signing on the same protected environment as Terraform apply
  • Read a September billing window, the first outside two service trials
Professional signal
  • Cloud security engineering on live AWS, not a sandbox
  • Detection and response written as tested code
  • Kubernetes hardening with runtime detection that was attacked on purpose
  • Supply-chain provenance enforced at admission