PontoAntiCrack halfway through: 167 tests and no permission to remediate

PontoAntiCrack has 167 passing tests.

It still does not have permission to remediate the AWS lab.

Those statements are not in conflict. The tests show that the code behaves consistently against the inputs I have given it. They do not prove that AWS will emit exactly those inputs, that the EventBridge patterns will match them in the deployed account, or that the planned mutations are safe against real resource state.

This is the point where a project can look complete in GitHub while remaining operationally unproven.

What the test suite covers

The shared runner now enforces the sequence I planned at the start.

A detection builds a plan. The exclusion tag is checked. The before state is written to DynamoDB. The circuit breaker evaluates recent volume. Dry-run prevents mutation. Enforcement calls apply(). The audit entry receives the final result. Slack is notified without receiving sensitive data.

The suite checks failure paths as well as the successful path:

1. A snapshot write failure prevents remediation.

2. An open circuit breaker prevents remediation.

3. A dry-run event still increments the breaker count.

4. A notification failure does not erase the remediation result.

5. A benign plan returns without creating a misleading "remediated" record.

6. An ambiguous case can finish as ESCALATED.

7. Reprocessing the same event does not create uncontrolled duplicate changes.

The handler-specific tests cover S3 policies, Security Group permissions, and credential findings. Positive fixtures are the smaller part. Most of the test count comes from boundaries and failure behavior.

sg-open is the cleanest detection

Security Group rules have edge cases, but the unsafe slice is usually visible.

If a permission exposes SSH to 0.0.0.0/0 or the IPv6 equivalent, the plan identifies that public range. If the same permission also includes a private CIDR, the private range stays.

Open HTTPS does not automatically become a remediation target. Neither does SSH restricted to the intended internal range.

The planned change is narrow because deleting the entire permission would be easy and wrong. A response tool should not turn a security finding into an outage by removing unrelated access.

I now test the exact before and after structures rather than checking only that an AWS revoke call occurred. A mocked API success is weak evidence if the proposed arguments are overbroad.

S3 is still the difficult one

My early S3 classifier treated the presence of a condition too generously.

That approach avoided false positives for organization-restricted policies, but it also created an obvious blind spot. A condition that requires HTTPS does not restrict who can use the bucket. Calling the policy non-public because a Condition block exists would be a false sense of safety.

The current version recognizes the restrictive cases it knows, including the organization boundary used by this lab. Conditions that affect transport or do not narrow principal access do not automatically make a wildcard principal safe.

Unknown combinations are escalated.

I could keep adding condition keys until the code resembles a partial IAM policy engine. I do not think that is a good direction for this Lambda. AWS policy evaluation has enough complexity that an incomplete reimplementation can become more dangerous than a conservative detector.

The automation needs a defined competence boundary. Outside it, a human gets the snapshot and the reason.

Credential findings need two kinds of restraint

The IAM handler filters for relevant resource types and severities before planning a response.

A finding that happens to come from the same AWS service but concerns another resource should not touch credentials. A low-value informational event should not become an urgent containment action because the EventBridge source matched.

The second restraint is root.

Standard IAM access keys follow their configured containment path. Root findings are recorded as escalations with no automatic root mutation. That rule is enforced in the handler and covered by negative tests.

This produces a less impressive demo than "the system automatically fixes every credential incident." It is a more defensible design.

The circuit breaker is not an error handler

The breaker exists for valid code receiving a dangerous amount of work.

Suppose an EventBridge pattern is broader than expected and starts sending thousands of events. Every invocation can parse correctly. Every API call can succeed. The combined effect can still be disastrous.

The breaker counts planned remediations in a time window. Once the configured limit is reached, later events keep their audit path but do not reach apply().

Dry-run contributes to the count so I can observe whether a proposed deployment would have opened the circuit. Waiting until enforcement to discover the rate would defeat the point.

I have tested the state transitions locally. The AWS test still needs to show how the chosen values behave with actual event timing.

Terraform is valid, AWS is not yet involved

The Terraform configuration validates. Static analysis, formatting, linting, and security scans pass.

I have not applied it.

The next step is terraform plan against the Lab account and the actual roles created for this project. I want to inspect the IAM policies from the AWS perspective, not just the source files.

Each detection role should be limited to the reads and writes used by that handler. The common audit and notification permissions are shared requirements. S3 remediation does not need EC2 permissions. Security Group remediation does not need IAM credential permissions.

After plan review, the first apply will keep dry-run enabled.

Documentation fixtures are a hypothesis

The event fixtures came from AWS documentation and the service schemas I could inspect offline.

They have been useful. They are also synthetic.

CloudTrail event details can vary with the API path, request form, and service behavior. GuardDuty findings include fields that may be present, absent, or nested differently across finding types. EventBridge pattern semantics need to be checked by the service that will evaluate them.

I have a capture checklist ready for each lab scenario. The captured events will be stored with sensitive values removed, and the fixtures will be updated only after I compare them field by field.

If the real event disagrees with the test, the real event wins.

The next decision

The project is at a useful pause.

The internal architecture is far enough along that deploying it will test the design rather than a stub. It is also still restricted enough that an unexpected match cannot modify the lab.

My next sequence is:

1. Review the Terraform plan and generated IAM policies.

2. Apply with dry-run enabled.

3. Trigger each scenario in the AwLZ Lab account.

4. Capture the real events and compare them with the fixtures.

5. Test the EventBridge patterns through AWS.

6. Observe the audit records, breaker counts, and Slack summaries.

7. Enable one remediation at a time.

The number 167 is satisfying. It is not the finish line.

The finish line is a real AWS resource changing only when it should, with its original state already stored somewhere else.