AwLZ is complete: what changed after building a real AWS landing zone

AwLZ is finished.

All six Terraform stacks have been applied to the real AWS organization. The account structure exists, the organization trail is delivering logs to the archive account, the security services are centralized, the guardrails have been exercised, and GitHub can plan and apply without a stored AWS access key.

There are still follow-up items. "Finished" means the core scope is implemented and tested, not that AWS has run out of services I could add.

This is the first TrustStack project to cross that line.

What exists now

The organization is split into management, security, logging, and workload responsibilities.

The management account owns AWS Organizations and the administrative path. Log Archive receives organization-wide CloudTrail data in an S3 bucket protected by KMS and Object Lock. Security is the delegated administrator for GuardDuty, Security Hub, AWS Config, and IAM Access Analyzer. Dev and Lab sit in the Workloads OU under the organization guardrails.

Terraform state has its own protected S3 and KMS foundation. The CI path uses GitHub OIDC with separate plan and apply roles. The plan role cannot quietly become an apply role just because both jobs run in the same platform.

That is the infrastructure summary. The more important part is what I can now demonstrate.

The tests that made the project real

I ran the guardrail checks from the contexts they were designed to restrict.

A member account administrator cannot simply ignore an organization deny. Attempts to interfere with protected audit controls fail even when the local role would otherwise have broad permissions.

CloudTrail events from member accounts appear in the separate archive. A retained object cannot be removed before its retention period expires. The workload account that caused an event does not control the evidence store that receives it.

The delegated security account sees the organization-level service state. The member accounts are enrolled under centralized administration rather than each maintaining unrelated settings.

The CI workflow receives short-lived AWS credentials through OIDC. There is no permanent access key in the GitHub secrets page waiting to be copied, forgotten, or reused somewhere else.

These tests are recorded under docs/evidence. I kept the failures that changed the design, including the applies blocked by my own SCPs.

The SCP mistake was the best part

Twice, a guardrail prevented the deployment from configuring the environment.

I could present that as an implementation bug I eventually removed. That would miss the point.

The policy had real authority. It blocked a call even though the role making the call had high IAM permissions. The failure forced me to separate a dangerous outcome from an API action that can also participate in valid configuration.

The correction was not "let Terraform bypass security." It was a narrower model of which actions were destructive, which maintenance operations were necessary, and which principal context was acceptable.

This is the sort of thing a static policy review can miss. The JSON can look reasonable. The actual service sequence decides whether it works.

What I would change if I started again

I would define the evidence plan earlier.

At the beginning, I treated evidence as documentation to collect after the infrastructure existed. By the middle, it was clear that evidence changes the implementation. If I cannot state how a control will be tested, the control is probably underspecified.

I would also prototype service delegation in smaller slices. GuardDuty, Security Hub, Config, and Access Analyzer share the phrase "centralized security service," but their organization workflows are not interchangeable. I spent time trying to make their Terraform structure look more uniform than their behavior.

Finally, I would add expected cost categories to the design document before the first apply. Cost is part of secure operations. A control that is disabled later because nobody understood its ingestion volume did not have a sustainable design.

What remains open

I am leaving a small, visible follow-up list:

1. Let Security Hub stabilize long enough to record a useful CIS posture snapshot.

2. Reconfirm GuardDuty enrollment from the organization and member views.

3. Collect a complete billing cycle and publish the real lab cost.

4. Add permission boundaries for selected workload roles.

5. Add a few operational alarms that were outside the original core scope.

6. Record the short final demonstration.

None of those items invalidates the foundation. Hiding them would make the project look more complete and the handoff less useful.

What AwLZ taught me

Landing zones are often shown as hierarchy diagrams. Those diagrams are helpful, but they make the work look like account organization.

The difficult part is authority.

Who can change the trail? Who owns the KMS key? Which account can read a finding? Which policy wins when a local administrator and the organization disagree? Which identity is allowed to deploy a change? What evidence survives if the workload account is fully compromised?

Terraform expresses the answer, but it does not choose the answer for me.

AwLZ also changed how I think about "administrator." Inside a member account, an administrator is still operating within an organizational boundary. That is the intended design, not a limitation to work around.

Moving from cloud accounts to a cluster

The next TrustStack project is KateClusters.

This changes the subject from AWS organization governance to Kubernetes control plane and runtime security. The same principle carries over: configuration does not count as proof.

I will build the cluster with kubeadm, capture a CIS baseline before hardening it, then work through audit logging, encryption at rest, Pod Security Admission, RBAC, NetworkPolicy, Falco, Loki, Grafana, and four attack scenarios.

The lab will be a single node. I won't pretend that proves high availability or tenant isolation. It does give me ownership of the API server configuration, scheduler, controller manager, kubelet, audit policy, and encryption provider.

AwLZ started with a state bucket. KateClusters starts with a Debian VM and a snapshot taken before kubeadm.

For the first time in a few weeks, no SCP can block me.

I am sure Kubernetes will find another way.