pose

Purpose

Untested systems are unvalidated assumptions. The discipline of Test and Evaluation, borrowed from defense acquisition and applied to cybersecurity operations, ensures that systems, processes, and controls perform as designed under realistic conditions before they are exposed to production environments and adversarial pressure.

T&E is not a single gate at the end of a project. It is a continuous discipline that runs alongside development and deployment, and it does not end when a system goes live. Every change, patch, and new component reopens the question T&E exists to answer: does this actually work the way we assumed it would, under real conditions, against a real adversary?

EO compliance runs through this discipline the same way. An Executive Order requirement is a claim until T&E tests it. Documentation and attestations describe what an organization asserts is true; T&E is the checkpoint that confirms it.

Core Components

T&E Planning

A formal test plan developed before any system or significant change enters deployment. The plan defines test objectives, test environment requirements, acceptance criteria, and roles. Testing that starts without a plan is not testing. It is hoping.

Security Testing

Static analysis, dynamic analysis, penetration testing, and red team operations conducted at defined points in the development and deployment lifecycle. Security testing is not a one-time gate passed once and forgotten. It is a continuous practice that runs as systems change.

Acceptance Criteria & User Acceptance Testing (UAT)

Explicit, measurable thresholds that a system or change must meet before production deployment is authorized. Security criteria carry equal weight to functional criteria. A system that works but is not secure has not met acceptance criteria. A system that is secure but does not work has not met them either.

User Acceptance Testing is the mechanism that validates a system does what the mission owner actually needs, not just what the written specification says. The two are not the same thing, and the gap between them is where failures live.

This is not theoretical. In large organizations, public and private alike, UAT is frequently absent entirely from major systems and changes. The absence is not a paperwork gap. It means systems move into production with no formal mechanism confirming the people who will use them have actually validated that they work as intended. Acceptance criteria exist on paper. Nothing enforces them in practice.

Where a system or change carries a training component, that training content and the process for delivering it fall inside UAT as well, not outside it. Training that has not been validated by the people who will actually use it is subject to the same failure mode as an untested system: it exists on paper, and no one has confirmed it works.

Third-Party & Open-Source Component Verification (EO 14306)

Software Bills of Materials and Hardware Bills of Materials are only as good as the verification behind them. T&E is the checkpoint that confirms the components declared in an SBOM are the components actually present in the deployed build, not just the components claimed on an attestation form. EO 14306 compliance is a claim until it is tested. Third-party patches represent the highest-risk category of SBOM change event, because they introduce new code into a system outside the organization's own development pipeline, often on a timeline the organization does not control.

This component connects directly to Process Engineering (Pillar 3), where open-source provenance and SBOM accountability are established as an ongoing obligation rather than a one-time declaration. T&E is where that obligation gets checked.

Red Team Operations

A red team is a group authorized to attack an organization's own systems the way a real adversary would, in order to find weaknesses before an actual attacker does. A blue team is the group of defenders responsible for detecting and responding to that simulated attack. Red team operations are conducted independently of the blue team, so the exercise tests real detection and response capability rather than a rehearsed outcome.

After-Action Reporting

Documented findings, remediation tracking, and lessons learned from all T&E activities. Test results are not filed away. They feed directly back into the Risk Register (Pillar 2) and the Process Engineering pillar (Pillar 3), because a finding that does not change anything downstream was not worth finding.

Stakeholder Approval & SOP Updates

Findings require formal stakeholder sign-off, not informal acknowledgment. Where testing reveals the process itself was wrong, not just the system, the relevant Standard Operating Procedure gets updated. This is the mechanism that keeps Process Engineering and Test & Evaluation connected rather than operating as disconnected silos. Testing without this step validates systems while leaving the underlying process uncorrected.

Engineering-to-CO/COR Change Notification

Engineering is obligated to notify the Contracting Officer or Contracting Officer's Representative when change management events occur, hardware, software, or network, that carry attestation or EO compliance implications. The CO/COR owns the supplier and contract lifecycle: new suppliers, contract renewals, and the attestations tied to them. Without a formal notification obligation, the CO/COR has no reliable way to know a change occurred that should trigger a new or updated attestation. This is not a case of anyone refusing to comply. It is a structural gap where two roles each own half of the same problem with no required handoff between them.

CO/CIO Attestation Design & Federal Sign-Off (EO 14306)

EO 14306 assigns responsibility for designing the attestation instrument itself jointly to the Contracting Officer and the Chief Information Officer. This is not a passive concurrence role. The CO and CIO are responsible for the form the attestation takes, not just for signing what Engineering hands them.

Federal sign-off closes the pillar: the completed T&E package, including results, gaps identified, corrective actions taken, and EO compliance status, goes to the CO/COR for formal concurrence. In practice, this has been a persistent point of failure rather than a clean handoff. Attestations that should exist as a matter of course have, in real cases, simply not been produced. Naming that plainly is more useful than describing an idealized process that does not reflect how this actually breaks down in the field.

Cross-Industry Configuration

The T&E process architecture is consistent across industries; the mission layer configures the specifics. Federal agencies apply defense acquisition T&E standards and EO-driven attestation requirements. Financial institutions apply Payment Card Industry Data Security Standard (PCI-DSS) penetration testing requirements and internal audit sign-off. Healthcare organizations layer HIPAA Security Rule validation into acceptance testing. The testing disciplines themselves, planning, security testing, acceptance criteria, red team exercises, are universal. The compliance framework and the sign-off chain are mission-specific.

Closing

A system that has not been tested against real conditions and a real adversary is a system running on assumptions. T&E exists to replace those assumptions with evidence, and to make sure that evidence reaches the people with the authority to act on it: the stakeholders who approve corrective SOPs, and the CO/COR who holds the attestation that says compliance was verified, not just claimed.

 

​