SOC2 Compliance Through Infrastructure Automation: Evidence Collection at Scale

SOC2 audits used to consume six weeks of engineering time annually. Engineers scrambled to produce screenshots, export configurations, and write narratives explaining security controls. The evidence was stale before the auditor reviewed it. We built an automated evidence collection pipeline using AWS Config, custom conformance packs, and a continuous compliance dashboard that reduced audit preparation to three days. Here is the system we built.

The Problem: Manual Compliance Is Unsustainable

Our first SOC2 Type II audit was painful. The auditor requested evidence for 147 controls. Each request triggered a chain of Slack messages, screenshot sessions, and hastily written explanations. The evidence was point-in-time, meaning it proved compliance at the moment of capture but said nothing about the other 364 days.

Manual compliance reality:

• 6 weeks of engineering distraction per annual audit
• 147 evidence requests requiring manual collection
• Evidence gaps in 23% of controls due to stale documentation
• $340K annual cost (engineering time + external consultants)
• Zero continuous visibility into compliance posture

Architecture: Continuous Compliance Pipeline

We designed the system around three principles: evidence should be generated automatically, compliance should be continuously validated, and auditor access should be self-service.

The pipeline continuously evaluates AWS resources against our SOC2 control framework, generates evidence artifacts, stores them in a tamper-evident S3 bucket, and surfaces compliance posture in a real-time dashboard.

AWS Config Conformance Pack

We authored a custom conformance pack mapping AWS Config rules to SOC2 Trust Service Criteria. Each rule generates evidence automatically.

ConformancePackName: soc2-type2-controls
Parameters:
  RequiredTagKeys:
    Type: String
    Default: "team,environment,data-classification"

Resources:
  # CC6.1 - Logical and Physical Access Controls
  EncryptionAtRest:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: soc2-cc61-encryption-at-rest
      Description: "All storage must be encrypted (SOC2 CC6.1)"
      Source:
        Owner: AWS
        SourceIdentifier: ENCRYPTED_VOLUMES
      Scope:
        ComplianceResourceTypes:
          - AWS::EC2::Volume

  # CC6.6 - System Boundaries
  SecurityGroupRestriction:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: soc2-cc66-no-unrestricted-ingress
      Description: "No security groups allow 0.0.0.0/0 ingress (SOC2 CC6.6)"
      Source:
        Owner: AWS
        SourceIdentifier: RESTRICTED_INCOMING_TRAFFIC
      InputParameters:
        blockedPort1: "22"
        blockedPort2: "3389"

  # CC7.2 - System Monitoring
  CloudTrailEnabled:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: soc2-cc72-cloudtrail-enabled
      Description: "CloudTrail enabled in all regions (SOC2 CC7.2)"
      Source:
        Owner: AWS
        SourceIdentifier: CLOUD_TRAIL_ENABLED

  # CC6.3 - Access Revocation
  IAMPasswordPolicy:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: soc2-cc63-password-policy
      Description: "IAM password policy meets SOC2 requirements (SOC2 CC6.3)"
      Source:
        Owner: AWS
        SourceIdentifier: IAM_PASSWORD_POLICY
      InputParameters:
        RequireUppercaseCharacters: "true"
        RequireLowercaseCharacters: "true"
        RequireNumbers: "true"
        RequireSymbols: "true"
        MinimumPasswordLength: "14"
        MaxPasswordAge: "90"

This conformance pack covers 43 of our 147 controls directly. Each rule evaluates continuously, providing evidence that compliance was maintained throughout the audit period, not just at a single point in time.

Automated Evidence Generation

For controls that AWS Config cannot evaluate natively, we built Lambda functions that generate evidence artifacts on a schedule:

import boto3
import json
from datetime import datetime, timedelta
from typing import Any

def generate_access_review_evidence(event: dict, context: Any) -> dict:
    """Generate evidence for CC6.2 - User Access Reviews.
    
    Produces a structured report of all IAM users, their last activity,
    and group memberships for quarterly access review.
    """
    iam = boto3.client('iam')
    s3 = boto3.client('s3')
    
    evidence = {
        "control_id": "CC6.2",
        "control_name": "User Access Reviews",
        "generated_at": datetime.utcnow().isoformat(),
        "review_period": {
            "start": (datetime.utcnow() - timedelta(days=90)).isoformat(),
            "end": datetime.utcnow().isoformat(),
        },
        "users": [],
    }
    
    paginator = iam.get_paginator('list_users')
    for page in paginator.paginate():
        for user in page['Users']:
            user_detail = iam.get_user(UserName=user['UserName'])
            
            # Get last activity
            try:
                last_used = iam.get_access_key_last_used(
                    AccessKeyId=get_access_key(iam, user['UserName'])
                )
                last_activity = last_used.get('AccessKeyLastUsed', {}).get('LastUsedDate')
            except Exception:
                last_activity = None
            
            # Get group memberships
            groups = iam.list_groups_for_user(UserName=user['UserName'])
            
            evidence["users"].append({
                "username": user['UserName'],
                "created": user['CreateDate'].isoformat(),
                "last_activity": last_activity.isoformat() if last_activity else "never",
                "groups": [g['GroupName'] for g in groups['Groups']],
                "mfa_enabled": has_mfa(iam, user['UserName']),
                "days_inactive": calculate_inactive_days(last_activity),
                "risk_flags": assess_risk(user, last_activity, groups['Groups']),
            })
    
    # Store evidence with integrity hash
    evidence_key = f"evidence/CC6.2/{datetime.utcnow().strftime('%Y/%m/%d')}/access-review.json"
    s3.put_object(
        Bucket='compliance-evidence-production',
        Key=evidence_key,
        Body=json.dumps(evidence, indent=2, default=str),
        ServerSideEncryption='aws:kms',
        Metadata={
            'control-id': 'CC6.2',
            'integrity-hash': compute_sha256(evidence),
        }
    )
    
    return {
        "statusCode": 200,
        "control_id": "CC6.2",
        "users_reviewed": len(evidence["users"]),
        "risk_flags_found": sum(1 for u in evidence["users"] if u["risk_flags"]),
    }

Evidence artifacts are stored in an S3 bucket with Object Lock enabled (WORM compliance), ensuring auditors can verify that evidence has not been tampered with after generation.

Compliance Dashboard and Alerting

We built a real-time compliance dashboard that shows current posture across all 147 controls:

The dashboard categorizes controls into three states: compliant (green), at-risk (yellow, trending toward non-compliance), and non-compliant (red). At-risk detection uses trend analysis. If a control shows degrading compliance over seven days, it triggers an alert before becoming non-compliant.

def evaluate_compliance_trend(control_id: str, lookback_days: int = 7) -> str:
    """Determine if a control is trending toward non-compliance."""
    evaluations = get_recent_evaluations(control_id, lookback_days)
    
    if not evaluations:
        return "unknown"
    
    compliant_ratio = sum(1 for e in evaluations if e["status"] == "COMPLIANT") / len(evaluations)
    
    if compliant_ratio == 1.0:
        return "compliant"
    elif compliant_ratio >= 0.95:
        # Check if trend is degrading
        recent = evaluations[-3:]
        if any(e["status"] != "COMPLIANT" for e in recent):
            return "at-risk"
        return "compliant"
    else:
        return "non-compliant"

Auditor Self-Service Portal

Instead of fielding evidence requests via email, we built a portal where auditors can browse and download evidence directly. The portal maps each Trust Service Criterion to its evidence artifacts, providing:

• Historical compliance timeline (was this control compliant on date X?)
• Raw evidence artifacts (JSON exports, configuration snapshots)
• Remediation history (when was a gap detected and resolved?)
• Control owner and escalation path

This self-service approach eliminated the back-and-forth that traditionally dominates audit cycles.

Results

Metric	Before	After	Improvement
Audit preparation time	6 weeks	3 days	-93%
Engineering hours per audit	480 hours	24 hours	-95%
Controls with continuous evidence	0	147/147	100%
Compliance gaps at audit time	23%	1.4%	-94%
Annual compliance cost	$340K	$42K	-88%
Time to detect compliance drift	Days/weeks	< 15 minutes	-99%

Key Lessons

Map controls to infrastructure before building. We spent two weeks mapping every SOC2 control to specific AWS resources and configurations. This upfront investment made automation straightforward because we knew exactly what to evaluate.

Object Lock is worth the cost. Auditors specifically asked about evidence integrity. S3 Object Lock with a 400-day retention period gave them confidence without custom solutions.

Trend detection beats binary compliance. Knowing a control is "currently compliant" is less valuable than knowing it is "compliant but degrading." Early warning gave us time to remediate before the auditor noticed.

Give auditors direct access. The self-service portal transformed our auditor relationship. Instead of adversarial evidence requests, auditors became collaborators who trusted the system.

Conclusion

SOC2 compliance does not have to be a quarterly fire drill. By treating compliance controls as infrastructure (codified, continuously evaluated, automatically evidenced), we turned a six-week annual burden into a three-day formality. The system pays for itself by freeing engineering time and catching compliance drift before it becomes an audit finding. Start by mapping your controls to AWS resources, deploy Config conformance packs for what they cover natively, and build Lambda-based evidence generators for the rest.