Emerging Threats Snort Processor — Design

Overview

Fetches Snort IDS rules from the Emerging Threats open ruleset, parses CVE references, and creates Exploit records (source=emergingthreats) linked to CVEMetadata records (source=vulnetix). The Snort rule text is stored as a remediation description in a markdown code block in CVEDescription.

Only rules with at least one reference:cve are processed. Disabled rules (lines starting with #alert, #drop, etc.) are parsed with Disabled=true.

The parser is version-specific: this processor uses the Snort 2.x parser (parser_v2.go). Snort 3.x (Lua-based) rules will need a separate parser_v3.go.

Feed

Snort Rule Format (v2.9.0)

action protocol src_ip src_port -> dst_ip dst_port (options)

Example with CVE references:

alert tcp $EXTERNAL_NET $HTTP_PORTS -> $HOME_NET any (
  msg:"ET WEB_SERVER Cisco ASA Clientless SSL VPN XSS";
  flow:to_client,established;
  reference:url,tools.cisco.com/security/center/viewAlert.x?alertId=18442;
  reference:cve,2009-1201;
  reference:cve,2009-1202;
  classtype:web-application-attack;
  sid:2010730; rev:4;
  metadata:affected_product Web_Server_Applications, attack_target Web_Server,
    created_at 2010_07_30, confidence Medium, signature_severity Major,
    tag XSS, tag Cross_Site_Scripting, updated_at 2010_09_08;
)

Key Fields Extracted

Parsing

Comment vs. Disabled Rule Detection

• # Copyright ... → pure comment, skip
• #alert tcp ... → disabled rule, parse with Disabled=true

Detection: after stripping #, check if first word is a valid Snort action (alert, drop, pass, log, activate, dynamic, sdrop).

Body Tokenization

The rule body is split on ; while respecting:

• Double-quoted strings ("...")
• Hex blocks (|xx xx|)

This prevents false splits in content patterns like content:"|4e 3d;2c|".

Storage

CVE Source Resolution

For each CVE referenced by a rule, the processor ensures a CVEMetadata record exists with source="vulnetix":

1. Exists check: SELECT 1 FROM "CVEMetadata" WHERE cveId=$1 AND source='vulnetix'
2. Clone from priority source: cve.org → nist-nvd → github → anchore_adp → any other
3. Fallback: Create minimal record (state=PUBLISHED, dataVersion=5.1)

Incremental Strategy

• Resume key: Exploit.exploitId (SID) for source=“emergingthreats”
• At startup, load all processed SIDs via LoadProcessedExploitIDs()
• Skip rules whose SID is already in the set (unless --force or --all)
• The rules file is always re-downloaded (no ETag/If-Modified-Since)

Flags

ECS Schedule

Daily at 08:00 UTC. CPU: 256, Memory: 512 MB.

Key Files

Verification

-- Exploit records
SELECT COUNT(*) FROM "Exploit" WHERE source = 'emergingthreats';

-- ExploitCVE junctions
SELECT COUNT(*) FROM "ExploitCVE" ec
JOIN "Exploit" e ON ec."exploitUuid" = e.uuid
WHERE e.source = 'emergingthreats';

-- CVEMetadata vulnetix records
SELECT COUNT(*) FROM "CVEMetadata" WHERE source = 'vulnetix';

-- Remediation descriptions
SELECT LEFT("value", 200) FROM "CVEDescription"
WHERE source = 'vulnetix' AND "value" LIKE '%Snort IDS%' LIMIT 3;

S3 Persistence

Not used. This processor does not currently archive payloads or quarantine failures to S3. Per the S3 Persistence Contract this is non-compliant — see the compliance matrix for the implementation roadmap.

Expected paths when implemented:

• Archive: emerging-threats/files/{sha256}/{filename}
• Quarantine: failed-feeds/emergingthreats-snort-processor/{YYYY-MM-DD}/{reason}/{filename}
• Likely reasons: (none documented)