Feature Request: Add Authentication File Support and Burp Import Functionality

[kenyon-wong]

Summary

Request to add two key features to httpx:

1. Authentication file support (-sf, --secret-file) similar to nuclei
2. Burp Suite import functionality (-im burp, --input-mode burp) for processing Burp XML exports

Motivation

Current Limitations

• httpx currently lacks built-in authentication management for complex scanning scenarios
• No native support for importing request data from Burp Suite, a widely used security testing tool
• Users must manually handle authentication headers and cookies for each request
• Difficult to integrate httpx into existing security testing workflows that rely on Burp Suite

Use Cases

1. Authenticated Web Application Scanning: Need to scan applications with session-based authentication, API keys, or custom headers
2. Security Testing Workflow Integration: Import requests captured in Burp Suite for further analysis with httpx
3. Automated Security Pipelines: Manage authentication credentials centrally across different targets
4. Penetration Testing: Seamlessly transition from manual testing in Burp to automated scanning with httpx

Proposed Features

1. Authentication File Support (-sf, --secret-file)

Command Line Interface

# Basic usage
httpx -sf secrets.yaml -l targets.txt

# With other httpx features
httpx -sf secrets.yaml -l targets.txt -tech-detect -title -status-code -ms 200

Configuration Format (YAML)

secrets:
  - id: "web-app-auth"
    domains:
      - "example.com"
      - "*.example.com"
    headers:
      Cookie: "session=abc123; csrf_token=xyz789"
      Authorization: "Bearer token123"
    
  - id: "api-auth"
    domains:
      - "api.service.com"
    headers:
      X-API-Key: "api_key_here"
      Authorization: "Basic dXNlcjpwYXNz"
    
  - id: "custom-auth"
    domains:
      - "internal.company.com"
    credentials:
      username: "admin"
      password: "password123"
    headers:
      X-Custom-Header: "custom_value"

Features

• Domain-based matching: Apply different credentials to different domains/subdomains
• Wildcard support: Use *.domain.com for subdomain matching
• Multiple authentication methods: Support for cookies, headers, basic auth, bearer tokens
• Environment variable support: Reference environment variables in configuration
• Credential inheritance: Hierarchical credential application

2. Burp Suite Import Support (-im burp, --input-mode burp)

Command Line Interface

# Import from Burp XML export
httpx -im burp -l burp_export.xml

# Combine with authentication
httpx -im burp -l burp_export.xml -sf secrets.yaml

# Extract specific information
httpx -im burp -l burp_export.xml -title -tech-detect -status-code

Supported Features

• Full request parsing: Extract method, URL, headers, and body from Burp XML
• Request filtering: Filter by method, status code, or URL patterns
• Header preservation: Maintain original request headers while allowing authentication override
• Body handling: Properly handle POST data and form parameters
• URL reconstruction: Rebuild complete URLs from Burp's host/path format