Audit Log

Comprehensive audit logging system for tracking user actions and system changes with detailed input/output information for compliance and security monitoring

Overview

FlagFlow's audit logging system, introduced in version 1.5.0, provides comprehensive tracking of all user actions and system changes. This feature is essential for:

Compliance requirements - Meet regulatory standards with detailed audit trails
Security monitoring - Track unauthorized access attempts and suspicious activities
Change tracking - Monitor who changed what flags and when
Debugging - Understand the sequence of actions leading to issues
Accountability - Maintain clear records of user actions

⚠️ Important: Audit Log Size

Audit logs can become very large! Every user action with full input and output data is logged. Plan for adequate storage capacity and consider log rotation policies for production deployments.

Configuration

Audit logging is controlled by the AUDITLOG_ENABLED environment variable and operates at log level 100 (a special level for audit events).

Enable Audit Logging Environment Variable

Environment Configuration

# Enable audit logging
AUDITLOG_ENABLED=true

# Optional: Set general log level (audit logs use level 100 regardless)
LOGLEVEL=info

Note: Audit logs operate at log level 100, which is separate from the general LOGLEVEL setting. This ensures audit events are always captured when audit logging is enabled.

Routing to Special Log Storage Log Routing

Since audit logs use a specific log level (100), they can be easily routed to specialized storage systems using log management tools:

Using Fluentd/Fluent Bit

fluentd-audit-routing.conf

<filter flagflow.**>
  @type grep
  <regexp>
    key level
    pattern ^100$
  </regexp>
</filter>

<match flagflow.audit>
  @type s3
  s3_bucket audit-logs-bucket
  s3_region us-west-2
  path audit-logs/flagflow/
  time_slice_format %Y%m%d
</match>

Using Logstash

logstash-audit-routing.conf

filter {
  if [level] == 100 {
    mutate {
      add_tag => ["audit"]
    }
  }
}

output {
  if "audit" in [tags] {
    elasticsearch {
      hosts => ["audit-elasticsearch:9200"]
      index => "flagflow-audit-%{+YYYY.MM.dd}"
    }
  }
}

Using Docker Logging Driver

docker-compose.yml with audit log routing

version: '3.8'
services:
  flagflow:
    image: flagflow/flagflow:latest
    environment:
      - AUDITLOG_ENABLED=true
    logging:
      driver: "fluentd"
      options:
        fluentd-address: "audit-collector:24224"
        tag: "flagflow.audit"
        labels: "service"
        env: "ENVIRONMENT"

Audit Log Format

Audit logs are structured JSON entries containing comprehensive information about each user action. Here's the format and examples:

Audit Log Entry Structure Standard Format

Complete Audit Log Entry

{
  "level": 100,
  "time": "2024-12-01T15:30:45.123Z",
  "hostname": "flagflow-prod-01",
  "pid": 1234,
  "service": "flagflow",
  "traceId": "abc123def456",
  "userId": "admin",
  "userPermissions": ["flags:read", "flags:write", "users:manage"],
  "action": "flag.update",
  "resource": "feature_flags/payment/allow_crypto",
  "method": "PUT",
  "endpoint": "/rpc/protected/flag.update",
  "inputs": {
    "flagName": "payment/allow_crypto",
    "value": true,
    "previousValue": false
  },
  "outputs": {
    "success": true,
    "flagId": "payment_allow_crypto_001",
    "updatedAt": "2024-12-01T15:30:45.120Z"
  },
  "clientIp": "192.168.1.100",
  "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)",
  "sessionId": "sess_xyz789",
  "duration": 45,
  "status": "success"
}

Audit Log Fields Field Descriptions

Field	Type	Description
level	number	Always 100 for audit logs
time	string	ISO 8601 timestamp of the action
traceId	string	Unique identifier for request tracing
userId	string	User who performed the action
userPermissions	array	User's permissions at time of action
action	string	Type of action performed (flag.create, flag.update, user.delete, etc.)
resource	string	Resource affected by the action
inputs	object	Complete input data for the action
outputs	object	Complete output/response data
clientIp	string	IP address of the client
duration	number	Action duration in milliseconds
status	string	success, error, or warning

Example Audit Log Entries

Flag Creation Flag Operations

Flag Creation Audit Log

{
  "level": 100,
  "time": "2024-12-01T14:15:30.456Z",
  "traceId": "trace_001",
  "userId": "developer_jane",
  "action": "flag.create",
  "resource": "features/new_checkout_flow",
  "inputs": {
    "flagName": "features/new_checkout_flow",
    "type": "boolean",
    "defaultValue": false,
    "description": "Enable new checkout flow for testing"
  },
  "outputs": {
    "success": true,
    "flagId": "features_new_checkout_flow_001",
    "createdAt": "2024-12-01T14:15:30.450Z"
  },
  "status": "success",
  "duration": 23
}

Flag Value Change Flag Updates

Flag Update Audit Log

{
  "level": 100,
  "time": "2024-12-01T16:45:12.789Z",
  "traceId": "trace_002",
  "userId": "admin",
  "action": "flag.update",
  "resource": "features/new_checkout_flow",
  "inputs": {
    "flagName": "features/new_checkout_flow",
    "value": true,
    "previousValue": false
  },
  "outputs": {
    "success": true,
    "flagId": "features_new_checkout_flow_001",
    "updatedAt": "2024-12-01T16:45:12.785Z",
    "propagatedToInstances": 3
  },
  "status": "success",
  "duration": 67
}

User Creation User Management

User Creation Audit Log

{
  "level": 100,
  "time": "2024-12-01T10:30:45.123Z",
  "traceId": "trace_003",
  "userId": "admin",
  "action": "user.create",
  "resource": "users/developer_bob",
  "inputs": {
    "username": "developer_bob",
    "permissions": ["flags:read", "flags:write"],
    "roles": ["developer"]
  },
  "outputs": {
    "success": true,
    "userId": "user_dev_bob_001",
    "createdAt": "2024-12-01T10:30:45.120Z"
  },
  "status": "success",
  "duration": 34
}

Unauthorized Access Attempt Failed Actions

Failed Action Audit Log

{
  "level": 100,
  "time": "2024-12-01T18:22:15.666Z",
  "traceId": "trace_004",
  "userId": "guest_user",
  "userPermissions": ["flags:read"],
  "action": "flag.delete",
  "resource": "critical/payment_enabled",
  "inputs": {
    "flagName": "critical/payment_enabled"
  },
  "outputs": {
    "success": false,
    "error": "Insufficient permissions",
    "errorCode": "PERMISSION_DENIED",
    "requiredPermission": "flags:delete"
  },
  "clientIp": "203.0.113.45",
  "status": "error",
  "duration": 12
}

Migration Import System Events

Migration Import Audit Log

{
  "level": 100,
  "time": "2024-12-01T09:00:00.000Z",
  "traceId": "trace_005",
  "userId": "system_migration",
  "action": "migration.import",
  "resource": "migration/staging_to_prod_20241201",
  "inputs": {
    "sourceEnvironment": "staging",
    "flagsCount": 47,
    "usersCount": 12,
    "migrationFile": "flagflow_migration_staging_20241201-090000.json"
  },
  "outputs": {
    "success": true,
    "importedFlags": 47,
    "importedUsers": 12,
    "skippedDuplicates": 3,
    "completedAt": "2024-12-01T09:00:15.456Z"
  },
  "status": "success",
  "duration": 15456
}

Storage and Performance Considerations

⚠️ Storage Requirements

Audit logs can consume significant storage space. Consider these factors:

Every user action generates a complete audit entry with full input/output data
High-activity environments can generate thousands of audit entries per day
Each entry can be 1-5KB depending on the complexity of inputs/outputs

Capacity Planning Storage Planning

Estimate your audit log storage needs:

Example Calculations:

Small team (10 users, 50 actions/day): ~250KB/day, ~90MB/year
Medium team (50 users, 500 actions/day): ~2.5MB/day, ~900MB/year
Large organization (200 users, 2000 actions/day): ~10MB/day, ~3.6GB/year

Log Management Best Practices Best Practices

Implement log rotation: Archive old audit logs to cold storage after 90 days
Use compression: Compress archived audit logs to save 70-80% space
Monitor disk space: Set up alerts when audit log storage exceeds thresholds
Index efficiently: When using search systems, index key fields (time, userId, action)
Regular cleanup: Establish retention policies (e.g., keep for 7 years for compliance)

Compliance and Security

FlagFlow's audit logging supports various compliance requirements and security standards:

Compliance Standards

✅ SOX (Sarbanes-Oxley): Complete change tracking
✅ GDPR: User action accountability
✅ HIPAA: Access logging for protected data
✅ ISO 27001: Information security management
✅ PCI DSS: Access control monitoring

Security Features

🔒 Immutable logs: Cannot be modified after creation
🔍 Complete context: Full input/output capture
👤 User attribution: Every action tied to a user
⏱️ Precise timing: Millisecond-accurate timestamps
📍 Request tracing: Trace IDs for correlation

Troubleshooting

Audit Logging Not Working Common Issues

No Audit Logs Appearing

Check that AUDITLOG_ENABLED=true is set and the application has restarted. Verify your log level configuration allows level 100 messages.

Audit Logs Too Large

This is expected behavior. Implement log rotation and archiving. Consider filtering specific actions if needed using your log management system.

Missing Trace IDs

Trace IDs are automatically generated for each request. If missing, check that the request is going through FlagFlow's standard middleware pipeline.

On this page