Skip to content

Troubleshooting

This guide helps you diagnose and resolve common issues when running Tracee.

Installation Issues

BTF Not Available

Problem: Error message about missing BTF support

ERRO error loading eBPF program: BTF is required

Solution:

  1. Check if BTF is available: ls /sys/kernel/btf/vmlinux
  2. If missing, upgrade to a kernel version that includes BTF support (most modern distributions)
  3. See Prerequisites for more details

Permission Denied

Problem: Tracee fails to start with permission errors

ERRO permission denied loading eBPF program

Solutions:

  1. Run as root: sudo tracee
  2. Use required capabilities:
    # Docker - minimal required capabilities
    docker run \
      --cap-add SYS_RESOURCE \
      --cap-add SYS_ADMIN \
      --cap-add SYS_PTRACE \
      --cap-add NET_ADMIN \
      --cap-add SYSLOG \
      tracee
    
    # Kubernetes
    securityContext:
      capabilities:
        add: 
          - "SYS_RESOURCE"
          - "SYS_ADMIN" 
          - "SYS_PTRACE"
          - "NET_ADMIN"
          - "SYSLOG"
    
  3. For recent kernels (>=5.8), you may use CAP_BPF + CAP_PERFMON instead of CAP_SYS_ADMIN
  4. See Prerequisites for complete details and justifications

Kernel Version Incompatibility

Problem: Unsupported kernel version errors

Solution: - Check supported kernels: Prerequisites - Tracee requires kernel 5.4 or newer (4.18 for RHEL 8) - Consider upgrading to a supported kernel version

Runtime Issues

No Events Generated

Problem: Tracee starts but produces no events

Troubleshooting steps:

  1. Check scope filters:

    # Test with minimal configuration
    tracee --events execve
    

  2. Verify events are enabled:

    # List enabled events
    tracee --events help
    

  3. Check policy configuration:

    # Ensure policy scope matches your workload
    scope:
      - container  # Or specific process filters if needed
    

High CPU Usage

Problem: Tracee consuming excessive CPU

Solutions:

  1. Reduce event scope:

    scope:
      - container  # Limit to containers only
    

  2. Filter events:

    rules:
      - event: execve
        filters:
          - uid!=0  # Example: exclude root processes
    

  3. Use specific events instead of sets:

    rules:
      - event: execve        # Specific event
      # - event: syscalls    # Avoid broad sets
    

Container Issues

Container Events Not Captured

Problem: Missing container-related events

Solutions:

  1. Mount container runtime socket:
    # Docker
    docker run -v /var/run/docker.sock:/var/run/docker.sock tracee
    
    # Containerd
    docker run -v /run/containerd/containerd.sock:/run/containerd/containerd.sock tracee
    

Performance Issues

Events Being Dropped

Problem: Warning about dropped events

WARN events dropped due to buffer overflow

Solutions:

  1. Increase buffer size:

    tracee --perf-buffer-size 1024
    

  2. Reduce event frequency:

    rules:
      - event: openat
        filters:
          - data.pathname!=/tmp/*  # Filter noisy paths
    

Slow Event Processing

Problem: Events arrive with significant delay

Solutions:

  1. Check system load: Use top, htop to verify system isn't overloaded
  2. Optimize event selection: Use specific events instead of broad event sets

Output Issues

JSON Parsing Errors

Problem: Invalid JSON output

Solutions:

  1. Use proper output format:

    tracee --output json --output option:parse-arguments
    

  2. Check for mixed output:

    # Separate logs from events
    tracee --output json --log file:/var/log/tracee.log
    

Missing Event Fields

Problem: Expected fields not present in events

Solutions:

  1. Enable argument parsing:

    tracee --output option:parse-arguments
    

  2. Check event definition: Some events may not include all expected fields

Debugging

Enable Debug Logging

# Enable debug logs
tracee --log debug

# Or via environment
TRACEE_LOG_LEVEL=debug tracee

Capture System Information

# System info for bug reports
tracee --version
uname -a
cat /etc/os-release
ls -la /sys/kernel/btf/vmlinux

Test Minimal Configuration

# Minimal test configuration
sudo tracee --events execve

Getting Help

If you continue experiencing issues:

  1. Search existing issues: GitHub Issues
  2. Check discussions: GitHub Discussions
  3. Join Slack: Aqua Community Slack

When reporting issues, include:

  • Tracee version (tracee --version)
  • Operating system and kernel version
  • Container runtime (if applicable)
  • Complete error messages
  • Minimal reproduction steps