General overview of the Event Pipeline and Processes

The content below includes a list of all technical notes published under QRadar by category and sorted by popularity. Users can expand or collapse each section below using the + / - buttons. As new documentation is released, this content will be updated and new articles added. Click Expand All before starting a CTRL-F search.

Proofpoint on Demand customers can use this add-on to collect email security logs that can be stored and indexed in Splunk to search, report and investigate email delivery. This technology add-on maps the message and mail logs to Splunk Common Information Model (CIM) for email.

Security Information and Event Management (SIEM) solutions are used by many organizations to identify and correlate various security events occurring in their point products. Examples of SIEM products include HP's ArcSight, IBM's QRadar, and Splunk. 

When you create a log source extension, you might encounter some parsing issues. Use these XML examples to resolving specific parsing issues.

You create log source extensions (LSX) when log sources don't have a supported DSM, or to repair an event that has missing or incorrect information, or to parse an event when the associated DSM fails to produce a result.

The QRadar Support team writes articles for users to assist with technical resolutions or common problems. This page includes a searchable list of all published articles. Users can filter the table by keyword to quickly locate support write-ups.

How do I modify an existing event format and using a routing rule to forward the data to another log server using Syslog?

The project SIEM Analytics is designed to assist professionals in choosing SIEM systems, to talk about the strengths and weaknesses of the most common SIEM systems, as well as to give a preliminary comparative analysis of SIEM systems.

Restoring a backup archive is useful if you want to restore previously archived configuration files, offense data, and asset data on your IBM® Security QRadar® system.

In this post my intention is just to give some quick points on QRadar High Availability (HA)

1.    HA Overview
    -    Uses Primary and Secondary HA hosts
    -    Uses Virtual IPs
    -    Network connectivity is tested via hearbeat (pings) to all managed hosts
    -    HA Can be configured for either  console or managed host
    -    Both devices must have the same versions of the software
    -    Both devices must support the same DSM, scanner and protocols RPMs
    -    Uses data synchronization or shared external storage 
    -    Consistency is maintained locally by using Distributed Replicated Block Device (DRDB)
    -    If using external storage data consistency is maintained through iSCSI or Fibre Channel
    -    Data is synchronized in real time
    -    Note: Asset profiler can impact DRDB speed
    -    "/store" partition on secondary is automatically replicated to the secondary host
    -    Ensure min 1 Gbps between primary and secondary HA hosts
    -    Initial synchronization can take greater than 24 hours
        This may be an understatement. I've seen initial synchronization take upwards of 72 hours.
    -    Secondary host goes into "standby" after synchronization
    -    Primary HA hosts status becomes "offline" when restored from a failover
    -    Primary needs to be placed "online" before it becomes active
    -    Disk replication is enabled while primary is "offline"
    -    Post disk failover synchronization is faster
    -    Basically uses deltas
    -     When the primary host is restored, only the data collected by the secondary during the period the primary was unavailable is synchronized
    -    Replacing or reformating the disk on the primary can result in longer synchronization time in the event of a failback

    IP Considerations
        -    Uses Virtual IPs
        -    Needs 3 IP address - VIP, Primary and Secondary
        -    The IP address initially configured on the primary host is automatically made the cluster VIP
        -    A new IP will need to be assigned to the primary once HA configuration is started
        -    Primary host can act as a standby for secondary 
        -    VIP is used by a host that has a status of active
        -    All IPs must be in the same subnet
        -    Latency must be less than 2ms for traffic crosing the WAN

    HA Wizard
        -    Used to configure Primary, Secondary and cluster VIP
        -    Verifies the secondary has a valid HA activation key
        -    Verifies the secondary is not part of an existing HA cluster
        -    Verifies software version is the same on both devices
        -    Verifies external storage (if configured) on primary and then secondary
        -    Verifies both support the same DSM, scanner and protocol RPMS

    Failover scenarios
        -    Power supply failure
        -    network failure (detected by connectivity tests)
        -    OS malfunction that delays or stops hearbeat tests
        -    RAID failure
        -    Manual failover
        -     Management interface failure on primary hosts

        -    Primary does not take back its role as primary in the case of a failover. 
        -    Secondary stays as primary while primary acts as standy
        -    Primary must be switched to "active" to take over its role

        -    No failover for software errors or disk capacity issues
        -    If both primary and secondary are unable to ping a managed hosts no failover occurs
        -    If primary cannot but secondary can ping a managed host, failover occurs

    HA Failover event sequence
        -    File systems are mounted
        -    Management interface alias is created eth0 is eth0:0
        -    VIP is assigned to the alias
        -    QRadar services are started
        -    Secondary connects to console and downloads configuration files

    Tips for manual synchronization
        -    Ensure primary and secondary hosts are sync'd
        -    Secondary must be in standby
        -    Secondary to offline and power off the primary
        -    DO NOT MANUALLY FORCE FAILOVER DURING PATCHES OR SOFTWARE UPGRADES

2.    HA Planning
    -    File systems on both devices much match - ext-3, etc
    -    Secondary's "/store" partion must be equal to or greater than the primary
    -    Both devices should have the same number of interfaces
    -    Both must use the same management interface
    -    Only 1 VIP
    -    Port 7789 is needed for Distributed Replicated Block Device (DRDB)
    -    DRBD traffic is bidirectional
    -    Disk replication ensures software updates are applied to the secondary
    -    Ensure the host has a valid activation key

3.    HA Management
    -    Uses System and License management window to:
        -    monitor HA
        -    Force failover
        -    Disconnect cluster
        -    Modify cluster settings
        -    Modify heartbeat interval
        -    Place the device in "offline" mode before maintenance

I provide support for european customers of QRadar and all of the family products (QVM, QRM, QRIF, QNI). Nowadays, I am more IT consultant than Civil engineer (MSc Eng), but I still have great passion for all reinforced concrete construction around the world.

The version  of the DSA utility differs based Operating systems and appliance Model types. QRadar 7.2.x uses a different build than QRadar 7.3.x.  M3 and M4 appliances use a different build of the DSA than M5+ appliances. This technote lists the builds required for your base Operating and Appliance type.

QRadar: HA synchronization progress resets to 0%

What are the sequence of events during an High-Availability (HA) failover and how are these experienced?

Event categories are used to group incoming events for processing by IBM® Security QRadar®. The event categories are searchable and help you monitor your network.

Is there a way to test the high-availability (HA) crossover connection?

You can manually map a normalized or raw event to a high-level and low-level category (or QID).

Click a check mark in the following matrix to go to the log source that you're most interested in. For each log source, the relevant ATT&CK framework categories are listed. The Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) framework was developed by Mitre Corp.

Because Azure and Office 365 are widely used, I decided to start with this. I hope you will find it useful because unfortunately, there is a lack of good resources other than Microsoft when it comes to monitoring Azure with a SIEM and I had to spend many hours to study the logs and figure out what was relevant.