Changes in Traffic Analysis in 7.3.1
Among new features introduced in version 7.3.1, one of the most important would be a change in Traffic Analysis.
Change reasons
Many users have had issues with incorrectly auto detected log sources. In some extreme cases, incorrectly detected devices can have a major performance impact, which would lead to degradation on ecs-ec. The solution for this problem was to move this configuration into the database. Also, the addition of API, as well as replication and script to manage, helped to avoid these issues.
Despite change introduced, all customers still have an option to use the XML configuration file. If this would be necessary, it will override settings locally on each event collector.
Only the fresh installations of 7.3.1 are using this new autodetection configuration method. Existing installations from the upgrade of 7.2.x and 7.3, maintain their XML file usage on upgrading.
With the recent changes in /opt/qradar/support folder, provided by autoupdates, IBM introduced a new CLI script. It is called autodetection_config.py, which allows now for an import option of the existing configuration file to the database.
The script supports import of the only one customized file, which from this moment become global for the whole deployment.
Traffic Analysis prior to 7.3.1
In the previous versions of QRadar, each managed host had an XML configuration file for the log source autodetection process. Therefore each managed host kept own separated settings. Configservices didn’t manage any of these files during deploying changes to the other components, what could cause problems
Traffic Analysis after 7.3.1
Since the version 7.3.1 Postgres database stores autodetection configuration. Autodetection configuration tables are replicated out to all managed hosts during the replication process running on each host every minute. CLI support script interface provides configuration changes made via API calls. In future, this configuration will be available from User Interface, too.
Autodetection, which is part of ecs-ec, automatically detects changes in Postgres without any restart. For each event collector, you can override global settings with local. Simply you can revert back to the xml configuration file.
Using the Event Collector options in Admin Tab / System & License Management / Edit Host / Component Management let you switch from XML configuration file to Database, and back, .
Troubleshooting
In case of troubles, the best start would be comparing the output from mbeans with the query in Postgres database. Run jmx.sh script against “Traffic Analysis” mbean, which will show the number of log source types loaded into memory. Run the query in the database can show the number configuration files stored. Difference between these two numbers show some issues with autodetection of configuration.
[root@robertrojek ~]# psql -U qradar -c "select count(*) from ta_dsm_info where enabled = 't'" count ------- 170 (1 row) [root@robertrojek ~]# /opt/qradar/support/jmx.sh -p 7777 -b "com.q1labs.sem:application=ecs-ec.ecs-ec,type=filters,name=Traffic Analysis Engine”com.q1labs.sem:application=ecs-ec.ecs-ec,type=filters,name=Traffic Analysis Engine ---------------------------------------------------------------------------- SensorDevicesCreated: 0 EventsProcessedSuccessfully: 1 EventsProcessedUnsuccessfully: 178 CurrentNumberOfAddressesMonitored: 3 CurrentNumberOfDeviceTypesUsed: 170 TotalEventsAttempted: 882 TotalAddressesAbandoned: 3 TotalEventsLostDueToMaxDevicesMonitored: 0 TotalEventsDroppedByPayloadFilters: 0