Missing /store partition in QRadar

Missing /store partition can sometimes seem in your QRadar, due to unsafe close of your server (hard reboot or power fail incident). In result,  you can run into troubles caused by xfs file system corruption. This ends up with the  /store partition not properly mounted by QRadar.

Normally, in Red Hat 7, during boot up, you you can get advise to run xfs_repair command. This can manage corrupted filesystem and it can restore system to the useful state.

Unfortunately, in QRadar, this info is missing. Thus, you can find your system stuck on booting process or even if you can login then you can’t run any service. This is obviously caused by missing /store partition. You can confirm this problem using the following command:

# df -hT

After that, the next step would be assured is this partition available in the system using command like lvs or parted

Despite that, when trying to mount this partition using the following commands like:

mount /dev/mapper/storerhel-store /store

you will get error saying that structure needs cleaning.

Repair

After that I decided to run xfs_repair.  xfs_repair repairs corrupt or damaged XFS filesystems. The filesystem is specified using the device argument which should be the device name of the disk partition or volume containing the filesystem. If given the name of a block device, xfs_repair will attempt to find the raw device associated with the specified block device and will use the raw device instead

Unfortunately when I ran:

# xfs_repair /dev/mapper/storerhel-store

got error in response. The only option was to run xfs_repair with force option:

# xfs_repair -L /dev/mapper/storerhel-store

-L stands for Force Log Zeroing. Forces xfs_repair to zero the log even if it is dirty (has metadata changes). When using this option the filesystem will likely seem to be corrupt, and can cause the loss of user files and/or data. Nevertheless in this case when I ran it,

it resolved this issue. I could mount recovered /store partition and then reboot server.

This issue applies only to QRadar 7.3.x which use Red Hat 7 as operating system.