[Linux] How to configure Kdump
- Red Hat Linux 7
in this tutorial we are going to configure Kdump on a Red Hat Linux 7 server. Kdump is a useful utility that dumps all the memory when the server crashes.
1 – Kernel configuration
First of all, Kdump requires the following package to be installed:
yum install kexec-tools
Next, allocate the memory for the crash kernel that is started after a failure/crash and that has to dump the memory. To to this, modify in the grub file setting the crashkernel memory size in the GRUB_CMDLINE_LINUX entry and then regenerate the configuration:
vi grub ... GRUB_CMDLINE_LINUX="rd.lvm.lv=rhel/root crashkernel=512M@64M rd.lvm.lv=rhel/swap vconsole.font=latarcyrheb-sun16 vconsole.keymap=us rhgb quiet nomodeset" ... #for legacy BIOS systems grub2-mkconfig -o /boot/grub2/grub.cfg #for modern UEFI systems grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
Note that for systems with more than 128 GB of memory, the recommended setting is crashkernel=512M@64M.
Then, issue a reboot for changes to take effect.
2 – Kdump configuration
Kdump could potentially dump the entire content of memory, so it might require a very large filesystem. Let’s start from the following situation:
#about 750GB of memory free -h total used free shared buff/cache available Mem: 755G 598G 136G 5.5G 19G 136G Swap: 23G 0B 23G #about 1TB of free space in the volume group vgs VG #PV #LV #SN Attr VSize VFree VGExaDb 2 6 0 wz--n- 1.63t <995.08g
So, in the worst case a 850GB filesystem is required to dump all the memory. Let’s proceed to create it:
#creation of the logical volume lvcreate -L 850G -n LVKDUMP1 VGExaDb #create an ext4 filesystem on the logical volume mkfs -t ext4 /dev/VGExaDb/LVKDUMP1 #create a label for the logical volume e2label /dev/mapper/VGExaDb-LVKDUMP1 KDUMP #check that the label has been successfully created ls -latr /dev/disk/by-label/ total 0 drwxr-xr-x 8 root root 160 Jun 6 17:17 .. ... lrwxrwxrwx 1 root root 10 Jun 6 17:30 KDUMP -> ../../dm-5
NOTE: It is important to create the label, because it will be used in kdump configuration file. Logical volume names are not accepted.
NOTE: Creating a dedicated logical volume is useful because if there is the need to take back the allocated space, it can easily be done with LVM.
Next, create the directory and mount the logical volume on it. Edit the /etc/fstab file to make configuration persistent after a reboot.
mkdir /kdump mount /dev/VGExaDb/LVKDUMP1 /kdump mkdir /kdump/crashfiles #directory where dumps will be actually written vi /etc/fstab ... /dev/VGExaDb/LVKDUMP1 /kdump ext4 defaults 1 1
Next, modify the kdump configuration file (the default is /etc/kdump.conf):
#add or edit the follwing lines ext4 LABEL=KDUMP path crashfiles
Finally, start the kdump daemon:
systemctl enable kdump systemctl start kdump systemctl status kdump
3 – Test – optional
To test if it is working, reboot the system and check if kdump automatically starts up.
systemctl is-active kdump
Then, issue the following commands to force a kernel crash:
echo 1 > /proc/sys/kernel/sysrq echo c > /proc/sysrq-trigger
Once the system is up again, vmcore files should be located under /kdump/crashfiles directory.
That’s all, see you in the next tutorial!