Linux RAID Alerting

In this KB, we will show you how to use mdadm to notify you should a device in your Linux Software RAID array fail.  Monitoring your RAID array is important because you must replace a failed device as soon as possible.  This is because different levels of RAID have varying abilities to sustain device loss (see our Linux RAID How-to for a description of the different RAID levels and their sustainability with failed devices) so it's important you treat failures with urgency and replace the failed devices as quickly as possible.

We have seen in some of our other RAID KB's (Removing Failed RAID Devices and Staring and Stopping RAID Arrays) some examples of commands that can be issued from the shell to get the status of the array ('cat /proc/mdstat' and 'mdadm --detail /dev/md0'.)  However these commands won't help you detect or send an alert should a failure occur.

In the following examples, we will show you how to be proactive and configure mdadm for alert notification.  One final word of advice before we begin, this is a quick and dirty way to monitor RAID (not to mention the lack of scalability in the enterprise.)  There are plenty of open source and commercial tools that can also be used to monitor RAID in the enterprise.  If you plan to have several systems running software RAID, I would highly encourage the use of an external monitoring package.  That said, Let's begin!

1. We will begin by sending a test message from our system to ensure mail routing is working as expected.
   
   

   [root@Linux01 /]# mdadm --monitor --scan --mail= This e-mail address is being protected from spambots. You need JavaScript enabled to view it /dev/md0 -t -1 [root@Linux01 /]#

   
   If the test worked, you should have received an e-mail that looks something like the following:
   
   

   This is an automatically generated mail message from mdadm running on Linux01.tcpdump.com A TestMessage event had been detected on md device /dev/md0. Faithfully yours, etc. P.S. The /proc/mdstat file currently contains the following: Personalities : [raid6] [raid5] [raid4] md0 : active raid5 sdc1[1] sdb1[0] sdd1[2] 8385664 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU] unused devices:

   
   
2. Once we have confirmed we can receive mail from the system, we can either setup an init script to run mdadm with the monitor command line switch (in daemon mode) or setup a cronjob. 
   
   Syntax to run mdadm in daemon mode:
   

   mdadm --monitor --scan --mail= This e-mail address is being protected from spambots. You need JavaScript enabled to view it --delay=3600 /dev/md0 -f

   
   Cron Syntax to check the array every 5 minutes:

   */5   *    *    *    *    /sbin/mdadm --monitor --scan --mail= This e-mail address is being protected from spambots. You need JavaScript enabled to view it /dev/md0 -1

I'll leave it to you to figure out how best to setup alerting on your system.  Hopefully these examples helped! Best of luck and thanks for reading!

Add this page to your favorite website

Comments

Add New Search

Write comment
Name:
Email:	do not notifynotify
Website:
Title:
UBBCode:	-color-aquablackbluefuchsiagraygreenlimemaroonnavyolivepurpleredsilvertealwhiteyellow -size-tinysmallmediumlargehuge
	Please input the anti-spam code that you can read in the image.

!joomlacomment 4.0 Copyright (C) 2009 Compojoom.com . All rights reserved."