LVM Mirroring Overview

Continuing with our LVM how to series, in this KB, we will build on the knowledge from our LVM Configuration KB, and explore two sided logical volume (LV) mirroring.  In this KB, we will guide you through the creation of a two sided mirror from a new LV and the steps necessary to add mirroring to an existing LV.

Note: Mirroring is usually thought of as two way (in which two physical copies of the volume exist.)  However, when creating LVM mirrors, you can define as many mirrors of a LV as you wish limited only by the number of physical volumes (PVs) available in the Volume Group (VG).  Each side of the mirror requires one or more PVs.

LVM mirroring is most commonly used for data protection and high availability of mission critical data.  Similar to RAID 1, LVM mirroring ensures fault tolerance without any associated downtime should either leg of the mirror experience a hardware failure.  In the event of a failure, LVM converts the mirrored volume into a linear volume and continues to operate as before, but without the mirrored redundancy.

LVM mirroring at a minimum requires three physical volumes:

1. A volume for one side of the mirror
2. A volume for other side of the mirror
3. A volume for the log

Note: We recommend for recoverability purposes that the PV's be three separate physical disks.  Although not recommended, the PV's could also be a combination of partitions from one more physical disks but understand that a hardware failure could render the data on the LVM mirror useless - which somewhat defeats the purpose of LVM mirroring.

Creating New Mirror Volumes

For our first example, we will show you how to create a new mirrored logical volume (LV) from free space on our volume group (VG).  We are going to create a new 3GB mirrored LV called DumpMirror on the volume group TCPDumpVolGRP Let's begin:

1. Let's first check our TCPDumpVolGRP volume group to see if it has enough free space to support a 3GB mirrored LV
   
   

   [root@Linux01 ~]# vgs VG #PV #LV #SN Attr VSize VFree TCPDumpVolGRP 3 1 0 wz--n- 11.91G 9.91G VolGroup00 1 5 0 wz--n- 9.88G 0 [root@Linux01 ~]#

   Note: Since our 3GB mirrored LV will require ~6GB (3GB x 2 legs = 6GB + Log Volume), the 9.91GB free should suffice.
   
   
2. Let's also check to ensure the TCPDumpVolGRP volume group has enough free physical volumes (PV) available
   
   

   [root@Linux01 ~]# pvs PV VG Fmt Attr PSize PFree /dev/sda2 VolGroup00 lvm2 a- 9.88G 0 /dev/sdb1 TCPDumpVolGRP lvm2 a- 3.97G 3.97G /dev/sdc1 TCPDumpVolGRP lvm2 a- 3.97G 1.97G /dev/sdd1 TCPDumpVolGRP lvm2 a- 3.97G 3.97G [root@Linux01 ~]#

   Note: Two sided mirrors require three physical volumes which we appear to have with free space available on them
   
   
3. We will now create the mirrored LV.  Creating a mirrored LV is much the same as the creation of a non-mirrored LV with the exception of the '-m 1' command line switch.  The '-m 1' tells LVM you wish to create a two sided mirror
   
   

   [root@Linux01 ~]# lvcreate -L 3G -n DumpMirror -m 1 TCPDumpVolGRP Logical volume "DumpMirror" created [root@Linux01 ~]#

   
   
4. Creating the mirrored LV can take sometime.  Monitor the sync process until it reaches 100%
   
   

   [root@Linux01 ~]# lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert 4GLV TCPDumpVolGRP -wi-ao 2.00G DumpMirror TCPDumpVolGRP mwi-ao 3.00G DumpMirror_mlog 5.21 LogVol00 VolGroup00 -wi-ao 1.00G LogVol01 VolGroup00 -wi-ao 1.94G LogVol02 VolGroup00 -wi-ao 1.00G LogVol03 VolGroup00 -wi-ao 2.00G LogVol04 VolGroup00 -wi-ao 3.94G [root@Linux01 ~]#

   Note: The sync process is only ~5% complete, this could take a while
   
   

   [root@Linux01 ~]# lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert 4GLV TCPDumpVolGRP -wi-ao 2.00G DumpMirror TCPDumpVolGRP mwi-ao 3.00G DumpMirror_mlog 100.00 LogVol00 VolGroup00 -wi-ao 1.00G LogVol01 VolGroup00 -wi-ao 1.94G LogVol02 VolGroup00 -wi-ao 1.00G LogVol03 VolGroup00 -wi-ao 2.00G LogVol04 VolGroup00 -wi-ao 3.94G [root@Linux01 ~]#

   Note: The LV is now synced and ready for use
   
   
5. Inspect the LV to confirm the devices that are in use by the mirror
   
   

   [root@Linux01 ~]# lvs -a -o +devices LV VG Attr LSize Log Copy% Devices 4GLV TCPDumpVolGRP -wi-ao 2.00G /dev/sdc1(0) DumpMirror TCPDumpVolGRP mwi-a- 3.00G DumpMirror_mlog 100.00 DumpMirror_mimage_0(0), DumpMirror_mimage_1(0) [DumpMirror_mimage_0] TCPDumpVolGRP iwi-ao 3.00G /dev/sdb1(0) [DumpMirror_mimage_1] TCPDumpVolGRP iwi-ao 3.00G /dev/sdd1(0) [DumpMirror_mlog] TCPDumpVolGRP lwi-ao 32.00M /dev/sdc1(64) LogVol00 VolGroup00 -wi-ao 1.00G /dev/sda2(126) LogVol01 VolGroup00 -wi-ao 1.94G /dev/sda2(254) LogVol02 VolGroup00 -wi-ao 1.00G /dev/sda2(158) LogVol03 VolGroup00 -wi-ao 2.00G /dev/sda2(190) LogVol04 VolGroup00 -wi-ao 3.94G /dev/sda2(0) [root@Linux01 ~]#

   Note: Some columns were removed from the above output.  You will notice DumpMirror uses /dev/sdb1 and /dev/sdd1.  You will also notice the logs for DumpMirror are held on /dev/sdc1
   
   
6. Format DumpMirror
   
   

   [root@Linux01 ~]# mke2fs -j /dev/TCPDumpVolGRP/DumpMirror mke2fs 1.39 (29-May-2006) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) 393216 inodes, 786432 blocks 39321 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=805306368 24 block groups 32768 blocks per group, 32768 fragments per group 16384 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912 Writing inode tables: done                            Creating journal (16384 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 21 mounts or 180 days, whichever comes first.  Use tune2fs -c or -i to override. [root@Linux01 ~]#

   
   
7. Mount DumpMirror on the mount point of /mirror
   
   

   [root@Linux01 ~]# mkdir /mirror [root@Linux01 ~]# mount /dev/TCPDumpVolGRP/DumpMirror /mirror

   
   
8. Add DumpMirror to the /etc/fstab file
   
   

   /dev/TCPDumpVolGRP/DumpMirror /mirror ext3 defaults 0 0

   
   

Nice work.  You know have a fault tolerant logical volume! We are now going to continue with the KB and show you how to add mirroring to an existing logical volume.

Mirror From Existing Volume

We are now going to look at how to create a mirror from an existing logical volume (LV.)  For this portion of the KB, we will assume you have an LV in use on your system and would like it mirrored.

Note: If you do not have any LV's on your system and need help creating your first LV, see our LVM Configuration KB.

In the following example, we will convert the logical volume DataLV from a linear to mirrored LV.  Lets begin!

1. Review the output of pvs, vgs and lvs so we know the layout of DataLV and confirm the requirements for the mirror
   
   

   [root@Linux01 ~]# lvs -a -o +devices LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices DataLV TCPDumpVolGRP -wi-ao 2.5G /dev/sdb1(0) LogVol00 VolGroup00 -wi-ao 1.00G /dev/sda2(126) LogVol01 VolGroup00 -wi-ao 1.94G /dev/sda2(254) LogVol02 VolGroup00 -wi-ao 1.00G /dev/sda2(158) LogVol03 VolGroup00 -wi-ao 2.00G /dev/sda2(190) LogVol04 VolGroup00 -wi-ao 3.94G /dev/sda2(0) [root@Linux01 ~]#

   Note: You will notice that DataLV is 2.5GB in size and resides on /dev/sdb1.
   
   

   [root@Linux01 ~]# pvs PV VG Fmt Attr PSize PFree /dev/sda2 VolGroup00 lvm2 a- 9.88G 0 /dev/sdb1 TCPDumpVolGRP lvm2 a- 3.97G 1.47G /dev/sdc1 TCPDumpVolGRP lvm2 a- 3.97G 3.97G /dev/sdd1 TCPDumpVolGRP lvm2 a- 3.97G 3.97G [root@Linux01 ~]#

   Note: pvs confirms our assumptions showing that some of sdb1 is consumed by DataLV.  We are also able to confirm we have the three required PV's necessary for the mirror.
   
   

   [root@Linux01 ~]# vgs VG #PV #LV #SN Attr VSize VFree TCPDumpVolGRP 3 1 0 wz--n- 11.91G 9.41G VolGroup00 1 5 0 wz--n- 9.88G 0 [root@Linux01 ~]#

   Note: A free space check on our volume group shows we have enough free space for the 2.5GB mirror required for DataLV
   
   
2. Convert the logical volume to a mirror
   
   

   [root@Linux01 Data]# lvconvert -m1 TCPDumpVolGRP/DataLV TCPDumpVolGRP/DataLV: Converted: 12.5% TCPDumpVolGRP/DataLV: Converted: 26.2% TCPDumpVolGRP/DataLV: Converted: 40.0% TCPDumpVolGRP/DataLV: Converted: 53.8% TCPDumpVolGRP/DataLV: Converted: 68.8% TCPDumpVolGRP/DataLV: Converted: 82.5% TCPDumpVolGRP/DataLV: Converted: 96.2% TCPDumpVolGRP/DataLV: Converted: 100.0% Logical volume DataLV converted. [root@Linux01 Data]#

   Note: The lvconvert command can take a considerable amount of time depending on how big a mirror is to be created.  Go grab a beer or read some more articles on TCPDump while you wait, but do not break out of the process.
   
   
3. Confirm the mirror has been created
   
   

   [root@Linux01 ~]# lvs -a -o +devices LV VG Attr LSize Log Copy% Devices DataLV TCPDumpVolGRP mwi-a- 2.50G DataLV_mlog 100.00 DataLV_mimage_0(0), DataLV_mimage_1(0) [DataLV_mimage_0] TCPDumpVolGRP iwi-ao 2.50G /dev/sdb1(0) [DataLV_mimage_1] TCPDumpVolGRP iwi-ao 2.50G /dev/sdc1(0) [DataLV_mlog] TCPDumpVolGRP lwi-ao 32.00M /dev/sdd1(0) LogVol00 VolGroup00 -wi-ao 1.00G /dev/sda2(126) LogVol01 VolGroup00 -wi-ao 1.94G /dev/sda2(254) LogVol02 VolGroup00 -wi-ao 1.00G /dev/sda2(158) LogVol03 VolGroup00 -wi-ao 2.00G /dev/sda2(190) LogVol04 VolGroup00 -wi-ao 3.94G /dev/sda2(0) [root@Linux01 ~]#

   Note: Some columns were removed from the above output.  You will notice DataLV uses /dev/sdb1 and /dev/sdc1.  You will also notice the mirror logs for DumpMirror are held on /dev/sdd1

Note: For additional details on the Attr column (LVM display attributes) of the lvs, vgs, and pvs commands, please see our LVM Attributes KB.

Nice work, your logical volume is now fault tolerant!

Add this page to your favorite website

Comments

Add New Search

sims  - why a log |27/03/2010 01:47:24 Question: why does one need a whole partition for log? What is the logs purpose? Is this not a mirror? I'm a bit confused. Why would a mirror require 3 equal sized partitions/LV/etc? Reply | Quote

CoolKoon  - Because..... |12/05/2011 16:32:48 The way mirroring works in LVM is that the system creates arbitrary data blocks (e.g. 512 KB in size) and it's these individual blocks that are kept in sync across the drives. The log is the essential mechanism for the system to keep the data of sync status in across reboots, hence it can't be on any of the mirrored drives. You can also configure the system to keep the log in the RAM, but this means that your drives will have to be synced every time your system's rebooted (which's a VERY bad idea, because syncing's among the slowest LVM/RAID operations ever). Reply | Quote

Anonymous |09/08/2011 18:43:37 That does not sound like a real software mirror. Sounds more like strip+ parity... where the log acts as recovery/management. Sounds point less as a fault tolerant solution... If you are going to go the route of 3 physical drives why not just create a software raid? Reply | Quote

Dark night |10/09/2011 09:48:00 U confusing man Reply | Quote

Anonymous |12/10/2011 13:20:33 I think the confusion people are experiencing is over the lvcreate command and how it automaticaly picks /dev/sdb1 and /dev/sdd1 as the mirrors and /dev/sdc1 as the log. FYI - thanks for the article. Reply | Quote

LinuxIsFun  - re: |12/10/2011 13:25:13 I'm wordering if lvm will let you set preferred reads to a specifc device in the mirror like to /dev/sdd1 instead of /dev/sdb1. We use veritas to mirror between low speed drives and SSD cache drives and we set the preferred reads to the SSD drives to get exceptional read performance. If lvm can do this it would be awesome. Reply | Quote

jonty  - Order of legs in a mirror |26/10/2011 08:28:04 @LinuxIsFun - I want to know the exact same thing: if I mirror an SSD and a hard drive which of these two devices will be used when I read from the mirror. I did an hour of googling and came up with this: * LVM2 is built on top of device mapper. LVM2 decides which parts of the physical volumes will be used in the mirror and then passes instructions to device mapper. Then device mapper creates the mirror, handles all the I/O for the mirror, and lets LVM2 use it. * Device mapper used to use round-robin scheduling to divide the workload between the physical volumes in the mirror. But this has changed in recent versions of device mapper and it now does what you and I would both like: the mirror reads from the first physical volume and only switches to the second if there is a fault with the first. This was discussed on the dm-devel mailing list between Miklos Vajna and Jonathan Brassow in February 2011. You can see a copy of this at: http://www.spinics.net/lists/dm-devel/msg14676.html Reply | Quote

Rahul  - LVM |15/10/2011 15:56:16 The requirement pf third drive/partition is definitely to store mirror logs. When we run pvdisplay, vgdisplay and lvdisplay kind of commands, such commands fetch info from these logs. Also after reboot, system gets to know about such mirroring info from mirror logs like how much percent of data is mirrored. Now we can also NOT have a third drive for logs. It's done by using an option mirror core log with lvcreate, while creating a mirror lv. In this case, logs will be stored on to the core drives/partitions i.e. on drives carruing data. Try searching mirror core log Reply | Quote

Michal Belica  - LVM core log |11/11/2011 06:43:33 Hi, if I'm reading it correctly, the core log mode keeps the log in memory, and not on disk with the data so you'll get resync on reboot. I've also been searching for a way to get rid of the log part without the need to resync everytime, but without any success. Regards. MB Reply | Quote

Anonymous |21/11/2011 10:45:32 Nice post ... Really help.. Reply | Quote

zafer |21/11/2011 10:47:17 This is really a nice post.. If possbile please let us know how to unmirror it. And how to mirror in cluster filesystem ? Reply | Quote

Alexis Wilke |26/12/2011 02:08:57 The lvconvert command will let you switch between mirror (-m) and stripes (-s). You may want to read the manual pages of the lvconvert command first. The manual pages are actually really good quality. Reply | Quote

Alexis Wilke |26/12/2011 04:49:04 Actually, you may want to use -m 0 (--mirror followed by the number zero) to return to linear and "break" the mirror functionality. Reply | Quote

Anonymous |16/05/2012 23:29:38 Reply | Quote

Write comment
Name:
Email:	do not notifynotify
Website:
Title:
UBBCode:	-color-aquablackbluefuchsiagraygreenlimemaroonnavyolivepurpleredsilvertealwhiteyellow -size-tinysmallmediumlargehuge
	Please input the anti-spam code that you can read in the image.

!joomlacomment 4.0 Copyright (C) 2009 Compojoom.com . All rights reserved."