[WBEL-users] Adding to RAID 1, only sees new partition as spare... {Scanned} {Scanned}

Kirby C. Bohling kbohling at birddog.com
Tue May 17 18:42:03 CDT 2005


On Tue, May 17, 2005 at 03:59:47PM -0700, Scott Silva wrote:
> Kirby C. Bohling wrote:
> 
> >On Tue, May 17, 2005 at 03:15:57PM -0700, Scott Silva wrote:
> >  
> >
> >>Benjamin Smith wrote:
> >>    
> >>
> >>>I have a server that does backups, and I want to replace an existing 160 GB 
> >>>drive, with a RAID 1 300 GB array. 
> >>>
> >>>I don't have enough IDE slots to mount all at the same time. So, I installed a 
> >>>single 300 GB HDD, configured it as /dev/md0 RAID 1, made a filesystem, 
> >>>mounted, and copied all the data over. It's seen and mounts fine with all 
> >>>data present. 
> >>>
> >>>Now, I've removed the original 160 GB HDD, and put the other new 300 GB in 
> >>>it's place. 
> >>>
> >>>I've tried and tried, and I can't get the 2nd 300 GB drive to sync up to the 
> >>>first 300 GB drive. It keeps seeing it as a spare... 
> >>>
> >>>What am I missing? 
> >>>
> >>>1st 300 GB partn in array (with data on it) /dev/hdh1 
> >>>2nd 300 GB partn in array (seen as spare) /dev/hde1 
> >>>      
> >>>
> >>Did you properly partition the new drive as type fd (linux raid)?
> >>
> >>Maybe you should use mdadm. I have had better luck with it over raidtools.
> >>you would use ;
> >>
> >>sfdisk -d /dev/hdh |sfdisk /dev/hde
> >>
> >>to clone the partition data, then
> >>
> >>mdadm -a /dev/md0 /dev/hde1
> >>    
> >>
> >
> >For whatever it's worth, I've had lots of problems doing just that.
> >I've attempted at various points to add a third mirror to a two disk
> >set.  I'll explain with a scenerio:
> >
> >I have drives /dev/sd{a,b,c}.  /dev/sda and /dev/sdb are in md0.
> >/dev/sdb failes.  I add sdc to md0.  I get a replacement in for sdb.
> >What I want to do is add sdb back to md0 let it re-sync, and then
> >remove sdc.  I've never been able to do that.  Instead, I have to
> >remove sdc (leaving only sda in the array), and then add sdb.
> >
> >I believe the problem is that the array configuration says, there is
> >one drive on the RAID metadata.  I have yet to find a tool that will
> >allow you to change that.  Any additional drives that get added,
> >must be spares.  I'm curious what it would take to edit the metadata
> >on the RAID array to convince it that you have two drives in the
> >array instead of one.
> >
> >You are probably stuck.  Maybe using mdadm you can create a new
> >array.  If you know which disk is going to get used as the source of
> >the mirror (I've never known that, so I'm afraid to try it).
> >
> >Possibly this has something that can modify the on disk metadata:
> >http://unthought.net/raidreconf/
> >
> >I've never played with that, but it's supposed to be really nifty,
> >if not production quality.
> >
> >    Thanks,
> >  
> >
> AFAIK you can only have 2 active drives in a raid mirror. The
> third drive seems a nifty idea, but I don't think it practical.
> You must remove one drive from the array before you can add
> another.  

I'm confident you can create a RAID1 devices with three disks.
Here, lets go see:

[root at hawk root]# lvcreate --size 500M --name disk1 /dev/std_fs
[...output removed...]
[root at hawk root]# lvcreate --size 500M --name disk2 /dev/std_fs
[...output removed...]
[root at hawk root]# lvcreate --size 500M --name disk3 /dev/std_fs
[...output removed...]

[root at hawk root]# mdadm --create /dev/md0 --level 1 --raid-devices 3 /dev/std_fs/disk{1,2,3}
mdadm: array /dev/md0 started.
[root at hawk root]# cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
Event: 1
md0 : active raid1 lvmh[2] lvmg[1] lvmf[0]
      524224 blocks [3/3] [UUU]
      [>....................]  resync =  2.5% (13440/524224)
finish=1.8min speed=4480K/sec
unused devices: <none>

There's an example session where I did exactly that on three LVM
devices.  What you can't do is change the number of disks in an
RAID array.  My guess is that this is a limitation because changing
the number of disks in a RAID0/5 array would completely change the
addressing mechanism.  No one considered that being able to change
the number of devices would be handy.  It might also be that
changing the number of devices has a lot of race conditions
involved, and very difficult to manage failure modes.

> In your example above you would add sdb as a spare, and software
> fail sdc. That will get sdb to resync.  Or just leave sdb as a
> spare, all ready to go, and it should auto failover and resync if
> sda or sdc fails.  

Yes, that's precisely how I did it.  It's not like I failed to get
the drive replaced.

> Yes there is a possibility of the other drive failing before the
> sync is done, but that is why you have backups.  

I'm not sure about you, but I have terrabyte filesystems.  Bringing
them off tape is reserved for fires, earthquakes, or other forms of
extreme badness.  Willfully putting myself one disk failure away
from disaster recovery isn't good risk management. Downtime is
fairly expensive where I work.

I have hot fail-over machines, because backups are neccessary, but a
last resort.   I have machines that the building empties out if they
don't work.   We generally are very low risk, keep spares of
absolutely everything in the building.

> The overhead of writing data to 3 drives and keeping them in sync
> would probably not be worth it.  Raid1 is already somewhat slower
> on writes, you would penalize it even more.  If you are truly
> paranoid, you could do raid10, but writes would be even worse.
> 

I already do RAID 10.  It's plenty speedy in my experience.
Finally, it's not like I swap out drives during heavy I/O when I
have two working drives.  I'm not a complete idiot.  I'm not running
triple mirror all the time, just during the phase where it avoids
the risk of a single drive.  If your drives has lots of reads (and
lots of writes), my guess is that re-sync'ing from 2 drives during
I/O load is after then resync'ing from 1.  You have two heads to
provide data to the readers.  Yes, the writes should have to wait
until all three have gotten the data written, but the writes are
what take a long time, the head seeks are.  Two heads are better
then one.  Unless you are almost exclusively writes is should be
faster from two disks rather then one.

    Thanks,
        Kirby









More information about the Whitebox-users mailing list