Recovering MD array and mdadm: Cannot get array info for /dev/md0

What a case! A long story short one of our disks got a bad disk in a software RAID1 setup and when we tried replacing the disk in a recovery Linux console we got the strange error of an MD device:

mdadm: Cannot get array info for /dev/md125

And ccording to the /proc/mdstat the device was there and mdadm -E reported the array was “clean”.

root@631019 ~ # mdadm --add /dev/md125 /dev/sdb2
mdadm: Cannot get array info for /dev/md125

root@631019 ~ # cat /proc/mdstat                                                        πŸ™
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md122 : inactive sda4[0](S)
      33520640 blocks super 1.2
       
md123 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
md124 : inactive sda3[0](S)
      4189184 blocks super 1.2
       
md125 : inactive sda2[0](S)
      1048512 blocks
       
unused devices: <none>

root@631019 ~ # mdadm -E /dev/sda2                                                      πŸ™
/dev/sda2:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : aff708ee:16669ffb:1a120e13:7e9185ae
  Creation Time : Thu Mar 14 15:10:21 2019
     Raid Level : raid1
  Used Dev Size : 1048512 (1023.94 MiB 1073.68 MB)
     Array Size : 1048512 (1023.94 MiB 1073.68 MB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 126

    Update Time : Thu Jul 11 10:22:17 2019
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
       Checksum : c1ee0a10 - correct
         Events : 103


      Number   Major   Minor   RaidDevice State
this     0       8        2        0      active sync   /dev/sda2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2

The important piece of information here is that the RAID1 is in an inactive state, which is really strange! It is perfectly normal to be started with one disk missing (the raid as you can see consists from 2 disks) and in read-only mode before mounting it. But here it is in an inactive state! The output of /proc/mdstat shows a sign of inappropriate assembly of all those arrays probably during the boot of the rescue Linux system – missing information or old version of mdadm utility or some other configuration loaded! In such states – inactive and as you see no information about the type of the arrays it is normal mdadm to report error it could not get current array info. The key word here is CURRENT despite mdadm misses it in the error output:

root@631019 ~ # mdadm --add /dev/md125 /dev/sdb2
mdadm: Cannot get array info for /dev/md125

Because in fact mdadm tries adding a disk in the currently loaded configuration, not the real one in your disks!

The solution

  1. Remove ALL current configuration by issuing multiple stop commands with mdadm, no inactive raids or any raids should be reported in “/proc/mdstat”.
  2. Remove (or better rename) mdadm configuration files in /etc/mdadm.conf (in some Linux distributions is /etc/mdadm/mdadm.conf).
  3. Rescan for MD devices with mdadm. The mdadm will load the configuration from your disks.
  4. Add the missing partitions to your software raid devices.

STEP 1) Remove all current MD configuration

Use mdadm with “–stop”. This will not delete any data (metadata for the raid and real data). It just remove the currently loaded configuration (which might be wrong!) from your kernel. We want to rescan it cleanly, again.

root@631019 ~ # mdadm --stop /dev/md122
mdadm: stopped /dev/md122
root@631019 ~ # mdadm --stop /dev/md123
mdadm: stopped /dev/md123
root@631019 ~ # mdadm --stop /dev/md124
mdadm: stopped /dev/md124
root@631019 ~ # mdadm --stop /dev/md125
mdadm: stopped /dev/md125
root@631019 ~ # cat /proc/mdstat 
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
unused devices: <none>

STEP 2) Remove all mdadm configuration files

Because they might be the problem. Imagine there was an old configuration file, which the rescue Linux (or your newly installed by some strange method server or ) mdadm may try assembling your raids using it and miss-configuring everything! Do not replace the file with a new mdadm scan, because it will be the same (or sort of) wrong configuration. Just remove the files!

root@631019 ~ # rm /etc/mdadm.conf 
root@631019 ~ # rm /etc/mdadm/mdadm.conf

STEP 3) Rescan for MD devices with mdadm

The mdadm will rescan your disks for metadata and will load a new configuration in the kernel reflecting what it has just found in the disks. A verbose option may be included but it is not mandatory. With the verbose option the user see what mdadm is doing and it can be discovered errors.

root@631019 ~ # mdadm --assemble --scan --verbose
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/loop0
mdadm: no recogniseable superblock on /dev/sdb5
mdadm: no recogniseable superblock on /dev/sdb4
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: Cannot assemble mbr metadata on /dev/sdb
mdadm: No super block found on /dev/sda2 (Expected magic a92b4efc, got 00000041)
mdadm: no RAID superblock on /dev/sda2
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got af461b01)
mdadm: no RAID superblock on /dev/sda1
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sda5 is identified as a member of /dev/md/srv4.local:5, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/srv48.local:5
mdadm: added /dev/sda5 to /dev/md/srv48.local:5 as 0
mdadm: /dev/md/srv48.local:5 assembled from 1 drive - not enough to start the array.
mdadm: looking for devices for further assembly
mdadm: /dev/sda4 is identified as a member of /dev/md/2113894:root, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/2113894:root
mdadm: added /dev/sda4 to /dev/md/2113894:root as 0
mdadm: /dev/md/2113894:root has been started with 1 drive (out of 2).
mdadm: looking for devices for further assembly
mdadm: /dev/sda3 is identified as a member of /dev/md/2113894:swap, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/2113894:swap
mdadm: added /dev/sda3 to /dev/md/2113894:swap as 0
mdadm: /dev/md/2113894:swap has been started with 1 drive (out of 2).
mdadm: looking for devices for further assembly
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md/2113894:swap
mdadm: no recogniseable superblock on /dev/md/2113894:root
mdadm: no recogniseable superblock on /dev/loop0
mdadm: no recogniseable superblock on /dev/sdb5
mdadm: no recogniseable superblock on /dev/sdb4
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: Cannot assemble mbr metadata on /dev/sdb
mdadm: /dev/sda5 is busy - skipping
mdadm: /dev/sda4 is busy - skipping
mdadm: /dev/sda3 is busy - skipping
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda1
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sda2 is identified as a member of /dev/md/126_0, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/126_0
mdadm: added /dev/sda2 to /dev/md/126_0 as 0
mdadm: /dev/md/126_0 has been started with 1 drive (out of 2).
mdadm: looking for devices for further assembly
mdadm: /dev/sda5 is busy - skipping
mdadm: /dev/sda4 is busy - skipping
mdadm: /dev/sda3 is busy - skipping
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md/126_0
mdadm: no recogniseable superblock on /dev/md/2113894:swap
mdadm: no recogniseable superblock on /dev/md/2113894:root
mdadm: no recogniseable superblock on /dev/loop0
mdadm: no recogniseable superblock on /dev/sdb5
mdadm: no recogniseable superblock on /dev/sdb4
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: Cannot assemble mbr metadata on /dev/sdb
mdadm: /dev/sda5 is busy - skipping
mdadm: /dev/sda4 is busy - skipping
mdadm: /dev/sda3 is busy - skipping
mdadm: /dev/sda2 is busy - skipping
mdadm: no recogniseable superblock on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda

A lot of information above, but if you inspect it deeply you will see multiple arrays are found and started with a single disk. And here is the new current configuration in the kernel, which looks a lot better than the previous one:

root@631019 ~ # cat /proc/mdstat 
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md124 : active (auto-read-only) raid1 sda2[0]
      1048512 blocks [2/1] [U_]
      
md125 : active (auto-read-only) raid1 sda3[0]
      4189184 blocks super 1.2 [2/1] [U_]
      
md126 : active (auto-read-only) raid1 sda4[0]
      33520640 blocks super 1.2 [2/1] [U_]
      
md127 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
unused devices: <none>

In fact, the 3 RAID1s are started successfully and we could use them, only one RAID is in an inactive state, but it is RAID0 and it is normal – this device cannot be recovered because it is RAID0 with a missing device (but this might be expected if you use RAID1).

STEP 4) Add the missing partitions to your software raid devices.

First, you should partition accordingly your new disk (probably with parted) if you haven’t done it yet use this (first copy then randomize the GUIDs):

sgdisk /dev/sda -R /dev/sdb
sgdisk -G /dev/sdb

And we can add the partitions to recover the RAID devices successfully:

root@631019 ~ # mdadm --add /dev/md124 /dev/sdb2
mdadm: added /dev/sdb2
root@631019 ~ # cat /proc/mdstat                
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md124 : active raid1 sdb2[2] sda2[0]
      1048512 blocks [2/1] [U_]
      [===============>.....]  recovery = 75.0% (786432/1048512) finish=0.0min speed=196608K/sec
      
md125 : active (auto-read-only) raid1 sda3[0]
      4189184 blocks super 1.2 [2/1] [U_]
      
md126 : active (auto-read-only) raid1 sda4[0]
      33520640 blocks super 1.2 [2/1] [U_]
      
md127 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
unused devices: <none>
root@631019 ~ # mdadm --add /dev/md125 /dev/sdb3
mdadm: added /dev/sdb3
root@631019 ~ # mdadm --add /dev/md126 /dev/sdb4
mdadm: added /dev/sdb3

After a while all RAID1 are rebuilt successfully!

The whole story

A disk in a software RAID1 got faulty and we stopped the server to change the disk. The colocation replaced the disk with a new one and booted the server. Because it was CentOS 7 it did not boot normally because one of the raid devices included in the grub.cfg could not be started – Centos 7 Server hangs up on boot after deleting a software raid (mdadm device) Then we booted in the rescue console “4.19.0-1-grml-amd64 #1 SMP Debian 4.19.8-1+grml.1 (2018-12-11) x86_64 x86_64 x86_64 GNU/Linux”. We partitioned our new disk with the same layout like the old one and executed the first command to add the new raid partition to the raid device and we got a nasty error:

root@631019 ~ # mdadm --add /dev/md123 /dev/sdb5
mdadm: Cannot get array info for /dev/md123

Checking the mdstat show the first sign of troubles. Some strange raid0 and ddf mdadm driver loaded?

root@631019 ~ # cat /proc/mdstat                                                                                                                                       πŸ™
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md122 : inactive sda4[0](S)
      33520640 blocks super 1.2
       
md123 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
md124 : inactive sda3[0](S)
      4189184 blocks super 1.2
       
md125 : inactive sda2[0](S)
      1048512 blocks
       
md126 : active raid0 sdb[0]
      1952972800 blocks super external:/md127/10 64k chunks
      
md127 : inactive sdb[0](S)
      541784 blocks super external:ddf

It appeared no raids in “/dev/sdb”, which was normally the disk was new.

root@631019 ~ # mdadm -E /dev/sdb
/dev/sdb:
   MBR Magic : aa55
Partition[0] :   3907029167 sectors at            1 (type ee)

The kernel also had not tried building the RAIDs (there were version 1.2, so it was normal):

root@631019 ~ # dmesg|grep md    
[    0.000000] Linux version 4.19.0-1-grml-amd64 (team@grml.org) (gcc version 8.2.0 (Debian 8.2.0-12)) #1 SMP Debian 4.19.8-1+grml.1 (2018-12-11)
[    2.542302] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[    2.542381] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[    2.542402] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[    2.594464] usb usb1: Manufacturer: Linux 4.19.0-1-grml-amd64 ehci_hcd
[    2.614448] usb usb2: Manufacturer: Linux 4.19.0-1-grml-amd64 ehci_hcd
[    3.704709] md126: detected capacity change from 0 to 1999844147200
[   20.542235] systemd[1]: RTC configured in localtime, applying delta of 0 minutes to system time.
[   20.604080] systemd[1]: systemd 232 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
[   20.622746] systemd[1]: Detected architecture x86-64.
[   20.623153] systemd[1]: Running with unpopulated /etc.
[   20.635723] systemd[1]: Set hostname to <lswrescue>.
[   20.636175] systemd[1]: Initializing machine ID from random generator.
[   20.662804] systemd[1]: Populated /etc with preset unit settings.
[   20.775061] systemd-sysv-generator[866]: Overwriting existing symlink /run/systemd/generator.late/grml-reboot.service with real service.
[   20.818898] systemd[1]: Listening on Journal Socket.
[   20.819321] systemd[1]: Reached target Swap.
[   20.819727] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
[   20.820108] systemd[1]: Reached target Encrypted Volumes.
[   20.996378] systemd-journald[869]: Received request to flush runtime journal from PID 1

The we tried first to stop an array and then to assemble it again and then to add to rebuild the array. It failed with the same error:

root@631019 ~ # mdadm -E /dev/sda2                                                                                                                                     πŸ™
/dev/sda2:
          Magic : a92b4efc
        Version : 0.90.00
           UUID : aff708ee:16669ffb:1a120e13:7e9185ae
  Creation Time : Thu Mar 14 15:10:21 2019
     Raid Level : raid1
  Used Dev Size : 1048512 (1023.94 MiB 1073.68 MB)
     Array Size : 1048512 (1023.94 MiB 1073.68 MB)
   Raid Devices : 2
  Total Devices : 2
Preferred Minor : 126

    Update Time : Thu Jul 11 10:22:17 2019
          State : clean
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0
       Checksum : c1ee0a10 - correct
         Events : 103


      Number   Major   Minor   RaidDevice State
this     0       8        2        0      active sync   /dev/sda2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
root@631019 ~ # mdadm --manage /dev/md122 --add /dev/sdb4
mdadm: Cannot get array info for /dev/md122
root@631019 ~ # mdadm --assemble /dev/md122                                                                                                                            πŸ™
mdadm: /dev/md122 not identified in config file.
root@631019 ~ # mdadm --assemble /dev/md122 --scan                                                                                                                     πŸ™
mdadm: /dev/md122 not identified in config file.
root@631019 ~ # mdadm --assemble /dev/md122 --scan --force                                                                                                             πŸ™
mdadm: /dev/md122 not identified in config file.
root@631019 ~ # mdadm --stop /dev/md122                                                                                                                                πŸ™
mdadm: stopped /dev/md122
root@631019 ~ # mdadm --assemble /dev/md122 --scan --force
mdadm: /dev/md122 not identified in config file.
root@631019 ~ # cat /proc/mdstat                                                                                                                                       πŸ™
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md123 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
md124 : inactive sda3[0](S)
      4189184 blocks super 1.2
       
md125 : inactive sda2[0](S)
      1048512 blocks
       
md126 : active raid0 sdb[0]
      1952972800 blocks super external:/md127/10 64k chunks
      
md127 : inactive sdb[0](S)
      541784 blocks super external:ddf
       
unused devices: <none>
root@631019 ~ # mdadm --assemble /dev/md122 /dev/sda4
mdadm: /dev/md122 assembled from 1 drive - need all 2 to start it (use --run to insist).
1 root@631019 ~ # cat /proc/mdstat                                                                                                                                       πŸ™
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md122 : inactive sda4[0](S)
      33520640 blocks super 1.2
       
md123 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
md124 : inactive sda3[0](S)
      4189184 blocks super 1.2
       
md125 : inactive sda2[0](S)
      1048512 blocks
       
md126 : active raid0 sdb[0]
      1952972800 blocks super external:/md127/10 64k chunks
      
md127 : inactive sdb[0](S)
      541784 blocks super external:ddf
       
unused devices: <none>
root@631019 ~ # mdadm --add /dev/md122 /dev/sdb4
mdadm: Cannot get array info for /dev/md122

First, the assemble command failed, but specifying the exact device and partition inactive device appeared again in the current configuration in “/proc/mdstat”. We were in the same situation as before – inactive devices and no information about the RAID in the loaded configuration in the kernel. So the mdadm loaded the wrong configuration again and we saw there was a really strange configuration in the “/etc/mdadm/mdadm.conf”. As you can see the line “mdadm: /dev/md122 not identified in config file”? What?

# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default (built-in), scan all partitions (/proc/partitions) and all
# containers for MD superblocks. alternatively, specify devices to scan, using
# wildcards if desired.
#DEVICE partitions containers

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays
ARRAY /dev/md126 UUID=aff708ee:16669ffb:1a120e13:7e9185ae
ARRAY /dev/md/swap  metadata=1.2 UUID=2960efb0:54bfb2af:babacd0b:c95e34c0 name=2113894:swap
ARRAY /dev/md/root  metadata=1.2 UUID=e6367857:94aa2b7a:70a26151:63c60d0b name=2113894:root
ARRAY /dev/md/5  metadata=1.2 UUID=53a6a93b:769c450c:589a622e:00c66915 name=srv.local:5
ARRAY metadata=ddf UUID=9451717d:ca7996b6:e79e8c3c:e381784d
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=0 UUID=01d23819:8d2cb691:843b82dd:1998dabe
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=1 UUID=ed021fc6:45d423de:93fd03fd:f03785c3
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=2 UUID=daf0c60e:eca16660:7d65c7e0:0fa3c2cb
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=3 UUID=3ab9e3ab:0b90cfff:f5ecdf19:b523e1d5
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=4 UUID=9898dc76:e9f662e2:0ec3bbba:10affe69
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=5 UUID=77245b0c:b24d7261:4f7ee128:4d431b86
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=6 UUID=3b06e0d2:8883165a:e3ae19a5:1b403352
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=7 UUID=47440a60:a27caa49:e035dc9e:31d6771b
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=8 UUID=7b256bc9:4a8f465c:e4ff4318:df8361d2
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=9 UUID=d09e9945:9d22102d:b895e189:e498fe91
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=10 UUID=90d64ef2:eeb2d004:823cbfaa:6f2081e1
ARRAY container=9451717d:ca7996b6:e79e8c3c:e381784d member=11 UUID=18a87c3b:1993f6f8:30fb116f:51db1cfc

We removed the file and then stopped one of the inactive devices and this time we tried assembling the device with a verbose option and here we saw the problem!

root@631019 ~ # mdadm --stop /dev/md122                                                                                                                                πŸ™
mdadm: stopped /dev/md122
root@631019 ~ # mdadm --assemble /dev/md122 --update=summaries --verbose
mdadm: looking for devices for /dev/md122
mdadm: No super block found on /dev/loop0 (Expected magic a92b4efc, got 637ab963)
mdadm: no RAID superblock on /dev/loop0
mdadm: No super block found on /dev/sdb5 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb5
mdadm: No super block found on /dev/sdb4 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb4
mdadm: No super block found on /dev/sdb3 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb3
mdadm: No super block found on /dev/sdb2 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb2
mdadm: No super block found on /dev/sdb1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb1
mdadm: No super block found on /dev/sdb (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sdb
mdadm: /dev/sda5 has wrong uuid.
mdadm: /dev/sda3 has wrong uuid.
mdadm: No super block found on /dev/sda2 (Expected magic a92b4efc, got 00000041)
mdadm: no RAID superblock on /dev/sda2
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got af461b01)
mdadm: no RAID superblock on /dev/sda1
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: --update=summaries not understood for 1.x metadata

mdadm reported that some raid partitions has wrong uuid, so apperantly the currenly loaded configuration was totally wrong and we must delete all mdadm configuration files, to stop all of the software MD devices and rescan again with verbose option to see what is going on. And this time it worked the assemble command finished and the RAID1 devices appeared with one drive OK and one missing!

root@631019 ~ # mdadm --stop /dev/md122
mdadm: stopped /dev/md122
root@631019 ~ # mdadm --stop /dev/md123
mdadm: stopped /dev/md123
root@631019 ~ # mdadm --stop /dev/md124
mdadm: stopped /dev/md124
root@631019 ~ # mdadm --stop /dev/md125
mdadm: stopped /dev/md125
root@631019 ~ # cat /proc/mdstat 
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
unused devices: <none>
root@631019 ~ # rm /etc/mdadm.conf 
root@631019 ~ # rm /etc/mdadm/mdadm.conf
root@631019 ~ # mdadm --assemble --scan --verbose
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/loop0
mdadm: no recogniseable superblock on /dev/sdb5
mdadm: no recogniseable superblock on /dev/sdb4
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: Cannot assemble mbr metadata on /dev/sdb
mdadm: No super block found on /dev/sda2 (Expected magic a92b4efc, got 00000041)
mdadm: no RAID superblock on /dev/sda2
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got af461b01)
mdadm: no RAID superblock on /dev/sda1
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sda5 is identified as a member of /dev/md/srv.local:5, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/srv.local:5
mdadm: added /dev/sda5 to /dev/md/srv.local:5 as 0
mdadm: /dev/md/srv.local:5 assembled from 1 drive - not enough to start the array.
mdadm: looking for devices for further assembly
mdadm: /dev/sda4 is identified as a member of /dev/md/2113894:root, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/2113894:root
mdadm: added /dev/sda4 to /dev/md/2113894:root as 0
mdadm: /dev/md/2113894:root has been started with 1 drive (out of 2).
mdadm: looking for devices for further assembly
mdadm: /dev/sda3 is identified as a member of /dev/md/2113894:swap, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/2113894:swap
mdadm: added /dev/sda3 to /dev/md/2113894:swap as 0
mdadm: /dev/md/2113894:swap has been started with 1 drive (out of 2).
mdadm: looking for devices for further assembly
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md/2113894:swap
mdadm: no recogniseable superblock on /dev/md/2113894:root
mdadm: no recogniseable superblock on /dev/loop0
mdadm: no recogniseable superblock on /dev/sdb5
mdadm: no recogniseable superblock on /dev/sdb4
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: Cannot assemble mbr metadata on /dev/sdb
mdadm: /dev/sda5 is busy - skipping
mdadm: /dev/sda4 is busy - skipping
mdadm: /dev/sda3 is busy - skipping
mdadm: No super block found on /dev/sda1 (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda1
mdadm: No super block found on /dev/sda (Expected magic a92b4efc, got 00000000)
mdadm: no RAID superblock on /dev/sda
mdadm: /dev/sda2 is identified as a member of /dev/md/126_0, slot 0.
mdadm: no uptodate device for slot 1 of /dev/md/126_0
mdadm: added /dev/sda2 to /dev/md/126_0 as 0
mdadm: /dev/md/126_0 has been started with 1 drive (out of 2).
mdadm: looking for devices for further assembly
mdadm: /dev/sda5 is busy - skipping
mdadm: /dev/sda4 is busy - skipping
mdadm: /dev/sda3 is busy - skipping
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md/126_0
mdadm: no recogniseable superblock on /dev/md/2113894:swap
mdadm: no recogniseable superblock on /dev/md/2113894:root
mdadm: no recogniseable superblock on /dev/loop0
mdadm: no recogniseable superblock on /dev/sdb5
mdadm: no recogniseable superblock on /dev/sdb4
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: no recogniseable superblock on /dev/sdb1
mdadm: Cannot assemble mbr metadata on /dev/sdb
mdadm: /dev/sda5 is busy - skipping
mdadm: /dev/sda4 is busy - skipping
mdadm: /dev/sda3 is busy - skipping
mdadm: /dev/sda2 is busy - skipping
mdadm: no recogniseable superblock on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda                                                                                                                 πŸ™
130 root@631019 ~ # cat /proc/mdstat                                                                                                                                     πŸ™
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md124 : active (auto-read-only) raid1 sda2[0]
      1048512 blocks [2/1] [U_]
      
md125 : active (auto-read-only) raid1 sda3[0]
      4189184 blocks super 1.2 [2/1] [U_]
      
md126 : active (auto-read-only) raid1 sda4[0]
      33520640 blocks super 1.2 [2/1] [U_]
      
md127 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
unused devices: <none>

And adding raid partitions was successful:

root@631019 ~ # mdadm --add /dev/md124 /dev/sdb2
mdadm: added /dev/sdb2
root@631019 ~ # cat /proc/mdstat                
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md124 : active raid1 sdb2[2] sda2[0]
      1048512 blocks [2/1] [U_]
      [===============>.....]  recovery = 75.0% (786432/1048512) finish=0.0min speed=196608K/sec
      
md125 : active (auto-read-only) raid1 sda3[0]
      4189184 blocks super 1.2 [2/1] [U_]
      
md126 : active (auto-read-only) raid1 sda4[0]
      33520640 blocks super 1.2 [2/1] [U_]
      
md127 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
unused devices: <none>
root@631019 ~ # mdadm --add /dev/md125 /dev/sdb3
mdadm: added /dev/sdb3
root@631019 ~ # cat /proc/mdstat                
Personalities : [raid0] [linear] [multipath] [raid1] [raid6] [raid5] [raid4] [raid10] 
md124 : active raid1 sdb2[1] sda2[0]
      1048512 blocks [2/2] [UU]
      
md125 : active raid1 sdb3[2] sda3[0]
      4189184 blocks super 1.2 [2/1] [U_]
      [=>...................]  recovery =  6.2% (261888/4189184) finish=0.2min speed=261888K/sec
      
md126 : active (auto-read-only) raid1 sda4[0]
      33520640 blocks super 1.2 [2/1] [U_]
      
md127 : inactive sda5[0](S)
      1914583040 blocks super 1.2
       
unused devices: <none>

Leave a Reply

Your email address will not be published. Required fields are marked *