How to adjust Linux SW RAID Disk Rebuild Speed


How to adjust Linux SW RAID Disk Rebuild Speed

You are here:


The process of rebuilding the RAID can be a slow process. Viewing the console and repeatedly running cat /proc/mdstat only to see a few points of a percentage increase can be tedious (although, you can use watch in this case). There is of course a reason for the ‘slowness’, particularly if you are using older hardware that relies on the stability of a lighter load.  But if you are using large disks for high storage capacity or have large backup drives with several terabytes of data being configured with RAID, then you may wish to increase the process speed. Here is the method we use at Fraction Servers to adjust Linux SW RAID rebuild speed.

1) /proc/sys/dev/raid/{speed_limit_max,speed_limit_min} kernel variables

To see current limits, enter:



To increase speed, enter:

### echo value > /proc/sys/dev/raid/speed_limit_min ###
echo 50000 > /proc/sys/dev/raid/speed_limit_min



### sysctl -w ###
sysctl -w

Usually, the speed_limit_max variable is high enough that it won’t affect performance but it is usually possible to speed up RAID rebuilds by increasing the speed_limit_min value. It’s worth noting that while this can speed up RAID rebuilds it may have an effect on the system as a whole. Faster rebuilds take up more system resources like disk bandwidth, CPU, and RAM. Larger arrays are usually able to support higher read/write minimum values as the disk bandwidth can be shared between the disks.

If the speed_limit_max variable does need to be changed it can be updated in a similar way.

### echo value > /proc/sys/dev/raid/speed_limit_max ###
echo 50000 > /proc/sys/dev/raid/speed_limit_max


### sysctl -w ###
sysctl -w

Do the same for if necessary. 50000 is an example, so adjust value to what is necessary for the server you are applying this to.

It should reset back to the default after rebooting.


2) Set read-ahead option


Set readahead (in 512-byte sectors) per raid device. The syntax is:

### Set read-ahead to 32 MiB ###
blockdev --setra 65536 /dev/md0
blockdev --setra 65536 /dev/md1


3) Disable NCQ on all disks


Determine which disks are part of the array and adjust this accordingly

### Sample for loop ###
for i in sd[abcde] # make sure you have the right disks
   echo 1 > /sys/block/$i/device/queue_depth


4) Bitmap Option


Bitmaps optimize rebuild time after a crash, or after removing and re-adding a device. Turn it on by typing the following command:

mdadm --grow --bitmap=internal /dev/md0


Once array rebuild or fully synced, disable bitmaps:

mdadm --grow --bitmap=none /dev/md0


5) Set stripe-cache_size for RAID5 or RAID 6


This is only available on RAID5 and RAID6 and boost sync performance by 3-6 times. It records the size (in pages per device) of the stripe cache which is used for synchronising all write operations to the array and all read operations if the array is degraded. The default is 256. Valid values are 17 to 32768. Increasing this number can increase performance in some situations, at some cost in system memory. Note, setting this value too high can result in an “out of memory” condition for the system. Use the following formula:

memory_consumed = system_page_size * nr_disks * stripe_cache_size


To set stripe_cache_size to 16 MiB for /dev/md0, type:

echo 16384 > /sys/block/md0/md/stripe_cache_size


To set stripe_cache_size to 32 MiB for /dev/md3, type:

echo 32768 > /sys/block/md3/md/stripe_cache_size






Table of Contents