RAID контроллеры, непонятки со скоростью в разных схемах RAID Опции

[iwant2beahoster]

Получил доступ к тестовому стенду. И провел тесты.

3WARE 9650SE 16-port SATA 3Gb/s 256MB RAID 0,1,5,6,10,50,Single Disk, JBOD, PCI-E x8, KIT

[root@template ~]# tw_cli /c2 show all
/c2 Driver Version = 2.26.02.013
/c2 Model = 9650SE-16ML
/c2 Available Memory = 224MB
/c2 Firmware Version = FE9X 4.10.00.007
/c2 Bios Version = BE9X 4.08.00.002
/c2 Boot Loader Version = BL9X 3.08.00.001
/c2 Serial Number = L322626A8510931
/c2 PCB Version = Rev 032
/c2 PCHIP Version = 2.00
/c2 ACHIP Version = 1.90
/c2 Number of Ports = 16
/c2 Number of Drives = 16
/c2 Number of Units = 1
/c2 Total Optimal Units = 1
/c2 Not Optimal Units = 0
/c2 JBOD Export Policy = off
/c2 Disk Spinup Policy = 1
/c2 Spinup Stagger Time Policy (sec) = 1
/c2 Auto-Carving Policy = off
/c2 Auto-Carving Size = 2048 GB
/c2 Auto-Rebuild Policy = on
/c2 Rebuild Mode = Adaptive
/c2 Rebuild Rate = 1
/c2 Verify Mode = Adaptive
/c2 Verify Rate = 1
/c2 Controller Bus Type = PCIe
/c2 Controller Bus Width = 8 lanes
/c2 Controller Bus Speed = 2.5 Gbps/lane

К нему 16 штук 1TB SATA дисков Seagate ST31000340NS

Подключил 8 дисков к контроллеру и собрал их в RAID5

[root@template ~]# ./tw_cli /c4/u0 show
Unit UnitType Status %RCmpl %V/I/M Port Stripe Size(GB)
------------------------------------------------------------------------
u0 RAID-5 OK - - - 256K 6519.19
u0-0 DISK OK - - p0 - 931.312
u0-1 DISK OK - - p1 - 931.312
u0-2 DISK OK - - p2 - 931.312
u0-3 DISK OK - - p3 - 931.312
u0-4 DISK OK - - p4 - 931.312
u0-5 DISK OK - - p5 - 931.312
u0-6 DISK OK - - p6 - 931.312
u0-7 DISK OK - - p7 - 931.312
u0/v0 Volume - - - - - 6519.19

И проверил скорость чтения:

[root@template ~]# dd if=/dev/sdc of=/dev/null bs=8M iflag=direct
9530+0 records in
9529+0 records out
79935045632 bytes (80 GB) copied, 116.869 s, 684 MB/s
57059+0 records in
57058+0 records out
478637195264 bytes (479 GB) copied, 698.931 s, 685 MB/s
423807+0 records in
423806+0 records out
3555142402048 bytes (3.6 TB) copied, 5287.42 s, 672 MB/s

Собрал на том же стенде RAID10 с 16-тью дисками.

[root@template ~]# tw_cli /c2/u0 show
Unit UnitType Status %RCmpl %V/I/M Port Stripe Size(GB)
------------------------------------------------------------------------
u0 RAID-10 OK - - - 256K 7450.5
u0-0 RAID-1 OK - - - - -
u0-0-0 DISK OK - - p15 - 931.312
u0-0-1 DISK OK - - p11 - 931.312
u0-1 RAID-1 OK - - - - -
u0-1-0 DISK OK - - p7 - 931.312
u0-1-1 DISK OK - - p3 - 931.312
u0-2 RAID-1 OK - - - - -
u0-2-0 DISK OK - - p14 - 931.312
u0-2-1 DISK OK - - p10 - 931.312
u0-3 RAID-1 OK - - - - -
u0-3-0 DISK OK - - p6 - 931.312
u0-3-1 DISK OK - - p2 - 931.312
u0-4 RAID-1 OK - - - - -
u0-4-0 DISK OK - - p13 - 931.312
u0-4-1 DISK OK - - p9 - 931.312
u0-5 RAID-1 OK - - - - -
u0-5-0 DISK OK - - p5 - 931.312
u0-5-1 DISK OK - - p1 - 931.312
u0-6 RAID-1 OK - - - - -
u0-6-0 DISK OK - - p12 - 931.312
u0-6-1 DISK OK - - p8 - 931.312
u0-7 RAID-1 OK - - - - -
u0-7-0 DISK OK - - p4 - 931.312
u0-7-1 DISK OK - - p0 - 931.312
u0/v0 Volume - - - - - 8
u0/v1 Volume - - - - - 7442.5

Прогнал чтение.

[root@template ~]# dd if=/dev/sdb of=/dev/null bs=8M iflag=direct
469+0 records in
468+0 records out
3925868544 bytes (3.9 GB) copied, 6.84351 s, 574 MB/s
120888+0 records in
120887+0 records out
1014073655296 bytes (1.0 TB) copied, 1756.27 s, 577 MB/s

По моему это явная ерунда...

Пользователи 3ware отзовитесь! Что у вас со скоростью чтения?
Почему 8-ми дисковый RAID5 "делает" 16-ти дисковый RAID10 по чтению?

[iwant2beahoster]

Ivan, верно ли я вас понял?
RAID10 - 16 дисков.
Результаты тестов урезаны, оставлены первая и последняя запись и среднее.

[root@template ~]# disktest -K32 -ID -PT -h3 -T30 -B4k -r /dev/sdb
| 2011/11/21-20:24:24 | START | 21868 | v1.4.2 | /dev/sdb | Start args: -K32 -ID -PT -h3 -T30 -B4k -r (-N 15608053761) (-c) (-p R) (-D 100:0) (-t 0:2m) (-o 0)
| 2011/11/21-20:24:24 | INFO | 21868 | v1.4.2 | /dev/sdb | Starting pass
| 2011/11/21-20:24:27 | STAT | 21868 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 5016234.7B/s (4.78MB/s), IOPS 1224.7/s.
| 2011/11/21-20:24:54 | STAT | 21868 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 5135018.7B/s (4.90MB/s), IOPS 1253.7/s.
| 2011/11/21-20:24:54 | STAT | 21868 | v1.4.2 | /dev/sdb | Total read throughput: 5108940.8B/s (4.87MB/s), IOPS 1247.3/s.
| 2011/11/21-20:24:54 | END | 21868 | v1.4.2 | /dev/sdb | Test Done (Passed)
[root@template ~]# disktest -K64 -ID -PT -h3 -T30 -B4k -r /dev/sdb
| 2011/11/21-20:25:08 | START | 21911 | v1.4.2 | /dev/sdb | Start args: -K64 -ID -PT -h3 -T30 -B4k -r (-N 15608053761) (-c) (-p R) (-D 100:0) (-t 0:2m) (-o 0)
| 2011/11/21-20:25:08 | INFO | 21911 | v1.4.2 | /dev/sdb | Starting pass
| 2011/11/21-20:25:11 | STAT | 21911 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 6340608.0B/s (6.05MB/s), IOPS 1548.0/s.
| 2011/11/21-20:25:38 | STAT | 21911 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 6470314.7B/s (6.17MB/s), IOPS 1579.7/s.
| 2011/11/21-20:25:39 | STAT | 21911 | v1.4.2 | /dev/sdb | Total read throughput: 6390033.1B/s (6.09MB/s), IOPS 1560.1/s.
| 2011/11/21-20:25:39 | END | 21911 | v1.4.2 | /dev/sdb | Test Done (Passed)
[root@template ~]# disktest -K128 -ID -PT -h3 -T30 -B4k -r /dev/sdb
| 2011/11/21-20:25:50 | START | 21986 | v1.4.2 | /dev/sdb | Start args: -K128 -ID -PT -h3 -T30 -B4k -r (-N 15608053761) (-c) (-p R) (-D 100:0) (-t 0:2m) (-o 0)
| 2011/11/21-20:25:51 | INFO | 21986 | v1.4.2 | /dev/sdb | Starting pass
| 2011/11/21-20:25:54 | STAT | 21986 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 7609002.7B/s (7.26MB/s), IOPS 1857.7/s.
| 2011/11/21-20:26:21 | STAT | 21986 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 7588522.7B/s (7.24MB/s), IOPS 1852.7/s.
| 2011/11/21-20:26:21 | STAT | 21986 | v1.4.2 | /dev/sdb | Total read throughput: 7600128.0B/s (7.25MB/s), IOPS 1855.5/s.
| 2011/11/21-20:26:21 | END | 21986 | v1.4.2 | /dev/sdb | Test Done (Passed)
[root@template ~]# disktest -K256 -ID -PT -h3 -T30 -B4k -r /dev/sdb
| 2011/11/21-20:26:29 | START | 22122 | v1.4.2 | /dev/sdb | Start args: -K256 -ID -PT -h3 -T30 -B4k -r (-N 15608053761) (-c) (-p R) (-D 100:0) (-t 0:2m) (-o 0)
| 2011/11/21-20:26:30 | INFO | 22122 | v1.4.2 | /dev/sdb | Starting pass
| 2011/11/21-20:26:33 | STAT | 22122 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 7580330.7B/s (7.23MB/s), IOPS 1850.7/s.
| 2011/11/21-20:27:00 | STAT | 22122 | v1.4.2 | /dev/sdb | Heartbeat read throughput: 7544832.0B/s (7.20MB/s), IOPS 1842.0/s.
| 2011/11/21-20:27:00 | STAT | 22122 | v1.4.2 | /dev/sdb | Total read throughput: 7645593.6B/s (7.29MB/s), IOPS 1866.6/s.
| 2011/11/21-20:27:00 | END | 22122 | v1.4.2 | /dev/sdb | Test Done (Passed)

kpv спасибо за ссылку. Удивительно. Но может проблема всё же в 3Ware? Может у LSI, несмотря на то что они сейчас владеют 3ware, или скажем у Areca будут результаты получше?