Description:
Recently I bought Syba SD-PEX50055 for a planned upgrade of my server (I wanted to use it with Samsung 850 PRO and ORICO 3.5 HDD Enclouser). I read about this device a lot, people wrote different things, but I was sure that it will work fine with my hardware. But ... I was wrong (at least from the begining).
Device information:
06:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 01) (prog-if 01 [AHCI 1.0])
Subsystem: ASMedia Technology Inc. Device 1060
Flags: bus master, fast devsel, latency 0, IRQ 44, NUMA node 0
I/O ports at d000 [size=8]
I/O ports at cc00 [size=4]
I/O ports at dc00 [size=8]
I/O ports at d800 [size=4]
I/O ports at d400 [size=32]
Memory at fa5ee000 (32-bit, non-prefetchable) [size=512]
Expansion ROM at fa5f0000 [disabled] [size=64K]
Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
Capabilities: [78] Power Management version 3
Capabilities: [80] Express Legacy Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Kernel driver in use: ahci
Kernel modules: ahci
Problem:
...
[ 12.272045] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 12.272243] ata1.00: link online but device misclassified
[ 12.272245] ata1: link online but 1 devices misclassified, retrying
[ 12.272432] ata1: reset failed (errno=-11), retrying in 10 secs
[ 22.271375] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 22.271574] ata1.00: link online but device misclassified
[ 22.271575] ata1: link online but 1 devices misclassified, retrying
[ 22.271764] ata1: reset failed (errno=-11), retrying in 35 secs
[ 56.807161] ata1: limiting SATA link speed to 3.0 Gbps
[ 57.268952] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
[ 57.269152] ata1.00: link online but device misclassified
[ 57.269153] ata1: link online but 1 devices misclassified, device detection might fail
[ 57.573933] ata2: SATA link down (SStatus 0 SControl 300)
...
[ 175.914701] xhci_hcd 0000:07:00.0: ERROR: unexpected setup address command completion code 0x11.
[ 181.122351] xhci_hcd 0000:07:00.0: Command completion event does not match command
[ 181.122421] xhci_hcd 0000:07:00.0: Setup ERROR: setup address command for slot 1.
...
In my case all devices (SATA and USB) couldn't be initialized with above errors in dmesg. At first I thought about "old" kernel (3.10.0-514.16.1) because my second OS with a kernel 4.9.6 worked without any problems. That assumption was wrong. I found two cases alike my - first and second - which gave me a hint for a resolution.
After some investigations I realized that the cause was a kernel parameter - "intel_iommu=on" that I used for pci-passthrough. I disabled it and all works fine now.
BTW "intel_iommu=on acpi=off" works, but I didn't try to assign pci-e device to a virtual machine.
Benchmarks:
readtest: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
writetest: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.2.8
Starting 2 processes
^Cbs: 2 (f=2): [r(1),w(1)] [1.5% done] [59160KB/105.7MB/0KB /s] [14.8K/26.9K/0 iops] [eta 02h:14m:10s]
fio: terminating on signal 2
readtest: (groupid=0, jobs=1): err= 0: pid=7932: Wed May 17 20:15:08 2017
read : io=7086.1MB, bw=61252KB/s, iops=15312, runt=118478msec
slat (usec): min=2, max=118, avg= 4.73, stdev= 2.77
clat (usec): min=78, max=152389, avg=2083.22, stdev=8950.43
lat (usec): min=82, max=152392, avg=2088.11, stdev=8950.50
clat percentiles (usec):
| 1.00th=[ 243], 5.00th=[ 338], 10.00th=[ 414], 20.00th=[ 556],
| 30.00th=[ 700], 40.00th=[ 844], 50.00th=[ 988], 60.00th=[ 1128],
| 70.00th=[ 1288], 80.00th=[ 1480], 90.00th=[ 1784], 95.00th=[ 2040],
| 99.00th=[76288], 99.50th=[77312], 99.90th=[78336], 99.95th=[79360],
| 99.99th=[101888]
bw (KB /s): min=50625, max=80664, per=100.00%, avg=61557.70, stdev=7302.66
lat (usec) : 100=0.01%, 250=1.13%, 500=15.05%, 750=17.13%, 1000=17.41%
lat (msec) : 2=43.72%, 4=4.14%, 10=0.04%, 20=0.01%, 100=1.33%
lat (msec) : 250=0.05%
cpu : usr=4.57%, sys=10.35%, ctx=424836, majf=0, minf=3497
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=1814244/w=0/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=32
writetest: (groupid=0, jobs=1): err= 0: pid=7934: Wed May 17 20:15:08 2017
write: io=11871MB, bw=102600KB/s, iops=25650, runt=118479msec
slat (usec): min=2, max=197, avg= 5.29, stdev= 2.65
clat (usec): min=70, max=152377, avg=1240.46, stdev=6898.13
lat (usec): min=85, max=152381, avg=1245.91, stdev=6898.15
clat percentiles (usec):
| 1.00th=[ 112], 5.00th=[ 183], 10.00th=[ 227], 20.00th=[ 318],
| 30.00th=[ 422], 40.00th=[ 516], 50.00th=[ 620], 60.00th=[ 708],
| 70.00th=[ 812], 80.00th=[ 908], 90.00th=[ 1012], 95.00th=[ 1080],
| 99.00th=[ 1272], 99.50th=[75264], 99.90th=[76288], 99.95th=[76288],
| 99.99th=[100864]
bw (KB /s): min=82704, max=124280, per=100.00%, avg=103103.39, stdev=10485.98
lat (usec) : 100=0.04%, 250=12.18%, 500=25.85%, 750=25.96%, 1000=24.47%
lat (msec) : 2=10.66%, 4=0.01%, 100=0.79%, 250=0.03%
cpu : usr=13.65%, sys=22.27%, ctx=1416705, majf=0, minf=5182
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=3038995/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=32
Run status group 0 (all jobs):
READ: io=7086.1MB, aggrb=61251KB/s, minb=61251KB/s, maxb=61251KB/s, mint=118478msec, maxt=118478msec
WRITE: io=11871MB, aggrb=102600KB/s, minb=102600KB/s, maxb=102600KB/s, mint=118479msec, maxt=118479msec
Disk stats (read/write):
sdd: ios=1810243/3033742, merge=2/1, ticks=3705451/3709574, in_queue=7415507, util=99.97%
echo 3 > /proc/sys/vm/drop_caches
dd if=/mnt/ram/file of=/mnt/ssd/file bs=4M oflag=direct
1764+1 records in
1764+1 records out
7402856448 bytes (7.4 GB) copied, 22.2746 s, 332 MB/s
echo 3 > /proc/sys/vm/drop_caches
dd if=/mnt/ram/file of=/mnt/ssd/file bs=4M
1764+1 records in
1764+1 records out
7402856448 bytes (7.4 GB) copied, 9.67306 s, 765 MB/s