PCI pipe write speeds not matching documentation

Spartan-6 PCI Express Module
1

The user’s manual says that the PCI speeds are 170MB/s write, 200 MB/s read. I ran the demo PipeTest code and got these results. The read result isn’t too bad (190, ie 5% slower) but the write result is significantly slower (111 MB/s best case) instead of 170 MB/s.

Our application needs ~100MB/s throughput, so that is not very much overhead.

I’ve got a brand new Core i7-2600, with a Gigabyte X68Xp-UD3 motherboard, with 8GB memory on Windows7 64 bit.

[FONT=“Courier New”]
E:\pipetest>PipeTest.exe PipeTest-xem6110-lx45.bit bench
---- Opal Kelly ---- PipeTest Application v1.0 ----
FrontPanel DLL loaded. Built: Mar 13 2011 17:15:39
Found a device: XEM6110-LX45
Device firmware version: 1.0
Device serial number: 11420000QN
Device device ID: Opal Kelly XEM6110
FrontPanel support is not enabled.
UpdateWireIns (1000 calls) Duration: 0.089 seconds – 11235.96 calls/s
UpdateWireOuts (1000 calls) Duration: 0.140 seconds – 7142.86 calls/s
ActivateTriggerIns (1000 calls) Duration: 0.002 seconds – 500000.00 calls/s
UpdateTriggerOuts (1000 calls) Duration: 0.139 seconds – 7194.24 calls/s
Read BS:1024 SS:4194304 TS:67108864 Duration: 0.336 seconds – 190.48 MB/s
Read BS:1024 SS:4194304 TS:33554432 Duration: 0.169 seconds – 189.35 MB/s
Read BS:1024 SS:4194304 TS:16777216 Duration: 0.084 seconds – 190.48 MB/s
Read BS:1024 SS:4194304 TS:8388608 Duration: 0.042 seconds – 190.48 MB/s
Read BS:1024 SS:4194304 TS:4194304 Duration: 0.021 seconds – 190.48 MB/s
Read BS:1024 SS:1048576 TS:33554432 Duration: 0.168 seconds – 190.48 MB/s
Read BS:1024 SS:262144 TS:33554432 Duration: 0.175 seconds – 182.86 MB/s
Read BS:1024 SS:65536 TS:16777216 Duration: 0.101 seconds – 158.42 MB/s
Read BS:1024 SS:16384 TS:4194304 Duration: 0.036 seconds – 111.11 MB/s
Read BS:1024 SS:4096 TS:1048576 Duration: 0.016 seconds – 62.50 MB/s
Read BS:1024 SS:1024 TS:1048576 Duration: 0.048 seconds – 20.83 MB/s
Read BS:900 SS:1048500 TS:33553800 Duration: 0.169 seconds – 183.43 MB/s
Read BS:800 SS:1048000 TS:33554400 Duration: 0.168 seconds – 184.52 MB/s
Read BS:700 SS:1047900 TS:33553800 Duration: 0.168 seconds – 184.52 MB/s
Read BS:600 SS:1048200 TS:33554400 Duration: 0.168 seconds – 184.52 MB/s
Read BS:512 SS:1048576 TS:33554432 Duration: 0.168 seconds – 190.48 MB/s
Read BS:500 SS:1048500 TS:33554000 Duration: 0.169 seconds – 183.43 MB/s
Read BS:400 SS:1048400 TS:16777200 Duration: 0.085 seconds – 176.47 MB/s
Read BS:300 SS:1048500 TS:16777200 Duration: 0.085 seconds – 176.47 MB/s
Read BS:256 SS:1048576 TS:16777216 Duration: 0.084 seconds – 190.48 MB/s
Read BS:200 SS:1048400 TS:8388600 Duration: 0.041 seconds – 170.73 MB/s
Read BS:128 SS:1048576 TS:8388608 Duration: 0.042 seconds – 190.48 MB/s
Read BS:100 SS:1048500 TS:8388600 Duration: 0.042 seconds – 166.67 MB/s
Write BS:1024 SS:4194304 TS:67108864 Duration: 0.590 seconds – 108.47 MB/s
Write BS:1024 SS:4194304 TS:33554432 Duration: 0.296 seconds – 108.11 MB/s
Write BS:1024 SS:4194304 TS:16777216 Duration: 0.147 seconds – 108.84 MB/s
Write BS:1024 SS:4194304 TS:8388608 Duration: 0.073 seconds – 109.59 MB/s
Write BS:1024 SS:4194304 TS:4194304 Duration: 0.036 seconds – 111.11 MB/s
Write BS:1024 SS:1048576 TS:33554432 Duration: 0.293 seconds – 109.22 MB/s
Write BS:1024 SS:262144 TS:33554432 Duration: 0.298 seconds – 107.38 MB/s
Write BS:1024 SS:65536 TS:16777216 Duration: 0.153 seconds – 104.58 MB/s
Write BS:1024 SS:16384 TS:4194304 Duration: 0.042 seconds – 95.24 MB/s
Write BS:1024 SS:4096 TS:1048576 Duration: 0.014 seconds – 71.43 MB/s
Write BS:1024 SS:1024 TS:1048576 Duration: 0.049 seconds – 20.41 MB/s
Write BS:900 SS:1048500 TS:33553800 Duration: 0.293 seconds – 105.80 MB/s
Write BS:800 SS:1048000 TS:33554400 Duration: 0.294 seconds – 105.44 MB/s
Write BS:700 SS:1047900 TS:33553800 Duration: 0.294 seconds – 105.44 MB/s
Write BS:600 SS:1048200 TS:33554400 Duration: 0.294 seconds – 105.44 MB/s
Write BS:512 SS:1048576 TS:33554432 Duration: 0.294 seconds – 108.84 MB/s
Write BS:500 SS:1048500 TS:33554000 Duration: 0.293 seconds – 105.80 MB/s
Write BS:400 SS:1048400 TS:16777200 Duration: 0.147 seconds – 102.04 MB/s
Write BS:300 SS:1048500 TS:16777200 Duration: 0.147 seconds – 102.04 MB/s
Write BS:256 SS:1048576 TS:16777216 Duration: 0.147 seconds – 108.84 MB/s
Write BS:200 SS:1048400 TS:8388600 Duration: 0.073 seconds – 95.89 MB/s
Write BS:128 SS:1048576 TS:8388608 Duration: 0.073 seconds – 109.59 MB/s
Write BS:100 SS:1048500 TS:8388600 Duration: 0.074 seconds – 94.59 MB/s

E:\pipetest>
[/FONT]

2

The original tests were done on PCs that do not perform out-of-order PCI requests. Our current host interface core does not support these writes and so we had to disable support for them which dropped performance a bit as you’ve noticed.

We plan to improve the host interface to support out-of-order PCI soon, although we don’t have a public timeline for that. If you’d like to try to get early versions for evaluation, please contact [email protected].