Setup: XEM3005, C++ FrontPanel API, Mac OS X
I am trying to put a data acquisition system together and have had some trouble. I isolated my problem to timing in the block-throttled pipe-out transfer. When I do a transfer of 8 MB with a block size of 1024 bytes, there is a pause in the transfer after the first two blocks (2048 bytes). The pause can be between 15 and 30 ms. After that point, the rest of the transfer seems to go along just fine. I’m doing this in C++ under Mac OS X with a call to ReadFromBlockPipeOut.
When I transfer larger amounts of data with successive calls to ReadFromBlockPipeOut, I see a timing gap between the calls (which cannot be avoided), but I also see the pause mentioned above after 2048 bytes for each call.
I could get around the 15 to 30 ms lag by buffering data to RAM. The data rate is too large to buffer on the FPGA for this length of time. But, I wonder why there should be a lag within the transfer at all. The overall throughput is good at 25 MB/s.
My data acquistion code includes a FIFO between the data collection and the block throttled pipe. I stripped out everything except what I used to watch the timing and the resulting Verilog file is attached. The code counts cycles of ti_clk at 48 MHz and feeds bits 25 through 10 to the pipe so that each increment of the words in the output on the host corresponds to 21.33 us. I saw gaps between 750 and 1500 in the final data.
When I reduced the transfer block to 512 bytes, the lag was observed after 1536 bytes being transferred. Three blocks in this case as opposed to blocks in the tests with blocks of 1024 bytes.
I hope that this lag can be eliminated. Or documented.
I also would like to see the maximum transfer size increased from 16777215. Why not 4294967295? I’m not sure that I could use 4 GB, but I could use more than 16 MB.
Bob Jensen
pipe_time_test.txt (1056 Bytes)
Bob–
I’ll set this up and give it a shot on our Windows machine here. However, I suspect you’re up against an OS-related latency. USB does not guarantee any latency for bulk transfers. USB is not a real-time protocol, so it is difficult to make any guarantees.
Thanks for looking into this. I would have been less surprised if the pause existed randomly throughout the transfer. But after the first 2048 bytes, everything ran smoothly for the next 8 MB. And always after the first 2048 bytes. The behavior gives the impression that the software is waiting for something to happen, giving up, then doing the real work.
I don’t expect that you’ll see this with windows based on my looking at transfers via an XML interface to FrontPanel under XP. Probably in the library for the Mac OS.
Thanks again.
Bob–
If you already have a bitfile and XML file for testing purposes, could you send them along to [email protected]? It would save time on our end setting up this test.
I will send those along later today. Thanks.
Yes, it seems that the Mac version has this latency, but the Windows version does not. The only explanation I can provide now is that the Mac is scheduling the transfers at a lower priority and other system tasks are getting in the way.
On Windows, we use overlapping, “asynchronous” transfers to increase throughput. This is not presently being done on Mac or Linux, but we are looking into it. When this is implemented, it may help reduce this latency you are noticing.
We do not have a schedule for the implementation of asynchronous transfers on Mac or Linux at this time.
Thanks for looking into that. It still seems curious to me that there is a long pause after the first few thousand bytes are transferred and then no significant pause for the next few million. I’ve since gone around this problem and included an SDRAM-based FIFO between two on-chip FIFOs, with all that between the data collection and the pipe out. This easily soaks up the 20 or 30 ms delay at the transfer start and the 150 ms delay between calls to the pipe transfer between disk writes.
Thanks again for the prompt support.