I’m using XEM7320 with SYZYGY ADC to make a high-speed data acquisition application in python without losing any samples. Since the sampling rate is high (more than 1MSps), I put the pipe function into another thread to call it as soon as it returns. However, I realize that other data processing procedures in the main thread increase the delay between two consecutive pipe function calls. I would like you to share some recommendations with me about how to handle the data as soon as it arrives from the pipe function.
The procedure that I am using now is as follows and it works fine until 800kSps:
- User initiates data acquisition.
- A thread starts a function.
- This function has a ReadFromBlockPipeOut function inside while loop.
- As soon as FPGA prepares data, pipeout returns and data is appended to a queue, and pipeout is recalled again.
- In main thread, I process the data or just save it as txt file.
In theory by USB3.0 I can transfer ((300MB * 8 Mbit/MB) / 32 Mbit/Msample) 75 MegaSample/sec. How can I store that much data properly in Python? Also, my application should handle plotting a block of data.