Unstable readings from WireOuts

FrontPanel
1

I have a design that can use a wide range of clock frequencies so to make things simple, I am using hi_clock as the base clock of my circuit (which is pure synchronous).

My design has two synchronous counters, Counter1 of 32 bits and Counter 2 of 48 bits and they are read by the host using 2 and 3 WireOuts respectively (ep21, ep20 for Counter1 and ep 2a, 29 and 28 Counter2).

When I read the two counter values from the host API (using a single UpdateWireOuts() and then 5 calls to GetWireOutValue()), I get the correct values about 90% of the times but in the other 10%, some of the ep values are returned as 0xffff.

When the reading is bad, typically all the 5 eps are read as 0xffff but sometimes, just the eps of Counter1 or just the eps of Counter 2 are read as 0xffff and the other counter is read OK. I did not notice so far a case where only some of the eps of Counter1 or Counter2 are bad.

I tried to clock my circuit on both hi_clk, ti_clk, not(ti_clock) but for no avail.

I am pretty sure that my circuit operates properly and the 0xffff values are not actual counter values.

Below is more information that may help identifying the problem. If more inforamtion is needed, please let me know.

Thanks,

Kam

Board configuration:
Device firmware version: 1.3
Device serial number: JVCNkEpdGp
Device device ID: Opal Kelly XEM3010

Host:
HP Workstation xw4200 running Ubuntu.

Compiler
gcc version 3.3.5 (Debian 1:3.3.5-8ubuntu2.1)

Note:
During compilation I get the following warning: /usr/bin/ld: warning: “libstdc++.so.6, needed by ./libokjFrontPanel.so, may conflict with libstdc++.so.5.”

When I run ldd on my executable (called ‘test’) I get:
ldd test
libokjFrontPanel.so => /usr/lib/libokjFrontPanel.so (0xb7ee2000)
libusb-0.1.so.4 => /lib/libusb-0.1.so.4 (0xb7eda000)
libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0xb7e20000) < /lib/tls/i686/cmov/libm.so.6 (0xb7dfe000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7df3000)
libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7cc6000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0xb7bee000) < /lib/ld-linux.so.2 (0xb7f0f000)

FPGA Flow:
Verilog. Xilinx WebPack 8.1I (on Unbuntu )

PLL Initialization:
if (true == dev->IsOpen()) {
printf(“Found a device: XEM3010\n”);

            // Set output 1 to 50 MHz.
            okCPLL22393 pll;
            pll.SetPLLParameters(0, 400, 48, true);
            pll.SetOutputDivider(0, 8);
            pll.SetOutputSource(0, okCPLL22393::ClkSrc_PLL0_0);
            pll.SetOutputEnable(0, true);
            ((okCUsbXEM3010 *)dev)->SetPLLConfiguration(pll);;
    }

Log output from my test program

153 Reading wire outs, base=0x20, count=2
154   Reading value of ep 0x21: 0x0000 (0)
155   Reading value of ep 0x20: 0x004d (77)
156 Reading wire outs, base=0x28, count=3
157   Reading value of ep 0x2a: 0x0000 (0)
158   Reading value of ep 0x29: 0x0075 (117)
159   Reading value of ep 0x28: 0xd2d2 (53970)
160 Counter1 =       77 (0x004d), Counter2 = 7721682 (0x75d2d2)

161 Reading wire outs, base=0x20, count=2
162   Reading value of ep 0x21: 0x0000 (0)
163   Reading value of ep 0x20: 0x0052 (82)
164 Reading wire outs, base=0x28, count=3
165   Reading value of ep 0x2a: 0x0000 (0)
166   Reading value of ep 0x29: 0x007d (125)
167   Reading value of ep 0x28: 0x3720 (14112)
168 Counter1 =       82 (0x0052), Counter2 = 8206112 (0x7d3720)

169 Reading wire outs, base=0x20, count=2
170   Reading value of ep 0x21: 0xffff (65535)
171   Reading value of ep 0x20: 0xffff (65535)
172 Reading wire outs, base=0x28, count=3
173   Reading value of ep 0x2a: 0xffff (65535)
174   Reading value of ep 0x29: 0xffff (65535)
175   Reading value of ep 0x28: 0xffff (65535)
176 Counter1 = 4294967295 (0xffffffff), Counter2 = 281474976710655 (0xffffffffffff)

177 Reading wire outs, base=0x20, count=2
178   Reading value of ep 0x21: 0x0000 (0)
179   Reading value of ep 0x20: 0x005c (92)
180 Reading wire outs, base=0x28, count=3
181   Reading value of ep 0x2a: 0x0000 (0)
182   Reading value of ep 0x29: 0x008b (139)
183   Reading value of ep 0x28: 0xfe1e (65054)
184 Counter1 =       92 (0x005c), Counter2 = 9174558 (0x8bfe1e)

185 Reading wire outs, base=0x20, count=2
186   Reading value of ep 0x21: 0x0061 (97)
187   Reading value of ep 0x20: 0x0061 (97)
188 Reading wire outs, base=0x28, count=3
189   Reading value of ep 0x2a: 0xffff (65535)
190   Reading value of ep 0x29: 0xffff (65535)
191   Reading value of ep 0x28: 0xffff (65535)
192 Counter1 =  6357089 (0x610061), Counter2 = 281474976710655 (0xffffffffffff)

193 Reading wire outs, base=0x20, count=2
194   Reading value of ep 0x21: 0x0000 (0)
195   Reading value of ep 0x20: 0x0066 (102)
196 Reading wire outs, base=0x28, count=3
197   Reading value of ep 0x2a: 0x0000 (0)
198   Reading value of ep 0x29: 0x009a (154)
199   Reading value of ep 0x28: 0xc398 (50072)
200 Counter1 =      102 (0x0066), Counter2 = 10142616 (0x9ac398)
2

Kam-

A couple questions:

  1. I see you’re using the Java shared-object file (libokjFrontPanel…) Technically, this is only for use with Java and has been “SWIGged” to create this object file. I’m not sure what happens when you link to this from a non-Java application. Can you reproduce the same results with the statically-linked library?

  2. Have you been able to reproduce these results on a Windows machine? (to see if it is Linux specific)

  3. Any chance you could provide a test case including a C++ program (with the static lib) and HDL source to build with the Xilinx tools ?

3
  1. I see you’re using the Java shared-object file (libokjFrontPanel…)
    Technically, this is only for use with Java and has been “SWIGged” to create
    this object file. I’m not sure what happens when you link to this from a
    non-Java application. Can you reproduce the same results with the
    statically-linked library?

I had problem linking with the static library, this is why I used the dynamic lib. Static linking would be my first choice for this project. Can you provide instructions and a sample makefile how to compile/link for the example the DES sample under Linux?

  1. Have you been able to reproduce these results on a Windows machine?
    (to >see if it is Linux specific)

I did not try it on Windows (don’t have development suit on Windows). However, it tried it with a) feeding the WireOut ep’s with fixed values and b) clocking my circuit from clk1 (50MHZ) rather than ti_clock. In both cases, the host API provided the expected values (though I am not sure how safe it is to feed values from clk1 to ti_host without proper syncrhonization). This suggests to me that the problem is on the FPGA side and not the host side (?).

  1. Any chance you could provide a test case including a C++ program (with
    the static lib) and HDL source to build with the Xilinx tools ?

Sure, I can mail it to you but first I need to be able to statically link (please see #1 above).

Thanks,

Kam

4

Kam-

Attached to this post is the Makefile we use for the DESTester app.

Makefile.txt (260 Bytes)

5

Thanks. It is working now (but not with the compiler of my choice).

I tried the Makefile you sent and got the same linking error I got earlier with your static lib (this is why I used the dynamic lib).

Next I tried a newer version of gcc (4.0.0) instead of the old one I was using (3.3.5) and was able to link with the static lib without the conflict warning I got before.

Now, when I run my circuit on ti_clk and use binary linked with gcc 4.0.0, everything seems to be just fine.

So, it seems that the problem was due to compiler version mismatch.

I will check if moving to gcc 4.0.0 is feasable for us (we need to link with other stuff) and if not, will address it as a seperate issue.

Thanks for your help.

Tal

6

Tal-

I find that to be a strange manifestation. I’ll keep it in mind, however. We plan on moving everything to dynamic libraries soon and getting rid of the static libs on Windows and Mac. Hopefully that will solve your problem, too.

7

We decided to upgrade our project to gcc 4.x I think we should be OK with the current software you provide. Thanks for your help.

While I am writing, here are few minor assorted notes:

1.Our target platform does not have hotplug (though my desktop does) so as somebody mention in a different thread, the current trend is to move from hotplug to udev. You may want to change the installation notes to address udev based installation. (it is not urgent for us, we should be able to figure out ourselves how to do that).

2.Since interfacing with logic that is not synchronized with ti_clock requires careful synchronization (right?), how about solving it once yourself and sharing the solution with your users (similar to the buffered pipes you provide)?

3.UpdateWireOuts() with 5 WireOut ep’s instantiated takes about 1200 usec on my machine. The USB frame period is about 125 usec, right?. If so, how come it takes 10 times the frame period?

4.If it is not too late, I suggest to use a different name for the FrontPanel end points. The terms end points already has a different meaning in the context of USB.

  1. I have a small test design that use a counter to blink one of the LED’s. I use the USB to download the .bit file and it works as expected. However, when I disconnect the USB cable (after loading the .bit file), the LED stops blinking (the board is powered by the AC adapter). Any reason why the design stop working when I disconnet the USB cable? Is it because I am using ti_clock?

  2. Is there any disadvantage of clocking my circuit from ti_clk (or should I use hi_clock?) compared to using for example clk1? I want to avoid syncronization across clock domains and running my circuit on ti_clock (48mhz?) is ok for my design. Since ti_clk derived from the oscilator itself (pre PLL), it should have less jitter, right?

I really appreciate the cleanness of your design. It makes my job much easier.

Thanks,

Kam

8

Hi Kam-

To answer your questions / comments:

  1. Yes, we have since moved to udev and our next release will reflect this.

  2. The WireIn/WireOut endpoints are asynchronous endpoints and should not be considered synchronous. In the context of the triggers and pipes, pipes are used to provide a synchronous interface for multi-bit signals. Wires are more like switch banks or pushbuttons. In our designs, we generally send a “trigger” to latch the wire contents after they have changed – if we require synchronous delivery.

  3. I’m not sure on this. We would have to look into it. I’m not sure the OS is capable of hitting the bus back-to-back in cases like this. For example, for bulk transfers, asynchronous queueing is used to keep a command queue full.

  4. While unfortunate, we couldn’t come up with anything better :frowning: Most of our customers know little about USB and prefer to keep it that way.

  5. The TI_CLK and HI_CLK are the same. When the HI_CLK (sourced from the USB microcontroller) disappears, your design stops. The HI_CLK will disappear if the USB micro enters a suspend mode. We have discussed an option to keep the micro out of suspend mode.

  6. You can run from HI_CLK (or TI_CLK) if you wish. It will have slightly lower jitter than the PLL-produced clocks.