TEST_SE0_NAK

[Christos]

Hi,

I'm using an external USB device that uses TEST_SE0_NAK.

I do have installed libusb-win32.

It seems though that in some systems the behaviour is ok whereas in others (also having libusb-win32 driver installed), the performance is not good having lots of errors.

Is there something that  I can do to have proper Test_SE0_NAK support in my libusb driver installations?
Is Test_SE0_NAK supported in libusb-win32 1.2.4 ?

Christos

[Xiaofan Chen]

On Sun, May 22, 2011 at 2:23 AM, Christos <[hidden email]> wrote:
> I'm using an external USB device that uses TEST_SE0_NAK.

Hmm, that is an USB-IF USB Electric Test thingy, right? What
is the relation with libusb-win32?

> I do have installed libusb-win32.

Tell us more about the device and how libusb-win32 is used? Which
version? Filter driver mode or device driver mode?

> It seems though that in some systems the behaviour is ok whereas in others
> (also having libusb-win32 driver installed), the performance is not good
> having lots of errors.
>
> Is there something that  I can do to have proper Test_SE0_NAK support in my
> libusb driver installations?
> Is Test_SE0_NAK supported in libusb-win32 1.2.4 ?



--
Xiaofan

------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Christos]

It is a proprietary DSP device, its not a test thing.
Actually I'm utilizing the TEST_SE0_NAK from its firmware in order to signal that data are not yet available since the device cannot send NAK back in the usual way on a IN request.
The device needs USB2 highspeed bandwidth (~3MBytes/sec read and ~3MBytes/sec write concurrently).
It has only one configuration with two bulk endpoints,  one IN and one OUT, maxed to 512 bytes each.
The application that utilizes this device is a DSP application and there is a simple method on communicating with the device. The host first reads a buffer of 8192bytes with usb_bulk_read and then it writes a buffer of 8192 bytes with usb_bulk_write and these operations should happen within a predefined 12msec timeout.

<Read/Write thread pseudocode>
repeat loop
rc = usb_bulk_read(h->h, ep, (char *)buffer,  8192, 12);
rc = usb_bulk_write(h->h, ep, (char *)buffer, 8192, 12);
end loop

So the device if it has no data to send, it goes in TEST_SE0_NAK mode and when it has data it clears this condition and reply to IN host requests.
Thus when the host requests IN data, if the device has not prepeared them (and is in TEST_SE0_NAK), waits until they became available, when they are available it gets them and then it writes to OUT. Then it starts immediately to wait for the next 8192bytes with the next IN request.

The USB external device has data ready every ~5msec so that is well within the timeout range. Also the device is quite capable and can write/read data at a measured rate of ~15MByte/sec so I guess that is not a device speed issue. After all it works ok on some (~50%) of the tested PC systems.

I have installed the 'driver' part of release 1.2.4 (not a snapshot).

It works ok in some of the systems that got tested but not in others and that is a mystery since the same install procedure and driver is followed in all of them. They have though different host USB controllers/chipsets.
The tested systems are XP/Vista/W7, there are fail and success in all of them so the OS seems its not a factor.

The real baffling, and that is somethign weird indeed, comes to those systems that errors are experienced.
There, while I experience errors on the bulk communication of the device, if I start the Thesycon DPC latency checker in order to have a simultaneous test (this is a third party external application and not used in any other way, I just launch it in parallel sometimes to my own application from the desktop to see the latency) , then the USB errors stop and the driver works ok!! and when I close it, the errors in the USB bulk comms through the libusb-win32 driver start again!!

Any idea how this mind-boggling situation can be solved?
(Please dont tell me to have the DPC checker launched every time I want to use the libusb-win32 :-)  )

[Xiaofan Chen]

On Sun, May 22, 2011 at 4:32 PM, Christos <[hidden email]> wrote:
> It is a proprietary DSP device, its not a test thing.
> Actually I'm utilizing the TEST_SE0_NAK from its firmware in order to signal
> that data are not yet available since the device cannot send NAK back in the
> usual way on a IN request.

Why? That is kind of the standard way. Or maybe you can try a ZLP.

> The device needs USB2 highspeed bandwidth (~3MBytes/sec read and
> ~3MBytes/sec write concurrently).
> It has only one configuration with two bulk endpoints,  one IN and one OUT,
> maxed to 512 bytes each.
> The application that utilizes this device is a DSP application and there is
> a simple method on communicating with the device. The host first reads a
> buffer of 8192bytes with usb_bulk_read and then it writes a buffer of 8192
> bytes with usb_bulk_write and these operations should happen within a
> predefined 12msec timeout.

I am thinking 12msec might be a bit aggressive for Windows which
is not a real time OS.

> &lt;Read/Write thread pseudocode&gt;
> repeat loop
> rc = usb_bulk_read(h->h, ep, (char *)buffer,  8192, 12);
> rc = usb_bulk_write(h->h, ep, (char *)buffer, 8192, 12);
> end loop

What if you increase 12 to higher value like 1000?

> So the device if it has no data to send, it goes in TEST_SE0_NAK mode and
> when it has data it clears this condition and reply to IN host requests.
> Thus when the host requests IN data, if the device has not prepared them
> (and is in TEST_SE0_NAK), waits until they became available, when they are
> available it gets them and then it writes to OUT. Then it starts immediately
> to wait for the next 8192bytes with the next IN request.
>
> The USB external device has data ready every ~5msec so that is well within
> the timeout range. Also the device is quite capable and can write/read data
> at a measured rate of ~15MByte/sec so I guess that is not a device speed
> issue. After all it works ok on some (~50%) of the tested PC systems.

It may be good to know the different chipset of these PC systems. Not
all host controllers perform the same. Intel chipsets seem to be better
in terms of compatibility based on my experiences. VIA/SIS/Nvidia
are not that good. Not sure about AMD.

> I have installed the 'driver' part of release 1.2.4 (not a snapshot).

When you say you get a lot of errors, what are the errors?
You may want to give the debug version a try and then use Microsoft
DebugView to get the debug log. That would help to understand
the issue.

> It works ok in some of the systems that got tested but not in others and
> that is a mystery since the same install procedure and driver is followed in
> all of them. They have though different host USB controllers/chipsets.
> The tested systems are XP/Vista/W7, there are fail and success in all of
> them so the OS seems its not a factor.

Does the system loading place a part? What if you put your application
to have higher thread priority?

> The real baffling, and that is somethign weird indeed, comes to those
> systems that errors are experienced.
> There, while I experience errors on the bulk communication of the device, if
> I start the Thesycon DPC latency checker in order to have a simultaneous
> test (this is a third party external application and not used in any other
> way, I just launch it in parallel sometimes to my own application from the
> desktop to see the latency) , then the USB errors stop and the driver works
> ok!! and when I close it, the errors in the USB bulk comms through the
> libusb-win32 driver start again!!

Kind of strange and that leads me to think it is related to system loading
or the other drivers in the system.

> Any idea how this mind-boggling situation can be solved?
> (Please dont tell me to have the DPC checker launched every time I want to
> use the libusb-win32 :-)  )
>

:-)



--
Xiaofan

------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Christos]

The TEST_SE0_NAK method has been used since there was no other way for the firmware so that is something that we need to live with.

The error, is timeout, code -116.
For 1000 msec, there should be transmitted about 200 data packages,
the packet errors are ~20 per sec and that represents a 10% loss.

Values larger than 20 msec have no meaning in timeouts in this occasion.
Either way, modern systems (quads/i3/i5/etc) that are tested are quite capable machines and having extremely low values in DPC in the range of 40 to 50 usec so I think that the real-time responsivenes is not a real issue here.

The chipsets verified working (and identified) are Intel ICH8, ICH10, NVIDIA nF4
Not working Intel ICH4, ICH6, Intel 5 Series/3400 (brand new!)

Also, there is a (well known) USB commercial driver given to me from the DSP manufacturer (TI) for testing and it does not show this kind of problem in the exact same situation of TEST_SE0_NAK, yet it costs quite a lot in licensing so I'm trying to find a solution with libusb-win32 that more-or-less does the work, and I would be realy happy indeed if it was not having this behaviour in some systems.


I do understand though that the loading could play a role here, so I tried to elevate the application priority with
"p.PriorityClass = ProcessPriorityClass.RealTime"
yet it did not do anything to rectify the situation.

How can I elevate the libusb-win32 driver priority, are there any hints for that?
Is there support for TEST_ES0_NAK in the libusb-win32 ?

[Travis]

On 5/22/2011 7:55 AM, Christos wrote:
> The TEST_SE0_NAK method has been used since there was no other way for the
> firmware so that is something that we need to live with.
>
> The error, is timeout, code -116.
> For 1000 msec, there should be transmitted about 200 data packages,
> the packet errors are ~20 per sec and that represents a 10% loss.
>

I think your whole problem is that of a coding one.

Your trying to stream data with the synchronous transfer functions.
To stream data, you must always keep extra transfers pending so when one
completes another is always under way.

Here is an example of how to do this:
http://article.gmane.org/gmane.comp.lib.libusb.devel.windows/198

Are you using it in libusb-win32 in filter mode?


> How can I elevate the libusb-win32 driver priority, are there any hints for
> that?

I don't think this is really possible.  Any driver can get exclusive
access ti the system.

> Is there support for TEST_ES0_NAK in the libusb-win32 ?
>

An SE0 NAK isn't something that is or isn't supported by a USB function
driver.

I've never heard of a TEST_SE0_NAK until now.
I know, HS devices can NAK if they are not ready to transfer but this is
low-level stuff and has nothing to to do the libusb-win32 driver.

Furthermore, driver DPCs and se0 naks are apples and oranges. ;)
libusb-win32 certainly doesn't manage any of it's own custom DPCs and
it's ISR/completion callback routines are very efficient.

I still somewhat confused as to what your are trying to accomplish and
what isn't working?
Can you post the actual libusb-win32 usb code that is failing?

Regards,
Travis


------------------------------------------------------------------------------
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its
next-generation tools to help Windows* and Linux* C/C++ and Fortran
developers boost performance applications - including clusters.
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Christos]

Hi,

I have my usb device firmware changed and finally managed to get rid of the TEST_SE0_NAK so now proper NAK is send.
So the firmware got changed and libusb-win32 is been used. Even in W7/64.

Yet still I have bad blocks.
To remind you, there was also a funny thing, when I started on the
desktop in parallel to my application the Thesycon DPC latency
checker, all bad blocks dissapear magically and when I exit the DPC
checker bad blocks reappeared again. That does happen in my XP/32 AND
in my W7/64 !! Same behaviour !!

I also tried to use the libusbK because I wanted to see if with libusbK
that situation gets rectified due to KMDF been used.
So the libusbK is installed, YET, the bad blocks are still here !! and the funny thing with DPC is
also still here !!

So, I have logged via the debugview the two situations, one with the
errors (DPC checker not started) and one with everything been ok (the
DPC checker is started).

What is shown is  this
00035351        33.79257584     (libusbK.sys)[Pipe_Abort] pipeID=81h
00035352        33.79277039     [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h

on regular intervals in the log with the errors.


Here is also the portion of the source code that reads and writes to the USB device,
no other usb activity exists, no control endpoint sommunication etc, only these two libusb function calls
the code fragment is condensed for clarity without unnecessary code lines.
It is quite simple.

==================================================
SetPriorityClass( GetCurrentProcess(), REALTIME_PRIORITY_CLASS );
SetThreadPriority( GetCurrentThread(), THREAD_PRIORITY_TIME_CRITICAL );

while ( io_keep_running != 0 ) {

        rc = usb_bulk_read(h->h, ep, (char *)buffer, 8192, USBTimeOut);

        < code for read buffer processing >
                ...

        < code to prepare write buffer >
                ...

        rc = usb_bulk_write(h->h, ep, (char *)buffer, 8192, USBTimeOut);

}
=================================================

So, no cancel is sent from within the application, no usbread or usbwrite call gets cancelled from here.
Thus the question is, why there are requests cancelled as mentioned in the log files?
(And most weird, why no cancels happen when the DPC check utility runs?)

Any help is deeply appreciated.

Christos

[Travis]

On 6/15/2011 8:32 AM, Christos wrote:

> while ( io_keep_running != 0 ) {
>
> rc = usb_bulk_read(h->h, ep, (char *)buffer, 8192, USBTimeOut);
>
> <  code for read buffer processing>
> ...
>
> <  code to prepare write buffer>
> ...
>
> rc = usb_bulk_write(h->h, ep, (char *)buffer, 8192, USBTimeOut);
>
> }
> =================================================
>
> So, no cancel is sent from within the application, no usbread or usbwrite
> call gets cancelled from here.
> Thus the question is, why there are requests cancelled as mentioned in the
> log files?
> (And most weird, why no cancels happen when the DPC check utility runs?)

..but you are mistaken.  If these functions timeout they will cancel the
operation.  This is how all sync transfers work in libusb-win32.
Set USBTimeOut to 5000.  Then what happens?

Regards,
Travis


------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Travis]

On 6/15/2011 8:56 AM, Travis wrote:

> On 6/15/2011 8:32 AM, Christos wrote:
>> while ( io_keep_running != 0 ) {
>>
>>     rc = usb_bulk_read(h->h, ep, (char *)buffer, 8192, USBTimeOut);
>>
>> <  code for read buffer processing>
>>         ...
>>
>> <  code to prepare write buffer>
>>         ...
>>
>>     rc = usb_bulk_write(h->h, ep, (char *)buffer, 8192, USBTimeOut);
>>
>> }
>> =================================================
>>
>> So, no cancel is sent from within the application, no usbread or
>> usbwrite
>> call gets cancelled from here.
>> Thus the question is, why there are requests cancelled as mentioned
>> in the
>> log files?
>> (And most weird, why no cancels happen when the DPC check utility runs?)
>
> ..but you are mistaken.  If these functions timeout they will cancel
> the operation.  This is how all sync transfers work in libusb-win32.
> Set USBTimeOut to 5000.  Then what happens?
>

You mentioned earlier that one of the reasons you want to use
libusb-win32 is its brutally high performance.
Is it possible, it is in-fact the shear speed of the libusb-win32 that
is causing your device to timeout?

I should also note that the "timeout" parameter is not very accurate.  
This is because the timeout is managed in the DLL itself.

If you require this specific timeout value (for a reason I don't
understand), then you *might* consider porting your test application to
libusbK.dll.

Using libusbK.dll/sys there is an easy way to get a very accurate
(driver-level) timeout using the timeout pipe policy.

FYI: I'm still having trouble finding a link that would indicate a
problem with our driver(s). ..looks like they are working perfectly from
the logs.

Regards,
Travis



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Christos]

Ok,

Here is the snapsot of the debugview log
=====================================================
00035311 33.75938416 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035312 33.75960541 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035313 33.76009369 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035314 33.76011276 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035315 33.76141357 (libusbK.sys)[Pipe_Abort] pipeID=81h
00035316 33.76164246 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h
00035317 33.76165390 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=0 (bytes)
00035318 33.76183701 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035319 33.76238251 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035320 33.76245499 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035321 33.76474762 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035322 33.76494980 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035323 33.76549911 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035324 33.76556015 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035325 33.77000809 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035326 33.77033234 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035327 33.77087402 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035328 33.77093887 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035329 33.77537155 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035330 33.77559280 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035331 33.77612305 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035332 33.77618027 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035333 33.77701950 (libusbK.sys)[Pipe_Abort] pipeID=81h
00035334 33.77726364 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h
00035335 33.77728653 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=0 (bytes)
00035336 33.77744293 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035337 33.77799988 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035338 33.77806854 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035339 33.78074646 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035340 33.78094482 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035341 33.78149796 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035342 33.78155899 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035343 33.78599548 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035344 33.78623199 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035345 33.78675079 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035346 33.78682327 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035347 33.79137039 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035348 33.79159164 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035349 33.79208755 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035350 33.79210663 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035351 33.79257584 (libusbK.sys)[Pipe_Abort] pipeID=81h
00035352 33.79277039 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h
00035353 33.79278183 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=0 (bytes)
00035354 33.79311371 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035355 33.79362488 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035356 33.79368591 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035357 33.79674911 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
00035358 33.79693985 (libusbK.sys)[Xfer] [Bulk-Write] pipeID=02h length=8192 stageLength=4194304
00035359 33.79749680 (libusbK.sys)[XferComplete] [Bulk-Write] TotalTransferred=8192 (bytes)
00035360 33.79755402 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035361 33.80199432 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)
=====================================================


where you can see that periodically we have this
00035315 33.76141357 (libusbK.sys)[Pipe_Abort] pipeID=81h
00035316 33.76164246 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h

These cancelltions appear in the log with whatever timeout and I use, even with the 5000msec that you prosed !!  :-)



Now, some more info on my configuration

At first, what I've installed is the libusbK driver for this vid/pid.
(there is also a libusb-win32 1.2.4 for the initial boot vid/pid of the device)

In my application, (it is 32bit, x86), I'm linking the libusb.lib from lib-win32(1.2.4)\lib\msvc folder
and using the usb.h header.

I was assuming that this was compatible with the libusbK.sys driver. And it actually works (with the exception of that 'cancelled transfer' glitch).


Now, from what I understand, I might need to scrap the libusb.lib and usb.h and see how to link with the libusbK.dll or some libusbK.lib library, right?

[Travis]

On 6/15/2011 9:50 AM, Christos wrote:

>
>
> where you can see that periodically we have this
> 00035315 33.76141357 (libusbK.sys)[Pipe_Abort] pipeID=81h
> 00035316 33.76164246 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h
> ioStatus=C0000120h requestStatus=C0000120h
>
> These cancelltions appear in the log with whatever timeout and I use, even
> with the 5000msec that you prosed !!  :-)
>

You are not going to want to hear it, but your device is completely to
blame here.
It is truly failing to respond to the read request.

I do not have an explanation as to why it works with windriver, but I
can say with near absolute certainty:
if you are experiencing the same problem with both libusbK.sys and
libusb0.sys that it is not a driver issue.  It is a hardware/user
software issue.

>
> Now, some more info on my configuration
>
> At first, what I've installed is the libusbK driver for this vid/pid.
> (there is also a libusb-win32 1.2.4 for the initial boot vid/pid of the
> device)
>
> In my application, (it is 32bit, x86), I'm linking the libusb.lib from
> lib-win32(1.2.4)\lib\msvc folder
> and using the usb.h header.
>
> I was assuming that this was compatible with the libusbK.sys driver. And it
> actually works (with the exception of that 'cancelled transfer' glitch).
>

That is correct, libusb0.dll is fully compatible with libusbK.sys.

> Now, from what I understand, I might need to scrap the libusb.lib and usb.h
> and see how to link with the libusbK.dll or some libusbK.lib library, right?
>

If using a larger timeout didn't work, doing the above will not solve
your problem either.

Please post the the source code which uses windriver and works.
Post the full source if possible, or provide a link.

Regards,
Travis



------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Christos]

Ok,

Here is the code with windriver, its main loop is the same with libusb version
except the changed specific commands of read and write.
The main difference is in the streams preparation that are used.
In the libusb version there is no streams method used.
==================================================
dwPipeNum = 0x81;
dwError = WDU_StreamOpen(hDevice, dwPipeNum,
                             PERF_STREAM_BUFFER_SIZE,
                                                         PERF_DEVICE_TRANSFER_SIZE, TRUE, 0,
                             TRANSFER_TIMEOUT, &readStream);
dwPipeNum = 0x02;
dwError = WDU_StreamOpen(hDevice, dwPipeNum,
                                                          PERF_STREAM_BUFFER_SIZE,
                                                          PERF_DEVICE_TRANSFER_SIZE, TRUE, 0,
                                                          TRANSFER_TIMEOUT, &writeStream);
dwError = WDU_StreamStart(readStream);    
dwError = WDU_StreamStart(writeStream);    

SetPriorityClass( GetCurrentProcess(), REALTIME_PRIORITY_CLASS );
SetThreadPriority( GetCurrentThread(), THREAD_PRIORITY_TIME_CRITICAL );

while ( io_keep_running != 0 ) {

        rc = WDU_StreamRead(readStream, buffer, buffersize, &dwBytesTransferredSingle);

        < code for read buffer processing >
                ...

        < code to prepare write buffer >
                ...

        rc = WDU_StreamWrite(writeStream, buffer, buffersize, &dwBytesTransferredSingle);

}
=================================================

I do not want to argue if you think it is the usb h/w device thats malfunctioning.
The only thing that I see is that the libusb solution itself Does Work under certain conditions and not under all. I do see the libusb performing perfect and I can repeat the external conditions when I want (DPC) and I always see perfect libusb performance. When I change the external conditions, libusb has the cancel request issue.
So based on what I see here, it is not a h/w problem. Nor a user application issue. Its a driver thats behaving strangely to external conditions (memory/cpu_level).
Yet, this is only my own view and I might be proven wrong.
If you want my guess (if its worth something) there has to be something in the driver memory handling or something at the levels of the workitems in the driver and do a cancel/abort at times.

In any case, I'll try once more, by changing the libusb.lib and usb.h in my application with what the kBench example is utilizing, a more direct approach to libusbK, if that can make any difference at all.

Thank you for all the good work and effort you put here.

Christos

[Travis]

On 6/15/2011 12:00 PM, Christos wrote:
> Ok,
>
> Here is the code with windriver, its main loop is the same with libusb
> version
> except the changed specific commands of read and write.
> The main difference is in the streams preparation that are used.

Yeah, but that's a pretty big difference.
Those stream transfers are working asynchronous and I believe they also
create and run in their own thread.
You must do the same thing with libusb-win32 only there isn't a nice
stream API to help you along.  (libusbK will have a stream API but it
has not yet been implemented)

kBench does exactly  what you need, but it is not the best example
because it has lots of other features.

> In the libusb version there is no streams method used.
> ==================================================
> dwPipeNum = 0x81;
> dwError = WDU_StreamOpen(hDevice, dwPipeNum,
>                               PERF_STREAM_BUFFER_SIZE,
> PERF_DEVICE_TRANSFER_SIZE, TRUE, 0,
>                               TRANSFER_TIMEOUT,&readStream);
> dwPipeNum = 0x02;
> dwError = WDU_StreamOpen(hDevice, dwPipeNum,
>  PERF_STREAM_BUFFER_SIZE,
>  PERF_DEVICE_TRANSFER_SIZE, TRUE, 0,
>  TRANSFER_TIMEOUT,&writeStream);
> dwError = WDU_StreamStart(readStream);
> dwError = WDU_StreamStart(writeStream);
>
> SetPriorityClass( GetCurrentProcess(), REALTIME_PRIORITY_CLASS );
> SetThreadPriority( GetCurrentThread(), THREAD_PRIORITY_TIME_CRITICAL );
>
> while ( io_keep_running != 0 ) {
>
> rc = WDU_StreamRead(readStream, buffer, buffersize,
> &dwBytesTransferredSingle);
>
>          <  code for read buffer processing>
>                  ...
>
>          <  code to prepare write buffer>
>                  ...
>
> rc = WDU_StreamWrite(writeStream, buffer, buffersize,
> &dwBytesTransferredSingle);
>
> }
> =================================================
>
> I do not want to argue if you think it is the usb h/w device thats
> malfunctioning.
> The only thing that I see is that the libusb solution itself Does Work under
> certain conditions and not under all. I do see the libusb performing perfect
> and I can repeat the external conditions when I want (DPC) and I always see
> perfect libusb performance. When I change the external conditions, libusb
> has the cancel request issue.
> So based on what I see here, it is not a h/w problem. Nor a user application
> issue. Its a driver thats behaving strangely to external conditions
> (memory/cpu_level).

Your are claiming a problem with some older WDM transfer functions that
have worked flawlesly for years.
You are also claiming the exact same problem with new libusbK KMDF based
transfer functions.
This in itself is a near complete impossibility.

Right now your application code is apples and oranges.  e.g. Your
windriver routines are completely different from your libusb-win32 routines.

> Yet, this is only my own view and I might be proven wrong.
> If you want my guess (if its worth something) there has to be something in
> the driver memory handling or something at the levels of the workitems in
> the driver and do a cancel/abort at times.
>

No, that is not happening.  Made obvious by the big delay you receive
when using a 5000 timeout.

> In any case, I'll try once more, by changing the libusb.lib and usb.h in my
> application with what the kBench example is utilizing, a more direct
> approach to libusbK, if that can make any difference at all.
>

I think you need to implement your own stream API using windows threads
and the libusb-win32 async tranfer functions.
kBench does this, but there is also a libusb-win32 "benchmark.c" for
libusb0.dll which is nearly identical to kBench.

Regards,
Travis

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Christos]

>> Yet, this is only my own view and I might be proven wrong.
>> If you want my guess (if its worth something) there has to be something in
>> the driver memory handling or something at the levels of the workitems in
>> the driver and do a cancel/abort at times.
>>
>
>No, that is not happening.  Made obvious by the big delay you receive
>when using a 5000 timeout.

That is exactly why I say this. There is NO Big Delay even when 5000 timeout is specified.
If you examine the logs, you see that the abort happens before the timeout, in even less time than 1ms.
Time between request send and pipe abort is less than one msec

00035350 33.79210663 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00035351 33.79257584 (libusbK.sys)[Pipe_Abort] pipeID=81h
00035352 33.79277039 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h
00035353 33.79278183 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=0 (bytes)

I also did send you a zip file with two extended logs in your email a couple of days ago. You can see there the exact timings.

As said, I might make a mistake in all these as I'm not so inclined.

Anyway, I'll try to have a stream-like implementation as in kBench since appart from this occasional 'cancel' glitch, the libusb does an excellent job for me and I do not want to give up easily without trying.

[Xiaofan Chen]

On Thu, Jun 16, 2011 at 3:12 PM, Christos <[hidden email]> wrote:

> That is exactly why I say this. There is NO Big Delay even when 5000 timeout
> is specified.
> If you examine the logs, you see that the abort happens before the timeout,
> in even less time than 1ms.
> Time between request send and pipe abort is less than one msec
>
> 00035350 33.79210663 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192
> stageLength=4194304
> 00035351 33.79257584 (libusbK.sys)[Pipe_Abort] pipeID=81h
> 00035352 33.79277039 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h
> ioStatus=C0000120h requestStatus=C0000120h
> 00035353 33.79278183 (libusbK.sys)[XferComplete] [Bulk-Read]
> TotalTransferred=0 (bytes)

Yes this is strange. And as you said, no cancels happen when the DPC check
utility runs, this is also strange.

Do you have similar log for libusb0.sys?

My suggestion is to run USBCV for your device and see if it can pass
the tests (USB compliant), at least Chapter 9 test. Since you have control
on the firmware, you might be able to fix the firmware in case some tests
fail.
http://www.usb.org/developers/tools/

You mentioned before that "the chipsets verified working (and identified)
are Intel ICH8, ICH10, NVIDIA nF4. Not working Intel ICH4, ICH6, Intel 5
Series/3400 (brand new!)"

Is this still the case now?

> Anyway, I'll try to have a stream-like implementation as in kBench since
> appart from this occasional 'cancel' glitch, the libusb does an excellent
> job for me and I do not want to give up easily without trying.
>

Since Windriver streaming API works for your device, we are quite
sure that using stream-like implementation as in kBench will also
work for you.

It is in our to-do list to implement streaming API for libusbk.
http://code.google.com/p/usb-travis/#To-do_list

The goal libusb-win32 project is to try our best to match or have
better features of other generic USB drivers under Windows.
The generic drivers I mean here include free ones
(like Microsoft WinUSB) or non-free ones (Jungo WinDriver USB,
Thesycon USBIO, etc )


--
Xiaofan

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Christos]

I'll have another try implementing the usb communication in my application in a more stream-like way.

Trying to read again the logs and understand the problem, I've noticed that the 'cancelled' requests are not always with 0 bytes read, there are sometimes with multiples of 512b (probably my endpoint payload block) but also sometimes with the full data requested and that is the most interesting, why although it read a complete buffer (8K) it did a cancel and returned the valid 8k buffer. ?!

I can understand the case where 0 or a partial number of the required length is read and cancelled, it could be some reason,
YET why does it gets cancelled even though it reads succesfully the requested length?

Complete buffer read, yet it cancelled
00003927 4.13420105 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00003928 4.13867617 (libusbK.sys)[Pipe_Abort] pipeID=81h
00003929 4.13884830 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h
00003930 4.13885355 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=8192 (bytes)

Partial, 1K buffer read and cancelled
00003939 4.15031624 (libusbK.sys)[Xfer] [Bulk-Read] pipeID=81h length=8192 stageLength=4194304
00003940 4.15429211 (libusbK.sys)[Pipe_Abort] pipeID=81h
00003941 4.15448380 [WRN](libusbK.sys)[XferComplete] [Cancelled] pipeID=81h ioStatus=C0000120h requestStatus=C0000120h
00003942 4.15449381 (libusbK.sys)[XferComplete] [Bulk-Read] TotalTransferred=1024 (bytes)






>You mentioned before that "the chipsets verified working (and identified)
>are Intel ICH8, ICH10, NVIDIA nF4. Not working Intel ICH4, ICH6, Intel 5
>Series/3400 (brand new!)"
>
>Is this still the case now?

I'm not constantly running tests on All of those systems, you understand that this is quite time consuming appart from the fact that these belong to various others which I cannot call anytime I want.
Either way though, since that time, I have changed the firmware and there is no more a TEST_SE0_NAK status send back, instead normal NAKs are sent when there are no data available. I suppose (and want to believe) that the very rare combination of those controllers with libusb and the specific test status was giving the errors at that time.


For Chapter 9, since major (but pricey) usb stacks do work ok, I have no reason to go into USBCV and spend time there.

[Xiaofan Chen]

On Thu, Jun 16, 2011 at 7:36 PM, Christos <[hidden email]> wrote:
> For Chapter 9, since major (but pricey) usb stacks do work ok, I have no
> reason to go into USBCV and spend time there.
>

I do not understand what you mean here. USBCV Chapter 9 test
is a basic test to pass for any USB product from any vendor. I know
I know that there are product which do not pass. But if you are
developing a USB firmware and do not want to run USBCV test,
then I have absolutely no confidence of your product.

And it does not take a pricey USB stack to pass USBCV. I have
tested USBCV for some 8-bit USB PICs last time and the free
USB Stack can make it pass.


--
Xiaofan

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Xiaofan Chen]

On Thu, Jun 16, 2011 at 7:36 PM, Christos <[hidden email]> wrote:
> I can understand the case where 0 or a partial number of the required length
> is read and cancelled, it could be some reason,
> YET why does it gets cancelled even though it reads successfully the
> requested length?
>

I think this is possible if the returned data is bad, eg, data toggle error,
bus error, etc.



--
Xiaofan

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Travis]

On 6/16/2011 12:12 AM, Christos wrote:
> That is exactly why I say this. There is NO Big Delay even when 5000 timeout
> is specified.
> If you examine the logs, you see that the abort happens before the timeout,
> in even less time than 1ms.
> Time between request send and pipe abort is less than one msec

This is not possible.  You have confused your logs I'm afraid.

Regards,
Travis

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel

[Travis]

On 6/16/2011 4:36 AM, Christos wrote:
> I'll have another try implementing the usb communication in my application in
> a more stream-like way.

I suggest you continue using win driver;  apparently your USB routines
are very specific to it.

Regards,
Travis

------------------------------------------------------------------------------
EditLive Enterprise is the world's most technically advanced content
authoring tool. Experience the power of Track Changes, Inline Image
Editing and ensure content is compliant with Accessibility Checking.
http://p.sf.net/sfu/ephox-dev2dev
_______________________________________________
Libusb-win32-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-win32-devel