Quantcast

Re: Polling file descriptors for synchronous transfers

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Polling file descriptors for synchronous transfers

Mysingen
Hello everyone

I'm sorry if this post becomes a bit messy. I wanted to subscribe to
the mailing list before carrying on, so I have simply copied the last
message (from Peter Stuge) verbatim from the Nabble page to provide
the context. Here goes...


> Hej Klas,
>
>> I am writing a single threaded application that has a poll/select loop.
>> I don't need asynchronous USB transfers at all, but I do need to poll
>> the file descriptors associated with the USB device.

> Why? In the code you showed your single thread seems to be waiting in
> poll() only for libusb file descriptors. I suggest simply using the
> sync API if you do not need to consider any other file descriptors.

The application uses processes rather than threads but is intended to
keep track of a large number of Android devices. The process-oriented
design lets me simplify a lot of things, but I still need pollable
descriptors to be able to let each process wait on a fair number of
event sources. The main loop of each process would become messy if I
have to introduce a different waiting mechanism for the raw USB level
handling. Other event sources include sockets opened against TCP ports
open on the devices, applications developed specifically to monitor
the "machine park" of devices, etc.

The sync API is rather more complicated to use so I'd rather not, if
possible. Anyway I don't see how that would make the problem go away,
but perhaps it does. Can anyone confirm that descriptors behave
differently when only using the sync API?

>> Some pseudo code to give you some idea what's going on before the
>> first poll:
>>
>> libusb_init(NULL)
>> libusb_device_handle * handle = discover_my_device() // discover,
>> open & claim the device
>> libusb_bulk_transfer(handle, endpoint OUT, message, ...) // the
>> device will write an answer back

> And where do you read that answer?

Reading is typically not a problem. It's just that I don't want to
call the functions to do so unless I'm sure there really is something
to read as I'd otherwise end up with a busy-wait style event loop. I'm
implementing the ADB protocol as a library that provides functions
like get_fd(), read(), write(), and so on. Most of it is quite
straight forward stuff.

> It's impossible to say without seeing your full program. It will
> further be very helpful if you also attach a debug log from running
> your program. (Build libusb-1.0.9 with --enable-debug as parameter to
> configure.)

I'll see if I can provide a minimal example that demonstrates the problem.

BR / Klas

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Polling file descriptors for synchronous transfers

Peter Stuge
Hi Klas!

Klas Lindberg wrote:

> >> I am writing a single threaded application that has a poll/select loop.
> >> I don't need asynchronous USB transfers at all, but I do need to poll
> >> the file descriptors associated with the USB device.
>
> > Why? In the code you showed your single thread seems to be waiting in
> > poll() only for libusb file descriptors. I suggest simply using the
> > sync API if you do not need to consider any other file descriptors.
>
> The application uses processes rather than threads but is intended to
> keep track of a large number of Android devices. The process-oriented
> design lets me simplify a lot of things, but I still need pollable
> descriptors to be able to let each process wait on a fair number of
> event sources. The main loop of each process would become messy if I
> have to introduce a different waiting mechanism for the raw USB level
> handling. Other event sources include sockets opened against TCP ports
> open on the devices, applications developed specifically to monitor
> the "machine park" of devices, etc.

Thanks for the explanation!


> The sync API is rather more complicated to use so I'd rather not, if
> possible. Anyway I don't see how that would make the problem go away,
> but perhaps it does. Can anyone confirm that descriptors behave
> differently when only using the sync API?

If you use the sync API you do not have to pay any attention to
USB-related file descriptors, libusb handles all events for you.

(Please note that "descriptor" is a significant term within USB with
a meaning distinct from file descriptors, so remember to be clear
about which ones you mean. :)


> Reading is typically not a problem. It's just that I don't want to
> call the functions to do so unless I'm sure there really is something
> to read as I'd otherwise end up with a busy-wait style event loop. I'm
> implementing the ADB protocol as a library that provides functions
> like get_fd(), read(), write(), and so on. Most of it is quite
> straight forward stuff.

I don't know the ADB details, but it looks like the sync API might
work fine. Is it important that your main event loop does other event
handling while the USB transfer is running? Or is it OK to block
until the USB transfer completes or times out?

If can block then use the sync API.

If can not block then create a thread which calls
libusb_handle_events_check() in a loop.


> > debug log from running your program. (Build libusb-1.0.9 with
> > --enable-debug as parameter to configure.)
>
> I'll see if I can provide a minimal example that demonstrates the problem.

Thanks!


//Peter

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Polling file descriptors for synchronous transfers

Mysingen
Hello Peter

Sorry for not replying sooner. I got quite ill this Monday (and am
still not recovered) so it just had to wait. Anyway, I hope you have
an Android phone handy. Set it to USB debugging mode, connect it and
use the attached archive like so:

tar -xzf libadb.tar.gz
cd libadb
make
./main

Note that I have *not* run this yet against a home compiled libusb
1.0.9. That's still on my todo list, but I thought I'd at least give
you something. The interesting parts are the calls to libadb_poll().
They all time out even though there is data to read. (What is the
correct USB terminology again? Frames?)

BR / Klas

On Mon, May 28, 2012 at 12:43 AM, Peter Stuge <[hidden email]> wrote:

> Hi Klas!
>
> Klas Lindberg wrote:
>> >> I am writing a single threaded application that has a poll/select loop.
>> >> I don't need asynchronous USB transfers at all, but I do need to poll
>> >> the file descriptors associated with the USB device.
>>
>> > Why? In the code you showed your single thread seems to be waiting in
>> > poll() only for libusb file descriptors. I suggest simply using the
>> > sync API if you do not need to consider any other file descriptors.
>>
>> The application uses processes rather than threads but is intended to
>> keep track of a large number of Android devices. The process-oriented
>> design lets me simplify a lot of things, but I still need pollable
>> descriptors to be able to let each process wait on a fair number of
>> event sources. The main loop of each process would become messy if I
>> have to introduce a different waiting mechanism for the raw USB level
>> handling. Other event sources include sockets opened against TCP ports
>> open on the devices, applications developed specifically to monitor
>> the "machine park" of devices, etc.
>
> Thanks for the explanation!
>
>
>> The sync API is rather more complicated to use so I'd rather not, if
>> possible. Anyway I don't see how that would make the problem go away,
>> but perhaps it does. Can anyone confirm that descriptors behave
>> differently when only using the sync API?
>
> If you use the sync API you do not have to pay any attention to
> USB-related file descriptors, libusb handles all events for you.
>
> (Please note that "descriptor" is a significant term within USB with
> a meaning distinct from file descriptors, so remember to be clear
> about which ones you mean. :)
>
>
>> Reading is typically not a problem. It's just that I don't want to
>> call the functions to do so unless I'm sure there really is something
>> to read as I'd otherwise end up with a busy-wait style event loop. I'm
>> implementing the ADB protocol as a library that provides functions
>> like get_fd(), read(), write(), and so on. Most of it is quite
>> straight forward stuff.
>
> I don't know the ADB details, but it looks like the sync API might
> work fine. Is it important that your main event loop does other event
> handling while the USB transfer is running? Or is it OK to block
> until the USB transfer completes or times out?
>
> If can block then use the sync API.
>
> If can not block then create a thread which calls
> libusb_handle_events_check() in a loop.
>
>
>> > debug log from running your program. (Build libusb-1.0.9 with
>> > --enable-debug as parameter to configure.)
>>
>> I'll see if I can provide a minimal example that demonstrates the problem.
>
> Thanks!
>
>
> //Peter
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> libusb-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/libusb-devel

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

libadb.tar.gz (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Polling file descriptors for synchronous transfers

Peter Stuge
Klas Lindberg wrote:
> ./main
>
> Note that I have *not* run this yet against a home compiled libusb
> 1.0.9. That's still on my todo list, but I thought I'd at least give
> you something.

Thanks. I built current libusb.git master, and..


> The interesting parts are the calls to libadb_poll().
> They all time out even though there is data to read.

..it works for me.

/tmp/libadb $ LD_LIBRARY_PATH=/tmp/lu/lib ./main
probing device 1d6b:0002
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 0483:2016
interface with wrong number of endpoints
probing device 0bb4:0cb2
endpoints: 2
7 5 0x82 2 0x200 0
not an adb protocol interface
endpoints: 2
7 5 0x83 2 0x200 0
7 5 0x4 2 0x200 0
adb interface found
0x83 0x4
serial number length: 12
serial number: HT165V403130
found device on address 5
interface: 1
in endpoint: 131
out endpoint: 4
probing device 1d6b:0002
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 1d6b:0001
interface with wrong number of endpoints
probing device 0483:2016
interface with wrong number of endpoints
probing device 0bb4:0cb2
endpoints: 2
7 5 0x82 2 0x200 0
not an adb protocol interface
endpoints: 2
7 5 0x83 2 0x200 0
7 5 0x4 2 0x200 0
adb interface found
0x83 0x4
serial number length: 12
serial number: HT165V403130
found device on address 5
interface: 1
in endpoint: 131
out endpoint: 4
sent 24 / 24
sent 13 / 13
recd 24 / 24
recd 9 / 9
recd: CNXN 0x1000000 0x1000 9 740 0xb1a7b1bc device::
may poll fd=3 events=0x1
may poll fd=6 events=0x1
may poll fd=7 events=0x4
poll(3) = 0
fd=3, revents=0xffffffff
fd=6, revents=0xffffffff
fd=7, revents=0xffffffff
sent 24 / 24
sent 12 / 12
may poll fd=3 events=0x1
may poll fd=6 events=0x1
may poll fd=7 events=0x4
poll(3) = 0
fd=3, revents=0xffffffff
fd=6, revents=0xffffffff
fd=7, revents=0xffffffff
recd 24 / 24
recd: OKAY 0x2 0x1 0 0 0xa6beb4b0
recd 24 / 24
recd 512 / 825
recd 825 / 825
recd: WRTE 0x2 0x1 825 53868 0xbaabada8 drwxr-xr-x    3 root     root             0 Jun  1 03:30 acct
drwxrwxrwt    2 root     root            40 Jun  1 03:30 app-cache
drwxrwx--x    1 system   cache         4096 Apr 27 18:58 cache
dr-x------    2 root     root             0 Jun  1 03:30 config
lrwxrwxrwx    1 root     root            17 Jun  1 03:30 d -> /sys/kernel/debug
drwxrwx--x    1 system   system        4096 May 31 23:57 data
-rw-r--r--    1 root     root           118 Jan  1  1970 default.prop
drwxr-xr-x   13 root     root          1500 Jun  1 03:30 dev
lrwxrwxrwx    1 root     root            11 Jun  1 03:30 etc -> /system/etc
-rwxr-x---    1 root     root         98544 Jan  1  1970 init
-rwxr-x---    1 root     root          4487 Jan  1  1970 init.chacha.rc
-rwxr-x---    1 root     root          1677 Jan  1  1970 init.goldfish.rc

may poll fd=3 events=0x1
may poll fd=6 events=0x1
may poll fd=7 events=0x4
poll(3) = 0
fd=3, revents=0x1
fd=6, revents=0x1
fd=7, revents=0x5
recd 0 / 24
nothting transfered
failed to read CLSE
recd 0 / 24
nothting transfered
failed to read message
/tmp/libadb $


//Peter

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Polling file descriptors for synchronous transfers

Mysingen
> ..it works for me.

Do you mean that you don't experience a 5 second "freeze" just before
the file listing? The output looks like it should, with or without the
timeout.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate
star

Re: Polling file descriptors for synchronous transfers

Peter Stuge
Klas Lindberg wrote:
> > ..it works for me.
>
> Do you mean that you don't experience a 5 second "freeze" just before
> the file listing? The output looks like it should, with or without the
> timeout.

Ah - you're right that there was a bit of a delay. I expected an
actual error. I don't have access to the phone anymore, but will
try to test further later today.


//Peter

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel
Loading...