|
On Tue, Jan 26, 2010 at 9:18 PM, Michael Plante <[hidden email]> wrote:
I'd be in big trouble at work if you can't!
>>
>> If you never free the global mutex, I'd be tempted to bypass the compare/exchange if it isn't null: >> >> if ( global_mutex ) { >> WaitForSingleObject(global_mutex); >> } else { >> // Create mutex, do the exchange etc. >> } I think that's just an optimization. You still have a race where the mutex Right, it's an optimization.
My vote would be for private data first. If not, the named object makes things a little simpler at init time, a little less at exit. Both ways work and I don't have a real preference.
Orin.
------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
Orin Eman wrote:
>> Michael Plante wrote: >> >> Orin Eman wrote: >> >> >> I wouldn't worry about that... denial of service would be so easy by just >> >> >> talking directly to WinUSB or whatever other driver we were using that tinkering >> >> >> with our semaphore wouldn't be worthwhile! >> >> >> >> Are you allowed to talk to WinUSB if you're not an administrator? (I don't >> >> know; I'm asking.) >> >> I'd be in big trouble at work if you can't! Fair enough. But, to scale back my argument a bit, there may be different policies for terminal services (== remote desktop?) users than local users. Physical access is something to consider: just unplug the device! Regardless: 1) I still am not comfortable with introducing new security holes, even if there are more significant ones that Microsoft caused. Who knows? Maybe they'll clean this one up? Maybe they can't? Maybe, in the future, security descriptors will be attached to devices via udev-like rules (buried in the bowels of the registry, of course)? 2) During the time your interface is claimed (or whatever the action is that gives exclusive access), how much is the risk mitigated? I'd suggest that the semaphore is vulnerable during, before, and after libusb is used, so that's a broader window for attacks. 3) Someone can modify libusb (it's LGPL, after all), forget to change the semaphore name, and create a poorly-behaved program that interferes with a well-behaved program, creating seemingly-legit support requests for both. Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
On Tue, Jan 26, 2010 at 9:56 PM, Michael Plante <[hidden email]> wrote:
No difference for terminal services users with default WinUSB settings that I know of. The only problem I've had is having the device sulk and needing to physically unplug it...
See http://msdn.microsoft.com/en-us/library/ms794732.aspx for device security descriptors. You can even specify a security descriptor in the inf file. Have fun understanding SDDL!
Orin.
------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
Orin Eman wrote:
>> See http://msdn.microsoft.com/en-us/library/ms794732.aspx for device security >> descriptors. You can even specify a security descriptor in the inf file. Have >> fun understanding SDDL! Indeed. A more complicated security mechanism requires more complicated configuration. But big companies will have the resources to either decipher those in-house or pay for a support contract, and there are probably enough examples online. This kinda makes my point, though I was never really concerned about 1&2 as much as 3, as I run as administrator anyway, and there's no remote access to my Windows box, other than a few trojans I installed to keep my life interesting. If we do use this solution, let's at least append the process ID and make sure the semaphore or mutex gets destroyed in windows_exit(); that solves #3, I think. Ultimately, the first 2 are just concerns other people might have. I debug/develop on Windows, but the production target is an embedded Linux board w/o Internet access and usually surrounded by water on all sides, so I'm not as concerned about security as this email chain might imply. Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Michael Plante
Michael Plante wrote:
>> Yes, it presently uses a mutex. Like I said, that is not a difficult problem. The only >> issue with the default context is making sure that mutex only gets overwritten once, and >> this can be done with an InterlockedCompareExchange() (are HANDLE's 32 or 64-bits in >> 64-bit Windows? If 64, it might need to be InterlockedCompareExchangePointer().). Or, >> if it's done via static initialization, that it get torn down, and that we check for >> errors during init. >> >> The condition variable is difficult. I was merely trying to be complete by listing the others. Since none of the synchronization objects are exposed in libusb.h, would it make sense to define some inline functions in libusbi.h that call out to either Win32 or pthreads, and then create two typedefs: 1 for standalone mutexes, and 1 for a struct containing a mutex and a condition variable? (The reason for the struct would be because it may be something completely different inside the struct in Win32.) For the mutexes (even the static one), the preprocessor output should be identical to the old code, and for the condition variable, it'll only be slightly different. The primitives needing to be replaced are (assuming we ignore dpfp_threaded.c) : pthread_mutex_init/pthread_mutex_destroy pthread_mutex_lock/pthread_mutex_unlock pthread_mutex_trylock pthread_cond_init /pthread_cond_destroy pthread_cond_broadcast pthread_cond_wait pthread_cond_timedwait PTHREAD_MUTEX_INITIALIZER (used only for the default context lock) pthread_mutex_t pthread_cond_t What I would suggest for standalone mutexes is along the lines of: #ifdef OS_WINDOWS #define libusb_mutex HANDLE #else #define libusb_mutex pthread_mutex_t #endif #ifdef OS_WINDOWS static int _inline libusb_mutex_init(libusb_mutex *mutex, void const *attr) { if(! mutex) return ((errno=EINVAL)); *mutex = CreateMutex(NULL, FALSE, NULL); if(!*mutex) return ((errno=ENOMEM)); return 0; } #else static int inline libusb_mutex_init(libusb_mutex_t *restrict mutex, const pthread_mutexattr_t *restrict attr) { return pthread_mutex_init(mutex, attr); } #endif #ifdef OS_WINDOWS #define PTHREAD_MUTEX_INITIALIZER NULL static int _inline libusb_mutex_static(libusb_mutex *mutex) { libusb_mutex temp_mutex; // N.B.: (opaque) pointer; stack is ok if (libusb_mutex_init(&temp_mutex, NULL)) return 0; // fail if (InterlockedCompareExchangePointer( mutex, temp_mutex, NULL)) { libusb_mutex_destroy(&temp_mutex); } return 1; // succeed } #else static int inline libusb_mutex_static(libusb_mutex *mutex) { return 1; // don't have to fake it. } #endif #ifdef OS_WINDOWS static int _inline libusb_mutex_lock(libusb_mutex_t *mutex) { DWORD result; if(!mutex) return ((errno=EINVAL)); result = WaitForSingleObject(*mutex, INFINITE); if(result == WAIT_OBJECT_0 || result == WAIT_ABANDONED) return 0; // acquired (abandoned is bad, but I don't know mapping) return ((errno=EINVAL)); // don't know how this would happen, so don't know proper errno } #else static int inline libusb_mutex_lock(libusb_mutex_t *mutex) { return pthread_mutex_lock(mutex); } #endif In libusb_init(): + if(!libusb_mutex_static(&default_context_lock)) + goto err; pthread_mutex_lock(&default_context_lock); ...and so on. No change is needed to the static definition. Again, this leaks one mutex per process execution, but for many programs, that's a difference of milliseconds between when it's freed by the OS and when it could be freed by libusb_exit(), and a small known-constant # is acceptable, IMHO. I didn't try compiling these, so there may be problems, but I think it gets the idea across. I want to be careful about pthread_* name collisions, since when I use libusb on Win32, I already have a pthread dependency for unrelated reasons. Maybe it's not an issue... Thoughts? Thanks, Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Pete Batard
Hi,
2010/1/27 Pete Batard <[hidden email]>: > Hi Francesco, > >> could anyone who has the 64bit build of pthread-win32 upload it >> somewhere or send it to me :) ? Thanks in advance!! > > http://libusb-winusb-wip.googlecode.com/files/pthread-win32_x64.zip > Sorry it took so long to upload it. Thank you very much! In the past days I've successfully created the WinUSB-based driver for a project I've ported to libusb-1.0. Now I managed to find a MSVC++ 2008 Professional edition and used it to compile in 64bit-mode all dependencies of that project (namely wxWidgets, libusb-1.0). Unfortunately I'm having an hard time trying to debug a heap corruption assert which is popped-up semi-randomly by the final application. I think one possible cause of the problem could be that I'm linking in the final EXE libraries compiled with different /MDd or /MTd settings... so here's my new question: which setting (/MDd or /MTd or something else?) was used to compile the pthreadVC2_x64.dll ? ;) I have noticed that the msvc2008 project of libusb-1.0 uses /MTd and I've changed it to /MDd (which btw is the default used by wxWidgets) since I think it's better to use the DLL version of the CRT (see http://msdn.microsoft.com/en-us/library/abx4dbyh%28VS.71%29.aspx)... was /MTd chosen for some particular reason? Thanks!! Francesco ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
On 2010.01.30 12:23, Francesco wrote:
> In the past days I've successfully created the WinUSB-based driver for > a project I've ported to libusb-1.0. Great. > I think one possible cause of the problem could be that I'm linking in > the final EXE libraries compiled with different /MDd or /MTd > settings... so here's my new question: which setting (/MDd or /MTd or > something else?) was used to compile the pthreadVC2_x64.dll ? ;) The pthread-win32 I compiled are using /MD. I can produce /MT ones if required. > I have noticed that the msvc2008 project of libusb-1.0 uses /MTd and > I've changed it to /MDd (which btw is the default used by wxWidgets) > since I think it's better to use the DLL version of the CRT (see > http://msdn.microsoft.com/en-us/library/abx4dbyh%28VS.71%29.aspx)... > was /MTd chosen for some particular reason? On 2010.01.20 19:24, Orin Eman wrote: > Last I saw, the DLL was using the DLL C runtime. I'd strongly recommend > using the (multi-threaded) static C runtime, otherwise, binary > distributions would require the VC redistributables (and we start > worrying about side by side configurations and so on). I don't really mind either way, but if this is going to be an issue with pthread-win32, I'd rather pick the one that produces the less headache. Of course, we will remove pthread-win32 use after the first version is out, so that problem should soon be moot. Oh, and if you're having issues with threading, you might want to test with the "concurency" branch instead, which is available at: http://git.libusb.org/?p=libusb-pbatard.git;a=shortlog;h=refs/heads/concurrency This branch removes the need for pthread in the Windows specific source (but not on core), so it might help. Regards, /Pete ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
Pete Batard wrote:
>> The pthread-win32 I compiled are using /MD. I can produce /MT ones if >> required. For prebuilt binaries, that's probably a good idea. We are providing a "sources" file for libusb, which allows Francesco to change to /MD* there, but not in pthread. >> Francesco wrote: >> > I have noticed that the msvc2008 project of libusb-1.0 uses /MTd and >> > I've changed it to /MDd (which btw is the default used by wxWidgets) >> > since I think it's better to use the DLL version of the CRT (see >> > http://msdn.microsoft.com/en-us/library/abx4dbyh%28VS.71%29.aspx)... >> > was /MTd chosen for some particular reason? Did doing that to pthreads stop the corruption errors, since you now have access to a system that should build it? >> On 2010.01.20 19:24, Orin Eman wrote: >> > Last I saw, the DLL was using the DLL C runtime. I'd strongly recommend >> > using the (multi-threaded) static C runtime, otherwise, binary >> > distributions would require the VC redistributables (and we start >> > worrying about side by side configurations and so on). >> >> I don't really mind either way, but if this is going to be an issue with >> pthread-win32, I'd rather pick the one that produces the less headache. >> >> Of course, we will remove pthread-win32 use after the first version is >> out, so that problem should soon be moot. So I take it the code I posted to the list for this is not a direction we want to follow anytime soon? >> Oh, and if you're having issues with threading, you might want to test >> with the "concurency" branch instead, which is available at: It might be threading, but Francesco did not mention threads. It might also just be a matter of, as we discussed, having multiple copies of the CRT floating around. >> http://git.libusb.org/?p=libusb-pbatard.git;a=shortlog;h=refs/heads/concurre ncy >> This branch removes the need for pthread in the Windows specific source >> (but not on core), so it might help. Were you considering merging this into master? Thanks, Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
On 2010.01.30 17:58, Michael Plante wrote:
>>> The pthread-win32 I compiled are using /MD. I can produce /MT ones if >>> required. > > For prebuilt binaries, that's probably a good idea. We are providing a > "sources" file for libusb, which allows Francesco to change to /MD* there, > but not in pthread. Yeah. I'll produce /MT ones and upload them when I get a chance. >>> Of course, we will remove pthread-win32 use after the first version is >>> out, so that problem should soon be moot. > > So I take it the code I posted to the list for this is not a direction we > want to follow anytime soon? I'd say "anytime soon" depends on when we have a first official release. As I mentioned privately, I'm reluctant to add threading changes to core with the first release, even if they are trivial, because we are introducing enough modifications as it is (I'll be sending the core patches soon). > Were you considering merging (the concurrency branch) into master? It *will* be merged into master before the first release. In fact, I've been integrating changes from master into concurrency so that it can be merged back with master. I was just waiting to see if there was anything more that needed to be done in concurrency, but I'd say it's probably stable enough now. Regards, /Pete ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Michael Plante
Hi all,
thanks for your info. I've debugged the problem and actually there were 2 things to consider: 1) until I didn't switch from /MTd to /MDd in the MSVC project that I'm now using to build libusb-1.0, the heap problems were very probably caused by the conflicting CRT settings (as I said wxWidgets uses /MDd). Before realizing this, to help me in the process of debugging the heap corruption I started using the "AppVerifier" tool from MS (see http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=c4a25ab9-649d-4a1b-b4a7-c9d8b095df18). AppVerifier helps debugging heap problems (and many other types) by injecting some code in the application which e.g. checks the heap/stack after each malloc/free call. However it turned out to be misleading: 2) after fixing the /MTd => /MDd setting, I found that the application was stopping in the CancelIo calls done by _free_index() (which was triggered by handle_timeout() calls) with a message like: First-chance exception at 0x7794fec7 in usbpicprog.exe: 0xC0000008: An invalid handle was specified. It took me a bit of time to find out that CancelIo() is called on possibly-invalid handles by purpose and that the exception above was generated by AppVerifier. Now, after disabling AppVerifier and with all libs using /MDd (except for pthread-win32 with /MD) I don't have any heap-corruption problem. The application works but unfortunately I get rather often "time out" errors when calling libusb_bulk_transfer() for read or write. I'm now investigating these errors (they didn't show up with libusb-0.1)... 2010/1/30 Michael Plante <[hidden email]>: > Pete Batard wrote: >>> The pthread-win32 I compiled are using /MD. I can produce /MT ones if >>> required. ok, thanks. >>> Francesco wrote: >>> > I have noticed that the msvc2008 project of libusb-1.0 uses /MTd and >>> > I've changed it to /MDd (which btw is the default used by wxWidgets) >>> > since I think it's better to use the DLL version of the CRT (see >>> > http://msdn.microsoft.com/en-us/library/abx4dbyh%28VS.71%29.aspx)... >>> > was /MTd chosen for some particular reason? > > Did doing that to pthreads stop the corruption errors, since you now have > access to a system that should build it? I didn't try to recompile pthreads myself: I think mixing /MD and /MDd is fine. The problems probably arise when mixing /MT and /MD switches... >>> On 2010.01.20 19:24, Orin Eman wrote: >>> > Last I saw, the DLL was using the DLL C runtime. I'd strongly > recommend >>> > using the (multi-threaded) static C runtime, otherwise, binary >>> > distributions would require the VC redistributables (and we start >>> > worrying about side by side configurations and so on). >>> >>> I don't really mind either way, but if this is going to be an issue with >>> pthread-win32, I'd rather pick the one that produces the less headache. >>> >>> Of course, we will remove pthread-win32 use after the first version is >>> out, so that problem should soon be moot. >>> Oh, and if you're having issues with threading, you might want to test >>> with the "concurency" branch instead, which is available at: > > It might be threading, but Francesco did not mention threads. actually the program I'm developing initializes the USB device from its primary thread but then makes long read/write operations from a secondary thread, so indeeed multithreading could be a source of error. I'll eventually test the concurrency branch... Thanks, Francesco ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
Francesco wrote:
>> Before realizing this, to help me in the process of debugging the heap >> corruption I started using the "AppVerifier" tool from MS (see >> http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=c4a2 5ab9-649d-4a1b-b4a7-c9d8b095df18). >> AppVerifier helps debugging heap problems (and many other types) by >> injecting some code in the application which e.g. checks the >> heap/stack after each malloc/free call. >> However it turned out to be misleading: One other option you can try, if you're interested in checking the heap, is sticking this at the start of main() (or before you do any allocations): _CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF|/*_CRTDBG_CHECK_ALWAYS_DF|_CRTDBG_DELAY_ FREE_MEM_DF|*/_CRTDBG_LEAK_CHECK_DF); and remember to include <crtdbg.h> . You can read the docs on what the commented flags do, but the ones I left in don't seem to have much performance impact. Then, when you run in the debugger, it'll dump leaks when the program exits. Some of those flags also check before/after malloc/free, but I don't remember which ones. Some of the builtin CRT code exempts its own allocations from this checking, which is why you can get away with putting it in main(), rather than digging into the CRT code that calls main(). HTH, Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Pete Batard
On 2010.01.30 20:47, Pete Batard wrote:
>> Were you considering merging (the concurrency branch) into master? > > It *will* be merged into master before the first release. Concurrency branch now fully merged into master (r128). Note that with r127, the timer functions are now handled by a separate thread (with its affinity set to the first core), so libusb on Windows is now running at least 2 separate threads. Regards, /Pete ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Pete Batard
Pete Batard wrote:
>> I'd say "anytime soon" depends on when we have a first official release. >> As I mentioned privately, I'm reluctant to add threading changes to core >> with the first release, even if they are trivial, because we are >> introducing enough modifications as it is (I'll be sending the core >> patches soon). If you want, I can put up a new branch and start making those changes there. I was hoping to hear from Peter or Daniel whether they're clean enough. In any case, I can't do it until Monday night (UTC-6 here). Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
On 2010.01.31 00:19, Michael Plante wrote:
> If you want, I can put up a new branch and start making those changes there. I'm fine with that. In fact, I'll probably mirror your branch as soon as you create it, as these changes are definitely going to be part of the 2nd official release. Regards, /Pete ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Michael Plante
Hi Michael!
2010/1/30 Michael Plante <[hidden email]>: > One other option you can try, if you're interested in checking the heap, is > sticking this at the start of main() (or before you do any allocations): > > _CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF|/*_CRTDBG_CHECK_ALWAYS_DF|_CRTDBG_DELAY_ > FREE_MEM_DF|*/_CRTDBG_LEAK_CHECK_DF); > > and remember to include <crtdbg.h> . thanks for the tip! I'm going to add this call to my main() asap! Thanks, Francesco ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Pete Batard
Pete Batard wrote:
> Note that with r127, the timer functions are now handled by a separate > thread (with its affinity set to the first core), so libusb on Windows > is now running at least 2 separate threads. Sorry if this discussion has been had before, but can I ask a dumb question: Why is a timer needed at all, and why does it need to be high precision ? libusb 0.1 doesn't have OS specific timing structures like timespec visible in the API and doesn't deal with absolute time anyway, and the OS backends deal with the system specific issues. Why isn't libusb 1.0 taking the same approach ? Do you really need better than 1msec precision on this anyway ? (ie. why aren't the win32 primitives that take timeouts in msec + timeGetTime() sufficient for MSWindows back end ?) Graeme Gill. ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
Graeme Gill wrote:
>> Pete Batard wrote: >> > Note that with r127, the timer functions are now handled by a separate >> > thread (with its affinity set to the first core), so libusb on Windows >> > is now running at least 2 separate threads. >> >> Sorry if this discussion has been had before, but can I >> ask a dumb question: Why is a timer needed at all, and >> why does it need to be high precision ? >> >> libusb 0.1 doesn't have OS specific timing structures >> like timespec visible in the API and doesn't deal with >> absolute time anyway, and the OS backends deal with the >> system specific issues. Why isn't libusb 1.0 taking the same approach ? Well, looking at libusb-win32 0.1, anyway, the timeouts are relative, but they're relative to when you try to reap in the async api (usb_reap_async()). In the sync api, you don't have more than one going on from a given thread, so relative is fine. So I believe that the reason for absolute times is to have multiple operations going. As to *accuracy* (forget precision for a moment), yes, we did have this discussion, and Tim has seen up to 0.5 seconds of skew between processors on the HF performance counter: http://old.nabble.com/Re%3A--PATCH%3A-winusb--hires-timer-fix-p26749357.html The reason for the thread is so we can lock it to one processor without screwing up all the threads in an app. I expect some delay to be introduced in the context switches to get the time, but I think it's better than what we had. AFAIK, we really only need the thread for the monotonic clock, not the realtime clock, right? Maybe we can simplify/optimize things there. Is there any skew in GetSystemTimeAsFileTime? However, most of the time the monotonic clock IS what's used, except libusb_wait_for_event(), which is for pthread_cond_timedwait (of course, depending on how CV's are emulated, we may get away with a relative timeout there, but that obviously doesn't solve the timer thread problem). >> Do you really need better than 1msec precision on this anyway ? >> (ie. why aren't the win32 primitives that take timeouts in msec >> + timeGetTime() sufficient for MSWindows back end ?) Hrm. I don't recall if we've discussed that option. timeGetTime() sounds monotonic if your system can't stay up 50 days, which is a good bet on Windows... :) And we could mess around with it so the only requirement is that you have to call into libusb at least every 50 days (update a high-order DWORD counter). The docs don't talk about XP/later, but they say on NT/2k, the accuracy can be worse than 5ms in some cases, and that there are functions that *might* work which will lower that granularity. I don't know if any of this is a good idea. Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
Michael Plante wrote:
> Well, looking at libusb-win32 0.1, anyway, the timeouts are relative, but > they're relative to when you try to reap in the async api > (usb_reap_async()). In the sync api, you don't have more than one going on > from a given thread, so relative is fine. So I believe that the reason for > absolute times is to have multiple operations going. I've had a look at some of the doco but I can't say I'm understanding it. There is the statement: "* - libusb also needs to be called into at certain fixed points in time in * order to accurately handle transfer timeouts." but this seems to me to be complicating the API unnecessarily to cope with certain platforms. libusb 0.1 didn't have these complications, hiding platform dependencies in the implementation. The libusb 1.0 async scheme seems similar to MSWindows OVERLAPPED type approach, but it doesn't need absolute time, so I'm struggling to understand why libusb 1.0 does. Don't you kick off an operation with a given timeout, and then get called back when it completes or times out ? Why do you have to poll it ? Why should different operations in the same thread interfere with each other ? > As to *accuracy* (forget precision for a moment), yes, we did have this > discussion, and Tim has seen up to 0.5 seconds of skew between processors on > the HF performance counter: Right, but why use them if you don't need too given the complication of needing a thread with affinity ? > Hrm. I don't recall if we've discussed that option. timeGetTime() sounds > monotonic if your system can't stay up 50 days, which is a good bet on > Windows... :) It's supposed to be monotonic if you use the recommended 32 bit modulo arithmetic, even over and after 50 days. Since MSWindows timeout API's all take values in msec, and given the multi-core and speed throttling issues with the HP timers, I would guess this is what they are using internally Graeme Gill. ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
Graeme Gill wrote:
>> Michael Plante wrote: >> > Well, looking at libusb-win32 0.1, anyway, the timeouts are relative, but >> > they're relative to when you try to reap in the async api >> > (usb_reap_async()). In the sync api, you don't have more than one going on >> > from a given thread, so relative is fine. So I believe that the reason for >> > absolute times is to have multiple operations going. >> >> I've had a look at some of the doco but I can't say I'm understanding >> it. There is the statement: >> >> "* - libusb also needs to be called into at certain fixed points in time in >> * order to accurately handle transfer timeouts." >> >> but this seems to me to be complicating the API unnecessarily to cope >> with certain platforms. libusb 0.1 didn't have these complications, Because the timeouts were referenced to when the reap was attempted, not when you submitted the transfer, which can be at very different times (the former later). The latter is the logical place to reference the timeout to, but libusb only did this with the sync interface. I don't see much platform-dependent about it. Would you do it differently? Certainly it could be simpler if we only supported sync. >> hiding platform dependencies in the implementation. The libusb 1.0 >> async scheme seems similar to MSWindows OVERLAPPED type approach, I believe overlapped relies on something in the kernel to set an event at the appropriate time. Ultimately, poll does use a relative time, but it has the old relative time, minus whatever time has elapsed since you first set the timeout. This last part is, I believe, key. >> but it doesn't need absolute time, so I'm struggling to understand >> why libusb 1.0 does. Don't you kick off an operation with a given >> timeout, and then get called back when it completes or times out ? Not sure. I would assume that, in order to get called back, something in libusb has to be running first, right? >> Why do you have to poll it ? Well, it's not busy-waiting; your thread is suspended. Assuming you knew that, see the previous paragraph. >> Why should different operations in the >> same thread interfere with each other ? I don't follow. Could you clarify? >> Right, but why use them if you don't need too given the complication >> of needing a thread with affinity ? Agreed, *if* you don't need to. I'm not sure. >> It's supposed to be monotonic if you use the recommended 32 bit >> modulo arithmetic, even over and after 50 days. I don't know that the libusb core does use modulo arithmetic, though it probably could. That was what I meant when I said "mess with". That still requires you call into libusb often enough to increment an overflow counter, or call in often enough that it doesn't matter. Point being, if the difference exceeds 50 days, you have a problem. >> Since MSWindows >> timeout API's all take values in msec, and given the multi-core >> and speed throttling issues with the HP timers, I would guess this >> is what they are using internally They may boil down to the same system call (don't know), but the scheduler probably doesn't use mm* functions. Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
|
In reply to this post by Pete Batard
Pete Batard wrote:
>> On 2010.01.31 00:19, Michael Plante wrote: >> > If you want, I can put up a new branch and start making those changes there. >> >> I'm fine with that. In fact, I'll probably mirror your branch as soon as >> you create it, as these changes are definitely going to be part of the >> 2nd official release. Ok, the branch is up. Please help me test it. I have not yet made CV changes, and I have left the mutex associated with the CV using a pthread_mutex_t. There is a #define USE_PTHREAD in libusbi.h and windows_compat.h . The comments should explain it. 1&2 work. Once 0 works, 1 will be deprecated. 2 should behave identically to before, while 1 is the "new stuff". http://git.libusb.org/?p=libusb-mplante.git;a=shortlog;h=refs/heads/lesspthr ead Michael ------------------------------------------------------------------------------ The Planet: dedicated and managed hosting, cloud storage, colocation Stay online with enterprise data centers and the best network in the business Choose flexible plans and management services without long-term contracts Personal 24x7 support from experience hosting pros just a phone call away. http://p.sf.net/sfu/theplanet-com _______________________________________________ Libusb-devel mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/libusb-devel |
| Powered by Nabble | Edit this page |
