[PATCH 0/37] Official Windows backend integration

[Pete Batard]

Hi Daniel,

As requested, I am submitting the Windows backend integration patches
directly to you, with the list in CC. As some of these patches are
likely to be bigger than the maximum attachment size, some of them might
be filtered out, so, for people monitoring this on the list, gaps are
expected in the sequence.

For information, these patches are 100% identical to what can be found
in Michael's Windows Merge 2 branch
(http://git.libusb.org/?p=libusb-mplante.git;a=shortlog;h=refs/heads/windows_merge2)
which was started about 2 months ago, up to patch 35 (updated AUTHORS).

The last 2 patches are commits that were flagged "do NOT integrate" at
Peter's request, but which I think need to be part of the official
branch for the following reasons:

- the libusb-1.0.rc, which is automatically generated by running
configure from MinGW or cywgin (but NOT MSVC/DDK) is needed for all
users compiling the backend, and compilation will break unless the file
is present. Peter expects MSVC/DDK users to only download source from
source tarballs, where we would have pre-generated the file, so he
doesn't believe it needs to be included in official. On the other hand,
I estimate that, as long as we have a public git, we will have MSVC or
DDK users downloading the source from the official git tree, and if they
do, their compilation will fail. This is why I believe the file should
be included always.

- Tthe Chinese NLS patch is a well known contention point at this stage.
Currently, we have all reasons to believe that CJK MinGW users compiling
libusb from git will experience an issue, due to the locale settings,
which has been officially recognized by libtool as a bug. At this stage,
there doesn't exist any prebuilt libtool binaries for MinGW that
integrate this fix, and my experience has shown that recompiling libtool
from source with the latest version of MinGW, to have an up to date
fixed version, leads to further issues.
Therefore, the only course of action for people who want to avoid the
NLS issue is to patch libtool. Now, the NLS patch is not provided by
libtool as a standalone patch that we could just point to, so regardless
of what we do, we'd still need to provide that standalone patch for our
users somewhere.
Considering all this, as well as what is best for our users, I'm seeing
the provision of the patch and automated patching of libtool from
autogen as the most sensible option for the foreseeable future, and
that's the reason why I also believe this patch needs to be part of
official.

Now, regardless of these current 37 patches, I believe Peter expressed
some inkling to split/massage the git commits further, but right now,
both Michael and I are 100% happy with the breakdown we have, and, as
we've pretty much extended the integration review period by one month
(from the one month more or less originally planned) so that Peter had a
better chance to provide guidance (which he did), I think any further
delay will need to be seriously balanced with regards to whether it
actually makes sense for our users in the long run.

Finally, because this is a very large amount of new code, and there
should still be bugs lurking around, the Windows backend will need to be
flagged as EXPERIMENTAL one way or another, so that any developer
planning to use it is aware that additional changest / bugfixes are not
unexpected. I'm open to suggestion as to what would be the best place to
do just that.

There's also the small issue of prepackaging the sources for the
official release (with a prebuilt configure that works on all
platforms), but my tests show that this shouldn't be much of a problem,
and we can address that once you've had a chance to review these 37
patches and let us know if you are OK to integrate them.

Regards,

/Pete

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Michael Bender-3]

Pete Batard wrote:
> As requested, I am submitting the Windows backend integration patches
> directly to you, with the list in CC. As some of these patches are
> likely to be bigger than the maximum attachment size, some of them might
> be filtered out, so, for people monitoring this on the list, gaps are
> expected in the sequence.
So why, in the second decade of the 21st Century, are we using e-mail
to send out a 37-part code patch? Is this so that the three users that
live in Outer Vladivostok and connect to the Internet via a 300-baud
dialup link to a FidoNet BBS-to-USENET gateway in Slovinsk can be
the beneficiaries of all this fine work?

Don't we have technologies like web servers and FTP sites and
Sourceforge and all the rest of those fancy-schmancy new things
that can be used to post this kind of stuff these days?

mike



------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Greg KH-2]

On Tue, May 04, 2010 at 12:14:12PM -0700, Michael Bender wrote:

> Pete Batard wrote:
> > As requested, I am submitting the Windows backend integration patches
> > directly to you, with the list in CC. As some of these patches are
> > likely to be bigger than the maximum attachment size, some of them might
> > be filtered out, so, for people monitoring this on the list, gaps are
> > expected in the sequence.
> So why, in the second decade of the 21st Century, are we using e-mail
> to send out a 37-part code patch? Is this so that the three users that
> live in Outer Vladivostok and connect to the Internet via a 300-baud
> dialup link to a FidoNet BBS-to-USENET gateway in Slovinsk can be
> the beneficiaries of all this fine work?
>
> Don't we have technologies like web servers and FTP sites and
> Sourceforge and all the rest of those fancy-schmancy new things
> that can be used to post this kind of stuff these days?

It's easier to review code this way.

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Tim Roberts]

Michael Bender wrote:
>
> So why, in the second decade of the 21st Century, are we using e-mail
> to send out a 37-part code patch?

I know I'm going to be sorry for starting this, but it's still only the
FIRST decade of the 21st Century.  It's the END of the first decade, to
be sure, but the first decade nonetheless.

--
Tim Roberts, [hidden email]
Providenza & Boekelheide, Inc.


------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Peter Stuge]

Pete Batard wrote:
> - the libusb-1.0.rc, which is automatically generated by running
> configure from MinGW or cywgin (but NOT MSVC/DDK) is needed for all
> users compiling the backend, and compilation will break unless the file
> is present.

It's needed for all Windows users.


> Peter expects MSVC/DDK users to only download source from
> source tarballs, where we would have pre-generated the file, so he
> doesn't believe it needs to be included in official.

I don't think I actually commented on .rc, but in any case we need to
figure a way to not store the version numbers in more than
authoritative place..


> On the other hand, I estimate that, as long as we have a public git,
> we will have MSVC or DDK users downloading the source from the
> official git tree, and if they do, their compilation will fail.

Be that as it may I don't think we should support such users.

If they want to do development on libusb itself then they should get
the tools needed to do so. If they only want to *use* libusb, and if
fulfilling tool requirements is too difficult or even just a chore
then these *users* should grab a binary release.

Oh, and I think it would be very nice if all the VS noise was cleaned
out of the .rc since I believe it isn't neccessary? (This goes for
all other files too, .dsp .dsw .sln .vcproj etc. I haven't looked at
those in detail recently though.)


> This is why I believe the file should be included always.

Either the .rc is included, or the .rc.in is included, it's not
sensible to keep both. Having .rc instead of .rc.in duplicates the
version number from configure.ac and thus creates opportunity for
conflicting version numbers within one code base. This is obviously
not good. Maybe the risk for human error can be mitigated by
scripting, but I would much prefer to only have a single place be
authoritative, than to have to keep two places in sync.


> - Tthe Chinese NLS patch is a well known contention point at this
>   stage.

My argument is basically the same as above. It's not a bug in libusb,
it seems far-fetched to me that *users* who want to *use* libusb
would download libusb from the developer infrastructure, and go on to
build libusb, before they use it. It seems to me that we can avoid
lots of support issues from such users (regardless of where in the
world they are) by instead serving them with binary snapshots.


> MinGW users compiling libusb from git

There was brief discussion about a separate Makefile for MinGW
builds, similar to how there is now one or more build scripts per
Microsoft compiler on Windows.


> Therefore, the only course of action for people who want to avoid the
> NLS issue is to patch libtool.

There are several workarounds, and modifying ltmain.sh as installed
by libtoolize is just one of then.

export LANG=C

is probably the simplest one.


> Now, the NLS patch is not provided by libtool as a standalone patch
> that we could just point to,

Not sure what you mean here, but since they are also using git, it is
very easy to refer to the particular commit id, and maybe their web
interface for git also allows downloading a particular commitdiff
(ie. a patch).


> Now, regardless of these current 37 patches, I believe Peter expressed
> some inkling to split/massage the git commits further,

I wanted to review and maybe actually combined rather than split :)
but also I think there are still issues to solve.

For one, simple stuff like cleaning up files to be free of noise, but
also there's a matter of 32/64 builds that I think warrants
discussion.

When looking into mingw-w64 in order to create previously mentioned
snapshots automatically, I had the privilege to discuss 32/64
libraries with the main mingw-w64 guy in that area. He strongly
recommended that we simply use different filenames for 32 and 64 bit
versions of the library files, to avoid confusion of files. This may
already be the case for some of the MS-native build files that Pete
and Michael have been using but is e.g. not at all the case when
using autotools build, which is what at least Cygwin should use, and
possibly also builds on MinGW.


> I think any further delay will need to be seriously balanced

I expect there to still be delay. As you wrote it's a lot of code,
and it should still be reviewed.

Hopefully there will be feedback on the list - I will certainly also
try to comment further.


> There's also the small issue of prepackaging the sources for the
> official release

On the topic of packaging there's also the issue of binary packages.
As I mentioned I will set up something for automatic snapshots, but
what should the output look like?

Would it work to reuse the same layout as in the snapshots you've
made, Pete?


//Peter

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Michael Plante]

Peter Stuge wrote:
>> Oh, and I think it would be very nice if all the VS noise was cleaned
>> out of the .rc since I believe it isn't neccessary?

Maybe, since it is effectively source code (not C).


>> (This goes for all other files too, .dsp .dsw .sln .vcproj etc.

Doubtful, though perhaps they could be generated.  I'm wary of signing on to
making any more files "generated", since the per-commit auto-generation
process is still vaporware.


>> similar to how there is now one or more build scripts per
>> Microsoft compiler on Windows.

The MSVC ones (dsp/dsw/vcproj/sln) are not build scripts.  It's not that
simple.  They have metadata related to the IDE, as well as information
related to building, but neither is in a form that could be directly fed to
a compiler.  Tools exist to convert the project files into build scripts,
but there's no general way to reverse the process.  I.e., build scripts are
the derived product in this case, not the project files.  While the project
files could be given a  "derived" status, perhaps at the cost of some
information, all changes made in the IDE by libusb developers would be lost.

I should point out that I've already lost the ability to have complete
control over the resource settings (.rc) in the IDE because of our debug vs
release #ifdef stuff, even before we made that file generated.  Changes only
affect half the builds, essentially.  Fiddling with this too much raises
problems.

Technically, there's another file that I've refrained from committing, but
that would ideally be useful to have, the .opt file.  I kept it to myself,
as it's a binary/closed format that stores some other settings, and I didn't
want rocks thrown at me. :)  I never did figure out the complete breakdown
of what goes in there, and what goes in the dsp's, but I decided we can live
without it.  It is per-dsw, oddly enough.  And it changes (probably a
timestamp or something) every time you close the workspace...  I know the
debug command line and some of the source browser settings go in there, but
I'm not certain what else.


>> As I mentioned I will set up something for automatic snapshots, but
>> what should the output look like?

I'd worry about source snapshots first, and, if all goes well, a separate
process can be derived for binary snapshots from that.

Michael


------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Xiaofan Chen]

On Wed, May 5, 2010 at 3:14 AM, Michael Bender
<[hidden email]> wrote:
> Don't we have technologies like web servers and FTP sites and
> Sourceforge and all the rest of those fancy-schmancy new things
> that can be used to post this kind of stuff these days?
>

You know what, this is the way Linux kernel development works:
sending patches by email.

Greg KH can send out 100s of emails in the Linux usb mailing lists.
So 37 for libusb is not much compare to that. ;-)

Since it works well for big projects like Linux, it should work
for smaller projects like libusb as well.


--
Xiaofan http://mcuee.blogspot.com

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Pete Batard]

On 2010.05.04 23:57, Peter Stuge wrote:
> I don't think I actually commented on .rc,

Now that I think about it, that's right. Still the .rc in merge2 is
currently flagged not to be integrated, and I see it as an issue.

> but in any case we need to
> figure a way to not store the version numbers in more than
> authoritative place..

Agreed. The best thing would be to have a script that automatically
generates the .rc (and possibly the msvc/config.h) and places them in
the git tree so that they are available, while still avoiding any
duplication work for maintainers.

Right now, I believe that this is something that we can try to address
post release, rather than something that should warrant additional delay.

>> On the other hand, I estimate that, as long as we have a public git,
>> we will have MSVC or DDK users downloading the source from the
>> official git tree, and if they do, their compilation will fail.
>
> Be that as it may I don't think we should support such users.

I think we should, especially as we very easily can and it does not
introduce much of any risk, if any. I believe it should be our duty to
help our users when we can, rather than set what I still see as very
artificial (and unneeded) limitations.

> If they want to do development on libusb itself then they should get
> the tools needed to do so. If they only want to *use* libusb, and if
> fulfilling tool requirements is too difficult or even just a chore
> then these *users* should grab a binary release.

I'm uncomfortable with the idea that to spare duplication (which we plan
to address in the future) for less than a handful of people (the
maintainers of the git tree), we will take the risk of inconveniencing
many of our users. I really don't see the point of placing requirements
on our users ("don't use git - use snapshots only") when we're not ready
to place requirements on ourselves ("generate the .rc from running
autogen before a new release").

> Oh, and I think it would be very nice if all the VS noise was cleaned
> out of the .rc since I believe it isn't neccessary? (This goes for
> all other files too, .dsp .dsw .sln .vcproj etc. I haven't looked at
> those in detail recently though.)

The 2008 are not part of the commits, as they can be generated from the
2005 files, so they have already been removed. Using the MSVC6 file for
VS2005/2008 is not an option, as they are missing the 64 bit builds.

I'm all for trying to limit the MSVC "noise", but short of switching the
whole project to cmake, I don't really see how we can avoid providing
all the files you mentioned above. I'm afraid limiting the number of
project files is just not the way Microsoft designed their development
environment for.

> Either the .rc is included, or the .rc.in is included, it's not
> sensible to keep both. Having .rc instead of .rc.in duplicates the
> version number from configure.ac and thus creates opportunity for
> conflicting version numbers within one code base. This is obviously
> not good. Maybe the risk for human error can be mitigated by
> scripting, but I would much prefer to only have a single place be
> authoritative, than to have to keep two places in sync.

Well, the idea is that one the initial .rc is in official, the git
maintainer would update with .rc generated from the .rc.in if needed (or
we could add as a maintainer task to always run autogen and commit the
generated .rc for a new release). Short of breaking any possibility of
compiling the Windows backend from git, unless we have some scripted
generation for the git maintainers, I don't really see how we can to
avoid the risk of conflicting version numbers.

But at this stage, I'm really all for leaving Daniel decide what he
wants to do with regards to 36 & 37.

>> - Tthe Chinese NLS patch is a well known contention point at this
>>    stage.
>
> My argument is basically the same as above. It's not a bug in libusb,
> it seems far-fetched to me that *users* who want to *use* libusb
> would download libusb from the developer infrastructure, and go on to
> build libusb, before they use it. It seems to me that we can avoid
> lots of support issues from such users (regardless of where in the
> world they are) by instead serving them with binary snapshots.

On the contrary, I believe we can avoid a lot of support issues from
users who will use the official *public* git tree if we include the
patch. I'm counting on libusb users and developers to do whatever they
please, and on them retrieving the latest version of libusb using git,
be it only because it's there (As well as enterprise grade, we also need
to cater for developers interested in cutting edge and most up to date
versions). I also previously highlighted the scenario of why people who
might have even a remote inkling to develop for libusb would chose to
use git rather than any other method (because they then can submit
patches to this list if needed), and why, by not providing the libtool
patch, we would then be alienating users that can make potential
significant contributions to our project.

>> MinGW users compiling libusb from git
>
> There was brief discussion about a separate Makefile for MinGW
> builds, similar to how there is now one or more build scripts per
> Microsoft compiler on Windows.

Since this is a bug that can potentially affect all environments that
use libtool, including Linux, Darwin, we can't tell for sure that we're
not gonna get a non MinGW CJK user that's affected by it tomorrow.
Again, this is a problem that can be sorted once and for all, right
here, right now, by applying the *official* libtool fix, and leave our
users happy ever after... And again, I fail to see how we can chose not
to helping our users, when we have a very good (and non risky, since we
are reusing an official patch) opportunity to do so. As long as we are
relying on the libtool utility for our code generation, the argument
that libusb should not touch libtool patches does not quite cut it for me.

>> Therefore, the only course of action for people who want to avoid the
>> NLS issue is to patch libtool.
>
> There are several workarounds, and modifying ltmain.sh as installed
> by libtoolize is just one of then.
>
> export LANG=C
>
> is probably the simplest one.

Confirmed as NOT working by Xiaofan in a previous mail, as this won't
stay from one session to another.

>> Now, the NLS patch is not provided by libtool as a standalone patch
>> that we could just point to,
>
> Not sure what you mean here, but since they are also using git, it is
> very easy to refer to the particular commit id, and maybe their web
> interface for git also allows downloading a particular commitdiff
> (ie. a patch).

Their commit (7877d1d13f5f2f67b3d062603cb82791fc479257)includes a
modification of their changelog and also, the main part applies to
ltmain.m4sh (the libtool system file) rather than ltmain.sh.
=> Unusable as is for our users.

> For one, simple stuff like cleaning up files to be free of noise,

Not sure what you mean by that. Unless you can clarify the above, I'd
say that noise cleanup can wait for the next release. I did clean up
TODOs in my code some time ago (except the ones that I am actually
planning to implement in future official releases), and I plan to keep
some experimental stuff like autoclaim while the releases are
experimental, so that we can get valuable user feedback.

> also there's a matter of 32/64 builds that I think warrants
> discussion.
>
> When looking into mingw-w64 in order to create previously mentioned
> snapshots automatically, I had the privilege to discuss 32/64
> libraries with the main mingw-w64 guy in that area. He strongly
> recommended that we simply use different filenames for 32 and 64 bit
> versions of the library files, to avoid confusion of files. This may
> already be the case for some of the MS-native build files that Pete
> and Michael have been using but is e.g. not at all the case when
> using autotools build, which is what at least Cygwin should use, and
> possibly also builds on MinGW.

Again, is that something that should stall the current integration
process? I personally don't think so. We can fine tune issues like the
naming of the 64 bit libraries or any other problem that we might have
overlooked in later releases (do the MinGW-w64 have specific
recommendations wrt 64 bit naming? We might as well follow these if they
do rather than pick our own scheme). Also, if the 32/64 bit naming is
seen as that critical for our users, I expect to be hearing something
after the first official release, which should also help decide how we
want to address that specific problem.

>> I think any further delay will need to be seriously balanced
>
> I expect there to still be delay. As you wrote it's a lot of code,
> and it should still be reviewed.

Indeed. What we are currently proposing for integration needs to be
reviewed, while leaving what we are NOT currently proposing for
integration to be sorted our later, as I believe we've worked hard
enough over the past 2 months to make sure there wasn't anything that
might be considered as a showstopper in there.

We can always commit further changes in future releases, and that's
really the whole point of the first few releases being flagged as
experimental. At this stage, I'd much rather have user feedback from
official experimental releases than a handful of maintainers and
developers trying to guess what said users will want.

> On the topic of packaging there's also the issue of binary packages.
> As I mentioned I will set up something for automatic snapshots, but
> what should the output look like?
>
> Would it work to reuse the same layout as in the snapshots you've
> made, Pete?

Well, I'm not entirely sure I have the best binary snapshot breakdown
out there. I've fixed some issues, like including the libusb.h or making
sure that the libraries I generate can be run from XP onwards, but for
instance, I'm not including the static libs from MSVC/DDK, and I
wouldn't mind getting feedback as to what people think would make sense
to have in a binary package (size of the binaries might be part of the
equation. Maybe we should split them?)

You can reuse the layout from my automated scripts if you want, but I'm
open to suggestions as to what content our binary snapshots should have.

Regards,

/Pete

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Xiaofan Chen]

On Wed, May 5, 2010 at 6:57 AM, Peter Stuge <[hidden email]> wrote:

>> MinGW users compiling libusb from git
>
> There was brief discussion about a separate Makefile for MinGW
> builds, similar to how there is now one or more build scripts per
> Microsoft compiler on Windows.
>

That would help MinGW users. But now it is not done yet. It
may be more complicated than libusb-win32. In any case,
it should not delay the integration of the Windows backend.

>> Therefore, the only course of action for people who want to avoid the
>> NLS issue is to patch libtool.
>
> There are several workarounds, and modifying ltmain.sh as installed
> by libtoolize is just one of then.
>
> export LANG=C
>
> is probably the simplest one.

"export LANG=C" needs to be presented before both "configure"
and "make". In that case, it is okay.



--
Xiaofan http://mcuee.blogspot.com

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Xiaofan Chen]

On Wed, May 5, 2010 at 6:57 AM, Peter Stuge <[hidden email]> wrote:

> For one, simple stuff like cleaning up files to be free of noise, but
> also there's a matter of 32/64 builds that I think warrants
> discussion.
>
> When looking into mingw-w64 in order to create previously mentioned
> snapshots automatically, I had the privilege to discuss 32/64
> libraries with the main mingw-w64 guy in that area. He strongly
> recommended that we simply use different filenames for 32 and 64 bit
> versions of the library files, to avoid confusion of files. This may
> already be the case for some of the MS-native build files that Pete
> and Michael have been using but is e.g. not at all the case when
> using autotools build, which is what at least Cygwin should use, and
> possibly also builds on MinGW.
>

I am not so sure if it is a good idea to support MinGW-W64 as of now.
At least I have not figured how to build it with MinGW-W64 as yet. As
far as I know, it is not matured and the basic documentation of how
to build libusb-1.0 Windows backend is not there (what to download,
what to install, how to setup MSYS, etc).

As for the DLL name, it makes sense to differentiate. As for the
library, I am not 100% sure which way to go. At least libusb-win32
uses the same name for the lib file but different name for the DLL file.


--
Xiaofan http://mcuee.blogspot.com

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Peter Stuge]

Michael Plante wrote:
> Peter Stuge wrote:
> >> Oh, and I think it would be very nice if all the VS noise was
> >> cleaned out of the .rc since I believe it isn't neccessary?
>
> Maybe, since it is effectively source code (not C).

I've written lots of resource files. The noise is the many blank
lines, lines with nonsense comments and the AFX junk, which AFAIK is
some kind of precompilation trick that VS uses. That's the part I
don't know if we need - I hope those more experienced with VS can
answer that?

The point is that the source code should be in good condition, so
that it's easy to work with, and so that any future references into
the code history are also easy. I don't think that committing junk
now, and cleaning it up "after the release" is good enough. Best to
make it clean the first time. As the .rc says it was generated, and
it shows, but I don't think it should. Same for other generated
files.


> >> (This goes for all other files too, .dsp .dsw .sln .vcproj etc.
>
> Doubtful, though perhaps they could be generated.  I'm wary of
> signing on to making any more files "generated", since the
> per-commit auto-generation process is still vaporware.

Yeah, I wish there were more hours in a day, so that I could spend
more time on libusb. I might have made more progress on automated
builds already, but there's that 32/64 issue, and also our little
adventure re. building both static and shared at once.


> >> similar to how there is now one or more build scripts per
> >> Microsoft compiler on Windows.
>
> The MSVC ones (dsp/dsw/vcproj/sln) are not build scripts.

I'm sorry that I didn't explicitly write "project files" or something
else that would indicate that I do understand that these are more
than .bat or .cmd files.


> It's not that simple.  They have metadata related to the IDE,

..which I don't think we care much about, or? (I would even be happy
to only include one set of "compiler support" files, but I realize it
will irk the major contributors, who all happen to be using very
different development environments.)


> as well as information related to building,

..which we do care very much about.


> all changes made in the IDE by libusb developers would be lost.

I don't follow here I'm afraid. Can you give an example of a
setting that would be lost, and when it would be lost? :)

I think you mean that it will get lost when the files are
re-generated, if they were changed to be generated?


> I should point out that I've already lost the ability to have complete
> control over the resource settings (.rc) in the IDE because of our
> debug vs release #ifdef stuff,

Can you point me to that stuff? Which commit(s)?


> Changes only affect half the builds, essentially.

I think we should not make very many changes anyway, right?



> .opt binary/closed format
> I didn't want rocks thrown at me. :)

Good call. :)


> >> As I mentioned I will set up something for automatic snapshots,
> >> but what should the output look like?
>
> I'd worry about source snapshots first, and, if all goes well, a
> separate process can be derived for binary snapshots from that.

Source is easy, that infrastructure already exists in the form of
make distcheck.


//Peter

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Xiaofan Chen]

On Wed, May 5, 2010 at 12:42 PM, Peter Stuge <[hidden email]> wrote:

>> >> (This goes for all other files too, .dsp .dsw .sln .vcproj etc.
>>
>> Doubtful, though perhaps they could be generated.  I'm wary of
>> signing on to making any more files "generated", since the
>> per-commit auto-generation process is still vaporware.
>
> Yeah, I wish there were more hours in a day, so that I could spend
> more time on libusb. I might have made more progress on automated
> builds already, but there's that 32/64 issue, and also our little
> adventure re. building both static and shared at once.
>

Correct me if I am wrong. You basically do not like to see any
VS project files in the git tree, right?

What are the tools you intend to use for the automated builds?
Or you may have the desire to use Open Source tools to
Microsoft tools (VS or DDK/WDK) for the libusb-1.0 Windows
backend.

If the above is correct, I can understand your posts even if
I do not quite agree. The windows backend is anyway for
Windows users and many more software developer would
use Microsoft tools instead of Cygwin/MinGW.


--
Xiaofan http://mcuee.blogspot.com

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Michael Plante]

Peter Stuge wrote:
>> and the AFX junk, which AFAIK is some kind of precompilation trick that
VS uses.

Probably junk (have to look), but AFX is an old name for MFC that stuck.
(Application Framework)


>> > It's not that simple.  They have metadata related to the IDE,
>>
>> ..which I don't think we care much about, or? (I would even be happy
>> to only include one set of "compiler support" files, but I realize it
>> will irk the major contributors, who all happen to be using very
>> different development environments.)

The IDE is my whole reason to use Visual Studio, rather than MinGW or
Cygwin.  Without it, I wouldn't have bothered to buy Visual Studio.  Without
the project files...well, I'd just wind up re-creating them locally and
rebasing continuously, so there really is no "without".  I recognize Graeme
has some reason he prefers Visual Studio, aside from the IDE, but I'm
talking about me.  And besides, there are nmake scripts (similar to
makefiles) if you don't want the IDE.


>> > as well as information related to building,
>>
>> ..which we do care very much about.

Of course, but, again, we could just use nmake scripts if that were *ALL* we
cared about.


>> > all changes made in the IDE by libusb developers would be lost.
>>
>> I don't follow here I'm afraid. Can you give an example of a
>> setting that would be lost, and when it would be lost? :)
>>
>> I think you mean that it will get lost when the files are
>> re-generated, if they were changed to be generated?

Yes.  In the sense that the IDE can't generate .in files (or whatever "seed"
you propose).  This may not be a loss in the long run, since the project
files for all versions would stay in sync, but the disadvantage is it would
be *very* annoying.


>> > I should point out that I've already lost the ability to have complete
>> > control over the resource settings (.rc) in the IDE because of our
>> > debug vs release #ifdef stuff,
>>
>> Can you point me to that stuff? Which commit(s)?

Hrm, whenever we were first discussing it.  I'm not sure the editable
version even made it into git.  I'd have to look.  Basically, the moment we
decided to put #ifdefs in the rc file to support the OriginalFileName having
two different names.  I don't recall which, but editing it in visual studio
only changes one or the other, I think.  Last I checked.  Of course, it's
become so simple (just a version resource, no dialogs or anything) that I
don't mind editing it by hand.  On a grapical application this would be bad.
Here it is quite tolerable.

I was just pointing out that doing partially-supported things can cause
problems in general, some of which are tolerable.


>> > Changes only affect half the builds, essentially.
>>
>> I think we should not make very many changes anyway, right?

To the rc file?  See above.  To the project files, it'd be a pain, I think,
at least compared to how easy it *usually* is to change things.  Frankly,
it'd be no more painful than it is for people who use non-visual-studio
environments, so maybe that'll make you disregard this...  I think I'm
digging myself deeper. :(


>> Source is easy, that infrastructure already exists in the form of
>> make distcheck.

With a cross-compiler, though, right?  Or does .rc get generated regardless
of the build target?  (I guess I could check myself...)


Michael




------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Peter Stuge]

Pete Batard wrote:
> > but in any case we need to
> > figure a way to not store the version numbers in more than
> > authoritative place..
>
> Agreed. The best thing would be to have a script that automatically
> generates the .rc (and possibly the msvc/config.h) and places them in
> the git tree so that they are available, while still avoiding any
> duplication work for maintainers.

IMO anything generated does not belong in a source code repo. Might
as well commit binaries then.


> this is something that we can try to address post release

I think it should be addressed before commit.


> >> MSVC or DDK users downloading the source from the official git tree
> >
> > Be that as it may I don't think we should support such users.
>
> I think we should, especially as we very easily can and it does not
> introduce much of any risk, if any. I believe it should be our duty to
> help our users when we can, rather than set what I still see as very
> artificial (and unneeded) limitations.

I'm really sorry that you can't see the point of optimizing repo
contents for development only. There's a big difference between users
and contributing developers in every other project, and that has been
true for libusb too, as far as I can remember. Users prefer something
they can easily use over something that is more complicated to use
(e.g. they must build it first). Is your experience different?


> I really don't see the point of placing requirements on our users
> ("don't use git - use snapshots only")

Obviously not a requirement, but using git source has many implicit
requirements for the benefit of developers, and since git is their
tool that's what matters. If users very much do want to grab git
source they can - but they must then also fill the requirements for
developers.


> when we're not ready to place requirements on ourselves ("generate
> the .rc from running autogen before a new release").

Was this discussed? As you may know, autogen.sh *is* run as part of a
release, along with configure and make distcheck. I do agree that it
would be nice to generate e.g. .rc at packaging instead of at build.
I have been toying with the idea of a .cmd to do the job but need to
research cmd text processing capabilities.

> > Oh, and I think it would be very nice if all the VS noise was cleaned
> > out of the .rc since I believe it isn't neccessary? (This goes for
> > all other files too, .dsp .dsw .sln .vcproj etc. I haven't looked at
> > those in detail recently though.)
>
> The 2008 are not part of the commits, as they can be generated from the
> 2005 files, so they have already been removed. Using the MSVC6 file for
> VS2005/2008 is not an option, as they are missing the 64 bit builds.
>
> I'm all for trying to limit the MSVC "noise", but short of switching the
> whole project to cmake, I don't really see how we can avoid providing
> all the files you mentioned above.

Nor did I suggest that. Open the .rc in Notepad and I think you'll
see what I mean.


> I'm afraid limiting the number of project files is just not the way
> Microsoft designed their development environment for.

I'm sure it's great in other ways. :)

Now that you mention it there are many files. Could they go into a
subdirectory or would that be too awkward? Hm, maybe I asked before.


> Short of breaking any possibility of compiling the Windows backend
> from git,

I'd be fine with that. As long as it can be cross-compiled I will
be happy. :) I realize it's not useful for developers who prefer to
work on Windows though, so we should find a solution. Maybe that .cmd
file.


> But at this stage, I'm really all for leaving Daniel decide what he
> wants to do with regards to 36 & 37.

Or we can come up with a solution that requires no decision.


> cater for developers interested in cutting edge
..
> remote inkling to develop for libusb
..
> alienating .. potential significant contributions

One question is what development environments we want to support in
the libusb project. Personally I believe we hardly have resources to
support even the one, so it must be kept at a minimum. Three (other)
for only one platform seems way too ambitious.

In any case, developers always need to have more stuff than users.


> > separate Makefile for MinGW builds
>
> Again, this is a problem that can be sorted once and for all, right
> here, right now, by applying the *official* libtool fix, and leave our
> users happy ever after...

I assume you checked how ltmain.sh is created? Automatically trying
to apply the upstream fix is not guaranteed to always fit. We might
be able to come up with a variation of the fix that is more contrived
but more generic but I think we should spend time on some libusb
thing instead.


> I fail to see how we can chose not to helping our users

I think you're comparing apples with oranges. Users are not
developers.

Repos are developer tools. Users can access them too - but they will
need all developer requirements - so we should offer better
alternatives for users - and then there is no problem, because users
no longer have a reason to go to git, unless they actually want to do
development - which is a process that we need to create more
documentation for anyway.


> > export LANG=C
>
> Confirmed as NOT working by Xiaofan in a previous mail, as this
> won't stay from one session to another.

It works fine in what is quite likely the most common case;
sh autogen.sh && make

> >> Now, the NLS patch is not provided by libtool as a standalone patch
> >> that we could just point to,
> >
> > Not sure what you mean here, but since they are also using git, it is
> > very easy to refer to the particular commit id, and maybe their web
> > interface for git also allows downloading a particular commitdiff
> > (ie. a patch).
>
> Their commit (7877d1d13f5f2f67b3d062603cb82791fc479257)includes a
> modification of their changelog and also, the main part applies to
> ltmain.m4sh (the libtool system file) rather than ltmain.sh.
> => Unusable as is for our users.

People should patch the libtool files that are installed in their
system, rather than the output from those files that is ltmain.sh
in the libusb directory.


> > For one, simple stuff like cleaning up files to be free of noise,
>
> Not sure what you mean by that.

If this is still not clear then let me know and I'll make sure to
do the changes.


> Unless you can clarify the above, I'd say that noise cleanup can
> wait for the next release.

Wrong approach. I guess you haven't dealt much with repositories that
have cleanups in their history. It's really horrible to try to do any
kind of work across those cleanups.


> I did clean up TODOs

I don't mind TODOs unless they're small or important enough to
implement instead. :)


> I plan to keep some experimental stuff like autoclaim while the
> releases are experimental, so that we can get valuable user
> feedback.

I think this needs more investigation - the matter of autoclaim
should just depend on how libusb behaves on the other platforms.
I hope you agree that it's a bad idea to introduce special behavior
on just one platform.


> > 32/64 libraries
>
> Again, is that something that should stall the current integration
> process?

I think it is part of the process. It must certainly be resolved one
way or another before the process can be said to be completed. Do you
disagree?

I would be happy to postpone the issue; e.g. producing only 32
binaries for libusb initially.


> We can fine tune issues like the naming of the 64 bit libraries or
> any other problem that we might have overlooked in later releases

I think that the library filenames are more than a tuning issue.
I'd like to try to get it right the first time.


> (do the MinGW-w64 have specific recommendations wrt 64 bit naming?

They didn't, but putting 64 in the filename seems like a good start
to me.


> Indeed. What we are currently proposing for integration needs to be
> reviewed, while leaving what we are NOT currently proposing for
> integration to be sorted our later, as I believe we've worked hard
> enough over the past 2 months to make sure there wasn't anything that
> might be considered as a showstopper in there.

I think there are several "showstoppers" still.


> user feedback from official experimental releases

Too bad there's not much feedback from the users who want to use git.
I actually don't expect too much feedback after release either.


> > binary packages.
> > automatic snapshots
> > layout
>
> Well, I'm not entirely sure I have the best binary snapshot
> breakdown out there. I've fixed some issues, like including the
> libusb.h

Yes, the header needs to be there. :)


> or making sure that the libraries I generate can be run from XP
> onwards,

How do you do that?


> but for instance, I'm not including the static libs from MSVC/DDK,

I will build with mingw-w64 and will definately include both shared
and static libraries. I believe they will be usable everywhere.


> and I wouldn't mind getting feedback as to what people think would
> make sense to have in a binary package (size of the binaries might
> be part of the equation. Maybe we should split them?)

Disk is cheap, computers decompress quickly and libusb is small.
I don't think we need to split.


> You can reuse the layout from my automated scripts if you want, but I'm
> open to suggestions as to what content our binary snapshots should have.

I'm also eager for any input, but I think we came up with a pretty
good layout when it was discussed earlier.


Thanks

//Peter

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Michael Plante]

Peter Stuge wrote:
>> IMO anything generated does not belong in a source code repo.

Not in the official one, anyway.  So yes.


>> Might as well commit binaries then.

That is a bit of a leap.  It is difficult (frequently impossible) to diff
binaries.


>> Now that you mention it there are many files. Could they go into a
>> subdirectory or would that be too awkward? Hm, maybe I asked before.

They could.  Would that convince you to let them stay?


>> I think that the library filenames are more than a tuning issue.
>> I'd like to try to get it right the first time.

Agreed.



Michael


------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Peter Stuge]

Xiaofan Chen wrote:

> >> >> (This goes for all other files too, .dsp .dsw .sln .vcproj etc.
> >>
> >> Doubtful, though perhaps they could be generated.  I'm wary of
> >> signing on to making any more files "generated", since the
> >> per-commit auto-generation process is still vaporware.
> >
> > Yeah, I wish there were more hours in a day, so that I could spend
> > more time on libusb. I might have made more progress on automated
> > builds already, but there's that 32/64 issue, and also our little
> > adventure re. building both static and shared at once.
>
> Correct me if I am wrong. You basically do not like to see any
> VS project files in the git tree, right?

If practical then I would certainly prefer it, but it may not be so
easy, so maybe we'll need the files there, but I think that they
should then be manually fixed up before a commit, so that the
contents is really clean and easy to work with.


> What are the tools you intend to use for the automated builds?

Crosscompile on the server with mingw-w64. (I'll reply to your other
message about maturity shortly.)


> If the above is correct, I can understand your posts even if
> I do not quite agree. The windows backend is anyway for
> Windows users and many more software developer would
> use Microsoft tools instead of Cygwin/MinGW.

Yes, but see also recent message with thoughts about supported
development environments. I agree that it would help Windows
developers to include some project files - but on the other hand we
don't (and IMO shouldn't) have Xcode project files although many Mac
developers use that. (But one difference is that Apple development
tools already include all autotools and so on, AFAIU.)


//Peter

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Xiaofan Chen]

On Wed, May 5, 2010 at 2:09 PM, Peter Stuge <[hidden email]> wrote:
> Pete Batard wrote:
>> Short of breaking any possibility of compiling the Windows backend
>> from git,
>
> I'd be fine with that. As long as it can be cross-compiled I will
> be happy. :) I realize it's not useful for developers who prefer to
> work on Windows though, so we should find a solution. Maybe that .cmd
> file.

Now I understand you quite well. I would say you have a point
here since some libusb-1.0 developer may have no access to
Windows.

DDK/WDK cmd file would be an option. But I remember there
may be issues with the run-time msvcrt problem. If that can
be solved, IMHO it is probably okay to drop all the VC project files.

FYI: WDK will be the official build method for libusb-win32
project in the future. We will try to maintain the capability
of MinGW for 32bit library and DLL build, but not the kernel
driver due to issues during my testing of the MinGW build.
Cross-compiling will not be officially supported either.

>> > export LANG=C
>>
>> Confirmed as NOT working by Xiaofan in a previous mail, as this
>> won't stay from one session to another.
>
> It works fine in what is quite likely the most common case;
> sh autogen.sh && make

This is correct.

>> > 32/64 libraries
>>
> I would be happy to postpone the issue; e.g. producing only 32
> binaries for libusb initially.

Then probably you are wrong. One of libusb-1.0 Windows backend's
main attractiveness is the support of WinUSB. And the main
attractiveness is that WinUSB works nicely under 64bit
WIndows Vista and 7.

>> (do the MinGW-w64 have specific recommendations wrt 64 bit naming?
>
> They didn't, but putting 64 in the filename seems like a good start
> to me.

The DLL should have 64 in it (32bit and 64bit DLL can be reside
in the same directory). But not so sure about the lib file. Should
be fine to put a 64 in it.

>
>> but for instance, I'm not including the static libs from MSVC/DDK,
>
> I will build with mingw-w64 and will definitely include both shared
> and static libraries. I believe they will be usable everywhere.
>

Could you post the instruction for MinGW-W64? Thanks.


--
Xiaofan http://mcuee.blogspot.com

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Peter Stuge]

Xiaofan Chen wrote:
> I am not so sure if it is a good idea to support MinGW-W64 as of now.
> At least I have not figured how to build it with MinGW-W64 as yet.

On Gentoo Linux it's very simple at least, and I've seen talk about
some installer packagese for running it natively on Windows.


> As far as I know, it is not matured

Consensus from many sources is that MinGW-W64 is higher quality than
"plain" MinGW. They have more complete API headers, and of course 64
support.


> and the basic documentation of how to build libusb-1.0 Windows
> backend is not there (what to download, what to install, how to
> setup MSYS, etc).

As I understand it is not different from "plain" MinGW.

When cross-compiling, I run ./configure --host=i686-w64-mingw32
or ./configure --host=x86_64-w64-mingw32 for MinGW-W64, instead
of ./configure --host=i686-mingw32 like I do with "plain" MinGW.


> As for the DLL name, it makes sense to differentiate. As for the
> library, I am not 100% sure which way to go. At least libusb-win32
> uses the same name for the lib file but different name for the DLL
> file.

Which .lib do you mean?

Separating names of static library and import .lib for the DLL is one
thing, I think this is already done in Pete's layout, I don't know
about that one, for one it's not clear how to build to different
target filenames using MinGW, but anyway I'm thinking that all 64
binaries should at least have a suffix with 64 added to their name.


//Peter

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Xiaofan Chen]

On Wed, May 5, 2010 at 2:39 PM, Peter Stuge <[hidden email]> wrote:

>> As far as I know, it is not matured
>
> Consensus from many sources is that MinGW-W64 is higher quality than
> "plain" MinGW. They have more complete API headers, and of course 64
> support.
>
>
>> and the basic documentation of how to build libusb-1.0 Windows
>> backend is not there (what to download, what to install, how to
>> setup MSYS, etc).
>
> As I understand it is not different from "plain" MinGW.

Not really. Try it under Windows and post your documentation
and I will be grateful.

> When cross-compiling, I run ./configure --host=i686-w64-mingw32
> or ./configure --host=x86_64-w64-mingw32 for MinGW-W64, instead
> of ./configure --host=i686-mingw32 like I do with "plain" MinGW.
>

This is for cross build. libusb-1.0 should formost be able to be
built under Windows. Cross-build is an alternatives for the
libusb maintainers who do not want to install Windows.
For the real libusb-1.0 Windows developers, Windows is the
OS to use for developing and testing.

>> As for the DLL name, it makes sense to differentiate. As for the
>> library, I am not 100% sure which way to go. At least libusb-win32
>> uses the same name for the lib file but different name for the DLL
>> file.
>
> Which .lib do you mean?

I mean libusb.lib for MSVC32 and MSVC64. They will be put into
different directory. And it seems to me the norm is that they
have the same name for MSVC.

> Separating names of static library and import .lib for the DLL is one
> thing, I think this is already done in Pete's layout, I don't know
> about that one, for one it's not clear how to build to different
> target filenames using MinGW, but anyway I'm thinking that all 64
> binaries should at least have a suffix with 64 added to their name.
>




--
Xiaofan http://mcuee.blogspot.com

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel

[Peter Stuge]

Xiaofan Chen wrote:
> >> and the basic documentation of how to build libusb-1.0 Windows
> >> backend is not there (what to download, what to install, how to
> >> setup MSYS, etc).
> >
> > As I understand it is not different from "plain" MinGW.
>
> Not really. Try it under Windows and post your documentation
> and I will be grateful.

I don't have a Windows system, but I could try to set up something
virtual.

I think that these files may be a good start:
(running on 64, building for 64)

http://sourceforge.net/projects/mingw-w64/files/External%20binary%20packages%20%28Win64%20hosted%29/make/make-3.81-20100227.zip/download
http://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Release%20for%20GCC%204.4.1/mingw-w64-bin_x86-64-linux_4.4.1-1a.tar.bz2/download
http://sourceforge.net/projects/mingw-w64/files/3rd%20party%20development%20tools/autotools-20100118.tar.bz2/download

(Their autotools doesn't include libtool, so I guess it may need to
be built manually. I did mention to them that it would be nice to
have a recent libtool package. :)


> > When cross-compiling, I run ./configure --host=i686-w64-mingw32
> > or ./configure --host=x86_64-w64-mingw32 for MinGW-W64, instead
> > of ./configure --host=i686-mingw32 like I do with "plain" MinGW.
>
> This is for cross build. libusb-1.0 should formost be able to be
> built under Windows. Cross-build is an alternatives for the
> libusb maintainers who do not want to install Windows.
> For the real libusb-1.0 Windows developers, Windows is the
> OS to use for developing and testing.

I agree completely, and the project files are important because of that.


> >> As for the DLL name, it makes sense to differentiate. As for the
> >> library, I am not 100% sure which way to go. At least libusb-win32
> >> uses the same name for the lib file but different name for the DLL
> >> file.
> >
> > Which .lib do you mean?
>
> I mean libusb.lib for MSVC32 and MSVC64.

The static library, or the import .lib for the DLL?


> They will be put into different directory. And it seems to me the
> norm is that they have the same name for MSVC.

Hmm, maybe the mingw-w64 advice is still a good idea, so that
although files *should* be in different directories, they *could* be
in the same one? Does that actually work? 64-bit binaries in
system32?


//Peter

------------------------------------------------------------------------------
_______________________________________________
Libusb-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/libusb-devel