Craft Stable

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Craft Stable

Hannah von Reth
Hi everyone,

We recently got a stable branch and will soon have KF5 binaries.
So basically we'll have a KF5 Windows SDK soon.
Regarding those binaries I created a poll[1] and I'd like you to participate.
For more information please read my blog post about the Craft Cache[2].

[1] https://phabricator.kde.org/V2
[2] https://the2ring.blogspot.de/2017/05/the-craft-cache.html

Happy crafting,
Hannah
Reply | Threaded
Open this post in threaded view
|

Re: Craft Stable

Thomas Friedrichsmeier-2
Hi!

On Mon, 8 May 2017 12:08:01 +0000
Hannah von Reth <[hidden email]> wrote:
> Hi everyone,
>
> We recently got a stable branch and will soon have KF5 binaries.
> So basically we'll have a KF5 Windows SDK soon.

Wonderful! That was my topmost wish for KDE on Windows for a long time.
Thanks for making it happen!

Some questions:
- Does it cache sources (perhaps at least for "archive" downloads, might
  be difficult for version controlled sources)? I think one of the most
  common causes of build breakage (esp. breakage over time) has been
  upstream moving their downloads around, or even removing some
  downloads, entirely.
- What's the workflow / commit policy? Are the stable branches to be
  considered "frozen", or to what degree? Is it ok to update extragear
  applications to new versions, for instance? Is it ok to patch for
  compilation with a non-default compiler? Is it ok to patch for
  non-critical stuff?
- I suppose the current "stable" branch is "the branch that will
  become the _next_ version of the SDK (aka Qt5.6.2/KF5.33.0 or
  whatever). How does this relate to master, then? Or is "stable" just
  a placeholder label until the branch naming has been decided on?
- What - besides voting on the poll - can we do to help? Now, and in
  the long run?

Regards
Thomas

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Craft Stable

Hannah von Reth
Hi Thomas


On 18/05/2017 10:29, Thomas Friedrichsmeier wrote:

> Hi!
>
> On Mon, 8 May 2017 12:08:01 +0000
> Hannah von Reth <[hidden email]> wrote:
>> Hi everyone,
>>
>> We recently got a stable branch and will soon have KF5 binaries.
>> So basically we'll have a KF5 Windows SDK soon.
> Wonderful! That was my topmost wish for KDE on Windows for a long time.
> Thanks for making it happen!
>
> Some questions:
> - Does it cache sources (perhaps at least for "archive" downloads, might
>   be difficult for version controlled sources)? I think one of the most
>   common causes of build breakage (esp. breakage over time) has been
>   upstream moving their downloads around, or even removing some
>   downloads, entirely.
We will package also all tool to archive a full KD5 build.
Packaging sources isn't planned as for most targets we use tarballs and
patches.
A build can be reproduced by using Craft, re uploading all KF5 tarballs
and Qt tarballs wouldn't add much value.
> - What's the workflow / commit policy? Are the stable branches to be
>   considered "frozen", or to what degree? Is it ok to update extragear
>   applications to new versions, for instance? Is it ok to patch for
>   compilation with a non-default compiler? Is it ok to patch for
>   non-critical stuff?
Commits to extragear are welcome also changes to KDE Applications.
I'd see the frameworks category as semi frozen, only reviewed patches
can be applied there and only if they are really needed.
So everything that's cached is semi frozen.
Updating a 3party lib won't happen besides a severe security issue.

> - I suppose the current "stable" branch is "the branch that will
>   become the _next_ version of the SDK (aka Qt5.6.2/KF5.33.0 or
>   whatever). How does this relate to master, then? Or is "stable" just
>   a placeholder label until the branch naming has been decided on?
The branch is a place holder as soon as 2017.05 is branched the stable
branch will change again.
Currently the only difference in the stable branch are some patches and
the default targets for the recipes.
Maybe dev would be a better name for the stable branch, I don't know.
The stable branch is meant to be the next release.
> - What - besides voting on the poll - can we do to help? Now, and in
>   the long run?
Besides voting. For now please try to use craft stable, I uploaded a new
cache this morning and I plan to do the release this week.
The cache should work, but there still might be some issues, so pls
build some applications also some qmake projects with the stable branch.

After the release we need to upgrade the KF5 and application versions
maybe update a few 3rd party libs, but the most important step will be
to  make sure that the current kf5 tarballs are OK. Of course adding and
fixing recipes for KDE Applications is also important and welcome.

Well and if everything works, please spread the word after the release
and make KDE devs to support the Windows platform and provide a single
application installer for their project.
>
> Regards
> Thomas

Cheers,

Hannah


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Craft Stable

Thomas Friedrichsmeier-2
Hi,

On Thu, 18 May 2017 14:59:07 +0200
Hannah von Reth <[hidden email]> wrote:
> On 18/05/2017 10:29, Thomas Friedrichsmeier wrote:
[...]

> > Some questions:
> > - Does it cache sources (perhaps at least for "archive" downloads,
> > might be difficult for version controlled sources)? I think one of
> > the most common causes of build breakage (esp. breakage over time)
> > has been upstream moving their downloads around, or even removing
> > some downloads, entirely.  
> We will package also all tool to archive a full KD5 build.
> Packaging sources isn't planned as for most targets we use tarballs
> and patches.
> A build can be reproduced by using Craft, re uploading all KF5
> tarballs and Qt tarballs wouldn't add much value.
What I was thinking about was "what if I try to build using a non-cached
compiler/architecture three months after the release?" In the past
something similar often meant some of the _downloads_ would fail for
being (re-)moved upstream. So I was wonderings, whether it would be
feasible to cache those upstream tarballs. Not the KF5 and Qt ones, but
libssl, dbus, icu, etc.

Not the most pressing matter, I agree.

> > - What's the workflow / commit policy? Are the stable branches to be
> >   considered "frozen", or to what degree? Is it ok to update
> > extragear applications to new versions, for instance? Is it ok to
> > patch for compilation with a non-default compiler? Is it ok to
> > patch for non-critical stuff?  
> Commits to extragear are welcome also changes to KDE Applications.
> I'd see the frameworks category as semi frozen, only reviewed patches
> can be applied there and only if they are really needed.
> So everything that's cached is semi frozen.
> Updating a 3party lib won't happen besides a severe security issue.
>
> > - I suppose the current "stable" branch is "the branch that will
> >   become the _next_ version of the SDK (aka Qt5.6.2/KF5.33.0 or
> >   whatever). How does this relate to master, then? Or is "stable"
> > just a placeholder label until the branch naming has been decided
> > on?  
> The branch is a place holder as soon as 2017.05 is branched the stable
> branch will change again.
> Currently the only difference in the stable branch are some patches
> and the default targets for the recipes.
> Maybe dev would be a better name for the stable branch, I don't know.
> The stable branch is meant to be the next release.
Ok. I suppose any naming scheme is ambiguous. Still wondering a bit
about master vs. stable. For generic work on portage should I commit to
stable, and cherry-pick to master (or vice versa)? While master is
for ... anything that does not yet seem ready for the next release?

> > - What - besides voting on the poll - can we do to help? Now, and in
> >   the long run?  
> Besides voting. For now please try to use craft stable, I uploaded a
> new cache this morning and I plan to do the release this week.
> The cache should work, but there still might be some issues, so pls
> build some applications also some qmake projects with the stable
> branch.

Craft stable seems to work *really* well for me, so far. I certainly
don't recall ever building kdelibs4/KF5 on Windows without some kind
of manual intervention, before. Great! Some notes:
- Perhaps the craft setup script could give a hint as to which
  platforms are build-cached, or simply mention the fact that some are
  cached, and some are not. I went through most of a MinGW-32 build,
  thinking the cache feature simply wasn't online, yet, before I
  figured it out...
- Perhaps it was extraordinary bad luck, but somehow craft had cached a
  NULL for the build cache's manifest.json. That kept the cache
  disabled on my second attempt, too. Perhaps it would be worth
  decreasing the json cache lieftime from one day to one or two hours.
  It's not like _this_ is causing a whole lot of traffic, I guess.
- After setting up craft a few times, I ran into some interesting
  problems due to inconsistent drive letter substs. Problem is that if
  the drive letter substitutions already exist, then simply calling
    subst R: somewhere
  does not have any effect. You have to do
    subst R: /D
  first.

> After the release we need to upgrade the KF5 and application versions
> maybe update a few 3rd party libs, but the most important step will be
> to  make sure that the current kf5 tarballs are OK. Of course adding
> and fixing recipes for KDE Applications is also important and welcome.
>
> Well and if everything works, please spread the word after the release
> and make KDE devs to support the Windows platform and provide a single
> application installer for their project.

Regarding this, what is the current "state of the art" approach to
packaging? For RKWard we had already set up an NSIS installer. This
seems to work to some degree (produces a package, still struggling with
missing MinGW runtime, and failure to find Qt plugins). However I have
an inkling that NSIS wouldn't be the "default" approach to packaging,
anymore. Where can I read up on this?

Regards
Thomas

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Craft Stable

Hannah von Reth
Hi Thomas


On 20/05/2017 22:31, Thomas Friedrichsmeier wrote:

> Hi,
>
> On Thu, 18 May 2017 14:59:07 +0200
> Hannah von Reth <[hidden email]> wrote:
>> On 18/05/2017 10:29, Thomas Friedrichsmeier wrote:
> [...]
>>> Some questions:
>>> - Does it cache sources (perhaps at least for "archive" downloads,
>>> might be difficult for version controlled sources)? I think one of
>>> the most common causes of build breakage (esp. breakage over time)
>>> has been upstream moving their downloads around, or even removing
>>> some downloads, entirely.  
>> We will package also all tool to archive a full KD5 build.
>> Packaging sources isn't planned as for most targets we use tarballs
>> and patches.
>> A build can be reproduced by using Craft, re uploading all KF5
>> tarballs and Qt tarballs wouldn't add much value.
> What I was thinking about was "what if I try to build using a non-cached
> compiler/architecture three months after the release?" In the past
> something similar often meant some of the _downloads_ would fail for
> being (re-)moved upstream. So I was wonderings, whether it would be
> feasible to cache those upstream tarballs. Not the KF5 and Qt ones, but
> libssl, dbus, icu, etc.
>
> Not the most pressing matter, I agree.
Well it would be possible but I'm not sure how many users would be
affected. But it might be a thing for the next release.
I plan to do a blog post for 2017.05 soon.

>
>>> - What's the workflow / commit policy? Are the stable branches to be
>>>   considered "frozen", or to what degree? Is it ok to update
>>> extragear applications to new versions, for instance? Is it ok to
>>> patch for compilation with a non-default compiler? Is it ok to
>>> patch for non-critical stuff?  
>> Commits to extragear are welcome also changes to KDE Applications.
>> I'd see the frameworks category as semi frozen, only reviewed patches
>> can be applied there and only if they are really needed.
>> So everything that's cached is semi frozen.
>> Updating a 3party lib won't happen besides a severe security issue.
>>
>>> - I suppose the current "stable" branch is "the branch that will
>>>   become the _next_ version of the SDK (aka Qt5.6.2/KF5.33.0 or
>>>   whatever). How does this relate to master, then? Or is "stable"
>>> just a placeholder label until the branch naming has been decided
>>> on?  
>> The branch is a place holder as soon as 2017.05 is branched the stable
>> branch will change again.
>> Currently the only difference in the stable branch are some patches
>> and the default targets for the recipes.
>> Maybe dev would be a better name for the stable branch, I don't know.
>> The stable branch is meant to be the next release.
> Ok. I suppose any naming scheme is ambiguous. Still wondering a bit
> about master vs. stable. For generic work on portage should I commit to
> stable, and cherry-pick to master (or vice versa)? While master is
> for ... anything that does not yet seem ready for the next release?
Well I'm open for suggestions, this is my first distro release xD
I'm a bit undecided about in which direction to cherry-pick but I guess
cherry-picking -x is a good idea.

>
>>> - What - besides voting on the poll - can we do to help? Now, and in
>>>   the long run?  
>> Besides voting. For now please try to use craft stable, I uploaded a
>> new cache this morning and I plan to do the release this week.
>> The cache should work, but there still might be some issues, so pls
>> build some applications also some qmake projects with the stable
>> branch.
> Craft stable seems to work *really* well for me, so far. I certainly
> don't recall ever building kdelibs4/KF5 on Windows without some kind
> of manual intervention, before. Great! Some notes:
> - Perhaps the craft setup script could give a hint as to which
>   platforms are build-cached, or simply mention the fact that some are
>   cached, and some are not. I went through most of a MinGW-32 build,
>   thinking the cache feature simply wasn't online, yet, before I
>   figured it out...
The whole setup experience needs some attention, but setting the default
to a cached version seems reasonable :D
> - Perhaps it was extraordinary bad luck, but somehow craft had cached a
>   NULL for the build cache's manifest.json. That kept the cache
>   disabled on my second attempt, too. Perhaps it would be worth
>   decreasing the json cache lieftime from one day to one or two hours.
>   It's not like _this_ is causing a whole lot of traffic, I guess.
Hm the cache definitely has potential for improvements but right now
there is just one cache and recaching the search would have its downside
too.
Maybe separate time outs would be a thing. Anyway the cache = {} might
have been another bug I already fixed, only a 404 should be cached as {}.
 
> - After setting up craft a few times, I ran into some interesting
>   problems due to inconsistent drive letter substs. Problem is that if
>   the drive letter substitutions already exist, then simply calling
>     subst R: somewhere
>   does not have any effect. You have to do
>     subst R: /D
>   first.
As I said before, there is improvement potential in the setup :D

>
>> After the release we need to upgrade the KF5 and application versions
>> maybe update a few 3rd party libs, but the most important step will be
>> to  make sure that the current kf5 tarballs are OK. Of course adding
>> and fixing recipes for KDE Applications is also important and welcome.
>>
>> Well and if everything works, please spread the word after the release
>> and make KDE devs to support the Windows platform and provide a single
>> application installer for their project.
> Regarding this, what is the current "state of the art" approach to
> packaging? For RKWard we had already set up an NSIS installer. This
> seems to work to some degree (produces a package, still struggling with
> missing MinGW runtime, and failure to find Qt plugins). However I have
> an inkling that NSIS wouldn't be the "default" approach to packaging,
> anymore. Where can I read up on this?
Well NSIS is the current way and I think even so the syntax is a
nightmare its one of the best nightmares I had until today.
In some experiments I was even able to create a windows store installer
out of an NSIS installer (you'll need a certificate for that and you'll
need to sign everything).
But if you manage to well integrate wix, patches are welcome.
Regarding the runtime, pls add a dependency to lib/runtime to rkward, it
would be probably reasonable to only set the dep if the compiler is
mingw, as we have a different solution with NSIS to provide the runtime
with msvc.
>
> Regards
> Thomas

Cheers,
Hannah


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Craft Stable

Thomas Friedrichsmeier-2
Hi,

On Sat, 20 May 2017 23:20:55 +0200
Hannah von Reth <[hidden email]> wrote:
> On 20/05/2017 22:31, Thomas Friedrichsmeier wrote:
[...]
> > Regarding this, what is the current "state of the art" approach to
> > packaging? For RKWard we had already set up an NSIS installer. This
> > seems to work to some degree (produces a package, still struggling
> > with missing MinGW runtime, and failure to find Qt plugins).
> > However I have an inkling that NSIS wouldn't be the "default"
> > approach to packaging, anymore. Where can I read up on this?  
> Well NSIS is the current way and I think even so the syntax is a
> nightmare its one of the best nightmares I had until today.

I feel your pain... See
https://cgit.kde.org/rkward.git/tree/windows_nsis/installer.nsi
(not sure, if we'll still provide this, but this was our custom
installer to update RKWard in existing installations of R and KDE). But
fortunately, craft does a good job of hiding all the mess...

[...]

> Regarding the runtime, pls add a dependency to lib/runtime to rkward,
> it would be probably reasonable to only set the dep if the compiler is
> mingw, as we have a different solution with NSIS to provide the
> runtime with msvc.

Ok, thanks for the hint about the lib/runtime dependency. Solves the
first issue, well. (Although I'll try to remember to look into
providing a patch for NullsoftInstallerPackager.py; seems odd that I
would have to specify extra dependencies just for packaging).

That still leaves the issue of Qt not finding its plugins. This even
prevents startup due to the missing "windows" platform plugin. I can
work around this by copying "plugins/platforms" to "bin", on the target
system. But that does not seem to be the proper fix.

This problem isn't new, and I even tried to come up with a solution a
year ago:
https://mail.kde.org/pipermail/kde-windows/2016-March/009636.html

Yes, that was rejected for good reason, and I certainly don't intend to
re-open that discussion. But back then, I was left thinking that a
proper solution was on its way:
https://git.reviewboard.kde.org/r/127169/ . That has been committed
long ago, but the problem persists.

Long background, simple question: How to fix this, properly?

Regards
Thomas

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Craft Stable

Hannah von Reth


On 21/05/2017 21:14, Thomas Friedrichsmeier wrote:

> Hi,
>
> On Sat, 20 May 2017 23:20:55 +0200
> Hannah von Reth <[hidden email]> wrote:
>> On 20/05/2017 22:31, Thomas Friedrichsmeier wrote:
> [...]
>>> Regarding this, what is the current "state of the art" approach to
>>> packaging? For RKWard we had already set up an NSIS installer. This
>>> seems to work to some degree (produces a package, still struggling
>>> with missing MinGW runtime, and failure to find Qt plugins).
>>> However I have an inkling that NSIS wouldn't be the "default"
>>> approach to packaging, anymore. Where can I read up on this?  
>> Well NSIS is the current way and I think even so the syntax is a
>> nightmare its one of the best nightmares I had until today.
> I feel your pain... See
> https://cgit.kde.org/rkward.git/tree/windows_nsis/installer.nsi
> (not sure, if we'll still provide this, but this was our custom
> installer to update RKWard in existing installations of R and KDE). But
> fortunately, craft does a good job of hiding all the mess...
>
> [...]
>
>> Regarding the runtime, pls add a dependency to lib/runtime to rkward,
>> it would be probably reasonable to only set the dep if the compiler is
>> mingw, as we have a different solution with NSIS to provide the
>> runtime with msvc.
> Ok, thanks for the hint about the lib/runtime dependency. Solves the
> first issue, well. (Although I'll try to remember to look into
> providing a patch for NullsoftInstallerPackager.py; seems odd that I
> would have to specify extra dependencies just for packaging).
>
> That still leaves the issue of Qt not finding its plugins. This even
> prevents startup due to the missing "windows" platform plugin. I can
> work around this by copying "plugins/platforms" to "bin", on the target
> system. But that does not seem to be the proper fix.
The current way is to move the plugins into the application folder,
before the actual packaging
occurs def preArchive() is called, where you can move the plugins to the
binary folder.
See
https://github.com/KDE/craft/blob/master/portage/kde/applications/kate/kate.py#L65
 

>
> This problem isn't new, and I even tried to come up with a solution a
> year ago:
> https://mail.kde.org/pipermail/kde-windows/2016-March/009636.html
>
> Yes, that was rejected for good reason, and I certainly don't intend to
> re-open that discussion. But back then, I was left thinking that a
> proper solution was on its way:
> https://git.reviewboard.kde.org/r/127169/ . That has been committed
> long ago, but the problem persists.
Craft already sets KDE_INSTALL_USE_QT_SYS_PATHS :)

Btw: I created the 2017.05 branch I didn't announce it yes as I hadn't
the time nor the energy to blog about it :P
>
> Long background, simple question: How to fix this, properly?
>
> Regards
> Thomas



signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Runtime packaging (was: Re: Craft Stable)

Thomas Friedrichsmeier-2
In reply to this post by Thomas Friedrichsmeier-2
On Sun, 21 May 2017 21:14:53 +0200
Thomas Friedrichsmeier <[hidden email]>
wrote:

> Hannah von Reth <[hidden email]> wrote:
> > Regarding the runtime, pls add a dependency to lib/runtime to
> > rkward, it would be probably reasonable to only set the dep if the
> > compiler is mingw, as we have a different solution with NSIS to
> > provide the runtime with msvc.  
>
> Ok, thanks for the hint about the lib/runtime dependency. Solves the
> first issue, well. (Although I'll try to remember to look into
> providing a patch for NullsoftInstallerPackager.py; seems odd that I
> would have to specify extra dependencies just for packaging).
Hm. Now I found out that I already did, once:
https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=e1e82b5d3a75d27635c474f76d38090be069939e

But you removed it, some time later:
https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=8e0ee6b658912f34aa80f45fb65d3fd4cd5399a6

Well, ok, I can see that libs/runtime handles this much cleaner, but at
the same time, having to add such a basic dependency, explicitly, _and_
only for packaging (meaning, I'll look in all the wrong places, first),
_and_ in a compiler-specific way, just seems really messy.

I think:
a) Since the problem is relevant for packaging, only, it should be
handled by the Packager.
CollectionPackagerBase::internalCreatePackage() (or one of its
helpers), would seem a good place where the runtime could be injected).
This could in fact be done by pulling in libs/runtime.
b) I really don't understand why this should be handled in a
compiler-specific way, and I suspect the only reason for this is
historic. libs/runtime does appear to cover the VC-redistributable
files. However, NullsoftInstallerPackager still adds considerable code
in order to detect the correct version of vcredist.exe, pass it to
NSIS, and unpack it, from inside the installer.

So - is there any particular reason to prefer the current way of
packaging the VC-runtime? Otherwise I'd look into providing patch to
include libs/runtime at packaging time, automatically, for both
compilers.

Regards
Thomas

attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Runtime packaging

Hannah von Reth
Well, yes it is a bit messy.

This difference is that with msvc, when you have installed the compiler
the runtime is already installed.

Providing the redist is also a bit cleaner than grapping dlls from
somewhere in the system.

Additionally you need runtime on mingw to be able to run crafted
applications outside of the craft env.

I guess one solution would be to drop the msvc code in runtime.py and
make virtual/base depend on runtime so that is always installed?


Cheers,

Hannah


On 23/05/2017 20:43, Thomas Friedrichsmeier wrote:

> On Sun, 21 May 2017 21:14:53 +0200
> Thomas Friedrichsmeier <[hidden email]>
> wrote:
>> Hannah von Reth <[hidden email]> wrote:
>>> Regarding the runtime, pls add a dependency to lib/runtime to
>>> rkward, it would be probably reasonable to only set the dep if the
>>> compiler is mingw, as we have a different solution with NSIS to
>>> provide the runtime with msvc.  
>> Ok, thanks for the hint about the lib/runtime dependency. Solves the
>> first issue, well. (Although I'll try to remember to look into
>> providing a patch for NullsoftInstallerPackager.py; seems odd that I
>> would have to specify extra dependencies just for packaging).
> Hm. Now I found out that I already did, once:
> https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=e1e82b5d3a75d27635c474f76d38090be069939e
>
> But you removed it, some time later:
> https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=8e0ee6b658912f34aa80f45fb65d3fd4cd5399a6
>
> Well, ok, I can see that libs/runtime handles this much cleaner, but at
> the same time, having to add such a basic dependency, explicitly, _and_
> only for packaging (meaning, I'll look in all the wrong places, first),
> _and_ in a compiler-specific way, just seems really messy.
>
> I think:
> a) Since the problem is relevant for packaging, only, it should be
> handled by the Packager.
> CollectionPackagerBase::internalCreatePackage() (or one of its
> helpers), would seem a good place where the runtime could be injected).
> This could in fact be done by pulling in libs/runtime.
> b) I really don't understand why this should be handled in a
> compiler-specific way, and I suspect the only reason for this is
> historic. libs/runtime does appear to cover the VC-redistributable
> files. However, NullsoftInstallerPackager still adds considerable code
> in order to detect the correct version of vcredist.exe, pass it to
> NSIS, and unpack it, from inside the installer.
>
> So - is there any particular reason to prefer the current way of
> packaging the VC-runtime? Otherwise I'd look into providing patch to
> include libs/runtime at packaging time, automatically, for both
> compilers.
>
> Regards
> Thomas


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Runtime packaging

Thomas Friedrichsmeier-2
Hi,

On Tue, 23 May 2017 21:34:06 +0200
Hannah von Reth <[hidden email]> wrote:

> Well, yes it is a bit messy.
>
> This difference is that with msvc, when you have installed the
> compiler the runtime is already installed.
>
> Providing the redist is also a bit cleaner than grapping dlls from
> somewhere in the system.
>
> Additionally you need runtime on mingw to be able to run crafted
> applications outside of the craft env.
>
> I guess one solution would be to drop the msvc code in runtime.py and
> make virtual/base depend on runtime so that is always installed?
Sounds reasonable (and easy!). Just to mention one downside:
Installation of the vcredist.exe is still specific to the NSIS
packager, so far. Any other present or future packager would still
require custom code for including the VC runtime. Not sure how
important that is to you. (Personally, I'm perfectly fine with one
functional installer type, whichever it may be.)

Regards
Thomas

>
>
> Cheers,
>
> Hannah
>
>
> On 23/05/2017 20:43, Thomas Friedrichsmeier wrote:
> > On Sun, 21 May 2017 21:14:53 +0200
> > Thomas Friedrichsmeier <[hidden email]>
> > wrote:  
> >> Hannah von Reth <[hidden email]> wrote:  
> >>> Regarding the runtime, pls add a dependency to lib/runtime to
> >>> rkward, it would be probably reasonable to only set the dep if the
> >>> compiler is mingw, as we have a different solution with NSIS to
> >>> provide the runtime with msvc.    
> >> Ok, thanks for the hint about the lib/runtime dependency. Solves
> >> the first issue, well. (Although I'll try to remember to look into
> >> providing a patch for NullsoftInstallerPackager.py; seems odd that
> >> I would have to specify extra dependencies just for packaging).  
> > Hm. Now I found out that I already did, once:
> > https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=e1e82b5d3a75d27635c474f76d38090be069939e
> >
> > But you removed it, some time later:
> > https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=8e0ee6b658912f34aa80f45fb65d3fd4cd5399a6
> >
> > Well, ok, I can see that libs/runtime handles this much cleaner,
> > but at the same time, having to add such a basic dependency,
> > explicitly, _and_ only for packaging (meaning, I'll look in all the
> > wrong places, first), _and_ in a compiler-specific way, just seems
> > really messy.
> >
> > I think:
> > a) Since the problem is relevant for packaging, only, it should be
> > handled by the Packager.
> > CollectionPackagerBase::internalCreatePackage() (or one of its
> > helpers), would seem a good place where the runtime could be
> > injected). This could in fact be done by pulling in libs/runtime.
> > b) I really don't understand why this should be handled in a
> > compiler-specific way, and I suspect the only reason for this is
> > historic. libs/runtime does appear to cover the VC-redistributable
> > files. However, NullsoftInstallerPackager still adds considerable
> > code in order to detect the correct version of vcredist.exe, pass
> > it to NSIS, and unpack it, from inside the installer.
> >
> > So - is there any particular reason to prefer the current way of
> > packaging the VC-runtime? Otherwise I'd look into providing patch to
> > include libs/runtime at packaging time, automatically, for both
> > compilers.
> >
> > Regards
> > Thomas  
>
>


attachment0 (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Runtime packaging

Hannah von Reth
I hope its working :P

https://cgit.kde.org/craft.git/commit/?id=8e1cfcf995b325e119904990e5dd13c78b93e605

https://cgit.kde.org/craft.git/commit/?id=da91c6cb5b1708c6aeb5545d06d753d63f30a045


On 23/05/2017 22:09, Thomas Friedrichsmeier wrote:

> Hi,
>
> On Tue, 23 May 2017 21:34:06 +0200
> Hannah von Reth <[hidden email]> wrote:
>> Well, yes it is a bit messy.
>>
>> This difference is that with msvc, when you have installed the
>> compiler the runtime is already installed.
>>
>> Providing the redist is also a bit cleaner than grapping dlls from
>> somewhere in the system.
>>
>> Additionally you need runtime on mingw to be able to run crafted
>> applications outside of the craft env.
>>
>> I guess one solution would be to drop the msvc code in runtime.py and
>> make virtual/base depend on runtime so that is always installed?
> Sounds reasonable (and easy!). Just to mention one downside:
> Installation of the vcredist.exe is still specific to the NSIS
> packager, so far. Any other present or future packager would still
> require custom code for including the VC runtime. Not sure how
> important that is to you. (Personally, I'm perfectly fine with one
> functional installer type, whichever it may be.)
>
> Regards
> Thomas
>
>>
>> Cheers,
>>
>> Hannah
>>
>>
>> On 23/05/2017 20:43, Thomas Friedrichsmeier wrote:
>>> On Sun, 21 May 2017 21:14:53 +0200
>>> Thomas Friedrichsmeier <[hidden email]>
>>> wrote:  
>>>> Hannah von Reth <[hidden email]> wrote:  
>>>>> Regarding the runtime, pls add a dependency to lib/runtime to
>>>>> rkward, it would be probably reasonable to only set the dep if the
>>>>> compiler is mingw, as we have a different solution with NSIS to
>>>>> provide the runtime with msvc.    
>>>> Ok, thanks for the hint about the lib/runtime dependency. Solves
>>>> the first issue, well. (Although I'll try to remember to look into
>>>> providing a patch for NullsoftInstallerPackager.py; seems odd that
>>>> I would have to specify extra dependencies just for packaging).  
>>> Hm. Now I found out that I already did, once:
>>> https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=e1e82b5d3a75d27635c474f76d38090be069939e
>>>
>>> But you removed it, some time later:
>>> https://cgit.kde.org/craft.git/commit/bin/Packager/NullsoftInstallerPackager.py?h=2017.05&id=8e0ee6b658912f34aa80f45fb65d3fd4cd5399a6
>>>
>>> Well, ok, I can see that libs/runtime handles this much cleaner,
>>> but at the same time, having to add such a basic dependency,
>>> explicitly, _and_ only for packaging (meaning, I'll look in all the
>>> wrong places, first), _and_ in a compiler-specific way, just seems
>>> really messy.
>>>
>>> I think:
>>> a) Since the problem is relevant for packaging, only, it should be
>>> handled by the Packager.
>>> CollectionPackagerBase::internalCreatePackage() (or one of its
>>> helpers), would seem a good place where the runtime could be
>>> injected). This could in fact be done by pulling in libs/runtime.
>>> b) I really don't understand why this should be handled in a
>>> compiler-specific way, and I suspect the only reason for this is
>>> historic. libs/runtime does appear to cover the VC-redistributable
>>> files. However, NullsoftInstallerPackager still adds considerable
>>> code in order to detect the correct version of vcredist.exe, pass
>>> it to NSIS, and unpack it, from inside the installer.
>>>
>>> So - is there any particular reason to prefer the current way of
>>> packaging the VC-runtime? Otherwise I'd look into providing patch to
>>> include libs/runtime at packaging time, automatically, for both
>>> compilers.
>>>
>>> Regards
>>> Thomas  
>>


signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Runtime packaging

Thomas Friedrichsmeier-2
Hi!

On Wed, 24 May 2017 00:00:26 +0200
Hannah von Reth <[hidden email]> wrote:

> I hope its working :P
>
> https://cgit.kde.org/craft.git/commit/?id=8e1cfcf995b325e119904990e5dd13c78b93e605
>
> https://cgit.kde.org/craft.git/commit/?id=da91c6cb5b1708c6aeb5545d06d753d63f30a045

Not quite, unfortunately. The reason appears to be that libs/runtime is
added as a _build_ dependency, and - worse - virtual/base is also only a
_build_ dependency of everything.

That _could_ be changed, without harm, I believe, because everything
else defined in virtual/base is - again - build dependencies. But that
would mean changing a whole lot of recipes.

The alternative way to do it is still adding libs/runtime during
packaging, only.

Regards
Thomas

attachment0 (836 bytes) Download Attachment