Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Boudewijn Rempt-2
On Sat, 6 May 2017, Ben Cooksley wrote:

> Which brings me to the third point of attention. We'll only be adding
> projects to the Next Gen CI system at their request going forward.

Krita should be part of this, too. We don't need anything special or
up to date, our frameworks requirements are very modest.

--
Boudewijn Rempt | http://www.krita.org, http://www.valdyas.org
Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Dmitry Kazakov
The only trouble for Krita will be Windows builds, but it is not a
regression, so we can proceed with the move.

For Windows we have the following problems:

1) Krita needs MinGW for compiling on Windows
2) We have a set of cmake scripts that builds all the deps on Windows.
Sometimes the scripts apply some patches. So if we would like to reuse
the builds done by CI system for "nightly" packages (and we would really
like to!), so these custom dependency builds should also be integrated
somehow. How to do that within the limits of the new dependencies framework?

Here is the list of the deps we build on Windows (and for Linux AppImage
packages):
https://cgit.kde.org/krita.git/tree/3rdparty

And here is the list of patches we apply on them:

./ext_exiv2/exiv2.patch
./ext_exiv2/tzname.patch
./ext_exiv2/patch_mingw.patch
./ext_iconv/iconv-src-1.13.1.patch
./ext_ocio/patch_mingw.patch
./ext_openexr/patch_mingw.patch
./ext_poppler/poppler_mingw.patch
./ext_vc/vc1.2_malloc_free.patch
./ext_qt/0001-Don-t-request-the-MIME-image-every-time-Windows-asks.patch
./ext_qt/0002-Hack-always-return-we-support-DIBV5.patch
./ext_qt/0003-Hack-for-fullscreen-workaround.patch
./ext_qt/mac-default.patch
./ext_qt/qtbase-configure.patch




On 16.05.2017 13:02, Boudewijn Rempt wrote:
> On Sat, 6 May 2017, Ben Cooksley wrote:
>
>> Which brings me to the third point of attention. We'll only be adding
>> projects to the Next Gen CI system at their request going forward.
> Krita should be part of this, too. We don't need anything special or
> up to date, our frameworks requirements are very modest.
>

--
Dmitry Kazakov

Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Ben Cooksley
In reply to this post by Boudewijn Rempt-2
On Tue, May 16, 2017 at 10:02 PM, Boudewijn Rempt <[hidden email]> wrote:
> On Sat, 6 May 2017, Ben Cooksley wrote:
>
>> Which brings me to the third point of attention. We'll only be adding
>> projects to the Next Gen CI system at their request going forward.
>
> Krita should be part of this, too. We don't need anything special or
> up to date, our frameworks requirements are very modest.

I've now enabled Krita for the Extragear product.

>
> --
> Boudewijn Rempt | http://www.krita.org, http://www.valdyas.org

Cheers,
Ben
Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Ben Cooksley
In reply to this post by Dmitry Kazakov
On Tue, May 16, 2017 at 10:14 PM, Dmitry Kazakov <[hidden email]> wrote:
> The only trouble for Krita will be Windows builds, but it is not a
> regression, so we can proceed with the move.

*Nod*. We haven't had Windows CI at all up until now.

>
> For Windows we have the following problems:
>
> 1) Krita needs MinGW for compiling on Windows

How hard a requirement is this? From what I understand it's related to
the G'Mic code correct?
Chances are we'll introduce Mingw at some point, but it's not on my
immediate plans (the design makes it not too hard, but i'd prefer to
push MSVC a good part of the way to workable first)

> 2) We have a set of cmake scripts that builds all the deps on Windows.
> Sometimes the scripts apply some patches. So if we would like to reuse the
> builds done by CI system for "nightly" packages (and we would really like
> to!), so these custom dependency builds should also be integrated somehow.
> How to do that within the limits of the new dependencies framework?
>
> Here is the list of the deps we build on Windows (and for Linux AppImage
> packages):
> https://cgit.kde.org/krita.git/tree/3rdparty
>
> And here is the list of patches we apply on them:
>
> ./ext_exiv2/exiv2.patch
> ./ext_exiv2/tzname.patch
> ./ext_exiv2/patch_mingw.patch
> ./ext_iconv/iconv-src-1.13.1.patch
> ./ext_ocio/patch_mingw.patch
> ./ext_openexr/patch_mingw.patch
> ./ext_poppler/poppler_mingw.patch
> ./ext_vc/vc1.2_malloc_free.patch
> ./ext_qt/0001-Don-t-request-the-MIME-image-every-time-Windows-asks.patch
> ./ext_qt/0002-Hack-always-return-we-support-DIBV5.patch
> ./ext_qt/0003-Hack-for-fullscreen-workaround.patch
> ./ext_qt/mac-default.patch
> ./ext_qt/qtbase-configure.patch
>
>

I guess the first question is: have you tried to upstream these? In
general we don't want to patch stuff at all on the CI as it creates
maintenance burden. (Patching of KDE software is completely out of the
question for obvious reasons)

At the moment we source the binaries for everything except Qt from
Craft. How many of these patches are applied by them?
For the other deps, if Craft has tooling available for those we can
probably make them available.

Currently i've not built any mechanisms for registering non-KDE
projects within the CI system framework (and was sort of hoping to
avoid that, but we'll see how we go)

Cheers,
Ben

>
>
>
> On 16.05.2017 13:02, Boudewijn Rempt wrote:
>>
>> On Sat, 6 May 2017, Ben Cooksley wrote:
>>
>>> Which brings me to the third point of attention. We'll only be adding
>>> projects to the Next Gen CI system at their request going forward.
>>
>> Krita should be part of this, too. We don't need anything special or
>> up to date, our frameworks requirements are very modest.
>>
>
> --
> Dmitry Kazakov
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Boudewijn Rempt-2
On Tue, 16 May 2017, Ben Cooksley wrote:

> On Tue, May 16, 2017 at 10:14 PM, Dmitry Kazakov <[hidden email]> wrote:
> > The only trouble for Krita will be Windows builds, but it is not a
> > regression, so we can proceed with the move.
>
> *Nod*. We haven't had Windows CI at all up until now.
>
> >
> > For Windows we have the following problems:
> >
> > 1) Krita needs MinGW for compiling on Windows
>
> How hard a requirement is this? From what I understand it's related to
> the G'Mic code correct?

And Vc -- that also doesn't build correctly with msvc.

> I guess the first question is: have you tried to upstream these? In
> general we don't want to patch stuff at all on the CI as it creates
> maintenance burden. (Patching of KDE software is completely out of the
> question for obvious reasons)

Part of these patches are also in craft -- adding cmake build systems
to libraries, for instance. The Qt patches aren't relevant for Qt > 5.7,
iirc. We also patch frameworks, actually, for things like localization
problems and the location of config files. But that's not important
for CI.

--
Boudewijn Rempt | http://www.krita.org, http://www.valdyas.org
Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Dmitry Kazakov
In reply to this post by Ben Cooksley

>> For Windows we have the following problems:
>>
>> 1) Krita needs MinGW for compiling on Windows
> How hard a requirement is this? From what I understand it's related to
> the G'Mic code correct?
> Chances are we'll introduce Mingw at some point, but it's not on my
> immediate plans (the design makes it not too hard, but i'd prefer to
> push MSVC a good part of the way to workable first)

Well, of course, if we disable GMic and Vc, we will still be able to
build Krita itself. But, then, it will be a question of the purpose of
CI. Just testing for compilation error and unittest failures is a nice
purpose. But we would also like to have it build the user-targeted
nightly/stable packages. And that will be impossible with without at
least Vc.

--
Dmitry Kazakov

Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Boudewijn Rempt-2
On Tue, 16 May 2017, Dmitry Kazakov wrote:

> Well, of course, if we disable GMic and Vc, we will still be able to build
> Krita itself. But, then, it will be a question of the purpose of CI. Just
> testing for compilation error and unittest failures is a nice purpose. But we
> would also like to have it build the user-targeted nightly/stable packages.
> And that will be impossible with without at least Vc.

Gmic should be gone soon, and maybe there's a new version of Vc that builds
with msvc? I haven't checked recently...


--
Boudewijn Rempt | http://www.krita.org, http://www.valdyas.org
Reply | Threaded
Open this post in threaded view
|

Re: Next Gen CI Will Be Moving to Production Shortly: Upcoming Changes

Ben Cooksley
In reply to this post by Dmitry Kazakov
On Tue, May 16, 2017 at 11:29 PM, Dmitry Kazakov <[hidden email]> wrote:

>
>>> For Windows we have the following problems:
>>>
>>> 1) Krita needs MinGW for compiling on Windows
>>
>> How hard a requirement is this? From what I understand it's related to
>> the G'Mic code correct?
>> Chances are we'll introduce Mingw at some point, but it's not on my
>> immediate plans (the design makes it not too hard, but i'd prefer to
>> push MSVC a good part of the way to workable first)
>
>
> Well, of course, if we disable GMic and Vc, we will still be able to build
> Krita itself. But, then, it will be a question of the purpose of CI. Just
> testing for compilation error and unittest failures is a nice purpose. But
> we would also like to have it build the user-targeted nightly/stable
> packages. And that will be impossible with without at least Vc.

I would suggest we have a different setup for producing nightlies &
final binaries as the CI builds in full debug mode, which means
asserts are enabled and no optimization is done by the compiler.

This means that performance probably will suck to an extent especially
in performance sensitive code, and we probably want to give a more
optimal user experience, even with Nightly binaries...

With G'Mic going away, shall we give it a try and see what happens
with Vc with the latest MSVC?

>
> --
> Dmitry Kazakov
>

Cheers,
Ben