Considering to switch away from KDEPIM/Akonadi

classic Classic list List threaded Threaded
44 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Considering to switch away from KDEPIM/Akonadi

Martin Steigerwald
Hi.

I did not write in a long time about own issues with KDEPIM/Akonadi and
instead tried to help others along here and there, but now I feel quite
frustrated…

also due to own resistance to actually report a bug, I deal with a
regular crash of KMail¹ – regular as in several times a day.

However, that is not the main issue.

The main issue are two performance related issues (despite MariaDB
tuning and all that):

1) During mail retrieval KMail sometimes just stops displaying new mails
coming in. It GUI is responsive, but when I click on another folder or
so… I just get the waiting message. These stalls, that do not happen all
of the time, take up to a minute or even longer.

2) Filtering of new mail into folders takes very long. Very long means
up to 5 minutes or move of sustained activity of both Akonadi and
MariaDB for filtering 1000-2000 new mails. That is with me even making
sure to remove old mails from Akonadi's maildir entirely. Most folders
do not have more than 30000-40000 mails, some considerable less. A few
have 80000-100000 mails and I have the idea to remove mails out of them
as well.

3) Then I had one of the LRCONFLICTs displayed in KMail's status line
without even able to expand it to see what is is about. I solved it by
clearing Akonadi cache of incoming folder.

At the same time due to introduction of Office 365 at work² I partly use
Evolution with EWS and I found the following:

1) Evolution did not crash *once* so far.

2) Evolution just works. For me so far, that is.

3) Evolution does not get stuck.

3) So far I do not see much of a performance issue. There are some short
stalls at time, but nothing even almost as unbearable as with KMail and
Akonadi.

Granted the amount of mails in my work-around is considerably lower. And
while KMail does not crash several times a day with my work setup, it
still crashed once in a while. Also Akonadi gets stuck with IMAP or EWS
connection or whatever else.


Now, I know how much I defended KDEPIM / Akonadi, but… I also see the
amount of effort you, Daniel, and others put into fixing things up and
even changing Akonadi in some fundamental ways in order to make it work…
I truly appreciate that work…

but I am also tired.

Simply put: I am fed up with it.

Yes, I see the recent improvements in Akonadiconsole to help others
report meaningful bugs and all that…

but I do not like to be a database admin or a server administrator just
to read my mail and respond to them or write a bug report.

KMail is still *just* a mail program.

Akonadi does not have issues since yesterday. I'd say it had issues till
the beginning. I might have been lucky at times where it mostly worked
for me, but also I moved through a ton of issues and pain with KDEPIM
and Akonadi.

I am sorry to write but I have found myself seriously considering to
switch away from KDEPIM, as much as I like KMail.

The only other idea I had is to just switch from MariaDB to PostgreSQL.
It would mean a PostgreSQL upgrade once in a while, but it if works
better, I'd probably do that. Especially after I somewhere skimmed that
you, Daniel, wrote about a kind of locking issue with MariaDB/MySQL. And
also since you, Daniel, as the only one I am currently aware of tackling
deep into Akonadi, AFAIK use PostgreSQL as well.

I am reluctant as akonadictl fsck reported a lot of issues the last time
I used it, including about items without RID. I may like to export those
into files before attempting a switch as Akonadi still cannot reply
those. And still I also do not get how a maildir resource could ever
fail to write out the mail into the maildir as long as the filesystem is
healthy.  I am also reluctant due to having to reconfigure about 100 or
more filter rules.

Or I might give it another change till the rewrite of the search
infrastructure as one of the last steps to improve Akonadi's
architecture.

Or at least wait till 18.12 or 19.04 packages are available and test
again.


But as much as it saddens me I am quite close to just being done with
it… and take the huge step to migrate away from KDEPIM / Akonadi.

How long has Akonadi been in development? Way more than 5 year, even a
decade probably? It ought to be stable.

Yet, just by monitoring KDEPIM bugs mailing list I know it is far from
being stable.

Yes, I know I have argued in favor for attempting to fix up Akonadi… and
maybe its really close now with only one of the major steps in the major
architectural overhaul missing – new search infrastructure.

And yes, I'd like to be convinced to hold out a little bit longer, but
on the other hand I did so again, and again, and again…

and it still does not work as stable and robust as I'd like it to. I
still put up with issues that a software that is being developed for
more than 5 years – correct me if my memory is wrong – simply should not
have anymore.

So maybe, just maybe it is important to start admitting that Akonadi is
still not in the state it is supposed to be… and consider all options…


I end it now, its already long enough. I am preparing for a recreational
time out. I will most likely be offline or mostly offline from Friday to
Monday.

I bet it might spur quite a discussion or so… all I ask of you is to
stay friendly in tone. I truly see that KDEPIM developers are working
hard, just AFAIK there are only a few who dare to dig into Akonadi. Dan
and well David recently.

I hope that this "rant" while honest is still friendly in tone. I still
remember walking around angrily in this room, venting "such a piece of
crap software" at quite some times. In hind sight I do not mean it that
harshly. I see the good ideas behind Akonadi as well, but the 18.08
implementation of it, feels like a major burden to me. 17.12 has been
better, as it did not have the stalls during receiving mails and KMail
did not crash several times a day. But it still used a lot of resources
and mail filtering was still slow. However it was a lot more bearable
than 18.08 appears to me.


[1] Bug 404052 - Crash during/after filtering inbox, probably related to
Qt WebEngine integration
https://bugs.kde.org/404052

[2]  I do not agree with that move, but that is how it is currently…

Thanks,
--
Martin


Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Martin Steigerwald
Martin Steigerwald - 04.04.19, 13:46:
> But as much as it saddens me I am quite close to just being done with
> it… and take the huge step to migrate away from KDEPIM / Akonadi.

I have been again considering notmuch¹

While I am not aware of a KDE app for Notmuch, I really like the
philosophy:

> "Not much mail" is what Notmuch thinks about your email collection.
> Even if you receive 12000 messages per month or have on the order of
> millions of messages that you've been saving for decades. Regardless,
> Notmuch will be able to quickly search all of it. It's just plain not
> much mail.

[1] https://notmuchmail.org/

Thanks,
--
Martin
Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Vojtěch Zeisek-2
In reply to this post by Martin Steigerwald
Hi

Dne 4.4.2019 13:46, Martin Steigerwald napsal:
> 1) During mail retrieval KMail sometimes just stops displaying new
> mails
> coming in. It GUI is responsive, but when I click on another folder or
> so… I just get the waiting message. These stalls, that do not happen
> all
> of the time, take up to a minute or even longer.

I watch this since this morning. It made KMail unusable right now, so
using not-so-good web interface right now... I didn't do any change in
configuration or upgrade. I'm using KDE PIM 18.12.

> 2) Filtering of new mail into folders takes very long. Very long means
> up to 5 minutes or move of sustained activity of both Akonadi and
> MariaDB for filtering 1000-2000 new mails. That is with me even making
> sure to remove old mails from Akonadi's maildir entirely. Most folders
> do not have more than 30000-40000 mails, some considerable less. A few
> have 80000-100000 mails and I have the idea to remove mails out of them
> as well.

I don't see such problems. I use most of filters on the IMAP server side
and when I search within even large folder (well, for me up to ~10-20
000 mails), it is basically instant.

> 3) Then I had one of the LRCONFLICTs displayed in KMail's status line
> without even able to expand it to see what is is about. I solved it by
> clearing Akonadi cache of incoming folder.

Few of this issues I also saw recently. Sometimes it happened that I
manually moved mail to some folder, it was moved (well, rather copied),
but it stayed in the original folder. Bit annoying.
Is there way how to clear Akonadi cache for all
folders/accounts/resources in once?

> At the same time due to introduction of Office 365 at work² I partly
> use
> Evolution with EWS and I found the following:

I use Akonadi-ews and it works perfectly. Well, if You believe Exchange
could work at all. ;-)

> 1) Evolution did not crash *once* so far.

I don't remember when my KMail crashed last time.

> 2) Evolution just works. For me so far, that is.

I like Evolution, but KMail has so much more features...

> 3) Evolution does not get stuck.

KMail does sometimes... :-(

> I am reluctant as akonadictl fsck reported a lot of issues the last
> time
> I used it, including about items without RID. I may like to export
> those
> into files before attempting a switch as Akonadi still cannot reply
> those. And still I also do not get how a maildir resource could ever
> fail to write out the mail into the maildir as long as the filesystem
> is
> healthy.  I am also reluctant due to having to reconfigure about 100 or
> more filter rules.

I has been running akonadictl fsck for several hours already (! it has
never happened before) and now it reports tones of messages like
Cleaning up missing external file:
/home/vojta/.local/share/akonadi/file_db_data/65/617965_r0 for item:
220946 on part: 617965
I wonder how it will look on my other computer (both running openSUSE
Tumbleweed).

I don't have much annoying issues with KMail, for me it's still the best
e-mail client. I don't wish to leave it. :-)
So big thanks to all developers.

--
Vojtěch Zeisek

Komunita openSUSE GNU/Linuxu
Community of the openSUSE GNU/Linux

https://www.opensuse.org/
https://trapa.cz/
Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Martin Steigerwald
Vojtěch Zeisek - 04.04.19, 14:10:
> I don't have much annoying issues with KMail, for me it's still the
> best e-mail client. I don't wish to leave it.
> So big thanks to all developers.

Thank you for your feedback.

Great to hear it is working much better for you.

Just to make it clear:

I am grateful for KDEPIM. KMail is a marvelous mail program, Korganizer
and KAddressBook are also really nice.

So yes, big thanks to all developers. No matter what my recent
experiences where.

Thanks,
--
Martin


Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Colin Close
In reply to this post by Vojtěch Zeisek-2
I have found that postgresql provides a far more reliable kmail/akonadi installation. The most telling part is how quickly the ‘akonadi fsck’ and ‘akonadi vacuum’ complete compared to MySQL. Maybe this is a clue to some of the MySQL issues.
Colin Close
OpenMandriva

Sent from my iPhone

> On 4 Apr 2019, at 13:10, Vojtěch Zeisek <[hidden email]> wrote:
>
> Hi
>
> Dne 4.4.2019 13:46, Martin Steigerwald napsal:
>> 1) During mail retrieval KMail sometimes just stops displaying new mails
>> coming in. It GUI is responsive, but when I click on another folder or
>> so… I just get the waiting message. These stalls, that do not happen all
>> of the time, take up to a minute or even longer.
>
> I watch this since this morning. It made KMail unusable right now, so using not-so-good web interface right now... I didn't do any change in configuration or upgrade. I'm using KDE PIM 18.12.
>
>> 2) Filtering of new mail into folders takes very long. Very long means
>> up to 5 minutes or move of sustained activity of both Akonadi and
>> MariaDB for filtering 1000-2000 new mails. That is with me even making
>> sure to remove old mails from Akonadi's maildir entirely. Most folders
>> do not have more than 30000-40000 mails, some considerable less. A few
>> have 80000-100000 mails and I have the idea to remove mails out of them
>> as well.
>
> I don't see such problems. I use most of filters on the IMAP server side and when I search within even large folder (well, for me up to ~10-20 000 mails), it is basically instant.
>
>> 3) Then I had one of the LRCONFLICTs displayed in KMail's status line
>> without even able to expand it to see what is is about. I solved it by
>> clearing Akonadi cache of incoming folder.
>
> Few of this issues I also saw recently. Sometimes it happened that I manually moved mail to some folder, it was moved (well, rather copied), but it stayed in the original folder. Bit annoying.
> Is there way how to clear Akonadi cache for all folders/accounts/resources in once?
>
>> At the same time due to introduction of Office 365 at work² I partly use
>> Evolution with EWS and I found the following:
>
> I use Akonadi-ews and it works perfectly. Well, if You believe Exchange could work at all. ;-)
>
>> 1) Evolution did not crash *once* so far.
>
> I don't remember when my KMail crashed last time.
>
>> 2) Evolution just works. For me so far, that is.
>
> I like Evolution, but KMail has so much more features...
>
>> 3) Evolution does not get stuck.
>
> KMail does sometimes... :-(
>
>> I am reluctant as akonadictl fsck reported a lot of issues the last time
>> I used it, including about items without RID. I may like to export those
>> into files before attempting a switch as Akonadi still cannot reply
>> those. And still I also do not get how a maildir resource could ever
>> fail to write out the mail into the maildir as long as the filesystem is
>> healthy.  I am also reluctant due to having to reconfigure about 100 or
>> more filter rules.
>
> I has been running akonadictl fsck for several hours already (! it has never happened before) and now it reports tones of messages like
> Cleaning up missing external file: /home/vojta/.local/share/akonadi/file_db_data/65/617965_r0 for item: 220946 on part: 617965
> I wonder how it will look on my other computer (both running openSUSE Tumbleweed).
>
> I don't have much annoying issues with KMail, for me it's still the best e-mail client. I don't wish to leave it. :-)
> So big thanks to all developers.
>
> --
> Vojtěch Zeisek
>
> Komunita openSUSE GNU/Linuxu
> Community of the openSUSE GNU/Linux
>
> https://www.opensuse.org/
> https://trapa.cz/

Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Anders Lund
In reply to this post by Martin Steigerwald
To read your mail, I had to

1. kill kmail (leading to two kbugthingy dialogs that requires 3 clicks each
to get rid of, and to korganizer ALSO crashing, leading to two more dialogs)
2. restart akonadi
3. wait
4. start kmail again
5. wait while it did its lengthy mail checking.

In my experience, Kmail is often in a defunct state, where new mail is not
fetched, and as such not a reliable mail client. Apart from that, my
experience is about in line with yours. Long waits, unreliability, lots of
BSOD time.

So YES, it is quite frustrating. Alternatives would be wellcome, as kmail is a
hazzard.

Kindly,
Anders

torsdag den 4. april 2019 13.46.42 CEST skrev Martin Steigerwald:

> Hi.
>
> I did not write in a long time about own issues with KDEPIM/Akonadi and
> instead tried to help others along here and there, but now I feel quite
> frustrated…
>
> also due to own resistance to actually report a bug, I deal with a
> regular crash of KMail¹ – regular as in several times a day.
>
> However, that is not the main issue.
>
> The main issue are two performance related issues (despite MariaDB
> tuning and all that):
>
> 1) During mail retrieval KMail sometimes just stops displaying new mails
> coming in. It GUI is responsive, but when I click on another folder or
> so… I just get the waiting message. These stalls, that do not happen all
> of the time, take up to a minute or even longer.
>
> 2) Filtering of new mail into folders takes very long. Very long means
> up to 5 minutes or move of sustained activity of both Akonadi and
> MariaDB for filtering 1000-2000 new mails. That is with me even making
> sure to remove old mails from Akonadi's maildir entirely. Most folders
> do not have more than 30000-40000 mails, some considerable less. A few
> have 80000-100000 mails and I have the idea to remove mails out of them
> as well.
>
> 3) Then I had one of the LRCONFLICTs displayed in KMail's status line
> without even able to expand it to see what is is about. I solved it by
> clearing Akonadi cache of incoming folder.
>
> At the same time due to introduction of Office 365 at work² I partly use
> Evolution with EWS and I found the following:
>
> 1) Evolution did not crash *once* so far.
>
> 2) Evolution just works. For me so far, that is.
>
> 3) Evolution does not get stuck.
>
> 3) So far I do not see much of a performance issue. There are some short
> stalls at time, but nothing even almost as unbearable as with KMail and
> Akonadi.
>
> Granted the amount of mails in my work-around is considerably lower. And
> while KMail does not crash several times a day with my work setup, it
> still crashed once in a while. Also Akonadi gets stuck with IMAP or EWS
> connection or whatever else.
>
>
> Now, I know how much I defended KDEPIM / Akonadi, but… I also see the
> amount of effort you, Daniel, and others put into fixing things up and
> even changing Akonadi in some fundamental ways in order to make it work…
> I truly appreciate that work…
>
> but I am also tired.
>
> Simply put: I am fed up with it.
>
> Yes, I see the recent improvements in Akonadiconsole to help others
> report meaningful bugs and all that…
>
> but I do not like to be a database admin or a server administrator just
> to read my mail and respond to them or write a bug report.
>
> KMail is still *just* a mail program.
>
> Akonadi does not have issues since yesterday. I'd say it had issues till
> the beginning. I might have been lucky at times where it mostly worked
> for me, but also I moved through a ton of issues and pain with KDEPIM
> and Akonadi.
>
> I am sorry to write but I have found myself seriously considering to
> switch away from KDEPIM, as much as I like KMail.
>
> The only other idea I had is to just switch from MariaDB to PostgreSQL.
> It would mean a PostgreSQL upgrade once in a while, but it if works
> better, I'd probably do that. Especially after I somewhere skimmed that
> you, Daniel, wrote about a kind of locking issue with MariaDB/MySQL. And
> also since you, Daniel, as the only one I am currently aware of tackling
> deep into Akonadi, AFAIK use PostgreSQL as well.
>
> I am reluctant as akonadictl fsck reported a lot of issues the last time
> I used it, including about items without RID. I may like to export those
> into files before attempting a switch as Akonadi still cannot reply
> those. And still I also do not get how a maildir resource could ever
> fail to write out the mail into the maildir as long as the filesystem is
> healthy.  I am also reluctant due to having to reconfigure about 100 or
> more filter rules.
>
> Or I might give it another change till the rewrite of the search
> infrastructure as one of the last steps to improve Akonadi's
> architecture.
>
> Or at least wait till 18.12 or 19.04 packages are available and test
> again.
>
>
> But as much as it saddens me I am quite close to just being done with
> it… and take the huge step to migrate away from KDEPIM / Akonadi.
>
> How long has Akonadi been in development? Way more than 5 year, even a
> decade probably? It ought to be stable.
>
> Yet, just by monitoring KDEPIM bugs mailing list I know it is far from
> being stable.
>
> Yes, I know I have argued in favor for attempting to fix up Akonadi… and
> maybe its really close now with only one of the major steps in the major
> architectural overhaul missing – new search infrastructure.
>
> And yes, I'd like to be convinced to hold out a little bit longer, but
> on the other hand I did so again, and again, and again…
>
> and it still does not work as stable and robust as I'd like it to. I
> still put up with issues that a software that is being developed for
> more than 5 years – correct me if my memory is wrong – simply should not
> have anymore.
>
> So maybe, just maybe it is important to start admitting that Akonadi is
> still not in the state it is supposed to be… and consider all options…
>
>
> I end it now, its already long enough. I am preparing for a recreational
> time out. I will most likely be offline or mostly offline from Friday to
> Monday.
>
> I bet it might spur quite a discussion or so… all I ask of you is to
> stay friendly in tone. I truly see that KDEPIM developers are working
> hard, just AFAIK there are only a few who dare to dig into Akonadi. Dan
> and well David recently.
>
> I hope that this "rant" while honest is still friendly in tone. I still
> remember walking around angrily in this room, venting "such a piece of
> crap software" at quite some times. In hind sight I do not mean it that
> harshly. I see the good ideas behind Akonadi as well, but the 18.08
> implementation of it, feels like a major burden to me. 17.12 has been
> better, as it did not have the stalls during receiving mails and KMail
> did not crash several times a day. But it still used a lot of resources
> and mail filtering was still slow. However it was a lot more bearable
> than 18.08 appears to me.
>
>
> [1] Bug 404052 - Crash during/after filtering inbox, probably related to
> Qt WebEngine integration
> https://bugs.kde.org/404052
>
> [2]  I do not agree with that move, but that is how it is currently…
>
> Thanks,




Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

René J.V. Bertin
On Thursday April 04 2019 14:32:19 Anders Lund wrote:

> 1.... two kbugthingy dialogs that requires 3 clicks each
> to get rid of

FWIW, DrKonqi closes with a single Escape key stroke if you don't tell it to obtain a backtrace first.
Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Anders Lund
torsdag den 4. april 2019 15.12.35 CEST skrev René J.V. Bertin:
> On Thursday April 04 2019 14:32:19 Anders Lund wrote:
> > 1.... two kbugthingy dialogs that requires 3 clicks each
> > to get rid of
>
> FWIW, DrKonqi closes with a single Escape key stroke if you don't tell it to
> obtain a backtrace first.

No. I press escape, and have them in the system tray.



Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

René J.V. Bertin
On Thursday April 04 2019 15:35:40 Anders Lund wrote:
>torsdag den 4. april 2019 15.12.35 CEST skrev René J.V. Bertin:
>> On Thursday April 04 2019 14:32:19 Anders Lund wrote:
>> > 1.... two kbugthingy dialogs that requires 3 clicks each
>> > to get rid of
>>
>> FWIW, DrKonqi closes with a single Escape key stroke if you don't tell it to
>> obtain a backtrace first.
>
>No. I press escape, and have them in the system tray.

That happens to me too, and means 1 click to get them out of there (if you can avoid them from getting into the hidden systray bits) and then 1 Escape stroke.
There should be a way to prevent DrKonqi from hiding in the systray, possibly even a global setting to disable it alltogether (for specific apps).

R.
Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Anders Lund
1. Dialog appears. Click or press ESC, dialog moves to systray
2. Click systray icon to show dialog
3. Click close.

3 clicks.


torsdag den 4. april 2019 16.04.42 CEST skrev René J.V. Bertin:

> On Thursday April 04 2019 15:35:40 Anders Lund wrote:
> >torsdag den 4. april 2019 15.12.35 CEST skrev René J.V. Bertin:
> >> On Thursday April 04 2019 14:32:19 Anders Lund wrote:
> >> > 1.... two kbugthingy dialogs that requires 3 clicks each
> >> > to get rid of
> >>
> >> FWIW, DrKonqi closes with a single Escape key stroke if you don't tell it
> >> to obtain a backtrace first.
> >
> >No. I press escape, and have them in the system tray.
>
> That happens to me too, and means 1 click to get them out of there (if you
> can avoid them from getting into the hidden systray bits) and then 1 Escape
> stroke. There should be a way to prevent DrKonqi from hiding in the
> systray, possibly even a global setting to disable it alltogether (for
> specific apps).
>
> R.





Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

René J.V. Bertin
On Thursday April 04 2019 17:48:56 Anders Lund wrote:
>1. Dialog appears. Click or press ESC, dialog moves to systray
>2. Click systray icon to show dialog
>3. Click close.

I just tried

%> kate &
%> killall -SEGV kate

My Plasma4 desktop shows an alert with a "Report bug" button that does nothing. 1 click on the beetle icon in the systray, and then hit Escape and DrKonqi exits. On a non-Plasma desktop DrKonqi will appear, and still exits when I hit Escape. So that's at most 1 click.

It's possible that you have a later version which reacts differently to Escape (mine is built from git master of Dec. 6th 2018). Easy to check by hitting Escape in your step 3. If that also closes it's quicker & easier than clicking 'Close' (less eye-hand co-ordination required) ;)


Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Con Hennessy-2
In reply to this post by Martin Steigerwald

Hi Martin,

 

I'm also a longtime (and content) user of kmail.

 

I've occasionally experienced what you describe (or something very similar) and what I've ended up doing is to effectively reinitialise akonadi. This is relatively easy to do by stopping akonadi, moving ~/local/share/akonai, and restarting akonadi.

 

It may take some time to re-download various emails by akonadi but it has proven successful. I've only had to do this twice in the last 12 months.

 

I hope this might help you and others.

 

Regards,

Con

 

On Thursday 4 April 2019 and Martin Steigerwald wrote:

> Hi.

>

> I did not write in a long time about own issues with KDEPIM/Akonadi and

> instead tried to help others along here and there, but now I feel quite

> frustrated…

>

> also due to own resistance to actually report a bug, I deal with a

> regular crash of KMail¹ – regular as in several times a day.

>

> However, that is not the main issue.

>

> The main issue are two performance related issues (despite MariaDB

> tuning and all that):

>

> 1) During mail retrieval KMail sometimes just stops displaying new mails

> coming in. It GUI is responsive, but when I click on another folder or

> so… I just get the waiting message. These stalls, that do not happen all

> of the time, take up to a minute or even longer.

>

> 2) Filtering of new mail into folders takes very long. Very long means

> up to 5 minutes or move of sustained activity of both Akonadi and

> MariaDB for filtering 1000-2000 new mails. That is with me even making

> sure to remove old mails from Akonadi's maildir entirely. Most folders

> do not have more than 30000-40000 mails, some considerable less. A few

> have 80000-100000 mails and I have the idea to remove mails out of them

> as well.

>

> 3) Then I had one of the LRCONFLICTs displayed in KMail's status line

> without even able to expand it to see what is is about. I solved it by

> clearing Akonadi cache of incoming folder.

>

> At the same time due to introduction of Office 365 at work² I partly use

> Evolution with EWS and I found the following:

>

> 1) Evolution did not crash *once* so far.

>

> 2) Evolution just works. For me so far, that is.

>

> 3) Evolution does not get stuck.

>

> 3) So far I do not see much of a performance issue. There are some short

> stalls at time, but nothing even almost as unbearable as with KMail and

> Akonadi.

>

> Granted the amount of mails in my work-around is considerably lower. And

> while KMail does not crash several times a day with my work setup, it

> still crashed once in a while. Also Akonadi gets stuck with IMAP or EWS

> connection or whatever else.

>

>

> Now, I know how much I defended KDEPIM / Akonadi, but… I also see the

> amount of effort you, Daniel, and others put into fixing things up and

> even changing Akonadi in some fundamental ways in order to make it work…

> I truly appreciate that work…

>

> but I am also tired.

>

> Simply put: I am fed up with it.

>

> Yes, I see the recent improvements in Akonadiconsole to help others

> report meaningful bugs and all that…

>

> but I do not like to be a database admin or a server administrator just

> to read my mail and respond to them or write a bug report.

>

> KMail is still *just* a mail program.

>

> Akonadi does not have issues since yesterday. I'd say it had issues till

> the beginning. I might have been lucky at times where it mostly worked

> for me, but also I moved through a ton of issues and pain with KDEPIM

> and Akonadi.

>

> I am sorry to write but I have found myself seriously considering to

> switch away from KDEPIM, as much as I like KMail.

>

> The only other idea I had is to just switch from MariaDB to PostgreSQL.

> It would mean a PostgreSQL upgrade once in a while, but it if works

> better, I'd probably do that. Especially after I somewhere skimmed that

> you, Daniel, wrote about a kind of locking issue with MariaDB/MySQL. And

> also since you, Daniel, as the only one I am currently aware of tackling

> deep into Akonadi, AFAIK use PostgreSQL as well.

>

> I am reluctant as akonadictl fsck reported a lot of issues the last time

> I used it, including about items without RID. I may like to export those

> into files before attempting a switch as Akonadi still cannot reply

> those. And still I also do not get how a maildir resource could ever

> fail to write out the mail into the maildir as long as the filesystem is

> healthy. I am also reluctant due to having to reconfigure about 100 or

> more filter rules.

>

> Or I might give it another change till the rewrite of the search

> infrastructure as one of the last steps to improve Akonadi's

> architecture.

>

> Or at least wait till 18.12 or 19.04 packages are available and test

> again.

>

>

> But as much as it saddens me I am quite close to just being done with

> it… and take the huge step to migrate away from KDEPIM / Akonadi.

>

> How long has Akonadi been in development? Way more than 5 year, even a

> decade probably? It ought to be stable.

>

> Yet, just by monitoring KDEPIM bugs mailing list I know it is far from

> being stable.

>

> Yes, I know I have argued in favor for attempting to fix up Akonadi… and

> maybe its really close now with only one of the major steps in the major

> architectural overhaul missing – new search infrastructure.

>

> And yes, I'd like to be convinced to hold out a little bit longer, but

> on the other hand I did so again, and again, and again…

>

> and it still does not work as stable and robust as I'd like it to. I

> still put up with issues that a software that is being developed for

> more than 5 years – correct me if my memory is wrong – simply should not

> have anymore.

>

> So maybe, just maybe it is important to start admitting that Akonadi is

> still not in the state it is supposed to be… and consider all options…

>

>

> I end it now, its already long enough. I am preparing for a recreational

> time out. I will most likely be offline or mostly offline from Friday to

> Monday.

>

> I bet it might spur quite a discussion or so… all I ask of you is to

> stay friendly in tone. I truly see that KDEPIM developers are working

> hard, just AFAIK there are only a few who dare to dig into Akonadi. Dan

> and well David recently.

>

> I hope that this "rant" while honest is still friendly in tone. I still

> remember walking around angrily in this room, venting "such a piece of

> crap software" at quite some times. In hind sight I do not mean it that

> harshly. I see the good ideas behind Akonadi as well, but the 18.08

> implementation of it, feels like a major burden to me. 17.12 has been

> better, as it did not have the stalls during receiving mails and KMail

> did not crash several times a day. But it still used a lot of resources

> and mail filtering was still slow. However it was a lot more bearable

> than 18.08 appears to me.

>

>

> [1] Bug 404052 - Crash during/after filtering inbox, probably related to

> Qt WebEngine integration

> https://bugs.kde.org/404052

>

> [2] I do not agree with that move, but that is how it is currently…

>

> Thanks,

Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Paul Vixie
i do this every two weeks, because forced.

i fsck and vacuum and restart twice a day, because forced.

i use post gres, because mariadb was even worse.

don't like akonadi.

On 5 Apr 2019, at 00:43, [hidden email] wrote:

Hi Martin,

 

I'm also a longtime (and content) user of kmail.

 

I've occasionally experienced what you describe (or something very similar) and what I've ended up doing is to effectively reinitialise akonadi. This is relatively easy to do by stopping akonadi, moving ~/local/share/akonai, and restarting akonadi.

 

It may take some time to re-download various emails by akonadi but it has proven successful. I've only had to do this twice in the last 12 months.

 

I hope this might help you and others.

 

Regards,

Con

 

On Thursday 4 April 2019 and Martin Steigerwald wrote:

> Hi.

>

> I did not write in a long time about own issues with KDEPIM/Akonadi and

> instead tried to help others along here and there, but now I feel quite

> frustrated…

>

> also due to own resistance to actually report a bug, I deal with a

> regular crash of KMail¹ – regular as in several times a day.

>

> However, that is not the main issue.

>

> The main issue are two performance related issues (despite MariaDB

> tuning and all that):

>

> 1) During mail retrieval KMail sometimes just stops displaying new mails

> coming in. It GUI is responsive, but when I click on another folder or

> so… I just get the waiting message. These stalls, that do not happen all

> of the time, take up to a minute or even longer.

>

> 2) Filtering of new mail into folders takes very long. Very long means

> up to 5 minutes or move of sustained activity of both Akonadi and

> MariaDB for filtering 1000-2000 new mails. That is with me even making

> sure to remove old mails from Akonadi's maildir entirely. Most folders

> do not have more than 30000-40000 mails, some considerable less. A few

> have 80000-100000 mails and I have the idea to remove mails out of them

> as well.

>

> 3) Then I had one of the LRCONFLICTs displayed in KMail's status line

> without even able to expand it to see what is is about. I solved it by

> clearing Akonadi cache of incoming folder.

>

> At the same time due to introduction of Office 365 at work² I partly use

> Evolution with EWS and I found the following:

>

> 1) Evolution did not crash *once* so far.

>

> 2) Evolution just works. For me so far, that is.

>

> 3) Evolution does not get stuck.

>

> 3) So far I do not see much of a performance issue. There are some short

> stalls at time, but nothing even almost as unbearable as with KMail and

> Akonadi.

>

> Granted the amount of mails in my work-around is considerably lower. And

> while KMail does not crash several times a day with my work setup, it

> still crashed once in a while. Also Akonadi gets stuck with IMAP or EWS

> connection or whatever else.

>

>

> Now, I know how much I defended KDEPIM / Akonadi, but… I also see the

> amount of effort you, Daniel, and others put into fixing things up and

> even changing Akonadi in some fundamental ways in order to make it work…

> I truly appreciate that work…

>

> but I am also tired.

>

> Simply put: I am fed up with it.

>

> Yes, I see the recent improvements in Akonadiconsole to help others

> report meaningful bugs and all that…

>

> but I do not like to be a database admin or a server administrator just

> to read my mail and respond to them or write a bug report.

>

> KMail is still *just* a mail program.

>

> Akonadi does not have issues since yesterday. I'd say it had issues till

> the beginning. I might have been lucky at times where it mostly worked

> for me, but also I moved through a ton of issues and pain with KDEPIM

> and Akonadi.

>

> I am sorry to write but I have found myself seriously considering to

> switch away from KDEPIM, as much as I like KMail.

>

> The only other idea I had is to just switch from MariaDB to PostgreSQL.

> It would mean a PostgreSQL upgrade once in a while, but it if works

> better, I'd probably do that. Especially after I somewhere skimmed that

> you, Daniel, wrote about a kind of locking issue with MariaDB/MySQL. And

> also since you, Daniel, as the only one I am currently aware of tackling

> deep into Akonadi, AFAIK use PostgreSQL as well.

>

> I am reluctant as akonadictl fsck reported a lot of issues the last time

> I used it, including about items without RID. I may like to export those

> into files before attempting a switch as Akonadi still cannot reply

> those. And still I also do not get how a maildir resource could ever

> fail to write out the mail into the maildir as long as the filesystem is

> healthy. I am also reluctant due to having to reconfigure about 100 or

> more filter rules.

>

> Or I might give it another change till the rewrite of the search

> infrastructure as one of the last steps to improve Akonadi's

> architecture.

>

> Or at least wait till 18.12 or 19.04 packages are available and test

> again.

>

>

> But as much as it saddens me I am quite close to just being done with

> it… and take the huge step to migrate away from KDEPIM / Akonadi.

>

> How long has Akonadi been in development? Way more than 5 year, even a

> decade probably? It ought to be stable.

>

> Yet, just by monitoring KDEPIM bugs mailing list I know it is far from

> being stable.

>

> Yes, I know I have argued in favor for attempting to fix up Akonadi… and

> maybe its really close now with only one of the major steps in the major

> architectural overhaul missing – new search infrastructure.

>

> And yes, I'd like to be convinced to hold out a little bit longer, but

> on the other hand I did so again, and again, and again…

>

> and it still does not work as stable and robust as I'd like it to. I

> still put up with issues that a software that is being developed for

> more than 5 years – correct me if my memory is wrong – simply should not

> have anymore.

>

> So maybe, just maybe it is important to start admitting that Akonadi is

> still not in the state it is supposed to be… and consider all options…

>

>

> I end it now, its already long enough. I am preparing for a recreational

> time out. I will most likely be offline or mostly offline from Friday to

> Monday.

>

> I bet it might spur quite a discussion or so… all I ask of you is to

> stay friendly in tone. I truly see that KDEPIM developers are working

> hard, just AFAIK there are only a few who dare to dig into Akonadi. Dan

> and well David recently.

>

> I hope that this "rant" while honest is still friendly in tone. I still

> remember walking around angrily in this room, venting "such a piece of

> crap software" at quite some times. In hind sight I do not mean it that

> harshly. I see the good ideas behind Akonadi as well, but the 18.08

> implementation of it, feels like a major burden to me. 17.12 has been

> better, as it did not have the stalls during receiving mails and KMail

> did not crash several times a day. But it still used a lot of resources

> and mail filtering was still slow. However it was a lot more bearable

> than 18.08 appears to me.

>

>

> [1] Bug 404052 - Crash during/after filtering inbox, probably related to

> Qt WebEngine integration

> https://bugs.kde.org/404052

>

> [2] I do not agree with that move, but that is how it is currently…

>

> Thanks,

Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

ianseeks-3
In reply to this post by Martin Steigerwald
On Thursday, 4 April 2019 13:20:06 BST Martin Steigerwald wrote:

> Vojtěch Zeisek - 04.04.19, 14:10:
> > I don't have much annoying issues with KMail, for me it's still the
> > best e-mail client. I don't wish to leave it.
> > So big thanks to all developers.
>
> Thank you for your feedback.
>
> Great to hear it is working much better for you.
>
> Just to make it clear:
>
> I am grateful for KDEPIM. KMail is a marvelous mail program, Korganizer
> and KAddressBook are also really nice.
>
> So yes, big thanks to all developers. No matter what my recent
> experiences where.
>
> Thanks,
>
Try converting to Postgres - i did that a few years ago and most of the problems you list went away.  The only thing that still didn't work well was the filtering on incoming email as it created duplicates (i've not tried it since) so i let all emails arrive first then selected all then applied the filters manually. I'm only getting the occasional duplicate now.

--
opensuse:tumbleweed:20190402
Qt: 5.12.2 KDE Frameworks: 5.56.0 - KDE Plasma:  5.15.3 - kwin 5.15.3
kmail2 5.10.3 - akonadiserver 5.10.3 - Kernel:  5.0.5-1-default  - xf86-video-nouveau:  1.0.15



Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Peter Humphrey-3
On Friday, 5 April 2019 08:37:20 BST Ianseeks wrote:

> The only thing that still didn't work well was the filtering on incoming
> email as it created duplicates (i've not tried it since) so i let all emails
> arrive first then selected all then applied the filters manually. I'm only
> getting the occasional duplicate now.

That's a good idea - I must try it. At present new emails are being filtered
into their proper folders, and marked unread, but the Unread total is not
updated so I don't know they're there. I have to stop and start akonadi for
them to appear.

I've several times created a new user for myself, setting KMail up again and
importing my my mails. I get the same behaviour every time.

I've also found that importing from KMail archive is not a good idea: it
doesn't remove duplicates, which continue to cause problems thereafter. I
first extract the archive, then import the directory structure; that does
remove duplicates.

There's another problem with archives: they only store the emails themselves,
not things like Important marks. They aren't proper archives in the usual
sense.

--
Regards,
Peter.



Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Jerome Yuzyk-2
In reply to this post by Martin Steigerwald
On Thursday, April 4, 2019 5:46:42 AM MDT Martin Steigerwald wrote:
> Hi.
>
> I did not write in a long time about own issues with KDEPIM/Akonadi and
> instead tried to help others along here and there, but now I feel quite
> frustrated…

And further, the whole point of Akonadi was to collect all "social desktop"
data in one place for some higher purpose that's never been realized and
instead introduced middleware that's more often than not an impediment to
program use without, after many years, any benefit to the operation of the
programs I use most. It's just something to fear, or work around, or live
with, in hopes that it'll work one day, after many years of promises.



Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

Erik Quaeghebeur
On vrijdag 5 april 2019 18:32:40 CEST, Jerome Yuzyk wrote:
> And further, the whole point of Akonadi was to collect all "social desktop"
> data in one place for some higher purpose that's never been realized and
> instead introduced middleware that's more often than not an impediment to
> program use without, after many years, any benefit to the operation of the
> programs I use most. It's just something to fear, or work around, or live
> with, in hopes that it'll work one day, after many years of promises.

I agree that Akonadi works sub-optimally. However, its purpose is quite
clear: to make ‘groupware’ data available to in a standardized way, with
reusable components so that not every application needs to roll their own.
Its use outside of kdepim has remained limited. But for example Zanshin
also uses it as its backend (thankfully, so my todos sync together with my
calendar data) and plays nicely together with ktasks (each presents a
different view of the same data). As another example, Trojita directly uses
the contacts data in Akonadi, so it doesn't have to implement its own
contacts component.

So, to be honest, I think the idea of Akonadi is good. The implementation
is buggy and possibly a bit over-engineered. I think there is just a
manpower problem to fix the bugs. As far as I know there is no corporate
backing; Kolab is now sponsoring Kube <https://kube-project.com/>
development.


Best,

Erik
Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

ianseeks-3
In reply to this post by Peter Humphrey-3
On Friday, 5 April 2019 10:53:05 BST Peter Humphrey wrote:

> On Friday, 5 April 2019 08:37:20 BST Ianseeks wrote:
>
> > The only thing that still didn't work well was the filtering on incoming
> > email as it created duplicates (i've not tried it since) so i let all emails
> > arrive first then selected all then applied the filters manually. I'm only
> > getting the occasional duplicate now.
>
> That's a good idea - I must try it. At present new emails are being filtered
> into their proper folders, and marked unread, but the Unread total is not
> updated so I don't know they're there. I have to stop and start akonadi for
> them to appear.
I configured the toolbar to have "Apply All Filters" on it to make it easy.
I occasionally get an "Unread" total on the inbox even though there are none.  It seems that sometimes the process doesn't complete.

> I've several times created a new user for myself, setting KMail up again and
> importing my my mails. I get the same behaviour every time.
>
> I've also found that importing from KMail archive is not a good idea: it
> doesn't remove duplicates, which continue to cause problems thereafter. I
> first extract the archive, then import the directory structure; that does
> remove duplicates.
>
> There's another problem with archives: they only store the emails themselves,
> not things like Important marks. They aren't proper archives in the usual
> sense.

i've not tried that for years.  I think it tried it when migrating from one version to another and it didn't seem to work well.
>


--
opensuse:tumbleweed:20190403
Qt: 5.12.2 KDE Frameworks: 5.56.0 - KDE Plasma:  5.15.3 - kwin 5.15.3
kmail2 5.10.3 - akonadiserver 5.10.3 - Kernel:  5.0.5-1-default  - xf86-video-nouveau:  1.0.15



Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

ddumont
In reply to this post by Martin Steigerwald
On Thursday, 4 April 2019 13:46:42 CEST Martin Steigerwald wrote:
> The only other idea I had is to just switch from MariaDB to PostgreSQL.
> It would mean a PostgreSQL upgrade once in a while, but it if works
> better, I'd probably do that.

I've just switched from MySQL backend to PostgreSQL (on Debian/unstable) and
kmail (through kontact) works much better.

With Mysql backend, I had to restart akonadi 2 or 3 times a day and purge the
db every week or so.

I've switched to PostgreSQL 2 days ago (which is admittedly not statistically
significant) and had no problem since.

HTH


Reply | Threaded
Open this post in threaded view
|

Re: Considering to switch away from KDEPIM/Akonadi

ddumont
In reply to this post by Vojtěch Zeisek-2
On Thursday, 4 April 2019 14:10:56 CEST Vojtěch Zeisek wrote:
> I use Akonadi-ews and it works perfectly. Well, if You believe Exchange
> could work at all.

I've heard of this EWS connector a few months ago, but could not figure out how
to to use it with kmail...

I've just found that the EWS setup was hidden behind the "Add custom..."
account button.

Thanks for the info which led me to search better in kmail configuration menus.

All the best

Dod



123