Kmail not syncing read/unread state with imap mailbox

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
I'm using Kmail 5.11.0 on an updated Arch machine.

I've set up this yahoo.it mailbox via Imap, the db has been set to
PostgreSQL and it works well except from this annoying issue:

If I read mails from Thunderbird (on another machine) and then I come
back to Kmail, messages I've just read are marked as non read. This
happens even if I hit the "check for new mail" button.

Messages are actually marked as read by Thunderbird since their state is
displayed correctly if I check those on the web interface of my mailbox
or by k-9 mail on android.

On the contrary, reading mails on Kmail and then back on Thunderbird (or
others), messages are shown with their correct read state.

I don't know if there is the same issue with madiaDB too since I managed
to use PGSQL from the beginning.

I haven't find any solution/workaround at the moment, so if you can
confirm this, I think it is a bug.

Thank you


Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

René J.V. Bertin
On Saturday April 27 2019 12:42:15 Francesco wrote:

>If I read mails from Thunderbird (on another machine) and then I come
>back to Kmail, messages I've just read are marked as non read. This
>happens even if I hit the "check for new mail" button.

>Messages are actually marked as read by Thunderbird since their state is
>displayed correctly if I check those on the web interface of my mailbox
>or by k-9 mail on android.
>
>On the contrary, reading mails on Kmail and then back on Thunderbird (or
>others), messages are shown with their correct read state.

I get exactly the same thing between KMail 4.14 (on my Mac) and KMail 4.13 (on my Linux rig). The former will ignore most of the state changes made by the latter but not vice versa (read state usually does update, but things like "flagged" or "action item" don't). I would have guessed that bug be fixed in an early 5.x version .... but maybe it's a feature instead?
Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
In data sabato 27 aprile 2019 14:56:49 CEST, René J.V. Bertin ha
scritto:

> On Saturday April 27 2019 12:42:15 Francesco wrote:
> >If I read mails from Thunderbird (on another machine) and then I come
> >back to Kmail, messages I've just read are marked as non read. This
> >happens even if I hit the "check for new mail" button.
> >
> >Messages are actually marked as read by Thunderbird since their state
> >is displayed correctly if I check those on the web interface of my
> >mailbox or by k-9 mail on android.
> >
> >On the contrary, reading mails on Kmail and then back on Thunderbird
> >(or others), messages are shown with their correct read state.
>
> I get exactly the same thing between KMail 4.14 (on my Mac) and KMail
> 4.13 (on my Linux rig). The former will ignore most of the state
> changes made by the latter but not vice versa (read state usually
> does update, but things like "flagged" or "action item" don't). I
> would have guessed that bug be fixed in an early 5.x version .... but
> maybe it's a feature instead?

A feature? I don't know, but It feels odd when such things are happening
to me :)

All other clients sync correctly the "read/unread" status, so I can't
think this as a feature. It seems like some kind of inconsistency
between devices that are supposed to be in sync via the imap protocol,
but their are not.

--
F



Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

René J.V. Bertin
On Saturday April 27 2019 15:57:52 Francesco wrote:
>A feature? I don't know, but It feels odd when such things are happening
>to me :)

You know the tongue-in-cheek "it's not a bug, it's a feature" thing? ;)

It can be both too, a buggy feature (you never know (as a mere user) where kmail/akonadi store what information...)

>between devices that are supposed to be in sync via the imap protocol,
>but their are not.

Do you get the correct status on mail that hasn't been seen by kmail yet but which you read in another client? If that works then there must be an issue with the refreshing of email headers.

R
Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
In data sabato 27 aprile 2019 18:52:24 CEST, René J.V. Bertin ha
scritto:
> Do you get the correct status on mail that hasn't been seen by kmail
> yet but which you read in another client? If that works then there
> must be an issue with the refreshing of email headers.

Other clients (e.g. Thunderbird and K-9) always shows the correct read/
unread state, even if Kmail doesn't.

Kmail only shows the correct read state when it has opened by itself
those mails. If those have been already opened by other clients, the
read state isn't synced up properly.

Honestly I don't know where this "propriety" is stored (in the header or
somewhere else) but I think KM should be able to read it from the Imap
server.

--
F


Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
In data domenica 28 aprile 2019 11:23:50 CEST, Francesco ha scritto:
> If those have been already opened by other clients, the
> read state isn't synced up properly.

... not even if clicking on Messages > Update folder (F5) or File >
Check new mails (CTRL+L).

--
F



Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

René J.V. Bertin
In reply to this post by Francesco Porro-2
On Sunday April 28 2019 11:23:50 Francesco wrote:

>Honestly I don't know where this "propriety" is stored (in the header or
>somewhere else) but I think KM should be able to read it from the Imap
>server.

A priori IMAP uses a header to store read and flagged state on the server. So KMail should at least respect that when it syncs mail it has never seen before (but which you already read, e.g. from another computer or your phone while the system running KMail was off/suspended).

After that, akonadi probably stores this kind of state info in its database first, and may fail to keep that in sync with the information on the server (because of flawed reasoning which of that state info has priority?) If those assumptions are correct you should be able to force-update the info by quitting kmail (kontact), running `akonadictl vacuum` (which should remove all cached emails from the db) and then restarting kmail.


R.
Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
On domenica 28 aprile 2019 12:36:36 CEST René J.V. Bertin wrote:

> On Sunday April 28 2019 11:23:50 Francesco wrote:
> >Honestly I don't know where this "propriety" is stored (in the header
> >or somewhere else) but I think KM should be able to read it from the
> >Imap server.
>
> A priori IMAP uses a header to store read and flagged state on the
> server. So KMail should at least respect that when it syncs mail it
> has never seen before (but which you already read, e.g. from another
> computer or your phone while the system running KMail was
> off/suspended).
>
> After that, akonadi probably stores this kind of state info in its
> database first, and may fail to keep that in sync with the
> information on the server (because of flawed reasoning which of that
> state info has priority?) If those assumptions are correct you should
> be able to force-update the info by quitting kmail (kontact), running
> `akonadictl vacuum` (which should remove all cached emails from the
> db) and then restarting kmail.
>
>
> R.

So,

- closed kmail
- run: akonadictl fsck && akonadictl vacuum
- read messages on Thunderbird (on another host)
- akonadictl restart
- reopened kmail
- pushed "Check mail"
- looked into a mail thread which I had just read on TB...

messages still not marked as read!

I've also tried two things:
- replaced postgreSQL with mariadB
- tested a Gmail account
 
nothing. The issue is still there.


--
F



Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

René J.V. Bertin
On Monday April 29 2019 11:35:08 Francesco wrote:

>messages still not marked as read!

Even messages that came in while kmail wasn't running?

NB: to be 100% certain you should have stopped akonadi completely, or suspended the host on which it is running. *Usually* email accounts are taken offline when you quit kmail but if for some reason they come back online they could have been syncing (if not only the email headers) in the background.

Evidently you don't need to use another computer for Thunderbird if you stopped akonadi (`akonadictl stop`).

If your issue persists even after that you probably should file a bug report because your kmail build is violating the very principle behind IMAP (keeping all state on the remote mail server so you can read your email from any host without having to remember which messages you already read etc).

R.
Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
On lunedì 29 aprile 2019 11:50:33 CEST René J.V. Bertin wrote:
> On Monday April 29 2019 11:35:08 Francesco wrote:
> >messages still not marked as read!
>
> Even messages that came in while kmail wasn't running?

I think so, but I'll do further checks.
However I don't get that point of this question.
 
> NB: to be 100% certain you should have stopped akonadi completely, or
> suspended the host on which it is running. *Usually* email accounts
> are taken offline when you quit kmail but if for some reason they
> come back online they could have been syncing (if not only the email
> headers) in the background.

Kmail was off.
akonadi was stopped.
Sure 100%.
 
> Evidently you don't need to use another computer for Thunderbird if
> you stopped akonadi (`akonadictl stop`).

Uh?

> If your issue persists even after that you probably should file a bug
> report because your kmail build is violating the very principle
> behind IMAP (keeping all state on the remote mail server so you can
> read your email from any host without having to remember which
> messages you already read etc).
>
> R.

Indeed.

--
F


Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
On lunedì 29 aprile 2019 13:12:38 CEST Francesco wrote:
> On lunedì 29 aprile 2019 11:50:33 CEST René J.V. Bertin wrote:
> > On Monday April 29 2019 11:35:08 Francesco wrote:
> > >messages still not marked as read!
> >
> > Even messages that came in while kmail wasn't running?
>
> I think so, but I'll do further checks.
> However I don't get that point of this question.

I've done further investigations... keeping kmail closed and akonadi
stopped, I retrieved new mails on Thunderbird and read those.
Back to Kmail, checked new mail... they are correctly marked as read!

So the issue seems to be limited to mails that are already fetched by
Kmail and later read on other clients. For those mails already marked as
read, KMail seems to retrieve the correct status.

Odd...

--
F


Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Erik Quaeghebeur
Just FYI, here read status (and other IMAP flag changes) is *immediately*
synced between KMail and other clients, in both directions, both for
setting and unsetting. (KMail/KIMAP 18.12.3, sqlite backend, regular
syncing is unset, so apparently that only refers to fetching new/expunging
deleted messages.)

Erik
Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
On Monday, 29 April 2019 18:43:26 CEST Erik Quaeghebeur wrote:
> Just FYI, here read status (and other IMAP flag changes) is *immediately*
> synced between KMail and other clients, in both directions, both for
> setting and unsetting. (KMail/KIMAP 18.12.3, sqlite backend, regular
> syncing is unset.)
>
> Erik

I'm glad for you.

Today I brought up a new Kubuntu 19.04 VM (kmail 5.10, mysql  Ver 14.14
Distrib 5.7.26); configured Kmail & accounts from scratch.

The issue is still present...

> so apparently that only refers to fetching new/expunging deleted messages

This issue refers to all messages which were already fetched by Kmail and then
read on another client. Newly fetched messages by Kmail but already read on
Thunderbird are marked correctly.

--
F


Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Daniel Vrátil -2
In reply to this post by Francesco Porro-2
On Monday, 29 April 2019 16:23:07 CEST Francesco wrote:

> On lunedì 29 aprile 2019 13:12:38 CEST Francesco wrote:
> > On lunedì 29 aprile 2019 11:50:33 CEST René J.V. Bertin wrote:
> > > On Monday April 29 2019 11:35:08 Francesco wrote:
> > > >messages still not marked as read!
> > >
> > > Even messages that came in while kmail wasn't running?
> >
> > I think so, but I'll do further checks.
> > However I don't get that point of this question.
>
> I've done further investigations... keeping kmail closed and akonadi
> stopped, I retrieved new mails on Thunderbird and read those.
> Back to Kmail, checked new mail... they are correctly marked as read!
>
> So the issue seems to be limited to mails that are already fetched by
> Kmail and later read on other clients. For those mails already marked as
> read, KMail seems to retrieve the correct status.
>
> Odd...
Does this happen exclusively with Gmail? We do handle Gmail a little bit
differently from other IMAP server (because Gmail's broken). Could you try and
see if you can reproduce the same problem with some other email provider,
please?

- Dan


--
Daniel Vrátil
www.dvratil.cz | [hidden email]
IRC: dvratil on Freenode (#kde, #kontact, #akonadi, #fedora-kde)

GPG Key: 0x4D69557AECB13683
Fingerprint: 0ABD FA55 A4E6 BEA9 9A83 EA97 4D69 557A ECB1 3683

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
On Thursday, 2 May 2019 18:17:28 CEST Daniel Vrátil wrote:

> On Monday, 29 April 2019 16:23:07 CEST Francesco wrote:
> > On lunedì 29 aprile 2019 13:12:38 CEST Francesco wrote:
> > > On lunedì 29 aprile 2019 11:50:33 CEST René J.V. Bertin wrote:
> > > > On Monday April 29 2019 11:35:08 Francesco wrote:
> > > > >messages still not marked as read!
> > > >
> > > > Even messages that came in while kmail wasn't running?
> > >
> > > I think so, but I'll do further checks.
> > > However I don't get that point of this question.
> >
> > I've done further investigations... keeping kmail closed and akonadi
> > stopped, I retrieved new mails on Thunderbird and read those.
> > Back to Kmail, checked new mail... they are correctly marked as read!
> >
> > So the issue seems to be limited to mails that are already fetched by
> > Kmail and later read on other clients. For those mails already marked as
> > read, KMail seems to retrieve the correct status.
> >
> > Odd...
>
> Does this happen exclusively with Gmail? We do handle Gmail a little bit
> differently from other IMAP server (because Gmail's broken). Could you try
> and see if you can reproduce the same problem with some other email
> provider, please?
>
> - Dan

Actually, I see this happening with my yahoo mailbox accessed via Imap.

But yes, occurring also with Gmail.

Maybe later I'll try also with hotmail and gmx.

--
F



Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

René J.V. Bertin
In reply to this post by Daniel Vrátil -2
On Thursday May 02 2019 18:17:28 Daniel Vrátil wrote:

> Does this happen exclusively with Gmail? We do handle Gmail a little bit
> differently from other IMAP server (because Gmail's broken). Could you try and

Again, FWIW because related to 4.14, but there I indeed only see state syncing issues with GMail accounts.
And I just observed something there that may give an indication as to what's going on:

- I had an email from this morning in my inbox which I hadn't yet read; KMail showed it as such
- I went to the account's "All Mail" folder, waited for it to re-sync then read that mail
- going back to the inbox the email was still there and still showed as unread
- attempts to "Update Folder" and "Update Folder and Subfolders" had no effect as if they were ignored (not even a progress bar)

I could confirm this with other emails. I then went to do something else and when I came back an hour or so later the emails did show as read.

The most straightforward explanation would be that state changes are not picked up and/or that gmail's IMAP server doesn't generate whatever notification standard imap servers generate in such a situation. If in addition kmail doesn't "bother" to re-read email for the duration it is configured to keep downloaded copies you (probably) end up with what we're seeing.

R.
Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
In reply to this post by Francesco Porro-2
On giovedì 2 maggio 2019 19:08:25 CEST Francesco wrote:

> On Thursday, 2 May 2019 18:17:28 CEST Daniel Vrátil wrote:
> > On Monday, 29 April 2019 16:23:07 CEST Francesco wrote:
> > > On lunedì 29 aprile 2019 13:12:38 CEST Francesco wrote:
> > > > On lunedì 29 aprile 2019 11:50:33 CEST René J.V. Bertin wrote:
> > > > > On Monday April 29 2019 11:35:08 Francesco wrote:
> > > > > >messages still not marked as read!
> > > > >
> > > > > Even messages that came in while kmail wasn't running?
> > > >
> > > > I think so, but I'll do further checks.
> > > > However I don't get that point of this question.
> > >
> > > I've done further investigations... keeping kmail closed and
> > > akonadi
> > > stopped, I retrieved new mails on Thunderbird and read those.
> > > Back to Kmail, checked new mail... they are correctly marked as
> > > read!
> > >
> > > So the issue seems to be limited to mails that are already fetched
> > > by
> > > Kmail and later read on other clients. For those mails already
> > > marked as read, KMail seems to retrieve the correct status.
> > >
> > > Odd...
> >
> > Does this happen exclusively with Gmail? We do handle Gmail a little
> > bit differently from other IMAP server (because Gmail's broken).
> > Could you try and see if you can reproduce the same problem with
> > some other email provider, please?
> >
> > - Dan
>
> Actually, I see this happening with my yahoo mailbox accessed via
> Imap.
>
> But yes, occurring also with Gmail.
>
> Maybe later I'll try also with hotmail and gmx.

So, after some further testing, I can reproduce the same problem with
Gmail, hotmail, gmx and email.it.

All of these accounts set up as imap.

Version of Kmail tested:
- 5.11.0 on Arch linux (updated)
- 5.10.3 on Kubuntu 19.04 (updated)

--
F



Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

Francesco Porro-2
In reply to this post by René J.V. Bertin
On giovedì 2 maggio 2019 19:34:13 CEST René J.V. Bertin wrote:

> Again, FWIW because related to 4.14, but there I indeed only see state
> syncing issues with GMail accounts. And I just observed something
> there that may give an indication as to what's going on:

Just for knowledge, is KMail 4.14 still supported/developed nowadays?

I thought it was superseded by Kmail 5 and thus reached EOL.

--
F


Reply | Threaded
Open this post in threaded view
|

Re: Kmail not syncing read/unread state with imap mailbox

René J.V. Bertin
On Sunday May 05 2019 16:33:17 Francesco wrote:

>Just for knowledge, is KMail 4.14 still supported/developed nowadays?
>
>I thought it was superseded by Kmail 5 and thus reached EOL.

You're right. But it works, and I plan to stick with it as long as possible.