[bug] Manual off-peak email synchronisation settings are not respected

Question

23

[bug] Manual off-peak email synchronisation settings are not respected

Tracked by Jolla (In release)

asked 2016-01-08 11:59:18 +0300

slowcyclist
621 ●8 ●12 ●13

updated 2017-03-17 09:02:41 +0300

Mohjive

8581 ●166 ●226 ●250

http://www.y3k.se/

I have two IMAP email accounts on two different servers. I have set them up similarly with "personalized" sync schedule, such that they sync every hour during peak period and "manually" off-peak, so that if I do nothing they should just not sync off-peak and I should have no notification. In spite of this the two accounts perform one spurious sync each day after midnight (as can be seen on the 4th line of the attached log). It is not very pleasant to receive a notification in the middle of the night when you expect your phone to be perfectly quiet. My guess is that there is something wrong in the logic of the sync algorithm related to the date changing.

I attach the settings of one of the accounts and the corresponding log. The second account has different start and stop peak hours, but the problem is just the same.

[nemo@Jolla ~]$ su-devel cat  .cache/msyncd/sync/syncemail-111.xml
<?xml version="1.0" encoding="UTF-8"?>  <profile type="sync" name="syncemail-111">
<key value="111" name="accountid"/>
<key value="online" name="destinationtype"/>
<key value="syncemail-XXXXX" name="displayname"/>
<key value="true" name="enabled"/>
<key value="true" name="hidden"/>
<key value="true" name="scheduled"/>
<key value="true" name="sync_always_up_to_date"/>
<key value="false" name="sync_externally"/>
<key value="30" name="sync_since_days_past"/>
<key value="true" name="use_accounts"/>
<profile type="client" name="syncemail">
    <key value="from-remote" name="Sync Direction"/>
</profile>
<schedule interval="0" syncconfiguredtime="" enabled="true" days="5,2,3,1,6,7,4" time="">
    <rush end="17:00:00" interval="60" enabled="true" externalsync="false" days="5,2,3,1,4" begin="09:00:00"/>
</schedule>

[nemo@Jolla ~]$ su-devel cat  .cache/msyncd/sync/logs/syncemail-111.log.xml
<?xml version="1.0" encoding="UTF-8"?> <synclog name="syncemail-111">
<syncresults minorcode="0" scheduled="true" time="2016-01-07T15:02:12" majorcode="0"/>
<syncresults minorcode="0" scheduled="true" time="2016-01-07T15:17:23" majorcode="0"/>
<syncresults minorcode="0" scheduled="true" time="2016-01-07T16:17:14" majorcode="0"/>
<syncresults minorcode="0" scheduled="true" time="2016-01-08T00:17:18" majorcode="0"/>
<syncresults minorcode="0" scheduled="true" time="2016-01-08T09:17:22" majorcode="0"/>

Edit: Bug still present in 2.1.0.9

edit retag flag offensive close delete

Comments

1

You know, you could silence the phone at night? Or turn of connections? That would also save battery (and according to some people it might save you from cancer as well). There is also the Situations app which you could use to turn sounds or connections on/off at certain times.

jollailija ( 2016-01-08 15:34:10 +0300 )edit

1

Oh, you're right, thanks! I had not though about that ;). More seriously, I guess you understand this is meant to be a well-documented bug report. I have not found any other obvious way/place to report bugs... is there one? Is this report going to be forwarded to a person in charge? Is the code available somewhere, so that I can fix it for myself and propose a patch?

slowcyclist ( 2016-01-08 21:45:53 +0300 )edit

1

I just noticed this as well. In my case the account was previously set up to be always up to date, and I changed it to be always up to date during peak but sync manually during off-peak.

@jollailija: you might want people to be able to contact you in the middle of the night in case of emergency but not want to wake up from newsletters sent at 3 in the morning. If the sound is off, you won't hear your phone ring. If connections are off, you won't receive any messages sent through things like WhatsApp. Also, this is clearly a bug. If you set an account to only synchronise when manually prompted, it shouldn't start synchronising on its own, ever.

nthn ( 2016-09-05 10:12:31 +0300 )edit

1

Sounds like one I reported earlier could be related to this. Exchange Sync schedule issue

coley ( 2016-09-05 15:17:09 +0300 )edit

1

I can confirm this as well. From my corresponding files as in OP:

Mohjive ( 2017-03-17 09:00:44 +0300 )edit

You know, you could silence the phone at night? Or turn of connections? That would also save battery (and according to some people it might save you from cancer as well). There is also the Situations app which you could use to turn sounds or connections on/off at certain times.
jollailija ( 2016-01-08 15:34:10 +0300 )edit
Oh, you're right, thanks! I had not though about that ;). More seriously, I guess you understand this is meant to be a well-documented bug report. I have not found any other obvious way/place to report bugs... is there one? Is this report going to be forwarded to a person in charge? Is the code available somewhere, so that I can fix it for myself and propose a patch?
slowcyclist ( 2016-01-08 21:45:53 +0300 )edit
I just noticed this as well. In my case the account was previously set up to be always up to date, and I changed it to be always up to date during peak but sync manually during off-peak.
@jollailija: you might want people to be able to contact you in the middle of the night in case of emergency but not want to wake up from newsletters sent at 3 in the morning. If the sound is off, you won't hear your phone ring. If connections are off, you won't receive any messages sent through things like WhatsApp. Also, this is clearly a bug. If you set an account to only synchronise when manually prompted, it shouldn't start synchronising on its own, ever.
nthn ( 2016-09-05 10:12:31 +0300 )edit
Sounds like one I reported earlier could be related to this. Exchange Sync schedule issue
coley ( 2016-09-05 15:17:09 +0300 )edit
I can confirm this as well. From my corresponding files as in OP:
<schedule days="2,3,1,6,7,4,5" enabled="true" interval="0" syncconfiguredtime="" time=""> <rush begin="08:30:00" days="2,3,1,6,7,4,5" enabled="true" end="22:00:00" externalsync="true" interval="15"> </rush></schedule>
<syncresults majorcode="0" minorcode="0" scheduled="true" time="2017-03-16T22:06:45"></syncresults>
Mohjive ( 2017-03-17 09:00:44 +0300 )edit

Accepted Answer

10

answered 2017-11-01 10:14:24 +0300

Damien Caliste
6535 ●105 ●139 ●161

https://github.com/dcalis...

updated 2019-09-12 13:03:37 +0300

2019-09-12 news: I've finally reworked my patch one last time to follow review comments. Today, @Pekka Vuorela accepted it. I'm using it for several weeks now without wake up in the night anymore.

2017/11/16 news: I've reworked my patch and updated the merge request upstream. I'm waiting for review. In the meantime, I've compiled patched msyncd on OBS if some brave users would like to try. You can find the compiled package in this OBS repository. One need to install buteo-syncfw-qt5 and buteo-syncfw-qt5-msyncd. Then restart the daemon with systemctl --user restart msyncd. To revert back to Jolla version, issue pkcon install buteo-syncfw-qt5 buteo-syncfw-qt5-msyncd in a directory where the downloaded RPMs are not present otherwise it may use the local ones.

Feel free to try and send me feedback. With the simple implementation of the patch, there is a little issue : last rush hour sync overshoots the end time by an amount lower than the rush interval period.

2017/11/09 later news: Ok, thank you @Mohjive questioning me on the bug. I think I've now understand fully the bug and can explain what was observed. I will update the MR later to properly solve this. Let me explain for those interested:

This is fully related to how iphb is working and the fact that frequencies are used. To understand, I need to explain what is iphb. This is module in the kernel that can write something to a socket periodically, depending on what user is demanding as waiting duration. To save battery life, if two processes are requesting a 30s interval wake up, they will be woken up at the same time. If another process is requesting a 60s period, it will be woken up on the same time than the 30s ones, but only once over two periods. Important notice: but we don't know which one every two is chosen. For instance, let say that 30s period is waken at every hh:mm:[00,30] and the 60s period is waken at every hh:mm:00. If the submission for wake up is submitted at 12:34:56, then both the 30s and the 60s periods will be awaken in the coming 4 seconds. To sumarize, using frequency (and particularly long period) can lead to awaken sooner than expected.

Here is the output of a small C program I wrote to convinced myself:

start waiting 30s...
plop 0 0 0
plop 0 0 0
plop 0 0 0
0: eating 4.
1: eating 4.
plop 1 1 0
plop 1 1 0
plop 1 1 0
0: eating 4.
2: eating 4.
plop 2 1 1
plop 2 1 1
plop 2 1 1
0: eating 4.
1: eating 4.
plop 3 2 1
plop 3 2 1
plop 3 2 1
0: eating 4.
plop 4 2 1
plop 4 2 1
plop 4 2 1
0: eating 4.
1: eating 4.
2: eating 4.
plop 5 3 2
plop 5 3 2
plop 5 3 2
0: eating 4.
plop 6 3 2
plop 6 3 2
plop 6 3 2
0: eating 4.
1: eating 4.
plop 7 4 2
plop 7 4 2
plop 7 4 2
0: eating 4.
2: eating 4.
done.

Each plop correspond to 10s of passing time. I ran 3 scheduler with frequency 30s, 60s and 90s. "x: eating" means that event 30s × (i + 1) was awaken. The id after the plop is the number of times for each three periods they have been awaken. We clearly see that event 60s is called too soon (after 30s only), but then later on it is called indeed every 60s.

To conclude, when the rush period is over and ask to wait for more than 12 hours, frequency is set to 24 hours shift, but the reset of the 24h period is done at midnight so the call is done at that moment. As I put earlier in bold, with frequencies, we don't control when the call happen.

2017/11/09 news: I've submitted a merge request upstream with a patch that IMHO should solve the issue. By the way, thank you for sharing this kind of configuration. I was not aware of that possibility but is now using it actively ;)

I've just open a Mer bug to follow this. Thank you for providing detailed information already. It should be possible to reproduce and fix. I'll give a look when I have time.

edit flag offensive delete publish link

Comments

I've read the mr comment, but don't understand the first paragraph about stopping between rush hours. How is a frequency less than 24 hours making the period longer then the rush hour time?

For me I sometimes get a sync in the middle of the night, several hours after the rush period has ended, so it's not a simple rounding error. Also see OP, where one sync was in the middle of the night (00:17) while the rush hour was set to 09:00-17:00.

Mohjive ( 2017-11-09 10:59:32 +0300 )edit

@Mohjive there is still something puzzling me in the fact that wake up appears in the middle of the waiting period, indeed. What I understand from actually reading the code is that:

at the last sync of the rush period (let say at 16:55 for instance), the next sync time is computed. Since there is no interval given (value is 0), next sync is set to the beginning of the rush period next day, so sync is set to D+1 9:00). I've checked (and reenable the test) that this calculation is done well for a lot of various conditions.
then the SyncBackground::set() function is called with a duration of number of seconds between now (i.e. exact current time) and the next sync date (here D+1 9:00 in our example), which is $((560+360020)) = 72300.
in SyncBackground::frequencyFromSeconds(), the sleeping duration is then set to 24 hours (because duration is more than 12h but less than 24h). Result will be that waking time will be wrong.

I agree that this scenario is not explaining the waking up in middle of the night as reported here. But IMHO, the current code as it is, is broken for not periodic waiting times like what can happen when setting rush hours without all-day intervals.

Damien Caliste ( 2017-11-09 11:18:49 +0300 )edit

In the second bullet, surely you mean

5*60+3600*14

I've also made a comment in the MR.

Mohjive ( 2017-11-09 15:41:43 +0300 )edit

@Mohjive you're right for the typo in the comment. I've seen your comment in the MR. Thank you for looking at it. I will answer it later. I'm still testing strange behaviours with wake up in the wait period when having another account that should sync during the wait period of another…

Damien Caliste ( 2017-11-09 16:58:59 +0300 )edit

I can share the C program if someone wants to play with it.

Damien Caliste ( 2017-11-09 23:08:06 +0300 )edit

[bug] Manual off-peak email synchronisation settings are not respected

Comments

1 Answer

Comments

Question tools

Stats

Related questions

[bug] Manual off-peak email synchronisation settings are not respected

Comments

1 Answer

Comments

Question tools

Public thread

Stats

Related questions