CalDAV/CardDAV testing volunteers requested

Fails to sync or access the local database when connecting to my Nextcloud 11.0.2 instance (the default in 2.1.0.9 works). Sent an email about it.

Oops - I forgot that it has a new dependency on an updated libsailfishkeyprovider. I've edited my original post to include a download link to that one also, now. Sorry for the confusion!

1. installation without issues
2. caldav was changed to "manually", set back to "15 min"
3. calendar app shows caldav calendar (did not since 2.1.0.9), yay!
4. Sent some meetings from outlook, recieved them on the phone, imported them into the calendar
5. all meetings where automatically synced to my caldav calendar.

What is about the new SailfishOS 2.1.0.10 EA ? They also updated the package to fix the CalDav issues, but the version is lower than yours. Is it the same update, but with different version-numbers?

@ExPLIT: It shouldn't. Taking the presented information from chris.adams here into account, that doesn't make sense :)
2.1.0.9 therefore uses the old CalDav implementation and received extra fixes in 2.1.0.10, 2.1.1.x will probably contain the here mentioned new boteo-plugin.

@hoschi that's exact. The CalDAV corrections in 2.1.0.10 include corrections for MER1625 and MER1712.

Nextcloud 11.0.2:

The last releases (including todays 2.1.0.10) didn't work for me as my calendars showed up in the settings but not in the calendar app. Even by deleting and recreating dav accounts the calendars were discovered but didn't show up in calendar app. This is fixed now with the testing release you provided here. :)

I opened an ics file from an email attachment and imported it to my calendar and it was synced to my nextcloud. So this carddav version works well for me so far. I only experienced that one of my calendars doesn't get synced from my nextcloud. Probably it's because it is not a calendar created by me but an external calendar imported into my nextcloud (new feature): http://i.cal.to/ical/116/redbullmuenchen/spielplan/c15985ec.0e3f21aa-82bc21ab.ics

If you want to test this new feature I can provide you a test account at my installation. Actually you already have this account but I can reset your password or create a new account if you don't remember your credentials.

So far Nextcloud 11.0.2 with buteo-sync-plugin-caldav-0.1.40-1 has worked for me without any problems after the initial dependency problem was resolved.

In addition to my normal personal calendar usage (which I have to admit isn't that crowded), I tried to create some trickier scenarios by setting recurring several day events and editing single instances of them, and it still worked fine. This is kind of stuff that usually breaks quite easily with activesync.

Our company's Zimbra server had upload issues with events with alarms created on SFOS calendar, but I already reported that to OP.

Works against Nextcloud 11.0.2 (as the 2.1.0.10 patch did). No amok running caldav service till now.

Thanks to everyone for your efforts in testing!

A couple of potential issues were identified and I'll try to investigate those further over the coming couple of weeks. It seems to me that those issues are probably already existent in the plugin (e.g., one was related to all-day events spanning multiple days, and one was related to upsync of alarm details; neither of those are related to codepaths changed by the MER#1646 fix) so at this stage I'm happy to include this fix in the 2.1.1.x codeline.

The download links for those packages will probably expire in the next few days. If it turns out we need another testing round, I'll create new packages and new links. Thanks again to everyone for helping!

Hello @chris.adams regarding that you have some setup in house about ddifferent cloud solutions.
Soon the cozy team will pull a v3 of cozy out, with a contact and calendar clients (as i understand) reviewed .
Would it be possible that you'll be ready for it, if there is still interest on your side?
Should happened this summer, end of june or july.
https://blog.cozycloud.cc/post/2017/04/13/Release-Cozy-V3-Alpha?lang=en

If you follow the instructions from https://sailfishos.org/wiki/CalDAV_and_CardDAV_Community_Contributions#Adding_A_New_Service we can look to get the new cozy instance into the Mer test services. I can't spend much time on that personally, so we'd be relying on community contributions to get it working up to the point where we can turn it on in the infra instance.

@chris.adams I haven't encountered any problems with the latest set, at least yet. This is with Nextcloud 12.0.0.

Everything synced fine, so I put some 40 minutes into creating, editing and deleting contacts and calendar events on both Sailfish and server end. I didn't have any problems. I'll use this now and report back in case of problems.

@chris.adams that a clear answer, and i understand it. that just sad that i can't really participate on it. i'd just need a test device. and currently finance are just so short that i can't invest on it... :-(
But i find really great what you do until today . cool :-)
Have good continuation

@cemoi71 No problem, I understand, and thank you for your kind words.

@Manatus thank you very much for testing! Please do let me know if you encounter issues with these.

More testers would also be appreciated! Thanks!

So far no issues, only a bug with single quote: everytime I have it on my mailbox.org account it gets encoded in '

@chris.adams Ok, now I got some kind of issue. I have a specific event in the past and Nextcloud 12.0.0 reports that the event keeps updated several times a day. Probably every time my calendar syncs.

I'll see what the logs say later today...

@magullo, may you describe the issue in more details ? You may ask a new question about this, so we can keep track of it there.

for me, this new version does not introduce any improvements. Instead, I get even more random calendar duplication and syncing errors. Thus I set up a test user on my personal baikal 0.4.6 server, did some experiments also in combination with another client (Thunderbird Lightning) and sent a whole bunch of corresponding journalctl logs alongside device and server-database snapshots to @chris.adams. Hope this helps with fixing :-) Unfortunately, I was not yet able to reproduce the calendar duplication! But I could reproduce losing changes made to single occurences of recurring events by another client because the Jolla phone seems to have ignored the server-side.

How does one reset the caldav plugin back to the repository version? I'm not so experienced with rpm package management.

Normally, pkcon install buteo-sync-plugin-caldav should be enough. With devel-su also...

I have installed the newest version shortly after published, and got no duplications so far with my "usual methods". The issue seems to be fixed! :)

@chris.adams Thank you for your ongoing work on this subject. If I stay with this platform, I guess I have to abandon EAS, so I have to give your DAV plugins a thorough test anyway. I wiped the EAS account, installed your 29 Jun 2017 versions of the plugins and created two new accounts (IMAP, *DAV) connecting to the mailbox.org service. For the *DAV account, I only had to enter the root address of the DAV server, every other setting/service was auto discovered via the .well-known mechanism. Well done so far :-)

I will test two-way sync of calendar and contact entries and have an eye on the duplication issue. I'll report if I encounter any unexpected behaviour.

@chris.adams Version uploaded in 29 Jun didn't help with the events that keep updating at every sync.

@chris.adams nice to get new testing stuff on 29th June 17. (my sfos 2.1.0.11)

1. I tested a little on the 9.1.tst owncloud, created a calendar named "jolladiho" and synced with my JP-1301 and thunderbird lightning. I only recognised, that the owncloud event editor did not use a event serie from the jollaphone as a serie. It seems to be a single event. But thunderbird did it right.

2. I also tested against my radicale 0.10 caldav and carddav (home server). Creating and deleting works. I had no duplicated events or contacts today. Deleting an official holiday serie, that was duplicated with earlier versions, worked too.

All together, on todays test: No problems found. Maybe fixed ;-)

Is it correct, that on the update from the 29th june the version of carddav has changed, but the caldav version is still the same as the test release before?

@Fellfrosch: chris.adams wrote: "The CalDAV plugin is the main one which has been updated, the other packages are provided also but haven't changed as far as I can remember.". So indeed, CardDAV didn't change.

@Maus: So why has the version number of carddav changed but the caldav version number is still the same??????

@chris.adams: I still had duplication of calendars. Now I deleted the database an until now I got no duplication. I hope it stay like this. What is still broken: If my owncloud calendar isn't reachable, the calendars get lost. Ive sent you logs for this problem long time ago. Do you need further lgs for this?

@Fellfrosch: you are probably right. Iijoki seems to have come with carddav-0.0.25-1/caldav-0.1.38.1-1. Both have changed, and I updated both with the versions offered here. I believe what was meant was that the carddav plugin didn't change between Mar 20 and Jun 29 2017.

@chris.adams: Well it worked for nearly two days flawlessly. But now I unfortunately still have duplications of calendars. Even worse now the calendars got synced, so I could see all of the 6 times. But the events were not shown in the calendars. After a restart of the phone. The calendars are now just duplicated (2 times) and the events are here again...

I will have a look if I can make some deeper tests on weekend. If so, I will send you the logs.

If you are still seeing duplications with this particular package, yes I'd be very interested in seeing those logs. The logs you gave me previously were used in my initial analysis (and I was also able to test the updates against an account which was provided by another community member), however of course every server will potentially produce different output which might trigger a duplication bug in same cases.

Which is to say: while the package includes some patches which definitely fixed at least one duplication bug I'm aware of, if you can still repro a duplication with these packages then the logs will help me to identify the possible cause of other duplication bugs.

Thanks for your patience with this - I know it's frustrating.

After the last test files I did not have duplicates of calendars anymore. Just installed the new files, if something goes wrong, I will give you a note.

@chris.adams: I did some tests on the weekend. For having clean conditions, I've deleted the database and my sync account (and updated my nextcloud server). Than I renewed the account. So far I got no duplications. So no log files for this. But last time it also took 2 days so we have to wait if it is really fixed for my environment. BUT what I still can report is, that the calendars get lost after a while when the phone has no contact to the syncserver. Do you need logs for that? Do others also have this problem? I have this problem on two Phones - My servers go to sleep at night, so nearly every morning the Calendars are lost and reappear as soon as the syncserver is online again.

@Fellfrosch: I also see the events disappearing if no sync is possible (my work calendar can only sync if I am in my company's network). I did not yet install the patches provided here.

@Flaury: In my opinion this bug is much more serious than the duplication thing. The way it works at your company was the way, I did it in the past in my home network. I never wanted to open my network to the www. But with this bug I was forced to.

In the past it was just a simple owncloud installation. Never had to think about security issues, so it was just http and quite outdated. Nowadays I have to be always up to date and have to think about things i've never been engaged in before.

@chris.adams: It took some days but I still have duplications. The Problem is, I don't know how to log this, because it only appears every now and then and randomly. I think it has to do with the problem, that calendars get lost when the sync server isn't available. Why? Well, i have two calendar servers which i sync to. The one at my company (egroupware) and my own at home (nextcloud). With the new files i never got again duplication of events in my company's calendar. So yes it seems like you fixed that part. But i still get them with my private calendars at home. Three times so far, and I always recognized it in the morning after my server were down at night. If you want, I probably can send you a log where my calendars get lost, because that happens quite steadily. But it's quite unlikely that i can send you a log for the duplication problem.

@chris.adams: Unfortunately, I experience similar effects as Fellfrosch. Two days ago, many recurring events spontaneously multiplied. I had a busy day, and I ended up with some thirty entries, some showing up twice, others up to six times.

Another problem I noticed is past events that get their flag for being "acknowledged" cleared, apparently by the SFOS client. Once, I booted my phone and soon after, my desktop calendar notified me about some due events from two weeks ago. I could only get rid of these notifications by not using the SFOS connection or deleting those entries. I ended up with deleting those entries.

Jämsänjoki (2.1.1) seems to have worsened the situation: after installation of 2.1.1.23, removal and re-adding of *DAV accounts, all calendar entries immediately showed up nine-fold. This happens with a Open-Xchange server.

On the other hand, I somehow doubt that this has to do with Jämsänjoki. More likely, this is just coincidence and another loop of removing the *DAV account and re-adding it will yield a different factor. This is very frustrating. I have to dismiss my calendar notifications nine times :-(

@chris.adams: I just noticed that the calendar app, when offering me to select a target calendar for a new appointment, shows my CalDAV calendar nine times. Thus, not the events are duplicating in 2.1.1, but the calendar itself. I hope this helps in some way.

@Fellfrosch@chris.adams I can also confirm that the calendars disappear when the sync server is unavailable (jämsänjoki -> nextcloud box). this is a showstopper for me and the only getaround is to manually sync (although, does it work now?). also had the duplication errors, but did not erase the caldav after the jämsänjoki update. pardon me if the jämsänjoki update does not contain the latest caldav plugin discussed above. however, as @Fellfrosch also mentioned, at least the disappearing calendars is a serious issue imho.

@edgar and @Fellfrosch : I also set up a Radical server on my local network for CalDAV sync between my devices, but up to now I cannot reproduce the disappearance of calendars (once a first valid sync have been obtained) when not being at home. I would like to debug your issue anyway. Can we switch to this TJC question to further investigate ? I'm requesting some debug info there if you don't mind.

@chris.adams: The MD5 sum of all packages released on 10 Oct 2017 do not match those given above. For the caldav plugin, it is 98a9c83e619ef2f6b4f9fc41e2feef5a. Please confirm or update the question.

@Maus, it's a sha1sum, not a md5sum !

@Damien Caliste: thanks it clearly reads sha1sum above so I blame my age and the small screen of my JP1301, should think about an F5251, but that looks huge :-/

But this gives me the opportunity to thank you, M Caliste, and @chris.adams, for your continued effort to make *DAV usable on SFOS.

Most of the thanks goes to dcaliste recently, as most of my time has been taken up with other company priorities! Huge thanks to dcaliste, and of course the other community members who have been contributing (testing packages, reporting bugs and providing sync logs, triaging TJC issues and reporting findings in the contributors meetings, etc).

@chris.adams Looks like I'm not having constant update issue with events in Nextcloud anymore. But then I didn't see it immediately after the last OS update either so I'll keep checking it for a week or so.

Just a notice: The Oct 2017 packages don't fix the undead reminders bug, but they didn't address this problem specifically, anyway.

Thank you all for testing, sorry I haven't been too active on TJC to respond to feedback personally. I will review the MRs which Damien has created (including the one to address the undead reminders issue!) within the next week.

I've been using the test packages since approximately two weeks and have not seen any duplication since then.

I updated to kyminjoki and changed caldav sync from manual to twice a day, resulting in duplicate calendars. ( i had it on manual symc to avoid calendar duplication).

@edgar, version 2.1.3 doesn't contain the patchrs discussed here yet. So duplications may indeed appear in this version.

@chris.adams@Damien Caliste I'd really love to test the packages, but they're not available anymore under the links mentioned above, these only lead to an Owncload "file not found" page.

I've re-shared those now, the links will expire at the end of this month.

I get strange effect syncing my calendars (to be honest, the problem may be with presentation rather than synchronization): Unicode characters are replaced with numeric entities (in form ☾ for example) both in on-device calendar and notification screen (screenshots: https://www.lingua-it.pl/dl/?t=6f34ec8558958a7b7c711bf592f6fc8c and https://www.lingua-it.pl/dl/?t=4b97bbb7807fca0a7116e721151d9bb5 ).

Funny thing is, as far as I can tell, server gives raw UTF8 instead of entities, with charset given in headers: "Content-Type: text/calendar; charset=utf8" (tested with cadaver and wget), so conversion probably happens on device...

Events created on the phone are shown OK on phone, server and other clients.

Setup:

• phone: Xperia X
• OS: Sailfish X (official), 2.1.3.5
• server: SOGo 2.3

Tested with default libraries and updated ones from the original post -- no difference.