[Bug] Received SMS mixed together

Question

39

[Bug] Received SMS mixed together

bug
sms

asked 2015-07-21 23:19:11 +0200

This post is a wiki. Anyone with karma >75 is welcome to improve it.

updated 2020-05-24 15:40:32 +0200

Pohli

188 ●5 ●13 ●19

Last few days it sometimes happend, that recieved SMS does not make sense. So I asked sender and it is proved by screenshot that received message is corrupted.

It seems as an very old message is mixed at the end of the new one. Maybe it occures at the place where two SMS linked together to a long message. Up to now it only happend with the two most longest chats (more than 1000). Maybe an overflow in database?

I know there are already other threads according to SMSbugs, but I think mine is different to: phone receive same SMS many times or Old SMS text is sent instead of new text.

P.S.: Sailfish is still on 1.1.6.27

Edit: Still happening in 2020 with SFOS 3.3.0.16.

edit retag flag offensive close delete

Comments

I started noticing this as well on 1.1.6 with recipients that have way more than 1000 messages. There is an old message appended locally. It happened as I left the phone to lock itself on its own and also after forcing lock with the hardware key and then unlocking again. Hard to replicate.

Mr.Pancake ( 2015-07-22 02:17:42 +0200 )edit

I can't confirm that it is in relation with autolock. I know know I'm not the only one. So it is not a provider problem. If this bug is still on 1.1.7, I have to open an support case. This bug is very disgusting...

SaimenSays ( 2015-07-26 01:51:41 +0200 )edit

I can now confirm that extra (old and sent by me previously) sms randomly appears immediately after sending sms (contact with a few thousand messages).

Mr.Pancake ( 2015-07-31 02:21:33 +0200 )edit

4

There is also: Problems with "long" SMS messages (>160 chars.) including emojis

Problems are existing with receiving and sending!

I have < 1024 messages with a contact. Received a message which made no sense (sender likely placed emojis in it). The beginning is missing and I just received the end.

Even worse:

Contact with some more messages, don't know if > 1024 but I don't think so. I sent a long message. It's in the history exactly as I wrote it. But the recipient got the begin with content from a sms I've sent 5 months ago, and the end with content of the actual one. Thats a no go which can have terrible consequences depending on whats getting mixed up here.

So has anyone of you opened a ticket? Is Jolla aware of this terrible bugs?

It could be a reason for me to switch mobile os. Because I now feel very insecure and uncomfortable as I don't know what content my device will really send.

A day or so after the receive bug appeared about every second message didn't get sent (correctly) (provider answered with an automated sms that it could not be transfered). To overcome this the message had to be written new i.e. with copy and paste. I backed up and imported my messages with the backup function after that. This solved the provider does not "want" to send issue. But the bug of sending wrong content happened after that, so with a relatively new imported database. Receiving bug was while on 1.1.4, sending bug on 1.1.6!

This all happened after I used xmpp for some days. But this messages were already deleted and the protocol was deactivated.

Halo2 ( 2015-07-31 19:04:41 +0200 )edit

4

I have this problem since the beginning of the Jolla phone. Normally I do not delete messages from the conversation with my girlfriend. Every time the conversation has too much messages, SMS with more than 160 characters are mixed up. It seems that the more messages are in the conversation, the more messages are mixed up. In these cases I delete the conversation. After clearing the conversation, long SMS are not mixed up until the conversation has too much messages again. This is very annoying. The phone should be able to handle with huge conversations.

szobi ( 2015-07-31 23:32:51 +0200 )edit

I started noticing this as well on 1.1.6 with recipients that have way more than 1000 messages. There is an old message appended locally. It happened as I left the phone to lock itself on its own and also after forcing lock with the hardware key and then unlocking again. Hard to replicate.
Mr.Pancake ( 2015-07-22 02:17:42 +0200 )edit
I can't confirm that it is in relation with autolock. I know know I'm not the only one. So it is not a provider problem. If this bug is still on 1.1.7, I have to open an support case. This bug is very disgusting...
SaimenSays ( 2015-07-26 01:51:41 +0200 )edit
I can now confirm that extra (old and sent by me previously) sms randomly appears immediately after sending sms (contact with a few thousand messages).
Mr.Pancake ( 2015-07-31 02:21:33 +0200 )edit
There is also: Problems with "long" SMS messages (>160 chars.) including emojis
Problems are existing with receiving and sending!
I have < 1024 messages with a contact. Received a message which made no sense (sender likely placed emojis in it). The beginning is missing and I just received the end.
Even worse:
Contact with some more messages, don't know if > 1024 but I don't think so. I sent a long message. It's in the history exactly as I wrote it. But the recipient got the begin with content from a sms I've sent 5 months ago, and the end with content of the actual one. Thats a no go which can have terrible consequences depending on whats getting mixed up here.
So has anyone of you opened a ticket? Is Jolla aware of this terrible bugs?
It could be a reason for me to switch mobile os. Because I now feel very insecure and uncomfortable as I don't know what content my device will really send.
A day or so after the receive bug appeared about every second message didn't get sent (correctly) (provider answered with an automated sms that it could not be transfered). To overcome this the message had to be written new i.e. with copy and paste. I backed up and imported my messages with the backup function after that. This solved the provider does not "want" to send issue. But the bug of sending wrong content happened after that, so with a relatively new imported database. Receiving bug was while on 1.1.4, sending bug on 1.1.6!
This all happened after I used xmpp for some days. But this messages were already deleted and the protocol was deactivated.
Halo2 ( 2015-07-31 19:04:41 +0200 )edit
I have this problem since the beginning of the Jolla phone. Normally I do not delete messages from the conversation with my girlfriend. Every time the conversation has too much messages, SMS with more than 160 characters are mixed up. It seems that the more messages are in the conversation, the more messages are mixed up. In these cases I delete the conversation. After clearing the conversation, long SMS are not mixed up until the conversation has too much messages again. This is very annoying. The phone should be able to handle with huge conversations.
szobi ( 2015-07-31 23:32:51 +0200 )edit

Answer 1

9

answered 2015-08-20 04:57:20 +0200

chris.adams
2306 ●19 ●33 ●32

Thank you for this report. I have created a bug internally to track this issue and will update this thread with information as it becomes available.

If anyone has further information (e.g., a more concrete/precise way of reproducing the issue than "sometimes, when you are sending a long (>160 chars) SMS to a person to whom you have previously sent >1000 SMSes and to whom you have previously sent many other long (>160 chars) SMSes" -- e.g., can you trigger this behaviour specifically, or does it happen only randomly when those conditions are met?) then please reply to this answer.

Thanks,

Chris.

edit flag offensive delete publish link

Comments

It was a very new contact where it got "mixed" up first - with the 39th message. I only received the end of the message I should have received. A few days / or one-two weeks after using xmpp integration. Propably it was the first message with an emoticon in it (Somehow unlikely, but on the other side my people seem to know how to do it old-school ;-) ).

A few days later I only could send every third message or so. I did a backup and restore of the message database (with the phone built-in backup) to get back normal functionality. But:

A older contact, still < 1000 messages (but that are propably more than 1000 160 character sms), got my message flavored with old content.

Yea sorry not an manual to reproduce it but a few things which I think are of interest (error on receiving correct even with short history, a general send error where restore helps to send again, but now even incorrect content gets sent).

Halo2 ( 2015-08-20 19:55:24 +0200 )edit

Thanks Chris for official feedback!! In my case it only happens with two contacts wich have the most SMS in. And it does only appear sometimes. I don't use an XMPP account, so I think it's a problem with database, because it seems to be independent on message type.

SaimenSays ( 2015-08-20 22:42:15 +0200 )edit

1

See Matt's comment below for more information about this issue. While there's not much we can do to fix the way Android handles these SMS reference numbers, it is possible (but not certain) that we might be able to increase the size of the message reference number if we can convince upstream. We also need to check whether or not our own receiving code has the same bug that Android does.

chris.adams ( 2015-09-23 05:02:44 +0200 )edit

1

I have had this problem twice when receiving messages. The contact in question is the only one I have thousands of messages from/to (I copied the messages database and counted, I believe it was 2500-ish at the time of the first occurrence), and they are using iOS. I have received and sent sms:es and mms:es to that remoteUid, probably no calls as I generally don't call people and encourage people to not call me :P. I have also sent a few facebook XMPP messages to that contact's facebook (thought I'd mention it since the messages app shows all the messages)

The messages that got mixed up where both in the middle of a conversation with a lot of messages with a fairly short time between them, of which some where fairly long (160+), but I have also have long conversations which included long messages and not having this occur. I am sure that the second time it happened, both the message with the original text (A) and the new one which magically got text inserted from the first time (B), where shorter than 160 characters.

I'm not sure what you mean by SMS reference numbers, but messages A and B have unique messageToken:s in the database.

The first time I don't know the length or messageToken:s for as I don't remember anything that is easy to search for.

Sorry that I don't have any steps to reproduce either, but I hope it can help in some way. If there's anything else you wish to know about these messages just ask, I can find them fairly easily again.

tonireha ( 2015-09-24 19:10:46 +0200 )edit

3

@tonireha: thanks for that report. The SMS message reference number exists in the communication between the modem and the GSM infrastructure - it's lower down than anything the sailfish OS sees or records, so it doesn't appear in the message database. The Android problem (which this report indicates we may have too) is that the modem software is keeping an old message fragment around after all fragments of the message have been received and it has been dispatched to the phone software for display. Then it sits around until the modem receives a new message fragment that happens to use the same reference number (which is just a cycling counter) and the modem incorporates the old fragment into the newly received message.

The 160 characters limit is the most characters that can fit into a single SMS, but that's only possible if there are no special characters. If the message uses any characters that don't fit into the basic GSM character set, or if it includes emoji characters, or if just Apple decided to send 16-bit characters for some reason, then the limit per SMS is more like 65 characters. So you could certainly have fragmented messages with fewer than 160 obvious characters.

If there are obsolete fragments on your device, they will be found under the "/var/lib/ofono/[SIM-ID]/sms_assembly/" directory. Note that anything you have already received incorrectly will no longer be there, as it has already been (incorrectly) used to create an incoming message, and removed.

Thanks for the detailed report!

MattVogt ( 2015-10-02 00:53:37 +0200 )edit

Accepted Answer

9

answered 2017-04-09 06:31:50 +0200

fLegmatik
106 ●2 ●4 ●5

updated 2017-04-11 19:55:16 +0200

Workaround to solve this problem.

devel-su
cd /var/lib/ofono/<tab to select sim>
mv sms_assembly sms_assembly_`date +%F`
reboot

My purpose for Jolla: move obsolete parts from sms_assembly to commhistory.db after 2-5 hours from receiving first part, writing in text body <--partial message--> instead of missing part.

edit flag offensive delete publish link

Comments

3

I have cleanup sms_assembly yesterday. Today i requested 18 new messages with passwords from bank. All of them are very long, cyrillic (in UTF-16) and needed by my work every month. All 18 messages received fine, no sms was lost or corrupted. In previous months i always requested most of passwords again and again due to this problem. Then i look at sms_assembly and found 12 new folders, 10 of them with 003 part, 2 of them with 002 part. All this parts already stored in commhistory.db in their messages. So maybe this bug is related with other sms bugs like duplicate receiving.

Therefore i think that the best way now is just remove all parts from sms_assembly automatically every day.

fLegmatik ( 2017-04-11 09:20:21 +0200 )edit

2

I tested it and this workaround helped me. Long SMS message received correctly. I recommend everyone with sms problems look at this workaround.

nauevg ( 2017-04-16 15:13:45 +0200 )edit

1

This makes sense. How can I explore those in sms_assembly parts? Exploring them with more command shows non human readable characters. Edit! I tried to analyse those 002 parts with various converters. They seemed to be very short and in UTF16-LE format. I could not to convert them readable with various UTF-converters.

Rikujolla ( 2017-04-16 16:31:23 +0200 )edit

Rikujolla, i tried to analyze the backup of my fragments too. Today i decoded some text in files with "iconv -f utf-16be" and some files with "cat 002|tail -c +2|iconv -f utf-16be" (utf-16le in last case have a similar but worse result).

I made this on home computer with debian. When i tried to start iconv on sailfish it says only "illegal input sequence at position 0".

fLegmatik ( 2017-04-18 20:30:49 +0200 )edit

1

Can this workaround be scheduled, and how? If the problem can be "fixed" this way, why isn't it implemented yet? Does this workaround include any risks?

Also, how do I find out which SIM folder is the right one? The ones that end with "-3" are ofono's cache folders.

MMori ( 2018-03-08 13:24:32 +0200 )edit

Answer 3

4

answered 2015-11-02 23:31:53 +0200

SaimenSays

6943 ●92 ●146 ●141

Are there any news on solving this bug? I've talked to my peoble because of this behaviour and they have this mixed toghter SMS problems only with me. So it seems to be jolla dependent.

But I've some new details: For me it only happens when there are smilies in received SMS. So there may be some problems with encoding.

Do you use Emoji keyboard? Is there maybe some bug with this tool? I've deinstalled it now, and will have a look if it helps. Is there any native alternative to get a smily keyboard?

edit flag offensive delete publish link

Comments

I met this problem a week ago. I got a message hard to understand and I could check it from the sender. The original message in Nokia 515 phone had 283 characters of which one was smiley. I do not have any extra Emoji keyboard installed. I checked the Events table from ./local/shared/commhistory/commhistory.db with sqlite3 to analyze the messages. The old message had 150 unique characters in the beginning and 73 common characters to the new one. The new message had 153 unique characters in the beginning and 73 common to the old one. No smileys were transferred. The startTime column in Events table of the old message was unixtime 14333933563 (roughly June15). The unix time of the new message was 1466432966 (roughly June16). I noticed that the older message had in the lastModified column in the Events view the value "1970-01-01T02:00:00.000". The same column in the newer message was empty. At the time of the bug the sender had sent to me 953 messages. I have the latest Taalojärvi installed. On the time of the old message either Äijänpäivänjärvi or Aaslakkajärvi. I haven't used any early access updates in my phone.

Rikujolla ( 2016-06-27 08:37:48 +0200 )edit

Thanks for detailed inspection! So it really seems to depend on smileys. Maybe it would be better to convert your comment into an answer, so more people will notice it and check what can be done to fix it...

SaimenSays ( 2016-06-27 19:04:26 +0200 )edit

The sender sent to me two test messages with smileys again. Now smileys were transferred to characters nicely without any problems. The first message was less than 20 characters long and the other was the a same 283 character message which was messed like I told in earlier message. So I can't reproduce the error with smileys again. So for me it seems that the smiley may or may not be a problem:(. Hopefully the reason can be found.

Rikujolla ( 2016-06-28 09:09:09 +0200 )edit

I can also confirm that this cannot be reproduced easily. Sometimes messages are received correctly, sometimes with older fragments and dometimes not at all. Maybe you can try to re-send a message in question again identically.

SagaciousT ( 2016-06-28 09:25:34 +0200 )edit

Can not make it again. Have to wait the next event. Thanks for you for reporting that bug. Have to check this bug before blaming the sender of sending messy messages:)

Rikujolla ( 2016-06-28 10:27:22 +0200 )edit

Answer 4

3

answered 2015-08-18 18:00:30 +0200

This post is a wiki. Anyone with karma >75 is welcome to improve it.

updated 2015-08-19 18:42:01 +0200

Yo
2799 ●40 ●56 ●69

This is not an answer, but a poll. I have the feeling that this only happens when you

have tempered with the database or
have very many messages in the conversation.

Please vote in the comments below (by clicking the arrow icon).

BTW: This is a wiki page so no one gets karma for upvotes etc.

edit flag offensive delete publish link

Comments

20

Vote for "I have never tempered with the database but I see the weird mixing-up behaviour."

Yo ( 2015-08-18 18:01:27 +0200 )edit

Vote for "I have tempered with the database and I see the weird mixing-up behaviour."

Yo ( 2015-08-18 18:01:50 +0200 )edit

4

Vote for "I have only a few messages but I see the weird mixing-up behaviour."

Yo ( 2015-08-18 18:02:25 +0200 )edit

10

Vote for "I have a lot of messages and I see the weird mixing-up behaviour."

Yo ( 2015-08-18 18:02:44 +0200 )edit

2

Vote for "I have a lot of messages and never saw the weird mixing-up behaviour."

Yo ( 2015-08-18 18:03:13 +0200 )edit

Answer 5

2

answered 2016-10-07 16:27:46 +0200

Conrad
201 ●4 ●6 ●10

updated 2016-10-07 16:30:48 +0200

Interesting. So I've experienced this problem a few times, most recently today (20161007), and the mix-up always seems to be from the same conversation (i.e. the old SMS that gets mixed in comes from the same contact). That does argue that perhaps it's an originator error rather than Sailfish.

Both people I've noticed it with have Android 'phones. The messages involved don't contain emoji.

edit flag offensive delete publish link

Comments

In my case there are no android device. I am sure sms messages generated by some software. And i have problem with only one sender. Same time other phone can receive sms without problem. So i am sure the reason in Jolla phone or Jolla phone + some conditions. I want to find reason, but i don't know how to do it. I asked here - https://together.jolla.com/question/144812/how-to-debug-sms/, but had no answers... Maybe someone here can help me debug sms messages?

nauevg ( 2016-10-08 17:41:03 +0200 )edit

it happened to me today, with a contact with an iphone 6.

Simon ( 2016-10-21 02:03:36 +0200 )edit

Why can't we have a simple SMS functionality for Jolla ... it should show New Message , Received , Sent , Draft & Forwarding SMS , multiple selection to deleting sms etc... Jolla developers can you please confirm whether this can be implemented in next update

ashiagra ( 2017-04-11 10:13:35 +0200 )edit

I only get it SFOS -> SFOS, but we exchange ~800 msgs/month.

Tanghus ( 2020-05-24 14:06:00 +0200 )edit

Answer 6

1

answered 2015-09-18 06:11:40 +0200

MattVogt

2041 ●11 ●26 ●34

This is also not an answer, but a request for more information.

Android has an issue where it can keep obsolete incoming message fragments, and incorporate them into the result it delivers for a new incoming message: https://code.google.com/p/android/issues/detail?id=17769 https://code.google.com/p/android/issues/detail?id=28697

So, can anyone provide evidence that this problem has occurred where the receiving device is not an Android device?

edit flag offensive delete publish link

Comments

In my case I only write sms with android contacts. So I can not prove your question. Anyway your link is a bug from 2011 and android 2.0. So it should be fixed on actual devices, or is this bug still alive?

SaimenSays ( 2015-09-18 17:41:25 +0200 )edit

There's no indication that this bug has been subsequently fixed in Android, the bug is still open and people are still using the workaround application. If you have any information showing that it has been fixed, that would be helpful - please provide a link.

MattVogt ( 2015-09-19 00:30:54 +0200 )edit

I often have this issue from one contact that has an Android phone. Yesterday, I've encountered it the first time from an iOS phone.

SagaciousT ( 2016-05-31 14:38:50 +0200 )edit

I have never seen this problem with incoming sms (95% of my sms is with my girlfriend with an IOS) but she has seen corrupted messages from me.

jjvdgeer ( 2017-02-22 22:34:20 +0200 )edit

I only get it SFOS -> SFOS, but we exchange ~800 msgs/month. Some very long.

Tanghus ( 2020-05-24 14:07:18 +0200 )edit

[Bug] Received SMS mixed together

Comments

6 Answers

Comments

Comments

Comments

Comments

Comments

Comments

Question tools

Stats

Related questions

[Bug] Received SMS mixed together

Comments

6 Answers

Comments

Comments

Comments

Comments

Comments

Comments

Question tools

Public thread

Stats

Related questions