We have moved to a new Sailfish OS Forum. Please start new discussions there.
11

[Bug] Media doesn't find all music files correctly

asked 2015-09-30 16:29:43 +0200

Alex gravatar image

updated 2016-08-10 14:00:58 +0200

Affected Sailfish versions:

<= 2.0.2.48 Aurajoki

This bug does not just exist since 1.1.9.28 (time of report) but is already older and really annoys me.

Bug description & use case:

This bug appears everytime after transfering new Music (full Albums in Folders) to my 64GB micro SD card.

This bug causes the issue that after the successful music file transfer you won't find all the music files the Albums/Folders actually contain in the Album overview of the native Media app and that most of the newly added music files are not playable. Some part of the music files will be indexed without a problem with a working album cover, correctly shown id-tagged files and everything how it should work.

You will find some missing music files in an 'Unknown Album' in medias album overview with a wrong music duration of usually '0:00', an missing Artist name, a missing Cover and a messed up name with e.g. some numbers (e.g. 01) in front of the Title name but sometimes you are able to play them normally out of this 'Unknown Album'.

I have to manually re-build my media database to fix this issue everytime because a reboot doesn't fix this issue.

How to reproduce:

So you need a fully id-tagged Album with all music files in one folder. Also IMO there should be >10 music files in the folder so that the chances are better to reproduce this bug (even though I don't see there any pattern in correlation with the amount of music files). My music files usually are tagged with an Album cover, the Artist/Title name, the Album name, the track number, the year of release and the genre. I also tranfering my music via WinSCP so you need the SSH access to your Jolla activated but this shouldn't be necessary to reproduce this bug.

Also it seems that this bug is only reproducable after transfering many media files at once onto your device or only if your device already contains a significant amount of music/media files before the transfer of one album for reproduction. At the moment my Jolla contains about 10000 images listed in gallery and about 2000 mp3-files listed in media app (all media is located on my SD card).

  1. Connect your SSH client with your Jolla (I am using WinSCP)
  2. Navigate to your SD card (mine is a 64 GB Samsung microSD card formatted in FAT32)
  3. Drop your Album folder into another folder on your SD card, e.g. the Music folder
  4. Wait a few mintues to let the tracker index the music files
  5. Open the native Media App and navigate to the Album view (the album id tag is needed)
  6. You should find your Album in the Album list view - open your Album and mostly you will find out that some files are missing there. If you will notice some missing files also have a look in your 'Unknown Album' if you will finde some of them there with a 'half-indexed' messed naming etc.

Workarounds:

  • I have to execute tracker-control -es in terminal to fix this issue so that all music files will be indexed correctly
  • Use the 'Restore media database' option of the native 'Utilities' App from Jolla store (option is in Settings > Utilities)

Expected behaviour:

After transfering your favourite music onto your Jolla you shouldn't care about rebuilding your tracker database but you should just find all of your music correctly indexed inside of your native Media App. :)

Additional Information:

After indexing the files journalctl contains many of the following errors (but not for each added music file), even though the music is tagged:

[root@Sailfish nemo]# journalctl --no-pager | grep "tracker"
Aug 10 11:57:06 Sailfish tracker-extract[1418]: GLIB WARNING ** Tracker - Task for 'file:///path/to/file.mp3.filepart' finished with error: Could not get any metadata for uri:'file:///path/to/file.mp3.filepart' and mime:'audio/mpeg'
Aug 10 11:57:07 Sailfish tracker-miner-fs[1423]: GLIB WARNING ** Tracker - No urn for file:///path/to/file.mp3.filepart
Aug 10 11:57:08 Sailfish tracker-extract[1418]: GLIB CRITICAL ** libmediaart - media_art_set: assertion 'title != NULL' failed
Aug 10 11:57:08 Sailfish tracker-extract[1418]: GLIB WARNING ** Tracker - Could  not process media art for 'file:///path/to/file.mp3', No error given
Aug 10 11:57:10 Sailfish tracker-extract[1418]: GLIB CRITICAL ** libmediaart - media_art_set: assertion 'title != NULL' failed
Aug 10 11:57:29 Sailfish tracker-extract[1418]: GLIB WARNING ** Tracker - Could not process media art for 'file:///path/to/file.mp3', Title is required, but was not provided, or was empty
Aug 10 11:57:48 Sailfish tracker-extract[1418]: GLIB WARNING ** Tracker - Task for 'file:///path/to/file.mp3.filepart' finished with error: Could not get any metadata for uri:'file:///path/to/file.mp3.filepart' and mime:'audio/mpeg'
Aug 10 11:57:53 Sailfish tracker-extract[1418]: GLIB WARNING ** Tracker - Could not process media art for 'file:///path/to/file.mp3.filepart', Error when getting information for file '/path/to/file.mp3.filepart': File or directory not found
edit retag flag offensive close delete

Comments

Does the file format of the audio files make a difference? Or if you always use the same: which one?

ossi1967 ( 2015-09-30 16:47:05 +0200 )edit
1

@ossi1967 I am just using .mp3-files so I can't make any statement about other file formats.

Alex ( 2015-09-30 16:54:51 +0200 )edit

can not confirm. however, what i do notice when adding maaany files at once is that at some point you'll think tracker has finished even though the count shows it's still a couple of hundreds songs short. that's where i just wait and let it do whatever it is that it does there and after a while my collection is complete.

you say a simple tracker restart totally fixes your issue, which means your files must indeed have been tagged and transferred flawlessly all along. if you really want to get to the bottom of this try verbose from terminal, if you're lazy try the wait.

veloxneb ( 2015-10-01 13:47:13 +0200 )edit

wouaw! great report!
@Alex do you know (or could you know) if each files may have different compression?
After looking at the log info, maybe the problem is by the format of the name of some file or their metadata.
If you have the possibility to change it and avoid some special characters.
i don't know which one could be wrong...you may test it a little bit.
I guess that you download some of these files, as i remember the *.filepart is a file downloaded as long as it is not finished. Maybe some of them with *.filepart are not completely downloaded and some data are missed.

cemoi71 ( 2016-08-16 17:46:40 +0200 )edit

3 Answers

Sort by » oldest newest most voted
5

answered 2016-08-16 17:33:44 +0200

DaveRo gravatar image

updated 2016-08-30 18:34:28 +0200

New Jolla C user (2.0.0.43) I transferred about 50GB of mostly mp3s onto a empty 64GB card - and took it on holiday. 7300 songs according to media. Hundreds, possibly thousands, are in 'unknown'. All visible, tagged, playable from file manager - but that cannot play a whole album (directory). Tried some of the reindexing commands - no effect.

On returning home I reloaded most of the music, a few albums at a time, using sftp. All went well until I loaded a batch including some Jacques Brel albums. Some of these put all tracks into 'Unknown Album' 'Various artists' and with zero length. Subsequent albums, loaded in the same batch also went into unknown. I deleted the Brel and the rest indexed OK.

What these Brel albums have in common is accented characters in the tags. I've tried rewriting the tags with easytag but haven't so far got them to index. I have other albums with accented characters, so it's not a problem with accented characters per se. My guess is it's something to do with character encoding. Compare these exiftool outputs - the first from the offending Brel album, the second from an album that indexes OK

File Name                       : 02 - Dites, si c'était vrai (poème).mp3
...
ID3 Size                        : 221
Title                           : Dites, si c'�tait vrai (po�me)
Artist                          : Jacques Brel
Album                           : Chansons ou versions in�dites de Jeunesse
...
Encoder Settings                : Lavf55.19.104
Audio Bitrate                   : 168 kbps
Duration                        : 0:01:13 (approx)
====================================================
File Name                       : 10 - Rêve àdemi.mp3
...
ID3 Size                        : 1268
Title                           : Rêve àdemi
Artist                          : Gabriel Yacoub
Album                           : Babel
...                      : 
Audio Bitrate                   : 157 kbps
Date/Time Original              : 1997
Duration                        : 0:04:57 (approx)

The Brel tag appears to be in Latin-1, and the Yacoub is a two-byte code, so presumably unicode.

I've given up for now. Maybe I'll try removing the ID3 tags completely and rebuilding them. I'll keep a copy of the original album files in case anybody wants to investigate further.

Edit 1: same day

Set easytag to (re)write ID3v2.3 and unicode. (I don't know what they were before - I've had these files for >10 years.) Also removed the é in the file name - though I have other directory and file names with accented characters in them. Now indexes OK.

Edit 2: 30-8-16

I examined the Brel album that appeared to cause the problem with MP3 Diags. It reported:
No ID3V2.3.0 tag found, although this is the most popular tag for storing song information.
ID3V2 tag doesn't have an APIC frame (which is used to store images).

Tag: ID3V2.4.0
padding=10, unsynch=NO; frames: TIT2="A deux", TPE1="Jacques Brel", TALB="Chansons ou versions in�dites de Jeunesse", TRCK="01", TXXX="Year 1983", TCON="Soundtrack", TSSE="Lavf55.19.104"

It made no comment about the character coding. But the é glyph doesn't display on this system. So maybe that's a red herring.

edit flag offensive delete publish link more

Comments

1

+1 i find that is an interesting remark regarding the format of metadata...
i'm curious about the result be removing the id3 tags or reformat of it...

cemoi71 ( 2016-08-16 17:50:52 +0200 )edit

@DaveRo regarding your edit2, did you try to change the the one character or delete it?

cemoi71 ( 2016-08-30 19:17:01 +0200 )edit

do you have any information on which type it is encoded? this encoding byte which reveal if it is.
$00 – ISO-8859-1 (LATIN-1, Identical to ASCII for values smaller than 0x80).
$01 – UCS-2 (UTF-16 encoded Unicode with BOM), in ID3v2.2 and ID3v2.3.
$02 – UTF-16BE encoded Unicode without BOM, in ID3v2.4.
$03 – UTF-8 encoded Unicode, in ID3v2.4.
some infos are there:
https://en.wikipedia.org/wiki/ID3#cite_note-8
http://id3.org/id3v2-00
but i didn't understand how it works until now....

cemoi71 ( 2016-08-30 19:34:27 +0200 )edit

@cemoi71 No. I just used easytag to rewrite the tags for the whole folder (12 albums, some of them in a subfolder). Easytag was set to unicode. But I kept a copy of just one - the first one that went into unknown* - in case anyone wanted to diagnose the problem - and that's the one I just examined.

A hex editor shows that the é is 0xE9 which is correct in Latin-1 ISO 8859-1. (I wonder why neither exiftools nor MP3 diags displays the glyph? Probably a simple explanation.)

*Though that assumes that the tracker processed them in the order that I see them in my file manager, which it may not.

Edit: I can stick the album in dropbox if you want.

DaveRo ( 2016-08-30 19:40:42 +0200 )edit

don't if it make sense to have it. Because seems to me that you have more understanding about the stuff, and i would comes to the same result. Seems to me that the multimedia app has a bug. what do you think about it?
would be interesting to see the log of indexing as alex get.

cemoi71 ( 2016-08-31 12:00:35 +0200 )edit
1

answered 2015-09-30 20:19:34 +0200

ChristophAugen gravatar image

updated 2015-09-30 20:43:21 +0200

sorry just seen you have this problem with ssh. I experienced it only copying music via usb. with ssh it works fine

edit flag offensive delete publish link more

Comments

AFAIK if you transfer media files over USB with MTP it tries to do some media identification while the files are being transferred to make them show up faster in the media playing apps. On the other hand if you transfer files over ssh they will be scanned by tracker once it detects there are new files.

So it is quite possible that one of the detection mechanisms is broken (the MTP one) while the other one (the tracker one) works fine.

MartinK ( 2016-08-10 15:05:41 +0200 )edit

Jolla C owner here. I tried the reindexing commands too. No effects.

Francesco ( 2016-08-11 20:40:20 +0200 )edit

For me, reindexing by restarting tracker (tracker-control -r, tracker-control -se, or from Sailfish Utilities “control panel”) also doesn't help at all. +1 for a fix. I also have many MP3 and OGG files with national characters but many of them get indexed well. I also suspect that even albums without national characters get indexed badly. So it could be an issue with some versions of the ID3 tags. As stated in another question, the OGG files are more inclined not to get indexed (or to get indexed badly).

Cigydd ( 2016-08-29 10:05:25 +0200 )edit

What I found - see my answer - was not that tracks with international characters didn't get indexed - many did - but that such a track (and having those characters non-unicode) triggered the fault and that lots of subsequent tracks failed indexing.

I posted some links here which might be useful.

DaveRo ( 2016-08-30 12:07:52 +0200 )edit

@DaveRo Thank you for your clarification. So one single file can cause a fatal fault that prevents subsequent files from being indexed. And thank you for the links. Currently investigating what the tools discover about my files and what they can repair.

Edit: I discovered by one of the problematic albums with the MP3 Diags software that there was no track info at all. What caused this was the FFmpeg encoder I used to convert the songs from the OGG Vorbis format. It just didn't preserve the metadata, which I'm used to be preserved by it. Will see what the other tracks are missing.

Cigydd ( 2016-08-30 13:15:46 +0200 )edit
0

answered 2016-08-16 19:37:19 +0200

cemoi71 gravatar image

updated 2016-08-16 19:39:59 +0200

@Alex did you see what DaveRo made, and how was he successfull by his issue?
for your case i would suggest you to reformat some filename and IDv3 tags.
And remove characters which are not standard.
And give a look deeper on files with *.filepart ending, maybe they are not supported (for me they are not completed).

edit flag offensive delete publish link more

Comments

1

I don't have the time to test this out now but also this could be only a workaround, but we need a fix for that issue.

Yea the files with the *.filepart ending shouldn't be tracked at all until they are transfered completely. I think they are coming from transfering my music via (win)scp. I think this could be at least a part of the whole issue, since the tracker should only index files that are completely transfered.

Alex ( 2016-08-19 12:25:06 +0200 )edit
Login/Signup to Answer

Question tools

Follow
2 followers

Stats

Asked: 2015-09-30 16:29:43 +0200

Seen: 1,333 times

Last updated: Aug 30 '16