Huge amount of image files (i.e. cached map tiles) slows down or blocks MTP connection

asked 2015-07-22 10:21:13 +0300

wanderer gravatar image

updated 2015-07-24 09:54:36 +0300

For a few months I had troubles with connecting my Jolla to a Notebook or PC via MTP. Most times it took a minute or more until the MTP device showed up in the file manager, often the connection even broke up before it could be established. This was the case for my Linux Laptop, on a Windows machine the connection didn't work at all.

Yesterday I investigated this issue a little bit and I found the root for this troubles. It is the huge amount of map tile images which is stored in the internal memory when you use a map app like modrana, maep or JKartta. It looks like if the process msyncd, which handles the mtp connection, tries to make a thumbnail for every image file it finds. I was using modrana very heavily in the last months, so I had over 83000 files in the folder /home/nemo/.local/harbour-modrana/maps/. This is a huge amount of files msyncd has to deal with, and it slows it down critically. I deleted all the files in this folder and since this the MTP connection works fine again, on Linux as well as on windows.

If you have troubles with MTP, it is likely that you have this or a similar problem. It can be caused when you use a third party map application like modrana, maep, JKartta. Look for the cached data of this applications in the folder /home/nemo/.local/. With modrana, you can prevent this issue by storing the map tiles in SQLite databases instead of files (Setting in Menu -> Options -> Map).

Anyway, I think this is not a bug of the map apps, but of the msyncd process and the way MTP works on sailfish. Maybe it makes sense to isolate the /home/nemo/.local/ folder from MTP.

A few more technical details:

I saw that the process msyncd had a high CPU und RAM usage when I tried to make an MTP connection. To investigate this I connected to my Jolla via ssh (over WLAN), stopped the msyncd process via systemctl --user stop msyncd and started it directly via the command msyncd. By doing this I saw the output of the process with thousands of lines like this:

[W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for::  "file:///home/nemo/.local/share/harbour-modrana/maps/mapquest_eu/18/142193/91765.png"

It looks like if msyncd tries to make a thumbnail for every single image file it finds in the storage.

Edit 23.07.2015 - Further Details:

@Manatus had the idea that the MTP stack could work together with the tracker process. That doesn't seem to be the case. The MTP stack uses the process "tumblerd" to create thumbnails. This process is only active when the phone is connected via MTP and has a high CPU consumption as well during establiching the connection. I looked a little bit into the source code of the MTP stack (As ar as I could get with my lack of programming skills) and I found an interesting (but awkward) fact here:

# buteo-mtp can use org.freedesktop.thumbnails.Thumbnailer1to create
# thumbnails on request; at least Windows 8 requires thumbnails to be
# generated on the device.
Requires: tumbler

I found that the thumbnails are stored in the folder ~/.thumbnails on the phone. To look what happens, I deleted all the thumbnails in it, started msyncd manually as described above and connected the phone via msyncd. On the first connection I got the output

[W] meegomtp1dot0::Thumbnailer::recordRequest:141 - Thumbnailer::Exceeded max req size!

about 800 times. At the end the process crashed with

Error in 'msyncd': malloc(): memory corruption (fast): 0x44b2ed42

After restarting msyncd I got many error messages like

[W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for::  file:///home/nemo/android_storage/.aptoide/icons/47ec56493a130bff7758143e99a318ac_fgraphic_705x345.jpg

but at least the MTP connection was there after about half a minute. It looks like if the thumbnail storage exceeds a critical limit.

edit retag flag offensive close delete

Comments

Does it help if you create a file called .nomedia to the directory /home/nemo/.local/share/harbour-modrana?

This could work if MTP depends on tracker process in finding all of the images on device.

Manatus ( 2015-07-22 10:47:29 +0300 )edit
2

@Manatus: Wasn't the .nomedia file just an Android thing ? Provided it does help, I can make modRana add it by default.

BTW, on a general note - I have already considered MTP to be brain-dead stupid, but this (making thumbnail of every image file on the filesystem) takes this to a whole new level.

MartinK ( 2015-07-22 11:09:32 +0300 )edit
1

I think you are searching for an option to blacklist your modrana folder?! If yes then have a look here: usr/share/mtp/blacklist-home.conf

Alex ( 2015-07-22 11:10:28 +0300 )edit
1

@MartinK According to this question Sailfish uses it too. I think it didn't do that in early releases.

Manatus ( 2015-07-22 12:26:20 +0300 )edit
1

Removing the entire $HOME/.local from MTP is not a good idea as that might contain some user editable data (browser bookmarks, desktop files, application plugins, etc.) -- rarely needed though, but very similar to $HOME/.config. The separation between config and data is not always clear. I'd rather say implicitly downloaded map tiles belong under $HOME/.cache, which is blacklisted by default.

otsaloma ( 2015-07-22 16:53:50 +0300 )edit