Revision history [back]
 | 1 | initial version | posted 2015-07-22 10:21:13 +0200 |
Huge amount of image files (i.e. cached map tiles) slows down or blocks MTP connection
For a few months I had troubles with connecting my Jolla to a Notebook or PC via MTP. Most times it took a minute or more until the MTP device showed up in the file manager, often the connection even broke up before it could be established. This was the case for my Linux Laptop, on a Windows machine the connection didn't work at all.
Yesterday I investigated this issue a little bit and I found the root for this troubles. It is the huge amount of map tile images which is stored in the internal memory when you use a map app like modrana, maep or JKartta. It looks like if the process msyncd, which handles the mtp connection, tries to make a thumbnail for every image file it finds. I was using modrana very heavily in the last months, so I had over 83000 files in the folder /home/nemo/.local/harbour-modrana/maps/. This is a huge amount of files msyncd has to deal with, and it slows it down critically. I deleted all the files in this folder and since this the MTP connection works fine again, on Linux as well as on windows.
If you have troubles with MTP, it is likely that you have this or a similar problem. It can be caused when you use a third party map application like modrana, maep, JKartta. Look for the cached data of this applications in the folder `/home/nemo/.local/'. With modrana, you can prevent this issue by storing the map tiles in SQLite databases instead of files (Setting in Menu -> Options -> Map).
Anyway, I think this is not a bug of the map apps, but of the msyncd process and the way MTP works on sailfish. Maybe it makes sense to isolate the /home/nemo/.local/ folder from MTP.
A few more technical details: I saw that the process msyncd had a high CPU und RAM usage when I tried to make an MTP connection. To investigate this I stopped the msyncd process via systemctl and started it directly via ssh command line. By doing this I saw the output of the process with thousands of lines like this:
[W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for:: "file:///home/nemo/.local/share/harbour-modrana/maps/mapquest_eu/18/142193/91765.png"
It looks like if msyncd tries to make a thumbnail for every single image file it finds in the storage.
| | 2 | No.2 Revision |
Huge amount of image files (i.e. cached map tiles) slows down or blocks MTP connection
For a few months I had troubles with connecting my Jolla to a Notebook or PC via MTP. Most times it took a minute or more until the MTP device showed up in the file manager, often the connection even broke up before it could be established. This was the case for my Linux Laptop, on a Windows machine the connection didn't work at all.
Yesterday I investigated this issue a little bit and I found the root for this troubles. It is the huge amount of map tile images which is stored in the internal memory when you use a map app like modrana, maep or JKartta. It looks like if the process msyncd, which handles the mtp connection, tries to make a thumbnail for every image file it finds. I was using modrana very heavily in the last months, so I had over 83000 files in the folder /home/nemo/.local/harbour-modrana/maps/. This is a huge amount of files msyncd has to deal with, and it slows it down critically. I deleted all the files in this folder and since this the MTP connection works fine again, on Linux as well as on windows.
If you have troubles with MTP, it is likely that you have this or a similar problem. It can be caused when you use a third party map application like modrana, maep, JKartta. Look for the cached data of this applications in the folder `/home/nemo/.local/'. /home/nemo/.local/. With modrana, you can prevent this issue by storing the map tiles in SQLite databases instead of files (Setting in Menu -> Options -> Map).
Anyway, I think this is not a bug of the map apps, but of the msyncd process and the way MTP works on sailfish. Maybe it makes sense to isolate the /home/nemo/.local/ folder from MTP.
A few more technical details: I saw that the process msyncd had a high CPU und RAM usage when I tried to make an MTP connection. To investigate this I stopped the msyncd process via systemctl and started it directly via ssh command line. By doing this I saw the output of the process with thousands of lines like this:
[W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for:: "file:///home/nemo/.local/share/harbour-modrana/maps/mapquest_eu/18/142193/91765.png"
It looks like if msyncd tries to make a thumbnail for every single image file it finds in the storage.
| | 3 | No.3 Revision |
Huge amount of image files (i.e. cached map tiles) slows down or blocks MTP connection
For a few months I had troubles with connecting my Jolla to a Notebook or PC via MTP. Most times it took a minute or more until the MTP device showed up in the file manager, often the connection even broke up before it could be established. This was the case for my Linux Laptop, on a Windows machine the connection didn't work at all.
Yesterday I investigated this issue a little bit and I found the root for this troubles. It is the huge amount of map tile images which is stored in the internal memory when you use a map app like modrana, maep or JKartta. It looks like if the process msyncd, which handles the mtp connection, tries to make a thumbnail for every image file it finds. I was using modrana very heavily in the last months, so I had over 83000 files in the folder /home/nemo/.local/harbour-modrana/maps/. This is a huge amount of files msyncd has to deal with, and it slows it down critically. I deleted all the files in this folder and since this the MTP connection works fine again, on Linux as well as on windows.
If you have troubles with MTP, it is likely that you have this or a similar problem. It can be caused when you use a third party map application like modrana, maep, JKartta. Look for the cached data of this applications in the folder /home/nemo/.local/. With modrana, you can prevent this issue by storing the map tiles in SQLite databases instead of files (Setting in Menu -> Options -> Map).
Anyway, I think this is not a bug of the map apps, but of the msyncd process and the way MTP works on sailfish. Maybe it makes sense to isolate the /home/nemo/.local/ folder from MTP.
msyncd. By doing this I saw the output of the process with thousands of lines like this: [W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for:: "file:///home/nemo/.local/share/harbour-modrana/maps/mapquest_eu/18/142193/91765.png"
It looks like if msyncd tries to make a thumbnail for every single image file it finds in the storage.
Edit 23.07.2015 - Further Details:
@Manatus had the idea that the MTP stack could work together with the tracker process. That doesn't seem to be the case. The MTP stack uses the process "tumblerd" to create thumbnails. This process is only active when the phone is connected via MTP and has a high CPU consumption as well during establiching the connection. I looked a little bit into the source code of the MTP stack (As ar as I could get with my lack of programming skills) and I found an interesting (but awkward) fact here:
# buteo-mtp can use org.freedesktop.thumbnails.Thumbnailer1to create # thumbnails on request; at least Windows 8 requires thumbnails to be # generated on the device. Requires: tumblerI found that the thumbnails are stored in the folder
~/.thumbnailson the phone. To look what happens, I deleted all the thumbnails in it, started msyncd manually as described above and connected the phone via msyncd. On the first connection I got the output[W] meegomtp1dot0::Thumbnailer::recordRequest:141 - Thumbnailer::Exceeded max req size!about 800 times. At the end the process crashed with*** Error in 'msyncd': malloc(): memory corruption (fast): 0x44b2ed42 ***. After restarting msyncd I got many error messages like[W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for:: "file:///home/nemo/android_storage/.aptoide/icons/47ec56493a130bff7758143e99a318ac_fgraphic_705x345.jpg", but at least the MTP connection was there after about half a minute. It looks like if the thumbnail storage exceeds a critical limit.
| | 4 | No.4 Revision |
Huge amount of image files (i.e. cached map tiles) slows down or blocks MTP connection
For a few months I had troubles with connecting my Jolla to a Notebook or PC via MTP. Most times it took a minute or more until the MTP device showed up in the file manager, often the connection even broke up before it could be established. This was the case for my Linux Laptop, on a Windows machine the connection didn't work at all.
Yesterday I investigated this issue a little bit and I found the root for this troubles. It is the huge amount of map tile images which is stored in the internal memory when you use a map app like modrana, maep or JKartta. It looks like if the process msyncd, which handles the mtp connection, tries to make a thumbnail for every image file it finds. I was using modrana very heavily in the last months, so I had over 83000 files in the folder /home/nemo/.local/harbour-modrana/maps/. This is a huge amount of files msyncd has to deal with, and it slows it down critically. I deleted all the files in this folder and since this the MTP connection works fine again, on Linux as well as on windows.
If you have troubles with MTP, it is likely that you have this or a similar problem. It can be caused when you use a third party map application like modrana, maep, JKartta. Look for the cached data of this applications in the folder /home/nemo/.local/. With modrana, you can prevent this issue by storing the map tiles in SQLite databases instead of files (Setting in Menu -> Options -> Map).
Anyway, I think this is not a bug of the map apps, but of the msyncd process and the way MTP works on sailfish. Maybe it makes sense to isolate the /home/nemo/.local/ folder from MTP.
A few more technical details:
I saw that the process msyncd had a high CPU und RAM usage when I tried to make an MTP connection. To investigate this I connected to my Jolla via ssh (over WLAN), stopped the msyncd process via systemctl --user stop msyncd and started it directly via the command msyncd. By doing this I saw the output of the process with thousands of lines like this:
[W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for:: "file:///home/nemo/.local/share/harbour-modrana/maps/mapquest_eu/18/142193/91765.png"
It looks like if msyncd tries to make a thumbnail for every single image file it finds in the storage.
Edit 23.07.2015 - Further Details:
@Manatus had the idea that the MTP stack could work together with the tracker process. That doesn't seem to be the case. The MTP stack uses the process "tumblerd" to create thumbnails. This process is only active when the phone is connected via MTP and has a high CPU consumption as well during establiching the connection. I looked a little bit into the source code of the MTP stack (As ar as I could get with my lack of programming skills) and I found an interesting (but awkward) fact here:
# buteo-mtp can use org.freedesktop.thumbnails.Thumbnailer1to create # thumbnails on request; at least Windows 8 requires thumbnails to be # generated on the device. Requires: tumbler
I found that the thumbnails are stored in the folder ~/.thumbnails on the phone. To look what happens, I deleted all the thumbnails in it, started msyncd manually as described above and connected the phone via msyncd. On the first connection I got the output output
[W] meegomtp1dot0::Thumbnailer::recordRequest:141 - Thumbnailer::Exceeded max req size! size!
about 800 times. At the end the process crashed with with***
Error in 'msyncd': malloc(): memory corruption (fast): 0x44b2ed42 ***. 0x44b2ed42
After restarting msyncd I got many error messages like like
[W] meegomtp1dot0::Thumbnailer::slotError:107 - Thumbnailer returned error for:: "file:///home/nemo/android_storage/.aptoide/icons/47ec56493a130bff7758143e99a318ac_fgraphic_705x345.jpg"
, file:///home/nemo/android_storage/.aptoide/icons/47ec56493a130bff7758143e99a318ac_fgraphic_705x345.jpg
but at least the MTP connection was there after about half a minute. It looks like if the thumbnail storage exceeds a critical limit.