Guide: Creating partitions on SD-card, optionally encrypted

asked 2018-12-17 00:23:26 +0300

olf gravatar image

updated 2020-08-28 19:21:47 +0300

This guide covers these use cases:

  • Reformatting a SDXC-card (pre-formatted with exFAT) to vFAT32
    Either perform this in the SailfishOS' settings app or (if this fails or if you want more control) walk through sections 1, 2.1, (omit 2.2.a), 2.2.b, 2.3, 3.1, 3.2.a (just set partition type to "c") and 4.1.
  • Reformatting an SD-card to a Linux-native filesystem (unencrypted or encrypted)
    Either perform this in the SailfishOS' settings app for unencrypted partitions or (if this fails, if you want more control or if you want to deploy an encrypted partition, avoiding the drawbacks of partitions encrypted with SailfishOS' 3.x.y own mechanism: no automatic mounting at boot time, no android_storage possible on encrypted SD-card etc.) walk through sections 1, 2.1, (you can omit 2.2.x), 3.1, 3.2.a, 4.2 (unencrypted) or 4.3 (encrypted) and finally 5.
  • Repartitioning an SD-card, deploying multiple filesystems (unencrypted and / or encrypted)
    As this is what this guide was developed for, there is no shortcut one can take (except for the obvious omission of section 3.2.a).
    Typical use cases / partitioning schemes are:
    • Create a small vFAT32 partition (e.g., slightly below 8 GB, as described in section 3.2.b) and a large second, encrypted partition.
    • Split the space on SD-card between an unencrypted and an encrypted partition.
    • Create a small vFAT32 partition (e.g., slightly below 8 GB, see section 3.2.b) and split the remaining space on SD-card between an unencrypted and an encrypted partition.
  • Alter an encrypted partition on SD-card created per Settings -> Storage -> Format: Encrypt (which has to be manually mounted) to be automatically mounted (when booting the device), while retaining the data on it
    As Jolla introduced its SD-card encryption with SailfishOS 3.0.0 (i.e., after this guide was written), this is provided as a short addendum (section 9), which references the few relevant sections.


This is an enhanced, updated and streamlined guide created from the original "[How-to] Creating partitions on SD-card, optionally encrypted". It only covers the most general use cases with SailfishOS 2.2.0 and newer.
For details, variations, technical background and SailfishOS versions before 2.2.0, please consult the original how-to. The section topics and numbers are mostly the same for both guides.

This is part 1 "Creating partitions on SD-card, optionally encrypted", you might want to continue with part 2 "Externalising android_storage and other directories / files to SD-card" to utilise the space on SD-card thereafter.

0. Objectives etc.

  1. Automatically mount all partitions (both, encrypted and unencrypted) on SD-card, when SailfishOS boots.
  2. Primarily rely on SailfishOS provided tools and perform the partitioning and formatting directly on the SailfishOS device (this avoids compatibility issues).
  3. Not altering SailfishOS' basic filesystem structure and avoid altering existing files. Thus no /home/nemo etc. on SD-card.
  4. But with the ability to move android_storage and other files and directories under /home/nemo to an (optionally encrypted) partition on SD-card (see part 2 for this).
  5. Interoperable with SailfishOS GUI apps, specifically the functions Backup and Storage in the Settings app.
  6. A solution for all devices with SailfishOS, so an SD-card can be moved between different devices, while keeping data on it accessible. Hence the filesystems used are primarily EXT4 and vFAT32, because this is what Jolla officially supports.
  7. Optionally protect partition(s) on SD-card by encryption, in order to keep the data on it safe, when the device is lost. I.e. the keys for DM-Crypt are based on a file stored at the internal mass store, which is accessible, when the SailfishOS device is unlocked. Additionally, one can copy that file to other SailfishOS devices or Linux PCs to access the data on the SD-card there, too.
  8. And / or create unencrypted partition(s) for data exchange with any device and OS (by physically moving the SD-card), plus as "spare area".
  9. Applicable to SDHC- and SDXC-cards of at least 4 GB (up to the maximum size the device hardware supports, as the technical limit of this scheme is 2 TBytes).
  10. Keeping the reserved area of an SD-card intact.
  11. Ensure that trim / discard commands are properly propagated through the whole storage stack (from the top filesystem layer to the storage device proper), no matter if using encryption or not.
  12. Aligning partition, DM-Crypt and filesystem data-structures to the FLASH memory's alignment size in order to prevent excessive write amplification, which would result in a vastly degraded lifetime and lower performance of a reformatted SD-card.

README 1st:
Before you start, please read the sections relevant for your use case completely and decide thereafter, if you (at least) roughly understand what these commands will do (you may read their man pages, to be sure).
Mistyping (e.g., /dev/mmcblk0 instead of /dev/mmcblk1) may brick your SailfishOS device or quickly wear out your SD-card.
All this was tested for years on Jolla 1 phones, Xperia Xs, XA2s and 10s, plus Gemini PDAs with SailfishOS versions from 2.2.0 to 3.2.1 and various SD-cards, still this guide may have flaws. Please report your experience in this case, but do not blame anyone (e.g., me) for mishaps.
Contributions and constructive suggestions are welcome.

1. Preparation (installing needed utilities etc.)

Starting a terminal session as user nemo in /home/nemo (as usual).

mkdir sd-card_layouts # Create a directory for backing up the original and altered partition and file system layouts of the SD-card
devel-su # Become root user

  • Only needed for SailfishOS 2.2.0: ssu ar mer-tools && pkcon refresh

pkcon install util-linux dosfstools # Install fdisk (from util-linux) and dosfsck (from dosfstools)

2. Analysing a specific SD-card

2.1 Analysing an SD-card's partition layout

cd sd-card_layouts # Change to the backup directory created in section 1

First take a look at the partition layout with fdisk:
fdisk -l /dev/mmcblk1 | tee fdisk-l_mmcblk1-original.txt
Output of a 32 GB SDHC-card as an example:

Disk /dev/mmcblk1: 29 GiB, 31104958464 bytes, 60751872 sectors
Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device         Boot Start      End  Sectors Size Id Type
/dev/mmcblk1p1       8192 60751871 60743680  29G  c W95 FAT32 (LBA)

2.2.a Analysing an existing FAT32 filesystem on an SDHC-card

To analyse an SDHC-card's pre-formatted FAT32 filesystem (advertised size typically "4 GB" to "32 GB"), use dosfsck:
dosfsck -nv /dev/mmcblk1p1 | tee dosfsck-nv_mmcblk1p1-original.txt
Exemplary output of aforementioned 32 GB SD-card:

dosfsck 3.0.10 (12 Sep 2010)
dosfsck 3.0.10, 12 Sep 2010, FAT32, LFN
Checking we can access the last sector of the filesystem
Boot sector contents:
System ID "        "
Media byte 0xf8 (hard disk)
       512 bytes per logical sector
     32768 bytes per cluster
     1556 reserved sectors
First FAT starts at byte 796672 (sector 1556)
         2 FATs, 32 bit entries
   3795968 bytes per FAT (= 7414 sectors)
Root directory start at cluster 2 (arbitrary size)
Data area starts at byte 8388608 (sector 16384)
    948864 data clusters (31092375552 bytes)
63 sectors/track, 255 heads
      8192 hidden sectors
  60743680 sectors total
Checking for unused clusters.
Checking free cluster summary.
/dev/mmcblk1p1: 3 files, 2507/948864 clusters

2.2.b Analysing an existing exFAT filesystem on an SDXC-card

To analyse an SDXC-card's pre-formatted exFAT filesystem (advertised size typically "64 GB" or larger), use dumpexfat (from the exfat-utils).
Install exfat-utils from OpenRepos (e.g., per Storeman), then execute:
dumpexfat /dev/mmcblk1p1 | tee dumpexfat_mmcblk1p1-original.txt
Exemplary output of an 128 GB SD-card:

Volume label                        
Volume serial number      0x64323432
FS version                       1.0
Sector size                      512
Cluster size                  131072
Sectors count              249704448
Free sectors               249670912
Clusters count                975280
Free clusters                 975277
First sector                   32768
FAT first sector               16384
FAT sectors count              16384
First cluster sector           32768
Root directory cluster             4
Volume state                  0x0000
FATs count                         1
Drive number                    0x80
Allocated space                   0%

2.3 Determining offsets and checking them for correctness

2.3.1 Determining crucial offsets and sizes

  • "Sector size" (dumpexfat) is equivalent to "X bytes per logical sector" (dosfsck). It must equal the "Sector size" displayed by fdisk in section 2.1.
  • "First sector" (dumpexfat) is equivalent to "hidden sectors" (dosfsck) and is the partition offset. It must equal the "Start" sector displayed by fdisk in section 2.1.
    Also use this value as alignment size in sectors.
  • "FAT first sector X" is equivalent to "First FAT starts at Y (sector X)" and "X reserved sectors" (for which dosfsck ought to display the same value), which are the X reserved sectors.
  • "First cluster sector X" is equivalent to "Data area starts at byte Y (sector X)", which is the data area offset.
  • "Cluster size X" is equivalent to "X bytes per cluster", which is the cluster size in Bytes. Divide this value by the sector size to obtain the cluster size in sectors, as all calculations in the following sections are carried out in sectors.

2.3.2 Checks

  1. The data area offset must be a multiple (i.e., 1, 2, 3, ... times) of the partition offset.
  2. The displayed sector size in sections 2.1 and 2.2 ought be consistently 512 Bytes (though supporting other sector sizes should theoretically be feasible).
  3. Check, if your SD-card supports discards:
    When cat /sys/block/mmcblk1/queue/discard_zeroes_data outputs "1" it definitely does, otherwise it likely does not.
  4. The alignment size must be a 2^n multiple (i.e., 1, 2, 4, 8, ... times) of the erase-block size.
    To determine an SD-card's erase-block size (may not be set, if discards are not supported), look at the output of cat /sys/block/mmcblk1/queue/discard_granularity and cat /sys/block/mmcblk1/device/preferred_erase_size, which shall provide the same value (in Bytes, not sectors!). Divide this value by the sector size and compare the result with the alignment size.
  5. Multiply the value X from "FAT sectors count X" / "Y bytes per FAT (= X sectors)" with the value N from "FATs count N" / "N FATs, 32 bit entries" and add the number of reserved sectors: The result must equal the data area offset.
  6. Multiply the value X from "Clusters count X" / "X data clusters (Y bytes)" with the cluster size and add the data area offset: The result must equal the "Sectors count" / "sectors total" and the "Sectors" (with a capital "S") displayed for this partition at the end of fdisk's output.

Interpreting the results:

  • If a single of these checks does not provide a positive result and the SD-card has never been used before, consider not to use it for any valuable data! I would prefer to use such an SD-card in embedded devices (e.g., in digital cameras or MP3-players), but not for computers.
  • If one or some of these checks (except for "3. Discard support" and consequently maybe also "4. Erase-block size") fail, because the SD-card has been re-partitioned before, you might use the SD-card formatter from the SD Association (Windows and MacOS only, unfortunately) to reformat the card completely to a correct partitioning scheme.
  • If you cannot or do not want to use the aforementioned "official" SD-card formatter, you may continue, using the erase-block size as a rough estimate of the alignment size as last resort.

3. Re-partitioning an SD-card

Note, that all files on the SD-card will become inaccessible when carrying out the next steps, so now is the last chance to backup data residing on your SD-card.

3.1 Preparation for re-partitioning

Check, which partition(s) on SD-card is actually mounted and how often:
mount | grep mmcblk1 # Each partition is mounted twice (at different locations), if you have AlienDalvik running
Unmount the mounted partition(s) on the SD-card by issuing, e.g.:
umount /dev/mmcblk1pX # If the output is "umount: /dev/mmcblkXXX: not mounted", you picked the wrong partition
mount | grep mmcblk1 # Check again, it should be mounted once less, now
Continue to issue umount /dev/mmcblk1pX until the output is "umount: /dev/mmcblkXXX: not mounted" for all partitions on this SD-card (and consequently mount | grep mmcblk1 outputs nothing).

Discard ("trim") all data and metadata on the partition(s), on which the old filesystem(s) reside(s):
blkdiscard -v /dev/mmcblk1pX # Pay attention not to accidentally select the whole device /dev/mmcblk1

Become familiar with the fdisk tool:
fdisk /dev/mmcblk1

  • p lists the partition layout as obtained in section 2.1.
  • m provides a list of available commands.
  • Do not set the "DOS compatibility flag" (with the command c), which is solely needed for computers running a real MS-DOS and is really bad for FLASH-based storage (but it defaults to "off" in GNU fdisk since very long ago: more than 15 years).
  • If you have the impression something went wrong, simply exit fdisk (without writing the altered partition table) via q and start anew.

3.2.a Setting up a single FAT32, Linux-native or encrypted partition

If you do not want to create two or three partitions, just leave the partition layout as it is and solely alter the partition type (even that is optional): Hit t, by default the only existing partition is selected and set its partition type to 83 ("Linux") for an unencrypted partition, e8 ("LUKS") for an encrypted one or if you just want to reformat an SDXC-card (pre-formatted with exFAT) with FAT32 to c.
Check by entering p that the partition table now looks as it should, and finally write it to SD-card (and exit fdisk) via w (or q, if unaltered).

3.2.b Setting up two or three partitions: A FAT32 or native one plus a native or / and encrypted one

Prerequisites for FAT32 partitions:

  • When creating a FAT32 partition ("type 0c") on SD-card, it must become at least 16450560 sectors (with a sector size of 512 Bytes that results in 7,8 GBytes) large.
  • The maximum size for a FAT32 partition is 2 TB (which is also the limit for the DOS partitioning scheme used by fdisk).
  • A FAT partition shall be the first primary partition on SD-card, otherwise other operating systems / embedded devices may not handle the partitioning scheme correctly.
  • Thus for a dual partition layout including a FAT partition, the SD-card should be at least a "16 GB" one.

Continue in the fdisk /dev/mmcblk1 and delete the existing partition with d.

  1. Create a new, smaller first primary partition:
    • Create a new first primary partition by entering n, then p, then use the default of "1".
    • Enter the partition offset (as determined in section 2.3) as start sector.
    • Determine the last sector for the first partition by subtracting 1 from a desired size of n GBytes in sectors (hence the multiplier for 512 Byte sectors is 1024^3 Bytes divided by 512 Bytes = 2097152).
      In my example (still with that 32 GB SD-card with a partition offset of 8192 sectors) and aiming at creating a first primary partition of slightly less than 8 GBytes, this results in 8 * 2097152 - 1 = 16777215 as the last sector for the new partition (and a partition size of 16777215 +1 - 8192 = 16769024 sectors = 7,996 GBytes).
      This measure also ensures that the second partition starts at an n GByte boundary.
    • Then enter this calculated value as the last sector for the new first partition.
    • As Linux fdisk always sets the partition type of newly created partitions to 83 ("Linux"), hit t afterwards and set the partition type to c ("W95 FAT32 (LBA)"), if you want a FAT32 partition.
      If you want a native, unencrypted partition as first primary partition, leave it as it is (on 83).
  2. Create a new, second partition (which will be configured later to be used either natively or with DM-Crypt):
    • Start creating a new, second primary partition by entering n, then p for another primary partition (default, you may as well hit the Return key), then use the suggestion of "2".
    • Create this partition with the last sector of the first partition plus 1 as its start sector (e.g., for my exemplary 32 GB SD-card: 16777216). Note that fdisk will usually suggest a low sector number in the first partition's partition offset area (e.g., 2048), which must not be used.
    • Use the value obtained for the last sector of the original partition (in section 2; this is usually, but not necessarily the last sector of the SD-card) as last sector (on my exemplary 32 GB SD-card it was the last available sector, so if that was the case for your original partition layout too, you may as well just hit the Return key to avoid mishaps by mistyping).
      If a third partition is going to be created, use +nG (e.g., +10G) to set size of the second partition in GBytes.
    • When going to create an encrypted partition, set its partition type (by hitting t) to e8 ("LUKS", not documented by fdisk, but elsewhere), for an unencrypted Linux-native partition leave it on 83.
  3. If a third partition is wanted, repeat the steps for the second partition with fdisk for it:
    n, p, 3, <last sector of second partition + 1>, <noted last sector of original partition on sd-card>, t, e8.

Finally check by entering p that the partition table now looks as it should, and write it to SD-card (and exit fdisk) via w.

3.3 "Checks and balances", when calculating a partition layout

However you may deviate from this guide, you should follow these rules:

  • The first sector of any partition must be an even number, which must be evenly divisible by the erase-block size and should be evenly divisible by the alignment size.
  • The last sector of any partition must be an odd number.
  • The last sector of any partition plus 1 must be evenly divisible by the erase-block size (in sectors) and should be evenly divisible by the alignment size.
  • Consequently any partition size will be evenly divisible by the erase-block / alignment size, too (just to double-check).

Ultimately save the altered partition layout for reference / as a backup and perform a final check:
fdisk -l /dev/mmcblk1 | tee fdisk-l_mmcblk1-altered.txt

4. Creating filesystems

4.1 vFAT32 / FAT32 LFN (unencrypted)

Mind that FAT filesystems do not support access rights, hence are unsuited for putting android_storage on them (and for many other use cases), plus its anachronistic design, which does not fit well to FLASH-based mass storage. Furthermore FAT32 only supports files up to 2 GBytes size (though up to 2 TB filesystem size).
Thus a FAT partition is primarily suited for data exchange by physically moving an SD-card to other devices and as "spare area" for the FLASH controller. BTW, the same applies to exFAT, except for the file size limit.

Determine the parameters for mkfs.vfat based on the analysis of the output of dosfsck or dumpexfat in section 2.3:

  • Use the original cluster size (of typically 32, 64 or 128 sectors), which is set with option -s. If a preformatted exFAT partition had a cluster size of more than 128 sectors, use 128.
    When the partition is primarily used as an archive (i.e., not very often written to and not with many small files), as an optimisation for FLASH-media set the cluster size to its maximum value of 128 sectors (regardless of its original value).
  • Sector size as determined in section 2.3, set with option -S: This option is needed for file-based simulations, do not use it when creating a filesystem on SD-card (as its own sector size shall be implicitly used, then).
  • The partition offset ("hidden sectors count") as determined in section 2.3 is set with -h.
  • The number of FATs is set with the option -f.
    An optimisation for FLASH-based memory is to always create just a single FAT with -f 1, regardless if the original filesystem had a single FAT (typical for exFAT) or two (typical for FAT32), but this may create interoperability issues with older embedded devices (cameras etc.).

As the FATs are smaller now (due to the different capacity of the newly created filesystem), the number of reserved sectors (set with option -R) has to be adapted. As mkfs.vfat knows no "dry run" and I found no formula for calculating the exact size of a FAT according to the size of the complete file system, one simply starts a "trial run" within a sparse file on /tmp to determine the correct value for -R:
mkfs.vfat -S 512 -v -f 2 -s 64 -h 8192 -C /tmp/fat-test.img <filesystem size in KBytes> # "Trial run 1": Multiply the target partition size in sectors (created for a FAT filesystem in section 3.2) with 1024 divided by the sector size to calculate the filesystem size (in KBytes) and set cluster size, partition offset ("hidden sectors count"), number of FATs and sector size to the intended values
dosfsck -nv /tmp/fat-test.img # Compare the output with above determined values
Now calculate the number of reserved sectors, which are needed to add up with the size of the FATs (must be one or two) to any multiple (e.g., times 1, 2, 3, ...) of the determined alignment size. Note that the number of reserved sectors must be at least 32 or the cluster size (whichever is higher); in above example of an almost 8 GBytes large partition on my exemplary 32GB SD-card, each of the two FATs occupies 2048 sectors, so it takes additionally 4096 reserved sectors to sum up to the alignment size of 8192 sectors.
Then check, if this calculation is correct:
rm /tmp/fat-test.img # Delete the test file
mkfs.vfat -S 512 -v -f 2 -s 64 -h 8192 -R 4096 -C /tmp/fat-test.img 8384512 # "Trial run 2" with 4096 reserved sectors as an example, for a filesystem slightly smaller than 8 GB ending on sector 16777215
(As a different example, also for slightly smaller than 8 GB filesystem ending on sector 16777215, but for the aforementioned 128 GB SD-card: mkfs.vfat -S 512 -v -f 1 -s 128 -h 32768 -R 31744 -C /tmp/fat-test.img 8372224 # Note that the partition offset and consequently the filesystem size differ)
dosfsck -nv /tmp/fat-test.img # Check if everything looks as intended
Do apply the checks #5 and #6 in section 2.3.2 to the values of your FAT filesystem layout:
Are both O.K.?
rm /tmp/fat-test.img # Delete the test file

Finally perform the real creation of the FAT filesystem on SD-card:
mkfs.vfat -v -f 1 -s 128 -h 32768 -R 31744 /dev/mmcblk1p1 # "Real run" with example values for my 128 GB SD-card; do omit the sector size (i.e., no option -S) and filesystem size (as last parameter), as mkfs shall use the values from the SD-card
dosfsck -nv /dev/mmcblk1p1 | tee dosfsck-nv_mmcblk1p1-altered.txt # Last check, if everything is as intended, and backup

Side note: Whatever you may consider to do, never use the mkdosfs / mkfs.vfat options -a or -c on FLASH-based media (e.g., SD-cards or with an image file on internal eMMC)!

4.2 EXT4 (unencrypted)

Simply enter:
mkfs.ext4 -v /dev/mmcblk1pX # Substitute "X" with 1 or 2

Side note: Whatever you may consider to do, never use the mkfs.extX option "-c -c" on FLASH-based media as SD-cards!

4.3 DM-Crypt (encrypted)

4.3.1 Preparation

  • Only for SailfishOS 2.2.0: Install NielDK's Cryptsetup as described in section 4.3 of the original how-to, but prefer an upgrade to SailfishOS 2.2.1 to avoid trouble when upgrading later.
  • For SailfishOS 2.2.1: If you had installed Cryptsetup before upgrading to SailfishOS 2.2.1, please read section 4.3 of the original how-to thoroughly in order to employ a working Cryptsetup installation.
    If not, do nothing.
  • For SailfishOS 3.0.0 and newer: Do nothing.

Install crypto-sdcard from OpenRepos (e.g, per Storeman).
For Jolla 1 phones install crypto-sdcard ("sbj" edition) instead.

4.3.2 Creating a "key file"

Create a "key file", e.g. by creating a random passphrase file:
dd if=/dev/urandom of=/etc/crypto-sdcard/crypto_luks.key bs=64 count=1

4.3.3 Setting up Cryptsetup-LUKS

The LUKS header created with the following commands stores all relevant information to access this DM-Crypt partition, lateron only a valid "key" (corresponding to one of the keyslots) has to be provided in order to unlock and mount it.

Create a LUKS header on SD-card:
cryptsetup -v -q -h sha1 -s 256 -c aes-xts-plain --align-payload=8192 --type luks1 luksFormat /dev/mmcblk1pX /etc/crypto-sdcard/crypto_luks.key # Set "--align-payload=" to the alignment size in sectors and insert the right partition number once; omit the option "--type luks1" on SailfishOS < 3.0.3

Backup the LUKS header as text and binary:
cryptsetup -v luksDump /dev/mmcblk1pX | tee luksdump_mmcblk1pX.txt # Check & backup; substitute "X" twice
cryptsetup -v --header-backup-file luks-header-backup_mmcblk1pX.bin luksHeaderBackup /dev/mmcblk1pX # Backup the LUKS header, as without it being intact all encrypted data is inaccessible (e.g., when the LUKS header on SD-card is damaged); substitute "X" twice

Change the "key file" to its final name:
mv -v /etc/crypto-sdcard/crypto_luks.key /etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key # Mind to adapt the partition number "X"
Restrict access to read only:
chmod 0440 /etc/crypto-sdcard/*.key
Back up the "key file":
cp /etc/crypto-sdcard/*.key ./ # Better (for security) move the backed-up "key file" off-device later on, i.e. deleting it (and the binary backup of the DM-Crypt header) in ~nemo/sdcard-layouts/ after copying them away

4.3.4 Creating a filesystem on encrypted partition

Open the Cryptsetup container:
cryptsetup -v --allow-discards -d /etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key luksOpen /dev/mmcblk1pX crypto-sd # Create DM-device file; substitute "X" twice

Create a filesystem by entering:
mkfs.ext4 -v /dev/mapper/crypto-sd

When successfully created a filesystem, do:
cryptsetup -v close crypto-sd # Finishing setup steps by closing the DM-crypt mapper-file

4.3.5 Let SailfishOS automatically mount the encrypted partition

Ultimately reboot to see, if everything is working as intended: Issue as user nemo mount | grep crypto

If not, debug with journalctl -r as root user, while having less installed (e.g., per pkcon install less):
Search with / for crypto, use n to jump to the next hit and press PageUp / PageDown keys to look at adjacent messages.
Note that due to a flaw in older Linux kernels you may see journal messages as follows WRT mapped DM-crypt partitions, which are harmless and should be ignored:

systemd-udevd[ZZZZ]: conflicting device node '/dev/mapper/crypto_*' found, link to '/dev/dm-?' will not be created
systemd-udevd[YYYY]: inotify_add_watch(9, /dev/dm-?, 10) failed: No such file or directory

5. Finishing up

Finally "donate" all filesystems on the SD-card to user nemo, else you cannot use them as nemo:
devel-su
cd /run/media/nemo/
chown -v nemo:nemo */.

You can check that discards are correctly configured with lsblk -D /dev/mmcblk1 (crypto-devices do not return zeroes).

6. Notes

  • This guide can be easily adapted to USB storage devices attached via USB-OTG (if the device hardware supports USB-OTG; the Jolla 1 phone doesn't), using /dev/sda as device name for the whole storage device and /dev/sdaX as device names for the partitions on it (while having a single USB storage device attached).
    SailfishOS 3.0.0 and newer automatically mounts all supported filesystems (unencrypted) on USB storage devices and so does crypto-sdcard for filesystems on encrypted partitions with SailfishOS 2.2.0 and newer.
    Do not apply part 2 of this guide to (easily) removable devices as USB attached storage!
  • Interoperability with SailfishOS GUI apps
    • SailfishOS 3
      • The Backup and Storage functions in the Settings app are working flawlessly with encrypted and unencrypted filesystems.
        You can also manually mount and unmount filesystems on encrypted and unencrypted partitions at the GUI there, though apps depending on data residing on SD-card will fail (or AlienDalvik, if section 2 of part 2 was applied) after unmounting.
      • For all other SailfishOS's own and third party apps, plus AlienDalvik:
        If an app does not allow for setting its default path to SD-card, utilise part 2.
      • Before SailfishOS 3.0.2 (i.e., 3.0.0 and 3.0.1): The backup function in Settings -> SailfishOS updates and the storage chooser in Jolla's Camera app exhibit a slight flaw.
        It is "grayed out" when updating, until one has entered Settings -> Backup once since the last reboot (no need to create a backup there!) respectively manually chosen an encrypted partition in the Camera app.
        After doing so, either function (backup / storage chooser) is working fine.
        This affects partitions encrypted per SailfishOS' GUI or per this guide alike ([1], [2], [3]).
    • SailfishOS 2.2.x
      • Settings -> Backup only uses the first partition on SD-card, which must be unencrypted (stating it cannot write to it, if encrypted, even though it is successfully mounted and writable): If you rely on backing up your contacts etc. with SailfishOS' backup tool onto SD-card, create an unencrypted partition (formatted with any supported filesystem) at /dev/mmcblk1p1 (a couple of GBytes is sufficient, if a large encrypted partition is wanted).
        Consequently the backup function in Settings -> SailfishOS updates (when updating) behaves the same way.
      • Settings -> Storage fails to detect encrypted partitions, offering to format them!
        This is fine, as long as you do not follow this advice! 😉
      • For all other SailfishOS's own and third party apps, plus AlienDalvik:
        If an app does not allow for setting its default path to SD-card, utilise part 2.

7. Additional note

Use MicroSD-cards from SanDisk: From the "Ultra" series the red-gray or black models (reasonably priced), or their "Extreme" series (usually coloured red-gold and expensive). Alternatively MicroSD-cards from Samsung or the white / white-red models from the SanDisk "Ultra" series, but they both often do not support trim.
I personally regard all other brands as low quality (as they do not manufacture controller, flash memory and firmware by themselves, in contrast to SanDisk and Samsung) and "Noname" cards as rubbish.

8. Appendix: Practical trim / discard test

See section 8.2 of the original how-to.

9. Addendum: Alter an encrypted partition created per Settings app to be automatically mounted

  1. Prerequisites: A device with SailfishOS 3.0.0 or newer with an SD-card on which an encrypted partition was created per Settings -> Storage -> Format: Encrypt
  2. Thoroughly walk through the sections 1 "Preparation" and 2.1 "Analysing an SD-card's partition layout". Determine the device name of your encrypted partition by looking at fdisk's output (likely /dev/mmcblk1p1).
  3. Backup the original LUKS header of your encrypted partition as text and binary:
    cryptsetup -v luksDump /dev/mmcblk1pX | tee luksdump_mmcblk1pX-original.txt # Substitute "X" twice
    cryptsetup -v --header-backup-file luks-header-backup_mmcblk1pX-original.bin luksHeaderBackup /dev/mmcblk1pX # Backup the LUKS header to be able to return to the original state, in case anything goes wrong; substitute "X" twice
    (For reference, see the second last paragraph of section 4.3.3.)
  4. Install crypto-sdcard (rsp. on Jolla 1 phones crypto-sdcard ("sbj" edition)). See section 4.3.1 for details.
  5. Create a "key" file for the encrypted partition looked up in step 1 (can be done with a single command here, because the partition is already encrypted), restrict access to the "key" file and back it up:
    dd if=/dev/urandom of=/etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key bs=64 count=1 # Mind to adapt the partition number "X"
    chmod 0440 /etc/crypto-sdcard/*.key # Restrict access to read only
    cp /etc/crypto-sdcard/*.key ./ # Back up the "key file"
    Better (for security) move the backed-up "key file" off-device later on, i.e. deleting it (and the binary backup of the DM-Crypt header) in ~nemo/sdcard-layouts/ after copying them away.
    (For reference, see section 4.3.2 plus the last paragraph of section 4.3.3.)
  6. Determine if all eight "key slots" in the luks1 header are occupied, and decide if you want to keep the old passphrase-based keys

    See which "key slots" (of #0 to #7) are already occupied by looking at the output of more luksdump_mmcblk1pX-original.txt # Substitute "X" once
    Background: For luks1 headers, each added "key source" (a passphrase or file) is hashed down to 160 bits (20 bytes) and its hash value plus a salt value and some metadata stored in one of eight "key slots" (see cryptsetup manpage for details). Every configured "key source" is able to open ("unlock") the cryptsetup container; thus there always must be at least one configured "key source", otherwise one would not be able to access and manage the cryptsetup container anymore. Also note that for emptying ("freeing" / "killing") a "key slot", a valid "key source" has to be supplied.
    Side note: Since the introduction of the manual, passphrase-based unlocking and mounting of encrypted partitions on SD-cards per SailfishOS 3.0.0, a bug seems to fill up all "key slots" for some users. Sadly the code doing this seems to be proprietory and non-public (at least I was not able to find it although I have searched the net thoroughly), making further analysis of the reason for this behaviour impossible.

    • If there is at least one empty "key slot" and you are fine with keeping the extant "key slots" occupied and their "key sources" valid, just execute this and your are done with step 6:
      cryptsetup -v luksAddKey /dev/mmcblk1pX /etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key # Substitute "X" twice; you will be prompted for a valid passphrase
    • If there is no empty "key slot", you have to free one. As "key slot" #0 or #7 most likely have the current passphrase as "key source", free "key slot" #4:
      cryptsetup -v luksKillSlot /dev/mmcblk1pX 4 # Substitute "X" once; you will be prompted for a valid passphrase
      Then you might continue by working through the next bullet point (deleting all old key material) or simply execute the single luksAddKey command in the first bullet point (above).
    • In order to remove all old key material, first configure a free "key slot" with your "key file":
      cryptsetup -v luksAddKey /dev/mmcblk1pX /etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key # Substitute "X" twice; you will be prompted for a valid passphrase
      Then you can utilise this "key file" to authorise the removal of all other key material, i.e. emptying all the "key slots", which were occupied at the start of step 6 (you may re-execute the more command and take a look again), except the one you may have freed and reused in the bullet point before. Execute the command below for each "key slot #N" you want to empty:
      cryptsetup -v --key-file /etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key luksKillSlot /dev/mmcblk1pX N # Substitute "X" twice and "N" (the "key slot" number) once
      Note that you can always take a look at the current state of the luks header per cryptsetup -v luksDump /dev/mmcblk1pX # Substitute "X" once
      If anything goes seriously wrong, this command restores the original luks header: cryptsetup -v luksHeaderRestore /dev/mmcblk1pX --header-backup-file luks-header-backup_mmcblk1pX-original.bin # Substitute "X" twice
  7. After having finished configuring the key material for the luks header, perform a final, visual check and backup the luks header in its new state as text and binary (as in the second last paragraph of section 4.3.3):
    cryptsetup -v luksDump /dev/mmcblk1pX | tee luksdump_mmcblk1pX.txt # Substitute "X" twice
    cryptsetup -v --header-backup-file luks-header-backup_mmcblk1pX.bin luksHeaderBackup /dev/mmcblk1pX # Substitute "X" twice
  8. Work through section 4.3.5 "Let SailfishOS automatically mount the encrypted partition" and you are done with altering your encrypted partition to be auto-mounted.
edit retag flag offensive close delete

Comments

Thanks for this helpful guide.

As an improvement it would be nice to have a list of which exact SD-card models are checked against the requirements (Section 2.3.2 Checks).

I bought recently a 64GB SD-card from Samsung which eventually didn't support discards. I decided to stay with Kingston 32GB card which was faster and supported discards.

MRI-xa2 ( 2019-03-28 13:53:16 +0300 )edit
1

Discard ("trim") support is not a requirement, this guide works fine using SD-cards without it.
But as SATA-SSDs in 2014, good ones support it and I personally would not use an SD-card without discard ("trim") support in computers.

If one of the other checks fails for an SD-card's preformatted (ex)FAT filesystem, such a strange filesystem format indicates a low quality card. But these checks are primarily intended for creating a new FAT filesystem correctly and if you have altered the partition layout or filesystem(s) of a SD-card before.

MicroSD-cards I checked:

  • SanDisk Ultra 128 GB (red-gray) from 2017: Preformatted exFAT filesystem is fine, discard ("trim") is fully supported (discard_granularity, preferred_erase_size plus discard_zeroes_data) and works, reasonably fast
  • SanDisk Ultra 32 GB (red-gray) from 2017: Preformatted FAT filesystem is fine, discard is fully supported and works, reasonably fast
  • Samsung [standard series, name forgotten] 32 GB from 2017: Preformatted FAT filesystem is fine, discard support is not indicated (and does not work), significantly slower than the Sandisk Ultras.
olf ( 2019-05-11 17:09:30 +0300 )edit
1

Thanks for crypto-sdcard. Since I already had an encrypted SD card by Sailfish all I had to do was:

  • download crypto-sdcard
  • make the /etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key file
  • cryptsetup luksAddKey /dev/mmcblk1pX /etc/crypto-sdcard/crypto_luks_$(blkid -c /dev/null -s UUID -o value /dev/mmcblk1pX).key
aerique ( 2019-11-10 17:59:42 +0300 )edit

Yes @aerique, it is quite easy to convert an encrypted partition, which was created per Jolla's Settings at the GUI (and hence has to be manually mounted) to be automatically mounted by crypto-sdcard, while keeping the encryption and data intact.
Thanks for your mini-guide.

But there are caveats to be taken care of:

  • One should Backup the LUKS header as text and binary in the original (created with Jolla's tools) state first, as described in section 4.3.3.
  • All LUKS key slots (#1 - #7) may be occupied, as @jollajo observed.

Hence I am considering to extend and generalise your mini-guide a bit and to post it here.

olf ( 2019-11-10 18:46:49 +0300 )edit

Thanks for the very detailed guide!

However, I'm trying to setup a DM-crypted partition, but the luksFormat command fails with a Command failed with code -1 (wrong or missing parameters).

Digging deeper, if I add the debug option to cryptsetup, this error shows up in the long output: Userspace crypto wrapper cannot use aes-xts-plain (-95). I tried also to set cipher to aes-xts-plain64 or to not specify it and use the default but similar errors come up.

Device is a Xperia XA2 with latest SFOS 'Nuuksio' 3.2.1.20

simosagi ( 2020-01-18 19:20:08 +0300 )edit