Installing Gentoo Prefix
asked 2020-06-18 15:46:47 +0200
This post is a wiki. Anyone with karma >75 is welcome to improve it.
Here's a guide on how to get Gentoo Prefix running under SailfishOS. I've made it for the Pro1, but you might be able to use it for another device.
If you know what Gentoo Prefix is, do not expect it to be the smooth ride I'm sure it usually is.
If you know what Gentoo is, but not what Gentoo Prefix is: the short of it is that this basically lets you install Gentoo as a normal user in the directory of your choice, minus the kernel (since one is already running). You cannot mess up your SailfishOS install with this, so it's pretty safe to play around with.
If you don't know what Gentoo is, it's a distribution aimed at power-users that lets you customize quite a lot of stuff. Basically, a dedicated group of maintainers ensure that the configuration of your usual programs can be done through simple keywords in a standardized fashion (e.g. "ssl" to indicate you want your package to have the ssl options enabled). In practice, you basically have a set of text files in /etc/portage/ that describe the system you want, and a package manager (Portage) which will tell you if that's doable, or why not, and make it happen if it is. The downsides being that you are expected to have a coherent set of files in /etc/portage/, so you'll often face a "nope, you need to allow this in /etc/portage/ before I can do that" kind of issue if you're not careful. Also, since you really can personalize stuff, packages have to be compiled, which can be annoying when installing new software (not so much when updating, since you can just let it compile in the background). This also means you can tailor your programs to your hardware to get better performance. There is somewhat of an expectation for users to read documentation, so if you don't want to take the time to learn what things are and how they work, you shouldn't be trying to customize them, and thus probably shouldn't be using Gentoo. Oh, and the TL;DR of compiling on a phone: it wasn't a good plan 20 years ago, but you have a gaming PC in your hands nowadays, so the only issue is the rewrite limits of flash memory. Just use tmux or Screen when merging huge packages (e.g. llvm, xorg, firefox, webkit-gtk, icedtea, ...) so that you don't lose progress if the Sailfish terminal application stops for some reason (I've noticed it did that sometimes, and that was even before I installed the Prefix).
SailfishOS runs a 64bit kernel (aarch64) with a strictly 32bit userland (armhf). I'd ask, but I don't know who to. Whatever. The point is: you're not easily getting a 64bit toolchain set up on that, so this guide goes for a 32bit Gentoo Prefix (armhf). Now, the interesting thing is: Gentoo is very good at setting up cross-toolchains, so it might be possible to use the 32bit Gentoo Prefix to create a 64bit toolchain that you could use to install a 64bit Gentoo Prefix (this... is to go... even further beyond!). You can see what your toolchain is by running "gcc -v".
Note that ${EPREFIX} refers to the folder you want your Gentoo Prefix installed in. Mine is /gentoo, so if you see that in this guide, assume that this means you have to actually write the name of the folder and not use the environment variable (and only in this case, otherwise, prefer using the env variable). You should export this environment variable (which you'll need to do anyway):
$ export EPREFIX="/gentoo"
For reference, commands starting with "#" (e.g. "# mkdir ${EPREFIX}") are commands ran as root, whereas commands starting with "$" (e.g. "$ mkdir ${EPREFIX}") are commands run as nemo (or any normal user). You shouldn't be using root for anything past "Getting Started".
Getting Started
Setting up an SD Card
I strongly recommend using an SD card (and one targeted at dashcams, so that it withstands a lot of rewrites) to store your prefix. If you don't want to, feel free to skip this step and create the EPREFIX folder.
This assumes you've just put a dedicated SD card in your phone. If this is not the case, make sure the SD card is formatted in something that can support a Linux system (e.g. ext4).
Formatting the SD card:
# cfdisk /dev/mmcblk0
Choose Linux as partition type. Write the new partition table.
# mkfs.ext4 /dev/mmcblk0p1
This makes your partition use ext4.
Preparing the Prefix folder:
# mkdir ${EPREFIX}
This creates the folder (if you didn't know that, it's a strong sign that you should probably not be trying Gentoo yet).
As root, edit /etc/fstab to add the line:
/dev/mmcblk0p1 /gentoo ext4 defaults 0 0
This will make it be mounted automatically at boot.
# mount ${EPREFIX}
This mounts it right now.
# chown -R nemo:nemo ${EPREFIX}
This makes it owned by nemo, meaning that you don't need root privileges to read or write in there.
Installing the required packages:
SailfishOS has a package manager called pkcon. I'm very new to that OS, so there may be a better one, but this one will do.
You will need to install "make", "gcc", "gcc++", and "python" (that last dependency is not standard for a Gentoo Prefix install, but you'll need it for a workaround).
# pkcon install make gcc gcc++ python
I've actually done that with one command per program installed, but I assume you can do it in a single call.
Tinkering some stuff in the install script:
Download the Gentoo Prefix bootstrap script from https://wiki.gentoo.org/wiki/Project:Prefix and put it in ${EPREFIX}.
Make the script executable:
$ chmod +x ./bootstrap-prefix.sh
Open it in your favorite text editor (i.e. vim).
Find:
if [[ ${PN} == "m4" ]] ; then
# drop _GL_WARN_ON_USE which gets turned into an error with
# recent GCC 1.4.17 and below only, on 1.4.18 this expression
# doesn't match
sed -i -e '/_GL_WARN_ON_USE (gets/d' lib/stdio.in.h lib/stdio.h
Add before:
if [[ ${PN} == "tar" ]] ; then
# drop _GL_WARN_ON_USE which gets turned into an error with
# recent GCC 1.4.17 and below only, on 1.4.18 this expression
# doesn't match
sed -i -e '/_GL_WARN_ON_USE (gets/d' gnu/stdio.in.h gnu/stdio.h
fi
tar won't compile if you don't do that.
Find and comment out:
[[ ${PN} == "bash" && ${CHOST} != *-cygwin* ]] \
&& myconf="${myconf} --disable-readline"
bash won't compile if you don't do that.
Find:
einfo "running emerge -u system"
CPPFLAGS="-DGNUSTEP_BASE_VERSION" \
CFLAGS= CXXFLAGS= emerge -u system || return 1
Add before:
einfo "fixing virtual/libc"
CPPFLAGS="-DGNUSTEP_BASE_VERSION" \
CFLAGS= CXXFLAGS= emerge -1 --nodeps -n --ask virtual/libc glibc || return 1
The script won't be able to do the "emerge -u" you just saw without that.
Find:
[[ ${OFFLINE_MODE} ]] || type -P wget > /dev/null \
|| (bootstrap_wget) || return 1
Add before:
[[ ${OFFLINE_MODE} ]] || type -P wget > /dev/null \
|| (bootstrap_libpsl) || return 1
wget needs libpsl to compile.
Find:
bootstrap_wget() {
bootstrap_gnu wget 1.20.1 || \
bootstrap_gnu wget 1.17.1 || bootstrap_gnu wget 1.13.4
}
Replace with (yes the extra function should be added):
bootstrap_wget() {
bootstrap_gnu wget 1.20.1 #|| \
# bootstrap_gnu wget 1.17.1 || bootstrap_gnu wget 1.13.4
}
bootstrap_libpsl() {
bootstrap_gnu libpsl 0.21.0
}
wget will crash anyway, so let's not lose too much time trying the other versions. We also need to add something to install libpsl, hence the added function.
You're done with pre-installation stuff.
Stage 1
I'll repeat it again, just in case: from now on, no root, only nemo.
Go to ${EPREFIX}
$ cd ${EPREFIX}
Create a file called use_gentoo_prefix.sh, with the following:
export EPREFIX="/gentoo"
export CHOST="armv7hl-hardfloat-linux-gnueabi"
export CFLAGS="-march=armv8-a -mtune=cortex-a73.cortex-a53 -pipe -O2"
export PRESTAGE_1_PATH="${PATH}"
export PATH="${EPREFIX}/usr/bin:${EPREFIX}/bin:${EPREFIX}/tmp/usr/bin:${EPREFIX}/tmp/bin:${PATH}"
export LDFLAGS=""
export LD_LIBRARY_PATH=""
export CFFLAGS=""
export PKG_CONFIG_PATH=""
export PATH="${EPREFIX}/usr/sbin:${EPREFIX}/sbin:${EPREFIX}/tmp/usr/sbin:${EPREFIX}/tmp/sbin:${PATH}"
If you are indeed using the Pro1, you should only modify the EPREFIX line to match your own. Otherwise, I believe the CHOST is mostly linked to the toolchain provided by SailfishOS and should thus stay unchanged. the CLFAGS must match something that fits your CPU. Do not simply remove them, this will not work here and will force you to restart the whole thing way down the line (can you tell I'm speaking from experience? :o).
Source use_gentoo_prefix.sh. You'll need to do that again every time you close the terminal. I would recommend not setting it up as automatic in ~/.bashrc, but instead creating some alias like "alias gentoo='source /gentoo/use_gentoo_prefix.sh'", in case the prefix breaks for some reason.
$ source use_gentoo_prefix.sh
Stage 1 is now about to start for real...
$ ./bootstrap-prefix.sh "${EPREFIX}" stage1
If you see a nice logo in ASCII art, you failed to source use_gentoo_prefix.sh.
Bootstrapping WGET fails:
Yeah, it will do that in this install. I did warn about this not being a smooth ride, didn't I?
$ mv ${EPREFIX}/tmp/bin/wget{,_back}
This renames wget into wget_back. We'll create a shim. Using your favorite text editor, create the file ${EPREFIX}/tmp/bin/wget with the following content:
#!/bin/bash
LD_LIBRARY_PATH="/gentoo/tmp/lib" /gentoo/tmp/bin/wget_back $@
It is important that you do not use the ${EPREFIX} environment variable here. Hardcode the location.
Make it executable:
$ chmod +x ${EPREFIX}/tmp/bin/wget
Resume stage 1:
$ ./bootstrap-prefix.sh "${EPREFIX}" stage1
Bison will try multiple versions before succeeding.
Missing Profile:
Near the end of stage 1, you'll get a message about the profile for your setup not being automatically found.
$ ln -s ${EPREFIX}/var/db/repos/gentoo/profiles/default/linux/arm/17.0/armv7a/prefix/kernel-3.2+/ ${EPREFIX}/etc/portage/make.profile
You'll need to start stage 1 again ($ ./bootstrap-prefix.sh "${EPREFIX}" stage1), but let's do everything we need to do in ${EPREFIX}/etc/portage right now, so we don't have to later.
Modifications in ${EPREFIX}/etc/portage:
Here's my ${EPREFIX}/etc/portage/make.conf. Make yours match so that the compilation succeeds. You should ignore the PORTAGE_DIR and the DISTDIR if you don't want to compile things in RAM (but you really should be compiling in RAM, so check out the replies to this post to see how to set that up).
CFLAGS="-march=armv8-a -mtune=cortex-a73.cortex-a53 -O2 -pipe"
###############################################################################
# Added by bootstrap-prefix.sh for armv7hl-hardfloat-linux-gnueabi
USE="unicode nls"
#CFLAGS="${CFLAGS} -O2 -pipe"
CXXFLAGS="${CFLAGS}"
MAKEOPTS=""
CONFIG_SHELL="/gentoo/bin/bash"
DISTDIR="/gentoo/var/cache/distfiles"
# sandbox does not work well on Prefix, bug 490246
FEATURES="${FEATURES} -usersandbox -sandbox"
###############################################################################
CHOST="armv7hl-hardfloat-linux-gnueabi"
CHOST_arm="${CHOST}"
CHOST_default="${CHOST}"
PORTAGE_TMPDIR="/ram"
DISTDIR="${PORTAGE_TMPDIR}/portage"
ACCEPT_LICENSE="* -@EULA"
USE="${USE} X wayland xwayland egl gallium gles2 gbm wayland-compositor"
USE="${USE} -pam ssl"
USE="${USE} pulseaudio dbus"
${EPREFIX}/etc/portage/package.env:
dev-lang/perl perl
Perl... has some issues getting installed.
${EPREFIX}/etc/portage/env/perl:(You will need to create the directory first)
EXTRA_ECONF="-Dosname='linux' -Dhintfile='linux' -Duserelocatableinc='false'"
Perl considers that any OS with a Linux kernel in which /system/lib/libandroid.so exists must be Android. We need to really insist on being Linux. Also, it'll try and fail to install with incompatible options, so we disable the one that isn't hardcoded in the Gentoo package file.
Stage 2
We've got one last thing to do before starting Stage 2: fixing some stuff the script did incorrectly.
Print $PRESTAGE_1_PATH:
$ echo $PRESTAGE_1_PATH
Does it contain anything related to Gentoo Prefix? The goal here is to get the $PATH you were using before adding the Gentoo Prefix directories to it.
You'll need to edit two files, but their content are the same. Make it so ${EPREFIX}/tmp/usr/local/bin/{gcc,g++} contain only one copy of the three lines (you'll be able to see it clearly if the content has been duplicated). Make it so that their content is similar to:
#! /bin/sh
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/nemo/bin" export PATH
exec "${0##*/}" "$@"
Replacing the value I've put there with the one you have in $PRESTAGE_1_PATH (which is the same if you haven't modified your PATH outside of this guide).
You can now run Stage 2:
$ ./bootstrap-prefix.sh "${EPREFIX}" stage2
Stage 3
You can now run Stage 3:
$ ./bootstrap-prefix.sh "${EPREFIX}" stage3
Circular dependency
If you are hit by this bug, I've made a workaround to be able to continue while they fix it cleanly: download the workaround overlay, untar it in somewhere like a "/home/nemo/src" folder. You'll have to edit/create ${EPREFIX}/etc/portage/repos.conf and ${EPREFIX}/tmp/etc/portage/repos.conf, because I do not know which one is being used at this point. Make these files contain:
[libcryptbootstrap]
location=/home/nemo/src/libcryptbootstrap
priority=10
masters=gentoo
Edit the bootstrap script, find:
-pcre
-ssl
-python
Remove the "-" before ssl.
Start stage 3 again ($ ./bootstrap-prefix.sh "${EPREFIX}" stage3)
emerge --depclean failed:
Yeah, and you know what? Let's not bother fixing that. Look at the very last line there. It should tell you have successfully passed stage 3.
After the Install
Hopefully, you now have a Gentoo Prefix install on your phone. Happy hacking.
PS: If you're not used to Gentoo on ARM/AARCH64, but use it on some other architecture, you might be surprised by some packages being masked due to missing keyword (e.g. mednafen). Check the webpage for the package, since it might not be available even in unstable for this architecture, meaning that you will need to add a keyword to allow its installation using the package of another architecture.
You'll most likely want to follow the advice of Portage and set your locale in ${EPREFIX}/etc/locale.gen. I'm not going to include much in terms of guides for general Gentoo use, since there are many other better written resources for that. Just know of the "--autounmask" parameter for emerge, which really helps getting quickly over blocked/masked packages: if it tells you that you need to add a keyword looking like "your/package **", this means there's no ARM package available and it's trying to get the package from another architecture. It's not something you usually see when using Gentoo on a PC.
I'll post guides for these once I've reached them, but the next objectives are: - Getting Firefox running. Web browsers aren't exactly the easiest of packages to successfully merge. - Making sure hardware acceleration is used with X11 applications (and Wayland ones).
If you have access to it (you do with the Pro1), you should probably put the source code of your kernel in /usr/src/linux/ (doesn't look like putting it in the prefix will work), since quite a few packages attempt to read your configuration from there. I think they simply assume everything has been configured as they want it to be if they can't find the configuration file, but it would definitely be preferable to let them check.
I have uploaded a few screenshots, if people want to see it work, for some reason.
nsensfel ( 2020-06-18 19:30:16 +0200 )editI'm mostly going from failed merge to failed merge, learning new things. I'll test it out before editing the guide, but you might want to:
- add "-mfpu=crypto-neon-fp-armv8" to the CFLAGS. This should enable those extensions, and I believe they are available on this processor (so far, it fixed an issue, and hasn't caused any other).
- set ACCEPT_KEYWORDS="arm -~arm" so that it stops pulling unstable packages by default.
nsensfel ( 2020-06-25 12:30:17 +0200 )edit