Kubernetes on Novena
Kubernetes is a high-availability clustering package. It is available for ARM, but mostly is used on x86-64. These steps detail how to use it on Novena.
These steps assume you're starting from a base Novena image, such as the one found here on the Kosagi wiki.
Installing Kubernetes
Some steps I've taken to install Kubernetes on Debian. Adapted from https://blog.hypriot.com/post/setup-kubernetes-raspberry-pi-cluster/ for Novena.
0. Remove software that comes installed by default on Novena, but is unnecessary for servers.
apt-get install -y network-manager apt-get remove -y --auto-remove --purge hexchat pidgin bluez bluez-tools bluez-hcidump \ enigmail hexchat icedove iceweasel pavucontrol x11-xserver-utils xscreensaver \ libbluetooth3 keychain xserver-xorg-video-modesetting arandr android-tools-adb \ android-tools-fastboot android-tools-fsutils xfce4-goodies xfce4-power-manager \ xfce4-mixer xfce4-terminal mousepad orage dbus-x11 irssi synaptic qalc libqt5gui5 \ libqt5core5a libqt5widgets5 x11-apps x11-session-utils xbitmaps xfce4 xfce4-appfinder \ xfce4-notifyd xfce4-session xfce4-settings xfdesktop4 xfdesktop4-data xfonts-100dpi \ xfonts-75dpi xfonts-scalable xfwm4 xfwm4-themes xinit xorg xorg-docs-core \ libdrm-armada2-dbg libetnaviv-dev libetnaviv-dbg novena-disable-ssp novena-eeprom-gui \ novena-firstrun xorg-novena xserver-xorg-video-armada xserver-xorg-video-armada-dbg \ xserver-xorg-video-armada-etnaviv
1. Update the key.
wget http://repo.novena.io/debian/pool/main/k/kosagi-repo/kosagi-repo_1.2-r1_all.deb sudo dpkg -i kosagi-repo_1.2-r1_all.deb apt-get update DEBIAN_FRONTEND=noninteractive apt-get dist-upgrade apt-get install apt-transport-https # Used for accessing the newer repos
2. At this point, it's probably a good idea to move to a SATA disk.
dd if=/dev/mmcblk0 of=/dev/sda bs=1M count=1 fdisk /dev/sda # Re-create partitions 2 and 3. Make partition 2 at least 6GB # for swap (type 82), and make partition 3 the rest of the disk. # Type "x" for Expert mode, then "i", and change the ID to 0x4e6f7653. # Then "r" to return to the main menu, and "w" to write it to disk. mkfs.ext4 /dev/sda3 # or install btrfs-tools or xfsprogs and make a different type mount /dev/sda3 /mnt rsync -avxHAX --progress / /mnt/ novena-eeprom -w -f es8328,pcie,gbit,hdmi,eepromoops,sataroot reboot
3. Install Docker (borrowed from https://github.com/hypriot/image-builder-odroid-c1/blob/master/builder/chroot-script.sh). If you need a specific version, you can list them with:
apt-cache madison docker-engine
wget -q https://packagecloud.io/gpg.key -O - | apt-key add - echo 'deb https://packagecloud.io/Hypriot/Schatzkiste/debian/ jessie main' > /etc/apt/sources.list.d/hypriot.list apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 2C52609D echo 'deb [arch=armhf] https://apt.dockerproject.org/repo debian-jessie main' > /etc/apt/sources.list.d/docker.list apt-get update DEBIAN_FRONTEND=noninteractive apt-get install \ docker-engine=1.13.1-0~debian-jessie \ docker-compose="1.9.0-23" \ docker-machine="0.9.0-39"
4. Install kubeadm
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" > /etc/apt/sources.list.d/kubernetes.list apt-get update && apt-get install -y kubeadm
5. Run kubeadm on the master device to start the cluster.
kubeadm init --pod-network-cidr 10.244.0.0/16
It will output something like the following:
Your Kubernetes master has initialized successfully! You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: http://kubernetes.io/docs/admin/addons/ You can now join any number of machines by running the following on each node: kubeadm join --token=964a50.d8053ed3de195a11 10.0.245.169
6. Join the cluster from other machines.
kubeadm join --token=964a50.d8053ed3de195a11 10.0.245.169
7. Ensure cAdvisor doesn't start up. It's nice to have, but it leaks lots of information.
printf '[Service]\nEnvironment="KUBELET_EXTRA_ARGS=--cadvisor-port=0"\n' > /etc/systemd/system/kubelet.service.d/05-disable-cadvisor.conf systemctl daemon-reload systemctl restart kubelet
8. Install Flannel on the Master, which will manage the network for us.
curl -sSL https://rawgit.com/coreos/flannel/master/Documentation/kube-flannel.yml | sed "s/amd64/arm/g" | kubectl create -f -
9. Wait for all pods to be Running:
kubectl get po --all-namespaces