Kushal Das

FOSS and life. Kushal Das talks here.

kushal76uaid62oup5774umh654scnu5dwzh4u2534qxhcbi4wbab3ad.onion

Using rkt and systemd

Few days back, I wrote about my usage of rkt containers. As rkt does not have any daemon running, the simplest way to have a container running is to start it inside some screen or tmux session. I started following the same path, I used a tmux session.

But then I wanted to have better control over the containers, to start or stop them as required. Systemd is the solution for all the other services in the system, that makes it an ideal candidate for this case too.

Example of a service file

[Unit]
Description=ircbot
Documentation=https://github.com/kushaldas/ircbot
Requires=network-online.target

[Service]
Slice=machine.slice
MemoryLimit=500M
ExecStart=/usr/bin/rkt --insecure-options=image --debug run --dns=8.8.8.8 --volume mnt,kind=host,source=/some/path,readOnly=false  /mnt/ircbot-latest-linux-amd64.aci
ExecStopPost=/usr/bin/rkt gc --mark-only
KillMode=mixed
Restart=always

The path of the service file is /etc/systemd/system/ircbot.service. In the [Unit] section, I mentioned a super short Description, and link to the documentation of the project. I also mentioned that this service requires network-online.target to be available first.

The [Service] is the part where we define all the required configurations. The first value we mention is the Slice.

Slices, a way to do resource control

Systemd uses slices to group a number of services, and slices in a hierarchical tree. This is built on top of the Linux Kernel Control Group feature. In a system by default, there are four different slices.

  • -.slice : The root slice.
  • system.slice : All system services are in this slice.
  • machine.slice : All vms and containers are in this slice.
  • user.slice : All user sessions are in this slice.

We can see the whole hierarchy using the systemd-cgls command. For example:

Control group /:
-.slice
├─machine.slice
│ ├─ircbot.service
│ │ ├─11272 /usr/bin/systemd-nspawn --boot --register=true -Zsystem_u:system_r:container_t:s0:c447,c607 -Lsystem_u:object_r:container_file_t:s0:c447,
│ │ ├─init.scope
│ │ │ └─11693 /usr/lib/systemd/systemd --default-standard-output=tty
│ │ └─system.slice
│ │   ├─ircbot.service
│ │   │ └─11701 /usr/bin/ircbot
│ │   └─systemd-journald.service
│ │     └─11695 /usr/lib/systemd/systemd-journald
├─user.slice
│ └─user-1000.slice
│   ├─session-31.scope
│   │ ├─16228 sshd: kdas [priv]
│   │ ├─16231 sshd: kdas@pts/0
│   │ ├─16232 -bash
│   │ ├─16255 sudo su -
│   │ ├─16261 su -
│   │ └─16262 -bash

You can manage various resources using cgroups. Here, in our example service file, I mentioned that memory limit for the service is 500MB. You can read more here on resource management.

There is also systemd-cgtop tool, which will give you a top like view for the various resources consumed by the slices.

# systemd-cgtop -M rkt-250d0c2b-0130-403b-a9a6-3bb3bde4e934

Control Group                                                           Tasks   %CPU   Memory  Input/s Output/s
/machine.slice/ircbot.service                                             9      -   234.0M        -        -
/machine.slice/ircbot.service/system.slice                                -      -     5.0M        -        -
/machine.slice/ircbot.service/system.slice/ircbot.service                 -      -     5.0M        -        -

The actual command which we used to run the container is mentioned in ExecStart.

Using the service

I can now use the standard systemctl commands for this new ircbot service. For example:

# systemctl start ircbot
# systemctl enable ircbot
# systemctl stop ircbot
# systemctl status ircbot

You can also view the log of the application using journalctl command.

# journalctl -u ircbot

The documentation from rkt has more details on systemd and rkt.

Trying out rkt v1.14.0 on Fedora 24

Few days back we had rkt v1.14.0 release from CoreOS. You can read details about the release in their official blog post. I decided to give it a try on a Fedora 24 box, I followed the official documentation. The first step was to download rkt, and acbuild tools.

To download, and install acbuild tool, I did the following: (Btw, as it was a cloud instance, I just moved the binaries to my sbin path)

$ wget https://github.com/containers/build/releases/download/v0.4.0/acbuild-v0.4.0.tar.gz
$ tar xzvf acbuild-v0.4.0.tar.gz
$ sudo mv acbuild-v0.4.0/* /usr/sbin/

Now for rkt, do the following.

$ wget https://github.com/coreos/rkt/releases/download/v1.14.0/rkt-v1.14.0.tar.gz
$ tar xzvf rkt-v1.14.0.tar.gz
$ cd rkt-v1.14.0
$ ./rkt help
$ sudo cp -r init/systemd/* /usr/lib/systemd/

Now I had to modify a path inside ./scripts/setup-data-dir.sh file, at line 58 I wrote the following.

systemd-tmpfiles --create /usr/lib/systemd/tmpfiles.d/rkt.conf

Next step is to execute the script. This will create the required directories, and fix the permission issues. Before that I will also create a group, and add my current user to the group. Remember to logout, and login again for it.

$ sudo groupadd rkt
$ export WHOAMI=$(whoami); sudo gpasswd -a $WHOAMI rkt
$ sudo ./scripts/setup-data-dir.sh

rkt documentation suggests to disable SELinux for trying out, I tried to run it with SELinux in Enforcing mode, and then created local policy based on the errors. I have also opened a bug for the rawhide package.

# ausearch -c 'systemd' --raw | audit2allow -M localrktrawhide
# semodule -i localrktrawhide.pp

After all this we are finally in a state to start using rkt in the system.

The Try out document says to trust the signing key of etcd, I am going to do that, and then test by fetchin the image.

$ sudo ./rkt trust --prefix coreos.com/etcd
$ ./rkt fetch coreos.com/etcd:v2.3.7
image: searching for app image coreos.com/etcd
image: remote fetching from URL "https://github.com/coreos/etcd/releases/download/v2.3.7/etcd-v2.3.7-linux-amd64.aci"
image: keys already exist for prefix "coreos.com/etcd", not fetching again
image: downloading signature from https://github.com/coreos/etcd/releases/download/v2.3.7/etcd-v2.3.7-linux-amd64.aci.asc
Downloading signature: [=======================================] 490 B/490 B
Downloading ACI: [=============================================] 8.52 MB/8.52 MB
image: signature verified:
  CoreOS Application Signing Key <security@coreos.com>
  sha512-7d28419b27d5ae56cca97f4c6ccdd309

You can view the images, with a image list subcommand.

$ ./rkt image list
ID                      NAME                                    SIZE    IMPORT TIME     LAST USED
sha512-5f362df82594     coreos.com/rkt/stage1-coreos:1.14.0     162MiB  1 day ago       1 day ago
sha512-86450bda7ae9     example.com/hello:0.0.1                 7.2MiB  15 hours ago    15 hours ago
sha512-7d28419b27d5     coreos.com/etcd:v2.3.7                  31MiB   48 seconds ago  48 seconds ago

From here, you can just follow the getting started guide. I used the debug flag to see what is going on.

$ sudo ./rkt --insecure-options=image --debug run ../hello/hello-0.0.1-linux-amd64.aci                                
image: using image from local store for image name coreos.com/rkt/stage1-coreos:1.14.0
image: using image from file ../hello/hello-0.0.1-linux-amd64.aci
stage0: Preparing stage1
stage0: Writing image manifest
stage0: Loading image sha512-86450bda7ae972c9507007bd7dc19a386011a8d865698547f31caba4898d1ebe
stage0: Writing image manifest
stage0: Writing pod manifest
stage0: Setting up stage1
stage0: Wrote filesystem to /var/lib/rkt/pods/run/d738b5e3-3fe9-4beb-ae5c-3e8f4153ee57
stage0: Pivoting to filesystem /var/lib/rkt/pods/run/d738b5e3-3fe9-4beb-ae5c-3e8f4153ee57
stage0: Execing /init
networking: loading networks from /etc/rkt/net.d
networking: loading network default with type ptp
Spawning container rkt-d738b5e3-3fe9-4beb-ae5c-3e8f4153ee57 on /var/lib/rkt/pods/run/d738b5e3-3fe9-4beb-ae5c-3e8f4153ee57/stage1/rootfs.
Press ^] three times within 1s to kill container.
systemd 231 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK -SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT -GNUTLS -ACL +XZ -LZ4 +SECCOM$ +BLKID -ELFUTILS +KMOD -IDN)
Detected virtualization rkt.
Detected architecture x86-64.

Welcome to Linux!

Set hostname to <rkt-d738b5e3-3fe9-4beb-ae5c-3e8f4153ee57>.
[  OK  ] Listening on Journal Socket.
[  OK  ] Created slice system.slice.
         Starting Create /etc/passwd and /etc/group...
[  OK  ] Created slice system-prepare\x2dapp.slice.
[  OK  ] Started Pod shutdown.
[  OK  ] Started hello Reaper.
[  OK  ] Listening on Journal Socket (/dev/log).
         Starting Journal Service...
[  OK  ] Started Create /etc/passwd and /etc/group.
[  OK  ] Started Journal Service.
         Starting Prepare minimum environment for chrooted applications...
[  OK  ] Started Prepare minimum environment for chrooted applications.
[  OK  ] Started Application=hello Image=example.com/hello.
[  OK  ] Reached target rkt apps target.
[111534.724440] hello[5]: 2016/09/10 14:48:59 request from 172.16.28.1:35438

While the above container was running, I tested it out from another terminal, and then stopped it.

$ ./rkt list
UUID            APP     IMAGE NAME              STATE   CREATED         STARTED         NETWORKS
865b862e        hello   example.com/hello:0.0.1 running 8 seconds ago   8 seconds ago   default:ip4=172.16.28.2
$ curl 172.16.28.2:5000
hello
$ sudo ./rkt stop 865b862e
"865b862e-21f5-43e0-a280-3b4520dad97c"

I hope this post will help you to try out rkt on a Fedora system. Feel free to comment if you have question, or ask over twitter.