1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
|
---
title: "Abusing systemd-nspawn With Nested Containers"
date: 2023-03-01T04:03:13Z
description: "Pushing systemd-nspawn and my laptop to their limits with indefinitely nested containers"
type: "post"
tags: ["linux", "containers", "systemd"]
---
systemd-nspawn is some pretty insane stuff that you probably already have installed on your computer. Today's mission to wreck havoc will be to replicate an old project I did a few years ago, [Arch All the Way Down](https://git.exozy.me/a/Arch-All-the-Way-Down), but using some real power tools. I was able to achieve 4 nested containers with Docker and my old laptop. It shouldn't be too hard to improve that!
There's a really [cool trick you can do with systemd-nspawn](https://0pointer.net/blog/running-an-container-off-the-host-usr.html) where you can almost instantly create a container based on your existing root directory:
```bash
$ sudo systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b
```
This is all nice and great, and I was easily able to create containers nested 10 layers down, but it's annoying to have to log in to each one and run the command again. Let's automate this.
The first thing we can do is [auto login](https://wiki.archlinux.org/title/Getty#Nspawn_console). Cool, now our cursed setup looks like this:
```
$ cat autologin.conf
[Service]
ExecStart=
ExecStart=-/sbin/agetty -o '-p -f -- \\u' --noclear --keep-baud --autologin root - 115200,38400,9600 $TERM
$ sudo systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b --bind=autologin.conf:/etc/systemd/system/console-getty.service.d/autologin.conf
```
Now for some more fun: we will make the container auto-run this command by adding the thing above to its `/root/.bash_profile`. Don't try this at home.
```
$ cat bash_profile
systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b --bind=/etc/systemd/system/console-getty.service.d/autologin.conf:/etc/systemd/system/console-getty.service.d/autologin.conf --bind=/root/.bash_profile:/root/.bash_profile
$ sudo systemd-nspawn --directory=/ --volatile=yes --set-credential=passwd.plaintext-password.root:"" --set-credential=firstboot.locale:C.UTF-8 -b --bind=autologin.conf:/etc/systemd/system/console-getty.service.d/autologin.conf --bind=bash_profile:/root/.bash_profile
```
You might be a bit worried at this point (for many reasons), but one concern is that this spawns nested containers infinitely. Fortunately, it takes around a second to spawn a container so we won't DOS ourselves instantly, and you can easily hit Ctrl+] 3 times to terminate all containers.
As for measuring the nesting depth, container processes are visible on the host, so `ps aux | grep systemd-logind` or something like that will do the trick.
Now let's have some real fun! My laptop has 16GB of RAM, so I wonder how many nested containers it can handle. A few hundred maybe?
Nope, only 33. I still have plenty of RAM left over, but I'm getting a `Failed to fork inner child: No space left on device` error. Aw man.
Unfortunately, I spent way too much time trying to figure out this error, but it doesn't seem to be something you can search up online, for obvious reasons. [This](https://cgit.freedesktop.org/systemd/systemd/tree/src/nspawn/nspawn.c#n4412) is the line in the nspawn source code that raises the error, but it's not really helpful. If anyone has any ideas, I'd love to hear it!
|
