Comparing Sandboxing Tools

The motivation came from here:

https://xkcd.com/1200/

When programming in Node.js, a huge problem is that “npm install” downloads libraries you did not specify. It downloaded all dependencies listed in package.json, but it also downloaded their dependencies and the dependencies of their dependencies etc., which is code you did not explicitly ask for. While you can point your direct dependencies to trustworthy sources, you have no control about anything further down the line. In short: this is a (known) security hazard. A recent example is here. Auditing code in npm helps, but the whole concept is a fundamental problem.

Dart and Deno are reducing the problem significantly since you have to name all dependencies, but it does not necessarily help you if that dependency itself is compromised.

The runtime of Deno as well as wasmtime use a sandbox-approach to mitigate that: you have to enable access explicitly to anything: A Deno program has very few permissions otherwise. From a security point of view, this is much better.

Node.js nor Python have no sandbox model and when loading libraries from the Internet, which both do a lot, do you always know what you get? So I’m looking for choices how to retrofit programs with potentially questionable code.

My requirements:

  • Possible to use ad-hoc: I want to run a program with somehow limited access (e.g.: no root and no ability to become root, network access only when I allowed it, no access to files it does not need access to)
  • Protect my files from programs which run as me and thus with my normal privileges (e.g. very few program would need access to my ssh keys)

Test case:

  • Run a Node.js program which wants to read ~/.test_me and access http://www.google.com. It should not be able to do either unless it’s enabled.

Here the simple Node.js program:

const fs=require('fs');
const fetch=require('node-fetch');

async function accessStuff() {
  try {
    let f=await fs.promises.readFile(`${process.env.HOME}/.test_me`);
    console.log(`File: ${f}`);
  } catch(e) {
    console.log(`Error while accessing .test_me: ${e}`);
  }
  fetch('http://www.google.com', {
                method: 'get',
                })
    .then(res => res.text())
    .then(body => console.log(body.split('\n').slice(0,1)))
    .catch(e => {
      console.error(`Error: ${e}`);
    });
}

(async function () {
try {
  await accessStuff();
} catch(e) {
  console.error(`Error: ${e}`);
}
})();

and a sample run without limitations:

❯ node index.js 
File: test 1

[...many more lines from .test.me...]
[
  `<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage"
[...some HTML code from www.google.com...]

When it comes to security and sandboxing, those choices came up after a quick check with Google:

System-wide Mandatory Access Control (MAC)

SELinux needs special policies/contexts set up for the whole system. While this is great, it’s something root does. I see no simple way to do ad-hoc configurations to run a single command with the “correct” permissions. Plus the policy files are not easy to read nor write.

AppArmor is similar. A bit easier to read policy files, but they are all root owned, so not suitable for ad-hoc commands.

Both have a point to secure the complete system with the user explicitly not allowed to change the policies. Their purpose it not to protect the user from hurting themselves.

Sandbox Tools

Docker or containers in general provide a good way of isolation from the rest of the system and via bind mounts you can allow access to files or directories easily, but you have to create a container image first, upload it to a container registry and download and run it (Update: turns out that this is not required and a locally created image can be executed without problems). While it has its use, creating containers is a significant overhead if it’s needed for every program you are suspicious about.

minijail from Google looks good:

Minijail […] provides an executable that can be used to launch and sandbox other programs, […]

https://google.github.io/minijail/

Installing on Debian was straightforward (needs kernel-headers and libcap-dev). Running a command with a specific user-definable policy is possible:

 # minijail0 -S /usr/share/minijail0/$(uname -m)/cat.policy -- \\
             /bin/cat /proc/self/seccomp_filter
but the examples/ directory was a small shock to me: a single example, and not a well explained one.

❯ cat examples/cat.policy 
# In this directory, test with:
# make LIBDIR=.
# ./minijail0 -n -S examples/cat.policy -- /bin/cat /proc/self/status
# This policy only works on x86_64.

read: 1
write: 1
restart_syscall: 1
rt_sigreturn: 1
exit_group: 1

open: 1
openat: 1
close: 1
fstat: 1
# Enforce W^X.
mmap: arg2 in ~PROT_EXEC || arg2 in ~PROT_WRITE
fadvise64: 1

While there is a tool to record the uses system calls (via strace) to create a policy (similar to what SELinux’s audit2allow tool), that means running a potentially harmful program once without restrictions. Plus the policy file is not exactly easy to understand. And the documentation does not help.

This is a dead end for my purpose.

bubblewrap is used as security layer for FlatPack installations before it was spun out. Using it is very command-line-option intensive, but a wrapper script will handle this. A test run:

❯ cat bwrap.test 
#!/bin/sh

bwrap \
 --dev /dev \
 --ro-bind /lib /lib \
 --ro-bind /usr/bin /usr/bin \
 --ro-bind /bin /bin \
 --ro-bind /etc/resolv.conf /etc/resolv.conf \
 --ro-bind $HOME/js $HOME/js \
 --ro-bind $HOME/.test_me $HOME/.test_me \
 --tmpfs /tmp \
 --unshare-all \
 --share-net \
~/js/node/bin/node index.js

❯ ./bwrap.test 
~/.test_me contains: test 1
[
  `<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage"
[...some more HTML code...]

Removing the “–share-net” and removing the “–ro-bind” for .test_me stops both from being accessible:

❯ ./bwrap.test 
Error while accessing .test_me: Error: ENOENT: no such file or directory, open '/home/harald/.test_me'
Error: FetchError: request to http://www.google.com/ failed, reason: getaddrinfo ENOTFOUND www.google.com

Note that you also need /etc/resolv.conf too to allow resolving DNS names. Also the order of “–unshare-all” and “–share-net” is important as the last one wins.

firejail is conceptually similar to bubblewrap, but beside having a large list of command line options, it also has configuration files in /etc/firejail/ and it also allows user-owned configurations (default in ~/.config/firejail):

❯ cat ~/.config/firejail/nodejs.profile
whitelist /home/harald/js
#whitelist /home/harald/.test_me
net none
#quiet
include /usr/local/etc/firejail/whitelist-common.inc
include /usr/local/etc/firejail/default.profile

❯ firejail --profile=~/.config/firejail/nodejs.profile node index.js
Reading profile /home/harald/.config/firejail/nodejs.profile
Reading profile /usr/local/etc/firejail/whitelist-common.inc
Reading profile /usr/local/etc/firejail/default.profile
Reading profile /usr/local/etc/firejail/disable-common.inc
Reading profile /usr/local/etc/firejail/disable-passwdmgr.inc
Reading profile /usr/local/etc/firejail/disable-programs.inc
Parent pid 231521, child pid 231522
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Child process initialized in 94.84 ms
Error while accessing .test_me: Error: ENOENT: no such file or directory, open '/home/harald/.test_me'
Error: FetchError: request to http://www.google.com/ failed, reason: getaddrinfo ENOTFOUND www.google.com

Parent is shutting down, bye...
❯ firejail --quiet --net=none node index.js
Error while accessing .test_me: Error: EACCES: permission denied, open '/home/harald/.test_me'
Error: FetchError: request to http://www.google.com/ failed, reason: getaddrinfo ENOTFOUND www.google.com

The last sample shows that you don’t need to create a separate profile but similar to bwrap you can use command line options for most settings.

Uncommenting the “whitelist /home/harald/.test_me” line allows access to that file. Commenting out the “net none” allows network access. Per default network access is granted, but you can change this in /etc/firejail/default.profile. Once disabled in a profile, it cannot be re-enabled though. (Update: A “–ignore=net” option will ignore the “net none” in a profile).

After above tests I found out you can skip the “–profile=~/.config/firejail/PROFILENAME” if PROFILENAME is the binary name plus “.profile” as firejail will pick this up automatically. Very neat!

❯ firejail node index.js 
Reading profile /home/harald/.config/firejail/node.profile
Reading profile /etc/firejail/whitelist-common.inc
[...]

And you can make it less verbose too and with sensible defaults you don’t even need to create any profiles. E.g. shell history files are inaccessible by default:

❯ firejail --quiet bash
$ cd
$ cat .bash_history 
cat: .bash_history: Permission denied
$ ls -la .bash_history
-r-------- 1 nobody nogroup 0 Dec 30 23:39 .bash_history

My Conclusion

SELinux and AppArmor are not something users can manage by themselves. Different scope than what I am looking for.

Using containers, especially when running as non-root works as long as you want to use containers anyway. Otherwise it’s a huge overhead: create container image, store it in a registry, and then run it. Any code changes would need a new container image to be created. Good for certain workload, especially those which will run as containers anyway later on. While I use containers extensively, a lot of programs I run are not a container.

bubblewrap works. It needs an extensive list of options to be useful. That’s not hard to put into a script. Since you have to add a lot of options and there’s no default options you can specify, it’s very explicit about permissions which makes it easier to debug since everything is configured when running your suspicious program. As the order of options is important, I can see this getting complicated quickly for non-trivial programs. Here is an example. Luckily most programs are trivial: few accesses are needed plus some capabilities like network access.

firejail got the spot between security and ease-of-use right in my opinion: sensible defaults (e.g. disabling at and crontab commands) with profiles for many programs. You can also have user-configurable profiles and they are not hard to create. The amount of extra work when using firejail is low: just adding “firejail” before the command helps a lot already out-of-the-box by hiding sensitive files and disabling miss-usable commands. Creating a specific profile makes this very configurable. And if you name the profiles like the binary you plan to use, it’s both simple to use while still being configurable.

Note that no solution is 100% secure. There’s always a trade-off between convenience and security. Unless you enforce it, if it’s inconvenient, it won’t be done.

PS: While testing I experienced firejail to not be able run programs which have capabilities set if you use “caps.drop all” which is included in the default profile. See bug report. Can’t say yet if it’s a bug or badly worded option or lack of documentation or just unexpected behavior.

3 thoughts on “Comparing Sandboxing Tools

  1. Thanks for the writeup! I arrived at the same conclusion re. firejail vs. Docker.

    Re. Deno – you say it’s “reducing the problem significantly since you have to name all dependencies”, but
    * Deno deps are loaded from individually-controlled URLs, with no autorihy like NPM to run security scans or remove malware
    * these URLs can randomly deliver malware for a percentage of requests, so they can survive security scans
    * the content at these URLs can be legitimate today and malware tomorrow (then back)
    * can’t a Deno dependency load other dependencies in turn?

    I’m sure there are good answers to these questions, but I haven’t yet seen them in one place (e.g. in the Deno manual). On the contrary.

    Like

    1. Dan, thanks for your comment! You have a valid point about modules via URL not being safer than what NPM does.

      But what I like about Deno is not so much the explicit naming of the module/URL, but this part: Deno programs have no access to network or files by default. It needs to be explicitly enabled: See https://deno.land/manual/getting_started/permissions and thus you know what can be accessed.

      So it’s like bwrap in the sense that by default it allows nothing and you need to specify what access is needed.

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Create your website with WordPress.com
Get started
%d bloggers like this: