M5Stack & AWS IoT

Received my AWS IoT EduKit from M5Stack. First impression: it’s an improvement to the original M5Stack I have: display is nicer and power and reset button is now 2 instead of 1 button. Sensor touch instead of buttons too, but not sure this is an improvement or just cost cutting.

Time to test this via the Blinky Hello World example!

First notes: the instructions do clash with any Python environment you might have set up. It recommends to use miniconda. It’s not needed if you already have a virtual environment setup. Just activate your environment:

$ cd ~/git
$ git clone -b release/v4.2 --recursive https://github.com/espressif/esp-idf.git
$ cd esp-idf
$ . $HOME/esp/esp-idf/install.sh
ERROR: This script was called from a virtual environment, can not create a virtual environment again
$ . ./export.sh 
Detecting the Python interpreter
Checking "python" ...
Python 3.8.5
"python" has been detected
Adding ESP-IDF tools to PATH...
Using Python interpreter in /home/harald/venv/bin/python
Checking if Python packages are up to date...
Python requirements from /home/harald/git/esp-idf/requirements.txt are satisfied.
Added the following directories to PATH:
  /home/harald/git/esp-idf/components/esptool_py/esptool
  /home/harald/git/esp-idf/components/espcoredump
  /home/harald/git/esp-idf/components/partition_table
  /home/harald/git/esp-idf/components/app_update
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

You can ignore the error after executing . ./install.sh . All requirements should have been installed as expected into your virtual environment.

In case you get odd errors, delete ~/.espressif/ as it has the compiler tool chain. See also here if your code crash loops.

The AWS CLI tools need to be installed and configured (of course). Then finally the fun starts:

$ cd ~/git
$ git clone https://github.com/m5stack/Core2-for-AWS-IoT-EduKit.git
$ cd Core2-for-AWS-IoT-EduKit/Blinky-Hello-World/utilities/AWS_IoT_registration_helper/
$ pip install -r requirements.txt
$ python registration_helper.py -p /dev/ttyUSB0
[...lots of lines...]
Manifest was loaded successfully

That’ll throw an error if you use Python other than 3.7. The fix is simple: remove the bold lines in the registration_helper.py:

def check_environment():
    """Checks to ensure environment is set per AWS IoT EduKit instructions

    Verifies Miniconda is installed and the 'edukit' virtual environment
    is activated.
    Verifies Python 3.7.x is installed and is being used to execute this script.
    Verifies that the AWS CLI is installed and configured correctly. Prints
    AWS IoT endpoint address.
    """
    conda_env = os.environ.get('CONDA_DEFAULT_ENV')
    if conda_env == None or conda_env == "base":
        print("The 'edukit' Conda environment is not created or activated:\n  To install miniconda, visit https://docs.conda.io/en/latest/minico
nda.html.\n  To create the environment, use the command 'conda create -n edukit python=3.7'\n  To activate the environment, use the command 'con
da activate edukit'\n")
    print("Conda 'edukit' environment active...")
    
    if sys.version_info[0] != 3 or sys.version_info[1] != 7:
        print(f"Python version {sys.version}")
        print("Incorrect version of Python detected. Must use Python version 3.7.x. You might want to try the command 'conda install python=3.7'
.")
        exit(0)
    print("Python 3.7.x detected...")

    aws_iot_endpoint = subprocess.run(["aws", "iot", "describe-endpoint", "--endpoint-type", "iot:Data-ATS"], universal_newlines=True, capture_o
utput=True)
    if aws_iot_endpoint.returncode != 0:

Now in the AWS console you can find it:

and you can also get its endpoint from the AWS CLI:

$ aws iot describe-endpoint --endpoint-type iot:Data-ATS
{
    "endpointAddress": "c7xxxxxxxxxx0f-ats.iot.ap-northeast-1.amazonaws.com"
}

Use idf.py menuconfig to configure the endpoint name and the WiFi connectivity, then idf.py build flash monitor -p /dev/ttyUSB0 to build, flash and then connect to the serial port to watch it. Then you should see incoming MQTT messages:

And you make the LEDs blink by publishing into CLIENT_ID/blink. Stop blinking by publishing to the same topic. If you don’t know your CLIENT_ID, look it up on the display.

Given that AWS charges you for IoT traffic like those messages, don’t keep it messaging all day long. It only takes 5 days 18h only to hit the limit of free 500k messages to send (at 1/s).

Saleae Logic Analyzer – First Impressions

I bought myself a Saleae Logic 8 as I am constantly looking at oscilloscopes to help debugging problems with I/O on microcontrollers. It’s frustrating if you “see” nothing, beside you don’t get the results you want. Static signals are no problem: plug in an LED and you can see the state, but this does not work anymore as soon as frequencies larger than 10Hz are used. I2C runs at 100 or 400kHz…SPI even higher.

One fix would be a digital oscilloscope, but they are usually limited to 2 or 4 channels, with 4 channels being already expensive. Some can decode digital protocols too.

But there’s an alternative: Logic analyzers. And Saleae has a nice one; Logic 8: 8 channels, 100MHz digital and 10MHz analog sample rate, 8 channel. And for non-commercial use it’s 50% off!

Ordered one. 3 days later it arrived.

After few hours playing with it, it’s clear: I should have bought one much earlier. Seeing the I2C and SPI traffic or just pulses is so simple and so useful for debugging. Here an example:

1 is the I2C data channel, 2 is the clock. 3 is the same but recorded analog. 4 is the decoded I2C data and 5 is a decoder extension I wrote this afternoon for decoding the data for the PCF8583 I used here (in clock mode). This is easier and at the same time more useful than I thought.

PCF8583 and Espruino

Looking for a I2C device to test and play with it, I found an old PCF8583 board I bought a long time ago from Futurlec. I would have guess about 10 years ago. I remember it can handle 3.3V and has the needed pull-up resistors via jumpers. Perfect for testing!

Unexpectedly I found no library for that chip at http://www.espruino.com/modules/, but looking at the data sheet, it’s not really needed. Setting time and reading time is straightforward:

I2C1.setup({sda: D5, scl: D4});

pcf8583Addr=0x51;

function BCDToBinary(n) {
  return (n>>4)*10+(n&0x0f);
}
function binaryToBCD(n) {
  return ((n/10)<<4) + (n % 10);
}
function BCDToString(n) {
  return String.fromCharCode((n>>4)+48, (n&0x0f)+48);
}
    
function getPCFTime() {
  I2C1.writeTo(pcf8583Addr, 1);
  let d=I2C1.readFrom(pcf8583Addr, 4);
  return `${BCDToString(d[3])}:${BCDToString(d[2])}:${BCDToString(d[1])}.${BCDToString(d[0])}`;
}

function getPCFDate() {
  I2C1.writeTo(pcf8583Addr, 5);
  let d=I2C1.readFrom(pcf8583Addr, 2);
  let year=(2020+((d[0]&0xc0)>>6)).toString();
  let month=BCDToString(d[1] & 0x1f);
  let day=BCDToString(d[0] & 0x3f);
  return `${year}-${month}-${day}`;
}

function setPCFTime(h, m, s) {
  I2C1.writeTo(pcf8583Addr, [0, 0x80, 0, binaryToBCD(s), binaryToBCD(m), binaryToBCD(h)]);
  I2C1.writeTo(pcf8583Addr, 0, [0x00]);
}

function start(){
 g.clear();
 g.drawString("Starting...", 0, 0);
 g.flip(); 
}

var g = require("SSD1306").connect(I2C1, start, {height:64});
require("Font8x12").add(Graphics);
g.setFont8x12();

function updateDisplay() {
  g.clear();
  g.drawString(getPCFDate(), 0, 0);
  g.drawString(getPCFTime(), 0, 20);
  g.flip();
}

setInterval(updateDisplay, 1000);

By the way, the most amazing part of this test was that the included CR2032 Lithium battery still works after that many years.

openLRSng

Long time ago I purchased 3 of those pictured above. Originally for long distance remote control airplanes. I never used them as 2.4GHz took over and that worked for sufficient distance to my eyesight.

Where those modules can be used is for long range IoT uses though. LoRaWAN would be ok too but I don’t have any of those available. And for controlling servos, this receiver got everything out-of-the-box.

Setup

Software comes from the openLRSng repo, specifically the 433MHz release for hardware type 3: RX-3.hex and TX-3.hex from here. Yes, the receiver can be a transmitter, and for telemetry it’s even a requirement to send back data (e.g. battery or signal status).

Configuration is well described here and it uses a Chrome app for this. In order:

  • Connect the transmitter via an USB serial port converter (see Hardware Guide for pin-out details).
  • Flash one receiver with the receiver firmware RX-3.hex.
  • Flash one receiver with the transmitter firmware TX-3.hex.
  • Set up the transmitter to your liking.
  • Power up the receiver by connecting about 5V and GND to the receiver’s port 1. To force binding connect port 1 and 2 (the signal) via a jumper or a jumper cable. Should not be needed though. That power is used to power up the receiver and power the servos. It’s regulated down to 3.3V for the CPU.
  • Shortly after powering on the receiver it should bind to the transmitter and in the UI the receiver tab will be populated.
  • Set up all parameters (frequency) on the transmitter and save them in its EEPROM. Repeat for the receiver.

And you are done with the configuration work. From now on the two units don’t need a computer anymore: the transmitter will expect a CPPM (AKA PPMSum) input signal on port 5 and the receiver will have 8 channels of PWM data as output.

Now the part which took me longest: You have to create a CPPM signal on Port 5 on the transmitter (as per transmitter pin-out). That gets transmitted to the receiver which then shows servo-compatible PWM signals on its 8 ports (port 1 is used for RSS).

Since it’s quite time critical (it’s all about pulses in the 1000-2000µs range with ideally microsecond resolution and low jitter), a hardware timer solution is needed here. Luckily Arduino has PPMEncoder for exactly that. There was an unexpected snag though: the default of PPMEncoder is 500µs (0%) to 2500µs (100%) which causes the transmitter to not recognize the input CPPM signal anymore and then the receiver goes into “fail safe” mode (visible by the LEDs changing). If you vary the signal from 1000µs to 2000µs, all is well. In fact, 550..2450 works too. Since the initial pulse PPMEncoder creates is 500µs long, the total signal needs to be a bit more. However the servo I tested does not move smoothly outside the normal range: while it can move further left than 1000µs and further right than 2000µs, the movement is not linear anymore.

Here the rather simple Arduino program to create a CPPM signal on Pin 7:

#include "PPMEncoder.h"

#define OUTPUT_PIN 7

void setup() {
  ppmEncoder.begin(OUTPUT_PIN);
  for (int ch=0; ch<8; ++ch)
    ppmEncoder.setChannel(ch, 1500);
}

const int servoMax=2000;
const int servoMin=1000;

void loop() {
  while(1) {
  for (int i=servoMin; i<servoMax; ++i) {
    for (int ch=0; ch<4; ++ch)
      ppmEncoder.setChannel(ch, i);
    delay(20);
  }
  delay(500);
  for (int i=servoMax; i>=servoMin; i-=10) {
    for (int ch=2; ch<4; ++ch)
      ppmEncoder.setChannel(ch, i);
    delay(20);
  }
  delay(2000);
  }
}

A very boring video of the resulting servo movements is available here.

LEGO+ and M5Stack

When the Lego Mindstorm NXT came out, I bought one. Good fun, but I realized quite quickly that programming via GUI is…not fun at all. Attempts to program in NXC have been made, but it wasn’t as much fun as I hoped.

Enter the M5Stack with its LEGO+ module (now renamed to DC Motor) which can connect to the motors. Not the sensors, but there’s of course the way more useful M5Stack Units.

And since the M5Stack Core unit has WiFi, a display, buttons and can be programmed in microPython, it’s fun again!

However the latest firmware (v.1.7.2) misses the module for the LEGO+ module. v1.4.5 has it though.

Preview(opens in a new tab)

Task 1: Since you can only tell the motor to turn with a given direction and speed, make it so reach a certain amount of rotations. Classical PID control loop:

This rather simple program does the trick:

from m5stack import *
from m5ui import *
from uiflow import *
import module
import time

class PID:

    def __init__(self, P=0.2, I=0.0, D=0.0):
        self.Kp = P
        self.Ki = I
        self.Kd = D

        self.sample_time = 0.00
        self.current_time = time.ticks_ms()
        self.last_time = self.current_time
        self.clear()

    def clear(self):
        self.SetPoint = 0.0
        self.PTerm = 0.0
        self.ITerm = 0.0
        self.DTerm = 0.0
        self.last_error = 0.0

        # Windup Guard
        self.int_error = 0.0
        self.windup_guard = 10000.0

        self.output = 0.0

    def update(self, feedback_value):
        """Calculates PID value for given reference feedback
        .. math::
            u(t) = K_p e(t) + K_i \int_{0}^{t} e(t)dt + K_d {de}/{dt}
        .. figure:: images/pid_1.png
           :align:   center
           Test PID with Kp=1.2, Ki=1, Kd=0.001 (test_pid.py)
        """
        error = self.SetPoint - feedback_value

        self.current_time = time.ticks_ms()
        delta_time = self.current_time - self.last_time
        delta_error = error - self.last_error

        if (delta_time >= self.sample_time):
            self.PTerm = self.Kp * error
            self.ITerm += error * delta_time

            if (self.ITerm < -self.windup_guard):
                self.ITerm = -self.windup_guard
            elif (self.ITerm > self.windup_guard):
                self.ITerm = self.windup_guard

            self.DTerm = 0.0
            if delta_time > 0:
                self.DTerm = delta_error / delta_time

            # Remember last time and last error for next calculation
            self.last_time = self.current_time
            self.last_error = error

            self.output = self.PTerm + (self.Ki * self.ITerm) + (self.Kd * self.DTerm)
            lcd.print("PTerm: "+str(self.PTerm)+"    ", 0, 80, 0xffffff)
            lcd.print("ITerm: "+str(self.ITerm)+"    ", 0, 100, 0xffffff)
            lcd.print("DTerm: "+str(self.DTerm)+"    ", 0, 120, 0xffffff)

    def setWindup(self, windup):
        self.windup_guard = windup

lego_motor = module.get(module.LEGO)
setScreenColor(0x222222)

def buttonA_wasPressed():
  global speed
  lego_motor.M1.stop()
  pass
btnA.wasPressed(buttonA_wasPressed)

def buttonC_wasPressed():
  global speed
  lego_motor.M1.encode_clear()
  pass
btnC.wasPressed(buttonC_wasPressed)

speed = 0
lego_motor.M1.set_pwm(speed)

# Parameter tuned for motor@9V
pid=PID(0.3, 0.0, 40.0)

# Turn 1/8th of circle, that's 6 full rotations for the motor
pid.SetPoint=360*6
lego_motor.M1.encode_clear();

# If motor signal is less than about 50
# the motor won't turn. So stop then

too_small=50
too_small_count_max=20
too_small_count=0

for count in range(1000):
    pid.update(lego_motor.M1.encoder_read())
    speed=pid.output;
    lcd.print("Encoder: "+str(lego_motor.M1.encoder_read())+"      ", 0, 0, 0xffffff)
    lcd.print("Speed: "+str(speed)+" ", 0, 20, 0xffffff)
    lcd.print("Time: "+str(time.ticks_ms()), 0, 40, 0xffffff)
    if speed < -255:
        speed = -255
    if speed > 255:
        speed = 255
    lego_motor.M1.set_pwm(speed)
    if speed < too_small:
        too_small_count += 1
    else:
        too_small_count = 0
    if too_small_count > too_small_count_max:
        break;
    time.sleep(0.2)

lego_motor.M1.stop()

It was interesting to find usable PID parameters. In the end https://en.wikipedia.org/wiki/PID_controller#Ziegler%E2%80%93Nichols_method worked quite well. The motor has a lot of play and does not work at lowest speeds. I’ll instead use some servos I still have and see what I can PID with those.

Mijia LYWSD03MMC, BLE and Node.js

That hard-to-remember product name is a Bluetooth LE enabled small thermometer and hygrometer from Xiaomi. The special thing about it is that Aaron Christophel created a GitHub repo about how to re-flash it with firmware which makes it way more useful: it now advertises the temperature, humidity and battery level which makes it very easy to pick up.

So I got 4 of those and re-flashed them:

  1. Access https://atc1441.github.io/TelinkFlasher.html with a WebBluetooth capable browser
  2. Click on Connect and connect
  3. Click on Do Activation
  4. Click on Choose firmware and use the ATC_Thermometer.bin from above repo
  5. Click on Start Flashing
  6. Wait about 60s
  7. Wait until it reboot. The display will show the 3 last bytes of the MAC address for easier reference later on.

Connect to the newly flashed device. You can change some settings with the buttons at the end of the web page (e.g. disable the “Show battery in LCD” if you measure and record it anyway).

Scripts for finding them and reading data out of them is here.

Reading them out is very simple with above scripts:

❯ node read-thermometer.js -s ATC_C1CADA,ATC_D01337 -e 10 -t
environment,host=m75q,sensor=ATC_C1CADA temp=22.5,humidity=37,battery=100
environment,host=m75q,sensor=ATC_D01337 temp=20.7,humidity=44,battery=100

which you can feed via telegraf. Here the telegraf.conf snippet:

[[inputs.exec]]
  commands = ["node ~/git/LYWSD03MMC/read-thermometer/read-thermometer.js -s ATC_C1CADA,ATC_D01337 -t"]
  timeout = "12s"
  data_format = "influx"

And the result are very nice graphs in Grafana (3 sensors):

Temperatures recorded via BLE from 3 LYWSD03MMC sensors

firejail Command Completions for zsh

Command completions are awesome when they work. A great example is the docker command completion. It helps a lot: if you don’t know the possible commands or options, it shows them to you. If you do know them, you only have to type few letters and hit <TAB>. The best is when shows options which it dynamically generates, e.g. when you want to inspect an image, “docker image inspect <TAB><TAB>” will show you the images you can inspect. A real time saver.

Using firejail made me want to have command completion for it too as it has a ton of command line options. And some of them can have dynamic updates. There’s a completion script for bash part of the firejail repo, but no zsh.

So I created one. Was an interesting experience. And definitely helps me using firejail.

Most helpful was this blog post which is one of the very few simple zsh completion examples I found.

Comparing Sandboxing Tools

The motivation came from here:

https://xkcd.com/1200/

When programming in Node.js, a huge problem is that “npm install” downloads libraries you did not specify. It downloaded all dependencies listed in package.json, but it also downloaded their dependencies and the dependencies of their dependencies etc., which is code you did not explicitly ask for. While you can point your direct dependencies to trustworthy sources, you have no control about anything further down the line. In short: this is a (known) security hazard. A recent example is here. Auditing code in npm helps, but the whole concept is a fundamental problem.

Dart and Deno are reducing the problem significantly since you have to name all dependencies, but it does not necessarily help you if that dependency itself is compromised.

The runtime of Deno as well as wasmtime use a sandbox-approach to mitigate that: you have to enable access explicitly to anything: A Deno program has very few permissions otherwise. From a security point of view, this is much better.

Node.js nor Python have no sandbox model and when loading libraries from the Internet, which both do a lot, do you always know what you get? So I’m looking for choices how to retrofit programs with potentially questionable code.

My requirements:

  • Possible to use ad-hoc: I want to run a program with somehow limited access (e.g.: no root and no ability to become root, network access only when I allowed it, no access to files it does not need access to)
  • Protect my files from programs which run as me and thus with my normal privileges (e.g. very few program would need access to my ssh keys)

Test case:

  • Run a Node.js program which wants to read ~/.test_me and access http://www.google.com. It should not be able to do either unless it’s enabled.

Here the simple Node.js program:

const fs=require('fs');
const fetch=require('node-fetch');

async function accessStuff() {
  try {
    let f=await fs.promises.readFile(`${process.env.HOME}/.test_me`);
    console.log(`File: ${f}`);
  } catch(e) {
    console.log(`Error while accessing .test_me: ${e}`);
  }
  fetch('http://www.google.com', {
                method: 'get',
                })
    .then(res => res.text())
    .then(body => console.log(body.split('\n').slice(0,1)))
    .catch(e => {
      console.error(`Error: ${e}`);
    });
}

(async function () {
try {
  await accessStuff();
} catch(e) {
  console.error(`Error: ${e}`);
}
})();

and a sample run without limitations:

❯ node index.js 
File: test 1

[...many more lines from .test.me...]
[
  `<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage"
[...some HTML code from www.google.com...]

When it comes to security and sandboxing, those choices came up after a quick check with Google:

System-wide Mandatory Access Control (MAC)

SELinux needs special policies/contexts set up for the whole system. While this is great, it’s something root does. I see no simple way to do ad-hoc configurations to run a single command with the “correct” permissions. Plus the policy files are not easy to read nor write.

AppArmor is similar. A bit easier to read policy files, but they are all root owned, so not suitable for ad-hoc commands.

Both have a point to secure the complete system with the user explicitly not allowed to change the policies. Their purpose it not to protect the user from hurting themselves.

Sandbox Tools

Docker or containers in general provide a good way of isolation from the rest of the system and via bind mounts you can allow access to files or directories easily, but you have to create a container image first, upload it to a container registry and download and run it (Update: turns out that this is not required and a locally created image can be executed without problems). While it has its use, creating containers is a significant overhead if it’s needed for every program you are suspicious about.

minijail from Google looks good:

Minijail […] provides an executable that can be used to launch and sandbox other programs, […]

https://google.github.io/minijail/

Installing on Debian was straightforward (needs kernel-headers and libcap-dev). Running a command with a specific user-definable policy is possible:

 # minijail0 -S /usr/share/minijail0/$(uname -m)/cat.policy -- \\
             /bin/cat /proc/self/seccomp_filter
but the examples/ directory was a small shock to me: a single example, and not a well explained one.

❯ cat examples/cat.policy 
# In this directory, test with:
# make LIBDIR=.
# ./minijail0 -n -S examples/cat.policy -- /bin/cat /proc/self/status
# This policy only works on x86_64.

read: 1
write: 1
restart_syscall: 1
rt_sigreturn: 1
exit_group: 1

open: 1
openat: 1
close: 1
fstat: 1
# Enforce W^X.
mmap: arg2 in ~PROT_EXEC || arg2 in ~PROT_WRITE
fadvise64: 1

While there is a tool to record the uses system calls (via strace) to create a policy (similar to what SELinux’s audit2allow tool), that means running a potentially harmful program once without restrictions. Plus the policy file is not exactly easy to understand. And the documentation does not help.

This is a dead end for my purpose.

bubblewrap is used as security layer for FlatPack installations before it was spun out. Using it is very command-line-option intensive, but a wrapper script will handle this. A test run:

❯ cat bwrap.test 
#!/bin/sh

bwrap \
 --dev /dev \
 --ro-bind /lib /lib \
 --ro-bind /usr/bin /usr/bin \
 --ro-bind /bin /bin \
 --ro-bind /etc/resolv.conf /etc/resolv.conf \
 --ro-bind $HOME/js $HOME/js \
 --ro-bind $HOME/.test_me $HOME/.test_me \
 --tmpfs /tmp \
 --unshare-all \
 --share-net \
~/js/node/bin/node index.js

❯ ./bwrap.test 
~/.test_me contains: test 1
[
  `<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage"
[...some more HTML code...]

Removing the “–share-net” and removing the “–ro-bind” for .test_me stops both from being accessible:

❯ ./bwrap.test 
Error while accessing .test_me: Error: ENOENT: no such file or directory, open '/home/harald/.test_me'
Error: FetchError: request to http://www.google.com/ failed, reason: getaddrinfo ENOTFOUND www.google.com

Note that you also need /etc/resolv.conf too to allow resolving DNS names. Also the order of “–unshare-all” and “–share-net” is important as the last one wins.

firejail is conceptually similar to bubblewrap, but beside having a large list of command line options, it also has configuration files in /etc/firejail/ and it also allows user-owned configurations (default in ~/.config/firejail):

❯ cat ~/.config/firejail/nodejs.profile
whitelist /home/harald/js
#whitelist /home/harald/.test_me
net none
#quiet
include /usr/local/etc/firejail/whitelist-common.inc
include /usr/local/etc/firejail/default.profile

❯ firejail --profile=~/.config/firejail/nodejs.profile node index.js
Reading profile /home/harald/.config/firejail/nodejs.profile
Reading profile /usr/local/etc/firejail/whitelist-common.inc
Reading profile /usr/local/etc/firejail/default.profile
Reading profile /usr/local/etc/firejail/disable-common.inc
Reading profile /usr/local/etc/firejail/disable-passwdmgr.inc
Reading profile /usr/local/etc/firejail/disable-programs.inc
Parent pid 231521, child pid 231522
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Warning: cleaning all supplementary groups
Child process initialized in 94.84 ms
Error while accessing .test_me: Error: ENOENT: no such file or directory, open '/home/harald/.test_me'
Error: FetchError: request to http://www.google.com/ failed, reason: getaddrinfo ENOTFOUND www.google.com

Parent is shutting down, bye...
❯ firejail --quiet --net=none node index.js
Error while accessing .test_me: Error: EACCES: permission denied, open '/home/harald/.test_me'
Error: FetchError: request to http://www.google.com/ failed, reason: getaddrinfo ENOTFOUND www.google.com

The last sample shows that you don’t need to create a separate profile but similar to bwrap you can use command line options for most settings.

Uncommenting the “whitelist /home/harald/.test_me” line allows access to that file. Commenting out the “net none” allows network access. Per default network access is granted, but you can change this in /etc/firejail/default.profile. Once disabled in a profile, it cannot be re-enabled though. (Update: A “–ignore=net” option will ignore the “net none” in a profile).

After above tests I found out you can skip the “–profile=~/.config/firejail/PROFILENAME” if PROFILENAME is the binary name plus “.profile” as firejail will pick this up automatically. Very neat!

❯ firejail node index.js 
Reading profile /home/harald/.config/firejail/node.profile
Reading profile /etc/firejail/whitelist-common.inc
[...]

And you can make it less verbose too and with sensible defaults you don’t even need to create any profiles. E.g. shell history files are inaccessible by default:

❯ firejail --quiet bash
$ cd
$ cat .bash_history 
cat: .bash_history: Permission denied
$ ls -la .bash_history
-r-------- 1 nobody nogroup 0 Dec 30 23:39 .bash_history

My Conclusion

SELinux and AppArmor are not something users can manage by themselves. Different scope than what I am looking for.

Using containers, especially when running as non-root works as long as you want to use containers anyway. Otherwise it’s a huge overhead: create container image, store it in a registry, and then run it. Any code changes would need a new container image to be created. Good for certain workload, especially those which will run as containers anyway later on. While I use containers extensively, a lot of programs I run are not a container.

bubblewrap works. It needs an extensive list of options to be useful. That’s not hard to put into a script. Since you have to add a lot of options and there’s no default options you can specify, it’s very explicit about permissions which makes it easier to debug since everything is configured when running your suspicious program. As the order of options is important, I can see this getting complicated quickly for non-trivial programs. Here is an example. Luckily most programs are trivial: few accesses are needed plus some capabilities like network access.

firejail got the spot between security and ease-of-use right in my opinion: sensible defaults (e.g. disabling at and crontab commands) with profiles for many programs. You can also have user-configurable profiles and they are not hard to create. The amount of extra work when using firejail is low: just adding “firejail” before the command helps a lot already out-of-the-box by hiding sensitive files and disabling miss-usable commands. Creating a specific profile makes this very configurable. And if you name the profiles like the binary you plan to use, it’s both simple to use while still being configurable.

Note that no solution is 100% secure. There’s always a trade-off between convenience and security. Unless you enforce it, if it’s inconvenient, it won’t be done.

PS: While testing I experienced firejail to not be able run programs which have capabilities set if you use “caps.drop all” which is included in the default profile. See bug report. Can’t say yet if it’s a bug or badly worded option or lack of documentation or just unexpected behavior.

Humanoid Hunt! In Dart!

After doing a quick&dirty Node.js implementation of the Humanoid Hunt puzzle, I tried the same in Dart. It’s surprisingly similar, but this time I tried to create tests and I wanted to see what will be the result when I try the “real” puzzle.

And ‘lo and behold: after creating all the logic and test to confirm them working as expected, the first try with the actual puzzle data worked! Makes a good point about unit tests being fully capable to verify the logic.

That said, it took me easily twice as much time to write the tests and the code (as expected). But for any further changes, I can be assured to not have broken something.

Humanoid Hunt!

Sometimes advertising is fun. I turned off my ad-blocker for certain web pages, and while I usually get boring advertising, once in a while you get a good and interesting looking one.

In this case: https://hunt.reaktor.com/ : it’s a programming quest similar to Google Foobar Challenge, but much shorter and simpler, yet with a very similar sense of story telling and fun.

Here the 3 challenges I had and solved via Node.js (just because I can). No solutions here in this post. Maybe some spoilers.

First one. Simple decoding a number written in binary ASCII and simple processing the decoded array. The lines are 40 bytes each (=320 zeros or ones). You could do this manually.

Second one. Counting how many times characters were used. The signal is 30k in size.

Third one took me longest (as it should). Some hints: The two-dimensional space is 128×128 large. If you print it out, you’ll see the start near the top left corner and the finish area is on the top right (multiple finish points end there).

Looks like a maze, though, so first attempt of a simple dead-end-elimination did not work well. What works is flood-fill. https://www.youtube.com/watch?v=Zwh-QNlsurI has a nice explanation for this.

Out of the 11 finishes, 4 worked. The distances were 1071, 1072, 1073 and 1074 steps. If you look at the displayed maze, you’ll immediately spot the 7 false finishes (and not only as I marked them in red). So make sure you flood-fill starting at the finish.

As for the run time for the last test: it’s only 0.8s on a Ryzen 5 Pro 3400GE. The code itself is 280 lines long with 110 lines being the input signal data.

Summary

Complexity is low enough to solve them yourself, even if you did not attend computer science classes. This is a great method to weed out those who claim to be able to program but in reality cannot. And it’s fun so that some people (like me) do this for…fun! I was also curious about the story after the first very simple challenge.

I was a bit disappointed about it ending after only 3 puzzles. Google has 5 challenges (or more, as I could not solve the 5th one), so I expected harder and harder puzzles, but for the purpose of screening potential employees, this is sufficient. Google certainly can set the bar higher as their skill requirements are higher too.

I liked that the implementation language was free to choose. Google’s Foobar is limited to Java or Python only. Java is not my thing and Python I don’t enjoy that much.

All in all, good fun for an idle day! Recommended.

Create your website with WordPress.com
Get started