Dart and FFI and ffigen

While exploring how Go uses USB HID, I wondered if I can do the same in Dart. Node.js can do this via node-hid and it’s using its NAPI interface for this. Tested and worked: I could connect to the PS3 controller and turn on LEDs and motors at will. But what about Dart?

Dart has a new FFI interface since 2.12 and I never used it. Time to learn how this works in detail!

Unfortunately the second example I looked at (after hello_world()) was not using ffigen, but from what I read, it’s easier to use unless you prefer to create the Dart-C-bindings manually.

So I took the non-ffigen-using-example and ffigen’ified it: https://github.com/haraldkubota/dart-ffi

Notes: The varargs example does not work. The original sample here at line 36 even cheats: it defines MultiSum with 4 parameters. While this works, it won’t work with 3 or 5 parameters. So I skip this because I don’t think Dart can do varargs currently.

U2F on the CLI

U2F works well and easily via a web browser, but you can also use it directly on the command line. You “just” have to implement the USB protocol part of U2F, namely talk to /dev/hidrawX.

u2fcli did that and it worked on my R2S (ARMv8):

harald@r2s2:~/git$ git clone git@github.com:mdp/u2fcli.git
Cloning into 'u2fcli'...
remote: Enumerating objects: 57, done.
remote: Total 57 (delta 0), reused 0 (delta 0), pack-reused 57
Receiving objects: 100% (57/57), 19.26 KiB | 1.20 MiB/s, done.
Resolving deltas: 100% (21/21), done.
harald@r2s2:~/git$ cd u2fcli
harald@r2s2:~/git/u2fcli$ go mod init u2fcli
go: creating new go.mod: module u2fcli
go: to add module requirements and sums:
        go mod tidy
harald@r2s2:~/git/u2fcli$ go mod tidy
go: finding module for package github.com/flynn/u2f/u2ftoken
go: finding module for package github.com/flynn/hid
go: finding module for package github.com/mdp/u2fcli/cmd
go: finding module for package github.com/flynn/u2f/u2fhid
go: finding module for package github.com/spf13/cobra
go: found github.com/mdp/u2fcli/cmd in github.com/mdp/u2fcli v0.0.0-20180327171945-2b7ae3bbca08
go: found github.com/flynn/hid in github.com/flynn/hid v0.0.0-20190502022136-f1b9b6cc019a
go: found github.com/flynn/u2f/u2fhid in github.com/flynn/u2f v0.0.0-20180613185708-15554eb68e5d
go: found github.com/flynn/u2f/u2ftoken in github.com/flynn/u2f v0.0.0-20180613185708-15554eb68e5d
go: found github.com/spf13/cobra in github.com/spf13/cobra v1.2.1
harald@r2s2:~/git/u2fcli$ go build
harald@r2s2:~/git/u2fcli$ ls
cmd  go.mod  go.sum  LICENSE  main.go  README.md  u2fcli

Permissions for /dev/hidrawX needs to be given:

harald@r2s2:~/git/u2fcli$ sudo chmod a+rw /dev/hidraw0

And now a full cycle of register (once), sign+verify (log in):

harald@r2s2:~/git/u2fcli$ ./u2fcli reg --challenge MyComplexChallenge --appid https://test.com
Registering, press the button on your U2F device #1 [Yubico Security Key by Yubico]{
  "KeyHandle": "-374aUcG7iWqVc5rsX8jE_8yr1iS-EEDdt106-CAKec90Gg1VVK9dv5E_JmZRIyKVaas9vhLVHb7zbbJ6rNltg",
  "PublicKey": "BHBwVKLRYZZKZGaL96FQtzis8i01M2DMw4IQwuMIKbWa2dZJSC1GlXlYiWhycig4R3DdlipdR675o_e4QfpI-UU",
  "RegisteredData": "-374aUcG7iWqVc5rsX8jE_8yr1iS-EEDdt106-CAKec90Gg1VVK9dv5E_JmZRIyKVaas9vhLVHb7zbbJ6rNltjCCAr4wggGmoAMCAQICBHSG_cIwDQYJKoZIhvcNAQELBQAwLjEsMCoGA1UEAxMjWXViaWNvIFUyRiBSb290IENBIFNlcmlhbCA0NTcyMDA2MzEwIBcNMTQwODAxMDAwMDAwWhgPMjA1MDA5MDQwMDAwMDBaMG8xCzAJBgNVBAYTAlNFMRIwEAYDVQQKDAlZdWJpY28gQUIxIjAgBgNVBAsMGUF1dGhlbnRpY2F0b3IgQXR0ZXN0YXRpb24xKDAmBgNVBAMMH1l1YmljbyBVMkYgRUUgU2VyaWFsIDE5NTUwMDM4NDIwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAASVXfOt9yR9MXXv_ZzE8xpOh4664YEJVmFQ-ziLLl9lJ79XQJqlgaUNCsUvGERcChNUihNTyKTlmnBOUjvATevto2wwajAiBgkrBgEEAYLECgIEFTEuMy42LjEuNC4xLjQxNDgyLjEuMTATBgsrBgEEAYLlHAIBAQQEAwIFIDAhBgsrBgEEAYLlHAEBBAQSBBD4oBHzjApNFYAGFxEfntx9MAwGA1UdEwEB_wQCMAAwDQYJKoZIhvcNAQELBQADggEBADFcSIDmmlJ-OGaJvWn9CqhvSeueToVFQVVvqtALOgCKHdwB-Wx29mg2GpHiMsgQp5xjB0ybbnpG6x212FxESJ-GinZD0ipchi7APwPlhIvjgH16zVX44a4e4hOsc6tLIOP71SaMsHuHgCcdH0vg5d2sc006WJe9TXO6fzV-ogjJnYpNKQLmCXoAXE3JBNwKGBIOCvfQDPyWmiiG5bGxYfPty8Z3pnjX-1MDnM2hhr40ulMxlSNDnX_ZSnDyMGIbk8TOQmjTF02UO8auP8k3wt5D1rROIRU9-FCSX5WQYi68RuDrGMZB8P5-byoJqbKQdxn2LmE1oZAyohPAmLcoPO4wRgIhANIZ7Q_cty_UkWigyQ7Ot0pC0egyI_eSUJ52Hge95vz1AiEAzf7hX_XvNQvoPQ2IvjgJjUkV3wvDPctkac2Z_8fRaik"
}

harald@r2s2:~/git/u2fcli$ ./u2fcli sig --appid https://test.com --challenge SomethingElse --keyhandle "-374aUcG7iWqVc5rsX8jE_8yr1iS-EEDdt106-CAKec90Gg1VVK9dv5E_JmZRIyKVaas9vhLVHb7zbbJ6rNltg"
Authenticating, press the button on your U2F device
{
  "Counter": 50,
  "Signature": "AQAAADIwRQIhALlZyMmormC2b9JCaOXYAdKq4wvpdKg4wMu68fLgXmclAiADDHbFxKrm5eYCoCvC-m1vEEegXzWHfwuPLpUh81qHoA"
}

harald@r2s2:~/git/u2fcli$ ./u2fcli ver --appid https://test.com --challenge SomethingElse --publickey "BHBwVKLRYZZKZGaL96FQtzis8i01M2DMw4IQwuMIKbWa2dZJSC1GlXlYiWhycig4R3DdlipdR675o_e4QfpI-UU" --signature "AQAAADIwRQIhALlZyMmormC2b9JCaOXYAdKq4wvpdKg4wMu68fLgXmclAiADDHbFxKrm5eYCoCvC-m1vEEegXzWHfwuPLpUh81qHoA"
Signature verified

E-Bike!

I needed (wanted?) a new bicycle. The old one was still ok, but severely outdated in many ways and while I could replace and upgrade some parts, it just made no financial sense to do that. Thus a new bicycle it was. Assisted or not was an open question for a long time. I like those points:

  • easier climbing up hills (there’s plenty here)
  • being able to go further

I did not like those points though:

  • additional stuff which can break or needs maintenance (motor, battery)
  • additional weight
  • need to charge
  • price

In the end it became the Specialized Vado SL 4.0 EQ (2021 model) as it was almost perfect: it’s relatively light for an e-bike, without assist function it’s pretty much a normal bicycle (the motor does not interfere), and the price is, while not cheap, acceptable. The 2022 model is quite a bit more expensive and I would not have bought that one. I really wanted a removable battery as it would make charging easier, but it looks better as it is. I can live with that.

My new bicycle. The bag is an Ortlieb Single-Bag.

Update after some more tests: The motor is very smooth. It kicks in quickly at low speeds, which is important when starting from a stand-still, and by law it slowly decreases power output when you get over 10km/h. At 25km/h it disconnects completely. No jerking, no sudden drop in speed. And it kicks back in when I go below 25km/h and again it’s very subtle. Very smooth. Some engineers did a fabulous job here.

WiFi Signal Strength Graphs

WiFi generally works well, but sometimes it does not and I’d like to know why: Is the signal weak? Is the noise floor increasing? To measure that, get the data and collect and make nice graphs out of it!

Gather data

Mikrotik SXT Lite5

You need a method to get raw data out of your wireless device (Mikrotik SXT Lite5). In my case:

  • Create an account on the SXT to log in (via ssh key, see here)
  • A script to gather the data (I use telegraf with the InfluxDB line protocol). The REST API of RouterOS 6.x is unfortunately not yet usable.
#!/bin/bash
set -euo pipefail

# gather_one(router,interface)
function gather_one {
  ssh -i ~/.ssh/id_sxt17 telegraf@$1 "/interface wireless monitor $2 once" | tr -d '\r' >/tmp/f1
  radio_name=$(awk '/^ *radio-name:/{print $2}' /tmp/f1)
  if [[ -z "$radio_name" ]] ; then
    radio_name=$2
  fi

  tx=$(awk '/^ *tx-ccq:/{print $2/100}' /tmp/f1)
  rx=$(awk '/^ *rx-ccq:/{print $2/100}' /tmp/f1)
  tx_overall=$(awk '/^ *overall-tx-ccq:/{print $2/100}' /tmp/f1)
  nf=$(awk '/^ *noise-floor:/{print $2/1}' /tmp/f1)
  sn=$(awk '/^ *signal-to-noise:/{print $2/1}' /tmp/f1)

  if [[ ! -z "$rx" ]] ; then echo "signal,host=$1,radio=$radio_name rx=$rx" ; fi
  if [[ ! -z "$tx" ]] ; then
    echo "signal,host=$1,radio=$radio_name tx=$tx"
  else
    echo "signal,host=$1,radio=$radio_name tx_overall=$tx_overall"
  fi

  if [[ ! -z "$sn" ]] ; then echo "signal,host=$1,radio=$radio_name sn=$sn" ; fi
  if [[ ! -z "$nf" ]] ; then echo "signal,host=$1,radio=$radio_name nf=$nf" ; fi
}


gather_one sxt17.lan wlan1-gateway
gather_one router.lan wlan1
gather_one router.lan wlan2
  • telegraf.conf needs to run the above script and feed the data into InfluxDB:
[[inputs.exec]]
  commands = ["/home/harald/bin/wlan-influxdb.sh"]
  timeout = "10s"
  data_format = "influx"

Display Data

Create a new panel in Grafana:

Top left panel definition

And here the resulting graphs:

Signal Strength, Noise Floor and Signal Quality

Note that the various hardware variations don’t always show the same raw data about the wireless interfaces. E.g. my router does not show the signal strength, only the noise floor.

Mikrotik RouterOS and TLS

Browsers nowadays complain (rightly) about HTTP being used instead of HTTPS. My Mikrotik routers still use HTTP. Time to fix this!

❯ step ca certificate --provisioner=myCA@home --san=192.168.2.17 \
--san=sxt17.lan --not-after=8760h \
sxt17.lan sxt17.cer sxt17.key \
--provisioner-password-file ~/.step/pass/provisioner_pass.txt

❯ scp sxt17.key sxt17.cer admin@sxt17.lan:
❯ ssh admin@sxt17.lan

/certificate import file-name=sxt17.cer name=sxt17.lan passphrase=""
/certificate import file-name=sxt17.key passphrase=""
/file remove sxt17.cer
/file remove sxt17.key
/ip service set www-ssl certificate=sxt17.lan disabled=no
^D

Note that the certificate is valid for a year instead of the default of 24h. Renewing is simple:

❯ step ca renew --force ./sxt17.cer ./sxt17.key

❯ scp sxt17.key sxt17.cer admin@sxt17.lan:
❯ ssh admin@sxt17.lan

/certificate import file-name=sxt17.cer name=sxt17.lan passphrase=""
/certificate import file-name=sxt17.key passphrase=""
/file remove sxt17.cer
/file remove sxt17.key
^D

If the REST API of Router OS would work, I could use those instead of using ssh, but they don’t work for me. Seems version 7 of RouterOS will work with the REST API but I am not adventurous enough to verify that.

XKCD Userscript

Learning is easiest with a goal. The goal here was to recreate this and expand it a bit. Nothing earth-shaking. Still learning how to use querySelector().

To install, have Tampermonkey installed and click here.

Here the script:

// ==UserScript==
// @name         Display XKCD IMG title
// @namespace    https://hkubota.wordpress.com/
// @downloadURL  https://raw.githubusercontent.com/haraldkubota/xkcd-userscript/main/display_title.user.js
// @supportURL   https://hkubota.wordpress.com/2021/09/04/xkcd-userscript/
// @version      0.1
// @description  Show the IMG title and Explain XKCD Link
// @author       Harald Kubota
// @match        http*://*xkcd.com/*
// @icon         https://www.google.com/s2/favicons?domain=xkcd.com
// @grant        none
// ==/UserScript==

(function() {
    window.addEventListener('load', () => {
        const titleElement = document.querySelector('#comic [title]');
        if (titleElement) {
            const title = titleElement.title;
            let p = document.createElement('p');
            p.innerText = title;
            document.querySelector('#comic').append(p);
            const mystyle = {
                "font-variant": "none",
                "background": "lightgray",
                "padding": "10px"
            };
            Object.assign(p.style, mystyle);
        }
        const currentComic = document.querySelector('div#middleContainer.box > a');
        if (currentComic) {
            let uriPath = currentComic.text.split('/');
            let currentComicNumber=1;
            for (let i=uriPath.length-1; i>=0; --i) {
                let n = parseInt(uriPath[i]);
                if (!isNaN(n)) {
                    currentComicNumber = n;
                    break;
                }
            }
            let div = document.createElement('div');
            let a = document.createElement('a');
            var link = document.createTextNode('https://www.explainxkcd.com/wiki/index.php/'+currentComicNumber);
            a.appendChild(link);
            a.setAttribute('href', 'https://www.explainxkcd.com/wiki/index.php/'+currentComicNumber);
            div.appendChild(a);
            document.querySelector('#comic').append(div);
        }
    }, false);
})();

Now it looks like this:

The elements with the red arrows are added via above userscript

Deno and ES6 Modules

Continuing my expedition into the JavaScript front-end land…I like Deno. A lot. It does lots of things right (from my point of view, e.g. permissions), and it fixes many problem Node.js has. Using Deno for back-end work is straightforward, but since I focus currently on the front-end side, how can I create front-end JavaScript with Deno and still use Deno’s testing framework?

It’s actually not hard. Here a small TypeScript file:

import capitalize from "https://unpkg.com/lodash-es@4.17.15/capitalize.js";

function main() {
  console.log(capitalize("hello from the web browser"));
}

function sum(a: number, b: number): number {
    return a+b;
}

window.onload = () => {
  console.info(capitalize("module loaded!"));
};

export { main, sum }                                                                                                   

embedded in a simple index.html file:

<!DOCTYPE html>
<html>
<head>
    <title>Testing Deno</title>
</head>
<body>
    <div>Some Text</div>
<script type="module">
    import {main, sum} from './dist/browser.js';
    main();
    console.log(sum(5,6));
    console.log("Done.");
</script>
</body>
</html>

So far that’s straightforward: loading index.html runs main() and prints out the sum of 5+6 on the console.

Here the test script:

import { assertEquals } from "https://deno.land/std@0.106.0/testing/asserts.ts";

import { sum } from "./example.ts";

Deno.test("Sum 5 and 7", () => {
  assertEquals(sum(5, 7), 12);
});

Again, standard Deno. And deno test works as expected. And to deploy as plain-JavaScript, a simple deno bundle src/example.ts dist/browser.js does the trick.

And there you go: plain TypeScript and yet the logic can be tested with the normal deno test command. No extra tools needed. No Babel, WebPack, tsc. No require vs import either.

deno_dom is not yet working enough though (see the stub definition of addEventListener() here), so until then only basic DOM operations work.

Jest and ES6 Modules and userscripts

Greasemonkey (and thus userscripts) started when JavaScript was old and jQuery was well used. Since ES6 (ES2015) we have module imports, ES2017 brought us async/await and generally plenty useful features which should be used on modern browsers. Since Tampermonkey is only for newer browsers, we can use all those nice features instead of relying on old methods. Also testing…it’s a good thing once you get used to it. So developing userscripts nowadays should mean:

  • create testable code
  • use ES2017 code with module imports and async/await and fetch
  • no need for Babel and WebPack or jQuery

In the past I pretty much ignored the front-end, but now I have a use-case at work, so time to get the coding started: https://github.com/haraldkubota/userscripts-jest

Tampermonkey userscripts

After many years of wanting to use userscripts, I finally have a use case! Time to learn about how to create those. Docs for the API and headers are here.

Helpful idioms (from here):

Userscript idioms and code snippets

  • Query and remove a page element:
document.querySelector("#element_id").remove();
  • Store an object:
GM_setValue("MY_OBJECT_KEY", JSON.stringify(myObject));
  • Retrieve an object:
var myObject = JSON.parse(GM_getValue("MY_OBJECT_KEY", "{}"));
  • Replace the current URL and redirect the page:
var link = document.URL.replace("domain.fr", "domain.com");
window.location.href=link;
  • Add a button to execute some function on the page:
var btn = document.createElement("button");
btn.innerHTML = "My button";
btn.onclick = () => {
    alert("My button clicked !");
    return false;
};
document.querySelector("btn_predecessor_selector").after(btn);
  • Append a style to the page:
var css = "h1 { background: red; }"
var style = document.createElement("style");
style.type = "text/css";
style.appendChild(document.createTextNode(css));
document.head.appendChild(style);
  • Fetching and parsing an external page:
fetch("http://example.com/path-name")
.then(response => response.text())
.then(text => {
    var parser = new DOMParser();
    var htmlDocument = parser.parseFromString(text, "text/html");
    var content = htmlDocument.documentElement.querySelector("element_selector");
    alert("My fetched element content: " + content.textContent);
});

Deno on ARMv8 (Linux)

Running Deno on ARMv8 on Linux has one problem: there’s no released binary (see here for v1.13 which is the current version when I’m typing this). Luckily it’s very easy to build Deno: rustup and cargo install is all you need…

Unfortunately my little R2S with 1GB RAM is way too under-powered for this: E.g. the cargo build uses 2.5GB in /tmp. /tmp by default is 50% of physical RAM, so 0.5GB, which is way too small. While I could work around this by mounting a USB stick to the USB 2.0 port of the R2S, it won’t be fast.

AWS t4g.medium to the rescue! Spin up a t4g.medium instance (I use Terraform), and then execute

sudo yum groupinstall 'Development Tools'
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install deno

and some not-so-short time later you got an ARMv8 binary in ~/.cargo/bin:

[ec2-user@ip-172-31-45-62 ~]$ ls -l ~/.cargo/bin/
total 272456
-rwxr-xr-x 12 ec2-user ec2-user  14255928 Aug 12 09:13 cargo
[...]
-rwxrwxr-x  1 ec2-user ec2-user 107928560 Aug 12 09:59 deno
[...]

Copy that binary and it works fine on my little R2S.

Create your website with WordPress.com
Get started