One Million Robbery Queries

Today I had the very unpleasant surprise to find out that over the last month we had nearly one million requests from ChatGPT scraping bots. That was especially the case on our photos gallery website where they made a request every single second to check if there were any new pictures to steal so that our cats and dogs could be featured in an AI generated image.

First step was to ban them, but that might not be sufficient as this is just a random IP block within a Azure DC. Ironically, I asked ChatGPT to generate me a robots.txt file that bans the ChatGPT scraping bots. Here it is:

User-agent: ChatGPT-User
Disallow: /
User-agent: ChatGPT
Disallow: /
User-agent: GPTBot
Disallow: /

FreeBSD modules not loading correctly on ARM64

After upgrading to FreeBSD 14.2, I encountered a perplexing issue with kernel modules built from ports. They would load and show up in kldstat, but no message nor sysctl node would be created. In fact, it was as if the event_handler would not be called at all, yet it compiled and loaded successfully. On the other hand, modules shipped with the kernel and already compiled were working as intended. To investigate, I built a small test module:

#include <sys/param.h>
#include <sys/module.h>
#include <sys/kernel.h>
#include <sys/systm.h>

static int test_event_handler(module_t mod, int event, void *arg) {
    printf("Test module loaded, event: %d\n", event);
    return (0);
}

static moduledata_t test_mod = {
    "testmodule",
    test_event_handler,
    NULL
};

DECLARE_MODULE(testmodule, test_mod, SI_SUB_LAST, SI_ORDER_ANY);
MODULE_VERSION(testmodule, 1);

On FreeBSD 14.2 amd64, it would load and show the message in the log, whereas on FreeBSD 14.2 arm64, it would load but with no output. Yet, disassembling the module, the event_handler code was just there.

After some investigation, I found out that while the source were compiled with /usr/bin/cc (llvm clang), they were linked with /usr/local/bin/ld (GNU binutils ld). Uninstalling binutils and compiling again with both /usr/bin/cc and /usr/bin/ld, the module would load and show in the log. Why it only appeared with FreeBSD 14.2, however, is still a mystery.

Install awscli2 on FreeBSD 14.1-p3

Whilst I often use the awscli on Linux or mac for which there are binary installers, I also tend to work on personal projects on FreeBSD. Unfortunately this OS is not supported by AWS. Some would recommend to use the Linuxulator but surely running a native version would be better. So here is a quick step-through of how I got it running on FreeBSD 14.1-p3 with python311 thanks mostly to this github issue and also this one. Although this is done with python311 and py311-pip, you can probably use the same method for older version of python down to python39.

First we need to clone awscli2 from https://github.com/aws/aws-cli. Be careful to select the v2 branch.

git clone https://github.com/aws/aws-cli.git
cd aws-cli
git checkout v2

Then, I ran into a problem with pyOpenSSL. After installing it, executing awscli returned the following error message: ERR_UNABLE_TO_GET_ISSUER_CERT = _lib.X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT. Thanks to this other issue, it seems like a quick solution was to downgrade the version of pyOpenSSL (but this step might not be necessary to you, future reader blessed with a fix):

sudo pip install pyOpenSSL==23.1.0

Then it’s a matter of installing the requirements and building/installing the command line tool (note that it needs gcc tho, in my case gcc13-13.3.0):

sudo pip install -r requirements.txt
sudo CC=/usr/local/bin/gcc pip install -e .

And tada!

» aws --version
aws-cli/2.17.49 Python/3.11.9 FreeBSD/14.1-RELEASE-p3 source/amd64

Gesture lost on NeoReader

If you have one of those Onyx Boox e-ink tablet with NeoReader to read books and documents, you might have toyed with the settings and inadvertently disabled all gestures, long click, the floating bar and you are stuck without the ability to access the application settings, let alone turning pages left or right.

If you find yourself in this dire situation, you might have read on the Internet that the only solution left would be to reset the application entirely, deleting the application cache and data, losing all your settings in the process.

But fear not, because there is another way! Should you be able to connect an external keyboard to your tablet, via bluetooth for instance, go into the stuck NeoReader app and press F1 (or fn+F1 if your keyboard requires so). And voilà, the settings menu will pop up allowing you to restore the gestures and have the application usable again.

Install Arch Linux on EFI

Most what you will find in this post comes from this gist. I’m rewritting this here as a note in any case. Some more info about the installation process here on ArchWiki and also more info about the post-installation process.

  1. Boot USB flash drive and make sure it’s connected via Ethernet.
  2. Change terminal keys if you are not in qwerty:
    loadkeys fr
  3. Disable the beeping sound (this one will save your ears and sanity):
    setterm -blength 0
  4. Check if the system was booted with UEFI:
    cat /sys/firmware/efi/fw_platform_size

    It should exists and be 64 if it’s booted in UEFI x86_64. If that’s the case, continue.

  5. Check that you have an IP address and try a ping to check your Internet connectivity:
    ip address
    ping 8.8.8.8
    
  6. Update system clock and check status:
    timedatectl set-ntp true
    timedatectl status
  7. Enable SSH, this might be useful if you want to continue the installation from elsewhere or transfer files via sftp:
    systemctl start sshd
  8. List the disks then proceed with creating the partitions:
    fdisk -l
    cfdisk /dev/sda
    

    Create a EFI partition of 256M to 512M, a several GB swap partition and what is left with a Linux root partition.

  9. Format the partitions:
    mkfs.fat -F32 /dev/sda1
    mkfs.ext4 -L root -m 0 /dev/sda3
    
  10. Mount the root partition:
    mount -o noatime /dev/sda3 /mnt
    
  11. Install the base packages:
    pacstrap -Ki /mnt base linux linux-firmware
    
  12. Generate the fstab:
    genfstab -U -p /mnt >> /mnt/etc/fstab
    
  13. Chroot in the filesystem:
    arch-chroot /mnt
    
  14. Configure the terminal keyboard:
    vim /etc/vconsole.conf
  15. Set the timezone:
    ln -sf /usr/share/zoneinfo/Europe/Brussels /etc/localtime
    
  16. Update the hardware clock:
    hwclock --systohc
    
  17. Install other packages:
    pacman -S grub efibootmgr dosfstools openssh os-prober mtools net-tools inetutils netctl dhcpcd dhclient vim
    
  18. Edit and set-up the locale:
    vim /etc/locale.gen
    locale-gen
    
  19. Setup root password:
    passwd
    
  20. Create and mount EFI directory:
    mkdir /boot/EFI
    mount -o noatime /dev/sda1 /boot/EFI
  21. Time to install the GRUB bootloader and write the config:
    grub-install --target=x86_64-efi --bootloader-id=grub_uefi --recheck
    grub-mkconfig -o /boot/grub/grub.cfg
    
  22. If needed you might configure an extra entry within the GRUB boot list, for instance for a dualboot with FreeBSD. To that end, edit /etc/grub.d/40_custom and add (at the end of this file):
    menuentry FreeBSD {
      insmod ufs2
      set root='(hd0,gpt3)'
      chainloader /boot/loader.efi
    }
    

    Then update the grub configuration with:

    grub-mkconfig -o /boot/grub/grub.cfg
  23. Time to reboot:
    exit
    reboot
    

Charade 💥

We’ll meet againDon’t know whereDon’t know whenBut I know we’ll meet again some sunny day
Keep smiling throughJust like you always do‘Til the blue skies drive the dark clouds far away

So will you please say helloTo the folks that I knowTell them I won’t be longThey’ll be happy to knowThat as you saw me goI was singing this song

We’ll meet againDon’t know whereDon’t know whenBut I know we’ll meet again some sunny day

Today’s movie: The Boy and the Heron

How do you live?
The Boy and the Heron – 君たちはどう生きるか by Hayao Miyazaki, better titled in its original title translation “How do you live?” (based on a novel of the same name). That title alone could easily resume most of Studio Ghibli movies. But in this case, it fits even more.

So then, how do you live? How do you navigate the difficulties in life and see through it, opening new doors, then new paths, then new worlds? How do you negotiate with your own difficulties and losses, and then how do you negotiate with the difficulties of others? And that is one of the main takeaway here, it might be hard, so hard that we are all cowards in our own ways, but it’s not only about you. But then again, how do you do it?

One key clue, is that first and foremost, you ought to be sincere. The second one, is that you ought to build everything upon that, because this was but the foundation. Then you make choices, and through those choices you battle and get lost, and found, and lost and found again. Give up now and you’ll have so many occasions to give up later.

Where’s the good ending then? Well, there is none. Like most Ghibli movies to be honest. You are just put there, with all the things you have learned, and then invited to get on with it, left with a weird sense of wonder, sadness and anxiety.

InfluxDB and collectd time mismatch

Recently I’ve been playing a bit with Grafana and InfluxDB data sources populated by collectd for the most part. However when trying to explore the data in InfluxDB, all times were in 1970-01-01. The epoch is often used as a kind of null value in InfluxDB. However in this case, the time value were increasing. The problem was that there was a mismatch between the timestamp sent by collectd and the way InfluxDB was trying to parse them.

Collectd used the write_influxdb_udp write plugin which apparently sends timestamp in milliseconds, but the [[udp]] listener of InfluxDB was probably expecting them to be in nanoseconds. Hence, there factor 1000000 between the two timestamp formats. You can control which format InfluxDB expects on its [[udp]] listener using the following parameter:

# InfluxDB precision for timestamps on received points ("" or "n", "u", "ms", "s", "m", "h")
precision = "ms"

Port configure fails on ARM64

On FreeBSD, if you are trying to build a port but it fails at the configure step with a message similar to this:

checking build system type... Invalid configuration `arm64-portbld-freebsd13.2': machine `arm64-portbld' not recognized
configure: error: /bin/sh ./build-aux/config.sub arm64-portbld-freebsd13.2 failed

Here’s a quick-fix that might work for you:

export CONFIGURE_TARGET=arm64-unknown-freebsd13.2
make install

This is similar to passing --host arm64-unknown-freebsd13.2 to the configure script instead of trying to guess it.