As a server, we get our fair load of vulnerability scans. But it came out recently that it is more and more coming from Azure. Maybe we should ban them all?
Azure not so sure
Reply
One Million Robbery Queries
Today I had the very unpleasant surprise to find out that over the last month we had nearly one million requests from ChatGPT scraping bots. That was especially the case on our photos gallery website where they made a request every single second to check if there were any new pictures to steal so that our cats and dogs could be featured in an AI generated image.
First step was to ban them, but that might not be sufficient as this is just a random IP block within a Azure DC. Ironically, I asked ChatGPT to generate me a robots.txt file that bans the ChatGPT scraping bots. Here it is:
User-agent: ChatGPT-User Disallow: / User-agent: ChatGPT Disallow: / User-agent: GPTBot Disallow: /
FreeBSD modules not loading correctly on ARM64
After upgrading to FreeBSD 14.2, I encountered a perplexing issue with kernel modules built from ports. They would load and show up in kldstat, but no message nor sysctl node would be created. In fact, it was as if the event_handler would not be called at all, yet it compiled and loaded successfully. On the other hand, modules shipped with the kernel and already compiled were working as intended. To investigate, I built a small test module:
#include <sys/param.h>
#include <sys/module.h>
#include <sys/kernel.h>
#include <sys/systm.h>
static int test_event_handler(module_t mod, int event, void *arg) {
printf("Test module loaded, event: %d\n", event);
return (0);
}
static moduledata_t test_mod = {
"testmodule",
test_event_handler,
NULL
};
DECLARE_MODULE(testmodule, test_mod, SI_SUB_LAST, SI_ORDER_ANY);
MODULE_VERSION(testmodule, 1);
On FreeBSD 14.2 amd64, it would load and show the message in the log, whereas on FreeBSD 14.2 arm64, it would load but with no output. Yet, disassembling the module, the event_handler code was just there.
After some investigation, I found out that while the source were compiled with /usr/bin/cc (llvm clang), they were linked with /usr/local/bin/ld (GNU binutils ld). Uninstalling binutils and compiling again with both /usr/bin/cc and /usr/bin/ld, the module would load and show in the log. Why it only appeared with FreeBSD 14.2, however, is still a mystery.
Install awscli2 on FreeBSD 14.1-p3
Whilst I often use the awscli on Linux or mac for which there are binary installers, I also tend to work on personal projects on FreeBSD. Unfortunately this OS is not supported by AWS. Some would recommend to use the Linuxulator but surely running a native version would be better. So here is a quick step-through of how I got it running on FreeBSD 14.1-p3 with python311 thanks mostly to this github issue and also this one. Although this is done with python311 and py311-pip, you can probably use the same method for older version of python down to python39.
First we need to clone awscli2 from https://github.com/aws/aws-cli. Be careful to select the v2 branch.
git clone https://github.com/aws/aws-cli.git cd aws-cli git checkout v2
Then, I ran into a problem with pyOpenSSL. After installing it, executing awscli returned the following error message: ERR_UNABLE_TO_GET_ISSUER_CERT = _lib.X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT. Thanks to this other issue, it seems like a quick solution was to downgrade the version of pyOpenSSL (but this step might not be necessary to you, future reader blessed with a fix):
sudo pip install pyOpenSSL==23.1.0
Then it’s a matter of installing the requirements and building/installing the command line tool (note that it needs gcc tho, in my case gcc13-13.3.0):
sudo pip install -r requirements.txt sudo CC=/usr/local/bin/gcc pip install -e .
And tada!
» aws --version aws-cli/2.17.49 Python/3.11.9 FreeBSD/14.1-RELEASE-p3 source/amd64
Gesture lost on NeoReader
If you have one of those Onyx Boox e-ink tablet with NeoReader to read books and documents, you might have toyed with the settings and inadvertently disabled all gestures, long click, the floating bar and you are stuck without the ability to access the application settings, let alone turning pages left or right.
If you find yourself in this dire situation, you might have read on the Internet that the only solution left would be to reset the application entirely, deleting the application cache and data, losing all your settings in the process.
But fear not, because there is another way! Should you be able to connect an external keyboard to your tablet, via bluetooth for instance, go into the stuck NeoReader app and press F1 (or fn+F1 if your keyboard requires so). And voilà, the settings menu will pop up allowing you to restore the gestures and have the application usable again.
Install Arch Linux on EFI
Most what you will find in this post comes from this gist. I’m rewritting this here as a note in any case. Some more info about the installation process here on ArchWiki and also more info about the post-installation process.
- Boot USB flash drive and make sure it’s connected via Ethernet.
- Change terminal keys if you are not in qwerty:
loadkeys fr
- Disable the beeping sound (this one will save your ears and sanity):
setterm -blength 0
- Check if the system was booted with UEFI:
cat /sys/firmware/efi/fw_platform_size
It should exists and be 64 if it’s booted in UEFI x86_64. If that’s the case, continue.
- Check that you have an IP address and try a ping to check your Internet connectivity:
ip address ping 8.8.8.8
- Update system clock and check status:
timedatectl set-ntp true timedatectl status
- Enable SSH, this might be useful if you want to continue the installation from elsewhere or transfer files via sftp:
systemctl start sshd
- List the disks then proceed with creating the partitions:
fdisk -l cfdisk /dev/sda
Create a EFI partition of 256M to 512M, a several GB swap partition and what is left with a Linux root partition.
- Format the partitions:
mkfs.fat -F32 /dev/sda1 mkfs.ext4 -L root -m 0 /dev/sda3
- Mount the root partition:
mount -o noatime /dev/sda3 /mnt
- Install the base packages:
pacstrap -Ki /mnt base linux linux-firmware
- Generate the fstab:
genfstab -U -p /mnt >> /mnt/etc/fstab
- Chroot in the filesystem:
arch-chroot /mnt
- Configure the terminal keyboard:
vim /etc/vconsole.conf
- Set the timezone:
ln -sf /usr/share/zoneinfo/Europe/Brussels /etc/localtime
- Update the hardware clock:
hwclock --systohc
- Install other packages:
pacman -S grub efibootmgr dosfstools openssh os-prober mtools net-tools inetutils netctl dhcpcd dhclient vim
- Edit and set-up the locale:
vim /etc/locale.gen locale-gen
- Setup root password:
passwd
- Create and mount EFI directory:
mkdir /boot/EFI mount -o noatime /dev/sda1 /boot/EFI
- Time to install the GRUB bootloader and write the config:
grub-install --target=x86_64-efi --bootloader-id=grub_uefi --recheck grub-mkconfig -o /boot/grub/grub.cfg
- If needed you might configure an extra entry within the GRUB boot list, for instance for a dualboot with FreeBSD. To that end, edit
/etc/grub.d/40_customand add (at the end of this file):menuentry FreeBSD { insmod ufs2 set root='(hd0,gpt3)' chainloader /boot/loader.efi }Then update the grub configuration with:
grub-mkconfig -o /boot/grub/grub.cfg
- Time to reboot:
exit reboot
Charade 💥
We’ll meet againDon’t know whereDon’t know whenBut I know we’ll meet again some sunny dayKeep smiling throughJust like you always do‘Til the blue skies drive the dark clouds far away
So will you please say helloTo the folks that I knowTell them I won’t be longThey’ll be happy to knowThat as you saw me goI was singing this song
We’ll meet againDon’t know whereDon’t know whenBut I know we’ll meet again some sunny day
Today’s movie: The Boy and the Heron
So then, how do you live? How do you navigate the difficulties in life and see through it, opening new doors, then new paths, then new worlds? How do you negotiate with your own difficulties and losses, and then how do you negotiate with the difficulties of others? And that is one of the main takeaway here, it might be hard, so hard that we are all cowards in our own ways, but it’s not only about you. But then again, how do you do it?
One key clue, is that first and foremost, you ought to be sincere. The second one, is that you ought to build everything upon that, because this was but the foundation. Then you make choices, and through those choices you battle and get lost, and found, and lost and found again. Give up now and you’ll have so many occasions to give up later.
Where’s the good ending then? Well, there is none. Like most Ghibli movies to be honest. You are just put there, with all the things you have learned, and then invited to get on with it, left with a weird sense of wonder, sadness and anxiety.
InfluxDB and collectd time mismatch
Recently I’ve been playing a bit with Grafana and InfluxDB data sources populated by collectd for the most part. However when trying to explore the data in InfluxDB, all times were in 1970-01-01. The epoch is often used as a kind of null value in InfluxDB. However in this case, the time value were increasing. The problem was that there was a mismatch between the timestamp sent by collectd and the way InfluxDB was trying to parse them.
Collectd used the write_influxdb_udp write plugin which apparently sends timestamp in milliseconds, but the [[udp]] listener of InfluxDB was probably expecting them to be in nanoseconds. Hence, there factor 1000000 between the two timestamp formats. You can control which format InfluxDB expects on its [[udp]] listener using the following parameter:
# InfluxDB precision for timestamps on received points ("" or "n", "u", "ms", "s", "m", "h")
precision = "ms"
Port configure fails on ARM64
On FreeBSD, if you are trying to build a port but it fails at the configure step with a message similar to this:
checking build system type... Invalid configuration `arm64-portbld-freebsd13.2': machine `arm64-portbld' not recognized configure: error: /bin/sh ./build-aux/config.sub arm64-portbld-freebsd13.2 failed
Here’s a quick-fix that might work for you:
export CONFIGURE_TARGET=arm64-unknown-freebsd13.2 make install
This is similar to passing --host arm64-unknown-freebsd13.2 to the configure script instead of trying to guess it.