openQA bites

Due to recent changes in the worker configuration of the SUSE internal openQA instance, we needed to reconfigure some of the PowerPC jobs in openQA. This triggered a couple of questions regarding the availability of openQA worker, worker backends, their differences and their caveats. This blog post should act as a quickstart/overview guide for people getting into OpenQA testing on the PowerPC architecture.

openqa-mq is a small CLI tool that receives openQA related events from RabbitMQ. It is part of the openqa-mon packages and will work for OSD and for OOO.

A coworker recently faced the problem to copy&pasting a large amount of text into a VNC terminal for openQA. VNC doesn’t always allow copy&paste and when you have to manually type a longer string this is prone to typos and human error.

PostgreSQL is a capable and mature database, which comes in a major or minor version number (e.g. 16.0). Minor releases never change the internal storage, so the database always remains compatible with earlier and later minor releases. However major version releases do not have such a guarentee. We are running a single PostgreSQL database as a podman container and I recently (today) had the glorious task of migrating this database to the next major version. In this blog post I describe how we did this migration.

I recently automated the installation of updates on my openQA development instance. The goal was to make the instance updates itself over night, but only if it is idle, i.e. there are no running jobs. Sometimes when I’m busy, the instance needs to work overnight and despite openQA being able to restart cancelled jobs from a reboot, I prefer to avoid situations where this might result in problems in times, where I really can’t have that.

I recently encountered a new interesting openQA issue:

[2023-03-13T14:18:22.651705+01:00] [warn] [pid:18929] !!! : qemu-system-x86_64: -netdev tap,id=qanet0,ifname=tap3,script=no,downscript=no: could not configure /dev/net/tun (tap3): Operation not permitted

This is an error that you likely are encountering on a older openQA instance, after you setup multimachine jobs but haven’t used them in a while. For me the solution was to grant the CAP_NET_ADMIN capabilities to the qemu binary (again):

JeOS (Just enough OS) or MinimalVM images are minimal VM images (duh!) that can be used to quickly deploy VMs. Instead of a installation you only need to go through a first boot setup. This makes those images very handy if you need to spin up a bunch of test VMs as for instance if you need a custom cluster.

When developing new openQA tests you will have to run a lot of verification and debug test runs. This is why I typically encourage people to do all openQA testing on their own instances, to prevent spamming of the production instances.

It’s Hackweek and I’m back at working on the GeekOops project. One of the more annoying tasks that I have been postponing already since some time is to adjust the molecule workflow to work with cgroups 2.

The openqa-revtui tool is a neat CLI utility for helping you review openQA jobs and job groups. It is part of the openqa-mon project, which has grown in the last years and now consists on more than the job monitoring tool itself.