Sam Thursfield<p><strong>Status update, 13/12/24</strong></p><p>Its been an interesting and cold month so far. I made a successful trip to the UK, one of the first times I’ve been back in winter and avoided being exposed to COVID19 since the pandemic, so that’s a step forwards. </p><p>I’ve been thinking a lot about documentation recently in a few different places where I work or contribute as a volunteer. One such place is within openQA and the GNOME QA initiative, so here’s what’s been happening there recently.</p><p>The <a href="https://gitlab.gnome.org/GNOME/openqa-tests/-/wikis/QA-testing-monthly-call" rel="nofollow noopener" target="_blank">monthly Linux QA call</a> is one of my 2024 success stories. The goal of the call is to foster collaboration between distros and upstreams, so that we share testing effort rather than duplicating it, and we get issue reports upstream as soon as things break. Through this call I’ve met many of the key people who are do automated testing of GNOME downstream, and we are starting to share ideas for the future.</p><p>What I want for GNOME is to be able to run QA tests for any open merge request, so we can spot regressions before they even land. As part of the STF+GNOME+Codethink collaboration we got a <a href="https://gitlab.gnome.org/GNOME/gnome-shell/-/merge_requests/3419" rel="nofollow noopener" target="_blank">working prototype</a> of upstream QA for GNOME Shell, but to move beyond a prototype, we need to build a more solid foundation. The current GNOME Shell prototype has about 100 lines of copy-pasted openQA code to set up the VM, and this would need to be copied into every other GNOME module where we might run QA tests. I very much do not want so many copies of one piece of code.</p><a href="https://samthursfield.wordpress.com/wp-content/uploads/2024/12/image-1.png" rel="nofollow noopener" target="_blank"></a><p>I mentioned this in the QA call and Oli Kurz, who is the openQA product owner at openSUSE, proposed that we put the setup logic directly into os-autoinst, which is openQA’s test runner. The os-autoinst code has a bare ‘<a href="https://github.com/os-autoinst/os-autoinst/blob/master/basetest.pm" rel="nofollow noopener" target="_blank">basetest</a>‘ module which must be customed for the OS under test. Each distro maintains their own infrastructure on top of that to wait for the desktop to start, log in as a user, and so on.</p><p>Since most of us test Linux, we can reasonably add a <a href="https://github.com/os-autoinst/os-autoinst/issues/2583" rel="nofollow noopener" target="_blank">more specific base class specific to Linux</a>, and some further helpers for systemd-based OSes. I love this idea as we could now share improvements between all the different QA teams.</p><p>So the base test class can be extended, but how do we document its capabilities? I find openQA’s existing documentation pretty overwhelming as a <a href="http://open.qa/docs/" rel="nofollow noopener" target="_blank">single 50,000 word document</a>. It’s not feasible for me to totally rework the documentation, but if we’re going to collaborate upstream then we need to have some way to document the new base classes. </p><p>Of course I also wrote some <a href="https://gitlab.gnome.org/GNOME/gnome-build-meta/-/wikis/openqa/OpenQA-for-GNOME-developers" rel="nofollow noopener" target="_blank">GNOME specific documentation</a> for QA; but hidden docs like this are doomed to become obsolete. I <a href="https://gitlab.gnome.org/Teams/Websites/developer.gnome.org/-/merge_requests/143" rel="nofollow noopener" target="_blank">began adding a section on testing</a> to the GNOME developer guide, but I’ve had no feedback at all on the merge request, so this effort seems like a dead end.</p><p>So what should we do to make the QA infrastructure easier to understand? Let me know your ideas below.</p><a href="https://samthursfield.wordpress.com/wp-content/uploads/2024/12/img_20241203_155310.jpg" rel="nofollow noopener" target="_blank"></a><p>Looking at the problem from another angle, we still lack a collective understanding of what what openQA is and why you might use it. As a small step towards making this clearer, I wrote a comparison of four testing tools <a href="https://pad.gnome.org/tVzBIusTT2e76tpzP7IbKQ#" rel="nofollow noopener" target="_blank">which you can read here</a>. And at Oli’s suggestion I proposed a new <a href="https://en.wikipedia.org/wiki/Draft:OpenQA" rel="nofollow noopener" target="_blank">Wikipedia page for openQA</a>.</p><a href="https://en.wikipedia.org/wiki/Draft:OpenQA" rel="nofollow noopener" target="_blank"></a><p>Please suggest changes here or in the <a href="https://matrix.to/#/#openqa:opensuse.org" rel="nofollow noopener" target="_blank">openQA matrix channel</a>. If you’re reading this and are a Wikipedia reviewer, then <strong>I would greatly appreciate a review </strong>so we can publish the new page. We could then also add openQA to the Wikipedia <a href="https://en.wikipedia.org/wiki/Comparison_of_GUI_testing_tools" rel="nofollow noopener" target="_blank">“Comparison of GUI testing tools”</a>. Through small efforts like this we can hopefully reduce how much documentation is needed on the GNOME side, as we won’t need to start at “what even is openQA”.</p><p>I have a lot more to say about documentation but that will have to wait for next month. Enjoy the festive season and I hope your 2025 gets off to a good start!</p><p></p><p><a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://samthursfield.wordpress.com/tag/gnome/" target="_blank">#gnome</a> <a rel="nofollow noopener" class="hashtag u-tag u-category" href="https://samthursfield.wordpress.com/tag/openqa/" target="_blank">#openqa</a></p>