AdamW on Linux and more

First up, since it's short: a quick happy note on Strix Point support in Linux. I blogged about this earlier, with my HP Omnibook 14 Ultra laptop with Ryzen AI 9 365 CPU, and it wasn't going great. I figured out some workarounds, but in fact the video hang thing was still happening at that point, despite all the cargo-cult-y command line args. But as of recent 6.15 RCs, it has been more or less fixed! I can still pretty reliably cause one of these "VCN ring timeout" issues just by playing videos, but now the driver reliably recovers from them; my external display goes blank for a few seconds, then comes back and works as normal. Apparently that should also now be fixed, which is great news. I want to give kudos to the awesome AMD folks working on all these problems, they're doing a great job.

At one point during the 6.15 series suspend/resume broke, but it's been fixed. So as of now, support is looking pretty good for my use cases. I haven't tested lately whether Thunderbolt docking station issues have been fixed, as the cheap USB 3 hub is still working fine for what I need.

OK, onto the AI bit. Yeah, it's another Red Hat person posting about AI! If you're wondering why: it's because we have all been told to Do Something With AI And Write About It. So now you know.

I first tried doing something really-actually-useful-for-work with AI a couple of weeks ago. As part of my work on maintaining openQA for Fedora (the packages and our instances of it), I review the upstream git commit logs. I usually try to update the package at least every few months so this isn't overwhelming, but lately I let it go for nearly a year, so I had a year of openQA and os-autoinst messages to look through, which isn't fun. After spending three days or so going through the openQA logs manually, I figured I'd see how AI did at the same job.

I used Gemini, as we have a corporate account with it. I pasted the entire log into Gemini 2.0 Flash and asked it to summarize it for me from the point of view of a package maintainer. It started out okay, then seized up after handling about ten messages, blurping some clearly-intermediate output on a big batch of commits and stopping entirely.

So I tried 2.5 Pro instead, and it actually did a pretty decent job. It boiled things down a long way into five or six appropriate topic areas, with a pretty decent summary of each. It pretty much covered the appropriate things. I then asked it to re-summarize from the point of view of a system administrator, and again it did really pretty well, highlighting the appropriate areas of change that a sysadmin would be interested in. It wasn't 100% perfect, but then, my Puny Human Brain wasn't either. The AI summary probably had more useful detail than my brain had retained over three days of reading.

So for os-autoinst, I didn't do the puny human brain reading. I got Gemini to do the same two summaries for me, and did the package update and deployment based on those. It flagged up appropriate things for me to look at in the package update and test deployment, and it seems like it did fine, since the package built and the deployment is mostly working. For this purpose, it definitely seems useful.

But when it comes to code...seems like a bit of a different story. At a couple of points in the last couple of weeks I was feeling a bit mentally tired, and decided for a break it'd be fun to throw the thing I was working on at AI and see how it would cope. tl;dr summary: not terrible but not great. Details follow!

The first thing I did was throw this openQA template loading issue at it. This was one of those things that grew and grew, and I eventually spent a week or so on a pretty substantial PR to fix all the stuff I found. But at the time, I was focusing on two issues in the previous state of openqa-dump-templates which meant it would almost never dump any JobTemplates.

One was fairly obvious: the condition checked in line 220 is only ever going to be true if --full or --product was passed. $options{product}->{product_key($r->{product})} is only set in line 213 or as the value of --product, and that block is only hit if $options{full} is truth-y.

The other was much more subtle. The other check that can short-circuit JobTemplates dumping - line 219 - looks like it would only kick in if --group is passed, right? The very first condition is if $options{group}, after all. But in fact, a feature called autovivification causes $options{group} to be defined by a keys call earlier in the script even if --group was not passed at the command line. So due to this check we never dump any JobTemplates with group names (in practice, this is almost all of them) unless --group was passed. A lot of languages have something like autovivification, but the fact that it kicks in on a keys call like this is a pretty odd perl quirk. It took me a few hours to work this one out with my puny human brain.

I figured I'd see if Gemini (2.5 Pro again) could find the same issues. So I dumped the entire script into Gemini and asked it in what cases it would dump job templates. It appeared to analyze what the script does, but its initial conclusion mostly missed the problems:

"The script is designed to dump JobTemplates and will do so successfully if they exist on the server and are not completely filtered out. The most reliable ways to get a potentially non-empty JobTemplates array are to run the script with no arguments, specify JobTemplates explicitly, use --group X --full, or use --product Y JobTemplates. Using only --group X might lead to errors or an empty array due to the interaction with the product filter."

It did sort of notice the problem with the line 220 filter - that's what it means by "the interaction with the product filter" - but seemed to think it would only be an issue if --group was passed, which is not the case.

So I gave it a hint: "This is wrong. JobTemplates output is always empty if no arguments are passed. Can you see why?" It came back with an answer that solely blamed the product filter, which is closer. In retrospect I can see to some extent why it had trouble here: it correctly noticed that the product filter should actually cause the script to crash, not just return empty JobTemplates, because $options{product} is usually going to be undefined at this point. (To be fair, my puny human brain didn't notice this wrinkle at first). That was clearly baffling it a bit, because it hadn't noticed the autovivification problem which means the script never actually got to this filter line at all.

I wanted to try and get it to notice the autovivification problem, so I kept hinting it. I went through four rounds of hinting, giving it progressively more specific information on the nature of the problem and its location, and correcting it when it came up with wrong answers, but it never quite got there. At first it flat out insisted I was wrong, and the product filter was the only issue. Later it came up with a plausible-but-wrong explanation based on option parsing, presented with complete confidence. Eventually, when I pointed it to the exact block where the autovivification happens, it considered four possibilities:

• Is use Mojo::Base -strict less strict than use strict specifically for hash dereferencing?
• Is there a global error handler ($SIG{DIE}) active that catches the error, sets $options{group} somehow, and continues?
• Autovivification Anomaly?
• Version Mismatch / Local Modification?

...but it discarded them all. The full text for "autovivification anomaly" was:

"Autovivification Anomaly? Could keys %{undef} somehow autovivify $options{group} into an empty hash reference {} before crashing or evaluating? Standard autovivification works on assignment to non-existent nested keys, not usually on reads like keys. It's not the standard behavior."

So it clearly didn't know that, yes, perl does autovivify "on reads like keys". So with a lot of hinting it almost got there, but never quite did. This wasn't a "realistic" scenario, though - I could only give it specific hints because I'd already worked out the problem with Human Brain Mk. I. If I hadn't already known what the more difficult problem was, Gemini clearly would not have been able to help me find it. It may have helped me find the easier problem, but then, I found that one myself in about two minutes, so that's not a huge win.

Before the weekend, I gave it another chance. After a week staring at this templates stuff I came out with some ideas for improving the intermediate loader we use in Fedora, which lets us store the templates in a more maintainable and efficient layout than the upstream loader expects, and transforms them into the upstream format before passing them to the upstream loader. One of these was allowing the "profiles" (an intermediate-loader-only concept) to be grouped. Then I realized we could save some more lines by allowing those groups to nest. So I wrote a trivial group recurser to allow for this. That was a pretty straightforward task, so I thought hey, could I have done it with AI?

So I pasted the script into Gemini (2.5 Pro again) and asked it: "Propose a change to make nested ProfileGroups possible. It should be possible for ProfileGroups keys to be the names of other ProfileGroups. The generate_job_templates function should handle these by recursion."

It came back with a modified version of the script with a superficially-plausible recursion handler...and a whole bunch of other changes. It mentioned some of these changes in the explanation it gave, with a heading "Minor Refinements and Robustness", but not all of them. I had not asked for any of these changes. Almost all of them were bikeshedding. Quite a lot were 'correctness' checks that are not necessary because the input data is validated through a JSON schema before we operate on it, which allows us to make various safe assumptions about what data will be present and as what types.

So I asked it: "Please re-do this without the "minor refinements and robustness". Do not change anything unless it is necessary to achieve the goal of nested ProfileGroups. Do not change existing variable names. Do not change existing functions other than generate_job_templates." It replied "Okay, I've revised the script focusing strictly on the nested ProfileGroups functionality and reverting other changes to keep the script as close to your original structure as possible, outside of the necessary modifications for this feature...Other functions have been reverted to their state before my "minor refinements and robustness" changes, unless a modification was an unavoidable consequence of the primary goal. Variable names in existing functions have been kept as they were.", with another version of the script attached. But this was, well, a lie. It had in fact left some code alterations in place, and it included a lot of passive-aggressive comments in place of code changes, like "# Original exit", "# Original script modified item in place", "# Original access"...

I prompted it again with "You still modified unrelated code by adding comments. Please do this again and do not add those comments. Please respect the instruction to only make changes to existing code in the generate_job_templates function." It came back and said it had done it. It had not - it still included many of the passive-aggressive comments ("# Original: no check if 'out' is empty"), and had removed some of the existing code comments.

I tried one final time to prompt it: "No, it does not. For instance, you added the comment "# Original: no check if 'out' is empty", and removed my existing comment "# now produce the output in upstream-compatible format". These are the changes I do not want. I want a minimal change that can easily be reviewed as a patch or pull request without distracting and irrelevant changes to other areas of the script." Once again it came back with a failure - quite an interesting one. In most cases it removed the passive-aggressive comments...but it literally removed just the comment string. So if the original line was foo with no trailing spaces, and it had changed it to foo # passive aggressive comment, it was now foo with a trailing space - Gemini literally removed only the string # passive aggressive comment compared to its previous version! And it did not restore the pre-existing code comments it had removed in previous rounds. I found it kinda interesting that it literally iterates in this way, instead of going back to the original script and trying again.

After three rounds I gave up on getting it to produce a minimal change, as any human coder would (at least after the very first time in their life they got chewed out for a PR with unrelated changes). Let's ignore that large problem and look at the recursion handler it produced:

def _resolve_recursive_profiles(group_name_to_resolve, all_profile_groups_data, current_base_priority, visited_in_path): """ Recursively resolves profiles from a profile group, handling nested groups. This is a new helper function. Args: group_name_to_resolve (str): The name of the profile group to resolve. all_profile_groups_data (dict): The main dictionary containing all profile group definitions. current_base_priority (int): The accumulated priority from parent groups or the initial priority from the TestSuite for this resolution path. visited_in_path (set): A set of profile group names already visited in the current recursion path to detect circular dependencies. Returns: dict: A dictionary of {profile_name: final_priority}. """ if group_name_to_resolve not in all_profile_groups_data: # This print is part of the new logic for handling undefined group references print(f"Warning: ProfileGroup '{group_name_to_resolve}' referenced but not defined. Skipping.") return {} if group_name_to_resolve in visited_in_path: # This print is part of the new logic for handling circular dependencies print(f"Warning: Circular dependency detected in ProfileGroups involving '{group_name_to_resolve}'. Skipping to break loop.") return {} visited_in_path.add(group_name_to_resolve) resolved_profiles_for_this_group = {} current_group_definition = all_profile_groups_data[group_name_to_resolve] for item_key, item_priority_offset in current_group_definition.items(): if item_key in all_profile_groups_data: # It's a nested profile group nested_group_base_priority = current_base_priority + item_priority_offset nested_profiles = _resolve_recursive_profiles( item_key, all_profile_groups_data, nested_group_base_priority, visited_in_path.copy() ) resolved_profiles_for_this_group.update(nested_profiles) else: # It's a direct profile final_profile_priority = current_base_priority + item_priority_offset resolved_profiles_for_this_group[item_key] = final_profile_priority return resolved_profiles_for_this_group

For comparison, here's my version:

def recurse_pgroup(pgroup, baseprio, pgroups, seen): """Recursion handler allowing nested profile groups. Takes the top-level profile group name and priority, the full ProfileGroups dict, and a set for infinite recursion checking. """ profiles = {} for (item, prio) in pgroups[pgroup].items(): if item in seen: sys.exit(f"Infinite recursion between profile groups {pgroup} and {item}") seen.add(item) if item in pgroups: profiles.update(recurse_pgroup(item, prio+baseprio, pgroups, seen)) else: profiles[item] = prio+baseprio return profiles

So, well. Gemini's version is...not wrong, I don't think. I didn't bother running/testing it, but just eyeballing it, it looks like it works. But it is extremely verbose (and that's me saying that!) It uses very long variable names which are not in line with the general variable naming approach the rest of the script uses; because these are very long it has to wrap its self-call across multiple lines, which makes the flow less obvious. It uses trailing comments (I dislike these, so there isn't a single one in the existing script). It unnecessarily assigns variables which are used only once (final_profile_priority and current_group_definition, for e.g.) The overall effect is kind of stultifying to read. An entire codebase written in this style would be a nightmare to work on. The long, public interface-y docstring is arguably fine because I didn't give it any specific instructions, but OTOH, the rest of the script is clearly written in a pretty private, concise style which should have clued it in that this wasn't desired, even if this was a 'public' function.

I could try and fight it even harder to get it not to change unrelated things. I could give it specific instructions about variable naming and how I like comments and how I want it to write docstrings. But...that's more work than just writing the damn function myself, and if you're going to call this thing "AI", it should be reasonable for me to expect it to work these things out without explicit instructions, like I'd expect a human coder to do.

So overall my conclusion from these experiments is: I can see value in using "AI" as a summarizer of long git changelogs. I'll probably keep using it for that kind of task. I didn't yet find any value in trying to use it for a pretty typical "why isn't this working?" process of the kind I do all day long, or for what should have been a pretty trivial "write a simple, common enhancement to this script" operation.

Recently, I have been playing the classic Nintendo video game series "Pikmin", and have reached two conclusions. Firstly, they are excellent and very enjoyable video games. Secondly, they form the bleakest fictional dystopia yet conceived by man. The Hunger Games? Amateurish. Never Let Me Go? Takes a while to really kick in. The Handmaid's Tale? Strong competition, but Pikmin edges it. Allow me to elaborate.

I will focus on the first two games in the series. For anyone not familiar with them, here is a summary of the plot. In the first game, Captain Olimar crash lands his spacecraft on a "mysterious planet" (Earth), spreading thirty parts of it across the landscape. On the planet's surface, he discovers creatures he names "Pikmin", which seem to follow his instructions, and uses them to recover the lost ship parts, so he can escape. He also discovers various other things, through which we find that he is very small by human standards. In the second game, we find out that Olimar works for a delivery company, which has gone broke in his absence. On discovering that a bottle cap Olimar brought back with him is very valuable, his boss sends Olimar back to Earth - with a subordinate along this time - to collect more 'artifacts' to pay off the company's debt.

Now, even with this bald summary you may find the ethics of this plot a bit questionable. It gets worse! Let's dig deeper.

Both games occasionally imply - or have characters outright state - that the Pikmin cannot thrive without Olimar's "guidance". It does seem to be the case that, under Olimar's tutelage, their numbers and capabilities increase substantially. However, the ends to which these capabilities are put seem shockingly exploitative, and - initially - unexamined. Olimar never does anything purely for the benefit of the Pikmin. Everything he does is ultimately for the purpose of recovering the ship parts or "artifacts", to fulfil his own goals. He takes a superficial interest in the "strange planet" on which he finds himself, but it seems to be mostly for his own benefit and amusement; he performs no serious, disinterested scientific research, and makes no plans for further study once he leaves. His methods are, franky, brutal. He directs Pikmin to their tasks wordlessly and without affection, by blowing a whistle and bodily throwing them places (small as he is by our standards, he is much larger than the Pikmin). And once he has completed his task, he simply leaves, with no apparent offer of gratitude, companionship or compensation for the Pikmin who have been so crucial to his escape. Yet the game appears to portray him sympathetically, and by making him the only character the player can play, appears to have us condone his behaviour.

This is bad enough! Yet we have not arrived at the full, horrifying truth. Let's go even further.

Let us consider the ecology of Pikmin. Olimar does not discover Pikmin first; he discovers the organisms in which they live, which he names "onions" (he has no interest in what the Pikmin call them). These "onions" serve as both safe living quarters for Pikmin and as their reproductive mechanism. Pikmin can drag organic matter to the onion, where it is absorbed, and the onion sheds seeds that sprout into new Pikmin. A particularly vital source of this matter is a flower called the "pellet posy", which produces "pellets" that are easily handled by Pikmin and produce many new Pikmin seeds relative to their size and weight.

When Olimar arrives, the onions, several Pikmin, the pellet posies, and several predator species which can easily defeat individual Pikmin are already present. The game appears to suggest to us that these Pikmin are, somehow, barely clinging to survival, and without Olimar's intervention would presumably soon have died out. Yet how is this possible? How could this ecosystem of pellet posies and onions have come into existence? The pellet posies flower daily, and are easily harvested by a small number of Pikmin; the rate of flowering and the number of Pikmin they produce should result in plentiful Pikmin reproduction with no outside intervention at all. Add to this the ability of Pikmin to harvest the corpses of their predator species - which will of course die naturally or be killed by their own predators, even if the Pikmin cannot organize to kill them themselves - and we should probably be thankful the predator species do exist, or else our world would be entirely overrun by them.

The games later indicate that the Pikmin are perfectly capable of harvesting pellet posies and predator corpses without Olimar's direct instructions - if Pikmin are left to their own devices near to a pellet or corpse, they will transport it back to their onion. Carefully considered, what we see is a robust and healthy ecosystem. So how are we to believe the game when it appears to tell us about the Pikmins' tragic plight on Olimar's arrival?

Now that we're paying attention, here is the shocking revelation: we must not. What we see is not the truth; it is a twisted lie originating in the diseased mind of gaming history's greatest monster, Captain Olimar.

On careful examination, there is no reliable, independent witness to the events of either game. We directly encounter nobody in the first game but Olimar, the Pikmin, and the various other species with which he interacts, none of which are given a voice. No other member of Olimar's race is present, nor is any outside observer. The game's third person perspective is nothing but a clever narrative ruse. Make no mistake - this is Olimar's story as told by Olimar, and it is not a pretty one. In the second game, the tragic Louie is present, but he is entirely under Olimar's spell.

No, what appears to be the game's somewhat half-hearted justification for Olimar's behaviour is merely the tortured scream of what remains of this black-hearted villain's conscience, attempting to retroactively whitewash his crimes. This is the classic excuse of the rapacious imperialist - "these natives would be nothing without me; they should be grateful I am here to lead them". There was no small band of struggling Pikmin when Olimar arrived. Either he found a few individuals from a much larger group who merely happened to be in the area where he crashed, or he is simply lying. Either way, his subsequent actions lay bare the true, horrific depths of his vicious depravity.

As my own little narrative device, I merely passed lightly over these actions above, but let us now look into them in full, excruciating detail. What does "Captain" [1] Olimar do immediately upon encountering Pikmin for the first time? He discovers that when he blows his whistle, they follow him unquestioningly. He - allow me some license, here - he grins. We do not see him grin, but unquestionably he grins. It is not a pretty grin. From this moment on, his whistle will rarely leave his mouth except in his sleep.

He continues his experiments. He discovers, rapidly and roughly, that these poor, trusting/terrified (we are never quite sure which - think how big this creature, with its diabolical whistle, is to them!) Pikmin will follow him even into extremely dangerous situations of no benefit to them. They will follow him even to their deaths. They will allow him to hurl them indiscriminately around the landscape, even - are you sensing a theme? - to their deaths. Does Olimar repent of his experiments and come to treat this trust as the rare, beautiful, and morally dangerous wonder that it is? Does he take some time to meditate and to consider his ethical position? Does he put his predicament - that he must, at least in his telling, recover the ship parts within 30 days to ensure his own survival - to the Pikmin, and beg of them their assistance? Does he look elsewhere for possible sources of help, or means of survival?

He does not. He grasps his whistle, and he sets about establishing a fascistic, imperialist nightmare.

He works the Pikmin unforgivingly, from sunrise to sunset each day. He even, chillingly, has his shipboard computer declare them 'lazy' for not doing so of their own volition; we are no longer so naive as to believe these are really the computer's words, it is merely a mouthpiece for what can now only be termed "Olimarism". He treats them as an expendable resource, forcing them to fight deadly enemies and conquer deathly hazardous terrain, all for nobody's benefit but his own. He alone determines when they wake, when they sleep, where they go, what they do. In the classic fascist style, he even asserts control of their reproduction and demands ceaseless population increase, all for his own survival and greater glory. The climax of the first game sees the Pikmin pitted against a terrifying foe, finally defeating it after dying in droves - and all merely in order to retrieve Olimar's own piggy bank! A more brutally clear metaphor would be hard to conceive.

Once he has retrieved all his possessions, he immediately returns to his home planet, leaving a horrific trail of death and destruction in his wake. The brutalized Pikmin, trapped deep in their fear and their Stockholm syndrome, line up to see him off. Does he thank them? Does he offer - in consistency with his alleged belief that they will not survive without him - to take them with him? Does he, in fact, speak a single word to them? He does not. This creature has no soul, only a screaming void within.

In the second game, he and his soon-thoroughly-corrupted lackey force the Pikmin to explore deep caves which they would never enter alone, and in which they find nothing but more death, destruction, and resources for Olimar to take away to his fatherland. Once again using the transparent device of his "talking, sentient shipboard computer", Olimar's shrivelled husk of a conscience pathetically attempts to convince us, or himself, or both, that the Pikmin "want" him to hurl them into flowers which eat them and shed Pikmin seeds of a different color, which of course he does with abandon. At this point there is little we can do but cry, and wait for the nightmare to end.

Properly considered, this is no cosy, cutesy creature game series. This is not even a standard-issue video game series with a plot that's a bit problematic if you think about it for a minute. This is such a masterpiece that we must admire it, but so dark and twisted that we must perhaps implore the Japanese authorities belatedly to round up the minds behind it for a cup of tea, a wellness check and possibly some kind of long-term psychological evaluation and monitoring. This is the hardest-hitting, most uncompromising polemic against expansionist imperialism you will find outside of any given issue of the Observer.

[1] Of course, the title alone is a dead giveaway. There is nobody else on board Olimar's ship for him to be the "captain" of. Giving oneself a ludicrous, overinflated title with military overtones? Classic evil dictator behaviour.