Otros Blogs

Recently, I have been playing the classic Nintendo video game series "Pikmin", and have reached two conclusions. Firstly, they are excellent and very enjoyable video games. Secondly, they form the bleakest fictional dystopia yet conceived by man. The Hunger Games? Amateurish. Never Let Me Go? Takes a while to really kick in. The Handmaid's Tale? Strong competition, but Pikmin edges it. Allow me to elaborate.

I will focus on the first two games in the series. For anyone not familiar with them, here is a summary of the plot. In the first game, Captain Olimar crash lands his spacecraft on a "mysterious planet" (Earth), spreading thirty parts of it across the landscape. On the planet's surface, he discovers creatures he names "Pikmin", which seem to follow his instructions, and uses them to recover the lost ship parts, so he can escape. He also discovers various other things, through which we find that he is very small by human standards. In the second game, we find out that Olimar works for a delivery company, which has gone broke in his absence. On discovering that a bottle cap Olimar brought back with him is very valuable, his boss sends Olimar back to Earth - with a subordinate along this time - to collect more 'artifacts' to pay off the company's debt.

Now, even with this bald summary you may find the ethics of this plot a bit questionable. It gets worse! Let's dig deeper.

Both games occasionally imply - or have characters outright state - that the Pikmin cannot thrive without Olimar's "guidance". It does seem to be the case that, under Olimar's tutelage, their numbers and capabilities increase substantially. However, the ends to which these capabilities are put seem shockingly exploitative, and - initially - unexamined. Olimar never does anything purely for the benefit of the Pikmin. Everything he does is ultimately for the purpose of recovering the ship parts or "artifacts", to fulfil his own goals. He takes a superficial interest in the "strange planet" on which he finds himself, but it seems to be mostly for his own benefit and amusement; he performs no serious, disinterested scientific research, and makes no plans for further study once he leaves. His methods are, franky, brutal. He directs Pikmin to their tasks wordlessly and without affection, by blowing a whistle and bodily throwing them places (small as he is by our standards, he is much larger than the Pikmin). And once he has completed his task, he simply leaves, with no apparent offer of gratitude, companionship or compensation for the Pikmin who have been so crucial to his escape. Yet the game appears to portray him sympathetically, and by making him the only character the player can play, appears to have us condone his behaviour.

This is bad enough! Yet we have not arrived at the full, horrifying truth. Let's go even further.

Let us consider the ecology of Pikmin. Olimar does not discover Pikmin first; he discovers the organisms in which they live, which he names "onions" (he has no interest in what the Pikmin call them). These "onions" serve as both safe living quarters for Pikmin and as their reproductive mechanism. Pikmin can drag organic matter to the onion, where it is absorbed, and the onion sheds seeds that sprout into new Pikmin. A particularly vital source of this matter is a flower called the "pellet posy", which produces "pellets" that are easily handled by Pikmin and produce many new Pikmin seeds relative to their size and weight.

When Olimar arrives, the onions, several Pikmin, the pellet posies, and several predator species which can easily defeat individual Pikmin are already present. The game appears to suggest to us that these Pikmin are, somehow, barely clinging to survival, and without Olimar's intervention would presumably soon have died out. Yet how is this possible? How could this ecosystem of pellet posies and onions have come into existence? The pellet posies flower daily, and are easily harvested by a small number of Pikmin; the rate of flowering and the number of Pikmin they produce should result in plentiful Pikmin reproduction with no outside intervention at all. Add to this the ability of Pikmin to harvest the corpses of their predator species - which will of course die naturally or be killed by their own predators, even if the Pikmin cannot organize to kill them themselves - and we should probably be thankful the predator species do exist, or else our world would be entirely overrun by them.

The games later indicate that the Pikmin are perfectly capable of harvesting pellet posies and predator corpses without Olimar's direct instructions - if Pikmin are left to their own devices near to a pellet or corpse, they will transport it back to their onion. Carefully considered, what we see is a robust and healthy ecosystem. So how are we to believe the game when it appears to tell us about the Pikmins' tragic plight on Olimar's arrival?

Now that we're paying attention, here is the shocking revelation: we must not. What we see is not the truth; it is a twisted lie originating in the diseased mind of gaming history's greatest monster, Captain Olimar.

On careful examination, there is no reliable, independent witness to the events of either game. We directly encounter nobody in the first game but Olimar, the Pikmin, and the various other species with which he interacts, none of which are given a voice. No other member of Olimar's race is present, nor is any outside observer. The game's third person perspective is nothing but a clever narrative ruse. Make no mistake - this is Olimar's story as told by Olimar, and it is not a pretty one. In the second game, the tragic Louie is present, but he is entirely under Olimar's spell.

No, what appears to be the game's somewhat half-hearted justification for Olimar's behaviour is merely the tortured scream of what remains of this black-hearted villain's conscience, attempting to retroactively whitewash his crimes. This is the classic excuse of the rapacious imperialist - "these natives would be nothing without me; they should be grateful I am here to lead them". There was no small band of struggling Pikmin when Olimar arrived. Either he found a few individuals from a much larger group who merely happened to be in the area where he crashed, or he is simply lying. Either way, his subsequent actions lay bare the true, horrific depths of his vicious depravity.

As my own little narrative device, I merely passed lightly over these details above, but let us now look into them in full, excruciating detail. What does "Captain" [1] Olimar do immediately upon encountering Pikmin for the first time? He discovers that when he blows his whistle, they follow him unquestioningly. He - allow me some license, here - he grins. We do not see him grin, but unquestionably he grins. It is not a pretty grin. From this moment on, his whistle will rarely leave his mouth except in his sleep.

He continues his experiments. He discovers, rapidly and roughly, that these poor, trusting/terrified (we are never quite sure which - think how big this creature, with its diabolical whistle, is to them!) Pikmin will follow him even into extremely dangerous situations of no benefit to them. They will follow him even to their deaths. They will allow him to hurl them indiscriminately around the landscape, even - are you sensing a theme? - to their deaths. Does Olimar repent of his experiments and come to treat this trust as the rare, beautiful, and morally dangerous wonder that it is? Does he take some time to meditate and to consider his ethical position? Does he put his predicament - that he must, at least in his telling, recover the ship parts within 30 days to ensure his own survival - to the Pikmin, and beg of them their assistance? Does he look elsewhere for possible sources of help, or means of survival?

He does not. He grasps his whistle, and he sets about establishing a fascistic, imperialist nightmare.

He works the Pikmin unforgivingly, from sunrise to sunset each day. He even, chillingly, has his shipboard computer declare them 'lazy' for not doing so of their own volition; we are no longer so naive as to believe these are really the computer's words, it is merely a mouthpiece for what can now only be termed "Olimarism". He treats them as an expendable resource, forcing them to fight deadly enemies and conquer deathly hazardous terrain, all for nobody's benefit but his own. He alone determines when they wake, when they sleep, where they go, what they do. In the classic fascist style, he even asserts control of their reproduction and demands ceaseless population increase, all for his own survival and greater glory. The climax of the first game sees the Pikmin pitted against a terrifying foe, finally defeating it after dying in droves - and all merely in order to retrieve Olimar's own piggy bank! A more brutally clear metaphor would be hard to conceive.

Once he has retrieved all his possessions, he immediately returns to his home planet, leaving a horrific trail of death and destruction in his wake. The brutalized Pikmin, trapped deep in their fear and their Stockholm syndrome, line up to see him off. Does he thank them? Does he offer - in consistency with his alleged belief that they will not survive without him - to take them with him? Does he, in fact, speak a single word to them? He does not. This creature has no soul, only a screaming void within.

In the second game, he and his soon-thoroughly-corrupted lackey force the Pikmin to explore deep caves which they would never enter alone, and in which they find nothing but more death, destruction, and resources for Olimar to take away to his fatherland. Once again using the transparent device of his "talking, sentient shipboard computer", Olimar's shrivelled husk of a conscience pathetically attempts to convince us, or himself, or both, that the Pikmin "want" him to hurl them into flowers which eat them and shed Pikmin seeds of a different color, which of course he does with abandon. At this point there is little we can do but cry, and wait for the nightmare to end.

Properly considered, this is no cosy, cutesy creature game series. This is not even a standard-issue video game series with a plot that's a bit problematic if you think about it for a minute. This is such a masterpiece that we must admire it, but so dark and twisted that we must perhaps implore the Japanese authorities belatedly to round up the minds behind it for a cup of tea, a wellness check and possibly some kind of long-term psychological evaluation and monitoring. This is the hardest-hitting, most uncompromising polemic against expansionist imperialism you will find outside of any given issue of the Observer.

[1] Of course, the title alone is a dead giveaway. There is nobody else on board Olimar's ship for him to be the "captain" of. Giving oneself a ludicrous, overinflated title with military overtones? Classic evil dictator behaviour.

Figured I'd post an update on how things are going with the new laptop (HP Omnibook Ultra 14, AMD Ryzen AI 9 365 "Strix Point", for the searchers) and with Silverblue.

I managed to work around the hub issue by swapping out the fancy $300 Thunderbolt hub for a $40 USB-C hub off Amazon. This comes with limitations - you're only going to get a single 4k 60Hz external display, and limited bandwidth for anything else - but it's sufficient for my needs, and makes me regret buying the fancy hub in the first place. It seems to work 100% reliably on startup, reboot and across suspend/resume. There's still clearly something wrong with Thunderbolt handling in the kernel, but it's not my problem any more.

The poor performance of some sites in Firefox turned out to be tied to the hanging problem - I'd disabled graphics acceleration in Firefox, which helped with the hanging, but was causing the appalling performance on Google sites and others. I've now cargo-culted a set of kernel args - amdgpu.dcdebugmask=0x800 amdgpu.lockup_timeout=100000 drm.vblankoffdelay=0 - which seem to be helping; I turned graphics acceleration back on in Firefox and it hasn't started hanging again. At least, I haven't had random hangs for the last few days, and this morning I played a video on youtube and the system has not hung since then. I've no idea how bad they are for battery life, but hey, they seem to be keeping things stable. So, the system is pretty workable at this point. I've been using it full-time, haven't had to go back to the old one.

I'm also feeling better about Silverblue as a main OS this time. A lot of things seem to have got better. The toolbox container experience is pretty smooth now. I managed to get adb working inside a container by putting these udev rules in /etc/udev/rules.d. It seems like I have to kill and re-start the adb server any time the phone disconnects or reboots - usually adb would keep seeing the phone just fine across those events - but it's a minor inconvenience. I had to print something yesterday, was worried for a moment that I'd have to figure out how to get hp-setup to do its thing, but then...Silverblue saw my ancient HP printer on the network, let me print to it, and it worked, all without any manual setup at all. It seems to be working over IPP, but I'm a bit surprised, as the printer is from 2010 or 2011 and I don't think it worked before. But I'm not complaining!

I haven't had any real issues with app availability so far. All the desktop apps I need to use are available as flatpaks, and the toolbox container handles CLI stuff. I'm running Firefox (baked-in version), Evolution, gedit, ptyxis (built-in), liferea, nheko, slack and vesktop (for discord) without any trouble. LibreOffice and GIMP flatpaks also work fine. Everything's really been pretty smooth.

I do have a couple of tweaks in my bashrc (I put them in a file in ~/.bashrc.d, which is a neat invention) that other Atomic users might find useful...

if [ -n "$container" ] then alias gedit="flatpak-spawn --host /var/lib/flatpak/exports/bin/org.gnome.gedit" alias xdg-open=flatpak-xdg-open else alias gedit=/var/lib/flatpak/exports/bin/org.gnome.gedit fi

the gedit aliases let me do gedit somefile either inside or outside a container, and the file just opens in my existing gedit instance. Can't really live without that. You can adapt it for anything that's a flatpak app on the host. The xdg-open alias within containers similar makes xdg-open somefile within the container do the same as it would outside the container.

So it's still early days, but I'm optimistic I'll keep this setup this time. I might try rebasing to the bootc build soon.

New year, new blog post! Fedora's going great...41 came out and seems to be getting good reviews, there's exciting stuff going on with atomic/bootc, we're getting a new forge, it's an exciting time to be alive...

Personally I've spent a large chunk of the last few weeks bashing my head against two awkward bugs - one kernel bug, one systemd bug. It's a bit of a slog, but hey. Also now working up to our next big openQA project, extending test coverage for the new installer.

But also! I bought myself a new laptop. For the last couple of years I've been using a Dell XPS 13 9315, the Alder Lake generation. I've been using various generations of XPS 13 ever since Sony stopped making laptops (previously I used a 2010 Vaio Z) - I always found it to be the best thin-and-light design, and this one was definitely that. But over time it felt really underpowered. Some of this is the fault of modern apps. I have to run a dumb amount of modern chat apps, and while they're much nicer than IRC, they sure use a lot more resources than hexchat. Of course I have a browser with about 50 tabs open at all times, Evolution uses quite a lot of memory for my email setup for some reason, and I have to run VMs quite often for my work obviously. Put all that together, and...I was often running out of RAM despite having 16GB, which is pretty ridiculous. But even aside from that, you could tell the CPU was just struggling with everything. Just being in a video chat was hard work for it (if I switched apps too much while in a meeting, my audio would start chopping up for others on the call). Running more than two VMs tend to hang the system irretrievably. Just normal use often caused the fan to spin up pretty high. And the battery life wasn't great. It got better with kernel updates over time, but still only 3-4 hours probably.

So I figured I'd throw some hardware at the problem. I've been following all the chipset releases over the last couple of years, and decided I wanted to get something with AMD's latest silicon, codenamed "Strix Point", the Ryzen AI 3xx chips. They're not massively higher-performing than the previous gen, but the battery life seems to be improved, and they have somewhat better GPUs. That pretty much brought it down to the Asus Vivobook S 14, HP Omnibook Ultra 14, and Lenovo T14S gen 6 AMD. The Asus is stuck with 24GB of RAM max and I'm not a huge Asus fan in general, and the HP came in like $600 cheaper than the Thinkpad with equivalent specs, and had a 3 year warranty included. So I went with the HP, with 1TB of storage and 32GB of RAM.

I really like the system as a whole. It's heavier than the XPS 13, obviously, the bezels are a little bigger, and the screen is glossier. But the screen is pretty nice, I like the keyboard, and the overall build quality feels pretty solid. The trackpad seems fine.

As for running Fedora (and Linux in general) on it...well, it's almost great. Everything more or less works out of the box, except the fingerprint reader. I don't care about that because I set up the reader on the XPS 13 and kinda hated it; it's nowhere near as nice as a fingerprint reader on a phone. Even if it worked on the HP I'd leave it off. The performance is fantastic (except that Google office sites perform weirdly terribly on Firefox, haven't tried on Chromium yet).

But...after using it for a while, the issues become apparent. The first one I hit is that the system seems to hang pretty reproducibly playing video in browsers. This seems to be affecting pretty much everyone with a Strix Point system, and the only 'fix' is to turn off hardware video acceleration in the browser, which isn't really great (it means playing video will use the CPU, hurting battery life and performance). Then I found even with that workaround applied the system would hang occasionally. Looking at the list of Strix Point issues on the AMD issue tracker, I found a few that recommended kernel parameters to disable various features of the GPU to work around this; I'm running with amdgpu.dcdebugmask=0x800, which disables idle power states for the GPU, which probably hurts battery life pretty bad. Haven't had a hang with that yet, but we'll see. But aside from that, I'm also having issues with docks. I have a Caldigit TS3+, which was probably overpowered for what I really need, but worked great with the XPS 13. I have a keyboard, camera, headset, ethernet and monitor connected to it. With the HP, I find that at encryption passphrase entry during boot (so, in the initramfs) the keyboard works fine, but once I reach the OS proper, only the monitor works. Nothing else attached to the dock works at all. A couple of times, suspending the system and resuming it seemed to make it start working - but then I tried that a couple more times and it didn't work. I have another Caldigit dock in the basement; tried that, same deal. Then I tried my cheap no-name travel hub (which just has power pass-through, an HDMI port, and one USB-A port on it) with a USB-A hub, and...at first it worked fine! But then I suspended and resumed and the camera and headset stopped working. Keyboard still worked. Sigh. I've ordered a mid-range hub with HDMI, ethernet, a card reader and four USB-A ports on it off Amazon, so I won't need the USB-A hub daisychain any more...I'm hoping that'll work well enough. If not, it's a bit awkward.

So, so far it's a bit of a frustrating experience. It could clearly be a fantastic Linux laptop, but it isn't quite one yet. I'd probably recommend holding off for a bit while the upstream devs (hopefully) shake out all the bugs...

It's time for that rarest of events: a blog post! And it's another debugging adventure. Settle in, folks!

Recently I got interested in improving the time it takes to do a full compose of Fedora. This is when we start from just the packages and a few other inputs (basically image recipes and package groups), and produce a set of repositories and boot trees and images and all that stuff. For a long time this took somewhere between 5 and 10 hours. Recently we've managed to get it down to 3-4, then I figured out a change which has got it under 3 hours.

After that I re-analyzed the process and figured out that the biggest remaining point to attack is something called the 'pkgset' phase, which happens near the start of the process, not in parallel with anything else, and takes 35 minutes or so. So I started digging into that to see if it can be improved.

I fairly quickly found that it spends about 20 minutes in one relatively small codepath. It's created one giant package set (this is just a concept in memory at the time, it gets turned into an actual repo later) with every package in the compose in it. During this 20 minutes, it creates subsets of that package set per architecture, with only the packages relevant to that architecture in it (so packages for that arch, plus noarch packages, plus source packages, plus 'compatible' arches, like including i686 for x86_64).

I poked about at that code a bit and decided I could maybe make it a bit more efficient. The current version works by creating each arch subset one at a time by looping over the big global set. Because every arch includes noarch and src packages, it winds up looping over the noarch and src lists once per arch, which seemed inefficient. So I went ahead and rewrote it to create them all at once, to try and reduce the repeated looping.

Today I was testing that out, which unfortunately has to be done more or less 'in production', so if you like you can watch me here, where you'll see composes appearing and disappearing every fifteen minutes or so. At first of course my change didn't work at all because I'd made the usual few dumb mistakes with wrong variable names and stuff. After fixing all that up, I timed it, and it turned out about 7 minutes faster. Not earth shattering, but hey.

So I started checking it was accurate (i.e. created the same package sets as the old code). It turned out it wasn't quite (a subtle bug with noarch package exclusions). While fixing that, I ran across some lines in the code that had bugged me since the first time I started looking at it:

if i.file_path in self.file_cache: # TODO: test if it really works continue

These were extra suspicious to me because, not much later, they're followed by this:

self.file_cache.file_cache[i.file_path] = i

that is, we check if the thing is in self.file_cache and move on if it is, but if it's not, we add it to self.file_cache.file_cache? That didn't look right at all. But up till now I'd left it alone, because hey, it had been this way for years, right? Must be OK. Well, this afternoon, in passing, I thought "eh, let's try changing it".

Then things got weird.

I was having trouble getting the compose process to actually run exactly as it does in production, but once I did, I was getting what seemed like really bizarre results. The original code was taking 22 minutes in my tests. My earlier test of my branch had taken about 14 minutes. Now it was taking three seconds.

I thought, this can't possibly be right! So I spent a few hours running and re-running the tests, adding debug lines, trying to figure out how (surely) I had completely broken it and it was just bypassing the whole block, or something.

Then I thought...what if I go back to the original code, but change the cache thing?

So I went back to unmodified pungi code, commented out those three lines, ran a compose...and it took three seconds. Tried again with the check corrected to self.file_cache.file_cache instead of self.file_cache...three seconds.

I repeated this enough times that it must be true, but it still bugged me. So I just spent a while digging into it, and I think I know why. These file caches are kobo.pkgset.FileCache instances; see the source code here. So, what's the difference between foo in self.file_cache and foo in self.file_cache.file_cache? Well, a FileCache instance's own file_cache is a dict. FileCache instances also implement __iter__, returning iter(self.file_cache). I think this is why foo in self.file_cache works at all - it actually does do the right thing. But the key is, I think, that it does it inefficiently.

Python's preferred way to do foo in bar is to call bar.__contains__(foo). If that doesn't work, it falls back on iterating over bar until it either hits foo or runs out of iterations. If bar doesn't support iteration it just raises an exception.

Python dictionaries have a very efficient implementation of __contains__. So when we do foo in self.file_cache.file_cache, we hit that efficient algorithm. But FileCache does not implement __contains__, so when we do foo in self.file_cache, we fall back to iteration and wind up using that iterator over the dictionary's keys. This works, but is massively less efficient than the dictionary's __contains__ method would be. And because these package sets are absolutely fracking huge, that makes a very significant difference in the end (because we hit the cache check a huge number of times, and every time it has to iterate over a huge number of dict keys).

So...here's the pull request.

Turns out I could have saved the day and a half it took me to get my rewrite correct. And if anyone had ever got around to TODOing the TODO, we could've saved about 20 minutes out of every Fedora compose for the last nine years...