CopyFail works on ppc64le

Usually exploits like CopyFail tend to have PoCs or exploits that are architecture-specific. Not this one. Patches should be coming real soon now. Exploit tested using this deobfuscated version.

% arch
ppc64le
% make
cc -Wall -Wextra -Os payload.c -o payload
% python3 poc.py
uid=0(root) [...] context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.

Alternatively, disable the defective kernel module (adjust for your distro, as root). This may not always be possible; on my Fedora system I got rmmod: ERROR: Module algif_aead is builtin. In that case, you're going to be building a new kernel with AF_ALG disabled.

# echo "install algif_aead /bin/false" > /etc/modprobe.d/disable-algif-aead.conf
# rmmod algif_aead

To revert:

# rm /etc/modprobe.d/disable-algif-aead.conf
# modprobe algif_aead

PowerPC and the Western Digital My Book Live debacle

Users relying on the Western Digital My Book Live and My Book Duo NAS systems had an ugly surprise last week when they were abruptly and remotely reset to factory default, erasing all their data. A combination of multiple exploits and Western Digital commenting out a password check appear to be responsible not only for the injury of data loss, but also the added insult of being infected with malware at the same time to join a botnet.

The interest to us here is that the WD My Book Live and Duo family are 32-bit PowerPC devices, more specifically the 800MHz Applied Micro APM82181, which is an enhanced 90nm PowerPC 440 core with additional DSP instructions called the PPC 464. The PowerPC 464FP used here includes a 7-stage pipeline and floating-point unit, and the APM82181 adds a DDR2 controller (256MB onboard) and 256K of RAM configurable as L2 cache. You can boot Gentoo and OpenWRT on it, all of which is unsurprising because the My Book Live basically runs Debian. Western Digital has not issued updates for this device since 2015 and many distros (including Debian, starting with stretch) have dropped 32-bit PowerPC support, though it is still supported in the kernel (except for the PowerPC 601) and these operating systems plus Void PPC and others still support the architecture generally.

The attack abuses a zero-day (CVE-2021-18472) to drop a malware executable named .nttpd,1-ppc-be-t1-z. This is a 32-bit PowerPC-compiled ELF binary and is part of the Linux.Ngioweb family, which in its most recent iteration "supports" 32 and 64-bit x86, MIPS, ARM (32/64) and PowerPC, and there are rumours it's been spotted on s390x (!) and Hitachi SuperH. There is no "preferred device" and the new presence of this malware on PowerPC hosts simply means the authors write good portable code and are expanding to more targets (we'd rather they were porting more generally useful applications, of course).

The upshot of all this is platforms are only as good as their security. There's nothing about the vulnerability which is specific to PowerPC, merely to the spin of Debian they use and what they've layered on top of it. WD recommends disconnecting these NASes from the Internet, and technically as IoTs they probably shouldn't be out naked on a WAN in the first place, but a better idea is to put something on them that's actually supported and maintained. It's fortunate that these devices are "open enough" that you can do it. What about the systems or hardware that aren't?

Is the warrant canary still warranted?

UPDATE: Raptor will keep the canary but reduce the frequency to every six months. There appears to be some significant cost to them, so this seems like a good compromise to me.

Somebody is actually watching Raptor's warrant canary, and mentioned it hasn't been updated in 6 months (as of this writing the last date is March 3, 2019). Although my usual tendency is to glance at it before installing a firmware update, 1.06 is over a year old, so I hadn't noticed myself.

Conceptually, the warrant canary helps to protect purchasers by acting as a negative indicator if they are under a gag order regarding a subpoena or other state actor legal action: if the canary disappears or isn't reupped, then caveat emptor. Raptor's response in the Twitter thread suggests that the failure to update was inadvertent and my gut impression is this is probably true, but the real question is how likely Talos or Blackbird owners are to be targets for state-level threats. We're using niche machines here but the OpenPOWER workstation userbase tends to be more cognizant of how it can be monitored, and if we weren't on watchlists before OpenPOWER started getting more popular, especially in certain countries the number of workstations may now be at a level where such concerns are no longer preposterous.

Raptor, in the same thread, is asking users to speak up about whether the warrant canary is still useful. (They mention a cost/benefit ratio; I'm interested to hear what the cost is. Is it time, money, both?) Lest one think a smaller company could be pushed around more easily, I don't think size is really a factor here; in fact, I'd argue that a bigger company is even less likely to care about such things because of increased bureaucracy and potentially competing internal priorities over government contracts. I agree their point we really should get comfortable with rolling our own firmware is very well taken, but by the same token it's not necessarily a small task for an individual to audit Raptor's tree either. Particularly for critical or time-sensitive updates we will still have some level of vendor dependency and it would be nice to have the canary in those circumstances when using a pre-built firmware package becomes necessary, so put my vote down as "please keep it." We're using these machines for a reason, and the more failsafes there are, the more we're better protected from Mayhem — like meow.

(*not sponsored or endorsed by Allstate)

Serious hash table kernel bug (CVE-2019-12817)

A while ago when I was running my Talos II in hashed page table (HPT) mode for KVM-PR purposes, I started noticing some weird kernel behaviour with recent versions of Firefox. In dmesg were weird messages like this:

[337262.237052] ida_free called for id=170 which is not allocated.
[337262.237089] WARNING: CPU: 6 PID: 12276 at lib/idr.c:519 ida_free+0x114/0x1e0

Initially these were annoying but seemed innocuous. But later on I started getting some weird lockups and I wasn't sure what was going on, so as part of an attempt to figure it out I switched the machine back into radix MMU mode (i.e., I removed disable_radix from the kernel arguments) and the problems disappeared. I reported the phenomenon and the backtraces to the good folks at OzLabs, and Michael Ellerman said he'd look into it.

Turns out the problem was a little deeper than just some weird kernel warnings. From Michael's detailed report, way back around 4.17 code was introduced to support mappings above a 512TB address in the event of a segment lookaside buffer miss. (The SLB has been a tricky devil before.) This situation might seem exotic at first glance, but such addresses are eminently possible in a 64-bit address space. The new code enabled a subtle bug: if a process allocates memory in that range with mmap(), and then forks a new process, the child erroneously maintains the parent's "context ID," a handle-like structure, to that memory mapping and both the child and parent can stomp on the same range of memory. This is obviously bad, but it gets worse when the child process exits, because all of the context IDs it had (including the one it incorrectly inherited) are now freed and thus sets up a use-after-free error where a subsequent unrelated process might get that context ID and access the original parent's space or vice versa. The kernel messages I was seeing was the kernel detecting the situation when both the child and parent exited before a subsequent process got the bogus context ID (and complaining about the double free). The hangups may well have been when the kernel didn't.

There are currently no known circulating exploits for this flaw but it's pretty clear this could be the basis of a nasty attack if a malicious sort were able to trigger such a mapping and then use it to victimize a subsequent process. All 64-bit POWER and PowerPC systems that use a 64K page size are vulnerable in HPT mode (but the Adelie Linux people can smirk a little here, because they use 4K pages, which are not vulnerable). Prior to POWER9, all systems are HPT, so this means everything is affected from the G5 on up including the PA6T and POWER4/5/6/7/8. Additionally, if you use KVM-PR on your POWER9, emulate a POWER8 guest in KVM-HV, or use KVM-PR within a KVM-HV guest, you are also vulnerable because your machine/guest must be using HPT mode. 32-bit PowerPC systems are unaffected.

Most POWER9 systems are probably using the radix MMU by default. If you aren't, then you should, at least temporarily (I haven't switched back to HPT on my own Talos yet, fortunately). dmesg will tell you in the first few lines:

[0.000000] dt-cpu-ftrs: setup for ISA 3000
[0.000000] dt-cpu-ftrs: not enabling: system-call-vectored (disabled or unsupported by kernel)
[0.000000] dt-cpu-ftrs: final cpu/mmu features = 0x0000f86f8f5fb1a7 0x3c006041
[0.000000] radix-mmu: Page sizes from device-tree:

If you see hash tables referenced instead, then you are in HPT mode, and you should remove disable_radix from your kernel arguments and restart.

If you are on a pre-POWER9 system, however, there is currently no effective mitigation, but the good news is that the patch has landed in the kernel tree. It has made it to the RC for 5.1.15, so it should enter Fedora quickly (however, my F30 system is still showing 5.1.12 as current as of 7:30pm Pacific). Update: Red Hat is tracking this as bug 1720616. Debian has an advisory page and is tracking the flaw. Ubuntu is tracking this issue as USN-4031-1 and already has updates.