As the lack of updates might suggest, I’ve taken a break from my Tenchu: Stealth Assassins reverse-engineering/decompilation project. There’s a bunch of reasons for that and some of them are possibly relevant for whatever readership I have, so I figured I might as well write about them here.

I can’t believe it’s not Tenchu: Stealth Assassins news

For context, as part of my reverse-engineering/decompilation efforts I’ve developed a Ghidra extension that can export relocatable object files. I’ve been working on this tooling over the past two and a half years on my own, but lately something unexpected happened: it started gaining users.

This is a summary of what happened with it over the last couple of months.

Teaching an old elf COFF tricks

Exporting object files is a rather unconventional approach given that assembly files would be a more obvious and natural choice. The key insight here is that object files are essentially containers for arrays of bytes and the container part is mostly unopinionated about its contents. That makes them the real lingua franca of traditional toolchains.

In contrast, there are a lot of assemblers out there, each with their own syntax and quirks. Just for Windows x86 there’s MASM, FASM, NASM, YASM, GAS…

Simply put, object file formats are standardized, interoperable and far less numerous.

Taking this observation into account, I’ve made efforts to architecture my Ghidra extension in a manner that would allow supporting multiple object file formats and instruction sets, even back when all it supported was generating ELF object files for 32-bit little endian MIPS.

Eventually, I’ve added i386 support in order to carry out a case study, but the object file format consideration stayed theoretical until someone contributed a COFF object file exporter. It was very rough around the edges initially, but after spending some weeks fixing all the problems reported by another highly motivated user, it’s fully usable now.

What do you mean, you’re not relinking it?

Soon after, someone else stumbled upon my Ghidra extension with an unusual use-case: as part of a video game decompilation project, they wanted to use my extension to delink a program into object files. So far so good, but these object files would not be reused into a program or a library like I usually do.

Instead, these delinked object files would be used as a reference to compare against the output of their decompilation efforts using objdiff, a fancy object file comparator. I’ll admit I was at first a bit skeptical about this, because while my tooling can produce object files that are interchangeable with the original ones, I haven’t designed it to produce equivalent ones.

Fortunately, objdiff has an option for relaxed relocation diffs, where different but equivalent relocations are considered equal. The only actual issue was one of symbol visibility for labels and switch tables, which was accommodated with some new heuristics and options inside the exporters. I didn’t even have to implement section-relative relocations, a feature that I keep postponing.

This incidentally brought me to the decomp.me Discord server, where I’ve discovered that the decompilation community has a term for this kind of tooling: binary splitters, of which splat appears to the most well known. However, my Ghidra extension is quite different than what the community produced, where the focus is more on creating ready-to-use decompilation frameworks.

A new Ghidra setup GitHub action

As an aside, I’ve also received a pull request for an up-to-date Ghidra setup GitHub action. I’ve done the responsible thing, that is quickly scrolling through the GitHub action repository (which is written in a programming language I’m not familiar with) until I said fuck it, I’ll merge it.

After this decision, I started worrying about software supply-chain security for my Ghidra extension. It’s a niche tool with unusual applications, but it’s the kind of stuff that some people with… interesting life stories might use, which in turn might attract attention from other people with… different life stories.

All I’m saying is that while I do not plan on willingly adding undocumented features like a backdoor to my Ghidra extension (not that it has any documentation anyways), that doesn’t mean that Ghidra extensions aren’t a possible threat vector.

It’s very unlikely for my tooling to generate this kind of interest at this point in time, but the XZ Utils backdoor incident has shown that no overworked open-source maintainer is immune to malicious actors.

Big-endian and position-independent code adventures

Yet another person contacted me about using my tooling to delink artifacts that run on IRIX, an Unix operating system by Silicon Graphics, Inc. (or SGI). It seemed straightforward enough, as the ELF and MIPS support was by far the oldest and most mature part of my tooling.

How wrong I was.

The first unanticipated issue was that IRIX ran on big-endian MIPS systems, whereas my tooling was only ever used on little-endian systems up to that point. I had made some nominal accommodations for endianness, but I still ended up refactoring around a quarter of the entire extension in one massive commit anyway.

At least now it should be theoretically possible to delink code for the Nintendo 64 (which is a big-endian system) with my tooling. I’m not going to try though, because I’m far too deep into multiple rabbit holes already.

The second unanticipated issue was position-independent code, which has distinct ABI requirements and specific relocation types (like R_MIPS_GOT16) compared to the kind of MIPS code I usually delink. Fortunately, retrofitting this into the MIPS analyzers doesn’t appear to require large modifications, as it is fairly similar to the R_MIPS_GPREL16 scheme which is already supported.

24 hours of fame on Hacker News

Somewhere in the middle of all of this, I figured it was time to try my luck on Hacker News again and this time it took the bait. It stayed for a day on the front page and was even the top item for two hours, as can be seen from these statistics.

In parallel, the GitHub star count ballooned from around 75 stars to over 350 (making it the fourth most starred GitHub repository in the ghidra-extension topic) and the latest version of my Ghidra extension was downloaded 90 times, but for all I know it could’ve been caused by someone integrating my extension as part of a CICD pipeline. My GitHub account itself was even trending for a bit… near the bottom of the top 25 list in the Java category.

All that ruckus probably doesn’t amount to much. While I’ve put a lot of effort into polishing it, I refuse to believe that my Ghidra extension is so user-friendly that all these people managed to master a very esoteric reverse-engineering technique on their first try without anyone asking for help.

So what’s the problem?

On the surface, all of this sounds good: I’ve created a tool that solves a problem so well that it’s attracting users. While it is unconventional even by the standards of its niche (remember, the French copies nobody and nobody copies the French), somehow a couple of people managed to understand my ramblings and leveraged said tool successfully for their own use-cases.

A fractal of edge-cases

Under the surface, what was supposed to be a side-quest snowballed into a project that is a massive software engineering challenge. I’ve been meaning to write in detail about it for quite some time, but I keep trashing drafts because I sound like a rambling, stark raving lunatic in them.

Instead, I’ll just state that at this rate it’s simply not sustainable. Mucking around in the internals of ghidra-delinker-extension requires an in-depth technical knowledge about multiple ISAs, object file formats, platforms and toolchains. Making this work flawlessly requires a staggering level of exactness and failure to uphold it will invoke some very exotic undefined behavior.

Currently, my extension supports ELF/MIPS, ELF/x86 and COFF/x86. My grasp on x86, COFF and the MSVC toolchain is tenuous at best and yet it’s most of my known userbase right there.

There are hundreds of valid object file formats and ISAs combinations out there. If people start contributing random combinations to fulfill their use-case, I have no hope of keeping up with that.

Furthermore, despite all my efforts spent on code quality and regression testing, I’m having a hard time keeping the extension from imploding under its own weight. The many, many refactorings I’ve carried out in order to extend the data model and algorithms to fit the ever-increasing scope don’t lend to a sense of stability when the slightest of oversights will break the magic in downright malevolent ways.

What’s that over the horizon?

Even if one magically handwaves all of these problems away, there are many more hidden behind them. It’s one thing to delink a program, it’s another to reuse the pieces successfully.

The ability to debug the delinked code is an ongoing struggle because my extension does not generate debugging symbols. That would be another dimension for the support matrix on top of ISAs and object file formats, especially if call frame information is involved in order to enable stepping through it. It’s one rabbit hole that I keep restraining myself from jumping into, because that one looks especially cavernous.

My extension also allows delinking to a platform that is different from the original program, in spite of ABIs or common sense. I’ve successfully managed to slightly bend closely-related ABIs together in my experiments, but to really take advantage of this would require some sort of tooling, one that allows mixing and matching ABIs, possibly even ISAs with impunity.

Even if you don’t want to create unholy chimeras, it would be necessary to deal with link-time optimizations in a scalable manner. In that case, toolchains can basically choose to disregard ABIs in the name of size or performance, as long as they can get away with it.

I expect that for the purpose of delinking, this would translate mostly into functions with made-up, nonstandard calling conventions. That sort of tooling would paper over these digressions so that the API boundary presented by the delinked object file respects the ABI standard of its platform.

This one I don’t even know how to begin to solve.

So what’s the solution?

I don’t know.

All I’ve wanted was a way to divide and conquer my decompilation project where I rejected the perfect matching approach used by many others. Instead, I’ve stumbled upon what appears to be an entire unexplored field of computer sciences straight out of some post-apocalyptic cyberpunk universe.

Pages and pages of binary code, ripped out of the carcass of existing programs and stitched together to make new ones, like Frankenstein’s monsters made out of bits instead of gibs. Functions and data backflowing through the one-way toolchain like a big ball of wibbly wobbly, bitsy wimey stuff. Source code and object code blurred past the point where the implications for the GNU General Public License might give lawyers at the Free Software Foundations cirrhosis. Mad Max, but with people scavenging modules out of compiled programs instead of salvaging mechanical parts from cars.

It’s preposterous. Blasphemous, even. Downright heretical. Sorry about that.

What I do know is that I don’t have it in me to do a PhD, let alone several of them, which is what it would take to explore this. Heck, I can barely handle ghidra-delinker-extension by myself in its current form and at times developing it really was like having a strange mood from Dwarf Fortress. This abyss I’ve gazed into is far deeper than I’ve bargained for and it’s giving me vertigo at the moment.

I’m probably going to take a break from developing ghidra-extension-delinker for a couple of weeks or a couple of months. After two and a half years working on giving computer sciences the finger, it’s time for a much-needed break.

I’m not planning on abandoning this at all because it’s far too useful even in its current nascent state, but I need to touch grass with things that are easier on one’s sanity. There’s only so much one feeble human can buck against common wisdom at a time and I’m starting to sound like an insane person on Hacker News.