Merrow Postmortem: Patches

This whole randomizer business all started because of IPS.

IPS is an old-school patch file format, and the only one that I was aware of from old days of applying translation patches to games never released here, etc. There have been lots of newer patching formats since then, of course, but I was only vaguely aware of them.

So when I was asked if I could make a patch that would make one specific change in Quest 64, I thought of IPS. Searching for info about IPS patches and how I could make them, I found the ZeroSoft spec page for IPS that broke down how a patch is structured: https://zerosoft.zophar.net/ips.php.

The important thing this showed me about IPS was that despite not having any fancy features or protections, IPS is extremely simple and (mostly) human-readable.

If you open an IPS patch in a hex editor, it consists of this:

  • A 5-byte [header], which is just the word "PATCH" written out in ascii hex values: [50 41 54 43 48].
  • Some number of [records], which I'll detail below.
  • A 3-byte [footer], which is just the word "EOF" (End Of File) in ascii hex: [45 4F 46]. 

Records are little chunks of data of variable length, that are made up of 3 parts:

  • A 3-byte [offset], which is a hex address from [000000]-[FFFFFF], where data will be written.
  • A 2-byte [size], which is the number of bytes of data to write to a patched file.
  • [Data], which is [size] bytes of data in a row that you want to write at the location specified in [offset].

IPS can do a couple other things too, but reading that spec was enough for me to realize something a bit silly - this patching format is just a bunch of hexadecimal text. And while I didn't know then how to move data around, I definitely knew how to fiddle with text strings in C#.

Having clumsily learned a few different kinds of coding over the years, editing strings was easy. And it turned out that every patch (and every file, or game, or application) is actually just one giant wall of hexadecimal characters, which could be stored as a string.

[50415443481A2A3A025B6B454F46]: the content of an IPS patch that writes two [02] bytes of data [5B 6B] to the patched file starting at an address [1A2A3A]. 
Might be easier to read if I write it like this:
[[5041544348][1A2A3A][02][5B6B][454F46]].
So this patch changes the byte at address [1A2A3A] to [5B], and [1A2A3B] to [6B].

There's more to it than that, to be fair - that patch data I just wrote up there is ascii text, and you can't just rename a text file containing that to .IPS and expect it to work.
But you can take a text string like that one, and code a little function in almost any language that makes a binary file that contains that patch data, stored in hex rather than text. And that's an IPS patch.

So my randomizer kicked off from there. Once I realized I could just write addresses and data as strings, bash them together with "PATCH" and "EOF", and convert it to a binary file, I was set. Here's an example from the randomizer code to show what I mean:

/* Fast Monastery: write 00090002 as new door target ID at 4361A0 */
if (rndFastMonasteryToggle.Checked) 
{ 
    patchstrings.Add("4361A0");
    patchstrings.Add("0004");
    patchstrings.Add("00090002"); 
}

The code says, if the option "Fast Monastery" is enabled, my big list of patch strings will get three things added that will edit one in-game door's data: the offset, the size, and that many bytes of data. And that patch will tell the patcher to write the [04] bytes [00 09 00 02] to the file, starting at [4361A0], which changes where the door goes.

Each option enabled in the randomizer like the one above, will add to that giant pile of patch strings. And then at the end, after all the options are processed:

/* Patching mode: CREATE IPS PATCH */
if (expModePatchIPS.Checked) 
{
    patchbuild += headerHex;
    /* mash all the patch strings together */
    for (int ps = 0; ps < patchparts; ps++)  
    { 
        patchbuild += patchstrings[ps]; 
    }
    patchbuild += footerHex;
}
/* convert the string to a bytearray */
patcharray = StringToByteArray(patchbuild, true); 
/* write the bytearray to an ips file */
File.WriteAllBytes(filePath + fileName + ".ips", patcharray);

This function makes one big string, that contains the IPS header, all the offset/size/data triplets in order, and the IPS footer. Then it converts that string to a ByteArray, which turns it from hex text to hex binary, and then saves that to a new .ips file.

It's kind of a simple brute-force method with no error checking, but it works, which is the most important thing.
Here's the StringToByteArray function, just for reference:

/* Convert hex string to byte array */
public static byte[] StringToByteArray(string hex, bool addZero) { 
    int NumberChars = hex.Length;
    /* Add a leading zero to odd-length hex */
    if (NumberChars % 2 != 0) 
    {
        hex = "0" + hex;
        NumberChars++;
    }
    byte[] bytes = new byte[NumberChars / 2];
    /* Fill the byte array, two hex characters at a time */
    for (int i = 0; i < NumberChars; i += 2) 
    { 
      bytes[i / 2] = Convert.ToByte(hex.Substring(i, 2), 16); 
    }
    return bytes;
}

Not too long after writing this code, I realized I didn't even need to export IPS patches anymore. I could just tell the randomizer to edit files directly, using those same addresses and strings as instructions. But I'll get to that another time.

It's all just hex, if you dig deep enough. No matter how complex the game, or app, or file, it's eventually just hex. Once I realized that, the terrible enormity of the goofy stuff I could get away with started to hit me.

And no one stopped me, so a randomizer happened.