Tag: code

Here comes level names. This was the most painful part of the whole translation hacking. First, they are made of sprites. Second, there is a limited number of them. And last but not least, the table that tells which sprites to use for a given level name can’t be expanded.
The LSB of the pattern indices for the first level are stored at $5196 (file offset) and $519C for the second. After putting a read breakpoint on $*4f96 (remember that there’s usually a 512 bytes header to PC Engine roms) and $*4f9C, I ended up at $8E3F. A quick look at the code show me there is a address table stored at $8F81 where each address starts 1 byte before the LSB pattern index list. The first byte indicates the number of sprites. Here’s the complete index list:

8F95 : 05 00 02 04 06 08
8F9B : 05 0E 02 12 06 08
8FA1 : 05 18 0A 16 06 08
8FA7 : 05 14 00 16 06 08
8FAD : 05 1C 02 20 06 08
8FB3 : 04 18 1A 06 08
8FB8 : 04 22 24 06 08
8FBD : 05 02 10 18 06 08
8FC3 : 04 26 28 06 08

At this point, I managed to avoid using compressed data. Unfortunatelly the whole sprites were compressed. In fact, there are 4 “decompression” schemes. The first one is a direct transfer to VRAM using the tia instruction. The second is using a simple RLE. The third one and the most tricky mixes RLE and some XOR. The last one simply copies the data to VRAM using a handcrafted loop.

f796:   LDA <$7d
        ORA <$7e
        BEQ f7e1
        JSR f888
        CMP #$00
        BNE f7ac
                ; #1 Transfer to VRAM using tia
                TIA $3700, $0002, $0020
                BRA f7d2
f7ac:   CMP #$02
        BNE f7bc
                ; #2 RLE
                JSR f817
                TIA $3700, $0002, $0020
                BRA f7d2
f7bc:   CMP #$03
        BNE f7cf
                ; #3 RLE + XOR
                JSR f817
                JSR f857
                TIA $3700, $0002, $0020
                BRA f7d2
f7cf: ; #4 Loop transfer
        JSR f803

f7d2:   LDA <$7d
        SBC #$01
        STA <$7d
        LDA <$7e
        SBC #$00
        STA <$7e
        BRA f796

f7e1:

The RLE routine:

f817:   LDA [$78]
        STA <$83
        LDY #$04
        LDA #$09
        STA <$81
        STZ <$80
        CLX
        LDA #$20
        STA <$82
       
f828:   DEC <$81
        BNE f83a
        PHY
        LDA #$08
        STA <$81
        INC <$80
        LDY <$80
        LDA [$78], Y
        STA <$83
        PLY

f83a:   ROR <$83 ; the rle counter
        BCS f849
        STZ $3740, X
        INX
        DEC <$82
        BNE f828
        JMP f7f6

f849:   LDA [$78], Y
        STA $3740, X
        INY
        INX
        DEC <$82
        BNE f828
        JMP f7f6

The “XOR” routine:

f857:   PHX
        PHY
        LDY #$07
        CLX
       
f85c:   LDA $3740, X
        EOR $3742, X
        STA $3742, X
        LDA $3741, X
        EOR $3743, X
        STA $3743, X
        LDA $3750, X
        EOR $3752, X
        STA $3752, X
        LDA $3751, X
        EOR $3753, X
        STA $3753, X
       
        INX
        INX
       
        DEY
        BNE f85c
       
        PLY
        PLX
        RTS

The XOR is used here to maximize the number of repetitions in order to make the RLE more efficient. Well, that’s how I understood it. I didn’t make any benchmark to verify if it was really the case. The encoder was written pretty quickly. I reworked the font so that it more or less matches the ASCII set. This simplified the script encoder and the display routine.

Back to level names! I got help from a bunch of people to translate them. The last character of each level can be translated to country. Unfortunately that was far too big so I switched to “land”.

• Candy Land
• Toy Land
• Greenery Land
• Ice Land
• Time Land
• Water Land
• Sky Land
• Mirror Land
• Queen Land

Here’s the actual “font” made of 16×16 sprites.
The zigzag lines on the last 16×16 sprite are there to make the compressed data fits into 504 bytes. I must also keep an empty sprite for spacing.

I had the sprite index list done pretty quickly. But when I started working on the index pointer table, I realized that there were 9 entries. 9? Did I forgot one?
Yes, I forgot one. The water land… And I didn’t have enough room to put the missing letters (bloody size limit).

After a cautious review, it was decided to rename “Greenery land” into “Green land”. As There were already a “Queen land”, there’s now enough room for the missing “Water”.

The only issue remaining was that the sprite index list is 42 bytes long, and mine was 43. As each level name ends with the same sequence I can use the last index as the beginning of the next one (each sequence starts with the length of the index list). So I swapped the sprite for “nd” with the beginning of the one whose index was equal to a sequence length, namely “To” which was at index 6. Why 6? First because it’s the length of the longest index list and also because sprite indexes are always even.

Here’s the original index list (the fist 2 hexadecimal values are the values of the pointer table):
8F95 : 05 00 02 04 06 08
8F9B : 05 0E 02 12 06 08
8FA1 : 05 18 0A 16 06 08
8FA7 : 05 14 00 16 06 08
8FAD : 05 1C 02 20 06 08
8FB3 : 04 18 1A 06 08
8FB8 : 04 22 24 06 08
8FBD : 05 02 10 18 06 08
8FC3 : 04 26 28 06 08

The new one:
8F95 : 05 00 06 04 24 06
8F9B : 04 02 04 24 06
8FA0 : 05 08 0A 0C 24 06
8FA6 : 04 0E 10 24 06
8FAB : 05 12 14 28 24 06
8FB1 : 05 20 22 26 24 06
8FB7 : 04 16 04 24 06
8FBB : 06 18 1A 1C 28 24 06
8FC2 : 05 1E 0A 0C 24 06

Shortly after beating the evil level names I received the first version of the translated script. Unfortunately the first tests revealed that the current text layout was unreadeable. There were only 2 or 3 words per line. So I had to crawl brack into the rom and change the text box position and size. I was only some couples of values to change.

So the 24th of July 2011, the english patch was released. Nearly 2 years after I started working on it… Anyway, some days after the release a couple of angry frenchmen complained that we didn’t release a french patch. In order to silence those annoying creeters I had to modify the font and the script encoder in order to add accents. That was not a big deal. But I let the french translators handle the level names (with some kind of sadistic grin). After some weeks of hard work, they decided to give up and use the english names. The french translation was released in December 2011.

• English translation
  • patch
  • source code
• French translation
  • patch
  • source code

Oh hy! I didn’t notice you were here. Please take a seat.
So I were talking about the font. As shown in the following image, the original font already contained uppercase letters.

As I wanted to add lowercases and some punctuations, I had to find where and how the font was stored in ROM. The visual editor told me that the font was stored from $7300 to $7a10 in VRAM. So under the code debugger I set an aux write breakpoint (ctrl+w) to these addresses. The breakpoint was triggered at $f7c6 on :

TIA $3740, $0002, $0020

If you look closely at the following image, you’ll notice that there are 2 similar preceding occurrences. This is it! You are looking at the piece of code in charge of decompressing the tile data into VRAM.
This code supports 4 kind of compression:

• The tile is filled with 0.
• Uncompressed. The data is directly transfered to VRAM.
• Simple RLE encoding.
• Simple RLE encoding with some kind of postprocessing involving XORing consecutive lines.

Laziness told me to avoid writing a little tool to compress the font. So I checked how many uncompresserd char were used and if it was enough to store lowercases. Hopefully there was enough room for all of them.

At this point I had nearly everything I wanted. There were no need to modify the string display routine. Most of the work done would have been done by the script encoder as the lowercases characters where not contigous. It was not a big deal. Nevertheless I wanted to check if there will be enough room for the translated texts and how much characters could be displayed per screen. Basically you had 16 characters per line and 7 or 11 line depending on the screen. This gave me an idea of the maximal script size. Unfortunately it was too big. So I looked for a fast/small/simple text compression/decompression for 6502 based cpu. I had a quick look at the romhacking.net forum. And it seems that one of the most used technique is DTE (Dual Tile Encoding). It’s a pretty simple approach. A text does not use all the ascii values. Basically you only use upper and lowercases (52 chars), numbers (10) and a dozen punctuation marks. This leaves you with 182 unused values. You can use them to store couples of characters. Bytes with values between 0 and 74 represent a single character and all the others represent a tuple. For example 182 represents ‘ab’, 183 ‘en’ and so on. Here is quick example: Let’s compress the string “abcacbabcabcabba”. First we extract all the characters couples and count them:

• ab : 4
• bc : 3
• ca : 3
• ac : 1
• cb : 1
• ba : 2
• bb : 1

“ab” is the tuple with the highest occurence. We insert it in the symbol table: a,b,c,d=’ab’. And let’s replace it in the original string:“dcacbdcdcdba”. DTE only works on the original symbols. So in the second pass we must avoid the newly inserted symbols. The remaining tuples are :

• ca : 1
• ac : 1
• cb : 1
• ba : 1

The new symbol table is : a,b,c,d=’ab’,e=’ca’,f=’cb’,g=’ba’. In the end “abcacbabcabcabba” becomes “defdcdcdg”. The tricky part is to choose the right substitution sequence that will lead to the smallest possible string. It’s a whole new game and you can stick with choosing the tuple with the highest occurence.
Let’s get back to Alice. I grabbed the infamous lorem ipsum and made a script out of it. I compressed it with DTE. And I still had a lot of unused values. Remember the second pass of our example when I excluded the newly inserted symbol for the occurence counting. If I included it, DTE would in fact have been what is called BPE (Binary Pair Encoding). Here’s the mandatory academical reference : Byte Pair Encoding: A Text Compression Scheme That Accelerates Pattern Matching (1999). Using BPE we would have had for “dcacbdcdcdba”.

• dc : 3
• ca : 1
• ca : 1
• ac : 1
• cb : 1
• bd : 1
• cd : 2
• db : 1
• ba : 1

Let’s add “dc” in the symbol table a,b,c,d=’ab’,e=’dc’. We now have “eacbeedba”. BPE is basically a recursive algorithm. So when we are decoding a string, we must check if it’s part of the tuple symbols (let’s call them “special” symbols). If that’s the case we will have to loop until the decoded symbol does not contain any special symbol. If speed is a concern we may limit the depth of recursion. This means that when we must take the depth of each symbol into consideration. For example imagine we have 2 tuples “dc” and “ca”. Both have the same number of occurence in the string. We may choose “ca” instead of “dc” because it only contains “base” symbols.
Using BPE the “maximal” script was small enough to fit. I made a silly test using some part of Hamlet (here’s the script). You can find the BPE compression/decompression as long as code for generating IPS files in this [archive].

Next : Levels names. Shitty part : they are sprites. Describe pointer tables and all.
All compressed. So I can’t avoid it anymore. Describe the compression scheme.
Got help for level name translation. (show the level names and their translation).
As I have a compressor, reworked the font so that it more or less match ascii. Had to spot where the font was rendered to vram. (link to final font). Simplify script encoder and display routine.

Coming next: Level names. A world of pain and horror!