A penguin, an USB device and a video game console walk into a bar.

Some months ago I bought a cheap usb video capture device (dx.com). Using it with vlc under Windows is pretty straightforward. What about Linux?
First things first, if you are running a Debian Wheezy with a 3.2.x kernel you have nothing to do. The em28xx module is already here. You can use vlc or mplayer. Unfortunatly I didn’t managed to configure vlc properly. After some tweeking and intense internet browsing, I get the following parameters:

mplayer tv:// -tv driver=v4l2:width=720:height=480:outfmt=uyvy:norm=NTSC-M:device=/dev/video1:input=1:fps=30:alsa:amode=1:forcechan=2:audiorate=48800:adevice=plughw.2,0:forceaudio:immediatemode=0 -ao sdl -vo sdl

The input lag is rather small but it can be troublesome if you intend to record longplays. On the other hand, it’s a must-have if you are a coder. No need for an extra screen! This gadget saved my life at Datastorm 2014. Our intro was nearly done and I borrowed Flubba’s usb EasyCap device to test it on the PC Engine hardware. It was running like a charm under Mednafen but it failed miserably on the hardware (blame crappy bank management code). Hopefully it was all fixed before the deadline. This little usb gizmo was also very helpful when I was working on the Everdrive SD card driver or on the Memory Base 128.

Fire Pro Wrestling 3 title screen   Monster Puroresu title screen

Violent Soldier title screen   In game capture.

Roger Quartermain et les parchemins perdus de l’ère Heisei.

This is a french translation of Roger Quartermain and the lost script of the Heisei era originally published on ROMHack forum(a french forum about videogame fan translations).

Roger est heureux. Il a récemment fait l’acquisition à vil prix d’un jeu PC-Engine CDROM, “Nishimura Kyotaro Mystery.: Hokutosei No Onna”. Selon toute vraisemblance, il s’agirait d’un jeu d’enquête policière comme J.B. Harold Murder Club, Jake Hunter, Ace Attorney, Le Manoir de Mortevielle ou encore Maupiti Island.
Il fut publié à l’aube de l’année 1990 par Naxat (maintenant Kaga Create).
Alors qu’il glisse délicatement le disque dans sa bonne vieille DUO, il se rappelle que certains jeux offrent la possibilité de changer la langue. Par exemple, la version japonaise de J.B Harold Murder Club peut être joué en anglais. Malheureusement une telle option est introuvable dans Hokutosei No Onna… Qu’à cela ne tienne! Roger Quatermain attrape ses bottes, son chapeau et plonge vaillamment dans les entrailles sombres et fumeuses du jeu.
L’introduction montre un pauvre quidam se faire poignarder, deux jeunes femmes insouciantes se rendant à une gare et un personnage louche en train de les espionner.



S’agissant d’un jeu CDRom, ce dernier utilise la fonction du BIOS pour afficher le texte. Roger saisit ses vieilles notes et les parcourt frénétiquement en quête d’informations sur une quelconque routine d’affichage de texte. Et la voici! ex_fnt à l’adresse $e060. Il dégaine alors son émulateur favori et positionne un point d’arrêt à l’adresse indiquée.

Le point d’arrêt est atteint après l’introduction sur ce qui semble être l’écran de sélection des sauvegardes à charger.

Contrairement à ce qu’il pouvait laisser entendre Roger n’est pas intéressé par ex_fnt mais par le code appelant cette routine. Pour ce faire il place des points d’arrêts sur toutes les instructions rts d’ex_fnt. Une brève pression sur la touche R l’amène sur $f1e2. Une autre pression sur la touche S l’envoie sur l’appelant.

Il se retrouve donc à $566b.

D’après les notes, le symbole shift-jis est chargé depuis $41, $42 vers $f8 et $f9. La logique lui dicte donc de placer un point d’arrêt en écriture sur les adresses $41 et $42.

Le point d’arrêt est levé en $5b20. Le code est somme toute assez simple.

5b1c: ldy #$00
      lda ($3c), Y
5b20: sta $41
      iny
      lda ($3c), Y
      iny
      sta $42

L’étape suivante consiste à chercher où le pointeur $3c est initialisé. Roger place donc un autre point d’arrêt $3c et $3d.

5a18: lda #$90
      bra $5a22
      lda #$40
      bra $5a22
      lda #$80
5a22: sta $c0
5a24: sty $3d
      cpy #$00
      beq $5a2f
      stx $3c
      smb5 $c0
      rts

Malheureusement le code appelant est tout sauf trivial. Par contre la bonne nouvelle est que le texte n’est pas compressé. En notant attentivement les valeurs stockées en $41 et $42, Roger réussit à extraire la chaîne:

40 81 40 81 40 81 C7 82 CC 82 54 8B E4 88 59 8C 96 8E C5 82 6E 8E DF 82 DC 82 B7 82 A9 82 48 81 0D 00 FF 00

Le boutisme doit être inversé. Heureusement il a sous le coude un script Perl fort utile, swap_endian.pl. Les 4 derniers caractères ressemblent à des codes de contrôle. Sa grande et longue expérience lui permet de déduire que 00 0D doit symboliser un saut de ligne et 00 FF la fin du texte. Voici la chaîne décodée.

   どの亀井刑事で始めますか?

Mais tout cela ne lui dit pas où le pointeur $3c est initialisé. Roger sait que sur CDRom le code est d’abord transféré en RAM avant d’être exécuté. Cela signifie aussi que le code peut être auto-modifiable. Et manque de chance, c’est le cas ici. Roger doit encore mettre un point d’arrêt à l’emplacement où le saut est effectué et suivre le fil d’exécution jusqu’à atteindre l’emplacement où le pointeur est initialisé. Plusieurs heures passent et il échoue finalement en $724d
724d: ldx #$fc
      ldy #$74
      jmp $5a18

Il redémarre le jeu en ne laissant qu’un point d’arrêt sur $41 et $42. Comme prévu il se retrouve là où il a découvert la première chaine. Il décide donc de continuer son chemin et se retrouve en $acb4.

acae: lda $4f
      asl A
      tay
      lda ($50), Y
acb4: sta $41
      iny
      lda ($50), Y
      beq $acd8
      sta $42
      lda $2922
      sta $3f
      lda $2923
      sta $40
      lda #$01
      jsr $5655

Le pointeur $50 est initialisé en $ac92.

ac8a: tay
      lda $312a, Y
      asl
ac8e: tay
ac90: lda ($9f), Y
      sta $50
      iny
      lda ($9f), Y
      sta $51

Ainsi donc le pointeur est initialisé à partir d’une table. Ses valeurs sont

  • b1b6
  • bdb6
  • cdb6
  • d9b6
  • e7b6

Par chance les indexes pour ces pointeurs sont 2,3,4,5,6. Roger cherche dans l’iso la chaîne b1 b6 bd b6 cb b6 d9 b6 e7 b6. Il la trouve à l’offset $11f49. L’endroit semble baigner dans une soupe de pointeurs. Quelques octets plus loin il reconnait du texte en shit-jis. Il commence en $11f53. La “soupe” doit donc contenir des valeurs ressemblant à $53Xf. Il remonte et trouve $53af à $11d45. Tadam, il a sa première table et son premier bloc de texte. Comme tout les pointeurs sont en ordre croissant, Roger en conclue que la liste contenant les noms des options est stockée de $11f53 à $126f4. Il remarque d’autres symboles shift-jis à la suite. Ils vont de $126f5 à $132f5.
Une fois les options affichées, Il passe sur un autre écran où un vieux briscard de la crim’ lui taille le bout de gras.

Une fois encore il met un point d’arrêt sur $41 et $42. Mais à cet instant il se rappelle de $5a18. Cela commence par:

5a18: lda #$90
      bra $5a22
5a1c: lda #$40
      bra $5a22
5a20: lda #$80
5a22: sta $c0

Il peut donc y arriver depuis $5a18, $5a1c ou $5a20. Cela lui donne 3 points d’arrêt. $5a20 est le grand gagnant et comme il s’en doutait il y arrive par un bout de code auto-généré ($59ab). Une longue et pénible route se prépare à l’horizon. Et pénible elle est. Néanmoins, Il découvre que $5a20 est atteint depuis $6c5e.

6c5e: ldy $a5
      lda ($a8), Y
      sta $00
      iny
      lda ($a8), Y
      sta $01
      iny
      sty $a5
      and $00
      cmp #$ff
      beq $6c79
      ldx $00
      ldy $01
      jmp $5a20

La première phrase du vieux briscard se trouve en $89c9 ($1339c9 dans l’iso). Cette adresse provient d’une table pointée par $a8. Lui même initialisé en $707e.

707e: lda #$00
      sta $00
      lda #$80
      sta $01
      lda ($00), Y
      sta $a8
      iny
      lda ($00), Y
      sta $a9
      stz $a5
      stz $a6
      lda #$03
      jmp $619c

Le pointeur est stocké en $13308c. Roger est perplexe. Il ne s’agit pas d’une simple table de pointeur. Elle contient d’autres données. Que signifient-elles? Mais il est exténué et il y a Extra Sangsues sur le câble.
Il s’étire, ferme l’émulateur et son portable et se prépare pour 90 minutes de pur puissance.

Harry Dumper and the order of the Griffin

The Magic Griffin is a game copier. From the menu and the case, it seems to have been made by a taiwanese company called Front Fareast and JSI.
I don’t know if the Front Fareast listed on wikipedia is the same [link]. And I’ve found nothing about JSI.
About the hardware. You plug it in the Hucard slot and the joypad port. Oh I forgot, you can only use it on a PC Engine/CoreGrafx. On the front you have a standard Hucard slot and 2 joypad ports. Only the right one is working. I don’t know what’s the left one is used for. There are 2 parallel ports on the right. One for an external floppy disk drive and another one labeled COM port. I only tested the floppy disk.

Magic Griffin Mugshot Top view
Side connectors
Front View

 

What’s inside?

An Altera chip, a Motorola chip, some RAM, some TTL chips and a UV EEPROM (on the onther side). I haven’t identified the chips yet.
So what happens when you power it on? Well you see this:

It may work like the other flash cards (TotoTek PCE Pro, Krizz Everdrive). This means that it acts like a standard Hucard. But some special addresses are used to communicate with the special hardware. For example the Everdrive can access the SD card via some addressess that are mapped to the SPI registers. I pretty sure the code is hosted on the UV EEPROM.
About the menu.

  • RUN FILE
  • RUN IC CARD
  • SAVE IC CARD
  • RENAME FILE
  • DELETE FILE
  • FORMAT DISK

RUN FILE: Run a file from the floppy disk.

 

RUN IC CARD: Run the Hucard.
Unfortunately I was unable to run any card. It either tells me that there’s no Hucard or it will freeze. This surely comes from oxidized card connector.

SAVE IC CARD: Save Hucard to floppy disk.
It’s the menu that would have sent you to jail in the 90’s :D You name the file, press I and it will start dumping the Hucard on the floppy disk. Unfortunately the controls are awful. Pressing RUN will spit a whole lot of chars. Same for SEL, it’s more likely that it’ll delete the whole filename. You can’t write a single char. So you’ll end up with filenames like ‘ssssooooolll’ or ‘bbbbooommbbb’ :)
As I said before I was unable to run any Hucard. Despite the fact that it seems to have dump the game, I was unable to run any of the game I’ve dumped. I’ll check the dumped files as soon as I get my hands on an USB Floppy drive. But I’m pretty sure I’ll have a bunch of 0xff.

 

RENAME FILE: Rename one of the file on the floppy disk.

DELETE FILE: Delete a file from the floppy disk.

 

FORMAT DISK: Format floppy disk.
You can either format a HD (1.44M) or DD (720K) floppy drive. Be carefull, it’ll start formatting the disk once you enter the menu.

 

That’s all.
I’ll try to clean/fix the connector. And once I receive the USB floppy drive I’ll check the dumped files and verify which filesystem the disk was formatted.
The coolest stuff will be to dump the UV EEPROM. So if someone knows how to safely read it, I’ll be more than pleased to hear it.

Note: This was originally posted on the PC-Engine dev forum [here].

Roger Quartermain and the lost script of the Heisei era.

Roger is happy. He bought a cheap PC-Engine CDRom game called “Nishimura Kyotaro Mystery.: Hokutosei No Onna”. It seems to be a detective game like J.B. Harold Murder Club, Jake Hunter, Ace Attorney, Le Manoir de Mortevielle or Maupiti Island. It was published by Naxat (now Kaga Create) in 1990.
He remembers that some games have a language option. For example, the japanese version of J.B Harold Murder Club can be played in english. Unfortunately such option was nowhere to be found in Hokutosei No Onna… Nevermind! Roger Quartermain grabs his boots, his hat and jumps into the game dark bowels!
The introduction seems to be about some dude stabbed in his room, 2 girls going to a train station and another guy who is apparently stalking them.

Nishimura-0000Nishimura-0013
Nishimura-0004Nishimura-0005
Nishimura-0006Nishimura-0007

As it’s a CDRom game, it must use the BIOS function for displaying text. Roger took his old notes and search for any text related routine. Here it is! ex_fnt located at $e060. He draw his favorite emulator and set a breakpoint to it.
debugger00It fires just after the introduction when what seems to be the savegame selection screen appears.

Nishimura-0008debugger01

Roger is not really interested in the ex_fnt routine but more in the calling environment. So he sent breakpoints to the potential rts. A gentle pression to the R key sends him to $f1e2. Another push to the S key brings him the the caller.

debugger02

Here it is at $566b.
debugger03
According to the notes, the shift-jis is loaded from $41, $42 to $f8 and $f9. The logical next step is to put a write breakpoint to $41 and $42.
debugger04
The write breakpoint is triggered at $5b20. The code is pretty simple.

5b1c: ldy #$00
      lda ($3c), Y
5b20: sta $41
      iny
      lda ($3c), Y
      iny
      sta $42

The next step is to search where the $3c pointer. Once again Roger has to set a new write breakpoint at $3c and $3d.

5a18: lda #$90
      bra $5a22
      lda #$40
      bra $5a22
      lda #$80
5a22: sta $c0
5a24: sty $3d
      cpy #$00
      beq $5a2f
      stx $3c
      smb5 $c0
      rts

Unfortunately the caller is not that obvious to discover. But the good news is that the text is not compressed. By keeping track of the values stored at $41 and $42, Roger manages to extract the string

40 81 40 81 40 81 C7 82 CC 82 54 8B E4 88 59 8C 96 8E C5 82 6E 8E DF 82 DC 82 B7 82 A9 82 48 81 0D 00 FF 00

The endiannes must be swapped. He hopefully has some handsome perl script at hand swap_endian.pl. The last 4 characters looks like some control code. His long time experience enables him to deduce that 00 0D is some kind of newline code and 00 FF may be for end of text. Here’s the string.

   どの亀井刑事で始めますか?

But this doesn’t tell Roger where the $3c pointer is set. Roger knows that on CDRom systems, the data is first transfered to RAM before being executed. This means that the code can be self-modifiable. Unluckily this is the case here. Roger has to set a breakpoint where the jump is done and run it until he finally reaches the pointer initialization code. Time passes and he ends up at $724d
724d: ldx #$fc
      ldy #$74
      jmp $5a18

He restarts the game with only the write breakpoint at $41 and $42. As expected he ends up at the same code as before for the first sentence. The next run lands at a different location acb4.

acae: lda $4f
      asl A
      tay
      lda ($50), Y
acb4: sta $41
      iny
      lda ($50), Y
      beq $acd8
      sta $42
      lda $2922
      sta $3f
      lda $2923
      sta $40
      lda #$01
      jsr $5655

The pointer $50 is initialized at ac92.

ac8a: tay
      lda $312a, Y
      asl
ac8e: tay
ac90: lda ($9f), Y
      sta $50
      iny
      lda ($9f), Y
      sta $51

So the pointer is set up from a table. He gets the values

  • b1b6
  • bdb6
  • cdb6
  • d9b6
  • e7b6

Hopefully the indices for this pointers are 2,3,4,5,6. Roger searchs in the iso for b1 b6 bd b6 cb b6 d9 b6 e7 b6. He ends up at the offset $11f49. The place is filled with what looks like pointers. Some bytes after the string he recognises byte swapped shit-jis values. It starts at $11f53. So The pointer soup must contain some value equals to $53Xf. He scrolls up and finds $53af at $11d45. Tadam, he has his first table and string bloc. As all the pointers are in increasing order, Roger deduces that the option strings are stored from $11f53 to $126f4. But just after this, he notices some other shift-jis symbols. It goes from $126f5 to $132f5.
Once the options are display, he jumps to another screen where some grumpy old cop is talking to you.
Nishimura-0009
Once again, he set his breakpoints to $41 and $42. But then he remembers $5a18. It starts with

5a18: lda #$90
      bra $5a22
5a1c: lda #$40
      bra $5a22
5a20: lda #$80
5a22: sta $c0

So he can get here from $5a18, $5a1c or $5a20. This gives him 3 more breakpoints. $5a20 is the one and as expected it comes from RAM initialized code ($59ab). He prepares for a painful ride. And painful it is. Nevertheless, he finds out that that $5a20 is reached from $6c5e.

6c5e: ldy $a5
      lda ($a8), Y
      sta $00
      iny
      lda ($a8), Y
      sta $01
      iny
      sty $a5
      and $00
      cmp #$ff
      beq $6c79
      ldx $00
      ldy $01
      jmp $5a20

The first sentence of the grumpy cop comes from $89c9 (iso offset $1339c9). This address comes from a table pointed by $a8. It’s setup at $707e.

707e: lda #$00
      sta $00
      lda #$80
      sta $01
      lda ($00), Y
      sta $a8
      iny
      lda ($00), Y
      sta $a9
      stz $a5
      stz $a6
      lda #$03
      jmp $619c

The pointer is store at $13308c. Roger is perplex. It’s not just a simple pointer table. There is some extra data. He needs to figure out how it works but he is tired and there’s Night of the Creeps on TV.
He streches, closes the emulator and his laptop and prepares himself for 90 minutes of pure 80’s badassery.

You see this? This… is my boomstick!

(10/23/2012) WARNING! As Tomaitheous pointed out, there’s a problem with this board! Please refer to comments for more informations.

Yeah a new post!

Some years ago (more like a decade than a year), I bought a Turbo Stick. It’s the standard 2 buttons stick for NEC PC Engine console. Unfortunately the stick makes an awfull squealing nose. Forget 1 life run with this plastic beast. After 5 minutes all I wanted to do was to use it as a soccer ball. According to the Wataru’s PC Engine hardware list, 4 six buttons sticks were released. They are not small and given shipping costs I decided to build one. So I bought a stick and some buttons from Sparkfun. Later a friend gave me a Sanwa stick and buttons.

There are several places on the internet where to find PC Engine pad schematics. The main source are Emanuele Bettidi‘s pad schematics, PCE Wiki, NFG PC engine/TG 16 article. I first built a 2 buttons circuit with no auto fire. But there are a few games that requires a 6 buttons stick (Street Fighter 2 for example). The Avnue Pad 6 comes with auto fire and slow mode. The later is a particular ugly hack. The slow mode is acheived by toggling the pause rapidly. It’s as if you drank too much coffee and go berserk on the RUN button. Here are the guts shots and schematic of an Avenue Pad 6. I spent hours trying to make PCB board in Eagle for the “complete” version (with auto fire and slow mode). I didn’t find a local source for the DTC114Y. A replacement was found on the interweb. It was told that this component can be replaced by a 2N3904 and 2 resistors. This is rather straightforward if you look at the DTC114Y datasheet. Anyway, in the end I didn’t liked the way the board looked. The most logical step was to remove the rapid fire and slow mode (schematic). And by the way, I think I’ll never use slow mode and barely ever used auto-fire.

Here are the current version of the schematic and pcb board:

Here’s a 3d view of the nearly version of the board (made with WebGerber)


I need to add mounting holes in the board. Make some board. And last but not least, build the enclosure!

Alice’s bubbles (Part 3)

Here comes level names. This was the most painful part of the whole translation hacking. First, they are made of sprites. Second, there is a limited number of them. And last but not least, the table that tells which sprites to use for a given level name can’t be expanded.
The LSB of the pattern indices for the first level are stored at $5196 (file offset) and $519C for the second. After putting a read breakpoint on $*4f96 (remember that there’s usually a 512 bytes header to PC Engine roms) and $*4f9C, I ended up at $8E3F. A quick look at the code show me there is a address table stored at $8F81 where each address starts 1 byte before the LSB pattern index list. The first byte indicates the number of sprites. Here’s the complete index list:

8F95 : 05 00 02 04 06 08
8F9B : 05 0E 02 12 06 08
8FA1 : 05 18 0A 16 06 08
8FA7 : 05 14 00 16 06 08
8FAD : 05 1C 02 20 06 08
8FB3 : 04 18 1A 06 08
8FB8 : 04 22 24 06 08
8FBD : 05 02 10 18 06 08
8FC3 : 04 26 28 06 08

At this point, I managed to avoid using compressed data. Unfortunatelly the whole sprites were compressed. In fact, there are 4 “decompression” schemes. The first one is a direct transfer to VRAM using the tia instruction. The second is using a simple RLE. The third one and the most tricky mixes RLE and some XOR. The last one simply copies the data to VRAM using a handcrafted loop.

f796:   LDA <$7d
        ORA <$7e
        BEQ f7e1
        JSR f888
        CMP #$00
        BNE f7ac
                ; #1 Transfer to VRAM using tia
                TIA $3700, $0002, $0020
                BRA f7d2
f7ac:   CMP #$02
        BNE f7bc
                ; #2 RLE
                JSR f817
                TIA $3700, $0002, $0020
                BRA f7d2
f7bc:   CMP #$03
        BNE f7cf
                ; #3 RLE + XOR
                JSR f817
                JSR f857
                TIA $3700, $0002, $0020
                BRA f7d2
f7cf: ; #4 Loop transfer
        JSR f803

f7d2:   LDA <$7d
        SBC #$01
        STA <$7d
        LDA <$7e
        SBC #$00
        STA <$7e
        BRA f796

f7e1:

The RLE routine:

f817:   LDA [$78]
        STA <$83
        LDY #$04
        LDA #$09
        STA <$81
        STZ <$80
        CLX
        LDA #$20
        STA <$82
       
f828:   DEC <$81
        BNE f83a
        PHY
        LDA #$08
        STA <$81
        INC <$80
        LDY <$80
        LDA [$78], Y
        STA <$83
        PLY

f83a:   ROR <$83 ; the rle counter
        BCS f849
        STZ $3740, X
        INX
        DEC <$82
        BNE f828
        JMP f7f6

f849:   LDA [$78], Y
        STA $3740, X
        INY
        INX
        DEC <$82
        BNE f828
        JMP f7f6

The “XOR” routine:

f857:   PHX
        PHY
        LDY #$07
        CLX
       
f85c:   LDA $3740, X
        EOR $3742, X
        STA $3742, X
        LDA $3741, X
        EOR $3743, X
        STA $3743, X
        LDA $3750, X
        EOR $3752, X
        STA $3752, X
        LDA $3751, X
        EOR $3753, X
        STA $3753, X
       
        INX
        INX
       
        DEY
        BNE f85c
       
        PLY
        PLX
        RTS

The XOR is used here to maximize the number of repetitions in order to make the RLE more efficient. Well, that’s how I understood it. I didn’t make any benchmark to verify if it was really the case. The encoder was written pretty quickly. I reworked the font so that it more or less matches the ASCII set. This simplified the script encoder and the display routine.

Back to level names! I got help from a bunch of people to translate them. The last character of each level can be translated to country. Unfortunately that was far too big so I switched to “land”.

  • Candy Land
  • Toy Land
  • Greenery Land
  • Ice Land
  • Time Land
  • Water Land
  • Sky Land
  • Mirror Land
  • Queen Land

Here’s the actual “font” made of 16×16 sprites.
The zigzag lines on the last 16×16 sprite are there to make the compressed data fits into 504 bytes. I must also keep an empty sprite for spacing.

I had the sprite index list done pretty quickly. But when I started working on the index pointer table, I realized that there were 9 entries. 9? Did I forgot one?
Yes, I forgot one. The water land… And I didn’t have enough room to put the missing letters (bloody size limit).

After a cautious review, it was decided to rename “Greenery land” into “Green land”. As There were already a “Queen land”, there’s now enough room for the missing “Water”.

The only issue remaining was that the sprite index list is 42 bytes long, and mine was 43. As each level name ends with the same sequence I can use the last index as the beginning of the next one (each sequence starts with the length of the index list). So I swapped the sprite for “nd” with the beginning of the one whose index was equal to a sequence length, namely “To” which was at index 6. Why 6? First because it’s the length of the longest index list and also because sprite indexes are always even.

Here’s the original index list (the fist 2 hexadecimal values are the values of the pointer table):
8F95 : 05 00 02 04 06 08
8F9B : 05 0E 02 12 06 08
8FA1 : 05 18 0A 16 06 08
8FA7 : 05 14 00 16 06 08
8FAD : 05 1C 02 20 06 08
8FB3 : 04 18 1A 06 08
8FB8 : 04 22 24 06 08
8FBD : 05 02 10 18 06 08
8FC3 : 04 26 28 06 08

The new one:
8F95 : 05 00 06 04 24 06
8F9B : 04 02 04 24 06
8FA0 : 05 08 0A 0C 24 06
8FA6 : 04 0E 10 24 06
8FAB : 05 12 14 28 24 06
8FB1 : 05 20 22 26 24 06
8FB7 : 04 16 04 24 06
8FBB : 06 18 1A 1C 28 24 06
8FC2 : 05 1E 0A 0C 24 06

Shortly after beating the evil level names I received the first version of the translated script. Unfortunately the first tests revealed that the current text layout was unreadeable. There were only 2 or 3 words per line. So I had to crawl brack into the rom and change the text box position and size. I was only some couples of values to change.

So the 24th of July 2011, the english patch was released. Nearly 2 years after I started working on it… Anyway, some days after the release a couple of angry frenchmen complained that we didn’t release a french patch. In order to silence those annoying creeters I had to modify the font and the script encoder in order to add accents. That was not a big deal. But I let the french translators handle the level names (with some kind of sadistic grin). After some weeks of hard work, they decided to give up and use the english names. The french translation was released in December 2011.

Alice’s bubbles (Part 2)

Oh hy! I didn’t notice you were here. Please take a seat.
So I were talking about the font. As shown in the following image, the original font already contained uppercase letters.
Marchen Maze original font
As I wanted to add lowercases and some punctuations, I had to find where and how the font was stored in ROM. The visual editor told me that the font was stored from $7300 to $7a10 in VRAM. So under the code debugger I set an aux write breakpoint (ctrl+w) to these addresses. The breakpoint was triggered at $f7c6 on :

TIA $3740, $0002, $0020

If you look closely at the following image, you’ll notice that there are 2 similar preceding occurrences. This is it! You are looking at the piece of code in charge of decompressing the tile data into VRAM. Mednafen debugger view
This code supports 4 kind of compression:

  • The tile is filled with 0.
  • Uncompressed. The data is directly transfered to VRAM.
  • Simple RLE encoding.
  • Simple RLE encoding with some kind of postprocessing involving XORing consecutive lines.

Laziness told me to avoid writing a little tool to compress the font. So I checked how many uncompresserd char were used and if it was enough to store lowercases. Hopefully there was enough room for all of them. Marchen maze brutal lowercases insertion

At this point I had nearly everything I wanted. There were no need to modify the string display routine. Most of the work done would have been done by the script encoder as the lowercases characters where not contigous. It was not a big deal. Nevertheless I wanted to check if there will be enough room for the translated texts and how much characters could be displayed per screen. Basically you had 16 characters per line and 7 or 11 line depending on the screen. This gave me an idea of the maximal script size. Unfortunately it was too big. So I looked for a fast/small/simple text compression/decompression for 6502 based cpu. I had a quick look at the romhacking.net forum. And it seems that one of the most used technique is DTE (Dual Tile Encoding). It’s a pretty simple approach. A text does not use all the ascii values. Basically you only use upper and lowercases (52 chars), numbers (10) and a dozen punctuation marks. This leaves you with 182 unused values. You can use them to store couples of characters. Bytes with values between 0 and 74 represent a single character and all the others represent a tuple. For example 182 represents ‘ab’, 183 ‘en’ and so on. Here is quick example: Let’s compress the string “abcacbabcabcabba”. First we extract all the characters couples and count them:

  • ab : 4
  • bc : 3
  • ca : 3
  • ac : 1
  • cb : 1
  • ba : 2
  • bb : 1

“ab” is the tuple with the highest occurence. We insert it in the symbol table: a,b,c,d=’ab’. And let’s replace it in the original string:“dcacbdcdcdba”. DTE only works on the original symbols. So in the second pass we must avoid the newly inserted symbols. The remaining tuples are :

  • ca : 1
  • ac : 1
  • cb : 1
  • ba : 1

The new symbol table is : a,b,c,d=’ab’,e=’ca’,f=’cb’,g=’ba’. In the end “abcacbabcabcabba” becomes “defdcdcdg”. The tricky part is to choose the right substitution sequence that will lead to the smallest possible string. It’s a whole new game and you can stick with choosing the tuple with the highest occurence.
Let’s get back to Alice. I grabbed the infamous lorem ipsum and made a script out of it. I compressed it with DTE. And I still had a lot of unused values. Remember the second pass of our example when I excluded the newly inserted symbol for the occurence counting. If I included it, DTE would in fact have been what is called BPE (Binary Pair Encoding). Here’s the mandatory academical reference : Byte Pair Encoding: A Text Compression Scheme That Accelerates Pattern Matching (1999). Using BPE we would have had for “dcacbdcdcdba”.

  • dc : 3
  • ca : 1
  • ca : 1
  • ac : 1
  • cb : 1
  • bd : 1
  • cd : 2
  • db : 1
  • ba : 1

Let’s add “dc” in the symbol table a,b,c,d=’ab’,e=’dc’. We now have “eacbeedba”. BPE is basically a recursive algorithm. So when we are decoding a string, we must check if it’s part of the tuple symbols (let’s call them “special” symbols). If that’s the case we will have to loop until the decoded symbol does not contain any special symbol. If speed is a concern we may limit the depth of recursion. This means that when we must take the depth of each symbol into consideration. For example imagine we have 2 tuples “dc” and “ca”. Both have the same number of occurence in the string. We may choose “ca” instead of “dc” because it only contains “base” symbols.
Using BPE the “maximal” script was small enough to fit. I made a silly test using some part of Hamlet (here’s the script). You can find the BPE compression/decompression as long as code for generating IPS files in this [archive].

Next : Levels names. Shitty part : they are sprites. Describe pointer tables and all.
All compressed. So I can’t avoid it anymore. Describe the compression scheme.
Got help for level name translation. (show the level names and their translation).
As I have a compressor, reworked the font so that it more or less match ascii. Had to spot where the font was rendered to vram. (link to final font). Simplify script encoder and display routine.

Coming next: Level names. A world of pain and horror!

Alice’s bubbles (Part 1)

Marchen Maze title screen
Or more humbly, the awesome Märchen Maze translation postmortem. It’s more or less a compilation/addendum for this pcedev forum [post].

Märchen Maze is a PCEngine HuCard game released by Namco the 11th of December 1990. It’s a conversion of the 1988 arcade game. It was also ported to the almighty Sharp X68000. As you may have guessed the X68000 version is a near perfect conversion of the original arcade game. It features some jolly 3D isometric view and colourfull introduction. Well, things are not that fancy on the PCEngine version. The 3D isometric view was changed to a top view (à la Mercs or Commando). And the intro is now in sepia tones. What about the story and gameplay? Check the Video Game Den review [here].

Ok! Back to business.
The PCEngine has some neat RPGs. Unfortunately only a few of them were translated into the idiom of the perfidious Albion (English). This was a perfect excuse for trying out that black art called romhacking. Monster Maker was my first choice. I must admin that cdrom games are not a really good choice for your first romhacking experience. A HuCard game is far more simpler to hack. So I went for Momotarō Katsugeki. It’s a cool platform game with tons of text. Unfortunately the text is displayed top to bottom and from left to right in bubbles. It was clearly out of my league. So I asked some people if they knew some easily translatable games. I can’t remember who suggested it. But Märchen Maze was designated as it doesn’t have too much text and no apparent string compression.

The first thing I had to do was to find where the text was stored in the ROM. After hours of pain, David Shadoff took pity. And like a mighty wizard, he managed to extract the whole script in no time. So I had the rom location of the strings. All I had to do was to put read breakpoint in Mednafen. I was able to locate the text output routine ($87D6). The string “parsing” was spotted from $89E5 to $8CD8 using step-by-step debugging starting from the the output routine. It made me realized that $5C,$5D held the pointer to the string to be displayed. The parsing routine was called from $E377 using jmp ($202C) instead of jsr. The return address was pushed by hand onto the stack. Here’s an snippet of the parsing routine. $ff is a special value. More on this later.

l_89e2: LDY     $271b
        LDA     [$5c], Y
        CMP     #$ff
        BNE     l_89ee

The most important information was that I now knew the zero page location holding the string pointer. Setting a write breakpoint to $5C:$5D led me to $8C4B.

        LDA     [$52], Y
        STA     <$5c
        INY     
        LDA     [$52], Y
        STA     <$5d

I was lucky because what precedes told me where the string table was.

        LDA     #$00
        STA     <$64
        LDA     #$56
        STA     <$65
        LDY     #$0c
        LDA     [$64], Y
        STA     <$52
        INY     
        LDA     [$64], Y
        STA     <$53
        LDA     $2716
        ASL     A
        TAY     
        LDA     [$52], Y
        STA     <$5c
        INY     
        LDA     [$52], Y
        STA     <$5d

So $5c:$5d is set using the data pointed by $52:$53 indexed by $2716. Let’s put $2716 aside. This is clearly the string index. So $52:$53 contains our string pointer. And the preceding lines tells us that the string table address is stored at $560C. A quick revealed that the string table was stored from $5614 to $5849. If we translate the address, it’s at the file offsets $11814 to $11A49 (this is with the annoying 512 bytes offset). Now that I had the string pointer table, I could make it point to whenever I want (technically). So the translated texts may not have to fit into the original string locations. I checked that I got it right by swapping the pointer table entries. And as expected the intro started with the second text and so on.
A quick look at the tile vram showed me that the font already contains capital letters. So I decided to manually edit the ROM and managed to get this:
Quick hack

The next step was to modify the font in order to have lowercase letters.
To continue…

Alice in GitHub

Some of the stuffs I posted here are now on [GitHub]. Namely, the [arduino dataflash library] and the [ips patcher]. Feel free to clone them :)
I’m taking a little break from OpenGL stuffs and the pcengine tracker. To relax I’m working on Marchen Maze translation. I’m nearly done with the introduction. I’m currently working on the insertion software. But I need to find someone to translate the script… Anyway you can check my progress on [pcedev].

And for more useless stuffs you can check my [twitter page].

Fireballs of fire!

EEK! 6 months !

It’s not that I had nothing to write. On the contrary, I have a lot to post. I just didn’t found the time to write… Erm.. Let’s go with a 7 months old stuff.

While working on the tracker, I wanted to add some kind of “live” mode. The user could choose which sequence to play, modify it, control volume, add an effect etc… So I came with the idea to associate a command to a key combo.

Combos are detected by walking down a tree. Each time a button is pressed, we look at the current node and check if one of its children is associated to this button. If that’s the case, this child becomes the new current node. Otherwise there’s no matching combo and we reset the current node to the tree root. If the button pressed didn’t not changed for a certain amount of time (timeout), we check there’s a child corresponding to the current button. This way, we can detect combo containing a sequence like “press A button during t seconds”.

Let’s look at a simple example.

Mkit will successively return : DOWNDOWN+RIGHTRIGHT0 (aka nothing) – A

We have 0 because when you go from RIGHT to A, there’s a moment when you don’t press any button. And as we check keys every vsync, we necessary get 0 (unless if you are Superman on Redbull).

Consider we want to handle the following combos :

The combo tree is very similar to a lexical tree. A node can have 0 to N children but only one parent.

The tree is store in level-order with the longest path first. This means that you’ll have to be very careful and store node children by their depth. We have for our example :

Root – Down – A – Down+Right – Down+Left – Nothing – Left – Right – A – Nothing – Nothing – Leaf – A – B – Leaf – Leaf

In the current implement, the tree is stored in 2 arrays. The first one contains the index to first child of a node. And the second the key for each node.

combo_tree : 1, 3, 5, 6, 7, 8, 9, 10, 128 | 2, 11, 12, 128 | 0, 128 | 1
combo_keys: .db 255, JOY_DOWN, JOY_I, JOY_DOWN | JOY_RIGHT, JOY_DOWN | JOY_LEFT
.db 0, JOY_RIGHT, JOY_LEFT, JOY_I, 0, 0, JOY_I, JOY_II

The root is always at index 0 and we associate the key value 255 to it.
The tree traversal algorithm is very simple. For a given node i, we know that its children are at indices j from combo_tree[i] to combo_tree[i+1]-1. And that we can reach them only if the current key is combo_key[j]. If the bit 7 of a value in the children list is set, this means that we have a transition a leaf. The remaining bits are used as an index to a jump table.
The traversal looks like this:

 i // current node index
for(j=combo_tree[i]; j<combo_tree[i+1]; ++j)
    if(combo_key[j] == key)
        if(combo_tree[j] & 128) // is it a leaf?
            fnctl = combo_tree[j] & 127
            call(fnctl)
            i = 0 // Back to root
        else
            i = j // The child becomes the new current node
        return
i = 0 // We didn't find a suitable child. So we jump back to root.

The only thing missing here, is the key timeout and repetition. The key timeout is the number of vsync to wait before going back to root (COMBO_DEFAULT_DELAY in combo.asm). Note that in current code combo_tree, combo_keys and combo_jumpTable are in RAM. So you’ll have to initialize them before-hand.

If you have any question, feel free to leave a comment or go and join the [PCEdev Forum].

Talking about web stuff, I finally updated the gallery. Hurray!