Very important, this case study covers translating krikiriZ txt.scn (e-mote) files and bytecoded tjs2 files as part 2 of working with the kirikiriZ game engine. Part 1 is the Koakuma-chan no Yuuwaku case study.
Part 1, Koakuma-chan no Yuuwaku, establishes a core understanding the kirikiriZ game engine which this post builds on. This part 2 assumes all content in part 1 has been read and fully understood, including the parts about cli usage and automation.
Yet another disclaimer, The intent of these guides is to show people the mindset and real-world approaches I use when tackling a new proof of concept for a translation project. Ideally, these guides should empower you to figure things out the various issues related to translation yourself by solving the most common ones in a step-by-step way for a particular title. These guides do not necessarily outline an optimized workflow for translating a particular title or titles from a particular game engine because they are meant to show the process, not just the final answer.
This case study covers kirikiriZ title 紬の花嫁, Tsumugi no Hanayome, which could be translated as The Bride of Tsumugi. This is the story of how the english MTL patch was created.
It looks like the game's story involves yuri + futanari + romance, and some supernatural elements. Right...
It was released in late 2024, so it is a relatively recently released game and an english mtl patch for it was published two weeks after the game's release. That is a much better turn around time than the years of waiting it used to take for translations to happen prior to having well developed translation tools a decade ago. With AI MTL, the MTL is half readable as well and any human translator or editor that wants to pick this project up can do so knowing they have a fairly good template to work from.
To keep everything organized, I am using a base folder on the desktop named "Desktop\Tsumugi no Hanayome" and putting everything related to this project underneath it in more subfolders: bin\, extracts\, tools\, MTL\. Most paths in this case study are relative to that one since it is serving as the main project directory.
Code:
Tsumugi no Hanayome/
Tsumugi no Hanayome/bin/
Tsumugi no Hanayome/extracts/
Tsumugi no Hanayome/tools/
Tsumugi no Hanayome/MTL/
Tsumugi no Hanayome/other/
For me, the main executable for this game is under "Desktop\Tsumugi no Hanayome\bin\Tsumugi no Hanayome\Tsumugi.exe". The folder can moved to other places, and the game should still launch, at least for the repackage linked above.
Here are the raw installed files under 紬の花嫁/ for the installed version.
The .sig files are just text files that can optionally be used to check the file integrity of certain files using RSA public-private key crypography. The contents look similar to this.
Interestingly, there is an external file corruption tool, ファイル破損チェックツール.exe, "File Corruption Checking Tool", that can read and verify the hashes of the files using signature data in .sig. Here is what running the tool looks like after replacing Tsumugi.exe with the no-DVD patch.
- The .sig files in the main directory are not needed for the game to launch, so they were moved to Extras/backup/. However, the plugins/*.sig files are required or the game will not load the associated .dll files. Then, after refusing to load them, it will complain the classes that were supposed to be loaded from the .dll files do not exist because it never loaded the .dll files they were in.
- Tsumugi.cf is an optional configuration file that tells the game where to keep savedata. If this file is not present in the root directory of the game, then the game will use a folder called "savedata" at the root of the game's directory instead of somewhere else under the User/ folder.
- BootStrap.exe launches an small menu before the game launches with some options. It is not necessary and having multiple .exe files in the game's directory can be confusing, so it was moved to Extras/backup/. Move it back to the main directory to restore its functionality. It was not translated in the published patch.
Here is also what the repackage should look like with the translation patch applied.
First, launch Tsumugi.exe using a Japanese, Japan locale emulator to make sure the game actually works. Then take a screenshot or otherwise record the first dialogue line of the game. For the initial proof of concept, we need to translate that one line.
Next, we need to verify the game engine. This is a kirikiriZ title, but how do we know that? In addition to the heuristic of .xp3 files in the root of the directory, here is the Garbro ouput for the tested working noDVD executable Tsumugi no Hanayome\Tsumugi.exe\RT VERSION\00001. The original executable can't be opened as a resource archive in Garbro.
Code:
BLOCK "041104b0"
{
VALUE "FileDescription", "紬の花嫁"
VALUE "FileVersion", "1.2.0.3"
VALUE "InternalName", "tvp2/win32"
VALUE "LegalCopyright", "(KIRIKIRI core) (C) W.Dee and contributors All Rights Reserved. This software is based in part on the work of Independent JPEG Group. For details: Run this program with '-about' option."
VALUE "OriginalFilename", "tvpwin32.exe"
VALUE "ProductName", "TVP(KIRIKIRI) Z core / Scripting Platform for Win32"
VALUE "ProductVersion", "1.2.0.3"
VALUE "Comments", "Hending"
}
The Garbro output at the bottom left indicates the archive type when opening the .xp3. That information means the file is actually a valid kirikiri archive (.xp3) instead of some other archive type re-named to have a .xp3 extension.
All together, that makes it very likely this is a kirikiri game, specifically kirikiriZ.
In order to translate the first line of dialogue, we first need to extract the file that line is located in.
Unfortunately, the file names and contents in the .xp3 are obfuscated, likely with some form of encryption instead of just earlier generation obfuscation. The encrypted contents mean we need to either specify a scheme in Garbro to extract the assets or find a different way to extract the contents.
The file names are part of the metadata, information describing the contents, rather than the contents of the files themselves. Encrypted metadata, not knowing the original structure of the archives and file names, is annoying since it interferes with creating that archive structure again for the patch to keep everything organized, but ultimately, it does not affect much more than that.
We could search through Garbro's list for a working algorithm, but if the developer is using RSA PKI on a modern kirikiriZ title, then the odds of that working is pretty low. Nothing worked in that list worked for Koamuma-chan, remember? Instead, let's just use crskycode's KrkrDump again. This tool for extracting assets is very likely to work for Tsumugi since it is known to work with a different game that operates on the same kirikiriZ game engine, Koakuma-chan no Yuuwaku.
Here is the KrkrDump.json for Tsumugi. For the path separator in the "outputDirectory" entry, either use double \\ notation as in the original example or a single /. Both work, but a single \ will likely not.
Let's play the game while using KrkrDumpLoader.exe and extract all of the files. Multiple playthroughs might be required to select every choice combination.
There is usually an option in the Config screen to skip unread lines which can be used to make this take less time. This is usually a toggle, so keep toggling options on and off until it is found. Tsumugi no Hanayome has another option somewhere in the Config menus to make sure the game keeps going when in the background instead of pausing. Let's enable that too.
With the files fully extracted, now we need to organize them in order to begin to make sense of them and eventually find the dialogue scripts. Thankfully, the original file names including the extensions have been restored, but we can't use the original structure for guidance since the developer encrypted the file and folder name metadata in the .xp3 files. The alternative way of doing this is by file type (extensions) and the fastest way to do that is to use command prompt wildcards.
- Open "Tsumugi no Hanayome/extracts/Tsumugi_extracts" in Windows Explorer.
- Enter "cmd" in the address bar.
- Press enter.
With a comand prompt, organize the files into subfolders. Here is the structure I am using for Tsumugi.
The images folder is an absolute mess. CGs, backgrounds, character sprites, and gui overlays are all mixed together. It needs a couple more subfolders before some semblance of organization takes root. We can deal with that later though. For now, we need to focus on translating the first line of the game.
Browsing the extracted .ks files, those are all very small system scripts that define macros and such. Character dialogue lines are not there and neither is the game's core logic.
After some searching and browsing headers with notepad++, the game's scripts are likely in the .txt.scn files since they are valid e-mote files and start with the e-mote header "PSB".
描いたら、動かそう。 キャラクターアニメーションツールE-mote
Once you've drawn it, move it. Character anime tool E-mote
「E-mote」はキャラクターアニメーションを制作する際の『パーツ配置』『ボーン設定』『ボーン割り当て』などの煩雑なセットアップ工程を完全カットし、低い工数でイラストを動かすことを実現した技術です。
また制作されたデータは、様々なプラットフォームでリアルタイムで動かすことができ、動画データよりも圧倒的に少ないデータ量で、フルアニメーションを再生することが可能です。
"E-mote" is a technology that completely cuts out the complicated setup process such as "parts placement", "bone setting", and "bone assignment" when creating character anime, and realizes moving illustrations with low man-hours.
In addition, the produced data can be moved in real time on various platforms, and it is possible to play back full anime with an overwhelmingly smaller amount of data than video data.
ハイクオリティアニメーション
High quality anime
「E-mote」では『工数の削減』と『繊細な完成度の高い動き』を両立しました。
演奏にあわせてコードを押さえる指の動きや、ラーメンを箸でつまんで口に運ぶ動きなど、これまでに様々なハイクオリティアニメーションが「E-mote」を用いて作製されています。
"E-mote" achieves both "reduction of man-hours" and "delicate and highly complete movements". Various high-quality anime productions have been created using "E-mote", such as the movement of fingers holding down chords according to the performance, and the movement of picking up ramen with chopsticks and bringing it to the mouth.
「E-mote」のコスト
The cost of "E-mote"
"E-mote" provides "SDK for 1 platform + 1 editor" for medium and large companies for the amount of "300,000 yen for 1 year" or "300,000 yen for 2 years for 1 title only", and supports support during the license period "free of charge". If you are using it for personal use, you can use "Emofuri", which has almost the same function as "E-mote", for free. The new "Emofuri" has been removed from the limit on the number of frames in the video, and can be freely used in apps and games in combination with the Unity SDK and Tyranno Script.
In other words, the emphasis of the E-mote format is animation, not dialogue, and it has some cross platform features. KirikiriZ's fileformat documentation also curiously omits mentioning it.
The E-mote SDK does cost about $3000, but that depends on the current exchange ¥ vs $ exchange rate, which means low budget games probably would be better off using kirikiriZ's k2compat system and .ks files for the dialogue. That probably explains why Koakuma-chan used .ks files for dialogue even though those files are traditionally limited to shift-jis and utf-16-le-bom encoding instead of supporting utf-8 natively like the e-mote format. As someone trying to translate stuff, utf-8 support is important and better than utf-16-le-bom, but is it important to the tune of $3000? IDK honestly. The game developer has to decide that. There are also some modern KirikiriZ games that support utf-8 encoded .ks files, so it is possible at least theoretically.
That cost of the E-mote SDK also makes me grateful that UlyssesWu's FreeMote project exists which allows us to work with E-mote files in a comprehensive way without the E-mote SDK. The documentation being in easy to understand english is a big plus.
There are a lot of alternatives to UlyssesWu's FreeMote for working with E-mote/PSB files, however, at least for translating kirikiriZ .txt.scn files, no FreeMote alternatives are worth using because FreeMote provides comprehensive unpacking and packing support.
The alternative to FreeMote are tools that use regex to blindly extract strings which makes using them likey to cause crashes since those other tools do not try to figure out what is translatable and what is not. If FreeMote did not exist, those other approaches could be useful, but FreeMote exists, so please avoid using anything else when translating .txt.scn files, even if it means taking the time to learn cli's and how to parse json with python. There are some FreeMote json parsing examples in my codeberg repository under translation projects.
If FreeMote is too difficult to use, then spend some time learning how to use a cli, read the FreeMote documentation, and learn python. It is simpler than it sounds. Instead of using other tools, that will help more in the long run, especially when it comes to automation which can dramatically reduce the amount of work it takes to translate a game down to managable levels so even a single person can fully publish a translation of a game within two weeks of its release, excluding the actual translation step of course.
First, let's download the FreeMote Toolkit from the releases page, and then read up on the documentation linked above. In my case, I extracted FreeMote Toolkit 3.x to Tsumugi no Hanayome/tools/FreeMoteToolkit/, but 4.x probably also works. Here is a snippet of the FreeMote Toolkit wiki.
Guide for rookies
- Make sure you have .NET Framework 4.8 installed.
- Drag and drop a xxx.psb/xxx.scn/xxx.pimg file to PsbDecompile.exe to get json files and images.
- Drag and drop a xxx.json file (you got from PsbDecompile.exe) to PsBuild.exe to get a PSB file.
- Drag and drop a dx_xxx.psb file (you got from some galgames) to FreeMoteViewer.exe to view it.
- For other operations, you must learn how to use CMD or Windows Terminal first. Then read commandline usages.
- Keep in mind that there is no "double-click" or keyboard input involved in a "drag and drop" operation.
Type <tool_name.exe> -h command in CMD or Terminal for tools' instructions. For example, if you don't know how to use PsBuild, just type PsBuild -h and press Enter; If you don't understand PsBuild "info-psb" command (what's that or how to use that), type PsBuild info-psb -h and press Enter.
Here is some information on the Microsoft .Net framework. It is already included in most newer versions of Windows 10 and newer, including Windows 11, but Windows 10 <=1809 will need to install it.
Code:
C:\Users\User>"C:\Users\User\Desktop\Tsumugi no Hanayome\tools\FreeMoteToolkit\PsbDecompile.exe" --help
FreeMote PSB Decompiler
by Ulysses, [email protected]
18 Plugins Loaded.
Usage: [command] [options] <Files>
Arguments:
Files File paths
Options:
-?|-h|--help Show help information
-s|--seed <SEED> Set MT19937 MDF seed (Key+FileName)
-l|--length <LEN> Set MT19937 MDF key length. Default=131
-k|--key Set PSB key (uint, dec)
-raw|--raw Output raw resources
-oom|--memory-limit Disable In-Memory Loading
-1by1|--enumerate Disable parallel processing (can be very slow)
-hex|--json-hex (Json) Use hex numbers
-indent|--json-array-indent (Json) Indent arrays
-dfa|--disable-flatten-array Disable represent extra resource as flatten
arrays
-e|--encoding <ENCODING> Set encoding (e.g. SHIFT-JIS). Default=UTF-8
-t|--type <TYPE> Set PSB type manually
Allowed values are: PSB, Pimg, Scn, Mmo,
Tachie, ArchiveInfo, BmpFont, Motion,
SoundArchive, Map
-dci|--disable-combined-image Output chunk images (pieces) for image (Tachie)
PSB (try this if you have problem on image type
PSB)
Commands:
image Extract (combined) images from image (Tachie)
type PSBs (with "imageList")
info-psb Extract files from info.psb.m & body.bin
(FreeMote.Plugins required)
unlink Unlink textures from PSBs
Run ' [command] -?|-h|--help' for more information about a command.
Plugins:
FreeMote.Lz4 by Ulysses : LZ4 support.
FreeMote.Mdf by Ulysses : MDF (ZLIB) support.
FreeMote.Mxb by Ulysses : MXB (XMemCompress) support.
FreeMote.Mzs by Ulysses : MZS (ZStandard) support.
FreeMote.Psd by Ulysses : PSD export.
FreeMote.Psp by Ulysses & morkt : PSP (LZSS) unpack support.
FreeMote.Psz by Ulysses : PSZ (ZLIB) support.
FreeMote.Mfl by Ulysses : MFL (FastLZ) support.
FreeMote.Astc by Ulysses : ASTC support.
FreeMote.Bc7 by Ulysses : BC7 support via BCnEncoder.NET.
FreeMote.Tlg by Ulysses : TLG support via TlgLib.
FreeMote.Wav by Ulysses : Wav/P16 support.
FreeMote.Audio.File by Ulysses : Export/Import audio file.
FreeMote.At9 by Ulysses : At9 support via VGAudio.
FreeMote.NxAdpcm by Ulysses : NX ADPCM support via VGAudio.
FreeMote.NxOpus by Ulysses : NX Opus support via VGAudio.
FreeMote.Vag by Ulysses : VAG support.
FreeMote.Xwma by Ulysses : XWMA support.
Examples:
PsbDecompile -k 123456789 sample.psb
Done.
So, the core syntax to make something happen is just this.
Code:
PsbDecompile.exe script.psb
The .psb extension in the above example and extensions in general are inconsequential. While operating systems can use them to associate which file should be opened with which program, the file contents are what matters to actually render them properly.
About PSB
FreeMote is a set of tool/libs for M2 Packaged Struct Binary file format. The file header usually starts with PSB/PSZ/mdf, and the file extensions usually are .psb|.psz|.mdf|.pimg|.scn|.mmo|.emtbytes|.mtn|.dpak|.psb.m
E-mote files, at least in Tsumugi, begin with a "PSB" header, and are renamed to .scn or .pimg but can be various other extensions as well for different games. In our case, the extension used by Tsumugi is .scn. There are also a secondary .txt extensions before the final .scn extension to improve the clarity on the type of E-mote file when looking at the file name, meaning that they are intended to hold text.
E-mote files can also have an optional pin number that weakly encrypts their contents. Since using a pin number is always a terrible idea for security, if present, these pins can be determined using brute force. There is this old tool, EMotePrivateKey, by xmoezzz that can probably do it, but this is not necessary for Tsumugi since the .txt.scn are in plaintext E-mote format already.
I have not personally encountered any KirikiriZ games that use encrypted e-mote, .txt.scn, dialogue files. If you have, please leave a comment letting me know of a title that does so I can work out how to brute force them into a series of more tangible steps. Or if you already know the steps to do brute force them, please post or link to them.
Let's make sure PsbDecompile.exe and repacking any changes actually works since that is enough to finish the proof of concept. Then we can automate all of it.
The files in Tsumugi no Hanayome\extracts\Tsumugi_extracts\scripts are actually numbered sequentually, so I am assuming "紬の花嫁01 プロローグ.txt.scn" is the first one.
What if we did not want to assume that file is the first one?
If we go back to the KrkrDump.log, it lists files in the order the game reads them, so it is simple browse the log for the very first .txt.scn file that appears using ctrl+f. That will likely display the .txt.scn file that was extracted first when the first line of dialogue appeared in the game.
Code:
C:\Users\User>"C:\Users\User\Desktop\Tsumugi no Hanayome\tools\FreeMoteToolkit\P
sbDecompile.exe" "C:\Users\User\Desktop\Tsumugi no Hanayome\extracts\Tsumugi_ext
racts\scripts\紬の花嫁01 プロローグ.txt.scn"
FreeMote PSB Decompiler
by Ulysses, [email protected]
18 Plugins Loaded.
Decompiling: 紬の花嫁01 プロローグ.txt
Done.
C:\Users\User>
It says it decompiled something, but it did not say where it wrote the output. There is nothing in the C:\Users\User folder. Over at Tsumugi no Hanayome\extracts\Tsumugi_extracts\scripts there are now two new files called "紬の花嫁01 プロローグ.txt.json" and "紬の花嫁01 プロローグ.txt.resx.json". In other words, the PsbDecompile.exe tool creates its output relative to the target instead of the current directory of the command prompt.
The non-resx .json is several megabytes in size, but the .resx.json file is just 156 bytes. Here are the contents of the .resx.json file.
It looks like the .resx.json contains metadata about the main .json file. The Psb file it decompiled was a "Scn" PsbType which maybe means "scenario" or "scene", without any platform, no encryption, and was version 3. That will likely be useful to know when packing it again.
Opening up "紬の花嫁01 プロローグ.txt.json", the chaos begins. PsbDecompile.exe converts the compressed .scn.txt to uncompressed .json. JavaScript Object Notation (.json) is a normal way to represent complex data when it needs to be done in a language and program independent way. There are other ways to do this, but PsbDecompile prefers .json evidently.
Does this script.json have the line we are looking for? Instead of finding it manually, it would be nice to be able to search for it with ctrl+f just like we searched through the KrkrDump.log file. Let's try running ocr on the first line of the game to get the line we are searching for.
- Go back to first_line.png, which is the screenshot of the first line taken earlier.
- Cut the first line into first_line_snip.png
In my case, I did ocr.py since that should already be set up from the Koakuma-chan case study.
Code:
cd C:\Users\User\Desktop\Tsumugi no Hanayome\tools\translation_tools
python ocr.py "C:\Users\User\Desktop\Tsumugi no Hanayome\MTL\ocr\first_line_snip.png" -o
or if using ocr_text_recognition.py part of codeberg/ocr_tools, that should be
Code:
cd C:\Users\User\Desktop\Tsumugi no Hanayome\tools\ocr_tools
python ocr_text_recognition.py "C:\Users\User\Desktop\Tsumugi no Hanayome\MTL\ocr\first_line_snip.png" -o
That created "Tsumugi no Hanayome\MTL\ocr\first_line_snippet.png.ocr.txt" with the following text 人は、罪を犯しながら生きている。 Searching for it in the extracted.json leads to a single hit here.
We will have to deal with that mess eventually, but for now, let's just ignore absolutely everything else and only translate that one string. Once changed, here is what the line becomes.
Code:
"texts": [[null,[[null,"People live their lives while committing sins.",16]],null,208,{
Let's save the modified .json to "Tsumugi no Hanayome/MTL/dialogue_scn_json_translated/紬の花嫁01 プロローグ.txt.json". Never overwrite the original. Now, how do we pack it up so the game can read that string? We've done
data.xp3 -> file.txt.scn -> file.txt.json
To pack it up, we need to do
file.txt.json -> file.txt.scn -> patch.xp3
For the file.txt.json -> file.txt.scn step, we can use FreeMote again. Tsumugi no Hanayome\tools\FreeMoteToolkit has a couple different executables. EmtConvert.exe, EmtMake.exe, FreeMoteViewer.exe, PsbDecompile.exe, PsBuild.exe. EmtConvert.exe and PsBuild.exe seem promising but let's read the wiki instead.
While organizing the extracted files earlier, there was this interesting obviously an image file called hending_brandlogo.pimg that was not properly categorized when moving all the .tlg and .png files into the Tsumugi no Hanayome/extracts/Tsumugi_extracts/images/ subfolder. However, it does not open up in any image viewing program. Why not?
Curious, I opened it in notepad++, and it does have a PSB header. So, it is some sort of renamed .psb file? Well, no image viewing program would work then since it needs to be rendered in a special way as a .psb file. Then, while looking through the CommandLine-Usage-for-Tools wiki entry just now, it said this.
How interesting. Up until now, there were only dialogue "Scn" type psb files to play around with, but maybe the above syntax would work?
Code:
C:\Users\User>"C:\Users\User\Desktop\Tsumugi no Hanayome\tools\FreeMoteToolkit\P
sbDecompile.exe" image "C:\Users\User\Desktop\Tsumugi no Hanayome\extracts\Tsumu
gi_extracts\hending_brandlogo.pimg"
FreeMote PSB Decompiler
by Ulysses, [email protected]
18 Plugins Loaded.
Done.
That created a file called extracts/Tsumugi_extracts/hending_brandlogo.resx.json, meaning the tool has the same behavior as when it works with the dialogue "Scn" type psb files.
So the PsbType is "Pimg". How interesting. That explains the otherwise seemingly odd choice of extension, .pimg, for the psb file. It also created a folder called "hending_brandlogo". Inside that folder it had a few recognizable images.
When Tsumugi no Hanayome/Tsumugi.exe launches, the first thing that gets displayed is Hending's logo which has a cute animation. These files are the likely the ones that get displayed and animated by logic found elsewhere. Since this is a logo, it is better not to mess with it, but now we know how to mess with .pimg files in general which may or may not be helpful in the future.
So EmtConvert is for encryption and decryption. That is not what we want. PsBuild says it "Convert description jsons and resources to PSB." Does it work for .json files that are not descriptions and do not have external resources? Let's find out!
Code:
C:\Users\User>"C:\Users\User\Desktop\Tsumugi no Hanayome\tools\FreeMoteToolkit\P
sBuild.exe" --help
FreeMote PSB Compiler
by Ulysses, [email protected]
18 Plugins Loaded.
Usage: [command] [options] <Files>
Arguments:
Files File paths
Options:
-?|-h|--help Show help information
-v|--ver <VER> Set PSB version [2,4]. Default=3
-k|--key <KEY> Set PSB key (uint, dec)
-p|--spec <SPEC> Set PSB platform (krkr/common/win/ems)
Allowed values are: none, common, krkr, win, ems,
psp, vita, ps3, ps4, nx, citra, and, x360, other
-nr|--no-rename Prevent output file renaming, may overwrite your
original PSB files
-ns|--no-shell Prevent shell packing (compression)
-double|--json-double (Json) Use double numbers only (no float)
-e|--encoding <ENCODING> Set encoding (e.g. SHIFT-JIS). Default=UTF-8
Commands:
info-psb Pack files to info.psb.m & body.bin
(FreeMote.Plugins required).
link Link textures into an external texture PSB
port Re-compile a PSB to another platform
replace In-place Replace the images in PSB
Run ' [command] -?|-h|--help' for more information about a command.
Plugins:
FreeMote.Lz4 by Ulysses : LZ4 support.
FreeMote.Mdf by Ulysses : MDF (ZLIB) support.
FreeMote.Mxb by Ulysses : MXB (XMemCompress) support.
FreeMote.Mzs by Ulysses : MZS (ZStandard) support.
FreeMote.Psd by Ulysses : PSD export.
FreeMote.Psp by Ulysses & morkt : PSP (LZSS) unpack support.
FreeMote.Psz by Ulysses : PSZ (ZLIB) support.
FreeMote.Mfl by Ulysses : MFL (FastLZ) support.
FreeMote.Astc by Ulysses : ASTC support.
FreeMote.Bc7 by Ulysses : BC7 support via BCnEncoder.NET.
FreeMote.Tlg by Ulysses : TLG support via TlgLib.
FreeMote.Wav by Ulysses : Wav/P16 support.
FreeMote.Audio.File by Ulysses : Export/Import audio file.
FreeMote.At9 by Ulysses : At9 support via VGAudio.
FreeMote.NxAdpcm by Ulysses : NX ADPCM support via VGAudio.
FreeMote.NxOpus by Ulysses : NX Opus support via VGAudio.
FreeMote.Vag by Ulysses : VAG support.
FreeMote.Xwma by Ulysses : XWMA support.
Examples:
PsBuild -v 4 -k 123456789 -p krkr sample.psb.json
Done.
Which means the syntax is essentially this.
Code:
PsBuild.exe --ver 3 --encoding utf-8 file.psb
Version 3 and utf-8 are default values, but it is always better to be explicit about it since that makes it clearer what is happening, especially in case there is a bug later or a software update changes the defaults or interface.
It says it compiled something, but there is nothing at Tsumugi no Hanayome/MTL/dialogue_scn_json_translated/ besides the source .json file. However, C:\Users\User has a new file now called "紬の花嫁01 プロローグ.txt.psb". PsbDecompile.exe outputs relative to the target directory, but PsBuild.exe outputs to the current location of the command prompt.
Despite being part of the same toolkit, these tools use inconsistent logic. Consistency would be nice, but the only thing that really matters is whether it works or not. As long as it does, we can workaround the inconsistencies as long as we are aware of them.
The next step is to make the game read this file. Since the game is expecting a file named "紬の花嫁01 プロローグ.txt.scn", let's rename the file by swapping the .psb extension to .scn.
Older kirikiri games, usually kirikiri2, do not necessarily need any special handling for the created .xp3 archives. However, newer kirikiriZ games tend to require assistance to read from unencrypted .xp3 archives, although this of course depends on the developer.
First, let's try using Garbro without any special handling, then arcusmaximus's KiriKiriTools since that worked before, then other programs.
What should the archive.xp3 created with Garbro be named? patch.xp3, patch2.xp3 and so forth are the most common names for patch archives in kirikiri since that is the default naming scheme the engine uses, but developers are able to customize it to whatever they want.
Having patch.xp3 in the game's directory does not seem to change anything. The first line is not translated, and the game still runs normally. If the game's displayed content was altered in some way or if it gave an error, then we would have a sign that it was attempting to read the archive's contents, but we are not running into that sort of issue. Since the game's behavior does not change at all, it seems the file is being completely ignored.
Using KiriKiriTools v1.7, Xp3Pack.exe to create the .xp3 from a folder and version.dll placed in the game's main folder, results in the same behavior. The archive is simply ignored, so the translated "紬の花嫁01 プロローグ.txt.scn" file never loads. Why not?
If patch.xp3 does not work, even with KiriKiriTools, then what names for archives is Tsumugi expecting? If "patch.xp3" does not work, is the game expecting the patch to be called something else? That information is found in the Initialize.tjs file extracted earlier.
Let's open extracts/Tsugumi_extracts/system/initialize.tjs in notepad++.
That is not normal tjs2 script. It seems to have been altered in some way. Let's refer back to the kirikiri engine documentation.
Byte-code
Since it is easy to analyze the TJS2 script when it is stored as it is, the script is bytecoded.
Bytecoding is possible with Scripts.compileStorage.
Even if the archive is encrypted, it can still be breached, so it makes sense to make it difficult to parse the script.
Also, bytecode can be loaded faster than scripts (since the scripts are pre-compiled).
Tjs2 scripts are scripts meaning they need to be fed into an interpreter before being compiled into native machine code. Once in machine code, the code can be executed by a processor just like other compiled code.
The Initialize.tjs script for Tsumugi is in .tjs2 bytecode. Some implementations of languages use bytecode as an intermediary between the "human readable" source code and machine code before eventually turning that bytecode into machine code at runtime. Think of it like having to be compiled twice.
CPython also does this transparently (with .pyc files) to speed up interpretation during runtime for subsequent executions of the same code. This means that creating bytecode is a normal part of running interpreted scripts for optimized implementations of certain programming languages.
For translation, the important part is that tjs2 in bytecode format cannot get rid of the original strings that we need to translate because it still needs the original strings in order to display them eventually at runtime, like the game's title and other game engine strings. All of that together makes it very unlikely there is any encryption going on with the tjs2 scripts files.
Therefore, it should still be possible to extract and translate them by deciphering the tjs2 bytecode intermediary language. We just need to create or find a program that can work with tjs2 bytecode.
Since we would like to understand what archive name initialize.tjs is expecting, the most direct way to find that out is with a tjs2 disassembler or decompiler that can convert the tjs2 bytecode into something more readable, or at least searchable with notepad++. The only one I am aware of is Furikiri described as "Managed (& the world's first) TJS2 Decompiler 'FreeKiri' (In Dev)".
A disassembler takes the assembled code or bytecode and adds semantics to help humans understand what the code literally does. Disassembled code is theoretically alterable and convertable back into assembler or bytecode using a special program that understands the semantics added by the disassembler.
A decompiler takes the assembled code or bytecode and tries to recreate the original source code. In the case of tjs2 bytecode, that means recreating the .tjs scripts before they were turned into bytecode. Decompiled code is theoretically alterable and should be able to be compiled again using a normal compiler for that language without any knowledge of special semantics.
In the case of UlyssesWu's Furikiri, in addition to the readme.md, here is this comment made by the developer to clarify its use.
> I've found out how to decompile and compile.
> First, Furikiri seems to be the only decompiler that exists for tjs, everything else is a dissembler.[...]
>The main thing to know is that you can't use a compiler successfully on a dissembled tjs, only a decompiled one.
Basically you're correct, there is no "TJS2 decompiler"(*) other than my Furikiri. But Furikiri is not finished (the TJS2 language is a bloated, badly-implemented language after all), and currently the decompilation result can only be used as a reference of TJS2 bytecode's behaviour - that's already useful in my cases. You cannot expect the decompilation result can be directly compiled again, it's impossible for now.
However, it might be more practical to implement a tool to compile the disassmbled assembly. Furikiri has a plan (issue) to do that, but I currently just focused on decompilation.
Furikiri also provides the fastest TJS2 disassemble feature - definitely faster and more complete than the official one. That's the main feature of girigiri for now.
(*): Xmoe said on weibo (at 2019-5-25) that he will release a tjs decompiler after a while. Maybe he can make a better tool than my shit
Here is the referenced xmoe's tjs2Decompiler. Yeah... that never happened. xmoe did release a tjs2Compiler, but that is not really relevant for our purposes.
The main problem with using furikiri is that it is not compiled.
.sln files are Microsoft Visual Studio project files. Compiling them requires Microsoft Visual Studio, the Windows development SDK, the required build tools which is 5-20 GB minimum + a supported version of Windows for that Visual Studio version + the source code + the dependencies, and all of that must be cross compatibile with exactly the correct versions of one another before anything can work. It is entirely unreasonable for anyone except the original developer to know how to compile their software since only they know what versions of what and on what platforms and architectures they used to make their software work.
Compiling software is hell on earth, and I will not be covering how to do it here. Attached to this post is a compiled version of furikiri obtained via black magic and vs 2022. It is also available on Mediafire. Source code.
It requires the .net cli runtime 8.0. If it does not work on your machine or you encounter issues, bother the original developer about how to compile their software by creating a new issue in their repository.
So anyway, let's install the furikiri dependencies, whatever they may be since they are not actually officially documented anywhere as documentation, and extract the tool to Tsumugi no Hanayome/tools/furikiri.cli_girigiri.cli_net8.0/.
Code:
C:\Users\User>"C:\Users\User\Desktop\Tsumugi no Hanayome\tools\furikiri.cli_giri
giri.cli_net8.0\girigiri.exe" --help
Furikiri TJS2 Disassembler/Decompiler
by Ulysses, [email protected]
Usage: [options] <Files>
Arguments:
Files File paths
Options:
-?|-h|--help Show help information
-da|--disassemble Disassemble byte code
-d|--dec Decompile byte code
-p|--print Print result
Examples:
Girigiri init.tjs
All done!
The following syntaxes should work. The first one is for disassembly, and the second one is for decompilation.
Since we do not know how these tools will behave (besides poorly due to being written in c#) and do not want to risk altering anything in the Tsumugi no Hanayome/extracts/ folder, lets create a new "tjs" subfolder as tools/furikiri/tjs and copy all .tjs files there.
Code:
cd Desktop\Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0
girigiri.exe tjs\initialize.tjs
girigiri.exe --dec tjs\initialize.tjs
Every invocation of the tool seems to need user input as in Enter or ctrl+c meaning that this tool is not at all scripting friendly which contradicts one of the main advantages of having a cli. After some further testing, the "All done!" at the end does not take into account whether or not the tool crashed during execution. I stand by my poorly written comment earlier. Moving on.
The first command without any options created tjs/initialize.tjsasm which is a disassembled version of the tjs2 bytecode. This tool assumes --disassemble if no option was specified. Running the same command with --dec produced tjs/initialize.dec.tjs which is a more readable version of the disassembled version.
Opening the files in notepad++ results in some interesting code to look at, but first, while we are here, let's dissassemble and decompile every single .tjs script so we can use them as reference later.
Interestingly, not every .tjs script is in bytecode. A tjs2 script in bytecode will start with the following characters.
Code:
TJS2100
So, let's open all the .tjs files and look for that header. If the .tjs script does not start with that header, then the .tjs script is in plaintext. Let's move all the plaintext .tjs files into a folder called tools\furikiri.cli_girigiri.cli_net8.0\tjs\plaintext. As a notepad++ tip, after opening them all at once, press ctrl+w to quickly close a file and move on to the next one.
For Tsumugi no Hanayome, here is the list of plaintext .tjs files.
cd C:\Users\User\Desktop\Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0
set tool="C:\Users\User\Desktop\Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0\girigiri.exe"
for /f %i in ('dir /b "tjs\*.tjs') do %tool% --disassemble tjs/%i
After it says "All done!" press Enter, and repeat for all 139 bytecoded tjs files. Yeah... If the above for loop does not work, open the spoiler to read more about Windows code pages for handing character encoding at the command prompt.
Windows command prompts can still work with characters that are not valid in the active command prompt encoding (code page) since Windows dynamically uses ? as a cosmetic placeholder instead of displaying the real value when running programs. This only works as long as there is no enumeration of those file names. That cosmetic substitution allows the real file names to be fed directly into a program as input despite the incorrectly displayed substitute character.
However, the batch scripts listed below process multiple files sequentially and need the real file names in a readable sequence in order to process each one in that sequence. In other words, commands like "dir /b" enumerate the file names into a list for processing which requires the characters output by dir to be valid in the code page the command prompt is using. Any values that are not valid in that code page will have that character replaced by ? as a literal ?, which is not the original file's name. This makes commands that work normally when entered directly into the command prompt no longer work when attempting to automate them in the batch language when using an invalid code page.
The most direct workaround is to use a command prompt encoding that supports the enumeration of the file names. This is possible by using either the Windows code page for utf-8 which is 65001 on newer versions of Windows 10 or newer such as Windows 10 1809+, or, for Japanese media, Windows code page 932, which is the code page for shift-jis. cp932 is available only after changing the Windows locale to Japanese, Japan, and rebooting. Changing the locale is not necessary if the command prompt uses utf-8/65001 encoding already on a new enough version of Windows.
To check the current encoding, type "chcp" at the command prompt and press enter. That will print out the active code page.
Code:
chcp
will print out something like
Code:
Active code page: 437
To change the active code page, type the number of the code page after "chcp" as in "chcp 65001", "chcp 932", or "chcp 437". The command prompt requires an appropriate locale be set for Windows before the command prompt can change to a code page specific to that locale.
While it is possible to rename the files temporarily or to proxy the commands using Python's subprocess library, for simplicity, this guide assumes either that Windows code page 65001 is the active Windws code page in the command prompt or the locale was changed to Japanese, Japan in order to use Windows code page 932.
Code:
mkdir tjs\asm
move tjs\*.tjsasm tjs\asm
for /f %i in ('dir /b "tjs\*.tjs') do %tool% --dec tjs/%i
Disassembly should work with every .tjs script in bytecode, but decompiling will crash ...a lot. Just close the program as necessary and continue until it finishes processing every file. Then move any decompiled files into a subfolder.
Code:
mkdir tjs\dec
move tjs\*.dec.tjs tjs\dec
Of the 139 bytecoded tjs files, all 139 could be disassembled, but only 74 could be decompiled. The disassembled and decompiled files cannot be converted back into a format the game can read, but the files can provide hints as to how the game engine works to help us translate it.
Unfortunately the disassembled code is not very readable. It tends to look like this.
The information we are looking for right now is the exact name of the archive.xp3 this game uses to apply patches, if any. That name should be present in Initalize.tjs. Luckily, initalize.tjs did decompile so we do not have to look through the disassembled output. Here is the relevant section in the decompiled initialize.tjs file.
Code:
useArchiveIfExists("patch.xp3");
for (var v3 = 2; Storages.isExistentStorage(v4); v3++)
{
var v4 = System.arcPath + "patch" + v3 + ".xp3";
Storages.addAutoPath(v4 + ">");
continue;
}
var v3 = patch_aftercall.count;
for (var v4 = 0; v4 < v3; v4++)
{
var v5 = patch_aftercall[v4];
if (v5 instanceof "Function")
{
v5();
}
}
delete patch_aftercall;
Humm. That looks for patch.xp3, then patch2.xp3, then patch3.xp3 and so forth. However, our patch.xp3 is not being read either alone or after adding KiriKiriTools's version.dll with a patch.xp3 created using Xp3Pack.exe.
With this information, we can tentatively conclude that the issue is with the game ignoring the patch.xp3 archive completely despite it having the correct name of "patch.xp3".
Since the KiriKiriTools method of patching the game engine is not working despite the correct name for the archive, let's try another method.
Other methods listed in the Koakuma-chan case study were crskycode's KrkrPatch and bynejake's KrkrPatch and Galpatch. bynejake's two projects share the same code meaning they are logically the same project. Galpatch is just the KrkrPatch repository merged with another repository for another game engine and will be the one developed going forward.
Let's try both crskycode's and bynejake's projects to see how they each work. Here is the KrkrPatch readme.md.
Kirikiri Z File Patch
The universal patch extension for some new Kirikiri game.
How To Use
You need to create KrkrPatch.json with the following format.
After downloading and moving KrkrPatch.dll, the loader, the .json, and the garbro created patch.xp3 to the game directory, here is the result.
What did we learn? With KrkrPatch, we might be able to name the patch.xp3 anything we want, but will always need those four files for the patch to work. In order for the user to run the patch, any user would need to run a separate executable to apply the patch which is always a questionable thing to ask a user to do.
That is technically enough for a proof of concept for the dialogue for this game, but let's try GalPatch now anyway. Here is the GalPatch readme.md.
KrkrPatch
Auto load patch for krkr game.
Usage
- Rename krkr-version.dll to version.dll then put into game root dir.
- Put patch files into unencrypted folder without directory structure.
-- (Optional) Pack unencrypted folder to unencrypted.xp3 archive, use GARbro or Xp3Pack
- Put unencrypted | unencrypted.xp3 | both(priority : folder > archive) into game root dir.
- Supports multiple patches : unencrypted, unencrypted2~9 (larger seq, higher priority).
-- Picontinuous sequences are not necessary, sample : unencrypted3.xp3, unencrypted7.
WafflePatch
Patch for waffle game to prevent crashes.
Usage
- Rename waffle-version.dll to version.dll then put into game root dir.
In other words, we need krkr-version.dll and patch.xp3 renamed to unencrypted.xp3. There is also a claim that loading files from folders works. Unlike KrkrPatch, Galgame hardcodes the name of the folder and patch to load custom assets to "unencrypted" and "unencrypted.xp3", but it does not require additional configuration .json files or a custom executable.
Does it actually work though? Here is what happens when copying krkr-version.dll to the game directory as version.dll and renaming the garbro created .xp3 to unencrypted.xp3
It seems to work.
When removing unencrypted.xp3 and using a folder named "unencrypted", the line is still translated. GalPatch adds support for both unencrypted patch.xp3 archives and for reading assets directly from folders. Every KirikiriZ game needs these features and GalPatch does it by only adding one file, version.dll.
The biggest downside is that the instructions explicitly state that Galpatch patch files must be "without [a] directory structure", but that is a pretty minor downside all things considered.
This case study and my actual patch will use GalPatch for this game, but feel free to use KrkrPatch instead.
There is not much new to say about translating images since the workflow for now is exactly the same as Koakuma-chan's.
- Organize the files under Tsugumi no Hanayome/extracts/Tsugumi_extracts using cli wildcards like *.tlg
- Use Garbro to find all files that need translation
- Use Garbro to batch convert all of the .tlg files that need conversion. The FreeMote Toolkit 4.0+ can also do this with EmtConvert, and so can Garbro.
- Copy the .tlg -> .png converted images to Tsugumi no Hanayome/MTL/images/
- Edit each of them in Krita or another image editing program
- Convert the translated .png images back to .tlg using the image format converter tool in the Kirikiri2 SDK kr2_s3r2.zip/kirikiri2/tools/krkrtpc.exe
- Copy the .tlg images or set the output of krkrtpc.exe to Tsumugi no Hanayome/bin/Tsumugi no Hanayome/unencrypted/
Remember to extract files from the Kirikiri2 SDK archive by using 7z.exe and the -mcp=shift-jis flag to avoid corrupt filenames. This is necessary for .zip archives created using Windows Explorer because Explorer always corrupts non-ascii filenames when creating .zip files at least up to and including Windows 10 22H2 and likely newer.
Code:
7z.exe x -mcp=shift-jis kr2_232r2.zip
Here is a translation for the image format converter tool.
And here is proof of concept for images. As usual, I am using Krita to edit the images, but other image editing programs should also work, probably.
When translating copyright notices and content warnings, always use Yandex OCR. Compared to other more accurate translators, Yandex consistently produces the most hilariously appropriate translations for such content.
For more detailed instructions for translating images in kirikiriZ games, see the Koakuma-chan case study.
Part E) Translating the title and game engine strings
Next is the title.
Given all of the bytecoded tjs2 files, this should have been a long difficult process. But actually, before trying to figure out how to alter the bytecoded tjs2 files, let's look through the tjs\plaintext files first. The title is actually found in Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0\tjs\plaintext\appconfig.tjs. In other words, it is one of the very few plaintext .tjs files. The relevant line in AppConfig.tjs is this one.
Code:
global.ENV_GameName = "紬の花嫁";
Just copy the file to Tsumugi no Hanayome/bin/Tsumugi no Hanayome/unencrypted/AppConfig.tjs and update it.
Code:
global.ENV_GameName = "Tsumugi no Hanayome";
Done with updating the title. That was surprisingly simple!
While we are here, at Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0\tjs\plaintext, Storages.tjs in the same folder has some very interesting information about the subfolders in the .xp3 to look at. Information about how the game organizes its files are striped from the metadata in the .xp3 files, but that information is also present in the plaintext Storages.tjs script. That makes Storages.tjs a good reference for restoring that structure later.
Next, we need to translate the game engine strings. Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0\tjs\plaintext\default.tjs, a plaintext .tjs script, has a significant number of translatable game engine strings. Those are mostly the ones for the shortcut buttons when dialogue is playing.
After that, we finally have to deal with the bytecoded .tjs scripts head on.
Freemote understands e-mote files, but not bytecode TJS2 files. The SacanaWrapper program with all of the plugins can do both. This approach is not recommended for dialogue because
1. The extracted strings loose all context which makes guessing if they can be translated or not outright impossible from just the output. That makes it extremely easy to make a mistake and cause the game to crash.
2. It is not possible to recreate any intermediary structures appropriately in the format the game expects.
For Tsumugi, the translatable lines in the emote .txt.scn files look like this.
The length_of_dialogue is an integer, a number, that displays the length of the string that makes up the dialogue. From my testing, it is not necessarily clear what this being wrong actually affects for Tsumugi, but giving the game wrong information only opens the door to potential crashes.
Does SacanaWrapper correctly recreate this information? Who knows, but almost certainly not. With manual parsing using the FreeMote toolkit, it is possible to toggle this on and off during parsing to see if it matters since the FreeMote toolkit gives much greater control over the data.
Another example is that the game also supports a custom name for one of the characters and has specific rules for counting the length of strings that include that raw custom name. Parsing the .json manually allows for applying these special rules for calculating the length correctly.
There are also some phonechat entries from the characters texting each other and many duplicate entries for those texts that do not need to be translated. Parsing the .json makes it clearer what is happening in both cases.
For the reasons above, I would only recommend SacanaWrapper and other blind/regex based string extraction tools if all other attempts at extracting the contents of the files has failed.
The tool itself is poorly documented without any usage instructions or way to get it working offline. Using anything written in c# online is questionable because Microsoft embeds spyware in .Net.
Also,
- Compiling anything takes like 5 GB of space + Visual Studio on MS Windows because all practical use of c# is tied to .Net and requires the Windows Development SDK.
- c#, although there is some automatic garbage collection to deal with more common errors, is not entirely a memory safe language because it supports manipulating memory and pointers directly.
But it only affects code that is flagged as unsafe you say? Guess what is in StringTool.exe.config.
So you are not getting any of the memory safety, but you do get all of the object oriented requirements, all of the compiling baggage, and all of the Microsoft garbage including having to use windows. Adding all the downsides to using anything written by Microsoft together with the downsides for compiled languages, I just do not get development on c# at all.
And the cross platform promise of .net is mostly a lie too since developers tend to not publish either binaries or instructions to compile their software for other platforms. And even if they did, they would still require visual studio on windows, right? sigh Moving on.
So unlike the c# developer for girigiri, Ulysses, the c# developer, marcussacana, was kind enough to provide a binary of their stuff so we do not have to compile it from source, but also not kind enough to include usage instructions, the necessary dependencies, or any plugins, all of which are required to actually use the software. Sigh. The signs, moans, and groans just never stop with c#. Anyway.
Not quite official usage instructions for using SacanaWrapper.
1 - Download the Plugin Manager and SacanaWrapper from the github "Releases" page
2 - Extract both to the same directory
3 - Open the PMan.exe (Plugin Manager)
4 - Search for NUT, and install the NUTEditor plugin [and all other plugins]
5 - Drag&Drop your NUT [and other] files to the stringtool.exe [to extract the strings]
If you want to skip all of that, here is SacanaWrapper_2.1.5_with_plugins.7z. It is also available as an attachment to this post. Just extract and use it. For more help, read the readme.txt.
Extract it to Tsumugi no Hanayome/tools/SacanaWrapper_2.1.5/. Then install the .net 8.0 requirement. Click on "Run console apps - Download x64" which downloads "dotnet-runtime-8.0.8-win-x64.exe" or a newer minor version. Then install it and run StringTool.exe --help.
Now let's batch extract all of the translatable strings in the bytecoded .tjs files. Open a command prompt and enter the following.
Code:
set tool=C:\Users\User\Desktop\Tsumugi no Hanayome\tools\SacanaWrapper_2.1.5\StringTool.exe
cd "C:\Users\User\Desktop\Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0\tjs"
for /f %i in ('dir /b *.tjs') do "%tool%" -Dump "%i" "%i.txt"
mkdir txt
move *.txt txt
It would be nice to be able to revert back any changes we make in case something breaks which it inevitably will when translating game engine strings. Create a backup of Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0\tjs\txt. One way is to create an archive txt.7z and rename it to _txt.7z.backup.7z.
Now, some super happy fun time with the Windows Snipping Tool and ocr.py begins.
ocr.py should already be set up from the Koakuma-chan case study. If not, refer to the case study instructions to get it working. codeberg/translation_tools/ocr.py has also been updated to output properly formatted .csv to make it more convenient when using it together with romaji.py and sugoi_mtl.py. It also also been moved to codeberg/ocr_tools and renamed to ocr_text_recognition.py, so check the updated repository for it.
Snip something easy to find. In my case, I did the top left menu.
Then run ocr.py.
Code:
cd Desktop\Tsumugi no Hanayome\tools
python translation_tools\ocr.py ..\MTL\ocr\main_menu\mainmenu.string1.png -o
or for ocr_text_recognition.py
Code:
cd Desktop\Tsumugi no Hanayome\tools
python ocr_tools/ocr_text_recognition.py ..\MTL\ocr\main_menu\mainmenu.string1.png -o
That outputs mainmenu.string1.png.ocr.txt which has ファイル inside. To output a .csv, run ocr.py in batch mode, instead of a specific .png file, input a folder.
Humm. Maybe I should resolve the relative file paths to absolute ones to get rid of the .. in the next version of ocr.py. Or maybe removing the entire path and keeping only the filename would be better? Anyway...
Next, open Tsumugi no Hanayome\tools\*.txt Notepad++ and search for the result from ocr.py, ファイル. Ctrl+f, enter the string, and "Find All in All Opened Documents".
Finding the "correct" instance of the string is really annoying. Unfortunately, there is not really a way to get this right the first time. For ファイル in the main menu, the one that worked to translate the main menu option was the instance in Override.tjs. Override.tjs actually happens to have a lot of the translatable strings.
To translate it, change the string to the translated version.
set tool=C:\Users\User\Desktop\Tsumugi no Hanayome\tools\SacanaWrapper_2.1.5\StringTool.exe
cd "C:\Users\User\Desktop\Tsumugi no Hanayome\tools\furikiri.cli_girigiri.cli_net8.0\tjs"
"%tool%" -Insert override.tjs txt\override.tjs.txt override.tjs.translated.tjs
Move the .translated.tjs file to Tsumugi no Hanayome/bin/Tsumugi no Hanayome/unencrypted/ and rename it to remove the extra .translated.tjs extension. Here are some changes to Override.tjs after copying that script to the game's patch directory with the updated translations.
Then repeat the process for every string. Unfortunately, there is not a lot of automation possible here to determine which strings are translatable since every change needs to be manually checked to see if it makes the game crash. Even if the game does not crash right away, if it is not the correct string obtained from ocr.py, then the change needs to be reverted (ctrl+z in notepad++) in order to not risk a crash, and the process repeated.
There are some small shortcuts though. While StringTool strips most context, there is some context from the name of the file and which strings are adjacent to one another. Take this example in Override.tjs.txt.
Code:
menu.vreplay
characterMenu
文字表示(&C)
chSpeedMenu
表示速度(&C)
chSlowMenuItem
遅い(&S)
speed
chSpeeds
slow
chNormalMenuItem
普通(&N)
normal
chFastMenuItem
速い(&F)
fast
There is a pattern to the above strings in Override.tjs.txt since they seem to be next to ch-something-MenuItem and the English word for a specific speed surrounds them. That might be code for altering the speed of the displayed text. If one of those strings is translatable, then all of them that fit the same pattern will probably also be translatable.
By the same token, some may seem like there are translatable, but are not. If one is not, then the rest following the pattern or that are very close to it are probably also not translatable.
Note that even the disassembled tjs2 code provides more context than the above difficult to work with StringTool output. "caption" and "setTextSpeed" both appear next to the translatable strings in the assembler code but not in the StringTool output. Even if assembler is harder to read than decompiled code, it can be worth cross checking certain strings from StringTool against it to provide more context.
Unfortunately, I am not aware of any way to reassemble the diassembled.tjs files, so their use is currently limited to providing additional context for StringTool.exe.
Of course, context is just a heuristic. Ultimately every translated string needs to be tested for crashes. Here are all of the .tjs scripts that I found with translatable strings. There were still a few that I missed, so this is not comprehensive.
While Override.tjs had quite a few, even with the above scripts included, that was not enough to translate all of the game engine's strings. These game engine strings were not translated.
- The infamous を実行します, screenshot, string and related strings in the menus
- Font selection menu in main menu and font names
- Text to speech menu in main menu
- Gamepad button assignment menu
- Help tips
If anyone knows how to translate the screenshot string or the help tips, please comment on how exactly. I could not find the strings in the bytecoded .tjs scripts, the .ks scripts with macros, or the plaintext .tjs scripts.
The help tips not being found is especially puzzling. I suspect some of the menu ones, like the screenshot string, are in .dll plugins. The fonts ones, text to speech, and gamepad strings are probably in the bytecoded .tjs files, but I did not look very hard for those since I doubted anyone would use those. There is another more obvious font selection menu in the config screen already and TTS is kinda obscure.
For completeness, there were some strings found in these associated screens as images that I did not translate for the patch. They may get translated later but those are a lot of strings to ocr and translate for a negligible impact on the player experience.
- Extras - music tracks
- Extras - character pose screen
So far, we can run the following snippet to generate the translated .tjs files.
Then the .translated.tjs files can be moved to unencrypted/ and renamed to omit the .translated.tjs extensions so the game read them. However, that process is overly cumbersome, so let's automate it while we are here.
Let's create a new folder called Tsumugi no Hanayome/tools/scripts and create a new text file inside of it called "update_gamestrings.cmd".
Code:
@echo off
set working_directory=C:\Users\User\Desktop\Tsumugi no Hanayome
set tool=%working_directory%\tools\SacanaWrapper_2.1.5\StringTool.exe
set raw_tjs_directory=%working_directory%\tools\furikiri.cli_girigiri.cli_net8.0\tjs
set translated_txt_directory=%raw_tjs_directory%\txt
set patch_directory=%working_directory%\bin\Tsumugi no Hanayome\unencrypted
"%tool%" -Insert "%raw_tjs_directory%\keybinder.tjs" "%translated_txt_directory%\keybinder.tjs.txt" "%patch_directory%\keybinder.tjs"
"%tool%" -Insert "%raw_tjs_directory%\movieqsel.tjs" "%translated_txt_directory%\movieqsel.tjs.txt" "%patch_directory%\movieqsel.tjs"
"%tool%" -Insert "%raw_tjs_directory%\nocfchkdialog.tjs" "%translated_txt_directory%\nocfchkdialog.tjs.txt" "%patch_directory%\nocfchkdialog.tjs"
"%tool%" -Insert "%raw_tjs_directory%\override.tjs" "%translated_txt_directory%\override.tjs.txt" "%patch_directory%\override.tjs"
"%tool%" -Insert "%raw_tjs_directory%\snapcapture.tjs" "%translated_txt_directory%\snapcapture.tjs.txt" "%patch_directory%\snapcapture.tjs"
"%tool%" -Insert "%raw_tjs_directory%\speech.tjs" "%translated_txt_directory%\speech.tjs.txt" "%patch_directory%\speech.tjs"
"%tool%" -Insert "%raw_tjs_directory%\stopdeactive.tjs" "%translated_txt_directory%\stopdeactive.tjs.txt" "%patch_directory%\stopdeactive.tjs"
"%tool%" -Insert "%raw_tjs_directory%\uigame.tjs" "%translated_txt_directory%\uigame.tjs.txt" "%patch_directory%\uigame.tjs"
"%tool%" -Insert "%raw_tjs_directory%\uimain.tjs" "%translated_txt_directory%\uimain.tjs.txt" "%patch_directory%\uimain.tjs"
Now we can just run that file every time we update a string and it will update the relevant file in Tsumugi no Hanayome/bin/Tsumugi no Hanayome/unencrypted so the game can read the changes. Unlike girigiri, isn't it nice to have cli tools that are automation friendly?
With the caveats mentioned above, that is enough to translate most of the game engine strings.
That covers translating dialogue, phonetext, images, title, game engine strings, some level of game engine string automation to help with manual debugging. The next step is to dive more into automation.
However, there is one more thing I would like to do before diving into more automation. Let's try to make unencrypted.xp3 support subfolders to keep everything nice and organized.
For that, we need to know a little but more about how Galpatch's krkr-version.dll actually works. Let's experiment. Then we can determine whether it is possible or not.
- Loading an empty initialize.tjs makes the game not launch at all anymore.
- Loading the decompiled initialize.tjs results in a "String exception raised Cannot convert given narrow string to wide string" which is the same error generated as when we were messing with the various utf-16 encodings, bom vs no bom, le vs be, back in the Koakuma-chan workflow.
- Converting the decompiled initialize.tjs from utf-8 to utf-16-le-bom changes the error to "Script exception raised Syntax error (syntax error)".
In other words, Galpatch can alter the loading of initialize.tjs despite how early that file is in the boot process. It can also alter Storages.tjs which gets loaded before the code that tries to load "patch.xp3" which is "useArchiveIfExists("patch.xp3");".
With this information, there are a lot of games we can play now with how the files load. Let's try doing the same thing as the Koakuma-chan workflow to create subfolders. Here is the basic syntax.
However, after moving 紬の花嫁01 プロローグ.txt.scn, the dialogue script that translates the first line of the game, to the unencrypted/data folder, the first line of dialogue was no longer translated after loading the contents of the unencrypted folder and the unencrypted.xp3. The title from AppConfig.tjs loading, the Storages.tjs script was loading or the game would be crashing without it, and 紬の花嫁01 プロローグ.txt.scn was already tested to translate the first line, but the files from the subdirectory never loaded.
At the bottom, there is another function called addAutoPathRecursive(). However using that syntax did not worth either.
Changing is to the following removed the error which implies the game could find the archive and so it launched successfully, but did not load any custom assets from the archive.
Using crskycode's KrkrPatch with the archive named patch.xp3 had the same behavior meaning that this is not an issue with the patch software. Rather, this is an issue with the game's willingness to load assets from unencrypted archives and plain folders separately from the patch software's logic of loading files from the root of the .xp3 archives.
Thinking back to Koakuma-chan, arcusmaximus's KiriKiriTools did not have this issue. That is likely because it adjusted the game's loading of the patch.xp3 to allow loading of all assets from it dynamically by working with the game engine's built in asset loading mechanism.
However, bynejake's GalPatch and crskycode's KrkrPatch use different logic for loading assets from patches. At least in GalPatch's case, that updated logic allows replacing initialize.tjs, which is an improvement, but that logic also does not seem to patch the game's built in patching system for loading updated assets. So, we cannot use the game's built in asset loading system since that code still does not support loading unencrypted .xp3 archives.
The obvious thing to do is that we could technically modify the source code of krkr-version.dll and recompile it to forcefully add support for subfolders. But, besides compiling software always being a living hell, it would also alter the hash of the version.dll that we need to include with the final patch.xp3 from the one published on bynejake's Github repository.
version.dll contains executable code. Asking people to run executable code always presents a risk to that user. Even if we do not intend to run code maliciously, a third party could alter the contents of the .dll and republish the patch without our knowledge or consent.
That means the exact version of the .dll is important because it allows users to verify they are using an easily obtainable public .dll with a known hash which proves the file was not modified from the source repository version by a third party, including us, the patch provider.
A hash is a mathematical algorithm that always outputs the same number of bits as output regardless of the input. Crucially, the same input will always result in the same output. Among other uses, hashes can be used to prove that two files are identical even if they have different file names, extensions, created dates, modified dates, or other metadata differences. Hashes check the content of the data not the filesystem level metadata.
Including version.dll is not technically needed to publish the final patch.xp3 at all. version.dll is only provided with the patch for convenience. But we still need some justification for asking a user to take that risk of running that file. We can prove convenience is that reason by making sure the included version.dll is the exact same file as in the official repository for GalPatch.
Essentially, being able to publish a patch using the original .dll from the publicly available repository means any user can check the hash of the provided .dll with the original .dll from the repository to verify for themselves that the file was not changed. While this is not flawless protection against persons who hijack our patch for malicious purposes, using provably unmodified files published by the source developer goes a long way towards establishing trust in our translation patch. It also speaks to motive that there was a genuine attempt to minimize the user's exposure to arbitrary code execution to what was genuinely necessary to translate the game.
If the hashes do not match proving they are different files, then that convenience argument falls apart and so does our credibility as the patch provider by opening up the user's exposure to additional arbitrary code execution.
What was changed? Where is the source for those changes? Why is the patch.xp3 now dependent on this specific .dll instead of the official version? Was it really necessary to change it, or was there any possible way the publicly accessible version.dll could have worked?
While those questions have answers, it is better to not create a situation where they need to be asked in the first place. That makes compiling version.dll from source an unrealistic solution to this very minor problem of not having organized subfolders in the patch archive. Unless there is a technical reason that makes it genuinely necessary to compile version.dll ourselves, we are better off using the official version.dll. Mere better organization does not meet that high bar where we can justify exposing users of the patch to higher risk because there is an alternative that poses less risk.
It should be noted that the patch.xp3 includes altered bytecoded .tjs files which are fundamentally unverifiable. They are not human readable as bytecode, not guaranteed to decompile into normal tjs2, and the disassembled code is difficult to interpret at best. However, at least that danger is confined to the patch.xp3 file itself and a sufficiently technically literate user can always cut out all the .tjs scripts if they feel that is too much risk to them. The justification of being "necessary to translate the UI" is also quite reasonable for a patch that is meant to be a translation.
Since that failed and recompiling is ill-advised, the best thing to do is to add an alternative code path for the storages subsystem as a function that does the same thing but actually works for unencrypted archives and folders. Ideally, that should be written entirely in plaintext tjs2 to make sure it is portable and readable. Unfortunately, while such a thing is undoubtedly possible, I do not actually have any code to actually do that. For the time being, the patch will have to go without organized subfolders.
Part G) Automating dialogue extraction, translation, and insertion
The next step is to automate the extraction, translation, and reinsertion of dialogue.
The basic goal when automating stuff is to make sure one operation works, and then to automate the process for all of the files using that syntax that was tested to work. That syntax in CLI apps can be automated by using a shell script with variables for all of the inputs.
In my case, I am using a folder called Desktop\Tsumugi no Hayayome\MTL to contain the intermediary files related to translating the dialogue and splitting every aspect of it into subfolders. The actual .cmd files, however, are placed at tools/scripts since they are more tools meant to manipulate static assets than static assets themselves.
One alternative to subfolders is to keep everything besides the final output in the same folder, but that confuses me since that mixes different types of content. I would rather have all of the files in seperate folders since the folder names can be used to clarify the nature of the files inside of it. Separate folders also simplifies selecting the correct files while automating everything with batch scripts.
The structure I am using for translating the dialogue inside the e-mote files is
- dialogue_scn_raw - the raw KrKrDump extracted e-mote .txt.scn dialogue scripts
- dialogue_scn_json - the dialogue scripts converted from e-mote format (.txt.scn) scripts to txt.json using FreeMote
- csv - the spreadsheets in .csv format created by the ScriptExtractInsertTool.py that contain the dialogue extracted from the json files. The dialogue in these .csv files can translated using romaji.py and sugoi_mtl.py
- dialogue_scn_json_translated - the output of ScriptExtractInsertTool.py inserting the translated dialogue from the spreadsheet.csv files back to .json
- dialogue_scn_translated - the final translated .txt.scn e-mote scripts created using FreeMote
If following this guide, be sure to create the above folders and copy all of the dialogue .txt.scn scripts to MTL/dialogue_scn_raw.
As was mentioned earlier, automating stuff with the batch language requires the command prompt to be able to enumerate the characters in the file names correctly. Otherwise, it will not be able to find the files to work on them due to interpreting '?' literally. That means the following scripts will only work in the command prompt code pages for utf-8, 65001, or shift-jis, 932. Check the current code page by entering chcp into the command prompt wiindow.
Code:
chcp
Active code page: 932
To change the code page, either change the locale to Japanese,Japan or enter either "chcp 932" or "chcp 65001". Changing to chcp 932 also requires changing the locale to japanese,japan.
We need a script to actually perform the above actions using the syntax discovered earlier in this case study using the data in the above folders.
Let's create Desktop/Tsumugi no Hanayome/tools/scripts/extract.txt.scn_to_json.cmd
If following this guide, the paths or script names might need to be adjusted since the game specific tools were moved from codeberg/translation_tools/ to codeberg/translation_projects/ after this guide was written. I also normally put all .cmd files at tools/scripts/*.cmd The *.cmd scripts below are also all available at codeberg/translation_projects/games/Hending/Tsumugi no Hanayome/scripts.
Code:
@echo off
set working_directory=%userprofile%\Desktop\Tsumugi no Hanayome
:: this converts the .txt.scn to .txt.scn.json
set tool=%working_directory%/tools/FreeMoteToolkit/PsbDecompile.exe
set dialogue_scn_raw=%working_directory%\MTL\dialogue_scn_raw
set dialogue_scn_json=%working_directory%\MTL\dialogue_scn_json
set csv_directory=%working_directory%\MTL\csv
set dialogue_scn_json_translated=%working_directory%\MTL\dialogue_scn_json_translated
echo 'dir /b "%dialogue_scn_raw%\*.txt.scn"'
for /f "delims=;" %%i in ('dir /b "%dialogue_scn_raw%\*.txt.scn"') do "%tool%" "%dialogue_scn_raw%/%%i"
if not exist "%dialogue_scn_json%" mkdir "%dialogue_scn_json%"
if not exist "%dialogue_scn_json%/resx" mkdir "%dialogue_scn_json%/resx"
echo move /y "%dialogue_scn_raw%\*.resx.json" "%dialogue_scn_json%\resx"
move /y "%dialogue_scn_raw%\*.resx.json" "%dialogue_scn_json%\resx"
echo move /y "%dialogue_scn_raw%\*.txt.json" "%dialogue_scn_json%"
move /y "%dialogue_scn_raw%\*.txt.json" "%dialogue_scn_json%"
Now lets open a command prompt and run extract.txt.scn_to_json.cmd.
Code:
cd Desktop\Tsumugi no Hanayome\tools\scripts
extract.txt.scn_to_json.cmd
If all of the paths were correct, that should create a lot of .json files and move them to dialogue_scn_json. Next, we need to extract out the translatable strings from the .json files.
Most of the two weeks from the game's release date to the patch release date was spent writing a script to extract and insert the dialogue from those .txt.scn.json files without missing lines or generating errors.
Getting the phone texts translated also made the process take twice as long as it should have especially since the developer used a koala emote which took quite a while to figure out how to deal with. Koalas are not text! In the end, I just gave up and hardcoded some fixes for the koala.
But now that it has been done, it is just a matter of using that script in the same way as the Koakuma-chan version.
From my Codeberg, download translation_tools or translation_projects/games/Tsumugi no Hanayome/TsumugiNoHanayome_ScriptExtractInsertTool.py. A link is in my signature for signed in users. On Codeberg, click on the name of the script to preview the contents and then click on the "Download file" icon next to "History" to download it. Git also works.
Code:
set working_directory=%userprofile%\desktop\Tsumugi no Hanayome
cd %working_directory%
cd tools
git clone https://codeberg.org/entai2965/translation_projects.git
The script needs Python3.7+ or CPython3.6+ to run. It should already be installed from the Koakuma-chan workflow and from the ocr.py/ocr_text_recognition.py script used above.
Code:
python "TsumugiNoHanayome_ScriptExtractInsertTool.py" --help
usage: TsumugiNoHanayome_ScriptExtractInsertTool.py [-h] [-s SPREADSHEET]
[-o OUTPUT]
[-cn CHARACTER_NAMES]
[-c COLUMN] [-w WORDWRAP]
[-cd CSV_DIALECT] [-v]
[-d]
mode inputfile
Extract and insert text into kirikiriz .txt.scn files after they have been
extracted from archive.xp3. Any folders in the paths must already exist. For
usage, 'python tool.py -h' version=0.1.0
positional arguments:
mode must be extract or insert
inputfile the source file to extract strings from and insert
them into
options:
-h, --help show this help message and exit
-s SPREADSHEET, --spreadsheet SPREADSHEET
the file name to read and write the extracted strings
to and from, he first row is reserved for column
headers, must be .csv if pyexcel is not installed with
'python -m pip install pyexcel pyexcel-xls pyexcel-
xlsx pyexcel-ods pyexcel-ods3 pyexcel-odsr
openpyxl==3.0.10'
-o OUTPUT, --output OUTPUT
the output file name for the resulting file, only used
for mode=insert
-cn CHARACTER_NAMES, --character_names CHARACTER_NAMES
a .csv or spreadsheet mapping the raw character name
to a translation, the first row is reserved for column
headers
-c COLUMN, --column COLUMN
the column number in the spreadsheet to use as
replacements, only used for mode=insert, starts from
1, column A is the 1st column, so enter 1, column D is
the 4th column, so enter 4, column C is 3, 0 is a
special flag that uses the right most column, default
is 0
-w WORDWRAP, --wordwrap WORDWRAP
word wrap setting, enter the number of characters per
line, word wrap assumes a maximum of 3 lines, default
is 50
-cd CSV_DIALECT, --csv_dialect CSV_DIALECT
specify the csv dialect when reading spreadsheet.csv
files, normal settings are used otherwise, ignored for
non .csv formats, valid options are unix, excel,
excel-tab
-v, --version print version information
-d, --debug print debug information
The interface was copied from the Koakuma-chan script, so it should look very familiar. The basic syntax to extract and insert translatable text is this.
To see it working, run the string extraction tool on one of the .txt.scn files.
Code:
cd Desktop\Tsumugi no Hanayome\tools
python "translation_tools\games\Tsumugi no Hanayome\TsumugiNoHanayome_ScriptExtractInsertTool.py" extract "C:\Users\User\Desktop\Tsumugi no Hanayome\MTL\dialogue_scn_json\紬の花嫁01 プロローグ.txt.json"
That should create "dialogue_scn_json\紬の花嫁01 プロローグ.txt.json.csv" and complain that a lot of character names were not found.
Let's fix that by translating the character names. Create a text file called character_names.csv, TsumugiNoHanayome_characternames.csv, or similar with these contents.
Code:
Japanese name,latin name
葵,
山吹,
So, who are 葵 and 山吹? I don't know (IDK). Those names were output from the tool.py. Running "python romaji.py 葵" outputs "Aoi". "python romaji.py 山吹" outputs "Yamabuki". Let's assume that is correct for now.
Code:
Japanese name,latin name
葵,Aoi
山吹,Yamabuki
We could leave it at that, but let's check the vndb page. It looks like vndb uses the same spelling for the character names as UniDic which is a good sign that we can continue to use the output of romaji.py for the rest of the character names. From the character information on vndb, we can also add entries for 吐瀉丸, Toshamaru, and 速水 遥, Hayami Haruka.
Code:
Japanese name,latin name
葵,Aoi
山吹,Yamabuki
吐瀉丸,Toshamaru
遥,Haruka
Then rerun the above command with the updated character dictionary. With the character names it becomes.
set working_dir=C:\Users\User\Desktop\Tsumugi no Hanayome
set tool=python "%working_dir%\tools\translation_tools\games\Tsumugi no Hanayome\TsumugiNoHanayome_ScriptExtractInsertTool.py"
set character_names=%working_dir%\TsumugiNoHanayome.characternames.csv
set file=%working_dir%\MTL\dialogue_scn_json\紬の花嫁01 プロローグ.txt.json
set spreadsheet=..\MTL\dialogue.txt.scn.json.csv
%tool% extract "%file%" --spreadsheet "%spreadsheet%" -cn "%character_names%"
There should not be any missing character name errors when processing 紬の花嫁01 プロローグ.txt.json.
The above assumes the commands get entered directly into the command prompt. If there are issues running the commands as a .cmd file, then it may be an encoding issue. If the filename, 紬の花嫁01 プロローグ.txt.json, becomes distorted on older versions of windows (<Windows 10 1803+), then use shift-jis encoding for .cmd files. Once it extracts correctly, extract the next one.
Code:
set file=%working_dir%\MTL\dialogue_scn_json\紬の花嫁02 吐瀉丸(ev213放尿あり).txt.json
%tool% extract "%file%" -cn "%character_names%"
Code:
？？？ nested character name not found
おかあさん character not found
としゃまる nested character name not found
From here, we can either continue to run the script on the files one by one fixing the errors in the character names as we go or processes all of the files at once. Both approaches are valid.
Since we mostly already have the code we need to processes all of the files together, let's just move on. Here is what the characternames.csv looks like when complete.
Some of those translations are not so good, but without any context, it is difficult to know what is accurate and what is not, especially the DQN ones.
In particular, ヘビィボウガンマン, Hebibougan Man might be heavy bone man? Some of the translation engines I put that into transliterated it to Heavy Bow Man and Heavy blowgun man. If the bone version is what is intended, then maybe something like "Large boned man" would be an accurate translation? IDK, so let's just leave it as the raw romaji for now.
We can fix quality issues later. For now let's keep automating. Here is a script that can automatically extract the translatable dialogue.
Code:
@echo off
set working_directory=%userprofile%\Desktop\Tsumugi no Hanayome
:: this converts the .txt.scn to .txt.json
::set tool=%working_directory%/tools/FreeMoteToolkit/PsbDecompile.exe
:: this converts the .txt.json to .txt.scn
::set tool=%working_directory%/tools/FreeMoteToolkit/PsbBuild.exe
:: this converts the .txt.json to csv and back
set tool=python "%working_directory%/tools/translation_tools/games/Tsumugi no Hanayome/TsumugiNoHanayome_ScriptExtractInsertTool.py"
set dialogue_scn_raw=%working_directory%\MTL\dialogue_scn_raw
set dialogue_scn_json=%working_directory%\MTL\dialogue_scn_json
set csv_directory=%working_directory%\MTL\csv
set dialogue_scn_json_translated=%working_directory%\MTL\dialogue_scn_json_translated
set character_names=%working_directory%/TsumugiNoHanayome.characternames.csv
if not exist "%csv_directory%" mkdir "%csv_directory%"
echo 'dir /b "%dialogue_scn_raw%\*.txt.json"'
for /f "delims=;" %%i in ('dir /b "%dialogue_scn_json%\*.txt.json"') do %tool% extract "%dialogue_scn_json%/%%i" -cn "%character_names%" -s "%csv_directory%/%%i.csv"
Once the paths are fixed, running it works just like the koakuma-chan automation scripts, just with more steps.
Start a command prompt.
Code:
set working_directory=%userprofile%\Desktop\Tsumugi no Hanayome
cd "%working_directory%"
tools\scripts\extract.json_to_csv.cmd
The phonechats needed special handling compared to regular dialogue, but I automated almost all of it. That means just running the scripts is enough to translate the dialogue mostly. There are some name tags in the phone chats that are not translated.
This does assume that the paths in the *.cmd files are updated to work with the local environment and the scripts are invoked in the right environment where cutlet with Unidic and a sugoi compatible server are both available. However, that should both already be set up. If not, refer to the romaji.py, sugoi_mtl.py and the automation section in the Koakuma-chan guide.
Once the local environment is set up and the paths to the tools adjusted, it is just a matter or running the scripts.
For koakuma-chan, there were 4 scripts, extract, add romaji, add sugoi, insert. Since Tsumugi no Hanayome has an extra set of converting the files .txt.scn <-> .txt.json, there are 2 extra scripts to handle that, for a total of 6 that are part of the main dialogue. There is also a 7th script for updating game strings, update_gamestrings.cmd.
1. .txt.scn -> .txt.json
Code:
extract.txt.scn_to_json2.cmd
2. .txt.json -> .csv
Code:
extract.json_to_csv.cmd
3. add romaji to .csv's
Code:
@echo off
set working_directory=%userprofile%\Desktop\Tsumugi no Hanayome
set tool=python "%working_directory%/tools/translation_tools/romaji.py"
set dialogue_scn_raw=%working_directory%\MTL\dialogue_scn_raw
set dialogue_scn_json=%working_directory%\MTL\dialogue_scn_json
set csv_directory=%working_directory%\MTL\csv
set dialogue_scn_json_translated=%working_directory%\MTL\dialogue_scn_json_translated
set character_names=%working_directory%/TsumugiNoHanayome.characternames.csv
for /f "delims=;" %%i in ('dir /b "%csv_directory%\*.csv"') do %tool% "%csv_directory%/%%i" -s
4. add sugoi translations to the .csv's
Code:
@echo off
set working_directory=%userprofile%\Desktop\Tsumugi no Hanayome
set tool=python "%working_directory%/tools/translation_tools/sugoi_mtl.py"
set dialogue_scn_raw=%working_directory%\MTL\dialogue_scn_raw
set dialogue_scn_json=%working_directory%\MTL\dialogue_scn_json
set csv_directory=%working_directory%\MTL\csv
set dialogue_scn_json_translated=%working_directory%\MTL\dialogue_scn_json_translated
set character_names=%working_directory%/TsumugiNoHanayome.characternames.csv
for /f "delims=;" %%i in ('dir /b "%csv_directory%\*.csv"') do %tool% "%csv_directory%/%%i" -s
5. .csv -> .txt.json
Code:
@echo off
set working_directory=%userprofile%\Desktop\Tsumugi no Hanayome
:: this converts the .txt.json to csv and back
set tool=python "%working_directory%/tools/translation_tools/games/Tsumugi no Hanayome/TsumugiNoHanayome_ScriptExtractInsertTool.py"
set dialogue_scn_raw=%working_directory%\MTL\dialogue_scn_raw
set dialogue_scn_json=%working_directory%\MTL\dialogue_scn_json
set csv_directory=%working_directory%\MTL\csv
set dialogue_scn_json_translated=%working_directory%/MTL/dialogue_scn_json_translated
set character_names=%working_directory%/TsumugiNoHanayome.characternames.csv
for /f "delims=;" %%i in ('dir /b "%dialogue_scn_json%\*.txt.json"') do %tool% insert "%dialogue_scn_json%/%%i" -cn "%character_names%" -s "%csv_directory%\%%i.csv" -o "%dialogue_scn_json_translated%/%%i" --column 0
6. .txt.json -> .txt.scn
Code:
@echo off
set working_directory=%userprofile%\Desktop\Tsumugi no Hanayome
:: this converts the .txt.json to .txt.scn
set tool="%working_directory%/tools/FreeMoteToolkit/PsBuild.exe"
set character_names=%working_directory%/TsumugiNoHanayome.characternames.csv
set dialogue_scn_raw=%working_directory%\MTL\dialogue_scn_raw
set dialogue_scn_json=%working_directory%\MTL\dialogue_scn_json
set csv_directory=%working_directory%\MTL\csv
set dialogue_scn_json_translated=%working_directory%\MTL\dialogue_scn_json_translated
set dialogue_scn_translated=%working_directory%\MTL\dialogue_scn_translated
pushd "%dialogue_scn_translated%"
for /f "delims=;" %%i in ('dir /b "%dialogue_scn_json_translated%\*.txt.json"') do %tool% -v 3 "%dialogue_scn_json_translated%/%%i"
echo renaming files...
for /f "delims=;" %%i in ('dir /b "%dialogue_scn_translated%\*.txt.psb"') do ren "%dialogue_scn_translated%\%%i" "%%~ni.scn"
popd
The scripts above use the same syntax as was discovered during the dialogue proof of concept above. There are still some automation specific optimizations we make though and some other final notes about the workflow.
We do not really want to send newlines to Sugoi translator. Sugoi will understand the line as a contigious idea better without new lines. That is also true for other translators too, including DeepL and LLMs. in translation_tools/sugoi_mtl.py there is this line
and that should make sure new lines are not sent to Sugoi.
Also note that there are 67 txt.scn files but only 66 have translatable dialogue. "紬の花嫁61_4_staff.txt.json" does not have any text dialogue to translate. That is probably the staff credit roll at the end of the game as opposed to regular dialogue. Since the tool.py is coded to only extract dialogue and phonechats, it will not extract anything from that .json file.
The 66 .txt.scn files with translatable text will generate 70 .csv files because 4 of the files have phonechat conversations, and those produce their own .csv files.
The nameplate for the koala emote was also never automated, so that has to be fixed manually for the final patch.
disclaimer, The image editing for the above .png file was done in mspaint.exe and saved to .jpg to ensure no .png files were harmed in the creation of that image.
Just like with Koakuma-chan, let the game run with the skip setting to 'on' to see if it crashes. If everything is translated and nothing crashes, then the automation workflow is done, otherwise troubleshoot as needed.
The final patch also does not seem to work if Tsumugi.exe.sig is still in the parent folder.
The .sig/signature file is only used to check if the file was modified before loading it, so perhaps GalPatch's version.dll changes the signature of Tsumugi.exe which results in the game aborting its own load code?
However, without the .sig file present, the game just works normally since it has nothing to verify itself against.
It took ages to figure out the Tsumugi.exe.sig issue since I only ever worked on the repackage version with does not have any .sig files at the root of the main game at all. The repackage would always work, but the download versions would not, so fixing it took obtaining a download version and comparing hashes against the repackage version with translation_tools/hashes_organize.py.
With that, this guide is done. Looking back, a lot of software from a lot of different developers, some working for organizations, was used to create a translated version of Tsumugi no Hanayome. Here is an enumeration of that.
General software:
- The internet (anime-sharing.com, search engines, vndb.org, github, codeberg, mediafire, etc)
- Microsoft Windows + the windows command prompt/batch scripting
- various Windows utilities (mspaint.exe, image viewer, snipping tool)
- Microsoft Visual Studio 2022 Community + Win 10 SDK for compiling stuff, especially Furikiri
- CPython, cross platform open source core dependency for a lot of software
- Notepad++ for editing all text files
- wxHexEditor for debugging
- Krita for image editing
Even that is a partial list. The translation patch would not have been published without the above software and the work put into developing it done by the developers involved. I sincerely thank everyone involved for their contributions that helped this project finish in a mere 2 weeks!
Hi, Esan. Do you still have these ancient games to reupload? Sorry, I know it's a lot, so I'll be patient. 南国搾乳☆みるくアイランド娘 OH!!マイクロマン せーえき瞬間
They're all from the same dev. Please take your time.
