Page 1 of 2 12 LastLast
Results 1 to 10 of 19
Like Tree2Likes

Thread: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

  1. #1
    ピュア&イノセント Checkmate's Avatar
    Join Date
    Oct 2010
    Location
    Castle Groundgroup
    Posts
    6,597
    Thanks
    202
    Thanked 5,187 Times in 813 Posts
    Points: 145,472, Level: 100
    Points: 145,472, Level: 100
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Three FriendsOverdriveRecommendation First ClassTagger Second ClassVeteran50000 Experience Points

    [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    The Preface

    When I first come to the eroge world, I've always wondered how do people rip the encrypted CG out of the package files. I've come to learn that the easy way is to use ExtractFile or Crass to do it. Unfortunately, not all the eroge works with those two overpopular software, especially newer title, so I tried to google the way to rip CG.

    Unfortunately, most of the tutorial is in Chinese. However, I've come across a blog from a person called Haibara, although most of his blog is in Chinese, some very useful article were available in English. So I've decided to trial and error with his guide, and here I am, writing an introduction to Ripping and Processing CG.

    The Intro

    Kii.re isn't about just ripping the CG and dumping everything we ripped on it and the truth is ripping is just one of the task of being an uploader for Kii.re. There are actually 3 stages before a package is being uploaded on Kiire: Extracing the CG from encrypted game files, Processing the CG, and Optimization.

    Extracting the CG involve in using appropriate tools to extract the game CG without loss of quality. Different game engines requires different tools. For example, Kirikiri2 engine can be extracted using xp3viewer, or Minori Musica engine can be extracted using asmodean's exef2paz. Most game can be done using Westside's tool, a commercial solution packages.

    Once you dumped out the CG, you would find out that most of them have separated faces, bodies, and probably other parts as well (especially genitals and body fluid) to save space and processing power for the game to load fast and efficiently. Processing the CG meaning that one would merge these file into something usable for everyone.

    Once you have merged these files together, you will need to delete all the smaller resolution files, trim out all of the unnecessary pixels, blank spaces for example, converting any bmp format into png format that have much better compression while retaining the quality, and optimizing the png (using PNGOUT) before uploading it to Kii.re

    In this tutorial, I'll also include a link to download those tool, so you won't have them yourself. I'll also share most of my tools available (which is taken from Haibara's box.net), some other tools I got from ShareEX (Westside packages) and whatever I collected along the way. I'll also share my setup, how did I rip & process the eroge that I did in the upload sheet as I go.

    This is the first tutorial I write for Kii.re, also, the first eroge to be uploaded onto Kii.re:
    Kurano-kunchi no Futago Jijou
    倉野くんちのふたご事情
    http://vndb.org/v10793

    The game shall be referred as kurafuta hence forth.

    The Details

    Before you continue, I have to warn you that is not an easy task at all, you would probably need a fairly powerful PC with a bunch of software. Personally, I make use of an independent Core i5 2500K, 16GB of RAM, about 3TB of HDDs and a cached 120GB SSD dedicate to this task. You must use Windows: either XP or Windows 7, nothing middle, nothing newer. I control this machine through Remote Desktop Connection. It also connect to a ZFS Storage Server - so that I can safely store the RAW files and the Packaged files. RAW files are original, unprocessed files, and Packaged files are processed, optimized, ready to be uploaded to Kii.re.

    You would probably need loads of software aside of the ripping tools on it: Photoshop CS5 or CS6 for some post processing. I always go for the latest - CS6. ImageMagick (for post processing), PNGOUT (for optimization), Java Runtime (JRE) (certain processing software requires Java, Kii.re mass uploader also requires Java), ACDSEE Pro (for quickly previewing and assets the files) and dependencies that those above software requires (dotNetFX 3.5, 4.0 for example).

    There are many alternative to those software but that's what I use. Feel free to provide suggestion of the alternatives.

    kurafuta use Kirikiri2 engine, and the images is in xp3 package format. Kirikiri2 engine is an opensource visual novel engine in Japan, it's free to use, so you would find many eroge companies and doujinshi circles use it as the start. Most (if not all, although I don't rip enough to say all) Kirikiri2 game package the files in XP3. The best tools for kirikiri2 and xp3 packages is XP3Viewer. For older game (before 2011), you can probably use ExtractData too.

    You have to have the kurafuta game installed.

    Download XP3Viewer.
    Extract it to somewhere.
    Drag the kurafuta executable file or shortcut onto XP3Viewer.exe
    The game should run along with XP3Viewer, you must leave it alone, there are music but you may want to mute it so, it won't be looping over and over again.
    Drag the xp3 files you want to extract into XP3Viewer, set it to 32-bit PNG, don't use TLG Decoder.
    char.xp3 = character sprites, most post processing works shall be this
    event.xp3 = event (or ev) CG , most fapper want this.
    data.xp3 = scripts, game engine files, may contain useful sprite such as game_logo
    parts.xp3 = may contain many elements, in this case, backgrounds.
    voice.xp3 = contain voice or music file, you can extract this and use it as mobile phone notification, or alarm sound. Oniichan Daisuki! for example works pretty well to make my day better.

    Other games may have other files, it's best to extract them all.

    I'd probably go grab something to eat or play something while it processes the files. This shouldn't take too long if your PC is strong enough. However, it might lags you use it for something else (like watching youtube). This is probably my number one reason to build another PC for ripping and processing.

    Once you're done, you will notice that: the background and the event CG is fine, you don't have to merge anything, this is great. However, you will need to merge the faces in the char CG.

    There are also a bunch of csv files in the char CG. These contain position value for the face, you will need these files to merge the faces together. For this, I use Kirikiri2CSV.jar to make it work.

    You will need JRE 7 and ImageMagick 6.7.2 binary for Windows or newer, always use 32-bit version even if you're on Windows 64bit. So before continue, make sure you have them installed.

    There are a few other things you need to adjust before JRE work on your system: go to "Environment Variables" in the System Setting of Windows, Find the PATH in System Variables and append the following line in the last:
    Code:
    ;C:\Program Files (x86)\Java\jre7\bin
    If you had Java installed else where, change the path as necessary. the ; must stay there or else you will break your system totally. Press Ok.


    Now run cmd.exe as Admin, type java, if it display a bunch of switches such as /a /b etc etc then it works, if it say java is not recognized as an internal or external command, then you probably did it wrong somewhere, check the path, relaunch cmd (you must relaunch cmd everytime you change the PATH value)

    Go to the folder where you store your Kirikiri2CSV.jar, the command is:
    Code:
    kirikiri2CSV.jar -C "path_to_csv_folder" -d "path_to_char_folder"
    Mind the cap. For me, it's
    Code:
    kirikiri2CSV.jar -C "D:\Ripping\Kurafuta\char" -d "D:\Ripping\Kurafuta\char"
    I extracted the char in D:\Ripping\Kurafuta. The csv is located inside char folder, and the folder I want it to merge is also char folder. Now make sure the files are in PNG format and not BMP format, this requirement however differs from tools to tools, games to games.

    kirikiri2CSV.jar will create a bat file inside its folder named merge.bat, if you look at it, it will show:

    Code:
    composite "D:\Ripping\Kurafuta\char\CA01002L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +263+192 "D:\Ripping\Kurafuta\char\merged\CA01002L.png"
    
    composite "D:\Ripping\Kurafuta\char\CA01003L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +263+191 "D:\Ripping\Kurafuta\char\merged\CA01003L.png"
    
    composite "D:\Ripping\Kurafuta\char\CA01004L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +257+193 "D:\Ripping\Kurafuta\char\merged\CA01004L.png"
    
    composite "D:\Ripping\Kurafuta\char\CA01005L.png" "D:\Ripping\Kurafuta\char\CA01001L.png" -geometry +261+193 "D:\Ripping\Kurafuta\char\merged\CA01005L.png"
    
    composite "D:\Ripping\Kurafuta\char\CA01002M.png" "D:\Ripping\Kurafuta\char\CA01001M.png" -geometry +147+109 "D:\Ripping\Kurafuta\char\merged\CA01002M.png"
    These are ImageMagick command to merge the files according to the position value in CSV. Run the bat as Admin and wait for it to finish it work. If you have any error, check if you had ImageMagick installed properly.

    Once it's done, you can find the merged folder inside the char folder, all the faces are merged with the sprites nicely. Now, you have done with processing the images. Next step is Optimizing the Package.

    The First step of Optimization is to filter which files are going to be uploaded. If you take a look at the char folder for example, you will see the file list like this:
    Code:
    CA01002L.png
    CA01002M.png
    CA01003L.png
    CA01003M.png
    Upon further investigation, those L and M files are identical except that the L has a higher resolution than M. Before we conclude that we should discard all M and only keep L. Note that many games, the L files doesn't have the same counts as M. What it means is that in certain game, there are only a few files that are available in higher resolution, not all.

    To be able to tell if this was the case, I'll actually sort all M and L files and count it. My method is to key in *L.png in the Windows search bar - Windows will automatically search out all files that has L in the end. Then I'll cut all the search result, make a new folder call Large, and place those L files inside there.

    My results are: M files count: 532, L files count: 532. It means that we can safely discard the M files and only keep L files for upload. However, discarding the files doesn't mean you delete, discarding mean you're not going to include them in the final upload package. Remember, all RAW files should be kept in case of a ripping error.

    The same process is apply to all other folders. In this particular rip, I only see the event CG that has similar format. So I count the L files to make sure it has the same number of the non L one before conclude that I won't be uploading them onto Kii.re

    So the result are:
    char: only L files in merged folders, plus L files that has full body in char folders - these are already there and the kirikiri2CSV left them untouched, you have to take a look at them and copy them out manually (666 files)
    event: only files that has 1900 x 1080 dimensions (754 files)
    parts/bg: all files
    parts/face: looks like a bunch of faces again, though files counts are 666, char files counts are 666. We conclude that the we already had them all but I suggest you take a look a bit to make sure. In this case, we got them all. So this folder is discarded
    parts/frame: this is the game interface package, you can scoop anything you deem useful here. To me, the company logo and game logo is the two that could be very useful, so I take two of them out.
    parts/rule: these are effects mask to be used in game engine, currently, I don't see how it's being useful.
    parts/sd: sd art, saved them all
    parts/se: sound fx, I'd love to collect them all but not for kii.re
    parts/thumb: save game thumbnail, discard it.
    voice: voice file, all the moaning, ferachio, etc etc, enjoy them at your convenience.

    Now what I'd do is that I'd copy all of the files into a master Folder call Upload, my Upload folder will have the following folder:
    Code:
    bg
    char
    event
    frame
    sd
    Checking char folder, I see there are a lot of blank space from all sides of the sprites images to be trimmed, hence I run the char through a trim batch automation command in Photoshop. It will take an hour or two depending on how many it has to process. You can use imagemagick command too but I don't know how to batch a lot of files yet, I'll update when I figure that out. It should be faster than Photoshop.

    Finally, I run all of the images through PNGOUT. This process is long and brutal: it takes all 100% processing power from your computer for several hours: anywhere from 6-24 hours. This is the second reason why I built myself a reasonably powerful computer seperated from my workstation. During the process, you won't be able to use your computer for anything.

    The process shall reduce the PNG file to smallest, out of my test, a 1.7MB PNG when run through PNGOUT will have a size of 1.15MB, that's about 550KB saving per file, and there are 1500 files to run through, that could be potentially 800MB of space to save for per game, much more on bandwidth in the long run. In this case, I saved 600MB of space after nearly 20 hours of running everything through PNGOUT.

    THIS IS A WORK IN PROGRESS
    Last edited by Checkmate; March 4th, 2013 at 12:34 PM.
    foxyscorpion and peez like this.

    I only post what I bought. My uploads are for those who share the same taste but could not easily import them. If you can buy the game, please do it to support the company.
    Bugs report belong to the Site Support, not my inbox.

  2. The Following 38 Users Say Thank You to Checkmate For This Useful Post:

    + Show/Hide list of the thanked


  3. #2
    KILLER! Killer cell's Avatar
    Join Date
    Sep 2013
    Location
    HFIL (Home For Infinite Losers)
    Posts
    323
    Thanks
    11
    Thanked 67 Times in 11 Posts
    Points: 972, Level: 16
    Points: 972, Level: 16
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    31 days registeredThree Friends500 Experience Points

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    Thank you! found it very handy!
    Ignore This Signature

  4. The Following 34 Users Say Thank You to Killer cell For This Useful Post:

    + Show/Hide list of the thanked


  5. #3
    Member yonakayume's Avatar
    Join Date
    Aug 2013
    Posts
    82
    Thanks
    70
    Thanked 84 Times in 23 Posts
    Points: 2,668, Level: 31
    Points: 2,668, Level: 31
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Three Friends1000 Experience Points1 year registered

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    This is a really useful guide, thank you! The only problem I had was figuring out which XP3Viewer to download.

  6. The Following 27 Users Say Thank You to yonakayume For This Useful Post:

    + Show/Hide list of the thanked


  7. #4
    ピュア&イノセント Checkmate's Avatar
    Join Date
    Oct 2010
    Location
    Castle Groundgroup
    Posts
    6,597
    Thanks
    202
    Thanked 5,187 Times in 813 Posts
    Points: 145,472, Level: 100
    Points: 145,472, Level: 100
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Three FriendsOverdriveRecommendation First ClassTagger Second ClassVeteran50000 Experience Points

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    I still don't have time to put up screenshots and the collections of the program listed in this post though

    I only post what I bought. My uploads are for those who share the same taste but could not easily import them. If you can buy the game, please do it to support the company.
    Bugs report belong to the Site Support, not my inbox.

  8. The Following 27 Users Say Thank You to Checkmate For This Useful Post:

    + Show/Hide list of the thanked


  9. #5
    Senior Member pingtoryan's Avatar
    Join Date
    Apr 2012
    Posts
    1,405
    Thanks
    2
    Thanked 511 Times in 289 Posts
    Points: 8,989, Level: 63
    Points: 8,989, Level: 63
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Three FriendsVeteran5000 Experience Points
    Awards:
    WOW #1 ParticipantWOW #2 ParticipantWOW #4 Participant

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    For some reason kirikiri2csv.jar failed to work for me (I think java problem because ImageMagick works after all the trouble I went to install it).

    Good thing I found a better way to take care of those CSVs and merging operations (Before I found that I made do by taking the ImageMagick merge code from here and manually input the x/y value from the CSV into the code. Tedious, but at least it works).

  10. The Following 25 Users Say Thank You to pingtoryan For This Useful Post:

    + Show/Hide list of the thanked


  11. #6
    ピュア&イノセント Checkmate's Avatar
    Join Date
    Oct 2010
    Location
    Castle Groundgroup
    Posts
    6,597
    Thanks
    202
    Thanked 5,187 Times in 813 Posts
    Points: 145,472, Level: 100
    Points: 145,472, Level: 100
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Three FriendsOverdriveRecommendation First ClassTagger Second ClassVeteran50000 Experience Points

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    There are new, easier way to merge kirikiri game but I don't have time to put out tutorial yet. There's a lot more other engines to do this, some tools were provided to me by genius exclusively for kii.re, I plan to release them gradually.

    I only post what I bought. My uploads are for those who share the same taste but could not easily import them. If you can buy the game, please do it to support the company.
    Bugs report belong to the Site Support, not my inbox.

  12. The Following 24 Users Say Thank You to Checkmate For This Useful Post:

    + Show/Hide list of the thanked


  13. #7
    Junior Member
    Join Date
    Mar 2014
    Posts
    2
    Thanks
    5
    Thanked 12 Times in 1 Post
    Points: 3,489, Level: 36
    Points: 3,489, Level: 36
    Overall activity: 1.0%
    Overall activity: 1.0%
    Achievements:
    Veteran1000 Experience Points

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    Does anyone know how to extract file .cpz? i'm trying to rip Hapymaher -Fragmentation Dream- using crass but unsuccessful. i think it's because the cmvs.cui is outdated (the newest one is 0.6.9 but i couldnt find any download link)

  14. The Following 12 Users Say Thank You to thienthan246 For This Useful Post:


  15. #8
    Senior Member Haiyami's Avatar
    Join Date
    Oct 2011
    Posts
    438
    Thanks
    5
    Thanked 53 Times in 24 Posts
    Points: 7,491, Level: 57
    Points: 7,491, Level: 57
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Veteran5000 Experience Points

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    I haven't had much success in extracting the data. But I need character sprites, background pictures and effect pictures extracted from Tsujidou-san no Jun'ai road and Tsujidou-san no Virgin road. I'm thinking of making an animated video of the drama cd and translating it while I go. But I need the extracted image data from the games. I've used every program I can think of and I haven't had much success. I'd really appreciate it someone could be able to extract the picture data from the games for me.

  16. The Following 2 Users Say Thank You to Haiyami For This Useful Post:


  17. #9
    Senior Member pingtoryan's Avatar
    Join Date
    Apr 2012
    Posts
    1,405
    Thanks
    2
    Thanked 511 Times in 289 Posts
    Points: 8,989, Level: 63
    Points: 8,989, Level: 63
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Three FriendsVeteran5000 Experience Points
    Awards:
    WOW #1 ParticipantWOW #2 ParticipantWOW #4 Participant

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    Quote Originally Posted by Haiyami View Post
    I haven't had much success in extracting the data. But I need character sprites, background pictures and effect pictures extracted from Tsujidou-san no Jun'ai road and Tsujidou-san no Virgin road. I'm thinking of making an animated video of the drama cd and translating it while I go. But I need the extracted image data from the games. I've used every program I can think of and I haven't had much success. I'd really appreciate it someone could be able to extract the picture data from the games for me.
    Check this out.

    Wish I could help more, but those two games aren't in my line of interest and with KO'd internet I can't check them out anytime soon.
    Ignore This Signature
    Alone. It is said that our kind suffer, separated from the glory of Love. But none of us are ever truly alone. For our scholarly hearts are bound by knowledge...discipline...
    Learning is pursued in the name of the many...the studious, who generation after generation, chose the mantle-...of Dire Students.

  18. The Following 2 Users Say Thank You to pingtoryan For This Useful Post:


  19. #10
    Senior Member Haiyami's Avatar
    Join Date
    Oct 2011
    Posts
    438
    Thanks
    5
    Thanked 53 Times in 24 Posts
    Points: 7,491, Level: 57
    Points: 7,491, Level: 57
    Overall activity: 0%
    Overall activity: 0%
    Achievements:
    Veteran5000 Experience Points

    Re: [WIP] The First Kiire Tutorial: Ripping & Processing CG For Dummy

    Quote Originally Posted by pingtoryan View Post
    Check this out.

    Wish I could help more, but those two games aren't in my line of interest and with KO'd internet I can't check them out anytime soon.
    Thanks as bunch! But I can't get the program to work. I get an error "The file msvcp110.dll is missing."
    Last edited by Haiyami; June 13th, 2014 at 09:51 AM.

  20. The Following 2 Users Say Thank You to Haiyami For This Useful Post:


Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. CG ripping help~
    By Sakimichi in forum Tech Support and Talk
    Replies: 7
    Last Post: September 28th, 2013, 09:18 AM
  2. 3D Hentai Movie WIP ~ need a bit of help...
    By Thelostmask in forum Hentai Corner
    Replies: 2
    Last Post: July 3rd, 2012, 04:19 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •