Announcement

Collapse

Please use the Hentai ID thread for all hentai ID requests. Click me for link!

The Identification Thread is Here:

http://www.hongfire.com/forum/showthread.php/447081
See more
See less

Interactive Text Hooker - new text extraction tool

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    Windows XP SP3
    ITH v1.5
    When started, after 2 seconds, ITH flashes message "Can't monitor process" and immediately terminates. Tried quickly attach VS to it:
    Spoiler
    Previous versions work OK.
    読んでみた。 読んでみようとした。 読めたらいいな、と思った。

    Comment


    • #32
      If you plan on doing that, I might be able to assist you with an sqlite database filled with game information for some ~4100 games including:
      • ID_Getchu
      • ID_Company
      • Date_Getchu
      • Name_Romaji
      • Name_Kanji
      • Package_Cover
      • Company_Romaji
      • Company_Kanji
      • Company_Website
      • Agth
      • Walkthrough
      • Savedata
      • Notes
      • Character_Names
      • Character_Bios
      • Character_Romanized
      • Voice_Actors
      • Story
      • Vndb_ID
      • Vndb_Title
      • Vndb_Rating
      • Vndb_Votes
      • Vndb_Tags
      • Vndb_Synopsis
      • Vndb_Relations
      • Vndb_Publisher
      • Vndb_Developer
      • Artist
      • Scenario
      Data was scraped directly from Getchu and then updated certain fields from VNDB via Curl. Not all of the romaji / agth / walk / savedata fields are present but it's a start. Also theres 252mb of getchu package images + 1.5g character images and sample cg.
      Last edited by Devocalypse; 11-04-2010, 07:02 AM.
      devOSD nEXT - advanced eroge translation overlay

      Comment


      • #33
        Setsumi: I didn't see ITH terminate on my SP3, but there do exist big bugs in the process monitor thread.
        I have fixed them, have a try and see if this issue still exist.

        Devocalypse: Glad to see people playing with DB join in. To be honest I'm not quite familiar with DB programming.
        Without help from people like you I don't think I can carry out that function. Let's discuss it in detail then.

        fredericklim: Thanks for your idea, I will think of this issue carefully and come out a plan.
        Last edited by kaosu; 11-04-2010, 05:49 PM.
        sigpic
        Got stuck at AGTH H-codes? Have a try of ITH, supports more game engine.
        http://www.hongfire.com/forum/showthread.php?t=208860

        Comment


        • #34
          Forgot to mention I had the same ITH closing happen in XPsp3 vm.
          The new version does it as well except every 3-4 starts it remains open.. weird.
          I'm no good in cpp but if i had to guess I'd say its got something to do with the initialization routine or some thread not responding in time.

          Concerning the DB, it's not so much DB programming, as it is using an abstraction layer or class to read/write to a common database format (SQLite3 in this case). For cpp this might be an interesting read.

          I populated the DB with php, it started as MySQL but was later migrated to SQLite3 as that one is more portable.

          If there is an online server with the latest database, I'd suggest using MySQL on the server and an SQLite db local to the client. Synchronization would be automatic or on-demand. A client would make a request with his "LastUpdated" timestamp to the server and it would receive all of the records newer than the timestamp given. That would be quite a strain on the server though depending on the amount of data the clients are allowed to harvest from the server in order to keep in their own db.
          A distributed DHT like model would be good except that in that case if everyone is able to contribute to the DB, and there is no centralized version tracking system, it would be easy for malicious users to sabotage the content.
          Another possible solution would be to occasionally generate a "latest version" of the local db and put it on a torrent or ddl yet again that would defeat the purpose of being simple and easy to use.

          In the OLD days DLH.net used to allow you to download the big packs of updates and add them manually to DLH while downloading minor updates / changes automatically within the client.
          So a sane solution would be to dump and bundle the base database with the program so that the clients would only request the latest changes.

          I've been planning an Interface similar to Steam, where you would be able to browse all eroge, search for titles, artists, seiyuus, tags or just about anything. There would be no downloading of games, but you would be able to flag a game as installed and you'd get all information updates on the installed games locally, as well as a centralized interface to launch them from. Think Windows game explorer on steroids. Alas time is not very fond of me so this remains just a bunch of ideas.
          devOSD nEXT - advanced eroge translation overlay

          Comment


          • #35
            Originally posted by kaosu View Post
            Setsumi: I didn't see ITH terminate on my SP3, but there do exist big bugs in the process monitor thread.
            I have fixed them, have a try and see if this issue still exist.
            Looks okay now. Thanks.
            読んでみた。 読んでみようとした。 読めたらいいな、と思った。

            Comment


            • #36
              I suggested an online database that contains only the information necessary to find a H-code for a game. At least the hash of the executable file, the H-code and the game name. ITH should just get the H-Code for the game you want to play and store it locally. There's no need to download the whole database. I also don't see the need to support anything more than that in ITH. I suggest contacting the author of http://agthdb.bakastyle.com/ and asking him about adding hashes for executable files.

              BTW there is already VNDB and ErogameScape.
              AGTH wiki

              Comment


              • #37
                @Kaosu
                1. About thread linking support, it might be good to make a GUI for it.
                The idea is to make people easily manage the linked thread. Because after linking more than 3 threads, IMO it's gonna confusing. Moreover if we made a mistake by linking a wrong thread after linked many threads, we must restart the hook again.

                2. Still about Kirikiri engine, and related with no.1 above :
                It seems the thread address along the subcontext is static (see the bolded text), even if the game is restarted. By using that same pattern maybe you could safely create a user-defined thread linking automation, so it'll autolink threads when it comes out.
                Though I'm not sure if this case can be applied to another engines too.
                Here's an example in WLO :
                0012:0100:0x00541254:0x00547A1A:0x00000018:KiriKiri1 --- normal dialogue
                001A:0100:0x00541254:0x00547A1A:0x00000014:KiriKiri1 --- dialogue when using phone, or 3rd person dialogue
                0022:4262:0x00541254:0x00547A1A:0x00000016:KiriKiri1 --- dialogue when using mail, or novel mode

                3. Also on ConsoleOutput, this "0100: Pending" text flooding is annoying. Could you add an option to hide it so another information logs not flooded by it?

                Comment


                • #38
                  Depending on how well planned a hcode db project should be there are quite a few aspects to keep track of. For example when it comes to hashes, nodvd and/or nojapan patches will create .exe files with a new hash although they share the same code. I do believe you'd need some sort of user management for people to post codes, remove wrong codes or request codes. And while building a db it could be argued that it actually might be also interesting to know which games don't need a code or just something like /KF or /X3. I probably could continue in that style for a bit.

                  Comment


                  • #39
                    My report of ITH:

                    ITH work 100% perfect, without any special configuration with all these games:

                    ましろ色シンフォニー
                    http://www.getchu.com/soft.phtml?id=596025

                    絶対★妹原理主義!! 初回版
                    http://www.getchu.com/soft.phtml?id=686398

                    いつのまにか彼女は……
                    http://www.dieselmine.com/itukano/index.html

                    愛妻【reverse side】
                    http://www.dieselmine.com/2010/aisai/index.html

                    ディスグレイス
                    http://www.getchu.com/soft.phtml?id=8570

                    普通じゃないッ!!
                    http://www.getchu.com/soft.phtml?id=689885

                    NOTE #1: 普通じゃないッ!! need to use a H-CODE to work with AGTH, but, using ITH is UNNECESSARY any code.

                    NOTE#2: in AGTH need to use /ks "いつのまにか彼女は……" and "愛妻【reverse side】" and AGTH need to use /ks4 in 絶対★妹原理主義!!, however in ITH IS UNNECESSARY TO USE /KS and /KS4.

                    NOTE #3: I am using Windows XP SP3.

                    NOTE #4: 普通じゃないッ!! is a wonderful game and everybody NEED to play it !
                    Last edited by fhc; 11-06-2010, 07:02 PM.
                    ...

                    Comment


                    • #40
                      Thanks for you all thinking seriously about this topic. Online H-code query is fantastic but still some problem with it.
                      After considered many aspect of the text extract process, I decided to carry out a new hook represent way to extend h-code.
                      H-code process has its limitation of fixed address. It's calculation power is also not enough to describe sophisticated text finding process.
                      e.g. pointer arithmetic. Since the game is a program, the text display function is describe in some kind of programming language.
                      In order to extract 100% text from it, we also need the calculation power of programming language too.
                      So my plan is to design a mechanism to allow you write your own plugin(DLL) for ITH's engine.
                      You may do anything you want within your DLL, and tell ITH where the text is. Actually ITH_engine.dll is doing this thing.
                      If this feature is implemented, hook is no more game or version dependent. You can write your own engine detect function.
                      This also correspond to ITH's philosophy - interactive. Then the content we need to store online is no more h-code but plugin DLLs.
                      Restriction of this is that it requires people playing this stuff to be familiar with a programming language.
                      But I think people who play with h-codes and read asm are more or less have knowledge of programming language.
                      Last edited by kaosu; 11-06-2010, 08:42 PM.
                      sigpic
                      Got stuck at AGTH H-codes? Have a try of ITH, supports more game engine.
                      http://www.hongfire.com/forum/showthread.php?t=208860

                      Comment


                      • #41
                        Isn't downloading and executing DLLs insecure?
                        I can find h-codes, but I know neither C++ nor ASM.
                        H-Codes are enough for most games and they are easy to create, there's a very good tutorial by freaka for that. Writing C++ and ASM is much much harder. I'm certainly not going to learn it.
                        AGTH wiki

                        Comment


                        • #42
                          I'm second in request for proper thread linking GUI.

                          Originally posted by kaosu View Post
                          H-code process has its limitation of fixed address.
                          Then, maybe "found address by code snapshot" function will help?
                          For example, I have a game and H-code for initial version (/HA14:-14@4ABB10).
                          - Fresh install game, run it.
                          - In ITH select process, press "Generate Snapshot", enter H-code.
                          - ITH returns string with code snapshot around the hook: [offset=10 snap="40 31 79 7C 9F 65 E2 F1 45 0F 15 15 1E 35 8C E1 75 D9 B8 F0 56 4C"]

                          Then I patch game with official and unofficial(nodvd/region) patches and use code like this:
                          /HA14:-14@4ABB10 [offset=10 snap="40 31 79 7C 9F 65 E2 F1 45 0F 15 15 1E 35 8C E1 75 D9 B8 F0 56 4C"]

                          If blue part is present then ITH ignores address and find it dynamically.
                          I believe this approach will work in most cases regardless of any patches and at the same time acts as hash.

                          About DLL plugins, I think they are good only for difficult cases. Very few people will make it. Distributing binary files is a disaster compared to text strings which anyone can copy/paste anywhere and can fish out even from google cache in case of site goes down.
                          読んでみた。 読んでみようとした。 読めたらいいな、と思った。

                          Comment


                          • #43
                            ITH 1.6 is out. Added GUI to manager links.

                            @errzotl80: Currently I haven't come up a good design to automate thread linking.
                            This may be implemented in the future.
                            Last edited by kaosu; 11-13-2010, 07:57 PM.
                            sigpic
                            Got stuck at AGTH H-codes? Have a try of ITH, supports more game engine.
                            http://www.hongfire.com/forum/showthread.php?t=208860

                            Comment


                            • #44
                              When I turn on suppress repetition, the auto copy to clipboard seems to always miss the first two characters of a new game line/paragraph. I tried this on Engage Links from Alcot. Does anyone have a solution to this?


                              ITH Version 1.5 and 1.6
                              WinXP SP3
                              Last edited by skuludo; 11-15-2010, 09:20 PM.

                              Comment


                              • #45
                                @kaosu: Thanks for your good work

                                AGTH has one limit with /ks. When the we use /ks to prevent hooked text repeating by an even number of characters larger than 2 (ex: 4 6 8 ...), AGTH will still break words with 2 similar characters (for example: まま ) and make the hooked sentence become meaningless. Can you fix this disadvantage in ITH? I know if the number of repeated words is 2, it can't be helped, but if it's different from 2, maybe it's possible to do something about it...

                                BTW, good luck with your project :">

                                P/s: my English isn't good, so please forgive me for that

                                Comment

                                Working...
                                X