Announcement

Collapse
No announcement yet.

[Plugin] YAT - Yet Another Translator 1.4.2.0 + subtitles

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • [Utilities] [Plugin] YAT - Yet Another Translator 1.4.2.0 + subtitles

    What? But we already have UTP and TranslationPlus. Why would we need...

    Yet Another Translator(or YAT for short)

    Yet Another Translator (YAT) is a translation plug-in for CM3D2 aimed to provide working translation capabilities for CM3D2 versions 1.49 and up.

    The project is open source and dedicated to the public domain. The plug-in itself was written from scratch while maintaining compatibility with currently available translations.

    Main features
    • Patchers for Sybaris and ReiPatcher
    • String and asset replacing (pretty much like in UTP and TranslationPlus)
    • Fixed for CM3D2 version 1.49
    • CM3D2 VPVR support
    • Some optimisations to string translating
    • Logging and dumping
    • Subtitles for VPVR and other events
    • Support for Head-mounted displays
    • Text-to-clipboard

    You can find more information on YAT project page

    DOWNLOAD
    The latest version is: 1.4.2.0
    Remember to read the README, as it contains requirements and an installation guide

    Installation, troubleshooting, help
    You can find all help you need on YAT Wiki.

    If you need help or encounter a problem, consult the wiki first.
    I won't be looking at this thread too often because of HF servers being sometimes quite unstable -- moreover, it's hard to keep track of already resolved issues on a forum.
    Until I find a better way to report problems and propose features, consider creating an issue ticket on YAT GitHub page to get your problem looked into quickly.

    More info about contributing is in YAT Wiki/Contributing.

    AutoTranslate for YAT

    For best translation experience (at least the best one can afford with such a vast game as CM3D2), it's suggested to use Google AutoTranslate plug-in by whootwhoot. You can find the plugin's releases on GitHub and support thread on HongFire.

    Raw Subtitles for YAT

    As of version 1.2.0.0, YAT supports transcribing voice files and displaying translations as subtitles. However, YAT itself does not come with any subtitles: any attempt to display them in-game will only show voice files' names instead any kind of text.

    However, most voices are transcribed in the game's ARC files (most of the text is just uncommented, though). Thus YAT provides a separate tool, SubtitleDumper that can read ARC files and output found Japanese transliterations of voice files as text files compatible with YAT.

    Thus, to get raw subtitles (aka. Japanese transliterations), you can either download pre-dumped transliterations or download the tool and dump them yourself.

    Here are the download links:

    RAW SUBTITLES
    FOR VPVRDumped 20.08.2017
    FOR KPVRDumped 01.10.2017

    SUBTITLEDUMPER
    SubtitleDumper is released alongside YAT, so the tool is always in a separate ZIP file alongside the latest YAT.
    Attached Files
    Last edited by denikson; 02-25-2018, 03:11 PM.

  • #2
    AWESOME WORK!!! We need to make AutoTranslate plugin to work with this, we have to contact the author

    Comment


    • #3
      Finally, version 1.2.0.0 is out! As main features, you can now create subtitles for VPVR and other events alike.

      Since subtitles require transcribing voice file into strings, the translation process can be quite tedious. However, KISS decided to add Japanese transliterations as comments into the VPVR scripts. That is why I made SubtitleDumper, a very simple command line tool to extract these raw subtitles and apply them to use with YAT. In addition, in the OP you can find some simple, pre-extracted subtitles. Notice that the raw subtitles are for VPVR only and may not translate EVERYTHING.

      If you want to have full Japanese subtitles, use SubtitleDumper yourself.

      As a semi-related note, version 1.2.0.0 supports additional translation aides. For instance, you can use text-to-clipboard to translate the game with 3rd party tools, like Translation Aggregator.

      In addition, because of requests, I fixed AutoTranslate to work with YAT.

      Update: whootwhoot has updated AutoTranslate to version 1.3.0 with support for YAT and ignore list*! You can find the official release from AutoTranslate's GitHub.

      *The official version does not support request limiting (as of now), but I doubt many will miss the feature.

      Spoiler

      You can find the fixed AutoTranslate for YAT (and the source code, as per license) in the attachments below.

      Update: Revision 2
      Added ignore list (AutoTranslateIgnore.txt) to ignore translating some text. Use this to prevent the plug-in from breaking CM3D2 in some places where the game assumes the buttons/labels are hard-coded to Japanese.

      Update: Revision 3
      Ignore list (AutoTranslateIgnore.txt) now supports ignoring whole levels. Just add
      Code:
      $LEVEL <level>
      on a single line to disable AutoTranslate for level <level>. You can ignore multiple levels by adding multiple $LEVEL-tags (one per line, of course).

      In addition, the number of concurrent translation requests sent to Google can now be limited. That can be done by editing CoroutinePoolSize option in AutoTranslatePlugin.ini under the [General] section. The default value is 100. If you get a lot of time-out messages by AutoTranslate (or translating fails in general), consider lowering the value. In extreme cases, this may also increase performance.
      Last edited by denikson; 09-24-2017, 12:07 PM.

      Comment


      • #4
        Amazing, it really works! Great Work!
        By the way, is there a way to adjust the position of the text on the screen and size?
        Auto translate is working perfectly too, even translating things it didn't translate before.
        Attached Files
        Last edited by animexzon; 08-14-2017, 12:16 PM.

        Comment


        • animexzon
          animexzon commented
          Editing a comment
          awesomeguy247 & denikson That did the trick, tyvm!

        • junsuiakuma1991
          junsuiakuma1991 commented
          Editing a comment
          What the ?! Subtitles in Vacation Mode ? Is that really done by this plug-in ?

        • animexzon
          animexzon commented
          Editing a comment
          After installing Yat, put the RAW SUBTITLES (FOR VPVR) in Sybaris\Plugins\UnityInjector\Config\Strings then install CM3D2.AutoTranslate fixed by denikson in the post above, and you will have subs in VPVR.

      • #5
        Great job again man! I do have one small problem with SubtitleDumper though. The .\CM3D2.SubtitleDumper.exe .\script*.arc command doesn't work on powershell for some reason. When i try it this message comes out [WARNING].\script*.arc is not a file[WARNING].\script*.arc is not a file. But when I remove the * it works fine but i need to do every script.arc separately one by one.

        Comment


        • denikson
          denikson commented
          Editing a comment
          Ah, my bad. And I thought PowerShell is as simple as using Command Line or normal Bash.

          I fixed the README on GitHub: in fact with PowerShell you want to use the following TWO commands:

          $files = Get-ChildItem script*.arc | % { $_.Name }
          .\CM3D2.SubtitleDumper.exe $files

          Again, replace script*.arc with the pattern you'd like.

      • #6
        If someone could upload a full raw dump of the subtitles, that would be superb, I tried to use the dumper and failed miserably. I am not sure what is missing though, since I am testing on a fresh install.

        Comment


        • #7
          Okay so i finished dumping every script arc and put the whole thing into the strings folder. Apparently it's not easy to load almost 200k strings and every loading screen takes around 1-2 min to get past.

          Comment


          • denikson
            denikson commented
            Editing a comment
            Pretty much what awesomeguy247 said. Notice that StringDumper does only one thing: it dumps all the possible subtitles it can find. By all I mean ALL: normal events, VIP, yotogi, VPVR. Moreover, StringDumper does not tag which strings must be loaded for which level. Thus YAT will load ALL subtitles every single scene.
            If you do want all sensible subtitles, you should remove the unneeded ones, leave the necessary and add level tags to the files (by simply adding a level number to the text file name -- see raw subtitles' file names as an example). DO NOT use StringDumper's result as-is with YAT (unless you have a supercomputer or you know what you're doing).

            All in all, I suggest mainly dumping the following ARCs: script.arc (then remove everything except the sex folder), VPVR arcs and some other DLC-related arcs for yotogi. I don't have all DLC (nor the time to tag all subtitles properly), which is why I dumped only the necessary VPVR ones (and even tagged them all with level 43).
            Last edited by denikson; 08-14-2017, 11:03 PM.

          • Maximum92
            Maximum92 commented
            Editing a comment
            denikson awesomeguy247 I completely forget about the non sex/vip stuff lol. I will delete all that crap and hopefully the load times will reduce. I'm trying to keep only actual sentences from yotogi but there are probably 100's of strings of just sucking and moaning sounds to delete. Also tagging them all will take awhile probably. And correct me if i'm wrong, is the level number for free mode 24?
            Last edited by Maximum92; 08-15-2017, 02:08 AM.

        • #8
          I've replaced the old translation loader with YAT but some of the texts are missing, i've made sure all of the translation resources are in the right folder which they are. Any ideas what could be the cause?

          Comment


          • denikson
            denikson commented
            Editing a comment
            Sounds like issue #1 (https://github.com/denikson/CM3D2.YATranslator/issues/1). As a TL,DR: Make sure you have Cecil.Inject 1.2.6 installed. In addition, make sure to remove any previous translation loader DLLs.

          • kuzno12
            kuzno12 commented
            Editing a comment
            Yep, looks like the Cecil.Inject plugin was outdated even though I downloaded the latest version of Sybaris. Sorry for not spotting that a Ticket regarding this issue was already resovled and thanks for the quick reply!

        • #9
          Hey everyone!

          A minor bump to notify of a few updates:

          Updated SubtitleDumper
          SubtitleDumper can now merge all subtitles into a single file! More information in the tool's README.
          You can find the updated tool in the OP.

          Updated raw VPVR subtitles
          With the updated SubtitleDumper, all raw VPVR translations are now merged into a single TXT file!
          You can find the updated subtitles in the OP. REMEMBER to remove previous raw subtitles before installing these.

          Small update to AutoTranslate-YAT
          Added the ability to ignore strings from being translated using AutoTranslateIgnore.txt. The text file is created in UnityInjector's Config folder. Put each string on its own line in the text file.
          Use this list to prevent some game-breaking strings from being translated (for instance CM3D2's edit mode pose editor buttons and VPVR's farming GUI).
          You can find the updated plug-in in post #4.

          Comment


          • #10
            Big Thanks to you denikson, that you did port my AutoTranslatePlugin to your YAT. I wanted to do it myself, but recently got a new Job, so currently I'm busier than ever.
            Currently I'm backporting your changes into the main branch of my plugin, so that there aren't two different versions of my plugin floating around, but it may take a while because of my job.
            I just wanted to let you all know, I'm not dead yet

            So again, thank you for the work.

            Comment


            • denikson
              denikson commented
              Editing a comment
              That is good to hear! I was considering making a PR, but I made so vast changes to the code (pretty much removing all reflection to simplify the code and make tool YAT-only, reorganising classes, reformatting code and other possible ad hockery) that it would've been (most likely) quite a pain to merge it all.

              All in all, it is truly grand that you're back in "business"!

          • #11
            meanwhile I just found out the hard way that AutoTranslate crashes during maid edit very frequently indeed and now I am quite lost. Even though you implemented an autotranslateignore.ini. I simply do not know which strings make the game crash and do not know how to find out.

            So in short I just gotta ask for how one can do that and utilize the feature or if someone can make an all-encompassing .ini so that not everybody has to do it themselves (to begin with, is it in theory possible to do that in a reasonable amount of time?). Otherwise one will just be pressing F10 to deactivate the whole plugin during Maid Edit and Vacation and woe anyone who forgets and did not save recently...
            I am a cream whip, nothing more and nothing less.
            Oh, and I am a cream whip that made a List of CM3D2 Mods [Always WIP and totally NSFW], mind you.

            Comment


            • denikson
              denikson commented
              Editing a comment
              Darn. Given the "random" crashes, the (somewhat) vague error logs and my inability to recreate these errors, I would argue that AutoTranslate plays but a minor role in this issue. After all, VPVR and maid appearance edit work fine for me.

              Nevertheless, I updated the AutoTranslate to R3 which now allows to ignore levels by adding $LEVEL <level> to the AutoTranslateIgnore.txt. For instance,
              Code:
              $LEVEL 27
              will disable AutoTranslate for level 27 (photo editor). YAT (or some other plug-in) usually tells the level in the console when the game switches to it.
              Moreover, you can now limit how many strings AutoTranslate translates at once.
              Unfortunately I am no thaumaturge, but perhaps these features could be of some help.

              At this very moment my best recommendation is to try and dump the debug console (Mirror option in DebugPlugin.ini) and set plugins' verbosity levels to max (for instance, VerbosityLevel=5 to AutoTranslatePlugin.ini). Then just play. When the game crashes, inspect debug.log and look if you find any patterns (i.e. which plug-in was the last to log something before the crash).

              The last resort would be trying to disabling plug-ins one-by-one (while keeping AutoTranslate and YAT enabled) and trying to find the culprit. That would be quite a pain to do, however.

              Or again, maybe the issue is in AutoTranslate. I have but quickly analysed the code of whootwhoot, so there still may be bugs/issues I am not aware of. I'll try to look through the code again.
              Last edited by denikson; 08-29-2017, 12:01 PM.

            • whootwhoot
              whootwhoot commented
              Editing a comment
              So I'm comming back from the dead

              Recently my computer totally crashed, but now I have new hardware and are back in the game

              I'll also have a look at your crash problems, but it all looks very mysterious. Do you play the VPVR content in VR or 'normally'?

              Also, hongfire notifications are totally broken for me, so don't wonder if I don't answer for a long time.

            • SakuraKoi
              SakuraKoi commented
              Editing a comment
              whootwhoot , just normal VPVR it was, I have no VR

          • #12
            Poof! A new version appears!

            For any of you who has a plethora of string translations (or subtitles), version 1.3.0.0 might help with loading times, for new YAT introduces a new, more balanced string translation loader!
            Notice that it is balanced, not optimized: the trade-off for preformance is memory. With this version you might notice that CM3D2 suddenly may take up about 50-100 MB of more RAM (in very extreme cases, that is).

            If you are short on memory, with the new translation loader comes the ability to make string loading much lazier: instead of loading all strings at runtime YAT can now load only strings needed for the current level.

            For more info I highly suggest checking out the changelog (which always comes packaged as CHANGELOG.html in any YAT release).

            Comment


            • SakuraKoi
              SakuraKoi commented
              Editing a comment
              holy hell... I should have complained/spoken about the loading times between scenes way earlier... so that I could have seen this instead of playing over a week (considering my first save on the 4th, I played about 300 days) with them teribad loading times.

              Thank you very much for this update (very swift now), even if it costs 100MB RAM, I kinda doubt it should be too much, due to the fact that meanwhile, if you just got a view tabs open in any browser, a few GB may simply be taken (not even sure why I have 26 processes with 18 tabs, most take more than 50MB)

          • #13
            I don't understand something, is it possible to see what a maid says during yotogi? Because previously those lines didn't have text. I don't have VR or anything
            Attached Files

            Comment


            • thehangman23
              thehangman23 commented
              Editing a comment
              I am trying to figure this out as well. I have never used Visual Studio, so I don't really know what is meant by or how to go about building the tool with it. The build.bat file from the github says that I need MSBuild 15 to compile it, but again, I don't know what I'm doing with Visual Studio.

              Despite not building the tool, it does seem to work in PowerShell, since it still returns the line for "Searches CM3D2 Arc files for subtitle transcripts and dumps them." However, when I try to run arc files, the exe crashes and gives me this:

              [INFO]Beginning to write new data
              [WARNING]script*.arc is not a file
              [WARNING]script*.arc is not a file

              Unhandled Exception: System.ArgumentException: Delegate to an instance method cannot have null 'this'.
              at System.MulticastDelegate.ThrowNullThisInDelegateTo Instance()
              at CM3D2.SubtitleDumper.Program.Main(String[] args)

              I'm assuming this is happening because I did not follow the step with building the tool, though I would if I knew how.

            • denikson
              denikson commented
              Editing a comment
              I suggest reading my comment for the post below. However, here's a small reiteration:

              Dumping everything at once indeed seems to be an issue. I'll fix it in the next version.

              Since it seems that you people are really into getting those yotogi (and other) dialogs, I'll probably just make a simple GUI tool to automate the process. Expect an update in the nearest of futurue.

              And awesomeguy247, you'd have to be a tad more specific about what isn't being dumped correctly -- after all, I need to verify and be able to reproduce the issue too.

            • denikson
              denikson commented
              Editing a comment
              awesomeguy247 (Gee willickers, and I thought the "247" was just a pure coincidence, jk jk). Unfortunately, the names of rental maids don't tell me very much, as they are not referenced in the scripts (at least in latin -- my lack of understanding moonrunes really backfires on me here).

              Nevertheless, I did a very extensive check: I unpacked VPVR script ARCs, collected all voice files that are referenced in all the scripts, and matched them with the voice files that are dumped by SubtitleDumper (yes, you can do all that from a command line).
              In the end, I got the following list of voices that SubtitleDumper "didn't" dump:

              Code:
              S0_vr_21897
              S0_vr_21898
              S0_vr_21899
              S0_vr_21900
              S0_vr_21901
              S0_vr_21902
              S0_vr_21903
              S0_vr_21904
              S0_24520a
              S3_23194b
              S5_21275a
              N23_00345a
              N23_00343a
              N23_00347a
              N23_00341a
              N23_00349a
              N22_00589a
              N22_00587a
              N22_00591a
              N22_00585a
              N22_00593a
              Here's the "shocker": none of these voice files have a direct transcript. For instance, there are no transcripts for S0_vr_21897, but there are for S0_21897 (and unused, but dumped!). No transcripts for S0_vr_21898, but for S0_21898 (also unused, but dumped!). See a pattern here?
              Unfortunately, I can't make SubtitleDumper smart enough to detect these kind of oddities without significantly killing the performance of dumping, since the aforementioned pattern does not always hold (for instance, N23_00345a vs N23_00345).

              Having said that, here's what I'm going to do: instead of skipping those untranslated voices, SubtitleDumper will just output the file names without the transcript instead. That way it will be a manual task to match/duplicate the transcript. Not too hard a task, in my opinion, since there shouldn't be too many of these empty transcripts (for instance, in VPVR it's just 11 voices).

              I'm a bit busy, but expect a "fixed" version this weekend.

          • #14
            https://mega.nz/#!yolQgBQR!om5VHaM_T...tui9niLUwRV0ZA

            Here's what all the contents of the script files had to offer. If anyone's willing to sift through these files to remove the unneeded ones (i.e. Conversations, events, anything that has dialogue box) then upload their work that'd be great.

            The dumper keeps on throwing an error when I'm trying to dump script_personal003_2.arc
            It also throws an out of bounds exception when I'm trying to dump all scripts at once for some reason.
            Attached Files
            Last edited by YBañez; 09-13-2017, 01:57 AM.

            Comment


            • denikson
              denikson commented
              Editing a comment
              And that's really the reason I don't release pre-dumped strings. Do you even have all DLC? What if KISS releases a new one (which will make this dump incomplete)?

              Some scripts cannot be dumped indeed -- that is because they cannot be even opened by CM3D2.Toolkit most likely because they don't contain any proper files. Nothing (reasonably simple) I can do to remedy that (not that it matters, that script is likely empty and has only about 2K worth of bytes).

              As for failing to dump everything at once, it indeed seems to be an issue. Yes, I did forget that CMD does not support glob patterns (like 'script*') internally. I'll fix it in the next version.

              Since it seems that you people are really into getting those yotogi (and other) dialogs, I'll probably just make a simple GUI tool to automate the process. Expect an update in the nearest of futurue.

          • #15
            For yotogi dialogue, what about the technique I used here?

            The TagHitRet function in YotogiKagManager still gets sent the text from script files, it just doesn't do anything but clear it. So I patched the function to take the text first, do the same preprocessing that TagHitRet does in ADVKagManager, and put it in a label where TextToClipboard would pick it up. I'm sure you could adapt this to get the text to where you need it, which would obviate the need for script dumps for yotogi support, and from the sound of it probably improve performance too.

            Comment


            • jojonick
              jojonick commented
              Editing a comment
              I never saw that before nice, can you make it work with yat?

            • hiremi
              hiremi commented
              Editing a comment
              *I* can't, because I don't really know how yat does things and currently have other (mostly RL-type) priorities keeping me from digging into it right now. But I thought maybe denikson would find the idea useful.

            • denikson
              denikson commented
              Editing a comment
              Thanks, I am well aware of that. In fact, I was thinking about initially implementing subtitles using this method -- after all, all CM3D2 voices have these kind of voice transliterations in the script files. However, KISS devs were "nice" enough to comment out the transliterations in most scripts, which makes them inaccessible, for instance, in VPVR.

              Thus to keep subtitle system as generic as possible, I opted for just dumping the subtitles into text files (the same method people use to translate Illusion's games). Thus subtitles are easily translatable and still functional even if the devs decide to comment yotogi transcripts out as well (to be honest, nothing stops them from doing so, as the transcripts serve no purpose in the base game). Doing it through hooking into the script managers (which there are a multitude of for different kind of scenes) is just a bunch of monkey patching that can easily be broken.
              Note, that we want the subtitle system to be flexible and rely on stage-specific hacks as little as possible (since these things can change easily over time).

              Nonetheless, I guess I'll have to look into it again. It will be messy (I bet most of you don't look into the sources either way, so I guess it will not matter), but should be doable.
              Last edited by denikson; 09-14-2017, 08:08 AM.
          Working...
          X