Error when importing translations when using Custom File Format

Hi Team, hope you’re doing great

I’m reaching since we’re having a problem when importing translations using the Custom File Format (CFF) tool.

We already configured our project (ID: 732337) with 3 languages (en, es-ES and jp) and set up the CFF importer and exporter. When uploading the file (using the UI), the source (english) is uploaded without issues, but the translations are empty despite them being present in the imported file, as you can see in the following screenshot:

We also tried with the CLI using crowdin upload sources, but it behaves the same way. We tried in different ways: deleting the file, uploading one more time after the first upload, crowdin upload translations, among others, and still the same issue.

As a way of debugging the importer, we are sending the string entries to the context property, and also added the targetLanguages that is specified in the documentation.

When checking the context in the UI, we noticed the targetLanguages is empty. here’s an example of a string that is pushed to the string env variable (you can see it on the previous screenshot too):

{
   "identifier":"pm00_01-start.00",
   "text":"If the time I had to live was predetermined, \\nI wonder how I would handle it.",
   "context":"Original Text: もし、自分の命の時間が、あらかじめ決まっていたとしたら、俺ならどう受け止めるだろう。",
   "labels":[
      "character:voice-off",
      "scene-title:「プロローグ」"
   ],
   "isHidden":false,
   "customData":"character:voice-off",
   "translations":{
      "en":{
         "text":"If the time I had to live was predetermined, \\nI wonder how I would handle it.",
         "status":"untranslated"
      },
      "ja":{
         "text":"もし、自分の命の時間が、あらかじめ決まっていたとしたら、俺ならどう受け止めるだろう。",
         "status":"translated"
      },
      "es-ES":{
         "text":"abc",
         "status":"translated"
      }
   },
   "targetLanguages": [  ]
}

I don’t if the previous is worth to be mentioned, but just wanted to added the context. Due to targetLanguages env variable being empty, we are using a hardcoded list of languages.

Could you please help us figure out what could be happening?

I’m attaching our importer, the crowdin.yaml and an example of the file we want to import using the CFF. Our project ID is: 732337

cff.importer.js

const contentObj = JSON.parse(content);
const forcedTargetLanguages = [
  {id: 'en'},
  {id: 'ja'},
  {id: 'es-ES'},
];

for (const scene of Object.values(contentObj['texts'])) {
  for (const textId in scene) {

    const item = scene[textId];
    const stringObj = {
      identifier: textId,
      text: item.text,
      context: item.context,
      labels: item.labels,
      isHidden: item.isHidden || false,

      // Max 4k of custom data
      customData: item.customData,

      // Translations p
      translations: {}
    };


    // If importing translations
    let contexTranslations = {};
    const languages = forcedTargetLanguages || [];

    if (languages.length > 0) {

      stringObj.translations = {};
      for (const lang of languages) {

        // Continue if target translation doesn't exist
        if (!Object.keys(item.translations).includes(lang.id)) {
          continue;
        }

        const translation = item.translations[lang.id];
        stringObj.translations[lang.id] = {
          text: translation.text,
          status: "translated"
        };

        contexTranslations[lang.id] = stringObj.translations[lang.id];
      }
    }

    contexTranslations['item'] = item;
    
    // here we're pushing the `targetLanguages` value to the context
    stringObj['targetLanguages'] = targetLanguages || 'empty'; 
    stringObj['context'] = JSON.stringify(stringObj);

    strings.push(stringObj);
  }
}

crowdin.yml

"project_id": "732337"

"base_path": "."
"base_url": "https://api.crowdin.com"

"preserve_hierarchy": true

"files": [
  {
    "source": "pm00_01.txt.json",
    "translation": "pm00_01.txt.json",
    "scheme": "crowndefined",
    "crowdin_format": "custom_multilingual",
    "languages_mapping": {
      "two_letters_code": {
        "ja": "ja",
        "es-ES": "es-ES",
        "en": "en"
      }
    }
  }
]

file-to-import.json

{
  "filename": "pm00_01.txt",
  "labels": [
    "filename:pm00_01.txt"
  ],
  "texts": {
    "start": {
      "pm00_01-start.00": {
        "character": "voice-off",
        "text": "If the time I had to live was predetermined, \\nI wonder how I would handle it.",
        "translations": {
          "ja": {
            "text": "もし、自分の命の時間が、あらかじめ決まっていたとしたら、俺ならどう受け止めるだろう。",
            "status": "approved"
          },
          "en": {
            "text": "If the time I had to live was predetermined, \\nI wonder how I would handle it.",
            "status": "approved"
          },
          "es-ES": {
            "text": "abc",
            "status": "untranslated"
          }
        },
        "isHidden": false,
        "context": "Original Text: もし、自分の命の時間が、あらかじめ決まっていたとしたら、俺ならどう受け止めるだろう。",
        "labels": [
          "character:voice-off",
          "scene-type:default",
          "scene-label:start",
          "scene-title:「プロローグ」",
          "filename:pm00_01.txt"
        ],
        "customData": "character:voice-off"
      },
      "pm00_01-start.01": {
        "character": "voice-off",
        "text": "It was a winter evening. The arrival of spring was \\nstill far away.",
        "translations": {
          "ja": {
            "text": "まだ、春の訪れにはほど遠い、冬の日の夜。",
            "status": "approved"
          },
          "en": {
            "text": "It was a winter evening. The arrival of spring was \\nstill far away.",
            "status": "approved"
          },
          "es-ES": {
            "text": "texto prueba",
            "status": "untranslated"
          }
        },
        "isHidden": false,
        "context": "Original Text: まだ、春の訪れにはほど遠い、冬の日の夜。",
        "labels": [
          "character:voice-off",
          "scene-type:default",
          "scene-label:start",
          "scene-title:「プロローグ」",
          "filename:pm00_01.txt"
        ],
        "customData": "character:voice-off"
      },
      "pm00_01-start.02": {
        "character": "voice-off",
        "text": "The shopping mall was just about to close. \\nI managed to slip through the elevator door \\nright as it slid shut.",
        "translations": {
          "ja": {
            "text": "閉店間際のショッピングモール。ちょうど閉まりかけたそのエレベーターに、ほとんど滑り込むようにして乗り込んだのは――",
            "status": "approved"
          },
          "en": {
            "text": "The shopping mall was just about to close. \\nI managed to slip through the elevator door \\nright as it slid shut.",
            "status": "approved"
          },
          "es-ES": {
            "text": "texto prueba",
            "status": "untranslated"
          }
        },
        "isHidden": false,
        "context": "Original Text: 閉店間際のショッピングモール。ちょうど閉まりかけたそのエレベーターに、ほとんど滑り込むようにして乗り込んだのは――",
        "labels": [
          "character:voice-off",
          "scene-type:default",
          "scene-label:start",
          "scene-title:「プロローグ」",
          "filename:pm00_01.txt"
        ],
        "customData": "character:voice-off"
      }
    }
  }
}

CLI screenshots


image


Thank you Team!

Hi @enoshima.memo.team ,

Please kindly let us double-check it with the team and come back

Will keep you posted,

thanks a lot @Natalia ! :grinning_face:

@Natalia do you have any updates about the issue we reported?

thank you

Hi @enoshima.memo.team !

Let me please consult with my colleague on the matter. Once we have some news, we will inform you as soon as possible!

Let’s keep in touch!

hi @Tania , just wanted to confirm if you have any updates about our issue

thank you

Hi @enoshima.memo.team ! We are deeply sorry for the delay, but we are still investigating everything. Let me please speed up the process!

Will keep you posted on the progress!

Hi @enoshima.memo.team ! can you please try uploading translations via Translations → Upload existing translations tab?

hi @Ira , sorry for my late reply

I tried again using the Translations -> Upload existing translations tab and it worked this time.

I had to upload the file twice (once per lang) but it worked smoothly without issues

I also tried with the crowdin CLI and it worked too. I used crowdin upload sources to load a new file and then crowdin upload translations and it uploaded the translations of both langs at the same time, since my languages_mapping configuration has all them listed

Thanks a lot @Ira and also to the Crowdin team! we can continue working on our project as we originally intended!

hi @Ira , I just remember something I forgot to post in the previous message

I don’t know if this is expected, but there are times that I reupload the translations (for example, after making some local changes first, then reupload them), but it’s not taking the new changes.

I don’t remember exactly, but I think that after the first upload, I made some changes using the UI, so the translations differed from the local files. I didn’t sync the local ones and made some local changes to the same strings, then upload the files. And none of the new local changes were updates

Is this the expected behavior with the translations? can I overwrite the same strings with a new crowdin upload translations if they were already changed in the UI?

btw, I used both the Translations -> Upload existing translations tab and crowdin CLI

thanks a lot!

Hi @enoshima.memo.team,

It’s not uncommon for previously uploaded translations (if they are approved) to take precedence over new uploads, especially if changes have been made through the UI.

To ensure that your new local changes are reflected in Crowdin, you might want to try the following steps:

  1. Make sure that all these options are checked when you upload your translations. This will allow the new translations to overwrite any existing suggestions in the system.

  2. If you’re using the CLI, ensure that your languages_mapping configuration is set up correctly to include all the languages you’re working with.

  3. After uploading, check the translation history for the affected strings to see if the new translations have been captured correctly.

If you continue to experience issues, please provide us with the exact steps you’re taking, along with any error messages or unexpected behaviors you encounter.

You can email us directly to support@crowdin.com and provide file samples + URL to the exact strings in Editor that have issues.

This will help us to better understand the problem and assist you further.

thanks a lo @Dima for the details and the clarification about the precedence.

I’ll try again following your recommendations, and if anything not expected happens I’ll contact you again for sure!

thanks a lot to you and the whole Team!