LecygneNoir
/
thea2Shattering_i18n_tools
1
0
Fork 0
Code Issues 0 Pull Requests 0 Releases 9 Wiki Activity
Labels Milestones
New Issue

#1 New lines disappear when uploading file to Zanata

Closed
opened 5 years ago by LecygneNoir · 5 comments
LecygneNoir commented 5 years ago
Owner

Hello,

When modules files are compiled to pot and uploaded to Zanata, it seems some new lines are deleted.

It implies they are still missing when compiling back the files to the game :-/

It does not happen with DES and UI files which are XML, only with the txt modules' files..

It could be a problem between the Windows char ISO and the UTF-8 translation when uploading to Zanata.

We need to check the full process to be sure when and why they disappear.

One exemple:

I am all on my own,Wounded by iron weapons and scarred by swords.I often see battle.I am tired of fighting.I do not expect to be allowed to retire from warfareBefore I am completely done for.At the wall of the city, I am knocked aboutAnd bitten again and again.Hard-edged things made by the blacksmith's hammer attack me.Each time I wait for something worse.I have never been able to find a doctor who could make me betterOr give me medicine made from herbs.Instead the sword gashes all over me grow bigger day and night.
Hello, When modules files are compiled to pot and uploaded to Zanata, it seems some new lines are deleted. It implies they are still missing when compiling back the files to the game :-/ It does not happen with DES and UI files which are XML, only with the txt modules' files.. It could be a problem between the Windows char ISO and the UTF-8 translation when uploading to Zanata. We need to check the full process to be sure when and why they disappear. One exemple: ``` I am all on my own,Wounded by iron weapons and scarred by swords.I often see battle.I am tired of fighting.I do not expect to be allowed to retire from warfareBefore I am completely done for.At the wall of the city, I am knocked aboutAnd bitten again and again.Hard-edged things made by the blacksmith's hammer attack me.Each time I wait for something worse.I have never been able to find a doctor who could make me betterOr give me medicine made from herbs.Instead the sword gashes all over me grow bigger day and night. ```
LecygneNoir commented 5 years ago
Poster
Owner

The application causing the problem is msguniq

It search for duplicate in the pot file then merge all duplicates, as pot does not allow duplicate string when translating.

Unfortunately, it also break existing end of line to reconstruct its own file...

I'll search for options and/or bugs regarding this!

The application causing the problem is `msguniq` It search for duplicate in the pot file then merge all duplicates, as pot does not allow duplicate string when translating. Unfortunately, it also break existing end of line to reconstruct its own file... I'll search for options and/or bugs regarding this!
LecygneNoir commented 5 years ago
Poster
Owner

Problem was not what I thought at first.

gettext standard do not use where the line break as breakline, eg:

"I am all on my own,"
"Wounded by iron weapons and scarred by swords."
"I often see battle."

does not mean at all that there are newline at the end of each sentences.

Indeed, everything outside " " is not interpreted and purely text formating...

Thus I need to add newline inside the " " directly when extracting from txt to be able to preserve newline in translation.

Problem was not what I thought at first. gettext standard do not use where the line break as breakline, eg: ``` "I am all on my own," "Wounded by iron weapons and scarred by swords." "I often see battle." ``` does not mean at all that there are newline at the end of each sentences. Indeed, everything outside `" "` is not interpreted and purely text formating... Thus I need to add newline inside the `" "` directly when extracting from txt to be able to preserve newline in translation.
LecygneNoir commented 5 years ago
Poster
Owner

commit 60c281a14f add newlines in the multilines story, but it creates another problem.

When uploading newly generated files to Zanata, it break a large part of already done translation 😭

Indeed newline changes dramatically the syntax of source text, thus Zanata consider it as a modified text, and do not copy existing translation...

It's very similar to bug #2 so I think similar solution should work, I'll test on Zanata debug-version

commit 60c281a14f63172be98e68c288147e04d6f7b050 add newlines in the multilines story, but it creates another problem. When uploading newly generated files to Zanata, it break a large part of already done translation :sob: Indeed newline changes dramatically the syntax of source text, thus Zanata consider it as a modified text, and do not copy existing translation... It's very similar to bug #2 so I think similar solution should work, I'll test on Zanata debug-version
LecygneNoir commented 5 years ago
Poster
Owner

Okayyy, there was another bug which insert spaces at the end of lines (before the last ")...

It's also patched by the previous fix, I am unsure from where it comes, but the fact is it does help Zanata to recognize old translations :-/

Perhaps I'll need to do some manual action for the next version, then it should be definitively patched.

Okayyy, there was another bug which insert spaces at the end of lines (before the last `"`)... It's also patched by the previous fix, I am unsure from where it comes, but the fact is it does help Zanata to recognize old translations :-/ Perhaps I'll need to do some manual action for the next version, then it should be definitively patched.
LecygneNoir commented 5 years ago
Poster
Owner

After tests inside Zanata and some other little fix, the extract_module.sh script is patched.

I also add escapement for double quota that break pot sometimes.

Fixed in f02f8da1a8

After tests inside Zanata and some other little fix, the extract_module.sh script is patched. I also add escapement for double quota that break pot sometimes. Fixed in f02f8da1a8dca4c7ef124df93001a267a13cf858
LecygneNoir closed this issue 5 years ago
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Apply labels
Clear labels
No items
No Label
Milestone
Set milestone
Clear milestone
No items
No Milestone
Assignees
Assign users
Clear assignees
No Assignees
1 Participants
Notifications
Due Date

No due date set.

Dependencies

This issue currently doesn't have any dependencies.

Write Preview
Loading…
Cancel
Save
There is no content yet.
Delete Branch '%!s(MISSING)'

Deleting a branch is permanent. It CANNOT be undone. Continue?

No
Yes