The beauty of the way the localization is designed is exactly that if you do it the right way (add to the bottom *or* maintain your own numbering), there are no bad consequences of some translators not having updated their translations immedately, the worst that will happen is that new strings which are not present in the old catalogs just fall back to the built-in (english) version. But when strings are inserted, the subsequent strings get their IDs shifted, so the wrong translation will be picked, and you might e.g. get an important requester where the OK button says Cancel (in that language) so the user presses the wrong button at the wrong time.
Something like this happened with AmiUpdate a couple of years ago, and users had to be helped for a long time afterwards (essentially they had to switch to English in Locale prefs and update both AmiUpdate and the catalogs before switching back to their own language again).
@asymetrix:
Quote:
maybe have a universal catalogue file for all software to use
I can see you are from an English-speaking country, and maybe you are not that familiar with the problems of translating, but I can tell you that this approach is really not good, because the same word in English in different contexts can have very different counterparts in other languages. This is partly the same problem that makes Google translations so bad.
But when strings are inserted, the subsequent strings get their IDs shifted, so the wrong translation will be picked, and you might e.g. get an important requester where the OK button says Cancel (in that language) so the user presses the wrong button at the wrong time.
If i insert things on the same place (not at bottom) and in CD and in all .CT files and rebuild all Catalogs , all should be fine then and without issues, right ?
One caveat still: which order does OS 4.1 locale.library look for catalogs, is it just PROGDIR:Catalogs first, and if so and there is a duplicate what happens? Because the old convention was to use native language names instead of english language names. Yet there are version numbers as well.
If i insert things on the same place (not at bottom) and in CD and in all .CT files and rebuild all Catalogs , all should be fine then and without issues, right ?
Yes, assuming you have decided to add strings in the middle and not at the bottom. If so, you have to follow this schede: any time you add a new "MSG_xxx" line in your owb.cd, you have to add the same line in any .ct files too, otherwise the old and not updated catalogs will not be able to pick the ordering of the strings correctly.
There is no need you fills both lines (MSG_xxx and the translation line), you may even leave the second line (the translation line) blank, and only filling the first "MSG_xxx" line and then putting an extra space below ..
Translators will then be able to fill this space with their own translation later ... but if they do NOT, their old catalogs will continue to work as before
An example is quicker than any other words, let's say you have this situation:
owb.cd
Quote:
MSG_MENU_PROJECT Project ; MSG_MENU_NEW_WINDOW New Window... ; MSG_MENU_NEW_TAB New Tab ;
And in italian.ct we have this translation
Quote:
MSG_MENU_PROJECT Progetto ; MSG_MENU_NEW_WINDOW Nuova finestra... ; MSG_MENU_NEW_TAB Nuova scheda ;
This italian translation is 100% correct, when you open owb in italian all 3 italian strings will be placed perfectly "replacing" the 3 corresponding english one
But now what happen if you in the first or in the second line of your .cd you added something new ? Like this:
Quote:
MSG_MENU_PROJECT Project ; MSG_MENU_ICONIFY Iconify ; MSG_MENU_NEW_WINDOW New Window... ; MSG_MENU_NEW_TAB New Tab ;
Now we have added an iconify option at second line, right ? All correct for english, when you run owb with the default english language the strings will be perfectly fine ... but what happens if you now open owb in italian using the old and not updated italian catalog ?
It happens that the italian strings will messed up completely, because in the "id" line of where you added the new iconify, italian catalog instead will search and then will use the "New Window..." line instead of the iconify one
So what can you do in this situation to make also italian working ?
You have 2 solutions:
1 - Or just wait until someone will add that new italian lines according to your latest modification in the .cd catalog descriptor 2 - Or just add manually on your own the corresponding "MSG_MENU_ICONIFY" lines into the italian.ct (and the same for all the other .ct files we have)
As we said earlier, you can avoid completely this boring and manual work by just adding all that new strings at the end of the cd file .. in this way there is zero risk to messed up the other old catalogs, and there is no necessity to wait until other translators will update their translations according to your latest modifications in .cd file ... not found strings in catalogs will just pick and use the english strings from your newest addition in .cd catalog descriptor
Edited by samo79 on 2020/4/12 9:19:24 Edited by samo79 on 2020/4/12 9:23:22 Edited by samo79 on 2020/4/12 9:25:16 Edited by samo79 on 2020/4/12 9:26:35
One caveat still: which order does OS 4.1 locale.library look for catalogs, is it just PROGDIR:Catalogs first, and if so and there is a duplicate what happens? Because the old convention was to use native language names instead of english language names. Yet there are version numbers as well.
If i remember correctly locale in AmigaOS4 will search for: "PROGDIR:Catalogs" path first Then if not found it, it will automatically search in Locale/Catalogs/Languages
About folder names: yes OS4 by default prefer the english naming scheme for the languages, for example:
"italian" instead of "italiano" "german" instead of "deutsch" etc.
The old scheme was typical of the old AmigaOS3.x era, but also MorphOS and AROS still followed this scheme
OS4 developers decided to change this by using an english naming scheme by default, however we still mantain the compatibility with the old naming scheme so everything still working even on OS4
@samo There wasn't needs for such big post to explain obvious like 2+2 = 4 :) its all pretty clear that things shifts if you change only CD file and forgot to change all CTs and regenerate new catalogs from them.
Quote:
and there is no necessity to wait until other translators will update their translations according to your latest modifications in .cd file ... not found strings in catalogs will just pick and use the english strings from your newest addition in .cd catalog descriptor
With insterting things to any place , but the same place for CD and all CTs and regenerated Catalogs with new strings being in english, there is also nothing to wait. They the same will pick new entry in english. No differences.
@Thematic Same as with LIBS probabaly : if found in Progdir:Catalogs, then used firstly, if not , then system's Catalogs.
I wish there was detailed information about the conversion process and how AmigaOS deals with them.
Do you have any links with problems of translating etc.
@kasie
Why do we need 'inserting things to any place', why not just have English tags to search for ?
If you are searching for Italian 'MSG_MENU_NEW_WINDOW' its either in the old Italian file or its not.
If you match 'MSG_MENU_NEW_WINDOW' , then just get its Italian translation with next read.
If any new message TAG does not exist in an old file, read the version of the file in use, look up on central server if a new version of the file exists, download and use, otherwise notify of a missing message Tag.
Obviously you will need different code to read data looking for the message Tags in serial, having the files temporarily in memory could speed up the search.
If you really don't want to read in serial then have a count of the tags at the beginning of the file, then follow a list of all Tag IDs in the file.
A quick match would know if it exists or not.
Just throwing some thoughts to topic I have no experience with...
Flexcat will update CT files with new entries in the CD file. The new entries are placed in the same position in the CT files as they occur in the CD file. No need to add entries at the bottom of the CD file. The new entries are added to the CT files and marked as ***NEW*** so a translator can simply search in a good editor and add translations at the ***NEW*** locations.
Flexcat will update CT files with new entries in the CD file. The new entries are placed in the same position in the CT files as they occur in the CD file. No need to add entries at the bottom of the CD file.
Wrong.
Precisely because the entries are placed in the same place in the .ct as in the .cd, you cannot insert new strings in the middle without the following ones shifting their IDs, causing a mess when the program is used with old catalogs. (Unless you either keep all entries explicitly numbered, or do it the kas1e way).
Quote:
If you update the version in the CD file, it should be copied to the CT files so that old catalogs won't be loaded by a program.
Also wrong.
The #version in a .cd file (which BTW is only a version, without a revision) serves to tell Locale that only .catalogs with that specific version (and any revision) are allowed to be used. The version.revision in a .ct file (which is a regular AmigaOS version tag, i.e. $VER: etc.) is the translator's own tool to keep track of the translations, except as I wrote earlier that the version part (the number in front of the dot) must be the one required in the .cd file, if any, i.e. neither lower nor higher. The version.revision of the .ct file is carried over into the catalog.
Edit: Maybe I am reading your statement wrong. If you mean to say that the translator (or the author) must make sure that the version of the .cd file is also used as the first part of the version.revision pair in the .ct file, then I agree. The above is written because I read your explanation as the compiler (CatComp) would automatically copy the version from the .cd to the .ct, which it won't.
I wish there was detailed information about the conversion process and how AmigaOS deals with them.
I agree, it's hard to find.
Quote:
Do you have any links with problems of translating etc.
There are some documents about the Locale system in the SYS:Documentation drawer, mostly about fonts, keyboards, charsets etc., and there is also the doc file for CatComp, which deals with a lot of this stuff as well.
But it would be useful to have something on the Wiki, for instance, which describes the whole process, the rules/recommendations and the tools.
Quote:
Why do we need 'inserting things to any place', why not just have English tags to search for ?
If you are searching for Italian 'MSG_MENU_NEW_WINDOW' its either in the old Italian file or its not.
If you match 'MSG_MENU_NEW_WINDOW' , then just get its Italian translation with next read.
If any new message TAG does not exist in an old file, read the version of the file in use, look up on central server if a new version of the file exists, download and use, otherwise notify of a missing message Tag.
Obviously you will need different code to read data looking for the message Tags in serial, having the files temporarily in memory could speed up the search.
If you really don't want to read in serial then have a count of the tags at the beginning of the file, then follow a list of all Tag IDs in the file.
A quick match would know if it exists or not.
Just throwing some thoughts to topic I have no experience with..
I'm not sure you have understood what Locale does.
It actually takes over all of the work you describe here, so as a programmer you only need to specify (instead of a literal string) that you want the catalog string with a certain ID, e.g. MSG_ABOUT, and Locale will then supply that string in the language the user has set in Locale prefs. If the string has not been provided in the catalog (e.g. because the translator for that language has not yet updated the translation), Locale will fall back to supplying the same string in another language (if the user has more than one preferred language) or in English - the built in default - as a last resort. The program(mer) doesn't need to search for anything.
With insterting things to any place , but the same place for CD and all CTs and regenerated Catalogs with new strings being in english, there is also nothing to wait. They the same will pick new entry in english. No differences.
Of course it can be done this way.
The reason I advise against it is that more or less all other projects with translators do it the other way, so translators are used to building their own new .ct files based on the new .cd plus the old .ct, as described by xenic in a later post. This can be done either with FlexCat or with CatComp, no (big) difference. In both cases it is very easy for the translator to see which strings need translation as they are empty (CatComp) or marked with ***NEW*** (FlexCat). The way you propose, you will already have the new strings set in the .ct file (to the English version), making them harder to spot for the translator.
And the way you propose is actually giving you extra work with no change in the result (except that you get to keep the related strings together in the .cd file, which seems important to you). In both cases, a string which hasn't been translated will output the English version, either because you neticuously placed it there, or because Locale automatically falls back to it when the translation was left empty.
you cannot insert new strings in the middle without the following ones shifting their IDs, causing a mess when the program is used with old catalogs. (
Probably Xenic mean there that catalogs of course will be rebuilded, and old ones will be not used of course (as the catalogs which will be used distributed with the programm and placed in Catalogs, from where they will be taken first, even if system have other one in sys:catalogs).
Another question, is if i need to change the line which already present, but should be changed. For example "_WebM (VP8) support " should be changed to "_WebM (VP8,VP9) support". Now, if i didn't rebuild catalogs from CT files, then chances are 99% that any translator later will see nothing and will translate nothing, and they always will have older version. Instead, if i myself will create CT files from current catalogs, then i can everywhere in translated strings for any language add that small change, and rebuild catalogs back, so every locale will have that change.
In other words, if i need to change a string which already present, and want it to be in all locales : i need then rebuild all catalogs. Then why not put new entries i need in any place (and not in the bottom), if i anyway have needs to create CT from current catalogs , and rebuild catalogs back if i want to change presented strings a little (and note in release notes what string were changed)
Besides, i think better to release then CT files as well with programm, so only those CT files will be used for catalogs, and no other ones which only tranlators have (and with any change, CT files can be updated and send back, same as catalogs anyway, which only will be better : other tranlators can work on that CTs and CTs will not holds somewhere on only one tranlator's hdd)
If you only plan to "update" some words in strings, maybe you can considere "old" ones as typos and fix'em on the "real" string/line and the do a rebuild. Just make somewhere a warning/notice that .CD is updated (and strings if possible) then translators can update them too.
Like: "_WebM (VP8) support " should be changed to "_WebM (VP8,VP9) support"
I myself udpated such string in spansh .CT a few weeks ago (when you added/fixed webp/webm support): .. MSG_PREFSWINDOW_MEDIA_USE_WEBM Soporte _WebM (VP8/VP9) ; _WebM (VP8) support ..
@all Is it possible to make a check like "no older catalogs allowed" ? I.e. i will create new ones, with some the same version/revision, and i need to be sure, that nothing less (meaning older catalogs) can be loaded. Should be some special code done about, or its all can be handled on the level of catalogs VER$ and co ?
Why make life even more complicated, just don't include old catalogs in release archive, and then step by step re-add them when translators updates them
@samo79 I fear there can be some smart ones who will by any of reassons use old catalogs. Or, when install new beta didn't overwrite catalogs directory, even if i will wrote that in readme, and we will have reports about all sort of issues caused by that and loosing time on, until those ones will say "ops, i forget to copy new catalogs, was it necessary ?"
Yep, however in the worst case the unique problem we can get will be a messing of the localized strings, annoying maybe, but not a big issue If you add a mandatory version number to follow for the catalogs, then you will force us to an unecessary extra work to follow everytime a specific version release
@nbache Maybe I wasn't explicit enough. With Flexcat you would do the following:
Add a new entry in the middle of the CD file (.cd). Bump the version number in the CD file (#version not the AmigaDOS $VER). Update the CT (.ct) files with Flexcat. Use Flexcat to produce a new command file (.c) and header file (.h). Use Flexcat to produce a blank CT file for translators. Compile the CT files to catalogs with Flexcat. Compile the command file and header file with the main program.
I do all the above in a makefile.
If you don't bump the #version in the CD file then you are right that old catalogs won't work properly. The entire catalog creation should probably be done with Flexcat if you are going to add new entries in the middle of the CD file somewhere.
My only problem with Flexcat is that more recent versions get picky with elipsus (...) warnings and return a warning code (5) which causes the OS4 make command to stop the compile. As a result, if you are using Flexcat in a makefile then you need to preceed the Flexcat command with a dash (e.g. -Flexcat) so make won't stop the whole compile.
Amiga X1000 with 2GB memory & OS 4.1FE + Radeon HD 5450
News feed
OS4 Feedback forum
OS4Depot Feedback forum
Amigabounty forum
OpenAmiga forum
AmiCygnix forum
ABC shell forum
AmiKit forum
Cinnamon Writer forum
CodeBench forum
Digital Universe forum
Dopus 5 forum
E-UAE forum
Gnash forum
Ibrowse forum
JAmiga forum
Odyssey forum
OWB forum
Qt forum
SmartFileSystem forum
Timberwolf forum
TouchDevice forum
TuneNet forum
Unsatisfactory Software forum
Donate
and fix'em on the "real" string/line and the do a rebuild.
