The latest Studio update doesn't recognise Icelandic characters as being in the same word, i.e. as forming a word together with the surrounding letters

Question

Since I installed the latest Studio 2019 update, Studio doesn't recognise source language terms which have some of the special Icelandic letters/characters in them (including at least &aacute;, &eacute;, &iacute;, &oacute; and &ouml;). Note that such letters are very common in Icelandic words, but I have been using Trados and later Studio for nearly twenty years without this problem every occurring. Now these characters suddenly aren't perceived right by Term Recognition, so I have to remember and type the whole target word or phrase myself, or even look it up in Multiterm since it doesn't appear in Term Recognition. 
 This is a huge pain in the neck and costs a lot of time on each translation. In fact, if I copy such a word out of Studio to text boxes of some other programs, for instance a digital dictionary I use considerably, the characters don't copy properly, so that I for instance get ste&acute;ttarfe&acute;lag instead of the properly spelled word, stéttarfélag. 
 I myself saw this as an installation problem, since it started with the last Studio update, but when I wrote to SDL, the answer was to say this was not an installation problem. It it's not, it's at least a mistake in the SDL update that needs fixing very quickly, and I don't at all agree that I am supposed to pay SDL to fix a problem that their change in Studio has caused. 
 Do you have a quick fix? I've already tried repairing the installation and of course restarting and the simple methods!

Oana Nagy · Answer

Hello Lingua Norðan Jökuls , 
 Could you email me please (swhale@sdl.com) the details of who you spoke to in support about this as I cannot see any recent support tickets logged and I will follow up on this for you.

Daniel Hug · Answer

Hi Lingua Norðan Jökuls 
 I am also using Studio 2019 SR1 CU3, but I can't reproduce your issue:

Is your Multiterm the latest version? (There was a Multiterm update at the same time as the latest Studio update.) 
 I can confirm that Studio does not mark the whole word when you click on it: . 
 Although it does mark whole words containing German umlauts: . 
 But when I mark and copy the text, it copies as one word into other applications like Notepad++ and the Linguee site:

However, I do notice you used an unusual character to get the accent onto the "e", and I think that's at least part of the problem: The "usual" way is to use the "Latin small letter e with acute", Unicode U+00E9. You used the letter e (U+0065) followed by a combining acute accent (U+0301). It displays the same, but in the file it looks really different: 
 
 If you look at how these two same-looking words are actually represented: 
 
 What's the remedy? I don't speak Icelandic, but is it possible that your source files should not contain the strange "e" plus combining accent acute and that they should contain the normal "e plus acute" instead? (and all the other characters that cause problems - I am sure they are all concatenations of normal letters with some diacritical mark). 
 If they are not necessary, you could replace them in the source text? (It works with Notepad++.) 
 I think your issue is not caused by Studio, but by your source text. 
 Hope this helps. 
 Daniel

Daniel Hug · Answer

Hello and Lingua Norðan Jökuls 
 allow me to add that if your Termbase contains the word stéttarfélag in the spelling with "e plus acute" (U+00E9) and the source text contains the odd way of encoding it, the term will not be recognized. 
 
 But ... termbase search does find both. That is really confusing and shows that these two functions treat unicode differently: 
 
 If you have the word in Multiterm in one encoding and want to add the other one, you get a duplicate message, so that part of Multiterm equalizes both ways of encoding these characters. The Term Recognition does not. I guess you can argue both ways regarding which is the right way to treat this case. This is such a special case that I find it hard to call this a bug, but it would be less confusing if Studio and Multiterm had one way of treating it throughout, not two different ways. 
 
 Daniel 
 PS: Please forgive my total ignorance of Icelandic.

Lingua Norðan Jökuls · Answer

I had already checked a few days ago (and confirmed it once more now) that I had the latest version of both Studio and Multiterm. I am no longer sure exactly when and into what programs words with these characters weren't copying correctly out of Studio, but these words definitely didn't do so in the digital dictionary T&ouml;lvaor&eth;ab&oacute;kin. The trouble is that you have to pay a subscription to get access to it. I didn't mention it before, but Find is incapable of locating such words in Studio, unless I take only part of the word outside of such characters. I confirmed that now once more. Note that the Icelandic texts I have been translating are from a private party, and I have mostly been working on them since May - and in fact only on them since I performed the update. However, some passages in it are copied from legal acts, so I can show you one of them: 
 Kynfer&eth;isleg áreitni: Hvers kyns kynfer&eth;isleg heg&eth;un sem er í ó&thorn;ökk &thorn;ess sem fyrir henni ver&eth;ur og hefur &thorn;ann tilgang e&eth;a &thorn;au áhrif a&eth; misbjó&eth;a vir&eth;ingu vi&eth;komandi, einkum &thorn;egar heg&eth;unin lei&eth;ir til ógnandi, fjandsamlegra, ni&eth;url&aelig;gjandi, au&eth;mýkjandi e&eth;a mó&eth;gandi a&eth;st&aelig;&eth;na. Heg&eth;unin getur veri&eth; or&eth;bundin, táknr&aelig;n og/e&eth;a líkamleg. 
 Considering everything you say above, I now opened some texts from other Icelandic clients and Studio worked right on the characters in all of them, so I suppose you're correct about these current texts being the problem. Note however that it would be lots of trouble to replace all of the troublesome characters formed in this unfortunate way if I had to do it in many brief texts that come in the future. I wouldn't even know it was a problem until I started working on them, and batch texts to analyse (in cases of fixed price offers) and pre-translate wouldn't work entirely right, because Studio doesn't recognise enough of the words in some short passages. 
 The Icelandic looks perfectly natural and correct in all the texts in this major project, other than that it doesn't work right in the current Studio update. The same texts were working just fine in Studio until this update. Therefore, I am certainly unwilling to ask any clients to change their programs and send me different texts. 
 The current text is the last one in this project, and I'm nearly finished with it. Hopefully, none processed in this way will arrive from that client (who is one of long and faithful standing) or any other clients in the near future. What I am mainly hoping, therefore, is that Studio technicians will manage to find a way to avoid this problem and can include that repair in the next update from SDL.

Daniel Hug · Answer

Lingua Norðan Jökuls 
 It works you install CleanUp Tasks, a free app from the SDL AppStore: https://appstore.sdl.com/language/app/cleanup-tasks/963/ 
 Then you prepare your project as normal, but before you open the files for translation, you select them in the files view and choose "Batch tasks" -> "Cleanup Source". I always have to enlarge the window a bit to see everything (it does not scroll), but you deselect Lock Segments and Remove Tags and select Use Conversions. 
 Now you can create a new file and enter the character conversions you need. I tried this and here is the settings file, which you can load by clicking "Add". 
 odd_encoding_fix.zip 
 This is how it looks: 
 
 You an add search/replace pairs as you need them. In future, all you need to do is to run the task, you don't have to touch the settings again once they are complete. 
 Daniel

Trados Studio > 1. Trados Studio

The latest Studio update doesn't recognise Icelandic characters as being in the same word, i.e. as forming a word together with the surrounding letters

Top Replies