Sarah D 

The message seems to be saying "you have TUs in your file that are not nl-BE".  The implications being that your project in Extract is nl-BE.  So if you change all your codes to something other than nl-BE you'll make things worse, not better.

Perhaps you can confirm the languages of your Extract project, and then confirm the languages in your original TMX?

Thank you for your reply Paul.

I'm not sure what you mean? The target language I used for the extract project was nl-BE, given that the tmx target language was also nl-BE.

 Sarah D 

ok -so read what you have written again.  I can only go on what you're telling us.

Sarah D said:

"Warning! There were translation units (tu) found in file (name of my TMX file) which did not have a translation unit variant (tuv) with target language "Dutch (Belgium)"

It says it found TUs in your file that did not have nl-BE language codes.

You then said this:

Sarah D said:

I was able to change all target tuvs from nl-BE to nl, but that did not help. Changing them to nl-NL did not help either.

Why?  If the TMX contained TUs that were not nl-BE then you have made certain there aren't any at all.

Or of course I'm completely misunderstanding what you have written here?

Maybe you can share your TMX so I can try it?

 Sarah D 

ok -so read what you have written again.  I can only go on what you're telling us.

Sarah D said:

"Warning! There were translation units (tu) found in file (name of my TMX file) which did not have a translation unit variant (tuv) with target language "Dutch (Belgium)"

It says it found TUs in your file that did not have nl-BE language codes.

You then said this:

Sarah D said:

I was able to change all target tuvs from nl-BE to nl, but that did not help. Changing them to nl-NL did not help either.

Why?  If the TMX contained TUs that were not nl-BE then you have made certain there aren't any at all.

Or of course I'm completely misunderstanding what you have written here?

Maybe you can share your TMX so I can try it?

I realise it's contradictory, but it was the solution that was suggested in the only Q&A I found, to the same problem I had, just with a different target language.

I would be grateful if you could give it a try. What would be a safe way to share my tmx to you privately?

Sarah D 

Sure... pfilkin at sdl dot com

Thank you! I sent you the export yesterday by e-mail.

Sarah D 

Thanks for the file... found some time to look at it this evening.  I get the same error you mentioned, but the application still extracts the terminology:

So, I split the file and kept splitting it until I found the problem.  It relates to the segment contents rather than the language codes so this was a misleading error message.  The segment (<seg>) for both the fr-BE and nl-BE languages contains placeholders (<ph>, <bpt>, and <ept>) but no actual translatable content in one single <tu>. The placeholders refer to certain tag formats or structures that were present in the original document.  If you're extracting terminology, the actual terms or sentences should be inside the <seg> tags and since this file you provided has a single TU with only placeholders and no content Extract seems to get a bit upset!

This is the offending translation unit variant:

      <tuv xml:lang="fr-BE">
        <seg><bpt i="1" type="110" x="1" /><ph x="2" type="114" /><ept i="1" /></seg>
      </tuv>
      <tuv xml:lang="nl-BE">
        <seg><bpt i="1" type="110" x="1" /><ph x="2" type="114" /><ept i="1" /></seg>
      </tuv>

If you search for that and delete the entire translation unit and then try again the error is not there.

Sarah D 

Just to close this off... you can use this regex to quickly find segments like this... also segments with no content at all... and that should save some time trying to find the offending segments:

<seg>(?:\s*<(bpt|ept|ph)[^>]*\/?>\s*)*<\/seg>

Hi Paul

Thank you for your detective work!

You are right, the extraction works despite the bug on the small file (207 kb) with 161 TUs.

Hoewever it doesn't work on our actual database of +265.000 KB and +200.000 TUs I'm actually trying to extract from.

The regex you provided works, but only to find just one faulty TU at a time.

Do you know of a regex that could erase the faulty TU's in our big extraction file from <tu to </tu>?

Sarah D 

Sarah D said:

Do you know of a regex that could erase the faulty TU's in our big extraction file from <tu to </tu>?

Do I know of one... no ;-) Can I create one... yes.  But it may not work for all files as it could be dependent on the specific format of your TMX files.  So based on the sample you sent me this seems to work.

Search for this:

<tu[^>]*>(?:(?!<\/tu>).)*(<seg>(?:\s*<(?:bpt|ept|ph)[^>]*\/?>\s*)*<\/seg>|<seg>\s*<\/seg>)(?:(?!<\/tu>).)*<\/tu>

Replace with nothing!

Hm.. We're not quite there yet ...

I tried the string (in Notepad++ I have to add, search mode Regular expression), and 0 results were found.

That shouldn't be possible as it includes the tu of the sample I sent you.

Do you have any more ideas?

Sarah D 

It's often troublesome to use something without fully understanding it!  Make sure you check the "dot matches newline" checkbox... seems to work just fine:

That may be, but I'm determined to find a solution, so willing to learn trying to understand!

I'm getting this error message now: