Crash on translate

[tmerten]

I just tried to run vue translation manager on a bigger project and received a stack trace. (more info below)

./node_modules/.bin/vue-translation-manager translate

<--- Last few GCs --->

[17890:0x104001c00]     1688 ms: Mark-sweep 1192.5 (1214.3) -> 1192.5 (1198.3) MB, 13.9 / 0.0 ms  (average mu = 0.937, current mu = 0.002) last resort GC in old space requested
[17890:0x104001c00]     1700 ms: Mark-sweep 1192.5 (1198.3) -> 1192.5 (1198.3) MB, 11.8 / 0.0 ms  (average mu = 0.874, current mu = 0.002) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x2df284c4fc7d]
Security context: 0x22367c38c491 <JSObject>
    1: indexOf [0x2236aa524e39](this=0x2236e12b0c01 <Very long string[620757201]>,0x2236bf4a61e1 <String[2]: {{>)
    2: extractTemplateExpression [0x2236e817d731] [/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/index.js:78] [bytecode=0x2236bf4be529 offset=11](this=0x223671a045a1 <JSGlobal Object>,0x2236e12b0c01 <Very long stri...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Writing Node.js report to file: report.20190321.123121.17890.001.json
Node.js report completed
 1: 0x100064183 node::Abort() [/usr/local/bin/node]
 2: 0x1000647f1 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x10017d12f v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x10017d0d0 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x100439f30 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x10044001a v8::internal::Heap::SetUp() [/usr/local/bin/node]
 7: 0x100420613 v8::internal::Factory::AllocateRawWithImmortalMap(int, v8::internal::PretenureFlag, v8::internal::Map*, v8::internal::AllocationAlignment) [/usr/local/bin/node]
 8: 0x100422175 v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/local/bin/node]
 9: 0x1004feeea v8::internal::String::SlowFlatten(v8::internal::Isolate*, v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/usr/local/bin/node]
10: 0x100519ede v8::internal::String::IndexOf(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>, v8::internal::Handle<v8::internal::String>, int) [/usr/local/bin/node]
11: 0x10063bc91 v8::internal::Runtime_StringIndexOfUnchecked(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x2df284c4fc7d
13: 0x2df284f2a9ce
Abort trap: 6

As the project is private I cannot paste/refer to the code, but a quick grep -R "\$t(" * | wc -l results in about 1200 calls (maybe missing some in plain .js files, but not much) with rather long keys currently (I wanted to setup vue translation manager to refactor this ;) ).

However, I was wondering if it makes sense to add an option to pass a path or a single .vue file to translate.
This way one would be able to translate only parts of the project (which is what I want to do most of the time).

[tmerten]

And already an update ;). If I run it with increased heap size I get:

NODE_OPTIONS=--max_old_space_size=4096 ./node_modules/.bin/vue-translation-manager translate src/views/overview/
/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/node_modules/yargs/yargs.js:1148
      else throw err
           ^

RangeError: Invalid string length
    at extractTemplateExpression (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/index.js:90:52)
    at checkTemplateExpression (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/index.js:101:20)
    at matches.map (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/index.js:121:27)
    at Array.map (<anonymous>)
    at TranslationManager.getStringsForComponent (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/index.js:120:33)
    at containsUntranslatedStrings (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/bin.js:268:25)
    at files.filter (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/bin.js:157:55)
    at Array.filter (<anonymous>)
    at launchInteractiveTranslationPrompt (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/bin.js:157:38)
    at Object.require.command [as handler] (/Users/thorsten/src/development/frontend/node_modules/vue-translation-manager/bin.js:20:5)

which might be due to our too long translation keys at some places (?).

[MaxGfeller]

Hey @tmerten, sorry for only responding now.

Could you try to prepare a small example that reproduces this problem? And how long are your translation keys anyway?

[tmerten]

Hi @MaxGfeller . Nevermind.
Unfortunately it is hard to make it reproducible in a small example. We have some dusty places where the translation keys are the English sentence (as VueI18N uses the key as default). I found keys of 100+ chars (including whitespaces) if that helps (however, we are working on getting rid of those as it is clearly not even close to a best practice ;) ).
Anyways, if I find some time, I can check if I can reproduce it in an empty project using some long lorem ipsum keys.

[freakypie]

Hey, i was able to replicate this. I did some debugging and i think sometimes it gets confused and then starts matching where the closing index is lower than the opening index. When this happens the text string starts to grow until it runs out of memory

I fixed it by hacking the code in place to check if the close index is less than open index

[tmerten]

@freakypie : That is great news, thanks for digging. And, hey, I am sorry for not putting more effort in from our side.
Meanwhile we also did clean up the project files a lot so I'll give it another go.