Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Modern javascript localization with c-3po and the good old gettext

484 views

Published on

Gettext format overview. Javascript localization with c-3po.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Modern javascript localization with c-3po and the good old gettext

  1. 1. Modern javascript localization with good old gettext https://c-3po.js.org/
  2. 2. Mostovenko Alexander Company: EVO company Twitter: @MostovenkoA Github: github.com/AlexMost Me
  3. 3. About this talk
  4. 4. About this talk Localization process in general and more about gettext format.
  5. 5. About this talk Localization process in general and more about gettext format. What is c-3po and how it helped us to improve existing process?
  6. 6. About this talk Localization process in general and more about gettext format. What is c-3po and how it helped us to improve existing process? Continuous translations.
  7. 7. Our initial tasks
  8. 8. Our initial tasks 1. Localize frontend on several projects: (cabinet, company sites) Bigl.ua others...
  9. 9. Our initial tasks 1. Localize frontend on several projects: (cabinet, company sites) Bigl.ua others ... 1. Setup translation process when devs and translators can work independently.
  10. 10. Gettext (.po) ICU ... Localization formats
  11. 11. ICU - International Components for Unicode http://userguide.icu-project.org/
  12. 12. ICU - International Components for Unicode http://userguide.icu-project.org/ ICU features >>> gettext features
  13. 13. ICU is cool, but ...
  14. 14. ICU is cool, but ... 1. Poor tooling (editors for translators e.t.c).
  15. 15. ICU is cool, but ... 1. Poor tooling (editors for translators e.t.c). 2. Our backend already used gettext.
  16. 16. ICU - cool but ... "{gender_of_host, select, " "female {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to her party.}" "=2 {{host} invites {guest} and one other person to her party.}" "other {{host} invites {guest} and # other people to her party.}}}" "male {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to his party.}" "=2 {{host} invites {guest} and one other person to his party.}" "other {{host} invites {guest} and # other people to his party.}}}" "other {" "{num_guests, plural, offset:1 " "=0 {{host} does not give a party.}" "=1 {{host} invites {guest} to their party.}" "=2 {{host} invites {guest} and one other person to their party.}" "other {{host} invites {guest} and # other people to their party.}}}}"
  17. 17. GNU gettext https://www.gnu.org/software/gettext/
  18. 18. GNU gettext https://www.gnu.org/software/gettext/ 1. Simple format.
  19. 19. GNU gettext https://www.gnu.org/software/gettext/ 1. Simple format. 2. Has good ecosystem for translation process.
  20. 20. gettext idea
  21. 21. ‘Hello world’ Source code gettext idea
  22. 22. gettext(‘Hello world’) Source code gettext idea
  23. 23. gettext(‘Hello world’) Source code ‘Hello world’ .pot file gettext idea
  24. 24. gettext(‘Hello world’) Source code msgid: ‘Hello world’ msgstr: ‘’ .po file ‘Hello world’ .pot file gettext idea
  25. 25. gettext(‘Hello world’) Source code msgid: ‘Hello world’ msgstr: ‘Здоровеньки були’ .po file ‘Hello world’ .pot file gettext idea
  26. 26. gettext(‘Hello world’) Source code msgid: ‘Hello world’ msgstr: ‘Здоровеньки були’ .po file ‘Hello world’ .pot file gettext idea
  27. 27. gettext workflow Code .pot Extracting translation strings from code (AST traverse)extract
  28. 28. gettext workflow Code .pot Extracting translation strings from code (AST traverse) .pot .po Merge new extracted translations from .pot file with existing. msgmerge extract merge .po.po
  29. 29. gettext workflow Code .pot Extracting translation strings from code (AST traverse) .pot .po Merge new extracted translations from .pot file with existing. msgmerge extract merge Translator adds translations translation .po.po .po.po.po .po.po.po
  30. 30. gettext workflow Code .pot Extracting translation strings from code (AST traverse) .pot .po Merge new extracted translations from .pot file with existing. msgmerge extract merge Translator adds translations translation .po.po .po.po.po .po.po.po resolve Code Placing translations back to the code (by locale).po.po.po
  31. 31. .po file structure
  32. 32. Example .po file msgid "" msgstr "" "Project-Id-Version: c 3po-webpack-startn" "PO-Revision-Date: 2017-02-01 22:22+0200n" "Language-Team: Ukrainiann" "Language: ukn" "MIME-Version: 1.0n" "Content-Type: text/plain; charset=UTF-8n" "Content-Transfer-Encoding: 8bitn" "Plural-Forms: nplurals=2; plural=(n!=1);n" #: app.js:6 msgid "Choose locale" msgstr "Оберіть локаль" #: app.js:10 msgid "${ 0 } second" msgid_plural "${ 0 } seconds" msgstr[0] "${ 0 } секунда" msgstr[1] "${ 0 } секунди" msgstr[2] "${ 0 } секунд" #: app.js:14 msgid "webpack with c-3po localization demo" msgstr "Демо локалізації з c-3po та webpack"
  33. 33. msgid "" msgstr "" "Project-Id-Version: c 3po-webpack-startn" "PO-Revision-Date: 2017-02-01 22:22+0200n" "Language-Team: Ukrainiann" "Language: ukn" "MIME-Version: 1.0n" "Content-Type: text/plain; charset=UTF-8n" "Content-Transfer-Encoding: 8bitn" "Plural-Forms: nplurals=2; plural=(n!=1);n" #: app.js:6 msgid "Choose locale" msgstr "Оберіть локаль" #: app.js:10 msgid "${ 0 } second" msgid_plural "${ 0 } seconds" msgstr[0] "${ 0 } секунда" msgstr[1] "${ 0 } секунди" msgstr[2] "${ 0 } секунд" #: app.js:14 msgid "webpack with c-3po localization demo" msgstr "Демо локалізації з c-3po та webpack" Headers
  34. 34. msgid "" msgstr "" "Project-Id-Version: c 3po-webpack-startn" "PO-Revision-Date: 2017-02-01 22:22+0200n" "Language-Team: Ukrainiann" "Language: ukn" "MIME-Version: 1.0n" "Content-Type: text/plain; charset=UTF-8n" "Content-Transfer-Encoding: 8bitn" "Plural-Forms: nplurals=2; plural=(n!=1);n" #: app.js:6 msgid "Choose locale" msgstr "Оберіть локаль" #: app.js:10 msgid "${ 0 } second" msgid_plural "${ 0 } seconds" msgstr[0] "${ 0 } секунда" msgstr[1] "${ 0 } секунди" msgstr[2] "${ 0 } секунд" #: app.js:14 msgid "webpack with c-3po localization demo" msgstr "Демо локалізації з c-3po та webpack" Translations
  35. 35. "Plural-Forms: nplurals=2; plural=(n!=1);n" Plural-forms header
  36. 36. "Plural-Forms: nplurals=2; plural=(n!=1);n" Plural-forms header Plurals count
  37. 37. "Plural-Forms: nplurals=2; plural=(n!=1);n" Plural-forms header Plurals formula
  38. 38. #: app.js:6 msgid "Choose locale" msgstr "Оберіть локаль" 1 to 1 translations
  39. 39. #: app.js:10 msgid "${ n } second" msgid_plural "${ n } seconds" msgstr[0] "${ n } секунда" msgstr[1] "${ n } секунди" msgstr[2] "${ n } секунд" Plural forms example
  40. 40. #. Notes for the translator #: app.js:6 #, flag msgid "Choose locale" msgstr "Оберіть локаль" Comments
  41. 41. msgctx "email" msgid "Hello" msgstr "Привіт" msgctx "main page" msgid "Hello" msgstr "Вітаємо" Context example
  42. 42. c-3po https://c-3po.js.org/
  43. 43. c-3po Code .pot Extracting translation strings from code (AST traverse)extract resolve Code Placing translations back to the code (by locale).po.po.po
  44. 44. Library: https://github.com/c-3po-org/c-3po Core functionality. Translation functions (tags): t, ngettext, gettext, jt Babel-plugin: https://github.com/c-3po-org/babel-plugin-c-3po Extract, Resolve, Validation
  45. 45. Key ideas
  46. 46. Key idea 1. Tagged template strings for formatting
  47. 47. c-3po translations are based on tagged template strings import { t } from 'c-3po' const name = 'Mike'; console.log(t`Hello ${name}`);
  48. 48. What is a template string?
  49. 49. What is template string? console.log('Hello ' + name) console.log(`Hello ${name}`)
  50. 50. What is a tagged template string? const name = ‘Mike’ t`Hello ${name}` function t(strs, ...exprs) { // strs -> [‘Hello’, ‘’] // exprs -> [‘Mike’] }
  51. 51. Valid javascript t `Hello ${name}` t(5)`Hello ${name}` t(5) `Hello ${name}`
  52. 52. Why just not simple functions but tags?
  53. 53. Why just not simple functions but tags? c-3po must work without transpilation
  54. 54. Why just not simple functions but tags? // Without babel transpile !!! const name = ‘Mike’ t(`Hello ${name}`) function t(str) { // strs -> ‘Hello Mike’ // no chance to get ‘Hello ${name}’ msgid here :( }
  55. 55. Existing solutions Force you to use sprintf formatting (i18next, jed e.t.c)
  56. 56. sprintf gettext("Hello %s", user) gettext("%2$s %3$s a %1$s", "cracker", "Polly", "wants") gettext("Current timestamp: %d", Date.now)
  57. 57. sprintf ● Alien formatting for js
  58. 58. sprintf ● Alien formatting for js ● No reason to use es6 template strings
  59. 59. sprintf ● Alien formatting for js ● No reason to use es6 template literals ● Extra CPU work on the client (sprintf parsing)
  60. 60. Key idea 2. Precompile translation on a build step
  61. 61. Traditional translations resolve flow
  62. 62. Traditional translations resolve flow Load assets to the client (browser)
  63. 63. Traditional translations resolve flow Load assets to the client (browser) Resolve locale (from cookie, params e.t.c)
  64. 64. Traditional translations resolve flow Load assets to the client (browser) Resolve locale (from cookie, params e.t.c) Fetch locale data
  65. 65. Traditional translations resolve flow Load assets to the client (browser) Resolve locale (from cookie, params e.t.c) Fetch locale data Apply translations
  66. 66. Wanted translations resolve flow
  67. 67. Wanted translations resolve flow Load assets to the client (browser)
  68. 68. Wanted translations resolve flow Load assets to the client (browser) That’s it, they are already localized !!!
  69. 69. Benefits of precompiled translations:
  70. 70. Benefits of precompiled translations: ● Smaller bundle size
  71. 71. Benefits of precompiled translations: ● Smaller bundle size ● Less work on the client
  72. 72. Benefits of precompiled translations: ● Smaller bundle size ● Less work on the client ● No wait until translations are loaded and applied
  73. 73. Key idea 3. Works with and without babel
  74. 74. Demo on jsfiddle
  75. 75. Key idea 4. Can extract and resolve translations from everything that works with babel (jsx)
  76. 76. Extract and resolve translations with simple config Extract: { "extract": { "output": "extract.pot" } } Resolve: { "resolve": { "translations": "uk.po" } } https://c-3po.js.org/quick-start.html
  77. 77. Key idea 5. Efficient dev and prod setup
  78. 78. Requirements for the dev setup
  79. 79. Requirements for the dev setup ● Faster builds
  80. 80. Requirements for the dev setup ● Faster builds ● Simple integration
  81. 81. Dev setup with c-3po
  82. 82. Dev setup with c-3po ● Use whole c-3po lib (import c-3po).
  83. 83. Dev setup with c-3po ● Use whole c-3po lib (import c-3po). ● Babel plugin only for extraction translations
  84. 84. Requirements for the prod setup
  85. 85. Requirements for the prod setup ● Smaller result assets
  86. 86. Requirements for the prod setup ● Smaller result assets ● Faster locale load
  87. 87. Prod setup with c-3po
  88. 88. Prod setup with c-3po ● Use c-3po mock lib (alias in webpack e.t.c).
  89. 89. Prod setup with c-3po ● Use c-3po mock lib (alias in webpack e.t.c). ● Babel plugin for transformations.
  90. 90. Prod setup with c-3po ● Use c-3po mock lib (alias in webpack e.t.c). ● Babel plugin for transformations. ● Separate build for each locale.
  91. 91. Key idea 6. Validate translation strings
  92. 92. Validation problem msgid "http://some.random.domain.link" msgstr ? msgid "" msgstr ? msgid "translate ${ test ? result1() : result2() }" msgstr ? msgid "5" msgstr ?
  93. 93. c-3po validation (empty str) t`` Module build failed: SyntaxError: Can not translate empty string 5 | 6 | > 7 | t`` | ^
  94. 94. c-3po validation (only numbers) t`3243243` ERROR in Error: Module build failed: SyntaxError: Can not translate '3243243' 2 | 3 | > 4 | t`3243243` | ^
  95. 95. c-3po validation (programs inside translations) t`Translate ${someVar ? callFn1() : callFn2()}` ERROR in Error: Module build failed: SyntaxError: You can not use ConditionalExpression '${someVar ? callFn1() : callFn2()}' in localized strings 2 | 3 | > 4 | t`Translate ${someVar ? callFn1() : callFn2()}`; | ^ 5 |
  96. 96. c-3po validation (only var wrap) t`${someVar}` ERROR in Error: Module build failed: SyntaxError: Can not translate '${ someVar }' 2 | 3 | > 4 | t`${someVar}`; | ^ msgid:’${someVar}’
  97. 97. Validate not translated strings
  98. 98. Validation (no translated str) t`non translated str` ERROR in Error: Module build failed: SyntaxError: No "non translated str" in "uk.po" file 2 | > 3 | t`non translated str`; https://c-3po.js.org/configuration.html#configresolveunresolved-string-one-of-fail-warn-skip
  99. 99. Key idea 7. Can use any default locale in sources (standard gettext uses only English locale)
  100. 100. The problem Standard ngettext accepts only 2 plural forms char * ngettext(const char * msgid1, const char * msgid2, unsigned long int n);
  101. 101. Ukrainian/Russian strings in sources How to be with plurals?
  102. 102. Ukrainian/Russian strings in sources How to be with plurals? Standard ngettext accepts only 2 plural forms?
  103. 103. Ukrainian/Russian strings in sources How to be with plurals? Standard ngettext accepts only 2 plural forms? What if we have more or less than 2 plural forms?
  104. 104. Ukrainian/Russian strings in sources How to be with plurals? Standard ngettext accepts only 2 plural forms? What if we have more or less than 2 plural forms?
  105. 105. ngettext for other locales ( plural forms > 2 )
  106. 106. ngettext for other locales ( plural forms > 2 ) Doc - https://c-3po.js.org/ngettext.html Change defaultHeaders setting to: 'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);'
  107. 107. ngettext for other locales ( plural forms > 2 ) Demo on jsfiddle Doc - https://c-3po.js.org/ngettext.html Change defaultHeaders setting to: 'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);' ngettext(msgid`${n} банан`, `${n} банана`, `${n} бананів`, n)
  108. 108. c-3po tags and functions
  109. 109. t - tag gettext Demo on jsfiddle Doc - https://c-3po.js.org/tag-gettext--t-.html
  110. 110. t - tag gettext t`Hello ${name}` < Usage Demo on jsfiddle Doc - https://c-3po.js.org/tag-gettext--t-.html
  111. 111. t - tag gettext t`Hello ${name}` msgid "Hello ${ name }" msgstr "Привіт ${ name }" < Usage < Extract Demo on jsfiddle Doc - https://c-3po.js.org/tag-gettext--t-.html
  112. 112. t - tag gettext t`Hello ${name}` msgid "Hello ${ name }" msgstr "Привіт ${ name }" < Usage < Extract `Привіт ${name}` < Resolve Demo on jsfiddle Doc - https://c-3po.js.org/tag-gettext--t-.html
  113. 113. ngettext - plural forms
  114. 114. ngettext - plural forms ngettext(msgid`${n} time clicked`, `${n} times clicked`, n) < Usage
  115. 115. ngettext - plural forms ngettext(msgid`${n} time clicked`, `${n} times clicked`, n) msgid "${ n } time clicked" msgid_plural "${ n } times clicked" msgstr[0] "${ n } time clicked [translated]" msgstr[1] "${ n } times clicked [translated]" < Usage < Extract
  116. 116. Forgot msgid ? ERROR in Error: Module build failed: SyntaxError: First argument must be tagged template expression. You should use 'msgid' tag 2 | 3 | > 4 | ngettext(`${hours} hour`, `${hours} hours`, hours); | ^ 5 | Demo on jsfiddle Doc - https://c-3po.js.org/ngettext.html
  117. 117. ngettext - resolve function _tag_ngettext(n, args) { return args[+(n != 1)]; } _tag_ngettext(n, [n + " time clicked [translated]", n + " times clicked [translated]"])); Demo on jsfiddle Doc - https://c-3po.js.org/ngettext.html
  118. 118. First contribution !!! jt (jsx-gettext)
  119. 119. First contribution !!! jt (jsx-gettext)
  120. 120. jt - jsx gettext
  121. 121. jt - jsx gettext const btn = <button key="btn">{ t`me` }</button>; <span>{jt`Click ${ btn }`}</span> < Usage
  122. 122. jt - jsx gettext const btn = <button key="btn">{ t`me` }</button>; <span>{jt`Click ${ btn }`}</span> < Usage < Extractmsgid "Click ${ btn }" msgstr "Click ${ btn } [translated]"
  123. 123. jt - jsx gettext const btn = <button key="btn">{ t`me` }</button>; <span>{jt`Click ${ btn }`}</span> < Usage < Extractmsgid "Click ${ btn }" msgstr "Click ${ btn } [translated]" ['Click ', btn, ' [translated]'] < Resolve
  124. 124. gettext
  125. 125. gettext gettext('simple gettext'); < Usage
  126. 126. gettext gettext('simple gettext'); < Usage < Extract msgid "simple gettext" msgstr "simple gettext [translated]"
  127. 127. gettext gettext('simple gettext'); < Usage < Extract msgid "simple gettext" msgstr "simple gettext [translated]" 'simple gettext [translated]' < Resolve
  128. 128. Why add gettext func if we have ‘t’ tag?
  129. 129. Legacy - *.coffee *.eco (100 K+ LOC) 1. Wrap string in gettext inside *.coffee *.eco 2. Add babel loader after coffee loader to the webpack pipeline 3. Profit !!!
  130. 130. Even more features
  131. 131. Aliasing https://c-3po.js.org/aliasing.html
  132. 132. Babel plugin will extract aliased translation import { t as i18n } from 'c-3po' i18n`this translation will work`
  133. 133. Multiline https://c-3po.js.org/multiline-strings.html
  134. 134. import { t } from 'c-3po'; function test(name) { return t`multi line string with multiple line breaks and with formatting ${name}` } Multiline (auto dedent)
  135. 135. Translator comments
  136. 136. Translator comments // translator: some description for the extracted string. t`some string`
  137. 137. Translator comments // translator: some description for the extracted string. t`some string` #. some description for the extracted string msgid "some string" msgstr ""
  138. 138. Summary
  139. 139. Summary ● Native js tagged template strings for formatting
  140. 140. Summary ● Native js tagged template strings for formatting ● Can precompile translation on a build step
  141. 141. Summary ● Native js tagged template strings for formatting ● Can precompile translation on a build step ● Works with and without babel
  142. 142. Summary ● Native js tagged template strings for formatting ● Can precompile translation on a build step ● Works with and without babel ● Extracts and resolves translations from everything that works with babel (jsx)
  143. 143. Summary ● Native js tagged template strings for formatting ● Can precompile translation on a build step ● Works with and without babel ● Extracts and resolves translations from everything that works with babel (jsx) ● Efficient dev and prod setup
  144. 144. Summary ● Native js tagged template strings for formatting ● Can precompile translation on a build step ● Works with and without babel ● Extracts and resolves translations from everything that works with babel (jsx) ● Efficient dev and prod setup ● Validation for translated strings
  145. 145. Summary ● Native js tagged template strings for formatting ● Can precompile translation on a build step ● Works with and without babel ● Extracts and resolves translations from everything that works with babel (jsx) ● Efficient dev and prod setup ● Validation for translated strings ● Can use any default locale in sources (not only English)
  146. 146. Continuous translations process
  147. 147. Devs VCS repo Continuous translation process Translator
  148. 148. Devs VCS repo Continuous translation process push Translator
  149. 149. Devs VCS repo Continuous translation process push notify Translator
  150. 150. Devs VCS repo Continuous translation process push translate notify Translator
  151. 151. Devs VCS repo Continuous translation process push translate notify update Translator
  152. 152. Tools for translators Poeditor - https://poeditor.com/ Transifex - https://www.transifex.com/ Weblate - https://weblate.org
  153. 153. Devs VCS repo Continuous translation process push push pull update Translator notify translate
  154. 154. Weblate example
  155. 155. Weblate example
  156. 156. Weblate - https://docs.weblate.org/en/latest/
  157. 157. Links ● c-3po doc - https://c-3po.js.org/ ● c-3po quick start - https://c-3po.js.org/quick-start.html ● gettext - https://www.gnu.org/software/gettext/ ● ICU - http://userguide.icu-project.org/ ● Weblate - https://weblate.org
  158. 158. Thx :)

×