Gettext format overview. Javascript localization with c-3po.

1. Modern javascript
localization with good old
gettext
https://c-3po.js.org/

2. Mostovenko Alexander
Company: EVO company
Twitter: @MostovenkoA
Github: github.com/AlexMost
Me

3. About this talk

4. About this talk
Localization process in general and more
about gettext format.

5. About this talk
Localization process in general and more
about gettext format.
What is c-3po and how it helped us to
improve existing process?

6. About this talk
Localization process in general and more
about gettext format.
What is c-3po and how it helped us to
improve existing process?
Continuous translations.

7. Our initial tasks

8. Our initial tasks
1. Localize frontend on several projects:
(cabinet, company sites)
Bigl.ua
others...

9. Our initial tasks
1. Localize frontend on several projects:
(cabinet, company sites)
Bigl.ua
others ...
1. Setup translation process when devs and translators
can work independently.

10. Gettext (.po)
ICU
...
Localization
formats

11. ICU - International Components for Unicode
http://userguide.icu-project.org/

12. ICU - International Components for Unicode
http://userguide.icu-project.org/
ICU features >>> gettext features

13. ICU is cool, but ...

14. ICU is cool, but ...
1. Poor tooling (editors for translators e.t.c).

15. ICU is cool, but ...
1. Poor tooling (editors for translators e.t.c).
2. Our backend already used gettext.

16. ICU - cool but ...
"{gender_of_host, select, "
"female {"
"{num_guests, plural, offset:1 "
"=0 {{host} does not give a party.}"
"=1 {{host} invites {guest} to her party.}"
"=2 {{host} invites {guest} and one other person to her party.}"
"other {{host} invites {guest} and # other people to her party.}}}"
"male {"
"{num_guests, plural, offset:1 "
"=0 {{host} does not give a party.}"
"=1 {{host} invites {guest} to his party.}"
"=2 {{host} invites {guest} and one other person to his party.}"
"other {{host} invites {guest} and # other people to his party.}}}"
"other {"
"{num_guests, plural, offset:1 "
"=0 {{host} does not give a party.}"
"=1 {{host} invites {guest} to their party.}"
"=2 {{host} invites {guest} and one other person to their party.}"
"other {{host} invites {guest} and # other people to their party.}}}}"

17. GNU gettext
https://www.gnu.org/software/gettext/

18. GNU gettext
https://www.gnu.org/software/gettext/
1. Simple format.

19. GNU gettext
https://www.gnu.org/software/gettext/
1. Simple format.
2. Has good ecosystem for translation process.

20. gettext idea

21. ‘Hello world’
Source code
gettext idea

22. gettext(‘Hello world’)
Source code
gettext idea

23. gettext(‘Hello world’)
Source code
‘Hello world’
.pot file
gettext idea

24. gettext(‘Hello world’)
Source code
msgid: ‘Hello world’
msgstr: ‘’
.po file
‘Hello world’
.pot file
gettext idea

25. gettext(‘Hello world’)
Source code
msgid: ‘Hello world’
msgstr: ‘Здоровеньки були’
.po file
‘Hello world’
.pot file
gettext idea

26. gettext(‘Hello world’)
Source code
msgid: ‘Hello world’
msgstr: ‘Здоровеньки були’
.po file
‘Hello world’
.pot file
gettext idea

27. gettext workflow
Code .pot Extracting translation strings from code (AST
traverse)extract

28. gettext workflow
Code .pot Extracting translation strings from code (AST
traverse)
.pot .po Merge new extracted translations from .pot file
with existing. msgmerge
extract
merge .po.po

29. gettext workflow
Code .pot Extracting translation strings from code (AST
traverse)
.pot .po Merge new extracted translations from .pot file
with existing. msgmerge
extract
merge
Translator adds translations
translation
.po.po
.po.po.po
.po.po.po

30. gettext workflow
Code .pot Extracting translation strings from code (AST
traverse)
.pot .po Merge new extracted translations from .pot file
with existing. msgmerge
extract
merge
Translator adds translations
translation
.po.po
.po.po.po
.po.po.po
resolve Code
Placing translations back to the code (by locale).po.po.po

31. .po file structure

32. Example .po file
msgid ""
msgstr ""
"Project-Id-Version: c 3po-webpack-startn"
"PO-Revision-Date: 2017-02-01 22:22+0200n"
"Language-Team: Ukrainiann"
"Language: ukn"
"MIME-Version: 1.0n"
"Content-Type: text/plain; charset=UTF-8n"
"Content-Transfer-Encoding: 8bitn"
"Plural-Forms: nplurals=2; plural=(n!=1);n"
#: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
#: app.js:10
msgid "${ 0 } second"
msgid_plural "${ 0 } seconds"
msgstr[0] "${ 0 } секунда"
msgstr[1] "${ 0 } секунди"
msgstr[2] "${ 0 } секунд"
#: app.js:14
msgid "webpack with c-3po localization demo"
msgstr "Демо локалізації з c-3po та webpack"

33. msgid ""
msgstr ""
"Project-Id-Version: c 3po-webpack-startn"
"PO-Revision-Date: 2017-02-01 22:22+0200n"
"Language-Team: Ukrainiann"
"Language: ukn"
"MIME-Version: 1.0n"
"Content-Type: text/plain; charset=UTF-8n"
"Content-Transfer-Encoding: 8bitn"
"Plural-Forms: nplurals=2; plural=(n!=1);n"
#: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
#: app.js:10
msgid "${ 0 } second"
msgid_plural "${ 0 } seconds"
msgstr[0] "${ 0 } секунда"
msgstr[1] "${ 0 } секунди"
msgstr[2] "${ 0 } секунд"
#: app.js:14
msgid "webpack with c-3po localization demo"
msgstr "Демо локалізації з c-3po та webpack"
Headers

34. msgid ""
msgstr ""
"Project-Id-Version: c 3po-webpack-startn"
"PO-Revision-Date: 2017-02-01 22:22+0200n"
"Language-Team: Ukrainiann"
"Language: ukn"
"MIME-Version: 1.0n"
"Content-Type: text/plain; charset=UTF-8n"
"Content-Transfer-Encoding: 8bitn"
"Plural-Forms: nplurals=2; plural=(n!=1);n"
#: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
#: app.js:10
msgid "${ 0 } second"
msgid_plural "${ 0 } seconds"
msgstr[0] "${ 0 } секунда"
msgstr[1] "${ 0 } секунди"
msgstr[2] "${ 0 } секунд"
#: app.js:14
msgid "webpack with c-3po localization demo"
msgstr "Демо локалізації з c-3po та webpack"
Translations

35. "Plural-Forms:
nplurals=2;
plural=(n!=1);n"
Plural-forms header

36. "Plural-Forms:
nplurals=2;
plural=(n!=1);n"
Plural-forms header
Plurals count

37. "Plural-Forms:
nplurals=2;
plural=(n!=1);n"
Plural-forms header
Plurals formula

38. #: app.js:6
msgid "Choose locale"
msgstr "Оберіть локаль"
1 to 1 translations

39. #: app.js:10
msgid "${ n } second"
msgid_plural "${ n } seconds"
msgstr[0] "${ n } секунда"
msgstr[1] "${ n } секунди"
msgstr[2] "${ n } секунд"
Plural forms example

40. #. Notes for the translator
#: app.js:6
#, flag
msgid "Choose locale"
msgstr "Оберіть локаль"
Comments

41. msgctx "email"
msgid "Hello"
msgstr "Привіт"
msgctx "main page"
msgid "Hello"
msgstr "Вітаємо"
Context example

42. c-3po
https://c-3po.js.org/

43. c-3po
Code .pot Extracting translation strings from code (AST
traverse)extract
resolve Code
Placing translations back to the code (by locale).po.po.po

44. Library:
https://github.com/c-3po-org/c-3po
Core functionality.
Translation functions (tags):
t, ngettext, gettext, jt
Babel-plugin:
https://github.com/c-3po-org/babel-plugin-c-3po
Extract, Resolve, Validation

45. Key ideas

46. Key idea 1.
Tagged template strings for formatting

47. c-3po translations are based on tagged template
strings
import { t } from 'c-3po'
const name = 'Mike';
console.log(t`Hello ${name}`);

48. What is a template string?

49. What is template string?
console.log('Hello ' + name)
console.log(`Hello ${name}`)

50. What is a tagged template string?
const name = ‘Mike’
t`Hello ${name}`
function t(strs, ...exprs) {
// strs -> [‘Hello’, ‘’]
// exprs -> [‘Mike’]
}

51. Valid javascript
t `Hello ${name}`
t(5)`Hello ${name}`
t(5) `Hello ${name}`

52. Why just not simple functions but tags?

53. Why just not simple functions but tags?
c-3po must work without transpilation

54. Why just not simple functions but tags?
// Without babel transpile !!!
const name = ‘Mike’
t(`Hello ${name}`)
function t(str) {
// strs -> ‘Hello Mike’
// no chance to get ‘Hello ${name}’ msgid here :(
}

55. Existing solutions
Force you to use sprintf formatting
(i18next, jed e.t.c)

56. sprintf
gettext("Hello %s", user)
gettext("%2$s %3$s a %1$s", "cracker", "Polly", "wants")
gettext("Current timestamp: %d", Date.now)

57. sprintf
● Alien formatting for js

58. sprintf
● Alien formatting for js
● No reason to use es6 template strings

59. sprintf
● Alien formatting for js
● No reason to use es6 template literals
● Extra CPU work on the client (sprintf parsing)

60. Key idea 2.
Precompile translation on a build step

61. Traditional translations resolve flow

62. Traditional translations resolve flow
Load assets to the client
(browser)

63. Traditional translations resolve flow
Load assets to the client
(browser)
Resolve locale (from
cookie, params e.t.c)

64. Traditional translations resolve flow
Load assets to the client
(browser)
Resolve locale (from
cookie, params e.t.c)
Fetch locale data

65. Traditional translations resolve flow
Load assets to the client
(browser)
Resolve locale (from
cookie, params e.t.c)
Fetch locale data
Apply translations

66. Wanted translations resolve flow

67. Wanted translations resolve flow
Load assets to the client
(browser)

68. Wanted translations resolve flow
Load assets to the client
(browser)
That’s it, they are already
localized !!!

69. Benefits of precompiled translations:

70. Benefits of precompiled translations:
● Smaller bundle size

71. Benefits of precompiled translations:
● Smaller bundle size
● Less work on the client

72. Benefits of precompiled translations:
● Smaller bundle size
● Less work on the client
● No wait until translations are loaded and applied

73. Key idea 3.
Works with and without babel

74. Demo on jsfiddle

75. Key idea 4.
Can extract and resolve translations from
everything that works with babel (jsx)

76. Extract and resolve translations with simple
config
Extract:
{ "extract": { "output": "extract.pot" } }
Resolve:
{ "resolve": { "translations": "uk.po" } }
https://c-3po.js.org/quick-start.html

77. Key idea 5.
Efficient dev and prod setup

78. Requirements for the dev setup

79. Requirements for the dev setup
● Faster builds

80. Requirements for the dev setup
● Faster builds
● Simple integration

81. Dev setup with c-3po

82. Dev setup with c-3po
● Use whole c-3po lib (import c-3po).

83. Dev setup with c-3po
● Use whole c-3po lib (import c-3po).
● Babel plugin only for extraction translations

84. Requirements for the prod setup

85. Requirements for the prod setup
● Smaller result assets

86. Requirements for the prod setup
● Smaller result assets
● Faster locale load

87. Prod setup with c-3po

88. Prod setup with c-3po
● Use c-3po mock lib (alias in webpack e.t.c).

89. Prod setup with c-3po
● Use c-3po mock lib (alias in webpack e.t.c).
● Babel plugin for transformations.

90. Prod setup with c-3po
● Use c-3po mock lib (alias in webpack e.t.c).
● Babel plugin for transformations.
● Separate build for each locale.

91. Key idea 6.
Validate translation strings

92. Validation problem
msgid "http://some.random.domain.link"
msgstr ?
msgid ""
msgstr ?
msgid "translate ${ test ? result1() : result2() }"
msgstr ?
msgid "5"
msgstr ?

93. c-3po validation (empty str)
t``
Module build failed: SyntaxError: Can not translate empty string
5 |
6 |
> 7 | t``
| ^

94. c-3po validation (only numbers)
t`3243243`
ERROR in Error: Module build failed: SyntaxError: Can not translate '3243243'
2 |
3 |
> 4 | t`3243243`
| ^

95. c-3po validation (programs inside translations)
t`Translate ${someVar ? callFn1() : callFn2()}`
ERROR in Error: Module build failed: SyntaxError: You can not use
ConditionalExpression '${someVar ? callFn1() : callFn2()}' in localized strings
2 |
3 |
> 4 | t`Translate ${someVar ? callFn1() : callFn2()}`;
| ^
5 |

96. c-3po validation (only var wrap)
t`${someVar}`
ERROR in Error: Module build failed: SyntaxError: Can not translate '${ someVar }'
2 |
3 |
> 4 | t`${someVar}`;
| ^
msgid:’${someVar}’

97. Validate not translated strings

98. Validation (no translated str)
t`non translated str`
ERROR in Error: Module build failed: SyntaxError: No "non translated str" in "uk.po"
file
2 |
> 3 | t`non translated str`;
https://c-3po.js.org/configuration.html#configresolveunresolved-string-one-of-fail-warn-skip

99. Key idea 7.
Can use any default locale in sources
(standard gettext uses only English locale)

100. The problem
Standard ngettext accepts only 2 plural forms
char * ngettext(const char * msgid1, const char * msgid2, unsigned long int n);

101. Ukrainian/Russian strings in sources
How to be with plurals?

102. Ukrainian/Russian strings in sources
How to be with plurals? Standard ngettext accepts only 2
plural forms?

103. Ukrainian/Russian strings in sources
How to be with plurals? Standard ngettext accepts only 2
plural forms? What if we have more or less than 2 plural
forms?

104. Ukrainian/Russian strings in sources
How to be with plurals? Standard ngettext accepts only 2
plural forms? What if we have more or less than 2 plural
forms?

105. ngettext for other locales ( plural forms > 2 )

106. ngettext for other locales ( plural forms > 2 )
Doc - https://c-3po.js.org/ngettext.html
Change defaultHeaders setting to:
'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);'

107. ngettext for other locales ( plural forms > 2 )
Demo on jsfiddle
Doc - https://c-3po.js.org/ngettext.html
Change defaultHeaders setting to:
'plural-forms':'nplurals=3; plural=(n%10==1 && n%100!=11 ? 0 : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2);'
ngettext(msgid`${n} банан`, `${n} банана`, `${n} бананів`, n)

108. c-3po tags and functions

109. t - tag gettext
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html

110. t - tag gettext
t`Hello ${name}` < Usage
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html

111. t - tag gettext
t`Hello ${name}`
msgid "Hello ${ name }"
msgstr "Привіт ${ name }"
< Usage
< Extract
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html

112. t - tag gettext
t`Hello ${name}`
msgid "Hello ${ name }"
msgstr "Привіт ${ name }"
< Usage
< Extract
`Привіт ${name}` < Resolve
Demo on jsfiddle
Doc - https://c-3po.js.org/tag-gettext--t-.html

113. ngettext - plural forms

114. ngettext - plural forms
ngettext(msgid`${n} time clicked`, `${n} times clicked`, n) < Usage

115. ngettext - plural forms
ngettext(msgid`${n} time clicked`, `${n} times clicked`, n)
msgid "${ n } time clicked"
msgid_plural "${ n } times clicked"
msgstr[0] "${ n } time clicked [translated]"
msgstr[1] "${ n } times clicked [translated]"
< Usage
< Extract

116. Forgot msgid ?
ERROR in Error: Module build failed: SyntaxError: First argument must be tagged
template expression. You should use 'msgid' tag
2 |
3 |
> 4 | ngettext(`${hours} hour`, `${hours} hours`, hours);
| ^
5 |
Demo on jsfiddle
Doc - https://c-3po.js.org/ngettext.html

117. ngettext - resolve
function _tag_ngettext(n, args) {
return args[+(n != 1)];
}
_tag_ngettext(n,
[n + " time clicked [translated]", n + " times clicked [translated]"]));
Demo on jsfiddle
Doc - https://c-3po.js.org/ngettext.html

118. First contribution !!! jt (jsx-gettext)

119. First contribution !!! jt (jsx-gettext)

120. jt - jsx gettext

121. jt - jsx gettext
const btn = <button key="btn">{ t`me` }</button>;
<span>{jt`Click ${ btn }`}</span>
< Usage

122. jt - jsx gettext
const btn = <button key="btn">{ t`me` }</button>;
<span>{jt`Click ${ btn }`}</span>
< Usage
< Extractmsgid "Click ${ btn }"
msgstr "Click ${ btn } [translated]"

123. jt - jsx gettext
const btn = <button key="btn">{ t`me` }</button>;
<span>{jt`Click ${ btn }`}</span>
< Usage
< Extractmsgid "Click ${ btn }"
msgstr "Click ${ btn } [translated]"
['Click ', btn, ' [translated]']
< Resolve

124. gettext

125. gettext
gettext('simple gettext'); < Usage

126. gettext
gettext('simple gettext'); < Usage
< Extract
msgid "simple gettext"
msgstr "simple gettext [translated]"

127. gettext
gettext('simple gettext'); < Usage
< Extract
msgid "simple gettext"
msgstr "simple gettext [translated]"
'simple gettext [translated]' < Resolve

128. Why add gettext func if we have ‘t’ tag?

129. Legacy - *.coffee *.eco (100 K+ LOC)
1. Wrap string in gettext inside *.coffee *.eco
2. Add babel loader after coffee loader to the
webpack pipeline
3. Profit !!!

130. Even more features

131. Aliasing
https://c-3po.js.org/aliasing.html

132. Babel plugin will extract aliased translation
import { t as i18n } from 'c-3po'
i18n`this translation will work`

133. Multiline
https://c-3po.js.org/multiline-strings.html

134. import { t } from 'c-3po';
function test(name) {
return t`multi line string
with multiple line breaks and
with formatting ${name}`
}
Multiline (auto dedent)

135. Translator comments

136. Translator comments
// translator: some description for the extracted string.
t`some string`

137. Translator comments
// translator: some description for the extracted string.
t`some string`
#. some description for the extracted string
msgid "some string"
msgstr ""

138. Summary

139. Summary
● Native js tagged template strings for formatting

140. Summary
● Native js tagged template strings for formatting
● Can precompile translation on a build step

141. Summary
● Native js tagged template strings for formatting
● Can precompile translation on a build step
● Works with and without babel

142. Summary
● Native js tagged template strings for formatting
● Can precompile translation on a build step
● Works with and without babel
● Extracts and resolves translations from everything
that works with babel (jsx)

143. Summary
● Native js tagged template strings for formatting
● Can precompile translation on a build step
● Works with and without babel
● Extracts and resolves translations from everything
that works with babel (jsx)
● Efficient dev and prod setup

144. Summary
● Native js tagged template strings for formatting
● Can precompile translation on a build step
● Works with and without babel
● Extracts and resolves translations from everything
that works with babel (jsx)
● Efficient dev and prod setup
● Validation for translated strings

145. Summary
● Native js tagged template strings for formatting
● Can precompile translation on a build step
● Works with and without babel
● Extracts and resolves translations from everything
that works with babel (jsx)
● Efficient dev and prod setup
● Validation for translated strings
● Can use any default locale in sources (not only
English)

146. Continuous translations process

147. Devs
VCS
repo
Continuous translation process
Translator

148. Devs
VCS
repo
Continuous translation process
push
Translator

149. Devs
VCS
repo
Continuous translation process
push notify
Translator

150. Devs
VCS
repo
Continuous translation process
push
translate
notify
Translator

151. Devs
VCS
repo
Continuous translation process
push
translate
notify
update Translator

152. Tools for translators
Poeditor - https://poeditor.com/
Transifex - https://www.transifex.com/
Weblate - https://weblate.org

153. Devs
VCS
repo
Continuous translation process
push
push
pull
update
Translator
notify
translate

154. Weblate example

155. Weblate example

156. Weblate - https://docs.weblate.org/en/latest/

157. Links
● c-3po doc - https://c-3po.js.org/
● c-3po quick start - https://c-3po.js.org/quick-start.html
● gettext - https://www.gnu.org/software/gettext/
● ICU - http://userguide.icu-project.org/
● Weblate - https://weblate.org

158. Thx :)