Подробнее о некоторых улучшениях производительности. Huge pages.

1. ×òî íîâîãî â PostgreSQL 9.4 c òî÷êè çðåíèÿ ïðîèçâîäèòåëüíîñòè
Èëüÿ Êîñìîäåìüÿíñêèé
ik@postgresql-consulting.com

2. Ïëàí
Óëó÷øåíèÿ ïðîèçâîäèòåëüíîñòè
Materialized views
Huge pages
WAL
Íåáîëüøèå íî ïðèÿòíûå ìåëî÷è
Ìîíèòîðèíã ïðîèçâîäèòåëüíîñòè
Çàäåë íà áóäóùåå

3. Materialized views
View, êîòîðàÿ íå îáíîâëÿåòñÿ ïðè îáðàùåíèè, à õðàíèò äàííûå ëîêàëüíî

4. Materialized views
View, êîòîðàÿ íå îáíîâëÿåòñÿ ïðè îáðàùåíèè, à õðàíèò äàííûå ëîêàëüíî
Î÷åíü öåííî äëÿ OLAP è äåíîðìàëèçàöèè

5. Materialized views
View, êîòîðàÿ íå îáíîâëÿåòñÿ ïðè îáðàùåíèè, à õðàíèò äàííûå ëîêàëüíî
Î÷åíü öåííî äëÿ OLAP è äåíîðìàëèçàöèè
Çàäà÷à îáíîâëåíèÿ äàííûõ â Mat. View - ñëîæíàÿ

6. Materialized views
View, êîòîðàÿ íå îáíîâëÿåòñÿ ïðè îáðàùåíèè, à õðàíèò äàííûå ëîêàëüíî
Î÷åíü öåííî äëÿ OLAP è äåíîðìàëèçàöèè
Çàäà÷à îáíîâëåíèÿ äàííûõ â Mat. View - ñëîæíàÿ
Äî 9.3 â PostgreSQL íå áûëî âñòðîåííîé ðåàëèçàöèè

7. Materialized views
View, êîòîðàÿ íå îáíîâëÿåòñÿ ïðè îáðàùåíèè, à õðàíèò äàííûå ëîêàëüíî
Î÷åíü öåííî äëÿ OLAP è äåíîðìàëèçàöèè
Çàäà÷à îáíîâëåíèÿ äàííûõ â Mat. View - ñëîæíàÿ
Äî 9.3 â PostgreSQL íå áûëî âñòðîåííîé ðåàëèçàöèè
Most desired feature â TODO

8. Materialized views
View, êîòîðàÿ íå îáíîâëÿåòñÿ ïðè îáðàùåíèè, à õðàíèò äàííûå ëîêàëüíî
Î÷åíü öåííî äëÿ OLAP è äåíîðìàëèçàöèè
Çàäà÷à îáíîâëåíèÿ äàííûõ â Mat. View - ñëîæíàÿ
Äî 9.3 â PostgreSQL íå áûëî âñòðîåííîé ðåàëèçàöèè
Most desired feature â TODO
Materialized views â 9.3 ýòî PoC

9. Materialized views â 9.4
REFRESH MATERIALIZED VIEW CONCURRENTLY

10. Materialized views â 9.4
REFRESH MATERIALIZED VIEW CONCURRENTLY
Íå òðåáóåòñÿ ýêñêëþçèâíàÿ áëîêèðîâêà

11. Materialized views â 9.4
REFRESH MATERIALIZED VIEW CONCURRENTLY
Íå òðåáóåòñÿ ýêñêëþçèâíàÿ áëîêèðîâêà
Òî åñòü óæå ìîæíî ïîëüçîâàòüñÿ

12. Materialized views â 9.4
REFRESH MATERIALIZED VIEW CONCURRENTLY
Íå òðåáóåòñÿ ýêñêëþçèâíàÿ áëîêèðîâêà
Òî åñòü óæå ìîæíî ïîëüçîâàòüñÿ
 ñèëó îñîáåííîñòåé ðåàëèçàöèè òðåáóåòñÿ Primary Key/ Unique Index

13. Huge pages - ÷òî òàêîå è çà÷åì
Ñåðâåðà ñ áîëüøèì êîëè÷åñòâîì ïàìÿòè (128-256Gb) òåïåðü ñòîÿò ðàçóìíûõ
äåíåã

14. Huge pages - ÷òî òàêîå è çà÷åì
Ñåðâåðà ñ áîëüøèì êîëè÷åñòâîì ïàìÿòè (128-256Gb) òåïåðü ñòîÿò ðàçóìíûõ
äåíåã
Äåðæàòü áàçó â ïàìÿòè õîðîøî (ëþáóþ õîðîøî, PostgreSQL ñîâñåì õîðîøî)

15. Huge pages - ÷òî òàêîå è çà÷åì
Ñåðâåðà ñ áîëüøèì êîëè÷åñòâîì ïàìÿòè (128-256Gb) òåïåðü ñòîÿò ðàçóìíûõ
äåíåã
Äåðæàòü áàçó â ïàìÿòè õîðîøî (ëþáóþ õîðîøî, PostgreSQL ñîâñåì õîðîøî)
Ïî óìîë÷àíèþ ïàìÿòü âûäåëÿåòñÿ ïî 4kB - òàê áûñòðåå

16. Huge pages - ÷òî òàêîå è çà÷åì
Ñåðâåðà ñ áîëüøèì êîëè÷åñòâîì ïàìÿòè (128-256Gb) òåïåðü ñòîÿò ðàçóìíûõ
äåíåã
Äåðæàòü áàçó â ïàìÿòè õîðîøî (ëþáóþ õîðîøî, PostgreSQL ñîâñåì õîðîøî)
Ïî óìîë÷àíèþ ïàìÿòü âûäåëÿåòñÿ ïî 4kB - òàê áûñòðåå
ÎÑ òðàíñëèðóåò âèðòóàëüíûå àäðåñà â ôèçè÷åñêèå, ðåçóëüòàò êýøèðóåòñÿ â
Translation Lookaside Buffer (TLB)

17. Huge pages - ÷òî òàêîå è çà÷åì
Ñåðâåðà ñ áîëüøèì êîëè÷åñòâîì ïàìÿòè (128-256Gb) òåïåðü ñòîÿò ðàçóìíûõ
äåíåã
Äåðæàòü áàçó â ïàìÿòè õîðîøî (ëþáóþ õîðîøî, PostgreSQL ñîâñåì õîðîøî)
Ïî óìîë÷àíèþ ïàìÿòü âûäåëÿåòñÿ ïî 4kB - òàê áûñòðåå
ÎÑ òðàíñëèðóåò âèðòóàëüíûå àäðåñà â ôèçè÷åñêèå, ðåçóëüòàò êýøèðóåòñÿ â
Translation Lookaside Buffer (TLB)
1Gb
4kB = 262144 è 64-áèòíàÿ àäðåñàöèÿ - íà áîëüøèõ îáúåìàõ ïàìÿòè îâåðõýä è
íèçêàÿ ýôôåêòèâíîñòü TLB

18. Huge pages - ÷òî òàêîå è çà÷åì
Ñåðâåðà ñ áîëüøèì êîëè÷åñòâîì ïàìÿòè (128-256Gb) òåïåðü ñòîÿò ðàçóìíûõ
äåíåã
Äåðæàòü áàçó â ïàìÿòè õîðîøî (ëþáóþ õîðîøî, PostgreSQL ñîâñåì õîðîøî)
Ïî óìîë÷àíèþ ïàìÿòü âûäåëÿåòñÿ ïî 4kB - òàê áûñòðåå
ÎÑ òðàíñëèðóåò âèðòóàëüíûå àäðåñà â ôèçè÷åñêèå, ðåçóëüòàò êýøèðóåòñÿ â
Translation Lookaside Buffer (TLB)
1Gb
4kB = 262144 è 64-áèòíàÿ àäðåñàöèÿ - íà áîëüøèõ îáúåìàõ ïàìÿòè îâåðõýä è
íèçêàÿ ýôôåêòèâíîñòü TLB
Âûäåëÿåì ïàìÿòü áîëüøèìè (íàïðèìåð 2Mb) ïîðöèÿìè

19. Huge pages è PostgreSQL
vm:nr_hugepages = 3170 (sysctl, ÿäðî äîëæíî ïîääåðæèâàòü)

20. Huge pages è PostgreSQL
vm:nr_hugepages = 3170 (sysctl, ÿäðî äîëæíî ïîääåðæèâàòü)
Äî 9.2 âêëþ÷èòåëüíî - ÷åðåç áèáëèîòåêó hugetlb

21. Huge pages è PostgreSQL
vm:nr_hugepages = 3170 (sysctl, ÿäðî äîëæíî ïîääåðæèâàòü)
Äî 9.2 âêëþ÷èòåëüíî - ÷åðåç áèáëèîòåêó hugetlb
9.3 óëó÷øåíà ðàáîòà ñ shared memory c ïîìîùüþ mmap - huge pages íå
ðàáîòàþò

22. Huge pages è PostgreSQL
vm:nr_hugepages = 3170 (sysctl, ÿäðî äîëæíî ïîääåðæèâàòü)
Äî 9.2 âêëþ÷èòåëüíî - ÷åðåç áèáëèîòåêó hugetlb
9.3 óëó÷øåíà ðàáîòà ñ shared memory c ïîìîùüþ mmap - huge pages íå
ðàáîòàþò
huge_pages = try jonjo (postgresql.conf)

23. Huge pages è PostgreSQL
vm:nr_hugepages = 3170 (sysctl, ÿäðî äîëæíî ïîääåðæèâàòü)
Äî 9.2 âêëþ÷èòåëüíî - ÷åðåç áèáëèîòåêó hugetlb
9.3 óëó÷øåíà ðàáîòà ñ shared memory c ïîìîùüþ mmap - huge pages íå
ðàáîòàþò
huge_pages = try jonjo (postgresql.conf)
try - äåôîëò, íå ïîëó÷èëîñü, íå èñïîëüçóåì

24. Huge pages è PostgreSQL
vm:nr_hugepages = 3170 (sysctl, ÿäðî äîëæíî ïîääåðæèâàòü)
Äî 9.2 âêëþ÷èòåëüíî - ÷åðåç áèáëèîòåêó hugetlb
9.3 óëó÷øåíà ðàáîòà ñ shared memory c ïîìîùüþ mmap - huge pages íå
ðàáîòàþò
huge_pages = try jonjo (postgresql.conf)
try - äåôîëò, íå ïîëó÷èëîñü, íå èñïîëüçóåì
Ñåé÷àñ ðàáîòàò òîëüêî íà linux, try íà FreeBSD - OK

25. Îïòèìèçàöèÿ çàïèñè â WAL
Áîëåå îäíîãî áýêåíäà ìîæåò ïèñàòü â áóôåðû WAL

26. Îïòèìèçàöèÿ çàïèñè â WAL
Áîëåå îäíîãî áýêåíäà ìîæåò ïèñàòü â áóôåðû WAL
Ïðè àïäåéòå â WAL ìîæåò çàïèñûâàòüñÿ òîëüêî èçìåíåííîå ïîëå, à íå âñÿ
ñòðîêà (åñëè tuple ïèøåòñÿ â òó æå ñòðàíèöó)

27. Íåáîëüøèå íî ïðèÿòíûå ìåëî÷è
Ïðîèçâîäèòåëüíîñòü WINDOW ôóíêöèé

28. Íåáîëüøèå íî ïðèÿòíûå ìåëî÷è
Ïðîèçâîäèòåëüíîñòü WINDOW ôóíêöèé
Ïðîèçâîäèòåëüíîñòü COPY

29. Íåáîëüøèå íî ïðèÿòíûå ìåëî÷è
Ïðîèçâîäèòåëüíîñòü WINDOW ôóíêöèé
Ïðîèçâîäèòåëüíîñòü COPY
Çàòðàòû ïàìÿòè íà àíîíèìíûå áëîêè (DO)

30. Íåáîëüøèå íî ïðèÿòíûå ìåëî÷è
Ïðîèçâîäèòåëüíîñòü WINDOW ôóíêöèé
Ïðîèçâîäèòåëüíîñòü COPY
Çàòðàòû ïàìÿòè íà àíîíèìíûå áëîêè (DO)
Óëó÷øåíèÿ îïòèìèçàòîðà

31. Ìîíèòîðèíã
pg_stat_all_tables:n_mod_since_analyze

32. Ìîíèòîðèíã
pg_stat_all_tables:n_mod_since_analyze
pg_stat_archiver

33. Ìîíèòîðèíã
pg_stat_all_tables:n_mod_since_analyze
pg_stat_archiver
backend_xmin

34. Ìîíèòîðèíã
pg_stat_all_tables:n_mod_since_analyze
pg_stat_archiver
backend_xmin
Óëó÷øåí EXPLAIN

35. Çàäåë íà áóäóùåå - ïàðàëëåëèçì è ëîãè÷åñêàÿ ðåïëèêàöèÿ
ïîëüçîâòåëüñêèé Ñ-êîä ìîæåò èñïîëíÿòüñÿ êàê îòäåëüíèûé ïðîöåññ
(background workers)
Ïàðàëëåëüíàÿ çàïèñü â áóôåðû WAL
Logical Decoding