This presentation is about the work that I did during the Google Summer of Code 2014 to PostgreSQL. The project is about change an Unlogged Table to Logged and vice-versa. Project wiki page: https://wiki.postgresql.org/wiki/Allow_an_unlogged_table_to_be_changed_to_logged_GSoC_2014 I presented this work at PGCon2015 in Ottawa/Canada (2015-06-19)

1. Sharing Code and
Experiences
@fabriziomello

2. About me

3. I was born in Dom Pedrito in the South of Brazil

4. But I lived in Bagé since I was 1 year old

5. Bagé is 236 miles away from Porto Alegre

6. Porto Alegre is the FISL city

7. In my hometown Bagé is common
people...
● … be born there
● … grow up there
● … build a family there
● … spend the entire life there
● … and die there.

8. Until 2005 this lifestyle
fitted with my “old” needs

9. Then I had an opportunity
to work 100% of my time
with FOOS

10. Background
● Bachelor in Information Systems in 2002
● Entrepeneur at http://timbira.com
● Agile Methodologies Specialization student 2014/2015
● PostgreSQL colaborator since 2008 (Brazilian
community and now the international too)

11. FOSS and me
● My first contact was using Linux in 1997
● I fell in love with this culture since then
● In 1999 I met PostgreSQL so since then I
knew this would be part of my life

12. But FOSS bite me awile :-)
● Because of this decision I had a lot of
troubles, including financial…
● But here I am :-)

13. Is a global program that
offers students stipends to
write code for open source
projects.
We have worked with the
open source community to
identify and fund exciting
projects for the upcoming
summer.

14. Connect students to
open source communities

15. GSoC and PostgreSQL
● Since 2006
● Cool projects
○ Fast GiST index build
○ New phpPgAdmin Plugin Architecture (brazilian)
○ pgAdmin database designer
○ Better indexing for ranges
○ Document collection Foreign-data Wrapper

16. And now my project ...
PostgreSQL 9.1 introduced a new kind of table
Unlogged Tables

17. What means “Unlogged”?
First we need to know what means “WAL”
PostgreSQL is Full-ACID and to guarantee data
integrity uses a standard method called
WAL (Write-Ahead Logging)

18. WAL (Write-Ahead Logging)
“In computer science, write-ahead logging (WAL) is a family
of techniques for providing atomicity and durability (two of
the ACID properties) in database systems.
In a system using WAL, all modifications are written to a log
before they are applied. Usually both redo and undo
information is stored in the log.”
http://en.wikipedia.org/wiki/Write-ahead_logging

19. Ok, and what means “Unlogged” ?
● Unlogged means that the data written in
these tables is not written to WAL.
● So it makes written really, really fast
compared to written into regular tables.

20. So I’ll use it to all of my tables...
● However you won’t want to do that, because
○ They are neither crash-safe (an unlogged table is
automatically truncated after a crash or unclean
shutdown)
○ And they are nor replicated using SR

21. But there are some cool use cases
● Speed ETL jobs
● Cache
● Session State
● Queues?!
● ...

22. And now we have the power to ...
● change from UNLOGGED to LOGGED
○ ALTER TABLE name SET LOGGED;
● change from LOGGED to UNLOGGED
○ ALTER TABLE name SET UNLOGGED;

23. Already committed
commit: f41872d0c1239d36ab03393c39ec0b70e9ee2a3c
author: Alvaro Herrera <alvherre@alvh.no-ip.org>
date: Fri, 22 Aug 2014 14:27:00 -0400
Implement ALTER TABLE .. SET LOGGED / UNLOGGED
This enables changing permanent (logged) tables to unlogged and
vice-versa.
(Docs for ALTER TABLE / SET TABLESPACE got shuffled in an order that
hopefully makes more sense than the original.)
Author: Fabrízio de Royes Mello
Reviewed by: Christoph Berg, Andres Freund, Thom Brown
Some tweaking by Álvaro Herrera

24. Current implementation
1. Acquire AcessExclusiveLock
2. Check dependencies
a. Cannot change temp tables
b. Check Foreign Keys
3. Create new heap/toast with new relpersistence
4. Rewrite heap/toast
5. Rewrite indexes

25. Currently Caveats
● AccessExclusiveLock
● Rewrite all datafiles

26. PostgreSQL GSoC2015
Improve the performance of
ALTER TABLE SET LOGGED
UNLOGGED statement

27. Improvements
● Don’t rewrite datafiles when wal_level =
minimal
● Create wal records for datafiles when
wal_level != minimal

28. Main Problems
● To change an Unlogged to Logged (or vice-
versa) we need
○ Drop/Create the InitFork
○ and it is not a transactional operation

29. GSoC2015 : Main Problems
● ALTER TABLE ... SET LOGGED
○ if we drop the InitFork
○ and a crash occurs we're in a inconsistent state
○ in catalog the relation is marked as unlogged and
we don't have the InitFork anymore

30. GSoC2015 : Main Problems
● ALTER TABLE ... SET UNLOGGED
○ if we create the InitFork and the transaction
rollback or a crash occurs we're in a inconsistent
state because in the catalog the relation is marked
as logged and during the crash recovery we
truncate the relation if the InitFork exists leading us
to a inconsistent state

31. Idea if “wal_level = minimal”
● Acquire AcessExclusiveLock
● Check dependencies
○ Cannot change temp tables
○ Check Foreign Keys
● Create a “TransientInitFork” to crash recovery detection
● FlushRelationBuffers and fsync relation
● Drop/Create the “InitFork”
● Change Catalog
● Drop the “TransientInitFork”

32. Idea if “wal_level != minimal”
● Same of the “wal_level = minimal”
● Plus
○ xlog all pages of datafiles (heap, toast, index)

33. Questions?

34. Special thanks to
● GSoC2014
○ Stephen Frost (mentor)
○ Josh Berkus and Thom Brown (organizers)
○ Christoph Berg (patch review)
○ Álvaro Herrera (patch review and commit)
● GSoC2015
○ Josh Berkus and Thom Brown (organizers)
○ Ashutosh Baspat (mentor)

35. PGBR2015 call for papers is open!!
● When?
○ November 18, 19 and 20
● Where
○ Porto Alegre, Brasil
● http://pgbr.postgresql.org.br