The talk from PgconfUS 2015

1. Enterprise grade deployment
and security with PostgreSQL
PGConf US 2015
March 25 - 27, 2015
Himanchali
himamahi09@gmail.com

2. #Who Am I
himanchali@inmobi.com
Technical Lead – Production & Infrastructure Engineering

3. www.inmobi.com
 Independent mobile ad network
 Customer engagement platform through a suit of product ranging from app
distribution & monetization to brand advertisement
 Serves billion of ads
 Hosting the first official PGDay India event in April 
http://www.meetup.com/India-PUG/events/220553997/

4. What kind of application do we have..
 OLTP
 OLAP

5. What exactly am I talking today ??
 Production Deployment
 All main transactional and CI reporting DBs
 Replication across globe
 Load balancing and connection pooling through PGBouncer
 Automation through puppet
 Version control for DB level changes
 Data security
 Data security
 User access control
 OS level security

6. Data growth
0
10
20
30
40
50
60
70
80
90
2010 2011 2012 2013 2014 2015
Main Transactional DBs
(GB)
Reporting DB (TB)
Others (GB)

7. Current stats of Postgres DB Servers
 About 60 hosts spread across 4 regions
 DB size ranges from 10 GB – 20 TB
 The critical transactional DBs :
 Average size 80GB
 Average 18K QPS with peak of 54K QPS
 Reporting DB
 Average 200 GB data load everyday

8. Simple expectation of management
 99.99% Uptime for DB servers
 No data loss
 Good performance

9. Achieving 99.99% Uptime
 A proper DB Architecture
 Multiple disaster recovery Plan
 A time to time disaster drill
 Proper data backup
 PITR setup

10. A proper DB Architecture

11. The architecture..
 Proper data modeling
 Use of constraints carefully
 Minimize the dependency between DBs
 Use of proper data types
 Proper retention policies

12. Disaster Recovery plan
 Question is when to touch the trigger ??
 What if standby doesn’t come up ?? (rare condition)
 Are my backups up to date ??
 Do I have an option to revert some drop of a table?

13. Data availability in different regions
 Cascaded SR setup
 A boon for cross region replication
 One node as entry point to the region
 Cascade slaves within LAN from above node
 A little precaution needed with this setup
 When the master is down and its standby is being triggered as master
 Proper monitoring that no slave is out of sync

14. Read distribution across multiple slaves
 Load balancer
 Pgbouncer
 Connection pooling
 Connection mode control , session or transaction
 Maintenance without downtime

15. What about performance ??
 Proper index creation
 Query optimization
 Table partitioning
 Proper vacuum

16. What else …
 How can I forget those many stored procedures in my DB .
 User defined functions for complex business logic
 Saving time between application and DB
 Get rid of dependency of language used by application

17. Practice adopted for DB changes
 Version control of schema changes
 Version control of data changes

18. Minimize the deployment effort
 Using puppet and chef modules
 No direct change on the host manually
 Configuration changes pushed via puppet
 Packages installation using chef-solo

19. Adding spice of monitoring…
 Use of enriched statistics collector
 User stats
 DB stats
 Table stats
 Index stats
 And many more…
 pg_log monitoring
 Log everything if you want to monitor everything

20. Security aspects

21. Control user access
 Limited access to application users
 Give only what they need
 Control at DB, schema or table level
 Use user groups to manage if many users
 Manage and standardize DBA access
 A common user
 Maintain default privilege

22. Some example
REVOKE CREATE ON SCHEMA public FROM PUBLIC;
REVOKE ALL on schema public from write_user,read_user;
GRANT USAGE on schema public to write_user,read_user ;
GRANT all on all tables in schema public to write_user;
GRANT all on all sequences in schema public to write_user;
REVOKE ALL on all functions in schema public from write_user,read_user;
GRANT EXECUTE on all functions in schema public to write_user,read_user;
GRANT SELECT on all sequences in schema public to read_user;
GRANT SELECT on all tables in schema public to read_user;

23. ALTER DEFAULT PRIVILEGES FOR ROLE enterprisedb,postgres IN SCHEMA public GRANT all on tables to write_user;
ALTER DEFAULT PRIVILEGES FOR ROLE enterprisedb,postgres IN SCHEMA public GRANT EXECUTE on functions to
write_user,read_user;
ALTER DEFAULT PRIVILEGES FOR ROLE enterprisedb,postgres IN SCHEMA public GRANT all on sequences to write_user;
ALTER DEFAULT PRIVILEGES FOR ROLE enterprisedb,postgres IN SCHEMA public GRANT SELECT on sequences to
read_user;
ALTER DEFAULT PRIVILEGES FOR ROLE enterprisedb,postgres IN SCHEMA public GRANT SELECT on tables to read_user;

24. Access restriction to DB
 Use pg_hba.conf and pg_ident.conf
 Avoid using trust
 Use MD5 authentication
 IP / Subnet access allocation for DB or user level
 Restrict the connection for any particular subnet for any user/DB
 Use pg_ident.conf if using ident authentication in pg_hba.conf

25. If that’s not enough…
 Use secure TCP/IP Connections with SSL
 IPTables
 Control the traffic by making some rules
 Allow or reject
 SE-Linux - control access at process level
 setenforce=1

26. Myth broken !!

27. Thank You !!
QUESTIONS??