This document provides an overview of NginX, HAProxy, and DNS stack technologies presented at a WordCamp conference. It discusses how NginX uses an asynchronous event-driven architecture to handle high loads more efficiently than threaded architectures. It then demonstrates configuring NginX as a reverse proxy, including logging, caching, and gzip compression. Finally, it briefly introduces HAProxy as an open-source load balancer and discusses installing and enabling it on Ubuntu.
1. NginX, HAProxy and DNS Stack
Presentation at WordCamp Belgrade 2015.
April 19th
Authors:
Ivan Dabic, General Manager @MaxCDN - NginX
Jovan Katic, Support Engineer @MaxCDN - HAProxy
Karlo Butigan Markovic, NOC Engineer @MaxCDN - DNS
4. ○ we don’t want to have “wild” requests cached even though they will never be
requested again
○ we assume whatever is requested two times is valid request as it’s probably
going to be requested 3rd, 4th,... time.
● proxy_cache_valid defines status code we treat as valid and how long we want to
cache it in nginx cache. In this case status codes 200 and we’ll cache it for 10
seconds (NOT a good practise but, for the same of showing the load balancing
method below we wanted short caching time). You’ll usually set this to at least one
week or more.
What we may want to deal with separately is the cache key. To show the purpose of it I am
setting the cache key to following:
proxy_cache_key $request_uri$http_accept_encoding;
This will, basically, define caching parameters that distinguish cached asset by:
1. Requested asset (uri)
2. AcceptEncoding request header
What showe to be the perfect setup is:
proxy_cache_key $scheme$request_uri$http_accept_encoding$param$args;
Above setup defines:
1. $scheme: Local nginx variable that holds the value of protocol used to access/request
cached asset (http, https,...)
2. $request_uri: Same as in default example, it’s the nginx variable holding the value
of requested asset uri
3. $http_accept_encoding: variable holding the value of request header
“AcceptEncoding”
4. $param: Custom variable we can use to alter the cache key in certain scenarios use
it with caution! Changing cache key may affect cache clearing!
5. $args: Query strings in request
So, let’s show an example of cache affection by cache_key. We have defined the cache_key
“distinguisher” by usign “$http_accept_encoding” variable. This means that any request with
different AcceptEncoding request header value for the same file will result in different cache
entry:
~$ curl I http://vps2.net/index.html
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Sun, 26 Apr 2015 22:56:13 GMT
3
9.
~$ service haproxy restart
* Restarting haproxy haproxy [ OK ]
service haproxy reload
* Reloading haproxy haproxy [ OK ]
service haproxy status
haproxy is running.
service haproxy stop
* Stopping haproxy haproxy [ OK ]
service haproxy status
haproxy not running.
To be honest, you won't be able to do anything with the init script before you configure the
load load balancer itself. So let's check what do we get “out of the box”:
~$cat /etc/haproxy/haproxy.cfg
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
maxconn 2000
user haproxy
group haproxy
daemon
defaults
log global
mode http
option httplog
option dontlognull
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
8
14. BIND
BIND is the oldest host > IP translator. It uses root name servers, TLD name servers, and
authoritative name servers to translate domains into IP addresses. For a full description and
more on DNS please look below the setup and presentation in regards to the WordCamp
presentation.
To install BIND on Ubuntu server use the following command:
aptget install bind9
(suggestion: aptget install dnsutils)
To install BIND CentOS use the following command:
yum install bind9
Configuring BIND to be an authoritative DNS server:
Open the /etc/bind/named.conf.options file with the text editor that you are most comfortable
with (vi, nano, etc.) and input:
options {
directory "/var/cache/bind";
recursion no;
allowtransfer {none;};
dnssecvalidation auto;
authnxdomain yes; # conform to RFC1035
listenonv6 { any; };
};
This tells BIND where the directory for caching is. It also tells it not to be in the recursion
mode which is important for security reasons. Allowing transfer can be set to none or to a
slave or master/slave server IP address (or multiple addresses). dnssecvalidation option tells
the server if it the domains should be signed and validate using dnssec. authnxdomain tells
the server to answer authoritatively (the AA bit is set). listenonv6 sets the IPv6 IP address
on which the server should listen on.
Save the file and then open the /etc/bind/named.conf.local and in our case we have set the
zone name to maxcdn.com, set the type as master (as in master DNS server), location of the
file of the zone itself and allowtransfer when the allowtransfer is set to none in the options
file.
zone "maxcdn.com" in {
type master;
file "/etc/bind/zones/maxcdn.com";
allowtransfer {none;};
};
13
15.
Save that file and create a directory called zones in /etc/bind using mkdir /etc/bind/zones then
using then go to that directory using cd /etc/bind/zones and create a new file called
maxcdn.com using your favorite text editor like so:
vi maxcdn.com
and input the following:
$TTL 86400 ; 24 hours could have been written as 24h or 1D
maxcdn.com. IN SOA @ root (
2002022401 ; serial
3H ; refresh
15 ; retry
1w ; expire
3h ; minimum
)
IN NS localhost.
IN A 178.62.160.79
www IN A 178.62.160.79
The above file tells BIND that the time to live for this zone is 24 hours and that it is the Start of
Authority record. Incrementing the serial number tells the slave server with the same zone to
update the zone record. IN NS tells which is/are the default name server/s of the zone. IN A
gives the translation of the domain/host to IP.
Once everything is configured restart BIND service so that it can accept all of the new
settings.
When testing the zone from the local BIND server using dig command you would get an
answer like so:
dig @localhost maxcdn.com
; <<>> DiG 9.9.53ubuntu0.2Ubuntu <<>> @localhost maxcdn.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; >>HEADER<< opcode: QUERY, status: NOERROR, id: 44418
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 3
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
14
18. DNS - bind
1) DNS brief background
Paul Mockapetris designed the Domain Name System in 1983 at the University of California
Jon Postel was the person who actually asked Paul to write the first implementation for DNS
The Stanford Research Institute was the one who held the largest HOSTS.TXT file at that
time and that file was taken
UC Berkeley students Douglas Terry, Mark Painter, David Riggle and Songnian Zhou are
the first people to write the code for Unix DNS implementation and they called it BIND
(Berkeley Internet Name Domain) (1984)
Kevin Dunlap of DEC substantially revised the DNS implementation in 1985
Mike Karels, Phil Almquist, and Paul Vixie have maintained BIND since then.
In 1987, RFC822 and RFC823 get supressed by RFC1034, RFC1035 and a few more. (For
more details look in the links section)
In the days before DNS either you remembered all of the IPs that you needed to visit, you had
your own hosts file written or you downloaded the hosts.txt file from Stanford Research
Institute.
(Basically you had to ask the person for their IP address so that you can visit their website)
Host file locations that can still be used and are used in some cases:
/etc/hosts Unix based systems
%WinDir%HOSTS Win 3.1
%WinDir%hosts Win 95, 98, ME
%SystemRoot%System32driversetchosts all versions above, and including, Win NT.
Short version:
People around the world who had access to the Internet were using hosts file to store all of
the
host> IP translations
You would have to call the person and ask them for their IP address to get to their website
US government advanced research agency decided to invest in the DNS project
1983 first implementation was written
BIND came to life in 1984
2) What is BIND/PDNS (PowerDNS) and differences between the two
BIND/pdns is a Domain Name System software that communicates with the root servers to
get a translation of a hostname (to an IP address) or acts as a authoritative master/slave
server depending on the configuration.
The difference between BIND and PowerDNS:
BIND is the first a is the most widely used Domain Name System (DNS) software on the
Internet
Uses flat files (only)
17
20. You can see all of the current root servers and news regarding them on
http://www.rootservers.org/
Ex.:
Generic: .com, .net ...
CountryCode: .rs (RNIDS)
Sponsored: .mil, .gov, .xxx (must be eligible to get it)
infrastructure: .arpa (Used for instance in reverse lookup of IPv4 and IPv6)
"The Internet Assigned Numbers Authority (IANA) is responsible for the global coordination of
the DNS Root, IP addressing, and other Internet protocol resources." https://www.iana.org/
5) What are authoritative servers master/slave
Authoritative server can be master or slave and it holds the authority over a domain name.
When registering for a domain name, person registering the domain is aked to insert at least
two domain servers
Usually named ns1 and ns2, ns1 being the master and ns2 the salve
Those two servers usually have either a similar IP address with a different third octet or
completely different IPs
In a redundant network the two servers would be on separate locations/ISPs or just on
separate ISPs
"A second name server splits the load with the first server or handles the whole load if the
first
server is down." O'Reilly, DNS and BIND (Fourth Edition)
6) Brief bind config file explanation for authoritative servers
Master
options {
directory "/var/cache/bind";
# do NOT want your authoritative server to be recursive as well because of
# security and performance reasons
recursion no;
allowtransfer { none; }; # or put the IP of the slave server or slave/master
dnssecvalidation auto;
authnxdomain no; # conform to RFC1035
listenonv6 { any; };
};
zone "site.edu" in {
type master;
file "/path/to/file/movie.edu";
# IP of the slave or slave/master that are allowed to receive the specific zone file
allowtransfer { xxx.xxx.xxx.xxx; };
};
19
22. mail will go to mail.example.org. If both values are equal (ex. MX10 and MX10) then it will
loadbalance between the two in a way that smtp hosts would then round robin between the
two
hosts.
Round Robin: http://en.wikipedia.org/wiki/Roundrobin_DNS
8) DNS uses TCP and UDP port 53
DNS uses TCP and UDP port 53 for queries. TCP port 53 is used for transfer of zones over
the
external network and is usually blocked for protection purposes which will change in the future
because. As Scott Hogg, CTO for Global Technology Resources, Inc. (GTRI), nicely said
"The reality is that DNS queries can also use TCP port 53 if UDP port 53 is not accepted."
"the
practice of denying TCP port 53 to and from DNS servers is starting to cause some problems.
There are two good reasons that we would want to allow both TCP and UDP port 53
connections to our DNS servers. One is DNSSEC and the second is IPv6."
9) The path of DNS resolution of a host name
21
23. 10) When a page is asked for from a site that uses CDN: What a browser gets and from
where.
Explanation: When your browser requests example.com it has to get the IP address from your
IPS's DNS server (this process is explained in “The path of DNS resolution of a host name”).
After the browser gets the IP address it opens a connection to it and gets the page. That page
consists of:
1. Dynamic content: Which is loaded from the Origin Server
2. Third Party content: Which is loaded from the Third Party Server (which can be google ads,
images from pinterest, facebook images, youtube videos, etc.)
3. Static content: Which is loaded from MaxCDN edge/flex boxes nearest to the client to the
client using Anycast or GeoDNS.
Explain anycast:
Anycast is networking technique where the same IP prefix is advertised from multiple
locations. It then uses one of the two methods to determine where to route. The first method
is determining the routing protocol costs and also the status of the server (response time,
number of requests, etc.). The other method being the upstream provider, partially, manually
setting the shortest path to the IP. As soon as a BGP announcement drops in one part of the
network, traffic will be rerouted to the other nearest advertised location with the same ASN.
22
24.
Explain geoDNS:
GeoDNS is basically routing to unicast IPs, usually with the same service and the same
content, depending from which part of the world request came from. GeoDNS uses the DNS
server and a plugin with a list of GeoIPs which can be found for free or recieved monthly
using a commercial service. What the plugin does is basically create ACL (access control
lists) and connect those ACLs to BIND views. Bind views have the ability to give a different
zone file for the same domain/zone thus controlling which server is hit (reached) when
requesting a resource.
Example links:
History of DNS:
http://cyber.law.harvard.edu/icann/pressingissues2000/briefingbook/dnshistory.html
http://en.wikipedia.org/wiki/Domain_Name_System#History
http://www.cybertelecom.org/dns/history.htm
http://tools.ietf.org/html/rfc882
http://tools.ietf.org/html/rfc883
http://tools.ietf.org/html/rfc1034
http://tools.ietf.org/html/rfc1035
ARPANET
http://en.wikipedia.org/wiki/ARPANET
List of root servers and their IPs
https://www.iana.org/domains/root/servers
http://www.internic.net/domain/named.root
Updated list of TLDs
https://data.iana.org/TLD/tldsalphabydomain.txt
https://www.iana.org/domains/root/db
PDNS or BIND:
http://www.quora.com/DomainNameSystem%28DNS%29/WhichisbetterBindorPowerD
NS
Config file example for recursive DNS
https://www.digitalocean.com/community/tutorials/howtoconfigurebindasacachingor
forwardingdnsserveronubuntu1404
Root servers and news
http://www.rootservers.org/
https://www.iana.org/ Internet Assigned Numbers Authority
Configuring authoritative only DNS server
https://www.digitalocean.com/community/tutorials/howtoconfigurebindasanauthoritativeo
nly
dnsserveronubuntu1404
Example zone
https://www.centos.org/docs/5/html/Deployment_GuideenUS/s1bindzone.html
23