Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Nginx Internals

113,408 views

Published on

Nginx Internals

Published in: Technology, Education
  • If you are looking for customer-oriented academic and research paper writing service try ⇒⇒⇒ WRITE-MY-PAPER.net ⇐⇐⇐ liked them A LOTTT Really nice solutions for the last-day papers
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://redirect.is/fyxsb0u } ......................................................................................................................... Download Full EPUB Ebook here { https://redirect.is/fyxsb0u } ......................................................................................................................... Download Full doc Ebook here { https://redirect.is/fyxsb0u } ......................................................................................................................... Download PDF EBOOK here { https://redirect.is/fyxsb0u } ......................................................................................................................... Download EPUB Ebook here { https://redirect.is/fyxsb0u } ......................................................................................................................... Download doc Ebook here { https://redirect.is/fyxsb0u } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • You can try to use this service ⇒ www.HelpWriting.net ⇐ I have used it several times in college and was absolutely satisfied with the result.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Thankyou so much! Certainly the best customer service I've had in a while. Happy with the way you guys worked with me. ☞☞☞ https://w.url.cn/s/Aaxmqpl
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hi, I read your guide on how to make my breasts bigger and I have to report some good news! I've been using your techniques for about 2 weeks now and I'm already starting to feel more like a woman! I can wear all sexy clothes and please my boyfriend in ways I've never been able to before. Thank you thank you!! 》》》 https://t.cn/A6Li7BTH
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Nginx Internals

  1. 1. Nginx Internals Joshua Zhu 09/19/2009
  2. 2. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  3. 3. Source Code Layout  Files  $ find . -name "*.[hc]" -print | wc –l 234  $ ls src core event http mail misc os  Lines of code  $ find . -name "*.[hc]" -print | xargs wc -l | tail -n1 110953 total
  4. 4. Code Organization  core/  The backbone and infrastructure  event/  The event-driven engine and modules  http/  The HTTP server and modules  mail/  The Mail proxy server and modules  misc/  C++ compatibility test and the Google perftools module  os/  OS dependent implementation files
  5. 5. Nginx Architecture  Non-blocking  Event driven  Single threaded[*]  One master process and several worker processes  Resource efficient  Highly modular
  6. 6. The Big Picture
  7. 7. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  8. 8. Memory Pool  Avoid memory fragmentation  Avoid memory leak  Allocation and deallocation can be very fast  Lifetime and pool size  Cycle  Connection  Request
  9. 9. Memory Pool (cont’d)  ngx_pool_t  Small blocks  Large blocks  Free chain list  Cleanup handler list  API  ngx_palloc  memory aligned  ngx_pnalloc  ngx_pcalloc
  10. 10. Memory Pool Example (1 Chunk)
  11. 11. Memory Pool Example (2 Chunks)
  12. 12. Buffer Management  Buffer  Pointers  memory  start/pos/last/end  file  file_pos/file_last/file  Flags  last_buf  last_in_chain  flush  in_file  memory  …
  13. 13. Buffer Management (cont’d)  Buffer chain  Singly-linked list of buffers  Output chain  Context  in/free/busy chains  Output filter  Chain writer  Writer context
  14. 14. String Utilities  ngx_str_t  data  len  sizeof() - 1  Memory related  String formatting  String comparison  String search  Base64 encoding/decoding  URI escaping/unescaping  UTF-8 decoding  String-to-number conversion
  15. 15. Data Structures  Abstract data types  Array  List  Queue  Hash table  Red black tree  Radix tree  Characteristic  Set object values after added  keep interfaces clean  Chunked memory (part)  efficient
  16. 16. Logging  Error log  Level  Debug  Access log  Multiple logs  Log format  variables  Per location  Rotation
  17. 17. Configuration File  Directive  name  type  set  conf  offset  post  Parsing  ngx_conf_parse  Values  init  merge
  18. 18. Configuration File (cont’d)  Block  events  http  server  upstream  location  if  Variables  Buildins  Other types  http_  sent_http_  upstream_http_  cookie_  arg_
  19. 19. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  20. 20. Master and Workers  Master  Monitor workers, respawn when a worker dies  Handle signals and notify workers  exit  reconfiguration  update  log rotation  …  Worker  Process client requests  handle connections  Get cmd from master
  21. 21. Master Process Cycle
  22. 22. Worker Process Cycle
  23. 23. Inter-process Communication  Signals  Channel  socketpair  command  Shared memory  Connection counter  Stat  Atomic & spinlock  Mutex
  24. 24. Event  ngx_event_t  Read  Write  Timeout  Callbacks  Handlers  ngx_event_accept  ngx_process_events_and_timers  ngx_handle_read_event  ngx_handle_write_event  Posted events  Posted accept events queue  Posted events queue
  25. 25. Time Cache  The overhead of gettimeofday()  Time cache variables  ngx_cached_time  ngx_current_msec  Time strings  ngx_cached_err_log_time  ngx_cached_http_time  ngx_cached_http_log_time  Timer resolution  Interval timer  setitimer()
  26. 26. Events and Timers Processing
  27. 27. Timer Management  Actions  Add a timer  Delete a timer  Get the minimum timer  Red black tree[*]  O(log n) complexity
  28. 28. Accept Mutex  Thundering herd  Serialize accept()  Lock/unlock  Listening sockets  Delay
  29. 29. I/O  Multiplexing  kqueue/epoll  NGX_USE_CLEAR_EVENT (edge triggered)  select/poll/dev/poll  NGX_USE_LEVEL_EVENT (level triggered)  …  Advanced I/O  sendfile()  writev()  direct I/O  mmap()  AIO  TCP/IP options  TCP_CORK/TCP_NODELAY/TCP_DEFER_ACCEPT
  30. 30. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  31. 31. Important Structures  Connection  ngx_connection_t  HTTP connection  ngx_http_connection_t  HTTP request  ngx_http_request_t  headers_in  headers_out  …
  32. 32. Virtual Servers  Address  Port  Server names  Core server conf
  33. 33. Locations  Location tree  Static  Regex  = ^~ ~ ~*  Per-location configuration  Value  inheritance  override  Handler  Named location  try_files/post_action/error_page
  34. 34. HTTP Contexts  Types  main_conf  srv_conf  loc_conf  Request  ngx_http_get_module_main_conf  ngx_http_get_module_srv_conf  ngx_http_get_module_loc_conf  Parse conf file  ngx_http_conf_get_module_main_conf  ngx_http_conf_get_module_srv_conf  ngx_http_conf_get_module_loc_conf  Module context  ngx_http_get_module_ctx  ngx_http_set_ctx
  35. 35. HTTP Handling  Receive data  Parse the request  Find the virtual server  Find the location  Run phase handlers  Generate the response  Filter response headers  Filter the response body  Send out the output to the client
  36. 36. Request Parsing  Request line  Headers  Interesting tricks  Finite state machine  ngx_strX_cmp
  37. 37. Phases and Handlers  Phases  POST_READ  SERVER_REWRITE  FIND_CONFIG  REWRITE  POST_REWRITE  PREACCESS  ACCESS  POST_ACCESS  TRY_FILES  CONTENT  LOG  Phase handler  Checker  Handler  Next
  38. 38. Phases and Handlers (cont’d)  Phase engine  Handlers  server_rewrite_index  location_rewrite_index  r->phase_handler  Default checkers  ngx_http_core_generic_phase  ngx_http_core_find_config_phase  ngx_http_core_post_rewrite_phase  ngx_http_core_access_phase  ngx_http_core_post_access_phase  ngx_http_core_try_files_phase  ngx_http_core_content_phase
  39. 39. Phases and Handlers (cont’d) phase modules POST_READ realip SERVER_REWRITE rewrite REWRITE rewrite PREACCESS limit_req, limit_zone, realip ACCESS access, auth_basic CONTENT autoindex, dav, gzip, index, random_index, static LOG log
  40. 40. Filter Chain  Singly-linked list like (CoR)  Filter response only  Header filter  Body filter  Send out the response  ngx_http_send_header  top_header_filter  ngx_http_output_filter  ngx_http_top_body_filter  ngx_http_header_filter  ngx_http_copy_filter  ngx_http_write_filter  Process order
  41. 41. Filter Chain Example
  42. 42. HTTP Handling Example  curl -i http://localhost/
  43. 43. HTTP Keep-Alive  Request memory reuse  Connection memory shrink  Keep-alive timeout  Request count
  44. 44. Subrequest  Filters  Addition filter  SSI filter  Maximum subrequests
  45. 45. Internal Redirect  Return a different URL than originally requested  Examples  try_files  index/random_index  post_action  send_error_page  upstream_process_headers
  46. 46. Upstream  Hooks  input_filter_init  input_filter  create_request  reinit_request  process_header  abort_request  finalize_request  rewrite_redirect  Modules  FastCGI  Proxy  Memcached  Event pipe  Load balancer
  47. 47. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  48. 48. Mail Proxy  Sequence diagram
  49. 49. Mail Proxy (cont’d)  Mail session  Command parsing  Packets relay  Things you can do  Load balancing  Authentication rewriting  Black lists/white lists
  50. 50. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  51. 51. General Module Interface  Context  index & ctx_index  Directives  Type  core/event/http/mail  Hooks  init_master  called at master process initialization  init_module  called when the module is loaded  init_process  called at worker process initialization  exit_process  called at worker process termination  exit_master  called at master process termination
  52. 52. Core Module Interface  Name  Hooks  create_conf  init_conf  Examples  Core  Events  Log  HTTP
  53. 53. Event Module Interface  Name  Hooks  create_conf  init_conf  event_actions  add  del  enable  disable  add_conn  del_conn  process_changes  process_events  init  done
  54. 54. Mail Module Interface  Protocol  type  init_session  init_protocol  parse_command  auth_state  create_main_conf  init_main_conf  create_srv_conf  merge_srv_conf
  55. 55. HTTP Module Interface  Hooks  preconfiguration  postconfiguration  create_main_conf  init_main_conf  create_srv_conf  merge_srv_conf  create_loc_conf  merge_loc_conf
  56. 56. A “Hello World” HTTP Module • Creating a hello world! module  Files  ngx_http_hello_module.c  config  Build  ./configure –add-module=/path/to/hello/module  Configuration  location & directive
  57. 57. Agenda  Source code layout  Key concepts and infrastructure  The event-driven architecture  HTTP request handling  Mail proxying process  Nginx module development  Misc. topics
  58. 58. Auto Scripts  Handle the differences  OS  Compiler  Data types  Libraries  Module enable/disable  Modules order
  59. 59. Reconfiguration
  60. 60. Hot Code Swapping
  61. 61. Thank You!  My site: http://www.zhuzhaoyuan.com  My blog: http://blog.zhuzhaoyuan.com

×