Load Balancing & Reverse Proxies with Varnish & More

Artur Bergman (Wikia/Fastly)
10:45am Tuesday, 12/08/2009
Location: Online

Varnish is a modern application level reverse proxy for web sites & services written with performance in mind. Varnish incorporates some advanced features to stretch the kernel as far as possible. Wikia relies heavily on varnish to serve a peak traffic of close to a gigabit/sec out of 3 different datacenters. Each one with two Varnishes working as a pair serving thousands of requests a second.

It’s innovative features include:
  • VCL—a domain specific language for configuration that gets compiled to native code
  • mmap log file—logs all headers with no performance penalty 8* mmap or malloc storage—lets the kernel deal with disk/memory allocation
  • hybrid thread/event model
This session will include:
  • Startup options for high performance (based on Wikia)
  • Configuration of storage system
  • Example of Wikias experience (including SSD based storage)
  • VCL cookie handling
  • Advanced VCL with embedded C code
  • Monitoring tools
Photo of Artur Bergman

Artur Bergman

Wikia/Fastly

Artur Bergman, hacker and technologist at-large, is the VP of Engineering and Operations at Wikia. He provides the technical backbone necessary for Wikia’s mission to compile and index the world’s knowledge. He is also an enthusiastic apologist for federated identity and a board member of the OpenID Foundation. In past lives, he’s built high volume financial trading systems, re-implemented Perl 5’s threading system, wrote djabberd, managed LiveJournal’s engineering team, and served as operations architect at Six Apart. His current interests extend to encompass semantic search, large scale infrastructure, open source development, federated instant messaging, neurotransmitters, and the future of cyborgs.