High Performance at Massive Scale - Lessons Learned at Facebook

Data at facebook is growing at an incredible rate. More than half a million people sign up every day, and the 150 million existing users continue to add new data at an ever increasing rate. The rate of reads against the data is also increasing, with more than 50 million low latency random accesses a second. Building a system to handle these requirements is quite a challenge, and I’ll be sharing some of the lessons we’ve learned from it.

One of the critical components of the system is the memcached cluster. I’ll be discussing the ways we’ve maintained performance and reliability as the cluster has grown, and the techniques we’ve used to deal with the massive amounts of network traffic required. Some of these techniques are at the application layer such as clustering of data and analysis of access patterns. Others are at the system level such as dynamic throttling of network traffic and modifications to the kernel’s network stack.

I’ll also be discussing optimizations we’ve made to the PHP interpreter such as changes to reduce initialization cost with a large codebase, and lessons learned scaling MySql to tens of terabytes.

I’ll be sharing data on all of these topics, and discussing the impact that various optimizations have had on both scaling and performance.