Flickr implemented a real-time collection of referrers to Photos, streams, sets and collections. Currently the data is gathered real time for every single user on Flickr. The data is then cooked and reported with a 24 hour lag for filtering out referrer SPAM. MySQL myISAM/INNODB, curl, Java are the only component used in the setup and it scales linearly. This talk is about building a model for capacity planning, scaling for triple the request rate, and scaling linearly for an intensive application.
Former Database Architect of Various Inc, Friendster, Flickr and current Architect at Rockyou. I’m an expert at scaling all application teirs but with a special emphasis in data layout, millisecond data retrieval on multi-terabyte data stores all using mySQL. I share my tips on mysqldba.blogspot.com
View a complete list of MySQL contacts.