Just-In-Time Scalability: Agile Methods to Support Massive Growth

Eric Ries (Kleiner Perkins Caufield & Byers), Chris Hondl (IMVU)

In the course of six months IMVU’s user base quadrupled in size. At the start of this period we were bottlenecked on a single central database. During these six months we evolved IMVU’s architecture to use caching with memcached, replication, horizontal and vertical partitioning to support this growth.

This session will focus on the techniques used to incrementally add scalability without having to make large changes to the application layer or disrupt ongoing feature development by the rest of the team. We’ll look specifically at implementing horizontal partitioning in a way that makes writing scalable application code easy for non-DB experts.

  • not all or nothing
  • migrate one system or one table at a time
  • measure what to scale (based both on slow queries AND frequent queries)
  • how to write scripts to measure
  • mix of techniques, and when to use which technique (vertical, horizontal, cache, read replication)
  • implementing cache coherency (cacheclass), memory based and no disk based
  • have a plan, but invest in implementation just ahead of growth
  • avoid doing work that is uneccessary (e.g., Ries was absolutely convinced we needed to partition tables tracking virtual currency and virtual goods, yet we still haven’t partitioned them)
  • handling large volume usage and reporting data
  • “desharding:” pulling views of horizontally partitioned back onto a database for running reports
Photo of Eric  Ries

Eric Ries

Kleiner Perkins Caufield & Byers

Eric Ries became a Venture Advisor at Kleiner Perkins Caufield & Byers, after co-founding and serving as Chief Technology Officer of IMVU. He is the co-author of several books including The Black Art of Java Game Programming (Waite Group Press, 1996). While an undergraduate at Yale Unviersity, he co-founded Catalyst Recruiting. Although Catalyst folded with the dot-com crash, Ries continued his entrepreneurial career as a Senior Software Engineer at There.com, leading efforts in agile software development and user-generated content. In 2007, BusinessWeek named Ries one of the Best Young Entrepreneurs of Tech. He serves on the advisory board of a number of technology startups including pbWiki, Bunchball, FooMojo, Causes and KaChing.

Chris Hondl

IMVU

Chris Hondl was responsible for application scalability and availability of the IMVU web site during much of the past two years. Prior to joining IMVU, Hondl led the development of AlterCast, Adobe Systems’s first server software product, and co-founded Sandcastle, Inc., which was acquired by Adobe Systems in 1997.

Sponsors
  • Kickfire
  • Zmanda, Inc.
  • Continuent
  • EDS
  • JasperSoft
  • Sun Microsystems
  • Symantec Anti-Virus Software
  • XAware
  • Data Direct Technologies
  • Dolphin Interconnect Solutions
  • Hewlett Packard
  • Infobright, Inc
  • Linagora
  • Microsoft
  • OpSource
  • Oracle
  • Pentaho
  • R1Soft
  • Red Hat
  • Ticketmaster
  • TechRepublic

Contact Us

View a complete list of MySQL contacts.