High Availability and Scalability Patches from Google

Ben Handy (Google), Justin Tolmer (Google)
Average rating: *****
(5.00, 1 rating)

Like most users, Google wants MySQL to be perfect. Perfect means that it should scale linearly on large servers, never crash and be trivial to manage. We will describe the changes we have published, document their benefit with benchmark results, and describe our experience as a member of the MySQL developer community. Anything we implement is deployed on critical servers within a few months. This makes for interesting times.

We have enhanced InnoDB and MySQL to scale on servers with more than 4 cores for read-only and read-write workloads. We made the mutex code in InnoDB faster, refactored code to remove mutex hot spots, added instrumentation to report on mutex contention by mutex and by mutex caller and integrated MySQL with tcmalloc. We will provide benchmark results for 8 and 16 core servers to show that they run MySQL much faster than 4 core servers.

We have made replication more robust and easier to manage. This includes transparent slave failover, crash-safe slaves, checksums in binlog events and better support for hierarchical replication. Slaves are made crash-safe by fixing bugs that corrupt slave replication state on a crash. With these changes and the use of transactional replication, it is safe to crash and restart a slave without fear of duplicate transactions or a halted slave.

Global transactions IDs have been implemented. These provide unique binlog event identifiers to enable transparent slave failover and hierarchical replication. When a master fails a slave can change to the new master without losing transactions and without manually remapping its replication offset to the new master. Hierarchical replication is much easier to support when slave failover is transparent. In addition, slaves that restore a backup taken before a master failure can catch up on replication using other servers without remapping replication offsets.

We continue to add instrumentation to make the server easier to understand. New features include SQL commands that display mutex contention statistics, query plans for running queries and information on temporary tables open for all connections. We will describe interesting things that can be done with these details.

Ben Handy

Google

Ben Handy works on InnoDB scalability issues at Google.

Photo of Justin Tolmer

Justin Tolmer

Google

Justin Tolmer works on MySQL Replication and High Availability at Google.

Comments on this page are now closed.

Comments

Picture of Marian Marinov
Marian Marinov
02/12/2010 2:58pm PST

No slides, no video… is this a ghost lecture ?

Co-presented By:

O'Reilly Media MySQL/Sun Microsystems
  • Kickfire
  • Virident
  • Infobright, Inc
  • JasperSoft
  • Intel
  • Advanced Micro Devices
  • BIRT Exchange by Actuate
  • Calpont
  • Canonical
  • Continuent
  • Dolphin Interconnect Solutions
  • Facebook
  • HiT Software, Inc.
  • IBM
  • iDashboards
  • Oracle
  • Pentaho
  • R1Soft
  • Schooner Information Technology
  • SQLstream
  • Ticketmaster
  • Zmanda, Inc.
  • Linux Journal

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Sharon Cordesse at scordesse@oreilly.com

Download the MySQL Sponsor/Exhibitor Prospectus

Media Partner Opportunities

Download the Media & Promotional Partner Brochure (PDF) for information on trade opportunities with O'Reilly conferences or contact mediapartners@ oreilly.com

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

MySQL Conference Newsletter

To stay abreast of conference news and to receive email notification when registration opens, please sign up for the MySQL Conference newsletter.

Contact Us

View a complete list of MySQL contacts.