MySQL Row Change Event Extraction and Publish

Gene Pang (Google)
Replication and Scale-Out
Location: Ballroom A
Average rating: *****
(5.00, 1 rating)

This talk introduces a MySQL row change extraction system we built at Google. We have modified MySQL 5.1 source code to generate row-level change logs without use of triggers. The system continuously extracts row change events out of a large-scale replicated MySQL server cluster, converts these events sequentially to self-contained protocol buffer records, and then stores them to permanent GFS storage. The result real-time row change event streams are also replicated to various data centers and fed to various proprietary applications. As these row change event streams offer a complete and ordered sequence of transaction changes, they can be used for many purposes. They are used by many applications to track changes to databases and combined with simple filters, they are used to maintain derived views out of MySQL databases. As each row change record contains both the old and new table entry data, a lot of applications can avoid costly periodical full table extractions. Another usage case is to maintain database replicas in a non-relational storage infrastructure. Applications with high read bandwidth but low transaction consistency requirements can read from the non-relational storage infrastructure instead of MySQL servers.

Gene Pang

Google

Co-presented By:

O'Reilly Media MySQL/Sun Microsystems
  • Kickfire
  • Virident
  • Infobright, Inc
  • JasperSoft
  • Intel
  • Advanced Micro Devices
  • BIRT Exchange by Actuate
  • Calpont
  • Canonical
  • Continuent
  • Dolphin Interconnect Solutions
  • Facebook
  • HiT Software, Inc.
  • IBM
  • iDashboards
  • Oracle
  • Pentaho
  • R1Soft
  • Schooner Information Technology
  • SQLstream
  • Ticketmaster
  • Zmanda, Inc.
  • Linux Journal

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Sharon Cordesse at scordesse@oreilly.com

Download the MySQL Sponsor/Exhibitor Prospectus

Media Partner Opportunities

Download the Media & Promotional Partner Brochure (PDF) for information on trade opportunities with O'Reilly conferences or contact mediapartners@ oreilly.com

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

MySQL Conference Newsletter

To stay abreast of conference news and to receive email notification when registration opens, please sign up for the MySQL Conference newsletter.

Contact Us

View a complete list of MySQL contacts.