Many large scale web site architects and developers have long felt the constraints of living in the 4 KB limit imposed upon the cookie space in a browser. The desire and business need to associate valuable data with a session or user as they traverse through your web site can open many doors of opportunity for a web site, including the notion of personalization. Personalization’s goal is to deliver a highly relevant and optimized user experience in order to provide a better user experience and achieve increases in conversion and revenue.
Historically, tidbits of information have been squirreled away in cookies as the most scalable way of associating rapidly changing data with a user’s session, as most server side solutions that attempted to store and share this rapidly changing session state across several thousand stateless applications servers have had significant scalability issues for large scale sites like eBay, which require a stateless architecture for scalability and manageability. Approaches using traditional OLTP systems like Oracle have also suffered from significant scalability issues from the high transaction rate of write operations that lead to excessive cost in hardware and software licenses to support the load of a large scale site like eBay with over 2 billion URL requests per day.
eBay’s Kernel Framework team sought to provide a technology platform that would enable eBay to have a more rich and effective personalization strategy. The vision was to provide each of the several thousand application servers running the eBay site the ability to access and leverage an instantly updateable, globally shared, personalization data system on every URL request. This system would have to be both cost effective and highly scalable for future growth in both data size and number of transactions.
The eventual solution implemented by eBay was a system built upon the MySQL Memory Engine as the core cache engine. MySQL’s Memory Engine with its rich SQL capabilities and flexibility, along with its ability to perform significantly more read/write operations per second (loosely transactions per second) on a low cost hardware platform than several other technologies considered, while also having a very attractive price point, was a clear choice for eBay.
This presentation will discuss the challenges of attempting to provide such a large scale personalization technology platform and how eBay leveraged the MySQL Memory Engine and collaboration with MySQL AB (eBay leveraged the power of open source and donated a significant enhancement it had developed to MySQL, which MySQL productionalized) to produce a robust and scalable system.
Chris Kasten was the principal architect and engineer for eBay’s custom Java data access layer framework for over 5 years, which now processes over 45 billion database transactions per day for eBay’s website. He has 9 patents pending relating to data access layer, caching, and security technologies. He currently leads the eBay Kernel Data Access Layer Framework team, which architects, designs and develops technology solutions for eBay’s demanding data access and caching needs. Prior to eBay, Mr. Kasten was a consultant for IBM Software Group, as well as an independent consultant for several years, working in the telecom and aerospace industries.
View a complete list of MySQL contacts.