Using BlackRay As a Storage Engine

Felix Schupp (SoftMethod GmbH)

The BlackRay data engine is an open source in-memory relational database, providing hybrid features from relational databases and search engines alike. Transactions, SQL and standard interfaces such as JDBC, ODBC and object oriented relational APIs are more commonly associated with databases. Fulltext search, fuzzy matching algorithms such as phonetic and synonym searches, as well as millisecond response times even on complex queries play more into the field of search engines. BlackRay offers a superset of these features, along with arbitrary wildcards in all fulltext queries. The design of the in-memory index takes all query optimizations into consideration for optimal query plan execution. While BlackRay offers an SQL parser and JDBC/ODBC connectivity via the Postgres protocol, this solution is not yet perfect.

MySQL offers a broad variety of features, most of which can be exposed through the storage engine interface architecture. The core of this talk deals with implementing BlackRay as a storage engine in MySQL. Our experiences from writing a simple adaptor to finally implementing the intimacy of the BlackRay index structures and exposing these features to MySQL are the major focus of this presentation.

However, the current storage engine architecture has several serious shortcomings. Some of these issues have been identified when implementing the above mentioned architecture. A more sophisticated solution to the query optimizer as well as a solution for the multiple token match problem in a single column need to be addressed in order to leverage the performance of BlackRay natively in MySQL.

We are also planning on releasing the code along with the upcoming 0.11 release of BlackRay this summer (probably around OSCON in July). The BlackRay project can be found at http://www.blackray.org

Photo of Felix Schupp

Felix Schupp

SoftMethod GmbH

Felix has been at the helm of SoftMethod since 2004. He received his degree in Computer Engineering from San Jose State University in 2000. Subsequently he was co-founder and Chief Operating Officer of PhysicianPal Inc, a San Francisco based solution provider for the healthcare industry, connecting insurances, workers compensation agencies and ambulatory surgery centers. SoftMethod, his current venture is establishing a stronghold in open source consulting and performance engineering in telco and large enterprise environments.

From 2000 until 2003 he served as an Instructor and Program Manager for UC Berkeley International programs, focusing on advanced Internet business engineering.

He has been an active participant in the open source community, and has been a speaker at several (un-)conferences and congresses, most recently the OpenSQL Camp, SAPO CodeBits and AstriEurop. Felix is also a sponsor and mentor in the academic cooperative project “Telling Test Stories” with the University of Innsbruck. In his rare spare time he enjoys getting some wind into the sails of his 1962 dragon or getting some powder under his snowboard.

  • Oracle
  • Monty Program
  • Calpont
  • Facebook
  • Gear6
  • Infobright, Inc
  • JasperSoft
  • Joyent
  • Kickfire
  • NorthScale, Inc.
  • Percona
  • Schooner Information Technology
  • Solid Quality Mentors (SolidQ)
  • Intel
  • Pentaho
  • Linux Pro Magazine

Sponsorship Opportunities

For information on exhibition and sponsorship opportunities at the conference, contact Yvonne Romaine at yromaine@oreilly.com

Download the O'Reilly MySQL Conference & Expo Sponsor/ Exhibitor Prospectus

Media Partner Opportunities

Download the Media & Promotional Partner Brochure (PDF) for information on trade opportunities with O'Reilly conferences or contact mediapartners@ oreilly.com

Press and Media

For media-related inquiries, contact Maureen Jennings at maureen@oreilly.com

O'Reilly MySQL Conference Newsletter

To stay abreast of conference news and to receive email notification when registration opens, please sign up for the O'Reilly MySQL Conference newsletter (login required).

Contact Us

View a complete list of O'Reilly MySQL Conference contacts.