Open source database specialist MariaDB has released a new product called MariaDB Platform X3, aimed at helping customers combine their transactional systems with analytical systems for the first time, but what does that mean and why should you care?
Traditionally, transactional and analytical workloads have been very separate entities. Transactional databases tend to be comprised of row-based storage, with indexes and some degree of clustering or replication. Analytical systems tend to be historic data so are often columnar storage with aggregated queries and no indexes.
Bringing these together, into what Gartner calls a hybrid transactional/analytical processing (HTAP) environment, brings a range of potential advantages, from better access to real time transaction data for analytics, to more flexibility with infrastructure and hardware provisioning, which naturally impacts costs.
Speaking on stage this week during MariaDB's OpenWorks event in New York, the vendor's CEO Michael Howard said: "Convention told us that SQL can't scale, and we debunked that. Convention also tells us that you have to separate analytics from transaction processing, and we are going to debunk that too.
"The implication is, while you are overseeing the operations of your service, your competitor is focusing on new frontiers of monetisation. If your database doesn't have the capability to add analytics to a transactional workload, or is constrained by anaemic database templates, you will not be able to meet customer expectations, compete well or win."
How does it work?
The key to developing an HTAP system like this is to essentially allow different storage engines, which can handle different types of workloads - specifically analytics and transactional processing - to operate under the same schema and in the same application.
Shane Johnson, senior director of product marketing at MariaDB explained during OpenWorks that X3 is a combination of two existing MariaDB products: TX for transactional and AX for analytical workloads.
Combining the two breaks down what Johnson called the 'brick wall' that traditionally existed between app/dev teams and their OLTP environments, and the BI/data science function and their OLAP environment.
"What we are doing at MariaDB Platform is not intended to replace your data warehouse, that still exists and gathers data, but let's bring some analytics into the backend database for customer-facing applications," Johnson added.
Digging into those two products, TX is built out from a combination of the InnoDB or MyRocks storage engine with the MariaDB MaxScale database proxy. AX is roughly a combination of ColumnStore 1.2 for storage with MaxScale. So the combined X3 platform isn't one, unified database but an architecture that allows the two workloads to coexist.
"This is the first phase of the journey," Howard told Computerworld UK. "There will be an X4, further marrying this and putting machine learning in MaxScale to do [query routing] on your behalf.
"I have a smorgasbord of futures that is part of this. Today was more about saying, hey, you can do it, and more importantly not only can you do it but can you please think about it? Because, you are doing an injustice to your own creativity and what you can do for your customers...so I had to get that out."
Origins of X3
Howard told Computerworld UK this week that X3 originated from an argument he had with Oracle founder Larry Ellison during his time with the vendor.
"Larry Ellison and I got into a big argument about how to go to market in data warehousing," he said. "At that time I was steadfastly arguing that data warehousing was separate from transactional processing and Larry was steadfast in his belief that they would be coalesced. I ended up convincing him that they had to be separate and that is what happened at Oracle.
"When I learned about the architecture of MariaDB, having these different storage engines, the dream came together for me. It was just not possible at Oracle because they didn't have these storage engines."
Howard said he has been asking the product teams about the possibility of a combined transactional and analytical platform since "the day I got here" and that "it is still a journey and we still have things to do and coalesce and make better".
Why?
The benefits of combining transactional and analytical workloads tends to come down to scale and flexibility. As the vendor outlined at the time: "[X3] can uniquely scale out transactions or analytics to support business growth goals. With a single unified product, [it] reduces complexity and increases operational and analytical efficiency, giving application users full visibility and analytical access to historical data."
"Customers are outgrowing their OLTP, which constrains your analytics," Johnson said. "Some customers need more historical data than three to six months. Then there are those customers that have storage limitations or query times are too long. The other group is SaaS companies (like ServiceNow), which are data driven and they need an open-ended, self-service way to access that."
In terms of getting there, MariaDB is also well suited to hybrid deployments, which is handy for existing customers because "customers can start on-premises for transactional and just add analytics and place it in the cloud," as Johnson outlined.
This also gives customers flexibility when it comes to their hardware and, subsequently, costs.
"Under the covers these can run on different infrastructure, servers, locations," Johnson said. "From a hardware perspective this means for instances running transactions I might want very fast SSD but average CPU, for analytical I want a gigantic spinning disk and maybe a big processor with lots of cores. So you can get more efficient in allocating hardware."
Customers
The natural fit for customers appears to be large retailers or ecommerce operations, but Howard admits that there is no prototypical customer base for this product, aside from launch customer Qberg. "I think we will find that out, I don't really know the answer to that," Howard said.
Italian market research company QBerg has moved to X3 from the Percona distribution of MySQL, which was limiting its ability to query historical data to just the last four months.
Germán Castro Pérez, chief techniology officer at QBerg said as part of a statement: "By bringing MariaDB’s analytical solution together with our MariaDB transactional environment, we were able to store a considerably larger volume of historical data for analytics, with ample room for more still, and we greatly decreased overall processing time. Using MariaDB as a platform to connect our transactions and analytics has given us the ability to offer a new level of excellence to our customers."
