Holiday Shopping . . . Are You Prepared? More Importantly, is Your Database?

During this year’s holiday shopping season craziness, I thought it would be a good time to share some experiences we’ve had with our e-commerce customers—experiences that show why they chose to invest in ClustrixDB product licenses and why they’ve expanded their implementations multiple times. While modern e-commerce sites use a variety of scale-out database technologies, we see customers being very selective about where they purchase production licenses, rather than deploy free open source technology. ClustrixDB is used as a primary operational database for major e-commerce sites such as Rakuten:Global Market (#1 e-commerce company in Japan), Photobox (#1 photo printing site in Europe), MakeMyTrip (#1 travel reservation site in India), as well as fast-growing sites such as nomorerack (U.S.) and iOffer (U.S.). All of these customers use minimal DBA staff to run day-to-day database operations with ClustrixDB.

Transactions Matter and Availability is Job #1

In e-commerce, transactions really matter. ACID properties are taken for granted by developers and “four nines” availability is expected. In particular, transaction durability in failure modes becomes a must to ensure integrity when recovering customer purchases, reservations, or requests. And most of these transactions are relational transactions that connect data across customers to products to price. Some simple, noncritical transactions—such as retrieving product catalog information for display—in e-commerce sites are suitable for document databases, and for that part of the architecture we generally see customers quite happily using the free open source distributions.

Our experience with Rakuten has taught us just how important predictable transaction performance and four nines of availability is to e-commerce sites. The Rakuten CTO team put ClustrixDB through its paces during evaluation to validate consistently good transaction performance of ~10ms to deliver an engaging user experience for applications such as the Kobo book reader. And each failure mode was tested for time to full recovery. Large online shopping experience sites such as Rakuten have farms of databases behind application components, and Rakuten’s internal benchmark is to achieve the same TCO as a public cloud. Our ability to deliver on business-critical-level SLAs while beating those TCO targets has made ClustrixDB an essential part of the architecture.

As a result of our work with Rakuten, we constantly measure our production clusters to evaluate availability over time. In 2013 we’ve been consistently at four nines across all customers, and many customers have seen zero downtime over the entire year.

Big Unpredictable Peaks

Photobox largeSuccessful promotions, flash deals, or holiday events such as Cyber Monday can create particular challenges. Photobox is an early ClustrixDB customer that experienced a several fold increase in usage during the Christmas season. This peak event requires pre-provisioning of extra capacity, and we need to ensure that “no fuses blow” in the database when the customer surge hits. 

A more recent example is nomorerack (NMR), a website that expected a 15x to 20x peak workload on Cyber Monday alone. As a result of their success in 2013, they were already close to peak usage of their existing ClustrixDB implementation, so the Clustrix support team worked with them to temporarily expand their database online by 230% for the holiday shopping influx. They had a great business result of 600% increase in revenue as outlined in our Clustrix Blog. NMR has the choice now to “flex down” the database cluster to what they expect their steady state to be after the holiday shopping sprees are over.

Holiday shopping is not the only event that drives big spikes in demand. Our company worked with India’s top travel e-commerce site, MakeMyTrip, to deploy ClustrixDB in a master-master configuration across three locations in India to ensure that they could deal with peak user demand by using each location to load balance across traffic in different sub-regions. This configuration was put to the test when one of India’s leading airline carriers, SpiceJet, ran a “can’t be beat” ticket promotion that caused every relevant travel booking site in India to crash—except MakeMyTrip. 

Understanding the Customer = Real-Time Analytics

While most customers initially came to us for scale, we quickly discovered the most painful performance problems were complex analytic-style queries consisting of multiple joins and aggregate operations. Digging deeper, we found new e-commerce applications are getting smarter about tracking, analyzing, and adapting to online customer behavior in real time. The application starts to look more like a mixed OLTP/OLAP workload, which is a sweet spot for ClustrixDB technology.

Working with nomorerack, the ClustrixDB team used our deep embedded instrumentation to remotely identify nomorerack which queries were most problematic. Then we were able to make specific suggestions to NMR’s development team on their application or tune parameters on the ClustrixDB to optimize performance. This type of ongoing query optimization is particularly important for a fast-growing site such as NMR, where they are constantly improving their application to better monetize user traffic.

For iOffer, two ClustrixDB clusters are deployed in a primary and disaster recovery (DR) configuration. The synchronized DR configuration is also used for analytics workloads, providing ad hoc reporting on live operational data while isolating those complex queries from the transactional load on the primary cluster. The massively parallel processing (MPP) analytic capabilities in ClustrixDB, similar to those of leading parallel data warehouse architectures, allow iOffer to run their analytics jobs without any ETL complexity or the cost of a separate data warehouse.

The Takeaway

Our takeaway from working with the new generation of e-commerce sites can be succinctly summarized—companies are betting their business on the cloud. A cloud database holds their most valuable customer data, so it requires that:

  1. The database be rock solid in production, even in unpredictable workloads and failures.
  2. The scale-out implementation be able to quickly expand online to meet surges in demand.
  3. Real-time analytic workloads be able to monetize up-to-date customer data. New e-commerce workloads are not pure OLTP—smarter sites increasingly rely on these analytic workloads.
  4. The customer service team be exceptional, with a passion to proactively work as an extension of the company’s team.

These requirements have sharpened our focus on both reliability engineering and 24×7 support to deliver consistently high levels of availability (four nines or greater) with cloud economics for these new workloads. The result for our customers has been that they are able to monetize extreme holiday shopping peaks in demand that can make or break their business for the year—and often pay back their database investment in hours or less.