What’s in store for the database industry in the year ahead? I have three guaranteed predictions: One, data volumes will grow and become more complex. Two, business leaders will look for new ways to extract value from data. And three, data security and governance will be crucial—and mistakes will be costly.
How can I be so sure about those three points? Because they have been constants in the database market for more than 25 years, with no signs of changing whatsoever.
That’s the easy part of my data predictions for the year ahead, but there’s much more. Based on a fast-moving, rapidly changing, highly innovative cloud database market over the past 12 months, the momentum points to other major trends. Here’s how I see them.
- Exabyte-size databases become mainstream. To appreciate how quickly databases are growing, go back to 1997. That’s the year Microsoft hosted “Scalability Day,” where it demonstrated Windows NT’s ability to manage a terabyte of data. Today, many businesses are managing petabytes of data—a thousand times more. Exabytes are next. That’s 1 million times as much data as just 24/25 years ago. The term “big data” will no longer be sufficient to describe the largest datasets in 2022.
- If data migration doesn’t kill you, it will make you stronger. Gartner has predicted that 75% of databases will be moved to the cloud, or created in the cloud, by the end of 2022. That trend is data migration of unprecedented scale and scope—and the cause for sleepless nights by the teams that must support those business-critical projects, which can take months to complete. Vendors will likely offer dozens of new tools and services to help with it all.
- Snowflake creates an industry-wide snowstorm. It’s impossible to ignore the impact that Snowflake has had in the world of data analysis and data sharing, and there’s little doubt that its influence and reach will continue to grow. The company has shattered the old style of centralized data warehouses with its Data Cloud, an overarching model that puts data science into the hands of more users. More cloud database providers will try to “be like Snowflake,” or join the party through partnerships such as ones Snowflake has in place with AWS, Google Cloud, and Microsoft.
- Database management finally gets easier. Database management has long been an esoteric skillset of partitioning, sharding, tabular data, joins, stored procedures, and indexes. Now, cloud databases are becoming much easier to provision, manage, and scale through new serverless capabilities, fully managed cloud services, and Oracle’s “self-driving” Autonomous Database. This should allow IT teams to focus more on business strategy and less on the guts of the system.
- Databases are free—forever! For years, database providers have offered cloud database services on a trial basis, typically for a few weeks or credits that, once exhausted, required users to pay. More recently, a growing number of vendors such as Cockroach Labs, DataStax, and Neo4j have introduced entry-level cloud databases that are free for as long as you use them. This makes it easier for developers to get started and build and test prototypes.
- The Big 3 cloud providers are the new center of gravity. AWS, Google Cloud, and Microsoft Azure may not have the #1 position as the database market leader—that distinction still belongs to Oracle—but they have something better: momentum. With their comprehensive families of purpose-built databases, analytics, AL/ML, etc., the leading public cloud providers are also emerging as the leading cloud database providers.
- Immovable objects (incumbents) meet irresistible forces (startups). The age-old paradox of physics now characterizes the cloud database market, where traditional vendors like Oracle, Teradata, and IBM are nearly impossible to up-root, yet relative newcomers like Databricks, Couchbase, Firebolt, and Yellowbrick Data are chipping away at the legacy installed base. The old guard must continue reinventing themselves with new services and innovations.
- Data distribution becomes a business conversation. Centralized data management is giving way to distributed data architectures, enabled by new capabilities and services such as change data capture, data fabrics, and distributed databases that are designed to replicate data across clouds and geographies. Executives and business managers will begin to pay attention because there can be advantages (customer proximity) and risks (regulatory compliance) in where data is located.
- CEOs seek to monetize data. Business leaders know there is latent value in the petabytes of data their organizations aggregate and manage (see my “guaranteed predictions” above), and they will increasingly look for ways to capitalize on one of their greatest assets. Here too, Snowflake points the way with its data marketplace, which counts more than 500 listings and growing. That’s one way to do it, and there are many others, not all of which involve buying and selling datasets.
- The world will surpass 800 different database management systems. You might think that after 50-plus years, the pace of development of database platforms would begin to fizzle. You would be wrong. Carnegie Mellon University’s comprehensive “Database of Databases” now lists 775 different database management systems—and it keeps growing. Recently added were Dremio, ShardingSphere, UnumDB, PancakeDB, and Mongrel. If you believe that choice is good, then the cloud database market will bring more goodness in 2022.
In some ways, this list only scratches the surface of all that’s happening in the world of databases. There are also universal databases, multi-clouds, hybrid clouds, data clouds, customer data management, security, AI/ML, data science, and on. If you blink, you may miss the next innovation. That’s not a prediction—that’s the reality in the cloud database market.
This article appears in the Predictions 2022 Edition of the Acceleration Economy Journal Download the Full Journal Here
