Investing in the cloud is possibly one of the best—and most disruptive—decisions that a business does. With the plethora of positive information supporting the hosting of business data in the cloud, it becomes evident that the best thing for a modern business to do, is to either move data to the cloud or start everything within the cloud.
It is often assumed that managing data in the cloud is very easy and of course cheap; because of the minimum investment and affordability of hosting and storing data in the cloud. Yes, it is much cheaper than ‘on-prem’ storage, and somehow a lot easier, but be aware and cautious about managing data in the cloud. Wrong management could cause a very large disaster.
In my experience as a data scientist, and not as a data architect, let me describe what I consider the Top 5 pitfalls to avoid, in very plain language.
Avoid the Data Hoarder
In most cases, we assume that we have stored and archived everything. This is a big mistake. Just because storing data in the cloud is easy and affordable, it doesn’t mean that all data should be stored – only store the data that is truly need. In fact, start small by storing data that is highly transactional.
Once you get to that point, keep moving or storing new data in the cloud. If you decide to move and store everything in the cloud, you and/or your team will be confused about identifying the so-called ‘source of truth’. This is a big problem. When you need a very specific data point, it has to be accurate and consistent. Otherwise, your entire organization will spend more time refining and cleaning the data than making analyses and decisions.
In other words, what you saved in IT by moving data to the cloud will be wasted by your team trying to understand all the data in the cloud.
The Multi-Cloud Jungle
Yes, exactly as it sounds. No cloud technology meets all your organization’s needs. So, it’s not unusual for businesses and organizations to end up with multiple cloud solutions for multiple needs. And these solutions can span multiple teams such as HR, Finance & Accounting, Commercial, Marketing, Communications, social media, Logistics, etc. – this is great! Each team can deliver a ton of work as they have the best solution or technology.
However, there is a problem. When comprehensive analysis is needed, it takes forever, and it can’t be done easily! Everybody ends up downloading everything manually to Excel, copying, and pasting, then formatting, and more copying and pasting. We go back to the consequence of the previous point: if you ask 5 individuals about the same KPI, you will get 5 different numbers.
In multi-cloud environments, make sure that each technology has the right APIs, and your organization/team has the right tools capable to blend all data from each cloud tool within the same platform. This will ensure that the organization has visibility of the right data points.
Who-is-Who?
Unfortunately, this is one of the biggest issues, especially within multi-cloud environments. Not having the right profiles to administer and manage every layer within the cloud environment usually creates friction and confusion among analysts.
It is very important to set up the right profiles about ‘who-manages-what’, and ‘who-has-access-to-what’. The organization hierarchy does not mean data accessibility hierarchy. Make sure that everyone has access to the right and pertinent data and protect and restrict sensitive data.
Why is This Not Working?
Yes, often one of the things that I heard the most. The fact that everything is running in the cloud doesn’t mean that it will never fail to run.
It is very important to keep monitoring of the performance of your system or tool in the cloud. Make sure that all relevant operational metrics are followed so you can identify regular patterns.
The beauty of the cloud is that everything can be tracked with ease. Such as:
- Monitor how much data is stored and how the latency of your data is.
- Monitor the performance of your compute power as well as how much your CPU, memory, and RAM are used.
- Understand when the usage peaks are happening and why.
- Even individual activity can be tracked and analyzed.
All of that will provide understanding and insights about your operation and your team from a perspective that perhaps it was never seen before.
Data is Gone! How Could This Ever Happen?
I’m very sorry to say this, but not having the right data security strategy in place creates a huge risk.
When acquiring a cloud solution, work with the provider or a team of experts about how to create a contingency plan to always back up your data. Who is storing what, when, where, and how much does it cost? Do not assume that because you are using a tool or system in the cloud all the data that you produce belongs to you.
Even worse, make sure you have the right security and data protection strategy in place. The last thing that anybody wants is to fail to anticipate a cyber-attack. And you heard right, I said ‘anticipate’, because it will happen. It is just a matter of time. So, no need to panic, just anticipate a potential risk.
Also, don’t think that by having data ‘on-prem’ it’s more secure. A human being is still the most unsecured piece of the organization and can be easily “hacked” through social engineering ploys.
As a final thought, your business strategy should include a data strategy, without this, your business strategy will fail.
