1. Stakeholders don't fully appreciate how Big Data can radically change their business
Big Data really is the single biggest change in the world of computing since eCommerce; don’t underestimate its importance. The amount of data in the world is proliferating at a rate never imagined before. Because it’s so important, you need a champion at a senior level that sees the potential to use this as an opportunity to forge ahead rather than be left behind.
Get an executive sponsor and keep them engaged.
If you’re the CEO and your CMO, CIO or CDO isn’t already banging on your door to start a Big Data initiative – ask them what they’re doing to ensure your organisation is ready to embrace the future.
2. Lack of clearly defined business outcomes
It’s easy to dive into a Big Data project without having a clear view of exactly what you want out of it. How is this going to benefit my business? What can it transform? How will it increase sales through highly targeted marketing? What can it tell me about my business or market that I don’t already know? Will it shrink costs by reducing our Data Warehouse dependency?
Be specific with your initial Use Case.
Implementing the solution is just the start; what you want to get to as quickly as possible is the clever analytics coming out the other side, be they prescriptive, predictive, diagnostic or descriptive.
3. Absence of a data-driven culture
Big Data isn’t a silver bullet that will solve all your data problems, but it will certainly add value to the information you are already collecting. Your competitors are already doing this stuff and are improving customer retention, driving new sales, and optimising distribution. How? Because they are seeing into the future with a high level of confidence.
Wouldn't it be good to know as much about your customers as Facebook, Google and Amazon know about you? These are the visionary companies that invented Big Data and use these technologies every day. Join them.
4. Over promising, under delivering
Yes, this is extremely powerful rhetoric, so keep your goals small to start with. It’s easy to get a Proof of Concept up and running, implementing a secure and highly available production system is another matter. To paraphrase McDonalds, definitely do ‘Start small, think big’, but forget ‘Scale fast’. Great if you’re a start-up, unnecessary for your Big Data landscape.
Scale smart, not fast: scale up when you need the extra compute or disk, scale down during off-peak hours.
5. Lack of skilled team members
It’s a small resource pool out there. People who know what they are talking about, and more importantly, know what they are doing are hard to find.
AgilityWorks bring a healthy dose of battle scars to save you some of the same.
6. Being technology led, rather than outcome led
Ooh, look at the shiny, shiny! Hadoop, Spark, Pig, Hive, Kafka, Storm, Flume, Flink, Knox, Ranger, Python, Tableau. Irresistible names, aren’t they? And not without reason, they are truly exciting technologies that are changing the world. That’s just a taster of the wealth of options out there, so it’s important to balance creativity - in terms of what is technologically possible - and pragmatism - regarding complexity.
IT geeks love to play with new toys, so don’t lose sight of the end game. Big Data is still evolving and as always, some technologies will win out and persist while others may fall by the wayside.
7. Wrong technology or architectural decisions
You need to get off on the right foot, in terms of aligning your Big Data decisions, firstly with your business strategy, and then with your existing architecture and skillsets. Companies have invested in software for decades now so whatever you do needs to embrace an understanding of the existing solutions, while ensuring that the new capabilities are fully exploited. Many existing data warehouses and information hubs were built before this technology had been invented.
Make sure it all fits together and revisit what is best done where. This is a constant theme for our clients; those that use SAP, those that use Open Source and those that use a hybrid of solutions.