The Seven Deadly Sins of Big Data Projects
Brian White
By Brian White

Seven Deadly Sins .png


1. Stakeholders don't fully appreciate how Big Data can radically change their business

Big Data really is the single biggest change in the world of computing since eCommerce; don’t underestimate its importance. The amount of data in the world is proliferating at a rate never imagined before. Because it’s so important, you need a champion at a senior level that sees the potential to use this as an opportunity to forge ahead rather than be left behind. 


Big Data Tick.png

Get an executive sponsor and keep them engaged.

If you’re the CEO and your CMO, CIO or CDO isn’t already banging on your door to start a Big Data initiative – ask them what they’re doing to ensure your organisation is ready to embrace the  future. 


2. Lack of clearly defined business outcomes

It’s easy to dive into a Big Data project without having a clear view of exactly what you want out of it. How is this going to benefit my business? What can it transform? How will it increase sales through highly targeted marketing? What can it tell me about my business or market that I don’t already know? Will it shrink costs by reducing our Data Warehouse dependency?


 Big Data Tick.pngBe specific with your initial Use Case.

Implementing the solution is just the start; what you want to get to as quickly as possible is the clever analytics coming out the other side, be they prescriptive, predictive, diagnostic or descriptive.


 3. Absence of a data-driven culture

Big Data isn’t a silver bullet that will solve all your data problems, but it will certainly add value to the information you are already collecting.  Your competitors are already doing this stuff and are improving customer retention, driving new sales, and optimising distribution. How? Because they are seeing into the future with a high level of confidence.


Big Data Tick.pngWouldn't it be good to know as much about your customers as Facebook, Google and Amazon know about you? These are the visionary companies that invented Big Data and use these technologies every day. Join them. 


4. Over promising, under delivering

Yes, this is extremely powerful rhetoric, so keep your goals small to start with. It’s easy to get a Proof of Concept up and running, implementing a secure and highly available production system is another matter. To paraphrase McDonalds, definitely do ‘Start small, think big’, but forget ‘Scale fast’. Great if you’re a start-up, unnecessary for your Big Data landscape.


Big Data Tick.pngScale smart, not fast: scale up when you need the extra compute or disk, scale down during off-peak hours.



5. Lack of skilled team members

It’s a small resource pool out there. People who know what they are talking about, and more importantly, know what they are doing are hard to find. 


Big Data Tick.pngAgilityWorks bring a healthy dose of battle scars to save you some of the same. 



6. Being technology led, rather than outcome led

Ooh, look at the shiny, shiny! Hadoop, Spark, Pig, Hive, Kafka, Storm, Flume, Flink, Knox, Ranger, Python, Tableau. Irresistible names, aren’t they? And not without reason, they are truly exciting technologies that are changing the world. That’s just a taster of the wealth of options out there, so it’s important to balance creativity - in terms of what is technologically possible - and pragmatism - regarding complexity.


Big Data Tick.pngIT geeks love to play with new toys, so don’t lose sight of the end game. Big Data is still evolving and as always, some technologies will win out and persist while others may fall by the wayside.



7. Wrong technology or architectural decisions

You need to get off on the right foot, in terms of aligning your Big Data decisions, firstly with your business strategy, and then with your existing architecture and skillsets. Companies have invested in software for decades now so whatever you do needs to embrace an understanding of the existing solutions, while ensuring that the new capabilities are fully exploited. Many existing data warehouses and information hubs were built before this technology had been invented.


Big Data Tick.pngMake sure it all fits together and revisit what is best done where. This is a constant theme for our clients; those that use SAP, those that use Open Source and those that use a hybrid of solutions. 



New Call-to-action


Brian White
By Brian White

Brian heads up AgilityWork’s Big Data practice. He started his career as a Lotus Notes developer and is currently studying for a Master’s Degree in Big Data Analytics. Brian enjoys the incredible pace of technology change, helping our clients implement new ways of storing, managing and gaining insight from their data. Outside of work, his two young sons keep him busy, so he likes to keep fit. He’s completed the Iron Man triathlon (and has the “M-Dot” tattoo to prove it!)

Follow us on Social

And get access to even more digital insights


Just want to talk, call us on:
+44 (0)844 5610930

Join us

Life doesn't wait and neither should you. If you want to join a bunch of people intent on changing the world, you've come to the right place.


Latest opportunities