Even though the finishing line is clear, the implementation is complex, because we have a hard truth in front of us: having data doesn’t mean that you are ready to perform big data analysis. In fact, before being able to use big data, we have to make sure that we have the necessary information infrastructure to be able to capture and store data; that eventually will be useful to perform the type of analysis that will inform our decisions. Now we will explain what’s needed to perform a big data analysis.
1. What’s the difference between big data and a common statistical analysis?
First of all, we should answer the following question: what’s the difference between big data and statistical analysis that most of the companies perform? The central core of the big data is the statistical analysis, but the difference resides on the volume, the velocity, and the variety of the analyzed data. The common statistical analysis can be performed with small samples of data using a program as simple as Excel. Big data, on the other hand, depends on stronger tools to be able to consume, transform and analyze data in a massive way and in real-time. Big data also is dependent on its own programming languages such as R, in order to manage the complexity of the required analysis. Even though it’s difficult to suggest a strong definition, the main difference is the volume and the complexity of the data that we are analyzing.
2. Why should you perform a big data analysis?
Big data analysis promises to tell stories with analytical support that helps us to be more efficient and profitable. In common statistical analysis, we use surveys to take a small sample and we assume that the population we are interested in behaves in the same way as the sample. On the contrary, with big data, we can analyze all of our users’ behavior, in a global and individual level. Having a complete view of the client’s behavior helps to make more accurate decisions regarding sales, marketing, inventory, etc. These decisions improve the company-client relationship and the ROI. According to Bain & Company, companies that use big data are more likely to execute decisions according to a plan, are more likely to have a growth of 10% + year after year, and make decisions more quickly and efficiently.
In a certain way, big data replaces the dependence on the manager’s intuitions. The manager still needs to use his own judgment, but instead of over-debating the merit of ideas, we use data to inform the decisions we make.
3. How to start using the big data?
When did big data start to be used a question arose: what problem could we solve with more data? Without a specific question, having a big data analysis is like having a website: the infrastructure itself is not worth much without the content that is intended to be presented. Once we have the question, we move on to the infrastructure phase, with another question:
4. What are our data sources?
What are the data sources we have available to tell stories about our company? We may have QuickBooks to manage providers and invoices; an ERP such as SAP or Oracle to manage inventory; transactional data in databases such as SQL; a CRM, etc. In the case of marketing, we have tools such as Google Analytics, AdWords, and Facebook, they are different sources that do not necessarily interact well with each other. Before being able to understand what we can do with big data, we need to understand what sources we have available and how accessible they are.
5. How is big data’s infrastructure?
According to Google, in order to be able to perform a data analysis you have to go through four phases: collect, transform, analyze, and visualize. To collect, we need to send the data to a joint storage so we can have all the sources integrated in just one place. Then we have to clean that data (the process of transforming) so it becomes useful. Data cleansing implies ensuring that all data is free of errors that could deviate the analysis.
Once the data is clean, the analyzing process begins. Analyzing data implies having software with enough capacity to process large amounts of data and then to produce results. Once again here we see a key difference with the common statistical analysis: Excel tends to close automatically when you demand a lot from it. Software like Google Bigquery has the analytical and processing capacity to perform this kind of analysis without breaking.
Finally, we have visualization, which allows us to reproduce data in a visual and friendly way. Some visualization programs also allow users without training in statistics or programming to manipulate the presentation of the data in order to draw their own conclusions.
For more information about how to implement big data in an organization please check this article.
6. How do you know you need big data?
Setting up the infrastructure to do big data analysis is expensive, and sometimes performing a statistical analysis is enough for a company. At Centrico Digital, we know that a manager is ready for big data when he/she begins to ask for more detailed reports, correlations, and attribution in marketing to understand better his/her operation and ROI. The main problem is that a lot of people want to go from scattered data sources such as AdWords, Analytics and Facebook, directly to the visualization, but the visualization programs don’t have the processing capacity to do more than representing data that has already been processed.
If we want to really make big data, we must put together the infrastructure of the future.
7. Why big data is important for your company?
In the information age, resisting decision-making with data is putting oneself at a strategic disadvantage. The future of any industry depends not on working more, but on working smarter. Big data is the future because it’s the door towards a new way of making decisions. Luckily, most of big data’s infrastructure can be set up in the cloud, avoiding high investment costs in buying and managing servers and allowing portability of the data between platforms.
Conclusion
Will 2019 be the year in which your company takes this step towards the analytical world? If you think this is the case, let’s start a conversation about how big data can transform your business, your team, and your decisions.
