The concept of Big Data has several definitions, mainly due to the fact that the term "large" is very subjective and inaccurate. Nevertheless, today we come across this notion in almost every sector, the key here is to clarify what it actually means.
Enthusiasts of number measurements, would say that a large amount of data means that it has more than a few 100 TB's, or even - at least one Petabyte. Indeed, it is seen a lot now, but will it still be seen as much in 5 or 10 years? Others say that everything that leverages Hadoop, or has massively parallel processes, we can also include into this category. But to be honest, the most universal and safest definition of Big Data is that it is data which is too big for Online Transaction Processing. The main argument behind this statement is that it adapts to technological progress and does not get outdated.
Now when we understand the raw definition, let's take a look at what is going on deeper. From time to time, here and there, we hear about the fact that Big Data equates to huge profit. This is such a hot topic on the Internet, that when we read the statements from all the excited people, we might think that it is everything that we need to do – just invest in Big Data and spend the rest of our lives relaxing on a Hawaiian beach. However, it is not so simple, trust me. You have to be careful with a couple of important decisions, before you get started. First of all, remember - you do not have to invest in Big Data just because everyone else does.
Basically, your first step should be to recognize whether you really need the management architecture. Do you have the funds, time and effort, that you need to invest in the establishment of a new architecture such as NoSQL databases that are actually necessary to contain your company data? Is the data used by your organization a resource that greatly increases in volume over time, or are you able easily to predict the amount of memory you will require? It is worth remembering that relational databases are generally weaker in terms of access speed, but nevertheless they provide greater consistency which, in some industries, is the most important aspect.
If you have carefully thought through all of the above issues, and you continue to believe that big data is definitely worth your company’s investment, before you actually get started, you also have to check what the advantages and disadvantages of Big Data implementation are:
In summary, you need to come up with a strategy. Start by researching the market, especially the major companies with which you are competing. Then make a list of pros and cons. Ask yourself the following questions. Seriously think about the technology that you want to use, keeping in mind its broad spectrum - SQL, NoSQL, NewSQL, and Hybrid Database. Which one is better, Hadoop or MPP? Why do we need Big Data? What will we use it for? Will it increase our income? After you have answered all of these questions you are ready to get down to work, good luck!