What is Big Data?

Big Data and its analysis techniques are at the centre of modern science and business. Every day, millions of transactions, emails, photos, video files, posts and search queries are generated that result in zettabytes of data. All that data is stored in databases on various places across the planet.

All that data potentially contains a wealth of information. By analyzing the data that is generated every day, governments, researchers and companies might discover knowledge that they could use for their benefit. For governments, this might be to prevent potential attacks. For researchers, it might be to develop new medicines. And for companies, it might be the best location to open a new store. The value of the knowledge is different for each type of organization, but the process to extract these insights out of the data is very similar.

Extracting valuable knowledge out of massive quantities of data is, however, more difficult than it sounds. Due to the sheer volume of data that is generated every day, databases grow massively, and it becomes difficult to capture, form, store, manage, share, analyze and visualize meaningful insights out of the data. For that reason, knowledge about how to deduce valuable information out of large sets of data has become an area of great interest. This domain of knowledge is collectively described as “Big Data.”

Although the importance of Big Data has been recognized over the last decade, people still have differing opinions on its definition. In general, the term Big Data is used when datasets cannot be managed or processed by traditional commercial software and hardware tools within a tolerable time. However, Big Data is more than just processing capabilities of the underlying data sets, and it has gradually evolved into an entire domain of study.

In the the Big Data Framework, we will therefore adhere to the following definition when we are discussing Big Data:

Big Data is the knowledge domain that explores the techniques, skills and technology to deduce valuable insights out of massive quantities of data.

The objective of the Big Data Framework is to discuss these techniques, skills and technologies in a structured approach, so that Big Data students are equipped with the knowledge to deduce valuable insights to support future decisions.