Data products and Big Data solutions

Working with data and performing data analysis and analytics operations requires specialized knowledge. In most organizations, business users are only interested in finding the outcomes and solutions to certain questions – they require a data product.

Data products

A data product is an application that runs data analysis or analytics operations upon a certain input, and mostly have an easy-to-understand user interface. The users of data products do not exactly understand all underlying algorithms and are only able to run certain queries to find specific answers. Building data products can therefore be considered one of the key objectives of Big data.

Due to the growing interest in Big Data and its increased use in enterprise organizations, many data products have been developed. Commercial software companies bundle many data products together, and license these as Big Data solutions to enterprise organizations.

Data solutions

Big Data solutions are a quick way for enterprises to start leveraging the potential of Big Data analysis, because enterprises do not need to develop all required data products in-house. The downside of (commercial) Big Data solutions is that they are often expensive, and it is difficult to alter any of the underlying algorithms of the Big Data solution.

There are many Big Data solutions available on the market and almost every large Enterprise IT provider (Google, Amazon, Microsoft, SAP, etc.) now offers one or more Big Data solutions. Additionally, start-ups play a very important role in the development of Big Data solutions because they come up with new and innovative data products. The Big Data Framework has been developed from a vendor-independent perspective and therefore does not recommend any specific Big Data solutions.

Figure 1: Examples of popular Big Data solutions

Hadoop

It would not be possible to discuss Big Data solutions without mentioning Hadoop. Hadoop is an open-source software framework used for distributed storage and processing of dataset of big data using the MapReduce programing model. It consists of computer clusters built from commodity hardware. All the modules in Hadoop are designed with a fundamental assumption that hardware failures are common occurrences and should be automatically handled by the framework.

However, it is important to note that most Big Data solutions make use of the Hadoop framework as their underlying software framework. The term ‘Hadoop’ has therefore also become known as the ecosystem that connects different Big Data solutions (and commercial vendors) together.

To learn more about Big Data, visit our Big Data Knowledge Base. For more information, contact us at info@bigdataframework.org or drop us a message in the chatbox.