What is BIG DATA

 

As Name suggests BIG Data is huge data created due to IOT, Apps and real time application data colelction

• Walmart handles more than 1 million customer transactions every hour.

• Facebook stores, accesses, and analyzes 30+ Petabytes of user generated data.

• 230+ millions of tweets are created every day.

• More than 5 billion people are calling, texting, tweeting and browsing on mobile phones worldwide.

The three different formats of big data are:

1. Structured: Organised data format with a fixed schema. Ex: RDBMS

2. Semi-Structured: Partially organised data which does not have a fixed format. Ex: XML, JSON

3. Unstructured: Unorganised data with an unknown schema. Ex: Audio, video files etc.

Main Challanges 

• Validity: correctness of data

• Variability: dynamic behaviour

• Volatility: tendency to change in time

• Vulnerability: vulnerable to breach or attacks

• Visualization: visualizing meaningful usage of data

Solutions -

Hadoop

Pig

Hive

CAssandra

Spark

Kafka

etc