Big Data Analytics
In modern society, data is considered as most precious wealth. To survive and compete in the digital world, the data centers for the business should be modernized, scalable, available, and secure. Big data is a large and complex dataset that cannot be handled by traditional software applications. There are a number of concepts related to big data: originally there were 3 concepts: volume, variety, and velocity. The term “big data” has a tendency to imply the utilization of predictive analytics, user behavior analytics, or certain methods that take out values from data.
Challenges of Big Data:
- Capturing data
- Data storage
- Data analysis
- Search, sharing, transfer, visualization, querying and updating information privacy
- Data source
Big data analytics is the mechanism to test the data which are large in size and varied. That is, big data is to discover hidden patterns, correlations between attributes, predict market trends, customer choices, and other effective information that can help an organization to make more-informed business decisions.
Big data analytics technologies and tools
Data warehouses may not be able to deal with the processing requests posed by sets of big data that have to be updated often or even more than once, as in the case of real-time data on the stock market, the online behaviours of website visitors, or the capability of mobile applications. Many organizations collect, process, and analyse big data using NoSQL databases. Hadoop is a supportive tool that includes YARN, MapReduce, Spark, HBase, Hive, Kafka, and pig.
YARN: A cluster management technology.
MapReduce: A software framework that allows developers to code
Spark: A unified analytics engine for large-scale data processing.
HBase: A column-oriented key/value data store built to run on top of the Hadoop
Distributed File System (HDFS).
Hive: Data warehouse system for querying and analysing large datasets stored in Hadoop files.
Kafka: A message broker.
Pig: A high-level mechanism for the parallel programming
In Hadoop, the data can be analysed directly in a cluster or run through Spark, which is a processing engine. Once the data is ready, it can be analysed with the commonly used software that can be used in advanced analytics processes. Data mining and statistical analysis software can also play a role in the big data analytics process. Programming languages used to
handle big data are R, Python, and Scala; and SQL, the standard language for relational databases that are supported via SQL-on-Hadoop technologies.
Properties of Big data analytics
Timely: Knowledge workers spend their time finding and managing data.
Accessible: Accessing the right data is difficult.
Holistic: Information is currently kept in silos within the organisation.
Relevant: The usage of efficient tools will filter out relevant data.
Actionable: Performing analysis of recent data will help to make decisions.
Secure: The secure infrastructures being built by big data hosting and technology partners can save annual revenues.
