Real-time data processing is a critical need for modern-day businesses. It involves processing data as soon as it is generated to derive insights and take immediate actions. Databricks Streaming and Apache Flink are two popular stream processing frameworks that enable developers to build real-time data pipelines, applications and services at scale. In this article, we […]
Posts Tagged ‘Spark Streaming’
Introduction to Kafka and MongoDB Integration
Introduction: Apache Kafka is a distributed streaming platform that enables businesses to build real-time streaming applications. First developed at LinkedIn in 2010, it has become one of the most widely used messaging systems for big data and real-time analytics. Kafka can process and transmit massive amounts of data in real-time, and its design ensures fault […]
Hadoop Ecosystem Components
The Hadoop Ecosystem Hadoop Ecosystem is a platform or a suite that provides various services to solve big data problems. It includes Apache projects and various commercial tools and solutions. 4 major elements of Hadoop are HDFS, MapReduce, YARN, and Hadoop Common. Hadoop is a framework that enables the processing of large data sets which […]
Real-time Machine Learning in Azure
In the war of cloud computing, AWS is currently the undisputed leader but Microsoft Azure seems to be chipping away at AWS’ lead with every passing quarter. While Google Cloud offers impressive tools as well, it is struggling to keep pace with Azure. This could be traced to Microsoft’s longer tenure in the enterprise market. […]
Hangzhou Spark and Flink Meetup 2017
Almost every year I will be attending the Hangzhou Spark meetup, where we can meet the Spark and Hadoop fans and expert from the local community. I love it because it is an open meetup and nothing to do with the commercial. In this year there are great knowledges and experiences sharing on Flink as […]