Lambda Architecture demo using WSO2 CEP

Introduction to Lambda Architecture

With the rapid growth of communication, the handling of data has become a problem. The batch processing was introduced as a solution where a batch of data is collected and processed at once. The problem with this approach was the outdated result. To overcome this issue the Lambda Architecture was introduced which is a combination of batch processing and stream processing. In stream processing the real time results.

Lets see what is the purpose of each layer in Lambda architecture.

Batch layer

This layer is responsible for storing all the incoming events and create batch views. The generation of the results in this layer takes a considerable time as it processes a large set of historical data.

Speed layer

This layer will catch up the data which are missed from the batch layer, at the time the user queries for a output. This will process a small set of data and gives a real time output.

Service layer 

This layer will aggregate the both results from batch layer and speed layer to give a final result.

Using WSO2 CEP and BAM for Lambda Architecture

Lets discuss how to use the WSO2 CEP and BAM to produce a real time result using Lambda Architecture.