AWS SQS — Case Study and Use Cases
How BMW uses AWS SQS?
What is SQS?
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware, and empowers developers to focus on differentiating work. Using SQS, you can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.
SQS is a great way to decouple services, especially when there is a lot of heavy-duty, batch-oriented processing required. For example, let’s say you have a service where people uploads photos from their mobile devices. Once the photos are uploaded, your service needs to do a bunch of things. This includes the processing of images eg: scaling them to different sizes, applying different filters, extracting metadata etc. So one way to accomplish this would be to post a message to SQS queue. The message describes the work that needs to be performed on the newly uploaded image file. Once the message has been written to SQS, your application can return a success to the user because you know now that user has uploaded image and has stored in a temporary database i.e the SQS queue. So we have scheduled the processing of image. Meanwhile, the user also has not to wait for the entire workflow completion and thus provide better user experience. In SQS, the server which sends the message to SQS queue is known as Producer.
In the background, we can have several servers who keep on polling for the messages from SQS and performing the work specified in the data/message. If one of the backend server crashes, another will pick up the message and perform the work. These backend servers here are called as consumers as they consume the message from SQS. But in this scenario, we are missing the most important thing i.e to delete the message from queue after server polls it. If we don’t delete other server can polls the SQS and reads the message and perform the same task again. So this creates a sort of loop. So, we can ask the consumer server to delete the message from SQS as soon as they retrieve the data/message from SQS. So, SQS guarantees that a message will be delivered eventually so you can be confident that the work/processing will also be done.
Typical architecture and message flow is described below.
- A producer (component 1) sends message A to a queue, and the message is distributed across the Amazon SQS servers redundantly.
- When a consumer (component 2) is ready to process messages, it consumes messages from the queue, and message A is returned. While message A is being processed, it remains in the queue and isn’t returned to subsequent receive requests for the duration of the visibility timeout.
- The consumer (component 2) deletes message A from the queue to prevent the message from being received and processed again when the visibility timeout expires.
BMW Group using AWS Services to leverage better services to user.
BMW group believes disruption leads to new opportunities. The factors which leads are
- More software and services
- Cloud Solutions augment the power of IT infrastructure.
- Agile and real-time innovation
- Deep personalization and engagement
- New competitors
The cloud based solutions allows directly engage with the consumers. BMW group has invested in AWS to leverage better performance and services.
Intelligent Connectivity and Data Processing — Learning Maps
All the BMW connected cars collects the data from the inbuilt sensors and it dumps in the AWS cloud. Now, this data is further processed and provides a dynamically updated Digital Map which helps to the drivers or other connected systems for forecasting and can also help in Autonomous Driving.
The major benefits are:
- Accurate speed limit
- Improved Road Geometries
- Fewer map errors
- Optimized routes
The BMW Group calls this integration of cloud based solution like AWS as Car as a Service. They gave name to this model as CARASSO.
The BMW Group is using AWS for its new connected-car application that collects sensor data from BMW 7 Series cars to give drivers dynamically updated map information. BMW Group is one of the leading manufacturers of premium cars and mobility services in the world, with brands such as Rolls Royce, BMW, and Mini. BMW built its new car-as-a-sensor (CARASSO) service in only six months leveraging Amazon S3, Amazon SQS, Amazon DynamoDB, Amazon RDS, and AWS Elastic BeanStalk. By running on AWS, CARASSO can adapt to rapidly changing load requirements that can scale up and down by two orders of magnitude within 24 hours.
Here, the main role of AWS SQS comes in play. Let’s understand the architecture first.
The architecture has a service deployed through route 53. All the cellular data is being fetched by this service. And there are Load Balancer at first which takes the data and pass it to its backend servers. This server is deployed by Elastic BeanStalk and is known as Receiver Servlet. At this point the data is being stored in a SQS Queue. As the data is coming from real time, if the worker servlet goes down then the entire workflow fails. So when the data is received at Receiver Servlet, it is put into separate queues according to the processing done on the data. Here, there are two queues, one the Worker Queue SQS and the other is Chunk Buffer S3. The worker servlet keeps on polling for the messages to the SQS queue and receives the message and perform the operation. After it fetches message from SQS, it deletes the message. Now, the processed data is stored in RDS Database and further it is managed into the DynamoDB. The processed data from the Worker Servlet helps to create the Map Data. The Map data is updated and provided to the user via cellular. Thus, in this CARASSO model, SQS play very important role as it helps to manage the data/messages. So, it provides Agility, Flexibility and Scalability.
