Easier DynamoDB Unit Testing with Python
Your Python app uses DynamoDB and you need some unit tests to verify the validity of your code, but you aren’t sure how to go about doing this. We can help!
When building an IoT platform, AWS IoT Core offers many out of the box services to help get you up and running faster. One of those services is the IoT Device Shadow.
Amazon’s description of this service is:
The AWS IoT Device Shadow service adds shadows to AWS IoT thing objects. Shadows can make a device’s state available to apps and other services whether the device is connected to AWS IoT or not. AWS IoT thing objects can have multiple named shadows so that your IoT solution has more options for connection your devices to other apps and services
There is more than one solution to any problem, however, and just because something is easier doesn’t necessarily mean it is the right solution for your particular application. In this post, we explore the pros and cons of utilizing the IoT Device Shadow and compare it to an alternative structure using DynamoDB to store the state of a device.
Use Of IoT Device Shadow
A typical use case for the IoT Core Device Shadow involves using it to start a firmware update. When a device comes online, the physical state of the device is pushed to the reported state of the IoT Device Shadow:
A client (such as a mobile app or web dashboard) can see that the device is online because the IoT Device Shadow exists and the reported state is ‘online’. The IoT Device Shadow data can be accessed through a custom API or through the AWS SDK. When the client wants to upgrade the firmware, it sets the desired state of the shadow to the firmware version it would like to upgrade to. The difference between the reported ‘firmwareVersion’ and the desired ‘firmwareVersion’ causes a delta to generate, and a message to be sent on the $aws/things/<thingName>/shadow/update/delta managed MQTT topic1:
This delta message is consumed by the device, and it takes action to have its reported state match the desired state (in this case, update the firmware and report the new version).
DynamoDB as a Store of Device State
The IoT Core Device Shadow is a great service and a way to get a lot of functionality with minimal development effort. However, using the device shadow frequently or having large fleets that use it several times per day can cause costs to rise significantly over time.2 We can design an architecture using DynamoDB, API Gateway, and Lambda that gives us similar functionality to that of the IoT Device Shadow but is more cost effective.
The mechanics of a firmware update are slightly different in this alternative approach. When the device comes online, it publishes a JSON representation of its state onto a non-shadow IoT Core MQTT topic. The payload will need to contain an attribute that indicates the source of the message (“source”: “device”). An IoT Rule on this topic does an update operation which places the payload into the DynamoDB table. The partition key of this table is the device’s unique serial number. When the update operation occurs, this generates a DynamoDB Stream record with the eventName equal to INSERT or MODIFY (depending on if the device settings already exist in the table). The stream processing lambda does not forward this message to the device on an MQTT topic because the source of the change was the device. To initiate the firmware update, an external client sends a post request through API Gateway, which updates the firmware version attribute in Dynamo for the targeted device. This generates a stream record of type UPDATE with source attribute = ‘api’. The stream processing lambda forwards this message to the device on an MQTT topic because the source of the change is not the device.
Pros and cons of each approach
So which should you choose? Below are a few things you might want to consider when weighing these options:
Pros of IoT Core
Cons of IoT Core
This is surprising but true! Rarely does an AWS managed service have minimal managed failover like this, but the AWS IoT documentation is very clear on this point
Pros of DynamoDB approach
Cons of DynamoDB Approach
Overall, using the IoT Core IoT Device Shadow is extremely simple and brings a lot of utility to the table. With careful design, the costs associated with using the IoT Device Shadow can be managed. But if IoT Device Shadow cost calculations or the lack of multi-availability zone redundancy are a cause for concern for you, it is good to know that there are other solutions available that can meet most of your needs, keep costs under control, and offer some built-in benefits. For more AWS architecture inspiration, check out the Cloud Prose Blog by Trek10 here.