This post is an extract of the articles “Towards Better Scalability for IoT-Cloud Interactions via Combined Exploitation of MQTT and CoAP” presented at IEEE 2nd International Forum on Research and Technologies for Society and Industry, Technologies for smarter societies (RTSI), Bologna, 2016 and “Scalability of Kura-extended Gateways via MQTT-CoAP Integration and Hierarchical Optimizations” presented at ACM 7th International Conference on MOBILe Wireless MiddleWARE, Operating Systems, and Applications (MobilWARE), Turin, 2016.

In the latest years, the IoT perspective has become more and more relevant, as well as the number of sensors/actuators involved in IoT scenarios of practical industrial interest has rapidly increased, see for instance the smart city application domain. In addition, as many related research activities are showing, an exponential growth of the number of IoT devices considered in realistic applications will continue in the near future. Therefore, scalability is emerging as one of the key factors for IoT development and exploitation, also considering the technical challenges connected with possible geographical distribution over wide areas, connectivity through federated networks and resources in general, as well as the need to enhance the applicability of traditional security, trust, and privacy management infrastructures in an IoT-specific and efficient way.

On the one hand, there is the adoption of a hierarchical tree solution to benefit from them topic-based publish/subscribe structure of MQTT (Message Queue Telemetry Transport) that exploits hierarchical namespaces: using MQTT to subscribe our IoT gateways and entities to multiple topics, to facilitate message management towards all the nodes of the hierarchy, and to add/remove IoT entities without changes in the whole topic hierarchy. On the other hand, MQTT has demonstrated some non-negligible limitations in terms of scalability, e.g., inefficient usage of TCP connections towards the broker when growing the number of IoT devices in a gateway locality. For this reason, it is important to efficiently and scalably integrate MQTT in IoT Kura gateways with some purpose-specific CoAP ( Constrained Application Protocol) usage (connection-less datagrams on top of UDP), especially on behalf of constrained devices, as detailed below. The idea is to extend the MQTT-oriented Kura gateway with more lightweight CoAP-based coordination functionality, thus achieving scalable interactions, especially in the challenging case of node/resource discovery.

The solution extends (and has been designed and implemented on top of) the Kura framework, a platform for IoT-cloud integration of strong industrial relevance, based on Java OSGi and on the role of infrastructure gateways for efficient message brokering. In particular, the Kura framework has been extended, with CoAP protocol support, to offer improved scalability and reduced latency for communication/coordination among wide-scale sets of geographically distributed IoT devices interworking via gateways. Based on a dynamic tree structure organization, gateways and IoT devices exploit MQTT and CoAP in a combined way to improve the efficiency of:

  1. node communications for efficient resource look-up
  2. IoT device discoverability
  3. resilience to device disappearance or unexpected disconnection
  4. dynamic and lightweight hierarchy reconfiguration, triggered by 2 and 3.

The node managements optimization has been performed via hierarchical tree organization:

architecture-levels

The internal node structure, replicated into all nodes, consists of multiple bundles, mainly for the sake of modularity and loose coupling:

  • CoAP Server bundle. It manages of all the properties of registered resources and can be coordinated with other servers to perform callback methods in relation to the received messages. It works locally on a node to create a high-performance CoAP support inside Kura; Resource Discovery (RD) bundle. It is a data structure used to store endpoints and resources belonging to different domains, groups, or subgroups. It supports automated support to node disconnection management and usual discovery operations, e.g., to register, maintain, lookup, remove endpoints and resource descriptions;
  • Remote Query resource (RQ) bundle. It supports resource lookup by CoAP servers in our hierarchy, thus allowing nodes to retrieve information from CoAP servers by interfacing CoAP requests with MQTT brokers. CoAP Resource bundle. It is a service dedicated to the external exposition of discoverable resources. It can add, de lete, or modify the exposed attributes.

The management of our hierarchical tree is performed by:

  • MQTT Broker bundle. It is a simple extension of a regular MQTT broker, based on Mosquitto [2], to exchange messages between nodes within our hierarchy;
  • CoAPTreeHandler bundle. It receives requests from Kura nodes and returns references to create the tree structure. It can perform different management operations, e.g., handling node replicas, providing hierarchy metadata, handling children nodes in case of parent disconnections. It is implement in an event-oriented way, where event sources are queue listeners to get updates from MQTT brokers.

More specifically, this solution is relevant because proposes an original IoT gateway extension, based on a dynamic and efficient tree structure to allow IoT resources on different devices to scalably interact even coap if deployed over a large-scale distributed area, on top of very promising and powerful open-source initiative, i.e.:

  • the Kura IoT Platform, with mature ancillary frameworks
  • Guava
  • Kryo
  • emerging open-standard protocols, such as MQTT and CoAP protocols.
  • SSL and DTLS integration for securing machine-to-machine communications.

The CoAP support is provided with Californium framework, an opensource framework that implements CoAP functionalities and provides web service RESTful api. Although Kura and Californium are widespread and industry-mature frameworks, their default configurations have demonstrated to be not specifically optimized for hierarchical IoT gateway organization.

Thus, some non-negligible implementation/configuration improvements are needed, focusing on the following primary critical scalability aspects:

On Kura side, the main optimizations and improvements are related to reducing the number of MQTT messages exchanged among levels and nodes, through:

  • Object Serialization. MQTT is content-less and only support byte array as payload content. Thus, serialization is performed very often during regular operation: each CoAP message is serialized, then used into the Kura DataService class for MQTT communication, and de-serialized during responses. We have improved performance by integrating serialization based on the efficient Kryo framework. Kryo is an open-source object graph serialization framework for Java.
  • Data Service and DataTransportService. The DataService class is the Kura component used to manage MQTT communication and offers several configuration options. When DataService receives a publish request, it stores the message into a DataStore and submits the message on the internal executor of DataTransportService. DataStore is a heavy storage structure that, in case of many messages published concurrently, may even cause unavailability of the device because of its high CPU consumption. Therefore, to reduce associated overhead, the support to the features irrelevant for our solution have been removed.

On Californium side, the optimizations and improvements are realetd to reducing the number of POST requests:

  • RD Parsing and String Refactoring. To optimize regular expression management and to speed up parsing, we have decided to represent any CoRE link with resource path in the first position of the RD attribute list (fixed static position). In addition, we adopt Guava libraries for efficient string management; for instance, we have replaced the Java Scanner class with the specialized Guava Splitter and used the optimized StringBuilder for string declaration.
  • Request Aggregation (RA). Since every node can send POST messages, it is likely that some nodes may send information about multiple resources. Thus, we optimize message management whenever possible (in absence of strict latency requirements) by aggregating multiple resources together into a single CoRE link format composition.

The implementation optimizations significantly improve the performance of both the standard Kura gateway and the exploited Californium framework for CoAP. Experimental results show that the solution, comphrehensive of optimizations, can preserve scalability also in high-load scenarios, with overall latency that is proportional to the number of incoming requests and limited overhead. It is possible to achieve good overall performance also with IoT devices with very strict resource constraints, like RaspberryPis.

test_environment

For further details, please also refer to this page.