- IOTTLY overall specifications Implementation details.
- IOTTLY Brokers are server-side physical implementations of communication protocols.
- IOTTLY DATA MODEL Model describes the fundamental entities involved in the design and management of IoT projects, and the relationships among such entities.
- IOTTLY Reliable Message Queue All the building blocks of the IOTTLY Distribution will communicate using IOTTLY’s Reliable Message Queue.
- IOTTLY Polyglot implements the interaction between IOTTLY Core and each IOTTLY broker.
- IOTTLY History storage is the physical database where incoming messages from the devices are stored.
- IOTTLY Trust is the integrated security layer responsible for authenticating and authorizing every single actor communicating within IOTTLY.
- IOTTLY RESTful APIs are the way IOTTLY exposes its core functionalities to third-party applications and IOTTLY tools.
- IOTTLY Message designer is an editor which helps project creators draft and annotate the messages which will be exchanged with the devices.
- Device admin console provides real-time administrative control of the devices.
- Config tools are the pages used to configure specific data for Organizations and Projects.
- Ready-to-use control app IOTTLY provides a ready-to-use web Control App, available on both desktop and mobile devices.
- Custom logic flows provide server side programming and execution of event driven logics
Iottly overall specifications
The first release of IOTTLY will run on top of Linux
- other operating systems will be supported in future releases
IOTTLY source code will be hosted on Github
IOTTLY overall architecture will follow the pattern of micro-services
- each component will run as an independent service
- services will communicate through IP protocols depending on the specific needs
- docker images for each service and a docker-composer script for the overall distro will be managed
IOTTLY messages format will be JSON, including:
- messages exchanged with devices (the JSON structure will be free and customizable)
- internal messages
- messages exchanged with third parties applications
Further implementation details will be specified in the following for each architectural component.
IOTTLY Brokers are server side physical implementations of communication protocols.
IOTTLY will run multiple broker instances at the same time.
- This will allow to support different protocols through the implementation of different brokers
- It will also allow the community to contribute different implementations of the same protocol or new and different protocols
IOTTLY will use docker to run each broker instance in a sandboxed environment
- different organizations or projects will have separated broker instances for scalability and security purposes
- IOTTLY will take care of automating all the required docker management tasks
The first release of IOTTLY will ship MQTT and HTTP support
- MQTT is a standard, ligthweight, bi-directional protocol which implements a powerful publish-subscribe paradigm
- HTTP has to be regarded as a “fallback” protocol to support legacy devices; in this case commands will be implemented by means of polling
Iottly Data model
IOTTLY Brokers are server side physical implementations of communication protocols.
The IOTTLY Data Model describes the fundamental entities involved in the design and management of IoT projects, and the relationships among such entities.
Project creators will work in IOTTLY either privately or within the scope of an organization
The Project entity will support creators in taking care of the overall technical details:
- the communication protocol with devices
- the kind and properties of devices to be used
- the kind of messages to be exchanged with devices and third parties applications
Hence devices belong to a project which in turn belongs to an organization (or to a private project creator)
- devices can be declared as private or public with respect to the scope of the organization
- public devices are accessible from any organization with different degree of controls (read and/or write)
- furthermore, private devices can be shared among different organizations
The Application entity takes care of the interaction relationships of devices with third parties applications.
IOTTLY Reliable Message Queue
All the building blocks of the IOTTLY Distribution will communicate using IOTTLY’s Reliable Message Queue.
This leads to asynchronous and scalable interaction among each IOTTLY component.
Data model relationships will be enforced on top of the IOTTLY Reliable Message Queue in order to implement secure communication policies.
Queue model design will target:
- data streams to enable event-driven manipulation of devices data
- job orders to enable execution of on-demand tasks on data
- Reliability of the queue resides on the consumer ability to acknowledge the producer about the job being correctly processed
- From this point of view, for example, Polyglot behaves like a consumer when it has to send a command to a device, and it behaves like a producer when it has to dispatch an event received from a device, for further processing
- the History Storage behaves like a consumer when it has to store a value received from a device, while being a producer when it is required to retrieve some data by the RESTful APIs
The choice of which Message Queue to adopt is still under investigation.
Disque is among the main options.
The queue model will be designed and constrained by means of an API each producer / consumer will have to derive from.
IOTTLY Polyglot implements the interaction between IOTTLY Core and each IOTTLY broker.
Each broker instance will have a Polyglot client counterpart running in IOTTLY Polyglot.
IOTTLY Polyglot will define the interface and the API which will be implemented by each Polyglot client.
Each Polyglot client will have to properly register on the Message Queue to exchange messages from/to the message queue to/from the broker.
Polyglot Clients will be responsible of mapping the producer-consumer internal paradigm of IOTTLY onto each of the implemented brokers
- for brokers natively supporting publish-subscribe (like MQTT) this will be a pass-through mapping
- for other brokers the relative client will map internal IOTTLY communication paradigm to the specific protocol
Iottly History storage
History storage is the physical database where the incoming messages from devices are stored.
A partitioned history space will be reserved to each project in Iottly
- projects space will be limited by quotas defined at the Organization / Project levels
Device messages will be stored in raw JSON format
- Raw data will follow retention policies based on Organization / Project History quotas
Iottly History Storage will automatically extract raw time-series from the more complex incoming JSON structures
- Raw time-series will be stored in flat, per project, per dimension tables
- Raw time-series exposure to RESTful APIs will be bounded in time
- future releases of Iottly will support time-series replication into off-line big-data databases, such as HBase
A set of standard time aggregations (mean, max/min, median) will be performed on raw time-series and stored on History storage for long-time reporting of data.
- Native support for JSON
JSON flattening to time-series will be performed in real-time within the History storage job responsible for persisting raw JSON messages.
Workers will be implemented to perform time aggregations asynchronously
IOTTLY Trust is the integrated security layer responsible for authenticating and authorizing every single actor communicating within IOTTLY.
- Authorization logics and communication policies are derived from the data model
It acts on devices
- enforcing devices authentication (specific of the particular protocol)
- managing the communication encryption
- preventing fake devices to impersonate real ones
- preventing devices data leakage
It acts on RESTful APIs
- enforcing APIs calls to be authenticated and verifying in real-time the caller effective rights
- enforcing the relationships between applications, users and devices, thus preventing the possibility of a user to take control over a device he is not authorized to
Authentication tokens are managed at the Organization / Project level, thus preventing by design data leakages between organizations
Reliable device provisioning
- When provisioned a device is required to know (fabric settings):
- the organization IOTTLY Broker end-point
- the IOTTLY project global device token
- The first time it connects the device will start the handshaking to receive its unique auth. token
- This procedure makes it easy to preserve device security while not requiring to hard code the specific auth token for each device
MQTT authentication is SSL based (both client and server auth)
APIs authentication is implemented by means of http headers tokens
IOTTLY RESTful APIs
RESTful APIs are the way IOTTLY exposes its core functionalities to third-party applications and IOTTLY tools.
- they provide access to IOTTLY History storage (raw JSON messages, time-series raw and aggregated data)
- they enable the automation of IoT management tasks, providing read-write access to IOTTLY data model
- they provide full control of devices, allowing applications and users to send remote commands to them
Real-time Webhooks APIs
- applications may register specific callback to receive real-time events from devices
RESTful HTTPS APIs
Payload is JSON
Iottly Message designer
The Message designer is an editor which helps project creators draft and annotate the messages which will be exchanged with the devices.
Coherently with its open nature, IOTTLY will be payload agnostic with respect to devices messages.
- defining a wide message semantic would result in a naive attempt, always leaving out some cases
The Message designer easily allows to annotate JSON tags, binding them with specific semantic.
Thus messages will be interpreted in order to
- properly being routed through the APIs
- properly being represented on Device admin console
- properly build the GUI command controls both in the Device admin console and in the ready to use sample app.
A dynamic JSON parser will be developed in python. Each project will have its own parser.
- Parser binding with message semantic will be persisted on database at project level
Semantically meaningful functions will be dynamically mapped to JSON tags.
Message designer will generate such functions in python snippet which will then be compiled into the specific project parser.
Iottly Device admin console
The Device admin console provides real-time administrative control of the devices.
- customized on payload semantic
near-real time charts of time-series device data
Over the air firmware upgrades
Geo-location of devices
Iottly Config tools
Config tools are the pages used to configure specific data for Organizations and Projects.
- list of users (project creators, technicians, end users)
- list or projects
- list of end user third parties applications
- Project protocol
list of available firmwares
- global project device token
- list of devices
- deviceid and token
- RESTful APIs tokens
- Applications webhooks
Iottly Ready-to-use control app
IOTTLY provides a ready-to-use web Control App, available on both desktop and mobile devices:
- to support rapid prototyping
- to make IoT instant projects
The Control App is customizable:
- Just drag a device message onto the App designer
- then select a control (switch, value selector, bar)
- choose an icon to complete your IoT App
Iottly Custom logic flows
Custom logic flows provide server side programming and execution of event driven logics:
- When cross device logic is needed, server side is the right place to put it
- For example if you want to send an e-mail based on some device message content.
IOTTLY will integrate a customized fork of Node-RED:
- since IOTTLY lets you to customize device messages at the project level, a node-red instance will be run for each project
- your custom device messages will appear as input nodes in the inputs pane
- simply dragging a specific message node will make the flow trigger when the specific message is received without the hassle of writing js code to parse the message
- flows will be persisted on IOTTLY database on a per project basis
Single Node-red instances (client and server) will be run separately for each IOTTLY project, on top of
Node-red instances automation tasks will be developed on top of docker-py