Comparison of Open Source Process Engines for Edge Integration

In February 2017, I attended the JFokus conference for the first time. This is the biggest developer event in Sweden. While the ‘J’ in the title shows that there is a lot of focus on the Java platform, the Internet of Things (IoT) was another hot topic. IoT, by nature, is not just about the Java platform, because you often need more lightweight technologies for small devices and edge computing. This article gives a summary of current IoT trends discussed at JFokus, and focuses on IoT edge integration, which becomes a key component in successful IoT projects.

The Unstoppable Growth of the Internet of Things (IoT)

Internet of Things refers to the ever-growing network of physical objects that feature an IP address for internet connectivity. It also refers to the communication that occurs between these objects, and other Internet-enabled devices and systems.

Today, many people think mostly about consumer devices like smart phones, smart watches or maybe even refrigerators. But most future business will be in the industrial IoT, beginning with more sensors everywhere in factories, assembly lines, cars, etc.

The numbers are growing significantly year by year. We should reach at least 20 Billion IoT devices globally in the next two years.

Challenges of IoT Application Development

The development of IoT applications is very different from classical application development with Java, .NET or Python. Kevin Hoyt from IBM highlighted this in his IoT keynote “Crossing the IoT Chasm”. Development with devices, and other small edge hardware like Beacons, completely changes development experiences. You have to think differently about things like debugging, testing, scaling or updating applications.

IoT raises several new challenges for development and deployment:

  • Devices are mostly resource-constrained
  • Some devices are not connected to the cloud
  • Devices have low bandwidth to connect
  • Latency of connectivity is significant
  • Connectivity is not reliable
  • Connectivity is not cost-effective

Kevin Hoyt showed live development for Arduino and other devices. Don’t expect the full power of your Eclipse or Visual Studio IDE for IoT application development. Everybody quickly recognized the limitations of small devices or sensors. You have much less functionality, and always need to think about how long the small battery can survive when executing some implemented logic.

Frameworks and standards

Great IoT standards and frameworks are available for IoT development, such as MQTT, CoaP, OPC US, Apache Kafka, and many more. In addition, proprietary solutions exist. For example, you might use TIBCO FTL for reliant, secure, and easy to operate high volume messaging. All these options use different abstraction levels, have different functionality, operation complexity, and other trade-offs. However, there is not the one single standard.

Often, it is to expensive or complex to make existing devices ready for Internet communication. Not every machine can use Wifi, Bluetooth, ZigBee, or another “IoT technology”. I have seen an interesting workaround two weeks ago at partner event to introduce IoT to old devices and machines, which are not internet-ready and where it is too expensive too add: A manufacturing company installed video cameras to record the monitors of the old machines. Then they do real time analysis with machine learning / deep learning techniques to recognize pictures and find issues or problems.

Security

Another key challenge with IoT is security. This is a huge challenge and “still mostly ignored today” in production deployments as Kevin Hoyt said. Shahid Raza from SICS discussed in his session “Is IoT security a nightmare?” how standards such as IPsec, DTLS or IKEv2 will (hopefully) solve security-related issues in the next years. Some people from the audience also confirmed that end-to-end integration is their key concern. Today, many IoT communications are simply unattended deployments.

Integration Matters – Key for Success in IoT Projects

300+ IoT platforms are already available to solve these challenges as discussed in “The IoT Landscape 2016 Edition” by Matt Turck and David Rogg, First Mark Capital. Interestingly, Gartner states in its Market Guide for IoT Integration 2016 that “through 2018, half the cost of implementing IoT solutions will be spent on integration.”

Therefore, Integration is Key for Success of IoT Projects. Without integration

there is no Internet of Things. However, there is no “one-size-fits-all” integration. IoT integration will be part of a Hybrid Integration Architecture; the new default in enterprises.

integration structure

Architecture

This architecture is built upon a Hybrid Integration Platform (HIP), which can include

  • Classical Enterprise Service Bus (ESB) deployments to run and integrate mission critical systems.
  • Cloud native middleware (based on orchestration platforms like Kubernetes / CloudFoundry, or hosted by the vendor as iPaaS offering) to develop and deploy agile microservices and build innovative new applications.
  • Data ingestion pipelines or Extract Transform Load (ETL) platforms to ingest and prepare massive datasets from IoT devices to data centres or cloud platforms.
  • Streaming Analytics to correlate events in real time.
  • IoT Integration at the edge, i.e. at a local IoT Gateway or directly on IoT devices.

Note the importance that these integration components work well together (technically, but also from perspective of vendor support). Take a look at “The Need for a Hybrid Integration Architecture” to learn more about Hybrid Integration Platforms.

One example where you leverage several components of this architecture is “Streaming Analytics to help Formula 1” where sensor integration and analytics at the edge in formula 1 cars and pits (“the things” in this case) are combined with real time messaging and streaming analytics to improve performance of the cars.

This article focuses on IoT edge computing and integration – and available open source frameworks. The goal is to keep data closer ‘to the edge’. Some vendors also call this “Fog Computing”. This allows more reliable IoT infrastructures, reduced latency, and less traffic / costs.

IoT Edge Integration with Open Source Process Engines

The edge integration and event processing is done with so-called process engines. They do not focus just on data ingestion or ETL, but implement logic at the edge. IoT process engines have the following characteristics and features in common:

  • Connectivity and orchestration of various IoT data sources and technologies such as MQTT, CoaP or REST
  • Wiring together hardware devices, APIs and online services
  • Data Cleaning: Transformation, filtering, routing, aggregation, enrichment of data
  • Data Processing: Error handling, Re-Try, Re-Routing, Waiting, Resuming
  • Synchronous and asynchronous communication
  • Typically, real time processing
  • Visual coding, testing and debugging with flow editor, but also option respectively need to write source code and extend components using the open SDKs and APIs.
  • Deployed at the edge, either at a local IoT Gateway or directly on devices and sensors

Quite a few open source process engines are available for IoT integration at the edge with the above characteristics. Of course, software vendors offer commercial services on top these open source IoT frameworks to deliver additional features and commercial support.

Let’s take a look at three examples in the following sections. We’ll start with Eclipse Kura in conjunction with Apache Camel, then Node-RED, and finally Flogo. All three frameworks focus on IoT edge integration, but have completely different reasons why they were built. Therefore, they are not really competitive, but complementary to each other.

Eclipse Kura + Apache Camel

Eclipse Kura is one piece of ~25 Eclipse IoT Projects with various contributors from different companies. They offer a best of breed approach with focus on coding instead of a visual designer. Some examples:

  • IoT Gateway: Eclipse Kura
  • Services like Eclipse SmartHome
  • Standard implementations like Mosquitto (MQTT Server)
  • Connectivity via Eclipse Paho (MQTT Client)

See “Eclipse IoT in 2016: A Year in Review” for more details about the evolution of these projects.

Eclipse Kura focuses on IoT Gateway duties:

  • Hardware and field abstraction (sensor connectivity, I/O access)
  • Management of network and connectivity (wireless devices, firewall, VPN; online / offline mode)
  • Management of applications (remote start / stop / configuration, installation / uninstallation of applications)
  • IoT device connectivity (MQTT / CoaP; data buffering and retries; provisioning, credentials and certificates)

Kura

Kura is a set of Java and OSGi services, including I/O services, Data Services, Cloud Services, Networking and a web UI for configuration of devices, network, protocols, etc. It is a mature framework (~3 years old, code based committed in December 2013) and available under Eclipse Public License 1.0.

Since a few months, Kura also offers an Apache Camel Connector. Apache Camel is an open source integration framework. With this combination, you can enhance the IoT Gateway features of Kura with process engine features of Apache Camel integration routes.

The IoT Gateway Dream Team: Eclipse Kura & Apache Camel” explains and demos this combination.

Integrating

Eclipse Kura and Camel are built for developers and integration specialists who are comfortable with writing source code. It is more complex to install and build a first IoT integration flow with this combination.

Eclipse Kura and Apache Camel can be run on a variety of platforms – on premise or in the cloud. Due to their basement on the Java platform, they are relative heavyweight (for IoT standards). Therefore, they are typically not used directly on small and lightweight edge devices, but on more powerful computers (even though you can deploy these “ somewhere at the edge”, of course).

Node-RED

Node-RED focuses mainly on IoT edge integration on an IoT Gateway, and needs to be deployed on a more powerful machine. It is built on Node.js and Javascript and therefore not possible to run on a small, lightweight device. Node-RED is very easy to install and building a first IoT integration flow takes just a few minutes. The node.js web application includes both the runtime and the web IDE.

License

Node-RED is available under Apache 2.0 License and built for developers, integration specialists and citizen integrators (i.e. technical people with less experience in integration projects). It offers intuitive visual coding via web designer and is already a mature framework (~3 years old) with good documentation and many examples. “Wiring the Internet of Things with Node-RED” shows a good introduction.

Integrations

As IBM is the main sponsor, Node-RED has native integration into IBM Bluemix cloud platform and can leverage other Bluemix services. You can also run Node-RED on a variety of other platforms, including on premise, cloud and edge devices. However, it is always needed to run the full node.js engine to execute a flow. There is no way to build a binary application which you could run on a small device. You just can export and share flows as JSON strings respectively files, but need another Node-RED engine to deploy them again.

Flogo

Flogo was built to realize very lightweight integration applications. These can be deployed many IoT devices and gateways. Flogo is still a very young project under developer preview (released in October 2016 by TIBCO under BSD-style License). It is very easy to install and build a first IoT integration flow using its web IDE. Flogo can be used by developers, integration specialists and citizen integrators. All very similar to Node-RED. “Open Source Project Flogo – Overview, Architecture and Live Demo” gives a detailed introduction.

Smaller footprint

The key difference to Node-RED is its ultra-lightweight footprint with its zero-dependency model – powered by Golang. Lightweight binaries with zero dependencies can be shared and deployed on small devices. In contrary to Node-RED, you do not need the whole engine running on that device, but need just a few megabytes for the binary each on disk and in memory. You can deploy it on any device that supports Go binaries.

The next step in the Flogo project will make it even more lightweight: Flogo nanoservices target microcontrollers like ARM M0 (where even Go is way too heavyweight). The goal is to offload I/O centric compute farther off to the edge.

These very lightweight binaries allow many IoT specific scenarios on very small devices. Note that a Rasperry Pi (a typical example often referenced in IoT examples) is not a small devices, but very powerful for IoT scenarios. Most IoT devices are much smaller and less powerful. Actually, many attendees agreed on this requirement after my talk. One of the JFokus organizers said to me, “only 1% of developers can write low level device code. We need higher level tools like Flogo to enable easier implementation of business logic and integration at the edge on very small devices”.

Fit with microservice

On the other side, it is also appropriate for other microservice scenarios where disk and memory consumption are very important. For example, you might deploy Flogo services to serverless infrastructures like IBM’s OpenWhisk or Fission (based on Kubernetes). Both allow running lightweight applications in “serverless containers” using technologies like Docker under the hood. With serverless architectures, you can utilize machines 100% and just pay exactly for what you use – in contrary to classical cloud infrastructures like AWS EC2, where you also pay for idle instances. Here, a very lightweight framework can make a key difference regarding cost, speed and agility compared to Java or node.js platforms.

Comparison of Open Source IoT Integration Frameworks

As you can see, all three frameworks focus on different IoT use cases. This gets even more apparent by the different infrastructure layers used:

Layers in IoT

Node-RED and Eclipse Kura use a complete platform (Javascript and Java, respectively) on top of the operating system. Therefore, you get different resource requirements, startup and build times:

kura vs node red vs flogo

IoT Edge Integration Matters – Choose the Right Tool for the Job!

All three discussed open source IoT frameworks have great features and their right to exist. A direct feature comparison is not appropriate. Choose the one which makes sense for your scenario. Do you need an IoT Gateway and powerful integration routes on Java platform? A nice tool for integrating many IoT interfaces with a mature visual IDE? Or do you need something really lightweight for very small IoT devices / edge integration, or maybe for serverless architectures? Choose Eclipse Kura in conjunction with Apache Camel, Node-RED or Flogo for your scenario. Or maybe choose something like Apache NiFi / MiNiFi or StreamSets if you “just” need to do some data ingestion from devices to your cloud platform and want to use another open source framework.

As you can see, plenty of options are available for developing IoT applications. Keep in mind that integration will be a key piece in your IoT puzzle. If you want to learn more about this topic, take a look at the slides from JFokus or a video recording walking through the slides and showing some live demos of open source IoT integration frameworks.