Skip to main content


Efficiently Collect, Transform and Transit Your Data With eKuiper 1.9 Release

By Blog, eKuiper, Project Release

eKuiper—a lightweight IoT data analytics and streaming software—is now available in its 1.9.0 release. eKuiper, an LF Edge project, migrates real-time cloud streaming analytics frameworks such as Apache Spark, Apache Storm and Apache Flink to the edge. eKuiper references these cloud streaming frameworks, incorporates any special requirements of edge analytics and introduces rule engine, which is based on Source, SQL (business logic) and Sink; rule engine is used for developing streaming applications at the edge.

eKuiper 1.9 release continues to enhance the source/sink connectors to make it easier to connect and transmit data with lower bandwidth. The community has also enhanced the data transformation ability to flexibly encode and compress any part of your data. The 1.9 release adds a number of significant new features, among them are

  • Multiple neuron connection to analyze collected data from multiple IOT gateways together
  • MQTT sink/source compression/decompression support, save bandwidth for edge cloud communication
  • HTTPPull source & REST sink support dynamic token based authentication, connect to more services out of box
  • More transformation and compression functions added, handle your data flexibly
  • Partial data export/import, migrate only interested rules and the dependencies
  • Run python plugin in conda virtual environment, separate the python runtime env

Learn more about these and other features of eKuiper’s 1.9 release in the release notes.

What’s next

In the next release, the community will adapt to the EdgeX Foundry‘s Minnesota version, while exploring the use of external states such as Redis states to achieve persistent states.

Baetyl New Release Integrates With eKuiper and Delivers Edge to More Devices

By Baetyl, Blog, eKuiper, Project Release

Baetyl—an LF Edge project that extends cloud computing, data and service seamlessly to edge devices— has released v2.4.3 release. With the efforts of many active contributors, new functions have been added, and some existing functions have been continuously optimized since the previous v2.3.0 release. These new features continue to follow the cloud-native philosophy and build an open, secure, scalable, and controllable intelligent edge computing platform.

Compared to the previous Baetyl v2.3.0 releases, the new features and optimizations in v2.4.3 include:

  • Device management functionality has been refactored with the addition of a device template interface, support for calculating OT data collection values, support for IEC-104 protocol, updated OPC-UA and Modbus drivers, and updated driver-node binding logic;
  • Support for Windows platform has been added with the ability to generate Windows – platform images for the Baetyl edge main module;
  • Remote invocation has been implemented, allowing for remote access to specified edge services with results returned from the cloud;
  • New container mode with eKuiper as an optional system application;
  • New baetyl-rule module supports HTTP Target;
  • Adaptation to the highest K3s 1.24.4 version;
  • Fix workload type creation failure bug;

These new features are available and now you can view the release note here. Other features can be further explored by the developers, and Baetyl will continue to improve and optimize its functionality.

Integration with eKuiper

Baetyl uses eKuiper as a system optional application on the edge-side for stream processing and data analytics. The collaboration and integration with eKuiper increase the linkage between LF Edge projects and promote innovation. Platform and version adaptation enables Baetyl to run on more devices. The optimization of device management and new driver support prepare for the access of more devices with different protocols. From the point of eKuiper, this means you can deploy eKuiper more quickly and conveniently.

In the new version, the integration between Baetyl and eKuiper makes the following changes, including:

  • set mqtt client as Baetyl broker; 
  • mount eKuiper’s data file to the host to ensure no configuration loss; 
  • add k8s service to the eKuiper application to enable calling eKuiper’s open API from the host as well as from within the cluster to enable edge configuration changes; 
  • built-in eKuiper as an optional application in the Baetyl framework, so that eKuiper can be installed directly through Baetyl, eliminating the need for separate installation; 

After the integration, the Baetyl framework enhances the ability of edge message processing, while users can use Baetyl’s ability to use eKuiper more easily and quickly.

What’s next?

For future releases, the project is working on strengthening the management of non-intelligent devices at the edge, including providing

  • more comprehensive management functions for device models, devices, and drivers on the cloud management platform, 
  • a software gateway management module on the edge to support the ability of devices to connect through various industrial drivers,
  • a unified northbound connection protocol blink for access implementation.

The Baetyl project is also further expanding and optimizing the implementation of cloud storage at the underlying level, providing support for database storage outside of k8s crds. It’ll also enhance the integration with K8s cloud-native, providing more edge cloud-native capabilities for access, such as providing the ability to view edge description information, and so on.

eKuiper Issues 1.7.0 Release, Adding Lookup Table, Analytic Functions and More

By Blog, eKuiper, Project Release

eKuiper—a lightweight IoT data analytics and streaming software—is now available in its 1.7.0 release. eKuiper, an LF Edge project, migrates real-time cloud streaming analytics frameworks such as Apache Spark, Apache Storm and Apache Flink to the edge. eKuiper references these cloud streaming frameworks, incorporates any special requirements of edge analytics and introduces rule engine, which is based on Source, SQL (business logic) and Sink; rule engine is used for developing streaming applications at the edge.

eKuiper 1.7.0 continues to improve the expressiveness of the rules by providing the lookup table, updatable sink, and more SQL syntax and functions. The 1.7.0 release adds a number of significant new features, among them are

  • Stream batch combined computation
    • Not all data changes frequently, even in real-time computation. In some cases, you may need to supplement stream data with externally stored static data. For example, user metadata may be stored in a relational database, and the stream data only has data that changes in real time, requiring a connection between the stream data and the batch data in the database to enrich the data. In the new version, eKuiper adds the new Lookup Table concept for binding external static data, which can be joined with stream data in the rules to realize the operation of stream-batch combination. Additionally, with the support of updatable sink, eKuiper can process CDC data and update the batch storage.
  • Enhanced analysis capabilities
    • Analytic functions refer to functions that perform state-related analysis by saving state, allowing users to complete some of the common stateful calculations without complex time windows or custom functions. In the new version, we have added the Partition By syntax to the analytic functions, allowing users to perform partitioned stateful calculations based on the dimensions defined by Partition By clause. We also added a new analysis function latest to get the latest value of data. It is used to collect unstructured data whose columns are not fixed and can be automatically stitched to calculate the complete data.
  • Expanding the connectivity ecosystem 
    • On one hand, we continue to add built-in and extended source/sink by adding Httppush source, which allows users to push data to eKuiper via HTTP protocol; adding built-in Redis lookup source, which supports using Redis as external lookup table; adding Influx V2 sink, which supports writing to InfluxDB 2.x version. At the same time, Memory, SQL, and Redis source are adapted to support lookup tables; Memory, SQL, and Redis sink are adapted to be updatable.
  • Enhance the Ops experience
    • The new release refactors the external connection configuration API to enable easier connection resource management. Another major update comes from the bulk import/export and initialization of streams and rules to facilitate the migration of eKuiper instances.

Learn more about these and other features of eKuiper’s 1.7.0 release in the release notes.

What’s next

In the next release, the community will work on enhancing graph API to support more nodes, such as switch node and script node, to provide a more powerful and easy-to-use rule pipeline composing tool. A new function to run tensor flow lite model will be provided to easily integrate with AI models. Check out the evolving Github 1.8.0 milestones and we welcome you to add your wisdom.

LF Edge Releases Industry-Defining Edge Computing White Paper to Accelerate Edge/ IoT Deployments

By Akraino, Announcement, Baetyl, EdgeX Foundry, eKuiper, Fledge, Home Edge, LF Edge, Open Horizon, Project EVE, Secure Device Onboard, State of the Edge

Collaborative community white paper refines the definitions and nuances of open source edge computing across telecom, industrial, cloud, enterprise and consumer markets

 SAN FRANCISCO – June 24, 2022 –  LF Edge, an umbrella organization under the Linux Foundation that aims to establish an open, interoperable framework for edge computing independent of hardware, silicon, cloud, or operating system, today announced continued ecosystem collaboration via a new collaborative white paper, “Sharpening the Edge II: Diving Deeper into the LF Edge Taxonomy & Projects.” 

A follow-up to the LF Edge community’s original, collaborative 2020 paper which provides an overview of the organization and details the LF Edge taxonomy, high level considerations for developing edge solutions and key use cases,the new publication dives deeper into key areas of edge manageability, security, connectivity and analytics, and highlights how each project addresses these areas. The paper demonstrates maturation of the edge ecosystem and how the rapidly growing LF Edge community has made great progress over the past two years towards building an open, modular framework for edge computing. As with the first publication, the paper addresses  a balance of interests spanning the cloud, telco, IT, OT, IoT, mobile, and consumer markets.  

“With the growing edge computing infrastructure market set to be worth up to $800B by 2028, our LF Edge project communities are evolving,” said Jason Shepherd, VP Ecosystem, ZEDEDA  and former LF Edge Governing Board Chair. “This paper outlines industry direction through an LF Edge community lens. With such a diverse set of knowledgeable stakeholders, the report is an accurate reflection of a unified approach to defining open edge computing.” 

“I’m eager to continue to champion and spearhead the great work of the LF Edge community as the new board chair,” said Tina Tsou, new Governing Board chair, LF Edge.  “The Taxonomy white paper that demonstrates the accelerated community momentum seen by open source edge communities is really exciting and speaks to the power of open source.” 

The white paper, which is now available for download,  was put together as the result of broad community collaboration, spanning insights and expertise from subject matter experts across LF Edge project communities: Akraino, EdgeX Foundry, EVE, Fledge, Open Horizon, State of the Edge, Alvarium, Baetyl, eKuiper, and FIDO Device Onboard. 

ONE Summit North America 2022

Join the broader open source ecosystem spanning Networking, Edge, Access, Cloud and Core at ONE Summit North America, November 15-16 in Seattle, Wash. ONE Summit is the one industry event focused on best practices, technical challenges, and business opportunities facing decision makers across integrated verticals such as 5G, Cloud, Telco, and Enterprise Networking, as well as Edge, Access, IoT, and Core. The Call for Proposals is now open through July 8, 2022. Sponsorship opportunities are also available. 


About The Linux Foundation 

Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more.  The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: Linux is a registered trademark of Linus Torvalds.



eKuiper Issues 1.4 Release & Discusses 2022 Roadmap

By Blog, eKuiper

Written by Jiyong Huang, Chair of the eKuiper Technical Steering Committee and Senior Software Engineer of EMQ

eKuiper, a Stage 1 (at-large) project under the LF Edge umbrella, is a Go language implementation of the lightweight stream processing engine which can run in various IoT edge usage scenarios for real-time data analysis. Being able to run near the data source at the edge, eKuiper can improve system response speed and security, saving network bandwidth and storage costs.

The eKuiper project usually has one release per quarter with new features and several patch versions afterwards. The current active release is v1.4 which is the biggest update after joining LF Edge! We have solid community contributions for thee release, including requirement collections, code contributions, testing and trials. So we are grateful to share the updates and hope they can benefit our users.

Rule Pipeline: Building Complex Businesses Logic with Flexibility

eKuiper uses SQL to define business logic, which lowers the threshold of development. Simple SQL statements can efficiently define business requirements such as filtering, aggregation, and data conversion in practical usage scenarios. However, for some complex scenarios, it is difficult to address by defining a single SQL statement; even if you can, the SQL statement itself is too complex and difficult to maintain.

Based on the new in-memory source and sink, the rule pipeline can connect multiple SQL rules easily, efficiently, and flexibly. The readability and maintainability of SQL statements can be improved when implementing complex business scenarios. The rules are connected with in-memory topics similar to the MQTT topics and support wildcard subscriptions, enabling an exceptionally flexible and efficient rules pipeline. While improving business expressiveness, rule pipelining can also improve runtime performance in certain complex scenarios. For example, multiple rules need to process data that has been filtered by a certain condition. By extracting that filtering condition as a predecessor rule, you can make the filtering calculated only once, significantly reducing the computation in the case of many rules.

Portable Plugin: Making Extensions Easier

The original version of eKuiper supported an extension scheme based on the Go native plug-in system, supporting individual extensions to source, sink and function (UDF). However, due to the limitations of the Go plugin system, writing and using plugins is not easy for users familiar with Go, let alone users of other languages. eKuiper has received a lot of feedback from users in the community about plugin development, operation and deployment, and various operational issues.

To balance development efficiency and runtime efficiency, v1.4.0 adds a new Portable plugin system to lower the threshold of plugin development. The new Portable plug-in is based on the nng protocol for inter-process communication and supports multiple languages, currently providing go and python SDKs, with more SDKs to be added in subsequent versions according to user requirements; simplifies the compilation/deployment process, and runs like a normal program written in various languages without additional restrictions. Due to the different operation mechanisms, portable plugin crashes will not affect eKuiper itself.

Native plug-ins and portable plug-ins can coexist. Users can choose the plug-in implementation or mix them according to their needs.

Shared Connections: Source/Sink Multiplexed Connections

eKuiper provides a rich set of sources and sinks to access and send results to external systems. Many of these sources and sinks are input/output pairs of the same external system type. For example, MQTT and EdgeX both have corresponding source and sink, and in the new version, users can do all connection-related configurations in the connection.yaml file; in the source/sink configuration, you can specify which connections to use without repeating the configuration. Shared connection instances reduce the additional consumption of multiple connections. In some scenarios, users may be limited in the number of connections and ports to external systems, and using shared connections can meet this limit. Also, based on shared connections, eKuiper can support connections to the EdgeX secure data bus.

Other enhancements

  • Support for configuration via environment variables
  • By default, eKuiper uses SQLite, an embedded database, to store metadata such as streams and rules, allowing for no external dependencies at runtime. In the new version, users can choose Redis as the metadata storage solution
  • Rule status returns error reason
  • Optimized SQL Runtime, reduce CPU usage up to 70% in a shared source user scenario
  • sink dynamic parameter support, e.g., MQTT sink can set the topic to a field value in the result so that the data received can be sent to a dynamic topic
  • Authentication support: user-configurable JWT-based authentication for REST API

2022 Roadmap

As a young project, eKuiper is still far from perfect. There are a long list of feature requests from the users and community. eKuiper will continue to grow in the new year and open to anyone who is interested to make edge computing powerful and easier to use at the same time. The team will focus on the four themes:

  • Enrich API and syntax
    • more SQL clause (in, between, like etc.)
    • streaming feature (dynamic table etc.)
    • more handy functions (changed, unnest, statistic functions)
  • Stability & Performance
    • Allow to select feature by build tag and provide a minimal core version
    • Incremental window
    • Continuously memory footprint and CPU optimization
  • Extension
    • Stabilize portable plugin runtime and support more languages
    • More http source type
  • Build & Deployment
    • Cluster and HA
    • Android support

Please check the 2022 Roadmap in our Github project for details, and stay tuned in with the eKuiper community.