Skip to main content
Category

EdgeX Foundry

EdgeX Foundry Turns 5 Years Old & Issues its 10th Release

By Blog, EdgeX Foundry

By Jim White, EdgeX Foundry Technical Steering Committee Chair

EdgeX Foundry, an LF Edge project, turned five years old in April and celebrated with its 10th release!

Today, the project announced the release of Kamakura – version 2.2 of the edge/IoT platform.  EdgeX has consistently released two versions a year (spring and fall) since its founding in 2017.  Looking back to the founding and announcement of the project at Hannover Messe 2017, I would be lying if I was to say I expected the project to be where it is today.

Many new startup businesses don’t last five years. For EdgeX to celebrate five years, steadily provide new releases of the platform each year, and continue to grow its community of adopters is quite an achievement.  It speaks both to the need for an open edge / IoT platform and the commitment and hard work of the developers on the project.  I am very proud of what this community has accomplished.

We’ll take a closer look at EdgeX highlights over the past five years in an upcoming blog post, so for now, let’s dive into the Kamakura release. 

 

Here’s an overview of what’s new:

Kamakura Features

While this “dot” release follows on the heels of the project’s first ever long-term support release (Jakarta in the fall of 2021), the Kamakura release still contains a number of new features while still maintaining backward compatibility with all 2.x releases.  Notably, with this release, EdgeX has added:

  • Metrics Telemetry collection: EdgeX is about collecting sensor/device information and making that data available to analytics packages and enterprise systems in a consistent way so it can be acted on.  However, EdgeX had only minimal means to report on its own health or status.  An adopter could watch EdgeX service container memory or CPU usage or use the “ping” facility to make sure the service was still responsive, but that is about it.  In the Kamakura release, new service level metrics can be collected and sent through the EdgeX message bus to be consumed and monitored.  Some initial metrics collection is provided in the core data service (ex: number of events and reading persisted) for this release, but the framework for doing this system wide is now in place to offer more in the next release or allow an adopter to instrument other services to report their metrics telemetry as they see fit.  The metrics information can be easily subscribed to by a time series database, dashboard or custom monitoring application.

 

  • Delayed Start Services:  When EdgeX is running with security enabled, an initialization service (called security-secretstore-setup) provides each EdgeX micro service with a token which is used to access the EdgeX secrets store (provided by Vault).  These tokens have a time to live (TTL) which means if they were not used or renewed, they expired.  This created problems for services that may need to start later – especially those that are going to connect to future sensors/devices.  Furthermore, not all EdgeX services are known when the platform initializes.  An adopter may choose to add new connectors (both north and south) all the time to address new needs.   These types of issues and others often meant that adopters often had to shutdown EdgeX and restart all of EdgeX in order to provide new security tokens to all services.  In Kamakura, this issue has been addressed with a new service that uses mutual authentication TLS and exchanges a SPIFFE X.509 SVID for getting secret store tokens.  This new service allows a service to get or renew a token used to access the EdgeX secrets store anytime and not just at the bootstrapping / initialization of EdgeX as a whole.

 

  • New Camera Device Services:  While EdgeX somewhat supported connectivity to ONVIF cameras through a camera device service, the service was incomplete (for example not allowing the camera’s PTZ capability to be accessed).  With Kamakura, EdgeX will now have two camera device services.  A new ONVIF camera device service is more complete and tested against server popular camera devices.  It allows for PTZ and even the discovery of ONVIF compliant cameras.  A second camera service – a USB camera service – will help manage and monitor simple webcams.  Importantly, EdgeX will, through the USB camera service, be able to get a USB camera’s video stream to other packages (such as an AI/ML engine) to incorporate visual inference results into the EdgeX data sensor fusion capability.

 

  • Dynamic Device Profiles: in prior releases, device profiles were considered mostly static.  That is, once a device profile was used and associated to a device or any other EdgeX object like an event/reading, it was not allowed to change.  One could not, for example, have an empty device profile and add device resources (the sensor points collected by a device service) later as more details of the sensor or use case emerged.  With the Kamakura release, device profiles are much more dynamic in nature.  New resources can be added all the time.  Elements of the device profile can change over time.  This allows device profiles to be more flexible and adaptive to the sensors and use case needs without having to remove and re-add devices all the time.

 

  • V2 of the CLI:  EdgeX has had a command line interface tool for several releases.  But the tool was not completely updated to EdgeX 2.0 until this latest release.  For that matter, the CLI did not offer the ability to call on all 100% of the EdgeX APIs.  With Kamakura, the CLI now provides 100% coverage of the REST APIs (any call that can be done via REST can be done through the CLI) in addition to making the CLI tool compatible with EdgeX 2.0 instances.  Several new features were added as well to include tab completion, use of the registry to provide configuration, and improved result outputs.

The EdgeX community worked on many other features/fixes and behind the scenes improvements to include:

  • Updated LLRP/RFID device service and application services
  • Added or updated GPIO and CoAP services 
  • Ability to build EdgeX services on Windows platform (providing optional inclusion of ZMQ libraries as needed)
  • CORs enablement
  • Added additional testing – especially around the optional use of services
  • Added testing to the GUI
  • Optimized the dev ops CI/CD pipelines
  • Kubernetes Helm Chart example for EdgeX
  • Added linting (the automated checking of source code for programmatic, stylistic and security issues) part of the regular build checks and tests on all service pull requests
  • Formalized release and planning meeting schedule
  • Better tracking of project decisions through new GitHub project boards

Tech Talks

When EdgeX first got started, we provided a series of webinars called Tech Talks to help educate people on what EdgeX is and how it works.  We are announcing that a new Tech Talk webinar series will begin May 24 and run weekly through June.  The topic list includes:

  •   May 24 – Getting Started with EdgeX
  •   June 7 – Build an EdgeX Application Service
  •   June 14 – Build an EdgeX Device Service
  •   June 21 – Creating a Device Profile
  •   June 28 – Getting started with EdgeX Snaps

These talks will be presented by distinguished members of our community.  We are going to livestream them through YouTube at 9am PDT on the appointed days.  These new talks will help provide instruction in the new EdgeX 2.0 APIs introduced with the Ireland release.  You can register for the talks here:

What’s Next: Fall Release – “Levski”

Planning for the fall 2022 release of EdgeX – codenamed Levski – is underway and will be reported on this site soon.  As part of the Kamakura release cycle, several architectural decision records were created which set the foundation for many new features in the fall release.  This includes a north-south messaging subsystem and standardization of units of measure in event/readings. 

 EdgeX is 5, but its future still looks long and bright.

 

EdgeX 3.0 – the Future of Edge Computing

By Blog, EdgeX Foundry

By Jim White, EdgeX Foundry TSC Chair

Recently, the EdgeX Foundry project (a Linux Foundation project and part of the LF Edge umbrella of projects), released a dot release (v2.1) on top of our second major release which came out this summer.  This release was codenamed Jakarta and it is the community’s 9th overall release.  Jakarta was a stabilization release and is our first long-term support (LTS) release.

In reading my blog post title, you may be asking, what is this joker-of-a-technical-steering-committee-chairman talking about?  The paint on EdgeX 2.0 and our first ever LTS isn’t even dry and he’s talking about the third major release?

To put everyone’s mind at ease, EdgeX 3.0 is a long way off.  EdgeX 2.0 was a very big release.  The LTS release was a big commitment of support on the part of our community (we will support Jakarta for 24 months as defined by our LTS policy).  Furthermore, the community recently held its semi-annual planning meeting for the next release (codenamed Kamakura), and we know the spring 2022 release will be another dot release (EdgeX 2.2) with some additional new features but still backward compatible with all 2.x releases.  So, there is nothing on the immediate horizon that says EdgeX 3.0 is eminent.

I am not going to put a timeline on EdgeX 3.0 availability.  As a project, I am very proud of the fact that we have regularly released twice a year.  In 4 years of existence, EdgeX has had just 2 major releases.  EdgeX 3.0 would be the third, and given our current cadence.  Do the math and you can see that we are looking at more than year before EdgeX 3.0 is even remotely possible.  EdgeX 3.0 here serves as a metaphor of what big things are on the horizon for the project.

What I am going to pose in this post is a vision and a roadmap for the future of EdgeX Foundry.  I have the privilege of owning a front row seat in the creation and use of EdgeX for more than 6 years now.  I don’t suggest that I have all the answers or perfect vision with regard to the future, but I think my position grants me enough context and understanding to make some forecasts.  History combined with current requirements sprinkled with a few strong technical indicators can present a pretty good directional.  

To provide future me and my prognostications some escape, I include a couple of caveats with regard to my “vision”:  major industry disruption and time.  What I am presenting is a vision based on technology we know today or expect tomorrow.  Significant disruption (i.e., Internet or smartphones size impacts) is not something I can predict and it would potentially make this roadmap useless.  I am also not suggesting everything I am envisioning will be or even has to be in EdgeX 3.0.  The vision is a guidepost, but the journey may take a bit longer and more than one major release to accomplish – so again EdgeX 3.0 is metaphorical.  

OK – that said, allow the crazy TSC chairman to paint the picture of EdgeX 3.0.

Take the Bus but Allow Walking

When EdgeX was initially introduced, all service communications were via REST.  Whether your application needed something from an EdgeX service or two EdgeX services needed to talk, we used REST over HTTP.  This was a conscious decision.  We felt that REST was clean, simple and well understood.  If you are looking to get adopted and you have a complex tool to solve complex problems, you at least want to make it easy to communicate with.  REST is pretty easy to work with.

Quality of service needs, throughput efficiency, pub/sub model vs point-to-point communications, and message size are reasons why architects choose a message bus approach over REST.  These reasons are valid reasons to adopt messaging in edge/IoT computing too.

Over time, EdgeX has instituted more message bus communications for our services.  As of EdgeX 2.0, All communications from sensor to enterprise/cloud (the northward travel of the sensor data), can be done via message bus to deliver edge data.

 

In EdgeX 3.0, adopters must have the opportunity to use a message bus for all communications.  Most notably, an enterprise application or cloud system should be able to send a message on a message bus (MQTT, AMQP, Redis Streams, etc.) to request actuation on a device/sensor or to get details about what is happening at the edge.  Today, communications from north to south are all via REST.  Queries into EdgeX services or requests to change metadata, schedules or any type of configuration are also via REST.  Going forward, EdgeX 3.0 needs to allow communications with all services to happen by message bus.  That means all EdgeX services will need to be outfitted with topic endpoints and subscriptions to receive messages as well as the means to publish a message to a message bus.

This does not mean REST should be removed.  EdgeX is also about flexibility and simplicity.  REST interfaces suffice for some production use cases.  Further, REST makes it easier for a student of EdgeX to get up to speed and explore the platform.  REST also offers a great deal of assistance when debugging a broken system.  Use of REST allows users to walk around the services with a simple browser (whether trying to learn it or fix it), and with no additional setup or tools.

Finally with regard to EdgeX 3.0 and messaging, the platform should also embrace AMQP as an alternative to MQTT and Redis Pub/Sub (and ZMQ in some limited cases).  AMQP offers more features such as better support for cache and proxy.  AMQP is popular among some industries (finance and business) and is used in some IoT circles (notably supported in Microsoft’s Azure IoT Hub).  Because AMQP requires more resources (see issues with resource constraints below), it should not be the default message bus implementation.  Providing an optional implementation of the EdgeX message bus via AMQP, however, allows EdgeX to compete with products at the edge using this protocol and give it one more tentacle of flexibility.

Size Matters and Time is Relative

Since the beginning of my journey in IoT/edge computing, we were always working in limited resource environments but with the hope and expectation that Moore’s law would eventually be applied at the edge just as it has on our desktops, phones, enterprises and clouds.  This is a false assumption.  

I do think that the availability of resources at the edge is trending upward – it just happens much more slowly than every two years (per the law).  Why?  Two reasons: scale and time.

If your company was to replace all of its employee laptops, how many laptops would they be replacing?  Hundreds?  Thousands?  Sizeable yes.  And not cheap, but IT organizations rotate our laptops about every 2-3 years as a matter of course.  Agile development practices have our enterprise applications updated every few weeks.

Now imagine you are a city and you want to put an edge compute platform on every traffic signal.  Back of the napkin math says there are about 200 to 6000 compute platforms you are going to need for a square mile of city (using 4-12 lights per intersection, and about 45-550 intersections per square mile in a city – thanks Google).  How many square miles in a city?  Given Philadelphia has around 130 mi2 and Seatle has around 80 mi2, let’s just say 100 mi2 as a good working number.  That means we would need to field 20K to 600K platforms!  The cost to stick a high-end server with lots of resources on each traffic light would break a city.  The sheer scale of the deployment is too big to expect the use of something other than smaller/cheaper edge platforms.  This is not an isolated case.  IoT / edge deployments tend to get very big, very quickly.

Second reason – Moore’s law generally applies to information technology (IT) areas much better than it applies to operational technology (OT) areas.  Einstein was right – time is relative.  In OT realms, the rate of change is slower – much slower.  So even if companies, governments and institutions had access to cheaper, faster and bigger, stronger technology, deploying it into the hot, smelly, dirty, wet, corrosive and generally hard to reach places that edge computing goes to doesn’t happen overnight.  Getting to these systems can be a nightmare.  Most systems in OT stay in place at least 7 years.  It is not unusual to find edge technology that is in place for 15 years or more.  This is a far cry from the 2-3 year (or less) upgrade cycles found in IT environments.

Consequently, the environments that a platform like EdgeX needs to run in are constrained and likely to remained constrained for some considerable time into the future.

EdgeX started life as a Java platform.  It was too big to meet edge resource constraints.  We’ve worked hard to get our micro services lean and to operate in limited resource environments.  We also made an increasing number of services optional (use a minimal set of EdgeX services to support bare bones use cases).

As a project, we have to fight the urge to add to the platform such that it can only be used in expensive, non-resource constrained environments.  From the project’s onset, we used a small Raspberry Pi as our guidepost.  We were not endorsing Raspberry Pi.  It was a measuring stick.  If we could run EdgeX on that platform, we were within the realm of being able to run in a smaller, resource constrained, and yes cheaper, edge environment.

That measuring stick is still relevant to EdgeX 3.0 for reasons stated above.  

  • EdgeX needs to run in 1GB of RAM or less.  
  • EdgeX can run on a single core platform
  • EdgeX needs to run within 32GB of storage space.  
  • EdgeX needs to be able to ingest sensor data, make a decision (from a rules engine or other analytics package), and actuate a device in less than one second (within a single instance of EdgeX).
  • EdgeX needs to startup and be operational is 10 seconds or less.

There are some implications of this that are mentioned in other future platform decisions below (such as use of 3rd party components).  But importantly, as we add to EdgeX, we must take care not to allow EdgeX to get so big as to grow beyond the constraints of its slow-changing OT environment.

Adopt Versus Build – Finding Edge Worthy Components

For expediency and because the areas were not our specialty, the EdgeX community choose to adopt some enterprise grade open-source products.  We chose Drools initially to give us a rules engine to drive low latent decisions at the edge.  We incorporated Kong and Vault to provide our security API gateway and secret store.  We use Consul for our configuration/registry service by default.  We initially chose MongoDB for persistence.  

These were the right decision at the time.  We took a path where we do the edge and we let others do the things that they are good at.  “You do you” we said to 3rd party components and “we’ll do edge.”  Security, in particular, is not something we felt we should be creating on our own.

Over time, we have chosen alternates to help lower the EdgeX resource footprint.  We replaced MongoDB with Redis.  We chose eKuiper to replace Drools.  Still, it was a relatively hands off approach to incorporating best of breed 3rd party components.  Again – “you do you” and “we’ll do edge.”

But many of these choices, due to their enterprise and cloud native nature, are big – too big.  They are wonderful, but offer more capability than what will ever be used at the edge. They were not built for the edge.  eKuiper being an exception, but even there we have seen a large expansion of their feature set to address more (and perhaps more than the edge needs?).  The you-do-you, and we-do-edge approach is something we need to reconsider for EdgeX 3.0.  As a customer said to me recently, “EdgeX needs a diet plan.”

Have a look at the EdgeX performance numbers in the table below.  With regard to EdgeX services, most are under 25MB or less image footprint.  Memory usage by any EdgeX service is around 11MB and CPU utilization is miniscule.  Now note the size of the non-EdgeX services (those without the EdgeX prefix).  The numbers are stark.  Over the last few releases, EdgeX services are trending within a few megabytes of their original performance numbers (some even getting smaller).  3rd party services are getting bigger.    Its not that EdgeX needs a diet plan so much as we need to put our 3rd party components on a diet plan.

It is the 3rd party components that are costing EdgeX most on its ability to fit on resource constrained platforms.  There is a chance that there are still other, smaller options for some of these components.  But are the options built for the edge and with an edge attitude toward resource constraints? 

EdgeX has come to a point where it must consider one of a few options in adoption of 3rd party components:

  • Locate 3rd party component providers that are thinking edge resource constraints and use their smaller components to replace the current enterprise sub-systems we use today.
  • Partner with the current 3rd party component providers; give them our edge and resource requirements, and see if they are willing to create smaller, lighter versions of their products for the edge.
  • Create our own smaller, lighter components to address edge needs.  Perhaps start from trying to take an existing open-source product and cut it down; more limited in functionality but fine for the edge.
  • Allow adopters to unplug the heavy bits or implement their own.  Some use cases don’t require the 3rd party components.  When they are required, and by providing abstractions for all 3rd party components, EdgeX can continue to use the same 3rd party components as implementations for those abstractions, but make it easier for adopters and commercial implementers to create or select alternate implementations as use case resource constraints dictate.

I prefer these in order.  Finding other alternates has so far proven to be a challenge.  I recognize that other projects have their own priorities and supporting edge computing may not rise to a level of their concern (although one would hope that the size of the IoT edge opportunity would attract some).  Almost certainly, a project like eKuiper (now a sister project in LF Edge) would be open to finding ways to modularize or otherwise offer resource constrained functionality for resource constrained edge platforms.

Where we cannot find or convince a third party, open-source project to help meet our component need, EdgeX may have to look at implementing their own.  A dubious sounding task, but in actuality may not be that daunting given that edge needs are a map-reduce function against the bigger enterprise versions.  Depending on the architecture, we may find we can take a selective scalpel to the 3rd party component and create an edge worthy edition reasonably quickly.

There are some use cases where the security components are not needed (in physically restricted environments for example).  Some adopters may just drop the weight of our enterprise level components if we make it easy for them.  Forcing adopters to have to create their own components based on our abstractions should be used only as temporary fix or the answer when well-known, proprietary options exist for adopters to use in production deployments.

No longer can we just do “edge parts”.  If a 3rd party component was built for the enterprise / cloud and is not meeting our edge needs, we must begin to explore alternatives – to include our own implementation – to satisfy edge resource constraints.

We are going to want to add additional capability to EdgeX (as discussed in various parts of this document).  We are going to want to improve the platform.  If we want to stay within the resource constraints of our target host platform, we are going to need to reduce resource consumption in 3rd party areas to allow EdgeX to grow in other areas.

Not Cloud Native – Aim for Edge Native

Cloud native is the current rage in software development.  It is a term used to describe building and running applications that exploit cloud computing delivery models.  That is, the ability for an application to access compute resources in more of an on-demand way with scale, resiliency and flexibility in mind.  Specifically, cloud native applications take advantage of micro service architectures, containerized deployment and orchestration, agile development process, 12-factor app patterns, and good DevOps automation to usher work from code to deployable artifact.

There is a movement to drive cloud native principles in software engineering to the edge.  While there are some good aspects of cloud native computing that can be applied directly, using a cloud native approach on edge applications forgets many of the constraints of the edge (see here for a good list of some of those differences).  Others are calling for a modified approach that considers the unique requirements of the edge.  That is, they are cherry-picking elements of cloud native but still considering the constraints of the edge.  This approach is called edge native.  

EdgeX has adopted some elements of the edge native approach (small micro services, service availability tolerance, good DevOps automation, etc.).  In fact, EdgeX adopted some of these elements even before the term edge native existed.  I believe that our EdgeX community would agree with the general philosophy in edge native computing.  That is, we would agree that edge applications should be “built from the ground up with the Edge in mind – just like Cloud-Native applications are built for the Cloud.”  

The definition of edge native is still somewhat nebulous and debated.  Depending on the source, EdgeX adheres to some elements of edge native, but does not adhere to other characteristics.  And there are some guidelines of developing edge native applications that I would not suggest EdgeX adhere to blindly. 

As an example, use of containers is considered a staple (required?) in many edge native guidelines.  EdgeX has always provided containers, but doesn’t require use of containers.  In fact, the project supports alternate delivery technologies (like snap packages – developed by Canonical) and understands the reality of today’s edge infrastructure.  

  • Some OT groups have not embraced containerized workloads and have suggested it may be several years before they support containers in production.  
  • Some resource constrained platforms make use of containers impossible.  
  • Certain devices/sensors are going to make an all-container strategy a challenge.  

While EdgeX supports container use, it does not dictate it.  Flexibility is key so long as the edge is very heterogenous.  

What principals of edge native computing should EdgeX look to embrace that it does not do today?

Distributable

Edge native applications should be able to move as resources (compute, network, storage, etc.) dictate.  Services should be able to scale out to the edge or scale back to the enterprise / cloud as resources warrant.  For example, an application service could run at the far edge and provide for low-latency decisions when resources are available.  But they may also run in the enterprise or in the cloud when resources aren’t available.

In theory, EdgeX services were designed to be distributed on different hosts (on physical systems or virtual machines with a different IP address).  However, in reality, there are a number of issues with the current architecture that might make distributing services across hosts difficult.   Chief among them is that when the services are distributed, there isn’t an easy way to secure the communications between services – a necessity for edge native applications.  Providing central configuration and secrets across the distributed services is not fully addressed in EdgeX.  EdgeX 3.0 needs to allow any EdgeX micro service to live anywhere, anytime and still operate securely.

Resiliency and Rapid Recovery

Enterprise and cloud platforms and associated resources (compute, network, storage, etc.) are relatively stable.  Edge platforms are notoriously unstable.  EdgeX 3.0 needs services to be more resilient to edge failures and outages.  When service failures do occur, the services need to be brought back up quickly.

This does not mean EdgeX services need to be “highly available.”  High availability (HA) is the ability of a system or service to operate continuously without failing for long, agreed upon lengths of time.  It often requires some orchestration capability to monitor the services and offer “backup” or redundant services in the face of failure.  The resource constrained nature of edge platforms makes offering HA at the edge a particular challenge.

Dependent services will go down.  Resources like the network will become unavailable.  The EdgeX 3.0 services must be built to be withstand these issues.  When a service does go down, it must be able to be restarted quickly and not require a lot of new configuration or setup on the part of the user (such as new security keys that cannot automatically be provisioned).

I submit that in EdgeX 3.0, a service remains up and continues to try to acquire any dependent services or resources indefinitely unless trying to do so creates other issues.  It should also be able to use the alert/notifications service (if it is up) to alert on the situation to a default HTTP REST endpoint, send an email, SMS message or otherwise alert the systems overseer.  When the dependency is available, the service should come back on line and continue normal functions without the need for other intervention on the part of the user.

When a service fails, it should be able to restart within the one-minute system startup time and without other intervention on the part of the user that had to issue the start (or restart) command.

The community should also offer example scripts or services that would check that all services are functioning and when a service (or dependent facility such as the database) are down, it attempts to restart them.  The example “restart” service or script does not have to be part of a default EdgeX 3.0 deployment, but should serve to help adopters think about how to keep an edge system alive even during partial failures.

Indeed, when something like Kubernetes is used to deploy and orchestrate EdgeX services, resiliency and rapid recovery may be offered taken care of by that environment.  But EdgeX must always be able to handle itself when platform constraints don’t allow for CNCF types of deployment management.

The Other Data – EdgeX Control Plane Telemetry and Health / Monitoring

Today, EdgeX does not report on its own health and operations.  Therefore, there is no way to automate any type of higher order management of the platform.  For example, there would be no way to know if a device service is reporting sensor data as expected – at least not without manual intervention and exploration of log files (provided the right log levels are set).

Starting with the Delhi release, EdgeX offers a system management service.  This service was created to be able to perform a limited set of monitoring and control plane functions.  This includes:

  • Starting, stopping, restarting the EdgeX services
  • Providing memory and CPU usage of the service
  • Provide a service’s configuration
  • Provide an indication of whether the service is operational based on its response to ping requests

Because some of these functions (start, stop, restart and metrics collection) are dependent on how the services are deployed and running (via container or on a native Linux OS), the architecture of the service requires an “agent” request information from a specific platform-dependent “executor” to carry out much of the system management service functionality.  The system management service, therefore, suffers from several issues:

  • It doesn’t work for all environments (Windows)
  • It has to be informed of all new or removed EdgeX services (not easy to do in dynamic situations)
  • The architecture (two components versus a single service) makes it more difficult to setup and run

Perhaps, most importantly, the service isn’t providing any capability that couldn’t be obtained from use of other tools depending on how EdgeX is deployed and where it is running.  For example, if EdgeX is deployed via Docker containers, Portainer or Linux tools could be used to do most of the system management functions except for providing configuration which is always available via Consul.

The intention was that the system management service would eventually be extended to get more information from each service (telemetry and event information that was specific to EdgeX) and make that available to monitoring services and/or other applications.  The telemetry and event information is not something an outside tool could provide since it would require knowledge and access to EdgeX service internals.  

Where that leaves the project is with a service that is redundant to better tools and technology and unable to provide much needed (and otherwise unobtainable) telemetry data that it should be providing.  For this reason, the community has decided to mark the service deprecated with the Jakarta release.  It may not be removed from the platform until something else provides for its replacement, but at least the community is signaling that the system management service in its current form is nearing end of life.

By the way, as reported earlier in this post, the system management agent (SMA) is the most expensive service in the EdgeX inventory by a factor of 10.  So, removing this service as it exists today in EdgeX paves the way for additional features going forward.

EdgeX 3.0 needs to offer much more data to adopters about its health and operations.  Telemetry or metrics from each service can be used to understand whether a sensor is reporting correctly, if the system has the proper resources to support adding additional sensors, or even if sensors are being used to flood the system with information (denial of service through sensors).

Rather than collecting telemetry by having an outside service request (pull) it, each service needs to be able to publish (push) telemetry out.  The telemetry can be pushed anywhere.  In early implementations, EdgeX services may just be configured to send telemetry to a designated message topic where it is up to the adopter to figure out how to collect, use or respond to any telemetry.  Later, the telemetry data can be treated as alternate edge data (control plane data vs sensor data) that is consumed by EdgeX application services, rules engines or other analytics services.  Telemetry data does not originate from a “thing”, but its data can help drive operations and actuation at the edge as necessary.  

For example, imagine a device service reported telemetry about the number events it produced over a given period of time.  If the telemetry – in the form of control plane Events/Readings – were picked up by application services and routed to the rules engine, the rules engine could be configured to look out for sensors reporting more events than expected.  This could trigger the shutdown of that rogue sensor until a user explores the reason for the extra reporting.

Application services could also subscribe to service telemetry messages in order to filter, aggregate and otherwise prepare and ship telemetry data out of EdgeX – just as they are used to export sensor data today.

Each service is responsible for certain functions and responsibilities within an EdgeX instance.  Each service knows (or should know) what is critical to its operations and functions.  Therefore, each service should have the means to report telemetry specific and important to that service, and make this EdgeX specific information available to EdgeX as well as external systems.

The Event/Reading structure and services that handle Event/Readings may need to be modified slightly in recognition that the Event/Reading may contain sensor data or service telemetry data.   Hopefully, Telemetry data and sensor data Event/Readings will differ only slightly in EdgeX 3.0 thereby requiring only minimal change.

EdgeX 3.0 services will need additional configuration so as to control how much and what telemetry gets reported.  The user should be able to increase or decrease the telemetry reporting based on operational circumstances and use case needs (not unlike how logging output is adjusted today).  Telemetry or metrics collection can impact the performance of EdgeX services.  There may be a need, based on resource constraints in some deployments, to complete turn off telemetry collection and publication for the entire instance.

The basic premise of initial EdgeX telemetry collection is already specified in a proposed EdgeX ADR.

More EdgeX 3.0 Features

In addition to the vision provided above, EdgeX 3.0 will need to add some additional capability that it does not have today.

Alternate Language App Functions SDK

EdgeX has two device service (DS) software development kits (SDKs); one in Go lang and the other in C.  Since most of the platform is created in Go lang, it seems fitting and natural to have a DS SDK in Go.  We also have a DS SDK in C because it is the most natural fit for most low-level protocols and “thing” communications.

On the north side, we have just a single App Functions SDK to create new/custom application services (AS) in Go Lang.  Again, given EdgeX’s foundation in Go, a Go lang SDK here also makes sense.  Going forward, I anticipate more interfaces with artificial intelligence (AI) / machine learning (ML) and other analytics packages.  One of the more popular languages for the AI/ML communities is Python.  It would seem appropriate that EdgeX 3.0 provide a north side SDK that is more familiar to those more likely to need and build AS.  Other organizations are using EdgeX to translate from one OT protocol (like Modbus) to another OT protocol (like BACnet or OPC UA).  OT people tend to use C and C++ and it stands to reason that an App Functions SDK in one of these languages may better support their needs.  In general, the SDKs at either north or south end of EdgeX need to tend toward the language tendencies of the user groups most likely to use them. 

Distributed Ledger Support

IoT platforms are anticipated to generate 80 zettabytes of data by 2025 by one recent report.  As that data begins to be shared – potentially even bought and sold on the open market – the origination, legitimacy, and ownership of that data is going to need to be monitored and managed.  This screams for distributed ledger technology (DLT).  The closer that the data can be tracked and attributed, the better.  This means that IoT/edge platforms like EgdeX will need to integrate with DLT platforms in order to put its sensor data into a digital ledger.

EdgeX 3.0 will need to offer some initial integration to popular, open source DLT.  DLT can be resource intensive.  So DLT at the edge may be an option to be used only when the use case dictates it and the edge platform can support it.

K8s Is Coming

I said earlier that EdgeX 3.0 will move closer to edge native, but that doesn’t mean it has to offer high availability.  However, without a doubt, Kubernetes (K8s) is moving to support the edge.  K3s, MicroK8s, KubeEdge and other efforts stemming from the Kubernetes are just the forerunners to what will be some type of K8s support at the edge.

Not every organization will support its use at the edge – it can be complex.  Not all edge environments will have the resources to run some form of K8s at the edge.  And using K8s in whatever form it takes to deploy and orchestrate edge native applications will have to make some allowances for edge constraints and challenges.  But make no mistake, like winter, K8s is coming.  EdgeX 3.0 will need to do more for those looking to use K8s.  Providing deployment / orchestration assistance equal to that which we do for Docker Compose and Snaps will be imperative.

We have always claimed to be deployment/orchestration technology agnostic.  We will remain so.  But at some point, we should expect some form of K8s technology (probably a far cry from what we see today which is largely inadequate to edge native) to be the predominant means of deploying edge applications.  We will also have to educate those coming from a cloud native world why K8s is not always a fit for edge native.

UoM

We have already begun a conversation in the community about how to support unit of measure standards with regard to our sensor data collection.  That is, how to attribute sensor data to a well-defined unit of measure.

IoT / edge platforms have been be specific about the sensor data collected.  For that matter, devices are pretty lack with regard to how they send data.  I own two IoT temperature probes that send an integer of “793” when trying to tell me the temperature reading is 79.3 Fahrenheit.

In order for the sensor data to be more trusted and offer better value (see DLT comments above), the data will need to be tied to an appropriate unit of measure and that unit of measure should be well defined by some body.

EdgeX 3.0 will not create or dictate the unit of measure standard, but it will allow those that need more specificity to label sensor data with a unit of measure and the standard it comes from.

This discussion and the premise for a unit of measure solution are already specified in a proposed EdgeX ADR.

Automate Thing Provisioning

The “last mile” is a term that originated in supply chain management and then adopted by telecommunications. It was used to describe the most difficult part of a journey or furthest part in a network – which usually was found at the end or literally the last mile.

In edge computing, the last mile is that between our platform and the actual device or sensor.  Connecting the sensors or devices with all the protocols, data formats, complex hardware, etc. is a real challenge and is why EdgeX is so important.  EdgeX, as we know, helps to simplify and standardize how “things” of the OT world get connected to the IT world.

While EdgeX makes the last mile shorter, so to speak, there is still a fair amount of ceremony and work required to provision a new sensor.  As an EdgeX user, you still have to provide a device profile, issue the correct Metadata calls or providing the right device configuration in order to onboard a new “thing”.  We provide some amount of device discovery and automated provisioning, but it is limited to a few protocols and doesn’t go far enough.  

Many sensors/devices today can provide more information about the resources they have to offer or the actuation commands they support.  Many sensors/devices advertise their presence and offer platforms like EdgeX the ability to discover and onboard them automatically.

EdgeX 3.0 needs to complete the journey and make the last mile of connectivity easier for adopters.  Where possible, and with appropriate safeguards, EdgeX 3.0 should be able to onboard a new senor/device as soon as it is powered up and in communication range of EdgeX and the host it runs on.

There will be challenges, especially with more legacy protocols.  However, an edge platform that enables “thing” connectivity simply by powering the sensor on will help to make IoT / edge computing truly more ubiquitous.  

Wrap Up

The opinions and vision expressed in this post are my own.  While I am currently the EdgeX Foundry TSC chairman, the vision depicted here is not yet the opinion adopted by the EdgeX community.  It is not the codified roadmap for EdgeX.  I hope it will be – or that the community will take this vision and improve upon it, which they usually do.  “A goal is not always meant to be reached; it often serves simply as something to aim at.”

We’ve spent the last two years working on EdgeX 2.0 and our first LTS release.  I am so proud of the work that was accomplished and where EdgeX is at, but I don’t want adopters or the community to think we have reached the end.  We are not at the end.  We are at the beginning of creating the best open source IoT / edge platform on the planet.  

 

EdgeX Performance Update

By Blog, EdgeX Foundry

Written by James Butcher, EdgeX Foundry Core Working Group Chair and Edge Xpert Product Manager at IOTech Systems

 Following the recent release of EdgeX Foundry version 2.1, codenamed “Jakarta”, I thought it would be useful to provide a quick update on some of the performance metrics of the platform as it has evolved over the last couple of release cycles.

This release of EdgeX is also the first long term support (LTS) edition, whereby the EdgeX community will support this version with critical fixes for major flaws or bugs. The project’s detailed testing strategy helps to provide the confidence that this version is robust and reliable – and is key to the community making those LTS statements. See here for more details about the LTS policy.

Recent EdgeX Working Group Changes

You may know that the EdgeX QA & Test Working Group was previously responsible for the creation and operation of the community’s testing strategy and its main Test Automation Framework (TAF). This summer, the QA & Test group was merged into the EdgeX Core Working group, and I was pleased to be given the opportunity to chair the new combined group.

A key part of the EdgeX Core Working Group is its commitment to testing which helps ensure the quality and robustness of the framework. An EdgeX version is only released, for example, when all key requirements are reliably tested and preferably automated as part of TAF.

EdgeX Performance Metrics 

Another key part of the EdgeX testing strategy is the recording and monitoring of performance metrics. Since EdgeX 1.1 (Fuji), we have been running dedicated performance tests with each release and producing formal performance reports that describe the findings.

I mentioned some of the performance testing advancements in my blog around the release of EdgeX 1.3 (Hanoi). We cover key points such as footprint, CPU usage and latency of data flow through the platform. 

In the last couple of cycles (Ireland and Jakarta) we are now also recording performance related to running with the EdgeX Security Services.

Please find the Jakarta Performance Report here or click on the image below. 

The data continues to show that the EdgeX microservices developed by the community are generally pretty small and lightweight. One of the EdgeX Core Services, Core Metadata, for example, has a Docker image footprint of around 17MB.

Some of the third-party services we bring in, such as the Security Services or the Registry Service are a little larger, but still the footprint of the complete EdgeX stack requires less than 1GB of disk space. Note also the microservices architecture means not all services need to be deployed in all scenarios. It’s easy to pick and choose the services needed for each use case or physical hardware capability.

So whilst quite lightweight, we are still pushing for EdgeX to be smaller and faster where possible. The next couple of EdgeX development cycles (Kamakura and Levski) are devoting time to this, but a nice reduction in Jakarta is a drop in the run-time memory usage for the API Gateway Security Service. In previous EdgeX versions, memory consumption of the complete stack was recorded as around 1GB RAM, but an optimization such as configuring the API Gateway Service to run with a specific number of worker processes, means we can be much lower than that when needed. These types of config options are invaluable in helping to tune the framework, if physical resources are a concern.

Full Commercial Support and Value Add

I also wanted to mention IOTech’s commercially supported edition of EdgeX, named Edge Xpert. Edge Xpert 2.1, based on EdgeX Jakarta, will be available very soon so stay tuned for more info. Head to the IOTech website to understand how Edge Xpert features and its technical support offerings can help users deploy the EdgeX based technology more easily.

Open Community

Finally, please feel free the join our EdgeX Core meetings where we discuss progress and other issues that need to be addressed each week. We meet every Thursday at 8am PST. You can find the meeting links on our page here

 

 

EdgeX Foundry to Host “EdgeX Smart * Challenge,” a Virtual Global Hackathon, in Early 2022

By Announcement, EdgeX Foundry

 

SAN FRANCISCO – December 9, 2021EdgeX Foundry, the open source, vendor-neutral IoT/edge platform hosted by the Linux Foundation as part of the LF Edge project umbrella, is pleased  to announce the EdgeX Smart * Challenge – a virtual global hackathon – to begin in early 2022.

The Smart * (“Smart Star”) Challenge will highlight EdgeX’s applicability in use cases such as

  • smart building – the use of EdgeX in a solution that improves the safety, security, or efficiency of a building or provides building inhabitants more comfort, quality of life or productivity enhancements.
  • smart agriculture – the use of EdgeX in a solution that increases the quantity or quality of crop and animal products, reduces labor costs or improves safety of ag workers or the food supply they deliver.
  • smart energy – the use of EdgeX in a solution that improves energy (electric, gas, etc.) efficiency, optimizes energy storage or transportation, reduces environmental impact from energy use, improves energy sustainability, or helps ensure or protect energy grid safety.
  • smart manufacturing – the use of EdgeX in a solution that improves manufacturing/factory output, optimizes supply chain flow in the manufacturing process, reduces loss, prevents maintenance or other manufacturing system outages, reduces energy use, or improves worker safety

Sponsors for the Smart * Challenge include Intel, HP, BSI and IOTech Systems.  If you are an organization that has interest in helping sponsor the event, please email interest to info@edgexfoundry.org

The challenge will be held virtually over five weeks between January 24 and March 4, 2022. 

Edge/IoT teams interested in competing in this challenge should explore the contest Wiki site and send an email to info@edgexfoundry.org to receive a notification when registration for the event is open.

Details on past hackathons, including the EdgeX Foundry Challenge, Shanghai 2020 and the inaugural EdgeX hackathon, The EdgeX Open  2019

About the Linux Foundation

Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open-source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more.  The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.

 ###

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds. 

How Do You Say Thank You to Contributors? EdgeX is Trying to!

By Blog, EdgeX Foundry

By Jim White, EdgeX Foundry TSC Chair

As I write this, it is Thanksgiving week in the United States.  A time for everyone to reflect, share some time with loved ones, and show appreciation for the year’s blessings.  I cannot think of a better time to also say “thank you” to the EdgeX Foundry community of volunteers.  We just released our 2nd release of the year (the Jakarta release which is our 9th release overall).  Saying “thank you” seems insufficient and I really wish there was a better way to express the gratitude I have to all the men and women that continually do such great work and allow us to consistently deliver EdgeX releases cycle after cycle. 

I have said it before but it bears repeating, I consider myself fortunate to be associated with such a great group of people that work on EdgeX Foundry – both our volunteers and those that assist from the Linux Foundation.  Co-founding the project and serving as the project’s leader has been the highlight of my career.  Importantly, it is the people that I have had a chance to meet and interact with through the project that has made the experience so wonderful.

As anyone that has the fortune to lead an open-source project will tell you, volunteers – that is the contributors to the project – are the lifeblood of the project.  If you don’t have enough volunteers or if you don’t have a community of people with great attitudes and willingness to work together to make something great, your project will soon flounder and fail.  Because of organizational or personal commitments, an open-source project must constantly seek out new contributors who also bring new ideas and energy to a project.  This is not easy and project leaders often have to play the role of cheerleader and recruiter to find people willing to part with their most precious commodities: time and knowledge.

That is why EdgeX Foundry is excited to announce a new program meant to thank those that contribute to the project and hopefully entice new contributors to spend time on the project.

As of  November, the EdgeX Developer Badge program will send a digital badge to contributors on two occasions:

  1. When a contributor submits their first pull request that gets accepted and merged into the EdgeX Foundry code base (via GitHub)
  2. When a contributor fixes two bugs that have been documented via GitHub issues in one of the project’s repositories

The new EdgeX Foundry Developer Badges for bug fixing (the “Bug Hunter” Badge) and first contribution (First Time Contributor Badge)

Additional badges may be awarded by the project in the future.  The badges will be automatically issued via the project’s CI/CD processes and through Credly.com.  Developers will receive an email notification from the project when they have made their badge-worthy contribution.   They will also receive an email from Credly that allows them to accept their digital badge and then share their verifiable credential on LinkedIn, Twitter or other social media platform of their choosing.  It is a small token of appreciation on the part of the project to say “thank you for your work” that also allows a contributor to post recognition of their effort to professional or personal media outlets.  In other words, it’s an official way to provide some “street cred” to our volunteers, which we hope will attract their peers and co-workers to seek the same.

To our knowledge, this is a first of its kind program for an open-source project.  If there is an open-source project out there that would like to copy our program, please feel free to reach out to me for more information.  We’d like to see more contributors in more open-source projects and if this can help, we would be happy to share what we have done.

While saying thanks, I’d like to thank Aaron Williams who was the LF Edge Developer Advocate until this summer for coming up with the initial program idea.  I’d also like to thank Ernesto Ojeda, our EdgeX Dev Ops Working Group Chairman, and his Intel DevOps team for implementing the program.  It’s another example of the great people and work found in the LF Edge and EdgeX communities.

On behalf of the EdgeX Foundry project, we wish everyone a joyous holiday season.

 

It’s Here! Announcing EdgeX 2.0 – the Ireland Release

By Blog, EdgeX Foundry

By Jim White, EdgeX Foundry Technical Steering Committee Chair

 

 

 

 

 

 

 

When you think of Ireland, most people envision bright green, fresh landscapes.  An island cleaned regularly by plentiful rain and always made new.  The Ireland name is appropriate to our latest release – which is EdgeX made fresh,  a major overhaul.  

EdgeX Foundry 2.0 is a release over a year in the making!  

This is the EdgeX community’s second major release and our 8th release overall.  Indeed, the Ireland release contains both new features and non-backward compatible (with any EdgeX 1.x release) changes.  In general, this release took a huge step to eliminate, or at least significantly reduce, about four years of technical debt accumulation, while also adding new capabilities.

Why Move to EdgeX 2.0?

Whether you are an existing EdgeX user or new to the platform and considering adoption, there are several reasons to explore the new EdgeX 2.0 Ireland release:

  • First, we’ve completely rewritten and improved the microservice APIs.  I’ve provided some more details on the APIs below, but in general, the new APIs allow for protocol independence in communications, better message security, and better tracking of data through the services.  They are also more structured and standardized with regard to API request / response messages.
  • EdgeX 2.0 is more efficient and lighter (depending on your use case).  For example, Core Data is now entirely optional.
  • EdgeX 2.0 is more reliable and supports better quality of service (depending on your setup and configuration).  For example, you can use a message bus to shuffle data from the sensor collecting Device Services to exporting Application Services.  This will reduce your reliance on the REST protocol and allow you to use a message broker with QoS levels elevated when needed.
  • EdgeX 2.0 has eliminated a lot of technical debt and includes many bug fixes.  As an example, EdgeX micro service ports are now within the Internet Assigned Numbers Authority (IANA) dynamic/private port range so as not to conflict with well-known system or registered ports.

Introducing the V2 APIs

During one of our semi-annual planning meetings back in 2019, the community agreed that we needed to address some challenges in the way that EdgeX microservices communicated – both with each other and with third- party systems.  We identified some technical debt and knew that it had to be addressed in order to provide the platform the base from which new features, better security, and additional means of communications could be supported in the future.  This meant changing the entire API set for all EdgeX microservices.

What was “wrong” with the old V1 APIs?  Here were some of the issues:

  • Intertwined data model/object information with poor URI paths that exposed internal implementation details and made the APIs difficult to use.  Here is an example comparing the v1 and v2 calls to update a device.

V1: PUT /device/name/{name}/lastconnected/{time}

V2:  PATCH /device [with JSON body to specify the device info]

  • In many cases, requests often lacked any formal response object.  The HTTP status code was all that was returned.
  • The APIs and model objects were REST specific and didn’t support other communication protocols (such as message bus communications via MQTT or the like).
  • Used a model that was very heavy – often containing full embedded model objects instead of model object references.  For example, in v1, when requesting device information, the device profile and Device Service were also returned.
  • Used inconsistent naming and included unnecessary elements.  This made it hard for users to figure out how to call on the APIs without checking the documentation.  
  • Lacked a defined header that could be used to provide security credentials/tokens in the future.

Changing the entire API set of a platform is a monumental task.  One that required the collective community to get behind and support.  Given the size of the task, the community decided to spread the work over a couple of release cycles.  We began the work on the new APIs (which we called the V2 APIs) last spring while working on the Hanoi release and completed the work with this release.

The new APIs come with common sense request and response objects.  These objects share new features such as a correlation ID which allows requests and responses among services to be associated, and it also allows the flow of data through the system to be tracked more easily.  When a piece of sensor data is collected by a Device Service, the correlation ID on that data is the same all the way through the export of that data to the north side systems.  Below is a simple example of the new request and responses – in this case to add a new Device Service.

 

Importantly, the V2 APIs set us up for new features in the future while also making EdgeX easier to interact with.

EdgeX 2.0 Feature Highlights

Technical debt removal and the new V2 APIs aside, what else was added in EdgeX 2.0?  Plenty!  Here is a sample of some of the more significant features.

Message Bus and Optional Core Data

In EdgeX 1.0, all communications from the Device Services (the services that communicate with and collect data from “things”) and Core Data was by REST.  In EdgeX 2.0, this communication can be performed via message bus (using either Redis Pub/Sub or MQTT implementations).  In addition to more “fire and forget” communications, the underlying message bus brokers used in message bus communications offer guarantees of delivery of the message.

Moreover, EdgeX 2.0 takes the message bus implementation a step further by allowing Device Services to send the data directly to Application Services via message bus.  Core data becomes an optional secondary subscriber in this instance.  For organizations that do not need to persist sensor data at the edge, this option allows the entire Core Data service to be removed from deployment helping to lighten EdgeX’s footprint and resource needs.

REST communications from Device Services to Core Data can still be done, but it is not the default implementation in EdgeX 2.0.  The diagrams below depict the old versus new message bus service communications.

Improved security

In the 2020 IoT developer survey, security remained one of the top 3 concerns of people building and fielding IoT/edge solutions.  It also remains a prime concern of the members of our community.  The EdgeX project, with each release, has worked to improve the security of the platform.  In this release, there were several additions.

  • We created a module and a pattern that provides a common means for all the services to retrieve secrets from the secret store (EdgeX uses Vault as the secret store by default).  We call this feature “secret provider for all.”
  • EdgeX uses Consul for service registry and configuration.  In this release, the API Gateway is used to allow access to Consul’s APIs.  Access to the Consul APIs in EdgeX 1.0 was denied making changes in Consul difficult.
  • Consul is now bootstrapped and started with its access control list system enabled – offering better authentication/authorization of the service and better protection of the key/value configuration it stores.
  • Fewer processes and services are required to run as root in Docker containers.
  • The API Gateway (Kong) setup has been improved and simplified.
  • EdgeX now prevents Redis, the persistent store for EdgeX, from running in insecure mode.

Simplified Device Profiles

Device profiles are the way that users define the characteristics about sensors/devices connected to EdgeX – that is the data they provide, and how to command them.  For example, a device profile for BACnet thermostats provides information on data a BACnet thermostat sends, such as current temperature and humidity level. It also defines which types of commands or actions EdgeX can send to the BACnet thermostat such as the ability to set the cooling or heating point.  

Device profiles are specified in YAML or JSON and uploaded to EdgeX.  As such, they are the critical descriptions that makes EdgeX work.  Device profiles allow users to describe their sensors and devices to EdgeX without having to write lots of programming code. 

Writing device profiles in previous releases could be long and tedious depending on the device and what type of data it provided.  We greatly simplified and shortened device profiles in EdgeX 2.0.  As an example, here is the same essential profile in EdgeX 1 and EdgeX 2.

New Device Services

The growth of an open-source community can often be measured by how well it is attracting new contributors into the project.  During this release, thanks to some new contributors, we added several new Device Services.  Here is the new Device Services you can now find in the project:

  • Constrained Application Protocol (CoAP) is a web transfer protocol used in resource constrained nodes and networks.
  • GPIO (or general pin input/output) is a standard interface used to connect microcontrollers to other electronic devices.  It is also very popular with Raspberry Pi adopters given its availability on the devices.
  • Low Level Reader Protocol (LLRP) is a standardized network interface to many RFID readers.
  • Universal Asynchronous Receiver/Transmitter (UART) is serial data communications and is used in modems and can be used with USB in a USB to UART bridge.

These Device Services were contributed to EdgeX 1, but are being updated this summer to EdgeX 2.0.

New Graphical User Interface

In this release, a new and complete interface using Angular.JS (with some Go code for backend needs) has been added.  This GUI, complete with EdgeX branding, aligns with industry standards and technologies for user interfaces and should provide a platform for future GUI needs.

In addition to providing the ability to work with and call on EdgeX services, it provides a GUI “wizard” to help provision a new device (with associated Device Service and device profile).  What would typically take a user several complex REST API calls can now be done with a wizard interface that carefully walks the user through the creation process providing simple fill-in forms (with context help) to create the device.

The GUI also provides a visual status display that allows users to track the memory and CPU usage of the EdgeX services.  In the future, additional visualization will be added to be able to display sensor data.

Application Services

Application Services are used to export EdgeX data off the edge to enterprise and cloud systems and applications.  Application Services filter, prepare and transform the sensor data for easy consumption.  Several additions were made to the Application Services and the SDK that helps create them (called the application functions SDK).  The following is a list of some of the new features and functions:

  • New functions to filter sensor readings by device profile name and device resource name before exporting
  • Allowing multiple HTTP endpoints to be specified for export by one Application Service
  • Subscription to multiple message bus (enabling multiple filters by subscriptions)
  • Provided a new template for easier/faster creation of custom Application Services

Additionally, a new LLRP inventory Application Service was contributed to help correlate data coming from RFID readers to backend inventory systems.

EdgeX Ready

EdgeX Ready is a program designed to highlight member organizations that demonstrate the ability to work with EdgeX.  The EdgeX Ready program was launched with the EdgeX 2.0 release in order to promote awareness of users and their organizations that have EdgeX expertise.  This is the first step or a precursor to potentially exploring an EdgeX certification program.

Today, the EdgeX Ready self-assessment process requires an EdgeX adopter to:

  • Get and run an instance of EdgeX (and associated services)
  • Create and validate a device profile.
  • Use the profile with an EdgeX Device Service to provision a device and send sensor data into EdgeX.

When an organization  completes the EdgeX Ready process, they signal to other community members that they have successfully demonstrated they have EdgeX knowledge and experience.  In return, the EdgeX community highlights EdgeX Ready members on the EdgeX Foundry website with an EdgeX Ready badge (shown below) next to their logo.

Additionally, the program hopes to promote sharing of EdgeX device connectivity elements and sample data sets, which are central to the current submission process and helpful to others learning EdgeX.  Future EdgeX Ready programs may highlight additional levels of ability and experience. 

Under the Hood and Behind the Scenes

Effort by DevOps, Testing, Quality Control and Outreach teams often go unnoticed in any software effort.  These teams are composed of unsung heroes of the EdgeX 2.0 as they contributed substantially to this massive undertaking.  In addition to their normal roles of building, testing and marketing EdgeX, they managed to further the project in many ways.

Our DevOps team cleaned up the EdgeX Docker image names (removing unnecessary image prefix/suffix names) and provided descriptions and appropriate tags on all of our images in Docker Hub – helping adopters more quickly and easily find the EdgeX Docker images they need.  They also added informative badges to our GitHub repositories to help community developers and adopters understand the disposition and quality of the code in the repositories.

Our Test/QA team added new integration tests, performance tests and “smoke” tests to give the project an early indicator when there is an issue with a new version of a 3rd party library or component such as Consul – allowing the community to address incompatibilities or issues more quickly.

Finally, our marketing team revamped and upgraded our www.edgexfoundry.org website and create a Chinese version of our website in support of our ever growing Chinese community of adopters.  The welcome mat of our project now has a clean new look, better organization, and a lot more information to provide existing and potential adopters.  The website should help EdgeX adopters the information and artifacts they need more quickly and easily, while also highlighting the accomplishements of the community and its membership.

If all this wasn’t enough, here is a list of some of the other accomplishments the project achieved during this release cycle:

  • Improved the Device Service contribution review process.
  • Incorporated use of conventional commits in the code contribution process.
  • Started a program to vet 3rd party dependencies; insuring 3rd party dependencies are of sufficient license, quality and development activity to support our project.
  • Help launch the Open Retail Reference Architecture project to foster the development of an LF Edge reference architecture for retail adopters.
  • Entered into liaison agreements with the Digital Twin Consortium and AgStack (a new LF project) to figure out how EdgeX can be better integrated into digital twin systems and help facilitate solutions in the agricultural ecosystem.

More on Ireland

Find more information about the release at these locations.

On to Jakarta

As always, I want to take this opportunity to thank the wonderful EdgeX team and community.  EdgeX exists because of the incredibly smart and dedicated group of people in the community.  I also want to thank all the fine people at the Linux Foundation.  This release was especially long and difficult, but this team was up to the challenge and they are providing you another great release.  I am proud of their work and privileged to be the chairman of this group.

But our work is not done.  The EdgeX community has already held its planning session for our next release – codenamed Jakarta – that will release around November of 2021.  Look for more details on our planning meeting and the Jakarta release in another blog post I’ll have out soon.

Enjoy the summer, stay safe and healthy (let’s get this pandemic behind us), and please give EdgeX 2.0 a try.

EdgeX Foundry Releases the Most Modern, Secure, and Production-Ready Open Source IoT Framework

By Announcement, EdgeX Foundry

Four-plus years of collaboration, 190+ contributors, 8M+ container downloads, new retail project ORRA, EdgeX Ready and foundation for future, long-term support pave the way for Ireland release

SAN FRANCISCO August 3, 2021 EdgeX Foundry, a project under the LF Edge umbrella organization within the Linux Foundation, today announced it’s Ireland release. Focused on edge/IoT solutions, EdgeX Foundry’s second major release overhauls API sets, removes technical debt, provides more message-based communications, and simplifies and secures interface for adopters and developers, making the platform significantly easier to use and more reliable. 

“As a leading stage 3 project under LF Edge, the EdgeX Ireland release has expanded use cases across retail, building automation, smart cities, process control, and manufacturing,” said Arpit Joshipura, general manager, Networking, Edge & IoT, at the Linux Foundation. “It’s a key to standardizing IoT frameworks across market verticals.”

“This release sets in motion the opportunity for EdgeX to offer its first ever LTS or long-term support release as soon as the fall.  This is a significant commitment on the part of our open-source community to all adopters that says we stand with you, prepared to help support your use of EdgeX in real world, scalable, production deployments,” said Jim White, chief technical officer,  IoTech,  and EdgeX Foundry Technical Steering Committee Chair. 

Ireland Feature Highlights

  • Standardized and modernized northbound and southbound APIs enrich ease of interoperability across the IoT framework
  • Advanced security is built into the APIs, message bus, and internal architecture of EdgeX
  • New device services (southbound) and new app services (northbound) included in Ireland are also inherently secure (e.g., GPIO, CoAP, LLRP, UART)

Commercialization & Use Case Highlights

  • Open Retail Reference Architecture (ORRA): a new sub-project that provides a common deployment platform for edge-based  solutions and IoT devices. ORRA is a collaboration with fellow LF Edge projects Open Horizon and Secure Device Onboard, incubated by EdgeX Foundry.
  • The new Edgex Ready program highlights users and organizations that have integrated their offerings with solutions leveraging EdgeX;  a precursor to a community certification program. Learn how to become EdgeX Ready through the project’s Wiki page

Learn more about Ireland’s feature enhancements in this blog post

Plans for the next EdgeX release, codenamed ‘Jakarta’ are expected in Q4’ of 2021. 

For more information about LF Edge and its projects, visit https://www.lfedge.org/

About the Linux Foundation

Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation’s projects are critical to the world’s infrastructure including Linux, Kubernetes, Node.js, and more.  The Linux Foundation’s methodology focuses on leveraging best practices and addressing the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.

The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our trademark usage page: https://www.linuxfoundation.org/trademark-usage. Linux is a registered trademark of Linus Torvalds.

Additional Quotes and Community Support

”Beechwoods Software has been a contributing member of EdgeX Foundry since its inception and chairs the Certification Working Group. EdgeX technology is at the core of our EOS IoT Edge platform offering for which we are readying our version 2 release based on the latest EdgeX code base. Beechwoods is pleased with the growing momentum of EdgeX Foundry and look forward to continuing our support and collaboration,” said Michael Daulerio, Vice President of Marketing and Business Development at Beechwoods Software, Inc.

“Canonical is a founding member of the EdgeX Foundry project and has provided technical leadership in the technical steering committee from day one. The Ireland (aka 2.0) release of EdgeX introduces much improved V2 REST APIs, a transition to a secure message bus for data ingestion, and many additional improvements to the security of EdgeX. The cross-company cooperation that contributed to the success and timeliness of this release once again demonstrates the power of open source development. Snaps of the Ireland release of EdgeX are available from the Snap Store using the new 2.0 track, and can be used to build secure enterprise-grade EdgeX deployments using Ubuntu Core 20,” said Tony Espy, technical architect / IoT & Devices, Canonical, and at-large  EdgeX Foundry TSC member. 

“EdgeX Foundry continues to serve as the basis for our Edge Xpert product.  As such, we see the release of EdgeX 2.0 as critical to our company’s success in support of our customers.  It provides the ability for IOTech to add new features and add more value given the new APIs, support for more messaging and overall simplifications of the platform.  On top of that, the move toward an LTS release in the fall based on EdgeX 2.0 is an important milestone of support shown by the EdgeX community.  LTS tells adopters like IOTech that the EdgeX ecosystem stands behind them and is there to provide a scalable, reliable, and robust platform that can be used in production ready solutions,” said Keith Steele, CEO, IOTech Systems. 

Resources:

 

###

 

EdgeX Foundry Launches Chinese Website

By Blog, EdgeX Foundry

By Melvin Sun (Sr. marketing manager in Intel IoT Group, EdgeX TSC member, co-founder & maintainer of EdgeX China Project) and Kenzie Lan (Operations specialist of EdgeX China community)

EdgeX Foundry, a project under the LF Edge umbrella, is a vendor-neutral open source project on edge computing. 

To support the growing community of EdgeX Chinese developers and ecosystem partners, the project established the EdgeX China project in Q4’19 to drive collaboration in the region.  Since then,  the project community in China has set up local meetups, working groups, and events. In addition, local and language-specific developer and social media channels were created to generate awareness about the project and improve the adoption and experience of Chinese users and developers. 

As of today, the EdgeX China community has gained more than 300,000 followers on mainstream developer platforms of CSDN and OSChina, as well as six WeChat groups! In addition to developers, users and ecosystem partners, there are  ~10 commercial offerings from China based on EdgeX. 

To further support the EdgeX China community growth, the project officially launched its Chinese website (https://cn.edgexfoundry.org/) on June 30th, 2021 to provide awareness of EdgeX Foundry for new developers and users and to keep the local community up to date on events and news. 

The EdgeX Foundry Chinese site is divided into “Home Page”, “Why EdgeX”, “Getting Started”, “Software”, “Ecosystem”, “Community”, “Latest”, and “EdgeX Challenge China 2021”. This site provides a systematic introduction to EdgeX’ s architecture, methods of use, services, community, latest news, and overview and information on the ongoing 2021 China EdgeX Challenge for EdgeX users to compete for prizes by creating solutions based on EdgeX. Visitors can easily jump from the English website (https://www.edgexfoundry.org/)to the Chinese through the link between them.

CTOs, architects, developers, students, and business folks all are welcome to learn and exchange practical experience in the EdgeX China community. The EdgeX community will continue to grow and improve in collaboration with members around the world.

Thanks to all involved in creating and maintaining this China-focused version of the EdgeX Foundry website!

AgStack joins with LF Edge and EdgeX Foundry to Create Agriculture IoT Solutions

By Blog, EdgeX Foundry

This post originally published on the AgStack website

The newest Linux Foundation project, AgStack, announces a liaison with LF Edge and EdgeX Foundry to dock existing EdgeX software for IoT into AgStack’s IoT Extension. AgStack has also agreed to participate in the EdgeX Foundry Vertical Solutions Working Group to explore the adoption and use of the EdgeX IoT/edge platform as an extension to its design of digital infrastructure for agriculture stakeholders.  Through mutual LF project cooperation, the two projects hope to create a complete edge-to-enterprise open-source solution as part of AgStack that can help the global agricultural ecosystem.

The goals of the cooperation between the two projects are to:

  • Utilize the existing EdgeX Foundry framework to quickly accelerate AgStack’s reach into the agriculture edge – providing a universal platform for communicating with the ag industry’s sensors, devices and gateways.
  • Extend the EdgeX framework to handle agricultural edge use cases and unique ag ecosystem protocols, models, data formats, etc.
  • Jointly work on edge-to-enterprise market ready solutions that can be easily demonstrated and used as the foundation for real world products benefiting ag industry creators and consumers.
  • Setup an exchange (as fellow LF projects) to mutually assist and share lessons learned in areas such as project governance, devops, software testing, security, etc.

“We are in the early stages of defining and building the AgStack platform and we prefer not to have to start from scratch or reinvent the wheel as we build our industry leading open-source platform” said Sumer Johal, Executive Director of AgStack.  “EdgeX Foundry gives us the opportunity to leapfrog our IoT / edge efforts by several years and take advantage of the ecosystem, edge expertise, and lessons learned that EdgeX has acquired in the IoT space.”

“As a versatile and horizontal IoT/edge platform, we are excited to partner with AgStack who can help to highlight how EdgeX can be used in  agriculture IoT use cases and how the AgStack and EdgeX communities can collaborate to scale digital transformation for the agriculture and food industries” said Jim White, Technical Steering Committee Chairman of EdgeX Foundry.  “Even though a fellow LF project, we view AgStack as one of our vertical customers – applying EdgeX to solve real world problems – and what better place to demonstrate that than in an industry that feeds the world.”

 About AgStack

  AgStack was launched by the Linux Foundation in May 2021.  AgStack seeks to improve global agriculture efficiency through the creation, maintenance and enhancement of free, re-usable, open and specialized digital infrastructure for data and applications.  Founding members of AgStack include Hewlett Packard Enterprise, Our Sci, bscompany, axilab, Digital Green, Farm Foundation, Open Team, NIAB and Product Marketing Association.  To learn more visit https://agstack.org.

 

EdgeX Foundry Reaches Four Years!

By Blog, EdgeX Foundry

In four short years, EdgeX has become the global standard at the IOT Edge; this blog examines why and looks to the future.

By Keith Steele, CEO IOTech, and member of LF Edge Board and EdgeX Technical Steering Group.

The emergence of Edge Computing

Back in 2017, “edge computing” was a forecast in a Gartner report, it was called “the mother of all infrastructures.” A new computing model, edge computing promised to alter how businesses interact with the physical world.

With edge computing, data from physical devices—whether it be a drone, sensor, robot, HVAC unit, autonomous car, or other intelligent device—is acquired, processed, analyzed, and acted upon by dedicated edge computing platforms. The processed data can be acted upon locally and then sent to the cloud as required for further action and analysis.

Edge computing helps businesses very rapidly and inexpensively store, process, and analyze portions of their data closer to where it is needed, reducing security risks and reaction times, and making it an important complement to cloud computing. It is, however, a complex problem.

As more devices generate more data, existing infrastructure, bandwidth restrictions, and organizational roadblocks can stand in the way of extracting beneficial insights. What is more, there is no one-size solution that fits everyone. Different applications require different types of compute and connectivity and need to meet a variety of compliance and technology standards.

EdgeX – A Strategic Imperative

This inherent complexity at the edge was recognized as a major barrier to market take- up at the edge; in the same way that common standards and platforms are applied across most of the IT stack, there was recognition that a common ‘horizontal’ software foundation at the edge was needed.

In June 2017, in Boston MA, under the auspices of the Linux Foundation, around 60 people from many technology companies, gathered from around the World, to constitute the EdgeX Foundry open-source project;  the attendees had one aim: To create EdgeX Foundry as the global open edge software platform standard!

.       

At the outset, the EdgeX team saw an open edge software platform as a strategic imperative.  EdgeX enables successful edge implementation strategies and makes the IT/OT boundary a key value-add in building new end-to-end IoT systems. Platforms like EdgeX support heterogeneous systems and ‘real time’ performance requirements on both sides of the boundary, promote choice and flexibility, and enable collaboration across multiple vendors.

Four years later with literally millions of downloads, thousands of deployments across multiple vertical markets, and a truly global ecosystem of technology and commercialization partners, we can   justifiably claim to have achieved our goal.

Over the years the project has had something like 200 code contributors, from companies as diverse as Dell, Intel, HP, IOTech, VMWare, Samsung, Beechwoods, Canonical, Cavium Networks, and Jiangxing Intelligence.  Some made small contributions, while others have stayed for the whole journey. This blog is a tribute to all who have contributed to the project’s continued success.

The Importance of Ecosystem

Open-source projects without a global ecosystem of partners to develop, productize, test, and even deploy stand little chance of success.

The EdgeX ecosystem was greatly enhanced in January 2019 when EdgeX, along with project Akraino, became one of two founding projects in LF Edge, which is an umbrella organization created by the Linux Foundation that “aims to establish an open, interoperable framework for edge computing independent of hardware, silicon, cloud, or operating system.” This project aims to bring together complementary open-source technologies.

EdgeX standalone pre- 2019 was already of great value. However, as a founding project under the umbrella of LF Edge, it proved even more valuable as the momentum increased with additional global collaboration. The additional amplification and support across LF Edge projects, community, and members, helped turn EdgeX into a real high velocity project.

     

\EdgeX Fundamentals

There are several fundamental technical and business tenets we set out to solve with EdgeX:

Leveraging the power of Edge and Cloud Computing

Our starting point is that edge and cloud complement one another.

Cloud computing and data centers offer excellent support for large data storage and computing workloads that do not demand real-time insights. For example, a company may choose cloud computing to offload long-term data processing and analysis, or resource-intensive workloads like AI training.

However, the latency and potential costs of connections between the cloud and edge devices makes cloud computing impractical for some scenarios – particularly those in which enterprises need faster or real-time insights, or the edge produces massive amounts of data that should first be filtered to reduce cloud costs. In these cases, an edge computing strategy offers unique value.

Open vs. Proprietary

The idea behind EdgeX was to maximize choice so users did not have to lock themselves into proprietary technologies that, by design, limit choice.

Given the implicit heterogeneity at the Edge, ‘open’ at a minimum means the Edge X platform had to be silicon, hardware, operating system, analytics engine, software application and cloud agnostic. It would seem odd to embrace IoT diversity at the edge, but then be tied to a single cloud vendor, hardware supplier, application vendor or chip supplier.

Secure, Pluggable and Extensible Software Architecture

To offer choice and flexibility we chose a modern, distributed, microservices based software architecture, which we believe supported the inherent complexities at the edge. The other really big thing we defined was a set of open standard APIs that enable ‘plug and play’ of any software application at the Edge. Coming back to ‘product quality’ we also wanted these maintained in a way that any changes to the APIs did not mean huge rewrites for application providers.

Edge Software Application ‘Plug and Play’

A key promise of EdgeX is that it provides a standard open framework around which an ecosystem can emerge, offering interoperable plug-and-play software applications and value add services providing users with real choice, rather than having to deal with siloed applications, which may potentially require huge systems integration efforts.

EdgeX ensures that any application can be deployed and run on any hardware or operating system, and with any combination of application environments. This enables real flexibility to deliver interoperability between connected devices, applications, and services across a wide range of use cases.

Time-Critical Performance and Scalability

Many of the applications we want to run at the edge, including specialist AI and analytics applications, need access to ‘real-time’ data. These can be very challenging performance constraints, e.g., millisecond or even microsecond response times, often with absolute real-time predictability requirements. Edge systems can also be very large-scale and highly distributed.

The hardware available to run time-critical Edge applications is often highly constrained in terms of memory availability or the need to run at low power. This means edge computing software may need to be highly optimized and have a very small ‘footprint’.

Access to real time data is a fundamental differentiator between the edge and cloud computing worlds. With EdgeX we decided to focus on applications that required round trip response times in the milliseconds rather than microseconds.

Our target operating environments are server and gateway class computers running standard Windows or Linux operating systems. We decided to leave it to the ecosystem to address Time Critical Edge systems, which required ultra-low footprint, microsecond performance and even hard real time predictability. (My company – IOTech- just filled that gap with a product called Edge XRT. Its important real-time requirements are understood in full, as decisions taken can significantly impact success or failure of edge projects.)

Connectivity and Interoperability

A major difference between the edge and cloud is inherent heterogeneity and complexity at the edge. This is best illustrated in relation to connectivity and interoperability requirements, south and northbound:

  • Southbound: The edge is where the IT computer meets the OT ‘thing’ and there is a multitude of ‘things’ with which we will want to communicate, using a range of different ‘connectivity’ protocols at or close to real time. Many of these ‘things’ are legacy devices deployed with some old systems (brownfield). EdgeX provides reference implementations of some key protocols north and southbound along with SDK’s to readily allow users to add new protocols where they do not already exist; ensuring acquired data is interoperable despite the differences in device protocols. The commercial ecosystem also provides many additional connectors, making connectivity a configuration versus a programming task
  • Northbound: Across industry, we also have multiple cloud and other IT endpoints; therefore, EdgeX provides flexible connectivity to and from these different environments. In fact, many organizations today use multi-cloud approaches to manage risk, take advantage of technology advances, avoid obsolescence, obtain leverage over cloud price increases, and support organizational and supply-chain integration. EdgeX software provides for this choice by being cloud agnostic.

How does the EdgeX Vendor Ecosystem deliver customer solutions?

There are many companies offering value add products and services to the baseline open-source product, including mine, IOTech. There are also may examples of live deployments in vertical markets such as manufacturing and process automation, retail, transportation, smart venues, and cities etc.  See the EdgeX Adopter Series presentations for some examples.

Where Next for EdgeX?

The EdgeX project goes from strength to strength, with huge momentum behind its V1 Release and we will soon release EdgeX 2.0, a major release which includes all new and improved API set (eliminating technical debt that has incurred over 4 years), more message bus communications between services (replacing REST communications where you need more quality of service and asynchronous behavior), enhanced security services, and new device/sensor connectors.  The EdgeX 2.0 release will also emphasize outreach, including much more of a focus on users as well as developers.  With this release, the community launches the EdgeX Ready program.  The program is a means for organizations and individuals to demonstrate their ability to work with EdgeX.

Some Closing Thoughts

The full promise of IoT will be achieved when you combine the power of cloud and edge computing: delivering real value that allows businesses to analyze and act on their data with incredible agility and precision, giving them a critical advantage against their competitors.

The key challenges at the edge related to latency, network bandwidth, reliability, security, and OT heterogeneity cannot be addressed in cloud-only models – the edge needs its own answers.

EdgeX and the LF Edge ecosystem maximize user choice and flexibility and enable effective collaboration across multiple vertical markets at the edge helping to power the next wave of business transformation. Avoid the risk of getting left behind. To learn more, please visit the EdgeX website and LF Edge website and get involved!