Architecting for the Internet of Things creates a whole new ballgame for network pros.
Gartner predicts there will be more than 20 billion IoT devices in the world by 2020. About 40% of them will be deployed in the enterprise – monitoring air quality in factories, improving energy efficiency in offices, tracking assets across the supply chain, detecting equipment failures, and much more.
Read more at Network World.
Jim White, Dell Technologies IoT Platform Development Team Lead & EdgeX TSC Vice Chair
This week, EdgeX Foundry announces the release of Delhi, our third major release in 12 months. With each release that this growing community kicks out – I get a little nostalgic. EdgeX started its life in my kitchen. Like a lot of open source software efforts – it started humbly. In the case of EdgeX, it started on my laptop, propped open on my kitchen island one summer weekend about 4 years ago with an adult beverage or two.
Today, Edgex is being developed around the globe by a lot of tremendous software engineering talent. I was proud to start EdgeX, but I am even more proud to be part of an exciting group of international engineers building the most flexible, interoperable, open source IoT platform on the planet. This new release, Delhi, contains a new service and several key features that have our community and potential customers excited about the future of EdgeX Foundry.
Most importantly, this release contains the first EdgeX system management capability. A new service – the System Management Agent (SMA) – serves as the coordinator for control plane information (status, configuration and metrics around EdgeX services) and control actions on EdgeX services (start, stop, and restart). Cloud or third party systems can call on the API provided by the SMA to trigger the actions or to get the control plane data they need. The SMA can serve as a one-stop shop for managing an instance of EdgeX. Each EdgeX micro service has a corresponding management API that the SMA calls on to help control that service (example – stop the service) or pull back its latest configuration or metrics. The SMA and management API provided by each service will be expanded in future releases of EdgeX and will one day offer control plane data and actions via alternate protocols (like LWM2M or SNMP).
The original platform that started on my kitchen island was a Java platform. Last year, the community embarked on an endeavor to replace the bulky and slow Java services and tools with Go and C services and tools. Device Services – the EdgeX services that connect “things” to the platform are still in Java but not for long. With the Delhi release, two new Device Service SDKs have been created. The Go and C SDKs are allowing the community to create smaller, lighter, faster device/sensor connecting services that will allow EdgeX to operate in very limited resource compute environments – the thin edge in IoT solutions.
In the past, the EdgeX microservice dependencies required the services be brought up with wait times between the startup of each service to allow the system time to bring things up in order. The services now are able to automatically check and detect when a dependent is in place and to become fully functional only after a dependent is up. This greatly reduces the start of all of EdgeX from minutes to seconds – around 5-10 seconds on average. The services now detect a dependent has gone down and will keep retrying that service until it comes back up. This gives EdgeX more resiliency than it had in the past.
EdgeX was built to facilitate machine to machine communications. As such, it did not have a user interface. With the Delhi release, user interfaces were created – largely to help visually showcase EdgeX functionality and to help developers – these UI also serve to show how EdgeX can be driven and operated.
Many of the core and supporting services in EdgeX were improved and more unit testing was added to allow the community to keep an eye on the quality of the overall system and compatibility of future features of EdgeX. The Scheduling Service was also transitioned from Java to Go.
Security services were initiated in our last release (California). The Delhi release includes the next wave of security features such as access control to grant access to appropriate services, and improved security service bootstrapping.
As mentioned, more unit tests were added to the EdgeX services. Additionally, automated blackbox tests are in place for each service to make sure the public API of each service performs as documented.
In addition to all this hard work, our EdgeX Foundry ecosystem continued to expand during this release cycle. Among the new companies to join EdgeX Foundry are Basking Automation, Data Ahead, Intel, Redis Labs, and ZEDEDA. This community also launched the availability of a developer kit and community demonstrator (for showcasing EdgeX during conferences and IoT events). Find out more about these here. Stay tuned to the EdgeX news and announcement pages as some big organizations are preparing to announce their joining early in the New Year.
OK – no time to rest on our laurels. What’s next EdgeX? I am glad you asked! Indeed, the EdgeX technical steering committee (TSC) along with several community members met in Edinburgh Scotland to plan and scope our next release – code named Edinburgh – which is scheduled for April 2019 (I assure you the release name and TSC meeting place alignment happened a little serendipitously).
As Keith Steele, Chair of the EdgeX Foundry Technical Steering Committee, reported in his blog post, the results of the meeting were “superb” – as the Scots like to say.
The Edinburgh release has been scoped to include a rather significant amount of new features as well as making improvements on the current code base (“cleaning up technical debt” as we like to say). Given the increase in community membership and number of contributors to the project, we feel this scope is within reach. Here are some of the major efforts the membership has planned for the next 5-6 months.
The Edinburgh release of EdgeX will support the ingestion, use, and export of binary data (using CBOR format). To date, EdgeX has supported ingesting, using and exporting integer, float, string, and boolean types of discrete data collected from sensors. Many of the edge use cases involve video images, audio data, and other data that is binary data serialized.
EdgeX has come a long way over the past few releases in increasing and improving its quality through testing. In the Edinburgh release, performance and security testing will be automated.
The new Go and C Device Service SDKs created with the Delhi release allow the community to create smaller, faster, less-resource consuming replacements for the Java Device Services that now serve to connect “things” to EdgeX. This includes Modbus, BACNet, BLE, MQTT and SNMP Device Services. As there has been a bit of a pent up demand to get new Device Services for EdgeX due to the development of the SDKs, we believe this release will contain even more thing connectors than we could have imagined a year ago.
The paint on the new SDKs isn’t even dry and we already have the community showing off some early prototypes of new Device Services today and the numbers are impressive. In one small example, our old Java Modbus Device Service was 141MB in size and consumed 184MB of memory. The Go version produced with the new SDK is 16MB in size and uses just 8MB of memory.
The current EdgeX Export Services, while functional, create scalability issues. The current EdgeX capability relies on the Export Services to know and understand all possible endpoint distribution mechanisms and transformation capability necessary to support all clients – even if the use case requires only a single, simple client (like sending the raw sensor event data to an MQTT pipe in JSON). As the number and type of EdgeX north side clients (cloud, enterprise and on-prem IoT servers) expands, this service will be too big and complex to support the north side needs.
The Edinburgh release will feature a new type of “exporting” service – the Application Service. The initial and simple Application Services in the Edinburgh release will be designed and implemented to support smaller, more tailored exportation needs. These services will contain just the functionality (filtering, transformation, enrichment, etc.) needed for a single endpoint. To address multiple endpoints, multiple Application Services will be created (versus having one massive export service as EdgeX has today). In a way, the Application Services will begin to look more like the south side Device Services – that is functionality dedicated to a particular client protocol and data need. Long term over the course of a number of releases, Application Services will replace EdgeX Export Services as the north side distribution facility.
EdgeX Foundry has been a developer driven effort. As the platform is and will continue to be used in real world IoT solutions, it becomes imperative that the community improve the quality and volume of documentation, tutorials, examples, videos, etc. to help user communities go from day 0 to production as quickly as possible. It is also imperative that EdgeX institute a long term support plan/strategy to give the user community assurances about what they can expect from the open source effort they put into their products. The TSC Chair (Keith Steele) and TSC Vice-Chair (me) are dedicating our resolve to address these two concerns as part of the Edinburgh release cycle.
A certification program will be outlined in concert with the Edinburgh release. A certification program will enable third parties creating EdgeX services to verify their services as alternative or enhancing capability to those provided by the EdgeX open source effort. This will allow 3rd parties to add value (proprietary or open source) to EdgeX that customers can rely on to meet the EdgeX APIs and work without additional code change (enabling a plug-and-play ecosystem). Various levels of certification are being considered, from micro service replacement certification (validating alternate or commercial implementations of EdgeX micro services satisfy API requirements along with performance metrics and quality checks) to full EdgeX deployments (for commercial versions of EdgeX). Additional certification processes may be developed around particular cross cutting features such as security.
For a full list of the Edinburgh release scope and roadmap, see the project Wiki.
It’s a large scope, but the community and team of developers is growing. Contact our Developer Advocate, Michael Hall, if you and your team would like to be a part of our effort – it’s an exciting time to be part of an exciting and industry impacting project. I have every confidence the EdgeX Edinburgh release will make some news in April 2019… as long as I can keep the team away from all the Scottish whisky we brought home from our trip!
By Keith Steele, Chair of the EdgeX Foundry TSC
I wrote this blog on a train from Edinburgh back to my home town in Newcastle UK, ending two weeks of spirited activity for the EdgeX community. On October 15-17, we gathered in Barcelona for IOT World Solutions Congress and then we met in Edinburgh for our bi-annual Technical Steering Committee (TSC) meeting.
Both weeks were by hugely successful for the project with some major milestones achieved, market momentum demonstrated and collaboration that solidified plans for our impressive roadmap, which agreed for the next EdgeX release in April.
Most striking of all, however is the growing ecosystem of contributors driving the project forward, their talent, enthusiasm and unselfish collaboration. It was just a pleasure to witness and a lot of fun – I’ll never quite forget the gradual rise in volume in the room as we progressed through the Scotch Whiskey tasting with the TSC!
Anyway, I digress, let’s start this blog in Barcelona, or for some of us Toulouse! The IOTech team had a slight travel diversion due to storms and ended up in nice Southwestern French city, but didn’t get to explore. Undeterred, the team developed an alternate travel route and finally showed up in Barcelona to see the EdgeX Foundry booth ready and looking sharp with the new community demo as a nice focal point.
The New Community Demo
EdgeX Foundry’s new community demo set out to demonstrate the full capability of EdgeX in the context of a realistic use case, we chose Buildings Automation but it could have been any edge vertical. When we say the full capability of EdgeX we mean:
If you weren’t at the show, you’ll be able to see a video of it soon. In the meantime, James Butcher, Senior Solutions Architect at IOTech, shares the details of how the demo came together in this blog post.
So, how was it all received?
I say without hesitation this was a great show for EdgeX (and I say this as someone who usually regards trade shows as only marginally more useful than a Celine Dion tribute act).
The EdgeX Foundry stand was continuously busy – attendees came to see the community demo and meet with EdgeX members Basking Automation, CloudPlugs, Dell, Enigmedia, IOTech, and Mainflux as well as Redis Labs, RSA, VMware and ZEDEDA with their own presence demonstrating their integration with EdgeX.
At the show, we announced Intel and eight other tech influencers joined the project and there was huge interest for this and the new EdgeX dev kits at the show. Jason Shepherd, Chair of the EdgeX Foundry Governing Board, and I were busy briefing customers, analysts and media, while traffic to the booth was heavy and constant. James, one of the people manning the community demonstrator only managed lunch at 4pm the 1st day of the show!
My own takeaways
Barcelona, for me, proved without a shadow of doubt an open approach at the Edge is now a well-established need and EdgeX is regarded as the leading open implementation.
The vendor neutral approach is the clear driver, the Edge is by its nature a very heterogeneous and users want a solution that is designed to support that heterogeneity rather than a proprietary solution that locks them in to a specific vendor.
The flexibility of choice of cloud and hardware providers were big winners with visitors to the EdgeX booth. Some of the bigger users are hedging their bets with dual approaches, but there was evidence from the many vendors visiting the booth that they recognized they were going to have to support EdgeX.
Another big motivator was an open approach as a basis for bringing together the 6-10 vendors/technology partners typically required to deliver an end to end IOT solution using EdgeX as an integration point allowing vendors to keep their proprietary value add while collaborating across a standard and open infrastructure. Interesting times!
On to Edinburgh for the EdgeX Technical Steering Committee meeting.
Good to see the numbers growing with 40-50 people attending across the 3 days of the meeting, again from all corners of the globe. Nice to also to see many new faces from Intel, Siemens, Thales, Analog Devices, Canonical, Dell, ForgeRock, Intel, IOTech, Mainflux, Redis Labs, Samsung Electronics, Siemens AG, Technische Universität Berlin, University of Edinburgh, VMware, and Wanxiang Group.
The main meeting discussion topics were the roadmap for the next release, called ‘Edinburgh’ (scheduled for release in April 2019) and the status of the Delhi release, which we just announced the code freeze with release in mid-November.
The Delhi release is a very important milestone for EdgeX as the core services have now all been refactored in Go and C providing a big improvement in performance, footprint and scalability.
A dot release to Delhi is going to be released in December to provide Redis as the underlying database option (in place of MongoDB) for several services including Core Data, Core Metadata and Export Client. This work will be even more of a focus for Edinburgh where the easy replacement and swap of databases that support EdgeX is desired.
In parallel to the formal Delhi release of EdgeX, several community developer kits will also be made available, the first based on the Samsung Pi and Artik developer boards (link). The dev kits will go a long way to onboarding future EdgeX developers and improving developer onboarding will be a key priority for the TSC this year.
Here’s a short list of what was scoped for the Edinburgh release:
Look for a future post by Jim White, our EdgeX Foundry Vice Chair of the TSC, to provide more details on the upcoming April 2019 release.
A few other points of notes from the meeting
We agreed it was time to have a release manager to oversee the bi-annual EdgeX releases and this task would be undertaken by the major contributors. Ideally, we’d rotate ownership of this task and it would be a good way forward with potentially 4-member companies taking it in turns by rotation. A call will be set up shortly to discuss this process and move forward with it.
The DevOps and QA/Test Working Groups will, for the next few months, run combined calls given the close cooperation needed and the focus in the EdgeX Edinburgh release on incorporating automated performance and scalability tests as part of the overnight runs. Look out for the new time for these meetings shortly.
Now the C and the GO SDK’s are available a big part of the Device and DeviceSDK Working Group’s work will now switch to acceleration and collaboration around accelerating new Device Service connectivity. This is obviously seen as key to widespread deployment of EdgeX going forward.
As mentioned, we had to have a little bit of fun while in Edinburgh. There’s nothing quite like Scotch Whiskey tasting in Edinburgh.
Remember, if you can’t make the meetings you can join by phone and the recordings of the meeting can be found here.
The next F2F TSC meeting will be hosted in Seoul, South Korea on April 22 -25.2019. You can register here and find more information is available on the EdgeX Wiki.
It was also announced the October 2019 meeting will take place at Intel’s facility in Chandler Arizona. We look forward to collaborating with you there!!
Finally, a couple of thank you notes, the first one to Jeremy Phelps, who is stepping down as chair from the DevOps Working Group, Jeremy’s enthusiasm, skill and effort levels have been just stellar and he will be a hard act to follow!
Lastly, thanks to the team at the Linux Foundation for their massive efforts in bringing both Barcelona and Edinburgh to fruition with professionalism and untold patience having to deal with me. Maybe next time, we all get an Octopus T-shirt!
Best Regards,
Keith Steele, TSC Chair
If you have questions or comments, visit the EdgeX Rocket.Chat and share your thoughts in the #community channel.
As IoT devices proliferate, cloud just isn’t fast enough. But that’s exactly where edge computing comes in, enabling analysis at speeds the future demands.
Read more at TechTarget.
From MQTT and HMIs to digital twins, IoT app connectivity and the EdgeX Foundry, the concept of decoupling software from hardware is quickly moving to the forefront of automation technology development.
Read more at Automation World.
