September 2019

Snaps are self-contained application packages designed to run on any system that supports them. Each snap is a compressed SquashFS package (bearing a .snap extension), containing all the assets required by an application to run independently, including binaries, libraries, icons, etc.

Snaps are managed by snap, the command-line userspace utility of the snapd service. With the snap command, users can query the Snap Store, install and refresh (update) snaps, remove snaps, and perform many other tasks such as managing application snapshots, setting command aliases, enabling or disabling services, etc.

The EdgeX Foundry project publishes a  snap named edgexfoundry, which includes all of the EdgeX core, export, security and support services, Mongo/Redis, and all of the 3rd party runtime dependencies (Consul, Vault, Kong, …). Hackathon participants should use the version from the latest/stable channel in the Snap Store, which is the 1.0.1 release (Edinburgh). See the snap’s README for details. To install from latest/stable run:

sudo snap install edgexfoundry

After installing the edgexfoundry snap view the services by issuing the snap services command.

sudo snap services edgexfoundry

Notice that after running this one simple snap install command, all of the services required to runEdgeX will have been installed and are already running in the background on your system. Note which services are enabled and which are active by running the snap services command.

Upon installation, the following EdgeX services are automatically enabled:

• cassandra (persistent storage for Kong)
• consul (aka ‘the registry’)
• core-command
• core-config-seed (one-shot)
• core-data
• core-metadata
• edgexproxy (one-shot)
• kong-daemon
• mongod
• mongo-worker (one-shot)
• pkisetup (one-shot)
• sys-mgmt-agent
• vault
• vault-worker (one-shot)

Note – some of the above services are “one-shot” services which run once and then exit. These will show up as “enabled” but “inactive”.

The following services are disabled by default:

• support-notifications
• support-logging
• support-scheduler
• export-client
• export-distro
• device-virtual
• device-random

Any disabled service can be enabled and started using snap set. For example:

sudo snap set edgexfoundry support-notifications=on

To turn a service off (thereby disabling and immediately stopping it) set the service to off:

sudo snap set edgexfoundry support-notifications=off

On install, all services in a snap have corresponding systemd service units dynamically generated by snapd, thus if enabled, each will automatically start running when the system boots.

To keep things simple for EdgeX Open the security services in the edgexfoundry snap should be disabled by running:

sudo snap set edgexfoundry security-services=off

Currently, all log files for the snaps can be found inside $SNAP_COMMON, which is /var/snap/edgexfoundry/common. Additionally, logs can be viewed using the systemd journal or snap logs. To view the logs for all services in the edgexfoundry snap use:

sudo snap logs edgexfoundry

Individual service logs may be viewed by directly specifying the service name:

sudo snap logs edgexfoundry.consul

Once the EdgeX snap has been installed and the desired services have been successfully started, we are ready to set up an application service and a device service.

As a quick test the Mosquitto snap can be installed and a test topic can be published using the mosquitto_pub command. Use the following command to install Mosquitto:

sudo snap install mosquitto

Edgex-device-mqtt is a device service which supports importing device/sensor data readings via the MQTT protocol. It requires a running MQTT broker, as well as a peer MQTT application. The device service listens on a specific topic for incoming readings which it injects into EdgeX via the device service SDKs’ asynchronous readings feature. A peer MQTT application will be provided for the hackathon which will generate a steady stream of incoming readings related to the hackathon retail scenarios. This peer application will be setup to publish multiple scenario-specific streams on different topics. The device-mqtt service configuration provides a configuration value called “IncomingTopic” which can be used to change the topic the service listens on for incoming readings. Use the following command to install edgex-device-mqtt from the stable channel:

sudo snap install edgex-device-mqtt

After installation, check the status of the edgex-device-mqtt device service:

sudo snap services edgex-device-mqtt

Notice that the edgex-device-mqtt device service is disabled and inactive. This was done intentionally as the configuration and device profile need to be specific to each installation. You’ll find the configuration and device profile in the writable area of the snap (aka $SNAP_DATA):

/var/snap/edgex-device-mqtt/current/config/device-mqtt/res

After making the necessary modifications to configuration.toml (e.g. setting the “IncomingTopic” or changing “Port”) and mqtt.test.device.profile.yml (e.g. modifying device-resources, device-commmands, or core-commands), start and enable edgex-device-mqtt like this:

sudo snap start --enable edgex-device-mqtt

You should now be ready to receive peer MQTT application readings.

Now let’s publish a topic with a randfloat32 value of 1.2:

mosquitto_pub -t DataTopic -m '{"name":"MQTT test device","cmd":"randfloat32","randfloat32":"1.2"}'

You can check the readings from core-data with the following command:

curl -s localhost:48080/api/v1/reading | jq

Note – jq is a lightweight and flexible command-line JSON processor. In its simplest form with no command-line arguments specified, it simply outputs JSON its been given in a human-readable format. If not available on your system, it can be installed using sudo snap install jq.

If you see something similar to the following output then edgex-device-mqtt is working correctly. Float values are encoded base64 by default. The value “P5mZmg==” is 1.2 in base64.

[
   {
     "id": "4f922fd7-5c0d-48e0-8a06-6218d4c05fa9",
     "created": 1568926981304,
     "origin": 1568926981290,
     "modified": 1568926981304,
     "device": "MQTT test device",
     "name": "randfloat32",
     "value": "P5mZmg=="
   }
 ]

Once the device service has been started the configuration information is now in Consul. If you need to make further modifications to either the configuration or device profile, you’ll need to make the modifications in Consul which can be accessed from a browser at http://localhost:8500. After making modifications in Consul, the device service needs to be restarted like this:

sudo snap restart edgex-device-mqtt

If you would like to manage your EdgeX instance using a web browser, adding devices, device profiles, as well as visualizing data sent through the export service, you can do this through the EdgeX Web management UI. To install the edgex-ui-go snap run the following:

sudo snap install edgex-ui-go --channel=latest/beta

Now navigate with your browser to http://localhost:4000. The default user credentials are username: admin password:admin. Add your gateway and you’re ready to manage your EdgeX instance through the web UI.

Extending EdgeX

This section gives some hints as to how you can build your hackathon entry by creating or using additional EdgeX services.

App-Service-Configurable

Edgex-App-Service-Configurable – this is a new snap built from the current development release of EdgeX (Fuji). It allows a variety of use cases to be met by simply providing configuration (vs. writing code). For more information about this service, please refer to the README. As with device-mqtt, this service is disabled when first installed, as it requires configuration changes before it can be run. As with the device-mqtt snap, the configuration.toml file is found in the snap’s writable area:

/var/snap/edgex-app-service-configurable/current/config/res/

Profiles

In additional to base configuration.toml in this directory, there are a number of sub-directories that also contain configuration.toml files. These sub-directories are referred to as profiles. The service’s default behavior is to use the configuration.toml file from the /res directory. If you want to use one of the profiles, use the snap set command to instruct the service to read its configuration from one of these sub-directories. For example, to use the push-to-core profile you would run:

$ sudo snap set edgex-app-service-configurable profile=push-to-core

In addition to instructing the service to read a different configuration file, the profile will also be used to name the service when it registers itself to the system.

Note, as this service is based on the latest development release of EdgeX, not all use cases are supported, in particular integration with the EdgeX rules-engine will not work when used in conjunction with the Edinburgh release of EdgeX. Perform the following steps to install the edgex-app-service-configurable application service using the mqtt-export-configuration example and Mosquitto to test:

sudo snap install edgex-app-service-configurable

sudo snap set edgex-app-service-configurable profile=mqtt-export

sudo snap start --enable edgex-app-service-configurable.app-service-configurable

mosquitto_sub -t "edgex-events"

Multiple Instances

Multiple instances of edgex-app-service-configurable can be installed by using snap Parallel Installs. This is an experimental snap feature and must be first be enabled by running this command:

sudo snap set system experimental.parallel-instances=true

Now you can install multiple instances of the edgex-app-service-configurable snap by specifying a unique instance name when you install the snap. The instance name is made of the name of the snap plus a unique suffix which starts with the character “_”. This name only needs to be specified for the second and susbequent instances of the snap.

sudo snap install edgex-app-service-configurable edgex-app-service-configurable_http

or

sudo snap install edgex-app-service-configurable edgex-app-service-configurable_mqtt

Note – you must ensure that any configuration values that might cause conflict between the multiple instances (e.g. port, log file path, …) must be modified before enabling the snap’s service.

New Application or Device Services

Participants can also choose to construct their entry by building new device services (using the device SDKs) or new application services (using the application SDK). As the hackathon is time-boxed, most participants may just build new services natively vs. choosing to create Docker containers and/or snaps. For those interested in creating snaps for their new services, the following services can be useful as templates:

• edgex-device-mqtt (snapcraft.yaml)
• edgex-device-modbus (snapcraft.yaml)
• edgex-app-service-configurable (snapcraft.yaml)

What is EdgeX Foundry?

The EdgeX Foundry is a Linux Foundation project with the goal of providing an open and extensible platform for IoT edge computing. As a platform, EdgeX doesn’t try to solve business problems directly, but rather to provide a solid base for such a solution to be built using off the shelf components, provided by the EdgeX Foundry project or 3rd parties, which all interoperate using the same open APIs.

EdgeX is built as a collection of microservices which communicate with each other over a set of standard, open REST APIs. This design gives it a few  distinct advantages over other approaches to edge computing. First, it allows you to run the microservices on different machines, or even on different parts of the network. For example, you can run device controlling services on the devices themselves, with the data collection and processing services on a computer with more storage and CPU capability. This design also means that individual components can be substituted with alternative implementations that fill a specific need, without having to replace or modify any of the other services. Finally, it allows you to add new services that bring in new capabilities. The most common types of services to add into EdgeX are custom Device and Application services.

How EdgeX fits into a smart edge solutions

Every IoT project needs to address the difficulties of connecting multiple, disparate devices together and to controlling software, often over slow or unreliable connections. As such, managing that communication, providing fault tolerance, security, and near-device compute capabilities are a functions any successful IoT project must provide. This common plumbing is precisely what EdgeX Foundry provides.

By using EdgeX components in your solution, you get a mature, tested, stable foundation on which you can build your own value-add products or services.

• If you are building smart IoT devices, you can use EdgeX to connect to existing cloud offerings or datacenter software without having to build those connections into your device.
• If you are developing analytics, automation or other kinds of business intelligence software, EdgeX lets you ingest data from devices that speak any number of protocols and use many different data models, without having to build those translations and transformations yourself.
• If you provide custom deployments, EdgeX gives you an ecosystem of plug-and-play components that you can use to build up complex solutions quickly and reliably.

Running EdgeX in a development environment

You can build the EdgeX Foundry services using the open source code on Github, but more often than now you just need to get these services running so that you can connect your own services to them. To support that, the project publishes Docker images based on the latest stable release of the open source code, as well as docker-compose.yml files that will run all the necessary services together on your development machine.

1. Install Docker and Docker Compose
2. Download the latest stable docker-compose.yml file
3. Run `docker-compose up -d`
4. Verify that the services are all working with `docker-compose ps`
5. Test the APIs with http://localhost:48080/api/v1/ping

Once you have the EdgeX services running, you can take our Quick Start guide, or more in-depth API Walkthrough to get a feel for how EdgeX services work together.

Extending EdgeX to hardware with Device Services

In order to connect new devices to your EdgeX instance you will need to use a Device Service. These specialized services are responsible for communicating to devices over their own protocol, sending data from them into the EdgeX core-data service, and taking commands from the EdgeX core-command service and using them to control the device.

The EdgeX Foundry provides reference device services for a number of common IoT and smart device protocols, including MQTT, Modbus and plain HTTP, and you can use those pre-written services if your device supports those protocols. But if you need something more, the EdgeX Foundry Device SDKs make it easy for you to build a new device service capable of supporting your device.

Using either the Go or C SDKs, you can bootstrap a new Device Service that will already handle all of the setup and configuration for your new service, including uploading your Device Profiles into EdgeX core-metadata, provisioning any devices you have pre-defined, and scheduling automatic readings from your devices. The SDK also provides functions for calling all the common EdgeX APIs in a language-friendly way. All you have to do is add your device-specific connection and control code.

Training Tip: You can do end-to-end testing of your new Device Service by connecting your local EdgeX deployment up to a cloud service or NodeRed instance, and watch the readings flow in and commands go out.

Extending EdgeX to the cloud with Application Services

Since EdgeX Foundry itself doesn’t try to do anything other than efficiently moving data off the edge, you’re eventually going to want to send that data to some software for further processing. To do that, EdgeX provides a couple of ways to publish data to software running either on the edge, in a data center, or the cloud.

The first method is to use the pre-built Export Services, which are capable of sending data as JSON or XML over simple HTTP or MQTT to another service. This is a good option if you are using a service that already knows how to handle these common protocols, and can do any data transformations you need.

But if those aren’t suitable, the EdgeX Foundry also provides an Application Functions SDK for writing custom handlers, which can do anything from publishing data to a remote host, to performing local transformations, analytics and responses right from the edge. The SDK allows you to trigger you application functions whenever data comes into EdgeX, or on-demand, and even provides functions for publishing your data (via HTTP or MQTT) when you’re done handling it locally. Application Functions can even be chained together to build complex pipelines, reusing the same Application Functions in different ways.

Training Tip: You can do end-to-end testing of your new Application Service by connecting the Virtual Device Service to your local EdgeX deployment, which can simulate device data coming in.