Blog

By LF Edge Community members Utpal Mangla (Industry EDGE Cloud – IBM Cloud Platform);  Atul Gupta (Lead Data Architect – IBM); and  Luca Marchi (Industry EDGE and Distribute Cloud – IBM Cloud Platform)

Diet and Nutrition plays a vital role in healthcare and often it is still left behind by medical practitioners, doctors, nurses, and alternative medical care professionals. Whether patient is recovering from regular sickness, injuries, surgeries, or chronic illness – all of them need special care for diet and nutrition provided to the patients. The need is to provide enough nutrition at pre-determined frequencies for faster recovery and making sure the normalcy returns for the most patients in moderate time and at optimal expense. But the diet and nutrition guidance provided is either very generic or not monitored and adjusted as per patients’ vitals and recovery progression. 

These traditional mechanisms of feeding the health, diet, nutrition tracking data to medical practitioners is one-way mechanism and siloed decision making is based on this disparate data and not tied up for common benefits of the patients.

With the ongoing digital transformation of the healthcare, there is organic growth of edge devices in the hands of patients, healthcare staff such as fit-bit, cellphones, nutrition scales, smart kitchen, and bed-care devices, etc. The questions are – Where do we take it from here? What do we do with this explosion of edge data? Are we creating another set of data-silos?

Edge computing has the solutions for most of these problems and it can provide connectivity to use this data effectively and efficiently. This can converge all the data silos from traditional devices, healthcare facilities, human diet, and nutrition data into common repository such as ‘Data producers at Edge’, which can feed the data into cloud computing environments. The specialized healthcare cloud computing environments are now available, but they lack the data ingestion components. 

The ‘Data producers at Edge’ ingest the healthcare data into cloud computing environments from the Edge devices, healthcare facilities, et al completing the full cycle of data usage. This complete data cycle can now start benefiting patients, their families and as much gain efficiency for healthcare staff and facilities. Hospitals and alternative health professionals can start treating patients remotely using this new paraphernalia of devices, solutions, and inter-connectivity. 

These Data producers have rich data about patients’ daily health, vitals, physical vicinities, living conditions, walk-score, mobility, and assistance provided, etc. The net impact of using this data in combined and constructive way can not only open new horizons for healthcare but can start a new journey with Edge transformation which is yet to be realized. There is a potential of new industry disruptor in this area by leveraging benefits of Edge, Cloud computing, IoT devices, Patient Literacy, et al. The combination is self-serviced components and interconnectivity which can expand and scale on demand, as needed and provide solutions to areas of high demand and growth. It is now known that many healthcare areas can only be scaled and gain service efficiency by technology solutions and not by putting more healthcare facilities and professionals. Also, the traditional diet and nutrition methods have always played effective role in patient recovery, which can yield accurate, efficient, and robust healthcare solutions if used in conjunction with Edge and IoT advancements.

This full cyclic approach will tie together the Diet and Nutrition aspects to other healthcare areas, components, and capabilities. The basics of patient’s recovery progression can ultimately start to play its role in more connected manner and predictive care can replace the reactive care approach. 

The Edge data can be used to generate patient analytics and align it to diet and nutrition data to come up with new predictive care capabilities. The health care staff can adjust, re-calibrate the exercises based on existing and proposed diet and nutrition for the patients. This can even extend into improving the supply-chain of the products and services needed for the healthcare industry.

The diet and nutrition data, combined with sophisticated cloud compute can yield better and improved predictive care for the patients and move to new era in Healthcare Digital Edge Transformation. 

By Utpal Mangla (IBM), Luca Marchi (IBM), Shikhar Kwatra (AWS)

Smart Edge devices across the world are continuously interacting and exchanging information through machine-to-machine communication protocols. Multitude of similar sensors and endpoints are generating tons of data that needs to be analyzed in real-time used to train deep complex learning models. From Autonomous Vehicles running Computer vision algorithms to smart devices like Alexa running Natural language processing models, the era of deep learning has widely extended to plethora of applications. 

Using Deep learning, one can enable such edge devices to aggregate and interpret unstructured pieces of information (audio, video, textual) and take ameliorative actions, although it comes at the expense of amplified power and performance measurements. Since Deep learning resources require huge computation time and resources to process streams of information generated by such edge devices and sensors, such information can scale very quickly. Hence, bandwidth and network requirements need to be tackled tactfully to ensure smooth scalability of such deep learning models. 

If the Machine and deep learning models are not deployed directly at the edge devices, these smart devices will have to offload the intelligence to the cloud. In the absence of a good network connection and bandwidth, it can be a quite a costly affair. 

However, since these Edge or IoT devices are supposed to run on low power, deep learning models need to be tweaked and packaged efficiently in order to smoothly run on such devices. Furthermore, many existing deep learning models and complex ML use cases leverage third party libraries, which are difficult to port to these low power devices. 

The global Edge Computing in Manufacturing market size is projected to reach USD 12460 Million by 2028, from USD 1531.4 Million in 2021, at a CAGR of 34.5% during 2022-2028. [1]

Simultaneously, the global edge artificial intelligence chips market size was valued at USD 1.8 billion in 2019 and is expected to grow at a compound annual growth rate (CAGR) of 21.3% from 2020 to 2027. [2]

Recent statistical results and experiments have shown that offloading complex models to the edge devices has proven to effectively reduce the power consumption. However, pushing such deep learning models to the edge didn’t essentially meet the latency requirements. Hence, offloading to edge may still be use case dependent and may-not work for all the tasks involved at this point. 

We have to now build deep learning model compilers capable of optimizing these models into executable codes with versioning on the target platform. Furthermore, it needs to be compatible or go in-line with existing Deep learning frameworks inclusive of but not limited to MXNet, Caffe, Tensorflow, PyTorch etc. Lightweight and low power messaging and communication protocol will also enable swarms of such devices to continuously collaborate with each other and solve multiple tasks via parallelization of tasks at an accelerated pace. 

For instance, TensorFlow lite can be used for on-device inferencing. First, we pick an existing versioned model or develop a new model. Then, the TensorFlow model is compressed into a compressed flat buffer in the conversion stage. The compressed tflite file is loaded into an edge device during the deployment stage. In such manner, various audio classification, object detection and gesture recognition models have been successfully deployed on the edge devices. 

Hence, if we migrate the deep learning models with specific libraries and frameworks (like tensorflow lite or ARM compute library for embedded systems) can be optimized to run on IoT devices. In order to scale it further with efficiency, first, we need to directly compile and optimize deep-learning models to executable codes on the target device and we need an extremely light-weight OS to enable multitasking and efficient communications between different tasks.

References:

1. https://www.prnewswire.com/news-releases/edge-computing-market-size-to-reach-usd-55930-million-by-2028-at-cagr-31-1—valuates-reports-301463725.html
2. https://www.grandviewresearch.com/industry-analysis/edge-artificial-intelligence-chips-market

By Shikhar Kwatra, Utpal Mangla, Luca Marchi

The sheer volume of information being processed and generated by Internet of things enabled sensors has been burgeoning tremendously in the past few years. As Moore’s law is still valid providing an indication of number of transistors in an integrated circuit doubling roughly every two years, growing ability of edge devices to handle vast amounts of data with complex designs is also expanding on a large scale. 

Most current IT architectures are unable to compete with the growing data volume and maintain the processing power to understand, analyze, process and generate outcomes from the data in a quick and transparent fashion. That is where IoT solutions meets Edge Computing. 

Transmitting the data to a centralized location across multiple hoops, for instance, from the remote sensors to the cloud involves additional transport delays, security concerns and processing latency. The ability of Edge architectures to move the necessary centralized framework directly to the edge in a distributed fashion in order to make the data to be processed at the originating source has been a great improvement in the IoT-Edge industry.

As IoT projects scale with Edge computing, the ability to efficiently deploy IoT projects, reduce security to the IoT network, and adding of complex processing with inclusion of machine learning and deep learning frameworks have been made possible to the IoT network. 

International Data Corporation’s (IDC) Worldwide Edge Spending Guide estimates that spending on edge computing will reach $24 billion in 2021 in Europe. It expects spending to continue to experience solid growth through 2025, driven by its role in bringing computing resources closer to where the data is created, dramatically reducing time to value, and enabling business processes, decisions, and intelligence outside of the core IT environment. [1]

Sixty percent of organizations surveyed in a recent study conducted by RightScale agree that the holy grail of cost-saving hides in cloud computing initiatives. Edge AI, in contrast, eliminates the exorbitant expenses incurred on the AI or machine learning processes carried out on cloud-based data centers. [2]

Edge is now widely being applied in a cross-engagement matrix across various sectors:

1. Industrial – IoT devices and sensors where data loss is unacceptable
2. Telecommunication – Virtual and Software Defined solutions involving ease of virtualization, network security, network management etc.
3. Network Cloud – Hybrid and Multi-cloud frameworks with peering endpoint connections, provisioning of public and private services requiring uptime, security and scalability
4. Consumer & Retail framework – Inclusion of Personal Engagement devices, for instance, wearables, speakers, AR/VR, TV, radios, smart phones etc. wherein data security is essential for sensitive information handling 

Since not all the edge devices are powerful or capable of handling complex processing tasks while maintaining an effective security software, such tasks are usually pushed to the edge server or gateway. In that sense, the gateway becomes the communications hub performing critical network functions inclusive of but not limited to data accumulation, sensor data processing, sensor protocol translation and understanding prior to the gateway forwarding the data to the on-premise network or cloud. 

From Autonomous vehicles running different Edge AI solutions to actuators with WIFI or Zigbee with the ability to connect to the network, with increased complexity of fusion of different protocols, different edge capabilities will continue to surface in coming years.

References: 

[1] https://www.idc.com/getdoc.jsp?containerId=prEUR148081021

[2] https://www.itbusinessedge.com/data-center/developments-edge-ai/

By: Shikhar Kwatra, Utpal Mangla, Luca Marchi

With the advancement in deep tech, the operationalization of machine learning and deep learning models has been burgeoning in the space of machine learning. In a typical scenario within the organization involving machine learning or deep learning business case, the data science and IT teams need to extensively collaboration in order to increase the pace of scaling and pushing of multiple machine learning models to production through continuous training, continuous validation, continuous deployment and continuous integration with governance. Machine Learning Operations (MLOps) has carved a new era of DevOps paradigm in the machine learning/artificial intelligence realm by automating end-to-end workflows.

As we are optimizing the models and bringing the data processing and analysis closer to the edge, data scientists and ML engineers are continuously finding new ways to push the complications involved with operationalization of models to such IoT (Internet of things) edge devices. 

A LinkedIn publication revealed that by 2025, the global AI spend would have reached $232 billion and $5 trillion by 2050. According to Cognilytica, the global MLOps market will be worth $4 billion by 2025. The industry was worth $350 million in 2019. [1]

Models running in IoT edge devices need to be very frequently trained due to variable environmental parameters, wherein continuous data drift and limited access to such IoT edge solutions may lead to degradation of the model performance over time. The target platforms on which ML models need to be deployed can also vary, such as IoT Edge or to specialized hardware such as FPGAs which leads to high level of complexity and customization with regards to MLOps on such platforms. 

Models can be packaged into docker image for the purpose of deployment post profiling the models by determining the cores, CPU and memory settings on said target IoT platforms. Such IoT devices also have multiple dependencies for packaging and deploying models that can be executed seamlessly on the platform. Hence, model packaging is easily implemented through containers as they can span over both cloud and IoT edge platforms.

When we are running on IoT edge platforms with certain dependencies of the device, a decision needs to be taken which containerized machine learning models need to be made available offline due to limited connectivity. An access script to access the model, invoke the endpoint and score the request incoming to the edge device needs to be operational in order to provide the respective probabilistic output. 

Continuous monitoring and retraining of models deployed in the IoT devices need to be handled properly using model artifact repository and model versioning features as part of MLOps framework. Different images of the models deployed will be stored in the shared device repository in order to quickly fetch the right image at the right time to be deployed to the Iot device. 

Model retraining can be triggered based on a job scheduler running in the edge device or when new data is incoming, invoking the rest endpoint of the machine learning model. Continuous model retraining, versioning and model evaluation become an integral part of the pipeline. 

In case the data is frequently changing which can be the case with such IoT edge devices or platforms, the frequency of model versioning and refreshing the model due to variable data drift will enable the MLOps engineer persona to automate the model retraining process, thereby saving time for the data scientists to deal with other aspects of feature engineering and model development. 

In time, the rise of such devices continuously collaborating over the internet and integrating with MLOps capabilities is poised to grow over the years. Multi-factored optimizations will continue to occur in order to make the lives of data scientists and ML engineers more focused and easier from model operationalization standpoint via an end-to-end automated approach.

Reference: 

[1] https://askwonder.com/research/historical-global-ai-ml-ops-spend-ulshsljuf

This post first published on the IBM blog at this link; it has been reposted here with permission. Some content has been redacted so as not to be seen as an endorsement by LF Edge. 

By Ashok Iyengar, Executive Cloud Architect & Gerald Coon, Architect & Dev Leader, IBM Cloud Satellite