In 2019 I delivered a design to integrate the IoT platform Device Simulator with IoT platform Connection and Analytics Services.
The objective of this design is to improve the customization of simulated device data and by automation of Data Management configurations enabling data to be stored in the DB2 data lake by the Connection Service.
“A LOB engineer running a POC can use out-of-the-box simulated device data and in a minute with no other actions required start exploring simulated entities and apply analytics functions to the simulated entity data”
Design Requirements
The requirements on this design
Device simulator shall by default be Activated
The Device Simulator editor shall provide a setting to “Save device events to the data lake”
The Device Simulator shall automatically generate and activate a physical and logical interface for a simulated device if the “Save device events” property is selected
The Device Simulator editor shall optionally allow a CSV file to be uploaded containing a simulated time series of data.
The Device Simulator shall construct an event payload schema from CSV file column names and data types.
The Device Simulator shall, by rows in the CSV file, send data events with column values as event payload.
The Device Simulator editor shall run (as-is) in the client browser. Closing the browser will terminate simulation and ingest of simulated data
The Device Simulator shall support the export/import of the simulator configuration.
Limitation: A user shall not be allowed to edit the simulated device type and the event payload schema after the physical and logical interface has been defined.
Production Design
The final Device Simulator production design.
Meet Chris is the Application Developer and Devon is the Edge Device Developer.
Chris and Devon are the most frequently used personas in the developer experience design across the platform capabilities.
Simulated Devices Design
The Device Simulator design in Watson IoT Platform enables developers to build and test the IoT applications without deploying physical devices beforehand.
In 2018 I delivered a design on the IoT Edge developer experience in the Watson IoT Platform.
What is IoT Edge?
Edge devices generally live on the edge of the network within an enterprise. They are specific hardware sensors, industrial controllers, or other industrial devices residing on private or potentially proprietary networks at a physical location.
Edge devices are creating quite a challenge for systems integrators since most are generally building cloud-based applications which rely on receiving information from IoT devices to accomplish the business goal.
There are many challenges that exist in developing a path for this information flow, including
Latency vs Time
Quality vs Time
Bandwidth and Cost
Security and Vulnerability
Compliance
Maintenance and operations
Read more about my design research on the IoT Edge experience in the Watson IoT Platform
Design Research on IoT Edge Design research on the IoT Edge experience in the Watson IoT Platform.
Hills
The Hills for IoT Edge may be viewed as a stack of capabilities that deliver value to the Business and the Solution.
“Chris, an IoT application developer, can develop and deploy applications that are transparent to the instrumentation of devices at the edge”
Hill 1 formulates the transparency of IoT Edge to solution development.
“Devon, an IoT Edge developer, can develop workloads for the edge using common cloud programming models, and automate workload distribution, management and updates on Edge Gateways”
Hill 2 expresses the WOW on adopting new Cloud programming models for the IoT Edge and moving away from the constraints of monolithic FW.
“Dave, an IoT Ecosystem Partner, can deliver IoT Gateway products with edge capabilities, managed by the IoT Edge”
Hill 3 that states the value to the partner ecosystem to leverage the IoT Edge as a platform for delivering edge-based services by shipping on or shipping with.
Design Vision
Chris, the IoT Application developer, can develop code to work with state and conditions of manageable assets, without knowledge of individual instrumentation, data workload distribution, or network connection state.
Using IoT Edge, Chris can
Create Edge Gateway types, select IoT Edge capabilities, and register Gateways, using the IoT Platform UI and APIs
Provision IoT Core capabilities to IoT Edge gateways
Distribute Data Management workloads to the edge to Filter Messages and Compute Device Device State and submit State Change Events
Connect devices to the Gateway to Ingest, Store and Forward MQTT messages
Develop and deliver cloud applications, independently of changes made to the edge instrumentation
Devon the IoT Edge developer, can develop workloads for the edge using common cloud programming models, and automatically distribute, manage and update workloads on Edge Gateways.
Using IoT Edge, Devon can
Create, build and upload custom containers to the IoT Edge component registry
Provision, Update and Manage custom containers on IoT Edge Gateways
Use the local MQTT message bus in IoT Core, to integrate messaging across custom containers, devices, and cloud
Create and test Cloud Functions on the cloud and configure as edge followers to run at the edge
Create aggregation of devices into Things
Create Rules and Actions, using Device and Thing state, and distribute as workloads to the edge
UX design
The User Experience of IoT Edge has three major parts
IoT Edge extensions to Watson IoT Platform Gateway Types
IoT Edge extensions to Watson IoT Platform Gateways
New IoT Edge Services
Developers will use existing concepts of IoT Platform Gateway Types. By choosing the option to make a Gateway an IoT Edge Gateway, developers get access to additional capabilities when defining a gateway type. The new capabilities are introduced together with the existing gateway properties or added as new tabs.
When adding a new Edge Gateway Type, a developer will create a gateway type, select the Edge option, and then choose the hardware architecture for gateways of this type. The options are Intelx86, ARM32, ARM64.
The developer can also optionally select Edge Services that will run on gateways of this type. The developer can browse the Edge Service Catalog for a list of services and available versions for gateways of this architecture. The latest version is shown by default. Alternatively, a different version can be viewed and selected.
For developers that want to get an overview of available Edge Services, the Edge Service Catalog is available for browsing and exploring the Docker images that have been configured for a service.
In 2017 I contributed to the design research and design concepts for Device Activity Policy for Risk and Security Management in the Watson IoT Platform.
What is Device Activity
Device Activity is an indicator of device health. In the Design Research on IoT Device Activity, it was proven that Time since the last message is a preferred metric on device activity and health. This new design for the Watson IoT Platform provides a new Risk and Security Management policy that defines the expected messaging activity for devices connected to the IoT platform. The policy monitors whether a device sends a request at least once every defined time period. A threshold for device activity can be defined generally for all devices, and exceptions defined for individual device types. Compliance with the Device Activity policy can be monitored on the dashboard, or reported on using drill-in reports.
Design Vision
The design vision for the new Device Activity policy is to call with the concepts already established for Risk and Security Management policers in the Watson IoT Platform. Learn more about the common design policies in Risk and Security Management Design. The new policy should be available in the Security section in the platform, next to other policies. The policy should have a simple default setting that applies generally across devices on the platform. This policy should set clear a clear threshold on the expended frequency of messages arriving from healthy devices exhibiting an active behavior. The policy should also allow exceptions to be set individually for any device type that differs from a default general policy. As an option, the threshold value may be set indefinitely to disable checking and make any devices of a type compliant.
Using compliance reporting, administrators and operators may identify any devices that exhibit unhealthy behavior. Drill-in reports allow users to view device status and perform diagnostic actions.
Personas
Risk and Security Management is primarily targeting two of the IoT personas
Adam is an IoT Security Operator. He ensures security and compliance by specifying policies that detect abnormalities and prevents devices to be compromised. He reports to audits on compliance with regulations and policy coverage on devices.
Sally is an IoT System Operator. She handles the day-to-day system operations on the LOB and client IoT organization. She makes sure that new device types and devices are registered, are behaving, and are up to date with recent secure firmware. She defines policies, creates and runs actions on policy alerts that act on misbehaving devices.
Hills
“Adam, the security analyst can define a Device Activity policy for devices, to determine whether a device is active or inactive by tracking how frequently a device talks to the platform, activating the policy, and reporting on the coverage of the policy, in under 5 minutes”
Use-Cases
As a security analyst, I can
View the default (disabled) device activity policy configuration
Modify the default device activity and set a messaging interval
Add a custom rule as exception device activity policy configuration for a device type
Preview the predicted compliance of the new policy configuration
Save and enforce the updated device activity policy configuration
UX Design
The UX design of the use-case above.
The UX design of the Device Activity policy.
Final Product Design
Final production design for Device Activity Policy editor.
Final production design for Device Activity Dashboard and Compliance Reporting.
Related Designs
Design Research on IoT Device Activity. Read more about the research on device behavior metrics to use for a Device Activity policy in Watson IoT Platform Risk and Security Management.
Risk and Security Management Design. Read more about the design of the Watson IoT Platform Risk and Security Management policies.
Compliance Reporting Design. Read more about the design for reporting compliance with Watson IoT Platform Risk and Security Management policies.
Watson IoT Platform Risk and Security Management Lab. DevZone Quick Lab at Think 2018 on Risk and Security Management.
Meet Sally the IoT System Operator.
Sally is one of the most frequently used operator personas in the design across the platform capabilities.
Also, meet Adam the IoT Security Operator.
Design Research on IoT Device Activity. Read more about the research on device behavior metrics to use for a Device Activity policy in Watson IoT Platform Risk and Security Management.
Risk and Security Management Design. Read more about the design of the Watson IoT Platform Risk and Security Management policies.
Compliance Reporting Design. Read more about the design for reporting compliance with Watson IoT Platform Risk and Security Management policies.
Watson IoT Platform Risk and Security Management Lab. DevZone Quick Lab at Think 2018 on Risk and Security Management.
In 2017 I delivered design concepts for an improved developer experience using a help hub capability in the Watson IoT Platform.
What is a Help Hub and a Side Panel
When users launch the Watson IoT service and open the IoT Platform web interface they are taken to the IoT platform dashboard. This default behavior is desirable and efficient for existing users of the IoT platform and takes them directly to an overview of the system state. The IoT platform dashboard is a flexible solution that allows users to use default boards, or configure their personal or team boards that present the right IoT system status to every user or team.
However, for new users entering the platform for the first time, this feature may be perceived as confusing. Users are left with the question “Ok, now what? You dropped me here, what do you want me to do?”
To improve the experience for developers exploring the platform we designed a Help Hub that answers just those questions. And, provides a structured journey for developers to quickly get started with the Watson IoT platform.
Design Vision
With the Help Hub design, we want to provide a simple onboarding experience for new users of the platform. The design should create a seamless learning experience within the Watson IoT Platform; gently guiding users toward the key ah-ha moments that allow them to form a mental model of the IoT platform and its resources. This will increase the likelihood that new users achieve their goals, gain satisfaction, continue engaging with key features, and happily refer the service to others.
The design should present a list of actions for the user to take to get started. The tasks should be context-sensitive. That is, actions are presented in a sequence of importance. Once an action is completed, new action(s) are activated and appear to the user. There are conditions that make actions active. For example, if there are no device types created the Help Hub should display a task to create a device type. Or, if there is only one user registered with the IoT platform, the Help Hub should provide an action to add additional team members as users.
Some users may find the pushing of tasks a bit annoying and will turn off such suggestions. Such users will find the Side Panel helpful as a way to, on-demand, see a list of recommended tasks and easy access to documentation.
User Research
In our research, we identified typical situations here the Help Hub and Side Panel are key to user success and satisfaction.
Getting started – Devon is an IoT developer in a new team formed to run a POC using IBM Cloud and Watson IoT Platform to build a demo IoT applications for a line of business in a week. The POC will provide new insights and optimizations for existing products or processes.
Devon, needs to
Solve a business problem through innovation
Build a PoC to secure ongoing funding & support
Anticipating production-level requirements: scale, DevOps, security, etc.
Needs to integrate with existing systems and processes.
Hills
“As a Developer with no prior IoT platform experience, I can within 15 minutes set up a new IoT platform organization and start developing and testing my IoT devices and applications.”
Help Hub Design Concept
The design concept for Help Hub identifies that
Developers will set up their development space in IBM Cloud, add a Watson IoT Platform service and launch the IoT Platform Dashboard
On the dashboard, the developer will see a clear call to action to get started
Enough context and information on the action will be provided directly on the dashboard
More detailed, deeper, information is available in the IBM Cloud documentation
Selecting an action will take the user directly to the right section of the IoT Platform
A pre-step is presented to the user. This describes the context, the resources, and the options and considerations to be taken.
When the action is completed, the user its taken to a post-step that salutes the achievement, and introduces the next action to be taken.
Use-cases
As a developer
I need to connect devices and get data flowing into the IoT Platform
I need to create my first Device Type
I Need to register my first Device
I need to connect my device to the IoT Platform
Or, I need to simulate a device
I need to enable my team members to access the IoT Platform dashboard
I need to add a Member
I need to enable my application to access the IoT Platform APIs
When launching the Watson IoT Platform for the first time, Devin is welcomed to the platform. The dashboard suggests that he registers a device or simulates a device. He is guided by the IoT platform on how to accomplish the action. New actions follow as he adds members and API keys to the platform.
Using the Help Hub
Final UX for Side Panel
When launching the Watson IoT Platform, Chris is presented with actions to connect devices and to register members and API keys. He clicks a button, indicating that he is not interested in any guidance. Later, Chris opens the Side Panel to view suggested actions and product documentation.
Using the Side Panel
Final Product Design
The final production design
Related Designs
IoT Platform Developer Experience Design research on the developer experience in the Watson IoT Platform.
Meet Chris is the Application Developer and Devon is the Edge Device Developer.
Chris and Devon are the most frequently used personas in the developer experience design across the platform capabilities.
IoT Platform Developer Experience Design research on the developer experience in the Watson IoT Platform.
In 2017 I delivered design concepts for an improved developer experience using simulated devices in the Watson IoT Platform.
What is a Simulated Device
Devices are smart sensors that instrument equipment at the edge. They connect to the IBM Cloud through the Watson IoT Platform. Devices send messages, using the MQTT messaging protocol, to the IoT platform. The messages contain sensor readings formatted as JSON payloads.
To connect a device to the IoT platform developers need an actual physical device. Most IoT developer has devices ready to connect. But in some cases, a lack of an available device may become a delay in exploring the IoT Platform and start developing IoT applications. The IoT Platform should support developers to get started even without a physical device to connect. This blog describes the design of a new flexible capability to set up simulated devices in the Watson IoT Platform.
Design Vision
With Simulated Devices, we want to improve the getting started experience for device and application developers.
The design vision is to allow users to create simulated virtual devices of a new or an existing device type. Users should be allowed to define the messages that the device should send, set up a schedule for the frequency of messages to be sent, and then turn on the device. Directly in the IoT Platform. As simple as that!
We want to deliver this capability independently of the IoT Platform backend and make the simulator run in the client browser. We also want to be conservative about the number of messages and data volume sent to avoid choking the IoT platform with high volume data or spending the free data plan for trial IoT Platform clients.
User Research
In our research, we identify two developer personas.
Devon, a Device Developer. Devon develops and tests code for a device that integrates sensors and actuators with the OS and CPU board capabilities. He integrates MQTT and device management protocols on the device. He uses SDKs provided by chip manufacturers and applies security by design principles.
Chris, an Application Developer. Chris is developing applications and services in a connected product IoT solution based on a cloud service architecture in IBM Cloud that integrates with the IoT Platform. He develops and tests his components locally and deploys to a team development space in the cloud configured with an IoT Platform using live or simulated device data.
In most cases, Chris may have a device delivered by Devon to use for application development and testing. In other cases, Chris needs a simulator to proceed with exploring the IoT Platform APIs, Data Management interfaces and transformation, State change events, and other aspects of the integration of his IoT Application with devices and data on the IoT Platform.
In this design, we primarily focus on Chris the application developer and his first steps in exploring and getting started with the IoT Platform. We want Chris to quickly get live devices and data. Secondarily we support Devon in his everyday use of the platform to develop new device firmware. Using the simulator, Devon can test and deliver new message schemas to Chris and early validate new firmware specifications with Chris applications running with the IoT Platform.
Hills
We defined the following hill for Chris the application developer
“As a Developer with no prior IoT platform experience, I can within 15 minutes set up a new IoT platform organization and start developing and testing my IoT devices and applications using simulated devices”
“As a Developer creating a new Device Type, I can choose to register a new physical device, or simulate a device of the new type”
Device Simulator Design Concept
The design concept for Simulated Devices identifies that
Simulated devices are a core part of the platform experience
As a developer you opt-in for this capability in IoT platform settings
The simulator is driven by a floating panel easily available from any screen in the IoT platform
Simulated devices act as any other device on the platform; they are of a device type, they have an identity, they send messages to the platform, like any physical device.
Messages from simulated devices act as any message on the platform; they are received on a topic, they are of a message type, they are put into the last message cache, they are passed to applications, like any message from any physical device.
There may be multiple simulated devices of a single device type
There may be simulated devices of multiple device types
There may be both simulated and physical devices of the same device types
Simulated devices may mimic physical devices and send events that are already the last message cache. (So if I once had a device connected that sent a message I have to use a simulator to act as that device and continue to send messages.)
I can set up variability in messages so data randomize between two end-points.
I can set up variability in messages so different simulated devices, of the same type, send different data messages.
The design should also provide shortcuts in other parts of the IoT platform to provide hints to developers to use simulated devices to make progress.
Shortcuts put on IoT platform panels, can set up simulations with one click.
Rather than pulling simulators out in the developer experience and out of context, they are integrated into the development flow
Use-cases
As a developer
I can opt-in and enable simulated devices and get a ‘Simulated” device type added to my organization
I can add (one or more) simulated devices and give them a name (pattern) “MyDevice” or “MyDevice#” and get a default event/payload sent to a topic, alternating over these instances.
I can browse the list of devices on the platform and view real and simulated device and their state (raw data)
I can specify the JSON payload schema for devices
I can specify a data range for simulated element values
I can add (one or more) additional simulated devices by id
I can specify a different JSON payload schema for different devices
UX Designs
The initial UX design explored two major screens in the IoT Platform.
The main simulator page for devices of a selected type. On this page, users can add new simulated devices, or select an existing registered device to be simulated. Users can also turn on/off the simulation.
In the first user journey below, Chris is entering the IoT platform. He enables Simulated devices on the settings page. When opening the simulator he is informed that he needs to create a device type. He switches to the Device Type section and creates a new temperatureSensor type. He then returns to the simulator, modifies the default message to send a temperature value. He creates a device that is given the default name TemperatureSensor-1. The simulator starts to send 20.0 C values. Chris modifies the message schema to send temperature values in a range between 15.0 and 20.0 C.
Configure a new simulated device.
In the second user journey below, Chris is using an existing device type TiSensorTag and an already registered device TiSensorTag-1. The device has previously been connected to the IoT Platform and an event exists in the last event cache. Chris configures the message to match the registered device.
Simulate a registered device.
The configuration of types, events, and devices in IoT Platform Device Simulator simulator can be Exported and Imported across sessions, users, and test cases.
Use the Import/Export tab to manage simulations using a JSON definition of the simulator configuration.
IoT Platform Developer Experience Design research on the developer experience in the Watson IoT Platform.
Getting Started with Simulated Devices DevZone Quick Lab at Think 2018 on Simulated Devices.
“All IoT Platform Design Partners strongly agree with the importance of simulated devices to their organisations and that the design meets their needs.”
Meet Chris is the Application Developer and Devon is the Edge Device Developer.
Chris and Devon are the most frequently used personas in the developer experience design across the platform capabilities.
IoT Platform Developer Experience Design research on the developer experience in the Watson IoT Platform.
Getting Started with Simulated Devices DevZone Quick Lab at Think 2018 on Simulated Devices.
In 2017 I contributed to the design concepts for Compliance Reporting for Risk and Security Management in the Watson IoT Platform.
What is Risk Management?
Businesses with IoT deployments have the challenge of ensuring that the entire IoT landscape operates within acceptable and expected boundaries. The consequences of IoT devices operating outside of defined criteria, or policies, could have a major impact on the security of the overall IoT deployment, the safe operation of connected devices, and business impact.
What is a Risk and Security Risk Management Policy?
The Watson IoT Platform allows the configuration of specific policies in relation to connection security.
Compliance reporting provides dashboard cards and drill-in reports on the policies configured in a Watson IoT Platform organization.
Personas
Risk and Security Management is primarily targeting two of the IoT personas
Sally is an IoT System Operator. She handles the day to day system operations on the LOB and client IoT organization. She makes sure that new device types and devices are registered, are behaving, and are up to date with recent secure firmware. She defines policies, creates and runs actions on policy alerts that act on misbehaving devices.
Adam is an IoT Security Operator. He ensures security and compliance by specifying policies that detect abnormalities and prevents devices to be compromised. He reports to audits on compliance with regulations and policy coverage on devices.
Hills
“A LOB engineer running a POC can use out-of-the-box simulated device data and in a minute with no other actions required start exploring simulated entities and apply analytics functions to the simulated entity data”
Use-Cases
As a systems operator, I need to:
See the high-level security status of the devices so I can quickly identify potential risks.
Drill down from a general view to more specific reports to identify problems
Drill into the current state and history of devices to analyze root-cause
Download, filter, and sort the report
UX Design
View Policy Compliance from Dashboard
The Watson IoT Platform Risk and Security dashboard provides an instant overview of the current security position and provides drilling to find more compliance-related information.
The dashboard has a collection of cards that can be used to monitor security-related KPIs.
Sally sees that the Connection Security policy has the most violation. She drills down to find out more information. She clicks on Connection Security
The compliance report shows overall compliance with the Connection Security policy and the current compliance for each rule. Sally clicks on a custom rule defined for TemperatureSensor device type to drill down to see all the devices that fail the default policy.
The next report presents a list of devices that fail the rule and the causes of failure.
Using the Watson IoT Platform Dashboard to view compliance.
View Policy Compliance from Policies Page
The Policies page in the Security section of the Watson IoT Platform provides an overview of Risk And Security Management policies.
Access the compliance report by clicking the “View Compliance” button.
The compliance report shows overall compliance with the policy and the current compliance for each rule
The next level of drill-down shows all the non-compliant devices that violated the rule. Search/filter/download the report.
Using the Watson IoT Platform Security Policy Editor to view compliance.
Final Production Design
Viewing compliance reports using the Dashboard in the Watson IoT Platform.
Viewing compliance reports using the Risk And Security Policies in the Watson IoT Platform.
Related Designs
Risk and Security Management Design The Watson IoT Platform Risk and Security Management provides an overview of compliance on policies and risks so that security administrators and operations personnel can quickly understand the security posture of their IoT deployment according to policies that they have configured within the Platform.
Watson IoT Platform Risk and Security Management Lab. DevZone Quick Lab at Think 2018 on Risk and Security Management.
Meet Sally the IoT System Operator.
Sally is one of the most frequently used operator personas in the design across the platform capabilities.
Also meet Adam the IoT Security Operator.
Risk and Security Management Design. Read more about the design of the Watson IoT Platform Risk and Security Management policies.
Watson IoT Platform Risk and Security Management Lab. DevZone Quick Lab at Think 2018 on Risk and Security Management.
In 2016 and 2017 I contributed to the design concepts and UX design for Information Management in the Watson IoT Platform.
What is Information Management
Watson IoT Platform Information Management provides an abstract model of identifiable assets that will allow IoT applications to be decoupled from the complexities of how specific devices are connected. Devices and logical composites often form a complex hierarchical relationship between maintainable assets & connected sensors and actuation devices. By providing a definition of a state, type schemas for data, abstract interfaces to state and behavior, and transformation of raw event data as part of the IoT Platform service the application layer benefits from decoupling of the instrumentation of the real-world and the management of the information model.
User Research
In our research, we observe that IoT solutions have
Multiple devices and sensors instrumenting connected assets
Devices integrated into an IoT solution come from an ecosystem of manufacturers
The message format defined by devices is often unique to the device type. Limited data standardization on formats and units exists.
Devices form the complex hierarchical relationship between the assets and their sensors and actuation devices.
Device relations may be static by configuration, or dynamic by condition.
Asset state depends on the conditional state of the related devices.
An IoT solution developer needs access to an abstract model of identifying assets that will allow IoT applications to be decoupled from the complexities of how specific devices are connected.
Our research uncovers that organizations need to
Improve productivity and speed of delivery on customer IoT deployment projects
Bring down development cost due to impact from variability across device models and variants
Reduce custom code required to create and manage Things on top of devices and their events and commands
Reduce steps to configure and deploy a customer IoT system of devices
Organizations also need
A simple programming model and an abstract interface model for devices and things to insulate from variability
The platform to take the load off developers and relationships, behavior, state, history, simple transformations, complex enrichment, and access control into a consistent Things design
A DEVOPS practice to run and manage the information model processing pipeline
Decomposing the needs for actions on data in the information management processing pipeline we find the following
Parsing – Devices send data in various formats, like text, binary, encrypted formats. Binary formats are used to reduce transmission costs. Applications need data in JSON format.
Normalize – Devices come in various models and versions. Each model uses its specific data format and schema. Applications need to interact with them all in the same way.
Transform – Devices use various data units. For example, temperatures may be sent in Fahrenheit, Celsius, or Kelvin degrees. Applications need to interact using one defined unit.
Filter – Only data that are relevant should be exposed. For example, only average data per hour may be required.
Enrich – Device data may be combined with from some other external source. For example weather at the device location.
Abstract – Devices operator in an event mode sending continuous data updates. My programming model may prefer a REST-based state model or a change based event-model.
Hills
Device Abstractions – Donna the information operator can manage the complexities of an IoT ecosystem while keeping the application insulated from data change
Aggregation into Things – Chris the developer can reference and work with manageable assets without being exposed to the individual instrumentation of them
“Donna the information operator can manage the complexities of an IoT ecosystem while keeping the application insulated from data change”
“Chris the developer can reference and work with manageable assets without being exposed to the individual instrumentation of them”
Design Concepts
IoT Platform Information Management ingest, transform and aggregate data from your IoT devices, diverse data sources, and platforms into asset-based data structures.
Information Management manages reusable abstractions of devices and aggregations of things.
Information Management provides an event-based processing pipeline to transform, enrich and aggregate the state of devices and things.
Information Management provides APIs and User Interfaces to manage the information model and interacts with the state model of device and thing instances
Devices connect to the IoT Platform and submit events with device data. The information model declares schemas for the event types to specify the payload data types. The model also transforms the physical device events to a logical property model with readable property names, JSON data types, REST APIs to property values, state change events.
Simple information mode with a TEMPERATURE device sending “t:37.5” MQTT messages that get exposed to Watson IoT Platform Platform consumers as property “temperature” of type Number with value 37.5 C.
The information management model also provides an abstraction of device interfaces. Each Application Interface interface declares properties mapped to the physical interface. Abstract interfaces may be used to implement common standards normalizing the interface across sets of similar device types. Application Interfaces may be used to manage versions of device interfaces, hence adding new properties and deprecating old properties in a new interface. Application Interfaces may also implement open standards.
Binary messages from the LED light is transformed using DFDL to JSON. The Application Interfaces are mapped to the event types and type schemas in the Physical Interface.
A pain point called out by our sponsors is the effort put on the application layer to manage, and keep up with, changes and additions to the set of device types. Device types may be declaring different event types and applications needs to be updated when new device types are added to the solution.
With the Information Management model devices may now be mapped from single abstract Application Interfaces to the specific Physical Interfaces individually for each device type, hence making the application layer independent of the variability across physical devices.
Any LED device type supports the color property by mapping to the specific event type on each device type.
The concepts can be summarised as
Model custom device data events and provide an information model
Multiple reusable abstract application interfaces to insulate applications from variability across device types, sensor models, variants, and versions
Use abstract application interfaces to insulate applications from variability across device types, sensor models, variants, and versions
UX design
The use of Information Management in the Watson IoT Platform is an optional capability.
Operators can view raw event data in JSON format sent by devices to the IoT platform
Developers can build IoT applications that depend on the custom JSON message payload retrieved from the last event cache using the IoT platform APIs
However, using Information Management in Watson IoT Platform developers can get the benefits of parsing, transforming, normalizing, enriching, and abstracting data using the Information Management data processing pipeline. The Operator or Developer configures Information Management using the Interface section on a Device Type.
The user experience with the Information Management user experience is separated into two modes; a simple flow and an advanced flow.
The user experience with the Information Management user experience is separated into two modes; a simple flow and an advanced flow. The simple flow uses a simple interface editor and lets users select and filter the data properties that represent the device state. The advanced flow uses an advanced interface editor and provides richer capabilities to define an event type schema. Developers can also perform filtering and transformation of event data to multiple reusable interfaces that normalize device states across multiple device types.
Designing a Simple User Experience for Device State Properties
The simple interface editor appeals to early adaptors getting started with IoT, Devices, and Device State. It is essential to quickly get started, connect devices, identify the properties exposed by the devices and make them available to other capabilities in the IoT platform. The simple model allows users to look at incoming events, select the data that is of interest, and create the readable name and data types for the exposed properties.
The simple interface editor user experience lets users view the message payload of a device event. The user then selects the properties that should be exposed and chooses a data type for the data property.
The Information Management processing pipeline will receive the event, filter the data, and expose the state properties. The state is exposed in the IoT platform dashboard and accessible through the IoT platform APIs.
Using the simple interface editor to expose a temperature property “t” from the TemperatureSensor device.
Designing an Advanced User Experience using Physical and Logical Interfaces
The advanced interface editor exposed a richer set of capabilities and flexibility in defining the information model. Two additional concepts are exposed to the user in the user experience; physical interfaces and logical interfaces.
A physical interface is used to model the interface between a physical device and Watson IoT Platform. Event types can be associated with physical interfaces. The physical interface editor has been designed to support multiple ways of defining an event-type schema. Developers can
Let the IoT platform listen for events from devices and reverse engineer an event type schema from the event payload.
Manually edit an event type schema by adding event properties
Import an existing event type schema from a file
Using the simple interface editor to define a ‘t’ : Number temperature state property for the TemperatureSensor device type
A logical interface is used to define the normalized view onto the device state in the Watson IoT Platform and is usually consumed by applications. A logical interface must be associated with a logical interface schema that defines the state properties. The state is updated in response to inbound device events. The advanced interface editor provides a guided user experience in creating a logical interface.
As logical interfaces are highly reusable across device types is recommended to use an interface name that states the purpose of the abstraction, rather than the name of the device type. For example, it is recommended to use names like ‘iTemperature’ and ‘iHumidity’ for logical interfaces than ‘TiSenstorTag-Temperature’ that makes it less reusable across an ecosystem of temperature sensor products. The advanced editor also exposes the library of schemas and interfaces for reuse.
State property in a logical interface is bound to event properties using a mapping expression. Such a mapping expression may be as simple as just referencing the event property. It may also be as complex as a JSONata expression that computes a typed value using event properties, other state property values, constant values. JSONata expression provides arithmetic functions, string, array and time functions, and other utilities.
Using the advanced interface editor to define a ‘iTemperature’ logical interface with an exposed temperature state property “Temperature” from a TiSensorTag device.
Designing an Advanced User Experience for Multiple Logical Interfaces
Developers using the advanced interface editor can add multiple logical interfaces to a device type. Such added interfaces can be new definitions or reused interfaces selected from the Interface Library. A shared logical interface provides a fixed set of state properties and a new set of mapping expressions that binds the fixed state properties to the event properties defined in the physical interface of the device type.
Using the advanced interface editor to define a second ‘iHumidity’ logical interface with an exposed temperature state property “Humidity” from a TiSensorTag device.
Designing an Advanced User Experience for Activating Draft Interfaces
The advanced interface editor will save changes to physical and logical interfaces as Draft versions of the resources. A draft resource needs to be Activated to be added to the Information Management processing pipeline. The design of the advanced interface editor clearly indicates the state of an interface resource. The timestamp of the activation (deployment) is shown in the user interface.
Updating a resource will create a draft version.
Changes to interface resources have to be activated to be deployed into the Information Management processing pipeline.
Designing an Advanced User Experience for Mapping Expressions
State property in a logical interface is bound to event properties using a mapping expression. Such a mapping expression may be as simple as just referencing the event property. It may also be as complex as a JSONata expression that computes a typed value using event properties, other state property values, constant values. JSONata expression provides arithmetic functions, string, array and time functions, and other utilities.
The expression editor user experience design provides two editor modes; an expression builder and a code editor mode. The expression builder provides a graphical interface for selecting expression elements like payload properties, values, arithmetical operators. The code editor design provides a text editor for JSONata expressions. Expressions use keywords like $event to reference physical interface event properties by name, or $state to reference logical interface state properties by name. For example, “= $event.d.ambientTemp” to reference the ambient temperature event property in the TiSensorTag status event.
Using the Expression Editor to bind state properties to event properties.
Note, a state property may be an object of values. In the example above a Temperature property exposes the Celsius value. The state object exposed computed temperature values converted to Fahrenheit, Celsius, and Kelvin using mapping expressions all bound to the TiSensorTag ambient temperature value.
Related Designs
New designs for Information Management are currently developed
Design for Things Types and Thing Instance user experience
Design for an Interface Library user experience
Design for new Types vs Instance navigation user experience
Getting Started with Watson IoT Platform Data Management DevZone Quick Lab at Think 2018 on Data Management.
The design of the new Device and Thing information model you design is great. It off-loads the application layer a lot of code managing the aggregation of devices. - Watson IoT Platform sponsor.
Getting Started with Watson IoT Platform Data Management DevZone Quick Lab at Think 2018 on Data Management.
In 2016 I contributed to the design concepts for Risk and Security Management in the Watson IoT Platform.
What is Risk Management
Businesses with IoT deployments have the challenge of ensuring that the entire IoT landscape operates within acceptable and expected boundaries. The consequences of IoT devices operating outside of defined criteria, or policies, could have a major impact on the security of the overall IoT deployment, the safe operation of connected devices, and business impact. The Watson IoT Platform allows the configuration of specific policies in relation to connection security, plus blacklist and whitelist options for IP addresses.
The design objectives are to
Provide assurance to clients of the security between device and cloud for data protection, authentication, authorization, and access control
Establish a most trusted IoT platform with all the associated security and standards
Reduce the risk and impact of IoT security incidents by enabling clients to efficiently and securely manage their IoT landscape
The design should
Provide a security dashboard that visualizes the security status and provides easy access to device management and operations
Provide security policy definition and management to quickly spot security and critical issues
Provide advanced device management and register devices for secure operations by defining and managing cryptographic key material
Provide advanced data protection through Authentication, Authorization & Access Control
User research
User research on Risk and Security Management is indicating the importance in the following areas
TLS authentication using certificates and/or tokens
Most devices today have or have plans to support client certificates
Many devices today support TLS authentication
Managing and reporting on IoT risk and security compliance through policies
Blacklists are considered the primary way to block device connections
Personas
User research on Risk and Security Management is impacting the following IoT personas
Adam is an IoT Security Operator. He ensures security and compliance by specifying policies that detect abnormalities and prevents devices to be compromised. He reports to audits on compliance with regulations and policy coverage on devices.
Sally is an IoT System Operator. She handles the day-to-day system operations on the LOB and client IoT organization. She makes sure that new device types and devices are registered, are behaving, and are up to date with recent secure firmware. She defines policies, creates and runs actions on policy alerts that act on misbehaving devices.
Lester is a Service Delivery Manager. He is responsible for an SLA with an IoT client to the LOB. He, and his team of maintenance engineers, are on or near the client site and manage equipment and use the IoT Foundation platform and LOB industry applications to monitoring, plan, and service equipment.
Rob is a Maintenance Engineer responsible for servicing managed assets in a site or region. He uses a mobile maintenance application to access assigned work orders, asset location, status, and history. He requests new firmware and configuration updates to resolve issues.
User research also concludes that organizations that are adopting IoT often combine system operations and security administration responsibilities into a single role. The organizational scope of system operations and security administration may vary. Global system operators like Sally may enforce common policies while service delivery leads like Lester may define exceptions for policies in the context of a specific deployment.
“Sally the system operator can secure device messaging by utilizing TLS mutual authentication using client and server certificates provided by her organization”
Hill 2 – Define policies.
“Sally the system operator can define a policy for devices, preview the coverage of the policy and activate the policy, in under 5 minutes”
Hill 3 – Overview of exposure.
“Adam the security analyst can from a single view get an overview of the security compliance and drill into detailed information and definitions of the policies”
UX Design
System operator deploys certificates. The organization adopts certificates to improve connection security. System operator needs to deploy certificates in the platform to support TLS authentication.
Deploying certificates to an IoT platform organization.
Security analyst configures policies. Security analyst open Risk Management policies, configure the connection security and blacklist policies for the organization and predicts device compliance.
Configuring the connection and blacklist policy.
Security analyst overviews security posture and policy compliance. Security analyst views the shared Risk and Security dashboard to understand the overall security KPIs and drill into the policy details for root causes.
Reporting on policy compliance using the Watson IoT Platform Dashboard.
Related Designs
Compliance Reporting Design. Read more about the design for reporting compliance with Watson IoT Platform Risk and Security Management policies.
Watson IoT Platform Risk and Security Management Lab. DevZone Quick Lab at Think 2018 on Risk and Security Management.
Meet Sally the IoT System Operator.
Sally is one of the most frequently used operator personas in the design across the platform capabilities.
Also, meet Adam the IoT Security Operator.
“Connection security, Device health, and Anomaly detection are important risk and security factors to achieve operational data quality.” – Watson IoT Platform sponsor
Compliance Reporting Design. Read more about the design for reporting compliance with Watson IoT Platform Risk and Security Management policies.
Watson IoT Platform Risk and Security Management Lab. DevZone Quick Lab at Think 2018 on Risk and Security Management.
Watson IoT Platform Risk and Security Management Lab DevZone Quick Lab at Think 2018 on Risk and Security Management.
In 2016 I contributed to the design concepts for Access Control in the Watson IoT Platform.
What is Access Control
Operational security is key to the Watson IoT Platform and includes
Authentication is the process of verifying that a user is who he claims he is and belongs to the selected IoTP organization. This is your typical log in system.
Authorization is verifying that subjects, e.g. users, can only perform the actions that you want them to perform and no more.
Access Control extends authorization by allowing further restricting access to the resources an action is executed on. Access control also includes non-resource-related restrictions, for example temporal restrictions, spatial restrictions or conditional restrictions.
The Access control design must provide configuration and management of Roles which enable controls to be defined for Users, Applications and Gateways. Configuration of Role permissions grant or restrict the ability to perform particular platform operations
User Research
User research on Access Control is impacting security scenarios for the following IoT personas
Sally is a System Operator
View the permissions granted in a role
Create a new custom role and select permissions granted
Modify and remove a custom role
She needs to assign roles to Users, Applications and Devcies
She needs to revoke access for Users, Applications and Devices
Rob is a Maintenance Engineer.
He only has access / permissions to a set of assets in the scope of his task, or the scope of his regional organization.
His access may be checked against his and the device geo location.
He may be delegated permissions to additional assets and tasks, e.g. to perform an FW upgrade.
Chris is a Developer
He needs API Keys to the Watson IoT Platform and other services
He needs to develop and test in a IoT development / test environment and stage into a IoT production environment
He has full access in dev/test and limited access in production
Devon is a 3rd party developer
He needs API Keys to Watson IoT Platform and other services
He needs to develop and test in his 3rd party development environment, optionally integrated with the client staging environment
He has full access in dev/test and limited access in production
He depends on client APIs, docs and samples
He and his applications may be a leakage point for access keys and data and access may be revoked
Hills
Sally the system operator can
Authorize user permissions to resources using predefined roles
Authorize user permissions to resources using custom roles
Authorize user permissions to groups of resources using custom roles
Design Concepts
Terminology
The design concept defined the access control model
Subject. Anauthenticated entity that requests platform access and which needs to be authorized. Examples of subjects are Users, Applications, Devices and Gateways. Subjects may be grouped and such subjects groups are also subjects.
Action. The action that the subject wishes to perform. Most actions can be categorized as either View or Manage e.g Create, Read, Update, Delete or Execute. Operations are a further grouping of Actions on resource types covered by a given API. Permitted operations are grouped in a Role.
Resource. The object(s) that the subject wishes to perform the action against. Resources may be associated with a Group, or Tagged. Groups and tags are used to define a resource scope.
Permission. A permission is the selection of a role and a scope of resources that defined the permission of a subject. Subjects may be given multiple permissions.
Predefined Actions
The actions in the IoT Platform user interface and APIs, available to users, applications and devices, are grouped into sections. The sections are related to resource types and goupings of related actions. The groups are
Devices
Logs
Cache
Historian
Organization
Access Control
Gateways
Real Time Analytics
Connector
Risk Management
Actions under there categories are support by selecting View actions resources. Administrators can configure permissions to be
No access – by deselecting actions in a category
Real-only – by selecting View resource actions
Read and Write – by selecting by selecting View actions resource actions
Predefined User Roles
The Watson IoT Platform provides predefined Roles for users with a predefined set of Actions. These one of these roles may be selected for a user.
Administrator
As an Administrator I’m permitted to all user commands in the IoT Platform
Operator
As an Operator I can view and manage (create, modify, remove) devices, types, data, rules, users, roles and access.
I can not configure IoT Platform storage, authentication and mail configurations
Developer
As a Developer I can view and manage devices, types, data and rules.
I can not view or manage IoT Platform configurations or API keys
I can not manage users or access
Analyst
As an Analyst I can view and manage analytics (cloud and edge) rules, actions and alerts
I can view types, devices, events, data, users and roles
Reader
As a Reader I can view types, devices, events, data, rule alerts, users and roles
Predefined ApiKey Roles
The Watson IoT Platform provides predefined Roles for ApiKeys with a predefined set of Actions. These one of these roles may be selected for an API Key.
APP_Operator
Application for Administrators that replace or extend the IoT Platform web interface.
Can not configure IoT Platform storage, authentication and mail configurations
APP_BackendTrusted
Applications for device provisioning, device operations, data transformation and information management
Can view and manage devices, types and and rules. Can publish and subscribe to events and commands
APP_DataProcessor
Applications for reading and processing data, like Analytics
Can view and manage rules. Can view devices and types. Can subscribe to events and publish commands.
APP_Visualizer
Applications for presenting and visualizing data
Can view devices, types and alerts. Can subscribe to events
APP_Device
Device simulator
Default Device
Can publish events and subscribe to commands. Can subscribe and publish to DM topics.
NonTrusted_Gateway
In addition to above
Can view devices and device types.
Can can pub/sub to Device Management topics for itself and in behalf of locally connected devices.
FullyTrusted_Gateway
In addition to above
Can activate devices
Predefined and Custom Roles UX Design
Users with permissions to View roles and explore the predefined roles in the Users and Application sections in the IoT Platform. The axes of the presentation can be switched so that Roles are presented Horizontally or Vertically. Users can also filter the categories of actions to be shown.
Managing user roles in the Watson IoT Platform.
Custom roles can be created by Administrators, or other authorised users. A custom role selects the actions that a user or application may perform.
Creating a new role.
Meet Sally the IoT System Operator.
Sally is one of the most frequently used operator personas in the design across the platform capabilities.
We need to control access control and permissions for our internal organization, and for our 3rd parties that extend our IoT services. – Watson IoT Platform sponsor.
In 2016 I delivered design concepts and a UX design for Custom Device Management extensions in the Watson IoT platform.
What is Custom Device Management
Device Management in the Watson IoT Platform provides capabilities for Operators to perform actions on devices connected to the IoT platform. Basic device management actions include; Download Firmware, Update Firmware, Reboot and Factory Reset.
The device management requires a device management agent, implementing the device management protocol, to be running on devices. The agent is responsible for responding to IoT platform MQTT commands that initiates a device management action.
Custom Device Management extends the basic Device Management actions and enabled client organizations to define their specific actions and have any such actions available in the list of device management actions in the IoT platform user interface.
Device Management on Watson IoT Platform.
User research
In my research I found that
Device and Gateway partners implement a rich set of device management actions on gateways, in device management services, or in external cloud platform. Users’ needs to integrated this external set of actions and make them seamlessly available as delegated actions on devices in the Watson IoT platform.
Partners are delivering advanced device management for enterprise scale management of firmware. Client using such advanced device management services needs to make them seamlessly available, replacing any standard device management action for selected devices types.
Custom device management actions will specific to a set of device types. In a typical deployment multiple device management action sets will be required, for multiple independent device partners and service providers.
Custom device management actions will be downloaded as a device management extension package from a device partner product web page and installed on the IoT Platform.
Custom device management extension packages will be versioned. Multiple device management extension package versions can run on the platform, or be replaced with a new version.
Custom device management extension packages needs to be internationalized.
Custom Device Management Design
Design Concepts
The custom device management design is enabling
Extending the built-in capabilities for Device Management in WIoTP
A consistent user experience for all build-in and custom DM commands
Supporting delegation of device management 3rd party device, gateway and platform partners
Definition of a Custom Command Package to declare and upload commands to WIoTP
Support version and variant management of Custom Command Packages
Hills and Use-cases
As an Operator I need to configure, manage and run 3rd party custom Device Management actions on devices and gateways managed in the IoT Platform
As an Operator I need to configure an extension for 3rd party DM actions on a device type
As an Operator I need to select and run a DM action
As an Operator I need to update my 3rd party DM extension to stay in sync with new FW updates
UX Design
Install a 3rd party Device Management extension package
The System Operator will go to the 3rd party device provider web or log into the 3rd party cloud service. A device management extension package will be downloaded for a set of 3rd party device types. The System Operator will log into the IoT platform, choose Settings > Extensions and upload and enable the Device Management extension package.
Installing a 3rd party Device Management extension package.
Run a run a 3rd party Device Management action
The System Operator logs into the IoT platform. The operator has permissions to run DM actions and opens the Devices > Actions section. The operator chooses a 3rd part DM action, selects a device / gateway type, selects devices and provides optional information for the action. The operator runs the action and monitors the progress.
Running a run a 3rd party Device Management action.
Run a run a 3rd party Device Management action from a Device Type or Device
The System Operator may also run the 3rd part DM action directly on supported device types or devices.
Run a run a 3rd party Device Management action from a Device Type or Device.
Update a 3rd party Device Management extension package
Operators will, as part of a FW update, go to the 3rd party web and retrieve a new Device Management extension package with updated custom commands. The operator may choose to replace the existing package, or alternatively add a new package to maintain devices with different FW version
Updating a 3rd party Device Management extension package.
Read more about Custom Device Management Actions on the Watson IoT Platform blog.
Custom Commands Extension Package Designs
The UX design in the previous section shows how to use the IoT platform user interface to upload and use Custom Commands Extension Packages.
An extension package is JSON file with a structure defined by Watson IoT Platform
An extension package has
A name
A version
A description
A list of supported device types
A list of custom device management commands
A command has
A name
A description
A list of parameters (string types) w/ name, description, optionally a default value
A command specification containing the command string with parameter substitution identifiers
A sample extension package is provided in IoTP documentation
Related Designs
The custom command design has enabled related designs in 2016
Usage of custom device management commands in Firmware over the air (FoTA) integrations with business partners
With Custom Device Management we have full control integrating with the MQTT commands implemented in our IoT devices and gateways. – Watson IoT Platform partner.