Version 1.2.0, see Changelog for all notable changes made in this version.
⚠️ This stack is for demonstration purposes only. It exposes a public API endpoint for the STF Core. Before using in production, please consider adding a mechanism for controlling and managing access to this API.
The Smart Territory Framework – STF – is an open-source framework that makes it easier and faster to create and operate interoperable platforms. STF supports the development and integration of smart and efficient solutions across multiple domains such as Smart Cities, Campuses and Regions; Energy and Utilities; Agriculture; Smart Building and Manufacturing.
The STF complies with the NGSI-LD open standard and leverages the NGSI-LD compliant Smart Data Models initiative that provides open-licensed data models for different domains such as Smart Cities, Smart Agrifood, Smart Environment, Smart Sensoring, Smart Energy, Smart Water, Smart Destination, Smart Robotics and Smart Manufacturing.
Modular and built on open source and standards, the STF makes it easy to integrate existing solutions and add new capabilities and modules over time to its core. The core of the STF - STF Core - consists of two modules: the STF IoT module and the open-source FIWARE Context Broker. There are multiple implementations of the FIWARE Context Broker. As of today, STF supports the NEC Scorpio Broker and Orion-LD.
This repository contains an AWS CDK application that enables you to deploy the STF Core stack. The STF Core stack consists of two nested stacks:
- a stack to deploy a FIWARE Context Broker,
- a stack to deploy the STF IoT module.
To deploy the STF Core stack, you need the following:
- An AWS account. If you don't have one, see Set up an AWS account.
- An AWS Identity and Access Management (IAM) user with administrator permissions
- AWS CDK Toolkit installed. See Getting started with the AWS CDK to install it.
- Understand the NGSI-LD standard and the FIWARE Context Broker. This blog post gives a good overview of both topics.
- The FIWARE Context Broker
- STF IoT
- Getting started
- How to use it
- Building Data Producers and Data Consumers
- Design Considerations
- Additional Resources
- Troubleshooting
The Context broker is an open source component that enables the connection and integration of different systems, applications and services within an organization.
It provides geo-located query capabilities as well as a subscription mechanism enabling an independent module like a mobile application to query data filtered by geographical location but also to be notified with a given frequency or when changes on data take place (e.g., an air quality measurement is above a specified threshold value).
Among the multiple implementations of the FIWARE Context Broker available, we have integrated the NEC Scorpio and the Orion-LD context brokers for this CDK application leveraging for both Serverless services on AWS for automatic scaling, built-in high availability, and a pay-for-use billing model to increase agility and optimize costs. When deploying the STF, you can select the one you want to use.
NEC Laboratories Europe and AWS have worked together to provide a scalable and available implementation of the NEC Scorpio Broker on AWS using:
- AWS Fargate to easily run and scale the NEC Scorpio Broker container.
- Amazon RDS or Amazon Aurora Serverless for the PostgreSQL database used by the NEC Scorpio Broker.
- Amazon MSK to make it easy to run the Apache Kafka used by the NEC Scorpio Broker.
Fiware Foundation and AWS have worked together to provide a scalable and available implementation of Orion-LD on AWS using:
- AWS Fargate to easily run and scale the Orion-LD container.
- Amazon DocumentDB (with MongoDB compatibility) for the MongoDB database used by the Orion-LD context broker.
Built around the FIWARE Context Broker, the STF IoT module expands its capabilities enabling you to ingest IoT data at scale from multiple and heterogeneous sources with advanced device management capabilities like for example a registry of all your devices and sensors, regardless of the operating model, the technology or the connectivity used.
The STF IoT module offers Digital Twin capabilities enabling you to store and retrieve in real-time the current state of every registered device.
It also consists of an IoT data lake built on Amazon S3 that you can use to store, query and generate insights about your IoT data and easily visualise them.
The STF IoT module provides a central registry to store all your devices using the Smart Data Model Device.
All measurements and data produced by a device are linked to the registered device and stored in its Device Shadows using the relevant data model like for example AirQualityObserved for air quality measurements.
This gives an exhaustive view in real-time of the state of the device and its measurements providing Digital Twin capabilities. The STF IoT module is synchronized in real-time with the Context Broker so any change that occurs in the STF IoT module is replicated in the Context Broker and stored as well in the STF IoT Data Lake.
The STF IoT Data Lake is built on top of Amazon S3. A bucket named stf-iot-datalake-${Aws.REGION}-${Aws.ACCOUNT_ID} is created when this stack is deployed. You can use an existing bucket by setting the new_bucket property to false (and if not default name used, type the name of the bucket in the property bucket_name) in the file parameters.ts. Note that if there is an existing bucket with the same name and you set the property new_bucket to true, the deployment will fail.
With a single endpoint, the STF IoT module provides an API for device management in addition to the NGSI-LD FIWARE Context Broker API.
Indeed, the STF Core provides a unified API that exposes two services:
- the NGSI-LD API to interact with the FIWARE Context Broker
- the STF IoT API to provision and manage your things in the STF IoT Registry but also query your things with all their associated entities.
See How to use it section for more details on how to use the STF Core and its unified API.
Note, you will incur costs for the AWS services used by this application. To track costs in your AWS account, consider using AWS Cost Explorer and AWS Billing and Cost Management. You can also set up a billing alarm to get notified of unexpected charges.
This repository contains an AWS CDK application that enables you to deploy and provision the STF Core stack code and its supporting infrastructure. AWS CDK provisions your resources in a safe, repeatable manner through AWS CloudFormation.
The application includes a file parameters.ts in which you can select the region in which you will deploy the stack, which Context Broker and configure the resources that you will deploy like the type and size of instances to use for the database (and kafka for the Scorpio Broker). The default parameters will provide you with a scalable and available platform.
You can see estimates of the monthly cost of running the STF Core using Scorpio Broker and Orion-LD. The AWS Pricing Calculator is an estimation tool that provides an approximate cost of using AWS services based on the usage parameters that you specify. The AWS Pricing Calculator is not a quote tool, and does not guarantee the cost for your actual use of AWS services. The cost estimated by the AWS Pricing Calculator may vary from your actual costs for a number of reasons. These estimations do not include messaging costs.
Make sure that all the AWS services used in this stack are available in the region you deploy in. You can check here.
Once the Context Broker selected and the configuration parameters checked, you can deploy the application using the following commands:
npm install
cdk bootstrap
cdk deploy
The deployment of the STF Core stack can take up to an hour.
Once the STF Core stack is deployed, you will find in the output all the information needed to operate and connect your smart solutions using the STF including:
-
The STF unified API endpoint
StfCoreEndpoint. This endpoint is a unified API that exposes two services: The NGSI-LD FIWARE Context Broker API (that fully complies with the NGSI-LD API) and the STF IoT API that you can use to provision and manage your things in the central and unified STF registry. Before using this stack in production, please consider adding a mechanism for controlling and managing access to theStfCoreEndpoint. -
The ARN of the Amazon SQS Queue of the STF IoT module
StfCoreIotQueueArn. The default value of the ARN of the queue isarn:aws:sqs:${Aws.REGION}:${Aws.ACCOUNT_ID}:StfIoTQueue-${Aws.REGION}. This queue is the single entry point for all data producers. Usually, the data producer transforms the source data format into the data model relevant to the use case and then publishes the transformed data into the queue. See Building Data Producers and Data Consumers section for more details.
See How to use it section for more details.
You can also find this information in the AWS CloudFormation console in the Outputs tab after selecting the STF Core Stack.
Now that the STF Core is up and running you can innovate at a fast pace. You can easily add new modules and capabilities, new data producers and consumers over time. you can integrate your existing applications and legacy systems.
Before this, let's check that the FIWARE Context Broker is up and running. You can send an HTTP GET to {StfCoreEndpoint}/actuator/info for the Scorpio Broker or to {StfCoreEndpoint}/ngsi-ld/ex/v1/version for Orion-LD.
In the example below with the Scorpio Broker, I use the CURL tool in my terminal but you can paste the URL into any web browser.
curl https://abcde5678.execute-api.aws-region.amazonaws.com/actuator/info
The response is:
{
"build":
{
"artifact": "AllInOneRunner",
"name":"AllInOneRunner",
"time":"2022-05-01T23:55:31.024Z",
"version":"1.1.0-SNAPSHOT",
"group":"eu.neclab.ngsildbroker"
}
}VoilĂ !
If you are familiar with the FIWARE Context Broker and the NGSI-LD specification, you can skip the next section an go directly to Register and manage your things using the STF IoT API.
Before you build solutions and start ingesting data using the STF Core, you need to understand the NGSI-LD Information model and the design considerations for builing a Smart Territory Platform using the STF.
The NGSI-LD specification provides an open API for managing and requesting information and an underlying information model based on Entities that are digital representations of real-world objects having Properties and Relationships with other entities.
Every information stored in the Context Broker and in the STF IoT module is an NGSI-LD entity.
Since NGSI-LD is based on JSON-LD, Entities are stored with their context.
Below an extract of the definition of the context from the JSON-LD specification:
When two people communicate with one another, the conversation takes place in a shared environment, typically called "the context of the conversation". This shared context allows the individuals to use shortcut terms, like the first name of a mutual friend, to communicate more quickly but without losing accuracy. A context in JSON-LD works in the same way. It allows two applications to use shortcut terms to communicate with one another more efficiently, but without losing accuracy. Simply speaking, a context is used to map terms to IRIs.
A context is basically a list of key-values defining aliases (shortcut terms) for longer terms (IRIs).
{
"@context": {
"temperature": "https://smart-data-models.github.io/data-models/terms.jsonld#/definitions/temperature",
"co2": "https://smart-data-models.github.io/data-models/terms.jsonld#/definitions/co2"
}
}See Understanding @context to learn more about JSON-LD context.
Alongside the built-in NGSI-LD default context, the STF leverages the Smart Data Models and its context. The URL of the Smart Data Models context is provided in the file parameters.ts.
Before creating or updating an entity in the Context Broker, the STF IoT will provide this context in the request so the Context Broker can store the corresponding expanded form of the entity.
For example, below an entity of type Device in its compacted form using the Smart Data Models context.
{
"id": "urn:ngsi-ld:Device:MyFirstDevice",
"type": "Device",
"serialNumber": {
"type": "Property",
"value": "49206C6F766520796F752042617961"
},
"@context": "https://raw.githubusercontent.com/smart-data-models/data-models/master/context.jsonld"
}The Context Broker will store it in its expanded form (using the context provided) as below:
{
"id": "urn:ngsi-ld:Device:MyFirstDevice",
"type": "https://uri.fiware.org/ns/data-models#Device",
"https://smart-data-models.github.io/data-models/terms.jsonld#/definitions/serialNumber": {
"type": "Property",
"value": "49206C6F766520796F752042617961"
}
}To get the entity in its compacted form, the GET request sent to the Context Broker must include the Smart Data Models context or any context defining the compacted term for each property.
If you need to change compacted terms for translation purposes for example, when requesting the Context Broker, you can provide a custom context defining the terms you want. See the example below for a context that maps the IRI of the initial property serialNumber to a new term NumeroSerie.
{
"@context": {
"NumeroSerie": "https://smart-data-models.github.io/data-models/terms.jsonld#/definitions/serialNumber"
}
}That being said, you are ready to create your first entity.
In this section, you will use the tool Postman to test the Context Broker API.
You will use the Context Broker service of the STF Core unified API to create your first entity in the Context Broker.
For CRUD (Create, Read, Update and Delete) operations on entities, the endpoint {StfCoreEndpoint}/ngsi-ld/v1/entities is used (See NGSI-LD API definition).
When using the POST method, you can provide the @context in the payload, as shown above, or in the HTTP headers:
Link: <https://raw.githubusercontent.com/smart-data-models/data-models/master/context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"
For the GET method, you can only use the HTTP headers to pass the context. In this section, you will use the HTTP headers to provide the context.
In Postman, select the POST method. Enter the {StfCoreEndpoint}/ngsi-ld/v1/entities endpoint.
In the tab Headers, provide the context as shown below:
In the tab Body, select raw. Then, in the drop-down menu, select JSON (instead of Text). Finally enter the payload below:
{
"id": "urn:ngsi-ld:Device:MyFirstDevice",
"type": "Device",
"serialNumber": {
"type": "Property",
"value": "49206C6F766520796F752042617961"
}
}As you can see, here you don't need to provide the @context as it is provided in the headers.
Then choose Send to create your first entity. The response code should be 201 if the operation succeeds.
You can check that the entity is stored by sending a GET request to the endpoint {StfCoreEndpoint}/ngsi-ld/v1/entities/urn:ngsi-ld:Device:MyFirstDevice
If you still provide the context in the header, the entity will be compacted.
You can now delete this entity by sending a DELETE request to the endpoint {StfCoreEndpoint}/ngsi-ld/v1/entities/urn:ngsi-ld:Device:MyFirstDevice
Now, you can start provision your things and ingesting data using the STF IoT Service API.
Implementing smart and efficient solutions using the STF requires some design considerations. While some of them will be discussed in this section, a whole section is dedicated to these design considerations.
In this section, you will use the STF IoT service of the STF Core unified API to register a thing in the central IoT registry and then check that its entity of type Device is stored in its shadow named Stf-Device and in the Context Broker. You will then explore the capabilities of the STF IoT API.
You use the tool Postman to send a POST request to the {StfCoreEndpoint}/iot/things endpoint with the following body:
{
"id": "urn:ngsi-ld:Device:SmartParking-BoschPLS-edd9ab00002022cc",
"type": "Device",
"thingGroups": {
"type": "Property",
"value": ["ParkingSpot", "LoRaWAN"]
},
"location": {
"type": "GeoProperty",
"value": {
"type": "Point",
"coordinates": [
3.1030936,
50.6584717
]
}
}
}The property thingGroups is not part of the data model and is removed before updating the Device Shadow. You can use this property to add your thing in up to 10 Thing Groups. In the example above, the thing is added to the groups ParkingSpot and LoRaWAN.
This will create a thing named SmartParking-BoschPLS-edd9ab00002022cc in the AWS IoT Core Things Registry and create or update its shadow named Stf-Device with the payload above.
Now you can check that this entity has been created in the Context Broker as well by sending a GET request to {StfCoreEndpoint}/ngsi-ld/v1/entities/urn:ngsi-ld:Device:SmartParking-BoschPLS-edd9ab00002022cc.
See Design consideration section to learn more about the naming conventions for things in the registry.
You can see in the screenshot above that the Fleet indexing status of the named shadow Stf-Device is indexed. Before version 1.1.O, Fleet Indexing of the named shadows Stf-Devicewas enabled by default so when you listed all the things registered in the registry, you also got their named shadow Stf-Device. Now, it is disabled by default, so querying the list would only give you the list of things registered and not their entities. You can get the list of all entities of type Device directly from the Context Broker with the request /ngsi-ld/v1/entities/?type=Device&limit=1000&offset=0. If you need Fleet Indexing of named shadows, you can set the property shadow_indexingto true in the file parameters.ts. Fleet Indexing will incur costs.
You can query {StfCoreEndpoint}/iot/things to get the list of all the things registered in the IoT Core Registry. If Fleet Indexing is enabled, this includes their named shadow Stf-Device in the property entity.
If Fleet Indexing is not enabled, the property entity will be an empty object.
With the single query {StfCoreEndpoint}/iot/things/{thingName}, you can also get all the entities associated to the thing.
Above, the array entities only contains the entity Stf-Device as this is the only one associated to the thing so far.
But when the sensor produces data, you will see all the entities associated in the array (see Building Data Producers and Data Consumers section).
You can filter the associated entities you want to retrieve with the parameter shadows and the type of the entities to retrieve separated with a comma.
You can for example only gets the entity of type ParkingSpot with the query GET {StfCoreEndpoint}/iot/things/{thingName}?shadows=ParkingSpot.
If you want to retrieve the entities of type Device and ParkingSpot, the query is GET {StfCoreEndpoint}/iot/things/{thingName}?shadows=ParkingSpot,Device.
Finally, with the STF IoT API you can also delete your things in the STF IoT Registry with the request DELETE {StfCoreEndpoint}/iot/things/{thingName}. The deletion of a thing in the STF IoT Registry will lead to the deletion of the associated entity of type Device in both the STF IoT Registry and the Context Broker but will keep its other associated entities. If you want to delete the things and all its associated entities in the STF IoT Registry and in the Context Broker, you can use the parameter recursive with the value true: DELETE {StfCoreEndpoint}/iot/things/{thingName}?recursive=true
The entities published into the STF IoT are stored in the STF IoT DataLake via Kinesis Data Firehose using Dynamic Partitioning. Dynamic partitioning enables you to continuously partition streaming data in Kinesis Data Firehose by using keys within data (for example the type of the entity) and then deliver the data grouped by these keys into corresponding Amazon Simple Storage Service (Amazon S3) prefixes.
Entities stored in the STF IoT Datalake are partitioned by type and time (yyyy-MM-dd-HH).
This makes it easier to run high performance, cost-efficient analytics on streaming data in Amazon S3 using various services such as Amazon Athena, Amazon EMR, Amazon Redshift Spectrum, and Amazon QuickSight.
The FIWARE Context Broker offers geospatial capabilities. You can perform geospatial queries to filter entities based on the distance of a geospatial point. For example, you can query all entities of specified type(s) that are located 200 meters radius from coordinates using the query:
GET {StfCoreEndpoint}/ngsi-ld/v1/entities?type=BikeHireDockingStation&georel=near;maxDistance==200&geometry=Point&coordinates=[-0.1094022, 51.5172398]
The query above is an example from a demo fetching every minute all bike docking stations and their status in London. The result of this query is a list a docking stations and their status that are located in less 200 meters from 60 Holborn Viaduct in London.
You can also subscribe to events (entities changes) filter by their geographical location using NGSI-LD Subscriptions.
NGSI-LD offers an asynchronous notification mechanism based on subscriptions. You can create subscriptions to be notified with a given frequency or when changes on data take place (e.g., an air quality measurement is above a specified threshold value).
{
"id": "urn:ngsi-ld:Subscription:mySubscription",
"type": "Subscription",
"entities": [
{
"type": "Vehicle"
}
],
"watchedAttributes": ["speed"],
"q": "speed>50",
"geoQ": {
"georel": "near;maxDistance==2000",
"geometry": "Point",
"coordinates": [-0.1094022, 51.5172398]
},
"notification": {
"attributes": ["speed"],
"format": "keyValues",
"endpoint": {
"uri": "http://my.endpoint.org/notify",
"accept": "application/json"
}
},
"@context": [
"http://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld",
"http://example.org/ngsi-ld/vehicle.jsonld"
]
}In the example above, the subscription will trigger a notification every time an entity of type Vehicle located in 2000 meters radius of the specified location is measured with a speed greater than 50. Every time the speed (watched Attribute) of a concerned vehicle, changes, a new notification (including the new speed value) will be received in the specified endpoint.
You are ready now to start building your data producers and data consumers.
When using the STF, Data Producers are decoupled from Data Consumers. You can build each of both independently as the information model is standardized and known in advance.
You can for example build a Grafana dashboard to visualise your data using Amazon Athena to query your IoT Data Lake.
You can build your Data Producers within hours as the only process needed to integrate a new source of data, is to transform the source data format into the data model relevant to the use case before sending the data to the STF IoT module.
Same for Data Consumers. the STF Core offers multiple ways to consume the data: directly from the Context Broker, from the STF IoT Registry or from the STF IoT Data Lake. You can also subscribe to all changes of named shadows making it easy to build real-time applications. Finally you can create subscriptions in the Context Broker to notify independent systems and applications
You can combine all theses capabilities to build comprehensive and scalable solutions easily, iterate and innovate at a fast pace.
The following video guides you through the implementation of a comprehensive solution using the STF including Data Producers and Data Consumers.
Below a reference architecture with examples of Data Producers and Data Consumers using AWS services.
For each Data Producer, it is about using the right tool to ingest the data and route it to a Lambda function that will transform the source data format into an NGSI-LD entity before publishing it into the STF IoT Queue.
For example, the steps to build a Data Producer that ingests data from LoRaWAN Parking Lot sensors using a private LoRaWAN Network managed by AWS IoT Core for LoRaWAN are:
- Register your parking lot sensors in the central registry using the STF IoT Service API following the naming conventions.
- Onboard your parking lot sensors to AWS IoT Core for LoRaWAN.
- Create a Lambda function that will decode the payload and transform it into an NGSI-LD entity and publish it into the STF IoT Queue.
- Assign a LoRaWAN destination to your LoRaWAN Parking lot sensors. The destination will route the messages to an AWS IoT Rule that will route the messages to the Lambda Function.
Below an example of the code of a Lambda function that decodes and transforms the payload received from AWS IoT Core for LoRaWAN for the Bosch PLS sensor:
const STF_IOT_SQS_URL = process.env.STF_IOT_SQS_URL
const THING_PREFIX = process.env.THING_PREFIX
const aws = require('aws-sdk')
const sqs = new aws.SQS({})
exports.handler = async (event) => {
try {
const { PayloadData, WirelessMetadata: { LoRaWAN : {FPort, DevEui, Timestamp} }} = event
const bf = Buffer.from(PayloadData, 'base64')
console.log(`From ${DevEui}: ${bf.toString('hex')}`)
let entity = {}
let thingName
entity.type = `ParkingSpot`
entity.id = `urn:ngsi-ld:${entity.type}:${THING_PREFIX}-${DevEui}`
entity.status = {
type: "Property",
observedAt: Timestamp,
providedBy: {
type: 'Relationship',
object: `urn:ngsi-ld:Device:${THING_PREFIX}-${DevEui}`
}
}
switch (FPort) {
//Uplink message
case 1:
entity.status.value = bf.readUInt8(0) == 0 ? "free" : "occupied"
break
//Heartbeat message
case 2:
entity.status.value = bf.readUInt8(0) == 0 ? "free" : "occupied"
break
//Startup message
case 3:
entity.status.value = bf.readUInt8(16) == 0 ? "free" : "occupied"
break
default:
break
}
let send = await sqs.sendMessage({
QueueUrl: STF_IOT_SQS_URL,
MessageBody: JSON.stringify(entity)
}).promise()
} catch (e) {
console.log(e)
}
}You can find a list of Data Producers and Data Consumers in the STF Catalog on GitHub.
The STF is designed to be flexible and efficient. It uses the following design guidelines to provide advanced device management capabilities and efficient data ingestion using naming conventions.
The STF IoT module provides a central registry in which you can store all your devices and sensors, regardless the operating model, the technology and the connectivity used. For all types of sensors and devices, the first step is to register a thing using the STF IoT service API.
To link the data produced by a device to its thing in the registry, there are some naming conventions to follow.
First, we recommend using a prefix containing the application (or the type) and the device or sensor model. This enables easy filtering and searching your index. You will be able for example to create an IoT Rule that only listens to changes of shadows of things which have a name starting with a specific prefix.
To continue with our Smart Parking example, using the LoRaWAN Parking lot sensor Bosch PLS, the prefix would be SmartParking-BoschPLS-.
Then, what comes after the prefix must be a unique id that enables the link between the incoming messages and the thing. For example, for LoRaWAN devices, we recommend using the DevEUI of the device (as it is part of the uplink message received in AWS IoT Core for LoRaWAN). So for the example above, the name of these things in the registry would be SmartParking-BoschPLS-{DevEUI}.
When receiving messages from these sensors, the Lambda Function in charge of creating the entity of type ParkingSpot only needs to add the static prefix SmartParking-BoschPLS- to the DevEUI to get the name of the thing that it needs to update the device shadow.
Depending the connectivity and the operating model, you will need to onboard your device to the service or platform used.
For example if using AWS IoT Core for LoRaWAN, you will need to onboard your LoRaWAN device to AWS IoT Core for LoRaWAN in addition to the central registry.
If using AWS IoT Core to connect using MQTT with a X.509 certificate, then you will need to attach a certificate and a policy to the thing registered in the IoT registry.
NGSI-LD entitites are digital representations of real-world objects having Properties and Relationships with other entities. Each entity is uniquely identified by an id which is the URI of the entity and characterized, by reference in the field type, to one NGSI-LD Entity Type.
These two fields id and type are mandatory.
Using the STF, the id of an entity must follow this pattern urn:ngsi-ld:{Type}:{thingName} in which {Type} is the type of the entity and {thingName} is the name of the thing registered in the STF IoT central registry (using the AWS IoT Core Things registry). The type must be the name of the Data Model used.
For example, for a thing named SmartParking-BoschPLS-edd9ab00002022cc, the entity that defines the attributes of this device will be:
{
"id": "urn:ngsi-ld:Device:SmartParking-BoschPLS-edd9ab00002022cc",
"type": "Device",
"location": {
"type": "GeoProperty",
"value": {
"type": "Point",
"coordinates": [
3.1030936,
50.6584717
]
}
}
}
When publishing an entity into the STF IoT, a mechanism will extract the thingName from the id and create or update the named shadow Stf-{type} of the thing.
In the example above, a shadow named Stf-Device will be created or updated for the thing named SmartParking-BoschPLS-edd9ab00002022cc.
This shadow contains the entity urn:ngsi-ld:Device:SmartParking-BoschPLS-edd9ab00002022cc as shown above in its reported state.
When this device sends its measurements (parking space availability in that case), it will publish an entity of type ParkingSpot into the STF IoT.
{
"id": "urn:ngsi-ld:ParkingSpot:SmartParking-BoschPLS-edd9ab00002022cc",
"type": "ParkingSpot",
"status": {
"type": "Property",
"value": "free",
"observedAt": "2022-03-15T12:05:02Z",
"providedBy": {
"type": "Relationship",
"object": "urn:ngsi-ld:Device:SmartParking-BoschPLS-edd9ab00002022cc"
}
}
}This will update or create a shadow named Stf-ParkingSpot.
In the example above, you see that there is no @context provided when interacting with the STF IoT module. The Smart Data Models context will be added before updating the Context Broker.
The Device Shadow service uses reserved MQTT topics to enable applications and devices to get, update, or delete the state information for a device (shadow). Any change of a Device Shadow is published in these reserved MQTT Topics.
An AWS IoT Rule subscribes to all these changes and route them to an AWS Lambda function that updates the Context Broker accordingly.
Below the IoT Rule that listens to changes in all named shadows starting with the prefix Stf and retrieves the shadow document:
SELECT current.state.reported.*
FROM '$aws/things/+/shadow/name/+/update/documents'
WHERE startswith(topic(6), 'Stf') AND NOT isUndefined(current.state.reported.type)You can subscribe only to changes or retrieve the entire shadow document. The document contains all the properties even those that have not been updated.
In that case we retrieve the entire document to get the property location even if it is not part of the incoming message. It saves the API call needed to enrich the message with this property.
Although optional for an NGSI-LD entity, the property location is mandatory when registering a Device in the STF central registry.
The STF IoT provides a mechanism to enrich with the property location every message or payload coming from a device without any API call.
Indeed, most static devices might not send the information about their location in the payload. That is the case for example for LoRaWAN parking lot sensors.
However, a data consumer like a mobile application for parking space availability needs this information as well as the availability of parking spots (which would be stored in entities of type ParkingSpot).
In the Lambda function that updates the Context Broker, there is a mechanism that checks for every entities that are not of type Device if the property location is present.
If not it retrieves the property from the shadow Stf-Device of the thing, enriches the message and updates the shadow Stf-{type} with the property.
As we retrieve the full shadow document, next time this entity will have the property location.
In other words, when the device SmartParking-BoschPLS-edd9ab00002022cc sends its availability for the first time, it updates its shadow Stf-parkingSpot without providing the property location. When the message is routed to the Lambda Function that updates the Context Broker, the mechanism described above retrieves the property location from its shadow named Stf-Device, enriches the entity of type ParkingSpot with the location and updates the shadow named Stf-ParkingSpot.
This triggers another change in the shadow Stf-ParkingSpot that is routed to the same function but from now, the property location will always be provided as it is part of the shadow document.
Data consumers consuming the ParkingSpot entities from the Context Broker or from the STF IoT Data Lake won't need any additional request to get the location of the parking spot.
You can find the code of the mechanism described abobe in the Lambda Function that updates the Context broker.
This section provides links to additional resources that may help implement solutions based on the Smart Territory Framework:
- FIWARE NGSI-LD Tutorial
- JSON-LD, NGSI-LD, Digital Twins and Smart Data Models
- Smart Data Models
- Scorpio Broker Doc
- Orion-LD Doc
- AWS CDK Workshop
- AWS IoT Hero Workshop
- Check issues in Scorpio Broker repository
- View your CloudWatch Logs data in the CloudWatch console.
- Elastic IP address limit
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.