From e0bf96b9c303beae54b00ae3bd2733fa72c2c36d Mon Sep 17 00:00:00 2001 From: Eric FELIXINE Date: Mon, 4 May 2026 22:39:27 -0400 Subject: [PATCH] =?UTF-8?q?Docs:=20Ajout=20r=C3=A9f=C3=A9rentiel=20data-mo?= =?UTF-8?q?dels=20(source/mqttTopic)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- references/data-models.md | 191 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 191 insertions(+) create mode 100644 references/data-models.md diff --git a/references/data-models.md b/references/data-models.md new file mode 100644 index 00000000..b9ca025e --- /dev/null +++ b/references/data-models.md @@ -0,0 +1,191 @@ +# Smart City Digital Twin - Data Models & Schemas + +## Vue d'ensemble + +Ce document décrit les schémas de données utilisés par les différents brokers (MQTT, Fiware) avec les champs de traçabilité. + +## 🔍 Champs de traçabilité (ajoutés le 05-05-2026) + +| Champ | Type NGSI-LD | Type SensorThings | Description | +|-------|----------------|---------------------|-------------| +| `source` | Property (valeur: string) | Thing.properties (valeur: string) | Broker MQTT source (ex: "EMQX", "Mosquitto", "BunkerM", "simulator") | +| `mqttTopic` | Property (valeur: string) | Thing.properties (valeur: string) | Topic MQTT d'origine (ex: "city/sensors/airquality/airquality_000") | + +**Note** : `source` est un champ standard NGSI-LD (ETSI). `mqttTopic` est une propriété personnalisée (autorisée par l'extension NGSI-LD). + +--- + +## 1. NGSI-LD Entities (Orion-LD, Stellio) + +### Structure de base (avec traçabilité) + +```json +{ + "@context": [ + "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld" + ], + "id": "urn:ngsi-ld:AirQualityObserved:airquality_000", + "type": "AirQualityObserved", + "dateObserved": {"type": "Property", "value": "2026-05-05T02:00:00Z"}, + "location": {"type": "GeoProperty", "value": {"type": "Point", "coordinates": [-61.0, 14.6]}}, + "name": {"type": "Property", "value": "Port de Fort-de-France"}, + "batteryLevel": {"type": "Property", "value": 85}, + "source": {"type": "Property", "value": "EMQX"}, + "mqttTopic": {"type": "Property", "value": "city/sensors/airquality/airquality_000"} +} +``` + +### Mapping Smart Data Models → NGSI-LD Types + +| Sensor Type | NGSI-LD Type | Exemple ID | +|-------------|--------------|----------| +| airquality | AirQualityObserved | `urn:ngsi-ld:AirQualityObserved:airquality_000` | +| traffic | TrafficFlowObserved | `urn:ngsi-ld:TrafficFlowObserved:traffic_000` | +| parking | ParkingSpotObserved | `urn:ngsi-ld:ParkingSpotObserved:parking_000` | +| noise | NoiseLevelObserved | `urn:ngsi-ld:NoiseLevelObserved:noise_000` | +| weather | WeatherObserved | `urn:ngsi-ld:WeatherObserved:weather_000` | +| light | LightObserved | `urn:ngsi-ld:LightObserved:light_000` | + +--- + +## 2. SensorThings API (FROST-Server) + +### Thing Properties (avec traçabilité) + +```json +{ + "name": "Thing_airquality_000", + "description": "Smart City airquality sensor in Martinique", + "properties": { + "sensorType": "airquality", + "region": "Martinique", + "source": "EMQX", + "mqttTopic": "city/sensors/airquality/airquality_000" + } +} +``` + +### Datastream Structure + +```json +{ + "name": "Datastream airquality/pm25_ugm3", + "description": "Datastream for airquality sensor airquality_000 - pm25_ugm3", + "observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement", + "unitOfMeasurement": {"name": "pm25_ugm3", "symbol": "µg/m³", "definition": "http://www.qudt.org/vocab/unit#MicrogramPerCubicMeter"}, + "Sensor": {...}, + "ObservedProperty": {...} +} +``` + +--- + +## 3. MQTT Topics (Brokers) + +### Format des topics + +``` +city/sensors/{sensor_type}/{sensor_id} +``` + +Exemples : +- `city/sensors/airquality/airquality_000` +- `city/sensors/traffic/traffic_000` +- `city/sensors/parking/parking_000` + +### Mapping Topic → Entity ID + +| MQTT Topic | NGSI-LD Entity ID | SensorThings Thing ID | +|------------|-------------------|------------------------| +| `city/sensors/airquality/airquality_000` | `urn:ngsi-ld:AirQualityObserved:airquality_000` | `Thing_airquality_000` | +| `city/sensors/traffic/traffic_000` | `urn:ngsi-ld:TrafficFlowObserved:traffic_000` | `Thing_traffic_000` | + +--- + +## 4. InfluxDB (Bucket: iot_data) + +### Measurements (v2 Flux) + +- `airquality` +- `traffic` +- `parking` +- `noise` +- `weather` +- `light` + +### Tags +- `location`: Nom du lieu (ex: "Port de Fort-de-France") +- `sensor_id`: ID du capteur (ex: "airquality_000") + +### Fields +- Valeurs spécifiques au type (ex: `pm25_ugm3`, `co_mgm3`, `vehicle_count`, etc.) + +--- + +## 5. Procédure de mise à jour des payloads + +⚠️ **CHAQUE FOIS** que les payloads sont modifiés (ajout/suppression de champs) : + +1. **Nettoyer les bases** : + ```bash + # Orion-LD + curl -s "http://localhost:2026/ngsi-ld/v1/entities?type=AirQualityObserved&limit=1000" | python3 -c "..." + # (Supprimer chaque entité) + + # Stellio + # Via API Stellio (http://localhost:8087/ngsi-ld/v1/entities) + + # FROST + # Via API FROST (http://localhost:8086/FROST-Server/v1.1/Things) + ``` + +2. **Mettre à jour ce document** (`references/data-models.md`) + +3. **Tester** avec un simulateur frais (entities recréées) + +4. **Committer et pousser** vers Gitea : + ```bash + git add references/data-models.md simulator.py + git commit -m "Update data models: add/remove fields" + git push origin master + ``` + +--- + +## 6. Identification des messages par broker MQTT + +Pour identifier quels messages ont été reçus par chaque broker Fiware : + +### Requête Orion-LD (source) +```bash +curl -s "http://localhost:2026/ngsi-ld/v1/entities?type=AirQualityObserved&limit=10" \ + -H "Accept: application/ld+json" | python3 -c " +import sys, json +data = json.load(sys.stdin) +for e in data: + print(f\"{e['id']} | source: {e.get('source', {}).get('value')} | topic: {e.get('mqttTopic', {}).get('value')}\") +" +``` + +### Requête Stellio (source) +```bash +curl -s "http://localhost:8087/ngsi-ld/v1/entities?type=AirQualityObserved&limit=10" \ + -H "Accept: application/ld+json" -H "NGSI-LD-Tenant: urn:ngsi-ld:tenant:default" | ... +``` + +### Requête FROST (Thing.properties) +```bash +curl -s "http://localhost:8086/FROST-Server/v1.1/Things" | python3 -c " +import sys, json +data = json.load(sys.stdin) +for t in data.get('value', []): + props = t.get('properties', {}) + print(f\"{t['name']} | source: {props.get('source')} | topic: {props.get('mqttTopic')}\") +" +``` + +--- + +**Dernière mise à jour** : 05 Mai 2026 +**Auteur** : Smart City Digital Twin Team +**Version** : 1.1 (avec traçabilité source/mqttTopic)