smart-city-digital-twin-martinique/references/data-models.md

# Smart City Digital Twin - Data Models & Schemas

## Vue d'ensemble

Ce document décrit les schémas de données utilisés par les différents brokers (MQTT, Fiware) avec les champs de traçabilité.

## 🔍 Champs de traçabilité (ajoutés le 05-05-2026)

| Champ | Type NGSI-LD | Type SensorThings | Description |
|-------|----------------|---------------------|-------------|
| `source` | Property (valeur: string) | Thing.properties (valeur: string) | Broker MQTT source (ex: "EMQX", "Mosquitto", "BunkerM", "simulator") |
| `mqttTopic` | Property (valeur: string) | Thing.properties (valeur: string) | Topic MQTT d'origine (ex: "city/sensors/airquality/airquality_000") |

**Note** : `source` est un champ standard NGSI-LD (ETSI). `mqttTopic` est une propriété personnalisée (autorisée par l'extension NGSI-LD).

---

## 1. NGSI-LD Entities (Orion-LD, Stellio)

### Structure de base (avec traçabilité)

```json
{
  "@context": [
    "https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld"
  ],
  "id": "urn:ngsi-ld:AirQualityObserved:airquality_000",
  "type": "AirQualityObserved",
  "dateObserved": {"type": "Property", "value": "2026-05-05T02:00:00Z"},
  "location": {"type": "GeoProperty", "value": {"type": "Point", "coordinates": [-61.0, 14.6]}},
  "name": {"type": "Property", "value": "Port de Fort-de-France"},
  "batteryLevel": {"type": "Property", "value": 85},
  "source": {"type": "Property", "value": "EMQX"},
  "mqttTopic": {"type": "Property", "value": "city/sensors/airquality/airquality_000"}
}
```

### Mapping Smart Data Models → NGSI-LD Types

| Sensor Type | NGSI-LD Type | Exemple ID |
|-------------|--------------|----------|
| airquality | AirQualityObserved | `urn:ngsi-ld:AirQualityObserved:airquality_000` |
| traffic | TrafficFlowObserved | `urn:ngsi-ld:TrafficFlowObserved:traffic_000` |
| parking | ParkingSpotObserved | `urn:ngsi-ld:ParkingSpotObserved:parking_000` |
| noise | NoiseLevelObserved | `urn:ngsi-ld:NoiseLevelObserved:noise_000` |
| weather | WeatherObserved | `urn:ngsi-ld:WeatherObserved:weather_000` |
| light | LightObserved | `urn:ngsi-ld:LightObserved:light_000` |

---

## 2. SensorThings API (FROST-Server)

### Thing Properties (avec traçabilité)

```json
{
  "name": "Thing_airquality_000",
  "description": "Smart City airquality sensor in Martinique",
  "properties": {
    "sensorType": "airquality",
    "region": "Martinique",
    "source": "EMQX",
    "mqttTopic": "city/sensors/airquality/airquality_000"
  }
}
```

### Datastream Structure

```json
{
  "name": "Datastream airquality/pm25_ugm3",
  "description": "Datastream for airquality sensor airquality_000 - pm25_ugm3",
  "observationType": "http://www.opengis.net/def/observationType/OGC-OM/2.0/OM_Measurement",
  "unitOfMeasurement": {"name": "pm25_ugm3", "symbol": "µg/m³", "definition": "http://www.qudt.org/vocab/unit#MicrogramPerCubicMeter"},
  "Sensor": {...},
  "ObservedProperty": {...}
}
```

---

## 3. MQTT Topics (Brokers)

### Format des topics

```
city/sensors/{sensor_type}/{sensor_id}
```

Exemples :
- `city/sensors/airquality/airquality_000`
- `city/sensors/traffic/traffic_000`
- `city/sensors/parking/parking_000`

### Mapping Topic → Entity ID

| MQTT Topic | NGSI-LD Entity ID | SensorThings Thing ID |
|------------|-------------------|------------------------|
| `city/sensors/airquality/airquality_000` | `urn:ngsi-ld:AirQualityObserved:airquality_000` | `Thing_airquality_000` |
| `city/sensors/traffic/traffic_000` | `urn:ngsi-ld:TrafficFlowObserved:traffic_000` | `Thing_traffic_000` |

---

## 4. InfluxDB (Bucket: iot_data)

### Measurements (v2 Flux)

- `airquality`
- `traffic`
- `parking`
- `noise`
- `weather`
- `light`

### Tags
- `location`: Nom du lieu (ex: "Port de Fort-de-France")
- `sensor_id`: ID du capteur (ex: "airquality_000")

### Fields
- Valeurs spécifiques au type (ex: `pm25_ugm3`, `co_mgm3`, `vehicle_count`, etc.)

---

## 5. Procédure de mise à jour des payloads

⚠️ **CHAQUE FOIS** que les payloads sont modifiés (ajout/suppression de champs) :

1. **Nettoyer les bases** :
   ```bash
   # Orion-LD
   curl -s "http://localhost:2026/ngsi-ld/v1/entities?type=AirQualityObserved&limit=1000" | python3 -c "..."
   # (Supprimer chaque entité)
   
   # Stellio
   # Via API Stellio (http://localhost:8087/ngsi-ld/v1/entities)
   
   # FROST
   # Via API FROST (http://localhost:8086/FROST-Server/v1.1/Things)
   ```

2. **Mettre à jour ce document** (`references/data-models.md`)

3. **Tester** avec un simulateur frais (entities recréées)

4. **Committer et pousser** vers Gitea :
   ```bash
   git add references/data-models.md simulator.py
   git commit -m "Update data models: add/remove fields"
   git push origin master
   ```

---

## 6. Identification des messages par broker MQTT

Pour identifier quels messages ont été reçus par chaque broker Fiware :

### Requête Orion-LD (source)
```bash
curl -s "http://localhost:2026/ngsi-ld/v1/entities?type=AirQualityObserved&limit=10" \
  -H "Accept: application/ld+json" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for e in data:
    print(f\"{e['id']} | source: {e.get('source', {}).get('value')} | topic: {e.get('mqttTopic', {}).get('value')}\")
"
```

### Requête Stellio (source)
```bash
curl -s "http://localhost:8087/ngsi-ld/v1/entities?type=AirQualityObserved&limit=10" \
  -H "Accept: application/ld+json" -H "NGSI-LD-Tenant: urn:ngsi-ld:tenant:default" | ...
```

### Requête FROST (Thing.properties)
```bash
curl -s "http://localhost:8086/FROST-Server/v1.1/Things" | python3 -c "
import sys, json
data = json.load(sys.stdin)
for t in data.get('value', []):
    props = t.get('properties', {})
    print(f\"{t['name']} | source: {props.get('source')} | topic: {props.get('mqttTopic')}\")
"
```

---

**Dernière mise à jour** : 05 Mai 2026  
**Auteur** : Smart City Digital Twin Team  
**Version** : 1.1 (avec traçabilité source/mqttTopic)