Add data flow diagram (Mermaid MD, HTML, PDF) for Smart City Digital Twin
This commit is contained in:
Eric FELIXINE
191
data-flow-diagram.md
Normal file
191
data-flow-diagram.md
Normal file
@@ -0,0 +1,191 @@
|
||||
# Smart City Digital Twin — Diagramme des Flux de Données
|
||||
|
||||
## Vue d'ensemble
|
||||
|
||||
Ce diagramme illustre le flux complet des données IoT du simulateur vers les différentes couches de traitement, de stockage et de visualisation.
|
||||
|
||||
---
|
||||
|
||||
## Diagramme Mermaid
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
%% ===== COUCHE 1 : SOURCES DE DONNÉES =====
|
||||
subgraph Sources["📡 COUCHE 1 : SOURCES DE DONNÉES"]
|
||||
SIM[Smart City Simulator<br/>Python • 10 capteurs<br/>MQTT + REST API]
|
||||
end
|
||||
|
||||
%% ===== COUCHE 2 : MQTT BROKERS =====
|
||||
subgraph Brokers["📡 COUCHE 2 : MQTT BROKERS"]
|
||||
EMQX[EMQX<br/>Port 11883<br/>MQTT]
|
||||
MOSQ[Mosquitto<br/>Port 1883<br/>MQTT]
|
||||
BUNKER[BunkerM<br/>Port 1900<br/>MQTTS]
|
||||
end
|
||||
|
||||
%% ===== COUCHE 3 : CONTEXT BROKERS (NGSI-LD) =====
|
||||
subgraph Context["🔄 COUCHE 3 : CONTEXT BROKERS (NGSI-LD)"]
|
||||
ORION[Orion-LD<br/>NGSI-LD<br/>Port 1026<br/>10 entités]
|
||||
STELLIO[Stellio<br/>NGSI-LD<br/>Port 8080<br/>14 payloads]
|
||||
FROST[FROST-Server<br/>SensorThings API<br/>Port 8080<br/>21k+ observations]
|
||||
end
|
||||
|
||||
%% ===== COUCHE 4 : PLATEFORME IoT =====
|
||||
subgraph IoT["🏠 COUCHE 4 : PLATEFORME IoT (OpenRemote)"]
|
||||
OR[OpenRemote Manager<br/>Realm: Smart City<br/>33 assets IoT<br/>Port 8080]
|
||||
KC[Keycloak<br/>Auth OpenID<br/>Token endpoint]
|
||||
end
|
||||
|
||||
%% ===== COUCHE 5 : STOCKAGE & MÉTRIQUES =====
|
||||
subgraph Storage["💾 COUCHE 5 : STOCKAGE & MÉTRIQUES"]
|
||||
INFLUX[InfluxDB<br/>IoT Data Bucket<br/>Port 8086]
|
||||
PROM[Prometheus<br/>Metrics + Alerting<br/>Port 9090]
|
||||
GEO[GeoServer<br/>WMS/WFS + PostGIS<br/>Port 8080]
|
||||
end
|
||||
|
||||
%% ===== COUCHE 6 : VISUALISATION & ANALYSE =====
|
||||
subgraph Viz["📊 COUCHE 6 : VISUALISATION & ANALYSE"]
|
||||
GRAF[Grafana<br/>Dashboards<br/>Port 3001<br/>Datasources: InfluxDB, FROST, Orion]
|
||||
MAP[MapStore<br/>Cartographie<br/>WMS/WFS<br/>GeoServer]
|
||||
ORUI[OpenRemote UI<br/>Manager Interface<br/>Realm: Smart City]
|
||||
end
|
||||
|
||||
%% ===== FLUX DE DONNÉES =====
|
||||
|
||||
%% Simulator → MQTT Brokers
|
||||
SIM -->|MQTT| EMQX
|
||||
SIM -->|MQTT| MOSQ
|
||||
SIM -->|MQTTS| BUNKER
|
||||
|
||||
%% MQTT Brokers → Context Brokers
|
||||
EMQX -->|NGSI-LD| ORION
|
||||
EMQX -->|NGSI-LD| STELLIO
|
||||
EMQX -->|SensorThings| FROST
|
||||
|
||||
%% Simulator → OpenRemote (REST)
|
||||
SIM -.->|REST API| OR
|
||||
OR -.-> KC
|
||||
|
||||
%% Simulator → InfluxDB
|
||||
SIM -->|HTTP| INFLUX
|
||||
|
||||
%% Context Brokers → Visualization
|
||||
ORION -->|Query| GRAF
|
||||
STELLIO -->|Query| GRAF
|
||||
FROST -->|Query| GRAF
|
||||
|
||||
%% Context Brokers → GeoServer (DB Sync)
|
||||
ORION -.->|PostGIS| GEO
|
||||
STELLIO -.->|PostGIS| GEO
|
||||
FROST -.->|PostGIS| GEO
|
||||
|
||||
%% GeoServer → MapStore
|
||||
GEO -->|WMS/WFS| MAP
|
||||
|
||||
%% OpenRemote → Grafana
|
||||
OR -->|API Query| GRAF
|
||||
|
||||
%% All → Prometheus (Metrics)
|
||||
EMQX -.->|Metrics| PROM
|
||||
ORION -.->|Metrics| PROM
|
||||
STELLIO -.->|Metrics| PROM
|
||||
OR -.->|Metrics| PROM
|
||||
|
||||
%% Style
|
||||
classDef simulator fill:#1e293b,stroke:#fb923c,stroke-width:2px,color:#fff
|
||||
classDef broker fill:#0f172a,stroke:#22d3ee,stroke-width:2px,color:#fff
|
||||
classDef context fill:#0f172a,stroke:#34d399,stroke-width:2px,color:#fff
|
||||
classDef iot fill:#0f172a,stroke:#a78bfa,stroke-width:2px,color:#fff
|
||||
classDef storage fill:#0f172a,stroke:#fbbf24,stroke-width:2px,color:#fff
|
||||
classDef viz fill:#0f172a,stroke:#22d3ee,stroke-width:2px,color:#fff
|
||||
|
||||
class SIM simulator
|
||||
class EMQX,MOSQ,BUNKER broker
|
||||
class ORION,STELLIO,FROST context
|
||||
class OR,KC iot
|
||||
class INFLUX,PROM,GEO storage
|
||||
class GRAF,MAP,ORUI viz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Description des flux
|
||||
|
||||
### 1. **Génération des données (Simulator)**
|
||||
- **Smart City Simulator** (Python) génère des données pour 10 capteurs (Traffic, Air Quality, Parking, Noise, Weather, Light)
|
||||
- Intervalle de publication : 10 secondes
|
||||
- Protocoles : MQTT + REST API
|
||||
|
||||
### 2. **Ingestion MQTT (Brokers)**
|
||||
- **EMQX** (port 11883) : Broker public, reçoit tous les capteurs
|
||||
- **Mosquitto** (port 1883) : Via Traefik, accès externe
|
||||
- **BunkerM** (port 1900) : MQTTS (TLS), accès sécurisé
|
||||
|
||||
### 3. **Context Brokers (NGSI-LD & SensorThings)**
|
||||
- **Orion-LD** : Reçoit les données au format NGSI-LD
|
||||
- 10 entités (TrafficFlowObserved, AirQualityObserved, etc.)
|
||||
- Smart Data Models utilisés
|
||||
- **Stellio** : Alternative NGSI-LD
|
||||
- 14 payloads entités
|
||||
- Contexte : `https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld`
|
||||
- **FROST-Server** : SensorThings API
|
||||
- 21 256+ observations
|
||||
- PostgreSQL + TimescaleDB
|
||||
|
||||
### 4. **Plateforme IoT (OpenRemote)**
|
||||
- **OpenRemote Manager** (realm `smartcity`)
|
||||
- 33 assets IoT configurés
|
||||
- Carte Martinique (mapsettings.json)
|
||||
- **Keycloak** : Authentification OpenID Connect
|
||||
- Client `openremote` avec Service Account
|
||||
- Token endpoint : `/auth/realms/smartcity/protocol/openid-connect/token`
|
||||
|
||||
### 5. **Stockage & Métriques**
|
||||
- **InfluxDB** : Stockage temporel pour Grafana
|
||||
- Bucket : `iot_data`
|
||||
- Datasource dans Grafana
|
||||
- **Prometheus** : Collecte des métriques
|
||||
- MQTT brokers, Context brokers, OpenRemote
|
||||
- **GeoServer** : Données géospatiales
|
||||
- PostGIS pour centralisation
|
||||
- WMS/WFS pour MapStore
|
||||
|
||||
### 6. **Visualisation & Analyse**
|
||||
- **Grafana** (port 3001)
|
||||
- Dashboard : `smartcity-martinique-2026`
|
||||
- Datasources : InfluxDB, FROST, Orion-LD
|
||||
- **MapStore** : Cartographie
|
||||
- Sources WMS/WFS depuis GeoServer
|
||||
- **OpenRemote UI** : Manager Interface
|
||||
- Visualisation des assets realm Smart City
|
||||
|
||||
---
|
||||
|
||||
## Technologies clés
|
||||
|
||||
| Composant | Technologie | Port | Statut |
|
||||
|-----------|-------------|------|--------|
|
||||
| Simulator | Python + paho-mqtt | Interne | ✅ Actif |
|
||||
| EMQX | MQTT Broker | 11883 | ✅ Connecté |
|
||||
| Orion-LD | NGSI-LD Broker | 1026 | ⚠️ À vérifier |
|
||||
| Stellio | NGSI-LD Broker | 8080 | ⚠️ À vérifier |
|
||||
| FROST-Server | SensorThings API | 8080 | ⚠️ À vérifier |
|
||||
| OpenRemote | IoT Platform | 8080 | ⚠️ 403 (Service Account) |
|
||||
| InfluxDB | Time Series DB | 8086 | ✅ Configuré |
|
||||
| Grafana | Visualization | 3001 | ✅ Dashboard créé |
|
||||
| GeoServer | GeoServer | 8080 | ⚠️ À intégrer |
|
||||
| Prometheus | Metrics | 9090 | ✅ En cours |
|
||||
|
||||
---
|
||||
|
||||
## Fichiers associés
|
||||
|
||||
- **Simulator** : `~/smart-city-digital-twin-martinique/simulator.py`
|
||||
- **Dashboard Grafana** : `~/smart-city-digital-twin-martinique/grafana_dashboard_smartcity.json`
|
||||
- **Ce diagramme** : `~/smart-city-digital-twin-martinique/data-flow-diagram.md`
|
||||
- **Session Resume** : `~/smart-city-digital-twin-martinique/session_resume_2026-05-04.md`
|
||||
|
||||
---
|
||||
|
||||
**Dernière mise à jour :** 04 Mai 2026
|
||||
**Projet :** Smart City Digital Twin Martinique
|
||||
**URL Grafana :** http://localhost:3001/d/smartcity-martinique-2026
|
||||
Reference in New Issue
Block a user