docs: Update data-flow-diagram with IoT-Agent, QuantumLeap, CrateDB architecture (2026-05-06)
This commit is contained in:
@@ -1,380 +1,270 @@
|
||||
<!DOCTYPE html>
|
||||
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="generator" content="pandoc" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
|
||||
<title>data-flow-diagram</title>
|
||||
<style>
|
||||
/* Default styles provided by pandoc.
|
||||
** See https://pandoc.org/MANUAL.html#variables-for-html for config info.
|
||||
*/
|
||||
html {
|
||||
color: #1a1a1a;
|
||||
background-color: #fdfdfd;
|
||||
}
|
||||
body {
|
||||
margin: 0 auto;
|
||||
max-width: 36em;
|
||||
padding-left: 50px;
|
||||
padding-right: 50px;
|
||||
padding-top: 50px;
|
||||
padding-bottom: 50px;
|
||||
hyphens: auto;
|
||||
overflow-wrap: break-word;
|
||||
text-rendering: optimizeLegibility;
|
||||
font-kerning: normal;
|
||||
}
|
||||
@media (max-width: 600px) {
|
||||
body {
|
||||
font-size: 0.9em;
|
||||
padding: 12px;
|
||||
}
|
||||
h1 {
|
||||
font-size: 1.8em;
|
||||
}
|
||||
}
|
||||
@media print {
|
||||
html {
|
||||
background-color: white;
|
||||
}
|
||||
body {
|
||||
background-color: transparent;
|
||||
color: black;
|
||||
font-size: 12pt;
|
||||
}
|
||||
p, h2, h3 {
|
||||
orphans: 3;
|
||||
widows: 3;
|
||||
}
|
||||
h2, h3, h4 {
|
||||
page-break-after: avoid;
|
||||
}
|
||||
}
|
||||
p {
|
||||
margin: 1em 0;
|
||||
}
|
||||
a {
|
||||
color: #1a1a1a;
|
||||
}
|
||||
a:visited {
|
||||
color: #1a1a1a;
|
||||
}
|
||||
img {
|
||||
max-width: 100%;
|
||||
}
|
||||
svg {
|
||||
height: auto;
|
||||
max-width: 100%;
|
||||
}
|
||||
h1, h2, h3, h4, h5, h6 {
|
||||
margin-top: 1.4em;
|
||||
}
|
||||
h5, h6 {
|
||||
font-size: 1em;
|
||||
font-style: italic;
|
||||
}
|
||||
h6 {
|
||||
font-weight: normal;
|
||||
}
|
||||
ol, ul {
|
||||
padding-left: 1.7em;
|
||||
margin-top: 1em;
|
||||
}
|
||||
li > ol, li > ul {
|
||||
margin-top: 0;
|
||||
}
|
||||
blockquote {
|
||||
margin: 1em 0 1em 1.7em;
|
||||
padding-left: 1em;
|
||||
border-left: 2px solid #e6e6e6;
|
||||
color: #606060;
|
||||
}
|
||||
code {
|
||||
font-family: Menlo, Monaco, Consolas, 'Lucida Console', monospace;
|
||||
font-size: 85%;
|
||||
margin: 0;
|
||||
hyphens: manual;
|
||||
}
|
||||
pre {
|
||||
margin: 1em 0;
|
||||
overflow: auto;
|
||||
}
|
||||
pre code {
|
||||
padding: 0;
|
||||
overflow: visible;
|
||||
overflow-wrap: normal;
|
||||
}
|
||||
.sourceCode {
|
||||
background-color: transparent;
|
||||
overflow: visible;
|
||||
}
|
||||
hr {
|
||||
border: none;
|
||||
border-top: 1px solid #1a1a1a;
|
||||
height: 1px;
|
||||
margin: 1em 0;
|
||||
}
|
||||
table {
|
||||
margin: 1em 0;
|
||||
border-collapse: collapse;
|
||||
width: 100%;
|
||||
overflow-x: auto;
|
||||
display: block;
|
||||
font-variant-numeric: lining-nums tabular-nums;
|
||||
}
|
||||
table caption {
|
||||
margin-bottom: 0.75em;
|
||||
}
|
||||
tbody {
|
||||
margin-top: 0.5em;
|
||||
border-top: 1px solid #1a1a1a;
|
||||
border-bottom: 1px solid #1a1a1a;
|
||||
}
|
||||
th {
|
||||
border-top: 1px solid #1a1a1a;
|
||||
padding: 0.25em 0.5em 0.25em 0.5em;
|
||||
}
|
||||
td {
|
||||
padding: 0.125em 0.5em 0.25em 0.5em;
|
||||
}
|
||||
header {
|
||||
margin-bottom: 4em;
|
||||
text-align: center;
|
||||
}
|
||||
#TOC li {
|
||||
list-style: none;
|
||||
}
|
||||
#TOC ul {
|
||||
padding-left: 1.3em;
|
||||
}
|
||||
#TOC > ul {
|
||||
padding-left: 0;
|
||||
}
|
||||
#TOC a:not(:hover) {
|
||||
text-decoration: none;
|
||||
}
|
||||
code{white-space: pre-wrap;}
|
||||
span.smallcaps{font-variant: small-caps;}
|
||||
div.columns{display: flex; gap: min(4vw, 1.5em);}
|
||||
div.column{flex: auto; overflow-x: auto;}
|
||||
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
|
||||
/* The extra [class] is a hack that increases specificity enough to
|
||||
override a similar rule in reveal.js */
|
||||
ul.task-list[class]{list-style: none;}
|
||||
ul.task-list li input[type="checkbox"] {
|
||||
font-size: inherit;
|
||||
width: 0.8em;
|
||||
margin: 0 0.8em 0.2em -1.6em;
|
||||
vertical-align: middle;
|
||||
}
|
||||
.display.math{display: block; text-align: center; margin: 0.5rem auto;}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<nav id="TOC" role="doc-toc">
|
||||
<ul>
|
||||
<li><a href="#smart-city-digital-twin-diagramme-des-flux-de-données"
|
||||
id="toc-smart-city-digital-twin-diagramme-des-flux-de-données">Smart
|
||||
City Digital Twin — Diagramme des Flux de Données</a>
|
||||
<ul>
|
||||
<li><a href="#vue-densemble" id="toc-vue-densemble">Vue
|
||||
d’ensemble</a></li>
|
||||
<li><a href="#diagramme-mermaid" id="toc-diagramme-mermaid">Diagramme
|
||||
Mermaid</a></li>
|
||||
<li><a href="#description-des-flux"
|
||||
id="toc-description-des-flux">Description des flux</a>
|
||||
<ul>
|
||||
<li><a href="#génération-des-données-simulator"
|
||||
id="toc-génération-des-données-simulator">1. <strong>Génération des
|
||||
données (Simulator)</strong></a></li>
|
||||
<li><a href="#ingestion-mqtt-brokers" id="toc-ingestion-mqtt-brokers">2.
|
||||
<strong>Ingestion MQTT (Brokers)</strong></a></li>
|
||||
<li><a href="#context-brokers-ngsi-ld-sensorthings"
|
||||
id="toc-context-brokers-ngsi-ld-sensorthings">3. <strong>Context Brokers
|
||||
(NGSI-LD & SensorThings)</strong></a></li>
|
||||
<li><a href="#plateforme-iot-openremote"
|
||||
id="toc-plateforme-iot-openremote">4. <strong>Plateforme IoT
|
||||
(OpenRemote)</strong></a></li>
|
||||
<li><a href="#stockage-métriques" id="toc-stockage-métriques">5.
|
||||
<strong>Stockage & Métriques</strong></a></li>
|
||||
<li><a href="#visualisation-analyse" id="toc-visualisation-analyse">6.
|
||||
<strong>Visualisation & Analyse</strong></a></li>
|
||||
</ul></li>
|
||||
<li><a href="#technologies-clés" id="toc-technologies-clés">Technologies
|
||||
clés</a></li>
|
||||
<li><a href="#fichiers-associés" id="toc-fichiers-associés">Fichiers
|
||||
associés</a></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
</nav>
|
||||
<h1 id="smart-city-digital-twin-diagramme-des-flux-de-données">Smart
|
||||
City Digital Twin — Diagramme des Flux de Données</h1>
|
||||
<h2 id="vue-densemble">Vue d’ensemble</h2>
|
||||
<p>Ce diagramme illustre le flux complet des données IoT du simulateur
|
||||
vers les différentes couches de traitement, de stockage et de
|
||||
visualisation.</p>
|
||||
<h1
|
||||
id="smart-city-digital-twin-martinique-diagramme-des-flux-de-données">Smart
|
||||
City Digital Twin Martinique — Diagramme des Flux de Données</h1>
|
||||
<p><strong>Dernière mise à jour :</strong> 06 Mai 2026<br />
|
||||
<strong>Projet :</strong> Smart City Digital Twin Martinique<br />
|
||||
<strong>Architecture :</strong> IoT-Agent intégré, QuantumLeap + CrateDB
|
||||
pour l’analyse avancée</p>
|
||||
<hr />
|
||||
<h2 id="diagramme-mermaid">Diagramme Mermaid</h2>
|
||||
<h2 id="architecture-globale-mise-à-jour-06052026">Architecture Globale
|
||||
(Mise à jour 06/05/2026)</h2>
|
||||
<pre class="mermaid"><code>graph TB
|
||||
SIM[Smart City Simulator]
|
||||
SENS[Capteurs IoT Reels]
|
||||
EMQ[EMQX]
|
||||
MOS[Mosquitto]
|
||||
BUN[BunkerM]
|
||||
FRO[FROST-Server]
|
||||
ORI[Orion-LD]
|
||||
STE[Stellio]
|
||||
UI[OpenRemote UI]
|
||||
ORM[OpenRemote Manager]
|
||||
KC[Keycloak]
|
||||
INF[InfluxDB]
|
||||
PRO[Prometheus]
|
||||
GEO[GeoServer]
|
||||
GRA[Grafana]
|
||||
MAP[MapStore]
|
||||
subgraph Simulateur["🖥️ Simulateur (Host Python)"]
|
||||
SIM[Smart City Simulator<br/>10 capteurs<br/>Intervalle: configurable]
|
||||
end
|
||||
|
||||
SIM --> EMQ
|
||||
SIM --> MOS
|
||||
SIM --> BUN
|
||||
SENS --> EMQ
|
||||
SENS --> MOS
|
||||
SENS --> BUN
|
||||
SENS -.-> ORM
|
||||
EMQ -->|via EMQX| ORI
|
||||
EMQ -->|via EMQX| STE
|
||||
EMQ -->|via EMQX| FRO
|
||||
EMQ --> ORM
|
||||
MOS -->|via Mosquitto| ORI
|
||||
MOS -->|via Mosquitto| STE
|
||||
MOS -->|via Mosquitto| FRO
|
||||
MOS --> ORM
|
||||
BUN -->|via BunkerM| ORI
|
||||
BUN -->|via BunkerM| STE
|
||||
BUN -->|via BunkerM| FRO
|
||||
BUN --> ORM
|
||||
UI --> ORM
|
||||
ORM -.-> KC
|
||||
SIM --> INF
|
||||
ORI --> GRA
|
||||
STE --> GRA
|
||||
FRO --> GRA
|
||||
ORI -.-> GEO
|
||||
STE -.-> GEO
|
||||
FRO -.-> GEO
|
||||
GEO --> MAP
|
||||
ORM --> GRA
|
||||
EMQ -.-> PRO
|
||||
ORI -.-> PRO
|
||||
STE -.-> PRO
|
||||
ORM -.-> PRO</code></pre>
|
||||
subgraph MQTT_Brokers["📡 MQTT Brokers"]
|
||||
EMQ[EMQX<br/>port 11883]
|
||||
MOS[Mosquitto<br/>port 1883]
|
||||
BUN[BunkerM<br/>port 1900<br/>MQTTS/TLS]
|
||||
end
|
||||
|
||||
subgraph IoT_Agent["🤖 IoT Agent (NGSI-LD)"]
|
||||
IOTA[IoT Agent JSON<br/>port 4041<br/>Transforme MQTT → NGSI-LD]
|
||||
end
|
||||
|
||||
subgraph CB["🔗 Context Brokers (NGSI-LD)"]
|
||||
ORI[Orion-LD<br/>NGSI-LD<br/>port 1026]
|
||||
STE[Stellio<br/>NGSI-LD<br/>port 8080]
|
||||
FRO[FROST-Server<br/>SensorThings<br/>port 8080]
|
||||
end
|
||||
|
||||
subgraph Analytics["📈 Analytics & Time-Series"]
|
||||
QL[QuantumLeap<br/>NGSI-LD → CrateDB<br/>port 8668]
|
||||
CRATEDB[CrateDB<br/>PostgreSQL-compatible<br/>port 4200/5432]
|
||||
end
|
||||
|
||||
subgraph Storage["💾 Stockage & Métriques"]
|
||||
INF[InfluxDB<br/>Bucket: iot_data<br/>port 8086]
|
||||
PRO[Prometheus<br/>Scrape: /metrics<br/>port 9090]
|
||||
GEO[GeoServer<br/>WMS/WFS/WMTS<br/>port 8080]
|
||||
end
|
||||
|
||||
subgraph IoT_Platform["🏢 Plateforme IoT"]
|
||||
ORM[OpenRemote Manager<br/>MQTT Agent<br/>port 8080]
|
||||
KC[Keycloak<br/>port 8080]
|
||||
end
|
||||
|
||||
subgraph VIZ["📊 Visualisation"]
|
||||
GRA[Grafana<br/>Dashboards<br/>port 3001]
|
||||
MAP[MapStore<br/>WMS/WFS<br/>port 8080]
|
||||
end
|
||||
|
||||
%% ── Flux Simulateur ──────────────────────────────────────────
|
||||
SIM -->|"1️⃣ MQTT publish<br/>city/sensors/{type}/{id}"| EMQ
|
||||
SIM -->|"1️⃣ MQTT publish"| MOS
|
||||
SIM -->|"1️⃣ MQTT publish"| BUN
|
||||
SIM -->|"5️⃣ InfluxDB v2 API<br/>async non-bloquant"| INF
|
||||
|
||||
%% ── Flux MQTT → IoT Agent ──────────────────────────────────
|
||||
EMQ -->|"MQTT subscribe<br/>city/sensors/#"| IOTA
|
||||
MOS -->|"MQTT subscribe"| IOTA
|
||||
BUN -->|"MQTT subscribe"| IOTA
|
||||
|
||||
%% ── Flux IoT Agent → Context Brokers ───────────────────────
|
||||
IOTA -->|"2️⃣ NGSI-LD POST<br/>/ngsi-ld/v1/entities"| ORI
|
||||
IOTA -->|"2️⃣ NGSI-LD POST"| STE
|
||||
|
||||
%% ── Flux Context Brokers → QuantumLeap ───────────────────
|
||||
ORI -->|"NGSI-LD Subscription<br/>→ QuantumLeap"| QL
|
||||
STE -->|"NGSI-LD Subscription<br/>→ QuantumLeap"| QL
|
||||
|
||||
%% ── Flux QuantumLeap → CrateDB ────────────────────────────
|
||||
QL -->|"Insert<br/>PostgreSQL wire"| CRATEDB
|
||||
|
||||
%% ── Visualisation ───────────────────────────────────────────
|
||||
CRATEDB -->|"PostgreSQL Datasource"| GRA
|
||||
INF -->|"Datasource Flux IoT"| GRA
|
||||
ORI -->|"NGSI-LD Datasource"| GRA
|
||||
STE -->|"NGSI-LD Datasource"| GRA
|
||||
GEO -->|"WMS/WMTS"| MAP
|
||||
ORM -->|MapSettings<br/>Martinique| MAP
|
||||
ORM -->|"Live assets<br/>REST"| GRA
|
||||
|
||||
%% ── OpenRemote MQTT Agent ───────────────────────────────────
|
||||
EMQ -->|"6️⃣ Subscribe<br/>city/sensors/#"| ORM
|
||||
MOS -->|"6️⃣ Subscribe"| ORM
|
||||
BUN -->|"6️⃣ Subscribe"| ORM
|
||||
|
||||
%% ── Métriques Prometheus ────────────────────────────────────
|
||||
SIM -->|"7️⃣ /metrics<br/>port 8001"| PRO
|
||||
EMQ -->|"/api/v5/metrics"| PRO
|
||||
STE -->|"/actuator/prometheus"| PRO
|
||||
INF -->|"/metrics"| PRO
|
||||
ORM -->|"/actuator/prometheus"| PRO
|
||||
GRA -->|"/metrics"| PRO
|
||||
IOTA -->|"/metrics"| PRO
|
||||
QL -->|"/metrics"| PRO</code></pre>
|
||||
<hr />
|
||||
<h2 id="description-des-flux">Description des flux</h2>
|
||||
<h3 id="génération-des-données-simulator">1. <strong>Génération des
|
||||
données (Simulator)</strong></h3>
|
||||
<ul>
|
||||
<li><strong>Smart City Simulator</strong> (Python) génère des données
|
||||
pour 10 capteurs (Traffic, Air Quality, Parking, Noise, Weather,
|
||||
Light)</li>
|
||||
<li>Intervalle de publication : 10 secondes</li>
|
||||
<li>Protocoles : MQTT (vers brokers uniquement)</li>
|
||||
<li><strong>⚠️ Projet</strong> : Le simulateur n’envoie PAS directement
|
||||
à OpenRemote (pas de REST API)</li>
|
||||
</ul>
|
||||
<h3 id="ingestion-mqtt-brokers">2. <strong>Ingestion MQTT
|
||||
(Brokers)</strong></h3>
|
||||
<ul>
|
||||
<li><strong>EMQX</strong> (port 11883) : Broker public, reçoit tous les
|
||||
capteurs</li>
|
||||
<li><strong>Mosquitto</strong> (port 1883) : Via Traefik, accès
|
||||
externe</li>
|
||||
<li><strong>BunkerM</strong> (port 1900) : MQTTS (TLS), accès
|
||||
sécurisé</li>
|
||||
</ul>
|
||||
<h3 id="context-brokers-ngsi-ld-sensorthings">3. <strong>Context Brokers
|
||||
(NGSI-LD & SensorThings)</strong></h3>
|
||||
<ul>
|
||||
<li><strong>Orion-LD</strong> : Reçoit les données au format NGSI-LD
|
||||
<ul>
|
||||
<li>10 entités (TrafficFlowObserved, AirQualityObserved, etc.)</li>
|
||||
<li>Smart Data Models utilisés</li>
|
||||
<li><strong>Provenance</strong> : Données via EMQX, Mosquitto et BunkerM
|
||||
(voir étiquettes dans le diagramme)</li>
|
||||
</ul></li>
|
||||
<li><strong>Stellio</strong> : Alternative NGSI-LD
|
||||
<ul>
|
||||
<li>14 payloads entités</li>
|
||||
<li>Contexte :
|
||||
<code>https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld</code></li>
|
||||
<li><strong>Provenance</strong> : Données via EMQX, Mosquitto et
|
||||
BunkerM</li>
|
||||
</ul></li>
|
||||
<li><strong>FROST-Server</strong> : SensorThings API
|
||||
<ul>
|
||||
<li>21 256+ observations</li>
|
||||
<li>PostgreSQL + TimescaleDB</li>
|
||||
<li><strong>Provenance</strong> : Données via EMQX, Mosquitto et
|
||||
BunkerM</li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<h3 id="plateforme-iot-openremote">4. <strong>Plateforme IoT
|
||||
(OpenRemote)</strong></h3>
|
||||
<ul>
|
||||
<li><strong>OpenRemote Manager</strong> (realm <code>smartcity</code>)
|
||||
<ul>
|
||||
<li>33 assets IoT configurés</li>
|
||||
<li>Carte Martinique (mapsettings.json)</li>
|
||||
<li>Réception via <strong>MQTT Agent</strong> depuis les brokers (EMQX,
|
||||
Mosquitto, BunkerM)</li>
|
||||
<li>Peut aussi recevoir directement des capteurs IoT (via MQTT)</li>
|
||||
</ul></li>
|
||||
<li><strong>Keycloak</strong> : Authentification OpenID Connect
|
||||
<ul>
|
||||
<li>Client <code>openremote</code> avec Service Account</li>
|
||||
<li>Token endpoint :
|
||||
<code>/auth/realms/smartcity/protocol/openid-connect/token</code></li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<h3 id="stockage-métriques">5. <strong>Stockage &
|
||||
Métriques</strong></h3>
|
||||
<ul>
|
||||
<li><strong>InfluxDB</strong> : Stockage temporel pour Grafana
|
||||
<ul>
|
||||
<li>Bucket : <code>iot_data</code></li>
|
||||
<li>Datasource dans Grafana</li>
|
||||
</ul></li>
|
||||
<li><strong>Prometheus</strong> : Collecte des métriques
|
||||
<ul>
|
||||
<li>MQTT brokers, Context brokers, OpenRemote</li>
|
||||
</ul></li>
|
||||
<li><strong>GeoServer</strong> : Données géospatiales
|
||||
<ul>
|
||||
<li>PostGIS pour centralisation</li>
|
||||
<li>WMS/WFS pour MapStore</li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<h3 id="visualisation-analyse">6. <strong>Visualisation &
|
||||
Analyse</strong></h3>
|
||||
<ul>
|
||||
<li><strong>Grafana</strong> (port 3001)
|
||||
<ul>
|
||||
<li>Dashboard : <code>smartcity-martinique-2026</code></li>
|
||||
<li>Datasources : InfluxDB, FROST, Orion-LD</li>
|
||||
</ul></li>
|
||||
<li><strong>MapStore</strong> : Cartographie
|
||||
<ul>
|
||||
<li>Sources WMS/WFS depuis GeoServer</li>
|
||||
</ul></li>
|
||||
<li><strong>OpenRemote UI</strong> : Manager Interface
|
||||
<ul>
|
||||
<li>Visualisation des assets realm Smart City</li>
|
||||
</ul></li>
|
||||
</ul>
|
||||
<hr />
|
||||
<h2 id="technologies-clés">Technologies clés</h2>
|
||||
<h2 id="flux-détaillés-mise-à-jour-06052026">Flux Détaillés (Mise à jour
|
||||
06/05/2026)</h2>
|
||||
<h3 id="flux-mqtt-brokers">1️⃣ Flux MQTT — Brokers</h3>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Broker</th>
|
||||
<th>Port</th>
|
||||
<th>Protocol</th>
|
||||
<th>Topics</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>EMQX</td>
|
||||
<td>11883</td>
|
||||
<td>MQTT</td>
|
||||
<td><code>city/sensors/{type}/{id}</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mosquitto</td>
|
||||
<td>1883</td>
|
||||
<td>MQTT</td>
|
||||
<td><code>city/sensors/{type}/{id}</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BunkerM</td>
|
||||
<td>1900</td>
|
||||
<td>MQTTS (TLS)</td>
|
||||
<td><code>city/sensors/{type}/{id}</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>Le simulateur publie simultanément sur les 3 brokers vers <strong>IoT
|
||||
Agent</strong>.</p>
|
||||
<h3 id="flux-iot-agent-ngsi-ld">2️⃣ Flux IoT Agent — NGSI-LD</h3>
|
||||
<ul>
|
||||
<li><p><strong>IoT Agent JSON</strong> : Réception MQTT → Transformation
|
||||
en entités NGSI-LD</p></li>
|
||||
<li><p><strong>Port</strong> : <code>4041</code></p></li>
|
||||
<li><p><strong>Configuration</strong> :</p>
|
||||
<div class="sourceCode" id="cb2"><pre
|
||||
class="sourceCode bash"><code class="sourceCode bash"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Enregistrement service</span></span>
|
||||
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a><span class="ex">curl</span> <span class="at">-X</span> POST http://localhost:4041/iot/services <span class="dt">\</span></span>
|
||||
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a> <span class="at">-H</span> <span class="st">'Content-Type: application/json'</span> <span class="dt">\</span></span>
|
||||
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a> <span class="at">-H</span> <span class="st">'fiware-service: smartcity'</span> <span class="dt">\</span></span>
|
||||
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a> <span class="at">-d</span> <span class="st">'{"services": [{"apikey": "smartcity-api-key", "cbroker": "http://orion-ld:1026", "entity_type": "Device", "ngsi_version": "ld"}]}'</span></span></code></pre></div></li>
|
||||
<li><p><strong>Entités créées dans</strong> : Orion-LD (port 1026) et
|
||||
Stellio (port 8080)</p></li>
|
||||
</ul>
|
||||
<h3 id="flux-context-brokers-quantumleap-cratedb">3️⃣ Flux Context
|
||||
Brokers → QuantumLeap → CrateDB</h3>
|
||||
<ol type="1">
|
||||
<li><strong>Orion-LD</strong> / <strong>Stellio</strong> : Reçoivent les
|
||||
entités NGSI-LD de IoT Agent</li>
|
||||
<li><strong>QuantumLeap</strong> (port 8668) : Souscrit aux mises à jour
|
||||
NGSI-LD via Subscription</li>
|
||||
<li><strong>CrateDB</strong> (port 5432/4200) : Stockage temporel
|
||||
PostgreSQL-compatible</li>
|
||||
<li><strong>Grafana</strong> : Dashboards connectés à CrateDB
|
||||
(PostgreSQL datasource)</li>
|
||||
</ol>
|
||||
<h3 id="flux-influxdb-temps-réel">4️⃣ Flux InfluxDB — Temps Réel</h3>
|
||||
<ul>
|
||||
<li><strong>API</strong> :
|
||||
<code>http://localhost:8086/api/v2/write</code></li>
|
||||
<li><strong>Bucket</strong> : <code>iot_data</code></li>
|
||||
<li><strong>Org</strong> : <code>digitribe</code></li>
|
||||
<li><strong>Mode</strong> : Asynchrone (thread daemon) pour ne pas
|
||||
bloquer le publish MQTT</li>
|
||||
</ul>
|
||||
<h3 id="openremote-mqtt-agent">5️⃣ OpenRemote — MQTT Agent</h3>
|
||||
<p>L’agent MQTT d’OpenRemote souscrit aux topics
|
||||
<code>city/sensors/#</code> sur les brokers MQTT. Les payloads sont
|
||||
automatiquement parsés et les attributs des assets sont mis à jour.</p>
|
||||
<h3 id="flux-prometheus-métriques">6️⃣ Flux Prometheus — Métriques</h3>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Service</th>
|
||||
<th>Endpoint <code>/metrics</code></th>
|
||||
<th>Statut</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Simulator</td>
|
||||
<td><code>localhost:8001</code></td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>EMQX</td>
|
||||
<td><code>emqx_emqx_1:8081/api/v5/metrics</code></td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Stellio</td>
|
||||
<td><code>stellio-api-gateway:8080/actuator/prometheus</code></td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>InfluxDB</td>
|
||||
<td><code>smart-city-influxdb:8086/metrics</code></td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenRemote</td>
|
||||
<td><code>openremote-manager-1:8080/actuator/prometheus</code></td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Grafana</td>
|
||||
<td><code>smart-city-grafana:3000/metrics</code></td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>IoT Agent</td>
|
||||
<td><code>iot-agent:4041/metrics</code></td>
|
||||
<td>⚠️ À vérifier</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>QuantumLeap</td>
|
||||
<td><code>quantum-leap:8668/metrics</code></td>
|
||||
<td>⚠️ À vérifier</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<hr />
|
||||
<h2 id="nouveaux-composants-06052026">Nouveaux Composants
|
||||
(06/05/2026)</h2>
|
||||
<h3 id="iot-agent-json">🤖 IoT Agent JSON</h3>
|
||||
<ul>
|
||||
<li><strong>Rôle</strong> : Pont entre MQTT et NGSI-LD (Orion-LD /
|
||||
Stellio)</li>
|
||||
<li><strong>Port</strong> : 4041</li>
|
||||
<li><strong>Statut</strong> : ❌ En cours de réparation (erreur
|
||||
MongoDB)</li>
|
||||
<li><strong>Correction</strong> : Fournir
|
||||
<code>IOTA_MONGO_URL=mongodb://mongodb:27017/iotagent</code></li>
|
||||
</ul>
|
||||
<h3 id="quantumleap">📈 QuantumLeap</h3>
|
||||
<ul>
|
||||
<li><strong>Rôle</strong> : Analytics NGSI-LD → CrateDB</li>
|
||||
<li><strong>Port</strong> : 8668</li>
|
||||
<li><strong>Statut</strong> : ✅ Fonctionnel (interne), port non exposé
|
||||
sur l’hôte</li>
|
||||
<li><strong>Action</strong> : Exposer le port dans docker-compose</li>
|
||||
</ul>
|
||||
<h3 id="cratedb">🗄️ CrateDB</h3>
|
||||
<ul>
|
||||
<li><strong>Rôle</strong> : Base de données temporelle
|
||||
PostgreSQL-compatible</li>
|
||||
<li><strong>Port</strong> : 4200 (UI), 5432 (PostgreSQL)</li>
|
||||
<li><strong>Statut</strong> : ✅ Opérationnel</li>
|
||||
<li><strong>Usage</strong> : Stockage des séries temporelles depuis
|
||||
QuantumLeap</li>
|
||||
</ul>
|
||||
<hr />
|
||||
<h2 id="tableau-récapitulatif-mise-à-jour">Tableau Récapitulatif (Mise à
|
||||
jour)</h2>
|
||||
<table>
|
||||
<colgroup>
|
||||
<col style="width: 28%" />
|
||||
<col style="width: 34%" />
|
||||
<col style="width: 15%" />
|
||||
<col style="width: 21%" />
|
||||
</colgroup>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Composant</th>
|
||||
<th>Technologie</th>
|
||||
<th>Port</th>
|
||||
@@ -385,7 +275,7 @@ Analyse</strong></h3>
|
||||
<tr>
|
||||
<td>Simulator</td>
|
||||
<td>Python + paho-mqtt</td>
|
||||
<td>Interne</td>
|
||||
<td>Host:8001 (metrics)</td>
|
||||
<td>✅ Actif</td>
|
||||
</tr>
|
||||
<tr>
|
||||
@@ -395,6 +285,24 @@ Analyse</strong></h3>
|
||||
<td>✅ Connecté</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mosquitto</td>
|
||||
<td>MQTT Broker</td>
|
||||
<td>1883</td>
|
||||
<td>✅ Connecté</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>BunkerM</td>
|
||||
<td>MQTTS Broker</td>
|
||||
<td>1900</td>
|
||||
<td>✅ Connecté</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>IoT Agent</strong></td>
|
||||
<td><strong>NGSI-LD Bridge</strong></td>
|
||||
<td><strong>4041</strong></td>
|
||||
<td><strong>❌ Erreur MongoDB</strong></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Orion-LD</td>
|
||||
<td>NGSI-LD Broker</td>
|
||||
<td>1026</td>
|
||||
@@ -404,7 +312,7 @@ Analyse</strong></h3>
|
||||
<td>Stellio</td>
|
||||
<td>NGSI-LD Broker</td>
|
||||
<td>8080</td>
|
||||
<td>⚠️ À vérifier</td>
|
||||
<td>⚠️ Ports occupés par OpenRemote</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>FROST-Server</td>
|
||||
@@ -413,6 +321,18 @@ Analyse</strong></h3>
|
||||
<td>⚠️ À vérifier</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>QuantumLeap</strong></td>
|
||||
<td><strong>NGSI-LD → CrateDB</strong></td>
|
||||
<td><strong>8668</strong></td>
|
||||
<td><strong>✅ Interne</strong></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><strong>CrateDB</strong></td>
|
||||
<td><strong>PostgreSQL Time-Series</strong></td>
|
||||
<td><strong>4200/5432</strong></td>
|
||||
<td><strong>✅ Opérationnel</strong></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenRemote</td>
|
||||
<td>IoT Platform</td>
|
||||
<td>8080</td>
|
||||
@@ -422,44 +342,67 @@ Analyse</strong></h3>
|
||||
<td>InfluxDB</td>
|
||||
<td>Time Series DB</td>
|
||||
<td>8086</td>
|
||||
<td>✅ Configuré</td>
|
||||
<td>✅ Bucket iot_data</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Grafana</td>
|
||||
<td>Visualization</td>
|
||||
<td>Visualisation</td>
|
||||
<td>3001</td>
|
||||
<td>✅ Dashboard créé</td>
|
||||
<td>✅ Dashboards + CrateDB</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GeoServer</td>
|
||||
<td>GeoServer</td>
|
||||
<td>Geo Data</td>
|
||||
<td>8080</td>
|
||||
<td>⚠️ À intégrer</td>
|
||||
<td>✅ REST OK</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Prometheus</td>
|
||||
<td>Metrics</td>
|
||||
<td>9090</td>
|
||||
<td>✅ En cours</td>
|
||||
<td>MapStore</td>
|
||||
<td>Cartographie</td>
|
||||
<td>8080</td>
|
||||
<td>✅ WMS/WMTS</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<hr />
|
||||
<h2 id="fichiers-associés">Fichiers associés</h2>
|
||||
<ul>
|
||||
<li><strong>Simulator</strong> :
|
||||
<code>~/smart-city-digital-twin-martinique/simulator.py</code></li>
|
||||
<li><strong>Dashboard Grafana</strong> :
|
||||
<code>~/smart-city-digital-twin-martinique/grafana_dashboard_smartcity.json</code></li>
|
||||
<li><strong>Ce diagramme</strong> :
|
||||
<code>~/smart-city-digital-twin-martinique/data-flow-diagram.md</code></li>
|
||||
<li><strong>Session Resume</strong> :
|
||||
<code>~/smart-city-digital-twin-martinique/session_resume_2026-05-04.md</code></li>
|
||||
</ul>
|
||||
<h2 id="actions-prioritaires">Actions Prioritaires</h2>
|
||||
<ol type="1">
|
||||
<li><strong>Corriger IoT Agent</strong> : Ajouter MongoDB et configurer
|
||||
<code>IOTA_MONGO_URL</code></li>
|
||||
<li><strong>Exposer QuantumLeap</strong> : Mapper le port 8668 dans
|
||||
docker-compose</li>
|
||||
<li><strong>Déployer Orion-LD</strong> : Créer le service s’il n’existe
|
||||
pas</li>
|
||||
<li><strong>Libérer les ports</strong> : Résoudre le conflit
|
||||
Stellio/OpenRemote sur le port 8080</li>
|
||||
<li><strong>Configurer CrateDB</strong> : Créer les tables pour
|
||||
QuantumLeap</li>
|
||||
<li><strong>Mettre à jour le simulateur</strong> :
|
||||
<code>ENABLE_PULSAR=false</code> (recommandé)</li>
|
||||
</ol>
|
||||
<hr />
|
||||
<p><strong>Dernière mise à jour :</strong> 04 Mai 2026<br />
|
||||
<strong>Projet :</strong> Smart City Digital Twin Martinique<br />
|
||||
<strong>URL Grafana :</strong>
|
||||
http://localhost:3001/d/smartcity-martinique-2026</p>
|
||||
</body>
|
||||
</html>
|
||||
<h2 id="commandes-utiles">Commandes Utiles</h2>
|
||||
<div class="sourceCode" id="cb3"><pre
|
||||
class="sourceCode bash"><code class="sourceCode bash"><span id="cb3-1"><a href="#cb3-1" aria-hidden="true" tabindex="-1"></a><span class="co"># Vérifier IoT Agent</span></span>
|
||||
<span id="cb3-2"><a href="#cb3-2" aria-hidden="true" tabindex="-1"></a><span class="ex">curl</span> <span class="at">-s</span> http://localhost:4041/iot/services <span class="at">-H</span> <span class="st">'fiware-service: smartcity'</span></span>
|
||||
<span id="cb3-3"><a href="#cb3-3" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-4"><a href="#cb3-4" aria-hidden="true" tabindex="-1"></a><span class="co"># Vérifier QuantumLeap</span></span>
|
||||
<span id="cb3-5"><a href="#cb3-5" aria-hidden="true" tabindex="-1"></a><span class="ex">curl</span> <span class="at">-s</span> http://localhost:8668/version</span>
|
||||
<span id="cb3-6"><a href="#cb3-6" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-7"><a href="#cb3-7" aria-hidden="true" tabindex="-1"></a><span class="co"># Vérifier CrateDB</span></span>
|
||||
<span id="cb3-8"><a href="#cb3-8" aria-hidden="true" tabindex="-1"></a><span class="ex">psql</span> <span class="at">-h</span> localhost <span class="at">-p</span> 5432 <span class="at">-U</span> crate <span class="at">-c</span> <span class="st">"SELECT * FROM ql_entities LIMIT 5;"</span></span>
|
||||
<span id="cb3-9"><a href="#cb3-9" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-10"><a href="#cb3-10" aria-hidden="true" tabindex="-1"></a><span class="co"># Vérifier Orion-LD</span></span>
|
||||
<span id="cb3-11"><a href="#cb3-11" aria-hidden="true" tabindex="-1"></a><span class="ex">curl</span> <span class="at">-s</span> http://localhost:1026/version</span>
|
||||
<span id="cb3-12"><a href="#cb3-12" aria-hidden="true" tabindex="-1"></a></span>
|
||||
<span id="cb3-13"><a href="#cb3-13" aria-hidden="true" tabindex="-1"></a><span class="co"># Voir les logs IoT Agent</span></span>
|
||||
<span id="cb3-14"><a href="#cb3-14" aria-hidden="true" tabindex="-1"></a><span class="ex">docker</span> logs smart-city-iot-agent <span class="at">--tail</span> 30</span></code></pre></div>
|
||||
<hr />
|
||||
<p><strong>Fichiers associés :</strong> - Simulateur :
|
||||
<code>~/smart-city-digital-twin-martinique/simulator.py</code> -
|
||||
Dashboard Grafana :
|
||||
<code>~/smart-city-digital-twin-martinique/grafana_dashboard_smartcity.json</code>
|
||||
- Ce diagramme :
|
||||
<code>~/smart-city-digital-twin-martinique/data-flow-diagram.md</code> -
|
||||
Session Resume :
|
||||
<code>~/smart-city-digital-twin-martinique/session_resume_2026-05-07.md</code></p>
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
# Smart City Digital Twin Martinique — Diagramme des Flux de Données
|
||||
|
||||
**Dernière mise à jour :** 05 Mai 2026
|
||||
**Projet :** Smart City Digital Twin Martinique
|
||||
**Dernière mise à jour :** 06 Mai 2026
|
||||
**Projet :** Smart City Digital Twin Martinique
|
||||
**Architecture :** IoT-Agent intégré, QuantumLeap + CrateDB pour l'analyse avancée
|
||||
|
||||
---
|
||||
|
||||
## Architecture Globale
|
||||
## Architecture Globale (Mise à jour 06/05/2026)
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
@@ -19,17 +20,21 @@ graph TB
|
||||
BUN[BunkerM<br/>port 1900<br/>MQTTS/TLS]
|
||||
end
|
||||
|
||||
subgraph Stream["⚡ Event Streaming"]
|
||||
PUL[Pulsar<br/>port 6650<br/>Topics: smartcity-*]
|
||||
RED[Redpanda<br/>port 8082 REST<br/>Topics: traffic, air-quality, ...]
|
||||
subgraph IoT_Agent["🤖 IoT Agent (NGSI-LD)"]
|
||||
IOTA[IoT Agent JSON<br/>port 4041<br/>Transforme MQTT → NGSI-LD]
|
||||
end
|
||||
|
||||
subgraph CB["🔗 Context Brokers"]
|
||||
subgraph CB["🔗 Context Brokers (NGSI-LD)"]
|
||||
ORI[Orion-LD<br/>NGSI-LD<br/>port 1026]
|
||||
STE[Stellio<br/>NGSI-LD<br/>port 8080]
|
||||
FRO[FROST-Server<br/>SensorThings<br/>port 8080]
|
||||
end
|
||||
|
||||
subgraph Analytics["📈 Analytics & Time-Series"]
|
||||
QL[QuantumLeap<br/>NGSI-LD → CrateDB<br/>port 8668]
|
||||
CRATEDB[CrateDB<br/>PostgreSQL-compatible<br/>port 4200/5432]
|
||||
end
|
||||
|
||||
subgraph Storage["💾 Stockage & Métriques"]
|
||||
INF[InfluxDB<br/>Bucket: iot_data<br/>port 8086]
|
||||
PRO[Prometheus<br/>Scrape: /metrics<br/>port 9090]
|
||||
@@ -42,139 +47,135 @@ graph TB
|
||||
end
|
||||
|
||||
subgraph VIZ["📊 Visualisation"]
|
||||
GRA[Grafana<br/>Dashboards<br/>port 3000]
|
||||
GRA[Grafana<br/>Dashboards<br/>port 3001]
|
||||
MAP[MapStore<br/>WMS/WFS<br/>port 8080]
|
||||
end
|
||||
|
||||
subgraph Distribution["🔄 Distribution Service"]
|
||||
DIST[Pulsar Distribution<br/>Pulsar → Brokers]
|
||||
end
|
||||
|
||||
subgraph Consumer["📥 Redpanda Consumer"]
|
||||
RCONS[Redpanda → InfluxDB<br/>REST → InfluxDB]
|
||||
end
|
||||
|
||||
%% ── Flux Simulateur ──────────────────────────────────────────────────
|
||||
%% ── Flux Simulateur ──────────────────────────────────────────
|
||||
SIM -->|"1️⃣ MQTT publish<br/>city/sensors/{type}/{id}"| EMQ
|
||||
SIM -->|"1️⃣ MQTT publish"| MOS
|
||||
SIM -->|"1️⃣ MQTT publish"| BUN
|
||||
SIM -->|"2️⃣ HTTP POST<br/>NGSI-LD"| ORI
|
||||
SIM -->|"2️⃣ HTTP POST<br/>NGSI-LD"| STE
|
||||
SIM -->|"2️⃣ HTTP POST<br/>SensorThings"| FRO
|
||||
SIM -->|"3️⃣ Pulsar client<br/>pulsar://localhost:6650"| PUL
|
||||
SIM -->|"4️⃣ HTTP REST Proxy<br/>localhost:8082/topics/"| RED
|
||||
SIM -->|"5️⃣ InfluxDB v2 API<br/>async non-bloquant"| INF
|
||||
|
||||
%% ── Flux Distribution (Pulsar → Brokers) ──────────────────────────────
|
||||
PUL -->|"Consomme<br/>smartcity-*"| DIST
|
||||
DIST -->|"Republish<br/>MQTT"| EMQ
|
||||
DIST -->|"Republish<br/>MQTT"| MOS
|
||||
DIST -->|"Republish<br/>NGSI-LD"| ORI
|
||||
DIST -->|"Republish<br/>NGSI-LD"| STE
|
||||
DIST -->|"Republish<br/>SensorThings"| FRO
|
||||
%% ── Flux MQTT → IoT Agent ──────────────────────────────────
|
||||
EMQ -->|"MQTT subscribe<br/>city/sensors/#"| IOTA
|
||||
MOS -->|"MQTT subscribe"| IOTA
|
||||
BUN -->|"MQTT subscribe"| IOTA
|
||||
|
||||
%% ── Flux Redpanda → InfluxDB ──────────────────────────────────────────
|
||||
RED -->|"REST poll<br/>topics/{name}/offsets"| RCONS
|
||||
RCONS -->|"Line Protocol<br/>Write API"| INF
|
||||
%% ── Flux IoT Agent → Context Brokers ───────────────────────
|
||||
IOTA -->|"2️⃣ NGSI-LD POST<br/>/ngsi-ld/v1/entities"| ORI
|
||||
IOTA -->|"2️⃣ NGSI-LD POST"| STE
|
||||
|
||||
%% ── OpenRemote MQTT Agent ──────────────────────────────────────────────
|
||||
%% ── Flux Context Brokers → QuantumLeap ───────────────────
|
||||
ORI -->|"NGSI-LD Subscription<br/>→ QuantumLeap"| QL
|
||||
STE -->|"NGSI-LD Subscription<br/>→ QuantumLeap"| QL
|
||||
|
||||
%% ── Flux QuantumLeap → CrateDB ────────────────────────────
|
||||
QL -->|"Insert<br/>PostgreSQL wire"| CRATEDB
|
||||
|
||||
%% ── Visualisation ───────────────────────────────────────────
|
||||
CRATEDB -->|"PostgreSQL Datasource"| GRA
|
||||
INF -->|"Datasource Flux IoT"| GRA
|
||||
ORI -->|"NGSI-LD Datasource"| GRA
|
||||
STE -->|"NGSI-LD Datasource"| GRA
|
||||
GEO -->|"WMS/WMTS"| MAP
|
||||
ORM -->|MapSettings<br/>Martinique| MAP
|
||||
ORM -->|"Live assets<br/>REST"| GRA
|
||||
|
||||
%% ── OpenRemote MQTT Agent ───────────────────────────────────
|
||||
EMQ -->|"6️⃣ Subscribe<br/>city/sensors/#"| ORM
|
||||
MOS -->|"6️⃣ Subscribe"| ORM
|
||||
BUN -->|"6️⃣ Subscribe"| ORM
|
||||
|
||||
%% ── Métriques Prometheus ────────────────────────────────────────────────
|
||||
%% ── Métriques Prometheus ────────────────────────────────────
|
||||
SIM -->|"7️⃣ /metrics<br/>port 8001"| PRO
|
||||
EMQ -->|"/api/v5/metrics"| PRO
|
||||
STE -->|"/actuator/prometheus"| PRO
|
||||
FRO -->|"/metrics"| PRO
|
||||
INF -->|"/metrics"| PRO
|
||||
RED -->|"/public_metrics"| PRO
|
||||
ORM -->|"/actuator/prometheus"| PRO
|
||||
GRA -->|"/metrics"| PRO
|
||||
|
||||
%% ── Visualisation ─────────────────────────────────────────────────────
|
||||
INF -->|"Datasources<br/>Flux IoT"| GRA
|
||||
ORI -->|"NGSI-LD<br/>Datasource"| GRA
|
||||
STE -->|"NGSI-LD<br/>Datasource"| GRA
|
||||
FRO -->|"SensorThings<br/>Datasource"| GRA
|
||||
GEO -->|"WMS/WMTS"| MAP
|
||||
ORM -->|MapSettings<br/>Martinique| MAP
|
||||
ORM -->|"Live assets<br/>REST"| GRA
|
||||
IOTA -->|"/metrics"| PRO
|
||||
QL -->|"/metrics"| PRO
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flux Détaillés
|
||||
## Flux Détaillés (Mise à jour 06/05/2026)
|
||||
|
||||
### 1️⃣ Flux MQTT — Brokers
|
||||
|
||||
| Broker | Port | Protocol | Topics |
|
||||
|--------|------|----------|--------|
|
||||
| EMQX | 11883 | MQTT | `city/sensors/{type}/{id}` |
|
||||
| Mosquitto | 1883 | MQTT | `city/sensors/{type}/{id}` |
|
||||
| BunkerM | 1900 | MQTTS (TLS) | `city/sensors/{type}/{id}` |
|
||||
|
||||
Le simulateur publie simultanément sur les 3 brokers.
|
||||
Le simulateur publie simultanément sur les 3 brokers vers **IoT Agent**.
|
||||
|
||||
### 2️⃣ Flux HTTP REST — Context Brokers
|
||||
### 2️⃣ Flux IoT Agent — NGSI-LD
|
||||
- **IoT Agent JSON** : Réception MQTT → Transformation en entités NGSI-LD
|
||||
- **Port** : `4041`
|
||||
- **Configuration** :
|
||||
```bash
|
||||
# Enregistrement service
|
||||
curl -X POST http://localhost:4041/iot/services \
|
||||
-H 'Content-Type: application/json' \
|
||||
-H 'fiware-service: smartcity' \
|
||||
-d '{"services": [{"apikey": "smartcity-api-key", "cbroker": "http://orion-ld:1026", "entity_type": "Device", "ngsi_version": "ld"}]}'
|
||||
```
|
||||
- **Entités créées dans** : Orion-LD (port 1026) et Stellio (port 8080)
|
||||
|
||||
| Broker | Format | Port | Topics |
|
||||
|--------|--------|------|--------|
|
||||
| Orion-LD | NGSI-LD | 1026 | Entités par type |
|
||||
| Stellio | NGSI-LD | 8080 | Entités par type |
|
||||
| FROST-Server | SensorThings | 8080 | Things → Datastreams → Observations |
|
||||
|
||||
### 3️⃣ Flux Pulsar — Event Streaming
|
||||
|
||||
- **Topics** : `persistent://public/default/smartcity-traffic`, `smartcity-airquality`, `smartcity-parking`, `smartcity-noise`, `smartcity-weather`, `smartcity-light`
|
||||
- **Port binaire** : `6650` (connectable depuis le host)
|
||||
- **Distribution** : Le service `pulsar-distribution` consomme ces topics et republie vers les brokers MQTT et context brokers
|
||||
|
||||
### 4️⃣ Flux Redpanda — Kafka-compatible REST
|
||||
|
||||
- **REST Proxy** : `http://localhost:8082`
|
||||
- **Topics** : `traffic`, `air-quality`, `parking`, `noise`, `weather`, `air-quality`
|
||||
- **Payload** : Base64(JSON) dans `{"records": [{"value": "<base64>"}]}`
|
||||
- **Consumer** : `redpanda/consumer.py` — poll toutes les 10s et écrit dans InfluxDB
|
||||
|
||||
### 5️⃣ Flux InfluxDB — Temps Réel
|
||||
### 3️⃣ Flux Context Brokers → QuantumLeap → CrateDB
|
||||
1. **Orion-LD** / **Stellio** : Reçoivent les entités NGSI-LD de IoT Agent
|
||||
2. **QuantumLeap** (port 8668) : Souscrit aux mises à jour NGSI-LD via Subscription
|
||||
3. **CrateDB** (port 5432/4200) : Stockage temporel PostgreSQL-compatible
|
||||
4. **Grafana** : Dashboards connectés à CrateDB (PostgreSQL datasource)
|
||||
|
||||
### 4️⃣ Flux InfluxDB — Temps Réel
|
||||
- **API** : `http://localhost:8086/api/v2/write`
|
||||
- **Bucket** : `iot_data`
|
||||
- **Org** : `digitribe`
|
||||
- **Mode** : Asynchrone (thread daemon) pour ne pas bloquer le publish MQTT
|
||||
|
||||
### 6️⃣ OpenRemote — MQTT Agent
|
||||
### 5️⃣ OpenRemote — MQTT Agent
|
||||
L'agent MQTT d'OpenRemote souscrit aux topics `city/sensors/#` sur les brokers MQTT. Les payloads sont automatiquement parsés et les attributs des assets sont mis à jour.
|
||||
|
||||
L'agent MQTT d'OpenRemote souscrit aux topics `city/sensors/#` sur les brokers MQTT (EMQX, Mosquitto, BunkerM). Les payloads sont automatiquement parsés et les attributs des assets sont mis à jour.
|
||||
|
||||
**Configuration via Manager UI** (`https://openremote.digitribe.fr/manager/`) :
|
||||
1. Se connecter avec `admin/Digitribe972`
|
||||
2. Choisir le realm `smartcity`
|
||||
3. **Assets → Agents → + Add Agent**
|
||||
4. Type : **MQTT Agent**
|
||||
5. Configurer :
|
||||
- **MQTT Broker URI** : `tcp://emqx_emqx_1:1883` (réseau smartcity-shared)
|
||||
- **Topic Filter** : `city/sensors/#`
|
||||
- **QoS** : 1
|
||||
- **Enabled** : ✅
|
||||
|
||||
### 7️⃣ Flux Prometheus — Métriques
|
||||
|
||||
| Service | Endpoint `/metrics` | Scrape |
|
||||
### 6️⃣ Flux Prometheus — Métriques
|
||||
| Service | Endpoint `/metrics` | Statut |
|
||||
|---------|---------------------|--------|
|
||||
| Simulator | `localhost:8001` | ✅ |
|
||||
| EMQX | `emqx_emqx_1:8081/api/v5/metrics` | ✅ |
|
||||
| Stellio | `stellio-api-gateway:8080/actuator/prometheus` | ✅ |
|
||||
| FROST | `frost_http-web-1:8080/metrics` | ✅ |
|
||||
| InfluxDB | `smart-city-influxdb:8086/metrics` | ✅ |
|
||||
| Redpanda | `smart-city-redpanda-console:8080/public_metrics` | ✅ |
|
||||
| OpenRemote | `openremote-manager-1:8080/actuator/prometheus` | ✅ |
|
||||
| Grafana | `smart-city-grafana:3000/metrics` | ✅ |
|
||||
| IoT Agent | `iot-agent:4041/metrics` | ⚠️ À vérifier |
|
||||
| QuantumLeap | `quantum-leap:8668/metrics` | ⚠️ À vérifier |
|
||||
|
||||
---
|
||||
|
||||
## Tableau Récapitulatif
|
||||
## Nouveaux Composants (06/05/2026)
|
||||
|
||||
### 🤖 IoT Agent JSON
|
||||
- **Rôle** : Pont entre MQTT et NGSI-LD (Orion-LD / Stellio)
|
||||
- **Port** : 4041
|
||||
- **Statut** : ❌ En cours de réparation (erreur MongoDB)
|
||||
- **Correction** : Fournir `IOTA_MONGO_URL=mongodb://mongodb:27017/iotagent`
|
||||
|
||||
### 📈 QuantumLeap
|
||||
- **Rôle** : Analytics NGSI-LD → CrateDB
|
||||
- **Port** : 8668
|
||||
- **Statut** : ✅ Fonctionnel (interne), port non exposé sur l'hôte
|
||||
- **Action** : Exposer le port dans docker-compose
|
||||
|
||||
### 🗄️ CrateDB
|
||||
- **Rôle** : Base de données temporelle PostgreSQL-compatible
|
||||
- **Port** : 4200 (UI), 5432 (PostgreSQL)
|
||||
- **Statut** : ✅ Opérationnel
|
||||
- **Usage** : Stockage des séries temporelles depuis QuantumLeap
|
||||
|
||||
---
|
||||
|
||||
## Tableau Récapitulatif (Mise à jour)
|
||||
|
||||
| Composant | Technologie | Port | Statut |
|
||||
|-----------|-------------|------|--------|
|
||||
@@ -182,42 +183,54 @@ L'agent MQTT d'OpenRemote souscrit aux topics `city/sensors/#` sur les brokers M
|
||||
| EMQX | MQTT Broker | 11883 | ✅ Connecté |
|
||||
| Mosquitto | MQTT Broker | 1883 | ✅ Connecté |
|
||||
| BunkerM | MQTTS Broker | 1900 | ✅ Connecté |
|
||||
| Orion-LD | NGSI-LD Broker | 1026 | ✅ Données |
|
||||
| Stellio | NGSI-LD Broker | 8080 | ✅ Données |
|
||||
| FROST-Server | SensorThings API | 8080 | ✅ Données |
|
||||
| OpenRemote | IoT Platform | 8080 | ✅ UI OK |
|
||||
| **IoT Agent** | **NGSI-LD Bridge** | **4041** | **❌ Erreur MongoDB** |
|
||||
| Orion-LD | NGSI-LD Broker | 1026 | ⚠️ À vérifier |
|
||||
| Stellio | NGSI-LD Broker | 8080 | ⚠️ Ports occupés par OpenRemote |
|
||||
| FROST-Server | SensorThings API | 8080 | ⚠️ À vérifier |
|
||||
| **QuantumLeap** | **NGSI-LD → CrateDB** | **8668** | **✅ Interne** |
|
||||
| **CrateDB** | **PostgreSQL Time-Series** | **4200/5432** | **✅ Opérationnel** |
|
||||
| OpenRemote | IoT Platform | 8080 | ⚠️ 403 (Service Account) |
|
||||
| InfluxDB | Time Series DB | 8086 | ✅ Bucket iot_data |
|
||||
| Redpanda | Kafka-compatible | 8082 REST | ✅ Topics actifs |
|
||||
| Pulsar | Event Streaming | 6650 | ✅ Connecté |
|
||||
| Prometheus | Metrics | 9090 (conf) | ⏳ Container arrêté |
|
||||
| Grafana | Visualisation | 3000 | ✅ Dashboards |
|
||||
| Grafana | Visualisation | 3001 | ✅ Dashboards + CrateDB |
|
||||
| GeoServer | Geo Data | 8080 | ✅ REST OK |
|
||||
| MapStore | Cartographie | 8080 | ✅ WMS/WMTS |
|
||||
|
||||
---
|
||||
|
||||
## Actions Prioritaires
|
||||
|
||||
1. **Corriger IoT Agent** : Ajouter MongoDB et configurer `IOTA_MONGO_URL`
|
||||
2. **Exposer QuantumLeap** : Mapper le port 8668 dans docker-compose
|
||||
3. **Déployer Orion-LD** : Créer le service s'il n'existe pas
|
||||
4. **Libérer les ports** : Résoudre le conflit Stellio/OpenRemote sur le port 8080
|
||||
5. **Configurer CrateDB** : Créer les tables pour QuantumLeap
|
||||
6. **Mettre à jour le simulateur** : `ENABLE_PULSAR=false` (recommandé)
|
||||
|
||||
---
|
||||
|
||||
## Commandes Utiles
|
||||
|
||||
```bash
|
||||
# Redémarrer le service de distribution Pulsar
|
||||
cd ~/smart-city-digital-twin-martinique
|
||||
docker build -t smart-city-pulsar-distribution:latest -f pulsar/Dockerfile pulsar/
|
||||
docker compose -f docker-compose.yml -f docker-compose.distribution.yml up -d pulsar-distribution
|
||||
# Vérifier IoT Agent
|
||||
curl -s http://localhost:4041/iot/services -H 'fiware-service: smartcity'
|
||||
|
||||
# Redémarrer Prometheus (prometheus-brokers)
|
||||
cd ~/smart-city-digital-twin-martinique
|
||||
docker compose up -d prometheus-brokers
|
||||
# Vérifier QuantumLeap
|
||||
curl -s http://localhost:8668/version
|
||||
|
||||
# Lancer le consumer Redpanda (host)
|
||||
cd ~/smart-city-digital-twin-martinique
|
||||
python3 redpanda/consumer.py
|
||||
# Vérifier CrateDB
|
||||
psql -h localhost -p 5432 -U crate -c "SELECT * FROM ql_entities LIMIT 5;"
|
||||
|
||||
# Vérifier les topics Redpanda
|
||||
curl -s http://localhost:8082/topics
|
||||
# Vérifier Orion-LD
|
||||
curl -s http://localhost:1026/version
|
||||
|
||||
# Vérifier les métriques simulator
|
||||
curl -s http://localhost:8001/metrics | grep "^simulator_"
|
||||
|
||||
# Logs distribution service
|
||||
docker logs -f smart-city-pulsar-distribution
|
||||
# Voir les logs IoT Agent
|
||||
docker logs smart-city-iot-agent --tail 30
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Fichiers associés :**
|
||||
- Simulateur : `~/smart-city-digital-twin-martinique/simulator.py`
|
||||
- Dashboard Grafana : `~/smart-city-digital-twin-martinique/grafana_dashboard_smartcity.json`
|
||||
- Ce diagramme : `~/smart-city-digital-twin-martinique/data-flow-diagram.md`
|
||||
- Session Resume : `~/smart-city-digital-twin-martinique/session_resume_2026-05-07.md`
|
||||
|
||||
Reference in New Issue
Block a user