Files
smart-city-digital-twin-mar…/DOCKER-ARCHITECTURE-2026-05-05.md
Eric FELIXINE 5f9da72aa7 Architecture: Add Message Broker (Pulsar/Redpanda) integration
- New section: Message Broker (Pulsar/Redpanda)
- Updated Mermaid diagram with Message_Broker_Network
- Added Scorpio (FIWARE) native Kafka integration note
- New data flow: MQTT -> Message Broker -> Backends
- Updated connections list (5. Message Broker)
2026-05-05 00:25:51 -04:00

188 lines
10 KiB
Markdown

# Smart City Digital Twin - Cartographie Docker (Architecture & Réseaux)
**Date** : 05 mai 2026
**Projet** : `smart-city-digital-twin-martinique`
**Auteur** : Éric FELIXINE (via Hermes Agent)
---
## 1. Vue d'ensemble
Cette cartographie présente l'architecture Docker complète du jumeau numérique Smart City (Martinique), incluant les conteneurs, images, réseaux et ports exposés.
---
## 2. Liste des Conteneurs Actifs (Projet Smart City)
| Conteneur | Image | Réseaux | Ports |
|-----------|-------|----------|-------|
| `frost-api-8090` | `fraunhoferiosb/frost-server-http:latest` | `bridge`, `frost_http_default` | `0.0.0.0:8090→8080/tcp` |
| `stellio-api-exposed2` | `stellio-api-gateway:exposed` | `stellio-context-broker_default` | `0.0.0.0:8087→8080/tcp` |
| `mosquitto-exporter` | `sapcc/mosquitto-exporter:latest` | `smartcity-shared`, `traefik-public` | `0.0.0.0:9234→9234/tcp` |
| `prometheus-brokers` | `prom/prometheus:latest` | `smartcity-shared`, `traefik-public` | `9090/tcp` |
| `digital-twin-grafana` | `grafana/grafana:10.2.0` | `fiware-gis-quickstart_fiware`, `frost_http_default`, `smartcity-shared`, `traefik-public`, `digital-twin_digital-twin`, `docker_default` | `0.0.0.0:3001→3000/tcp` |
| `geoserver_stack-geoserver-1` | `oscarfonts/geoserver:2.25.2` | `frost_http_default`, `traefik-public` | `8080/tcp` |
| `grafana_stack-grafana-1` | `grafana/grafana:latest` | `frost_http_default`, `traefik-public` | `3000/tcp` |
| `frost_http-database-1` | `postgis/postgis:16-3.4-alpine` | `frost_http_default` | `5432/tcp` |
| `openremote-keycloak-1` | `openremote/keycloak:latest` | `openremote_default`, `smartcity-shared` | `8080/tcp`, `8443/tcp` |
| `openremote-manager-1` | `openremote/manager:latest` | `smartcity-shared`, `openremote_default` | `1883/tcp`, `8080/tcp`, `8443/tcp`, `127.0.0.1:8405→8405/tcp` |
| `openremote-postgresql-1` | `openremote/postgresql:latest-slim` | `openremote_default` | `5432/tcp`, `8008/tcp`, `8081/tcp` |
| `traefik` | `traefik:v3.0` | `openremote_default`, `traefik-public` | `0.0.0.0:80→80/tcp`, `0.0.0.0:443→443/tcp`, `0.0.0.0:1884→1884/tcp`, `127.0.0.1:9080→8080/tcp` |
| `mosquitto-traefik` | `eclipse-mosquitto:2.0` | `smartcity-shared`, `traefik-public` | `0.0.0.0:1883→1883/tcp`, `127.0.0.1:38084→8081/tcp`, `127.0.0.1:38884→8883/tcp` |
| `emqx_emqx_1` | `emqx/emqx:latest` | `emqx_default`, `smartcity-shared`, `traefik-public` | `4370/tcp`, `5369/tcp`, `8083-8084/tcp`, `0.0.0.0:11883→1883/tcp`, `0.0.0.0:18081→8081/tcp`, `0.0.0.0:18883→8883/tcp`, `0.0.0.0:38083→18083/tcp` |
| `stellio-search-service` | `stellio/stellio-search-service:latest-dev` | `stellio-context-broker_default`, `traefik-public`, `smartcity-shared` | `8083/tcp` |
| `stellio-subscription-service` | `stellio/stellio-subscription-service:latest-dev` | `smartcity-shared`, `stellio-context-broker_default`, `traefik-public` | `8084/tcp` |
| `stellio-kafka` | `confluentinc/cp-kafka:8.1.0` | `stellio-context-broker_default` | `9092/tcp`, `0.0.0.0:29092→29092/tcp` |
| `stellio-postgres` | `stellio/stellio-timescale-postgis:16-2.24.0-3.6` | `stellio-context-broker_default` | `5432/tcp` |
| `stellio-api-gateway` | `stellio/stellio-api-gateway:latest-dev` | `stellio-context-broker_default`, `traefik-public`, `smartcity-shared` | `8080/tcp` |
| `digital-twin-nodered` | `nodered/node-red:3.1` | `docker_default`, `smartcity-shared`, `traefik` | `0.0.0.0:1880→1880/tcp` |
| `digital-twin-postgis` | `postgis/postgis:15-3.4` | `digital-twin_digital-twin`, `docker_default`, `smartcity-shared` | `0.0.0.0:5433→5432/tcp` |
| `digital-twin-connector` | `python:3.11-slim` | `digital-twin_digital-twin` | - |
| `digital-twin-influxdb` | `influxdb:2.7-alpine` | `digital-twin_digital-twin`, `docker_default`, `smartcity-shared` | `0.0.0.0:8086→8086/tcp` |
| `fiware-gis-quickstart-orionproxy-1` | `fiware-gis-quickstart-orionproxy` | `fiware-gis-quickstart_fiware` | `127.0.0.1:1026→80/tcp` |
| `fiware-gis-quickstart-orion-1` | `quay.io/fiware/orion-ld` | `fiware-gis-quickstart_fiware`, `smartcity-shared`, `traefik-public` | `127.0.0.1:2026→1026/tcp` |
| `honcho-grafana-1` | `grafana/grafana:11.4.0` | `honcho_default` | `127.0.0.1:3088→3000/tcp` |
| `honcho-prometheus-1` | `prom/prometheus:v3.2.1` | `honcho_default` | `127.0.0.1:9091→9090/tcp` |
---
## 3. Liste des Réseaux Docker (Projet)
| Réseau | Conteneurs Connectés |
|---------|----------------------|
| `smartcity-shared` | `digital-twin-grafana`, `mosquitto-exporter`, `prometheus-brokers`, `openremote-keycloak-1`, `openremote-manager-1`, `stellio-search-service`, `stellio-subscription-service`, `stellio-api-gateway`, `digital-twin-nodered`, `digital-twin-postgis`, `digital-twin-influxdb`, `emqx_emqx_1`, `fiware-gis-quickstart-orion-1` |
| `frost_http_default` | `frost-api-8090`, `geoserver_stack-geoserver-1`, `grafana_stack-grafana-1`, `digital-twin-grafana`, `frost_http-database-1` |
| `stellio-context-broker_default` | `stellio-api-exposed2`, `stellio-search-service`, `stellio-subscription-service`, `stellio-kafka`, `stellio-postgres`, `stellio-api-gateway` |
| `traefik-public` | `digital-twin-grafana`, `mosquitto-exporter`, `prometheus-brokers`, `geoserver_stack-geoserver-1`, `stellio-search-service`, `stellio-subscription-service`, `stellio-api-gateway`, `emqx_emqx_1`, `digital-twin-nodered`, `traefik` |
| `digital-twin_digital-twin` | `digital-twin-grafana`, `digital-twin-postgis`, `digital-twin-connector`, `digital-twin-influxdb` |
| `docker_default` | `digital-twin-grafana`, `digital-twin-postgis`, `digital-twin-influxdb`, `digital-twin-nodered` |
| `fiware-gis-quickstart_fiware` | `fiware-gis-quickstart-orionproxy-1`, `fiware-gis-quickstart-orion-1`, `digital-twin-grafana` |
| `openremote_default` | `openremote-keycloak-1`, `openremote-manager-1`, `openremote-postgresql-1`, `traefik` |
| `emqx_default` | `emqx_emqx_1` |
| `honcho_default` | `honcho-grafana-1`, `honcho-prometheus-1` |
| `bridge` | `frost-api-8090` |
---
## 4. Diagramme d'Architecture (Mermaid)
```mermaid
graph TD
subgraph Traefik_Public [Traefik Public Network]
TR[Traefik\n:80/:443] --- DG[digital-twin-grafana\n:3001]
TR --- MO[mosquitto-exporter\n:9234]
TR --- PR[prometheus-brokers\n:9090]
TR --- GE[geoserver\n:8080]
TR --- SG[stellio-search\n:8083]
TR --- SS[stellio-subscription\n:8084]
TR --- AG[stellio-api-gateway\n:8087]
TR --- EM[emqx\n:11883/:18883]
TR --- NR[digital-twin-nodered\n:1880]
end
subgraph SmartCity_Shared [SmartCity Shared Network]
DG --- IF[digital-twin-influxdb\n:8086]
DG --- PG[digital-twin-postgis\n:5433]
DG --- ORK[openremote-keycloak\n:8080]
DG --- ORM[openremote-manager\n:8405]
DG --- ST[stellio-services]
DG --- EM
end
subgraph FROST_Network [FROST Default Network]
FR[frost-api-8090\n:8090] --- FD[frost-http-database\n:5432]
FR --- GE
FR --- DG
end
subgraph Stellio_Network [Stellio Context Broker Network]
ST[stellio-services] --- SA[stellio-api-exposed2\n:8087]
ST --- SK[stellio-kafka\n:29092]
ST --- SP[stellio-postgres\n:5432]
end
subgraph Orion_Network [FIWARE Orion Network]
OR[fw-orion-1\n:2026] --- OP[fw-orion-proxy-1\n:1026]
OP --- DG
end
subgraph OpenRemote_Network [OpenRemote Default Network]
ORK --- ORM
ORM --- ODB[openremote-postgresql\n:5432]
ORK --- TR
end
subgraph DigitalTwin_Network [Digital Twin Custom Network]
DG --- IF
DG --- PG
DG --- DC[digital-twin-connector]
end
%% NOUVEAU : Message Broker Network
subgraph Message_Broker_Network [Message Broker (Pulsar/Redpanda)]
MB[Message Broker\n(Pulsar :6650 / Redpanda :9092)]
MB --- SK %% Kafka-compatibilité (Stellio Kafka)
end
%% Flux de données avec Message Broker
EM -->|MQTT| MB
MO -->|Metrics| MB
MB -->|Topics| OR
MB -->|Topics| ST
MB -->|Metrics| PR
MB -->|Topics| IF
OR -->|NGSI-LD| FR
OR -->|NGSI-LD| ST
IF -->|InfluxDB| DG
DG -->|Grafana Dashboards| User[Utilisateur]
%% Note Scorpio integration
SC[Scorpio (FIWARE)\n:Kafka native] -.->|Kafka| MB
```
---
## 5. Message Broker (Pulsar/Redpanda) - NOUVEAU
Pour faciliter l'ingestion et le routage des données vers plusieurs backends (Prometheus, Context Brokers, etc.), un **message broker** sera ajouté entre les brokers MQTT et les services en aval.
### Options préférées :
- **Apache Pulsar** : Alternative cloud-native à Kafka, avec support natif des topics persistants et de la messagerie multi-tenant.
- **Redpanda** : Compatible Kafka, mais sans dépendance ZooKeeper, plus simple à déployer.
### Intégration avec FIWARE Scorpio :
- **Scorpio** (Context Broker FIWARE) intègre nativement **Kafka**, ce qui facilitera l'interconnexion avec le message broker.
### Nouveau flux de données :
```
MQTT Brokers (EMQX, Mosquitto, BunkerM)
Message Broker (Pulsar/Redpanda)
↓ ↓ ↓
├─→ Context Brokers (Orion-LD, Stellio, Scorpio)
├─→ Prometheus (via exporter)
├─→ Time-Series DB (InfluxDB, CrateDB)
└─→ autres backends (OpenRemote, GeoServer, etc.)
```
## 6. Connexions Clés
1. **Traefik** (`:80`/`:443`) : Reverse proxy pour tous les services exposés à l'hôte.
2. **Brokers MQTT** (Mosquitto `:1883`, EMQX `:11883`) : Réception des données du simulateur et des capteurs IoT.
3. **Message Broker** (Pulsar `:6650`/Redpanda `:9092`) : **NOUVEAU** - Ingest et routage vers les backends.
4. **Context Brokers** (Orion-LD `:2026`, Stellio `:8087`, Scorpio `:?`) : Reçoivent les données NGSI-LD (Scorpio via Kafka natif).
5. **FROST-Server** (`:8090`) : Stockage des données OGC SensorThings, connecté à PostgreSQL (`frost_http-database-1`).
6. **Grafana** (`:3001`) : Visualisation des données depuis InfluxDB, Prometheus, et (via adaptateurs) Orion-LD/Stellio.
7. **OpenRemote** (`:8080`) : Gestion des assets IoT, authentification via Keycloak, proxyé par Traefik.
8. **GeoServer** (`:8080`) : Serveur de tuiles cartographiques, connecté au réseau FROST.
9. **Node-RED** (`:1880`) : Alternative possible pour le simulateur IoT (projeté).
---
## 6. Références
- **Projet** : `~/smart-city-digital-twin-martinique/`
- **Skills** : `postman-fiware`, `smart-city-traceability-setup`, `openremote-map-configuration`
- **Session Resume** : `session_resume_2026-05-04.md`
- **Documentation Grafana** : `GRAFANA-STATUS-FINAL.md`
---
*Cartographie générée automatiquement le 05 mai 2026 à 14:30 (UTC-4) par Hermes Agent.*