# Smart City Digital Twin Martinique — Diagramme des Flux de Données

**Dernière mise à jour :** 05 Mai 2026  
**Projet :** Smart City Digital Twin Martinique

---

## Architecture Globale

```mermaid
graph TB
    subgraph Simulateur["🖥️  Simulateur (Host Python)"]
        SIM[Smart City Simulator<br/>10 capteurs<br/>Intervalle: configurable]
    end

    subgraph MQTT_Brokers["📡  MQTT Brokers"]
        EMQ[EMQX<br/>port 11883]
        MOS[Mosquitto<br/>port 1883]
        BUN[BunkerM<br/>port 1900<br/>MQTTS/TLS]
    end

    subgraph Stream["⚡ Event Streaming"]
        PUL[Pulsar<br/>port 6650<br/>Topics: smartcity-*]
        RED[Redpanda<br/>port 8082 REST<br/>Topics: traffic, air-quality, ...]
    end

    subgraph CB["🔗  Context Brokers"]
        ORI[Orion-LD<br/>NGSI-LD<br/>port 1026]
        STE[Stellio<br/>NGSI-LD<br/>port 8080]
        FRO[FROST-Server<br/>SensorThings<br/>port 8080]
    end

    subgraph Storage["💾  Stockage & Métriques"]
        INF[InfluxDB<br/>Bucket: iot_data<br/>port 8086]
        PRO[Prometheus<br/>Scrape: /metrics<br/>port 9090]
        GEO[GeoServer<br/>WMS/WFS/WMTS<br/>port 8080]
    end

    subgraph IoT_Platform["🏢  Plateforme IoT"]
        ORM[OpenRemote Manager<br/>MQTT Agent<br/>port 8080]
        KC[Keycloak<br/>port 8080]
    end

    subgraph VIZ["📊  Visualisation"]
        GRA[Grafana<br/>Dashboards<br/>port 3000]
        MAP[MapStore<br/>WMS/WFS<br/>port 8080]
    end

    subgraph Distribution["🔄  Distribution Service"]
        DIST[Pulsar Distribution<br/>Pulsar → Brokers]
    end

    subgraph Consumer["📥  Redpanda Consumer"]
        RCONS[Redpanda → InfluxDB<br/>REST → InfluxDB]
    end

    %% ── Flux Simulateur ──────────────────────────────────────────────────
    SIM -->|"1️⃣  MQTT publish<br/>city/sensors/{type}/{id}"| EMQ
    SIM -->|"1️⃣  MQTT publish"| MOS
    SIM -->|"1️⃣  MQTT publish"| BUN
    SIM -->|"2️⃣  HTTP POST<br/>NGSI-LD"| ORI
    SIM -->|"2️⃣  HTTP POST<br/>NGSI-LD"| STE
    SIM -->|"2️⃣  HTTP POST<br/>SensorThings"| FRO
    SIM -->|"3️⃣  Pulsar client<br/>pulsar://localhost:6650"| PUL
    SIM -->|"4️⃣  HTTP REST Proxy<br/>localhost:8082/topics/"| RED
    SIM -->|"5️⃣  InfluxDB v2 API<br/>async non-bloquant"| INF

    %% ── Flux Distribution (Pulsar → Brokers) ──────────────────────────────
    PUL -->|"Consomme<br/>smartcity-*"| DIST
    DIST -->|"Republish<br/>MQTT"| EMQ
    DIST -->|"Republish<br/>MQTT"| MOS
    DIST -->|"Republish<br/>NGSI-LD"| ORI
    DIST -->|"Republish<br/>NGSI-LD"| STE
    DIST -->|"Republish<br/>SensorThings"| FRO

    %% ── Flux Redpanda → InfluxDB ──────────────────────────────────────────
    RED -->|"REST poll<br/>topics/{name}/offsets"| RCONS
    RCONS -->|"Line Protocol<br/>Write API"| INF

    %% ── OpenRemote MQTT Agent ──────────────────────────────────────────────
    EMQ -->|"6️⃣  Subscribe<br/>city/sensors/#"| ORM
    MOS -->|"6️⃣  Subscribe"| ORM
    BUN -->|"6️⃣  Subscribe"| ORM

    %% ── Métriques Prometheus ────────────────────────────────────────────────
    SIM -->|"7️⃣  /metrics<br/>port 8001"| PRO
    EMQ -->|"/api/v5/metrics"| PRO
    STE -->|"/actuator/prometheus"| PRO
    FRO -->|"/metrics"| PRO
    INF -->|"/metrics"| PRO
    RED -->|"/public_metrics"| PRO
    ORM -->|"/actuator/prometheus"| PRO
    GRA -->|"/metrics"| PRO

    %% ── Visualisation ─────────────────────────────────────────────────────
    INF -->|"Datasources<br/>Flux IoT"| GRA
    ORI -->|"NGSI-LD<br/>Datasource"| GRA
    STE -->|"NGSI-LD<br/>Datasource"| GRA
    FRO -->|"SensorThings<br/>Datasource"| GRA
    GEO -->|"WMS/WMTS"| MAP
    ORM -->|MapSettings<br/>Martinique| MAP
    ORM -->|"Live assets<br/>REST"| GRA
```

---

## Flux Détaillés

### 1️⃣ Flux MQTT — Brokers

| Broker | Port | Protocol | Topics |
|--------|------|----------|--------|
| EMQX | 11883 | MQTT | `city/sensors/{type}/{id}` |
| Mosquitto | 1883 | MQTT | `city/sensors/{type}/{id}` |
| BunkerM | 1900 | MQTTS (TLS) | `city/sensors/{type}/{id}` |

Le simulateur publie simultanément sur les 3 brokers.

### 2️⃣ Flux HTTP REST — Context Brokers

| Broker | Format | Port | Topics |
|--------|--------|------|--------|
| Orion-LD | NGSI-LD | 1026 | Entités par type |
| Stellio | NGSI-LD | 8080 | Entités par type |
| FROST-Server | SensorThings | 8080 | Things → Datastreams → Observations |

### 3️⃣ Flux Pulsar — Event Streaming

- **Topics** : `persistent://public/default/smartcity-traffic`, `smartcity-airquality`, `smartcity-parking`, `smartcity-noise`, `smartcity-weather`, `smartcity-light`
- **Port binaire** : `6650` (connectable depuis le host)
- **Distribution** : Le service `pulsar-distribution` consomme ces topics et republie vers les brokers MQTT et context brokers

### 4️⃣ Flux Redpanda — Kafka-compatible REST

- **REST Proxy** : `http://localhost:8082`
- **Topics** : `traffic`, `air-quality`, `parking`, `noise`, `weather`, `air-quality`
- **Payload** : Base64(JSON) dans `{"records": [{"value": "<base64>"}]}`
- **Consumer** : `redpanda/consumer.py` — poll toutes les 10s et écrit dans InfluxDB

### 5️⃣ Flux InfluxDB — Temps Réel

- **API** : `http://localhost:8086/api/v2/write`
- **Bucket** : `iot_data`
- **Org** : `digitribe`
- **Mode** : Asynchrone (thread daemon) pour ne pas bloquer le publish MQTT

### 6️⃣ OpenRemote — MQTT Agent

L'agent MQTT d'OpenRemote souscrit aux topics `city/sensors/#` sur les brokers MQTT (EMQX, Mosquitto, BunkerM). Les payloads sont automatiquement parsés et les attributs des assets sont mis à jour.

**Configuration via Manager UI** (`https://openremote.digitribe.fr/manager/`) :
1. Se connecter avec `admin/Digitribe972`
2. Choisir le realm `smartcity`
3. **Assets → Agents → + Add Agent**
4. Type : **MQTT Agent**
5. Configurer :
   - **MQTT Broker URI** : `tcp://emqx_emqx_1:1883` (réseau smartcity-shared)
   - **Topic Filter** : `city/sensors/#`
   - **QoS** : 1
   - **Enabled** : ✅

### 7️⃣ Flux Prometheus — Métriques

| Service | Endpoint `/metrics` | Scrape |
|---------|---------------------|--------|
| Simulator | `localhost:8001` | ✅ |
| EMQX | `emqx_emqx_1:8081/api/v5/metrics` | ✅ |
| Stellio | `stellio-api-gateway:8080/actuator/prometheus` | ✅ |
| FROST | `frost_http-web-1:8080/metrics` | ✅ |
| InfluxDB | `smart-city-influxdb:8086/metrics` | ✅ |
| Redpanda | `smart-city-redpanda-console:8080/public_metrics` | ✅ |
| OpenRemote | `openremote-manager-1:8080/actuator/prometheus` | ✅ |
| Grafana | `smart-city-grafana:3000/metrics` | ✅ |

---

## Tableau Récapitulatif

| Composant | Technologie | Port | Statut |
|-----------|-------------|------|--------|
| Simulator | Python + paho-mqtt | Host:8001 (metrics) | ✅ Actif |
| EMQX | MQTT Broker | 11883 | ✅ Connecté |
| Mosquitto | MQTT Broker | 1883 | ✅ Connecté |
| BunkerM | MQTTS Broker | 1900 | ✅ Connecté |
| Orion-LD | NGSI-LD Broker | 1026 | ✅ Données |
| Stellio | NGSI-LD Broker | 8080 | ✅ Données |
| FROST-Server | SensorThings API | 8080 | ✅ Données |
| OpenRemote | IoT Platform | 8080 | ✅ UI OK |
| InfluxDB | Time Series DB | 8086 | ✅ Bucket iot_data |
| Redpanda | Kafka-compatible | 8082 REST | ✅ Topics actifs |
| Pulsar | Event Streaming | 6650 | ✅ Connecté |
| Prometheus | Metrics | 9090 (conf) | ⏳ Container arrêté |
| Grafana | Visualisation | 3000 | ✅ Dashboards |
| GeoServer | Geo Data | 8080 | ✅ REST OK |
| MapStore | Cartographie | 8080 | ✅ WMS/WMTS |

---

## Commandes Utiles

```bash
# Redémarrer le service de distribution Pulsar
cd ~/smart-city-digital-twin-martinique
docker build -t smart-city-pulsar-distribution:latest -f pulsar/Dockerfile pulsar/
docker compose -f docker-compose.yml -f docker-compose.distribution.yml up -d pulsar-distribution

# Redémarrer Prometheus (prometheus-brokers)
cd ~/smart-city-digital-twin-martinique
docker compose up -d prometheus-brokers

# Lancer le consumer Redpanda (host)
cd ~/smart-city-digital-twin-martinique
python3 redpanda/consumer.py

# Vérifier les topics Redpanda
curl -s http://localhost:8082/topics

# Vérifier les métriques simulator
curl -s http://localhost:8001/metrics | grep "^simulator_"

# Logs distribution service
docker logs -f smart-city-pulsar-distribution
```