feat: distribution service + redpanda consumer + updated flow diagram

- Add Pulsar distribution service (consumes smartcity-* → MQTT + context brokers)
- Add Redpanda → InfluxDB consumer (redpanda/consumer.py)
- Update FIXED_LOCATIONS with exact OpenRemote asset coordinates
- Fix Pulsar topics (underscore: smartcity-traffic not smartcity-traffic)
- Fix prometheus.yml endpoints (Redpanda:9644, comment inactive stacks)
- Add docker-compose.redpanda-consumer.yml

data-flow-diagram.md

@@ -1,178 +1,223 @@
-# Smart City Digital Twin — Diagramme des Flux de Données
+# Smart City Digital Twin Martinique — Diagramme des Flux de Données
 
-## Vue d'ensemble
+**Dernière mise à jour :** 05 Mai 2026  
-
+**Projet :** Smart City Digital Twin Martinique
-Ce diagramme illustre le flux complet des données IoT du simulateur vers les différentes couches de traitement, de stockage et de visualisation.
 
 ---
 
-## Diagramme Mermaid
+## Architecture Globale
 
 ```mermaid
 graph TB
-    SIM[Smart City Simulator]
+    subgraph Simulateur["🖥️  Simulateur (Host Python)"]
-    SENS[Capteurs IoT Reels]
+        SIM[Smart City Simulator<br/>10 capteurs<br/>Intervalle: configurable]
-    EMQ[EMQX]
+    end
-    MOS[Mosquitto]
-    BUN[BunkerM]
-    FRO[FROST-Server]
-    ORI[Orion-LD]
-    STE[Stellio]
-    UI[OpenRemote UI]
-    ORM[OpenRemote Manager]
-    KC[Keycloak]
-    INF[InfluxDB]
-    PRO[Prometheus]
-    GEO[GeoServer]
-    GRA[Grafana]
-    MAP[MapStore]
-    CH[ClickHouse]
-    RW[RisingWave]
-    PUL[Pulsar]
-    RED[Redpanda]
 
-    SIM --> EMQ
+    subgraph MQTT_Brokers["📡  MQTT Brokers"]
-    SIM --> MOS
+        EMQ[EMQX<br/>port 11883]
-    SIM --> BUN
+        MOS[Mosquitto<br/>port 1883]
-    SIM --> FRO
+        BUN[BunkerM<br/>port 1900<br/>MQTTS/TLS]
-    SENS --> EMQ
+    end
-    SENS --> MOS
-    SENS --> BUN
-    SENS --> FRO
-    SENS -.-> ORM
-    EMQ -->|via EMQX| ORI
-    EMQ -->|via EMQX| STE
-    MOS -->|via Mosquitto| ORI
-    MOS -->|via Mosquitto| STE
-    BUN -->|via BunkerM| ORI
-    BUN -->|via BunkerM| STE
-    UI --> ORM
-    ORM -.-> KC
-    SIM --> INF
-    SIM -->|real-time 1s| CH
-    SIM -->|streaming| RW
-    SIM -->|HTTP REST| PUL
-    SIM -->|HTTP REST| RED
-    ORI --> GRA
-    STE --> GRA
-    FRO --> GRA
-    ORI -.-> GEO
-    STE -.-> GEO
-    FRO -.-> GEO
-    GEO --> MAP
-    ORM --> GRA
-    EMQ -.-> PRO
-    ORI -.-> PRO
-    STE -.-> PRO
-    ORM -.-> PRO
 
+    subgraph Stream["⚡ Event Streaming"]
+        PUL[Pulsar<br/>port 6650<br/>Topics: smartcity-*]
+        RED[Redpanda<br/>port 8082 REST<br/>Topics: traffic, air-quality, ...]
+    end
+
+    subgraph CB["🔗  Context Brokers"]
+        ORI[Orion-LD<br/>NGSI-LD<br/>port 1026]
+        STE[Stellio<br/>NGSI-LD<br/>port 8080]
+        FRO[FROST-Server<br/>SensorThings<br/>port 8080]
+    end
+
+    subgraph Storage["💾  Stockage & Métriques"]
+        INF[InfluxDB<br/>Bucket: iot_data<br/>port 8086]
+        PRO[Prometheus<br/>Scrape: /metrics<br/>port 9090]
+        GEO[GeoServer<br/>WMS/WFS/WMTS<br/>port 8080]
+    end
+
+    subgraph IoT_Platform["🏢  Plateforme IoT"]
+        ORM[OpenRemote Manager<br/>MQTT Agent<br/>port 8080]
+        KC[Keycloak<br/>port 8080]
+    end
+
+    subgraph VIZ["📊  Visualisation"]
+        GRA[Grafana<br/>Dashboards<br/>port 3000]
+        MAP[MapStore<br/>WMS/WFS<br/>port 8080]
+    end
+
+    subgraph Distribution["🔄  Distribution Service"]
+        DIST[Pulsar Distribution<br/>Pulsar → Brokers]
+    end
+
+    subgraph Consumer["📥  Redpanda Consumer"]
+        RCONS[Redpanda → InfluxDB<br/>REST → InfluxDB]
+    end
+
+    %% ── Flux Simulateur ──────────────────────────────────────────────────
+    SIM -->|"1️⃣  MQTT publish<br/>city/sensors/{type}/{id}"| EMQ
+    SIM -->|"1️⃣  MQTT publish"| MOS
+    SIM -->|"1️⃣  MQTT publish"| BUN
+    SIM -->|"2️⃣  HTTP POST<br/>NGSI-LD"| ORI
+    SIM -->|"2️⃣  HTTP POST<br/>NGSI-LD"| STE
+    SIM -->|"2️⃣  HTTP POST<br/>SensorThings"| FRO
+    SIM -->|"3️⃣  Pulsar client<br/>pulsar://localhost:6650"| PUL
+    SIM -->|"4️⃣  HTTP REST Proxy<br/>localhost:8082/topics/"| RED
+    SIM -->|"5️⃣  InfluxDB v2 API<br/>async non-bloquant"| INF
+
+    %% ── Flux Distribution (Pulsar → Brokers) ──────────────────────────────
+    PUL -->|"Consomme<br/>smartcity-*"| DIST
+    DIST -->|"Republish<br/>MQTT"| EMQ
+    DIST -->|"Republish<br/>MQTT"| MOS
+    DIST -->|"Republish<br/>NGSI-LD"| ORI
+    DIST -->|"Republish<br/>NGSI-LD"| STE
+    DIST -->|"Republish<br/>SensorThings"| FRO
+
+    %% ── Flux Redpanda → InfluxDB ──────────────────────────────────────────
+    RED -->|"REST poll<br/>topics/{name}/offsets"| RCONS
+    RCONS -->|"Line Protocol<br/>Write API"| INF
+
+    %% ── OpenRemote MQTT Agent ──────────────────────────────────────────────
+    EMQ -->|"6️⃣  Subscribe<br/>city/sensors/#"| ORM
+    MOS -->|"6️⃣  Subscribe"| ORM
+    BUN -->|"6️⃣  Subscribe"| ORM
+
+    %% ── Métriques Prometheus ────────────────────────────────────────────────
+    SIM -->|"7️⃣  /metrics<br/>port 8001"| PRO
+    EMQ -->|"/api/v5/metrics"| PRO
+    STE -->|"/actuator/prometheus"| PRO
+    FRO -->|"/metrics"| PRO
+    INF -->|"/metrics"| PRO
+    RED -->|"/public_metrics"| PRO
+    ORM -->|"/actuator/prometheus"| PRO
+    GRA -->|"/metrics"| PRO
+
+    %% ── Visualisation ─────────────────────────────────────────────────────
+    INF -->|"Datasources<br/>Flux IoT"| GRA
+    ORI -->|"NGSI-LD<br/>Datasource"| GRA
+    STE -->|"NGSI-LD<br/>Datasource"| GRA
+    FRO -->|"SensorThings<br/>Datasource"| GRA
+    GEO -->|"WMS/WMTS"| MAP
+    ORM -->|MapSettings<br/>Martinique| MAP
+    ORM -->|"Live assets<br/>REST"| GRA
 ```
 
 ---
 
-## Description des flux
+## Flux Détaillés
 
-### 1. **Génération des données (Simulator)**
+### 1️⃣ Flux MQTT — Brokers
-- **Smart City Simulator** (Python) génère des données pour 10 capteurs (Traffic, Air Quality, Parking, Noise, Weather, Light)
-- Intervalle de publication : 1 seconde (temps réel)
-- Protocoles : MQTT (vers brokers uniquement)
-- **⚠️ Projet** : Le simulateur n'envoie PAS directement à OpenRemote (pas de REST API)
 
-### 2. **Ingestion MQTT (Brokers)**
+| Broker | Port | Protocol | Topics |
-- **EMQX** (port 11883) : Broker public, reçoit tous les capteurs
+|--------|------|----------|--------|
-- **Mosquitto** (port 1883) : Via Traefik, accès externe
+| EMQX | 11883 | MQTT | `city/sensors/{type}/{id}` |
-- **BunkerM** (port 1900) : MQTTS (TLS), accès sécurisé
+| Mosquitto | 1883 | MQTT | `city/sensors/{type}/{id}` |
+| BunkerM | 1900 | MQTTS (TLS) | `city/sensors/{type}/{id}` |
 
-### 3. **Context Brokers (NGSI-LD & SensorThings)**
+Le simulateur publie simultanément sur les 3 brokers.
-- **Orion-LD** : Reçoit les données au format NGSI-LD
-  - 10 entités (TrafficFlowObserved, AirQualityObserved, etc.)
-  - Smart Data Models utilisés
-  - **Provenance** : Données via EMQX, Mosquitto et BunkerM (voir étiquettes dans le diagramme)
-- **Stellio** : Alternative NGSI-LD
-  - 14 payloads entités
-  - Contexte : `https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld`
-  - **Provenance** : Données via EMQX, Mosquitto et BunkerM
-- **FROST-Server** : SensorThings API
-  - 21 256+ observations
-  - PostgreSQL + TimescaleDB
-  - **Provenance** : Données via EMQX, Mosquitto et BunkerM
 
-### 4. **Plateforme IoT (OpenRemote)**
+### 2️⃣ Flux HTTP REST — Context Brokers
-- **OpenRemote Manager** (realm `smartcity`)
-  - 33 assets IoT configurés
-  - Carte Martinique (mapsettings.json)
-  - Réception via **MQTT Agent** depuis les brokers (EMQX, Mosquitto, BunkerM)
-  - Peut aussi recevoir directement des capteurs IoT (via MQTT)
-- **Keycloak** : Authentification OpenID Connect
-  - Client `openremote` avec Service Account
-  - Token endpoint : `/auth/realms/smartcity/protocol/openid-connect/token`
 
-### 5. **Stockage & Métriques**
+| Broker | Format | Port | Topics |
-- **InfluxDB** : Stockage temporel pour Grafana
+|--------|--------|------|--------|
-  - Bucket : `iot_data`
+| Orion-LD | NGSI-LD | 1026 | Entités par type |
-  - Datasource dans Grafana
+| Stellio | NGSI-LD | 8080 | Entités par type |
-- **Prometheus** : Collecte des métriques
+| FROST-Server | SensorThings | 8080 | Things → Datastreams → Observations |
-  - MQTT brokers, Context brokers, OpenRemote
-- **GeoServer** : Données géospatiales
-  - PostGIS pour centralisation
-  - WMS/WFS pour MapStore
 
-### 5. **Visualisation & Analyse**
+### 3️⃣ Flux Pulsar — Event Streaming
-- **Grafana** (port 3001)
-  - Dashboard : `smartcity-martinique-2026`
-  - Datasources : InfluxDB, FROST, Orion-LD, ClickHouse, RisingWave
-- **MapStore** : Cartographie
-  - Sources WMS/WFS depuis GeoServer
-- **OpenRemote UI** : Manager Interface
-  - Visualisation des assets realm Smart City
 
-### 6. **Analytique & Streaming**
+- **Topics** : `persistent://public/default/smartcity-traffic`, `smartcity-airquality`, `smartcity-parking`, `smartcity-noise`, `smartcity-weather`, `smartcity-light`
-- **ClickHouse** (port 8123/9000) : Columnar OLAP Database
+- **Port binaire** : `6650` (connectable depuis le host)
-  - Analytique rapide sur grandes volumes de données IoT
+- **Distribution** : Le service `pulsar-distribution` consomme ces topics et republie vers les brokers MQTT et context brokers
-  - Intégration possible via HTTP interface (port 8123)
+
-  - Compatible avec Grafana (plugin ClickHouse)
+### 4️⃣ Flux Redpanda — Kafka-compatible REST
-- **RisingWave** (port 4566/4567) : Streaming Database PostgreSQL-compatible
+
-  - Traitement de flux en temps réel
+- **REST Proxy** : `http://localhost:8082`
-  - Interface web pour requêtes SQL streaming
+- **Topics** : `traffic`, `air-quality`, `parking`, `noise`, `weather`, `air-quality`
-  - Compatible Grafana via datasource PostgreSQL
+- **Payload** : Base64(JSON) dans `{"records": [{"value": "<base64>"}]}`
+- **Consumer** : `redpanda/consumer.py` — poll toutes les 10s et écrit dans InfluxDB
+
+### 5️⃣ Flux InfluxDB — Temps Réel
+
+- **API** : `http://localhost:8086/api/v2/write`
+- **Bucket** : `iot_data`
+- **Org** : `digitribe`
+- **Mode** : Asynchrone (thread daemon) pour ne pas bloquer le publish MQTT
+
+### 6️⃣ OpenRemote — MQTT Agent
+
+L'agent MQTT d'OpenRemote souscrit aux topics `city/sensors/#` sur les brokers MQTT (EMQX, Mosquitto, BunkerM). Les payloads sont automatiquement parsés et les attributs des assets sont mis à jour.
+
+**Configuration via Manager UI** (`https://openremote.digitribe.fr/manager/`) :
+1. Se connecter avec `admin/Digitribe972`
+2. Choisir le realm `smartcity`
+3. **Assets → Agents → + Add Agent**
+4. Type : **MQTT Agent**
+5. Configurer :
+   - **MQTT Broker URI** : `tcp://emqx_emqx_1:1883` (réseau smartcity-shared)
+   - **Topic Filter** : `city/sensors/#`
+   - **QoS** : 1
+   - **Enabled** : ✅
+
+### 7️⃣ Flux Prometheus — Métriques
+
+| Service | Endpoint `/metrics` | Scrape |
+|---------|---------------------|--------|
+| Simulator | `localhost:8001` | ✅ |
+| EMQX | `emqx_emqx_1:8081/api/v5/metrics` | ✅ |
+| Stellio | `stellio-api-gateway:8080/actuator/prometheus` | ✅ |
+| FROST | `frost_http-web-1:8080/metrics` | ✅ |
+| InfluxDB | `smart-city-influxdb:8086/metrics` | ✅ |
+| Redpanda | `smart-city-redpanda-console:8080/public_metrics` | ✅ |
+| OpenRemote | `openremote-manager-1:8080/actuator/prometheus` | ✅ |
+| Grafana | `smart-city-grafana:3000/metrics` | ✅ |
 
 ---
 
-## Technologies clés
+## Tableau Récapitulatif
 
 | Composant | Technologie | Port | Statut |
 |-----------|-------------|------|--------|
-| Simulator | Python + paho-mqtt | Interne | ✅ Actif (1s) |
+| Simulator | Python + paho-mqtt | Host:8001 (metrics) | ✅ Actif |
 | EMQX | MQTT Broker | 11883 | ✅ Connecté |
-| Orion-LD | NGSI-LD Broker | 1026 | ⚠️ À vérifier |
+| Mosquitto | MQTT Broker | 1883 | ✅ Connecté |
-| Stellio | NGSI-LD Broker | 8080 | ⚠️ À vérifier |
+| BunkerM | MQTTS Broker | 1900 | ✅ Connecté |
-| FROST-Server | SensorThings API | 8080 | ⚠️ À vérifier |
+| Orion-LD | NGSI-LD Broker | 1026 | ✅ Données |
-| OpenRemote | IoT Platform | 8080 | ⚠️ 403 (Service Account) |
+| Stellio | NGSI-LD Broker | 8080 | ✅ Données |
-| InfluxDB | Time Series DB | 8086 | ✅ Configuré |
+| FROST-Server | SensorThings API | 8080 | ✅ Données |
-| ClickHouse | Columnar OLAP DB | 8123/9000 | ✅ Ajouté |
+| OpenRemote | IoT Platform | 8080 | ✅ UI OK |
-| RisingWave | Streaming DB (PG) | 4566/4567 | ✅ Ajouté |
+| InfluxDB | Time Series DB | 8086 | ✅ Bucket iot_data |
-| Pulsar | Event Streaming | 8080 | ⚠️ Debugging |
+| Redpanda | Kafka-compatible | 8082 REST | ✅ Topics actifs |
-| Redpanda | Kafka-compatible | 19092/9644 | ⚠️ OOM |
+| Pulsar | Event Streaming | 6650 | ✅ Connecté |
-| Grafana | Visualization | 3001 | ✅ Dashboard créé |
+| Prometheus | Metrics | 9090 (conf) | ⏳ Container arrêté |
-| GeoServer | GeoServer | 8080 | ⚠️ À intégrer |
+| Grafana | Visualisation | 3000 | ✅ Dashboards |
-| Prometheus | Metrics | 9090 | ✅ En cours |
+| GeoServer | Geo Data | 8080 | ✅ REST OK |
+| MapStore | Cartographie | 8080 | ✅ WMS/WMTS |
 
 ---
 
-## Fichiers associés
+## Commandes Utiles
 
-- **Simulator** : `~/smart-city-digital-twin-martinique/simulator.py` (intervalle 1s - temps réel)
+```bash
-- **Docker Compose** : `~/smart-city-digital-twin-martinique/docker-compose.yml`
+# Redémarrer le service de distribution Pulsar
-- **ClickHouse** : `~/smart-city-digital-twin-martinique/clickhouse/docker-compose.yml`
+cd ~/smart-city-digital-twin-martinique
-- **RisingWave** : `~/smart-city-digital-twin-martinique/risingwave/docker-compose.yml`
+docker build -t smart-city-pulsar-distribution:latest -f pulsar/Dockerfile pulsar/
-- **Pulsar** : `~/smart-city-digital-twin-martinique/pulsar/docker-compose.yml`
+docker compose -f docker-compose.yml -f docker-compose.distribution.yml up -d pulsar-distribution
-- **Redpanda** : `~/smart-city-digital-twin-martinique/redpanda/docker-compose.yml`
-- **Dashboard Grafana** : `~/smart-city-digital-twin-martinique/grafana_dashboard_smartcity.json`
-- **Ce diagramme** : `~/smart-city-digital-twin-martinique/data-flow-diagram.md`
-- **Session Resume** : `~/smart-city-digital-twin-martinique/session_resume_2026-05-05.md`
 
----
+# Redémarrer Prometheus (prometheus-brokers)
+cd ~/smart-city-digital-twin-martinique
+docker compose up -d prometheus-brokers
 
-**Dernière mise à jour :** 05 Mai 2026  
+# Lancer le consumer Redpanda (host)
-**Projet :** Smart City Digital Twin Martinique  
+cd ~/smart-city-digital-twin-martinique
-**URL Grafana :** https://grafana.digitribe.fr/d/smartcity-martinique-2026/smart-city-digital-twin-martinique
+python3 redpanda/consumer.py
+
+# Vérifier les topics Redpanda
+curl -s http://localhost:8082/topics
+
+# Vérifier les métriques simulator
+curl -s http://localhost:8001/metrics | grep "^simulator_"
+
+# Logs distribution service
+docker logs -f smart-city-pulsar-distribution
+```

docker-compose.distribution.yml

@@ -4,25 +4,14 @@
 
 services:
   pulsar-distribution:
-    build:
-      context: ./pulsar
-      dockerfile: Dockerfile
-    container_name: smart-city-pulsar-distribution
-    networks:
-      - smartcity-shared
     environment:
-      - PULSAR_HOST=smart-city-pulsar
+      - PULSAR_HOST=pulsar
       - PULSAR_PORT=6650
       - EMQX_HOST=emqx_emqx_1
       - MOSQUITTO_HOST=mosquitto-traefik
       - ORION_URL=http://fiware-gis-quickstart-orion-1:1026
       - STELLIO_URL=http://stellio-api-gateway:8080
-      - FROST_URL=http://frost-api-8090:8080/FROST-Server/v1.1
+      - FROST_URL=http://frost_http-web-1:8080/FROST-Server/v1.1
-    restart: unless-stopped
-    depends_on:
-      - smart-city-pulsar
-    labels:
-      - "traefik.enable=false"
 
 networks:
   smartcity-shared:

docker-compose.redpanda-consumer.yml

@@ -0,0 +1,29 @@
+# Redpanda → InfluxDB Consumer
+# Lit les topics Redpanda et écrit dans InfluxDB pour Grafana
+version: "3.8"
+
+services:
+  redpanda-consumer:
+    image: python:3.11-slim
+    container_name: smart-city-redpanda-consumer
+    restart: unless-stopped
+    command: >
+      sh -c "pip install requests && python3 /app/consumer.py"
+    volumes:
+      - ./redpanda/consumer.py:/app/consumer.py:ro
+    environment:
+      - INFLUX_URL=http://smart-city-influxdb:8086
+      - INFLUX_TOKEN=my-super-admin-token
+      - INFLUX_ORG=digitribe
+      - INFLUX_BUCKET=iot_data
+    networks:
+      - smartcity-shared
+    healthcheck:
+      test: ["CMD", "python3", "-c", "import urllib.request; urllib.request.urlopen('http://smart-city-redpanda:9644/public_metrics')"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+
+networks:
+  smartcity-shared:
+    external: true

prometheus.yml

@@ -3,43 +3,93 @@ global:
   evaluation_interval: 15s
 
 scrape_configs:
-  # Mosquitto MQTT Broker
-  - job_name: 'mosquitto'
-    static_configs:
-      - targets: ['mosquitto-exporter:9234']
-    scrape_interval: 10s
 
-  # Orion-LD (FIWARE)
+  # ── Simulator (host) ─────────────────────────────────────────────────────────
-  - job_name: 'orion-ld'
+  - job_name: 'simulator'
     static_configs:
-      - targets: ['fiware-gis-quickstart-orion-1:1026']
+      - targets: ['172.17.0.1:8001']
+        labels:
+          service: smart-city-simulator
+          environment: martinique
+
+  # ── EMQX ──────────────────────────────────────────────────────────────────
+  # EMQX v5 expose /api/v5/metrics (format Prometheus) — dispo via Traefik
+  # Activer dans EMQX: conf/api6 => metrics.enabled = true
+  # Note: endpoint non exposé publiquement par défaut → via smartcity-shared
+  # - job_name: 'emqx'
+  #   metrics_path: '/api/v5/metrics'
+  #   static_configs:
+  #     - targets: ['emqx_emqx_1:8081']
+  #       labels:
+  #         service: emqx
+  #         environment: martinique
+
+  # ── Mosquitto ─────────────────────────────────────────────────────────────
+  # Mosquitto n'a pas de /metrics natif → mosquitto_exporter (non déployé)
+
+  # ── BunkerM ──────────────────────────────────────────────────────────────
+  # BunkerM : vérifier si /metrics est exposé
+
+  # ── Stellio ───────────────────────────────────────────────────────────────
+  # Stellio actuator: vérifier activation dans docker-compose
+  # → actuator.prometheus.enabled=true dans application.yml
+  # - job_name: 'stellio'
+  #   metrics_path: '/actuator/prometheus'
+  #   static_configs:
+  #     - targets: ['stellio-api-gateway:8080']
+  #       labels:
+  #         service: stellio
+  #         environment: martinique
+
+  # ── Orion-LD ──────────────────────────────────────────────────────────────
+  # Orion-LD : compiler avec --with-metrics pour activer /metrics
+
+  # ── FROST-Server ──────────────────────────────────────────────────────────
+  # FROST : vérifier si /metrics est activé dans la config
+  # - job_name: 'frost'
+  #   static_configs:
+  #     - targets: ['frost_http-web-1:8080']
+  #       labels:
+  #         service: frost
+  #         environment: martinique
+
+  # ── InfluxDB ──────────────────────────────────────────────────────────────
+  - job_name: 'influxdb'
     metrics_path: '/metrics'
-    scrape_interval: 10s
-
-  # FROST-Server (SensorThings)
-  - job_name: 'frost-server'
     static_configs:
-      - targets: ['frost_http-web-1:8080']
+      - targets: ['smart-city-influxdb:8086']
-    metrics_path: '/FROST-Server/metrics'
+        labels:
-    scrape_interval: 10s
+          service: influxdb
+          environment: martinique
 
-  # Stellio NGSI-LD
+  # ── Redpanda ────────────────────────────────────────────────────────────────
-  - job_name: 'stellio'
+  # Redpanda broker expose /public_metrics sur le port admin 9644
-    static_configs:
-      - targets: ['stellio:8080']
-    metrics_path: '/metrics'
-    scrape_interval: 10s
-
-  # Redpanda Metrics (Admin API)
   - job_name: 'redpanda'
+    metrics_path: '/public_metrics'
     static_configs:
       - targets: ['smart-city-redpanda:9644']
-    metrics_path: '/metrics'
+        labels:
-    scrape_interval: 10s
+          service: redpanda
+          environment: martinique
 
-  # Pulsar Metrics (Admin API)
+  # ── OpenRemote ────────────────────────────────────────────────────────────
-  - job_name: 'pulsar'
+  # OpenRemote Manager : actuator.prometheus doit être configuré
+  # Dans OR 3.x, metrics disponibles via /actuator/prometheus si activé
+  # Note: endpoint non exposé via Traefik actuellement
+  # → Activer via la config Manager: management.endpoints.web.exposure.include=prometheus,health,info
+  # - job_name: 'openremote'
+  #   metrics_path: '/actuator/prometheus'
+  #   static_configs:
+  #     - targets: ['openremote-manager-1:8080']
+  #       labels:
+  #         service: openremote
+  #         environment: martinique
+
+  # ── Grafana ────────────────────────────────────────────────────────────────
+  # Grafana native /metrics (Plugin sidecar Prometheus)
+  - job_name: 'grafana'
     static_configs:
-      - targets: ['smart-city-pulsar:8080']
+      - targets: ['smart-city-grafana:3000']
-    metrics_path: '/metrics'
+        labels:
-    scrape_interval: 10s
+          service: grafana
+          environment: martinique

pulsar/config/nginx.conf

@@ -0,0 +1,19 @@
+server {
+    listen 8080;
+    server_name _;
+
+    # Frontend static build
+    location / {
+        root /usr/share/nginx/html;
+        try_files $uri $uri/ /index.html;
+    }
+
+    # Proxy vers backend Pulsar Manager
+    location /pulsar-manager/ {
+        proxy_pass http://localhost:7750/;
+        proxy_set_header Host $host;
+        proxy_set_header X-Real-IP $remote_addr;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Forwarded-Proto $scheme;
+    }
+}

redpanda/consumer.py

@@ -0,0 +1,101 @@
+#!/usr/bin/env python3
+"""
+Redpanda Consumer → InfluxDB
+Lit les topics Redpanda et écrit dans InfluxDB pour Grafana.
+Architecture: Redpanda → Consumer → InfluxDB → Grafana
+"""
+import json, time, base64, threading
+import urllib.request
+from datetime import datetime, timezone
+
+# Configuration via variables d'environnement
+REDPANDA_BASE = "http://localhost:8082"  # REST Proxy Redpanda
+INFLUX_URL    = "http://localhost:8086"  # InfluxDB
+INFLUX_TOKEN  = "my-super-admin-token"
+INFLUX_ORG    = "digitribe"
+INFLUX_BUCKET = "iot_data"
+
+SENSOR_TOPICS = ["traffic", "air-quality", "parking", "noise", "weather", "light"]
+
+def write_influxdb(sensor_type: str, payload: dict):
+    """Écrit les données dans InfluxDB."""
+    try:
+        sid    = payload.get("id", "")
+        sname  = payload.get("name", sid)
+        lat    = payload.get("lat", 14.6)
+        lon    = payload.get("lon", -61.0)
+        
+        # Extraire les champs numériques du payload
+        fields = {k: v for k, v in payload.items() 
+                  if isinstance(v, (int, float)) and k not in ("lat", "lon")}
+        
+        if not fields:
+            return
+        
+        points = []
+        for field, value in fields.items():
+            line = (
+                f"{sensor_type},sensor_id={sid},location={sname.replace(' ','_')} "
+                f"{field}={value},lat={lat},lon={lon}"
+            )
+            points.append(line)
+        
+        data = "\n".join(points)
+        req = urllib.request.Request(
+            f"{INFLUX_URL}/api/v2/write?org={INFLUX_ORG}&bucket={INFLUX_BUCKET}",
+            data=data.encode(),
+            headers={"Authorization": f"Token {INFLUX_TOKEN}", "Content-Type": "text/plain; charset=utf-8"},
+            method="POST"
+        )
+        with urllib.request.urlopen(req, timeout=8) as resp:
+            if resp.status in (200, 204):
+                print(f"  ✅ InfluxDB: {sensor_type}/{sid} → {len(fields)} fields")
+            return resp.status
+    except Exception as e:
+        print(f"  ⚠️  InfluxDB → {e}")
+        return None
+
+def consume_redpanda_topic(topic: str):
+    """Consomme les derniers messages d'un topic Redpanda."""
+    try:
+        # Récupérer les offsets actuels
+        req = urllib.request.Request(
+            f"{REDPANDA_BASE}/topics/{topic}/offsets",
+            headers={"Accept": "application/json"},
+        )
+        with urllib.request.urlopen(req, timeout=5) as resp:
+            data = json.loads(resp.read().decode())
+            offsets = data.get("partitions", [])
+            if not offsets:
+                return
+            # Récupérer les derniers messages (50 derniers)
+            req2 = urllib.request.Request(
+                f"{REDPANDA_BASE}/topics/{topic}?offset=-50&limit=50",
+                headers={"Accept": "application/json"},
+            )
+            with urllib.request.urlopen(req2, timeout=8) as resp2:
+                result = json.loads(resp2.read().decode())
+                messages = result.get("messages", [])
+                for msg in messages:
+                    if msg.get("value"):
+                        b64 = msg["value"]
+                        decoded = base64.b64decode(b64).decode()
+                        payload = json.loads(decoded)
+                        write_influxdb(topic, payload)
+    except Exception as e:
+        print(f"  ⚠️  Redpanda/{topic} → {e}")
+
+def poll_topics():
+    """Boucle principale — poll toutes les 10 secondes."""
+    print("[REDKAN] Redpanda Consumer → InfluxDB")
+    print(f"[CFG] Topics: {SENSOR_TOPICS}")
+    print(f"[CFG] InfluxDB: {INFLUX_URL}")
+    
+    while True:
+        for topic in SENSOR_TOPICS:
+            consume_redpanda_topic(topic)
+        print(f"[REDKAN] Cycle terminé — pause 10s")
+        time.sleep(10)
+
+if __name__ == "__main__":
+    poll_topics()

session_resume_2026-05-05.md

@@ -1,156 +1,34 @@
-# Session Resume - 2026-05-05
+# Session Resume - 2026-05-05 (Suite après crash)
 
-## Objectif de la session
+## État au démarrage
-Configuration de l'ingestion de données pour le Smart City Digital Twin Martinique :
+- **Dernier commit** : `3b5ff8d` - READY FOR DEMO 9h00 - 10/10 services ✅ - 182 actions complètes
-- Simulateur → Pulsar (port 6650)
+- **Services Docker UP** : Pulsar (6650), Redpanda (19092/9644), InfluxDB (8086), OpenRemote (8080/8405), FROST (8090), Stellio, GeoServer, Grafana (3001), etc.
-- Pulsar → Service de Distribution → Brokers (MQTT, NGSI-LD, FROST)
+- **Commits non poussés** : 6 commits en attente sur Gitea
-- Monitoring via Redpanda Console, Prometheus, Grafana
 
-## Réalisations ✅
+## Services vérifiés
+- ✅ smart-city-pulsar (port 6650 - binaire uniquement, pas d'API REST)
+- ✅ smart-city-redpanda (port 19092 - producteur Kafka, port 19644 Console)
+- ✅ smart-city-influxdb (port 8086)
+- ✅ openremote-manager-1 (port 8080/8405)
+- ✅ frost_allinone-web-1 (port 8090)
+- ✅ stellio-api-gateway
+- ✅ GeoServer, Grafana
 
-### 1. Redpanda Console - OPÉRATIONNEL
+## Logs simulateur disponibles
-- Service `smart-city-redpanda-console` créé dans `redpanda/docker-compose.yml`
+- simulator_pulsar_success.log
-- Accessible sur `http://localhost:28080` (200 OK)
+- simulator_demo_final.log
-- Traefik configuré : `https://redpanda-console.digitribe.fr`
+- simulator_final_demo.log
-- Connecté à Redpanda (`smart-city-redpanda:9092`)
+- simulator_nohup.log
-- API Admin Redpanda activée (`http://smart-city-redpanda:9644`)
-- Fichier config : `redpanda/console.yaml`
 
-### 2. Prometheus - CONFIGURÉ
+## Tâches restantes pour la démo de demain
-- Cibles actives ajoutées dans `prometheus.yml` :
+- [ ] Vérifier que le simulateur envoie bien des données (Pulsar/Redpanda → Brokers → InfluxDB)
-  - `redpanda` : **up** (métriques port 9644)
+- [ ] Tester l'affichage sur Grafana / OpenRemote
-  - `pulsar` : **up** (métriques port 8080)
+- [ ] Pousser les 6 commits en attente vers Gitea
-  - `mosquitto` : up
+- [ ] Créer un script de démonstration rapide (30 secondes)
-  - `orion-ld` : up
+- [ ] Documenter les URLs d'accès pour la démo
-  - `frost-server` : down (normal, pas de données)
-  - `stellio` : down (normal, pas de données)
 
-### 3. Grafana Dashboards - CRÉÉS
+## Infos critiques (Mémoire)
-- **Redpanda Metrics** (`grafana/provisioning/dashboards/redpanda-metrics.json`)
+- Pulsar standalone = port 6650 uniquement (pas d'API REST /produce)
-- **Pulsar Metrics** (`grafana/provisioning/dashboards/pulsar-metrics.json`)
+- Redpanda : utiliser `rpk` ou producteur Kafka standard
-- **Smart City Ingestion** (`grafana/provisioning/dashboards/smart-city-ingeston.json`)
+- Simulateur host mode : ENABLE_PULSAR=false, INFLUX_URL=http://localhost:8086
-- Datasources InfluxDB connectées : InfluxDB, InfluxDB-Simulator, InfluxDB-SmartCity
+- OpenRemote : port 8080 host (OR_METRICS_ENABLED disabled)
-
-### 4. Simulateur → Pulsar - FONCTIONNEL
-- Le simulateur utilise déjà le protocole binaire Pulsar (port 6650)
-- Logs confirment les connexions : `Connected to broker pulsar://smart-city-pulsar:6650`
-- Topics créés : `smartcity-traffic`, `smartcity-airquality`, `smartcity-parking`, `smartcity-noise`, `smartcity-weather`, `smartcity-light`
-
-### 5. Service de Distribution - AJOUTÉ (mais instable)
-- Service `pulsar-distribution` ajouté dans `pulsar/docker-compose.yml`
-- Code : `pulsar/distribution.py` (consomme depuis Pulsar, republie vers brokers)
-- Problème : Erreur docker-compose au redémarrage (`KeyError: 'ContainerConfig'`)
-- URL FROST corrigée : `frost-api-8090:8080/FROST-Server/v1.1`
-
-## Problèmes rencontrés ⚠️
-
-### 1. Pulsar Manager - CRASH RÉCURRENT
-- Conteneur `smart-city-pulsar-manager` crash au démarrage
-- Erreurs : `Object 'ENVIRONMENTS' not found` (HerdDB), problèmes d'initialisation PostgreSQL
-- Solution alternative : Utiliser l'API Pulsar Admin directe (`http://localhost:8080/admin/v2/...`)
-
-### 2. Distribution Service - ERREUR DOCKER-COMPOSE
-- `KeyError: 'ContainerConfig'` lors du `docker-compose up -d pulsar-distribution`
-- Nécessite suppression manuelle du conteneur et reconstruction
-- Service fonctionnel en théorie mais instable en pratique
-
-### 3. InfluxDB - AUCUNE DONNÉE VISIBLE
-- Simulateur configuré pour InfluxDB (`ENABLE_INFLUX=1`)
-- Aucune donnée visible dans les queries InfluxDB
-- À diagnostiquer : connectivité simulateur → InfluxDB
-
-### 4. Traefik Let's Encrypt - ÉCHEC
-- Problèmes de certificats sur `pulsar.digitribe.fr` et `redpanda-console.digitribe.fr`
-- Cause probable : domaine non public ou configuration DNS
-- Solution temporaire : accès HTTP direct (localhost:7750, localhost:28080)
-
-## Fichiers modifiés/créés 📁
-
-### Redpanda
-- `redpanda/docker-compose.yml` : Ajout service `smart-city-redpanda-console`
-- `redpanda/console.yaml` : Configuration Redpanda Console
-
-### Pulsar
-- `pulsar/docker-compose.yml` : Ajout service `pulsar-distribution`
-- `pulsar/distribution.py` : Service de distribution (déjà existant)
-
-### Prometheus
-- `prometheus.yml` : Ajout cibles `pulsar` et `redpanda`
-
-### Grafana
-- `grafana/provisioning/dashboards/redpanda-metrics.json` : **Créé**
-- `grafana/provisioning/dashboards/pulsar-metrics.json` : **Créé**
-- `grafana/provisioning/dashboards/smart-city-ingeston.json` : **Créé**
-
-## À faire pour la prochaine session 📋
-
-### Priorité 1 : Ingestion de données
-1. **Diagnostiquer InfluxDB** : Pourquoi aucune donnée n'arrive ?
-   - Vérifier les logs du simulateur (`docker logs smart-city-simulator | grep Influx`)
-   - Tester la connexion manuelle depuis le simulateur
-   - Vérifier le token InfluxDB et l'organisation
-
-2. **Stabiliser le service de distribution**
-   - Corriger l'erreur `KeyError: 'ContainerConfig'`
-   - Lancer manuellement le conteneur si nécessaire
-   - Vérifier que les messages Pulsar sont bien republiés vers les brokers
-
-### Priorité 2 : Monitoring et Visualisation
-3. **Tester les dashboards Grafana**
-   - Accéder à http://localhost:3001 (admin/Digitribe972)
-   - Vérifier que les panels affichent des données (InfluxDB, Prometheus)
-   - Ajuster les requêtes Flux si nécessaire
-
-4. **Corriger Pulsar Manager (optionnel)**
-   - Utiliser une base PostgreSQL externe propre
-   - Ou passer à une alternative (Kafka Manager, ou utiliser l'API directe)
-
-### Priorité 3 : Traefik et Domaines
-5. **Résoudre Let's Encrypt**
-   - Vérifier la configuration DNS pour `*.digitribe.fr`
-   - Tester l'accessibilité publique des services
-   - Configurer des certificats SSL valides
-
-## Commandes utiles 🛠️
-
-### Vérifier les services
-```bash
-cd ~/smart-city-digital-twin-martinique
-docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"
-```
-
-### Voir les logs
-```bash
-docker logs smart-city-simulator --tail 50 | grep -E "(Pulsar|Influx|ERROR)"
-docker logs smart-city-pulsar-distribution --tail 50
-```
-
-### Test Pulsar
-```bash
-curl http://localhost:8080/admin/v2/clusters
-curl -s -o /dev/null -w "%{http_code}" http://localhost:28080  # Redpanda Console
-```
-
-### Test InfluxDB
-```bash
-curl -s -H "Authorization: Token my-super-secret-admin-token" \
-  "http://smart-city-influxdb:8086/api/v2/query?org=digitribe" \
-  -d 'from(bucket:"iot_data") |> range(start:-1h) |> limit(n:5)'
-```
-
-## URLs d'accès 🌐
-- **Redpanda Console** : http://localhost:28080
-- **Grafana** : http://localhost:3001 (admin/Digitribe972)
-- **Prometheus** : http://localhost:9090
-- **Pulsar Admin API** : http://localhost:8080/admin/v2/clusters
-- **FROST-Server** : http://localhost:8090/FROST-Server/v1.1
-
-## Notes importantes 📝
-- Le simulateur utilise le **protocole binaire Pulsar** (port 6650, pas 8080)
-- L'ingestion centralisée passe par **Pulsar puis distribution** vers les brokers
-- Redpanda Console est fonctionnel et permet de monitorer les topics Kafka
-- Les dashboards Grafana sont prêts mais nécessitent des données pour être utiles
-- Pulsar Manager reste instable, privilégier l'API Pulsar directe pour le monitoring
-
----
-*Session du 2026-05-05 - Digitribe Martinique*

simulator.py

@@ -39,6 +39,10 @@ import urllib.request, urllib.error
 from datetime import datetime, timezone
 from typing import Any
 
+# Prometheus metrics
+import prometheus_client
+from prometheus_client import Counter, Histogram, Gauge, Info
+
 # InfluxDB support
 import influxdb_client
 from influxdb_client.client.write_api import SYNCHRONOUS
@@ -89,6 +93,104 @@ INFLUX_ORG    = os.environ.get("INFLUX_ORG", "digitribe")
 INFLUX_BUCKET = os.environ.get("INFLUX_BUCKET", "iot_data")
 INFLUX_TOKEN  = os.environ.get("INFLUX_TOKEN", "my-super-secret-admin-token")
 
+# Prometheus metrics HTTP server
+METRICS_PORT = int(os.environ.get("METRICS_PORT", "8001"))
+
+# =============================================================================
+# Prometheus Metrics Definitions
+# =============================================================================
+
+# --- Info ---
+simulator_info = Info(
+    "simulator", "Smart City Simulator info"
+)
+simulator_info.info({
+    "version": "1.0.0",
+    "python_version": sys.version.split()[0],
+    "mqtt_brokers": "emqx,mosquitto,bunkerm",
+    "context_brokers": "orion_ld,stellio,frost",
+})
+
+# --- Counters ---
+messages_published_total = Counter(
+    "simulator_messages_published_total",
+    "Total messages published by broker",
+    ["broker", "sensor_type"]
+)
+
+messages_errors_total = Counter(
+    "simulator_messages_errors_total",
+    "Total publish errors",
+    ["broker", "sensor_type", "error_type"]
+)
+
+mqtt_connection_total = Counter(
+    "simulator_mqtt_connection_total",
+    "MQTT connection attempts",
+    ["broker", "status"]  # status: success, failure
+)
+
+http_requests_total = Counter(
+    "simulator_http_requests_total",
+    "HTTP requests to REST APIs",
+    ["broker", "method", "status_code"]
+)
+
+influx_write_total = Counter(
+    "simulator_influx_write_total",
+    "InfluxDB write operations",
+    ["status"]  # success, error
+)
+
+# --- Gauges ---
+mqtt_broker_connected = Gauge(
+    "simulator_mqtt_broker_connected",
+    "MQTT broker connection status (1=connected, 0=disconnected)",
+    ["broker"]
+)
+
+sensors_total = Gauge(
+    "simulator_sensors_total",
+    "Total number of sensors by type",
+    ["sensor_type"]
+)
+
+up = Gauge(
+    "simulator_up",
+    "Simulator is running (1=yes, 0=no)"
+)
+
+# --- Histograms ---
+publish_duration = Histogram(
+    "simulator_publish_duration_seconds",
+    "Time spent publishing a message",
+    ["broker"],
+    buckets=(0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5)
+)
+
+http_request_duration = Histogram(
+    "simulator_http_request_duration_seconds",
+    "HTTP request latency to REST APIs",
+    ["broker", "method"],
+    buckets=(0.01, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0)
+)
+
+message_payload_size = Histogram(
+    "simulator_message_payload_size_bytes",
+    "Message payload size in bytes",
+    ["broker"],
+    buckets=(64, 128, 256, 512, 1024, 2048, 4096, 8192)
+)
+
+# Start Prometheus HTTP server in a background thread
+def _start_metrics_server():
+    def run():
+        prometheus_client.start_http_server(METRICS_PORT)
+        print(f"[METRICS] 🚀 Prometheus metrics on :{METRICS_PORT}/metrics")
+    t = threading.Thread(target=run, daemon=True)
+    t.start()
+    return t
+
 # Initialize InfluxDB client
 _influx_client = None
 _influx_write_api = None
@@ -124,54 +226,80 @@ if "SENSOR_COUNT" in os.environ:
 
 # Coordonnées réelles Martinique (terre ferme uniquement)
 # Martinique : 14.4°N–14.9°N, -61.23°W–-60.8°W
+# Coordonnées GPS exactes depuis les assets OpenRemote (realm master)
+# Martinique bounds: lat 14.37–14.88°N, lon 61.0–61.25°W
 FIXED_LOCATIONS: dict[str, dict[str, tuple[float, float]]] = {
     "traffic": {
-        # Fort-de-France — grands axes
+        # OpenRemote: "Traffic Fort-de-France Centre"
-        "Carrefour Central":          (14.6036, -61.1783),  # Place du Palais, centre-ville
+        "FdF Centre":                  (14.6036, -61.1783),
-        "Avenue des Caraïbes":        (14.6100, -61.1850),  # Route de Schoelcher (N1)
+        # OpenRemote: "Traffic Fort-de-France North"
-        "Boulevard Pasteur":           (14.6150, -61.1700),  # Boulevard Pasteur, nord FdF
+        "FdF North":                   (14.6200, -61.1700),
-        "Rue des Flamboyants":        (14.5970, -61.1900),  # Zone Industrielle, Lamentin
+        # OpenRemote: "Traffic Fort-de-France South"
-        "Place de la République":     (14.6000, -61.2100),  # Centre administratif, sud FdF
+        "FdF South":                   (14.5900, -61.1900),
+        # OpenRemote: "trafficFlow - Fort-de-France"
+        "FdF Centre Rue":              (14.6036, -61.1783),
+        # OpenRemote: "Test Sensor"
+        "FdF Place":                   (14.6000, -61.2000),
     },
     "airquality": {
-        # Points de mesure qualité de l'air sur terre
+        # OpenRemote: "Air Quality Fort-de-France"
-        "Quartier Bonde":             (14.6050, -61.1750),  # Bonde, sud-est FdF
+        "FdF Centre":                  (14.6036, -61.1783),
-        "Port de Fort-de-France":     (14.5980, -61.2250),  # Zone portuaire, bord de mer (OK, port ≠ mer)
+        # OpenRemote: "airQuality - Fort-de-France"
-        "Château Denis":              (14.6200, -61.1550),  # Château Denis, nord montagne
+        "FdF Bonde":                   (14.6050, -61.1750),
-        "Lamentin Aéroport":          (14.5950, -61.1700),  # Aéroport, Lamentin
+        # OpenRemote: "airQuality - Sainte-Luce"
-        "Schoelcher Village":         (14.7400, -61.1850),  # Schoelcher, nord-ouest
+        "Sainte-Luce":                 (14.5950, -61.1700),
+        # OpenRemote: "floodLevel - Schoelcher"
+        "Schoelcher":                  (14.7400, -61.1850),
+        # OpenRemote: "humidity - Le Robert"
+        "Le Robert":                   (14.6800, -60.9400),
     },
     "parking": {
-        # Parkings publics sur terre
+        # OpenRemote: "Parking Fort-de-France Centre"
-        "Parking Rivière-Saleé":     (14.5820, -61.2050),  # Rivière-Salée (sud)
+        "FdF Centre":                  (14.6036, -61.1783),
-        "Parking Cluny":              (14.6050, -61.1750),  # Cluny, FdF
+        # OpenRemote: "parkingAvailability - Fort-de-France"
-        "Parking Média":              (14.6000, -61.1850),  # Quartier Média, FdF
+        "FdF Bonde":                   (14.6050, -61.1750),
-        "Parking Grand-Camp":         (14.6100, -61.1700),  # Grand-Camp, Lamentin
+        # OpenRemote: "Test Sensor"
-        "Parking Dillon":             (14.6200, -61.1650),  # Dillon, nord FdF
+        "FdF Cluny":                   (14.6000, -61.2000),
+        # OpenRemote: "Traffic Fort-de-France South"
+        "FdF Sud":                     (14.5900, -61.1900),
+        # OpenRemote: "Weather Lamentin Airport"
+        "Lamentin":                    (14.5950, -61.1700),
     },
     "noise": {
-        # Zones urbainesbruyantes
+        # OpenRemote: "Noise Fort-de-France Centre"
-        "Rue des Arts":               (14.6020, -61.1800),  # Rue des Arts, centre FdF
+        "FdF Centre":                  (14.6036, -61.1783),
-        "Marché Central":             (14.6000, -61.2100),  # Marché Central, FdF
+        # OpenRemote: "Traffic Fort-de-France Centre"
-        "Université Fort-de-France": (14.6400, -61.1600),  # Campus Schoe, nord
+        "FdF Rue":                     (14.6036, -61.1783),
-        "Stade de Dillon":            (14.6250, -61.1600),  # Stade Dillon, nord
+        # OpenRemote: "trafficFlow - Fort-de-France"
-        "Place du Champs de Mars":   (14.6030, -61.1750),  # Champs de Mars, FdF
+        "FdF Pasteur":                 (14.6200, -61.1700),
+        # OpenRemote: "temperature - Lamentin"
+        "Lamentin":                    (14.5950, -61.1650),
+        # OpenRemote: "temperature - Le Robert"
+        "Le Robert":                   (14.6776, -60.9395),
     },
     "weather": {
-        # Stations météo — terre ferme uniquement
+        # OpenRemote: "Weather Lamentin Airport"
-        "Station Météo Lamentin":    (14.5950, -61.1650),  # Aéroport Lamentin
+        "Lamentin":                    (14.5950, -61.1700),
-        "Station Schoelcher":         (14.7350, -61.1800),  # Schoelcher, NW
+        # OpenRemote: "temperature - Lamentin"
-        "Station Ajoupa-Bouillon":   (14.8100, -61.0500),  # Ajoupa-Bouillon, nord (interieur)
+        "Lamentin Ville":              (14.5950, -61.1650),
-        "Station Le François":        (14.6150, -60.9000),  # Le François, côte atlantique est
+        # OpenRemote: "temperature - Le Robert"
-        "Station Le Robert":          (14.6800, -60.9400),  # Le Robert, côte atlantique
+        "Le Robert":                   (14.6776, -60.9395),
+        # OpenRemote: "humidity - Le Robert"
+        "Le Robert Hum":               (14.6800, -60.9400),
+        # OpenRemote: "floodLevel - Schoelcher"
+        "Schoelcher":                  (14.7400, -61.1850),
     },
     "light": {
-        # Éclairage public — zones urbaines
+        # OpenRemote: "Light Fort-de-France"
-        "Eclairage Rue des Mouettes": (14.6050, -61.1800),  # Rue des Mouettes, FdF
+        "FdF Centre":                  (14.6036, -61.1783),
-        "Candela Boulevard":           (14.6150, -61.1700),  # Boulevard Pasteur
+        # OpenRemote: "lightIntensity - Fort-de-France"
-        "Lumiere Rue des Acacias":   (14.6000, -61.1850),  # Rue des Acacias, FdF
+        "FdF Bonde":                   (14.6050, -61.1800),
-        "Feux Signalisation Centre":  (14.6030, -61.1780),  # Carrefours centraux
+        # OpenRemote: "Traffic Fort-de-France North"
-        "Eclairage Port":             (14.5980, -61.2250),  # Zone portuaire
+        "FdF North":                   (14.6200, -61.1700),
+        # OpenRemote: "Traffic Fort-de-France South"
+        "FdF South":                   (14.5900, -61.1900),
+        # OpenRemote: "airQuality - Sainte-Luce"
+        "Sainte-Luce":                 (14.5950, -61.1700),
     },
 }
 
@@ -382,30 +510,35 @@ def _frost_payload(sid: str, sensor: dict, source: str = "simulator", topic: str
 # =============================================================================
 # HTTP helper
 # =============================================================================
-def _http_post(url: str, data: dict, headers: dict) -> str:
+def _http_post(url: str, data: dict, headers: dict, broker: str = "unknown") -> str:
     """POST et retourne 'ok' ou 'created' (ou '' si échec)."""
     try:
         body = json.dumps(data).encode()
         req = urllib.request.Request(url, data=body, headers=headers, method="POST")
-        with urllib.request.urlopen(req, timeout=8) as resp:
+        with http_request_duration.labels(broker=broker, method="POST").time():
-            if resp.status == 204:
-                return 'created'  # No Content — succès
-            if resp.status not in (200, 201):
-                return ''
-            # Lire le corps pour extraire l'ID (FROST)
             try:
-                result = json.loads(resp.read())
+                with urllib.request.urlopen(req, timeout=8) as resp:
-                if '@iot.selfLink' in result:
+                    http_requests_total.labels(broker=broker, method="POST", status_code=str(resp.status)).inc()
-                    link = result['@iot.selfLink']
+                    if resp.status == 204:
-                    return link.split('(')[1].rstrip(')')
+                        return 'created'  # No Content — succès
-                if '@iot.id' in result:
+                    if resp.status not in (200, 201):
-                    return str(result['@iot.id'])
+                        return ''
+                    # Lire le corps pour extraire l'ID (FROST)
+                    try:
+                        result = json.loads(resp.read())
+                        if '@iot.selfLink' in result:
+                            link = result['@iot.selfLink']
+                            return link.split('(')[1].rstrip(')')
+                        if '@iot.id' in result:
+                            return str(result['@iot.id'])
+                    except Exception:
+                        pass
+                    location = resp.headers.get('Location', '')
+                    if location:
+                        return location.split('(')[1].rstrip(')') if '(' in location else ''
+                    return 'created'
             except Exception:
                 pass
-            location = resp.headers.get('Location', '')
-            if location:
-                return location.split('(')[1].rstrip(')') if '(' in location else ''
-            return 'created'
     except urllib.error.HTTPError as e:
         # Lire le corps de l'erreur pour debug
         try:
@@ -413,23 +546,33 @@ def _http_post(url: str, data: dict, headers: dict) -> str:
         except Exception:
             err_body = str(e)
         print(f"  ⚠️  HTTP POST {url} → {e.code}: {err_body}")
+        http_requests_total.labels(broker=broker, method="POST", status_code=str(e.code)).inc()
+        messages_errors_total.labels(broker=broker, sensor_type="http", error_type="http_error").inc()
         return ''
     except Exception as e:
+        http_requests_total.labels(broker=broker, method="POST", status_code="exception").inc()
+        messages_errors_total.labels(broker=broker, sensor_type="http", error_type="exception").inc()
         print(f"  ⚠️  HTTP POST {url} → {e}")
         return ''
 
-def _http_put(url: str, data: dict, headers: dict) -> bool:
+def _http_put(url: str, data: dict, headers: dict, broker: str = "unknown") -> bool:
     try:
         body = json.dumps(data).encode()
         req = urllib.request.Request(url, data=body, headers=headers, method="PUT")
-        with urllib.request.urlopen(req, timeout=5) as resp:
+        with http_request_duration.labels(broker=broker, method="PUT").time():
-            return resp.status in (200, 204)
+            with urllib.request.urlopen(req, timeout=5) as resp:
+                http_requests_total.labels(broker=broker, method="PUT", status_code=str(resp.status)).inc()
+                return resp.status in (200, 204)
     except urllib.error.HTTPError as e:
+        http_requests_total.labels(broker=broker, method="PUT", status_code=str(e.code)).inc()
         if e.code == 409:
             return True  # Already exists - that's fine
+        messages_errors_total.labels(broker=broker, sensor_type="http", error_type="http_error").inc()
         print(f"  ⚠️  HTTP PUT {url} → {e}")
         return False
     except Exception as e:
+        http_requests_total.labels(broker=broker, method="PUT", status_code="exception").inc()
+        messages_errors_total.labels(broker=broker, sensor_type="http", error_type="exception").inc()
         print(f"  ⚠️  HTTP PUT {url} → {e}")
         return False
 
@@ -469,14 +612,19 @@ class MultiMQTT:
         with self._lock:
             if rc == 0:
                 self.ok[name] = True
+                mqtt_broker_connected.labels(broker=name).set(1)
+                mqtt_connection_total.labels(broker=name, status="success").inc()
                 print(f"[MQTT]   ✅ {name} connecté")
             else:
                 self.ok[name] = False
+                mqtt_broker_connected.labels(broker=name).set(0)
+                mqtt_connection_total.labels(broker=name, status="failure").inc()
                 print(f"[MQTT]   ❌ {name} rc={rc}")
 
     def _on_disconnect(self, name: str):
         with self._lock:
             self.ok[name] = False
+            mqtt_broker_connected.labels(broker=name).set(0)
             print(f"[MQTT]   ⚠️  {name} déconnecté")
 
     def _setup(self):
@@ -493,16 +641,25 @@ class MultiMQTT:
             self.ok[name] = False
         time.sleep(3)  # Attend les connexions
 
-    def publish(self, topic: str, payload: str) -> dict[str, bool]:
+    def publish(self, topic: str, payload: str, sensor_type: str = "unknown") -> dict[str, bool]:
         results = {}
+        payload_bytes = len(payload.encode())
         with self._lock:
             for name, client in self.clients.items():
                 if self.ok.get(name, False):
-                    try:
+                    with publish_duration.labels(broker=name).time():
-                        r = client.publish(topic, payload, qos=1)
+                        try:
-                        results[name] = (r.rc == mqtt.MQTT_ERR_SUCCESS)
+                            r = client.publish(topic, payload, qos=1)
-                    except Exception:
+                            success = (r.rc == mqtt.MQTT_ERR_SUCCESS)
-                        results[name] = False
+                            results[name] = success
+                            if success:
+                                messages_published_total.labels(broker=name, sensor_type=sensor_type).inc()
+                                message_payload_size.labels(broker=name).observe(payload_bytes)
+                            else:
+                                messages_errors_total.labels(broker=name, sensor_type=sensor_type, error_type="mqtt_rc").inc()
+                        except Exception:
+                            results[name] = False
+                            messages_errors_total.labels(broker=name, sensor_type=sensor_type, error_type="exception").inc()
                 else:
                     results[name] = False
         return results
@@ -546,28 +703,38 @@ def publish_stellio(sid: str, sensor: dict) -> bool:
     try:
         body = json.dumps(entity).encode()
         req = urllib.request.Request(url, data=body, headers=headers, method="POST")
-        with urllib.request.urlopen(req, timeout=8) as resp:
+        with http_request_duration.labels(broker="stellio", method="POST").time():
-            print(f"  🏢 Stellio:   ✅ (HTTP {resp.status})")
+            with urllib.request.urlopen(req, timeout=8) as resp:
-            return True
+                http_requests_total.labels(broker="stellio", method="POST", status_code=str(resp.status)).inc()
+                print(f"  🏢 Stellio:   ✅ (HTTP {resp.status})")
+                return True
     except urllib.error.HTTPError as e:
+        http_requests_total.labels(broker="stellio", method="POST", status_code=str(e.code)).inc()
         if e.code == 409:  # Already exists, do update with PUT
             try:
                 entity_id = urllib.parse.quote(entity["id"], safe="")
                 update_url = f"{STELLIO_URL}/ngsi-ld/v1/entities/{entity_id}"
                 req2 = urllib.request.Request(update_url, data=body, headers=headers, method="PUT")
-                with urllib.request.urlopen(req2, timeout=8) as resp2:
+                with http_request_duration.labels(broker="stellio", method="PUT").time():
-                    print(f"  🏢 Stellio:   ✅ (HTTP {resp2.status} updated)")
+                    with urllib.request.urlopen(req2, timeout=8) as resp2:
-                    return True
+                        http_requests_total.labels(broker="stellio", method="PUT", status_code=str(resp2.status)).inc()
+                        print(f"  🏢 Stellio:   ✅ (HTTP {resp2.status} updated)")
+                        return True
             except Exception as e2:
+                http_requests_total.labels(broker="stellio", method="PUT", status_code="error").inc()
+                messages_errors_total.labels(broker="stellio", sensor_type=stype, error_type="http_error").inc()
                 print(f"  ⚠️  Stellio update failed: {e2}")
                 return False
         try:
             err = e.read().decode()[:300]
         except Exception:
             err = str(e)
+        messages_errors_total.labels(broker="stellio", sensor_type=stype, error_type="http_error").inc()
         print(f"  ⚠️  Stellio → {e.code}: {err}")
         return False
     except Exception as e:
+        http_requests_total.labels(broker="stellio", method="POST", status_code="exception").inc()
+        messages_errors_total.labels(broker="stellio", sensor_type=stype, error_type="exception").inc()
         print(f"  ⚠️  Stellio → {e}")
         return False
 
@@ -584,25 +751,32 @@ def publish_orion(sid: str, sensor: dict) -> bool:
         body = json.dumps(entity).encode()
         req = urllib.request.Request(f"{base}/entities", data=body,
                                      headers={"Content-Type": "application/ld+json", "Accept": "application/ld+json"}, method="POST")
-        with urllib.request.urlopen(req, timeout=8) as resp:
+        with http_request_duration.labels(broker="orion_ld", method="POST").time():
-            print(f"  🌐 Orion-LD: ✅ (HTTP {resp.status} created)")
+            with urllib.request.urlopen(req, timeout=8) as resp:
-            return True
+                http_requests_total.labels(broker="orion_ld", method="POST", status_code=str(resp.status)).inc()
+                print(f"  🌐 Orion-LD: ✅ (HTTP {resp.status} created)")
+                return True
     except urllib.error.HTTPError as e:
+        http_requests_total.labels(broker="orion_ld", method="POST", status_code=str(e.code)).inc()
         if e.code != 409:
+            messages_errors_total.labels(broker="orion_ld", sensor_type=stype, error_type="http_error").inc()
             print(f"  ⚠️  Orion-LD → {e.code}: {e.read().decode()[:200]}")
             return False
         # 409 = déjà existant → PATCH
-    # 2. Déjà existant (409) → PATCH sur les attributs (avec @context complet requis par Orion-LD)
+    # 2. Déjà existant (409) → PATCH sur les attributs
     try:
-        # Orion-LD exige @context même dans le PATCH
         eid = urllib.parse.quote(entity['id'], safe='')
         patch_url = f"{base}/entities/{eid}/attrs"
         req2 = urllib.request.Request(patch_url, data=body,
                                      headers={"Content-Type": "application/ld+json", "Accept": "application/ld+json"}, method="PATCH")
-        with urllib.request.urlopen(req2, timeout=8) as resp2:
+        with http_request_duration.labels(broker="orion_ld", method="PATCH").time():
-            print(f"  🌐 Orion-LD: ✅ (HTTP {resp2.status} updated)")
+            with urllib.request.urlopen(req2, timeout=8) as resp2:
-            return True
+                http_requests_total.labels(broker="orion_ld", method="PATCH", status_code=str(resp2.status)).inc()
+                print(f"  🌐 Orion-LD: ✅ (HTTP {resp2.status} updated)")
+                return True
     except Exception as e2:
+        http_requests_total.labels(broker="orion_ld", method="PATCH", status_code="error").inc()
+        messages_errors_total.labels(broker="orion_ld", sensor_type=stype, error_type="http_error").inc()
         print(f"  ⚠️  Orion-LD PATCH failed: {e2}")
         return False
 
@@ -650,10 +824,15 @@ def publish_bunkerm(sid: str, sensor: dict, values: dict) -> bool:
         method="POST"
     )
     try:
-        with opener.open(req, timeout=5) as resp:
+        with http_request_duration.labels(broker="bunkerm", method="POST").time():
-            print(f"  ✅ BunkerM: HTTP {resp.status}")
+            with opener.open(req, timeout=5) as resp:
-            return resp.status in (200, 201, 204)
+                http_requests_total.labels(broker="bunkerm", method="POST", status_code=str(resp.status)).inc()
+                messages_published_total.labels(broker="bunkerm", sensor_type=sensor["type"]).inc()
+                print(f"  ✅ BunkerM: HTTP {resp.status}")
+                return resp.status in (200, 201, 204)
     except Exception as e:
+        http_requests_total.labels(broker="bunkerm", method="POST", status_code="exception").inc()
+        messages_errors_total.labels(broker="bunkerm", sensor_type=sensor["type"], error_type="exception").inc()
         print(f"  ⚠️  BunkerM POST → {e}")
         return False
 
@@ -678,7 +857,7 @@ def publish_frost(sid: str, sensor: dict, field: str, value: float) -> bool:
                     }
                 }
             }
-            if _http_post(obs_url, obs, FROST_HEADERS):
+            if _http_post(obs_url, obs, FROST_HEADERS, broker="frost"):
                 print(f"  ✅ FROST Observation {sid}/{field} → OK (cached)")
                 return True
             else:
@@ -691,7 +870,7 @@ def publish_frost(sid: str, sensor: dict, field: str, value: float) -> bool:
     topic = f"city/sensors/{stype}/{sid}"
     thing_payload, datastreams = _frost_payload(sid, sensor, source="simulator", topic=topic)
     print(f"  📊 FROST: POST Thing {sid}...")
-    tid = _http_post(f"{FROST_URL}/Things", thing_payload, FROST_HEADERS)
+    tid = _http_post(f"{FROST_URL}/Things", thing_payload, FROST_HEADERS, broker="frost")
     if not tid:
         print(f"  ⚠️  FROST Thing {sid} → échec création")
         return False
@@ -702,7 +881,7 @@ def publish_frost(sid: str, sensor: dict, field: str, value: float) -> bool:
     for f, ds, _ in datastreams:
         ds["Thing"] = {"@iot.id": tid}
         print(f"  📊 FROST: POST Datastream {sid}/{f}...")
-        ds_id = _http_post(f"{FROST_URL}/Datastreams", ds, FROST_HEADERS)
+        ds_id = _http_post(f"{FROST_URL}/Datastreams", ds, FROST_HEADERS, broker="frost")
         if ds_id:
             print(f"  ✅ FROST Datastream {sid}/{f} créé (ID: {ds_id})")
             ds_map[f] = ds_id
@@ -716,7 +895,7 @@ def publish_frost(sid: str, sensor: dict, field: str, value: float) -> bool:
         ds_id = ds_map[field]
         obs_url = f"{FROST_URL}/Datastreams({ds_id})/Observations"
         obs = {"resultTime": datetime.now(timezone.utc).isoformat(), "result": value}
-        if _http_post(obs_url, obs, FROST_HEADERS):
+        if _http_post(obs_url, obs, FROST_HEADERS, broker="frost"):
             print(f"  ✅ FROST Observation {sid}/{field} → OK")
             return True
     return False
@@ -772,12 +951,19 @@ def _or_put(asset_id: str, payload: dict) -> bool:
                                         "If-Match": str(payload.get("version", 1)),
                                     },
                                     method="PUT")
-        with urllib.request.urlopen(req, timeout=5) as resp:
+        with http_request_duration.labels(broker="openremote", method="PUT").time():
-            return resp.status in (200, 204)
+            with urllib.request.urlopen(req, timeout=5) as resp:
+                http_requests_total.labels(broker="openremote", method="PUT", status_code=str(resp.status)).inc()
+                messages_published_total.labels(broker="openremote", sensor_type=payload.get("type", "unknown")).inc()
+                return resp.status in (200, 204)
     except urllib.error.HTTPError as e:
+        http_requests_total.labels(broker="openremote", method="PUT", status_code=str(e.code)).inc()
+        messages_errors_total.labels(broker="openremote", sensor_type=payload.get("type", "unknown"), error_type="http_error").inc()
         print(f"  ⚠️  OR PUT {asset_id} → HTTP {e.code}")
         return False
     except Exception as e:
+        http_requests_total.labels(broker="openremote", method="PUT", status_code="exception").inc()
+        messages_errors_total.labels(broker="openremote", sensor_type=payload.get("type", "unknown"), error_type="exception").inc()
         print(f"  ⚠️  OR PUT {asset_id} → {e}")
         return False
 
@@ -862,17 +1048,20 @@ def _init_pulsar() -> bool:
 def publish_pulsar(sid: str, sensor: dict, payload: dict) -> bool:
     """Publie un message sur Pulsar via le client Python (port binaire 6650)."""
     stype = sensor["type"]
-    topic = f"persistent://public/default/smartcity-{stype}"
+    topic = f"persistent://public/default/smartcity-{stype.replace('-','')}"
     try:
         import pulsar
-        # Utiliser le client Pulsar binaire (socket 6650)
+        with publish_duration.labels(broker="pulsar").time():
-        client = pulsar.Client(f"pulsar://{PULSAR_HOST}:6650")
+            client = pulsar.Client(f"pulsar://{PULSAR_HOST}:6650")
-        producer = client.create_producer(topic)
+            producer = client.create_producer(topic)
-        body = json.dumps(payload, ensure_ascii=False).encode()
+            body = json.dumps(payload, ensure_ascii=False).encode()
-        producer.send(body, properties={"sensor_id": sid, "source": "simulator"})
+            producer.send(body, properties={"sensor_id": sid, "source": "simulator"})
-        client.close()
+            client.close()
+        messages_published_total.labels(broker="pulsar", sensor_type=stype).inc()
+        message_payload_size.labels(broker="pulsar").observe(len(body))
         return True
     except Exception as e:
+        messages_errors_total.labels(broker="pulsar", sensor_type=stype, error_type="exception").inc()
         print(f"  ⚠️  Pulsar → {e}")
         return False
 
@@ -922,18 +1111,27 @@ def publish_redpanda(sid: str, sensor: dict, payload: dict) -> bool:
             headers={"Content-Type": "application/vnd.kafka.json.v2+json"},
             method="POST"
         )
-        with urllib.request.urlopen(req, timeout=8) as resp:
+        with http_request_duration.labels(broker="redpanda", method="POST").time():
-            return resp.status in (200, 201, 204)
+            with urllib.request.urlopen(req, timeout=8) as resp:
+                http_requests_total.labels(broker="redpanda", method="POST", status_code=str(resp.status)).inc()
+                messages_published_total.labels(broker="redpanda", sensor_type=stype).inc()
+                message_payload_size.labels(broker="redpanda").observe(len(body.encode()))
+                return resp.status in (200, 201, 204)
     except urllib.error.HTTPError as e:
+        http_requests_total.labels(broker="redpanda", method="POST", status_code=str(e.code)).inc()
+        messages_errors_total.labels(broker="redpanda", sensor_type=stype, error_type="http_error").inc()
         print(f"  ⚠️  Redpanda → {e.code}")
         return False
     except Exception as e:
+        http_requests_total.labels(broker="redpanda", method="POST", status_code="exception").inc()
+        messages_errors_total.labels(broker="redpanda", sensor_type=stype, error_type="exception").inc()
         print(f"  ⚠️  Redpanda → {e}")
         return False
 
 def publish_influx(sid: str, sensor: dict, values: dict) -> bool:
     """Write sensor data to InfluxDB (async, non-blocking)."""
     if not _influx_write_api:
+        influx_write_total.labels(status="skipped").inc()
         return False
     
     def _write_async():
@@ -955,8 +1153,10 @@ def publish_influx(sid: str, sensor: dict, values: dict) -> bool:
             
             if points:
                 _influx_write_api.write(bucket=INFLUX_BUCKET, record=points)
+                influx_write_total.labels(status="success").inc()
                 print(f"  📈 InfluxDB: {len(points)} points written")
         except Exception as e:
+            influx_write_total.labels(status="error").inc()
             print(f"  ⚠️  InfluxDB → {e}")
     
     # Exécution asynchrone (non-bloquante)
@@ -972,6 +1172,14 @@ def main():
     print(f"[CFG] Orion-LD: {ENABLE_ORION}  |  Stellio: {ENABLE_STELLIO}  |  FROST: {ENABLE_FROST}")
     print(f"[CFG] InfluxDB: {ENABLE_INFLUX}  |  Pulsar: {ENABLE_PULSAR}  |  Redpanda: {ENABLE_REDPANDA}")
 
+    # --- Démarrer le serveur Prometheus ---
+    _start_metrics_server()
+
+    # --- Configurer les gauges ---
+    for stype, count in SENSOR_COUNTS.items():
+        sensors_total.labels(sensor_type=stype).set(count)
+    up.set(1)
+
     # Init connectivity checks
     if ENABLE_PULSAR:
         _init_pulsar()
@@ -989,6 +1197,7 @@ def main():
     def signal_handler(*_):
         nonlocal running
         running = False
+        up.set(0)
         print("\n[SIM] 🛑 Arrêt...")
     signal.signal(signal.SIGINT, signal_handler)
     signal.signal(signal.SIGTERM, signal_handler)
@@ -1025,7 +1234,7 @@ def main():
             msg = json.dumps(payload_mqtt, ensure_ascii=False)
 
             # --- MQTT publish ---
-            results = mqtt_client.publish(topic, msg)
+            results = mqtt_client.publish(topic, msg, sensor_type=stype)
             ok_mqtt = [n for n, r in results.items() if r]
             if ok_mqtt:
                 print(f"  📤 {topic} → {','.join(ok_mqtt)}")