eric/smart-city-digital-twin-martinique
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
143 Commits 1 Branch 1 Tag
747741081352fd726bab91d668d4ed5b3acbe5e3
Commit Graph

53 Commits

Author SHA1 Message Date
Eric FELIXINE
7477410813 Session 2026-05-17: GeoServer, PostGIS dédié, MapStore, ChirpStack 
- GeoServer: workspace Digitribe + Data Store PostGIS dédié
- PostGIS dédié: conteneur postgis-smartcity (PostGIS 3.4)
- Couche sensors: 55 capteurs IoT importés depuis OpenRemote
- MapStore: GeoServer WMS ajouté au CORS
- ChirpStack: credentials réinitialisés (admin/admin1234)
- BunkerM: DNS corrigé (underscores → hyphens)
- Ditto: config MongoDB et auth devops
- Documentation: session_resume + TODO.md
 2026-05-17 19:18:24 -04:00
Eric FELIXINE
15e9851b9f Session 2026-05-13: Nettoyage infra, BunkerM+Traefik, agentLink→REST, ChirpStack 
- Nettoyage: suppression conteneurs TTS, anciens Chirpstack, exited/excess
- BunkerM recréé et ajouté à traefik-public (mosquitto2.digitribe.fr)
- Config Traefik mise à jour: 3 fichiers mosquitto2 → bunkerm-bunkerm-1
- AgentLink MQTT désactivé sur 25 assets (master+smartcity)
- REST OpenRemote activé dans simulateur (location GeoJSONPoint incluse)
- ChirpStack: nouveau docker-compose (postgres, redis, mosquitto, chirpstack)
- Session state documenté dans SESSION_STATE_2026-05-13.md
 2026-05-13 08:03:27 -04:00
Eric FELIXINE
5fde1a2c8d feat(lorawan): démarrage ChirpStack et The Things Stack 
- ChirpStack opérationnel (port 8080/8090, gateway bridge UDP 1700)
- The Things Stack opérationnel (port 1885/1884, gateway UDP 1701)
- Fichages de configuration créés
- Docker-compose corrigés (réseaux smartcity-shared)
- Désactivation agentLink sur 35 assets du simulateur
- Correction _or_put: suppression If-Match header (403)
- realm smartcity identifié pour les assets du simulateur
 2026-05-12 17:34:53 -04:00
Eric FELIXINE
7331dbc90b fix(simulator): corrections finales - topics MQTT, ASSET_MAP, location REST 
Corrections:
- Topics MQTT: index basé sur position du capteur (pas compteur global itération)
- ASSET_MAP: mise à jour avec bons asset IDs (agentLink + location)
- Payload REST: ajout attribut location (GeoJSONPoint)
- Désactivation PUT REST sur assets avec agentLink (403 Forbidden)
- MQTT OpenRemote: tentative connexion anonyme (rc=5 persistant)
- Keepalive augmenté à 120s pour stabilité

Note: connexion MQTT au broker Artemis d'OpenRemote échoue (rc=5 Not Authorized)
Le broker nécessite une authentification spécifique non documentée.
Les agents MQTT d'OpenRemote ne reçoivent donc pas les données du simulateur.
La location est déjà correctement définie dans les assets en BDD.
 2026-05-12 08:07:44 -04:00
Eric FELIXINE
4afed8ff2b fix(simulator): connexion MQTT OpenRemote sans auth, location dans payload REST, ASSET_MAP corrigé 
- MQTT OpenRemote: connexion anonyme (pas de credentials) au broker Artemis
- Payload REST: ajout attribut location (GeoJSONPoint) pour chaque capteur
- ASSET_MAP: mise à jour avec les bons asset IDs (ceux avec agentLink + location)
- Topics MQTT: index basé sur position du capteur (pas compteur global)
- Désactivation PUT REST sur assets avec agentLink (403 Forbidden)
- keepalive augmenté à 60s pour stabilité connexion Artemis
 2026-05-12 07:34:29 -04:00
Eric FELIXINE
8b87d95ca5 fix: OpenRemote REST - gestion version If-Match pour PUT assets 
- Récupère la version actuelle de l'asset avant PUT
- Ajoute la version au payload pour éviter HTTP 409 Conflict
- OpenRemote: ✅ les assets sont mis à jour en temps réel
- MQTT OK: 3/4 (EMQX, Mosquitto, BunkerM)
 2026-05-11 14:56:27 -04:00
Eric FELIXINE
918c03dffa fix: Simulateur MQTT 3/4 + OpenRemote master + Mapsettings 
- MQTT OK: 3/4 (EMQX, Mosquitto, BunkerM)
- OpenRemote: utilise realm master (token fonctionnel)
- Realm smartcity recréé dans Keycloak
- Assets IOTSensor créés dans master (30) et smartcity (30)
- Mapsettings: layers iot-sensors + labels pour master et smartcity
- INTERVAL=5s, réseau openremote_default ajouté
- Dockerfile: --no-cache rebuild
 2026-05-11 14:19:53 -04:00
Eric FELIXINE
dfaa240d5a fix: Stabilisation complète Smart City Digital Twin Martinique 
- Correction simulateur: nettoyage code FIWARE (erreurs syntaxe)
- Grafana: dashboard complet 10 panneaux sur grafana.digitribe.fr
- InfluxDB: datasource corrigée (bucket smartcity, org digitribe)
- Nettoyage: suppression services FIWARE (Orion-LD, Stellio, QuantumLeap)
- Pipeline validé: Simulator → 3 MQTT brokers → Telegraf → InfluxDB → Grafana
- Dashboard URL: https://grafana.digitribe.fr/d/smartcity-martinique-complete/

Architecture simplifiée:
- 3 MQTT brokers (EMQX, Mosquitto, BunkerM)
- Telegraf pour agrégation
- InfluxDB pour stockage time-series
- Grafana pour visualisation (Traefik: grafana.digitribe.fr)
 2026-05-08 01:10:30 -04:00
Eric FELIXINE
5a5234f868 FIX: Pulsar Manager credentials + MapStore static files + CrateDB tables + QuantumLeap persistence 2026-05-07 15:07:10 -04:00
Eric FELIXINE
66a22a2421 Fix: InfluxDB token + bucket iot_data créé 
- Token InfluxDB corrigé dans simulator.py (my-super-token)
- Bucket iot_data créé dans InfluxDB
- CrateDB-Stellio ports sécurisés (suppression exposition publique)
- Healthchecks MongoDB/Mosquitto corrigés
- Nettoyage container digital-twin-grafana
 2026-05-07 10:41:16 -04:00
Eric FELIXINE
c27c2c10af fix: QuantumLeap + Redis + simulateur MQTT-only + données test CrateDB 2026-05-06 19:26:13 -04:00
Eric FELIXINE
64022bd9ab fix: Simulateur publie sur 3 brokers (emqx, mosquitto, bunkerm) avec préfixe json/ 2026-05-06 17:50:06 -04:00
Eric FELIXINE
0c1b75fcd3 feat: Add IoT-Agent integration - simulator publishes to smartcity-api-key/{sid}/attrs via EMQX 2026-05-06 16:20:05 -04:00
Eric FELIXINE
b73b02f39d Fix MQTT broker name: mosquitto-traefik -> mainfluxlabs-mosquitto 2026-05-06 11:17:22 -04:00
Eric FELIXINE
c06acf4fe8 feat: distribution service + redpanda consumer + updated flow diagram 
- Add Pulsar distribution service (consumes smartcity-* → MQTT + context brokers)
- Add Redpanda → InfluxDB consumer (redpanda/consumer.py)
- Update FIXED_LOCATIONS with exact OpenRemote asset coordinates
- Fix Pulsar topics (underscore: smartcity-traffic not smartcity-traffic)
- Fix prometheus.yml endpoints (Redpanda:9644, comment inactive stacks)
- Add docker-compose.redpanda-consumer.yml
 2026-05-05 22:12:38 -04:00
Eric FELIXINE
ad31e2289f fix: replace random coords with fixed Martinique locations (no more sea sensors) 
- Replace random.uniform(±0.02°) with FIXED_LOCATIONS dict keyed by type+name
- All 30 named sensor locations mapped to real Martinique coordinates on land
- Coordonnées Martinique: 14.4°N–14.88°N, -61.25°W–-60.85°W
- OpenRemote DB: UPDATE all IOTSensor assets with wrong coords (PostgreSQL jsonb_set)
- All 34 sensor instances now validated as TERRE (100% on land)

Fixed sensors: traffic, airquality, parking, noise, weather, light
 2026-05-05 21:24:29 -04:00
Eric FELIXINE
3f06298819 fix: Coordonnées capteurs Martinique - réduit plage à ±0.02 pour éviter mer 2026-05-05 18:38:27 -04:00
Eric FELIXINE
9ecc237bdc fix: ENABLE_REDPANDA/STELLIO/INFLUX bugs + Redpanda content-type + topics 2026-05-05 17:10:30 -04:00
Eric FELIXINE
81de240b40 fix: ENABLE_INFLUX bug - accepter true/yes/on (pas seulement 1) 2026-05-05 16:20:32 -04:00
Eric FELIXINE
ad613beefb feat: Pulsar distribution service (Simulator → Pulsar → Brokers) 
- Fix Pulsar: use binary client (port 6650) instead of non-existent REST /produce API
- Add pulsar-client to Dockerfile
- Create pulsar/distribution.py: consumes Pulsar and republishes to MQTT (EMQX/Mosquitto), NGSI-LD (Orion/Stellio), FROST
- Add docker-compose.distribution.yml for the distribution service
- Tested: Messages successfully distributed to EMQX and Orion-LD
- Update session resume
 2026-05-05 10:20:13 -04:00
Eric FELIXINE
01c2be4930 feat(simulator): real-time (1s), fix ENABLE_PULSAR, add Pulsar/Redpanda publish, fix InfluxDB URL 
- Change INTERVAL to 1s for real-time sensor data
- Fix ENABLE_PULSAR comparison (accept 'true'/'false' strings)
- Add publish_pulsar() and publish_redpanda() functions
- Fix InfluxDB URL (smart-city-influxdb instead of digital-twin-influxdb)
- Add docker-compose.yml with simulator service
- Add redpanda config and start script
- Add session_resume_2026-05-05.md
 2026-05-05 02:53:43 -04:00
Eric FELIXINE
6ee9e5103e Fix OpenRemote: Use localhost:8080 for token URL 
- Replace openremote-keycloak-1 (internal Docker) with localhost:8080 (Traefik)
- Fixes [Errno -2] Name or service not known error
 2026-05-04 23:43:46 -04:00
Eric FELIXINE
92a3026a7b Fix Orion-LD: Clean up debug code 
- Remove debug print statements from publish_orion()
- Orion-LD now works: DELETE + POST fixes zombie entities
- All entities now created with source/mqttTopic fields
- Traceability fully functional for AirQualityObserved
 2026-05-04 23:29:51 -04:00
Eric FELIXINE
f3345ff7fe Debug: Add logging to publish_orion to trace POST vs PATCH 
- Print entity ID before POST
- Print 409 Conflict message explicitly
- This will help understand why entities are not being created
 2026-05-04 23:26:44 -04:00
Eric FELIXINE
8fcfb4046a Fix Orion-LD: Remove source from @context 
- Testing shows Orion-LD stores source properly WITHOUT defining it in @context
- When defined in @context, it's stored with full URI as key
- Without @context definition, source is stored and returned correctly
- Simulator now creates entities with proper source/mqttTopic fields
 2026-05-04 23:16:54 -04:00
Eric FELIXINE
1ed03b5a57 Fix Orion-LD: Add source to @context + PATCH with full payload 
- ORION_CONTEXT now includes source definition (uri.fiware.org)
- PATCH /entities/{id}/attrs now sends full entity (with @context)
- Orion-LD requires @context even in PATCH requests
- This fixes 400 Bad Request errors on update
 2026-05-04 23:12:56 -04:00
Eric FELIXINE
6c8949f20f Fix publish_orion: PATCH sends attrs only (not id/type/@context) 
- PATCH /entities/{eid}/attrs now sends only attributes
- This allows updating entities with new fields (source, mqttTopic)
 2026-05-04 23:03:02 -04:00
Eric FELIXINE
1f61982e56 Simulator: Fix FROST container (frost_http-web-1 image, port 8090) 2026-05-04 22:46:37 -04:00
Eric FELIXINE
5fe800af0d Simulator: Fix INFLUX_URL (localhost:8086 not Docker internal) 2026-05-04 22:45:03 -04:00
Eric FELIXINE
d9cb0531cb Simulator: Fix FROST port 8088 + traceability fields 
- FROST_URL: localhost:8088 (avoid 8086 conflict with InfluxDB)
- Orion-LD: localhost:2026 (not Docker internal hostname)
- source field (NGSI-LD standard) for broker identification
- mqttTopic field (custom) for MQTT topic tracing
- Updated references/data-models.md with schemas
 2026-05-04 22:41:45 -04:00
Eric FELIXINE
cad1c06422 Simulator: Add source+mqttTopic traceability for Fiware brokers 2026-05-04 22:25:23 -04:00
Eric FELIXINE
7f0543de85 Simulator: Stellio URL to localhost:8087 (exposed container) 2026-05-04 22:01:18 -04:00
Eric FELIXINE
a2502eff91 Simulator: FROST_URL default to localhost:8086 (expose frost_http-web-1:8080) 2026-05-04 21:57:57 -04:00
Eric FELIXINE
4fc233d138 Simulator: default MQTT hosts to localhost (not host.docker.internal) 2026-05-04 21:55:25 -04:00
Eric FELIXINE
88f0d1e675 Simulator: use localhost URLs for Fiware brokers (Orion:2026, Stellio:8080) 2026-05-04 21:49:59 -04:00
Eric FELIXINE
5abab6cc00 Simulator: BunkerM port 1900 is MQTT simple (not TLS) - connection fix 2026-05-04 21:35:39 -04:00
Eric FELIXINE
2660d5946a Simulator: re-add BunkerM (MQTTS) to broker list 2026-05-04 21:08:17 -04:00
Eric FELIXINE
25e490c758 Simulator: fix variable placement (outside docstring) + host.docker.internal support 2026-05-04 21:01:09 -04:00
Eric FELIXINE
2e15a48303 Simulator: use host.docker.internal as default (Docker robust) 2026-05-04 20:53:46 -04:00
Eric FELIXINE
816f5fcddc Simulator: use localhost for MQTT brokers (fix DNS resolution) 2026-05-04 20:53:10 -04:00
Eric FELIXINE
ee708fb4ab Fix: InfluxDB async write + Grafana Org rename + GeoServer workspace 2026-05-04 18:37:29 -04:00
Eric FELIXINE
42d1223b14 GeoServer workspace Digitribe + InfluxDB support + data flow diagrams 2026-05-04 18:13:48 -04:00
Eric FELIXINE
df725eadbc Fix OpenRemote auth (password grant + client_secret), add Grafana dashboard, update session resume 2026-05-04 2026-05-04 17:34:24 -04:00
Eric FELIXINE
818ebbce6d fix(simulator): add async threads for FROST/Orion/Stellio 
- Non-blocking calls via threading for external brokers
- FROST_URL fixed: frost_http-web-1:8080
- Healthcheck uses Python (no wget/curl)
- InfluxDB writes no longer blocked by slow brokers

✅ Simulator now HEALTHY with async broker calls
 2026-05-04 14:56:03 -04:00
Eric FELIXINE
aa42a213bb fix: Stellio/Orion - use NGSI-LD core context only (remove raw.githubusercontent.com) 
- STELLIO_INLINE_CONTEXT: replaced long inline dict with uri.etsi.org core URL
- ORION_CONTEXT: removed raw.githubusercontent.com URLs (not accessible from containers)
- publish_stellio(): added NGSILD-Tenant header (urn:ngsi-ld:tenant:default)
- publish_stellio(): keep @context in payload (Stellio needs it to resolve vocabulary)
- Added STELLIO_URL and STELLIO_TENANT env vars for host-based execution via Traefik

Fixes: Stellio 503 JsonLdError, Orion-LD context resolution failures.
Tested: Stellio 201, Orion-LD 207.
 2026-05-04 10:35:18 -04:00
Eric FELIXINE
ba13bf1321 fix: minimal NGSI-LD context without @vocab (Stellio compatible) 2026-05-04 10:09:08 -04:00
Eric FELIXINE
16c02c91dc fix: use Gitea raw context URL for Stellio (replaces blocked ETSI URL) 2026-05-04 09:57:33 -04:00
Eric FELIXINE
871194a5e3 feat: Smart Data Models + FROST/NGSI-LD fixes 
- Intégration Smart Data Models (AirQualityObserved, TrafficFlowObserved, etc.)
- Payloads NGSI-LD avec @context officiels smartdatamodels.org
- FROST: Ajout FeatureOfInterest dans Observations (fix 400)
- FROST: Migration HTTP-only + suppression Locations du Thing
- URLs unités QUDT corrigées
- OpenRemote: Token realm smartcity (en attente 401)
- Orion-LD/Stellio: 204 success avec Smart Data Models
 2026-05-03 10:53:55 -04:00
Eric FELIXINE
e8270b7d73 Fix: OpenRemote token with admin-cli password grant, add OR_TOKEN_REALM, fix FROST_URL 2026-05-03 08:47:47 -04:00
Eric FELIXINE
8bb0381fff fix: OpenRemote token Basic Auth + corrected URLs (Keycloak container name) 
- Use Basic Auth for client_secret_basic
- OR_TOKEN_URL points to openremote-keycloak-1 container
- Assets copied to smartcity realm DB
 2026-05-03 03:08:24 -04:00