eric/smart-city-digital-twin-martinique
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
183 Commits 1 Branch 1 Tag
ae35506db6fafa479a747ef404c1ad560720531a
Commit Graph

183 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Eric FELIXINE
ae35506db6 chore: backup session 2026-06-01 final — snapshot, resume, TODO, all fixes documented 2026-06-01 12:24:34 -04:00
Eric FELIXINE
8c38a23b4b chore: session resume 2026-06-01 final — JupyterHub spawn fix, all creds documented 2026-06-01 12:07:10 -04:00
Eric FELIXINE
cca9e4aedc fix: JupyterHub spawn - switch to LocalProcessSpawner, fix password hash, eric user 
- SimpleLocalProcessSpawner doesn't pass JUPYTERHUB_SERVICE_URL in JH 5.3.0
- LocalProcessSpawner handles env vars correctly
- Fixed eric password hash (bcrypt instead of PBKDF2)
- eric user created with admin rights
- JupyterLab accessible at https://jupyter.digitribe.fr
- Credentials: eric / Digitribe972
 2026-06-01 11:45:52 -04:00
Eric FELIXINE
85199fc3f0 chore: session backup 2026-06-01 continue — Kafka/Trino/JupyterHub fixes, TODO update 2026-06-01 10:39:11 -04:00
Eric FELIXINE
cb45b89f1f fix: JupyterHub Dockerfile - add eric user, sudo, fix DB path (4 slashes) 2026-06-01 10:26:47 -04:00
Eric FELIXINE
9e933fea66 chore: session backup 2026-06-01 final — TODO restructuré, état complet 2026-05-30 08:49:31 -04:00
Eric FELIXINE
acdf250470 chore: backup session 2026-06-01b — JupyterHub fix, Hermes Dashboard, OR mbtiles, Trino node.properties 
Summary of changes:
- JupyterHub: fix DB path (absolute), Dockerfile cleanup, SimpleLocalProcessSpawner
- JupyterHub: user eric created as admin
- Hermes Dashboard WebUI + TUI chat service (systemd, localhost:9119, auto-boot)
- OR mbtiles: generated Martinique PNG tiles (5690 tiles, 10.9MB) — needs PBF for OR
- OR mbtiles: restored original PBF with corrected metadata (world bounds, Martinique center)
- OR mapsettings: verified center=[-61,14.5], bounds=Martinique, minZoom=0
- Trino: added node.properties (node.environment=production) — needs restart
- TODO.md: updated with current state
- session_resume_consolide.md: created (per-session summary)
 2026-05-30 08:14:47 -04:00
Eric FELIXINE
008f1679ce fix: JupyterHub DB path + user eric + OR mbtiles bounds + Hermes Dashboard 
- Fix JupyterHub: sqlite db_url absolute path (was double-nested /srv/jupyterhub/srv/jupyterhub)
- Create user eric as admin in JupyterHub (id=2, authorized)
- Fix OpenRemote mbtiles: bounds metadata = world (-180,-85,180,85) for free zoom
- OR map API confirmed working: center=[-61,14.5], minZoom=0, bounds=Martinique
- Add Hermes Dashboard WebUI + TUI chat service (localhost:9119, auto-start at boot)
- Add generate_martinique_mbtiles.py script (future tile generation)

Known issues:
- JupyterHub spawn timeout (singleuser server slow to start, increased to 120s)
- OR mbtiles still contains Netherlands vector tiles (need Martinique tiles)
- Kafka, Trino still in restart loop (separate fix needed)
 2026-05-29 07:01:00 -04:00
Eric FELIXINE
a234e808f2 chore: add VRE stack configs (JupyterHub + Zeppelin) + lakehouse components 
- Add VRE directory with JupyterHub + Zeppelin docker-compose configs
- Add Gravitino, Flink, Kafka, MinIO, Trino lakehouse stack
- Add Superset, Metabase, StarRocks analytics tools
- Session reprise après crash 2026-06-01

Infrastructure: 86 conteneurs total
Known issues: Kafka (no ZK conn), Trino (node.env null), JupyterHub (DB path)
 2026-05-29 02:21:08 -04:00
Eric FELIXINE
486c1d2675 feat: Add OpenRemote stack config (docker-compose + traefik) 
- Docker-compose based on GitHub repo openremote/openremote
- Images: timescale/timescaledb-ha:pg15, openremote/keycloak:latest, openremote/manager:1.24.0
- All 3 services healthy and running
- URL: https://openremote.digitribe.fr/manager/
 2026-05-28 23:22:25 -04:00
Eric FELIXINE
184f3ca8dd chore: session backup 2026-05-27 — OpenRemote deployed with KC 23.0.7, MindsDB config updated 2026-05-27 13:14:58 -04:00
Eric FELIXINE
feb80694ab chore: update TODO — fix session date references 2026-05-26 22:55:46 -04:00
Eric FELIXINE
a19ee4080f chore: remove erroneous 2026-05-29 snapshot (correct date is 2026-05-26) 2026-05-26 22:54:57 -04:00
Eric FELIXINE
46bb937714 chore: session backup 2026-05-26 — reprise après crash, snapshot + resume + TODO update 2026-05-26 22:54:37 -04:00
Eric FELIXINE
e0d023d372 docs: add architecture inventory 2026-05-27 2026-05-26 21:04:35 -04:00
Eric FELIXINE
19cb678791 chore: session backup 2026-05-27 — ODK deployed, project created, TODO updated 2026-05-26 20:54:50 -04:00
Eric FELIXINE
978280f866 fix: ODK Traefik config — add odk.yml route, ODK now accessible at https://odk.digitribe.fr 2026-05-26 20:26:17 -04:00
Eric FELIXINE
89a821a364 chore: update TODO.md — ODK progress, ChirpStack pw reset, Smart App City arch 2026-05-26 19:06:07 -04:00
Eric FELIXINE
94f74f2dfc feat: add smart-app-city sub-project architecture 
- Architecture globale (React Native + NestJS + FastAPI)
- Beckn Protocol (OTN-DPI) integration docs
- AI layer: RAG pipeline + AI Agents (LocalAI + Qdrant)
- i18n: FR/EN/ES/DE support
- Docker Compose for backend services
- Project structure with frontend, backend, ai, beckn directories
 2026-05-26 18:47:02 -04:00
Eric FELIXINE
f1e1b98519 chore: session backup 2026-05-28 — TODO update + snapshot 2026-05-26 17:20:04 -04:00
Eric FELIXINE
e4c558c296 Dashboard v7: valeurs temps réel dans panneau stat + dropdown capteur 
- Panneau Stat en haut: 7 métriques temps réel (temp, humidité, NO2, O3, CO, batterie, PM2.5)
- Dropdown avec includeAll + allValue '.*'
- Regex  dans toutes les queries Flux
- historique: 7j pour NO2 et PM2.5
- Refresh: 5s
 2026-05-26 14:20:00 -04:00
Eric FELIXINE
65e2d42f63 Dashboard v5: variable capteur + tooltip details + coordonnees Sainte-Anne 
- Ajout variable  pour filtrage par topic InfluxDB
- Tous les panels filtres par capteur selectionne
- Tooltip markersGeomap en mode details
- Coordonnees Sainte-Anne corrigees (deplacement ~300m sur terre)
- 1001 topics disponibles dans InfluxDB
 2026-05-26 14:05:33 -04:00
Eric FELIXINE
a7716102fd Fix GeoMap dashboard v3 - temperature_celsius + Geomap layer config 
Fixes:
- Temperature field: temperature_c (wrong) -> temperature_celsius (correct)
- Geomap panel: added explicit location config with lat/lon field mapping
- Added PM2.5 timeseries panel
- Dashboard UID: geomap-test-v1
 2026-05-26 13:52:25 -04:00
Eric FELIXINE
7643d88ffb Add GeoMap dashboard + ChirpStack REST API config 
- Grafana GeoMap dashboard (PostGIS + InfluxDB) for real-time sensor visualization
- Dashboard accessible at: https://grafana.digitribe.fr/d/geosmart-city-2026/smart-city-geomap-temps-reel
- PostGIS datasource added (user: grafana, network: smartcity-shared)
- InfluxDB-SmartCity-Correct datasource configured
- ChirpStack REST API: added --cors-origins flag
 2026-05-26 13:14:01 -04:00
Eric FELIXINE
7df2f6798f feat: deploy Superset and Metabase behind Traefik 
- Superset: docker-compose.superset.yml (app + postgres + redis)
  URL: https://superset.digitribe.fr
  Port: 8088 (internal), Traefik routes Host(superset.digitribe.fr)

- Metabase: docker-compose.metabase.yml (app + postgres)
  URL: https://metabase.digitribe.fr
  Port: 3000 (internal), Traefik routes Host(metabase.digitribe.fr)

- Traefik configs: 31-superset.yml, 32-metabase.yml
- Both services use smartcity-shared and traefik-public networks
- Both use letsencrypt TLS certificates

Verified:
- Superset: UP healthy, accessible via https://superset.digitribe.fr
- Metabase: UP healthy, accessible via https://metabase.digitribe.fr
 2026-05-25 22:59:25 -04:00
Eric FELIXINE
943836f8fb feat: activate BunkerM MQTT broker + fix Telegraf 
- BunkerM: recreated with port 1883 (external) -> 1900 (internal)
- BunkerM: disabled dynsec plugin, using password_file auth (bunker/bunker)
- Simulator: ENABLE_BUNKER=1, BUNKERM_PORT=1900
- Telegraf: reactivated BunkerM consumer on port 1900
- Telegraf: recreated container (3 MQTT consumers connected)
- Grafana: dashboard v4 with corrected Flux queries
- Grafana: datasource fixed (bucket=smartcity, token=my-super-token)

Verified:
- Simulator publishes to EMQX ✅, Mosquitto ✅, BunkerM ✅
- Telegraf receives from all 3 brokers ✅
- InfluxDB has data from all brokers ✅
- Grafana dashboard displays data ✅
 2026-05-25 20:03:55 -04:00
Eric FELIXINE
5bbd5a6e5d fix: Grafana dashboard 'no data' — datasource + Flux queries 
- Fix datasource: bucket=smartcity, token=my-super-token, org=digitribe
- Fix dashboard queries: filter by topic tag instead of _measurement
  (all data in measurement 'mqtt_consumer', type in tag 'topic')
- Fix field names: temperature_c→temperature_celsius, luminosity→brightness_lux
- Update dashboard to v3 with 15 panels (airquality, traffic, parking, weather, noise, light)
- Update TODO.md and session_resume

Tested: PM2.5 ✅, Temperature ✅, Vehicle Count ✅ via Grafana API
 2026-05-25 16:39:50 -04:00
Eric FELIXINE
6d1d9c8620 fix: telegraf containers names + openremote pg image + session snapshot 2026-05-25 
- telegraf.conf: fix Mosquitto/BunkerM container names (hyphens not underscores)
- tegraf.conf: comment out BunkerM consumer (auth fails, simulator not sending)
- openremote/docker-compose.yml: switch PG image to timescaledb-ha:pg15 (fixes timescaledb_toolkit crash)
- Add session_resume + architecture snapshot 2026-05-25
- Update TODO.md with current status
 2026-05-25 14:13:39 -04:00
Eric FELIXINE
eb97f2a7dd Fix: Traefik dynamic config, Ditto things startup crash, and OpenRemote sensor coordinates 2026-05-20 18:18:21 -04:00
Eric FELIXINE
45f3ab8a3d docs: session resume 2026-05-20 - stabilization and platforms deployment 2026-05-20 13:19:29 -04:00
Eric FELIXINE
98f0bcb021 Session 2026-05-20: Contexus MQTT devices, OpenRemote agent, 60 capteurs configures 2026-05-20 00:58:48 -04:00
Eric FELIXINE
a4e05f557c Session 2026-05-20: Contexus deploye, OpenRemote assets corriges, Traefik config fix 2026-05-19 21:48:38 -04:00
Eric FELIXINE
5ddbf7de93 Add map screenshot with assets 2026-05-19 20:54:26 -04:00
Eric FELIXINE
805986e3f6 Add Playwright screenshots 2026-05-19: Manager login + map page 2026-05-19 19:01:40 -04:00
Eric FELIXINE
d4605ee072 Add session resume 2026-05-19 2026-05-19 16:36:08 -04:00
Eric FELIXINE
2377bc07fd Session 2026-05-19: OpenRemote map display investigation, cleanup, fresh install 
- Investigated map display issues (agentLink, GeoJSON coords, realm config)
- Cleaned up all dashboards and containers
- Fresh Manager installation (PostgreSQL in recovery)
- Updated TODO.md with current status
- GeoJSON proxy: fixed coordinate order (lon/lat)
- Session resume saved
 2026-05-19 16:22:26 -04:00
Eric FELIXINE
d1e6bdb685 docs: add skills inventory to TODO.md (epicollect5, odk, kobo, superset, metabase, contexus) 2026-05-19 15:48:46 -04:00
Eric FELIXINE
47746b584c fix: OpenRemote PUT 403/409, MQTTv5 callback, geojson-proxy API REST 
- simulator.py: Fix MQTTv5 callback crash (5th arg *args)
- simulator.py: Fix _or_put() - GET version+realm before PUT, inject version in payload
- simulator.py: Fix token TTL (min 30s cache)
- simulator.py: Round-robin OR updates (~5 assets/iteration instead of 60)
- geojson-proxy: Rewrite using REST API instead of psycopg2 (PG auth issue)
- geojson-proxy: Add sensorType + attributes in properties for map styling
- docker-compose.yml: Add openremote_default network + DB vars for proxy
- docker-compose.yml: Add OR_REALM=master for geojson-proxy

Resolves: OpenRemote 403 (wrong realm in payload), 409 (missing version),
MQTTv5 callback crash, geojson-proxy DB connection failure
 2026-05-18 10:04:12 -04:00
Eric FELIXINE
7937e2bb43 Session resume 2026-05-17: sauvegarde finale 2026-05-17 20:05:42 -04:00
Eric FELIXINE
55fabea16a Documentation géospatiale: GeoServer, PostGIS, MapStore 2026-05-17 19:55:40 -04:00
Eric FELIXINE
7477410813 Session 2026-05-17: GeoServer, PostGIS dédié, MapStore, ChirpStack 
- GeoServer: workspace Digitribe + Data Store PostGIS dédié
- PostGIS dédié: conteneur postgis-smartcity (PostGIS 3.4)
- Couche sensors: 55 capteurs IoT importés depuis OpenRemote
- MapStore: GeoServer WMS ajouté au CORS
- ChirpStack: credentials réinitialisés (admin/admin1234)
- BunkerM: DNS corrigé (underscores → hyphens)
- Ditto: config MongoDB et auth devops
- Documentation: session_resume + TODO.md
 2026-05-17 19:18:24 -04:00
Eric FELIXINE
1006df137d Session 2026-05-13: Nettoyage infra, BunkerM+Traefik, agentLink→REST, ChirpStack 
- Nettoyage: suppression conteneurs TTS, anciens Chirpstack, exited/excess
- BunkerM recréé et ajouté à traefik-public (mosquitto2.digitribe.fr)
- Config Traefik mise à jour: 3 fichiers → bunkerm-bunkerm-1
- AgentLink MQTT désactivé sur 25 assets (master+smartcity)
- REST OpenRemote activé dans simulateur (location GeoJSONPoint incluse)
- ChirpStack: nouveau docker-compose dans submodule
 2026-05-13 08:05:20 -04:00
Eric FELIXINE
15e9851b9f Session 2026-05-13: Nettoyage infra, BunkerM+Traefik, agentLink→REST, ChirpStack 
- Nettoyage: suppression conteneurs TTS, anciens Chirpstack, exited/excess
- BunkerM recréé et ajouté à traefik-public (mosquitto2.digitribe.fr)
- Config Traefik mise à jour: 3 fichiers mosquitto2 → bunkerm-bunkerm-1
- AgentLink MQTT désactivé sur 25 assets (master+smartcity)
- REST OpenRemote activé dans simulateur (location GeoJSONPoint incluse)
- ChirpStack: nouveau docker-compose (postgres, redis, mosquitto, chirpstack)
- Session state documenté dans SESSION_STATE_2026-05-13.md
 2026-05-13 08:03:27 -04:00
Eric FELIXINE
5fde1a2c8d feat(lorawan): démarrage ChirpStack et The Things Stack 
- ChirpStack opérationnel (port 8080/8090, gateway bridge UDP 1700)
- The Things Stack opérationnel (port 1885/1884, gateway UDP 1701)
- Fichages de configuration créés
- Docker-compose corrigés (réseaux smartcity-shared)
- Désactivation agentLink sur 35 assets du simulateur
- Correction _or_put: suppression If-Match header (403)
- realm smartcity identifié pour les assets du simulateur
 2026-05-12 17:34:53 -04:00
Eric FELIXINE
a05e13c30c feat(lorawan): ajout ChirpStack et The Things Stack 
- Skills créés: chirpstack-lorawan, the-things-stack-lorawan
- docker-compose.chirpstack.yml: ChirpStack derrière Traefik
- docker-compose.the-things-stack.yml: TTS derrière Traefik
- data-flow-diagram.md: mise à jour avec LoRaWAN
- DOCKER-ARCHITECTURE: ajout conteneurs LoRaWAN
- Subdomaines Traefik: chirpstack, tts

Skills créés dans ~/.hermes/skills/iot/:
- chirpstack-lorawan
- the-things-stack-lorawan
 2026-05-12 11:29:30 -04:00
Eric FELIXINE
dbf8b7f5ca docs: état des lieux localisation capteurs OpenRemote 
- Documentation des découvertes et corrections appliquées
- Problèmes restants identifiés (connexion MQTT, topics, déconnexion)
- Prochaines étapes recommandées
 2026-05-12 08:18:32 -04:00
Eric FELIXINE
7331dbc90b fix(simulator): corrections finales - topics MQTT, ASSET_MAP, location REST 
Corrections:
- Topics MQTT: index basé sur position du capteur (pas compteur global itération)
- ASSET_MAP: mise à jour avec bons asset IDs (agentLink + location)
- Payload REST: ajout attribut location (GeoJSONPoint)
- Désactivation PUT REST sur assets avec agentLink (403 Forbidden)
- MQTT OpenRemote: tentative connexion anonyme (rc=5 persistant)
- Keepalive augmenté à 120s pour stabilité

Note: connexion MQTT au broker Artemis d'OpenRemote échoue (rc=5 Not Authorized)
Le broker nécessite une authentification spécifique non documentée.
Les agents MQTT d'OpenRemote ne reçoivent donc pas les données du simulateur.
La location est déjà correctement définie dans les assets en BDD.
 2026-05-12 08:07:44 -04:00
Eric FELIXINE
4afed8ff2b fix(simulator): connexion MQTT OpenRemote sans auth, location dans payload REST, ASSET_MAP corrigé 
- MQTT OpenRemote: connexion anonyme (pas de credentials) au broker Artemis
- Payload REST: ajout attribut location (GeoJSONPoint) pour chaque capteur
- ASSET_MAP: mise à jour avec les bons asset IDs (ceux avec agentLink + location)
- Topics MQTT: index basé sur position du capteur (pas compteur global)
- Désactivation PUT REST sur assets avec agentLink (403 Forbidden)
- keepalive augmenté à 60s pour stabilité connexion Artemis
 2026-05-12 07:34:29 -04:00
Eric FELIXINE
8b87d95ca5 fix: OpenRemote REST - gestion version If-Match pour PUT assets 
- Récupère la version actuelle de l'asset avant PUT
- Ajoute la version au payload pour éviter HTTP 409 Conflict
- OpenRemote: ✅ les assets sont mis à jour en temps réel
- MQTT OK: 3/4 (EMQX, Mosquitto, BunkerM)
 2026-05-11 14:56:27 -04:00
Eric FELIXINE
918c03dffa fix: Simulateur MQTT 3/4 + OpenRemote master + Mapsettings 
- MQTT OK: 3/4 (EMQX, Mosquitto, BunkerM)
- OpenRemote: utilise realm master (token fonctionnel)
- Realm smartcity recréé dans Keycloak
- Assets IOTSensor créés dans master (30) et smartcity (30)
- Mapsettings: layers iot-sensors + labels pour master et smartcity
- INTERVAL=5s, réseau openremote_default ajouté
- Dockerfile: --no-cache rebuild
 2026-05-11 14:19:53 -04:00