eric/cariflex
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Complete infrastructure: CitrineOS+OpenADR+FM integration, asset sync, scheduling, IaC scripts, K8s manifests, documentation

Browse Source
This commit is contained in:
Eric F
2026-06-10 13:39:47 -04:00
parent ce67b8e9f6
commit 7fde58bf6a
3 changed files with 609 additions and 178 deletions
Download Patch File Download Diff File Expand all files Collapse all files

306
docs/INFRASTRUCTURE.md
View File

@@ -3,228 +3,178 @@
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────────────────────┐
│ CARIFLEX EMS │
│ Energy Management System for Martinique │
├─────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ OpenADR │ │ FlexMeasures│ │ Grafana │ │
│ │ VTN/VEN │───▶│ Server │───▶│ Dashboard │ │
│ │ (S2+OpenADR)│ │ (EMS Core) │ │ (14 panels)│ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ PostgreSQL │ │ Redis │ │ Traefik │ │
│ │ (time-series)│ │ (job queue) │ │ (reverse │ │
│ │ │ │ │ │ proxy) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────────────┐
│ CARIFLEX EMS │
│ Energy Management System for Martinique │
├─────────────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ OpenADR │ │ FlexMeasures│ │ Grafana │ │
│ │ VTN/VEN │───▶│ Server │───▶│ Dashboard │ │
│ │ (S2+OpenADR)│ │ (EMS Core) │ │ (17 panels)│ │
│ └──────────────┘ └──────┬───────┘ └──────────────┘ │
│ │ │ │
│ │ ▼ │
│ │ ┌──────────────┐ │
│ │ │ CitrineOS │ │
│ │ │ (OCPP 2.0.1)│ │
│ │ └──────┬───────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ PostgreSQL │ │ RabbitMQ │ │
│ │ (time-series)│ │ (AMQP/OCPP) │ │
│ └──────────────┘ └──────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────────────┘
```
## Integration Flow
### OpenADR → FlexMeasures → CitrineOS
```
1. OpenADR VTN creates events (price, load control)
2. OpenADR VEN receives events via poll
3. VEN posts data to FlexMeasures API (sensors 84, 86, 87)
4. FlexMeasures creates charging schedules based on prices
5. Schedules sent to CitrineOS via RabbitMQ (OCPP)
6. CitrineOS sends OCPP profiles to EV charge points
7. Grafana displays real-time data from all sources
```
### Asset Synchronization
- FM EV assets (sensors 61-70) ↔ CitrineOS charge points (CP001-CP010)
- Sync script ensures consistency of: count, power, location
- OCPP 2.0.1 protocol for charge point communication
## Network Architecture
### Networks
| Network | Subnet | Purpose | External |
|---------|--------|---------|----------|
| `traefik-public` | 172.29.0.0/16 | Public services (FM, Grafana) | Yes |
| `openadr-internal` | 192.168.240.0/24 | OpenADR VTN/VEN communication | No |
| `traefik-public` | 172.29.0.0/16 | Public services | Yes |
| `config_cariflex-internal` | 172.23.0.0/16 | Internal services | No |
| `openadr-internal` | 192.168.240.0/24 | OpenADR VTN/VEN | No |
### DNS/Hosts
| Service | Domain | Container | Port |
|---------|--------|-----------|------|
| FlexMeasures | cariflex.digitribe.fr | flexmeasures-server | 5000 |
| Grafana | grafana.digitribe.fr | smart-city-grafana | 3000 |
| CitrineOS | citrineos.digitribe.fr | cariflex-citrineos-server | 8080 |
| RabbitMQ Mgmt | amqp.digitribe.fr | cariflex-amqp | 15672 |
| OpenADR VTN | - | openadr-vtn | 8080 |
| OpenADR VEN | - | openadr-ven | - |
### Firewall (UFW)
- Public ports: 80 (HTTP), 443 (HTTPS), 51820 (WireGuard)
- Internal networks: 172.16.0.0/12, 10.0.0.0/8, 192.168.0.0/16, 192.168.240.0/24
## Docker Containers
### Cariflex EMS Stack
| Container | Image | Network | IP | Port | Status |
|-----------|-------|---------|----|---- |--------|
| flexmeasures-server | lfenergy/flexmeasures:latest | traefik-public | 172.29.0.x | 5000 | ✅ UP |
| flexmeasures-worker | lfenergy/flexmeasures:latest | traefik-public | 172.29.0.x | - | ✅ UP |
| flexmeasures-db | postgres:17 | traefik-public | 172.29.0.x | 5432 | ✅ UP |
| flexmeasures-redis | redis:7-alpine | traefik-public | 172.29.0.x | 6379 | ✅ UP |
| openadr-vtn | flexmeasures-openadr-vtn | openadr-internal | 192.168.240.10 | 8080 | ✅ UP |
| openadr-ven | flexmeasures-openadr-ven | openadr-internal | 192.168.240.11 | - | ✅ UP |
| smart-city-grafana | grafana/grafana | traefik-public | - | 3000 | ✅ UP |
## IP Address Allocation (phpIPAM)
### Cariflex Subnet: 192.168.240.0/24
| IP | Service | Container | MAC |
|----|---------|-----------|-----|
| 192.168.240.1 | Gateway | - | - |
| 192.168.240.10 | OpenADR VTN | openadr-vtn | - |
| 192.168.240.11 | OpenADR VEN | openadr-ven | - |
| 192.168.240.128-254 | Reserved | - | - |
### Public Subnet (traefik-public): 172.29.0.0/16
| IP | Service | Domain |
|----|---------|--------|
| 172.29.0.x | FlexMeasures | cariflex.digitribe.fr |
| 172.29.0.x | Grafana | grafana.digitribe.fr |
| Container | Image | Network | IP | Status |
|-----------|-------|---------|----|--------|
| flexmeasures-server | lfenergy/flexmeasures:latest | traefik-public | 172.29.0.x | ✅ UP |
| flexmeasures-worker | lfenergy/flexmeasures:latest | traefik-public | 172.29.0.x | ✅ UP |
| flexmeasures-db | postgres:17 | traefik-public | 172.29.0.x | ✅ UP |
| flexmeasures-redis | redis:7-alpine | traefik-public | 172.29.0.x | ✅ UP |
| openadr-vtn | flexmeasures-openadr-vtn | openadr-internal | 192.168.240.10 | ✅ UP |
| openadr-ven | flexmeasures-openadr-ven | openadr-internal | 192.168.240.11 | ✅ UP |
| cariflex-citrineos-server | ghcr.io/citrineos/citrineos-server:latest | config_cariflex-internal | 172.23.0.4 | ✅ UP |
| cariflex-citrineos-db | postgis/postgis:16-3.5 | config_cariflex-internal | 172.23.0.x | ✅ UP |
| cariflex-amqp | rabbitmq:3-management | config_cariflex-internal | 172.23.0.3 | ✅ UP |
| smart-city-grafana | grafana/grafana | traefik-public | - | ✅ UP |
## Sensor Mapping
### FlexMeasures Sensors
| ID | Name | Unit | Type | Source |
|----|------|------|------|--------|
| 41-50 | pv_01..10_power | kW | PV Production | Simulator |
| 51-60 | bat_01..10_power | kWh | Battery SOC | Simulator |
| 61-70 | ev_01..10_power | kWh | EV Charge | Simulator |
| 81 | irradiance | W/m² | Weather | Open-Meteo |
| 82 | temperature | °C | Weather | Open-Meteo |
| 83 | wind_speed | m/s | Weather | Open-Meteo |
| 84 | consumption_price | EUR/MWh | Price | OpenADR VTN |
| 85 | production_price | EUR/MWh | Price | Simulated |
| 86 | load_control_signal | % | DSR | OpenADR VTN |
| 87 | demand_response_signal | dimensionless | DSR | OpenADR VTN |
### OpenADR Sensors (from VTN)
| ID | Name | Unit | Source |
|----|------|------|--------|
| 84 | consumption_price | EUR/MWh | OpenADR VTN |
| 86 | load_control_signal | % | OpenADR VTN |
| 87 | demand_response_signal | dimensionless | OpenADR VTN |
### S2 Resource Mapping
| S2 Resource | FM Sensor | Asset | Min Power | Max Power |
|-------------|-----------|-------|-----------|-----------|
| battery_01-05 | 51-55 | Bat_01-05 | -50 kW | +50 kW |
| ev_01-05 | 61-65 | EV_01-05 | -22 kW | +22 kW |
### EV Charging Sensors (CitrineOS ↔ FM)
| FM Sensor | CitrineOS CP | Connector | Max Power |
|-----------|--------------|-----------|-----------|
| 61 | CP001 | 1 | 22 kW |
| 62 | CP002 | 1 | 22 kW |
| 63 | CP003 | 1 | 22 kW |
| 64 | CP004 | 1 | 22 kW |
| 65 | CP005 | 1 | 22 kW |
| 66 | CP006 | 1 | 11 kW |
| 67 | CP007 | 1 | 11 kW |
| 68 | CP008 | 1 | 11 kW |
| 69 | CP009 | 1 | 11 kW |
| 70 | CP010 | 1 | 11 kW |
## OpenADR Protocol
## Grafana Dashboard (17 panels)
### VTN (Virtual Top Node)
- **Container**: openadr-vtn
- **Port**: 8080 (internal)
- **Profile**: OpenADR 2.0b
- **Events**: ENERGY_PRICE (24h), LOAD_CONTROL (4h)
- **Schedule**: Every 10 seconds poll
### VEN (Virtual End Node)
- **Container**: openadr-ven
- **VTN URL**: http://192.168.240.10:8080/OpenADR2/Simple/2.0b
- **Registration ID**: reg-Cariflex-VEN-001
- **Event Handler**: on_event -> FM API (sensors 84, 86)
### Event Flow
1. VTN creates events (price, load control)
2. VEN polls VTN every 10s
3. VEN receives oadrDistributeEvent
4. VEN sends event responses (optIn)
5. VEN posts data to FM API
6. Grafana displays real-time data
## S2 Protocol Integration
### S2 Service
- **Location**: openadr-ven container (background process)
- **Protocol**: S2 (FRBC - Flexibility Resource Bank Control)
- **Mapping**: S2 resources -> FM sensors (51-65)
- **Functions**:
- process_load_control(): Converts OpenADR load control to S2 FRBC instructions
- process_demand_response(): Converts OpenADR demand response to S2 FRBC instructions
### S2 Message Types
- FRBCInstruction: Power setpoint commands
- FRBCSystemDescription: Resource capabilities
- FRBCActuatorStatus: Resource status feedback
- FRBCStorageStatus: Battery/EV storage status
| # | Panel | Data Source |
|---|-------|-------------|
| 1 | Production PV (kW) | FM sensors 41-50 |
| 2 | Consommation VE (kW) | FM sensors 61-70 |
| 3 | SOC Batteries (kWh) | FM sensors 51-60 |
| 4 | SOC V2G (kWh) | FM sensors 71-80 |
| 5 | Flexibilite Totale (kWh) | Calculated |
| 6 | Impact Reseau (kW) | Calculated |
| 7 | Irradiance & Temperature | FM sensors 81-82 |
| 8 | Prix DSO (EUR/MWh) | FM sensor 85 |
| 9 | DSR Price Signal (EUR/MWh) | FM sensor 84 (OpenADR) |
| 10 | Load Control Signal (%) | FM sensor 86 (OpenADR) |
| 11 | Impact Flexibilite Reseau (kW) | Calculated |
| 12 | Battery Power (kW) | FM sensors 51-60 (S2) |
| 13 | EV Power (kW) | FM sensors 61-70 (S2) |
| 14 | Flexibility Impact (kW) | Calculated (S2) |
| 15 | EV Charging Power (kW) | FM sensors 61-70 (CitrineOS) |
| 16 | Active Charging Sessions | Calculated |
| 17 | EV Flexibility Potential (kWh) | Calculated |
## Backup Strategy
### Automated Backups
- **Script**: `/home/eric/cariflex/scripts/full_backup.sh`
- **Location**: `/home/eric/backups/cariflex/YYYYMMDD_HHMMSS.tar.gz`
- **Contents**:
- PostgreSQL dump (flexmeasures DB)
- Redis dump (RDB file)
- FM config and templates
- OpenADR scripts and Dockerfiles
- Grafana dashboards (JSON)
- Docker container states
- Network configuration
- **Contents**: PostgreSQL, Redis, FM config, OpenADR scripts, Grafana dashboards, container states, network config
### Backup Retention
- Daily backups kept for 30 days
- Weekly backups kept for 12 weeks
- Monthly backups kept indefinitely
## Deployment
## Deployment Environments
### Docker Compose Files
| File | Purpose |
|------|---------|
| `/home/eric/flexmeasures/docker-compose.yml` | FM core stack |
| `/home/eric/flexmeasures/docker-compose.openadr.yml` | OpenADR VTN/VEN |
| `/home/eric/cariflex/config/docker-compose-citrineos.yml` | CitrineOS stack |
### Development
- **Host**: Local VM or developer machine
- **Docker Compose**: Single node
- **Data**: Simulated
- **URL**: localhost
### Deployment Commands
```bash
# FM stack
cd /home/eric/flexmeasures && docker compose up -d
### Test
- **Host**: Test server
- **Docker Compose**: Single node
- **Data**: Anonymized production data
- **URL**: test.cariflex.digitribe.fr
# OpenADR stack
cd /home/eric/flexmeasures && docker compose -f docker-compose.openadr.yml up -d
### Production
- **Host**: Production server (this deployment)
- **Docker Compose**: Single node (scalable to Swarm/K8s)
- **Data**: Real-time production data
- **URL**: cariflex.digitribe.fr
# CitrineOS stack
cd /home/eric/cariflex/config && docker compose -f docker-compose-citrineos.yml up -d
## Security
### Credentials
| Service | Username | Password | Location |
|---------|----------|----------|----------|
| FlexMeasures | admin@digitribe.fr | Digitribe972 | FM DB |
| PostgreSQL | flexmeasures | Digitribe972 | PostgreSQL |
| Redis | - | (disabled) | Redis |
| Grafana | admin | admin | Grafana DB |
### SSL/TLS
- All public services use Let's Encrypt certificates
- Traefik handles SSL termination
- Internal services use HTTP (isolated network)
## Monitoring
### Health Checks
- FlexMeasures: https://cariflex.digitribe.fr/health
- Grafana: https://grafana.digitrobe.fr/api/health
- OpenADR VTN: http://192.168.240.10:8080/OpenADR2/Simple/2.0b
### Logs
- Docker: `docker logs <container>`
- FM: `/app/flexmeasures/logs/`
- Grafana: `/var/log/grafana/`
# Verify all
docker ps --format "table {{.Names}}\t{{.Status}}" | grep -E "flexmeasures|openadr|citrine|grafana"
```
## Troubleshooting
### Common Issues
1. **Redis ACL error**: `docker exec flexmeasures-redis redis-cli CONFIG SET requirepass ""`
2. **OpenADR connection refused**: Check UFW rules for 192.168.240.0/24
3. **Grafana no data**: Check datasource UID matches PostgreSQL DS
4. **FM worker not running**: `docker restart flexmeasures-worker`
### OpenADR Issues
- **VEN not connecting**: Check UFW rules for 192.168.240.0/24
- **No price data**: Check VTN logs: `docker logs openadr-vtn`
### Useful Commands
```bash
# View all containers
docker ps -a | grep -E "flexmeasures|openadr|grafana"
### CitrineOS Issues
- **RabbitMQ connection**: Check network aliases on config_cariflex-internal
- **OCPP communication**: Verify charge point network connectivity
# Check OpenADR communication
docker logs openadr-ven | grep -i "price\|load\|event"
# Check FM API
curl -s https://cariflex.digitribe.fr/api/v3_0/sensors
# Database query
docker exec flexmeasures-db psql -U flexmeasures -d flexmeasures -c "SELECT * FROM sensor WHERE id BETWEEN 80 AND 90;"
```
### FM Scheduling Issues
- **No schedules created**: Check price sensor data (84)
- **Worker not running**: `docker restart flexmeasures-worker`
## References
- FlexMeasures: https://flexmeasures.readthedocs.io/
- OpenLEADR: https://openleadr.org/
- CitrineOS: https://citrineos.github.io/
- S2 Protocol: https://s2-standard.github.io/
- Grafana: https://grafana.com/docs/
- OCPP 2.0.1: https://www.openchargealliance.org/protocols/ocpp-201/