Infrastructure docs, IaC scripts, skills energy (VOLTTRON, EnergyPlus, OpenStudio, BuildingGym, OSeMOSYS)

2026-06-10 12:09:00 -04:00
parent a0577ed17c
commit 84667c7126
4 changed files with 430 additions and 0 deletions
--- a/docs/INFRASTRUCTURE.md
+++ b/docs/INFRASTRUCTURE.md
@@ -0,0 +1,230 @@
+# Cariflex EMS - Infrastructure Documentation
+
+## Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│                              CARIFLEX EMS                                    │
+│                    Energy Management System for Martinique                   │
+├─────────────────────────────────────────────────────────────────────────────┤
+│                                                                              │
+│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                   │
+│  │   OpenADR    │    │   FlexMeasures│    │   Grafana    │                   │
+│  │   VTN/VEN    │───▶│   Server      │───▶│   Dashboard  │                   │
+│  │   (S2+OpenADR)│    │   (EMS Core)  │    │   (14 panels)│                   │
+│  └──────────────┘    └──────────────┘    └──────────────┘                   │
+│         │                    │                    │                          │
+│         ▼                    ▼                    ▼                          │
+│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐                   │
+│  │ PostgreSQL   │    │    Redis      │    │   Traefik    │                   │
+│  │ (time-series)│    │  (job queue)  │    │  (reverse    │                   │
+│  │              │    │              │    │   proxy)      │                   │
+│  └──────────────┘    └──────────────┘    └──────────────┘                   │
+│                                                                              │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
+## Network Architecture
+
+### Networks
+| Network | Subnet | Purpose | External |
+|---------|--------|---------|----------|
+| `traefik-public` | 172.29.0.0/16 | Public services (FM, Grafana) | Yes |
+| `openadr-internal` | 192.168.240.0/24 | OpenADR VTN/VEN communication | No |
+
+### DNS/Hosts
+| Service | Domain | Container | Port |
+|---------|--------|-----------|------|
+| FlexMeasures | cariflex.digitribe.fr | flexmeasures-server | 5000 |
+| Grafana | grafana.digitribe.fr | smart-city-grafana | 3000 |
+| OpenADR VTN | - | openadr-vtn | 8080 |
+| OpenADR VEN | - | openadr-ven | - |
+
+### Firewall (UFW)
+- Public ports: 80 (HTTP), 443 (HTTPS), 51820 (WireGuard)
+- Internal networks: 172.16.0.0/12, 10.0.0.0/8, 192.168.0.0/16, 192.168.240.0/24
+
+## Docker Containers
+
+### Cariflex EMS Stack
+| Container | Image | Network | IP | Port | Status |
+|-----------|-------|---------|----|---- |--------|
+| flexmeasures-server | lfenergy/flexmeasures:latest | traefik-public | 172.29.0.x | 5000 | ✅ UP |
+| flexmeasures-worker | lfenergy/flexmeasures:latest | traefik-public | 172.29.0.x | - | ✅ UP |
+| flexmeasures-db | postgres:17 | traefik-public | 172.29.0.x | 5432 | ✅ UP |
+| flexmeasures-redis | redis:7-alpine | traefik-public | 172.29.0.x | 6379 | ✅ UP |
+| openadr-vtn | flexmeasures-openadr-vtn | openadr-internal | 192.168.240.10 | 8080 | ✅ UP |
+| openadr-ven | flexmeasures-openadr-ven | openadr-internal | 192.168.240.11 | - | ✅ UP |
+| smart-city-grafana | grafana/grafana | traefik-public | - | 3000 | ✅ UP |
+
+## IP Address Allocation (phpIPAM)
+
+### Cariflex Subnet: 192.168.240.0/24
+| IP | Service | Container | MAC |
+|----|---------|-----------|-----|
+| 192.168.240.1 | Gateway | - | - |
+| 192.168.240.10 | OpenADR VTN | openadr-vtn | - |
+| 192.168.240.11 | OpenADR VEN | openadr-ven | - |
+| 192.168.240.128-254 | Reserved | - | - |
+
+### Public Subnet (traefik-public): 172.29.0.0/16
+| IP | Service | Domain |
+|----|---------|--------|
+| 172.29.0.x | FlexMeasures | cariflex.digitribe.fr |
+| 172.29.0.x | Grafana | grafana.digitribe.fr |
+
+## Sensor Mapping
+
+### FlexMeasures Sensors
+| ID | Name | Unit | Type | Source |
+|----|------|------|------|--------|
+| 41-50 | pv_01..10_power | kW | PV Production | Simulator |
+| 51-60 | bat_01..10_power | kWh | Battery SOC | Simulator |
+| 61-70 | ev_01..10_power | kWh | EV Charge | Simulator |
+| 81 | irradiance | W/m² | Weather | Open-Meteo |
+| 82 | temperature | °C | Weather | Open-Meteo |
+| 83 | wind_speed | m/s | Weather | Open-Meteo |
+| 84 | consumption_price | EUR/MWh | Price | OpenADR VTN |
+| 85 | production_price | EUR/MWh | Price | Simulated |
+| 86 | load_control_signal | % | DSR | OpenADR VTN |
+| 87 | demand_response_signal | dimensionless | DSR | OpenADR VTN |
+
+### S2 Resource Mapping
+| S2 Resource | FM Sensor | Asset | Min Power | Max Power |
+|-------------|-----------|-------|-----------|-----------|
+| battery_01-05 | 51-55 | Bat_01-05 | -50 kW | +50 kW |
+| ev_01-05 | 61-65 | EV_01-05 | -22 kW | +22 kW |
+
+## OpenADR Protocol
+
+### VTN (Virtual Top Node)
+- **Container**: openadr-vtn
+- **Port**: 8080 (internal)
+- **Profile**: OpenADR 2.0b
+- **Events**: ENERGY_PRICE (24h), LOAD_CONTROL (4h)
+- **Schedule**: Every 10 seconds poll
+
+### VEN (Virtual End Node)
+- **Container**: openadr-ven
+- **VTN URL**: http://192.168.240.10:8080/OpenADR2/Simple/2.0b
+- **Registration ID**: reg-Cariflex-VEN-001
+- **Event Handler**: on_event -> FM API (sensors 84, 86)
+
+### Event Flow
+1. VTN creates events (price, load control)
+2. VEN polls VTN every 10s
+3. VEN receives oadrDistributeEvent
+4. VEN sends event responses (optIn)
+5. VEN posts data to FM API
+6. Grafana displays real-time data
+
+## S2 Protocol Integration
+
+### S2 Service
+- **Location**: openadr-ven container (background process)
+- **Protocol**: S2 (FRBC - Flexibility Resource Bank Control)
+- **Mapping**: S2 resources -> FM sensors (51-65)
+- **Functions**:
+  - process_load_control(): Converts OpenADR load control to S2 FRBC instructions
+  - process_demand_response(): Converts OpenADR demand response to S2 FRBC instructions
+
+### S2 Message Types
+- FRBCInstruction: Power setpoint commands
+- FRBCSystemDescription: Resource capabilities
+- FRBCActuatorStatus: Resource status feedback
+- FRBCStorageStatus: Battery/EV storage status
+
+## Backup Strategy
+
+### Automated Backups
+- **Script**: `/home/eric/cariflex/scripts/full_backup.sh`
+- **Location**: `/home/eric/backups/cariflex/YYYYMMDD_HHMMSS.tar.gz`
+- **Contents**:
+  - PostgreSQL dump (flexmeasures DB)
+  - Redis dump (RDB file)
+  - FM config and templates
+  - OpenADR scripts and Dockerfiles
+  - Grafana dashboards (JSON)
+  - Docker container states
+  - Network configuration
+
+### Backup Retention
+- Daily backups kept for 30 days
+- Weekly backups kept for 12 weeks
+- Monthly backups kept indefinitely
+
+## Deployment Environments
+
+### Development
+- **Host**: Local VM or developer machine
+- **Docker Compose**: Single node
+- **Data**: Simulated
+- **URL**: localhost
+
+### Test
+- **Host**: Test server
+- **Docker Compose**: Single node
+- **Data**: Anonymized production data
+- **URL**: test.cariflex.digitribe.fr
+
+### Production
+- **Host**: Production server (this deployment)
+- **Docker Compose**: Single node (scalable to Swarm/K8s)
+- **Data**: Real-time production data
+- **URL**: cariflex.digitribe.fr
+
+## Security
+
+### Credentials
+| Service | Username | Password | Location |
+|---------|----------|----------|----------|
+| FlexMeasures | admin@digitribe.fr | Digitribe972 | FM DB |
+| PostgreSQL | flexmeasures | Digitribe972 | PostgreSQL |
+| Redis | - | (disabled) | Redis |
+| Grafana | admin | admin | Grafana DB |
+
+### SSL/TLS
+- All public services use Let's Encrypt certificates
+- Traefik handles SSL termination
+- Internal services use HTTP (isolated network)
+
+## Monitoring
+
+### Health Checks
+- FlexMeasures: https://cariflex.digitribe.fr/health
+- Grafana: https://grafana.digitrobe.fr/api/health
+- OpenADR VTN: http://192.168.240.10:8080/OpenADR2/Simple/2.0b
+
+### Logs
+- Docker: `docker logs <container>`
+- FM: `/app/flexmeasures/logs/`
+- Grafana: `/var/log/grafana/`
+
+## Troubleshooting
+
+### Common Issues
+1. **Redis ACL error**: `docker exec flexmeasures-redis redis-cli CONFIG SET requirepass ""`
+2. **OpenADR connection refused**: Check UFW rules for 192.168.240.0/24
+3. **Grafana no data**: Check datasource UID matches PostgreSQL DS
+4. **FM worker not running**: `docker restart flexmeasures-worker`
+
+### Useful Commands
+```bash
+# View all containers
+docker ps -a | grep -E "flexmeasures|openadr|grafana"
+
+# Check OpenADR communication
+docker logs openadr-ven | grep -i "price\|load\|event"
+
+# Check FM API
+curl -s https://cariflex.digitribe.fr/api/v3_0/sensors
+
+# Database query
+docker exec flexmeasures-db psql -U flexmeasures -d flexmeasures -c "SELECT * FROM sensor WHERE id BETWEEN 80 AND 90;"
+```
+
+## References
+- FlexMeasures: https://flexmeasures.readthedocs.io/
+- OpenLEADR: https://openleadr.org/
+- S2 Protocol: https://s2-standard.github.io/
+- Grafana: https://grafana.com/docs/