cariflex/docs/architecture_diagrams.md

# Cariflex - Diagrammes d'Architecture

## 1. Architecture Générale

```mermaid
graph TB
    subgraph EXTERNAL["External Layer"]
        EPEX["EPEX SPOT<br/>Day-ahead Prices"]
        ENTSOE["ENTSO-E<br/>CO2/Prices"]
        WEATHER["Weather API<br/>Solcast/Météo"]
        DSO["DSO/TSO<br/>OpenADR/S2"]
    end

    subgraph INTEGRATION["Integration Layer"]
        FM["FlexMeasures EMS<br/>━━━━━━━━━━━━━━<br/>• Ingestion<br/>• Forecasting<br/>• Scheduling<br/>• Reporting"]
        
        CITRINEOS["CitrineOS<br/>CSMS<br/>━━━━━━━━━━━━━━<br/>• OCPP 2.0.1<br/>• Charge Points<br/>• Transactions"]
        
        EVEREST["EVerest<br/>Middleware<br/>━━━━━━━━━━━━━━<br/>• ISO 15118<br/>• Smart Charging<br/>• Power Mgmt"]
        
        OPENLEADR["OpenLEADR<br/>VEN Client<br/>━━━━━━━━━━━━━━<br/>• OpenADR 2.0b<br/>• DSO Signals<br/>• Flexibility"]
    end

    subgraph MARKET["Market Layer (R&D)"]
        GSY["Grid Singularity<br/>gsy-e<br/>━━━━━━━━━━━━━━<br/>• P2P Trading<br/>• Market Clearing<br/>• Price Discovery"]
        
        OPLEM["OPLEM<br/>━━━━━━━━━━━━━━<br/>• Local Market<br/>• Prosumer Opt."]
        
        HAMLET["HAMLET<br/>━━━━━━━━━━━━━━<br/>• Multi-agent<br/>• Simulation"]
    end

    subgraph DEVICES["Device Layer"]
        PV["10 PV Panels<br/>5kWc each"]
        BAT["10 Batteries<br/>100kWh each"]
        EV["10 EVs<br/>75kWh V2G"]
        CHG["10 EV Chargers<br/>22kW each"]
    end

    subgraph VIZ["Visualization"]
        GRAFANA["Grafana<br/>Dashboards"]
        METABASE["Metabase<br/>Analytics"]
        CARIFLEX_UI["Cariflex UI<br/>FM Frontend"]
    end

    %% External to Integration
    EPEX -->|"Prices"| FM
    ENTSOE -->|"CO2/Prices"| FM
    WEATHER -->|"Irradiance"| FM
    DSO -->|"Flex Requests"| OPENLEADR

    %% Integration to FM
    CITRINEOS -->|"OCPP/REST"| FM
    EVEREST -->|"OCPP"| CITRINEOS
    OPENLEADR -->|"REST API"| FM

    %% Market to FM
    GSY <--->|"gsy-e-sdk"| FM
    OPLEM <--->|"sensor data"| FM
    HAMLET <--->|"forecasts"| FM

    %% Devices
    PV -->|"Modbus"| FM
    BAT -->|"Modbus"| FM
    EV -->|"ISO 15118"| EVEREST
    CHG -->|"OCPP 2.0.1"| CITRINEOS

    %% FM to Viz
    FM -->|"PostgreSQL"| GRAFANA
    FM -->|"PostgreSQL"| METABASE
    FM -->|"REST API"| CARIFLEX_UI

    style FM fill:#2196F3,stroke:#1565C0,color:#fff
    style CITRINEOS fill:#4CAF50,stroke:#388E3C,color:#fff
    style EVEREST fill:#FF9800,stroke:#F57C00,color:#fff
    style OPENLEADR fill:#9C27B0,stroke:#7B1FA2,color:#fff
    style GSY fill:#00BCD4,stroke:#0097A7,color:#fff
```

## 2. Flux de Données - Ingestion

```mermaid
sequenceDiagram
    participant Device as Device (PV/BAT/EV)
    participant MQTT as MQTT Broker
    participant FM_API as FlexMeasures API
    participant Worker as FM Worker (RQ)
    participant DB as PostgreSQL
    participant Grafana as Grafana

    Device->>MQTT: Publish sensor data (30s)
    MQTT->>FM_API: REST POST /sensors/{id}/data
    FM_API->>Worker: Enqueue ingestion job
    Worker->>DB: Save beliefs (timed data)
    Grafana->>DB: Query data (timeseries)
```

## 3. Flux de Données - Forecasting

```mermaid
sequenceDiagram
    participant Cron as Cron Job / Scheduler
    participant FM_CLI as FM CLI
    participant Pipeline as TrainPredictPipeline
    participant DB as PostgreSQL
    participant Grafana as Grafana

    Cron->>FM_CLI: Trigger forecast (daily)
    FM_CLI->>Pipeline: Train model (30 days history)
    Pipeline->>Pipeline: Predict (24h horizon)
    Pipeline->>DB: Save 82 forecast beliefs
    Grafana->>DB: Display forecast vs actual
```

## 4. Flux de Données - Scheduling

```mermaid
sequenceDiagram
    participant DSO as DSO (OpenADR VTN)
    participant OpenLEADR as OpenLEADR (VEN)
    participant FM_API as FlexMeasures API
    participant Scheduler as StorageScheduler
    participant DB as PostgreSQL
    participant CitrineOS as CitrineOS (CSMS)

    DSO->>OpenLEADR: Flexibility request
    OpenLEADR->>FM_API: POST /sensors/{id}/schedules/trigger
    FM_API->>Scheduler: Create schedule job
    Scheduler->>Scheduler: Optimize (soc_min, soc_max, power)
    Scheduler->>DB: Save schedule beliefs
    Scheduler->>CitrineOS: Send charging profile
    CitrineOS->>CitrineOS: Apply to charge point
```

## 5. Intégration Grid Singularity

```mermaid
sequenceDiagram
    participant GSY as Grid Singularity Exchange
    participant SDK as gsy-e-sdk
    participant FM_API as FlexMeasures API
    participant Scheduler as FM Scheduler
    participant DB as PostgreSQL

    GSY->>SDK: Market prices / orders
    SDK->>FM_API: POST /sensors/{id}/data (price signals)
    FM_API->>DB: Save price beliefs
    Scheduler->>DB: Read prices + forecasts
    Scheduler->>Scheduler: Optimize schedule
    Scheduler->>DB: Save optimal schedule
    FM_API->>SDK: Report flexibility offered
    SDK->>GSY: Submit bids/offers
```

## 6. Intégration OPLEM

```mermaid
sequenceDiagram
    participant OPLEM as OPLEM Optimizer
    participant FM_DB as FM PostgreSQL
    participant Scheduler as FM Scheduler
    participant Devices as Devices

    OPLEM->>FM_DB: Read sensor data (SOC, power)
    OPLEM->>OPLEM: Optimize local market
    OPLEM->>Scheduler: Send optimal setpoints
    Scheduler->>Devices: Apply commands
    Devices->>FM_DB: Report actual values
```

## 7. Intégration HAMLET

```mermaid
sequenceDiagram
    participant HAMLET as HAMLET Simulator
    participant FM_API as FlexMeasures API
    participant Scheduler as FM Scheduler

    HAMLET->>FM_API: Read forecasts
    HAMLET->>HAMLET: Run multi-agent simulation
    HAMLET->>FM_API: Validate strategies
    FM_API->>Scheduler: Apply validated schedule
```

## 8. Architecture Réseau

```mermaid
graph LR
    subgraph Docker["Docker Network: cariflex-internal"]
        FM_SRV["flexmeasures-server:5000"]
        FM_WRK["flexmeasures-worker:5000"]
        FM_DB["flexmeasures-db:5432"]
        FM_REDIS["flexmeasures-redis:6379"]
        
        CITRINEOS["citrineos-server:8080"]
        EVEREST["everest:8080"]
        GSY["gsy-e:8080"]
    end

    subgraph Traefik["Traefik (websecure)"]
        ROUTER["Router: cariflex.digitribe.fr"]
    end

    ROUTER -->|"HTTPS"| FM_SRV
    FM_SRV -->|"REST"| FM_WRK
    FM_WRK -->|"RQ"| FM_REDIS
    FM_WRK -->|"SQL"| FM_DB
    FM_SRV -->|"SQL"| FM_DB
    
    CITRINEOS -->|"REST"| FM_SRV
    EVEREST -->|"OCPP"| CITRINEOS
    GSY -->|"REST"| FM_SRV
```

## 9. TODO - Intégrations à Développer

| Intégration | Priorité | Complexité | Statut |
|-------------|----------|------------|--------|
| CitrineOS ↔ FM | Haute | Moyenne | ⏳ À faire |
| EVerest ↔ CitrineOS | Haute | Moyenne | ⏳ À faire |
| OpenLEADR ↔ FM | Haute | Faible | ⏳ À faire |
| Grid Singularity ↔ FM | Moyenne | Élevée | ⏳ À faire |
| OPLEM ↔ FM | Moyenne | Moyenne | ⏳ À faire |
| HAMLET ↔ FM | Basse | Élevée | ⏳ À faire |
| MQTT Brokers Cariflex | Haute | Moyenne | ⏳ À faire |