# ENTSO-E forecasts & data

Importing data which can be relevant for energy flexibility services via ENTSO-E's API into FlexMeasures.

We start with data about the upcoming day.

- Generation forecasts for the upcoming day
- Based on these, CO2 content for the upcoming day
- Day-ahead prices


## Usage

Importing tomorrow's prices:

    flexmeasures entsoe import-day-ahead-prices

Importing tomorrow's generation (incl. CO2 estimated content):

    flexmeasures entsoe import-day-ahead-generation

Use ``--help`` to learn more usage details.


### October 1st 2025 go-live for ENTSO-E moving to 15-minute day-ahead prices

ENTSO-E is moving from 1-hour day-ahead prices 15-minute day-ahead prices on October 1st 2025.
To prepare for this transition, you have two choices:

1. resample your existing price sensor in FlexMeasures from 1 hour to 15 minutes, or
2. get a new sensor for the 15-minute data.

If you do this *after* the go-live moment, the `flexmeasures-entsoe` package just keeps resampling the 15-minute ENTSO-E data to hourly data.

#### 1. Resampling

**The upside** of resampling your existing price data is that the sensor ID of your price sensor in FlexMeasures will remain the same.
Depending on your system setup, `Forecaster`/`Reporter`/`Scheduler` configurations (such as an asset's `flex-context`) may depend on it, and your users may expect the 15-minute data to live under the same sensor.

**The downside** is that it quadruples your data for that sensor, due to the fact that FlexMeasures only supports a fixed resolution for any given sensor. Although there should be no noticeable hit in performance, it obviously leads to redundant data in the price history before October 1st 2025.  

**To resample** your historical data, use:

```bash
flexmeasures edit resample-data --sensor <ID of your day-ahead price sensor> --event-resolution 15
```

The `flexmeasures-entsoe` package already automatically resamples the ENTSO-E data to the resolution of your sensor.

If you use a `Reporter` to derive retail prices or to compute energy costs, there is no need to update its configuration; just resample these sensors too, using the previous command (replacing the sensor ID as needed).
Alternatively, if you want to keep these sensors in their original resolution, and find that your reporters fail with an `AssertionError` about mismatched resolutions, you may need to add the `--resolution PT1H` option when using the `flexmeasures add report` command.

#### 2. Getting a new sensor

**The upside** is that this doesn't quadruple your historic data (see *the downside* of resampling, above).

**The downside** is that you may need to revise `Forecaster`/`Reporter`/`Scheduler` configurations (such as an asset's `flex-context`) and notify users (see *the upside* of resampling, above).

**To get a new sensor**, rename your existing *Day-ahead prices* sensor in the FlexMeasures UI.

The `flexmeasures-entsoe` package will then automatically create a new 15-minute price sensor the next time `flexmeasures entsoe import-day-ahead-prices` is run, assigning it a new sensor ID.

If you have any price or costs sensors using a `Reporter` to derive values from the day-ahead wholesale prices, update the sensor ID in the configuration of each `Reporter`.
Finally, either resample each derived sensor using:

```bash
flexmeasures edit resample-data --sensor <ID of your derivative sensor> --event-resolution 15
```

or, if you want to keep these sensors in their original resolution, and find that your reporters fail with an `AssertionError` about mismatched resolutions, you may need to add the `--resolution PT1H` option when using the `flexmeasures add report` command.

## Installation

First of all, this is a FlexMeasures plugin. Consult the FlexMeasures documentation for setup.

1. Add the plugin to [the `FLEXMEASURES_PLUGINS` setting](https://flexmeasures.readthedocs.io/stable/configuration.html#flexmeasures-plugins). Either use `/path/to/flexmeasures-entsoe/flexmeasures_entsoe` or `flexmeasures_entsoe` if you installed this as a package locally (see below).

2. Add `ENTSOE_AUTH_TOKEN` to your FlexMeasures config (e.g. ~/.flexmeasures.cfg).
You can generate this token after you made an account at ENTSO-E, read more [here](https://transparencyplatform.zendesk.com/hc/en-us/articles/12845911031188-How-to-get-security-token). 

   Optionally, override other settings (defaults shown here):

       ENTSOE_COUNTRY_CODE = "NL"
       ENTSOE_COUNTRY_TIMEZONE = "Europe/Amsterdam"
       ENTSOE_DERIVED_DATA_SOURCE = "FlexMeasures ENTSO-E"

   The `ENTSOE_DERIVED_DATA_SOURCE` option is used to name the source of data that this plugin derives from ENTSO-E data, like a CO₂ signal.
   Original ENTSO-E data is reported as being sourced by `"ENTSO-E"`.

3. To install this plugin locally as a package, try `pip install .`.


## Testing

ENTSO-E provides a test server (iop) for development. It's good practice not to overwhelm their production server.

Set ``ENTSOE_USE_TEST_SERVER=True`` to enable this.

In that case, this plugin will look for the auth token in the config setting ``ENTSOE_AUTH_TOKEN_TEST_SERVER``.

Note, however, that ENTSO-E usually does not seem to make the latest data available there. Asking for the next day can often get an empty response.


## Supported FlexMeasures versions

This plugin targets two distinct FlexMeasures capability tiers:

| FlexMeasures version | Behavior |
|---|---|
| `< 0.32` | Uses the legacy `get_data_source` factory; no account is linked to the ENTSO-E source. |
| `>= 0.32` | Uses the account-linked source API (`get_or_create_source` with an `Account`). |

This package supports Python 3.10 through 3.12, following the Python support policy of the currently supported FlexMeasures releases.

The oldest supported FlexMeasures release line is `0.31.*`.
CI is run against `0.31.*` (minimum supported legacy release), `0.32.*` (first account-linked release), and the latest released FlexMeasures version across all supported Python versions.
When a new FlexMeasures release introduces breaking changes the matrix should be updated accordingly.


## Development

To keep our code quality high, we use pre-commit:

    pip install pre-commit black flake8 mypy
    pre-commit install

or:
    
    make install-for-dev

Try it:

    pre-commit run --all-files --show-diff-on-failure
