Add FlexMeasures plugins, USEF protocol, and Cariflex simulator

- flexmeasures-entsoe: ENTSO-E data plugin
- flexmeasures-weather: Weather data plugin
- USEF Flex Trading Protocol PDF (2.4MB)
- Cariflex simulator (publishes to Redis)
- Dashboard Grafana updated with correct InfluxDB queries
- All tools extracted in /tools/

tools/flexmeasures-entsoe/README.md

@@ -0,0 +1,133 @@
+# ENTSO-E forecasts & data
+
+Importing data which can be relevant for energy flexibility services via ENTSO-E's API into FlexMeasures.
+
+We start with data about the upcoming day.
+
+- Generation forecasts for the upcoming day
+- Based on these, CO2 content for the upcoming day
+- Day-ahead prices
+
+
+## Usage
+
+Importing tomorrow's prices:
+
+    flexmeasures entsoe import-day-ahead-prices
+
+Importing tomorrow's generation (incl. CO2 estimated content):
+
+    flexmeasures entsoe import-day-ahead-generation
+
+Use ``--help`` to learn more usage details.
+
+
+### October 1st 2025 go-live for ENTSO-E moving to 15-minute day-ahead prices
+
+ENTSO-E is moving from 1-hour day-ahead prices 15-minute day-ahead prices on October 1st 2025.
+To prepare for this transition, you have two choices:
+
+1. resample your existing price sensor in FlexMeasures from 1 hour to 15 minutes, or
+2. get a new sensor for the 15-minute data.
+
+If you do this *after* the go-live moment, the `flexmeasures-entsoe` package just keeps resampling the 15-minute ENTSO-E data to hourly data.
+
+#### 1. Resampling
+
+**The upside** of resampling your existing price data is that the sensor ID of your price sensor in FlexMeasures will remain the same.
+Depending on your system setup, `Forecaster`/`Reporter`/`Scheduler` configurations (such as an asset's `flex-context`) may depend on it, and your users may expect the 15-minute data to live under the same sensor.
+
+**The downside** is that it quadruples your data for that sensor, due to the fact that FlexMeasures only supports a fixed resolution for any given sensor. Although there should be no noticeable hit in performance, it obviously leads to redundant data in the price history before October 1st 2025.  
+
+**To resample** your historical data, use:
+
+```bash
+flexmeasures edit resample-data --sensor <ID of your day-ahead price sensor> --event-resolution 15
+```
+
+The `flexmeasures-entsoe` package already automatically resamples the ENTSO-E data to the resolution of your sensor.
+
+If you use a `Reporter` to derive retail prices or to compute energy costs, there is no need to update its configuration; just resample these sensors too, using the previous command (replacing the sensor ID as needed).
+Alternatively, if you want to keep these sensors in their original resolution, and find that your reporters fail with an `AssertionError` about mismatched resolutions, you may need to add the `--resolution PT1H` option when using the `flexmeasures add report` command.
+
+#### 2. Getting a new sensor
+
+**The upside** is that this doesn't quadruple your historic data (see *the downside* of resampling, above).
+
+**The downside** is that you may need to revise `Forecaster`/`Reporter`/`Scheduler` configurations (such as an asset's `flex-context`) and notify users (see *the upside* of resampling, above).
+
+**To get a new sensor**, rename your existing *Day-ahead prices* sensor in the FlexMeasures UI.
+
+The `flexmeasures-entsoe` package will then automatically create a new 15-minute price sensor the next time `flexmeasures entsoe import-day-ahead-prices` is run, assigning it a new sensor ID.
+
+If you have any price or costs sensors using a `Reporter` to derive values from the day-ahead wholesale prices, update the sensor ID in the configuration of each `Reporter`.
+Finally, either resample each derived sensor using:
+
+```bash
+flexmeasures edit resample-data --sensor <ID of your derivative sensor> --event-resolution 15
+```
+
+or, if you want to keep these sensors in their original resolution, and find that your reporters fail with an `AssertionError` about mismatched resolutions, you may need to add the `--resolution PT1H` option when using the `flexmeasures add report` command.
+
+## Installation
+
+First of all, this is a FlexMeasures plugin. Consult the FlexMeasures documentation for setup.
+
+1. Add the plugin to [the `FLEXMEASURES_PLUGINS` setting](https://flexmeasures.readthedocs.io/stable/configuration.html#flexmeasures-plugins). Either use `/path/to/flexmeasures-entsoe/flexmeasures_entsoe` or `flexmeasures_entsoe` if you installed this as a package locally (see below).
+
+2. Add `ENTSOE_AUTH_TOKEN` to your FlexMeasures config (e.g. ~/.flexmeasures.cfg).
+You can generate this token after you made an account at ENTSO-E, read more [here](https://transparencyplatform.zendesk.com/hc/en-us/articles/12845911031188-How-to-get-security-token). 
+
+   Optionally, override other settings (defaults shown here):
+
+       ENTSOE_COUNTRY_CODE = "NL"
+       ENTSOE_COUNTRY_TIMEZONE = "Europe/Amsterdam"
+       ENTSOE_DERIVED_DATA_SOURCE = "FlexMeasures ENTSO-E"
+
+   The `ENTSOE_DERIVED_DATA_SOURCE` option is used to name the source of data that this plugin derives from ENTSO-E data, like a CO₂ signal.
+   Original ENTSO-E data is reported as being sourced by `"ENTSO-E"`.
+
+3. To install this plugin locally as a package, try `pip install .`.
+
+
+## Testing
+
+ENTSO-E provides a test server (iop) for development. It's good practice not to overwhelm their production server.
+
+Set ``ENTSOE_USE_TEST_SERVER=True`` to enable this.
+
+In that case, this plugin will look for the auth token in the config setting ``ENTSOE_AUTH_TOKEN_TEST_SERVER``.
+
+Note, however, that ENTSO-E usually does not seem to make the latest data available there. Asking for the next day can often get an empty response.
+
+
+## Supported FlexMeasures versions
+
+This plugin targets two distinct FlexMeasures capability tiers:
+
+| FlexMeasures version | Behavior |
+|---|---|
+| `< 0.32` | Uses the legacy `get_data_source` factory; no account is linked to the ENTSO-E source. |
+| `>= 0.32` | Uses the account-linked source API (`get_or_create_source` with an `Account`). |
+
+This package supports Python 3.10 through 3.12, following the Python support policy of the currently supported FlexMeasures releases.
+
+The oldest supported FlexMeasures release line is `0.31.*`.
+CI is run against `0.31.*` (minimum supported legacy release), `0.32.*` (first account-linked release), and the latest released FlexMeasures version across all supported Python versions.
+When a new FlexMeasures release introduces breaking changes the matrix should be updated accordingly.
+
+
+## Development
+
+To keep our code quality high, we use pre-commit:
+
+    pip install pre-commit black flake8 mypy
+    pre-commit install
+
+or:
+    
+    make install-for-dev
+
+Try it:
+
+    pre-commit run --all-files --show-diff-on-failure