Add FlexMeasures plugins, USEF protocol, and Cariflex simulator

- flexmeasures-entsoe: ENTSO-E data plugin
- flexmeasures-weather: Weather data plugin
- USEF Flex Trading Protocol PDF (2.4MB)
- Cariflex simulator (publishes to Redis)
- Dashboard Grafana updated with correct InfluxDB queries
- All tools extracted in /tools/

tools/flexmeasures-weather/requirements/Readme.md

@@ -0,0 +1,21 @@
+# Requirements
+
+All requirements for flexmeasures-flexmeasures-weather are specified in this directory.
+We separate by use case:
+
+- app: All requirements for running the plugin
+- test: Additional requirements used for running automated tests 
+- dev: Additional requirements used for developers (this includes testing)
+
+Also note the following distinction:
+
+
+## .in files
+
+Here, we describe the requirements. We give the name of a requirement or even a range (e.g. `>=1.0.`).
+
+## .txt files
+
+These files are not to be edited by hand. They are created by `pip-compile` (or `make freeze-deps`).
+
+Each requirement is pinned to a specific version in these files. The great benefit is reproducibility across environments (local dev as well as staging or production).

tools/flexmeasures-weather/requirements/app.in

@@ -0,0 +1,6 @@
+flexmeasures
+pvlib
+# the following three are optional in pvlib, but we use them
+netCDF4
+siphon
+tables

tools/flexmeasures-weather/requirements/app.txt

@@ -0,0 +1,444 @@
+#
+# This file is autogenerated by pip-compile with Python 3.10
+# by the following command:
+#
+#    pip-compile --output-file=requirements/app.txt requirements/app.in
+#
+alembic==1.16.1
+    # via flask-migrate
+altair==5.5.0
+    # via flexmeasures
+argon2-cffi==23.1.0
+    # via flexmeasures
+argon2-cffi-bindings==21.2.0
+    # via argon2-cffi
+arrow==1.3.0
+    # via rq-dashboard
+async-timeout==5.0.1
+    # via redis
+attrs==25.3.0
+    # via
+    #   jsonschema
+    #   referencing
+babel==2.17.0
+    # via py-moneyed
+bcrypt==4.0.1
+    # via flexmeasures
+beautifulsoup4==4.13.4
+    # via siphon
+blinker==1.9.0
+    # via
+    #   flask
+    #   flask-mail
+    #   flask-principal
+    #   sentry-sdk
+blosc2==3.3.4
+    # via tables
+certifi==2025.4.26
+    # via
+    #   netcdf4
+    #   requests
+    #   sentry-sdk
+cffi==1.17.1
+    # via argon2-cffi-bindings
+cftime==1.6.4.post1
+    # via netcdf4
+charset-normalizer==3.4.2
+    # via requests
+click==8.2.1
+    # via
+    #   click-default-group
+    #   flask
+    #   flexmeasures
+    #   rq
+click-default-group==1.2.4
+    # via flexmeasures
+contourpy==1.3.2
+    # via matplotlib
+convertdate==2.4.0
+    # via workalendar
+cycler==0.12.1
+    # via matplotlib
+dill==0.4.0
+    # via openturns
+dnspython==2.7.0
+    # via email-validator
+email-validator==2.2.0
+    # via
+    #   flask-security-too
+    #   flexmeasures
+filelock==3.18.0
+    # via tldextract
+flask==3.1.1
+    # via
+    #   flask-classful
+    #   flask-cors
+    #   flask-json
+    #   flask-login
+    #   flask-mail
+    #   flask-marshmallow
+    #   flask-migrate
+    #   flask-principal
+    #   flask-security-too
+    #   flask-sqlalchemy
+    #   flask-sslify
+    #   flask-wtf
+    #   flexmeasures
+    #   rq-dashboard
+    #   sentry-sdk
+flask-classful==0.16.0
+    # via flexmeasures
+flask-cors==6.0.0
+    # via flexmeasures
+flask-json==0.4.0
+    # via flexmeasures
+flask-login==0.6.3
+    # via
+    #   flask-security-too
+    #   flexmeasures
+flask-mail==0.10.0
+    # via flexmeasures
+flask-marshmallow==1.3.0
+    # via flexmeasures
+flask-migrate==4.1.0
+    # via flexmeasures
+flask-principal==0.4.0
+    # via flask-security-too
+flask-security-too==5.6.2
+    # via flexmeasures
+flask-sqlalchemy==3.1.1
+    # via
+    #   flask-migrate
+    #   flexmeasures
+flask-sslify==0.1.5
+    # via flexmeasures
+flask-wtf==1.2.2
+    # via
+    #   flask-security-too
+    #   flexmeasures
+flexcache==0.3
+    # via pint
+flexmeasures==0.25.0
+    # via -r requirements/app.in
+flexparser==0.4
+    # via pint
+fonttools==4.58.1
+    # via matplotlib
+greenlet==3.2.2
+    # via sqlalchemy
+h5py==3.13.0
+    # via pvlib
+humanize==4.12.3
+    # via flexmeasures
+idna==3.10
+    # via
+    #   email-validator
+    #   requests
+    #   tldextract
+importlib-metadata==8.7.0
+    # via
+    #   flexmeasures
+    #   timely-beliefs
+importlib-resources==6.5.2
+    # via flask-security-too
+inflect==6.0.2
+    # via flexmeasures
+inflection==0.5.1
+    # via flexmeasures
+iso8601==2.1.0
+    # via flexmeasures
+isodate==0.7.2
+    # via
+    #   flexmeasures
+    #   timely-beliefs
+itsdangerous==2.2.0
+    # via
+    #   flask
+    #   flask-wtf
+jinja2==3.1.6
+    # via
+    #   altair
+    #   flask
+joblib==1.4.2
+    # via
+    #   scikit-learn
+    #   sktime
+jsonschema==4.24.0
+    # via altair
+jsonschema-specifications==2025.4.1
+    # via jsonschema
+kiwisolver==1.4.8
+    # via matplotlib
+lunardate==0.2.2
+    # via workalendar
+mako==1.3.10
+    # via alembic
+markupsafe==3.0.2
+    # via
+    #   flask
+    #   flask-security-too
+    #   jinja2
+    #   mako
+    #   sentry-sdk
+    #   werkzeug
+    #   wtforms
+marshmallow==3.26.1
+    # via
+    #   -r requirements/app.in
+    #   flask-marshmallow
+    #   flexmeasures
+    #   marshmallow-polyfield
+    #   marshmallow-sqlalchemy
+    #   webargs
+marshmallow-polyfield==5.11
+    # via flexmeasures
+marshmallow-sqlalchemy==1.4.2
+    # via flexmeasures
+matplotlib==3.10.3
+    # via timetomodel
+msgpack==1.1.0
+    # via blosc2
+narwhals==1.41.0
+    # via altair
+ndindex==1.10.0
+    # via blosc2
+netcdf4==1.7.2
+    # via -r requirements/app.in
+numexpr==2.10.2
+    # via
+    #   blosc2
+    #   tables
+numpy==1.26.4
+    # via
+    #   blosc2
+    #   cftime
+    #   contourpy
+    #   h5py
+    #   matplotlib
+    #   netcdf4
+    #   numexpr
+    #   pandas
+    #   patsy
+    #   properscoring
+    #   pvlib
+    #   scikit-learn
+    #   scipy
+    #   siphon
+    #   sktime
+    #   statsmodels
+    #   tables
+    #   timely-beliefs
+    #   timetomodel
+    #   uniplot
+openturns==1.24
+    # via timely-beliefs
+packaging==25.0
+    # via
+    #   altair
+    #   marshmallow
+    #   matplotlib
+    #   sktime
+    #   statsmodels
+    #   tables
+    #   webargs
+pandas==2.2.1
+    # via
+    #   flexmeasures
+    #   pvlib
+    #   siphon
+    #   sktime
+    #   statsmodels
+    #   timely-beliefs
+    #   timetomodel
+passlib==1.7.4
+    # via flask-security-too
+patsy==1.0.1
+    # via statsmodels
+pillow==11.2.1
+    # via
+    #   flexmeasures
+    #   matplotlib
+pint==0.24.4
+    # via flexmeasures
+platformdirs==4.3.8
+    # via
+    #   blosc2
+    #   pint
+ply==3.11
+    # via pyomo
+properscoring==0.1
+    # via timely-beliefs
+protobuf==6.31.1
+    # via siphon
+psutil==7.0.0
+    # via openturns
+psycopg2-binary==2.9.10
+    # via
+    #   flexmeasures
+    #   timely-beliefs
+pvlib==0.12.0
+    # via -r requirements/app.in
+py-cpuinfo==9.0.0
+    # via
+    #   blosc2
+    #   tables
+py-moneyed==3.0
+    # via flexmeasures
+pycparser==2.22
+    # via cffi
+pydantic==1.10.22
+    # via
+    #   flexmeasures
+    #   inflect
+pyluach==2.2.0
+    # via workalendar
+pymeeus==0.5.12
+    # via convertdate
+pyomo==6.9.2
+    # via flexmeasures
+pyparsing==3.2.3
+    # via matplotlib
+python-dateutil==2.9.0.post0
+    # via
+    #   arrow
+    #   matplotlib
+    #   pandas
+    #   timetomodel
+    #   workalendar
+python-dotenv==1.1.0
+    # via flexmeasures
+pytz==2025.2
+    # via
+    #   flexmeasures
+    #   pandas
+    #   pvlib
+    #   timely-beliefs
+    #   timetomodel
+pyyaml==6.0.2
+    # via flexmeasures
+redis==6.2.0
+    # via
+    #   flexmeasures
+    #   redis-sentinel-url
+    #   rq
+    #   rq-dashboard
+redis-sentinel-url==1.0.1
+    # via rq-dashboard
+referencing==0.36.2
+    # via
+    #   jsonschema
+    #   jsonschema-specifications
+requests==2.32.3
+    # via
+    #   blosc2
+    #   pvlib
+    #   requests-file
+    #   siphon
+    #   tldextract
+requests-file==2.1.0
+    # via tldextract
+rpds-py==0.25.1
+    # via
+    #   jsonschema
+    #   referencing
+rq==2.3.3
+    # via
+    #   flexmeasures
+    #   rq-dashboard
+rq-dashboard==0.8.2.2
+    # via flexmeasures
+scikit-base==0.12.3
+    # via sktime
+scikit-learn==1.6.1
+    # via
+    #   sktime
+    #   timetomodel
+scipy==1.15.3
+    # via
+    #   properscoring
+    #   pvlib
+    #   scikit-learn
+    #   sktime
+    #   statsmodels
+    #   timely-beliefs
+    #   timetomodel
+sentry-sdk[flask]==2.29.1
+    # via flexmeasures
+siphon==0.10.0
+    # via -r requirements/app.in
+six==1.17.0
+    # via python-dateutil
+sktime==0.37.0
+    # via timely-beliefs
+soupsieve==2.7
+    # via beautifulsoup4
+sqlalchemy==2.0.41
+    # via
+    #   alembic
+    #   flask-sqlalchemy
+    #   flexmeasures
+    #   marshmallow-sqlalchemy
+    #   timely-beliefs
+    #   timetomodel
+statsmodels==0.14.4
+    # via timetomodel
+tables==3.10.1
+    # via -r requirements/app.in
+tabulate==0.9.0
+    # via flexmeasures
+threadpoolctl==3.6.0
+    # via scikit-learn
+timely-beliefs[forecast]==3.2.0
+    # via flexmeasures
+timetomodel==0.7.3
+    # via flexmeasures
+tldextract==5.3.0
+    # via flexmeasures
+tomli==2.2.1
+    # via alembic
+types-python-dateutil==2.9.0.20250516
+    # via arrow
+typing-extensions==4.13.2
+    # via
+    #   alembic
+    #   altair
+    #   beautifulsoup4
+    #   flexcache
+    #   flexparser
+    #   pint
+    #   py-moneyed
+    #   pydantic
+    #   referencing
+    #   sqlalchemy
+    #   tables
+tzdata==2025.2
+    # via pandas
+uniplot==0.21.1
+    # via flexmeasures
+urllib3==2.4.0
+    # via
+    #   requests
+    #   sentry-sdk
+vl-convert-python==1.8.0
+    # via flexmeasures
+webargs==8.7.0
+    # via flexmeasures
+werkzeug==3.1.3
+    # via
+    #   flask
+    #   flask-cors
+    #   flask-login
+    #   flexmeasures
+workalendar==17.0.0
+    # via flexmeasures
+wtforms==3.2.1
+    # via
+    #   flask-security-too
+    #   flask-wtf
+xlrd==2.0.1
+    # via flexmeasures
+zipp==3.22.0
+    # via importlib-metadata
+
+# The following packages are considered to be unsafe in a requirements file:
+# setuptools

tools/flexmeasures-weather/requirements/dev.in

@@ -0,0 +1,11 @@
+-c app.txt
+-c test.txt
+
+pre-commit
+black
+flake8
+flake8-blind-except
+mypy
+pytest-runner
+setuptools_scm
+watchdog

tools/flexmeasures-weather/requirements/dev.txt

@@ -0,0 +1,86 @@
+#
+# This file is autogenerated by pip-compile with Python 3.10
+# by the following command:
+#
+#    pip-compile --output-file=requirements/dev.txt requirements/dev.in
+#
+black==25.1.0
+    # via -r requirements/dev.in
+cfgv==3.4.0
+    # via pre-commit
+click==8.2.1
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/test.txt
+    #   black
+distlib==0.3.9
+    # via virtualenv
+filelock==3.18.0
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   virtualenv
+flake8==7.2.0
+    # via -r requirements/dev.in
+flake8-blind-except==0.2.1
+    # via -r requirements/dev.in
+identify==2.6.12
+    # via pre-commit
+mccabe==0.7.0
+    # via flake8
+mypy==1.16.0
+    # via -r requirements/dev.in
+mypy-extensions==1.1.0
+    # via
+    #   black
+    #   mypy
+nodeenv==1.9.1
+    # via pre-commit
+packaging==25.0
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/test.txt
+    #   black
+    #   setuptools-scm
+pathspec==0.12.1
+    # via
+    #   black
+    #   mypy
+platformdirs==4.3.8
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   black
+    #   virtualenv
+pre-commit==4.2.0
+    # via -r requirements/dev.in
+pycodestyle==2.13.0
+    # via flake8
+pyflakes==3.3.2
+    # via flake8
+pytest-runner==6.0.1
+    # via -r requirements/dev.in
+pyyaml==6.0.2
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   pre-commit
+setuptools-scm==8.3.1
+    # via -r requirements/dev.in
+tomli==2.2.1
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/test.txt
+    #   black
+    #   mypy
+    #   setuptools-scm
+typing-extensions==4.13.2
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/test.txt
+    #   black
+    #   mypy
+virtualenv==20.31.2
+    # via pre-commit
+watchdog==6.0.0
+    # via -r requirements/dev.in
+
+# The following packages are considered to be unsafe in a requirements file:
+# setuptools

tools/flexmeasures-weather/requirements/test.in

@@ -0,0 +1,10 @@
+-c app.txt
+
+pytest
+pytest-flask
+pytest-sugar
+pytest-cov
+# lets tests run successfully in containers
+fakeredis
+# required with fakeredis, maybe because we use rq
+lupa

tools/flexmeasures-weather/requirements/test.txt

@@ -0,0 +1,88 @@
+#
+# This file is autogenerated by pip-compile with Python 3.10
+# by the following command:
+#
+#    pip-compile --output-file=requirements/test.txt requirements/test.in
+#
+async-timeout==5.0.1
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   redis
+blinker==1.9.0
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   flask
+click==8.2.1
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   flask
+coverage[toml]==7.8.2
+    # via pytest-cov
+exceptiongroup==1.3.0
+    # via pytest
+fakeredis==2.29.0
+    # via -r requirements/test.in
+flask==3.1.1
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   pytest-flask
+iniconfig==2.1.0
+    # via pytest
+itsdangerous==2.2.0
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   flask
+jinja2==3.1.6
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   flask
+lupa==2.4
+    # via -r requirements/test.in
+markupsafe==3.0.2
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   flask
+    #   jinja2
+    #   werkzeug
+packaging==25.0
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   pytest
+    #   pytest-sugar
+pluggy==1.6.0
+    # via pytest
+pytest==8.3.5
+    # via
+    #   -r requirements/test.in
+    #   pytest-cov
+    #   pytest-flask
+    #   pytest-sugar
+pytest-cov==6.1.1
+    # via -r requirements/test.in
+pytest-flask==1.3.0
+    # via -r requirements/test.in
+pytest-sugar==1.0.0
+    # via -r requirements/test.in
+redis==6.2.0
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   fakeredis
+sortedcontainers==2.4.0
+    # via fakeredis
+termcolor==3.1.0
+    # via pytest-sugar
+tomli==2.2.1
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   coverage
+    #   pytest
+typing-extensions==4.13.2
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   exceptiongroup
+    #   fakeredis
+werkzeug==3.1.3
+    # via
+    #   -c /home/nicolas/workspace/seita/flexmeasures-weather/requirements/app.txt
+    #   flask
+    #   pytest-flask