Add extracted tools: CitrineOS, OpenOCPP, ShapeShifter
- CitrineOS core extracted (CSMS OCPP 2.0.1) - OpenOCPP extracted (firmware OCPP 1.6J/2.0.1) - ShapeShifter library installed (pip install -e) - ShapeShifter specification extracted - EVerest extracted TODO updated with progress
This commit is contained in:
Eric F
487
tools/EVerest-main/docs/source/explanation/dev-tools/edm.rst
Normal file
487
tools/EVerest-main/docs/source/explanation/dev-tools/edm.rst
Normal file
@@ -0,0 +1,487 @@
|
||||
.. _exp_dev_tools_edm:
|
||||
|
||||
###
|
||||
edm
|
||||
###
|
||||
|
||||
edm stands for EVerest dependency manager. It helps you orchestrating the
|
||||
dependencies between the different EVerest repositories.
|
||||
|
||||
.. note::
|
||||
|
||||
The EDM tool was developed at a time when the EVerest source code was
|
||||
organized in many different repositories. Since 2026, EVerest has essentially
|
||||
changed to a (quasi-)mono-repository layout. Manual installation of EDM is
|
||||
usually not necessary to work with recent versions of EVerest.
|
||||
|
||||
Dependency Manager for EVerest
|
||||
##############################
|
||||
|
||||
Install and Quick Start
|
||||
***********************
|
||||
|
||||
To install the **edm** dependency manager for EVerest you have to perform the
|
||||
following steps.
|
||||
|
||||
Please make sure you are running a sufficiently recent version of **Python3 (>=3.6)** and that you are able to install Python packages from source.
|
||||
See the *python3* command below for upgrading the required packages. Refer to
|
||||
the
|
||||
`Python Installing Packages <https://packaging.python.org/tutorials/installing-packages/#requirements-for-installing-packages>`_
|
||||
documentation for indepth guidance if any problems arise. You may want to create and activate a virtual environment
|
||||
using `venv <https://docs.python.org/3/library/venv.html>`_
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
|
||||
before executing the commands below.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python3 -m pip install --upgrade pip setuptools wheel jstyleson jsonschema
|
||||
|
||||
Python packages needed to run edm
|
||||
*********************************
|
||||
|
||||
The following Python3 packages are needed to run **edm**. If you install edm
|
||||
using this guide they will be installed automatically.
|
||||
|
||||
+ Python >= 3.6
|
||||
+ Jinja2 >= 3.0
|
||||
+ PyYAML >= 5.4
|
||||
|
||||
Installing edm
|
||||
**************
|
||||
|
||||
Now you can clone this repository and install **edm**:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone https://github.com/EVerest/EVerest.git
|
||||
cd EVerest/applications/dependency_manager
|
||||
python3 -m pip install . --break-system-packages
|
||||
|
||||
or in short
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python3 -m pip install git+https://github.com/EVerest/EVerest.git@main#subdirectory=applications/dependency_manager --break-system-packages
|
||||
|
||||
.. note::
|
||||
|
||||
Alternatively, you can also install ``edm`` in a python virtual environment.
|
||||
Make sure edm is available in your PATH after the installation. You can verify
|
||||
this by running ``edm --version``.
|
||||
|
||||
Next you could run
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm init --workspace ~/checkout/everest-workspace
|
||||
|
||||
This creates a workspace in the ``~/checkout/everest-workspace``
|
||||
directory from the most recent release of EVerest. If you want the most recent
|
||||
main you can use:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm init main --workspace ~/checkout/everest-workspace
|
||||
|
||||
The workspace will have the following structure containing all current
|
||||
dependencies for EVerest:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
everest-workspace/
|
||||
├── everest-cmake
|
||||
├── EVerest
|
||||
├── everest-dev-environment
|
||||
├── everest-framework
|
||||
├── everest-sqlite
|
||||
├── everest-utils
|
||||
├── Josev
|
||||
├── libcbv2g
|
||||
├── libevse-security
|
||||
├── libfsm
|
||||
├── libiso15118
|
||||
├── liblog
|
||||
├── libnfc-nci
|
||||
├── libocpp
|
||||
├── libslac
|
||||
├── libtimer
|
||||
└── workspace-config.yaml
|
||||
|
||||
The ``workspace-config.yaml`` contains a copy of the config that was used to create
|
||||
this workspace.
|
||||
|
||||
Enabling CPM_SOURCE_CACHE and setting PATH
|
||||
******************************************
|
||||
|
||||
The EVerest dependency manager uses
|
||||
`CPM <https://github.com/cpm-cmake/CPM.cmake>`_
|
||||
for its CMake integration. This means you *can* and **should** set the
|
||||
``CPM_SOURCE_CACHE`` environment variable. This makes sure that dependencies
|
||||
that you do not manage in the workspace are not re-downloaded multiple times.
|
||||
For detailed information and other useful environment variables please
|
||||
refer to the `CPM Documentation <https://github.com/cpm-cmake/CPM.cmake/blob/master/README.md#CPM_SOURCE_CACHE>`_.
|
||||
|
||||
Also set the PATH variable:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export CPM_SOURCE_CACHE=$HOME/.cache/CPM
|
||||
export PATH=$PATH:/home/$(whoami)/.local/bin
|
||||
|
||||
Building EVerest
|
||||
****************
|
||||
|
||||
Make sure you have installed :doc:`ev-cli <ev-cli>` first.
|
||||
You can now use the following commands to build the repository EVerest:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd ~/checkout/everest-workspace/EVerest
|
||||
mkdir build
|
||||
cd build
|
||||
cmake ..
|
||||
make -j$(nproc) install
|
||||
|
||||
.. _cmake_integration_setup:
|
||||
|
||||
Setting up and updating a workspace
|
||||
###################################
|
||||
|
||||
For letting **edm** do the work of setting up an initial EVerest workspace,
|
||||
do this:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm init --workspace ~/checkout/everest-workspace
|
||||
|
||||
If you are currently in the everest-workspace directory the following command
|
||||
has the same effect:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm init
|
||||
|
||||
For using a dedicated release version, you can do this:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm init 2023.7.0
|
||||
|
||||
In this example, version 2023.7.0 is pulled from the server. This will only work
|
||||
if your previous code is not in a "dirty" state.
|
||||
|
||||
Using the edm CMake module and dependencies.yaml
|
||||
################################################
|
||||
|
||||
To use edm from CMake you have to add the following line to the top-level
|
||||
CMakeLists.txt file in the respective source repository:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
find_package(EDM REQUIRED)
|
||||
|
||||
To define dependencies you can now add a dependencies.yaml file to your source
|
||||
repository. It should look like this:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
---
|
||||
sigslot:
|
||||
git: https://github.com/palacaze/sigslot
|
||||
git_tag: v1.2.3
|
||||
cmake_condition: "EVEREST_DEPENDENCY_ENABLED_SIGSLOT"
|
||||
options:
|
||||
- "SIGSLOT_COMPILE_EXAMPLES OFF"
|
||||
- "SIGSLOT_COMPILE_TESTS OFF"
|
||||
pugixml:
|
||||
git: https://github.com/zeux/pugixml
|
||||
git_tag: v1.15
|
||||
cmake_condition: "EVEREST_DEPENDENCY_ENABLED_PUGIXML"
|
||||
|
||||
If you want to conditionally include some dependencies, e.g. for testing, you can
|
||||
do this in the following way:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
catch2:
|
||||
git: https://github.com/catchorg/Catch2.git
|
||||
git_tag: v3.4.0
|
||||
cmake_condition: "BUILD_TESTING"
|
||||
|
||||
Here *cmake_condition* can be any string that CMake can use in an if() block.
|
||||
Please be aware that any variables you use here must be defined before a call to
|
||||
*evc_setup_edm()* is made in your ``CMakeLists.txt``
|
||||
|
||||
Additionally you can set the ``EVEREST_MODIFY_DEPENDENCIES`` environment variable
|
||||
to a file containing modifications to the projects ``dependencies.yaml`` files when
|
||||
running cmake:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
EVEREST_MODIFY_DEPENDENCIES=../dependencies_modified.yaml cmake -S . -B build
|
||||
|
||||
The ``dependencies_modified.yaml`` file can contain something along these lines:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
nlohmann_json:
|
||||
git: null # this makes edm look for nlohmann_json via find_package
|
||||
libfmt:
|
||||
rename: fmt # if find_package needs a different dependency name you can rename it
|
||||
git: null
|
||||
catch2:
|
||||
git_tag: v1.2.3 # select a different git tag for a build
|
||||
|
||||
Selective library consumption
|
||||
#############################
|
||||
|
||||
If your external project only needs specific everest-core libraries (e.g.
|
||||
``liblog``, ``everest-util``, ``everest-io``, ``libocpp``, ``libiso15118``)
|
||||
without building the full module framework, you can use the
|
||||
``EVEREST_LIBS_ONLY`` and ``EVEREST_INCLUDE_LIBS`` CMake options.
|
||||
|
||||
**CMake options:**
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Option
|
||||
- Default
|
||||
- Description
|
||||
* - ``EVEREST_LIBS_ONLY``
|
||||
- OFF
|
||||
- Skip modules, applications, config, and code generation. Only build
|
||||
libraries under ``lib/everest/``.
|
||||
* - ``EVEREST_INCLUDE_LIBS``
|
||||
- (empty)
|
||||
- Semicolon-separated allowlist of libraries to build. Transitive
|
||||
internal dependencies are resolved automatically. When empty, all
|
||||
libraries are built.
|
||||
* - ``EVEREST_EXCLUDE_LIBS``
|
||||
- (empty)
|
||||
- Semicolon-separated blocklist of libraries to skip.
|
||||
|
||||
**Example: building only liblog, everest-util, and everest-io**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake -S . -B build \
|
||||
-DEVEREST_LIBS_ONLY=ON \
|
||||
-DEVEREST_INCLUDE_LIBS="log;util;io"
|
||||
cmake --build build
|
||||
|
||||
Transitive dependencies are resolved automatically. For example, requesting
|
||||
``io`` will automatically include ``util`` (since ``everest-io`` depends on
|
||||
``everest-util``).
|
||||
|
||||
**Example: building only libocpp**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake -S . -B build \
|
||||
-DEVEREST_LIBS_ONLY=ON \
|
||||
-DEVEREST_INCLUDE_LIBS="ocpp"
|
||||
cmake --build build
|
||||
|
||||
This resolves the full dependency chain: ``ocpp`` -> ``log``, ``timer``,
|
||||
``evse_security``, ``sqlite``, ``cbv2g``.
|
||||
|
||||
**Using from an external project's dependencies.yaml:**
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
everest-core:
|
||||
git: https://github.com/EVerest/everest-core.git
|
||||
git_tag: 2026.02.0
|
||||
options:
|
||||
- "EVEREST_LIBS_ONLY ON"
|
||||
- "EVEREST_INCLUDE_LIBS log;util;io"
|
||||
|
||||
The internal dependency map is defined in ``cmake/ev-lib-dependencies.cmake``.
|
||||
|
||||
.. note::
|
||||
|
||||
Libraries that depend on framework code generation (``tls``, ``helpers``,
|
||||
``conversions``, ``slac``, ``external_energy_limits``, ``everest_api_types``)
|
||||
are **not available** in ``EVEREST_LIBS_ONLY`` mode. Use
|
||||
``EVEREST_EXCLUDE_MODULES`` instead if you need those libraries.
|
||||
|
||||
Framework thread pool scaling policy
|
||||
####################################
|
||||
|
||||
The framework message handler uses a dynamically scaling thread pool for
|
||||
operation messages such as variable updates, commands, errors, GetConfig and
|
||||
ModuleReady messages. Its scaling policy can be selected at CMake configure
|
||||
time.
|
||||
|
||||
**CMake options:**
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Option
|
||||
- Default
|
||||
- Description
|
||||
* - ``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY``
|
||||
- ``latency``
|
||||
- Selects the policy. Supported values are ``latency``, ``greedy``,
|
||||
``conservative``, ``fixed_size`` and ``custom``.
|
||||
* - ``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_LATENCY_THRESHOLD_MS``
|
||||
- ``50``
|
||||
- Maximum queued task wait time, in milliseconds, before the ``latency``
|
||||
policy adds another worker.
|
||||
* - ``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_LATENCY_TICK_MS``
|
||||
- ``5``
|
||||
- Supervisor tick, in milliseconds, for the ``latency`` policy.
|
||||
* - ``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_FIXED_SIZE_THRESHOLD``
|
||||
- ``3``
|
||||
- Queue size threshold at which the ``fixed_size`` policy adds another
|
||||
worker.
|
||||
* - ``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY_CUSTOM_HEADER``
|
||||
- (empty)
|
||||
- Header to include when the policy is ``custom``.
|
||||
* - ``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY_CUSTOM_TYPE``
|
||||
- (empty)
|
||||
- Fully-qualified C++ policy type to use when the policy is ``custom``.
|
||||
* - ``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY_CUSTOM_INCLUDE_DIR``
|
||||
- (empty)
|
||||
- Additional include directory for the custom policy header.
|
||||
|
||||
The built-in policies are:
|
||||
|
||||
.. list-table::
|
||||
:header-rows: 1
|
||||
|
||||
* - Policy
|
||||
- Behavior
|
||||
* - ``latency``
|
||||
- Default. Adds workers when queued work has waited longer than the
|
||||
framework latency threshold.
|
||||
* - ``greedy``
|
||||
- Adds workers as soon as backlog is detected.
|
||||
* - ``conservative``
|
||||
- Adds workers only when the queue depth significantly exceeds the current
|
||||
worker count.
|
||||
* - ``fixed_size``
|
||||
- Adds workers once the queue size reaches the configured
|
||||
``EVEREST_FRAMEWORK_THREAD_POOL_SCALING_FIXED_SIZE_THRESHOLD``.
|
||||
|
||||
**Example: selecting a built-in policy for a full EVerest build**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake -S . -B build \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY=latency \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_LATENCY_THRESHOLD_MS=50 \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_LATENCY_TICK_MS=5
|
||||
cmake --build build
|
||||
|
||||
**Example: selecting fixed-size scaling**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake -S . -B build \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY=fixed_size \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_FIXED_SIZE_THRESHOLD=3
|
||||
cmake --build build
|
||||
|
||||
**Example: using a custom policy**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cmake -S . -B build \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY=custom \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY_CUSTOM_HEADER=my_policy.hpp \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY_CUSTOM_TYPE=my_project::MyPolicy \
|
||||
-DEVEREST_FRAMEWORK_THREAD_POOL_SCALING_POLICY_CUSTOM_INCLUDE_DIR=/path/to/include
|
||||
cmake --build build
|
||||
|
||||
A custom policy must provide the same interface as the built-in policies:
|
||||
|
||||
.. code-block:: cpp
|
||||
|
||||
struct MyPolicy {
|
||||
static constexpr std::optional<std::chrono::milliseconds> supervisor_tick =
|
||||
std::nullopt;
|
||||
|
||||
static bool should_grow(
|
||||
std::size_t current_workers,
|
||||
std::size_t queue_size,
|
||||
std::optional<std::chrono::steady_clock::time_point> oldest_arrival);
|
||||
};
|
||||
|
||||
Create a workspace config from an existing directory tree
|
||||
#########################################################
|
||||
|
||||
Suppose you already have a directory tree that you want to save into a config
|
||||
file. You can do this with the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm --create-config custom-config.yaml
|
||||
|
||||
This is a short form of:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm --create-config custom-config.yaml --include-remotes https://github.com/EVerest/*
|
||||
|
||||
and only includes repositories from the EVerest namespace. You can add as many
|
||||
remotes to this list as you want.
|
||||
|
||||
For example, if you only want to include certain repositories you can use the
|
||||
following command.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm --create-config custom-config.yaml --include-remotes https://github.com/EVerest/everest* https://github.com/EVerest/ext-switchev-iso15118.git
|
||||
|
||||
If you want to include all repositories, including external dependencies, in
|
||||
the config you can use the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm --create-config custom-config.yaml --external-in-config
|
||||
|
||||
.. _git_information_at_a_glance:
|
||||
|
||||
Git information at a glance
|
||||
###########################
|
||||
|
||||
You can get a list of all git repositories in the current directory and their
|
||||
state using the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm --git-info --git-fetch
|
||||
|
||||
If you want to know the state of all repositories in a workspace you can use
|
||||
the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
edm --workspace ~/checkout/everest-workspace --git-info --git-fetch
|
||||
|
||||
This creates output that is similar to the following example:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
[edm]: Git info for "~/checkout/everest-workspace":
|
||||
[edm]: Using git-fetch to update remote information. This might take a few seconds.
|
||||
[edm]: "everest-dev-environment" @ branch: main [remote: origin/main] [behind 6] [clean]
|
||||
[edm]: "everest-framework" @ branch: main [remote: origin/main] [dirty]
|
||||
[edm]: "everest-deploy-devkit" @ branch: main [remote: origin/main] [clean]
|
||||
[edm]: "libtimer" @ branch: main [remote: origin/main] [dirty]
|
||||
[edm]: 2/4 repositories are dirty.
|
||||
|
||||
Further information can be seen as shell output by calling edm with parameter
|
||||
**-h** or **--help**.
|
||||
|
||||
----
|
||||
|
||||
**Authors**: Kai-Uwe Hermann, Stefan Wahren, Andreas Heinrich, Manuel Ziegler
|
||||
Reference in New Issue
Block a user