diff --git a/README.md b/README.md index e4e7e73351c799f8e4fae7a5bc1f8da38a314f7c..3ed00091bdf5d8a8110740238ed1984633598898 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ in an Open AI-gym-inspired environment. The scenario is a simplified setting with the following components. -Detections are viewed very simplistic in this scenario. +Detections are viewed very simplisticly in this scenario. Each aircraft has its own model of Situational Awareness (SA) for its opponents. If an observation of an opponent is made (or received from a teammate), the own SA is updated. @@ -45,18 +45,23 @@ To create an virtual environment and install dependencies, run the following com >>> pip install -r requirements.txt # or python install -r requirements_deploy.txt ``` -## Example usage +## Using the environment -Here follows some examples of how the environment can be used. +The environment creates a scenario between two opposing teams, green and red. Each team uses a specific behaviour +that determines what radar actions the aircraft in the team make. All avaliable behaviours can be found in [behaviour.py](behaviour.py). -The main files for running tests are: + +The main files for running scenario tests are: - main.py -- analyse.py -- run_scenario.sh +- run_testing.sh +- run_scenarios.sh + +Before running any type of test, the virtual environment must activated. -### Perform a single test +### Performing a single test -Run a single test. +Use the [main.py](main.py) file to run a single test, which can be configured with several different flags. +See the `--help` flag for more information about all the available flags. ```bash >>> # source virtual environment @@ -66,20 +71,79 @@ Run a single test. >>> python main.py --help ``` +### Performing a test of developed behaviours. + +Three behaviours, based on mixed integer programming, reinforcement learning and algorithm selection, have been developed specifically to be analysed and evaluated in this environment. +To run a test that pitches these three behaviours against all other available behaviours, use the file [run_testing.sh](run_testing.sh). +There are some environment variables in the file that can be used to configure a run. + +```bash +>>> # source virtual environment +>>> sh run_testing.sh +``` + +For details about the developed behaviours, see [mip_behaviour.py](mip_behaviour.py), [rl_behavior.py](rl_behavior.py) and [algsel_behaviour.py](algsel_behaviour.py). + + ### Perform all tests -Run all tests. There are some environment variables in `run_scenarios.sh` that can be used to configure a run. +To run a test of all behaviours, use the file [run_scenarios.sh](run_scenarios.sh). +The same environment variables that are available in `run_testing` are available here as well. +The default settings pitches all behaviours in the test list against each other 200 times, which takes a considerable amount of time. ```bash >>> # source virtual environment >>> sh run_scenarios.sh ``` -### Perform analysis +## Analysis + +There are a few ways of analysing the results from the test runs. + +The main files for analysing the tests are: +- test_analysis.py +- analyse.py -Perform analysis with on CSV-file with data +### Perform basic analysis + +Perform analysis on CSV-file with data. Gives information about which team that won a given metric in a scenario. ```bash >>> # source virtual environment >>> python analyse.py PATH_TO_CSV_FILE ``` + +### Perform extended anlysis + +Perform more analysis on a CSV-folder with data. Prints win/draw/loss percentages in terminal +and can plot the score in the metrics in each time step. Use the `--help` flag for a list of options when analysing. + +```bash +>>> # source virtual environment +>>> python test_analysis.py PATH_TO_CSV_FOLDER +``` + +### Training an RL-behaviour + +Behaviours can be trained using the deep reinforcement learning interface availade in the mdptest branch. To influence how the behaviour is trained, setting regarding the training, and the reward function can be changed. To start the training, simply run the multiagent_rl_train.py script within the virtual environment. + +```bash +>>> # source virtual environment +>>> python multiagent_rl_train.py +``` + +### Changing the reward + +In the reward method, in the ScalarizedRewardWrapper class, in the python file scalarized_reward_wrapper.py, the reward can be set by changing the new_reward dict in the else statement. See the code for more information on the different options and how to select them. + +### Changing the settings + +In the config.py file there are many settings available to change the simulation environemtn, the training environment, and the network parameters. For the simulation environment settings, there are options to change the number of planes per team and what behaviour is used to train against, as well as if a specific seed should be used for randomising. For the training environemt settings, the number of simulations running in parallell, and wether to use the evaluation or not can be configured. For the network parameters, there are a multitude. See the config.py file for a description of all of them. + +### Continuing to train an old behaviour + +It is possible to continue training an old behaviour. To do this, change the model_dir config in config.py to the path of the models directory of the model you wawnt to keep training. If you want to start a new training from scratch, change the model_dir to None. + +### Rendering of the behaviour + +It is possible to render the scenario during the evaluation simulations. To enable this, make sure that use_eval is set to True in config.py. Then, also in config.py, set eval_render to True, and use the visualise_delay and only_delay_middle settings to change if and how the rendering is slowed down. Now when a behaviour is trained, during eval, a window will open and the evaluation simulation will be rendered. diff --git a/README_dev.md b/README_dev.md deleted file mode 100644 index 3ed00091bdf5d8a8110740238ed1984633598898..0000000000000000000000000000000000000000 --- a/README_dev.md +++ /dev/null @@ -1,149 +0,0 @@ -# Sensor Control - -This environment can be used to test methods for coordinating multi-platform radar actions -in an Open AI-gym-inspired environment. - -## Scenario setting - -The scenario is a simplified setting with the following components. - -Detections are viewed very simplisticly in this scenario. -Each aircraft has its own model of Situational Awareness (SA) for its opponents. -If an observation of an opponent is made (or received from a teammate), -the own SA is updated. -The detections are always perfect (and have perfect association) and the latest observation of a particular aircraft is always used. -Instead, uncertainty is introduced via the probability that an aircraft detects (or is being detected) via its radar, radar detector or cockpit (visual detection). - -The radar model is probability-based and the performance is impacted by: -- RCS-constant of observed aircraft -- Angle of radar cone and angle between objects -- Distance between objects - -The probability of detecting an opponent is evaluated once in each time step an aircraft has an active radar. -Simply put, if an opposing aircraft is within the radar observation angle of an aircraft with active radar, -the probability of detection is higher if the aircraft are close to each other. - -For details, see methods `is_observed_by_radar` and `is_observed_by_radar_probability` in [support.py](support.py). - -A radar detection model simulates the probability of detection and there is a high probability of detection if the angle radar cone and angle between objects intersects. -The radar dection model does not depend on distance between objects. -For details, see methods `is_observed_by_radar_detection` in [support.py](support.py). - -There is a delay in transmitting an observation to ones teammates. -The specific delay can be controlled in [constants.py](constants.py). - -## Installation - -The code has been tested using Python 3.9.10. - -To create an virtual environment and install dependencies, run the following commands - -```bash ->>> git clone ... ->>> python3.9 -m venv venv ->>> source venv/bin/activate ->>> pip install -r requirements.txt # or python install -r requirements_deploy.txt -``` - -## Using the environment - -The environment creates a scenario between two opposing teams, green and red. Each team uses a specific behaviour -that determines what radar actions the aircraft in the team make. All avaliable behaviours can be found in [behaviour.py](behaviour.py). - - -The main files for running scenario tests are: -- main.py -- run_testing.sh -- run_scenarios.sh - -Before running any type of test, the virtual environment must activated. - -### Performing a single test - -Use the [main.py](main.py) file to run a single test, which can be configured with several different flags. -See the `--help` flag for more information about all the available flags. - -```bash ->>> # source virtual environment ->>> python main.py # Default version ->>> python main.py --green simpletwo --red strmone # Behaviour arguments ->>> python main.py --visualise # Visualise ->>> python main.py --help -``` - -### Performing a test of developed behaviours. - -Three behaviours, based on mixed integer programming, reinforcement learning and algorithm selection, have been developed specifically to be analysed and evaluated in this environment. -To run a test that pitches these three behaviours against all other available behaviours, use the file [run_testing.sh](run_testing.sh). -There are some environment variables in the file that can be used to configure a run. - -```bash ->>> # source virtual environment ->>> sh run_testing.sh -``` - -For details about the developed behaviours, see [mip_behaviour.py](mip_behaviour.py), [rl_behavior.py](rl_behavior.py) and [algsel_behaviour.py](algsel_behaviour.py). - - -### Perform all tests - -To run a test of all behaviours, use the file [run_scenarios.sh](run_scenarios.sh). -The same environment variables that are available in `run_testing` are available here as well. -The default settings pitches all behaviours in the test list against each other 200 times, which takes a considerable amount of time. - -```bash ->>> # source virtual environment ->>> sh run_scenarios.sh -``` - -## Analysis - -There are a few ways of analysing the results from the test runs. - -The main files for analysing the tests are: -- test_analysis.py -- analyse.py - -### Perform basic analysis - -Perform analysis on CSV-file with data. Gives information about which team that won a given metric in a scenario. - -```bash ->>> # source virtual environment ->>> python analyse.py PATH_TO_CSV_FILE -``` - -### Perform extended anlysis - -Perform more analysis on a CSV-folder with data. Prints win/draw/loss percentages in terminal -and can plot the score in the metrics in each time step. Use the `--help` flag for a list of options when analysing. - -```bash ->>> # source virtual environment ->>> python test_analysis.py PATH_TO_CSV_FOLDER -``` - -### Training an RL-behaviour - -Behaviours can be trained using the deep reinforcement learning interface availade in the mdptest branch. To influence how the behaviour is trained, setting regarding the training, and the reward function can be changed. To start the training, simply run the multiagent_rl_train.py script within the virtual environment. - -```bash ->>> # source virtual environment ->>> python multiagent_rl_train.py -``` - -### Changing the reward - -In the reward method, in the ScalarizedRewardWrapper class, in the python file scalarized_reward_wrapper.py, the reward can be set by changing the new_reward dict in the else statement. See the code for more information on the different options and how to select them. - -### Changing the settings - -In the config.py file there are many settings available to change the simulation environemtn, the training environment, and the network parameters. For the simulation environment settings, there are options to change the number of planes per team and what behaviour is used to train against, as well as if a specific seed should be used for randomising. For the training environemt settings, the number of simulations running in parallell, and wether to use the evaluation or not can be configured. For the network parameters, there are a multitude. See the config.py file for a description of all of them. - -### Continuing to train an old behaviour - -It is possible to continue training an old behaviour. To do this, change the model_dir config in config.py to the path of the models directory of the model you wawnt to keep training. If you want to start a new training from scratch, change the model_dir to None. - -### Rendering of the behaviour - -It is possible to render the scenario during the evaluation simulations. To enable this, make sure that use_eval is set to True in config.py. Then, also in config.py, set eval_render to True, and use the visualise_delay and only_delay_middle settings to change if and how the rendering is slowed down. Now when a behaviour is trained, during eval, a window will open and the evaluation simulation will be rendered. diff --git a/README_old.md b/README_old.md new file mode 100644 index 0000000000000000000000000000000000000000..e4e7e73351c799f8e4fae7a5bc1f8da38a314f7c --- /dev/null +++ b/README_old.md @@ -0,0 +1,85 @@ +# Sensor Control + +This environment can be used to test methods for coordinating multi-platform radar actions +in an Open AI-gym-inspired environment. + +## Scenario setting + +The scenario is a simplified setting with the following components. + +Detections are viewed very simplistic in this scenario. +Each aircraft has its own model of Situational Awareness (SA) for its opponents. +If an observation of an opponent is made (or received from a teammate), +the own SA is updated. +The detections are always perfect (and have perfect association) and the latest observation of a particular aircraft is always used. +Instead, uncertainty is introduced via the probability that an aircraft detects (or is being detected) via its radar, radar detector or cockpit (visual detection). + +The radar model is probability-based and the performance is impacted by: +- RCS-constant of observed aircraft +- Angle of radar cone and angle between objects +- Distance between objects + +The probability of detecting an opponent is evaluated once in each time step an aircraft has an active radar. +Simply put, if an opposing aircraft is within the radar observation angle of an aircraft with active radar, +the probability of detection is higher if the aircraft are close to each other. + +For details, see methods `is_observed_by_radar` and `is_observed_by_radar_probability` in [support.py](support.py). + +A radar detection model simulates the probability of detection and there is a high probability of detection if the angle radar cone and angle between objects intersects. +The radar dection model does not depend on distance between objects. +For details, see methods `is_observed_by_radar_detection` in [support.py](support.py). + +There is a delay in transmitting an observation to ones teammates. +The specific delay can be controlled in [constants.py](constants.py). + +## Installation + +The code has been tested using Python 3.9.10. + +To create an virtual environment and install dependencies, run the following commands + +```bash +>>> git clone ... +>>> python3.9 -m venv venv +>>> source venv/bin/activate +>>> pip install -r requirements.txt # or python install -r requirements_deploy.txt +``` + +## Example usage + +Here follows some examples of how the environment can be used. + +The main files for running tests are: +- main.py +- analyse.py +- run_scenario.sh + +### Perform a single test + +Run a single test. + +```bash +>>> # source virtual environment +>>> python main.py # Default version +>>> python main.py --green simpletwo --red strmone # Behaviour arguments +>>> python main.py --visualise # Visualise +>>> python main.py --help +``` + +### Perform all tests + +Run all tests. There are some environment variables in `run_scenarios.sh` that can be used to configure a run. + +```bash +>>> # source virtual environment +>>> sh run_scenarios.sh +``` + +### Perform analysis + +Perform analysis with on CSV-file with data + +```bash +>>> # source virtual environment +>>> python analyse.py PATH_TO_CSV_FILE +```