**Latest update January 14th, 2025:Data file 13 added.
This homepage contains information about the SLIDe:Steam Line Intrusion Detection Benchmark.
The benchmark is part of the competition at the [The 36th International Conference on Principles of Diagnosis and Resilient Systems (DX36)](), Nashville, TN, USA.
The benchmark is part of the competition at the <b>The 36th International Conference on Principles of Diagnosis and Resilient Systems (DX36)</b>, Nashville, TN, USA.
</div>
...
...
@@ -32,19 +31,26 @@ The benchmark is part of the competition at the [The 36th International Conferen
</div>
## News
**Latest update January 15th, 2025:Page is public!
## FAQ
**Ask your questions to Anna Sztyber-Betley, <a href="mailto:anna.sztyber@pw.edu.pl">anna.sztyber@pw.edu.pl</a>.
## Benchmark organizers and contact
*Anna Sztyber-Betley, anna.sztyber@pw.edu.pl, [Faculty of Mechatronics](https://www.mchtr.pw.edu.pl/www_mchtr_eng), Warsaw University of Technology, Poland
*Michał Syfert, michal.syfert@pw.edu.pl, [Faculty of Mechatronics](https://www.mchtr.pw.edu.pl/www_mchtr_eng), Warsaw University of Technology, Poland
*Anna Sztyber-Betley, <a href="mailto:anna.sztyber@pw.edu.pl">anna.sztyber@pw.edu.pl</a>, [Faculty of Mechatronics](https://www.mchtr.pw.edu.pl/www_mchtr_eng), Warsaw University of Technology, Poland
*Michał Syfert, <a href="mailto:michal.syfert@pw.edu.pl">michal.syfert@pw.edu.pl</a>, [Faculty of Mechatronics](https://www.mchtr.pw.edu.pl/www_mchtr_eng), Warsaw University of Technology, Poland
## How to participate
<a name="participate"></a>
All solutions should be implemented in Python and will be evaluated using Docker.
Please ensure that the solution does not require any licensed software.
Code templates and instructions can be found in the [github repository](https://github.com/asztyber/DXC25_SLIDe).
Code templates and instructions can be found in the [github repository](https://github.com/asztyber/DXC25_SLIDe).
Please send your solution before the deadline to the following email address:anna.sztyber@pw.edu.pl or upload your files to github/shared folder and send the link in the email.
Please send your solution before the deadline to the following email address:<a href="mailto:anna.sztyber@pw.edu.pl">anna.sztyber@pw.edu.pl</a> or upload your files to github/shared folder and send the link in the email.
Please let Anna Sztyber-Betley know if you plan to participate. I will be happy to discuss any questions you may have.
Please let Anna Sztyber-Betley know if you plan to participate. I will be happy to discuss any questions you may have. See the <a href="#participate">How to participate</a> section for more details.
## Description
...
...
@@ -115,7 +121,7 @@ Key process variables are described in the table below:
| 4.2 | $SP_{4.2}$ | Set point of the steam temperature at the outlet of superheater 4 | °C | - | 540 |
| 4.2 | $CV_{4.2}$ | Control signal of temperature controller at the outlet of the superheater 4 | % | 0-100 | 52.69 |
Control loops and process variables are in the plots below:
Control loops and process variables are shown in the plots below:
...
...
@@ -193,13 +199,13 @@ Code for the submission and usage instructions can be found in the [github repos
<aname="evaluation"></a>
Participants will be evaluated on the following metrices:
- Diagnosis of process faults:
- True detection rate.
-1 - False detection rate.
-Isolation accuracy.
- True detection rate (TDR) - The percentage of samples does the diagnosis system state that a fault is detected when there is a fault in the system.
-False alarm rate (FAR) - The percentage of samples does the diagnosis system state that a fault is detected when there is no fault in the system. We use 1 - FAR as a metric.
-<ahref="#faults_isolation_accuracy">Fault isolation accuracy (FIA)</a> - The average probability given to the true diagnosis for all samples when a fault is correctly detected.
- Diagnosis of cybernetic attacks:
- True detection rate.
-1 - False detection rate.
-Isolationaccuracy (with loop granularity).
- True detection rate (TDR) - The percentage of samples does the diagnosis system state that a cybernetic attack is detected when there is a cybernetic attack in the system.
-False alarm rate (FAR) - The percentage of samples does the diagnosis system state that a cybernetic attack is detected when there is no cybernetic attack in the system. We use 1 - FAR as a metric.
The final score is a weighted sum of these metrics.
Each score is calculated independently for each given sample (simulation step). All scores are averaged over the whole evaluation dataset.
...
...
@@ -219,12 +225,14 @@ The above mentioned scores will be calculated based on these output signals.
### Detection rate
True and False Detection rate will be calculated for both, process fault detection signal and cybernetic attack detection signal.
### Process Faults isolation accuracy
### Faults isolation accuracy
<aname="faults_isolation_accuracy"></a>
The isolation accuracy score is the probability assigned to the simulated fault. All probabilities returned by the algorithm should be normalized to sum to 1.
For example, if the simulated fault vector is [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] (i.e. the fault $f_2$ is present) and the provided probabilities are [0.0, 0.9, 0.05, 0.05, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], the isolation accuracy score is 0.9 (the probability assigned to the correct fault $f_2$).
