Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
M
MLTestSelection
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Requirements
Code
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Deploy
Releases
Model registry
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Erik Frisk
MLTestSelection
Commits
f0c9988d
Commit
f0c9988d
authored
7 years ago
by
Erik Frisk
Browse files
Options
Downloads
Patches
Plain Diff
Slight cleanup of code and added SingleFaultIsolation.m to src directory
parent
9f4db90c
Branches
Branches containing commit
No related tags found
No related merge requests found
Changes
3
Hide whitespace changes
Inline
Side-by-side
Showing
3 changed files
README.md
+1
-1
1 addition, 1 deletion
README.md
code/main.m
+82
-72
82 additions, 72 deletions
code/main.m
code/src/SingleFaultIsolation.m
+14
-0
14 additions, 0 deletions
code/src/SingleFaultIsolation.m
with
97 additions
and
73 deletions
README.md
+
1
−
1
View file @
f0c9988d
This repository contains data and code to generate results from the paper
This repository contains data and code to generate results from the paper
_Residual Selection for Consistency Based Diagnosis Using Machine Learning Models_
_Residual Selection for Consistency Based Diagnosis Using Machine Learning Models_
by Erik Frisk and Mattias Krysander, Department of Electrical Engineering, Linköping Univeristy, Sweden
by Erik Frisk and Mattias Krysander, Department of Electrical Engineering, Linköping University, Sweden
\ No newline at end of file
This diff is collapsed.
Click to expand it.
code/main.m
+
82
−
72
View file @
f0c9988d
% This script produces the results shown in:
% This script produces the results shown in:
%
%
% Erik Frisk and Mattias Krysander (2018). Resid
a
ul Selection for
% Erik Frisk and Mattias Krysander (2018). Residu
a
l Selection for
% Consistency Based Diagnosis Using Machine Learning Models. In Safeprocess
% Consistency Based Diagnosis Using Machine Learning Models. In Safeprocess
% 2018, Warsaw, Polen.
% 2018, Warsaw, Polen.
%
%
...
@@ -52,47 +52,48 @@ plotResiduals(absdata)
...
@@ -52,47 +52,48 @@ plotResiduals(absdata)
clear
absdata
clear
absdata
%% Plot Fig. 5: Probability of residual alarm given a given mode
%% Plot Fig. 5: Probability of residual alarm given a given mode
no_residuals
=
size
(
data
.
res
,
2
);
no_residuals
=
size
(
data
.
res
,
2
);
all_residuals
=
[
1
:
no_residuals
]
;
% select all residuals
all_residuals
=
1
:
no_residuals
;
% select all residuals
plotDecisionMatrix
(
all_residuals
,
data
);
plotDecisionMatrix
(
all_residuals
,
data
);
clear
no_residuals
ans
clear
no_residuals
ans
%% Plot Fig 6: Structural isolability of all available residuals
%% Plot Fig 6: Structural isolability of all available residuals
load
model
.
mat
;
load
model
.
mat
;
figure
;
figure
(
16
)
;
model
.
IsolabilityAnalysisFSM
(
data
.
fsm
(:,
[
2
:
8
]
),
'permute'
,
0
);
model
.
IsolabilityAnalysisFSM
(
data
.
fsm
(:,
2
:
8
),
'permute'
,
0
);
title
(
'Structural isolability matrix using all available tests'
);
title
(
'Structural isolability matrix using all available tests'
);
ylabel
(
'Injected fault'
,
'FontWeight'
,
'bold'
);
ylabel
(
'Injected fault'
,
'FontWeight'
,
'bold'
);
xlabel
(
'Diagnsoed fault'
,
'FontWeight'
,
'bold'
);
xlabel
(
'Diagnsoed fault'
,
'FontWeight'
,
'bold'
);
clear
ans
clear
ans
%% Create thresholded residuals
%% Create thresholded residuals
thdata
=
data
;
thdata
=
data
;
thdata
.
res
=
abs
(
data
.
res
)
>=
1
;
thdata
.
res
=
abs
(
data
.
res
)
>=
1
;
%% Plot Fig 7: Confusion matrix obtained by consistency-based diagnosis (CBD) using all thresholded residuals
%% Plot Fig 7: Confusion matrix obtained by consistency-based diagnosis (CBD) using all thresholded residuals
figure
;
figure
(
17
)
;
CBDPlots
(
thdata
,
model
,
all_residuals
);
CBDPlots
(
thdata
,
model
,
all_residuals
);
clear
all_residuals
ans
clear
all_residuals
ans
%% Train a random forest of 100 classification trees using the all data.
%% Train a random forest of 100 classification trees using the all data.
trees
=
100
;
trees
=
100
;
rng
(
1
);
% For reproducibility
rng
(
1
);
% For reproducibility
t
=
templateTree
(
'PredictorSelection'
,
'interaction-curvature'
);
t
=
templateTree
(
'PredictorSelection'
,
'interaction-curvature'
);
fprintf
(
'Building random forest classifier ... '
);
fprintf
(
'Building random forest classifier ... '
);
tic
;
tic
;
Mdl
=
fitcensemble
(
single
(
thdata
.
res
),
thdata
.
mode
,
'Method'
,
'bag'
,
...
Mdl
=
fitcensemble
(
single
(
thdata
.
res
),
thdata
.
mode
,
'Method'
,
'bag'
,
...
'NumLearningCycles'
,
trees
,
'Learners'
,
t
);
'NumLearningCycles'
,
trees
,
'Learners'
,
t
);
fprintf
(
' Finished in %.2f sec.\n'
,
toc
);
fprintf
(
' Finished in %.2f sec.\n'
,
toc
);
clear
t
trees
clear
t
trees
%% Plot Fig 8: Random forest classification performance using all tests on out-of-bag data.
%% Plot Fig 8: Random forest classification performance using all tests on out-of-bag data.
Y
=
oobPredict
(
Mdl
);
Y
=
oobPredict
(
Mdl
);
Ival
=
confusionmat
(
categorical
(
thdata
.
mode
),
categorical
(
Y
));
Ival
=
confusionmat
(
categorical
(
thdata
.
mode
),
categorical
(
Y
));
nf
=
8
;
nf
=
8
;
C
=
Ival
.
/(
sum
(
Ival
')'
*
ones
(
1
,
nf
));
C
=
Ival
.
/(
sum
(
Ival
,
2
)
*
ones
(
1
,
nf
));
figure
;
figure
(
18
)
;
b
=
bar3
(
C
);
b
=
bar3
(
C
);
title
(
'Confusion matrix - Validation data set'
)
title
(
'Confusion matrix - Validation data set'
)
set
(
gca
,
'XTickLabel'
,
data
.
modes
,
'TickLabelInterpreter'
,
'none'
)
set
(
gca
,
'XTickLabel'
,
data
.
modes
,
'TickLabelInterpreter'
,
'none'
)
...
@@ -104,8 +105,9 @@ title('Fault Isolation Performance Matrix')
...
@@ -104,8 +105,9 @@ title('Fault Isolation Performance Matrix')
for
finjIdx
=
1
:
nf
for
finjIdx
=
1
:
nf
for
fdiagIdx
=
1
:
nf
for
fdiagIdx
=
1
:
nf
text
(
fdiagIdx
,
finjIdx
,
C
(
finjIdx
,
fdiagIdx
)
+
0.05
,
...
text
(
fdiagIdx
,
finjIdx
,
C
(
finjIdx
,
fdiagIdx
)
+
0.05
,
...
sprintf
(
'%.1f'
,
C
(
finjIdx
,
fdiagIdx
)
*
100
),
'HorizontalAlignment'
,
'center'
)
sprintf
(
'%.1f'
,
C
(
finjIdx
,
fdiagIdx
)
*
100
),
...
'HorizontalAlignment'
,
'center'
)
end
end
end
end
for
k
=
1
:
length
(
b
)
for
k
=
1
:
length
(
b
)
...
@@ -114,15 +116,15 @@ for k = 1:length(b)
...
@@ -114,15 +116,15 @@ for k = 1:length(b)
b
(
k
)
.
FaceColor
=
'interp'
;
b
(
k
)
.
FaceColor
=
'interp'
;
end
end
colormap
(
'Summer'
)
colormap
(
'Summer'
)
view
(
0
,
90
)
view
(
0
,
90
)
clear
Y
Ival
nf
C
b
fdiagIdx
finjIdx
k
zdata
thdata
clear
Y
Ival
nf
C
b
fdiagIdx
finjIdx
k
zdata
thdata
%% Plot Fig 9: Random forest out-of-bag classification error as a function of the number of trees.
%% Plot Fig 9: Random forest out-of-bag classification error as a function of the number of trees.
figure
;
figure
(
19
)
;
oobErr
=
oobLoss
(
Mdl
,
'mode'
,
'cumulative'
);
oobErr
=
oobLoss
(
Mdl
,
'mode'
,
'cumulative'
);
plot
(
oobErr
,
'LineWidth'
,
2
)
plot
(
oobErr
,
'LineWidth'
,
2
)
xlabel
(
'Number of grown trees'
,
'FontWeight'
,
'bold'
);
xlabel
(
'Number of grown trees'
,
'FontWeight'
,
'bold'
);
ylabel
(
'Out-of-bag classification error'
,
'FontWeight'
,
'bold'
);
ylabel
(
'Out-of-bag classification error'
,
'FontWeight'
,
'bold'
);
clear
oobErr
clear
oobErr
%% Estimate residual importance by permuting out-of-bag observations.
%% Estimate residual importance by permuting out-of-bag observations.
...
@@ -132,13 +134,14 @@ importance = oobPermutedPredictorImportance(Mdl);
...
@@ -132,13 +134,14 @@ importance = oobPermutedPredictorImportance(Mdl);
fprintf
(
' Finished in %.2f sec.\n'
,
toc
);
fprintf
(
' Finished in %.2f sec.\n'
,
toc
);
%% Plot Fig 10: Residual importance
%% Plot Fig 10: Residual importance
[
imp
,
residualRanking
]
=
sort
(
importance
,
'descend'
);
[
imp
,
residualRanking
]
=
sort
(
importance
,
'descend'
);
figure
;
figure
(
20
)
;
plot
(
imp
,
'LineWidth'
,
2
)
plot
(
imp
,
'LineWidth'
,
2
)
xticks
(
[
1
:
length
(
residualRanking
)
]
)
xticks
(
1
:
length
(
residualRanking
))
axis
([
0
43
-
0.4
1.6
])
axis
([
0
43
-
0.4
1.6
])
set
(
gca
,
'XTickLabel'
,
residualRanking
,
'TickLabelInterpreter'
,
'none'
,
'FontSize'
,
8
)
set
(
gca
,
'XTickLabel'
,
residualRanking
,
'TickLabelInterpreter'
,
'none'
,
...
'FontSize'
,
8
)
title
(
'Out-of-Bag Permuted Predictor Importance Estimates'
);
title
(
'Out-of-Bag Permuted Predictor Importance Estimates'
);
ylabel
(
'Estimates'
,
'FontWeight'
,
'bold'
);
ylabel
(
'Estimates'
,
'FontWeight'
,
'bold'
);
xlabel
(
'Predictors'
,
'FontWeight'
,
'bold'
);
xlabel
(
'Predictors'
,
'FontWeight'
,
'bold'
);
...
@@ -146,80 +149,87 @@ clear imp
...
@@ -146,80 +149,87 @@ clear imp
%% Performance evaluation of selected tests using consistency based diagnosis.
%% Performance evaluation of selected tests using consistency based diagnosis.
no_samples
=
size
(
data
.
res
,
1
);
% n
no_samples
=
size
(
data
.
res
,
1
);
% n
no_modes
=
numel
(
data
.
modes
);
% m
no_modes
=
numel
(
data
.
modes
);
% m
no_of_samples_per_mode
=
no_samples
/
no_modes
;
% k
no_of_samples_per_mode
=
no_samples
/
no_modes
;
% k
structuralIsolationMatrix
=
model
.
IsolabilityAnalysisFSM
(
data
.
fsm
(:,[
2
:
8
]),
'permute'
,
0
);
structuralIsolationMatrix
=
...
model
.
IsolabilityAnalysisFSM
(
data
.
fsm
(:,
2
:
8
),
'permute'
,
0
);
stim2
=
sum
(((
2.
^
(
0
:
6
))
.*
structuralIsolationMatrix
),
2
);
probabilityMaximumIsolation
=
zeros
(
42
,
no_modes
-
1
);
stim2
=
sum
(((
2.
^
[
0
:
6
])
.*
structuralIsolationMatrix
)
'
);
C
=
cell
(
length
(
residualRanking
),
1
);
probabilityMaximumIsolation
=
zeros
(
42
,
no_modes
-
1
);
isolationerror
=
zeros
(
length
(
residualRanking
),
1
);
C
=
{};
mean_md
=
zeros
(
length
(
residualRanking
),
1
);
fa
=
zeros
(
length
(
residualRanking
),
1
);
for
i
=
1
:
length
(
residualRanking
)
for
i
=
1
:
length
(
residualRanking
)
selected_resid
a
uls
=
residualRanking
(
1
:
i
);
% select the i best residuals
selected_residu
a
ls
=
residualRanking
(
1
:
i
);
% select the i best residuals
dx
=
SingleFaultIsolation
(
data
.
res
(:,
selected_resid
a
uls
),
...
dx
=
SingleFaultIsolation
(
data
.
res
(:,
selected_residu
a
ls
),
...
data
.
fsm
(
selected_resid
a
uls
,
[
2
:
8
]
));
% apply CBD.
data
.
fsm
(
selected_residu
a
ls
,
(
2
:
8
)
));
% apply CBD.
% Compute for each mode the probability of maximum isolation
% Compute for each mode the probability of maximum isolation
dx2
=
sum
(((
2.
^
[
0
:
6
]
)
.*
dx
(:,
[
2
:
end
]
))
'
);
dx2
=
sum
(((
2.
^
(
0
:
6
)
)
.*
dx
(:,
(
2
:
end
))
),
2
);
for
j
=
1
:
no_modes
-
1
% not NF
for
j
=
1
:
no_modes
-
1
% not NF
probabilityMaximumIsolation
(
i
,
j
)
=
sum
(
dx2
(
[
1
:
no_of_samples_per_mode
]
+
...
probabilityMaximumIsolation
(
i
,
j
)
=
sum
(
dx2
(
(
1
:
no_of_samples_per_mode
)
+
...
j
*
no_of_samples_per_mode
)
==
stim2
(
j
))/
no_of_samples_per_mode
;
j
*
no_of_samples_per_mode
)
==
stim2
(
j
))/
no_of_samples_per_mode
;
end
end
% Compute confusion matrix of CBD using the i best residuals
% Compute confusion matrix of CBD using the i best residuals
C
{
i
}
=
CBDPlots
(
data
,
model
,
selected_resid
a
uls
);
C
{
i
}
=
CBDPlots
(
data
,
model
,
selected_residu
a
ls
);
% Compute performance indicators
% Compute performance indicators
isolationerror
(
i
)
=
sum
(
sum
(
abs
(
structuralIsolationMatrix
-
C
{
i
}(
2
:
8
,
2
:
8
)
.
/((
1
-
C
{
i
}(
2
:
end
,
1
))
*
ones
(
1
,
7
)))))/
7
^
2
;
isolationerror
(
i
)
=
sum
(
sum
(
abs
(
structuralIsolationMatrix
-
C
{
i
}(
2
:
8
,
2
:
8
)
.
/((
1
-
C
{
i
}(
2
:
end
,
1
))
*
ones
(
1
,
7
)))))/
7
^
2
;
mean_md
(
i
)
=
mean
(
C
{
i
}(
2
:
end
,
1
));
mean_md
(
i
)
=
mean
(
C
{
i
}(
2
:
end
,
1
));
fa
(
i
)
=
1
-
C
{
i
}(
1
,
1
);
fa
(
i
)
=
1
-
C
{
i
}(
1
,
1
);
end
end
clear
stim2
dx
dx2
no_samples
no_modes
no_of_samples_per_mode
selected_resid
a
uls
C
clear
stim2
dx
dx2
no_samples
no_modes
no_of_samples_per_mode
selected_residu
a
ls
C
clear
structuralIsolationMatrix
i
j
clear
structuralIsolationMatrix
i
j
%% Plot Fig 11: False alarm probability, mean missed detection probability,
%% Plot Fig 11: False alarm probability, mean missed detection probability,
% and aggregated isolation error as a function of selected tests.
% and aggregated isolation error as a function of selected tests.
figure
;
figure
(
21
)
;
plot
([
mean_md
;
fa
;
isolationerror
]
'
,'
LineWidth
'
,
2
)
plot
([
mean_md
,
fa
,
isolationerror
],
'LineWidth'
,
2
)
hold
hold
selected_no_of_resid
a
uls
=
[
12
14
26
27
];
selected_no_of_residu
a
ls
=
[
12
14
26
27
];
plot
(
selected_no_of_resid
a
uls
,
isolationerror
(
selected_no_of_resid
a
uls
),
'x'
,
...
plot
(
selected_no_of_residu
a
ls
,
isolationerror
(
selected_no_of_residu
a
ls
),
...
'LineWidth'
,
2
,
'MarkerSize'
,
8
,
'Color'
,
'k'
)
'x'
,
'LineWidth'
,
2
,
'MarkerSize'
,
8
,
'Color'
,
'k'
)
legend
(
'Missed detection probability'
,
'False alarm probability'
,
'Fault isolation errors'
)
legend
(
'Missed detection probability'
,
'False alarm probability'
,
...
xlabel
(
'Number of selected residauls'
,
'FontWeight'
,
'bold'
)
'Fault isolation errors'
)
ylabel
(
'Probability'
,
'FontWeight'
,
'bold'
)
xlabel
(
'Number of selected residuals'
,
'FontWeight'
,
'bold'
)
ylabel
(
'Probability'
,
'FontWeight'
,
'bold'
)
grid
on
grid
on
clear
mean_md
isolationerror
selected_no_of_resid
a
uls
clear
mean_md
isolationerror
selected_no_of_residu
a
ls
%% Plot Fig 12: Consistency based isolation performance per mode as a function of number of selected residuals.
%% Plot Fig 12: Consistency based isolation performance per mode as a function of number of selected residuals.
figure
;
figure
(
22
)
;
plot
([
1
-
fa
;
probabilityMaximumIsolation
']'
,
'LineWidth'
,
2
)
plot
([
1
-
fa
,
probabilityMaximumIsolation
],
'LineWidth'
,
2
)
l
=
legend
(
data
.
modes
{
1
:
end
});
l
=
legend
(
data
.
modes
{
1
:
end
});
set
(
l
,
'Interpreter'
,
'none'
)
set
(
l
,
'Interpreter'
,
'none'
)
xlabel
(
'Number of selected residuals'
,
'FontWeight'
,
'bold'
)
xlabel
(
'Number of selected residuals'
,
'FontWeight'
,
'bold'
)
ylabel
(
'Probability of maximum isolation performance'
,
'FontWeight'
,
'bold'
)
ylabel
(
'Probability of maximum isolation performance'
,
'FontWeight'
,
'bold'
)
grid
on
grid
on
clear
probabilityMaximumIsolation
fa
l
clear
probabilityMaximumIsolation
fa
l
%% Plot Fig 13: Consistency-based diagnosis results obtained when running different numbers k of most important residuals.
%% Plot Fig 13: Consistency-based diagnosis results obtained when running different numbers k of most important residuals.
% Plots the cases when using 10, 12, 26, and 27 residuals:
% Plots the cases when using 10, 12, 26, and 27 residuals:
selected_no_of_resid
a
uls
=
[
10
12
26
27
];
selected_no_of_residu
a
ls
=
[
10
12
26
27
];
figure
;
figure
(
23
)
;
C
=
{}
;
C
=
cell
(
length
(
selected_no_of_residuals
),
1
)
;
for
i
=
1
:
length
(
selected_no_of_resid
a
uls
)
for
i
=
1
:
length
(
selected_no_of_residu
a
ls
)
subplot
(
2
,
2
,
i
);
subplot
(
2
,
2
,
i
);
selected_resid
a
uls
=
residualRanking
(
1
:
selected_no_of_resid
a
uls
(
i
));
selected_residu
a
ls
=
residualRanking
(
1
:
selected_no_of_residu
a
ls
(
i
));
C
{
i
}
=
CBDPlots
(
data
,
model
,
selected_resid
a
uls
);
C
{
i
}
=
CBDPlots
(
data
,
model
,
selected_residu
a
ls
);
title
([
'No of tests: '
num2str
(
selected_no_of_resid
a
uls
(
i
))]);
title
([
'No of tests: '
num2str
(
selected_no_of_residu
a
ls
(
i
))]);
end
end
clear
C
i
clear
C
i
%% Plot structural isolability using the k best residuals. Compare with previous plot.
%% Plot structural isolability using the k best residuals. Compare with previous plot.
figure
;
figure
(
24
);
for
i
=
1
:
length
(
selected_no_of_residauls
)
for
i
=
1
:
length
(
selected_no_of_residuals
)
subplot
(
2
,
2
,
i
)
subplot
(
2
,
2
,
i
)
selected_residauls
=
residualRanking
(
1
:
selected_no_of_residauls
(
i
));
selected_residuals
=
residualRanking
(
1
:
selected_no_of_residuals
(
i
));
model
.
IsolabilityAnalysisFSM
(
data
.
fsm
(
selected_residauls
,[
2
:
8
]),
'permute'
,
0
);
% full str isolability
model
.
IsolabilityAnalysisFSM
(
data
.
fsm
(
selected_residuals
,
(
2
:
8
)),
...
title
([
'No of tests: '
num2str
(
selected_no_of_residauls
(
i
))])
'permute'
,
0
);
% full str isolability
title
([
'No of tests: '
num2str
(
selected_no_of_residuals
(
i
))])
end
end
%clear selected_no_of_resid
a
uls selected_resid
a
uls residualRanking i ans
%clear selected_no_of_residu
a
ls selected_residu
a
ls residualRanking i ans
This diff is collapsed.
Click to expand it.
code/src/SingleFaultIsolation.m
0 → 100644
+
14
−
0
View file @
f0c9988d
function isol=SingleFaultIsolation( res, FSM )
N = size(res,1);
nf = size(FSM,2);
isol = zeros(N,nf + 1);
for k=1:N
alarms = abs(res(k,:))>=1;
if any(alarms)
isol(k,:) = [false all(FSM(alarms,:),1)];
else
isol(k,:) = [true boolean(zeros(1,nf))];
end
end
end
\ No newline at end of file
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment