Skip to content
Snippets Groups Projects
Commit e5c02cfc authored by Mariusz Wzorek's avatar Mariusz Wzorek
Browse files

Update README.MD

parent 4a38d9e2
No related branches found
No related tags found
No related merge requests found
...@@ -14,7 +14,7 @@ A full open-source release of the HFKN knowledge framework, including ROS2 drive ...@@ -14,7 +14,7 @@ A full open-source release of the HFKN knowledge framework, including ROS2 drive
Run the docker Running the docker
-------------- --------------
To fetch a prebuild docker: To fetch a prebuild docker:
...@@ -29,38 +29,39 @@ Then to run it: ...@@ -29,38 +29,39 @@ Then to run it:
make run make run
``` ```
The detail of each benchmark is described in our paper. The following is documentation on how to run the benchmarks and interpret the resulting data. The benchmark results are created in a folder `results` accessible from outside of the container. The command for the benchmarks are: Details of each benchmark are described in the paper. The following is documentation on how to run the benchmarks and interpret the resulting data.
The benchmark results are created in a folder `results` accessible from outside of the container. The commands for the benchmarks are:
- `run_benchmark_triplestore`: benchmark the triple store with or without versioning (used in 4.1 Evaluation of the versioning overhead)" - `run_benchmark_triplestore`: benchmark the triple store with or without versioning (used in 4.1 Evaluation of the versioning overhead)"
It outputs several csv files of the form `bts_[optype]_[has_vers]_[number]` where `optype`: It outputs several csv files of the form `bts_[optype]_[has_vers]_[number]` where `optype`:
* `in_del` meaning triples are inserted and deleted for each revisions * `in_del` meaning triples are inserted and deleted for each revision
* `only_in` meaning triples are inserted for each revisions * `only_in` meaning triples are inserted for each revision
* `only_del` meaning triples are deleted for each revisions * `only_del` meaning triples are deleted for each revision
And `has_ver` indicate if versioning is enabled (`versioning`), disabled (`no_versioning`) and if versioning is enabled, but signing is disabled (`versioning_no_signing`) And `has_ver` indicate if versioning is enabled (`versioning`), disabled (`no_versioning`) and if versioning is enabled, but signing is disabled (`versioning_no_signing`)
The benchmark is run 10 times, the `number` indicates the iteration. The benchmark is run 10 times, the `number` indicates the iteration.
The columns in CSV: The columns in CSV:
* `time` indicates the time it took to compute a revision * `time` indicates the time it took to compute a revision
* `insertions_count` the number of inserted triple since start * `insertions_count` the number of inserted triples since start
* `insertions_time` the time it took to insert the triples * `insertions_time` the time it took to insert the triples
* `deletions_count` the number of removed triple since start * `deletions_count` the number of removed triples since start
* `deletions_time` the time it took to deleteq the triples * `deletions_time` the time it took to delete the triples
- `run_benchmark_rebase_raw`: benchmark for the rebase algorithm (used in 4.3 Evaluation of the Rebase Algorithm)" - `run_benchmark_rebase_raw`: benchmark for the rebase algorithm (used in 4.3 Evaluation of the Rebase Algorithm)"
It outputs `rebase_raw.cvs` with the following three columns: It outputs `rebase_raw.cvs` with the following three columns:
* `changes`: number of changes per revisions (e.g. number of triples added or removed) * `changes`: number of changes per revision (e.g. number of triples added or removed)
* `revisions`: number of revisions to rebase * `revisions`: number of revisions to rebase
* `average_time`: average time it took to compute the rebase * `average_time`: average time it took to compute the rebase
- `run_many_benchmark_revision_throughput_limit` : benchmark the maximum amount of revision that can be created for many different agents and documents (used in 4.5 Evaluating the Maximum Revision Creation Rate)" - `run_many_benchmark_revision_throughput_limit` : benchmark the maximum amount of revisions that can be created for many different agents and documents (used in 4.5 Evaluating the Maximum Revision Creation Rate)"
It outputs `throughput_limit.cvs` with the following columns: It outputs `throughput_limit.cvs` with the following columns:
* `agents` the number of agents * `agents` the number of agents
* `documents` the number of documents * `documents` the number of documents
* `triples` the number of triples added in each revisions * `triples` the number of triples added in each revision
* `average_time` the average time for creating revisions and merging * `average_time` the average time for creating revisions and merging
Build the docker Building the docker
---------------- ----------------
In case you want to rebuild the docker: In case you want to rebuild the docker:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment