A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms

Recent discoveries of vulnerabilities in the design and implementation of Byzantine fault-tolerant protocols underscore the need for testing and exploration techniques to ensure their correctness. While there has been some recent effort for automated test generation for BFT protocols, there is no benchmark framework available to systematically evaluate their performance. We present ByzzBench, a benchmark framework designed to evaluate the performance of testing algorithms in detecting Byzantine fault tolerance bugs. ByzzBench is designed for a standardized implementation of BFT protocols and their execution in a controlled testing environment. It controls the nondeterminism in the concurrency, network, and process faults in the protocol execution, enabling the functionality to enforce particular execution scenarios and thereby facilitating the implementation of testing algorithms for BFT protocols.

Download publication

View on Dagstuhl Digital Library

Citation

João Miguel Louro Neto and Burcu Kulahcioglu Ozkan. A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms (Tool Paper). In 6th International Workshop on Formal Methods for Blockchains (FMBC 2025). Open Access Series in Informatics (OASIcs), Volume 129, pp. 13:1-13:11, Schloss Dagstuhl – Leibniz-Zentrum für Informatik (2025) https://doi.org/10.4230/OASIcs.FMBC.2025.13 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

@InProceedings{louroneto_et_al:OASIcs.FMBC.2025.13,
  author    = {Louro Neto, Jo\~{a}o Miguel and Kulahcioglu Ozkan, Burcu},
  title     = {{A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms}},
  booktitle = {6th International Workshop on Formal Methods for Blockchains (FMBC 2025)},
  pages     = {13:1--13:11},
  series    = {Open Access Series in Informatics (OASIcs)},
  ISBN      = {978-3-95977-371-3},
  ISSN      = {2190-6807},
  year      = {2025},
  volume    = {129},
  editor    = {Marmsoler, Diego and Xu, Meng},
  publisher = {Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  address   = {Dagstuhl, Germany},
  URL       = {https://drops.dagstuhl.de/entities/document/10.4230/OASIcs.FMBC.2025.13},
  URN       = {urn:nbn:de:0030-drops-230406},
  doi       = {10.4230/OASIcs.FMBC.2025.13},
  annote    = {Keywords: Byzantine Fault Tolerance, BFT Protocols, Automated Testing}
}