A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms

Abstract

Recent discoveries of vulnerabilities in the design and implementation of Byzantine fault-tolerant protocols underscore the need for testing and exploration techniques to ensure their correctness. While there has been some recent effort for automated test generation for BFT protocols, there is no benchmark framework available to systematically evaluate their performance. We present ByzzBench, a benchmark framework designed to evaluate the performance of testing algorithms in detecting Byzantine fault tolerance bugs. ByzzBench is designed for a standardized implementation of BFT protocols and their execution in a controlled testing environment. It controls the nondeterminism in the concurrency, network, and process faults in the protocol execution, enabling the functionality to enforce particular execution scenarios and thereby facilitating the implementation of testing algorithms for BFT protocols.

João Neto and Burcu Kulahcioglu Ozkan. 2025. A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms. In 6th International Workshop on Formal Methods for Blockchains (FMBC 2025). 13:1–13:11.

J. Neto and B. Kulahcioglu Ozkan, "A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms," in 6th International Workshop on Formal Methods for Blockchains (FMBC 2025), 2025, pp. 13:1–13:11.

Neto, J., & Kulahcioglu Ozkan, B. (2025). A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms. In 6th International Workshop on Formal Methods for Blockchains (FMBC 2025) (pp. 13:1–13:11).

@inproceedings{2025-FMBC,
  author       = {Neto, Jo{\~a}o and Kulahcioglu Ozkan, Burcu},
  title        = {A Benchmark Framework for Byzantine Fault Tolerance Testing Algorithms},
  booktitle    = {6th International Workshop on Formal Methods for Blockchains (FMBC 2025)},
  series       = {Open Access Series in Informatics (OASIcs)},
  volume       = {129},
  pages        = {13:1--13:11},
  year         = {2025},
  publisher    = {Schloss Dagstuhl -- Leibniz-Zentrum f{\"u}r Informatik},
  doi          = {10.4230/OASIcs.FMBC.2025.13},
}