With upcoming blockchain infrastructures, world-spanning Byzantine consensus is getting practical and necessary. In geographically distributed systems, the pace at which consensus is achieved is limited by the heterogenous latencies of connections between replicas. If deployed on a wide-area network, consensus-based systems benefit from weighted replication, an approach that utilizes extra replicas and assigns higher voting power to well connected replicas. This enables more choice in quorum formation and replicas can leverage proportionally smaller quorums to advance, thus decreasing consensus latency. However, the system needs a solution to autonomously adjust to its environment if network conditions change or faults occur. We present Adaptive Wide-Area REplication (AWARE), a mechanism which improves the geographical scalability of consensus with nodes being widely spread across the world. Essentially, AWARE is an automated and dynamic voting weight tuning and leader positioning scheme, which supports the emergence of fast quorums in the system. It employs a reliable self-monitoring process and provides a prediction model seeking to minimize the system's consensus latency. In experiments using several AWS EC2 regions, AWARE dynamically optimizes consensus latency by self-reliantly finding a fast weight configuration yielding latency gains observed by clients located across the globe.
In geo-replicated systems, the heterogeneous latencies of connections between replicas limit the system's ability to achieve consensus fast. State machine replication (SMR) protocols can be refined for their deployment in wide-area networks by using a weighting scheme for active replication that employs additional replicas and assigns higher voting power to faster replicas. Utilizing more variability in quorum formation allows replicas to swiftly proceed to subsequent protocol stages, thus decreasing consensus latency. However, if network conditions vary during the system's lifespan or faults occur, the system needs a solution to autonomously adjust to new conditions. We incorporate the idea of self-optimization into geographically distributed, weighted replication by introducing AWARE, an automated and dynamic voting weight tuning and leader positioning scheme. AWARE measures replica-to-replica latencies and uses a prediction model, thriving to minimize the system's consensus latency. In experiments using different Amazon EC2 regions, AWARE dynamically optimizes consensus latency by self-reliantly finding a fast weight configuration yielding latency gains observed by clients located across the globe.
Novel Byzantine fault-tolerant (BFT) state machine replication protocols improve scalability for their practical use in distributed ledger technology, where hundreds of replicas must reach consensus. Assessing that BFT protocol implementations meet their performance expectations requires careful evaluation. We propose a new methodology using scalable network simulations to predict BFT protocol performance. Our simulation architecture allows for the integration of existing BFT implementations without modification or re-implementation, offering a cost-effective alternative to large-scale cloud experiments. We validate our method by comparing simulation results with real-world cloud deployments, showing that simulations can accurately predict performance at larger scales when network limitations dominate. In our study, we applied this methodology to assess the performance of several “blockchain-generation” BFT protocols, including HotStuff, Kauri, Narwhal & Tusk, and BullShark, under realistic network conditions (with constrained 25 Mbit/s bandwidth) and induced faults. Kauri emerges as the top performer, achieving 6,742 operations per second (op/s) with 128 replicas, outperforming BullShark (2,318 op/s) and Tusk (1,952 op/s). HotStuff, using secp256k1 and BLS signatures, reaches 494 op/s and 707 op/s, respectively, demonstrating the efficiency of BLS-signature aggregation for saving bandwidth. This study demonstrates that state-of-the-art asynchronous BFT protocols can achieve competitive throughput in large-scale, real-world scenarios.
Blockchain technology sparked renewed interest in planetary-scale Byzantine fault-tolerant (BFT) state machine replication (SMR). While recent works predominantly focused on improving the scalability and throughput of these protocols, few of them addressed latency. We present Mercury, a novel transformation to autonomously optimize the latency of quorum-based BFT consensus. Mercury employs a dual resilience threshold that enables faster transaction ordering when the system contains few faulty replicas. Mercury allows forming compact quorums that substantially accelerate consensus using a smaller resilience threshold. Nevertheless, Mercury upholds standard SMR safety and liveness guarantees with optimal resilience, thanks to its judicious use of a dual operation mode and BFT forensics techniques. Our experiments spread tens of replicas across continents and reveal that Mercury can order transactions with finality in less than 0.4s, half the time of a PBFT-like protocol (optimal in terms of number of communication steps and resilience) in the same network. Furthermore, Mercury matches the latency of running its base protocol on theoretically optimal internet links (transmitting at 67% of the speed of light).