distrifein
Broadcast and Consensus Protocols
distrifein is a distributed decentralized focussed project with elements of networking and databases involved. This project encapsulates critical concepts from distributed systems, networking, and event-driven architecture, providing a modular foundation for building reliable distributed applications.
- Distributed Systems:
- Links:
- Reasonably Reliable P2P links with TCP(approximately Perfect Links)
- Broadcast Methods:
- Best Effort Broadcast
- Reliable Broadcast
- Uniform Reliable Broadcast
- Failure Detectors:
- Heartbeat FD
- Event System
- Shared Event Bus
- Subscription and Publishing
- Links:
- Networking:
- Channels with TCP based server: Handles incoming clients
- TCP based transfer of data with sockets
Let’s start with the architecture of the project. The architecture is based on the concept of processes and layers. The processes are independent and communicate with each other via events. The layers are designed to be modular and only communicate via events.
The elements of a process are mentioned below where I assume we have crash stop processes:
We have these layers also in every process I run where the different layers communicate with each other via events which are managed by shared event bus. All components are designed to be modular and only communicating via events.
One of the positive things about this project it emulates this below mentioned architecture really well where each component is independent, modular and communicates via event based system. We have implemented atleast three broadcast methods namely: Best Effort Broadcast, Reliable Broadcast, Uniform Reliable Broadcast. We also have failure detection via heartbeat based method. Channels and links are implemented with TCP.
Date: 5/12/25
Finally I got around to transferring images via broadcast methods like best effort broadcast, reliable broadcast and uniform reliable broadcast. Image files are in PPM from which when path is specified it is read and send via the event system to low level components. For uniform reliable broadcast, it would be app->urb->beb->link and for reliable broadcast it would be app->rb->beb->link. For best effort broadcast, it would be app->beb->link. The image is then sent to the link layer and then to the other process via TCP.
Each of these methods have their own protocol albeit with the implementations utilizing the same modular components with the event system. There has been a major change in the message format where before the messsages were fragmented in sense that their were many types of messages. Now we have a single message format which handles all the types of messages. It offers a lot of flexibility and modularity to the project albeit with a bit of complexity and space.
Example: image transfer via uniform reliable broadcast :
Displays the image of the uniform reliable broadcast protocol in action where I specify the file to be sent. It first puts the image in pending for itself and then waits for acks from the other nodes to deliver it. SID is the current sender id and Org SID is the original sender id. We wait for acks for all the correct processes to deliver the image before being saved to the disk:
Results: image transfer via uniform reliable broadcast:
I had 3 processes running on my local machine, each with a different port number. The processes were able to communicate with each other and transfer images using the uniform reliable broadcast protocol. Each node had a unique ID and a folder to store the images received. The images were transferred successfully which is shown below:
The transferred files on node 1 are shown below, where each node sent 1 photo to all other nodes. Terrain photo was sent by Node 2, Dead Pixel Correction photo was sent by node 1 and finally the distrifein photo was sent by node 0.
