PHASE 03 Distsys · ~3 hours

Distributed Systems

Blockchain is "a replicated log agreed on by mutually distrustful parties". You've built replicated logs (Postgres replicas, Kafka). The new variable is Byzantine — some replicas are lying.

1. The models you already know

2. CAP, FLP, and what you actually trade off

• CAP — under a network partition, you can keep either Consistency or Availability. Blockchains pick CP (the network halts finalization rather than accept conflicting writes).
• FLP impossibility — in an async network, no deterministic consensus with even 1 failure. Blockchain dodges it by using probabilistic finality (PoW) or by assuming partial synchrony (PoS/BFT).

3. Byzantine Fault Tolerance (BFT)

The classic result: you can tolerate up to f malicious nodes if you have 3f + 1 total. Need a 2/3 supermajority to decide.

Ethereum PoS (Gasper) is BFT-flavored: a block is finalized once 2/3 of staked ETH attests to it. Reverting requires either (a) burning 1/3 of the stake or (b) breaking the crypto. This is "economic finality".

4. Proof of Work vs Proof of Stake

5. P2P networking: gossip and DHTs

Ethereum's execution layer uses devp2p; the consensus layer uses libp2p with gossipsub. Gossip = each node tells k neighbors about a new message; with k=8 a 10k-node net saturates in ~4 hops.

# Gossip pseudo-code
on receive(msg):
  if seen(msg.id): return
  mark_seen(msg.id)
  for peer in pick_random(peers, k=8):
      send(peer, msg)

6. Project — simulate a P2P mesh

Spin up 10 Node.js processes, each listening on a different port. On boot, each picks 3 random peers and opens sockets. When you POST a message to any one, it should reach all 10 via gossip within a second. Log the hop count.

// node peer.js <myPort> <peer1> <peer2> ...
const net = require('net'), http = require('http');
const crypto = require('crypto');
const [,, myPort, ...peers] = process.argv;
const seen = new Set();

const forward = (msg) => {
  for (const p of peers) {
    const s = net.connect(+p, 'localhost', () => s.end(JSON.stringify(msg)));
    s.on('error', () => {});
  }
};

net.createServer(sock => {
  let buf = '';
  sock.on('data', d => buf += d);
  sock.on('end', () => {
    const msg = JSON.parse(buf);
    if (seen.has(msg.id)) return;
    seen.add(msg.id);
    console.log(myPort, 'got', msg.id, 'hop', msg.hop);
    forward({ ...msg, hop: msg.hop + 1 });
  });
}).listen(+myPort);

http.createServer((req, res) => {
  const msg = { id: crypto.randomUUID(), hop: 0, body: 'hi' };
  seen.add(msg.id); forward(msg); res.end('ok');
}).listen(+myPort + 1000);

7. Docker + libp2p (optional)

For a "real" taste, run js-libp2p nodes in Docker containers on a shared network. This is exactly the stack Polkadot and IPFS use.

Quiz