MONOKH

نخبه break things
Github @monokh Twitter @mo_nokh

Bitcoin From Scratch - Part 3 - The Network

25 August 2020

In Part 1, we discovered the data structures involved in bitcoin and the role they play in the system.

In Part 2, we combined these concepts into a “node”, which handled transactions and was able to include them in blocks.

This part will produce a running network and some tools to observe it:

So far, running 2 nodes of this software would produce 2 different branches from the genesis block. This is because the nodes do not know about each other. To solve this, we need to implement a network layer as a way for the nodes to communicate.

It turns out, to go beyond a local in memory node, there is a huge amount of architecture involved. The following pieces will be covered:

Due to the huge amount of code changes, we will mostly stick to explaining the concepts in this part.

As always the code can be found on GitHub: https://github.com/monokh/nibble

Storage

The first step on the path of making our node resilient is ensuring that any data produced is stored, such that restarting the node will not take us all the way back to the genesis block.

Due to the nature of the node’s operation, we need a database that is able to do quick storage and lookups. Additionally, it would be preferable if the database was embeddable and did not require setup to ease the process of starting a node. RocksDB fulfills these requirements, it is a key value database that is stored on disk.

Bitcoin uses a combination of LevelDB and binary files. Data Storage

Let’s go over the database collections and their purpose:

Blocks

blockHash → { block }

We need to store the blocks that the node knows about. Such that we can build on them when mining and to provide this information to other nodes/clients as requested.

Blocks Metadata

We need a way to figure out what the latest block hash and the current block height is. To achieve this using only the blocks database, we would need to retrieve all the blocks and sort them according to their contents. To ease this burden, the metadata database provides some useful indexes:

Balances

publicKey → balance

Similar to the metadata stored for blocks, we need to index balances for fast retrieval. Without this, we’d need to iterate through every block and transaction to aggregate a public key's final balance.

In Bitcoin, the concept of balance is not core to the data structures. Bitcoin's transactions have inputs and outputs. Outputs (or rather unspent outputs) are the amounts of Bitcoin your public key can spend. These outputs are saved on the node and can be aggregated easily to calculate the balance.

The storage code is very straightforward. Here's an example of updating the database when a new block is being added.

pub fn set_latest_block(db: &DB, block_hash: &String, height: u32) -> Result<(), String> {
    db.put(b"latest_block_hash", block_hash.clone()).map_err(|e| e.to_string())?;
    db.put(block_hash.clone(), height.to_string()).map_err(|e| e.to_string())?;
    db.put(height.to_string(), block_hash.clone()).map_err(|e| e.to_string())?;
    return Ok(());
}

P2P Wire Protocol

As briefly mentioned, to produce a network, nodes need to communicate with each other and reach some form of consensus.

To facilitate this communication, we first need to describe a language for these nodes to use.

const MESSAGE_NEW_PEER: &str = "NEW_PEER";
const MESSAGE_PING: &str = "PING";

const MESSAGE_GET_BLOCK: &str = "GET_BLOCK";
const MESSAGE_GET_BLOCKS: &str = "GET_BLOCKS";

const MESSAGE_NEW_BLOCK: &str = "NEW_BLOCK";
const MESSAGE_NEW_TRANSACTION: &str = "NEW_TRANSACTION";

A TCP server runs alongside the node that is able to handle and respond to these messages.

impl P2PServer {
    fn handle_get_blocks (&mut self) -> Result  {
        let block_hashes = storage::get_block_hashes(&storage::db::blocks_md(true))?;
        return Ok(serde_json::to_string(&block_hashes).unwrap());
    }
		...
}

The node will also need to sync up with the network when it starts:

  1. Send NEW_PEER for every node it knows about. If the node responds with a list of peers. Do NEW_PEER for each of those
  2. Send GET_BLOCKS to each peer. Calculate the difference between the retrieved block hashes and local latest_block_hash. If there are differences, Retrieve the missing blocks with GET_BLOCK and process them node.processBlock(block).

By the end of this process, the node knows about every peer in the network and has synced it’s version of the blockchain with the other nodes, all the while validating that the blocks are valid according to the Consensus rules.

Publish

From this point on, the nodes are ready to publish new information to each other. For example, if the node successfully mines a block, it will broadcast it to all of it’s peers:

pub fn publish (p2p_data_ref: Arc>, req: String, data: String) -> Result<(), Box>  {
    let p2p_data = p2p_data_ref.lock().unwrap();
    for peer in &p2p_data.peers {
        if let Err(_e) = send(peer, req.clone(), Some(data.clone())) {
            println!("{}", format!("Failed to publish to peer: {}", peer).red());
        }
    }
    Ok(())
}

pub fn publish_block(p2p_data_ref: Arc>, block: block::Block) -> Result<(), Box> {
    publish(p2p_data_ref, MESSAGE_NEW_BLOCK.to_string(), serde_json::to_string(&block)?)
}

Bootstrap Nodes

One question that may popup is, how does a node become aware of other nodes in the first place?

There is an inherent issue in P2P networking: it is not feasible to troll the vast internet in search of peers. We need a way to bootstrap into the network and find a set of other peers. This it the role of the “bootstrap” node. In our case, these are just regular nodes that are highly available and it’s address is hardcoded into the node clients.

New nodes will announce their presence to the bootstrap node by sending the NEW_PEER message and will receive other peers to connect to.

RPC

The P2P protocol is very lightweight and bare on purpose so that it is able to handle the high volume of interactions.

We also want a convenient interface to interact with the node. For this, blockchains generally use JSON RPC(“Remote Procedure Calls”) servers.

The RPC server provides methods for retrieving data as well as sending transactions using the node’s wallet.

A call looks like this:

{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "getbalance",
    "params": [
        "03a23b2369525bf4094db602cd35fa3af24b440a2daf4a3c5e92a54196395d4b41"
    ]
}

This interface is used under the hood to implement the CLI as well as the web interface.

Threading

Looking back on Part 2, we went from a local node that simply found PoW for a block and printed it to the console, to a network connected node that is running several processes at the same time. These processes are:

It’s important that all of these processes run in their own thread. Threading introduced a few challenges:

MPSC Channels were used to publish blocks and transactions as the occurred on the node. Mutexes were used to keep the mempool and peer list thread safe while they are being updated.

Config

A simple config file has been implemented to allow multiple nodes with different configurations to be run on the same machine.

rpc_port = 2337
tcp_port = 2338
web_port = 2339
data_dir = "node2_data"
miner_enabled = true
bootstrap_node = "127.0.0.1:1338"

Notice that the bootstrap node’s address is defined here.

Still not complete

It turns out that building a full robust implementation is extremely time consuming. While we have something here that is valuable for demonstration purposes, there are still a few strides to reach a reliable network. Some of these are:

Web UI

There is a web interface that runs by default on port 1339. Using this interface, you can send transactions and explore the blockchain.

Closing thoughts

This was a fun project. It was disappointing to discontinue it before there was an operational public network. It’s a testament to the meticulous care that has gone into the Bitcoin software.