Bitcoin From Scratch - Part 1

3 July 2020

Understanding the fundamental building blocks(😉) of Bitcoin can be daunting for some developers. I find it difficult to meaningfully understand a system without knowing how everything fits together.

A good way to go through this process is to lay down, however rudimentary, a set of processes by which the system operates. Hence the purpose of this guide is to explain through simple pieces of code, how Bitcoin works.

Note: The intention is not to create a real implementation of the Bitcoin protocol. It also does not aim to explain every concept. Look for the embedded links if you find a concept you need to go over. Also look out for notes where concepts have been simplified and differ from the real Bitcoin implementation.

Hopefully by the end of this series, we should have a bitcoin-like protocol anyone can run.

The code written until this point is available here.

From a very high level. This is bitcoin:

let keypair = crypto::KeyPair::new();
let keypair2 = crypto::KeyPair::new();

let tx1 = create_signed_tx(&keypair, keypair2.public_key, 123);
let tx2 = create_signed_tx(&keypair2, keypair.public_key, 75);

let txs = vec![tx1, tx2];

let proposed_block = block::ProposedBlock {
    transactions: txs,
};

println!("Mining block with {} txs.", proposed_block.transactions.len());
let block = proposed_block.mine(3); // Difficulty 3 i.e. hash must begin with "000"
println!("Mined block: {:#?}", block);

We can split what we see here into the following categories:

Keys
Transactions
Blocks
Mining

Keys

Keys are required as a means to achieve ownership. In other words, you need to have the key to spend a certain amount of bitcoin.

pub struct KeyPair {
    pub public_key: key::PublicKey,
    private_key: key::SecretKey,
}

Your keys actually consist of 2 parts. The private key and the public key. The public key can be shared with the world and is used to receive bitcoin, whereas the private key is something you keep secure as it can be used to spend your bitcoin. More on public key cryptography.

Creating keys

In bitcoin, eliptic curves are used for public key cryptography - specifically secp256k1. In essense, using a (psuedo)random number we can generate a pair of keys that can be thought of as a user's wallet.

let mut rand = rand::OsRng::new().unwrap();
let secp = Secp256k1::new();
let (private_key, public_key) = secp.generate_keypair(&mut rand);
KeyPair {
    secp,
    public_key,
    private_key,
}

Transaction

Transactions are the smallest unit of the Bitcoin system. A transaction is a data structure which encodes a payment of x bitcoins from a to b.

In Bitcoin, transactions are not as simple as transferring a balance (account based), but rather a collection of inputs and outputs. This is generally called the "UTXO model".

pub struct Transaction {
    pub from: PublicKey,
    pub to: PublicKey,
    pub amount: u32
}

The public keys here are precisely from the key pairs that were generated.

What then prevents anyone from transferring on behalf of others? Public key cryptography. To be a valid transaction and be accepted into the network by others, the transaction needs to be "Signed" by the `from` private key.

impl KeyPair {
    pub fn sign(&self, message: &[u8]) -> Signature {
        ...
        self.secp.sign(&message, &self.private_key);
    }
}

Signed transaction

As mentioned previously, private keys exercise control over a number of bitcoins. A's private key is used to sign a transaction so that anyone can verify that the transaction from A to B was in fact authorised by A.

let tx = transaction::Transaction {
    from: keypair.public_key,
    to,
    amount,
};

let sig = keypair.sign(&tx.hash());

return transaction::SignedTransaction{
    transaction: tx,
    sig: sig.to_string(),
};

Block

A block is a collection of transactions. These transactions should be signed and therefore valid.

let tx1 = create_signed_tx(&keypair, keypair2.public_key, 123);
let tx2 = create_signed_tx(&keypair2, keypair.public_key, 75);

let txs = vec![tx1, tx2];

let proposed_block = block::ProposedBlock {
    transactions: txs,
};

In fact, on the bitcoin network, the only data which is replicated across are the blocks. The purpose of the block is to effectively write it's containing transactions into history. It achieves this by mining.

Mining

The process of mining is to do a computation to ensure that there is something at stake. In other words, the miner has to prove they have done some work - "Proof of Work"

To provide the correct proof of work for a block:

Serialise the block
Append a number
Hash it
Increment the number
Go back to 1

The PoW is suffice if the hash begins with a certain number of 0s. The number of 0s required is called difficulty. The more difficulty - the more 0s are required.

In reality, the difficulty is defined on a more granular level. See difficulty on the bitcoin wiki for more information.

let mut nonce: u32 = 0;
let block_string = self.serialize();

loop {
    let block = format!("{}{}", block_string, nonce);
    let block_hash = hex::encode(crypto::sha256(block.clone()));
    if block_hash.starts_with(&"0".repeat(difficulty)) {
        return Block {
            hash: block_hash,
            nonce,
            transactions: self.transactions
        }
    }
    nonce += 1;
}

You should hopefully now see how this all fits together. To recap the high level code initially presented:

Generate keypairs for each user
Using the private key of the user, sign transactions that move bitcoin
Combine these transactions in a "block"
Perform the PoW on this block until the difficulty condition is met

Finally running this program will produce a block including the signed transactions. Take note of the hash. It begins with "000", therefore it is a valid block when difficulty is set to 3.

Mining block with 2 txs.
Mined block: Block {
    hash: "000cc7b72695c1ab8b7969cb400f73d763bce1cafa866b0aab59fe762eb7fbed",
    nonce: 96,
    transactions: [
        SignedTransaction {
            transaction: Transaction {
                from: PublicKey(
                    bf9d4f6c8142956ac683f73ef009369b43aec7d765e18c6ac3b48341bf14ebd9e0a06bec8897e392725dae273eaa51400320130cba64143e61c1f61b3bc4f324,
                ),
                to: PublicKey(
                    c1c6bd81ff1a657ae4053149682502c749fbc4b638e047d36c4bdf637ebcd0e93d0f987638c248c099deef8ec76199c1818f3b387e232a2d8da5de5ca91cf13b,
                ),
                amount: 123,
            },
            sig: "304402206fb0e6ab4a6acfcc0725f048c7fbfed734accc0a4bad864ac161302410ed3f94022035ef86194f8de575658b7e4b3bae2aee193c507d78fe8ba4e43e22bce948271a",
        },
        SignedTransaction {
            transaction: Transaction {
                from: PublicKey(
                    c1c6bd81ff1a657ae4053149682502c749fbc4b638e047d36c4bdf637ebcd0e93d0f987638c248c099deef8ec76199c1818f3b387e232a2d8da5de5ca91cf13b,
                ),
                to: PublicKey(
                    bf9d4f6c8142956ac683f73ef009369b43aec7d765e18c6ac3b48341bf14ebd9e0a06bec8897e392725dae273eaa51400320130cba64143e61c1f61b3bc4f324,
                ),
                amount: 75,
            },
            sig: "30440220752b3ea25bd2ab2b29d44659af665caddd7ce9b54144c41e6adcdfde7ef807fa022066682bdb49666a91fd28e14c05cf8229d668c061f404f218bb62619e4604eb59",
        },
    ],
}

There is much more to Bitcoin to uncover. We have not presented yet:

How blocks are validated (consensus rules)
How it's possible to derive balances.
How blocks are constructed such that they are not reversible
How the peers communicate over the internet

This and much more will be covered in future parts.