Building a blockchain has been an important milestone in my development as a software engineer. If you would like to explore the code or run your own node, please find the project on github.
The Story
The motivation for the project came from experiences at my previous job. I was working at an investment bank in San Francisco writing a report on trends in the blockchain industry. Hours of browsing coindesk and medium posts had given me a vague notion of the inner workings of a distributed ledger, but I was missing something. The basics were not good enough and online resources were limited. I needed more knowledge. So I grabbed my computer and set out to build my own blockchain.
I purposely did not read the bitcoin source code and built my chain from first principals as presented by Satoshi in his original 2009 white-paper. The challenges quickly amassed due to my personal knowledge deficits in the many moving parts of a blockchain. The system spans many disciplines of computer science requiring knowledge of cryptography, data structures, and peer-to-peer networks; not to mention the basics of managing, structuring, and debugging a large codebase.
I leaped head first down the rabbit hole, back filling the knowledge I needed to design my ledger. I needed to create a peer-to-peer network, which meant I was building both a client and a server on the same machine all while writing asynchronous code for the first time. I meticulously thought through my data structures to prevent any possible assaults on my system as there is no guarantee the peers will be running my code or be behaving honestly. Spending hours after work in the dark corner of my living room, I realized I really have a passion for coding. It was at this point I stumbled across the Recurse Center, a retreat for programmers looking to build skills and explore radical ideas. I applied and was elated when I received my acceptance.
I am currently attending from January through March 2019 and it has been even better than expected. I found a group of like-minded people to explore the world of software engineering. I have experienced exponential growth of my development skills and finally put the finishing touches on my blockchain. It is odd to be done with this project after months of hard work, but I am looking forward to diving deeper into programming and fully utilizing the rest of my time at the Recurse Center.
The Technical Details
Xcoin is an unspent transaction model blockchain utilizing proof of work consensus built in python utilizing the twisted framework for peer-to-peer networking. The system is byzantine fault tolerant and testing is conducted with docker containers allowing the easy creation of a multi-peer network.
Containerized nodes coming to consensus on a ledger
A blockchain is a distributed data structure in which a group of nodes work together to maintain a ledger of transactions. The consensus protocol is the algorithm used to reach agreement on the correct order of the transaction.
Data Structures
The project operates with three basic data structures: Transactions, Blocks, and the Ledger.
Transactions: Represent the transfer of value from one account to another. Private key signatures ensure the transactions were sent by the owner of the asset. Transactions reference a previous transaction to prove their value and nodes guarantee that the previous transaction has not already been spent.
Blocks: Store the transactions. The block includes the hash of the previous block and a timestamp creating a temporal dimension to our data structure to prevent double spending of the same asset. The creator of the block is decided by the consensus method discussed below.
Ledger: The ledger stores the blocks and is the object that contains most of the logic for making sure blocks are valid. When a node receives a new block created by someone else, the ledger object validates the block to ensure it is placed in the correct location and all the transactions within the block are valid.
Consensus
The consensus algorithm for this project is proof of work. Proof of work guarantees the nodes eventually reach agreement on the state of the ledger as long as at least 51% are behaving honestly. The algorithm requires the nodes to follow 4 simple rules:
- Solve the hash puzzle
- Always accept a longer chain
- Double check all blocks are valid
- A reward is given for mining a block
The hash puzzle must be solved to publish a block. The puzzle is find a number, called a nonce, such that hashing the nonce concatenated with the hash of the previous block results in a hash with a specific number of leading zeros (ie 00000xxxxxx). For this puzzle, it is hard to find an answer but easy to verify an answer is correct. This means that nodes must expend some energy to publish a block and makes it so there is some time between blocks for transactions to propagate around the network. The difficulty of the puzzle can be adjusted by increasing/decreasing the number of leading zeros required for the answer.
The nodes must also accept any chain that is longer then their own. This solves the problem of when multiple blocks are created at the same time. Whichever of the alternative chains receives a new block first will be accepted by the network as the new shared truth.
Every new block must be checked by each node to ensure the hashing puzzle was solved, all transactions are valid, and the mining reward was properly allocated.
Finally, a mining reward needs to be given to the block producer. This incentivizes nodes to operate on the network and process transactions.
Networking
My blockchain implements a peer-to-peer network with peer discovery and uses a gossip protocol to spread messages on the network. I utilized the twisted networking framework to build my own peer-to-peer protocol. Data is sent over TCP in a JSON format. Every object has methods to both serialize to JSON and be constructed from serialized JSON. The protocol includes the following functions:
- Share peer list
- Propagate transactions
- Propagate new blocks
- Request specific blocks
Conclusion
I really enjoyed working on this project and feel I achieved a much deep understanding of blockchain technology. Many thanks to the recurse center and all the helpful people along the way.