Cuprate Architecture

WIP

Cuprate's architecture book.

Sections are notated with colors indicating how complete they are:

Color	Meaning
⚪️	Empty
🔴	Severely lacking information
🟠	Lacking some information
🟡	Almost ready
🟢	OK

Continue to the next chapter by clicking the right > button, or by selecting it on the left side.

All chapters are viewable by clicking the top-left ☰ button.

The entire book can searched by clicking the top-left 🔍 button.

Foreword

Monero¹ is a large software project, coming in at 329k lines of C++, C, headers, and make files.² It is directly responsible for 2.6 billion dollars worth of value.³ It has had over 400 contributors, more if counting unnamed contributions.⁴ It has over 10,000 node operators and a large active userbase.⁵

The project wasn't always this big, but somewhere in the midst of contributors coming and going, various features being added, bugs being fixed, and celebrated cryptography being implemented - there was an aspect that was lost by the project that it could not easily gain again: maintainability.

Within large and complicated software projects, there is an important transfer of knowledge that must occur for long-term survival. Much like an organism that must eventually pass the torch onto the next generation, projects must do the same for future contributors.

However, newcomers often lack experience, past contributors might not be around, and current maintainers may be too busy. For whatever reason, this transfer of knowledge is not always smooth.

There is a solution to this problem: documentation.

The activity of writing the what, where, why, and how of the solutions to technical problems can be done in an author's lonesome.

The activity of reading these ideas can be done by future readers at any time without permission.

These readers may be new prospective contributors, it may be the current maintainers, it may be researchers, it may be users of various scale. Whoever it may be, documentation acts as the link between the past and present; a bottle of wisdom thrown into the river of time for future participants to open.

This book is the manifestation of this will, for Cuprate⁶, an alternative Monero node. It documents Cuprate's implementation from head-to-toe such that in the case of a contributor's untimely disappearance, the project can continue.

People come and go, documentation is forever.

— hinto-janai

monero-project/monero

2024-05-24: $143.55 USD * 18,151,608 XMR = $2,605,663,258

⁴

git log --all --pretty="%an" | sort -u | wc -l on cc73fe7

⁵

https://monero.fail/map

⁶

https://github.com/Cuprate/cuprate

Intro

Cuprate is an alternative Monero node implementation.

This book describes Cuprate's architecture, ranging from small things like database pruning to larger meta-components like the networking stack.

A brief overview of some aspects covered within this book:

Component designs
Implementation details
File location and purpose
Design decisions and tradeoffs
Things in relation to monerod
Dependency usage

Source code

The source files for this book can be found on at: https://github.com/Cuprate/architecture-book.

Who this book is for

Maintainers

As mentioned in Foreword, the group of people that benefit from this book's value the most by far are the current and future Cuprate maintainers.

Cuprate's system design is documented in this book such that if you were ever to build it again from scratch, you would have an excellent guide on how to do such, and also where improvements could be made.

Practically, what that means for maintainers is that it acts as the reference. During maintenance, it is quite valuable to have a book that contains condensed knowledge on the behavior of components, or how certain code works, or why it was built a certain way.

Contributors

Contributors also have access to the inner-workings of Cuprate via this book, which helps when making larger contributions.

Design decisions and implementation details notated in this book helps answer questions such as:

Why is it done this way?
Why can it not be done this way?
Were other methods attempted?

Cuprate's testing and benchmarking suites, unknown to new contributors, are also documented within this book.

Researchers

This book contains the why, where, and how of the implementation of formal research.

Although it is an informal specification, this book still acts as a more accessible overview of Cuprate compared to examining the codebase itself.

Operators & users

This book is not a practical guide for using Cuprate itself.

For configuration, data collection (also important for researchers), and other practical usage, see Cuprate's user book.

Observers

Anyone curious enough is free to learn the inner-workings of Cuprate via this book, and maybe even contribute someday.

Required knowledge

General

Rust
Monero
System design

Components

Storage

Embedded databases
LMDB
redb

RPC

axum
tower
async
JSON-RPC 2.0
Epee

Networking

tower
tokio
async
Levin

Instrumentation

tracing

How to use this book

Maintainers

Contributors

Researchers

⚪️ Bird's eye view

⚪️ Map

⚪️ Components

⚪️ Formats, protocols, types

⚪️ monero_serai

⚪️ cuprate_types

⚪️ cuprate_helper

⚪️ Epee

⚪️ Levin

⚪️ Storage

⚪️ Database abstraction

⚪️ Blockchain

⚪️ Transaction pool

⚪️ Pruning

RPC

⚪️ Types

⚪️ JSON

⚪️ Binary

⚪️ Other

⚪️ Interface

⚪️ Router

⚪️ Handler

⚪️ Methods

⚪️ ZMQ

TODO

⚪️ Consensus

⚪️ Verifier

⚪️ TODO

⚪️ Networking

⚪️ P2P

⚪️ Dandelion++

⚪️ Proxy

⚪️ Tor

⚪️ i2p

⚪️ IPv4/IPv6

Instrumentation

Cuprate is built with instrumentation in mind.

⚪️ Logging

⚪️ Data collection

⚪️ Binary

⚪️ CLI

⚪️ Config

⚪️ Logging

⚪️ Resource model

⚪️ File system

⚪️ Sockets

⚪️ Memory

Concurrency and parallelism

It is incumbent upon software like Cuprate to take advantage of today's highly parallel hardware as much as practically possible.

With that said, programs must setup guardrails when operating in a concurrent and parallel manner, for correctness and safety.

There are "synchronization primitives" that help with this, common ones being:

These tools are relatively easy to use in isolation, but trickier to do so when considering the entire system. It is not uncommon for the bottleneck to be the poor orchastration of these primitives.

Analogy

A common analogy for a parallel system is an intersection.

Like a parallel computer system, an intersection contains:

Parallelism: multiple individual units that want to move around (cars, pedestrians, etc)
Synchronization primitives: traffic lights, car lights, walk signals

In theory, the amount of "work" the units can do is only limited by the speed of the units themselves, but in practice, the slow cascading reaction speeds between all units, the frequent hiccups that can occur, and the synchronization primitives themselves become bottlenecks far before the maximum speed of any unit is reached.

A car that hogs the middle of the intersection on the wrong light is akin to a system thread holding onto a lock longer than it should be - it degrades total system output.

Unlike humans however, computer systems at least have the potential to move at lightning speeds, but only if the above synchronization primitives are used correctly.