Introduction to Hardware Manager

Problem

A few years ago, while developing one of many tarpit ideas, I found myself being unable to build a data plane for my app that could handle the load (classic building software for millions of users when you have 0™) that was not prohibitively expensive. As I dug deeper into the issue, it became quite obvious that hardware is not the issue. I could get extremely fast NVMes and networking in the cloud, but the data infra running on top was just not able to extract even a fraction of it.

The common thing underpinning all that software was Linux: a general-purpose operating system designed and built on standards during a time when hardware was scarce, timesharing was a big thing, and things like VMs and the cloud didn’t exist.

Linux and legacy abstractions enforce a lot of rules upon software engineers, and they did the best possible job inside those boundaries. However, at some point we need to move forward, or else infrastructure and software will become too expensive, too slow, too complex, and too insecure, especially when agents become the dominant users of the internet.

If the current trends continue, we think in five years, non-human traffic will be as much as 1,000 times as much as human traffic.

In other words, humans will be a rounding error on the internet, not because human traffic goes down, but that’s just how fast we’re seeing non-human traffic grow.

Cloudflare

To solve these issues, we created Hardware Manager.

Intro to Hardware Manager

Hardware Manager is a thin layer of code that creates a single-tenant runtime engineered from scratch for maximum performance and security.

It provides a bare-metal environment where software is coupled with the runtime rather than separated.

It is built entirely from bare-metal primitives. A bare-metal primitive is a Hardware Manager subsystem that cannot be optimized further depending on the app executing (such as drivers and networking protocols like TCP). All bare-metal primitives are exposed via a transparent API, granting the software direct control over everything (like reading NIC device queues or modifying page tables directly).

Hardware Manager was designed with these 3 core design principles:

Question everything: Traditional methods are not always the best; many times what is considered a correct way to do something is heavily limited by hardware constraints that existed 20 years ago, but are no longer relevant. We use history as a guide on how to implement something, not a blueprint.
Ignore POSIX: POSIX is an extremely complex standard that on one hand simplifies complex things, but on the other, adds loosely defined and undefined behaviors that introduce complexity on their own. It also hides and abstracts too many things, making it difficult (I would argue even impossible) to write high-performance software.

As a result of that, we decided from day 0 that POSIX support is an afterthought, not something that will shape how we architect Hardware Manager.
Keep everything open: engineer the system for raw performance by giving software full control over its execution.

Implementation at a glance

A more technical deep dive will be done in a different blog post. At a glance, it provides many things you would expect from a kernel + some more.

It was designed to run on modern, powerful x86_64 server CPUs (with a plan to support arm64 down the line) with full SIMD support. It has SMP and NUMA support.

Since we are in the cloud age, our primary deployment target is the cloud, and we are happy to say that we are one of the rare kernels that can run across all EC2 instances (including metal) with full support for networking and storage.

All bare-metal primitives are exposed via qAPI. Storage and networking stacks are specifically designed to be easy to extend. For example, the networking stack is designed so that we can easily support sockets, callback-based flows, and direct device queue polling.

This is what not making POSIX a first-class citizen enables us to do. We can provide “simple”, powerful interfaces to expose things that were constrained and hidden away by very complex libraries like DPDK and SPDK on Linux.

In addition to enabling developers (and agents) to write high-performance software more easily, QuinineHM strives to enable developers to keep using the tooling they know and love. It provides things like GDB support, observability (via tools like Prometheus), encryption libraries, etc.

But, to help everyone create better more stable software, it provides deterministic simulation testing out of the box. Enabling both humans and agents to iterate quicker and create safer software.

This is just a sneak peek into what we built, but we will follow up with more in-depth blogs.