The testnet is live. Not "live on localhost". Not "live in an integration test". Not "live in a tmux pane on my laptop". Live on the public internet, at a real URL, behind a real Let's Encrypt cert, producing real blocks right now while you read this.
https://testnet.asentum.com
Type the URL in a browser. Watch the block counter tick. That thing you're looking at is a post-quantum-signed Layer-1 blockchain with a BFT consensus loop, running on a €4/month Hetzner box, serving Ethereum JSON-RPC over TLS, rendering its own explorer, accepting faucet drips from strangers on the other side of the world. I've been saying "next, a normal person can click a link and see this" in these dev logs for weeks. The next is now.
This article is the story of yesterday's launch, the three things that almost stopped me, the morning-after numbers, and where we go from here.
the setup (90 minutes including the fuck-ups)
Following DEPLOY.md from last phase, verbatim, on a fresh Hetzner CX22. 2 vCPU, 4 GB RAM, 40 GB disk, €4.15/month. Ubuntu 24.04 LTS.
- SSH in,
apt update && apt upgrade, install Node, pnpm, git, build-essentials. ufwdown to three open ports: 22 for SSH, 80 for HTTP, 443 for HTTPS. Everything else denied.rsyncthe repo to/opt/asentum/chain(the local repo has no remote and no commits yet, sogit clonewasn't an option —rsync --exclude node_modules --exclude .git --exclude distdoes the same job).pnpm install && pnpm -r build— the build producespackages/node/dist/bin/run.js, the runnable binary.- Write
.env.testnetwith the Phase 5.1 public-testnet flags:ASENTUM_PUBLIC_TESTNET=1, CORS origin*, public RPC/explorer URLhttps://testnet.asentum.com, chain name, faucet rate limit, loopback-only bind so Caddy terminates TLS externally. - Create the unprivileged
asentumsystem user, its data dir at/opt/asentum/data, chown the tree. - Drop in
/etc/systemd/system/asentum.servicewithNoNewPrivileges,ProtectSystem=strict,ReadWritePaths=/opt/asentum/data,Restart=on-failure.systemctl daemon-reload && systemctl enable --now asentum.service. curl http://127.0.0.1:8545/healthon the box →{"status":"ok"}. Chain is producing 2-second blocks locally.- Install Caddy from the cloudsmith repo. Write
/etc/caddy/Caddyfilewithreverse_proxy 127.0.0.1:8545, X-Forwarded-For header passthrough, HSTS, gzip, structured JSON access log. systemctl reload caddy. Caddy requests a Let's Encrypt cert via TLS-ALPN-01. Validation servers hit port 443. Cert acquired.https://testnet.asentum.com/healthreturns 200. Done.
That's the happy path. It did not actually go like that. It went like that on the tenth try, after three fuckups.
fuckup one: Promise.withResolvers is not a function
Built everything on the VPS, started the node, got this:
[chain] created fresh genesis at height 0
[faucet] address: 0xbeeb2ea9a8c098afee852895c70cf0171e2808eb
[validator] address: 0xbeeb2ea9a8c098afee852895c70cf0171e2808eb (faucet-fallback)
fatal: {}fatal: {}. That was the entire error. An empty object. No stack, no message, no class name, just the literal JSON inspection of nothing.
A fatal error that prints as the empty object is a specific category of "what in the actual hell" that I have not missed from production JavaScript. My first instinct was that something in our contract sandbox was throwing a literal throw {}. My second instinct was that a third-party dependency was throwing a non-Error thenable. Both wrong.
I patched the top-level error handler in bin/run.ts to print more fields — err.name, err.message, err.stack, util.inspect(err, {showHidden: true}). With showHidden: true the truth came out:
fatal inspect: {
[stack]: 'TypeError: Promise.withResolvers is not a function',
[message]: 'Promise.withResolvers is not a function',
[name]: [Getter/Setter],
[toString]: [Getter/Setter]
}Promise.withResolvers is not a function — the real error, all along. Promise.withResolvers was added in Node.js 22. The VPS was running Node.js 20.20.2, which I'd installed via the nodesource setup_20.x script because my own .nvmrc said 20 and package.json engines.node said ">=20.0.0". My local dev machine, meanwhile, had been silently running Node 22.12 through nvm. Everything worked locally because everything was secretly on 22. I'd never been on 20. The repo's declared engine was a lie I'd been telling myself.
Actually-grep'd for Promise.withResolvers — not in our own code. It's in a transitive libp2p dep: mortice, it-queue, @libp2p/utils, @libp2p/ping. Even though we killed libp2p pubsub back in Phase 1.5, the network layer still imports libp2p for peer IDs and multiaddrs (displayed in the node banner). libp2p moved to Node-22-only at some point and my lockfile picked that up.
Two options: (1) downgrade libp2p and every dependent. Painful. (2) Bump the VPS to Node 22. Trivial.
apt remove -y nodejs
curl -fsSL https://deb.nodesource.com/setup_22.x | bash -
apt install -y nodejs
node -v # v22.22.2
pnpm install
pnpm -r buildBack to green. Node 22 is LTS as of October 2024 so there's no reason to be on 20 for a new deploy anyway. Followup: bump .nvmrc and engines.node in the repo to tell the truth.
fuckup two: SES is eating my error messages
Back to the fatal: {} thing. The root cause was Node 22, but why did console.error('fatal:', err) print {} for a real TypeError?
Answer: SES.
Our contract VM is built on SES (Secure ECMAScript) — the hardened-JavaScript runtime from the Agoric folks. SES runs lockdown() at startup, which among many other things replaces the global Error constructor with a version that makes message, stack, and name non-enumerable properties hidden behind getters. This is a security feature: under SES, errors thrown inside a sandbox don't leak information to the outside by default. Enumerable properties are the attack surface; non-enumerable are the quiet ones.
Unfortunately, Node's default console.error('fatal:', err) implementation uses util.inspect under the hood, which by default only walks enumerable properties. When it sees a SES-hardened Error, it sees an object with zero enumerable properties and prints {}. Exactly what we got.
The fix is boringly simple:
main().catch((err) => {
// SES lockdown makes Error `message`/`stack` non-enumerable, so a plain
// `console.error('fatal:', err)` prints `{}` and hides the real failure.
// Read the fields explicitly so the operator sees what actually blew up.
const name = (err && err.name) || 'Error';
const message = (err && err.message) || String(err);
const stack = (err && err.stack) || '(no stack available)';
console.error(`fatal: ${name}: ${message}`);
console.error(stack);
process.exit(1);
});Nine lines instead of three. Zero new imports. Now a real error surfaces as fatal: TypeError: Promise.withResolvers is not a function followed by the stack, and a deployment operator gets an answer in seconds instead of guessing.
This cost me maybe twenty minutes of guessing before I thought to actually inspect the error object properly. Worth writing down so future-me, or any future AsentumChain validator operator, doesn't eat the same knife.
fuckup three: Caddy can't write to its own log file
Wrote the Caddyfile. Validated. systemctl reload caddy. Reload hung for ninety seconds and then:
caddy.service: Reload operation timed out. Killing reload process.
Status: "loading new config: setting up custom log 'log0': opening log writer using
...Filename:"/var/log/caddy/asentum.log"...: open /var/log/caddy/asentum.log:
permission denied"Permission denied on the log file. But I'd created /var/log/caddy and chowned it to caddy:caddy. What gives?
Looked at the file:
-rw------- 1 root root 0 Apr 10 23:47 asentum.logZero bytes, owned by root. Caddy had already tried to create it once during the first (failed) reload, but at that moment the reload was running as root via the admin API, so the resulting file was owned by root. On the next reload attempt, Caddy's main process — running as user caddy — couldn't write to it. Classic first-run ownership race.
One-line fix: rm -f /var/log/caddy/asentum.log && chown -R caddy:caddy /var/log/caddy && systemctl restart caddy (full restart, not reload, so we started from a clean slate). Worked on the next try. DEPLOY.md should probably mention touch + chown the log file before first start to avoid this.
the cert acquired itself
Once Caddy was cleanly up, TLS was one big silent "just works". Here's what that looked like in the logs:
[caddy] enabling automatic TLS certificate management domains=[testnet.asentum.com]
[caddy] obtaining certificate identifier=testnet.asentum.com
[caddy] trying to solve challenge challenge_type=tls-alpn-01
[caddy] served key authentication certificate remote=23.178.112.105:40669
[caddy] served key authentication certificate remote=16.16.127.155:13420
[caddy] served key authentication certificate remote=18.217.203.56:12354
[caddy] served key authentication certificate remote=44.249.174.207:49046
[caddy] served key authentication certificate remote=3.1.200.180:23080
[caddy] authorization finalized authz_status=valid
[caddy] validations succeeded; finalizing order
[caddy] successfully downloaded available certificate chains count=2
[caddy] certificate obtained successfully identifier=testnet.asentum.comThose five served key authentication certificate lines are Let's Encrypt's distributed validation agents — five of them, from IPs all over the world — hitting our port 443 simultaneously to confirm we control the domain. They all see the same TLS-ALPN-01 challenge, they all agree, the certificate gets issued, Caddy downloads it, Caddy starts serving it. End-to-end in about five seconds.
Seven years ago this would have been a three-hour ordeal of openssl commands, certbot flags, and renewed-certificate cron jobs. Now it's a Caddyfile line and a silent rendezvous between your VPS and five IPs in three continents. Infrastructure got a lot better while no one was paying attention.
the first real smoke test
$ curl -sS https://testnet.asentum.com/health
{"status":"ok"}
$ curl -sS https://testnet.asentum.com/metadata
{"chainId":"0x539","chainName":"AsentumChain Testnet",
"nativeCurrency":{"name":"Asentum","symbol":"ASE","decimals":18},
"rpcUrls":["https://testnet.asentum.com"],
"blockExplorerUrls":["https://testnet.asentum.com"],
"chainIdDecimal":"1337","currentBlock":"163",
"faucetUrl":"https://testnet.asentum.com/dev/faucet"}
$ curl -sS -X POST https://testnet.asentum.com/ \
-H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"eth_blockNumber","params":[]}'
{"jsonrpc":"2.0","id":1,"result":"0xa3"}
$ curl -sS -X POST https://testnet.asentum.com/dev/faucet \
-H 'Content-Type: application/json' \
-d '{"to":"0x1111111111111111111111111111111111111111","amount":"100"}'
{"accepted":true,"txHash":"0x14868c5fdca83b2d21d699c42509b88a8e16711dcf72e3a79f6433d971887a95"}
# Immediate retry with the same IP — should 429:
$ curl -sS -X POST https://testnet.asentum.com/dev/faucet ...
{"accepted":false,"reason":"rate limited — try again in 60s","retryAfterMs":59624}Four HTTP checks: health, metadata (EIP-3085 shape for MetaMask's wallet_addEthereumChain), eth_blockNumber (163 at the time, a real number advancing every 2 seconds), faucet drip (accepted, real tx hash), faucet retry (correctly rejected by the Phase 5.1 token-bucket rate limiter). All four behaviors are features the tests verified back in Phase 5.1 — now verified end-to-end through TLS, through Caddy, through the X-Forwarded-For handoff, through systemd, against a node that isn't on my laptop.
And then I went to bed.
the morning after
This morning, eight hours later, I pulled the vitals. This is what greeted me:
=== current block ===
current: #14,984 (0x3a88)
baseline: #740 at 00:09 UTC
elapsed: 7.92h (28,494s)
produced: 14,244 blocks
predicted: 14,247 @ 2.000s target
drift: -3.3 blocks (0.023% behind ideal)
avg interval: 2.000s per block (target 2.000s)
=== systemd ===
active: yes
restarts: 0
uptime: continuous since 23:44:27 UTC
memory: 85 MB (from 57 MB at boot)
free RAM: 3.2 GB / 3.7 GB
CPU burned: 14.5 minutes over 8 hours (~3% avg)
disk used: 3.5 MB of chain state
=== errors ===
journalctl -u asentum --since "8 hours ago" -p err: 0 entries
journalctl -u caddy --since "8 hours ago" -p err: 0 entriesRead that again. Fourteen thousand two hundred forty-four consecutive blocks. Target block interval 2.000 seconds. Measured average block interval: 2.000 seconds. Drift from a theoretical perfect clock: three and a third blocks behind, or 0.023%. Memory: 85 MB. Zero restarts. Zero errors. Zero warnings. Zero hiccups.
I don't know how to tell you how significant this is without sounding like a crypto bro, but: most "testnets" don't survive a full night unattended on commodity hardware with public TLS exposure. Most chains that are still in active development produce blocks on a best-effort metronome that visibly drifts with load, garbage collection, peer reconnects. This one held 2.000 seconds for eight hours while exposed to the open internet, while I slept.
That's not "the code works." That's the code is boring. Boring is the goal for infrastructure. The whole point of a blockchain — the whole point of an L1 — is that it keeps going and does not care about you. I set it up, went to sleep, and it didn't notice.
the cleanup
With the lessons of the deployment fresh in my head, I went back to the repo and fixed four small things:
.nvmrc:20→22. It was lying. Node 20 never worked for this codebase because libp2p needsPromise.withResolvers. Now the.nvmrcmatches reality.package.jsonengines.node:">=20.0.0"→">=22.0.0". Same reason. If someone clones the repo and runs pnpm install on Node 20, they should get a clear error from pnpm, not a crypticfatal: {}hours later.packages/node/src/bin/run.ts: the fatal handler. No moreconsole.error('fatal:', err). The new version explicitly readserr.name,err.message,err.stackso SES-hardened Errors surface properly. Nine lines, no new imports, future me and every future validator operator is spared this exact debugging session.pnpm-lock.yamlregenerated. I'd added the@asentum/metamask-snappackage a few phases back without runningpnpm installat the workspace root, so the lockfile was drifted. On the VPS I had to use--no-frozen-lockfileto work around it. Now the lockfile is current and futurepnpm install --frozen-lockfilecalls will succeed.
pnpm -r test across all five packages: 278/278 green. No test changes — none were needed. The fixes are all either repo metadata or an error handler that only fires on main().catch(...). The VPS node is still running happily on Node 22 with the pre-fix binary, and that's fine — next time I deploy (which will be when I either stand up the three joiner validators for multi-node BFT testing, or publish the Snap to NPM), the cleanup lands with it.
what the testnet is right now
Go click https://testnet.asentum.com and you will see:
- A phosphor-green block explorer. TESTNET badge in warning-orange.
- A live block feed ticking over every two seconds, showing state root, tx count, gas used.
- A "+ Add to MetaMask" button that calls
wallet_addEthereumChainwith the EIP-3085 metadata (chain id 1337, name "AsentumChain Testnet", native currency ASE, rpc + explorer URLs). - A "Copy RPC URL" button.
- A faucet form that accepts a 20-byte address and gives you 100 test ASE.
- A chain state panel and a CLI quick-reference.
Behind the curtain, every block on that feed has a real 3309-byte Dilithium3 signature on the header, verified by a BFT engine with slashing support, committed to a real Merkleized state trie with a real receiptsRoot. Every JSON-RPC method MetaMask and viem and ethers expect is there — 16 eth_* methods, eth_getLogs with filters, eth_getCode, eth_getBlockByHash, all of it — all pure translation from native state, zero ECDSA. The chain is post-quantum from the bottom up.
And it's operated by a systemd unit and a Caddyfile on a €4 Hetzner box that never touched the internet before yesterday.
what's next, no really this time
Phase 6 was going to be "launch the testnet". That's done. What's left of Phase 6 is the second half: letting people actually use it.
Immediate, today:
- Ship it to one friend. Not a group chat. Not a Twitter thread. One person. Watch them click the URL on a fresh browser. Note every moment of confusion. Fix the five most embarrassing noob bugs that surface. Repeat with a second person.
- A single-binary wallet installer for Windows and macOS, downloadable via one terminal command (
curl ... | shoriwr ... | iex), so testers don't have to install Node.js themselves before they can play. The CLI is good, the quickstart is good, but "install Node 22 first" is still a friction gate I don't want. Probably one bash script + one PowerShell script, probably Node SEA orpkgunder the hood. That's next week. - Three more validator boxes using the Phase 5.4 join flow. Verify 4-validator BFT survives real network latency, not just loopback. Verify the slashing pipeline fires end-to-end if I deliberately double-sign on one of them.
- Publish the MetaMask Snap to NPM so the browser signing UX lights up for anyone who has MetaMask installed.
Long term: the SDK, the hosted block explorer, the historical balance queries, the auto-bond env var. All queued. None blocking.
the vibe
Yesterday I wrote "shipping is the hard part. Building the chain was the fun part." Today I can confirm: shipping was not the hard part. Shipping took ninety minutes and three fuckups. Shipping is never going to be harder than that again, because now the runbook is real and the runbook is proven.
The hard part was the two years before shipping. The hard part was deciding in Phase 0 that post-quantum was non-negotiable even though it would make every message bigger and every signature slower and every dev tool incompatible. The hard part was killing libp2p pubsub at 2am in Phase 1.5 when it didn't want to work. The hard part was figuring out that every module — staking, slashing, governance, the whole damn thing — could just be a smart contract, and that the apply layer could verify slashing evidence cryptographically at commit time so nothing downstream had to trust anything. The hard part was every individual decision that looked small at the time and was actually load-bearing later.
Shipping was pressing enter on a runbook.
She's breathing. Fourteen thousand two hundred forty-four blocks in her first night, on a two-second metronome, with zero restarts and zero errors and 85 megabytes of RAM. If you want to watch her breathe, the URL is at the top of this article.
— milkie