The Oasis "counter-hack" and the centralization of defi

Note: This essay is more technical than most of my writing, and so you may find the W3IGG glossary useful if there is unfamiliar jargon. If anything is too complex, feel free to drop a comment and I will do my best to explain further.

In February 2022, a hacker stole 120,000 wrapped Ethereum from Wormhole, a cross-blockchain bridge. It was (and still is) one of the largest hacks in the crypto world. Based on the Ethereum prices at the time, the hacker made off with around $320 million. Because it's crypto, there was no "undo".

A loss of this magnitude might spell the end for many crypto projects, but Wormhole had deep-pocketed backers. The crypto arm of the Chicago-based proprietary trading firm Jump Trading had acquired Certus One, the developers of Wormhole, some months prior.

Jump Crypto, the crypto subsidiary of Jump Trading, was launched in September 2021 following a $350 million raise, and has been heavily involved in the crypto ecosystem since: both by trading cryptocurrencies and market-making, but also developing software. Fellow Jump Trading subsidiary Jump Capital has also invested heavily in the industry as a venture fund. The Jump conglomerate has been in the news quite recently following the SEC charges against Do Kwon and Terra/Luna. Although the SEC complaint does not directly name Jump Trading, it has been reported that they are the "U.S. Trading Firm" mentioned in the lawsuit that profited enormously from the Terra fraud. The firm reportedly enjoyed an almost $1.3 billion (with a B) windfall after stepping in to rescue the Terra stablecoin's peg on at least one occasion in May 2021. This action was hidden from the public by Terra founder Do Kwon, and the others who knew about it, and instead Kwon claimed that Terra had naturally regained its peg via a "self-healing" mechanism. He suggested this was a demonstration that it was not at risk of the kind of devastating collapse that ultimately occurred only a year later, and this "U.S. Trading Firm" made no apparent effort to correct the record.

But back to Wormhole: when the February 2022 hack occurred, Jump Crypto stepped up — putting $325 million of their own funds into the project. Wormhole offered a $10 million "bounty"^a to the hacker if they returned the funds, but the attacker decided they'd rather just keep the $325 million, thankyouverymuch.

In the year since the hack, the hacker has taken a different strategy with their newfound riches than many other thieves. Rather than trying to launder and then cash out their profits into fiat, they have instead moved the funds through various decentralized finance (defi) protocols. In late January 2023, after a period of dormancy, they began to take highly-leveraged positions on the liquid Ethereum staking derivatives stETH (Lido) and rETH (RocketPool). In fact, between the capital they deployed and the leverage, they became the third-largest holder of wrapped stETH in existence. Some in the crypto industry were a little mystified, and wondered if perhaps the attacker was a crypto native taking "degen" positions.^b

In order to lever up, the hacker opened two vaults on the Oasis protocol. Oasis is a project that was originally created by members of the MakerDAO team, and it serves as a frontend to the MakerDAO project. Oasis has since branched off into its own company, though it still remains the favored platform by MakerDAO — the big green "Use DAI" button on the MakerDAO homepage links to Oasis.

It seems that Jump Crypto may have sprung into action when they saw the hacker had moved funds to Oasis in January, and began researching if there might be a way to "reverse-hack" the hacker to recover the assets. Oasis claims in a statement that they "first became aware of the possibility to assist in the retrieval of the assets after a whitehat group reached out to the team on the evening of Thursday 16th February 2023, that showed it would be possible to retrieve the assets … [via a] previously unknown vulnerability in the design of the admin multisig access."

Assuming we take Oasis at their word that they did not already know about this awfully convenient backdoor (sorry, "unknown vulnerability"), this suggests that either an unrelated whitehat disclosed its existence to Oasis and then Oasis almost immediately contacted Jump Crypto to tell them about it, or Jump was the mysterious whitehat in this equation. My guess is the latter, because Jump had a hundred-thousand-ETH incentive.

There was then a five-day period (including a weekend) between the disclosure and Jump obtaining a court order from the High Court of England and Wales. Oasis has not published the court order, but in their statement they say they were directed "to take all necessary steps that would result in the retrieval of certain assets involved with the wallet address associated with the Wormhole Exploit". They elaborate that "this was carried out in accordance with the requirements of the court order, as required by law, using the Oasis Multisig and a court authorised third party".

From this we can infer that Oasis made no attempt to challenge the court order, which would certainly have taken more than a few days to resolve. My theory is that Oasis knew the action would be unpopular among the crypto community were they to find out about it — and particularly so among the subset who are passionate about decentralized finance projects like MakerDAO and Oasis — and so they asked Jump to obtain a court order to give Oasis some air cover for acting in a way they knew would be perceived to be against their ethos.

Three days (no weekend) elapsed between Jump and Oasis recovering the assets and Blockworks Research noticing and then reporting on the "reverse-hack". Before Blockworks published, neither Jump nor Oasis had made public mention of it. Oasis published a brief statement on the event 25 minutes after Blockworks published.^c Neither had responded to requests for comment from Blockworks. Perhaps they planned to release a statement at some point down the line even if unforced by Blockworks, but it's also entirely possible they planned to keep what they'd done quiet and hope no one noticed.

The recovery

The Wormhole exploiter had opened vaults on Oasis and used them to take out levered long positions on ETH staking derivatives. In order to safeguard against losses, they also used Oasis's automation tools to set automatic stop-loss protection for the vault.

The smart contracts for Oasis's automation tools are what's known as upgradeable smart contracts, meaning they employ a technique to make typically immutable blockchain code mutable. This has become extremely commonplace in crypto, where project teams have realized that always writing perfectly bug-free code is a pipe dream, and that being able to patch that code is sometimes quite desirable. Upgradeability is also used for other purposes besides bugfixing, including adding functionality without requiring users to migrate to a completely new contract.

Upgradeable smart contracts are slightly controversial in crypto, though not as much as I would expect. In fact, they seem to be becoming accepted as the norm. They solve real problems: fixing bugs is good! Adding new features is good! It's annoying and expensive to migrate between contracts!

But blockchain developers didn't just decide to make it a huge pain to change smart contract code because they liked feeling a rush of terror anytime they deployed. It's intentionally difficult for ideological reasons. Namely, crypto is big on "trustlessness" — that is, the idea that you shouldn't have to trust any person or organization, you can simply trust the code. The thinking goes: if you can audit the smart contract code and if that code can never be changed, you don't have to trust the people who wrote it.

With upgradeable smart contracts, trustlessness is no longer a given. Anytime someone uses a project with an upgradeable smart contract, or a project that depends on some other project with an upgradeable smart contract (and so on up the dependency graph), they have to decide either that they trust whoever is in charge of deciding when and how to change the code, or that any possible changes to the upgradeable code couldn't negatively impact them.

Upgradeable smart contracts are sometimes controlled by a single entity, but are often managed by a group of some kind, such as a DAO or a multi-signature contract (multisig). Some people are willing to accept DAO-governed upgradeable contracts under the theory that trusting a sufficiently large DAO is close enough to trustlessness. That's more of a philosophical argument, and it's one for another day, because Oasis used a multisig. If Oasis wanted to upgrade a contract, four of its twelve multisig members needed to approve the decision. Sometimes multisig contracts are controlled by people from disparate organizations with relatively independent interests. Sometimes they're all controlled by employees of a single entity. Sometimes multisig members are anonymous — leaving them ostensibly less vulnerable to attack or coercion, but also less subject to scrutiny.

Because Oasis is controlled by a company, Oazo, the key holders are likely just a group of Oazo employees, and the multisig is more of a safeguard against, say, a rogue employee than any sort of attempt at real decentralization of power.

I will leave it in Blockworks' extremely capable hands to explain in detail how the reverse hack was accomplished, but the long and short of it is that a new wallet — almost certainly controlled by Jump — was added as a signer to the Oasis multisig. After that point, they upgraded the automation smart contract (enabled by the Wormhole exploiter for stop-loss protection) to a new proxy that allowed them to effectively reassign control of the vault to themselves.

Amusingly, Oasis had implemented a 30-minute time delay mechanism in their smart contract upgrade flow so that observers could potentially intervene, or perhaps at least try to safeguard their own assets, if an upgrade was malicious (or simply undesirable). However, the same multisig that can upgrade the contract can also change the delay, so they simply set the delay to 0 and carried on their merry way.

Once the funds were under Jump's control, they exited the positions in the Oasis vaults, repaid the $78 million loan that the hacker had taken out from Oasis, and returned the 120,695 wstETH (currently ~$214 million) and 3,214 rETH (~$5.5 million) to their own pockets for a total recovery of a bit more than $140 million at today's prices. Although ETH prices in USD have come down considerably from the initial hack, those staking derivatives could be exchanged for approximately 137,537 ETH, meaning Jump actually came out of this whole debacle with about 17,500 ETH more than the 120,000 that was stolen a year ago. I guess that "degen" trading paid off.

The implications

Most folks are happy that the stolen crypto has made its way back to its rightful owners (even those of us who don't particularly like said owners). Wrongs have been righted! Justice has been served!

However.

Crypto was ostensibly designed to create a trustless, censorship-resistant system in which no one — including governments — has the ability to intervene in a person's transactions. Even if that person is a hacker, or otherwise breaking the law, or simply doing something that people would rather they not do. There is certainly a reasonable debate to be had over whether this is a desirable state of affairs, and over whether altogether that many people who are into crypto today actually hold this strong ideological position, but that was certainly a large part of the original intent. For the purposes of this piece I will be simply evaluating risk to this intent — not arguing for or against it.

In order to create a trustless, censorship-resistant system, blockchain developers had to make a lot of tradeoffs. Broadly speaking, blockchains are slow. They don't scale well. They're expensive. There is no "undo" button. Blockchains are not this way because they are just poorly coded or because their developers don't wish for them to be fast and inexpensive — there are certainly plenty of excellent developers working on blockchains. Blockchains are this way because they have to be in order to try to achieve that trustless, censorship-resistant ideological goal.

But now we've just seen a very clear illustration of how multisig-controlled upgradeable contracts, which have proliferated throughout crypto, undermine trustlessness and censorship-resistance. If a crypto project has accepted the huge costs that come with building on a blockchain as worthwhile tradeoffs to achieve trustlessness and censorship-resistance, but then undermines its own trustlessness and censorship-resistance by using multisig-controlled upgradeable contracts, well, what was even the point?

In fact, this would seem to be a worse world than the traditional financial system. In traditional finance, trusted intermediaries (e.g. banks or credit card processors) routinely press the "undo" button (e.g. chargebacks) for customers who've been victims of fraudulent transactions or other theft. In crypto, it's sort of accepted that people take on the risk of irreversible hacks and theft in the pursuit of trustlessness and censorship-resistance. But now we see that when it comes to projects like Oasis, most users are up a creek if their assets are stolen, but entities that are wealthy and powerful enough to coerce the multisig (in this case via a court), play under a different set of rules entirely.

And this is not a one-off issue with Oasis, a platform that a lot of people even within crypto may not have heard about. In fact, not all blockchains treat contract immutability as standard like Ethereum does. Solana, for example, has mutable contracts built right in to the protocol: contracts are mutable by default, with authority delegated to the single deployer address unless otherwise specified. Most Ethereum layer 2 chains, such as Arbitrum, Polygon, and Optimism, have upgradeable contracts governed by small multisigs.

Many of these layer 2 projects claim to be "works in progress" that will move to a decentralized and more secure model at some unspecified point in the future, but they are by no means treated by developers or users as though they are alpha software:

And within the Ethereum ecosystem, major and popular defi projects including PancakeSwap, SushiSwap, and Balancer have upgradeable contracts governed by multisigs.² Some projects also put treasury control or oracle data control behind multisig governance, which introduces risks of treasury theft or oracle manipulation attacks. Some of these vulnerable projects, like Frax and its $1.2 billion of multisig-controlled assets, could feasibly have massive ripple effects on the rest of defi. And some projects that aren't directly exposed to this risk are exposed via third-party services that are — for example, Chainlink, which is widely used for price oracles by projects including Aave, and could massively impact the defi ecosystem if compromised.

While some might be satisfied that it's fairly unlikely that enough people on a multisig would spontaneously decide to sabotage a project to which they have close ties, the possibility of legal pressure is not to be discounted — especially not now that there's been a case where it's been applied quite effectively in a way that could inspire other similar actions. Some small number of people might be willing to go to jail over their strong ideological beliefs around crypto censorship-resistance, but how confident can we be that those will be the people on the multisigs that could feasibly wipe out large parts of the defi ecosystem? When push comes to shove, and multisig members actually find themselves facing life-changing financial consequences, threats to their freedom, punishment that could have knock-on effects on their families or other loved ones, or even more serious threats, will they still feel that way? Or will a sufficient number of them just agree to comply? Oasis didn't even push back on the court order, from the looks of it, much less defy it.

In this case, we've seen a state actor (the High Court of England and Wales) apply pressure to a multisig to reverse a fairly clear-cut theft. But there are certainly more scenarios in which state powers could apply pressure to projects to take far more controversial actions. Furthermore, the classic "wrench attack" is just as applicable to multisig members, and there are plenty of non-state powers, or more authoritarian states, who might seek to pressure multisig members in ways that go far beyond court orders.

Why do these projects use multisigs in this way, then, if they're so risky? Well, true decentralization, trustlessness, and censorship-resistance is difficult and painful. Multisigs are easier. Upgradeable contracts are easier. Multisig-controlled upgradeable contracts get you most of the way towards trustlessness, and substantially ease development, bugfixing, and migration pain. Furthermore, a lot of the creators of even so-called "defi" projects simply don't want to relinquish control.

Some projects have stayed a little closer to that ideal of trustlessness and decentralization. Some of the blockchains themselves, like Bitcoin and Ethereum, for example. Some projects like Uniswap also stick to the immutable smart contract paradigm, and so can't be changed out from under people. However, those projects also suffer from centralization, just in different places. There are still people who make code changes to Bitcoin (albeit rarely), Ethereum, and Uniswap. Only five people, for example, actually have the power to change Bitcoin Core code. With Ethereum, Vitalik Buterin holds massive influence.

The argument is often made that, sure, there are small groups that hold massive sway in these projects, but if they were to release a change that completely contradicted the crypto ethos, people could just not use the updated version, or they could fork. This, however, naively interprets these decisions as happening in a vacuum. In reality, large financial players using these projects also have major sway in what version of a project is broadly adopted.

We saw a great example of this with the Ethereum Merge in September 2022. Some wondered if perhaps the Ethereum community might simply not move over en masse to the new proof-of-stake Ethereum chain, and rather would continue operating on the proof-of-work chain. However, when major stablecoin issuers like USDC and Tether announced support for the Ethereum proof-of-stake chain (and only that chain), people stopped wondering. No one really wanted to use the chain where stablecoins were worthless, particularly the projects that heavily rely on them, and people wanted to go where their assets had value and where their favorite projects were. Today, ETHPoW is, predictably, a ghost town.

The same can feasibly happen to project-level (rather than blockchain-level) changes as well. For example, in order to use a project like Uniswap, you need liquidity. If large liquidity providers move to newer versions, the majority of people will follow, leaving the remaining ideologues dealing with frustration at best, if not complete lack of ability to complete trades.

As it turns out, true trustlessness, decentralization, and censorship-resistance is hard. Many so-called defi projects sacrifice these ideals to varying degrees in exchange for ease of development and other benefits. However, a lot of people simply aren't aware that these tradeoffs are being made, and are not cognizant of their resultant risk exposure — particularly when it comes in the form of counterparty risk that is a degree or two removed.

As governments and regulatory bodies have been more actively looking for ways to enforce regulations on the crypto industry, and to determine whether projects are truly decentralized for the purposes of regulation, it's likely that we will see more and more wedges driven into these cracks that have emerged.