Krzysztof Urbansky - Beyond audits - what recent hacks should teach us about securing DeFi protocols

Hello everyone. My name is Chris and I'm from Altobe. At L to beat, I'm responsible for governance and partnerships. And if you don't know L2 to beat, we are this community watchdog that oversees uh Ethereum L2 space. So all those questions about force transactions I'm happy to discuss it uh also afterwards I can I can fill you in on how exactly does it work and which O2 actually do implement it but you can check it on our website as well.

So that that's exactly what you can check um on L2 to beat and today I would like to tell you a bit about our approach to risk analysis and to security guarantees analysis and how how should we analyze um protocols especially DeFi protocols beyond audits. Uh but let me start with some history. So 10 years ago almost exactly 10 years ago on the 17th of June um I don't know how many of you remember that uh this was like I was this was a month after I decided that that crypto is really a thing for me that I I saw this project the DAO and I realized hey this is super cool like this is actually much more interesting than the Bitcoin itself. So I will I will get interested in that like I will I will dive into it. And a month after I got interested uh the DAO was hacked and this is Grief um announcing on on uh on the um internal channel that the the DAO got hacked and people should should basically be withdrawing the money.

Um this was at that time this was the hack that was um the biggest hack in crypto at that time. It was um as big that we had to hard fork Ethereum in order to um to survive and the way that the DAO was exploited was pretty novel at the time. Um it was the so-called re-entry uh exploit and it was something that that we simply didn't know like you know the whole crypto space was very early at this time. All these smart contracts that we were writing like these were the first smart contracts ever written and we didn't know much about how to how to do this stuff. So this back the rean fancy back like once explo once discovered this became like the canonical exploit in DeFi the canonical exploit in smart contracts and since then it's like it's the the first thing that any crypto protocol like needs to check like you can see this is the um solidity by example tutorial and it's literally the first thing that is being being um taught in the hacks section.

Um so and and this also started the requirement for auditing smart contracts. So the the the DA hack like made it that any um any future D5 protocol had to be audited to see if if it's not vulnerable to to those exploits and hopefully none other uh D5 protocols will be affected by that. But then 10 years into the future, a month ago, almost literally a month ago, um Celt was hacked and we we realized this this is one of the first um mentions of the CDA hack. And as you can see, it was still quite optimistic in terms of like how much is the um how big is the issue. It proved to be like almost three times bigger than um than initially assumed.

And this hack was different. Uh this this is this this this this vulnerability was different because it now we know that it was a vulnerability in not in the protocol itself. It was vulnerability in the interrupt mechanism in the OFT um mechanism delivered by layer zero. Uh and if you were following layer zero from from the very beginning, you could remember that uh like so so like layer zero basically announced that yes this is the this is this is the exploit of kelp dao in our mechanism but it's their fault. It's their configuration that uh that was faulty.

They made it very clear that layer zero protocol functioned exactly as intended. I would say that if the functioning as intended means that people lose like almost $300 million then probably we should change our intents. But the intents initially were different. This is the original white paper for layer zero. As you can see that initial promises were to to provide security.

Lazy was supposed to provide those security mechanisms but later they changed the the the framing. They said that they allow protocols to design their own security uh mechanisms. So they are not responsible for those protocols security mechanism. They allow them to to build them themselves and they are providing this mechanism that they call distributed validator networks DVNs. However, if you if you look closely at what this distributed validator network is, it usually is not that much distributed and usually not that much of a network.

Um, it more often than not it is simply uh a multisyc because technically a DVN is just a smart contract with some off-chain infrastructure. any offchain infrastructure. They have the examples like it can be super sophisticated if you if you do implement a super sophisticated like zero knowledge proof mechanism. It can be like that but it can it can also be like an EOA or a multisc. So, you know, many people were surprised um when when this happened.

People were like, "Hey, like weren't you supposed to actually provide those security guies? Like, weren't you weren't you supposed to be securing uh those protocols?" Actually, you know, who would have guessed we we did. So this is an article that we wrote three years ago, more than three years ago when we actually were calling out layer zero that their promises of of providing security guarantees do not stick as we said exactly what they are using right now to defend themselves that uh that layer zero by itself does not provide any security guarantees and therefore from our perspective like if something is using layer zero it does not guarantee you that it's safe. So any protocol that is using layer zero should be considered risky until proven otherwise until you can prove that this is actually secure.

Layer zero does not provide you any security guarantees. And of course as you can imagine when I published this three years ago I received a strong push back from layer zero team. I don't blame them. No they have business to run. Uh but but this is still true and this is unfortunately um uh still the case that you have to validate yourself whether layer like whether the protocol built on top of layer zero actually is safe.

So we have this you know we have this mantra at L2B trust code not claims do not trust whatever the provider promises you that they deliver like check actually the the beautiful thing about crypto about blockchain about the open like source available nature of what we do is that we have can actually verify whether those protocols do provide those security guarantees or not and that's what we do at L2 beat. So of course throughout the years we saw many many requests can we have L2B for D5? Can somebody create L2 bit uh for D5 even you know even in some like weird characters that I cannot understand but I still know that they are requesting L2 for DI. So we started thinking okay what would it mean? What would it mean to to to do L2 for D5?

What is L2B really? You probably know L2 to beat from this website. So, and in this website we provide risk assessments and like security guarantees assessment for different O2s. If you hover over our risk pizza or our stages, you will see the explanation of what are they missing in order to be treated as for by by us as good enough. But this is actually not like the the this website.

is just the tip of the iceberg. The actual meat behind the L2B is this website. Uh it's discovery where we look into the actual construction of those um of those L2 systems. So this is basically what we use internally for our work. This is arbitrum and we analyze those protocols to see where are external dependencies.

Are there any and for L2B for the L2 assessment that we do we are trying to look into those external dependencies to see if there are any because for L2s to be you know the promise of L2 is to inherit Ethereum's decentralization um properties therefore the in an ideal world you would have non external dependencies you would like your security would be guaranteed by Ethereum. That's what we are looking for. And here you can see that for like each of those boxes is a smart contract or an EOA that we analyze actually what is happening there and how they all rely on each other. So how would it look like uh how would it look like applied for DeFi? We we did this experiment.

So we we ask ourselves okay can we can we do something meaningful? Can we do something valuable by applying the same um methodology to D5 protocols? So we looked into a receive into kelp da by the way it's funny that it's called kelp dao. I don't know what dao they they they have like it's just a name. Uh but this is this is what it looks like.

And actually this is not a full picture. This is only partial picture of what RSF looks like. And all those different colors mean smart contracts for different systems. And this this red color is just kelp dowo uh uh smart contracts like provided by them and and secured by them. And here we can see already some external dependencies.

So this thing in the middle is is lido. Of course this um keld is a restaking protocol. So they are using state if and this is um um ls if and if x tokens as external dependencies. And here you can see also a you know you know we know that a is involved in this whole thing. So this is the dependency on a we see it from exploring onchain configuration onchain deployment of those smart contracts and what you can see here is layer because whats is it's a reststaking uh protocol like restaking token that takes your and uses it in layer infrastructure to secure other things that could be also secured by um Ethereum by so-called called AVSS's and here is the uh here is the you know the the demarcation line between um Kla and I layer and here is the different ass directory so different als you could think like what is it really So we looked into one of them and if you look under the hood, if you scratch the surface a bit, you will see that in those AVSS's there are also like EOAs like single EOAs that are owning those.

So you know while everybody is talking about the the the failure of the Deviant configuration that Arisdal Kda had, nobody's talking about this like you know the risks are beyond just the risks in uh in Interov. We had risks internally as well like we don't know we don't know how like this is secured if this EOA can actually rag your stiff or or not and of course a receive is a in you know crosschain token uh it's been actually the crosschain feature was implemented on top of the token built on um L1 and this is this is the the connection between different uh chains it's Ethereum base. Of course, it's it's multiplied for every chain that they intert introduce and each chain configuration can technically be different. So, in order to know what are you dealing with in terms of security guarantees and risks, you have to analyze all of them because they can be different. There is no like it should like you would expect it to be the same for every chain, but nothing guarantees you that.

So you have to check and here there is a there there is this DVN of course this is four out of four today it used to be one out of one and that's that's where the issue was uh and you have to you have to check this because this is this is what you are trusting right now in terms of this um of mechanism and the important thing is that this configuration needs to be constantly monitored because RS E was configured differently initially when it was deployed it was configured as two out of two but at some point they realize okay why why to you know to to put some so so much hustle into into having two DVNs when we can have one like isn't one good enough so they changed this configuration from two out of two to one out of one did they announce it anywhere of course not like nobody is bragging about like decreasing their security guarantees. So it's not enough for you to just check the security profile of any OFT token at any given point in time. Yesterday it could have been safe. Tomorrow it might not be. Today we know that RS E is four out of four DVN but do we know if it will be like that tomorrow?

So that's what we do like that's what we analyze um in L2. So we not only create a snapshot of the system, we monitor it con um con constantly to see if any security related property changed for this system. And we um we monitor more than 100 systems uh today and I can tell you that they change more often than you would expect. Like every single day we have anywhere between 10 and 20 different changes in those systems that are already filtered out for being like something that we should probably look into and all those changes need to be analyzed to see whether whether they have some impact on security. So and we keep this change log for all the systems that we monitor.

This is arbitrum and for arbitrum we have like three years worth of of of change log of what happened onchain on arbitrum and each change is described by us like this is currently this is not public but we are planning on actually making it public because this is used internally by us to see what changed like every day our researchers are looking into the change log from the previous day and they are analyzing okay what does that mean what does that change that we observed chain mean for the security um uh you know security profile of the given um given uh protocol and this one this this this change is actually something that was expected it's a vote like implementation of the vote that was passed by arbitum dao this one is interesting you know we you you we we could see when the uh arbitrum security council initiated the freeze of stolen funds on um on arbitum. We saw it on chain as well and we were able to market as as this but we also see things like that that some important multisc in this case security council is changing and in this case this was like three security council members rotated in um after the elections. So but but sometimes we see such change that we didn't expect. We see a change in multisig or in some threshold or in some fees um structure that that have an impact on um on security and we go to the protocol and ask them hey guys like why why is this change happening? Like what's happening here?

Should we raise an alarm? Should we warn the community? And just the fact that we are looking and they know that we are looking uh usually makes them much more uh reasonable in terms of what they are pushing um on chain. So what about stages? Because I I showed you the meat behind the scenes like how do we how the sausage is made and how do we analyze those protocols but everybody knows L2 from our risk assessment and for st from stages.

So can we apply the same requirements that we have for um for like L2s uh to DI and our answer is not really uh at least not now like we didn't figure out yet how to how to do it and why because for L2s as I mentioned what we are looking into is limiting the external dependency and limiting the um the the vectors in which the external dependency ies impact the protocol. But this is not practical for D5. When we looked into D5, you know, composibility and interdependency is the core and essence of DI. Defi does not live in isolation. Isolation.

Defi is meant to be like composed of Lego bricks that you stack on top of each other. So this in this dependencies in DeFi are built in the whole concept of of what we are building. So we cannot aim to limit those dependencies and to get rid of them. We have to simply be aware of how it is composed and how it stacks up. Another thing is that while in L2s we know what is the acceptable uh and and what are we aiming for and what is acceptable in terms of those external dependencies like we define this we we say what kind of training wheels we accept for stage one for stage zero and when do we say that something is not even an L2 from our perspective if if it doesn't have a working proof system for example we don't even consider it a proper roll up a proper L2 but in terms Defi the risk assessment depends on your risk appetite.

You know um Birkshshire headway has definitely different risk appetite than any random DENE memecoin trading fund, right? And something that would be like if we were to apply standards of bakes headway and Warren Buffett to to our risk assessment for DeFi, we would say that we we would have to close the shop because he he doesn't accept any risk that that we are exposed to in DeFi. So it's hard to say like it like the the actual risk profile like risk assessment depends on your risk appetite and like what are you actually looking for. So it you have to adapt it to your needs. Also the risk of losing funds may not be dependent on the protocol itself as in in Kulpd if uh case you know the the protocol that is most affected by this incident is a did a you know fail in anything I would argue that to some extent like they didn't have limits and how they and any you know um any thresholds of how much can they be uh impacted in one go.

So that's why the this had such a huge impact on them. But otherwise like their they weren't exploited. Their protocol did not change in any way. So it wasn't their fault like their their risk assessment would be probably still good yet still they were exposed. So like your exposure depends not on your protocol itself but basically on everything else like the weakest link is the the thing we are looking for.

And last thing that why this is a bit different than what we are doing in in uh L2 assessment and is that in this case like in L2's assessment we are trying to limit and to limit off-chain uh dependencies and offchain components. We are looking into them but not that deep as in like if there are offchain components we already kind of dismiss uh such a system as not a proper L2 from our standpoint or like we apply certain um criterias that we have for those external dependencies but we are not checking them um that much uh inside in this case in defi it cannot like we have to go deeper we have to um analyze them so we are looking into that as I said for assessments and for stages we are not there yet. We we do not know how to uh how to give a stamp of approval on any any protocol any any solution. But what we can do right now is provide more insight into where are you exposed, what are the things that you should be monitoring. um where are the components that that could be risky for you outside of your ecosystem and most importantly we can monitor it so we can alert you when those external dependencies change so that you should probably reassess your risk appetite again so I would like to add one more thing I'm already wrapping up but I would like to add one more thing so I started with the DAO hack u to to start the story and recently we saw the huge comeback back of DAO.

So DAO came back with the funds that they still have in their treasury and they started the uh DDA security fund where they support um security related projects in this ecosystem and this is a quadratic funding matching pool. So the work it work the way it works is that if you support a project like us but there are also other great projects that you can support like there is defy scan there is uh hacken proof you can see there is a ton of projects that really deserve your support. If you go to to give to this um donation page and you donate even small amount the only important thing that it should be above $1, then it will get matched from this uh it will get matched from this uh from this pool uh for from from the DA security fund. So I would ask you if you like what we are doing, if you you know if you value what we and others are doing, then please go to this page today because it's timesensitive. You have only less than 24 hours left to do this.

Please go there and find your favorite projects even with $1, even like ideally I don't know $10 some. We don't ask for much but we ask to for you to do this because the more supporters we have the bigger the matching we can receive. So basically I don't ask you to to to give us $100 but I would ask you to first like first of all go there validate that you are eligible to be matched. need to pass certain KYC require like not KYC anti-IBLE requirements so that this is not gamed. Uh please pass those requirements and donate to your favorite projects even small amounts like $5, $10 and ask your friends to do the same and do it today.

Yeah. So do it now because there is less than 24 hours left for this.