Who Pays for Security Audits, Bounties, and Incentives | Benjamin Sepanski - Veridise

All right, thanks everybody for joining. My name is Ben. I'm the CSO at Veradise. Uh, if you guys don't know Veradise, we're a security firm. We do audits and architecture reviews and develop custom tooling for ZK, smart contracts, and blockchain infra.

But I'm not really here to talk about that today. Today I'm here to talk about who pays for security and how this affects audits, bounties, uh first let me say if you agree with some of the ideas I have here, please find me after the talk. Let's uh chat or if you disagree, I'd be interested to talk as well. Uh but before uh we can talk about what we agree or disagree on, we should talk about what security evaluation looks like today. So here, imagine you are an investor.

You have some new protocol you found out about. You're interested in it. You want to invest, but you don't know how to assess whether or not the protocol is secure enough for you to put a sizable amount of funds into the project. And you may go about following kind of industry standards of checking for a couple core things. Do they have an audit or some contest done?

Have they put a bug bounty in place to catch things missed by the audit? Do they have monitoring in place to react to exploits live? All of these are going to be kind of the key factors today where a lot of people evaluate whether or not a project is secure. And I'm going to focus a lot of complaints today on how people use audits and bug bounties. Now, there are some kind of well-known complaints that everybody is aware of.

things like audits are run by third parties in a timelimited setting and limited in scope as well often excluding things like previously audited code or integrations uh highly permissioned actions. Bug bounties also run are run by external researchers and are passive disclosure channels which rely on discretionary payouts and and can be uncertain. But I really think the core issue with using these to assess security is not actually from these points, but rather from the view of how they provide security as a status to a piece of code. So before I talk more about what I mean by that, let me ground this discussion by picking out a couple of exploits that have happened recently on uh to teams that are doing their best to follow these industry standards and would kind of pass any reasonable evaluation against these standards. So, these are two recent hacks that happened last year.

Um, if you're not aware, I'll provide a very brief summary. So, you're suffered an exploit uh due to some unchecked underflows and rounding errors where around $9 million was stolen. They did have a bug bounty in place and they did have an audit done on this code. Uh, this was legacy code actually running safely for many many years uh managing millions of dollars until an exploit happened years down the road. In kind of similar fashion, Bouncer was also exploited last year leading to hundreds of millions of dollars lost.

also had a large bounty in place and had uh multiple audits from multiple firms over a course of years during which it was managing significant funds even applying formal verification efforts which unfortunately and as mentioned in the reports did exclude exclude kind of rounding effects and rounding was noted by different analysts as they continued to analyze the project but at the time of the audits they did not realize that rounding could be exploited in the ways that we know now that it can be and my point here is is not to pick on any particular protocol, but to point out that these are projects following industry standards that were audited, that did have bounties and were exploited years after uh managing funds successfully. Now, you may say I'm picking on edge cases. I can do the standard uh security talk of putting some big numbers on the screen. It has also become industry standard for multiple billions of dollars to be stolen every year in the crypto space. of these only about 350 million I'll say only about 350 million which is kind of a funny sentence uh come from directly from smart contract exploits which are covered with these so-called industry standards or security assessments and 190 million thereof come from people who have audited or maybe had uh changes after an audit but are generally trying to follow these standards and maintain security in good faith and there are two things we should note here one is the industry standard is not nothing it's still good to get audits you know it's a smaller fraction of exploits that are happening on projects that are audited.

But there are two interesting things. One is $190 million is still a lot. It's still a lot of funds getting stolen. We can probably do better. I believe we can.

And the second thing of interest is that these audits generally cover uh smart contract implementations. And if you compare, you know, $348 million to $2.2 billion, that is a lot of threats that are not getting covered when we are evaluating whether or not a project is secure. So let's talk about why we're doing these things this way. If you know there is a better way or if this way is not sufficient, why are all these smart people who are working together uh approaching security in this way and assessing security in this way?

The standard has evolved over time with our approach to blockchain. When we first uh thought of and conceptualized smart contracts, we pictured immutable code operating transparently, managing funds. And it made sense that if this isn't not going to change, we just really need to work hard to get it right the first time and then the code itself will be secure. But this is not really match modern practice. We see that upgradability is very common and often even necessary so that teams can respond to bugs which are not found by audits.

And as noted in these case studies, new exploit classes continue to be discovered. code that is secure today under kind of all known threats might not be secure in three or four years as attackers continue to come up with new lines of attack. Now this kind of first myth has led us to the second one which is that audits can let us buy security. We can kind of contract this out because as long as the code is secure the project is secure so why not bring in the best experts we can find to do this for us. Now this is lots of problems in itself.

You know obviously you have to trust security firms. Uh you have to also worry about the scope that security firms are applying. Uh tooling may scope out key behaviors. Uh the threat model may include governance actions or assume that keys are never stolen. This is a wide variety of threats that are just not covered by audits.

And finally, the approach to handling bugs missed by audits has focused on incentives like bug bounties, which are appealing to somebody economically minded, but have failed to provide the proper amount of incentive to prevent uh attacks at a sufficient rate. Large fractions of projects that are exploited do actually have bug bounties and people continually choose exploit or sell instead of report. Whether this is because of worries about discretionary pay discretionary payouts or nation state actors who are not worried about state reprisal uh this is something where we need to improve. So if we're going to think back about like what led us here and now try to think about how can we break out of some of these paradigms we might look to other industries for inspiration. And one thing that we'll find if we look at other types of assessments how other industries are assessing whether a project is secure we could look at information security like sock 2.

We could look at payment card industry uh standards or federal governance around uh cloud providers. All of these security evaluations look very different. They are evaluating controls over periods of 6 to 12 months uh not a single snapshot of code. They are requiring annual reassessments. These reassessments have continuous monitoring requirements have requirements that the team is engaging continuously with third parties to do pin test or secure code review.

They're uh evaluating what types of response incident response capabilities are present and what types of continuous review is being performed. And the point here is that when somebody is trying to determine if a project is secure, they're not looking to a certain commit on the code. They're looking to the processes implemented by the team that security as assessed by these other industries is something maintained by the developers, not a property of the code. So we solved it, right? We're done.

We just need to do these assessments and then everything will be great. All the hacks will disappear. Um this is of course not the case. You know, first of all, even a perfect process may not produce perfect results. Even if we are checking for internal security teams and ongoing security budgets and are doing conser uh continuous reviews and testing and monitoring, things may slip through the cracks.

Zero days will still exist and be found. But there's a bigger issue here of who is going to do these assessments. You know, how can we actually get people to work this way? Why aren't people already doing it this way if this is something that could actually reduce the frequency and magnitude of hacks? And to this, I think we have to look to incentives.

And this is where we'll start talking about security funding. Before we can start talking about funding, we need to talk about risk. For funds to get stolen from you, you've got to get funds first. All right? So if you're an investor and you're investing and you're evaluating the security of a project, you're worried about the risk from a hack.

If you're a developer building a project, you have another risk that you have to work on and worry about and that the hack risk is conditional on and that risk is whether or not your project itself blows up. You know, there are no hacks without survival. And in an area with network effects where firstto market can dominate and liquidity can determine protocol viability, this risk can be often the primary concern for developers where if you spend the extra resources that could have been spent on marketing or faster development or more AI credits, you might not make it to market first. Your protocol might not survive. Your secure secure protocol might not be securing any funds.

Added to this is that developer funds and protocol funds are often separate pools. The funds that are kind of fully wholly at risk are not the same funds that developers have, you know, raised from their VCs and are operating off of in their day-to-day development life cycle. If we look instead of investors, there's just a different risk profile. The people who are putting their money into the project bear the immediate downside of theft. When the money is stolen, that's their money and now it's lost.

And even worse, as we've seen for code that is completely stagnant, completely immutable after launch, risk still continues to accumulate. Uh obviously mutable CH code may change, integrations may ship, but as exploits continue to develop and attackers continue to become more advanced, risk to a project continues to accumulate over time. Now, those risk profiles alone are not a problem. What becomes a problem is when you mix it with how security funding is managed today. Primarily devs are the funders of security.

Devs bear the cost of audits. And so when they are working against this firsttomarket risk, security spend has to rationally be capped. On the flip side, auditors are competing for these clients, the devs, and so they need to make sure their products possible to their clients, which means that much of the focus of the auditors is directly on finding bugs. This is just a different thing than assessing the security of a project from the point of view of an investor because the developers are focused on getting to launch and primarily worried about this first risk, this first way that they can blow up. Additionally, these separate funds make bounties much less effective.

Protocol funds may be much larger than the pool of funds that developers have access to to set aside for bug bounties. This makes in addition to discretionary payouts, the financial incentives of bug bounties much more difficult to make effective as an alternative option for uh grey hat researchers. And so with kind of all this in mind, I've seen like where assessments are not going to happen just from developers naturally, where they're not going to go straight to this uh process-based approach that is going to take more time and leave them behind on the competition. We need to turn and look at the people who are bearing the most risk. And I'll talk for a minute just directly to investors, people who are putting their funds into these protocols.

First, you should know that you're already funding security. If you're deciding to put funds into a project, deciding how much, you're kind of implicitly pricing the risk in some way. If you're funding developers and you're funding those pools of developer funding, you are indirectly funding the audits. And if you're ever invested in a project that gave a so-called white hat bounty, those protocol uh protocol funds are the funds funding that bounty. So investors are already dealing with and having to fund security.

What I would argue is a good next step and something that people could use to bring the crypto industry closer to a state where billions of dollars of theft is abnormal and not standard is for investors, people who are putting funds into protocols to fund security evaluations. Now, you might think here, oh, the auditor wants more people to pay for audits. That's not what I'm pitching. I'm not saying investors should pay for audits. I do think the developer should still pay for the audits of the protocols to find those bugs to help them make the code itself more secure.

What I am talking about is in a different type of an assessment. An assessment of the developer security process. An external party focused on process evaluation. Making robust and continuous security process a requirement to receive large amounts of funding by only putting their funds into projects which are able to pass these assessments. This radically changes what the incentives are for developers by setting an external standard that they must play against.

Devs must build processes to pass the evaluation under this model. Which means that when you are competing with other teams who are underspending you on security, you are not necessarily losing out on that first tomarket risk. You still have an advantage because the security process that you build out is something they are still going to have to build out eventually in order to obtain the funding that is necessary for to have sufficient protocol liquidity for the project to survive. So uh this is my pitch. If any of you are investing in protocols, think about how you are deciding whether or not a project is secure and consider not looking to audits and bounties as a serviceable and sufficient way to assess project security.

If you're interested in funding process over artifacts, please reach out to learn more about audit hub's approach to continuous security and how we integrate that into the regular development workflows of our clients or just find us online or find me after the talk. Thanks.