Future of Smart Contract Security Audits: REKT or WAGMI

foreign [Music] I work at optimism I focus on security in our protocol I previously was an auditor and co-founded diligence with Gonzalo and have had the Good Fortune of working as a client with these three gentlemen and not yet sir Torah but have had an opportunity to get to know them really well so I really feel in my element and I'm excited to have this conversation so I'll let them introduce themselves quickly if you could you know give us your names and tell us about your firms and what makes you special I can go first hey guys um and thank you Richie yeah if you're watching um yeah we co-founded consensus diligence almost six years ago now or going to that um me and John who's now at optimism unfortunately and the consensus diligence is a crypto native security firm that has for the entirety of its life done Security Services and has also a product side which you might have seen so mythex and more recently diligence fuzzing and a bunch of other smaller tools that are all open source that you can check on our website hi everyone my name is Chandra and I am from sartora sartora is a automated auditing company you can think of it as that way what we do is we use formal verification techniques to prove certain properties about your smart contracts and I'm personally more on the r d side and one of the things that we are focusing on these days is how do you have more Trust on the specifications that you write because at the end of the day that's really what we're using as the ground Truth for verifying your smart contracts so if you have any questions about formal verification I would love to hear about those everyone my name is Maddie co-founder and director of Sigma Prime we're in information security consultancy focused on blockchain technology doing a lot of smart contract auditing as you can imagine but very comfortable diving deeper into other layers of the stack as John mentioned layer 2 systems new l1s and whatnot we're also the founders and maintain owners of Lighthouse the rust implementation of the ethereum consensus protocol some of you here I'm sure are running validators some of you may be running Lighthouse as well we had the merge a few weeks ago my six milestone for everyone massive milestone for Sigma Prime and yeah hi everybody I'm Jonathan Alexander I am CTO at open Zeppelin open Zeppelin we are a security advisor security auditor we also work on open Zeppelin contracts open Zeppelin Defender and we are core development contributors to Florida and hi everybody I'm Nick Selby I'm with trail of bits we are an audit and research company based in New York but people are all around the world uh in addition to audits we have a research group that that mainly works on just sort of pushing the edges of of a bunch of different Technologies including blockchain uh and where are things going next and we make some pretty popular open source tools like Slither and Echidna which probably several of your you know familiar with thanks I'm glad to be here awesome thanks everyone uh so first question I think is like maybe a bit of a softball just to get us going uh and I'll I'll pose this to Medi first and then just let the conversation get going so it's been three years since the last Devcon what do you see as the biggest changes in that time maybe it's something you're excited about something you're worried about we'd love to hear it yeah interesting um let's see I think what I've been noticing is the quality of the projects that we've been reviewing has somewhat improved over the past three years uh this caveats obviously we still get the occasional you know D5 projects with you know lack of access control and the typical unsafe and safe external calls and re-entrances but I'd like to think that thanks to some of the tooling that's been available it's been doing great work on that front developers have a lot of abilities these days to pick up the low hanging fruits contracts are getting more and more complex in the D5 space particularly some of the lending protocols that we've been reviewing are pretty hectic but yeah yeah I think you know the amount of money getting stolen on a weekly basis these days is probably greater than what it was three years ago I'd say but it's probably due to the fact that there's just so much more so many more people getting into the field a lot of new projects being deployed and obviously you know our time is you know constrained we can't unfortunately service everyone all the time um but yeah so goods and bads I guess we've seen a lot of projects come and go too and a lot of money come and go so yeah I think that like even I fully agree with Maddie the quality has definitely improved in I think the bear Market kind of helped at that okay anybody have uh something different to say than better quality I have a mixed bag all right so like three years ago it was it was a simpler time uh things were a lot easier to look at things were a lot more self-contained uh and as you just said right the the complexity the the Maze of dependencies the uh the the complexity has kind of exploded at the same time that there's more and more value so the stakes are much much higher I think on the other hand you're right the tooling has gotten a lot better but it's also I think all of us here working with with firms that are we're getting used to the we're used to the risks we're used to the threats we're we're inured to the patterns we we can see them a lot more clearly everybody here has a lot more experience in in finding things uh and on the other other hand um three years ago I remember being describing to board members of a of a pretty large crypto company uh about nation-state threats specifically talking about North Korea and they kind of looked at me with blank stairs and they were like well you know so what and I think that over the past three years that that nation-state involvement has gone from a kind of a risk to a definite and definable threat yeah uh Chandra do you so well this is my first Devcon so I wasn't there in the the first the one three years ago uh but what I've noticed is that um there seems to this is a very fast moving field and there's like a new terms new buzzwords new Concepts that keep getting thrown around a lot uh but I think you know the main thing that I would uh you know say here is I think it's important to remember that at the end of the day uh like companies like surtora or like you know formal verification companies or like any company that is building tools for uh verifying smart contracts they actually rely on a very fundamental uh concept and I think that even though this field is very fast moving uh the fundamentals are still the same and you know those ideas uh go really far and I'm actually excited I think that we should be able to scale some of these techniques uh pretty well to some of the more recent uh Concepts that are coming around in this field nice okay I'm gonna I'll give you the next question but let's we'll keep it moving uh so yeah so we're done with softballs um Jonathan what responsibility do Auditors bear when code they've reviewed has been exploited thanks for Daisy hey nope that's targeted um you take a sip of water for yeah well I I think um first of all I think all of us because we uh I think everybody up here we take our work so seriously that the first order of responsibility of course is dealing with the issue that's found and I think that I know we at open Zeppelin make every effort if we've had situations I'm not going to say we've had situations where things we've missed have been found but customers we work with maybe have a situation arise and you are a trusted security advisor to them you're going to Rally to help them in the situation and that may be in discussing what are the potential fixes in reviewing fixes and even talking about public Communications and that is a uh I think if you are with a trusted you know we view these relationships as Partnerships then that's what what we're going to try to do um I think that in the case in a specific case where something is missed in an audit which sometimes um like we've had some situations where uh we'll we'll do an audit and then before thankfully pre-release an issue is discovered that was missed in the audit maybe because there was uh another you know set of eyes on it or maybe there was internal testing and in that case uh we always view our responsibility is to do a very thorough postmortem on what happened exactly what was found have our team analyze itself as well as get other people in our team and other get the customer involved so that we can all learn from that and figure out and explain to ourselves and improve our processes going forward so I think that's the very common uh approach so so we've heard from Jonathan that it's I I think this is just like late to the preface on the same topic but Jonathan has been open that a top-tier auditor can miss something and so you're lucky right like it got caught before it hit production that can happen to anyone so let's like put that out there is anybody on the stage like actually had something they've added be exploited in a significant way I'm curious not really but we've had stuff being disclosed after it's in production yeah like I I think that like we establishing and we've said this many many times we're not insurance companies right so that's the Baseline but there's an ethical and moral code that like that we uphold to and so if something happens obviously this is also Case by case on a case-by-case basis it should be analyzed differently because if it's if it's our the if the problem was created by us specifically then we might react one way whether if if the if there it was just like a Miss we might react some other way but at the end of the day we'll try our best but we're not obligated to anything right nor can we be because that actually destroys the entire relationship well I'll go because I'm just gonna just jump onto it uh we we actually don't I don't think that we have a different way I liked what you said about the moral responsibility I think it's really it's really important um I I don't think that we have a different um process we have we have a pretty stringent process to to determine um whether whether something was missed but but much more important is what happened and how can we help and how can we reach out proactively they're going to be an incident response mode they're going to be their hair's on fire there's going to be media involvement they're they're in a world of hurt a lot of us at trail of bits have worked in the incident response field so we're we're very accustomed to understanding what they need to do right now uh to stop the burning to stop the leaking and and get things done so our first reaction is to to refresh ourselves as to what we know about this project and then uh reach out to try to offer as much assistance as we can I'm very very pleased to say that the entire Rec leaderboard does not have our name on it today that that will change but you know as of right now but but this is a process that we take very seriously because uh it it can happen there are no guarantees in audits we we always do our very best but I think to stick with the customer when they're in a bad time as well as when they're in a good time is the most important thing sorry Jenna just give us like an insight into what that process looks like like just one or two actually the the process of choosing whether to help the customer or not in in one of those situations so the the helping the customer I mean I let me be clear because I made this same mistake in Stanford I don't mean free uh necessarily right like we we will help obviously for nothing there are certain things that we can do as Auditors and and as experts in in the field where if there's more services required if it's fixed reviews if there's other things but the first thing we want to do is reach out and say hey we saw that this happened do you need any do you know do you want to talk to us about stuff that you're that you're doing right now I think you you said that pretty well can we agree that audits is probably the the one of the worst terms to describe what we actually do I feel so bad using the word auditing or audit every time I do but these are really time boxed security assessments let's just all agree on this as John was saying these things are not bulletproof right um bugs will be missed and we allocate a certain amount of time to do our best to find these uh vulnerabilities in your code but guess what if you give us double the amount of time chances are we'll probably find more bugs um so let's just you know make sure that we're all clear on what we do up here in terms of Security reviews and security assessments okay and by the way that's that's not just a cya it's not you you said the word partnership or you said the word partnership um people wait a long time for audits it's a very important part of the process of going live with with a product especially a product where a lot of money you know if it goes bad a lot of money can go out the door uh customers have their own priorities customers have budgets customers have realities and usually it ends up that we're not negotiating actually about money but about time we're not negotiating about money but about the number of Engineers who can be looking at it so while we can't guarantee that a three-week two engineer audit will find all the bugs we can guarantee that a three week two engineer audit will find all the bugs that can be found by three Engineers by two Engineers working for three weeks and that that's a different sort of guarantee yeah okay um so I think I think Medi set this up well in that uh by saying like like just the Magics of what this is what we're doing how is it defined um so we're saying time box Security reviews now do we need do we need a set of Standards to make it clear whatever word we choose do we need to have a set of standards for this and I'll give it to Chandra to to take a crack at it so standards for Auditors you mean or for yeah I mean one that's a good place to take it as well if you like I was thinking more for what an audit is for what what needs to happen during the course of this Security review well um so you know I I can perhaps speak from a more automated auditing point of view but then I think some of the ideas probably also apply to human-based auditing uh I think the hardest thing is to come up with what with the specification right like even when you're auditing like what are you actually looking for I think that's uh one of the main questions that you should very carefully think about right uh and you know I think one of the things that's kind of uh difficult for auditing is uh it you it's hard to be very precise and if you're not precise then you kind of think that you did a proof or like you have some kind of guarantee that this bug isn't there but I think you know unless it's very easy to uh feel like you you know audited really well if you don't have the specification correctly in your head and um perhaps a related uh thing is uh in formal verification there's this notion of trusted Computing base and I think that idea also uh you know maps to auditing like what part of the code did you audit what parts of the code did you not audit right and you know you can only say confidently well with some confidence that the code that you did audit maybe doesn't have some vulnerabilities but the parts that you didn't you can't say anything about them right so it's very important to be very explicit about what the TCB is and what you verified and what you checked or whatever you audited anyone else I would say I'm a little frightened of the word standard in what we do because I think sometimes it comes from the place of people hoping that we're going to simplify down to some very simple set of rules what needs to be covered and then anyone could cover it and so I do think we need common language we need in many just helped us with some of our common language and understanding um so that we're working together but I think what's important is um if you're if you're a team and you're engaging auditors um it as opposed to you want to have common language about what will be done what what won't be done but this is people also and people with experience so I think um getting to know who are the actual Auditors and not just I think we were talking about this before not just the team or the you know the label but who are the specific people what is their experience because those are the people you're going to be working with and and what's more important probably than standards is agreement with who who's going to provide you the security review and who are those people and what are their skills and what are their ex what's their experience yeah so I'm gonna like that I'm gonna go to the other side of that the interpretation of the question of standards and well I'm re I'm going to just steal the phrasing uh from of Rickard who I was talking to the other day um he said who odds the Auditors and and so you know as a project and I experience this now as somebody who's audited before and understands the industry far better than most still scary like talking to like trying to evaluate the auditor like for the code that I have that I want looked at does this is this the right audit firm and do they have the right people available when I it's there's so many moving Parts it's crazy um but back to that who are as the Auditors like is there some way that we can that we can or should be holding I don't want to go back to like holding otters accountable and the skin in the game thing that we can go around round about but yeah like how can how can we help non-honors evaluate the Otters that they're working with should there be some kind of third-party Watchdog that's ranking you all or what like there's a I think it's a place like this a good plug here would be like to talk about each trust and I think we all participated in the standard in one form or another and I think it's like a dangerous proposition because what the the proposition of each trust is that there's levels of how secure your code bases there's like kind of a risk assessment of how your code base will look like after you have done certain things right so so this this uh this is part of the eea the Enterprise ethereum Alliance and I think it was uh Tomlin who started it I think it is a dangerous proposition but it might actually be helpful if you are not super embedded into the security industry right again it has it is useful but it is it should be taken with a grain of salt because like if you take level three or like formal verification as I'm gonna say it that will probably be a disservice to you so I'm mixed based on getting standards for what a good security practice looks like all right this is a scary one so oh but I'll go ahead I actually think in terms of security work they're and and I think about other niches of Technology the work that let's say these teams here do is very public it's very public so in some sense we're watched yeah very closely and so this idea that so our reputations are on the line all the time all the time and honestly I know all of us up here we lose sleep over it right like and and I will say too that like it opens up when we work and opens up on contracts and we've had issues and you know so we're our reputations are very public and we're all glad we're not on the rec leaderboard and um so we can talk about I think it's perfectly healthy to have conversations about how could we provide more transparency and more info but I think we are actually very yeah well watched right now okay yeah now now there still is the thing about what's the right match for you as a project and I think that's an important consideration yeah yeah and I like how you said like you lose sleep over it I think you know everybody in this room loses sleep about either the code that they're running or the code that they're auditing so should all go easier on ourselves um so I just want like I think it'd be fun to get some questions from the audience uh so if you have a question raise your hand uh and meanwhile while the mics are getting handed out we're gonna try something overrated underrated lightning round no people no firms these are Concepts uh okay so where should we start uh Nick uh plain English specs overrated or underrated uh underrated twice it it I was actually Jocelyn Feist is here is the head of our blockchain and he was saying you know you can pick the tool you can you can pick how it is that you want if if you're not able to put down in English what what are the invariants what are the things that you're trying to accomplish here you're you know once you do that you're 75 or 80 of the way through it also it just speaks to a number of things not just for Security review but also just for your own understanding yeah to be able to declare this is what this thing does you can't put a value on that yeah God uh Gonzalo overrated underrated contract upgrade ability so overrated okay um expanding upgradability is a bug not always we went through this so there's a there's a longer conversation about this that happened at defy security Summit I think that is available on YouTube so I could rent on and on about this there's also a bunch of articles on the on the internet about this but generally speaking it you are just injecting another um attack Factory in your code base okay all right uh Chandra uh how do you feel about sorry overrated or underrated reading the code very carefully truly underrated I think we should read code extreme we can't just write rules and you can't just write rules no you definitely need to understand the code before you start writing your rules formal verification is not a silver bullet it's definitely not a silver bullet but really but really like how you know like it's uh how do you know what kind of guarantees you're getting from formal verification uh I mean yeah I think this is a very good question right like you I this kind of goes back to what I was saying before you know it's important to think carefully about what you're approving right and your spec is really all you have and uh you know when you say that okay we have a formal guarantee you should be very careful when you make these kinds of claims and I think it's very dangerous to make these claims because you know um the spec could itself be wrong and even if the spec is reasonable I mean there could be a bug in the formal verification tool and like then that's you know that's also bad and uh you know just because there is a specific bug that you verify verify your code against does not mean that there's uh you know it's just bulletproof and you can just trust it with your life right so um yeah okay um many yeah Europe um have I asked everybody something okay all right so you're after many you'll get something don't worry um Maddie alternate virtual machine designs so not the evm overrated or underage override it okay I don't know do you wish to qualify that or yeah I don't know I've been spending a lot of time in on the evm so yeah getting familiar and you know it's got its quirks.com syndrome probably yeah probably yeah um look now we've we've we've definitely done a bunch of security assessments targeting different execution environments don't get me wrong um but obviously most developers are operating with solidity Viper at the evm level it's got its quirks but it it's kind of working I'd be excited to see the upcoming upgrades uh particularly eof AVM object files that might change a lot of things with regards to security guarantees or assumptions that we make with that with the evm but yeah I don't know I quite like it by now all right uh Jonathan um how do you feel about like insurance against smart contract vulnerabilities over 800 that's where to start I guess I'd I mean if I probably say um I'm going to say underrated because I I don't really think it's widely adopted and used at this point and I think it's a it's a uh it's based on where we're at as as a you know our risk tolerance is really high and our you know our skin in the game is really high and um and and our risk levels are not well understood so it's hard to get insurance yeah if not impossible for a lot of us so I think that'll change that's gonna change for sure and so that's why I say underrated I think it'll change and then yeah there'll be Hedges and insurance and I mean it is happening it's not like it isn't happening but I think it'll it will happen more um and hopefully you know some of the work that some of us do up here will help everybody understand the risk better so that insurance will be obtainable and things like that for those who want it yeah I do I do like I'll I will give one I haven't given many opinions I don't think but I think that having somebody try to price risk uh who's not an auditor Auditors don't want to price risk they like look for bugs right but and it's a totally different thing to be in uh what is an actuary but there should be actuaries and maybe they are the ones maybe they audit the Auditors they have an incentive to use and we're working what I'm what I'm hoping and and we're doing some work with with the actuarials and and with the insurance companies to make sure that insurance in this industry does not go the route of insurance in quote cyber security where I mean I thought it was quite hilarious about a week ago when Lloyds of London got hit with with a cyber and like they can't set Actuarial terms if they can't even secure their own systems yeah what we're really hoping for is that that people doing Security reviews can set those standards so that that the insurance companies are actually as passionate about what they do that they're getting into it not just because oh look at that it's a new adjacent Market that we can get into and much more around wow this is really Innovative stuff we should get in there to make it safer and we're working with several people like that and I'm very hopeful so I'm with you about underrated interesting okay uh is the audience ready does anybody have a mic and a question yeah I do all right let's just uh Mike if you I can hear you so I'll repeat it oh somebody's somebody's set up okay go ahead please hey uh so I was wondering because of all the monitoring tools that are coming out like photo for example and there is also private mempools is it really overrated oh so you keep I like you're using the format uh I don't know who should I give that to you later what's the question uh is is is about monitoring um is monitoring like mempool monitoring overrated or underrated specifically mempool monitoring I think or just is that is your question monitoring in general is monitoring over yeah yeah so uh like for for example like uh there are products out there that claim that they could like maybe detect hacks and all that uh or maybe secure your smart contracts that kind of a thing but a hacker could use like a private mempool to like really send these transactions across and to even to react right you your transaction maybe I'm gonna say let's talk after not sure but what I'll say is that um I don't think there's anybody claiming that we've solved monitoring in the ecosystem nobody's claiming that or that we can stop threats but I I what I will say is that I think we're all I think there's a lot of agreement that security is not just a time box Security review if you if that's all you do good luck and and there are things you should be doing Upstream and considering like formal verification and fuzzing and various techniques time box Security review part is part of it and there's things you should do so pre-release pre-deployment post deployment there's things you should do in monitoring is part of it and so there's value to an end-to-end security and I think that's where we're at okay we're starting to appreciate that and I want to seek in quickly while we move the mic over uh go ahead Alex you're up next Mike don't worry hey uh do your clients request public reports and when you write reports who do you write the report for um yes so I was I was telling one of my team members the other day that uh what we sell effectively is PDFs obviously joking a bit but yeah uh um yes it's critical the output of the work that we do is a security assessment report which you know contains your standard executive summary detailed description of all the vulnerabilities identified and recommendations who do we write that for that's an excellent question we write that for our clients now the problem that we may see quite often in our space is uh you know readers um sort of going through well users of the protocols that we review going through our reports and not necessarily understanding again the concept of a timebox security assessment in a lot of cases the scope that we actually um go through is a limited part of the entire protocol these happens quite often particularly with very very complex systems um and unfortunately some users may use our security assessment reports and say look this protocol is absolutely safe because this auditing firm has only found a couple of you know mediums and lows and informationals and they've all been resolved it right exactly that's right they passed the audit big check mark none of us here provide you know any big green check mark in our auditing reports there's plenty of other firms that do that unfortunately um so yeah I think I think you're absolutely right the audience of these these security assessment report is um is a tricky one um let me sneak another question if a client asks you to remove something so or rephrase something are you writing it for a client yeah this happens quite often I'm sure I'm sure that like it happens quite often yeah I'll take it uh no no no no I mean I mean a client can argue on the severity of a bug right and that's absolutely fine you know we're more than happy to have that conversation and be like okay why do you think this is not a critical like look I've got systems of chain in place to prevent this from happening that I would actually drop down the likelihood of exploitation I'm like okay fair I'm happy to drop down the likelihood which would effectively impact the severity of the bug but removing a bug from a report if it's valid hell no now this goes against the trusted advisor role which we play and and how to read our reports or I think all of our reports is that it isn't that check mark and what I want to see from a customer is I want to see them come out with our original report which we will not change and so you as as the public can look at the report and see the problems that they faced and then you can look at the fix review report which is separate or an appendix it's just it's it's different and see okay you can actually watch the progression as they're taking the advice they're understanding not just how to fix that bug but how to fix bugs like that from happening at all and and some companies are afraid to do that because people will think that they're not good at security I look at it as the exact opposite a company that is proud to say look these are the challenges we face and here are the innovative solutions that we came up with to address them that's where I want to look because now they're they're actually taking our advice and they're and they're doing what they should do with an office is there I want to like is Chandra is there are a difference in the way that like a report needs to be presented when it's like heavy on formal verification I mean I think we similar reports and like Maddie said right like we have the severity of the bugs and we explain exactly what property we wrote and why that corresponds to the bug and I think the only additional thing that we very much emphasize is that you know uh like this like be careful that don't just take this as like the you know uh this is not a silver bullet and like the yeah I think there's no difference to be honest in terms of the reports we write um okay Mike thank you um so uh disclosure I work with Jonathan and open Zeppelin but I didn't tell about the question ahead of time so there's no front running here um if you could wave your hand and every single developer in the space but most especially your clients just automatically follow the best practice without you having to tell them ever again what would it be yeah yeah it was a bit echoey but uh if you could magically like if you could be sure that every client if there's one best practice that you wish every client would would adhere to what would it be um who's feel yeah all right I've got one do not change commits meet the reviews please thank you should I should I start formal verification at the very beginning and continuously throughout the project or only at the very end oh that's such a good question uh so the question is do you want should we start formal verification towards the end or do towards the beginning of your development so this is a very interesting question you should start as early as you can that's uh based on you know our experience I think that's the right place to start thinking about uh formal verification I think the most important thing there is you know you uh it really helps you sort of write your code in the most simple modular way because that's really what is best suited for formal verification and you know it just it automatically just prevents you from making certain mistakes and I think it's extremely important even if you are you know no matter what kind of verification technique you're using the earlier you start the better and along the same lines you know writing your specs out in English and also like you know gradually making them somewhat more formal I think the earlier you do it the better like once you've already implemented your code try to come up with an invariant or like trying to write down the pre and post conditions and stuff it's it gets really hard so I think there's like I like the shift left security meme which is like it basically just the idea if you think of your time and your process is moving from left to right writing writing your code then writing your specs and tests is doing it on the right hand side so the more that you can do any of these things your specs formal verification test writing earlier in the process generally the better your outcome will be absolutely yeah yeah I think yeah so I have a question for you I think Jonathan and Nick were touching upon it but crypto is a very narrative driven industry in general and if we zoom out a little bit and get outside of the security Eco Chambers what are some narratives that you want to see and what are some narratives you want to correct that are floating out right now in terms of security did you catch uh I'll try to so you're saying you said it was this is a very we're in an early stage right now we're in an echo chamber of security conversation and we need like I'm struggling yeah I was going to say if we make an assumption that security Community within crypto uh is is on the on the table and there is a broader developer Community outside as well uh who are all developing building the tools who are uh susceptible to certain narratives what are some narratives that are floating around around secured around how people see security audits around how people see Security in general that you want to take the chance to like correct if you see anything wrong what does some ways all right so I'll just repeat it for for everyone from what I heard and correct me if I mishad um bad narratives in our industry that should be corrected hopefully quickly and early on um is anybody got done what are you angry about see I know I know how to like um I'm angry about a a lot of the raises that happen in the security industry recently I think think the the the the money and capitalism is creating legitimacy for for people that have not demonstrated proficiency so this is a narrative that really grinds my gears for example um wow and uh yeah yeah I'll give someone else I'll sneak in a smart contract over availability here all right since Gonzalo started with things that make us angry I'll do one too that um well I think that the our desire for cost efficiency and time efficiency is not in our best interests and I mean gas savings and speed of transfer from a layer two to a layer one through the bridge that was just developed yesterday like and our lack of acceptance on latency on anything so that we could do more security verification so I think these things work against us and I understand why we're doing them and how they're helping initially but I hope the narrative changes so that we um aren't as worried about gas savings and we're more and we aren't as worried about speed and we're a little more worried about security and safety for the users hello everyone my question is is uh if auditing is good enough or would you consider like uh Alternatives like open arena or Bounty bolts like hats Finance for example or maybe deploying two medium tier uh Auditors or just one top tier auditor per contract considering timeline so I'm not sure I fully understood the question but I think it's please correct me if I'm wrong but I think you asked if auditing is good enough then why do we need uh the bounties let's make it more General I like we sort of I I think I think I'll get we will get this is like like just generally we have some new alternatives to audits that have emerged giant bounties uh and and like code Arenas and Sherlock doing these uh like auditing competitions that are a little bit like a Time boxed uh Bounty program so yeah um how does that uh how does that like substitute or compliment auditing I don't know this I don't know if anyone's like we might yeah yeah but my answer is is back to what I said a little bit before which is a bounty can give you that this bug which is interesting but it's not really as interesting as as how you are working with this is a this is a holistic view of of all of The Code by by Engineers who are deeply and intimately involved with the code to the same extent that that your internal people are but they're seeing it from through a different lens they're seeing it through the lens of people who have literally watched everything that could go wrong go wrong and and being able to see mistakes in everything from documentation to to just your overall maturity it's a different set of things if you want if you want to find specific bugs and fix those that's great there's other things that are that are outside the the general scope of what we do in a Security review uh I'm not suggesting that that audit or Security review is the the be-all and end-all but it's also uh different from a lot of things that I'm seeing out there and as far as automation goes we use automation to find sort of low-hanging fruit or to point us in the right direction so that humans can look at it and make educated decisions about what we think and there's a lot more than just run this tool see what you get and hand over the report anyone else quickly we're running low on time um sorry sorry sorry no no that's I meant on the stage like if anybody wanted to all right I'm gonna be selfish too there's something in the time we have I really want to plant this idea as much as possible is that it's not the auditor's job they're not selling you Security in my opinion they're selling you like insight into your ability to write secure code so you need to take responsibility for writing secure code uh and not act as if like you just get to write code that does cool stuff and then give it to the auditor and they will make it safe for you so you need to focus on being able to write secure code whatever process you think that might need whatever tool you think that requires and then you give the other your code so that you can understand how close you are to being able to write secure code amen it took me a long time to get there so it's a bit of an epiphany and I'm trying to unfortunately we see what John was describing way too often and yeah it'd be great if we can all collectively shift um towards a different um yeah definition of what our work here means um yeah thank you thank you okay there was another question so let's have it I think so um Foundry when Foundry first came out uh I was surprised by the fact that it included fuzzing so my question is when does fuzzing make sense on these code bases which are generally open source and small and how do you approach fuzzing in your use for auditing fuzzing makes a lot of sense all the time it's we're going to like the shift left the shift security left narrative and all that like you should use all the tools literally uh you you should write specs if you have specs fuzzing will work better right um so I I don't yeah you should just do it all the time use Foundry and use all the other tools use a kidney use Harvey by the way I forgot to say I've been announcing this the entire week but this is the panel we at diligence will be open sourcing all our tools just like trailer bits we are a bit behind schedule but at least very late than never so even Harvey yeah yeah yeah all the code bases so yeah foreign did you uh I just well yeah I guess I to add to what Gonzalo said I think uh using tools is actually good so fuzzing makes sense to me I mean you know again it's not a guarantee but you know I think uh it's just like good software engineering right like you write code and you want to write tests and you want to do fuzzing and you want to write specs and these are all good software engineering practices and I think they also just are they carry to this domain as well so it makes sense um the one the one comment about that you just made well both of you like uh that the Declaration of things the ability to set forth the spec uh and then and then fuzz around that that's really important that first part is what Jocelyn and and Gustavo are going to be doing tomorrow there's going to be a two-hour workshop on how do you declare these things how do you actually set out and and describe in variants and how how can you use those to make your fuzzing more effective so I would really recommend that that you go to that so just to be fair I know like we're at a time by this timer let's give every speaker like 30 seconds to sorry all right so we started the questions way early we've got tons of time um I should have read the manual um did uh Brian I know Brian had his hand up at one point um and I'll get you please just get just like I'll I'll repeat it through the mic so go ahead that's that's cool is there a way to make like the the output of of audits machine readable um and it's interesting because we we do actually that's really interesting and I think we're gonna take that up and talk to us hit us up afterwards um I I think it's gonna really depend on the writing style of of the security firm that you engage with um but it is something I'd like to think about a bit more yeah um perhaps we could parse some of those reports into a readable format or a machine readable format that shouldn't be too hard to do we write a report in reports in latex so it should be pretty trivial for us to do cool so uh so try actually we do have we have been working on a related things so um I don't know if you've ever heard like there's things like deoxygen or the oxygen I don't know how to pronounce it and like there's also uh Nat spec is another similar Library so uh the nice thing about you know if you're if you have a formal spec it's actually a little bit easier to like sort of convert that to a format because you can annotate your code and then you can automatically generate some kind of a formatted report from from that so I think it's a good idea uh Hey guys Kevin over here uh Hey friends um so having come from sort of the security side and now being protocol engineer and founder um I'm wondering about this this line you tread as an author of being very slow methodical you know heavy testing core protocol must be super secure to now okay now I'm wanting to move a bit faster maybe their integration contracts or something that's not in my core protocol um you know how much time am I spending and testing and auditing how much money am I spending there versus move fast and see if things work I wonder if there's I have my own opinion but interested to hear what's on the stage I'll just quickly I'll say um the most secure code is the code that never hits mainnet so just keep testing I also think it depends on how much money you raised or how much money you have and now it's like it's very it's a very practical thing you know like you can spend a lot of time and resources doing doing writing tests but at the end of the day if you run out of money and you only have the test you're probably okay cool it might come off as somewhat pedantic but I'm curious looking at the different way that auditing firms do severity or impact and then difficulty slash likelihood I'll be honest I like when I prefer likelihood so that it's like uh difficulty is low when likelihood is high so I was just wondering you know High likelihood and high impact would equal High severity have you seen clients get confused about this different and without being overly like strict as the industry is still developing I'm also curious about like how we might classify things trail of bits has a nice classification swc seems like outdated and not updated as well as or maybe it could be going forward are there things that we can do as an industry to improve and start to I hate to say solidify but structure our information architecture a little bit more so that clients can understand when they're getting multiple audit reports and reduce confusion I think I think it's uh yeah it's a great Point as you said uh well for us for us at CP a high likelihood high impact critical um and I think we all use perhaps similar types of risk rating matrices um it it's probably beneficial if you could standardize on on how we would write severity so as you said it's typically likelihood of exploitation impact of exploitation but there's different levels so we only have we got four of them at CP so informational low medium high and critical that's actually five um but yeah I think I think it's a great idea there's been a lot of initiatives over the past few years on The Ether security channel to try to standardize some of this stuff it's never happened I don't I don't really know why um but I think it would be it would be quite beneficial I'm going to give you a bit of a left field question so there's been a few regulatory proposals that includes uh software security best practices in The Proposal uh what sorts of standards that are driven by regulations could be do you think could be damaging to the space and what ones do you think could be good like so so almost like uh yeah the typical like like a regular comes out of nowhere and they're maybe a bit naive what kind of damage could they perhaps do by uh misunderstanding well I think I I don't know if it goes to like the intent of your question having to do with structure of code I think we all know we're concerned about regulations that would impose requirements on protocols to be able to block certain parties as an example so I think the biggest a lot of our concerns are around regulations that would have a requirement so I don't know if that goes to your thought about the question having to do with specific processes your contracts must be upgradable for whatever reason those things are always damaging if you're talking about regulations at the code level I think there's no way they can be good you know like I I think that um as an example having worked in uh security before and in the in the cloud space the sock 2 standard and actually I believe it has a lot of really good aspects to the software development the secure security focused processes of your software development life cycle and I think those are good practices and if teams were transparent about the processes they follow I think that's healthy and if held to a certain standard all those things always have downsides in terms of the effort that goes into auditing them and proving them but yeah the life cycle of Regulation is really tough um getting getting regulators and government officials to understand the key areas they have a goal of what they want to accomplish but usually they're they're rather far behind the technology I think it's up to us everybody here we certainly do this I bet everybody else up here does this we spend a lot of time talking to Regulators about what is possible and what isn't we spent a lot of time talking about the intent and trying to come up with sort of more more open suggestions to Industry as opposed to specific because if if you say thou shalt do the following it's always going to be out of date by the time it passes and and it just becomes very very difficult so I think that the best thing that we can do the most practical thing we can do is educate those those people who would regulate us thanks for clarifying Brian uh yeah that makes a lot more sense um just to give you guys maybe an exotic perspective I was catching up with some of the Regulators in Australia a few weeks ago and this is exactly what they were asking for they said what should we be you know requesting from projects to do in terms of security practices in terms of security development secure development and you know what should we be regulating in the space so I try to sort of convince them to go down the educational part and um and yeah this is this is spot on I think it's certainly something that is happening at least in Australia these people are starting these conversations and are starting to realize the need of you you can't enforce these things on projects but at least you can have these set of guidelines and point your users your you know citizens to these guidelines and get them to demand these standards these set of minimum requirements from the projects they interact with thank you [Applause]