Challenges Developing and Maintaining Open Source Software in Web3

[Music] [Music] hi uh I'm ever hen brand as was introduced and I'm the CEO of runtime verification and I'm here to talk to you about the challenges of developing open source software and maintaining it sustainably not necessarily in web 3 it doesn't really matter web 3 web 2 I mean there's some different challenges in web 3 but a lot of them end up being the same um so uh why are we qualified to talk about this at all uh we have you know a huge GitHub presence 170 public repositories depending on how you count 20 12 to 37 of them are active at any given time it's spanning 15 different programming languages lots of external collaborators obviously the internal developers at the team as well um you know we have dependency chains within that software that runs six deep uh in some cases and we have different teams that range in size from 1 to 10 we've grown teams we've shrank teams we've you know done all sorts of different projects over the course of the last six years as I've been working at runtime verification and we have super active and super awesome uh automation with CI CD so what are the challenges I'm going to start from the easiest challenge to solve which is the technical one and a lot of people people probably think oh this is the only challenge the technical one and you know with the the technical challenge with managing open- Source software is automation you need to automate everything uh so you want to enforce as much as possible we use GitHub for that you can use gitlab or whatever other version control um and publishing software you want to use we use CI for testing so every you know release is tested that just keeps our you know Keeps Us sane basically automated releases and updates and so we actually have it you know that when we have those six dependency chains going on we six deep dependency chains if there's an update to one of our pieces of software it automatically pushes down the chain to the next piece of software for us um and one of the kind of challenges that you might notice is you know that software goes through different life cycles right there's when you first start developing a piece of software you're in Rapid development mode you're prototyping you're trying to move as quickly as possible you don't want to be slowed down um and that looks a lot different than when you're in kind of the the later stages of a project where you're just evolving the software over time and you have a lot more tests there to guide you so you you need to accept that there are different life cycles to software development and for example keep GitHub enforcement turned off at the beginning of the development and then you know a couple months in or a couple weeks in turn on that GitHub enforcement the Second Challenge is personnel and this maybe it even deserves to be the third challenge um because this is you know the medium difficulty challenge but what I like to tell new hires at runtime verification is that learning to program is easy learning to program in a group is hard right and the key thing to remember is that you have to be respectful of other people's time right and uh you know one thing to keep in mind is it's always harder to read code than to write code and so you're submitting code changes or something like that and it was easy for you to read it because you have the intent in your mind and you translated it into code it's much harder to go from that code and translate it back into the intent as the pr reviewer so as a person who's authoring code you need to make sure that your code is you know small changes isolated changes easy to read well tested so that the pr reviewer has an easy time pressing the approv button um you know remote remote work obviously makes this a lot harder you know when you're all in the same office together you just turn around hey like what do you think of this change and you work on the same screen but now we're talking about different time zones and there's delays associated with that so you have to keep all that in mind and actually keep it in your mental model for your development um and so here process is key example at runtime verification I let everyone know you know beginning of your day you clear your slack messages first then you do your poll request reviews and then you do bulk code development on new code and then you open new poll requests at the end of the day you do your poll request reviews first because that's blocking someone right that is causing a synchronization point between you and someone else they might be on the other side of the world and suddenly if you don't do that PO request review you're delaying them by a day and then that maybe delays You by day and so suddenly you know some small change gets delayed two days maybe 3 days if we're talking about an 8we project and you have a handful of delays like that you've lost two weeks on that 8we project right it's not good um anyway uh next Slide the third challenge is money and you know if you know how to solve this please let me know uh the solution I found is to beg and the you know the the problem with begging is that everyone starts begging and you need to have like a begging process and have a little structure to it and so you have like grants and retr funding and side quests where you get people to fund one thing but then you develop another thing as well um and the problem with structured begging is that it becomes a game and big players are better at games than small players and so then the small players Resort back to unstructured begging and so I don't know we're back at square one so anyway that's my talk uh you know solve the technical challenges first prioritize those that's the easiest part make sure as you on board people that you make them aware of the Personnel challenges and then come let me know when you figure out the money thing thanks thank you any questions oh there's one here great presentation I just want to know more about runtime verification I just check in the website um but it's much better for you to like introduce what you do and why are you working on open source yeah so we do we do mostly formal verification and formal verification tooling so we have this project called the K framework that's kind of our biggest project that is a tool that enables us to do formal verification for any programming language basically um and so we have formal verification tools for solidity for web assembly for rust we also have for some other blockchain like cardano and tezos and stuff like that so that require that's a big formal verification is hard software to develop and it's used in a bunch of other contexts as well and so we just end up having a lot of different repositories targeting different ecosystems and different programming languages that we do that far and most of that is done for security audits basically so thank you MH there's a question over there thanks hi um because you already mentioned uh the workflow especially at the GitHub pool requests so I had a question about that um do you have any advice in term terms of leading the team and structuring the pr reviews uh because in my opinion when there is an update or is a bug fix it's quite straightforward it doesn't really matter the number of commits because you look at the change you look at the description you know kind of what's happening but perhaps uh you have an advice on how to structure the commits when it comes to a new change and when you said about like the me like it's hard to construct the intent by looking at the changes so perhaps having more commit and uh following them one by one would help with that or you know what I'm asking perhaps have up there uh I mean it's challenging and it's challenging not because it's a non-s solved problem it's challenging because people just don't do it right but uh I mean have good commit hygiene try to make it so I try to make it so every commit builds if not passing tests it might be that you're going through refactoring and tests have to be failing in the middle there but every commit should build in my opinion and so that should make it so every commit you can review in isolation if that's necessary which is not the ideal way to do it because that takes a little longer to review but then the second piece of advice I'd give on that is as a poll request reviewer just be really picky you know if someone sends you a poll request that is changing four different things and it's hard to review the whole thing and keep it all in your head don't bother reviewing it just send it back to them and said hey you weren't respectful of my time break this into four poll requests and I'll review each one independently because ideally as a poll request reviewer it's easy for you to look at a poll request and say this is a good change I'm gonna hit approve and move on right but if if the if you can't do that and you have to now dissect what is this change doing you're not the expert on it you didn't write that code it's much easier for the person who authored the PO request to go and split that up into multiple different PO requests than it is for you to dissect what they meant right so don't you as a poll request reviewer don't be shy just be like look be more respectful of my time please go break this up into multiple PO requests so I can review each independently that's my opinion on it all right thank you very much thanks thanks for the tips for PR reviews I think we can all Ben