Better Solidity Support in VSCode by Hardhat by John Kane | Devcon Bogotá

foreign [Music] T and we're going to be covering this is going to be like a jaunt quite an easy joint through the different virtual Studio code extensions for solidity there will be showing of the extension I work on and then more interestingly we'll move on to a couple of the technical blockers that are holding us back from giving you this solidity experience you all deserve and then you know a plan for getting around those blockers so my name is John Kane I'm from the nomik foundation you probably knows as the heart attack people and normik is a not-for-profit foundation pay for by donations with a remit to improve the ethereum developer experience hard Hat's a big part of that but the rematch General and we've identified a gap in editor tooling and that's why we have built and released a vs code extension called solidity by Dynamic Foundation of which I'm the the team lead so you can find me on GitHub or hang around our Discord server particularly the heart VSCO channel so if you want to come and chat or suggest some features or you've got a really detailed book report and a matching PR come chat come chat editor Tilly let's kick off by addressing the Microsoft branded elephant in the room right why are we focused on vs codes and the reason is we have good reasons to believe that vs code is the main way that circle is getting developed today so if we take a look at the most recent uh solidity developer survey you can see the vs code is dominant right in fact the 60 Visual Studio people we suspect actually meant Visual Studio code rather than the venerable.net IDE similarly the remix and that's an online solidity editor a bit booze in key components from vs codes right so you're actually down at then before you're not talking about vs code or vs code adjacent right and we suspect that the solidity developer survey skews towards those who are deep in the ecosystem right those who are on crypto Twitter and and click on the survey link when it goes out but another way it's over representing this long tail it's over representing the shadowy supercoders who are doing their development and then and a teamwork session over a VPN to a military satellite that they repurpose for the day actually let's just test right hands up anyone who when they do do solid development they do it and vs code and hands up if you're a shadowy supercoder who's doing it within alright see that's a track no shadowy supercoder with admit right and actually hands up if anyone any of the two emacs users are in the room because I'll buy them a drink I'm having my doubts I've created the wrong incentives here right and I'm not immediately going to renege on that promise right well be a lesson on uh soft promises and the value of escrow right so by Pareto analysis vs go to Japan vs code I might just write the development I'm looking at the marketplace what should I do and I'd say that there are three options if you have a different setup get in touch I want to to hear it but those would go down as solidity by Juan Blanco solidity by theronic Foundation and trouble for vs code right and we're going to say go through all three of them quickly and if we were doing it by installs we would start with solidity by Juan Blanco but as I work on this one and it's my talk we'll start with mate um as I'm a coward I'm not going to do live coding so here is uh earlier John to run you through the feature set so if we jump into our solid City contract as you can see we have table Stakes so syntax highlighting we also have Diagnostics or otherwise known as inline warnings and errors you can hover over variables or functions to get useful information note that currently we're not showing documentation though I will come back to that we have code completion on local variables or more sophisticated examples where table permission is required completions work inside of import statements as well both at the file system level but also direct Imports so in hard Hat's case I was looking inside of npm modules in terms of navigation we can find usages similarly we can jump to definition and this is working across the different packages of monore refactoring includes rename which obviously you shouldn't use to do any key role though we have implemented undo we have a set of quick fixes which appear in response to Salty warnings and errors we find them under the bulb icon so let's bring back public possibility and simply making sure that we set the overrated specifier these are small annoyances with function signatures but we have more complicated quick fixes for instance we have a contract that needs to implement an interface or multiple interfaces we can leverage the admin functions from interfaces quick fix and this will do some of the heavy lifting forms it is scanned through the inheritance hierarchy of interfaces and these contracts and figured out what the smallest set of functions that needs to for Saucy to to stop shouting at you so once I had four knew if I push play this is going to play again ah good if I could buy it I want to add three caveats to that video gave it number one uh like any suitably advanced technology is indistinguishable from a direct Tech demo right and what you just saw was a reg take demo you didn't see all of the places where jump to definition doesn't work or you you're aiming for completions we don't resolve bolts out of completions caveat number two I just re-emphasize that the documentation against function wasn't appearing on hovers and there's a technical box to that I want to come back to right and fee is the biggest caveat which is there are some key features in there actually like um Diagnostics and quick fixes that only work inside of a hardtop project right and that's a problem that I want to come back to again but let's take a look at the other extensions quickly solidity why Juan Blanco this is the OG it's been the Workhorse of statistic development for years um with uh over 800 000 downloads relative to ours uh let's just see we know how event feels it provides syntax highlighting integrated formatting and it has navigation and Diagnostics and the default structure assumes you're organizing your contracts under source and lib so setting it up for heart attack is is difficult um but it's going to work better if you're doing boundary or adapt tools or or brownie um and next we have truffle for the escort this leverages solidity by one Blanco for the syntax highlighting Diagnostics Etc that adds a layer of UI Integrations into the Truffle two Suite right so that you can run compile through um the the UI through the task subsystem in vs code for example right um and that's great for onboarding if you're new to truffle or you prefer the UI to the CLI right one other really cool feature is integration with the Truffle debugger just wanted to mention as well this isn't from General development but the diligence team at consensus has a set or Suite of extensions that are security Centric so they're going to give you analysis and visualizations when you're doing that Smart Auto plenty of options for solidity then lots of great programmers pushing the ecosystem forwards but I don't think any of us would claim that we're yet in the same week of editor support as other language ecosystems like dotnet or rust or typescript right so how do we move from where we are now to that Best in Class experience and one of the answers is just time and resources right but there are a few technical blockers I'm going to spend the rest of that stock taking into those first off feature fragmentation so we have features which are siloed by editor and also cycled by development framework the others feature productiveness right can you rely on a feature to always be there and to do the right thing and for fragmentation clearly extensions we have built-in vs codes um a maybe the lack of support and then is why there are as many people using them right if they had those Advanced features in film they would be using them huh so that's one form of fragmentation the other form of fragmentation though is in developer framework not all of the features that were listed there work for all development Frameworks right if you're working on a truffle code base you're probably using truffle for vs code and if you're in a heart attack code base hopefully you'll use the normic extension and if you're in Foundry you're likely using one blank was extension right and that makes sense for some development specific features but there are core editor features we should just work across all of them right you're going to want rename and Foundry just as much as you're going to want to in heart app but currently it's siled in hard times right how are we going to get around this this sorry there are multiple reasons for that and one of the reasons is is just Conway's law so if you've got four teams working on compiler and then you're going to end up with a four-phase compiler the code structure matches the communication structure of the teams right and the development teams that are uh rating editor support it's easier for them to add it for their own development framework it's easier for me to harangue the other hard hat devs or cajole them or convince them or go crying to the CTO to try and get it changed than it is for me to convince the entire ecosystem to just do what I see right but there's on another aspect that's part of solidity right which is that solidity doesn't specify how import loading works or sorry or doesn't specify that part it leaves it up to the development framework right so if you so if we've got an import statement here pulling in other solidity codes so C requires a custom loader to actually resolve that and it has to be provided by the development framework and if it's a relative path the development frame was probably going to do that as relative to the on the file system right from the current file but if it's a direct then for then there's multiple different interpretations which make sense right and heart hats we would interpret this as a look inside of node modules and then find open sample and resolve the rest of the path read the irc20 soul fail and that makes sense right heart out leverages GS including node modules but that doesn't make sense for Foundry right it's going to resolve this based on remappings in either the foundry.tomo or remapping txt right and that has profound implications for editor telling right so say you're on irc20 that's token here and you wanted to jump to definition the editor can only resolve that if it understands this our open ziplin import line right that implies the editor has to understand the import loading logic of the development framework that this file has been dealt with under okay so how are we going to work around these fragmentations and editors and different editors and fragmentation across development framework I'm going to in two steps first we're going to use the language server which as language efforts of technology that came out of yes code but tries to deal with the fragmentation you get across editors right so instead of each editor implementing language features for each language we gather up the language features under a Daemon and we have the editor put that demon up ask which features it provides turn those features on in its UI and whenever the whenever it was a request by the user it delegates them off to the Daemon and the communication between the editor and the demon is covered by a clearly specified protocol language server protocol and that's what we're doing in the nomic extension today right there is a a language server which is encoding all of the completions diagnostic servers um that we that we provide today and there's that is embedded in vs code but there's nothing stopping us from exposing that out so that other editors save in could take advantage of it right but what about the uh important or more difficult form of fragmentation right between developer Frameworks if you have a language server an economic extension already why is it that I don't have advanced completions in Foundry right and the reason because that is because currently the language server is really a hard language server rather than a solidity language server we are all through the code base encoding heart assumptions to deal with things like the import loading that we saw earlier right and the reason that we've done that is we use bits of the heart attack code base right which allowed us to get up and running quickly we can make hard hat dance right and subtraction is hard right but we are looking now to refactor the language server to isolate out these heart particular points the development framework particular Parts behind an adapter interface will have a heart attack adapter but we also can then have the implementations for Foundry truffle brownie right and in that way we can add feature once and have it available across the ecosystem one of our point I just wanted to mention recent versions of Southsea have a language server built-in right so why don't we leverage that and we're big fans of Library language servers to the northern area but the so C language server is embedded in a particular version of Soft C right and heart attack code bases can have versions that are older than the ones that provides the language server and suddenly we have many complex code bases and heart attack code bases which have multiple Soul C versions right and we need the language features to just work across those versions so so that's why right but just to summarize the normic extension has an in language server and we are making development framework agnostic and then we're going to make Standalone so it can be used in other editors and that's how we can build a feature once and leverage it across the ecosystem feature platforms right this is about uh quality the edits are just doing the right thing and so when you um to jump to definition you can do jump to an air finish in all the places you expect and actually goes to the right place anatomic extension has blind spots right there are lots of places where features don't quite work as we would like and there are several areas that we need to work on this and but I just wanted to focus down on one in particular today right and that is that we are not giving the best experience when people are annoyingly trying to use the editor to make edits walkthroughs you're popping about inside of the Department inside of the editor you're doing navigations you're exploring the code base yeah jump back again right and then you start making an edit and you remove a semicolon right shame on you shame on you but um suddenly drop the definition doesn't work right and why is that so the video just showed you a change in the document and that change dot gets passed to the language server right and the language server does a parse um and it builds and Abstract syntax stream right and that's a data structure which represents the syntactic elements in the in the code the code might represent our function and then sub nodes would represent substructures like the function name the parameter list the um the function body I know as well that the parse produces an abstract syntax tree right so some trevia is thrown away stuff that isn't strictly necessary like white space like comments right because they are not typically used in later phases by for instance the parser right and that's why um when we do hovers we don't show the documentation the abstract syntax tree is abstract because it's thrown those elements away and because the ASD doesn't have it we don't have it so we can't provide it and however we take the the language server takes the AST and it combines that EST with the asts of all the other code files right and then we do a further analysis stage we overlay it with type information we also scan through the nodes and we find definition nodes on your signals right so our definition node might be a function declaration and a usage node might be a function notification right and we layer on top of the AST that extra information these extra links so that you can see which nodes is connected to which other node and so I change so that when the user requests I'll jump to definition the language server receives that request including the cursor position it looks up the nodes underneath that particular underneath the cursor and finds a function notification right and then looks up that pre-calculated link to find the definition nodes and we return that location right in terms of its file uh line column actually both to start and the end and vs code then jumps to that location next the user makes an edit and introduces a syntax together the change dot comes through we do a part and it fields right and why did the powers feel well we like other projects in the ecosystem leverage uh solidity parser right for parsing but solidity parser is it's a GIS library and it's designed to give binary answers to the question of is this syntactically valid if it's syntactically valid we produce an AST otherwise here's list of errors and that makes sense right most use cases of a parts are stop if there's a syntax error you don't proceed onto compilation or generation steps after as a syntax error right editors have have other needs right and the editors case we want both the list of errors but also the syntax tree right or as much of syntax tree as possible at the same time right and that's not what a standard parts are is designed for the way that we get around this in other languages is by building specialist parsers that are more tolerant right and they're designed specifically for that editor use case and that's what we are building now at nomic through our slang project the the slang parts are one of the components of that project looks to leverage best practices from other languages right from from Rosalind from rust right things like red green trees and the goal is to always produce and put always to produce a syntax tree right even in the presence of errors it's just that sometimes the syntax tree will be valid and other times it will not but hopefully slang's error recovery will mean that a missing semicolon and one function doesn't mean we don't get the syntactic structures in all of the other functions so that we can take advantage of those and extra features right one of the other advantages of building uh parser specifically for the editor is that you can include that trivia that's normally missed out the white space right the comments so that we can show the comments and hovers for instance right or we can do Advanced formatting or sorry uh refactorings right right now we're blocked in refactorings because developers probably won't be too happy on doing a refactoring we stripped out all the comments in the file right that's probably unacceptable but with our partial design for the purpose we have that information and we can make sure it's there okay so what we're doing here is swapping out an excellent but general purpose component with our components specifically designed for the editor and that's how we drive up feature quality to summarize if you're using hardware and you're using vs code we think you should give a strategy by nomic uh go and we think it represents an excellent set of features and you've just heard how we're driving up the quality of those features by building um editor-specific components and swapping out the general components that we have and we are looking to make those features available across other development Frameworks right we want a credibly neutral language server and we're doing the refactoring and putting in the work to or putting in the workforce attraction to support that we also want it to be a standalone language server so we want to pull it out for being embedded in vs codes it's going to need to do a a small amount of cleanup this is actually one of the easier things we need to do and but hopefully sometime soon uh Shadow is super coder will be able to hack away in solidity to their hearts content okay so that's how we're going to improve the solidity developer experience for everyone thank you very much are there any questions for John before you oh sure are there any analysis tools that can benefit from these improved more robust a abstract syntax history sorry yes sorry so the questions are there any other tools that can take advantage of this and we would hope definitely yes the slant parts are we're building a component that could be used in devtooling across the ecosystem right and it's going to give more detailed information and work across uh different uh solidity versions right so it should be giving you rich information cross versions other parts of the solidity project also intend to add other components to write even more information so usage in definition would be the next one that we're we're looking to tackle so yes hopefully lots of analysis tools will be able to leverage these components hey thank you very much this is awesome um so two questions real quick uh it wasn't clear to me at least if the slang parser is being used right now by the extension and the second question is uh is there any um intention from you guys to support.nbeam which is like a yep yep so right now we do not have the slang parser we're using the solidity parser and As and we're looking to dog fit the slang part or within the vs code extension um I I'm not giving her estimates maybe like it's not up to me but um yes we're hoping to put on the slang bars are in the near future in terms of random COC and that's part of making the the language server Standalone and webcoc would be the first Target for making it work with that and that's something that is within just what team's remit so hopefully that should be in the near future when the sling parser comes out um how do we install like is do we need to install it or is it already built in how how can we get access to it when it will be available it's a separate library and I believe it's written in Rust um and we but we'll make it available uh in nodes and JavaScript as well [Applause]