Slang’s Query API: a better way to analyse Solidity code

Okay, good day. My name is Tony. I work at the NOMIC Foundation in Applied Research. I was previously a co-lead on the SLANG project and it's in this capacity that I'm going to speak to you today. I have a fantastic job working with excellent people at NOMIC.

NOMIC is a non-profit and our number one core value is kindness and we live this every day. And we are hiring. There's meant to be a QR code at the end of my slides but I'm afraid it didn't make it. And I want to give heartfelt thanks to the co-founders of Nomec, Pato and Fran for creating and maintaining such an amazing place to work. So today we're going to explore the slang query API which is a new feature which makes working with solidity code surprisingly straightforward.

Now this is not a tutorial so some of the details have been omitted. You can't copy and paste any of the code that I'm going to be providing. It's just so that you can, well I'm hoping to inspire you, so that you go and find out some more details. Okay, so 20 minutes is a bit of a sprint, so I'm going to speed up. Let's go.

Okay, so what is Slang? So Slang's our Solidity compiler library. It's basically a developer tool that enables you to write better tools. It parses and analyzes solidity code from version 0.4.

11 to the latest 0.8, which is basically all of the solidity code that is live. Our focus is on correctness and convenience. We're about to release version 1 which covers the typical front-end features of a compiler and our main objective as I said is to enable you to develop better tools. Slang has a radical open to clarity of meta architecture.

Slang's an error-correcting compiler, so it always produces output, even if you've got source code errors. A key feature of Slang, which is what we're here to talk about, is its query API for code analysis. So compiler front ends fall into two main categories. There are those that produce what's called concrete syntax trees, and then produce an abstract syntax tree from that. And there are those that produce abstract syntax trees directly.

So what's the difference between concrete syntax and abstract syntax? Well, concrete syntax, as you can see on this slide, is a complete representation of the source code. It includes every character, every bit of white space, every comment. It's like having, as I say, like having the full book, like a PDF. Whereas an abstract syntax tree is a simplified tree representation.

So it doesn't include white space and equal signs and punctuation. It's just got the essential details. It's like if you were reading the plot of a book but you didn't actually have the text. So here's an example of a concrete syntax tree. So we've got a simple variable declaration here and every single element of the original source is represented.

Nothing is removed or simplified. Now, this is different from an AST, which will eliminate some of these details. So if we look what we've got here, it's a simple variable declaration. Now, it consists of a type name, which is uint, and then some whitespace, an identifier, some whitespace punctuation, the equal sign, some whitespace, punctuation, equal sign, some whitespace, a number literal, one, and then the semicolon. Now, what's crucial to understand is that we're preserving every single character.

In fact, even if there's an error in the source code, you'll find it in the concrete syntax tree. So why is this important? Well, when we're building developer tools or transforming code, we often need the complete information so we can round trip. If we were just concerned with the meaning of the code, then we'd use an abstract syntax tree. But for many tools, like formatters, linters, or refactoring tools, we often want the full syntactic details.

You don't want to lose comments when you're refactoring and sometimes you want to know if you've left two blank lines and you want to preserve that. This level of detail is great but it's also challenging. It's quite difficult to process this. On the one hand we have complete information but working with detailed trees can be complex and that's the challenge that the query API solves. So from now on I'll admit trivia nodes and whitespace from the examples.

They're still there but I don't need to show them. So let's look at a slightly more complex example. This definition shows how different elements nest within each other. And once again, notice how everything is preserved here. We've got a function definition, we've got the keyword function, the identifier, we've got a parameter list, which in this case is empty, but we still preserve it.

And then nested in that is a block and a return statement and so on. Now, this level of detail allows us to maintain complete source fidelity. As I said, if we're going to round trip, we can track precise code locations. So if you want to do error reporting, it handles formatting preservation. And when you're building development tools this detailed structure is essential for certain class of development tools.

So to emphasize this point, I know I'm being repetitive here I really want to make this point though, why do we need a concrete syntax tree? Well we really want to in a lot of tools, for example in Pretia, which is now using slang by the way, using this technology, Pretia Solidity, we want to make sure that we can preserve every single character, especially comments. For example, you might have a tool that wants to deal with comments, wants to encode validation information in a special comment format. Not like the documentation comment, but your own special comment format, so you want to preserve those comments. An AST on the other hand is focusing on the essential structure which is good for semantic processing, doesn't contain the formatting information, simplifies expressions and emits all comments in white space.

So now that we understand what a CST is for, let's look at how you might traditionally process a CST. So this is the kind of code that you often see a lot of. know what it's like to traverse a tree especially a very deep and complex tree Which is what you typically get out of a compiler? And it's not pretty What we're trying to do is actually quite simple We just want to find all the variable declarations in some solidity code, but look at what we have to do So first of all we're writing a recursive function. We need recursion because we don't know how deep in the tree it is.

Could be at the contract, inside a function, inside a block, anywhere. And then when we find each node we've got to manually check its kind, we've got to search through the children, We've got to handle all the edge cases, and we've got to manage the recursion stack, especially if you're threading state. And this is a simple example. In real-life code, you'd have to handle error cases, deal with unexpected node types, which is what you can get if you've got a source code error, for example. Manage your state during traversal, and if you've written a visitor pattern, you know that it's a pain in the ass to manage your state.

You've got to handle parent references, and you've got to manage the position in the source code. Now, the problems with this approach are numerous. First, it's incredibly verbose. The amount of code you need to write to do even simple analysis is substantial, and the more code you write, the more you have to maintain, and the more places there are for bugs to hide. Second, it's error-prone, not only because of the volume of the code, but also it's quite tricky to write these when you're dealing with complicated trees.

For example, in Solidity, variable declarations can appear in for loop initialisers. I don't know if this code would handle that. It's a question for the reader. It's hard to maintain. When the language evolves, and as you know, solidity evolves significantly in all sorts of interesting ways, like in one of the 0.

5s where the exponentiation operator changed its associativity. So when the language evolves, you need to update your traversal code, and that's because the traversal code is mixed with the analysis code. So what you want to do for analysis is mixed in with the mechanics of traversal. To make changes, you're often going to break things. Finally, and more importantly, this forces you to think about what or how you want to do things rather than what you want to do.

And the intent of this code, finding variable declarations, is buried in here. You could look at a big bit of traversal code and you wouldn't know what it was meant to be doing. We need something better. So we'll have a look at some specific challenges with manual traversal. So recursion management.

You've got to handle deep nesting efficiently, avoiding stack overflow. You've got to make sure you don't have any infinite loops. You've got to maintain your context, state management, and you've got to know when to stop recursing. Once again, if you use a visitor pattern where you've got an external code driving your visitor, you often want to stop or you don't want to go into the children of this node. How do you do that?

That means you end up with a more complicated API. State tracking is quite complex. You need to keep track of the parent node when you're walking a tree. You need to maintain scope information in the case of a compiler, build up your composite results, handle cross-node relationships, i.e.

you want to link this node which you visited 30 minutes ago with this other node which you're visiting now. Now error handling just multiplies the complexity and edge cases keep appearing. For example, parenthesised expressions, nested structures, optional elements and language specific quirks of which there are plenty in Solidity. And finally you've got a big maintenance budget. Now I know I'm repeating this point but this is significant as you'll see when we get to queries, which I think is the next slide.

So this is where queries come in. So let's look at how we solve some of those same problems using queries. So this is our first example. We want to find all the unit variable, all the uint variable declarations. It's as simple as saying, in variable declarations where the type name is uint and it's got an identifier, find that and return it.

No traversal. No edge cases. Hardly any maintenance. No efficiency problems. So the second example shows something a bit more powerful.

In this case we want to find immediately nested function declarations, i.e. a function inside another function. So here we're saying in every contract definition there are some contract members. In the contract members there's a contract member.

For every function definition in there find the block and look for a statement in there that is itself a function definition, bind that to a variable called nested and return it. So why is this so powerful? Well first of all it's declarative, where it's saying what we want, not how we want to do it. It makes our code easier to understand and maintain, and when you look at this two months later it's obvious what it's doing. Whereas I'd suggest that you'd have a couple of pages of code traversing your tree if you were using manual traversal.

And it wouldn't be obvious what it was actually meant to do. So it's structure aware. So the query understands Solidity's syntax structure. It knows about scope and it knows about nesting. And it also knows where type name, for example, in the first case here, appears in the tree.

These patterns are composable. So we can build complex pattern matching from simple pattern matching. So if we wanted to find uint variables within these nested function definitions it would be as simple as putting that, appending that first query below the second query. But most importantly this is focusing once again on what we're trying to achieve. We can think about code patterns and structure design without getting bogged down in the implementation details.

So let's have a look at some more advanced query patterns. This one isn't that advanced, but for example in an unchecked block you want to find all function calls. It's easy as that and it's obvious what it does. You could write this on one line. The second example is a bit more complicated but here we're looking for variable declarations, state variable declarations that have a type that is either a mapping or an array.

And think about what this would take if you're doing manual traversal. You'd need scope tracking, type checking logic, complex conditional logic, and you'd need to carefully handle the nested structure here. So what are the use cases for this? Some use cases, because I'm hoping that people come up with far more use cases than we know. We want emergent benefits from this.

Well, if you wanted to have custom coding standards, you want to enforce custom coding standards. You've got project-specific restrictions that you want to check. Or you want to do version compatibility checks, so you want to make sure that you don't use certain features of solidity. Style checking. Well, if you've got specific naming patterns you want to enforce specific structural conventions and you can have context aware rules now you might be thinking well can't I use a linter to do this well yes but this is meant to write the linter this is designed for writing linters so that's why these are the use cases.

This is not an end user tool. This is a tool for developers to write tools. So you can do pattern detection. Anti-patterns, for example. Optimisation opportunities, which may not be immediately obvious if you've got a big library of things.

Detecting complex structural patterns that you might want to simplify or mark. So once again, this is a kind of an ESLinter equivalent. Code transformation. Automated refactoring. Refactoring is transformations that preserve the semantics.

Code modernisation. It's very easy to write patterns that then transform into more modern code. Automated modifications, i.e. things that aren't refactorings that actually change the code.

And formatting. And as I said, Prettier Solidity is already using slang. One important point I want to make here is that we support WASM and we support specifically the component spec and wit. So for those who know what that means, this means that all of this technology works in the browser. It's also available as a Rust API or as TypeScript, which is our first target because most people are using TypeScript, which is our first target, because most people are using TypeScript.

So, some more applications, code transfer, I've already done that. Documentation generation, it's easy to generate automatic documents by extracting function signatures, doing structure analysis, checking for particular usage patterns and documenting them, and processing comments. As I mentioned before, you might have validation information in a certain specialised comment format. Now, you can also use this technology in combination with AI tooling, for example, to produce diagrams from your code. You can use this by encoding the slang API as a rag.

So what are the key benefits? Well, structure-aware queries. Your queries naturally align with the structure of the code that you're trying to match. You don't have to think about trees and nodes and traversal. Complete syntax preservation.

You never lose information. Perform transformations without losing the formatting. And you can round trip from a CST back to source. Efficient pattern matching. So we can spend all the effort required to make this efficient.

If you know anything about the state of the art in terms of tree pattern matching, and in fact we're not a search engine, we are a unifier. So we use a prologue slash datalog based mechanism which returns you all the potential matching results, not just the first one. You can avoid unnecessary traversals. We can cache results. We can index the syntax tree.

Make it very efficient. It would take a lot of effort to get the same result if you were doing it yourself. Composable syntax rules, you can combine simple rules to make bigger ones, you can have a big library of these patterns and you can reuse them over your code, share queries between your projects. You've got maintainable analysis code. So the queries express your intent, not how you want to do it, but what you want to do.

Your code is shorter, it's easier to understand. Any changes to the language, we take care of that. Slang takes care of that, so you don't have to. Less code means fewer bugs. And these benefits compound.

It's not one plus one plus one. It's the whole is far greater than the sum of the parts. So what impact does this have on your development? Well, before using this query API, you'd spend significant time writing and debugging the traversal code. You'd struggle with maintaining the analysis logic, and you'd face challenges when adding new features.

After it, you end up with clear, focused code. It's easy to maintain. You've got a robust implementation, and it's highly extensible. And that is it. Any questions?

Fantastic. Thank you, Anthony. All right, let's get with the questions. So, folks, remember, you can scan that QR code, add questions to that list, smash that upvote button so that the most interesting question gets asked. Let's go with the first one.

What are our advantages to using slang compared to Semgrep? Well, I must admit I'm not familiar with Semgrep, but I know the general concept. The specific thing here that this is a programmatic tool that you use to build other tools. Now, of course, you can combine things like Semgrep, I imagine, but I don't know if that is something that you would include in a tool for analysing lots of different versions of solidity. That makes sense.

Thank you. How does slang differ from a fine-tuned LLM? Could slang's modularity allow to use LLMs in the future? Oh, this is... I'm in applied research, and this is something that is very actively under research.

I'm not making any commitment, but if you've used fine-tuned LLMs, and I've used them a lot, for example, a lot of the code that I write is actually written by Claude. A lot of this presentation was written by Claude. Amazing tool, but you always need a human in the loop. a lot of this presentation was written by Claude. Amazing tool, but you always need a human in the loop.

So, yes, you can do this using LLMs, but the thing with LLMs is that they are a human augmentation tool, whereas the approach that we're talking about here is exact and precise. If you've used LLMs to do this, you know that you spend a lot of time on prompt engineering, and then you cross your fingers, and something comes back, and it doesn't quite work, and you prompt it again, and so on and so forth. So, yes, this is certainly something that could be included in the future. There's a chance, but maybe not now. I mean, not immediately.

Certainly not now. Okay. Are the limitations of a tree visitor-based approach not solved by using an API with CFG and a terminated representation like Slither provides? Well, if by CFG, well, you're either meaning two things. You're either meaning the control flow graph, which is typically something you'd find in an abstract syntax tree, or if you mean control flow as exposed by rust for example and this is something where you reify your control flow.

So you return a token in your visitor which says do you want to continue, do you want to go down the trees. Now this tree visitor, that are, these solution, these slither may well solve the same kind of problem. We have a different set of constraints. For example, our query language is intended to be extended with semantic predicates and with the ability to do arbitrary recursion by skipping over arbitrary subtrees. So there's a lot that we want to do there.

So I wouldn't say that in Slither you can or cannot do that. I'm not familiar enough with Slither. Perfect, thank you. Is there a question you'd like Moose in there? Well will this work with different EVM versions?

Yes, it will, because we're dealing with source. We're well before EVM. Slang is a compiler, and it'll produce bytecode. So that's a question about the compiler, not about querying. Have we published the grammar for slang?

Yes, we have. Slang is actually a declarative meta project. So you can use it for any programming language, not just Solidity. So we have a very good open source, we have a very good declarative meta project, so you can use it for any programming language, not just solidity. We have published that.

Everything we do is open source. Nomic, foundation on GitHub, all of our development is in the open. You can reuse all of this. Awesome. How important is it to maintain code with different Solidity versions?

Why not just use one version? Because if you want to be able to provide tools that analyse a large number of contracts, which may be an earlier version of Solidity than the latest one, for example, 0411. There's a contract on Mainnet that is 0411. You want to get the source to that. You want to provide analysis tools for it.

That's why we want to support all of these versions. Now we don't go back to 001 or whatever the first version was. So we're pragmatic about it. But these are the versions that are live, and our goal is to support all the versions that are live. How do you determine the cut-off version?

What's pragmatic? Analysis of Mainnet and all the contracts that are live. Okay, is it the number of contracts? Is it the value stored in those? Is it the amount of transactions?

No, the earliest. I didn't do this analysis, so I must admit I don't know what process we used to determine that, but it is the case that the earliest contract in use, apparently, is 0411. Fantastic, thank you. What's the best practice for using slang to analyze a large number of programs? Where should the data be stored and in what format?

Imagine this is something you run locally, correct? Yes, it is. Perfect. Thank you. All right.

What update in Solidity was the hardest to adapt for? You mentioned quirks and weird updates in Solidity. Which one was the hardest? None of them was particularly hard. The volume of quirks is what is a challenge.

I mean, we've got hundreds of edge cases. We had to analyse Sol-C because that's the only definition of the language and go through the code and then test it and then go to Sanctuary and look at massive numbers of contracts, run our compiler over them or run our parser over them. What breaks? What doesn't? We've got extensive test cases.

So that was the challenge. It's the number, not any particular difficulty. Fantastic. Thank you. Do you dream in bytecode?

No, I don't dream in... I dream compilers, actually. All right. Anthony, thank you a lot for your time and all the best to you. Thank you a lot for your time and all the best to you.

Thank you. People, our next session will start in a few minutes. I am off for today. Thank you for those who spent a bit of time with me this morning. And I'll see you soon.