JSON-RPC Enhancement in Geth

Hi everyone, this is Jess Fieser and I come... Thank you. We can state DB and to replace the transitions. This is really slow, and it costs a lot of CPU. So we want to introduce a real-time tracing system, which will offer more efficient and Promoting insights into the EVM execution and it will also support indexing the left execution data such as traces, the state of differ, and the other things So let's see how can we go to how to index source left data.

Currently we support indexing traces, which is grouped by a block. And so this is a block number and minus two traces. And us indexing is based on the nonce. So this is when we want to index as a transition senders address plus its nonce to our transition hash. So in my presentation, I'm using KVDB, your status stores, newly created block traces and analysis.

This is just a genetic query database, and I use the ETLDB in GALS. For the KIN schema, for the traces, it is just a block number plus block hash and there's a trace tab. And for the NUNs, it is used as a KIN schema is the tracing sender address plus nonce And the failure is just the IOP encoded data So the implementation is very simple For the trace tapers, currently we support the code traces and other traces such as parity traces and the suppressed traces. Here is the workflow of indexing. For each block, we will index each transition based on the TSAT starter hook.

And in this hook, we will see if we enable non-traces. If so, we first store the non-traces in the KVDB. And if not, we just continue. And for each transition end, we get the transition result in our local cache. And when the block is ended, we will store the block traces into the KVDB.

And in the meantime, we will notify our RSS background for this to notify him we have just indexed a new block. And for this, we will for this newly created traces into rsdb for the efficiency. And so this is just for the indexing and let's see how can we retrieve those data. For the traces, we can retrieve it by the block number and the block hash. And for the transit hashes, for the transitions, We can retrieve it by the block number or block hash.

For the transitions, we can feature the block first by the transition hash, and then retrieve it by the block number and feature the transition hash data by its index. For the nonce, it's really simple. We can just retrieve it by the send address and the first nonce is really simple we can just achieve it by the senders address and the nonce and here is just a small follow of block traces which if index yeah it's very simple just read the block number first and check the block data in is the wizard in the kvdb or in the forest number first and check the block data in the KVDB or in the ForestDB and then return back to the column. So, how can I use this feature? As currently, this feature is not merged into the mainnet, into the master branch.

So if you are interested, you can follow this PR and see how to use it. The left side is the configuration of the tool. In the feature, you just need to add something like this in the left side. And on the right side, I will introduce every configuration. So first one, path, is just a dictionary which is used to store source data.

And the second one, enable nonce-translator, is index the left trace, whether to enable the nonce-trace or not. And the third one, mask-keep-blocks, is just a block limit. So after such blocks, we will print all the data just to save disk size. And for the config, it's a map. It's a map to configure each tracer.

And for each detail, you can refer And the first each details we can, you can refer to the built-in, guesses the built-in traces for more detail. Okay. Currently we support source RPCs. It's like as a paratest trace RPC. And the last one is the RPC, just this is just a proposal and it is not enabled for other clients, I think Besides all the parity co-traces, we can also retrieve other gases' native traces I think it is really useful when you want to get history data or history police data for one transaction on one block.

We can just like this just RBC, in this example we need to retrieve the police data tracing for block one. So next we can just talk about performance. Yeah, before I talk about performance, in my mind, performance is just a space and time trade-offs. In this implementation, we just use a low-cost disk to store the data instead of high usage of the CPU. So for the advantage of the feature is just you no need to reapply the traces based on this data and the time compensation is very small, it's just all one.

And the retrieving is real time, you can index the data and then retrieve it back. And the disadvantage is you need another disk space to store this data, and it may have a little impact on the chain block syncing. Here I just write some benchmarks to compare the results of the Chase R RPC and the Debug RPC. It's a script to retrieve the data from the guest-wise Chess RPC. And this is our results of the Chess RPC results.

The upper one is traditional Debug Chess block, and the lower one is traditional debug trace block, and the The later one is trace block. It is about 100 times faster Than the traditional one. Yeah. So it's all thank you. Thank you very much.

Your question? Hey, thanks for this. I really appreciate it. I think as a developer, it's sometimes confusing because there's no standardization on the debug or trace namespaces, right? The other RPC methods are in the execution API, so they tend to be standardized across clients.

When you move to debug tracing, it becomes a bit trickier. Other than performance, because some clients are going to implement debug, some implement trace namespace, some do both. Other than performance, why would you recommend trace over debug? Is there other reasons you would recommend that option for tracing? It is more customized.

You can just index data by your need yeah and the cost is a traditional debug trace which is also support only support such as the parity trace, code trace and the flat code trace and if you wanted to index other things you might need to write a JS, yeah. And the performance is really very, is some part is not very good. So, in this situation, we just introduced this living Chasers framework. And the best on set, you can just write your own Chasers. Right, thank you.

I have another question unless someone else, maybe. Probably means that this will be integrated into or when it's going to be added together? Actually, I'm not sure. Maybe this feature will be much into the master branch and insert for or maybe one. No need to have four.

Yeah, just add your connection. Maybe can you touch real quick on the, you were talking about some potential issues on the syncing side. You had a con, your second bullet point on the slide where you had frozen cons. Do you remember? There was like two cons you mentioned.

One was like additional storage space for trace was one of the disadvantages. And then the second one was you said potential syncing issues. Yeah. Can you touch on that or go a bit more and just explain what you meant by that? Or like at least I guess what you observed when you were benchmarking?

I think. I will attend it and maybe in other works. Thank you. Cool. Thanks.

So most of the speed improvement looks like it comes from caching and indexing the data. Did you look at indexing the built-in native debug trace? You mean the debug traces? Yeah, the built-in native debug trace? You mean the debug trace?

Yeah, the pre-stating the call tracer. So you're basically saying half of the argument is it's faster, but most of that, I suspect, comes from the fact that you're indexing all the data. Did you look at index in the debug trace calls as well? I'm sorry, I have a full answer. So most of the performance increase comes from?

Yeah. . Oh, really? Not from the index? Not from index in the data?

Yeah, yeah, yeah. Actually, the performance is really depends on the history data. Yeah, you need to feature all the history data, need to replace a transition, so the performance is really, yeah. Because the disk feature is really slow, yeah. I guess, like, piggybacking on that question, like, the improvements that you shipped with Trace, the training namespace, could you not backport them to debug as well, or is that not possible?

Yeah, because in the libchaser, the data is just generated and stored when the transaction is used once. And when you use the debug chaser, the data is just replied after the time when you request it. So in this intention, we just start once and the features at any time. That's right, but you're talking specifically about the live tracing here, the real-time tracing, or you're also speaking about when you're, because when you had that graph that was showing 100 milliseconds for trace block, right? And then you had debug trace block at the top.

That wasn't for live tracing, right? And then you had debug trace block at the top. That wasn't for live tracing, right? That was just historical calls where you can trace any block, correct? No, no, no.

Actually, for trace tracing, you need to index the state force. Yeah, just after the guess is started. Right. Yeah. We got to cut it up here.