Standardizing Ethereum metrics: PeerDAS and FOCIL retrospective

So, my name is Katya, I'm an Ethereum protocol contributor, and I made an initiative to standardize metrics in the Ethereum protocol. And today I would like to talk about this initiative and how it can help to build Ethereum, to test Ethereum. So this talk is mostly for DevOps and core developers, but maybe if you're a staking operator and you run many nodes, many different clients at a time, you probably would be also interested. So, quick agenda for this talk, I will share my EPF journey, how I build metrics, specs, the standardization process itself, what challenges I've had, what's going to be next, and a quick overview of this talk. So I joined the Ethereum Protocol Fellowship last year, it was Cohorts 5.

For those of you who are not familiar, what is EPF, it's the Ethereum Foundation Program for those who want to contribute into the protocol, mostly developers and researchers. And this is where you can start contributing. And at that time, PeerDesk was in early stage, and I was interested, how can I help in this process? And I had the conversation with Iskandar's team, and I found out that that would be great to have the same metrics for all the clients, because it was early testing phase. There was an initiative in 2020, but it wasn't adopted, because it takes a lot of resources to do this.

So, yeah, I started building metrics, specs, beacon metrics, specs, specifically, so for consensus clients. I had initial list of metrics, which was taken from the first implementation of PeerDesk. Then I started to build them all across the consensus clients. I also built Grafana dashboards for Cortosis. Cortosis is the tool for running blockchains locally, and Cordes mostly use it for testing Ethereum.

And I also built the dashboard for ETH Bundles. This is an example of how the dashboard can look like with the standardized metrics, so we can see all the clients on one dashboard. And this is super helpful, because before this dashboard, DevOps just need to monitor many dashboards with different clients and compare them or make different queries, but they are not sure if this metric is the same across the different clients. So, and here you have just one dashboard, and you can easily apply filters and just to explore, to monitor the clients and to explore the issues. Yeah, just discover any problems they may have.

And this is super helpful during the testnet, because currently we're testing PeerDesk, which is Fusaka next upgrade, the major, the flagship EIP for Fusaka, which is scaling Ethereum. So, I also started to standardize Fossil, which is another EIP, probably for Glastonburg, not sure yet. But from these two EIPs, PeerDesk and Fossil, what I brought here is like I tried to make some steps how standardization can look like. So, the first step is when EIP is in progress, this is the first, this is a good time to start standardizing. So, we can make an initial list of metrics, what is needed, what would you like to monitor.

And during the breakout room, we can discuss the initial list. So, the next step is prototyping. So, normally after EIP is more or less ready, the builders build the first prototype. And here we can already apply the initial list of metrics. And all and test it in Krutosis.

So, Krutosis has Grafana dashboard. So, we can already see the result of the metrics. Then, sorry, other clients join and we can have like a small interop between clients. As you know, Ethereum is not one client. It's many clients, client diversity.

So, we can also already, if it's the DevNet phase, we can already see all the clients with the same metrics on one dashboard. After that, definitely there will be corrections because specs evolve and metrics also will evolve. So, we will have discussions on do we need any new metrics or should we correct the current ones? And the last step is production. So, all these metrics, standardized metrics will go to production.

And this is where, for example, if you're a staking operator, so you will see the same metrics for clients on one dashboard. The challenges I have. So, it's hard to plan the list of metrics ahead because definitely you need at least one implementation just to see the metrics on the dashboard because it's not really clear so what buckets you need or what metric type you need. So, you can definitely correct them after you have first implementation. So, you just can't give the list, the pre-built list of metrics to the users and say, yes, please build this.

You definitely need to correct them. The client diversity is also a challenge because clients are written in different languages and metrics, for metrics it doesn't matter. But anyway, our clients have their own style on how to build metrics, the granularity of the metrics, the naming, et cetera. So, and even if one person tries to build the metrics all across the clients, it takes time and a lot of effort to do this. Naming in units is also a challenge because as I previously said, clients have their own style, what units they use, what namings they use.

So, to standardize metrics, which are already in production is definitely difficult. So, this is why I started from the EIP, which is not yet in production. So, what's next? So, currently we're standardizing metrics for Fusaka and the latest ones are GitLabs v2 and custody group count metrics, almost done. And also the metrics for execution client.

So, execution client joined Fusaka, I mean, appeared as specifically a little bit later. So, we're still in the process of standardizing it. The next goes Glamsterdam, which is currently in discussion. So, which EIPs joined Glamsterdam and there will be metrics for execution and consensus clients. And the next H upgrade, we definitely need to standardize the P2P and Gossip Submetrics because the bandwidth is a bottleneck for Ethereum and it's definitely important.

So, a quick overview, why we need standardized metrics. We definitely need them for testing. So, when you test a lot of different clients, you definitely want to see them one at a time and one dashboard. And that makes the testing process much, much easier because you can easily discover any issues. Second, monitoring.

If you want to monitor many clients after the metrics went into production, so it will be easier to do. Research teams also may need standardized metrics because they work in analysis and they build benchmarks. And of course, staking operators may need them because they run different consensus clients and execution as well. So, that's why they might want to have the same metrics and one dashboard for all the just for DevOps. Thank you.

Yeah, this is my experience. And if you have some cases, so why you need standardized metrics on Ethereum specifically, please let me know after this talk. Maybe you are interested. Yeah. Thank you.