Optimism Retro Funding: So Far, So Good, So What!?

Thanks everyone. Before getting into crypto, I did work in the coffee supply chain and I've been getting a lot of questions about where the best coffee is in Bangkok and I'm still looking for an answer. So if anybody knows where to go for some excellent specialty coffee, please hit me up. I'd love to check that out. Today I'm going to be talking about Optimism Retrofunding.

It's a experimental grants program being run by the Optimism Collective. And as a data enthusiast, it creates this incredible substrate for looking at the effectiveness of different grant programs. I'm the co-founder of an organization called Open Source Observer. We're building a free analytics platform and a community of data scientists who care about measuring impact in the open. And so today we're going to be talking about the impact that Optimism is achieving and identifying some opportunities to improve and double down on where that impact has been most effective.

So I guess just a quick check. How many of the people in this room are familiar with retrofunding optimism? All right, pretty good. So I have a few slides that will be a bit of a cursory introduction to optimism. I think the main point that I want to get across here is that I believe this is a really important experiment for the world to pay attention to and to understand.

Regardless of whether it succeeds or fails, in the long run, there's something special and different that is happening here, and it's worth paying attention to. So another chart, or first chart of the day. We have a lot of charts coming. But it has another poll. How many of you have seen this before?

How many of you are familiar with the Electric Capital Developer Report? Have you seen this? All right. Not as much awareness. This is one of the most looked at reports that the crypto industry puts out.

And they track a metric called monthly active developers. They've been putting this out for quite some time now. It's basically an indicator of how many developers there are working across all of crypto, not just Ethereum. Every ecosystem is captured here. And so what you can see from this chart is that things went up a lot in 2021.

They kind of plateaued in 2022. And now things have gone down in aggregate from a peak of around 30,000 to about 26,000 monthly active developers by this metric. To put this in comparison, I recently heard that Gemini, which is Google's AI platform, they have 1.5 million active developers by their count. I don't know if they're comparable, but orders of magnitude greater than what we have here across the crypto industry.

Here's another metric. This one is the amount of venture capital that has gone into crypto since the beginning, basically. This chart stops at 2022, but there's another number that they care about, which is the total amount of deal flow. If we add this up, we get a number of around $80 billion has gone into the space since 2016. So I think the interesting thing is actually when you start combining these types of data sets.

I don't have the ability to go in and look exactly how they calculate these things, but I can overlay the two. And when I do that, I can come up with a new metric of my own. And that metric is the dollars spent per developer retained. And what you can see here is that for every $3.1 million that has been invested prospectively as risk capital, we only have one retained developer in the industry still here to show for that.

So I think we can do better, and we hope to do better. The point is not just that the venture capital can improve, but really that prospective funding is difficult. Behind every one of these deals, there's a team of professionals that are researching these companies and making bets. And the reality is that it's often quite difficult to do this. And there's an acceptance that most of these bets are not going to work out.

So prospective funding is hard. Retrofunding is based on the principle that it is easier to identify something that has worked looking back than to predict it going forward. Vitalik helped design the mechanism. Optimism has been rolling it out. And it's a pretty unique way of creating markets around impact.

Now to be clear I don't think the most die-in-the-wool retrofunding maxi would say that retrofunding is the only way that anything in the world should be funded but the important point here is that right now we have a surplus of, not a surplus, but a large amount of prospective funding and we need more retrospective funding in the world to signal what things we value and drive more people to build in those directions. And so that's what we'll be talking about today. At launch, Optimism allocated 850 million tokens from its treasury to retrofunding. This is a big deal. And they made a promise that if you create impact on the Optimism collective, you will be rewarded for that impact.

What that impact is, how it will be rewarded, TBD, but that's what we're here to discuss. To date, optimism has deployed around 60 million OP, or at least allocated, not all that has gone out the door, which is a pretty small share of its treasury. So even though this is a large sum of tokens, we're still very much in the early innings. And each round has been designed around a specific set of experiments, hypotheses that they wanted to test. Round one, I think, was completely off chain.

It was a proof of concept, about 50 projects, no applications. The selection was done, I think, by the OP Labs team. Rounds two and three are where this program gained a lot of recognition. They were big mega rounds. 40 million OP went out in total.

More than 600 projects participated. There were 100 badge holders, also known as citizens, who were voting on projects. I was a badge holder. I think there are a few in the room still. But they were given this enormous task of reviewing all of these projects and identifying which ones they thought should deserve a certain amount of funding.

In response to the mega rounds in 2024, this past year, optimism changed the scope a little bit and instead of having these large mega rounds, they moved to a system where you have smaller, more tightly scoped rounds with tighter eligibility criteria. And in those rounds, they also tested different experiments around badge holders and guest voters and expertise. In total, about 18 million has gone out, and there's more allocated through the end of 2024. Again, we're going to be looking at the results of the program so far, but putting this in the scale of other public goods funding experiments, this is pretty unique. It's pretty large.

And my team and I at Open Source Observer, the fact that this was a real experiment of significant scale was what attracted us to want to pay attention and work at Optimism. There's a lot of economists that like writing long papers about how public goods funding is broken, but here was a real chance to throw magic internet money and test whether we could actually change the incentive landscape. So I think people should pay attention to this, regardless of whether they participate directly or not. But first, because the only thing that people like more than throwing tokens around is complaining about things on Twitter, I want to address a few myths about optimism and the program of retrofunding. So the first myth is that retrofunding comes with conditions.

The reality is the projects that receive retrofunding are getting with no strings attached. They can cash out, they can migrate their projects to Solana, they can delegate to governance, or they can keep building. And what we see continuously is that most people actually tend to keep building on optimism. The second myth is that retrofunding is optimism's only grant program. This is also not true.

It is also... Oops, sorry. It is not the only grants program. It's also not the largest grants program. Optimism also runs perspective funding.

Since 2024, they've given out more than 60 million, I believe, in perspective mission-based grants. And most of the projects that are receiving retrofunding have also received some form, or a large number of the products that have received retrofunding have also participated in these prospective grants. I think there's a narrative that Optimism expects all of its teams to work for free and pray for retrofunding. This is not the case. There are various mechanisms and retrofunding is just one of them, and obviously the most experimental at this point.

Myth number three, Optimism no longer cares about public goods. You've probably heard this before if you follow the optimism narrative on Twitter. I think this narrative really took root after optimism changed the name of the program, but the reality is that there was never a formal definition of what a public good is. Optimism and I kind of agree with this definition have always viewed it as a kind of a spectrum and the role of retrofunding is to fill the gap. It's identify where is the market not effective in rewarding impact and using tokens strategically to fill that gap.

Myth number four, VC-backed projects have an unfair advantage. We can actually check the data on this. And what we see is that VC-backed projects are a relatively small number of the projects that have received retrofunding. And historically, they have not performed as well as projects that do not have any VC funding. So at least to date, VC projects do not seem to have any unfair advantage, even though there are some large ones that have participated in the rounds.

And myth number five is that Optimism's badge holders are super good at deciding how much retrofunding each project deserves. The reality is that we are just humans. We have other jobs in many cases, other ecosystems that we are working on. And so it is very difficult for any one badge holder to have a God view of all of the projects and what they're doing and to be able to review each one with a level of care that that project most likely deserves. So I think that a lot of improving retrofunding is going to be about finding the right balance of figuring out what humans and algorithms are good at so that we can make it more efficient, but at the same time ensure that we are learning continuously how to improve the allocations and make sure we get closer to impact.

So far, so good. Now we're going to get into the so what. Has this program actually delivered on these promises? Is it effective at rewarding impact? Or is it just an expensive, well-branded marketing campaign?

To date, Optimism has supported a lot of projects, more than 700. And there is a power law distribution in terms of how much funding they have received. So what this graph is showing on the x-axis is all of the projects ranked by how much funding they have received since the start of retrofunding. And you can see that an average project that's been in the program has gotten about 40,000 OP, a top one about 160, and there's a few that have received over 500,000, at the time probably worth more than a million dollars. So this has been the distribution so far.

It is a power law. It's not a kind of flat, even distribution. And what you can see is that the projects that tend to receive more funding are projects that many of you may be familiar with. At the very top you have Protocol Guild. You also have teams that have focused on developing specifically for Optimism, like Test and Prod.

You have core primitives on the super chain, like Account Abstraction, 4337, EAS. You also have some of the major DeFi protocols, like Velodrome, and social apps like Zora. You also have developer libraries, like Ethers and WebM and OpenZeppelin contracts that historically have not received any funding of this scale. And so this is also creating a recurring revenue model for those types of projects. In any case, it isn't just funding a narrow set of DeFi protocols.

This is a pretty expansive and diverse set of projects that have been receiving retrofunding. And anecdotally, retrofunding has had a fair amount of impact. One project, L2Beat, they wrote this beautiful letter at the end of one of the rounds thanking retrofunding and the badge holders, but also explaining how it actually changed how they thought about their purpose in the ecosystem. We can Ethereum news. This is also one that I can really relate with, that it was kind of a game changer for them receiving retrofunding.

It allowed them to work on this and not pursue other things. At Open Source Observer, this was definitely a game changer for us and gave us the confidence to leave, start a new organization, and focus on building for Optimism. So are we a few outliers or is there some meaningful signal here? At Open Source Observer, the project that I'm working on, we track cohorts of other projects and collections to see how they perform over time. And one of the things that we can clearly see in the data, if you remember that developer capital report I showed at the beginning, the trend line for optimism projects, especially ones that have received retrofunding, is different.

On the x-axis here you have time, we have a few milestones related to retrofunding three, the biggest round, and you can see on the y-axis the number of monthly active developers that have been participating in the program, both before and then coming out the other side. And, you know, the line is up and to the right. But there could be a lot of things that are explaining this. So the next thing we want to look at is how would they have compared relative to some kind of a counterfactual or control group. We're not in a position to basically separate out the groups and do a pure experimental test.

But what we can do is create a synthetic control using data about other ecosystems. At Open Source Observer, we look at not just optimism, but a range of other crypto ecosystems. So we can create comparable metrics for how projects in those ecosystems have performed in the absence of receiving retrofunding. And so through this, we can see that there is actually a significant increase relative to this synthetic control for projects that received retrofunding from round three. We can do this for any number of metrics, and I've only shown a few here, but this is a comparison of on-chain activity for projects that receive funding from retro funding for around the focus specifically on on-chain builders and here you can see that there is a pretty significant difference that has happened both around the time of sign up but then after the results for retrofunding 4 were announced.

The challenge here is that there are other confounding factors. We had the blobs, excuse me, we had the DEN couldn't upgrade, and as a result of that, transaction activity spiked across the board. So we can strip that out. And in this case, we don't see as clear a signal as we saw previously with the developer activity. So this is a whole line of research that we need to do, that we need to get better at to isolate the impact that retrofunding has had on different cohorts of projects.

Now if we step back and look at the overall allocation that Optimism has had so far, we can see that most of the funding has actually gone towards these two big boxes over here. So that includes a range of governance, education, education analytics projects, as well as technical contributions to the OP stack. Those are these two big boxes over here. And then off to the right, we have some of the categories that are probably most critical to the success of the super chain, which have historically received relatively less funding. So that's about $14 million for the on-chain builders, and then about $4 million for the developer libraries.

So I know there's other programs that are, other rounds that are in the works right now to put more funding into developer tooling. But historically, more of the funding has actually gone in these other areas. And so that could partially explain why there's been less impact on the on-chain activity than general open-source developer activity. So what's next? This is just one of a number of areas that we are excited to explore together with the optimism community.

Obviously, there is some quantitative and qualitative evidence that retrofunding is having an impact, but we really aren't in a position right now to make any causal statements about that impact. For us at Open Source Observer, this is a really big area of focus in 2025. We also know, and you don't need to bring up Goodhart's Law just yet, but we know that it's very easy to turn metrics into targets and to game them. On the other hand, we also know that to scale this program, data is going to play a critical role. And so the real challenge here is finding the right balance of human judgment and data that can improve decision making.

Metrics should be evolutionary. As you learn more information, they should improve. And we should be able to backtest the results that we get from each of these rounds against the metrics and the decisions that were used to fund those things, and ideally incrementally improve each time around. This is also a big area of focus for Open Source Observer in 2025. One of the things that we learned from recent rounds is that technical experts evaluate impact differently than non-experts.

Now, this shouldn't be a surprise to anyone. Up to now, the badge holder pool that Optimism has assembled consists mostly of a mix of experts, technical experts and non-experts. And in round five, there was an experiment that was done to actually test the groups of experts and non-experts prefer different projects. Did they vote differently? The answer, probably not surprisingly, was yes.

There were pretty big differences in how they voted, and not just on the projects they picked, but also how much funding they thought top projects should receive. So this chart here shows some of the most divisive projects, ones that received a pretty big difference or big spread between what experts thought they deserved and what non-experts thought they deserved. And so going forward, I think that feedback is going to be used to improve the design of future rounds, and particularly ones that are more technically focused, ensure that the right composition of badge holders are equipped to, you know, vote on what this product should receive. Another issue that comes up are just, like, the incentives this sends to different types of builders. So you have some big projects, you have some small projects, and when you have an allocation which is not like a very steep power law distribution, what you can see is that smaller projects, there might be an incentive to create smaller projects or to atomize your work into kind of a set of very small individual contributions as opposed to capture the collective impact of a larger team we can see this at least in round five where some of the products that did the best had the smallest team on a per contributor basis and so this could create some kind of perverse incentive for teams to fractionalize their work as opposed to representing it as one cohesive team.

Trent from Protocol Guild wrote a very nice essay pointing out this fact and doing some comparisons. And then the fifth one is that there is a trade-off between mathematical accuracy and just the vibes and the spirit that comes with each of these rounds. We quants, we tend to suck the life out of things, and there's something very special about the spirit and the vibes that are created around each of these retrofunding rounds. And so as the program becomes more accurate and predictive, there could be a tradeoff in terms of the energy and the attention that this gains because there's less speculation and less hope for some builders. We definitely saw that round three kind of marked a high water point in terms of the social media attention, at least, that retrofunding received.

We believe that the upside to figuring out these things and working on these challenges has the potential to be huge. If you just look at the size of the number of developers that are contributing to open source, it's a global movement, and if this were a nation, it would be as large as the population of Thailand or Mexico or Germany. So there is a large, global, borderless movement around open source. And historically, we have not been very good at funding it. If this were a nation, the amount of funding that it is allocating towards its public infrastructure would be on par with Fiji and Swaziland, pretty low down the ranking.

So we need to do better. We at Open Source Observer are very excited to work on this problem. We're hoping to attract more data scientists to care about the intersection of public goods and impact and data science. And if that speaks to you, we'd love to hear from you. You can scan the QR code to get in touch, join our Discord.

And for any builders that are in the room, Optimism is having a seventh round of retrofunding. It'll be starting hopefully before the end of the year. You can go to retrofunding.optimism.io, get all the details there, and sign up.

Thank you very much. Awesome, Carl. So you know next retrofunding, it's coming close. So take the opportunity. Questions that you have for Carl.

Let's start that way. Meanwhile please use the QR code to place your questions uh to ease. So go ahead. Hi Carl. Thomas here.

Hey. Hi. Uh thanks for the presentation. I'm uh just want to say first of all I'm a huge advocate for open source software. I use it all the time.

My day to day activities always always using libraries, frameworks, which you don't even know how open source they are, right? So I wanted to go back on two slides that you had previously mentioned. One of them was around the development on-chain activity. You don't have to go back, but it's fine. I was wondering would you, I know you said there were a handful of other metrics while you were speaking but you outlined these two wondering, would you say these are the most important metrics for measuring and evaluating the impact of funded projects?

No I would say that they are good trailing metrics so ideally if you have if you're doing other things well then you'll get more developers you will get more transactions, you'll get more users but you probably don't want to fund these things directly. If you were to just fund people based on the size of their team, then you'd get very big, bloated teams, you wouldn't actually get the impact that you care about. On the other hand, they do probably correlate, in many cases, with, at a high level, the growth of an ecosystem. So I'm not in a position to say there's one metric where you should just fund this thing. And I think that's the whole point, is that we need to test these things out in the wild.

We need to combine them in different ways. And then I think over time, we'll see some kind of composite set of metrics, some which are really good as these trailing indicators and some which could actually be more predictive of future success. Yeah, it's a big search space. And just a follow-up question, how much time would you reckon you would need for that? For which part?

You mentioned you're gathering data to make a well-informed decision. How much time do you need to make that decision? We see this much more as a challenge for the community. So we can definitely come up with metrics ourselves, but increasingly we want to partner with data teams, you know, projects themselves, people that have an intuition about what metrics they think are most useful, and test those out, experiment with them, and, you know, everything, all the metrics that we're building are fully open, so people can come and test them out directly. Thank you.

Yeah. Now we have some questions here. One is, what are the opportunities for data scientists to get involved? Yeah, so scan the QR code. We have a number of contribution opportunities.

Some are pretty basic, just like helping add, update, process data. What we're really looking for help is people that have the ability to do modeling. So strong Python skills, they're able to do things like the causal analysis that we showed. They're able to process, help us connect new data sets and process larger and larger amounts of data. Anyone who has the ability to formulate research questions and then design a study around that would be super welcome.

But also people that just want to create front ends and dashboards on top of it. There's a lot of cool things you can do. Great. Awesome. And for how long is Optimism going to be running retro PGA fronts?

I think the answer is probably forever. That would be the goal. Certainly, the plan is that once the initial treasury runs out, that there's a continuous source of funding that will come in from sequencer fees or from other sources of on-chain revenue. I think the goal, eventually, is that everything that is happening in optimism gets funded by retrofunding. Whether that takes five years or 10 years remains to be seen.

But the goal is that this is a system that runs forever. Awesome, Carl. Appreciate it. And appreciate all the questions you have. We just run out of time.

So please give a great applause to Carl. Thank you, everyone. And Optimism.