Bringing peer-to-peer networks to ALL the peers
Devcon 7 SEA·Wed, Nov 13, 2024, 10:26 AM · 10:05
Speaker
The p2p networks of the Ethereum ecosystem generally draw the line to server nodes. True end users devices: mobiles, laptops, browsers, are excluded and use centralised APIs and gateways to access the p2p network. Removing sovereignty, censorship-resistance and privacy in the process. In this lightning talk, we’ll review everything that can go wrong when trying to include resource restricted devices in a peer-to-peer network, using the most popular tools and libraries.
Transcript
All right, thank you very much. Hi, everyone. Let's get started. So, bringing peer-to-peer networks to all the peers. What are we talking about?
So, we are trying to create a peer-to-peer model for end-user devices to be included in this model. And by end-user devices, I mean desktop app, mobile app, web app in the browser. And here, more specifically, we are talking about live P2P GossipSub-based networks. So for those who don't know, GoSipSub is what is used on the Ethereum network for the beacon chain as a network layer. So the current status quo across most peer-to-peer network is that we use REST APIs, Web3 APIs, and other web gateways to allow users in the browser, mobile, and desktop to access a peer-to-peer network.
Usually anyone could run such API if they have the resources. But when building applications, whether web apps or mobile apps, usually the users or the developer will select one API or maybe a couple. So this does create some problems. And we are aware of that in the Ethereum ecosystem. And initiatives like Light Client try to solve this problem around trust, data access, and propagation, privacy, and censorship of this API.
I'll go in more detail at the end for that. So the first problem we had when starting to use GossipSub on, again, laptop, mobile, and browser, is that it has very unstable connections. So if you want to compare that to ECOM nodes, for example, when you're staking, usually you run your node on maybe a Raspberry Pi or a DAP node or some on VPS, where connection is quite stable. But on a laptop, you are going to change Wi-Fi. You are going to go offline, back online.
And GoSysApp doesn't have any mechanism to help you with this specific scenario, so there's no acknowledgement. To know whether you're online is a bit hard. You can have some time out from TCP. And you can miss messages when you're online is a bit hard this can be had some time out from tcp and you can miss messages when you're on when you're offline right because once message goes through the network it's gone so solution we we develop is uses um gossip sub and dc5 for desktop as well as a smart contract and zero knowledge based rate limit to ensure that you don't have too much bandwidth too much traffic going on the network so you can support this network on a household bandwidth. And we have a number of LIRP2P request response protocol to enable browser and mobile to query for more peers, because this V5 can consume a bit too much bandwidth and resources.
And also to push messages and get messages from the Ghostlytops network. So you still have a lightweight to get messages, but it's much more decentralized than the REST API. Finally, in terms of sharding, if you start to put 10,000 or 100,000 of users on the Ghostlytops network, unlikely to work. So we do separate, we see the global network into smaller Ghostlytops network, unlikely to work. So we do separate, or we say the global network, into smaller Ghostlytops network via sharding.
So if you have a quick look at how it could work, so we have service nodes, VPS, that run in the cloud. They allow a mobile phone to go and query for messages they missed. Mobile phone could subscribe to a desktop node and say, hey, can you please forward messages that I'm entering in? Desktop node goes offline and then back online can also query messages to a service node in the cloud. And finally, a browser can push messages to a service node via WebSocket that will then get pushed on the GoC-Sub layer, which we call a relay in our case.
All right, so we did have to add redundancy, right? Gossetab has some in-built redundancy. We had to add that when, for example, pushing messages, we push to a couple of nodes. I also had to add end-to-end reliability protocol so that in the context of a chat application, you know whether or you have some idea whether your recipients or other participants in the group have seen your message. And the result is that now you have some redundancy, and you can check message presence using various nodes to access the network instead of trusting one single REST API.
We do have some improved privacy because instead of having all your traffic going through one REST API, you can select various peers and decide what traffic you ask from them or you push to them. And in terms of censorship as well, it means that you do not rely on a single DNS, a single domain name, for example, and a single REST API, which could be censored or blocked. But instead, you apply a peer discovery strategy, allowing you to access the network via different peers. That's it for me. Five minute presentation.
I think time for questions. Thank you very much. We're going to have a bit of time to ask some questions. Are there any questions? Please raise your hand.
Oh. I think you mentioned status as one of the users. Are there any other apps using this stack or approach you highlighted? Yeah, of course. So, yes, so the status chat app uses that, and they have a mobile app and desktop app.
We have Railgun. Railgun is a private DeFi system where they have a zero-nation contract, and you can do DeFi without revealing your e-mail address. And they do use Waku on their mobile, web, and desktop app where the wallet has Waku running inside with the D2P. And I can send messages, can send basically the transactions, the proof that they have a node to broadcasters which are not in the cloud, which then basically the transactions, the proof that they have a node, to broadcaster, which are nodes in the cloud, which then takes the transaction from the wallet and push it on chain. Okay, thanks.
And we also have other projects, Railgun and Status are the most advanced ones, the most mature ones. I think there's another question in the middle. No? Okay, no. Go ahead.
Hi. So my question was, first is why REST and second is why not gRPC or RPC? Why we're not using gRPC? Yeah, gRPC or RPC. So in the case of lower protocol stack, we're using TCP and WebSocket.
I'm guessing you're talking just above. We're not using gRPC. Instead of I guess. So does come with a number of useful tools. So when we start to build and starting to use for example, it means that we are introducing one technology stack into the system, right?
So we use DPP GossipSub. And when we want to look at request-response protocol, we could indeed start to use gRPC or REST API, but this means introducing new technology stack into the system, which means that having a heterogeneous technology stack means you may have more problems. And so it's just easier to continue using IP2P and define this request-response protocol on top of IP2P. And IP2P comes with good things such as noise handshakes, right? So you have a point-to-point encryption with IP2P out of the box.
So you can just use that out of the box. Good question. So you can just use that out of the box. Good question. So we have time for one last question over here.
Could you share a few more words about the sharding idea? Is it if I have an application that grows massively, would it start sharding itself automatically? Or ? ALESSANDRO CHIESA- Awesome question. So the idea with sharding first is that you basically have a billion users.
All the messages they generate will be too big to go through one household bandwidth. So the idea is to split the networks that you only look at messages you are testing in. That's the writing layer. We have a content topic, which is metadata, like just a string that you can attach to messages, which allow you to further filter messages you are looking at. So on mobile, you are going to only look at specific content topic, which will be a subset of messages on the shard.
So now you have several options. You can either come in and the application is one content topic, and you're going to just be part of one shard. And if you grow up, then you may want to look at seeing if you can use more shards. Or you can start in and have actually a big application from the get-go and just try to split all your traffic from the get-go. In terms of, you can start with one shard, two shards, or three shards.
Instead of growing to more shards, it's just a consensus between operators. So you can say, okay, we need two more shards on our network, and you can just tell your node operators, can we please support charge 10 and 11 now, and they can start to support it and you can start to divert traffic for it. So it's a bit of a manual process and we haven't gone through this course pen yet but it's many operator consensus here. Thank you very much. That was the last question.
So we'll give some applause to Frank.
Automatic transcript — names and jargon may be misspelled.