Ethereum P2P Networking / Sharding by Felix Lange and Péter Szilágyi in Taipei, March 2018

yeah so it seems to be on okay guys last session so p2p networking so we came here to learn about ideas on the research team and then we were asked to do a session so we don't didn't really know what to talk about so what we're gonna talk about this mostly like the existing system and then in the end I think we're gonna have some kind of reverse Q&A session where we are some questions and answers let's see okay so on this side in the p2p network that we have yeah what's the best way to say this so I guess the p2p network that we have is built on a bunch of tools and the tools that we have is like discovering mechanism that can relay node metadata so you can see what those are up to the mainly runs on an unstructured p2p network with more let's round up to quality so it doesn't really matter which exact note you're connected to you can just be connected to any node basically as long as is reasonably well connected everything works and specifically boy theorem we also have a structure clipping network on the wrong side and that thing can come in handy for some purposes but it's not used for the theorem in the protocol level commercialised is the deaf PDP protocol framework and that's present in all of these their implementations it was designed in 2014 and hasn't really changed much and yeah it contains separate protocol support for four for the four functions listed earlier so there's no discovery protocol which is a udp-based DHT there's a tcp transfer protocol that yeah transmits all the data and on top of the transfer protocol it's an application they are in which these sort of soft protocols can be mounted and we have the bunch of support of course and one of them is the e protocol and yeah like I said the each protocol is this protocol that notes use to talk about the blockchain and it uses this random topology and with the authority hole you can relay new blocks you can relay new transactions or existing transactions and you can download block cameras block all these receipts and state data so when you connect to someone you can kind of see it here yeah that you found some note using the DHT and then when you connect with you exchange information about the blockchain on the application there and that information includes a Genesis block and therefore Guardi and some other information that's not in this diagram like the total difficulty of both sites and then for block relay is kind of similar to Bitcoin I guess so we propagate blocks as long as they have or even before executing them the full block is sent to the square root of all the current peers and then for the rest of the peers there's a notification that there is a new block but most of the time the peer will have received at law firms from some of its peers and in case they haven't actually received the day what kind of just fetch it a couple of seconds data in the end to sort of tap in this relay there's a fourth voyage peer there's like a set of recent blocks that have have been seen and then for transactions is it's kinda similar so we also have to set and then when they in the beginning after connection we just sort of start dribbling all the transactions that we know about and from both sides and then yeah usually if it if everything works out we know yeah after this process both sides are gonna know about the same transaction right and then the final function of this protocol is the watch in sync and this is kind of where he comes in and we can explain all of our words we've got your question so I guess most of you are kind of know how in theory out how the whole synchronization works so the point that I would like to make through the last few slides is not really how the synchronization algorithm is works or how how we clearing benefits from it rather all the rather different challenges that we met while implementing these synchronization mechanisms from a networking perspective and the reason why we would like to talk about a bit about these is because in the last couple of days we've seen a few slides a few sentences wrapped here and there that yeah we'll just swap out this guy with network and everything is done and those can those are a bit strong sentences so not working wise there are a few quite important networking fallacies that people need to be aware and although most people here I guess are designing consensus engines and shrugging and whatnot but I think it's really really important not to lose sight of the networking issues otherwise we will end up with with a few gotchas that current synchronization really really suffers from which are designed got just upstream sorry higher-level protocols or high-level data structures so essentially a Goan theorem itself has three synchronization modes Hosting fasting and light sync from the highest level of perspective of hosting basically just fetches over the box from the network and runs all the transactions with them after it traveled transactions we are in sync now fast the problem with this method of course is that this works fine when the chain is small but eventually transaction execution really really floats up and then you everything just slows down so as far as idle gap it was the one who designed the fasting protocol and the benefit of was that instead of running all the transactions you could just download everything and including the state so download blocks down the receive download the state since the headers already have all the work of proofs for the state as long as you trust that the header chain is valid and you trust that because you have a huge group of work on it you can trust that the state will be valid too especially if you can mine a few blocks on top or a few classified we're adding in my on top so in theory it's a really really elegant way to replace computation with bandwidth and of course last this worked fine for quite long time in theory but obviously if you want to run on a mobile phone or small devices and sadly even consumer laptops start to count as small devices nowadays fasting even fasting is getting really really slow because simply the changes about 60 gigs in prune state so it's just too large and that's why we reintroduced the light sink where it's kind of similar to passing in that you download the entire heather chain but everything else is downloaded on demand so it's kind of like an on-demand fasting souped-up version so in theory this is how it works on high level and everybody everybody kind of expects that this works without too many challenges now I would like to put you wrong hopefully this do take care that you design the next charting stuff that we need to then put on the peer-to-peer network so well first I started posting so in theory first thing is get the blocks I mean want them though what could go wrong and well my first question is let's suppose that we have 25 beers connected to that I download the blocks from in theory I can have five different ears I can have peers that frosty and completely useless to me I can't have peers that have stale data maybe they have only half the blockchain which data might be higher than mine or might be lower than mine so it's again good question I have a few lucky pairs which are in sync with the network those are really valuable ones then I have I can have actually I will have quite a few peers that are on the fourth chain for example if you're in classic or there are a few other clones of material it's really funny when you see something some we are all going connected to you and of course one really really important class of knows that we must not forget about it are malicious nodes these are probably not frivolously present in the network the point is to make sure that they don't get present in the network so we kind of must ensure that it's not really possible to abuse it so if we have five of these types of nodes and we have absolute no idea who is who then who do we sing with but the only thing that we know and that we are stars currently are their block height which is completely useless or their latest block cache which is completely useless or their total difficulty now the total difficulty in theory is a very useful thing because I know that the person who has the highest total difficulty that one has the best chain now there's a single catch I cannot verify it so essentially I can connect with anybody and tell them that I have a tank gazillion a chain of Temuco Cillian blocks and our large difficulty and they won't be able to tell me otherwise and I can play around with with time with block block numbers and total difficulty if you got the calculation so I can't fake it quite well to put to fake basically truth to try and trick somebody to think that I have a higher difficulty than I have so the best that we currently can do is actually pick the highest form and then go from there but we when we do this we must actually be aware that that might be an attacker okay so we know who we want to sing from ideally but let's suppose that I have 1 billion block now which block do I start to sing from the obviou the naive answer is 1 billion and 1 the practical problem is that there might have been a reorg since I synchronized last or since I turned off my machine this means that actually there might have been a short we work that's it just a couple of blocks I can't handle this quite easily I just asked the remote no not just for his head or less sorry not suppose that I'm I've got 1 million instead of continuing at block 1 million n1i below it down a few blocks and if there were not note says me blossom I already know I'm happy we know that I have the common ancestor we can sing from there now if I have a deeper reorg which usually doesn't happen on main that but it's important let's suppose 1,000 locks that doesn't happen user in the main that but if trying to go out of sync for duty understand this issue and then all of a sudden one client goes off and that's supposed echoey theorem is the one who is at fault now we fix the causality issue but we have to reorder handed blocks then all of a sudden that's a problem because that's a deeply organ and we have to find our that our common ancestor between the other chain which is on a completely different fork and for that we can actually do a binary search to find a common ancestor basically just trying from my could say from the Genesis block to our chain ahead and just keep request doing a binary search and extinguish remote the feathers until I found find a common one and that will be most probably our Nestor an interesting thing that even here if I'm trying to sing Christ from attacker they can actually say that their common ancestor is way below their - - through chain so we actually do extra checks so that if if somebody says that they're our common ancestor is block 1 million and it turns out that we have to share common blocks above them then we immediately assume that it's too much is more than disconnected however we still have a third option so if I have a reorg of 1,000 talks that's nice but if I have a real rocker for 1 million blocks that's a problem so probably nobody can meaningfully attack the network at current due for difficulty so you can't just go and mind a hundred blocks but I can actually mind a hundred blocks on the Genesis state because that's fairly cheap easy so it's a also really important to know that you not only have to take care that you have no rewards but you have to have a limit on how deep you are willing to go I think ok so not really sure Goethe know I think maybe allows one or two epochs deep reorg the idea behind was that was to handle a crazy enough Forks or consent issues but without actually exposing us to attacks okay so that's suppose I know which appear I want to synchronize with and I know what block I want to download stuff from then the question is that let's say I start pulling blocks it's fairly going ok and then all of a sudden I see a new peer that joined me which has a lot faster network do I switch to it over let's say I have a new we are connected which it has a much higher total difficulty advertise do i connect to it what do I switch thinking from my old here to my new here and the answer in both of these cases is a huge no because the problem is that it doesn't really matter what kind of heuristics I add to switch peers an attacker can always fake those metrics so I can start up and loading the local network that will be the fastest and I can advertise a huge total difficulty on it that will be the largest so my mind would have had no chance against such an attacker so the answer here is that since the networking untrusted every crap that can come over the network will come over eventually the network you need to ensure that you've treated three network traffic accordingly and the really important thing here to know is that the crappy beer is infinitely better than at an attacker so if by some chance we found a here that is really slow really shitty record on every aspect but it does something then it's that you have to have a huge reason why you would switch away from it of course this opens a few problems how do we actually maximize bandwidth so I can let's suppose I can I have like my fears some of them are slow here some other my fast years some of them just don't want to give me any data if I keep downloading data from a single appear that won't scale they will just be a bottleneck so what I could do is I take my original fear that I've decided to sync with I download all the headers from the original appear those can be verified to talk work have passions everything and after I have the header chain downloaded I can actually download everything else concurrently because I don't think ass can be verified and matched up with the headers of course that's that kind of seems perfect but then comes the interesting question I have a really slow here and a really fast one then the slow one will always block the fast now in if you think about Victor and a similar flow of course this is not a problem because we just downloaded 10 gigabyte file eventually the fast one will feel most of it as long as it a few chunks and that's fine but in our case we don't really download so we cannot download the entire thing at one go we kind of have to download that process and I'm kind of like a stream kind of like watching a movie so you when you watch a movie you don't really care whether you have the last 10 minutes if you haven't watched the first 10 minutes yet so it's always important to prioritize the first blocks or the first data chance and this would still not be the biggest problem the larger problem is that I don't have I have a limited cue to cue blocks off so if if I want to download that's my cue is 1,000 headers slope your blocks of 100 headers I can use a fast girl to fill up the rest but I still have to wait for the slope you to finish sending the data so that I can process that 1,000 locks and then it happens again and again and again and essentially what I'm doing is that every for every single queue size and just stop I just stopped or fitted my latest it will be but my slowest here or give or take and that's a data problem and for that what do it does is that it constantly tries to estimate the capacity and the bandwidth of each peer and whenever we download something concurrently from the network if we know that is a fast year we asked for more data if we know it's a slow period we ask for less data and the idea is that we want that stream to be constantly running and this kind of looks ok it performs quite well but it still has a bottleneck so now the the massive amounts of data are downloaded concurrently but the small amount of data the header chain is still downloaded from a single year and you might say that yes but the header is 500 bytes so downloading a single family header chain from a single appear is okay that's true unless that single peer is raspberry pi in which case you're screwed and that's one one class of problems if it's a Raspberry Pi the other class of problems if that single here which from which we try to download headers from isn't malicious one who just doesn't send us headers or send us very slowly then again we are pretty much screwed because we will never finish thinking and so the the way you're going to have solved this issue is that instead of downloading you can have weak header from the oh yeah an important thing to know is that it's really hard to download headers concurrently because you cannot verify them so it to verify whether a header you need its parent however what we can do is we assume that the master beer is trusted it's good now what happens if we assume that is fear that without headers from is good but instead of downloading every header from it we can't download Featherston what that means is that that's suppose I got 200 headers but between every header I placed 200 empty blocks basically gap of 200 and then I can actually back fill those gaps from the network concurrently I can ask anyone for those data and it does what the gap the way I feel them is that they slightly overlap with the with the actual skeleton and this way as long as my master fear is trusted and didn't send me junk everything that I basically all the other headers can be correctly verified and linked up to this and if you actually have the same data as the master here now of course there are the problems here is that okay the advantages first the advantage is of course that it doesn't matter how slow my master beer is because it just needs to give me 200 headers and I can convert that to 40,000 headers concurrently from the network so as long as I have a fast beer I don't care about whether a master Peter slow or not but of course the problem is what happens if if my recipe is a good beer I'm golden my master fears problem then it might feed me joke but the good thing is that if I cannot fill the gaps from the network and I cannot feel like it's not even with the master fear itself that means that I got drunk and I Melissa disconnect and the huge benefit of this is that even though I said that we never ever disconnect or never ever swap out our master here it's really important that it has two properties one of the properties is that if we fix an attacker we try to detect it immediately as soon as possible so it has very little time to screw with us and the second is if yes Africa anyway so the idea is to try to keep attackers away and stick with notes now this way it kind of seems like a really really operational synchronization Marcus Marcus does stuff really fast but there are still one class of problems which turns up eventually or may not and the problem is that there are lots of those in the network we should have crappy connection and I myself might have crappy connection so for example if I rented the latest newest and greatest the virtual machine from Amazon with a huge pipe a huge processing capacity and everything and I started up what happens currently on Manor is that 25 Raspberry Pi calibre computers will immediately leech onto me simply because they are looking for peers and even if some of them are racing or some of them are halfway in sync it's a huge problem because they simply cannot saturate my capacity and and this is this is quite a significant issue because then I the reason I cannot sink is not because I have problems capacity problems because my peers have capacity problems so originally going through solve this with time outs I think the original implantation has been a shame that frontier had a three second timer it worked beautifully except when we got a bug before that going to doesn't sink and we try to figure out why he couldn't and in the end it turned out that the guy reporting the bug was in a remote village in New Zealand and had an agency of three seconds satellite agency so a three-second latency coupled with a 30-second timeout is of the best choice so when the obvious answer is just raise the time out but if I raise the time out that all of a sudden I just reintroduce all my other parts that that I simply cannot throw off slow peers fast enough and I just get stuck and so what we actually came up with our solution was that instead of timeouts which are hard to to sort we try to try to calculate an expected for our chip time that we expect from our peers originally this expected round-trip time started three seconds the original timeout so if I have a really fat machine with a fat pipe and I found find fast beer I enforced a three second timeout anyone about that forget job however if I don't if I cannot think if I don't get better than this the time up miss expected relative time slowly goes up and eventually I will find some peers that that can satisfy those constraints and that suppose that I this expected time goes up to one minute the moment is still quite huge but let's suppose I have shitty peers and I don't have a sudden I get a really good beer now since I got the a really good beer it doesn't really make sense to keep the old shitty ones around because that new beer alone can satisfy my entire download requirements so at that point this expected round-trip time starts going down and eventually only the really fast beers can satisfy the constraints and everything else gets robbed off now this might seem a bit like playing nasty with raspberry PI's or so machines but it's important to emphasize that we only do this you're in sync so one we initially synchronize a node then honestly we don't give a damn about raspberry PI's we want to sync up fast then after we're done saying that we can talk about helping others or at least having really really slow yes and more or less that is the entire fasting challenges that we we solved and we're kind of quite happy that this this protocol manages to saturate quite quite a nice bandwidth but of course hosting is always limited by transaction processing so this is where and this is essentially we're fasting comes in like instead of processing their transactions fasting kind of just downloads all the transactions or some download everything essentially now if we just implement fasting as is try to run it we'll still see that fasting cannot saturate our download link and well one of the answers is if I'd wanted to run the fasting on my old laptop about ten-year-old laptop importing downloading and importing the first three thousand blocks takes I don't know maybe three seconds and verifying the ET hash on top of them takes five seconds so essentially for fasting the first bottleneck was ET hash verification and so the trick that we actually did is that it doesn't really make so if if I'm processing blocks one by one at run all the transactions and it's actually important to verify that before work now if I have to download five million blocks then does it make sense to verify the code for every single one of them and the answer is no so if I gratify them at random so maybe verify every one out of two thousand I just gave a number I don't really remember the exact numbers if I perfect by only one out of a few thousand or one out of a few hundred now I can save a huge processing time on the bottom for clarification and but I still have the same security so first thing essentially just sparsely verifies the proof of course of the chain and as you reach the chain head the last blocks there it actually verifies everything so if you actually managed to verify the proof of work of the main that the current had lost and probably everything below it is fine and this actually really really really significantly speeds up fast synchronization so with this optimization I can actually saturate my home link which is which is nice of course we still have a problem ETH comes back anyway if I want to run on the lower power devices present on mobile phone then generating the et hashcash is again significantly slower than the network bandwidth so on my phone which is a fairly new phone it takes three and a half minutes to generate the Nita hash cache so that's the small one that's the 30 megabyte one after that's horrible and essentially what happens is normally if you just start downloading blocks and verify a charity cache cache when you need it then you will just have these three minute gaps in synchronization and this is you don't really see this on a laptop or on a big machine but it's a mobile devices it's really important that ETS traffic cash generation is actually run concurrently with synchronization with downloading stuff and even more so for us to really have to actually memory have everything so that when you restart your nose then they don't have to again regenerate these caches so these were two of their really optimizations that that are really weird that they have nothing to do with synchronization but they were the ones that we actually limited synchronization okay and then count the nasty challenges so would stay with fasting we have two interesting properties one of them is that the size of the state is significantly less than the size of the chain I don't have exact numbers but approximately that chain meaning blocks receipts are 56 eco bytes and the state is maybe 4 gigabytes just approximate numbers so in theory it should be really really fast to download the blockchain sorry the state and slower to download the chain itself the issue is that the number of state entries is significantly insane with larger than the number of chain entries the launching has 5 million odd components of the same and the state try has 105 million give or take and what this ends up with is that actually downloading 4 gigabytes is significantly longer than downloading 56 petabytes that's again something that really hits you and something that you wouldn't expect and the entire if you if you wonder what the issue is the issue is latency so I cannot download too much data in a single request and if I have to make a million requests and each one take 50 milliseconds then this is just crazy and so one thing that we actually tried to do here is a really a very old the trick is that whenever we fast interested on all this stuff and one of the fasiq is actually downloading the chain it is concurrently already downloading to the state and the only way to download the state is fetch the header the head header from the peer that we're setting with pray that it's actually not an attacker and start synchronizing the state try from it and if it turns out that it's an attacker then it should turn out work fairly quickly so I don't yeah I wasted some bad I discard that and if if the turns out that it was a good note that I'll go down and by the time I actually downloaded all the blocks I should have in theory most of the state available yeah that was the theory that lasted up until about a year ago it still holds for test nuts for me that the latencies and the whole hat at the dropship type is so much outweighs everything that for example on my machine and my home internet connection it takes about one hour to download the entire chain and ten hours to add the four gigabytes on top so that's that's something to think about okay then of course there are some other challenges for example that as far as I know so apparently I think always have state pruning we also introduced a pruning a few releases ago which means that the expected availability data availability of us the state it's at the state tonight 16 minutes so it doesn't matter what I what I start to download after sixty minutes it's not it's not available anymore well at least only the common stuff is available and it this one actually requires quite a few networking tricks to keep keep downloading the headers concurrently and keep jumping forward if the state sing not too essential to keep keep up with and I had the dynamically changing data and okay but what's so in theory fasting is kind of a nice logarithm as early as I can design but what's the we keep problems with but the first problem is that each block content may not kind of deletes a thousand channels and introduces about two thousand new trials which means that's about two hundred triangle modifications per second it's an insanely dynamically changing data the state try and horrible aspect of it that the state trying essentially mapping hashes to hashes so the key of the hash devotees a hash the contents are just one hash or multiple hash so it's data content while its revenue uniformly randomly distributed this means that it brought trashes my local databases if I download the state try I have to insert it all over the place it has a huge huge overhead on my local database it also has a huge overhead on the remote sites database because even looking up the data is the base just jumps over all over the place and this is a typical place where in theory the networking aspects should have been trivial and the higher level data structure does come just completely screwed it and still doesn't okay then I already have tried out that maintenance has 105 million State entries which means if I mean we need to download that in any current with any constant multiplier maybe which I can download 100 at a time one person at a time it doesn't matter it's still something that the latency itself is what what so fasting isn't slow because you cannot handle dependent it's so because you can't handle but you can see of course if your user experience is really really nasty that the state price sizes cannot be calculated on us so we do not know how many triangles I need to download so I just keep downloading downloading and the user that they have been downloading for two hours now and have no idea what's coming up and I still can't have them because I myself know a lot right now so what are kind of takeaways from from this fasting issues with fasting the Tito is that you all should make the network networking fallacies but one of them is that even network programmers kind of assume that latency is more that latency doesn't matter and as you can see here actually latency is what breaks it so that's the single thing that breaks fasting so whenever you design a cassavas protocol and assume that you can get something fast across you can't the other ultimately important thing to keep in mind or the other network for fallacy is that your network is homogeneous for example that's also a huge problem because you have noticed all kinds in your network some will be performant most of them will be horrible and some will just try to screw things up and if you don't if you cannot handle the presence of all of those nodes then everything can go wrong especially if you are the crappy one for example if you are playing or if you're on a three second agency satellite connection another important thing is ok is that again that problem many people have seen that network if connectivity is a given and it's kind of really really important to be able to gracefully handle issues so that if something fails it's important to be able to continue where you left off and this might sound like an obvious thing to do but it's not always the case now I just I haven't written a slide about this but I got to just imagine that parrot is working protocol which actually has a few really really nice properties which they actually did they think about these issues one of them is was that so low latency as I said the biggest issue that with the track sink was that it just takes too many Network requests go back and forth back and forth to download all the data now apparently did is that they prepared a snapshot of the state the entire state and then peers can actually instead of downloading it one by one trying out they just downloaded it in big lots of data and reconstruct it on the remote side now they're really good benefited but there are actually two benefits one of them is that network packet wise it's about the I can say a number but I would say maybe you can get it in a thousand network packets it's just a random number whereas fasting requires a million so it's a few orders of magnitude smaller so you completely cut it off in league dancing and the other really important aspect be a nice property of the warps in protocol is that it reduces the load on the serving node so if I if I have the fasting and if I have to look up three Tigers than 300 disk accesses and I get almost no data out of it whereas with working protocol if I if the remote node if I ask for a chunk is I know 2 megabytes that's the thing to look up and I get an insane amount of data out of it so from this perspective the working protocol is a huge improvement but it also has two downside it's always about trade-offs I guess one of the data science that I do not know how much it is impacting parity notes or not so maybe they can answer it is that fasting can handle dynamically changing data since I've downloaded tiny portions of the data I can swap over to new tries and just download the missing ones whereas the warps in protocol cannot update beta fast so to create a new snapshot it probably takes quite a while which has two consequences one of the consequences is that it's the lobe that knows to it in the background so you do have to prepare it that's probably the least of the problem I think I'm not sure the other problem is that if I prepare snapshot in advance and I keep it for an epoch that means that when no thinks it can get that snapshot really fast but then I have to reiax acute every transaction on top of it which depending on the node computing power might be fast or slow so that's that was one of the issues that I have with with with this protocol the other issue is that fasting is proven to be correct so to say in that I have I start out from the root - of the blog that I want to sync and every data that I download I can immediately verify whether that's good data or bad data whereas in fact that what to download a state snapshot that is 4 gigabytes in size and that contains a lot of stuff that is not directly tied to the route - then make okay this is I will admit that I have limited knowledge about the protocol so feel free to correct me but as far as I know the checksums and hashes of the snapshots are handled separately from the from the root hash of the try itself which means actually need to trust the hash and yeah so the question is when you are now any chance with warp sing can you verify that they are surely part of the state drive without downloading the entire snapshot I think it's a record of it is it not now yes so yeah you have to download all the state chunks and reconstruct the entire entire tree it's actually because the snapshots are basically just the leaves of the tree and the code with some specific kinds of compression applied late yeah so essentially yes yes so as long as I think the same thing kind of holds that if you if you got a rule - so to say for the snapshots that is trusted from somehow or you managed to get be lucky and find a good one that is a Nasim protocol but it's a bit vulnerable from this perspective now I will be baby from user experience point of view it's a good trade-off so that that's completely fair but it's still from security yes backers so one major kind of failure that I personally have seen both in Jeff and already fascinating is where you basically start the process and then halfway through it stops for whatever reason you have to be very from scratch it doesn't start from scratch just if you don't remember how many nodes you download it so that's a typical user interface calculation it was fixed in the last week's Oh but so generally I think how many attackers you have in the network is a bit so it's kind of like vaccines and birthing unity so if most of the nodes can handle most of the attacks and it's not really worth while to come up with a really really hard and expensive and hard to pull off attack whereas if you can find some vulnerability that Muslims are susceptible to then it kind of makes sense to try and push that hard so I think this is such a thing that now I don't think we have many attacks but it's nice to be able to somehow prevent at least try to prevent most of them now for example it's again addressed in mind so from this perspective I think fasting and working I both suboptimal and I would vote for something in between but I won't go into that maybe I can do some benchmarks and proposals back later on so actually there was an impact on fasting pretty recently it was kind of funny it was like one note it was trying to snipe I thought I like frosting he knows by standing and I think what was like oh yeah yeah there was a really awesome thing in that in may not about half a year ago so the the way fasting work in Goa theorem was that he started to synchronize and after you actually managed to fast sink for the first time the fasting was disabled because obviously is not safe to custody fasting it's better to let us go closing from that one point onward and the code that checked whether fasting was complete or not was whether he had a full lock in your database available now so if you had a full block apart from the Genesis block then you can see that fasting done and he had to import everything and there was a point in time but half a year ago one on the main network when you connected to the main network you almost immediately got block number one on a different before and that's it and the point was that fasting although most of the time it's stable as I said network connectivity sucks in general so it's almost it's hugely unprovable that you can start fasting and finish it without any connectivity issues so your master Pierre will eventually drop off now if your rest if we have dropped off and you got block number one from random Joe in the network then you just started to slow sing from block number two and that was a really nasty thing we're not really sure whether this was malicious or sort of other things were deliberate or accidental because in theory if somebody minds at law starts mining on top of the Genesis block then because they are not instinct we could assume that yeah he was just accidentally doing that but since this was happening for days on end and we always got different block number ones on different Forks somebody was actually mining block number one they were on I mean on top of that any anyway so there are attempts in and even present that does not drink because that were constantly sees various totally not that working layer attacks but various other attacks so people are trying to school stuff so let's see what the ideal protocol might be to be some kind of snapshot of like basically some subset of all accounts that starts not very nice so actually I have one of those implemented so still I think that one ideal protocol that I was thinking about actually is to keep up to date kind of like up to date list of of the accounts without maintaining the market rules just let's suppose the latest just the latest account list the advantage of that would be it's fairly cheap to maintain you still have to square out with Rios that's problem but the contest itself is feel fairly cheap to maintain so if I can maintain actually done it means I can iterate that and send that over to the remote site cheaply but what parity what would be different from where this works in protocol would be that instead of creating a snapshot in advance and hashing it and proving crevices in it in different way I could suggest the similar way standard chart of accounts contiguous accounts and then actually prove it while work of routes from the two directions that the accounts are actually part of the original states right now the challenge here is that I cannot since the triceps dynamically changing all the time and every 60 mins I guess tries us essentially I won't have the time to download the entire tribe in a one belonging to a single state try so this is an issue but it's solvable by actually just downloading these chunks that suppose I got one first 1 million account the second one the second video etc in chunks these may belong to different tribes but these should get me most of the data and if I run a fast account about that then that is an interesting question whether it could perform decent dish or not so this is something that the reason I didn't want to go into details because it's something that needs to be actually measured first because otherwise it's pointless yes if you maintain that data structure that you can also finally do the optimization where like if you have any kind of everything off code you just like visa that directly instead of what Indian Territory yeah so that's definitely would be doable there are a few interesting properties that we need that so a one one interesting aspect that it may or may not help is I don't think most contracts and if you may not read so much so if you access an account then you most probably will update it and if you update it then you need to rethink I'll try anyway yeah I guess like for me I'm not as concerned about average piece behavior I'm considering like especially now the transaction fees are about way back down the cell does and so what I was trying to do is that okay so so if somebody does a reading denial-of-service that this might have yes just average case I don't think it well but for for denying service it might actually be a really good time I think we have okay alright so I mean this is about shorting right so what chansik so yeah for the charting so I mean nobody really know was so at least I don't know so what I'm thinking about that after those of those two days is just yeah so photo sharing network because you know a lot of things are still unclear so we can't really come up with a protocol now I guess but even if so if some of these ideas actually get implemented in the end and where we have the role separation and things like that and I will likely require like a more complicated network topology you know different protocols for different things so I guess like the main things that you probably need to worry about is that number one you can't have all the data get broadcast and ever more you probably want like some notion of subnetwork for every year for every shard but then the other thing you need is to like elevators get or Co meters get reassigned between charts of what we so need to be able to miss like basically plop into whatever stops in yes for example one one interesting aspect that we're thinking about discussing yesterday was that basically everybody can become a proposer we will probably have a ton of them so those are pretty safe people everybody can be an executor we'll probably have a ton of them those who believe just mining will be pretty safe people but the collectors itself those might be an endangered species since they have huge collaterals deposited as far as I understand so I actually think that like the 1000s was probably I got really too high like in practice I expect maybe 10 100 ok last week 100 still I wouldn't like to get that getting slashed the point I wanted to make is that although I in theory I could have infinite proposers and infinite executors the collectors will be finite and have a lot to lose and the problem from the networking perspective it means that it might be good to think about whether you want the creator's to be public or not for example interesting trick is that if you can if you can actually instead of having a network for collectors or having the collectors your public instead of all the collectors would just mess themselves as proposers and executors both then they have access to both networks they can just behave as both of them and result will be that from the outside and here they are much harder to find much harder to the dose we personally think after those two days that that committed us gonna be a problem so that's why yeah so that's this one is one of the things yeah in general like like the design of I think the design of the PDP protocol for Starling doesn't actually start you just might know like I mean maybe there's ideas but nobody's really started so it's kind of quarter to keep mind that yeah like for for someone like me these questions are the other questions and then yeah like we so then in the end what what the answers to those questions are going to mean is that we will likely have to come up with like better mechanisms to achieve the topology that we want you have to come up with that the right topology in the first place we can we can make almost anything work it's just more questions like what what what do we you want things and I think that's like marketing yeah so yesterday we had a really good conversation on the way to dinner after the discussion and I've talked to a bunch of people about this but it's probably good to broadcast this everybody so the state size versus the chain data size is three gigs versus 57 gigs for the chain data so walking rent is focusing on those like three gigs whereas like the chain state itself is the majority of like usability problem sorry when I say usability problem I mean the footprint on my hard drive is giant so like personally like I don't think I should own take that each player is throwing 90 percent of the history so what we talked through a decent amount of this yesterday and it looked really feasible to do Network level sharding for that information and it's like a really easy opt-in thing so that's likely something that we're going to be looking into in the near term t-shirt doesn't require any protocol changes doesn't like that so be very reasonable to drop that hard drive footprint to like so ten gigs yeah but like with upcoming consensus changes we're going to move to a like a weak subject to any model anyway and then you can just throw everything before the recent client checkpoint fare these you may need it if you have some DAP where you care about like seeing what stock trades you made five years ago but then like it makes to just have secondly or more office for that yeah and the other thing I wanted to toss onto there was that with the idea of self authenticating vlogs of stateless clients and again that stuff not actually meeting a protocol change for us to execute those kind of things to fight global change because the circle's rating consensus protocol change or anything like that it and this is a like naive assertion because I have only been thinking about this for the last day or something but it seems really feasible for us to implement a new sink mode that it's similar weirdly in the middle of all of them we're assuming we can get our way out where we can get witness data then we can do a version of like ultra fast warp sink whatever you want to call it where it operates using benevolent witnesses generated by other people to stay up with the head of that fills everything yeah so actually I have two comments for your you fibre the problem is still that so for witness data the question is how large will that data be because if it's so it's a there was somewhere yesterday or today outside that we would replace this guy with bandwidth now this kind of may suck but it'll still be faster than that most of the time at least so in uh it's if we can it's a fine line to threat especially if for if you start talking about that's a so it can be a denial service attack vector let's see if I figure out a way to make the witness date 100 megabytes then good luck so that will complete this community pop things so there are specific contexts where you have what Watson wants a fan would but storage is expensive so I can be BSS particularly would probably find look we might even find a hundred it's very hard you mean oh yeah okay yeah yeah so I can imagine that there are instances but my issue is that so it fell if it would be something often but each note we decide whether they want to do or forward the test data or not that may work but that's but the problem is that witness data disappears at the first note that refuses to forward it which kind of means that unless we have a really strong backbone that keeps forwarding witness data it might matter actually yeah it basically it can happen that you'll have a small minor network that keeps forwarding witness victim between themselves and the main network might not benefit so much anyway and to react to the pruning the database I'm not sure whether you went there deleting all blocks and everything receipts or not okay so uh so everything before the the header checkpoint has just deleting his block headers receipts block bodies I mean the archival notes store that could be somewhat charted okay like you store these 2048 in these 2048 is 2048 based on your your node ID or something like that but so one problem that might arise that is with deaf developers in general that I've seen that a few dabs out there for example Akasha which rely heavily on logs - so basically they figured out that it's cheaper to debit a lot them to save a storage entry in the contract database for the contract storage and this means that the only way Akasha can look up opposed to user main is actually to filter the chain which is insanely insanely expensive locally plus schools with this spooning approach but it's cheap for them but like my thought about this is rather that we can do these kinds of things but we also need to be aware of who we're breaking people to use logs of all the time so you know nobody using it in its like a bad thing so I don't know what else like maybe I mean you had commerce and all the other talks maybe had some [Music] when anyone else more questions and any other experience poke about this I find yesterday but I think is important is to the other chari implementation teams so can you speak more about your thoughts on like extending that p2p or starting something completely from scratch such as working on Peter please oh I see yes so that thing so we didn't include it in this thing because I feel it's more of a low-level concern so like how this stuff gets transported is that relevant but it's still yeah so we with wind changes to that PDP because yeah just like mini old we know you know do you think it's extensible they're definitely trying out to the p2p stuff I mean 30 is working on an implementation of the PDP we have a prototype of whisper liquidity so we're trying to make you work and see how good it is on the discovery side yeah I have like kind of I presented at Def Con that you have some ideas like how to evoke the wulfhere discovery thing so we can make some things much easier yeah so like this this will be five things so yeah we still working on that just recently through those Eclipse attack thing we kind of realized that we need that maybe more sort of discovery mechanisms and just as the HTC so we're working on like sort of an alternative and that's going to be a lot used like super useful also for girls that just can't run a DHD which is kind of a common kind of case also and yes so these things but I feel like they're not as relevant now because it's more like a low-level like call the whole thing works and then yeah you know like these these are just facilities that you can use and I think I always think that yeah with the current network we kind of have almost all the facilities so we need to make the current ethan network okay and then yeah like we don't need anything much fancier but yeah I like Forge well the Charlotte network is like a real different storage it's just like way more things then we might eat and we just for me it's just always like we should just wait and see what we need and then we can make those things happen for example from a shopping perspective one kind of important question that needs to be answered is for example if we do we want to have a single overlay network single communication network for all the Sharks or do a separate ones for individual sharks the into the reason why this is an interesting question it's because if you look at the main network versus best network now if you want to join Robster good luck finding peers and the reason is that in etherium world the discover Pro doesn't differentiate between the peers from mania test net or all the other Forks you just find it to connect to it and then hope that it's something that you want and so it works okay for me in that because most nodes are not main that but if you want to find somebody not on main that then that's a huge issue now if if we introduced 10 shards and each shots will have 10 peers on it then maybe you can wait it out until you find peers but if you're a collector and you want to jump fast between them then your can because it's supposed to fix this issue that Seoul will at least be able to make find the other guys were also interested in that chart so that there's one thing but then the other question is yeah how how how should they be connected but like be the most appropriate oh yeah for for the Charlotte specifically is the most important part that color try the other guys of that chart and then yeah that's also on this variable so good so of this issue where yeah like if you want to find some rocks yes so yeah that's one thing yeah I'm a football structure there was other things that we've been looking that we keep looking for excuses to somehow like turn this into a structured network but we just don't know if it does it make sense so it can make sense for something so many people ask me this question like why don't you use like a more optimal sort of I don't know a multicast thing for for example so it's just always this question comes up a lot so there are structures to the Prado a structure that works is specifically that they can they are deterministic and they can be attacked yeah there is an outside to it but then the question is developing that's that to everything that means it's just you know there's just how these things are but yeah like more specifically about the shrining so maybe like one of my questions is like more more specifically like in some cases what I've seen there are two days is that in certain cases you know sort of communication between two specific nodes on the network so if you have like a proposal and you want to give back so you would give feedback about that or something like that then you actually need to talk to the originator of the proposal and in that case you know the situation that we don't have right now where you want to talk to a specific node and then in those cases it can be yeah it can be useful to think about like maybe we should yeah introduce Robbie or something like that yeah for for messages like that but it's just yeah it's just I'm here to me like it will be super nice if we could maintain the stable enough now where yeah your exact period doesn't matter so much because it makes things a lot easier and there's a lot more robust but yeah if we have to have something like that then you know we have to find it we have to find any more questions this is a bit more general but we have Collier's switching charts all the time right so it seems like they would have to switch which piers that connected to all the time just to deal with that yeah have you considered just having the colleges like security property is yes in the other thing yes that that like exactly help greatly we call them exactly help reap what we call leaders need shards also depends on the number of commuters so for example if you have like 100,000 colonies and the metric thousand corners are between and 100 shards then that would basically be like a new call either but on average be selected one every thousand areas which is what it really means once a day so you know it would be like that would only be an issue against the number of call eaters is fairly small so the other thing to consider about the school that quality of switching networks so for especially for the quality overall we can have pretty high standards when it comes to the network connectivity so they've gotta have like a decent thing they gotta support that you need be properly in things like that and probably same for the proposals as well we just have to have like a good you just basically can't run there that doesn't mean people vote well we could just prevent them from doing that by making the protocol hard enough I don't know there can be a lot of ways to do that but what I'm trying to say is that we can have pretty high requirements for these clients because in the end with especially with with the shoddy Network it turns out that because this protocol is supposed to be so flexible so you have this whole scale between very light clients and slightly less light clients in the end I mean most users are gonna run some for my client and for those people the the issues that came up earlier in the circuit they they they do kind of matter because they're these people have finished in connection before for the for the goods that actually run the network that make proposals that collate these proposals yeah you kind of have to yeah that they can have we can have much higher status and it also means that we can come up with a totally custom protocol to just optimize this for super fast switching for example like this not not totally off the question I still have the question that for example if you have you assume that you have to profess networks but what happens if you have to cross the firewall between China and the world so what do you just say that so you can't just say that okay we'll leave China out of ethereal bathroom because you have slow internet access so yeah that's like a whole different kind of worms for performance yeah but you shouldn't pick no er cases like this yeah yeah so we've had some respect they have to have discussions about this in the past so there's only so much we can do to work against you know let's call the internet restrictions right so there are some things we can do like we in there for the wallet this car will be 5 discussion huge topic that is it really resolved a testable question of the protocol obfuscation so how much obfuscation do you want you wanna frisk Asian and all that you want it for more encryption for that protocol is kind of hard to achieve that in the next corner but also a lot so maybe we don't want that but then in the end I quality things they always this is like a game you can never win because you can you can win it with like really strong encryption but then your protocol is like to secure so there's yeah we just have to find that good way to handle that I guess it's specifically for the issue of like qualia and China block by firewall I guess we should just keep by the way do we have any kind of like analytics that spell how well like my ears that noteworthy which I know we're like anywhere for that matter no well we know that's not true so we have some it you know we have some issues on the tracker where people said you know I'm in China so maybe that's the source of my problems but that's not really measurable yeah it's not really measurable so we generally the wulfing is pretty much a Majorana quick we tried before there's a lot of research about different that was network topologies one of the popular was today's dragonfly topology which you have some kind of path tree and the leaves are highly interconnected Network now these are destructive networks as peter denied it but there is also a lot of research on how to randomize these in these systems and I think there are some idea that could be extracted from from this literature and maybe maybe it's worth 2 grand yeah cool you can just send it to me or put it in the kitchen Victor said oh yeah there's a deaf p2p get a channel you should talk to each other gender history but [Music] Oh can everyone like many slightly crazy idea but finding fast it will be possible to financially incentivize oh yes that's actually totally isn't so they some people have already do this and you know there is definitely also a market for that and I think you will become even more caught in the future did you know he's doing that so I kind of saw it yesterday because someone opened like a pull request I going through to add some feature around here management and I was checking there it up and they have like a sort of like a dab where you can buy I think what's it like light client-server past you something that look really cool at the what sorry I can't just yell also with I don't really know like who who this person is or something I just thought it's cool and like people actually doing this like we do things we want that stuff yeah sure in four hours [Music] I guess thank you