Build Web2 and Web3 Infrastructure Using Xnode in Minutes Instead of Weeks
ETHGlobal·Wed, Oct 9, 2024, 12:00 AM
Join Ashton Hettiarachi and J Lynn of Openmesh for a workshop titled, "Build Web2 and Web3 Infrastructure Using Xnode in Minutes Instead of Weeks." Learn how to quickly build Web2 and Web3 infrastructure such as validator nodes, data clouds, and P2P infrastructure, data APIs and analytics engine in minutes. This workshop took place at ETHGlobal Sydney 2024, the first major Ethereum hackathon in Oceania, hosted by ETHGlobal, May 3-5 at The Venue Alexandria. ----------------- Follow us: https://x.com/ethglobal Check us out: https://ethglobal.com Email us at: hello@ethglobal.com
Transcript
[Music] so today we'll be talking about xn and open mesh so what is open mesh open mesh is a decentralized cloud and Oracle Network so we provide immutable data peer-to-peer compute and storage we also have an element for big data and analytics so the best way to think about open MH we provide we bring the best of AWS chain link and ipfs into one design in one solution so the best way I I look at open MH is we bring the traditional infrastructure like traditional Cloud infrastructure into web3 our goal is to accelerate web3 so we have very small team so we started four years ago so most of our team based out of Sydney so the question question we asked why cryptographically guaranteed data and why decentralized infrastructure so if you think about this chart the technology is no longer double every two years in our space in web 3 technology doubles sometimes every month so but what's the problem uh the problem personally if you think about the world it it and infrastructure is controlled by three to four companies and we supposed to trust them um if you think about a couple of years ago these very same companies they control pretty much a lot of data and they had the capabilities to do like a read and write capabilities to fabric of society using social media uh all the apps they provide now the the the what concerned me the most the very same companies they also happen to have the the Maj majority of the AI capabilities and they all in a race to compete each other to show who's bigger and better it's it's no longer Innovation it's it's about politics sometimes it's about Wall Street narrative if you think about this if you if you project this into another maybe 50 or 100 years right the question we have to ask ourselves if you move from type zero civilization into type one civilization the same companies will have like Mega corporations controlling the entire solar system right the data infrastructure will be controlled by these corporations and and I have a big problem with that so that's why I drawn into decentralized uh philosophy I got into web 3 in back in 20 20 2015 I used to run uh Meetup ethereum Meetup and Bitcoin Meetup here in Sydney and wongo so if you look at the design principle we choose uh so we we got to have a good design because if you don't get the design right at the beginning it's going to be a not it's not going to be a good thing so I don't have much time since we have only 10 minutes to cover the the the the core infrastructure but I'll keep it very high level because I wanted to show you some demos for our hackathon so you guys can take a look at our technology um so I'll talk about xn so just to be clear xn is a one product we've got about 14 uh products that we developed all open source um all designed for developers so we've got apis for developers to power applications we have infrastructure as a service which is X node um and then we also have analytics and Big Data specifically for web 3 space now what is X node so X node basically aggregate all the physical infrastructure available infrastructure um from bare metal providers uh from cloud infrastructure providers and also independent um resource providers these resources are um GPU storage and compute so we bring that to and we deliver that to the developers so developers can quickly design deploy and manage infrastructure very very quickly the the difference between going through AWS and GC P compared to this approach you have a full control of of your infrastructure ultimately you own the the infrastructure um I'm going to do a quick demo of this x nood um platform again this is still in beta so there will be a lot of bugs so if you think about like infrastructure development uh space we took a lot of inspiration um so this is the platform I can't see the screen so you have a website um where it's similar like a Shopify for infrastructure you've got a bunch of templates so if you want to run a web3 node simply you pick a no pick a template um so these are all modules so you have a web two and web 3 infrastructure in this uh in this page and then on the left hand side you've got lots of tools you can access so from infrastructure to data to if you want to if you want to connect with third party applications all in one place so let's say you want to run a phantom node you pick a template um and then it gives you information about what what that package look like um so now when you select a provider so it's similar like Skyscanner or Google flights like if you say I want to fly from Sydney to New York find me Air Airline so this is what happens so what it does is basically aggregate all the bare metal providers available resources for your specific infrastructure it just provides that like a list and then what you do is you simply select um so it's selecting a server for your Phantom node and you just hit deploy when you want to deploy it gives you um it gives you a kind of a rundown of your summary of your build and you can just go and build that directly into your bareal provider um either whether it's a bare metal provider or if you're using your own infrastructure you can you can do that here if you look at here this section if you if your server let's say if you use for a specific number of compute and specific number of storage and if you have leftover from your server you can rent that into a Marketplace since we don't have much time we we'll skip that part but the idea here is you can deploy infrastructure very very fast and you can save cost to give you an example if you were to run ethereum node on AWS you might be paying $600 a month so with this approach you can save up to like 50 40% so you you deploy it and then and then you connect with the provider within few few minutes you have your infrastructure up and running so this is a great Innovation for infrastructure um as a service uh use cases so we've got use cases for running RPC nodes you can build infrastructure um for web 2 infrastructure you can combine web two and web 3 infrastructures together so it's a really nice way to build things uh very very fast um and also only pay for bare metal providers uh bare metal services um so that way you can save time and money so I'm going to hand over to to Jay so he will talk about the little bit more behind the scene of xnot thank you everyone it's good to be here let me just get switched over okay there we go cool well welcome everyone um again as ason said um my name is Jay let me actually um I was going to come around the podium and say hello I brought this and well you have microphones here and I'm not miked up so I'll stay right here um yeah uh welcome uh you'll forgive me this is actually my first presentation um in large part uh especially and I'm glad it's here at eth Global uh here in Sydney um so uh recently how how long has it been Ashton three four months since I joined open yeah he's quiet now um so I've actually been in large part leading the engineering team there um in terms of the infrastructure today we're going to cover xnode OS there's a huge huge number of um systems and designs from core validator to the uh peer-to-peer network but we won't be covering that today we're actually going to do a pretty high level overview of xn building applications on it um obviously this early preview uh everything is subject to change now it's important to note that uh open mesh through and through open source um and so well what you're seeing here is an early preview and if you looked at open mesh's geub you won't find some of it it'll all be going up there up as we lead to Genesis and move forward so uh again just a basic in I can't see that uh and I'm not going to mess with the display so um all right well good to see that that is also not working because that's huge great isn't this always the way um there we go no that is not working great um it just works until it doesn't yeah sorry about that you know what we'll just continue and I will uh well like we can't what resolution is this I wish I could have duplicated the uh duplicated the venue so I could have tested this uh before I go to production um you know what we'll bring this over and we will rely on is that showing up all right well we'll go with this I hope okay so now we're back um all right so um where was I open mesch is open source um and I am over time um okay so there you are there I am there any developer is um Ashton mentioned studio console studio is the application that you saw Ashton go through templates with um this is a standard web application to be able to self-host it same thing with console console um obviously you have X node right which is an individual server bare metal or Cloud um console is responsible for deploying the X node image that is built uh we'll go over how we build those um here so um Studio there is a packaging repository which lives up on GitHub or wherever else you'd like generally the console would Point your X node to whichever packaging repository you would like to use in that case that points to all of the various use cases that we have on studio uh pythia various other applications um gu nodes Etc um and then obviously there's the settings repository which represents your individual X node these things will all be version controlled signed using standard get infrastructure structure um studio and console both make that easy to use for uh users who are maybe less experienced with the underlying technology but the whole system is uh well layered so that you can reach in and uh do what you have to do um getting back into xnode OS now obviously building your own Os from scratch is nonsense uh because open mesh is open source we like to also use open source and give back to the community therefore xnode OS is uh is based uh on uh NX OS as its primary source in the same way that auntu uses Debian for its source we use nyos we contribute back to it and in fact um if you'll note that in the hackathon this weekend we have uh two I believe uh packages uh to be built for the Nyx Os Os package system that will then be able to use on no um Okay so we've covered X node OS uh let's go ahead and cover the agent the agent is essentially um is essentially a python application that runs on every X node that goes and grabs uh configuration from the settings repository which can be hosted at any um any appropriate G repository location including ipfs with the get ipfs back end um and in fact the one of the things we're working toward is building uh NYX and xnos entirely from source and entirely from ipfs sources as well um so uh back to agent so agent will go ahead and pull both the settings and the package repository we'll pull them in and then merges them into its local repository so that everything is is auditable and you can see exactly what what going on if you're not familiar with nyos it's essentially a functional operating system and uh package management uh Suite of utilities so that uh in compare sorry when you pair it with Git um it allows you to have um essentially fully auditable all the way down to Source uh deployments which can be rolled back it also actually handles dependency management as well um so you can have multiple applications and inst on your X Noe at once each one of which uh brings along its own dependencies so for example you can have um uh just say particular application that uses a different version of lib C than another uh basically the only thing that's shared on the system is the kernel and if you're running an application inside of VM that doesn't even need to be the case um xn OS uh is essentially a layered set of opinions so initially what you basically have is just system Services which is is the uh the Nyx build Suite uh appropriately opinionated for for xote and web 3 uh like with ipfs support and various other things built in and also tuned to run uh core which we'll touch uh core is the um is essentially go Application uh which um maintains the lib PTP infrastructure public and private unified data apis um again I wish I had more time we wouldn't go into it how am I doing on time I've kind of lost track it's okay there's no one following you anyway so oh well that's good be plenty of time for questions I'm sure I haven't done a terribly great job of explaining a lot of apologies um so uh system Services we tal touched on core so um your application uh generally what we'll do uh in xnot OS is right now we're starting out with essentially a base system um nios uses system D internally uh and so when you deploy an application uh if you're deploying it to the bare system you'll be writing systemd units um but you'll be configuring those through the uh NYX Os or NYX configuration language which we'll we'll touch on in the next slide um well the next set of slides anyway so uh with all of this infrastructure around your application which actually manages like your storage and does Administration and cryptographic signing and validation of integrity and distributing your application over ipfs etc etc um we'll actually look and see how you would say build X noode and how we build X node internally um again um oh my goodness I'm sorry that is not going to zoom in uh so um where is that so this is actually one of the configuration files for um this is one of the configuration files for the iso image that we build for bare metal servers um actually now I can see it over here um uh in this stanza basically this is just NYX packaging language uh we utilize NYX OS generators if you're familiar with that if it's not it's sweet if you're not it's a suet of utilities and functions which allow um people to easily build uh system images for either install or running out of say memory or anything else um on say like digital lotion or AWS or you know any any bare metal or cloud provider major one [Music] gcp o sorry anyone else currently we're doing ISO we have a number of things if you look at uh the I if you remember Ashton you'll you'll see uh a number of logos there um all right so uh again uh this is essentially just the iso module the iso module actually like includes all of the configuration required for the utility Suite to build this ISO image we include um we include configuration uh which has documentation I'm not going to read out the whole thing for you uh but we disable the documentation you can see here we set the system greeting uh that needs to change and then the system label uh and so like doing configuration for all of these various system utilities is all in one file all in one case if we need to split it out obviously there's includes um but for things that change together they stay together instead of being split across a whole bunch of different files in the system or with some bespoke build Suite um yeah so again ISO image Force text mode make bootable that's that uh building the image uh you essentially just there's a make file or you can just do NYX build and there's that goes through there and then you get the iso image coming back here uh once that's built uh that gets deployed back out to your cloud provider uh we have the uh okay so from the console those images get deployed um the console essentially has a set of event Hooks and uses a state machine replication so when you use your console instance uh it essentially manages the life cycle of your servers uh it's stateless and it follows the kubernetes operator design philosophy where the the particular executable maintains no local state of its own it's just listening for events from various Cloud providers like say your vm's gone down you need to spin up another one various other things thank you yeah yeah I appreciate that um cool uh obviously going up here you'll need some way to get information from uh your application package into Studio which is through this uh again this package is repo studio will read in description files um I've gotone out of order all right so uh again if you're uh familiar with the studio which Ashton showed you it's essentially UI set of applications um to conf well it's a lot more than that but um in large part as it relates to X node the studio allows you to change and modify configurations of templates and packages that you have deployed into your X node um obviously to do this you need some sort of a schema to describe the UI that you want in studio um one of the things that uh we are experimenting with at this point and have started to integrate is um descriptions of uis using just simple Json generated from specific uh files in your in your package out of uh your your NYX package um and this is actually uh just an example for for GU uh and if I hadn't gone out of order I would have already introduced I would have introduced uh would have introduced this so um essentially uh this is this is uh the GU package from NX packages um you have essentially just a set of gu instances uh and that lists options so uh enable make enable option go ethereum Noe it's all quite uh self explanatory uh uh okay so configuring system Services now one of the things that your application's obviously going to have to to do is just going have to spin up system Services um your application server uh like say if you're deploying k3s to your X node um just use an example you would have to spin up uh the various k3s demons and run it um again uh this is um obviously from the the GU deployment um so uh you would essentially for uh each gu you just go through you set the system packages which includes um the GU name for the configuration uh and then for each uh gu instance sets the state directory sets the data directory sets it into system services and then um just basically creates ordering targets um and that's what you have to do to generate a new system Des service instead of writing a whole bunch of uh of unit files and in this case You' be configuring it with xnode console which would then also go into your system configuration as well we went over that so uh system service instantiation if you want to say uh set up um set up a GU node that does poia you just have Services ethereum G and sepolia and then you just set the uh config configuration options that we defined earlier um this actually quite straightforward so if you note that like this IP address if you there we go if you look at the uh IP for cider here that comes from there uh and that gets rendered down obviously into this configuration language that um and so packaging existing blockchain Technologies forx Noe is actually it's quite straightforward if there isn't already a NYX package for it um then it would be available really quickly and as I said like with the hackathon this weekend we'll be bringing several applications into Nix and thus xote uh as quickly as we can this also gives us a very large community of people who are already you know used to used to developing these systems so yeah U thank you J I appreciate it yeah questions like quite a bit yeah thank you [Music]
Automatic transcript — names and jargon may be misspelled.