Understanding Transactions in EVM-Compatible Blockchains Powered by Opensourceby Lefteris Karapetsas

foreign [Music] I've been in ethereum since 2014 I worked in circles client in the solid compiler did Dao staff's white hat and the ATC and darwars back then then I walked on L2 payments for many years and founded rotkey today I'm talking about user transactions and how we can start to understand them like what are they and how can we decode them in a human readable format everybody has used ethereum in here and we have all faced these two big problems like you do not know um when you have a transaction you do not know how to get it like there is no built-in way to get transactions for an address you need to utilize some kind of third-party service and that will never be decentralized also once you have these transactions you really don't know what I mean there is a complete lack of a universal tool that will decode a transaction to human readable format there is some third-party services but again they are centralized and they tend to be protocol specific what does the transaction look like everybody has opened everything and sees this you know hex block there is no metadata there is no human readable info that's what every new user to ethereum is greeted with there is no way to understand what um your transaction like three four years ago has done there are ways we are no longer in 2014 there are ways to gain insights so you can see stuff in etherscan with the graph and other centralized apis such as covalent Morales Etc what are the ways to um decode the transaction with the understanding is that it looks like this is a complicated transaction that does swaps over multiple protocols um it's easy to use you just type in etherscan you know transactions last and it gives you using insight and it's totally free uh of course it there are Cons with other kind of centralized um it's proprietary and closer that means that you there is no way for me as a developer to see how they do what they do or extend it in any way um they know everything about you I mean I know the guys from others can they are good guys but you never know who in there could be malicious they can match IPS to your addresses and know who owns what address and where they are located and it actually does not decode everything we've all seen that there are transactions that data can doesn't like have insights for uh and there are other tools that actually can do this the graph um is a way it's kind of an indexer per protocol you they have sub graphs and they index the chain for a particular protocol and then you can query this index and get any insight you want um the cons so the process is that it's very good for single protocol data so for example here we have the other V2 ethereum by massari this is a subgraph that say the other guys can run in their interface and query any information they want for their protocol but it has many many disadvantages it needs payment per query this is their Vision that everybody should be paying for every query uh it's built for single protocol data so they subdress so there is no generic solution um if you have a portfolio tracker or a wallet or something that wants to decode every single transaction no matter what protocol it is in subclass will not work you will have to basically query every single subgraph and create a subgraph for everything that doesn't is not supported yet and it does not work with uh local apps what I like to call true dubs so basically uh when the company that makes the application hosts um uh host the code they can have an API key and pay for the queries but when it's a local application there is no way for this to to happen and there are these other centralized apis like covalent Morales and Alchemy they're easy to use they have pretty cool apis um which can decode transactions uh give you all the transactions but the same concept letters can apply like it's centralized it knows everything about you it's proprietary so you cannot extend it it's not modular at all um so having seen the ways that we can get insights about data let's see how we can actually get an accurate historical data and let's go to the original scene of ethereum like this is absolutely bonkers if any of you has tried to get the history of transactions for your address you will find out very easily that there is absolutely no building way to do this for um for ethereum there is no RPC method this is all due to the way that evm works and how the clients are built but it is really like it's absolutely crazy that there is no way for you as a user to get all of the transactions for your address someone that comes as a developer outside from web3 and comes to ethereum and sees this the thing that we are we're just crazy that this is broken uh it's it's not all gloom and doomville there are ways to do this effort can again comes to the rescue they have many apis and if you combine free I think so this one for for transactions then there is one for reality transfer and one for nfts if you combine all of them you get a pretty accurate picture of what transactions your address has done of course it has Dropbox right it doesn't detect all others appearances it's rate limited but you can pay for bigger limits but rate limiting means that it takes time for your query to actually work it is centralized so it can go down they can cut access to the API or um they can do what I said before that they can monitor you and map IP to your address the truly decentralized way to go around this is something by my friend Thomas de Ross It's called true blocks it is really the best and most complete way to get transaction data um two blocks it takes all appearances of an address really I have seen demos um where TJ basically shows etherscan and then Compares it with true blocks and you can see that for some addresses through box does indeed detect more appearances than the other scanner it is decentralized so it runs on top of your local node so you do not need to to do any other network queries it's super fast like it's really like milliseconds or seconds depending on the amount of addresses that you query for uh and it's built to share the this index with others of course like like if everything is software there is drawbacks um it is hard to set up so TJ is like who's a long wolf since I met him in uh Shanghai in 2016. now he's a bit bigger team but it's it takes time to build something that's easy to use by others it does require a local node so you need to be running an ergon node I think I'm not sure if it works with others and of course you require true blocks itself to create the index um so building on this I would like to like present what I try to call the stack of 3D centralization which is something that we should be driving striving for in crypto so everybody should try to run their own node so something like a dab node or a Raspberry Pi with whatever setup you guys want to have um you should run your own client right like for whatever chain you have run a client for that saying that you want to um to use and triples actually works for all chains only VM chains so you can have an index that like Roblox on top that will index attain and provide you an answer to the question how the heck um so what addresses uh sorry what other access does my address have and on top of it all to come and bind it all um you have the aggregating and decoding level that something like rotkey but not uh um like a decoding so Roti right now is a an application but imagine a platform where you can have a generic way to go from transactions to a common readable format for what they do and it's actually consumable by humans going from how we get data to what actually would go into this decoding platform that was that my talk is about so once you have all the data like either from method or from True blocks what kind of data is this if you have tried to play with understanding transaction history of ethereum you know that there is two ways to get data it's either a trace a transaction trace or transaction receipts there are two kinds of traces one is the gift style trace and the other is a parity Trace I'm gonna go through them a bit fast for those of you who do not know what they are so give style traces the tracing that comes with the git client when a transaction happens it touches a multi so it touches multiple contracts right so you make a transaction to a contract and this one may make a call to another contract and so on and so forth and as they do this they touch the state of these contracts and they they make some changes so this is what the trace of the transaction is and the give style Trace is the most uh complete it's like super detailed it has every single step of the execution with um the op code the program counter the storage diff Etc it's super detailed it's very hard to use because for complicated transactions you have like a huge thing that you don't really know what it does and it can grow extremely huge like in the gigabytes for really complicated transactions then we have the parity style Trace which comes from the now defunct parity client but it is used I think everyone for sure another mind and maybe maybe better I'm not sure um they have three commands one is a trace that's pretty cool and useful um it gives you a call stack like this of what did your transaction do there like the cold trace of the transaction and this actually does not require an archive node by the way the screenshot is from um a very nice article by bantag on traces that came out like two months ago I think so Google like bunt against transaction traces and you will um you can read about it in more detail and the other thing that you can use to understand how a transaction um what has it done is the parity style Trace div it gives you a state the for each account that you touch it gives you the difference in Balance code nodes and storage the cool thing is here if you have the API you can play with it a bit and you can have readable names for the storage slots and how did they change so this is a very useful Insight on what did the transaction do and of course then transaction receipts we've all seen like the how a source code of a contract looks like they have events these events are actually concerned in something that's called the transactionary seat so let's say for a token transfer it's a I don't know like transfer Source destination and value or something um almost everything generates them um it looks like like this this it's all hex but if you have the ABI of the contract you can decode this into a human readable format this is how you gather the data but gathering this data is actually expensive it takes time um and exactly because this is expensive in in resources persistence is key so any kind of platform that you create and the thing that we have created the drop key needs to have data persistence um you can choose various ways we've gone with a simple masculi database for now but um this way when you have gathered all of the data and you know that they are true and will not change then you can just take it out of the database and reuse it instead of having to re-query again through blocks or etherscan or make a trace again we talked about where you can get data how to get it and then I'm gonna go to the mid of the presentation which is the decoders themselves so we want to get human readable format of the transaction data yes of the transaction data so we have gotten either receipts or traces and with this Amazing Escape graphic that I made guys I'm really proud of myself for this we can see that um you can check for its um receipt for it to log the address and then send it depending on the address to uh either the generic erc20 transfer decoder if it's a uni swap swap to the units of decoder and so on and so forth like Ave Etc and all of this will at the end emit a common uh event format what's more uh some decoders feed data to other decoders so the erc20 transfers create the rc20 transfer event and this gets fed to unisop which translates it into swabs such a decoder platform is made on modularity uh this is um rodky's repo and this is where we have all the decoders and it's like a huge list it doesn't have all of them um because they don't fit on the screenshot but the idea is that it's easy to write easy to use and drag and drop that uses drag so you drop it in there and it's caught by the the system and then a new decoder is taking into account whenever we um we we decode a transaction that's the idea uh we're not there yet we build binaries and uh this is not as modular as it should be but the idea is that it should just um big drag and drop um this is how the source code looks like uh this is I think a hop protocol two bridge to another chain so um it's hard to read probably but the idea is that you get from the ER security transfer decoder you get the erc20 transfer and then you see oh it's a spend the counterparty is the F bits of uh hop and the asset of fmatism the amount matches so then we transform it into the common event format for uh hope and give it a nice readable um um explanation which like breeds the amount of heat to um either your own address or to some other address in the chain so the name of the chain via hop protocol I talk a lot about the common event format it's kind of a POC because we are um we are only consumer right now uh it's uh changing so this is how it looks in the code um it has like a sequence index inside the transaction so where did it happen in the transaction timestamp location location is mostly uh something that we use in rotkey because we um subtract everything into this format not only ethereum transactions but pure Kraken trades your ethereum stating everything gets uh subtracted into this less common denominator format um we have the history type A History event type and history event subtype so this is what defines the myth of how you define um an event the asset and the balance change and then some extra stuff like the location label is along the counterparty is if I I send it from me to someone or if I got something else and some extra data like if it's a CDP for maker we have the CDP ID here Etc um as I said everything is broken down into this thing uh like a swap is three of these events so it's a amount out amount in and fee out uh or it can be two if there is no fee uh we are working on this uh it's not final but this is like the idea uh this is how a front-end can consume and so this um uh code um this should be read unfortunately because I didn't take a nice cases from the bottom to the top so I claim my Badger airdrop uh so it has two events the gas fee that's burnt in the airdrop claiming then approval to one inch V2 and the gas fee for this and then the swap for uh in one is for basically immediately dumping the uh the tokens this is the same thing that you saw in the in the previous in the previous screenshot basically these events each one of them is one line here So This Is How They are consumed in in rotkey itself which is a portfolio trading application um so uh I would like to actually also talk a little bit about the abstraction layer of of the vision of rotkey which is like if you take this a step further and not just focus on um uh just evm code but anything like any event um you can get into like an open source middleware that offers an abstraction layer for everything accounts balance is pnl over multiple protocols and jurisdictions so this is kind of the vision that we would like at some point to go with rodkey um so we went from a portfolio turning up to um a common evm decoder and now more towards a middleware that would offer an abstraction for everything in accounting for crypto why right like people would ask why the heck why would you need this because everybody is Reinventing the wheel there is again as I said neighboring protocols different exchanges chains jurisdictions it's impossible to keep up I have talked I have spoken with people in uh both small startups and big names in the field for portfolio trading and crypto accounting everybody is saying that this is just too much to give up and maintaining just one module is a full-time job um so I believe that there is a solution to this problem so the problem of everybody Reinventing the wheel has a solution that um we can have an open source uh platform middleware if you want maintained by a core team but with contributors uh from the entire industry and used by multiple projects and for the problem of different protocols and jurisdictions and being it impossible for a single organization to keep up I would like to propose a solution that we can have people incentivized from each stain and protocol with appropriate know how to come and Implement modules in this platform again my amazing inkscape skills um imagine a middleware where you have like someone who wants to use um to do Port authoritarian accounting has um the the core uses Bitcoin and ethereum plugs in the Yen module the other module he also wants to do accounting plugs in the accounting module and because he's in Germany he plugs in the German accounting uh with fifo multiple Depot and early for accounting methods uh and imagine this middleware basically being used by many people in the field and um in the end just everything plugging into this because it's better to use a common open source uh middleware rather than every single application Reinventing the wheel uh any such platform would have some super basic requirements it needs to be open source like everybody does tries to reinvent the wheel in a proprietary clusters way this is absolutely idiotic it needs to have a modular architecture so that as we saw before like be pluggable have pluggable modules um you you're in a different country than Germany you can just plug the I don't know Netherlands accounting module you don't use um ethereum you use kusama you do the substrate uh module and you can do all the polka.com Etc it needs to have a this is a hard requirement to achieve but it needs to be moved illegally it needs to have multilingual bindings because we attract your python uh house we know how to use other languages but most of our code is in Python such a middleware should not uh uh limit the user to so we cannot ask the entire industry to spot to python if they are to to use such a thing the platform should be built in a multilingual way um and as foreign should be incentivized to actually contribute to this platform if it becomes an open source standard then everybody should be like oh wait I mean we made this new platform uh we need to write a roti module for it also because otherwise it's just like nobody will use it and the core team that builds and maintains it also needs to be incentivized in in some way in order to be able to keep uh building the ways that this can be is through the support to the various teams or through uh slas for a software level agreements for companies that may not want to have open source code so you can have the Dual licensing so it's a bit funny I wanted to show the I saw the timeline thing and I was in the template that they gave us and I thought hey why not put a timeline so how the heck did we get here 2017 I just need to do my taxes in crypto and I was like okay I'm not gonna what is the the way to do this there was Bitcoin to attacks there was nothing else back then I'm not going to use a centralized service okay I just don't trust them so I just made some python CLI scripts it worked I've not been sued by the German government yet so it won't I don't know um and later build a UI around them in 2020 we made it into a company we were a team of two people and maybe we had 200 users 300 and maybe 10 paid um so last year the app had grown we hired one more developer and we were 2000 users and 200 paying users in the beginning of this year uh there is many people who use rotkey right now um some like it some complain with they always want more and more but it is at a level that many people can use it we are a team of seven now six thousand uh around six thousand users it's hard to know because we it's an open source app and we don't have Analytics um and 500 550 pen users we came all this way without uh anything like it was just completely bootstrapped and from um uh basically your donations through Bitcoin and from Integrations with um uh other uh companies like Optimus gave us a grant lately before that there were kusama and so on and so forth um so for getting this POC that I described here to the full rot division we would need to go further from here and try to grow and potentially get some funding because with the current team that we have it's very impossible to actually build this Vision the POC cannot grow to a level of something that can be used by the entire industry with just six people six developers is just impossible um so with this I'm coming to the closing notes um so if you like restaurant you had like uh open source Locker first the modular thing that can be used by everybody in the industry then please talk to me or check out yeah this thing again check out roti.com jobs we have some open positions um we came here thanks to you like seriously it's a boost up uh project and we would not be here without git coin donations and without uh our premium users so keep supporting us um you can donate in git going grants or in um buy premium uh subscription and unlock all of the features of uh rodkey and you can join our community in Twitter or Discord like that's where all the support is it would be pretty cool for you to if you can um uh join the chat and join our community if you're interested in helping us grow in realizing this Vision that I try to present here uh then talk to me either like any day in the conference or write an email to Lefteris rodkey.com um yes without that's all uh thank you very much [Applause] okay like you mentioned the graph and true Vlogs the one thing that I don't understand about these two tools is the two query historical data is that they use a client node and this node is basically in charge of the story in this data in an SQL database isn't this uh centralist way of saving data is this one ipfs comes into play and if that's the case can you explain how ipfs and SQL database work together to solve this doesn't come anywhere in there like uh the graph is uh and true looks completely different things so the graph is um it creates an index on top of um your already existing um uh node data and truebox does the same but true blocks does it in a generic way for all of your transactions while the graph has specialized subgraphs written by developers of particular protocol that uh uh basically write an index address for this protocol and this lives on top of your node data it is decentralized like the graph by Design is also decentralized true blocks is itself also decentralized it creates this index and this index is said um I think it's pinned in ipfs and shared with others who use true blocks I I'm not totally sure on the details of sharing of true blocks because I'm not a developer but I think that this is how they do it as for the graph they have a decentralized network if I'm understanding this right the idea is if we build this out um and get it out there then like we could get around using services like tenderly and just basically run tenderly at home for transaction tracing and simulations and all that yeah I view it more from a historical perspective and Tenderly is a current emulator but yeah I suppose that you could also do the same correct me if I'm wrong tenderly is proprietary right yeah yes yeah I think that they work with trades they're pretty good but they are proprietary with such a design yes I believe that we could uh uh yeah use this as an alternate hola awesome presentation great work uh from the developer perspective on on a solidity developer I think will be very useful for for example I write a smart contract and I write the decoder let's say I write a decoder and I host it on ipfs have you thought about that like how we can have that I can standardized I know let's say Parable where we Define the URI or hash where we set our decoder and you guys can use it because you have a list of the course on GitHub right yeah yeah yeah I mean this is just a different medium of of delivering the decoder but yes there is exactly what you described so I'm not going to write every decoder my team is not going to write every decoder that's impossible but the idea is exactly that you when you're at your smart contract for a protocol then you say okay I'm also going to write a decoder for this and then somehow it should be delivered to this middleware yes it can be through a link it can this is the POC of what we have right now the drag and drop in the folder in GitHub I think it does it a lot of value if you can also validate the decoder because if I do it I can buy the malicious decoder but we will need like a validation system and these are this is a place where I think you can add a lot of value okay yeah that's that's good feedback thank you guys