Zero Knowledge Machine Learning

foreign [Music] for those of you that aren't as familiar I like to say that zero knowledge proof started digital signatures as ethereum is to bitcoin we're replacing a digital signature with a any program right and the thesis of sort of zkml is that the most interesting program actually to put in a zero knowledge proof is a machine learning model and the reason for that is that it makes it as though the model the AI is actually running on chain so we're giving intelligence to ethereum or whatever chain it's running on um and it's kind of like we're giving it Eyes Ears you know sensory organs for which with which to perceive the physical world um with which to make decisions about physical reality and the most important bits of physical reality to us which are the humans right what our intent is um so it makes it possible for humans not just field elements to uh control their destination to control digital assets so imagining that in the future we'll be able to say just like we talk to our smart speakers you know hey ethereum you know please transfer 10 F to Alfia and that will work right because some Ensemble of machine learning models will be able to combine all the signals of my intent all the data exhausts that I've given off in my life and decide whether that is true or not for a human that exercise is Trivial if we've talked for 10 minutes um and I do that I make that request two years from now you'd be able to authenticate and send that message the fact we can't do it yet with uh with ethereum is just a just a technical problem um and finally you can think about it as a way to let smart contracts exercise judgment right deal with any kind of ambiguous situation decide if a contract is satisfied decide if a news story says that a hurricane has hit the coast that thing and this little chart here is just a reminder that both the input and the parameters we can choose either private or public and all four of those squares are interesting everything's public we can think about the news model scalable Oracle if everything is private maybe we're thinking about decentralized kaggle or some kind of um medical situation where on the one hand the model has to be kept secret for reasons in the patient data has to be kept secret um so what we've built is a program called Ezekiel it is a tool to build turn Onyx model which is an export that you can make from pytorch or tensorflow so a baked machine learning model that's been trained into a zero knowledge proof we can prove and verify at the command line export to a binary or a contract smart contract or wasm we're adding new layers to it daily and it's enough for sort of small production models the scale is increasing very quickly sort of two to eight times a month depending on how much time we choose to spend on optimization as opposed to adding features and I expect that we'll be able to do some really exciting kind of identity stuff within maybe six to 12 months maybe sooner um and we've been building in the open since July it's Apache 2.0 license and uh that's the repo we would love any contributed contributions from you if you find this interesting um so here's an example of the kind of thing we can do right now uh if you like python this is very relaxing as opposed to writing circuits you can define a forward model X Y and Z are tensors they're with a shape that's determined at runtime and I can just sort of Define somewhat arbitrary functions of those tensors Matrix multiplications Powers non-linearities compose them and then the tool will translate that into determine a quantization strategy figure out the runtime shape of the tensors and translate that into a something that can be run as a zero knowledge proof um so the pieces we can do right now are sort of the describe what we're going to do do a mock or a full proof is the back end is all Halo 2. um we can do a complete a proof and we can verify a proof um and then there's some sort of technical parameters and this applications talk so I want to kind of give you a sense of what's possible and then we'll talk about what applications are cool um right so this is an example of computing the table from a very simple model that just has an input a weight and a bias a matrix multiplication and a relu um we can do a proof uh looking at different uh back ends proving back ends and we can do a verification the the proof will construct a file a little Json file that can that contains the data of the proof that you can upload to whatever the verification thing is and then you can run also the command line to verifier to check that proof um and okay so now we go back to Applications I always like to show a little bit of code so that we can think about kind of what is the big what is it what stimulate imagination but I like to think about this is kind of allowing scalable automated oracles right so there's three stages to that the first stage is ingesting some kind of signed signed data now it works fine with not signed data but there's sort of an adversarial problem so we need to somehow rate limit the amount of data that's coming in or or make sure that it's been signed off on by like say a news organization or camera or something like that then we run a model maybe it's a text model maybe it's image classification it makes a decision about what the data was and finally there's a unchain verification that then feeds back into the sort of attestation Loop and can be used in the next model um so interesting things in the sign ingestion this is this thing this space is really an interesting stage so there are solutions there's something called a sxg s XG which is a signed HTTP standard promulgated by Google kind of replacement to amp um there's an nginx plug-in there's a one click button on cloudflare a lot of people promise to do it almost no one has there's email there's a few other people from Xerox Park that have worked on the email that you can hear about from ayush um there's images at the publisher so the news organization New York Times Etc might sign off on their images so that other people can say oh New York Times said this bad thing happened um and then they're third-party notaries like TRS notary lip protocol that you can use to um create signatures uh as sort of a semi-trusted third party and they all use things that we can now verify um but I think we need we're going to need an industry at a push like https to push everyone to sign their data it's technically easy it lets us compose things in zero knowledge proofs where people just don't bother to do it right now although they've all promised to uh so the second thing is the text model classification what kind of models can we build and that's sort of a scaling process um I describe our roadmap as kind of ontology recapitulates phylogeny which is to say we look at all the models that have been built over the history of uh of machine learning and then you can download them you try to run them to your tool on it doesn't quite work because they're scaling problems or quantitation problems or because you haven't implemented all the nodes and then you fix it right and then you repeat so as I said open source project would love you to join me on this rotation and then we have to worry about scaling and there's kind of five ions that we use for scaling so optimization I won't say much about that I think you will talk about that a little bit in his talk aggregation which is a tool for combining multiple proofs into one proof and checking it checking it once recursion which is being able to verify the last snark inside the new one and that kind of lets you do something like uh doing a separate proof for each layer right that gives you another scalability tool and between occurred aggregation and recursion what you're going to see is the is instead of memory constraint which is what usually the problem is in for the size of the models we can handle um it's just money constraint right how how much how much computation we like to spend it's not it has a lot of overhead but it kind of eliminates the limits on how big a model that's why I can make grandiose claims about the size of the model that we're going to get to um and then finally Fusion is kind of the strategy of once we have a higher level understanding of the intent of the programmer the computational circuit that they're expressing in python or whatever it's easy to swap out more sophisticated zero knowledge arguments instead of looking at the level of constraints I look at the level of matrix multiplication or convolution and you can use those arguments and make it invisible to the person who's created the um the the machine learning model the machine learning model doesn't change at all but it kind of gets faster and faster in the back end um right so finally on chain verification um uh this is as I think you've heard uh you know severely constrained by the pre-copiles we have available in ethereum and a lot of the the strategies just getting it to fit into a bn128 um so the way that it can be done now is you have and probably the future honestly just because of the nature of of zero knowledge proofs is you have a first stage which is you know some sort of input and then you have easy to make yet hard to verify proofs right sort of wide proofs those proofs are aggregated uh using an aggregation strategy which might require a fairly Hefty machine so one example I did use 450 gigabyte machine RAM and then that produces it hard to make but easy to verify proof that you can then be checked on ethereum and right now it costs about 600 000 gas to do that but there's some things that we should hopefully improve that over time um right so putting all this together what do we get we get kind of this scalable on-chain data feeds you can sort of think of it as oh ethereum can read the news or instead of having a network of nodes that make a decision like in a chain link about something that's happened and then vote on it and there's a complex crypto economic process by which they penalize people who who tested wrongly you just have one person they download the signed data from Bloomberg they produce the zero knowledge proof they upload it and now everyone can trust that information so that enables a much more scalable fire hose of data coming from off chain from web 2 to on-chain um and it's my belief that uh zkml will actually be table Stakes for chains over the next sort of five to ten years that while delivering on This Promise of blockchain to a mass audience means that we really need this kind of a human not a field element owns all my assets right there's lots of solutions I've heard at this at this meeting towards that and this is part of that you can imagine for example when the account abstraction starts working that part of the account abstraction check that you're doing is to submit a zero knowledge proof of your identity right using this kind of 10 000 Factor everything about me um as one of the pieces um so zkml oracles will be simpler faster and more scalable um to put arbitrary on off chain data on chain and just I think really opens the fire hose to what we can do um and finally as I said before a zkm model you can think of as a smart judge that interprets ambiguous events and in the remaining two and a half minutes let me just talk about a couple of examples ideas of things that I hope people will build with this um so one that's sort of timely is zkyc right so we can take a person and an ID and prove that they match and that the ID number is not in some sanction database right okay of course that's technically something we can do very soon however Regulators won't accept it right banks have a kyc rule it has that it's it says specifically that they have to know the customer not that they have to know the customer is not on a sanctioned list so it seems like it won't work however maybe if we had done that it would have prevented the tornado stations right it does if you're a D5 developer or your mixer developer you add that to your pipeline it sort of makes you less of a Target you know it lets you run faster than your friend when the bear is chasing you um or at least let it uh does something to prevent you know unwanted actors from interacting with your contract so even though it's not perfect it's interesting um of course I talked about prediction markets you could imagine setting up a a contract that pays if a news story classifies to a particular thing someone won an election a hurricane of a certain intensity to hit a coast a car received a lot of damage and then a small classification model because it's relatively few classes can be used to to decide whether that happened and anyone can you know download the science story run the model submit the proof another thing is kind of fraud checks so gut checks for smart contracts you could imagine uh the abstracted account or the smart contract just has another little zkml check that is a rate limiter and a fraud check and makes it harder for people to scam that contract right um a proof of humanity for example would work but lots of other kind of fraud checks or checks of the state of the network and you could imagine getting baked in a little bit like civil protection but it's okay if it's weak because it's it's just a layer of security um I think it's exciting to think about this as putting the a in Dao so really making taking a situation where humans make a judgment over what happened vote and then some multi-sig signatories uh actually execute that judgment and actually putting that all with an effectively on-chain AI right so instead of working for a doubt again we all work for the the all-knowing AI in the cloud um it's gonna be good or bad but interesting uh and you know they can use for example to test whether someone who promised to do some work for the Dow did a good job did the work right you can imagine making a classifier that did that um you can think about using it for genetic screening situations where for example the model has to be secret because it's been trained on on non-consented data the data has to be secret you know just like when someone wants to get an STD test or something they would like to choose whether or not to reveal it to anyone and maintain anonymity um so that's another application um and finally it composes well with not just NPC but also things like differential privacy you can imagine a data owner uh this has nothing to do with blockchain necessarily but you can imagine a data owner that wants to protect their data they can release noisy summaries in a differential privacy sense and then receive the the the query or can run the model and torture the data and send a final query which can then be run in a proof to show the real model matches the commitment it was used to create the noisy marginal summary and when it was run on the data It produced the real result okay so income out thank you thanks Jason um if you want to take one or two questions um well his son gets set up Assuming he's he's here here hello um uh Eastern is going to talk about another aspect of DK machine learning specifically in neural networks but did I see a question over seven questions yep machine learning models are often kind of already black boxes how do you verify that the right machine learning model did the prediction or the verification yeah you you can you commit to the model so that's particularly easy when the model is presented as an onyx file you just take the hash of the file and that's your model for example or you can commit to the parameters themselves so you make you make some kind of basically a hash of the parameters or hash of the whole thing and that's how you know it's the model you want it to be do we have one more quick one everybody else can take them offline sorry really cool stuff um has the CK kyc sort of being worked on is that a theoretical possible application of something like this or like what's the state of yeah I'd say it's it's absolutely being worked on I think it will be a little while before it's feasible um but you know it's definitely something that I expect to see probably multiple people create in the next year