How to Integrate Diverse Sources of Reputational Evidence: Learnings from the Lens Protocol Repu

foreign [Music] coaster I am a staff data scientist at the Ave companies and I'm the lead data scientist on the lens protocol and uh my talk today is going to answer this question how do I integrate different sources of reputational evidence reputation is something that lots and lots and lots of teams are working on right now and I'm seeing some patterns that I think are less than ideal and I want to sort of address that uh my whole talk really is like in this slide right so the answer to the question I just asked is what I want you to do is represent your beliefs about your reputational signals as beta distributions and then do Bayesian updating to combine them and I'll sort of explain what all these things mean okay so this is what you should not do right so lots of teams what they're doing is that they're taking a weight vector and a signal Vector they're doing the dot product and out comes your reputation right another way to think about that is like this so you know you have your weights you have your signals you you multiply them up and then you sum them up and then out comes a reputation number I think this is less than ideal and I'll explain why so um information about uh the variance of your signals is lost when you do what I just showed and variance is important to know because it's a measure of quality it's how you represent the quality of the signal and I'll explain a little bit more about what I mean by quality in a minute so what it means if you were throwing away the variance is that you're throwing away quality information so now you don't know if your final reputation is coming from lots of like sort of low quality signals or maybe a combination of high and low or you don't know anymore this is what waiting looks like without variance right it's just one number so in this case it's maybe like 0.8 and that's really all you get you get this like one weight but what if you did it like this right so you still have this uh 0.8 sort of mean uh Point estimate uh but now you have like confidence intervals around it right so the the underlying distribution behind and this weight is actually something kind of wide right that this is what I would call a low quality signal versus something like this which again is mean 0.8 but has much narrower confidence intervals so we're much more confident about the value of this weight than the previous one this is a high quality signal so you know yeah now if you have these two signals one with uh sort of a high value but maybe low quality and maybe one with like a low value but high quality you know what are you doing to combine these how do you do that um and uh part of the answer is that you have to represent these as beta distributions so a beta distribution is just a kind of statistical distribution which is bound between 0 and 1 and takes two parameters Alpha and beta and I'll explain sort of what these parameters mean in a moment so um look like just do some concrete examples of some potential signals so a really high a high quality signal is like do you have a pull app from Rave right so this is a very difficult to Sybil uh signal that maybe I would say maybe one out of every thousand uh users uh who have this Robbie poapp are sybils they somehow sibled it so that's pretty high quality versus something like um having an ens address which is harder to sip it's like medium hard disable because you still have to pay for it uh but it's you know that's anybody can sort of get many uh ens addresses so you know the beta distribution comes in where uh you know the the way to think about the alpha parameter is that it's the number of non-sibles that you have in like your sample and then you sort you you fix the uh beta uh parameter to one and you sort of adjust the alpha parameter accordingly to sort of match this like real intuition about like one in a thousand our sybils or one in a hundred or one in ten are sybils so then to combine them uh we use Bayesian updating and for the beta distribution Bayesian updating is really easy so all you have to do is just add up the alphas and betas so to give you an example if uh I was combining the three signals I just showed uh I'd add up you know the alphas and the betas I'm going to get this outcome like posterior distribution that better represents uh my belief about the true reputation and sort of how confident I am in the like the final value but here's like a little bit of a more wild example so um say you had 50 signals and they were all low quality uh so beta three one it's very like one in four are sybils uh so but if you you know do basing updating you combine all these beta distributions the outcome distribution is beta 150 uh comma 50 which is actually a pretty high quality distribution because the uh the confidence interval is really small on it so uh to sort of motivate this whole thing uh so let's say you have these two outcome reputations um again like let's say this one is like that that has that mean this one is like this and as this mean the means are identical but uh this one we're much more likely to trust than this user who might actually be kind of low potentially or maybe really high but maybe low right so uh what we're asking this guy we really know that that's sort of where reputationally uh this user lives uh which hopefully maybe has a little bit more contacts now and maybe it's a little bit easier to understand what I want you to do is to uh represent your beliefs about reputational signals as beta distributions and then combine them using Bayesian updating uh so that is essentially Bayesian inference in like seven minutes uh so thank you for listening um I am still out of breath from this this altitude uh my name is Pedro coaster these are my uh contact details I'm going to be releasing a lot more information about this kind of approach which I think is um kind of a very useful approach it's the one we're going to be using for the lens uh protocol reputation system so you can find me on the leaf apps payalco.lens uh the Bird app at Palco and then on telegram calco XYZ thank you