"From Proofs to People: the Moral Responsability of Cryptographers" by Sofia Celi

[applause] Hi everyone. Okay, I see myself. I'm going to share my screen and hope it work. Okay, great. Hi everyone.

So as it has been already said and I'm a senior uh cryptography researcher both at Brave and at the University of Bristol and today I wanted to share with you a little bit of um kind of how we have been thinking about ethics and morality on cryptography um especially and also on security and I'll try to make this talk um with a lot of logos because I like to do logos for all my research um but we are also going to be talking uh quite a thought about very serious topics. And because this conference was held in Argentina today, which I'm really happy because I'm a Latin American, so I'm really happy that these conferences are happening in Latin America as well. I wanted to start kind of like setting the scene on one specific episode of the life of and the history of Latin America as a whole. So for some people that maybe are from Latin America that are present on the screen on the conference today, maybe this picture was really familiar to them because this shows what's called usually in the Spanish as loivos or in English the English uh or in English sorry the archives of terror and this was indeed a big collection of documents that were found in history after the fact that a lot of dictatorship happened in Latin America and especially these archives refers to what happened in Argentina, Uruguay, Chile and Paraguay during the 70s up until the 90s after what was an state sponsor um torture and kidnapping many people across Latin America which was halfly supported by the United States in one specific operation that is usually known as operasion condor or operation condor. What this specific archive show which is publicly available and anybody can actually access and see this specific archive is held by the UNESCO nowadays.

But what this specific archive show is that this process of torture and enacting of violence that was performed uh and against people was actually done um via the state and via a lot of organizations but it was actually uh helped via the usage of technology especially during data collection. So that could people could have been easily monitored, surveiled and later tortured and disappeared in some cases. Um was really really this kind of this process of data collection. And what we can reflect upon this is that if during those years, not so long ago because it was some decades ago, um this was processed and process of data collection was actually uh allowed um which which in turn enhanced surveillance, monitoring and eventually torture of people. what we can reflect is what could happen nowadays nowadays that we have this uh enormous access to technologies and digital technologies again which are more and more used to collect the data of users.

So with this in mind um I want to start also by like kind of setting the scene of who I am. So as I said in the previous slide, I'm a Latin American um and I'm mostly showing in this specific slide because when we will see in the next slides that we will talk about actually designing cryptographic protocols or security protocols, you will see that the majority of times there will be a lot of peles from the security perspective because the majority of times the people who um these protocols are going to be pushed to production in a specific regions of the world. the majority of the specific protocols and not from those specific regions of the world. So I'm here to tell you that if you're someone from Latin America or from an underrepresented community that come from a degree that maybe is not the former degree on computer science or the former degree in mathematics or in cryptography that there's also a space for you because the majority of times what we need when we're actually designing this kind of technology is people that care about privacy that care of now providing more means for this data collection and more surveillance because we have historical examples that show that doing that actually ends up with more torture and more disappearance of people. So, I'm here to tell you that if you're interested in getting into cryptography and security, there's a lot of space for you, especially if you come from an under the represented community, especially if you come from the Latin American region.

And I'm happy to take any kind of emails that you want to send me if you want to know about how my random work eventually got into cryptography and security. So, but this slide was just kind of to give you reassurance that even if you don't have like the former degree or the former background, there's a still place for you and we need people like you in cryptography and security. And because I have this kind of like very informal and very different background on computer science and also cryptography and security. What I tried to shape in my research is that I want to create cryptography that mostly is called as boring cryptography and boring security. And the reason why it's called like this is because often times maybe this is not the most sexy theoretical research or most or the most sexy mathematical proofs or security proofs.

But at the end of the day we are designing something that is for the people and should take into account what whatever people's need have. Instead of trying to create protocols and then push them and try to fit them into people, what we have to do is change the mindset and mindset and actually talk to people first and then design second. So don't find a motivation after the fact that you have designed something but rather talk to people and create something for them. And the result of this oftent times is boring cryptography because sometimes we don't need to advance so much theoretically in order to push something that works for people in real life. [snorts] So that's why on my cryptographic research you will see that I actually create some systems that are very cheap that work very cheaply for any devices because the majority of people on the world have access only to cheap devices and and and also to low connectivity.

So actually designing something that works for all or works for many regions of the world means designing very cheap cryptography but I want also to create mechanisms that preserve privacy by design and diminish the surveillance that we have currently on the digital means. And also want to design cryptography that survives when you actually push the cryptography in production to bring proofs that not only care about the mathematical correctness of the algorithm but actually that when you actually put that in production that specific property is also preserved at the browser level or at the operating system level or whatever level that is also preserved on the device that you're working on. And I want to actually avoid the illusion that the illusion that cryptography out of the box solves comp complex social problems because in my career what I have found out is that a lot of people often think that certain technology or cryptography can solve complex social problems. I have heard things like oh I'm going to develop this blockchain or those this serial knowledge proof or those this specific algorithm so that this specific country that belongs to the developing world or the third world countries um its economy is going to be solved by via the usage of this specific blockchain that is a very reductionist approach maybe it will help in some very diminished way but it doesn't mean that technology completely will solve the complex social problem so again it is important when we design is a specific technology to bring ethical concerns into standardizations bodies and policy process to strengthen thread models to include broader and emerging threats to talk to the people when we are actually designing an specific system and working with community and experts. So as I said this is perhaps boring cryptography but it's the boring cryptography that I like to research and to actually push in in production.

Maybe it's not going to be the sexiest one or the one that gets you the most grants, but it's one that is one that is going to give you a lot of impact and is going to have a lot of impact into the real life of people. And a question that I often get asked is that if you do this boring cryptography, maybe you will not have the most sexy theoretical career, but it will mean that you will have a really successful career. So in my life for example um the process of actually designing this kind of cryptography has allowed me to for example be part of advisory committee committees of the United Nation to create communities of representation to won some international awards or to be part of some ethics committees of different conferences. So don't think that there's only one path which is the theoretically sexy path that you can have in your career but actually another path of actually thinking about people and deploying systems for people can also land you a very successful career. Why does it matter for people like us that maybe are cryptography or security experts that are designing systems uh for people?

Um is because the majority of times when we are thinking of computer computer science, we actually don't think about how those specific protocols that we designed are going to be actually affecting the people we design for. But we have to take into account that whatever system we put in production and whatever computer science system we put in production maybe can still provides more forms of surveillance of occlusion of governance process of governance problems of access problems of consent problems and safety problems. So we as protocol designers as designers of systems especially in the cryptography and security realm we have the moral responsibility to think beyond proofs and idealized models. So in this talk I'm going to be talking about two specific stories. Um I put three in this slide if someone is also interested about the second point but mostly today I'm going to be talking about two.

First that when I do my research what I want is to span the thread model of security cryptographic protocol. And the third one that I also want to assure that whatever cryptography we put out there is not just out of the box and that we don't pretend that we by having this specific cryptography we are completely solving complex so social issues. So let's go and see on the first one. So in the first one essentially what I want to do in general in my research is to also expand the thread model of security protocols. So if you are familiar with the standardization bodies you will know that there's this really big standardization body that is called the IETF which is one of the biggest one because it has a standardized for example TLS, DNS and IP.

Currently on the ITF there's also a research area of it which is called the ITF. on this specific uh research area. We have a research group which is called the human rights and protocol considerations research group. Why was it created at the ITF level is because we as the ITF realized that the pushing for these network protocols for example the IP protocol, the UDP protocol, the TLS protocol. It means that certain human rights consideration have to have to be taken into account when they actually design these protocols.

So for example, if you are pushing something as TLS is because you care on the privacy of the communication and the security of the communication of users. Hence, we're preserving the privacy um right that people have as it is enacted in the human rights um declaration on the universal human rights declaration. However, when we actually proposed to create this specific research group, we got a lot of push push back. And you might think that we got push back for like very widely known reasons, but actually we got a lot of push back because a lot of people said, why should we care about human rights when we sign in technology? Why should technology be not assumed to be completely neutral?

Why do we have to think about human rights considerations? Why do we have to talk to human rights experts and policy makers? an answer is specifically on one specific case. I will give you an example of why is this important. So one of the core um kind of research lines that we have on this specific research group is to think about how can technology be abused or used by design in the context of intimate partner violence.

So how can technology be abused in order to enhance the violence that is enacted in the process of a relationship that is uh currently undergoing a process of abuse. And you might think oh this only happens with a specific technology that is designed specifically to enhance this kind of abuse. So you might think for example of a stockware or spouseear um and that is true. This specific technology exists but it doesn't mean that other technology cannot be abused in order to enhance this kind of violence. So a lot of technology that for example currently is used in order to do location tracker or your browsing history or network details is heavily abused by abusers in order to enhance this specific type of violence.

So for example in the case of browsing history what we have found out is that a lot of attackers and a lot of abusers actually survey the browsing history of their victims to see if they are like for example contacting helplines or they are indeed uh visiting an a specific website that gives them resources or how to escape this specific relationship. So what we actually show is that even if this technology was not initially designed to actually enact this kind of abuse, it is still used by a lot of abusers. As you see here, there are quotes from real life of abuser actually abusing this technology in order to enhance violence. So what we see here is that there's a human rights consideration when we design this technology. even technology as location trackers.

So browsing history or network details or how to surveil someone from the router or how to preserve the privacy of whatever site you are connecting to because that technology even though it is not designed to that can be used to enhance this abuse that is currently happening by a lot of different things. But what we also found out in this specific research is that one of the things that happen in cryptography and security research papers is that this kind of attacker is not taking into account because more often when we design a threat model, we design a threat model that only cares about a network address. But in these cases that we see this adversary has access to the devices, it has access to the browser, it has access to the UI, it has access to many um things. It can even coheres the victim. And hence when we think about thread models, we should not only think about an adversary that has access to the network, but we should actually design with taking into account that there's also a wide array of attackers that could has access to your device or your authentication mechanisms.

So we pose here the question of actually let's expand this specific consideration of threat model and let's also design with having the human rights consideration so that the technology can be abused even though its design maybe was not made for that it is still being abused. What we decided to create is an specific SO document which is a native document that actually says that this is the ways that technology can be abused. This is the kind of attacks that can be launched with this kind of abuse of this kind of technology and this is mitigations that we can have and you can ask is this boring? Yeah, it's kind of boring because we know how this technology can be abused. But still we need a document at the standardization level.

So but we are sure that when whatever technology is being pushed to production is being pushed to standardization that whatever considerations when late in that document are taken into account into other documents. So maybe this is a boring job because it involves a lot of talking and a lot of discussions and a lot of like saying the same things that we have said many times but it still is that something that needs to happen. Now on the on the third point that I talked to you about sometimes also what we think to doing is actually using cryptography out of the box to try to solve some so some complex social issues. So for example, one of the second lines of work that we have currently in HRPC is also to think about how technology also is abused in the case of a child and father or mother case. So in this case for example yes technology can be abused in the cases of uh child sexual imagery also trafficking but also certain kind of technology that is quote that is quote unquote uh for the safety of the child can also be used to diminish privacy especially for the most vulnerable to undermine the rights of association to also harm gender equality and gender affirmation especially in the cases of the queer youth to create environments of constant surveillance and to enable familian institutional state cohesion.

So what we're currently also doing is creating a document that says yes this technology that provides the child safety can provide some safety to the child but it also undermine some of the human rights that these children have. Why does it matter now? It matters now more than ever because there's a lot of policy bodies that are currently pushing for age verification system with some link to what is called digital identity systems. And the reason why they're pushing this is for the safety quote unquote of the children by saying children should not have access to certain child related sites. And hence we're going to be checking this by checking the age of whatever user is trying to access these uh sites.

But some of these proposals go beyond the age verification check and also now they want to check on the gender and also some of the proposals now are also aiming to check the legality of this person um that is trying to access a service. So there are some proposals for example that someone should only be able to access an specific health related government website only if they are quote unquote legal in this country. And one of the things that people have been proposing is to use zero knowledge proofs in order to check the range of this specific user. But one of the things that we have found out in research is that zero knowledge proofs still have some security fragility at the implementation level. So they should not be pushed into production out of the box.

And there's also some privacy leakage via channels that can occur in these age verification checks. But more than anything what we have found out is that this kind of thinking in society is that it provides centralization and exclusion because what happens if for example if we say that we are only going to allowing that a website is only accessed by an specific subset of the population that is of age majority we're going to be having to provide certain kind of credential a digital credential but who is issued this credential it means that only legal people in a country are going to be having this credential then what about refugees or people who don't have a state or people who currently are renovating their visa permit or their resident permit authorization. Does it mean that that people will not be um will not have the ability to access these specific sites because their situation is kind of unclear at the moment? So what this create is this creates a a system of exclusion and a system of centralization where only certain government and certain account providers are the one providing with this specific digital credential. So yes, maybe zero knowledge PS provides a way to preserve privacy in this age verification case, but this does not solve the complex social problem because who determines what to block?

Who determines which websites are the ones that we're going to be blocking and why? Who determines who has access to these specific websites and why they should have access and who stops the system from expanding from later expanding to not only checking the age but also checking the gender, checking the legal status, checking your sexual identity. who will stop that and who issues this credential and who is excluding from being issued this specific credential. So as you see here, this is a really complex issue that like questions a lot of so social implications that yes, we might have privacy via zero knowledge groups, but that's only cryptographic privacy. Does that not prevent that the systems become a centralization process, an exclusion process and a discrimination process luckily there's a lot of standardizations thinking about it.

So there's some conversations at the IDF and W3C that what they try to create is a system that preserve privacies but at the same time that that enhance exclusion or censorship in the press in the process. But this still begs the questions do we even need this kind of systems? Yes, CKS are powerful but they cannot fix bad policy or harmful ecosystems or absent revocation models or cohesive environments or bad governance. So coming back to our first slide in which we talked about um these uh the archives of terror essentially by providing kind of like these age verification checks or these gender verification checks or these sexual identity checks or these legality checks what we're essentially providing is such a central central uh we're providing to certain centralization authorities while providing them with more data. We're providing them that now the user is in this location that they have been issued this specific credential by this specific government and they are also on this specific age bracket and what history shows and it as is shown in Argentina and Uruguay in Chile and in Paraguay and many of the other places in Latin America because all of them are happy is that the more that you provide data to a government the more that that can to a central authority the more that that data can be used later in order to enact violence to enact censorship to enact torture and even to enact disappearance.

So we should be very careful to think that maybe yes cryptography can preserve violent preserve privacy but does it mean that now it's integrated in a system but that by design is also providing centralization and exclusion and more data to be collected. So with that just to end get involved in cryptography because we need more people as Latin American or from underrepresented community that can bring this concerns to a standardization bodies and also to research bodies and say like hey maybe think on this problem on this specific consequence of putting this technology in production and we need people that think on design on cryptography and security with people first. So meet the people you're designing for for. Expand the threat models with live experience. Respect the complexity of social systems.

Don't over promise what cryptography can do or cannot do. Engage with the community and policies and think that this kind of boring cryptography matters but also it matters to create and expand the threat models that we have of different systems and also to not over promise that this cryptography is going to be preserving privacy but at the same time it's going to provide a system for centralization and formation. And with that, I invite you to join all of these communities. If you are from an underrepresented community, especially if you are Latin America and you want to get into cryptography, contact us. We have a group that is called Crypto Latinos.

And we'll be happy to also to get you involved because we need more people from different regions that bring this kind of problems into the table. And also say like, hey, if you design a technology, then think about the region of the world that you're designing for because whatever problems you have maybe are not the same equally across the globe. And with that, thank you very much. I think on on my time if there are questions I guess I'm happy to take them. [applause]

[groaning]