“We’re Losing the Web. Publishers Are Rewriting History.” - Brewster Kahle, Internet Archive

I'm Bruce Der Kale, digital librarian of the Internet Archive, a nonprofit research library trying to build universal access to all knowledge. [Music] Okay. Hi Brewster, thanks for coming. Um, yeah. Um so my first question will be just for people who don't know you to like quick introduction like what minute one minute like uh who are you and what you do?

I'm Bruce Jer Kale. I'm the digital librarian of the internet archive. Um the idea is to build universal access to all knowledge. That was the dream of the internet that I signed up to try to make happen.

Wow. It's beautiful. Um I really appreciate the things that you do. It's wonderful that we have you storing everything. But how did you like come up with the idea of making that I'm back in college 1980 uh tried to figure out something good you could do with technology and uh one was to try to protect people's privacy.

Um this was in the early days when cryptography was starting to become legal uh in the United States. Um but I found that was too hard. So, I went to plan B um which was to try to build the library of Alexandria for the digital age. Um and I've just been on that path and it's now been 40 some odd years and it's we're not there yet which is I find crushing. We s so could be there to have all the published works of humankind.

All the books, music, video, lectures, everything created for humans available to anybody anywhere wanting to have access. Um so I thought it was a technical problem. Turns out it was an institutional issue.

And what's at that? Can you explore it? It's kind of

Yeah. I mean technologically we've had the capacity to go and store these materials. I mean just the library systems in the United States is 12 billion dollars a year. The amount of storage that it takes to go and do all of this is just not that much. And even going and paying authors um out of that um in the traditional way of supporting I mean all of that's fine.

It's just these organizations, institutions that wanted to take advantage of the digital to go and create more power for themselves, more control, whether it's large scale publishers, uh really is a is a big problem. Um but also organizations didn't adapt. Universities have uh hospitals I we we've ended up also with not trusting uh very much. So the sharing is uh is impacted but it's um yeah I so still on it there's still more to go. Yeah and like um I wanted to talk about obstacles that you had to overcome.

We had a really long journey that I mean you have to face like with so many things like I don't know copyright with Google doing more or less same things and so so so many like what are the top challenges that you had to overcome during this long period of time

I'd say convincing people that in the early days it was convincing people that this was possible

I because people have been dreaming of the library of Alexandria for thousands of years um and when there's a new technology change, whether it's going to papyrus or to go to print um or now to digital, you can start to look think about all of something like how much is all ever written and um and it suddenly becomes kind of doable. So the the first challenge was just trying to convince people that it's a worthwhile thing to do. Um, but it's the we had to get people to see themselves in the picture. How do you make a game with many winners? Not so that there's just one gigantic corporation that owns and controls it all, but you can go and make it so that individuals want to participate.

They want to go and share. So there's a good news story and that that in October of this year the internet archive will have archived its one trillionth web page. One trillion web pages maybe by maybe a billion people. So people want to share. Um corporations are you know more about not sharing or defining crushing.

Governments are having trouble these days. Um but people are awesome.

That's nice to hear. We still believe in humanity. Hope people eventually will overcome all the bureaucracy and all the corporation. But uh how do you feel in general the change of like internet over this like 40 years or more like how it was back then and how is it different now? How do you feel?

the the biggest change really has been the consolidation and control. The uh the internet was designed to be a decentralized technology. Um and the worldwide web wasn't designed very rigorously. So the worldwide web has a problem that unlike in the publishing era where you have a publisher go and make things available, they'd sell things to people and to libraries and they would keep things going and the publishers would work on the next thing. um the publishers now think that they should own and control all of it

and then there's only one copy in one place. So if they change it, it's changed everywhere or delete it everywhere all at once. So we tried to put in a fix a cluge called the wayback machine by archiving the worldwide web and making a copy. So you could see things in the past where the publishers, you know, wants everybody to think something different. And um so this is but it's a cluge uh on top of a publishing system that in the paper era had much more resilience, much more um robustness to it.

you know publishers could go out of business completely and a publisher couldn't just go and say oh I want to change that book and it suddenly has changed in all libraries all at once you couldn't do that in the paper era in the electronic era they it can and it happens all the time so now that we have governments and the corporations going and trying to control what people think they're pulling things away and unless there are libraries and archives then you can't do this and so we need lots of them. So we wanted lots of libraries and archives to start archiving the worldwide web and it's been challenging. So I just went to the Netherlands um um national library and they they don't think they have the rights under the law to even collect Dutch websites.

Oh wow. And they don't even think they have the right to collect Dutch websites in such a way that you can go and see those Dutch websites in one room during right one hours on one terminal that you can't bring a camera in to see. They can't do they don't think they can do any of these things. So the laws have been pushed so hard by the publishers to make it so that libraries even the national libraries can't do their jobs.

Yeah, we all see this direction that things is going now. Everything is getting too restricted. But question here, how can we get bags a state of internet head?

Is that even possible?

Absolutely possible. We still have the technologies to make a game with many winners. How do you make it so that there isn't centralized points of control that will make it so there's a gatekeeper of either what's being said or where all the money goes? It all ends up with them. Um it is possible but it requires coordination and needs um actually often government law help.

Uh one of the things that I'm really surprised about my career um starting in the 1980s was the um the governments allowing corporations to grow gigantic and they it's easier to stop the companies from merging than it is to try to get them to break up after the fact. but they just haven't. They just keep allowing them to merge and become bigger and bigger and bigger. So, how do we make a decentralized web

again? How do we do that? How do we make it so that anybody can go and make the information available um under their own terms, even get paid for it um with privacy and robustness. And then how can we get so readers can find things no matter where they are even if they're not on some major platform. Do we need algorithms that are centralized and coordinated by large commercial players to make it so that words, videos, thoughts move around effectively.

And maybe we did at a time when it required mainframes and these big things, but now we know the techni techniques and we can do things like blue sky that are not that many people to run it. It's not thousands of people at Twitter uh that are at blue sky. Um so we can we can do that as long as people feel that they're not being taken advantage of. they're not being shafted by some company, by some government watching um from corporations surveilling. So we need privacy, we need trust.

Um we need a a sense of sharing and building something together. Um absolutely possible with the technologies of JavaScript and the browser allows your computer to be a computational node just by clicking on a web page. It's kind of awesome. Um, we have abilities to move beyond the cloud-based services to have more decentralized compute services. Um, I'm excited about making um web servers again that can be on your home machine, right?

you know, can you just go and download something and then make it private so that you can't be hunted down for making information available and the people that are reading your uh website aren't feeling like they are being tracked and we have the technologies with tour. Um we have um uh alternatives to the DNS system which is currently being attacked by large corporate players to go and try to knock people offline. um we have mechanisms of building around the centralization we mistakenly put into the internet infrastructure the first time

and what do you think is the role of web 3 here and like in general your take on web 3 and its possible role in like making internet great again. The two things I really want out of web 3 is a mechanism of doing uh registration of a next generation DNS. Can we go and register names for these large hash addresses of of things like tour uh onion services? Um can we go and make it easy to refer to them? And that's not just a easy like drop it on a chain.

there's going to be contention between people so that Coca-Cola or Internet Archive is not hijacked. So, how do you do that? And that I think is going to require some good strong thinking. Another uh piece that I think we can do well is make decentralized e-commerce. Can you make it so that people can make money by say publishing their songs on the net by going and having end users be able to go and maybe hear a sample, but then be able to buy something with some currency, probably a cryptocurrency, a stable coin or whatever, um, and then put a coin in the box and be able to get an unencrypted version of the song.

So, yes, you could go and take this unencrypted thing and go and put it out there. Yeah, but that's dorky. I think what people want to know is if they're going to buy something, it's the money's actually getting to the band or the singer or the And right now there's so many middlemen that I think people have kind of given up on the you know that you hear these stories of these authors making right and certainly a lot of money goes to the publishers and the collecting rights agencies but it doesn't get through to the authors the singers. So can we make direct connections between people um in a privacy uh enhanced way so that you can do free you know information sharing but also for pay that's what I think we could get uh from the web three community if there was sort of a bright north star picked out and made it crisp say if we solve this puzzle then a lot of good things would happen.

Yeah. Yeah. It sounds good like the simple thing is I don't know to have a web three version of Spotify right which is here which is popular and so many people use it and then like so so much money is taken by the middleman without any reason and then people just suffer

um and uh yeah actually there are these subscription services like Spotify I think it would be great to get back to actually buying things

like you buy things like I give you some money, you give me a file,

and you don't give me a license,

you give me a receipt,

right? And then I have that. And yes, I could be a bad player with it, but let's make it so that I now own that and I can keep it, preserve it, do the traditional things that we used to be able to do when we bought physical things. The big publishers now say there is no such thing as buying anything. everything is theirs forever and they can go and surveil and watch what's going to happen to it forever.

I don't think that's a great way to go. Maybe Spotify kind of thing is a good way to discover, but then you say, I like that band. Why don't I buy things from them and then the money goes to them? Maybe 80% 90% of the money that I would spend would actually go to the people. Wouldn't that be a nice welcome change?

Yep. Yep. Yep. some Yeah. like in good old times, buying CDs,

buying CDs, DVDs, MP3s, records, um going to a band and and um seeing them in a performance hall and know that it's not being skimmed by the ticket company. So, you just it's can we know that we're actually supporting the people we'd want to support?

That will invigorate people to really participate on both sides. Um, but in the digital world, there's opportunities for leeching off of these systems, spying onto these systems, um, that has been taken advantage of. That's the big thing that I think that we've ended up with in the last 40 years, which we could do with a let's hit a redo button and fix some of these things.

Yeah. There are so many things that are broken but like I wanted to ask you about the generational shift or gap like how do you feel from like when you have studied uh your journey in web like what was the generation of engineers and developers and hackers back then and how do you see people now and maybe what can like younger generation learn from the older ones from real cypher punks of the 80s. Oh, I think that we we got a lot of things wrong at every stage of the way. Um, but at least when I got going, it in the United States, um, back in the 70s and early 80s, um, you didn't sign your code. At MIT, when we were building the list machine operating system, you didn't even sign your code because it was thought of as an arrogant.

Oh, wow. So, it was all like common.

It was all commons.

Wow. And then uh when the United States followed the European system of copyright, they made everything copyrighted and forever. So everything expressed was copyrighted. So MIT said, "Well, if everything's copyrighted and owned, then we own that so we could fork it." Mhm.

And they destroyed an operating system called the list machine operating system which pissed off Richard Stolman so much that we he went and built the free and open- source software license uh which was an an attempt to get back to the more commonsoriented structure that we uh operated in. So, I think there's been mistakes and then there's been counters and mistakes and counters, mistakes and counters. And right now, we've got a mistake in way too much licensing

um and not enough buying. Um what I what is good is the amount of sharing people are up for doing. So, open source software is actually doing quite well, right? in terms of GitHub there's just a lot of people see it as their career is going and sharing which is kind of great. Um so open access journal literature is doing well.

Um the book industry is not doing well uh in terms of sharing. They just make it so fewer and fewer people see it. Um some of the mistakes that we see um is the Wikipedia um is a glorious project. Um but it's thin, right? And you need to go and click through to the references, but the references in Wikipedia, they're biased towards things you can click on and see, which a lot of the publishers say you can't see their thing.

It's pay walls, it's it's restricted, it's whatever. So people are linking from Wikipedia to blog posts, propaganda, things that are paid news. Um, and that's weakening our infrastructure, our mental infrastructure. We we're not allowing this next generation to stand on the shoulders of giants,

right, of of the literature that's come before because we're denying it to them. So, we're getting a generation that's confused of what's true or not, whether the Holocaust happened, whether basic facts, um, is just being flooded with propaganda without the library being available. We can make a change to go back to this idea of let's get access to the materials to this next generation. This next generation is going to be brought up on AI on these liars language models and we're right now making us making sure the large language models don't have very good things in it. You can go and trying them and they're like, "Oh, they get confused too because they're just they're learning from what's on the web, right?

The common crawl. It's the sort of the the the default way to learn and it's thin. So, if we're going to bring up the next generation, not maybe next time on Google searches, but on AI, let's make sure this technology really works. Let's build European values. Let's build public AI.

Let's build AI technologies that reflect European values or your nation's values. Um, it's not that expensive. You don't need to put up new nuclear power plants to power um uh building all language models. So why don't we go and leverage this technology to try to solve new and different possibilities like what can we do in climate? There's a uh internet archive Europe is helping sponsor climate GPT.

So you take the best we know about climate because it's legal to do this in Europe to go and put that into a large language model that's a nonprofit for the public public AI.

So can we go and solve that? And we make large language models of small languages, Bulgarian, Finnish, so that it actually isn't just translated into an English model and then comes back out. It's actually all it knows is Finnish and it just knows the like the literature and the thinking and the way of thinking to increase the resilience and and the diversity that for instance is the European uh treasure, the European tradition.

Yeah. 100%. Um, I really like that you made this hot take on AI because it was also one of my questions like how is it going but uh my next question will be about the cipher punk like you've seen it emerging and I don't know if you notice it now but this there is some sort of revival of cypher punk at least in web 3 space. So, um, what do you think about cypher punk in general? And what do you think it really means to be a cipher punk nowadays?

And how is it different from the 80s as well? At least what I'm seeing is a lot of applied cryptography to try to support very humanoriented tasks. um where maybe there was this much more sci-fi uh idea of cipher punk back in the in the 80s uh which might be a lot more detached from real social problems. Um I think that we're seeing people trying to use these things to protect people's identities, try to make it so that voting works better. Um can we use our phones to do voting and have any idea that it's going to actually work?

Um so we're seeing basic societal infrastructure um having trouble especially in the United States right now um and so a turning to those that understand these technologies to build very humanoriented systems so that we understand them they could be adopted and adapted by lots of people um that's some of the things I'm seeing out of the decentralized web community, some of the web 3 uh groups. Um, and it's not just all about, I don't know, making monies in NFTTS and the like. Um, NFTTS, the idea of registering digital ownership, I think, is a great idea, is terrific. Do we need more pictures of gorillas? I don't I don't think so.

Um, so I'm hoping that these technologies have sort of adapted from being rebel outlawed when cryptography was outlawed in the 70s and 80s to being really applicable towards helping people live a life that they kind of want. They just want to be able to talk to their friends and not have it be watched. Um, they want to be able to buy things in stores without it all being creepy. um they want to be able to move around without always being watched. Um how can we build these systems and that's what I'm seeing out of some of the this generation of the cryptography experts.

Yeah, that's a great take. Um I wanted to ask you maybe some recommendation for builders especially for builders who are trying to do a nonprofit organizations in web web three web two doesn't matter because it's a tough road right it is and um like maybe you can share the challenges you had to overcome or maybe lessons learned from your journey of building like nonprofit something

I'm a big believer in the nonprofit um that the 501c3 structure in the United states um has been a really good idea in the United States. There have been a lot of bad ideas out of the United States, but that I think is a good one. It it means that you don't have equity ownership. So, you don't have people feeling that they own and control other people's lives, right? There's it's owned by the public like uh the GNU public license of software.

It's it's it's the public's um so the internet archive. Um yeah, how do you get people going and funded in a particular direction? There are now more foundations and individuals that are funding uh these um you won't get rich, but hopefully you'll just get enough to keep going. Um, so it's it I like the nonprofit structures that they tend to orient themselves towards making everything as available as they can. So the internet archive or Wikipedia by being openly accessible, it shows that we're vulnerable and people step up to support.

So, the Internet Archive gets about 200,000 people every year going and hitting the little I'll donate button. It's kind of great. And that's about onethird. Then there's about one-third as major donors and about a one-third is is um earned income.

So, libraries pay the internet archive to collect web pages or digitize books. We work with 1,300 libraries and that helps build these collections. the libraries are perfectly welcome to and we try to ask them to take their data back and and store them in other places too. So it's not um it's not building sort of a moat it's just trying to build an ecosystem and by

as if you have an idea that pe other people want to have happen often they'll help you do it. Mhm.

The idea of being secret and having, you know, the whole venture capital sort of, you know, secret secret and then springing on the world and that you own things and um you try to make it so people can't get out of your system. Those are antithetical to the nonprofit uh structures. Like to see more of those in more parts of the world. A lot of the nonprofit laws not as well developed in other places. Um, Internet Archive Switzerland is taking 18 months to to to start.

That's too long. Um, so we I think there is a good idea here is based on public ownership that we could support if we could get more public entities like real governments going and funding some of the technologies that they use um for the public good. Um Ben Ser uh Cerviny has this idea of the foundation of public code which is another level up from open- source software. How do you build the governance structures and the funding structures for among public entities to go and build the apps that they all need? Every city needs the same apps.

Do you go to deoids to do this or some local developer or do you work together to build not just the modules

but a supported good service? What does it take? And we now know like signal is a really good service but it you know Brian Actton put in $50 million that was like that was a lot of dollars um to go and get it so that it was a quality level. Um so I think we have to understand that these things aren't always cheaper to produce there. Um they are as expensive but they can benefit many many more people.

So I think the nonprofit infrastructures, nonprofit foundations, individuals participating, governments participating, we can build bedrock of open access literature of code, a society worth living in. Uh talking about the society worth living in. Imagine we are I don't know 10 15 years from now and we have this bright future. We all imagine private, secure, distributed, whatever. How would it look like?

What's this perfect web of the future and how our our life will change if it's here? I think we'll have a a a wider feeling like you can win that, you know, you're graduating high school, that you can get a job, not not just making physical things, but on on, you know, digital things, and you can make money, you can get um feedback positively from your community. um they can give you guidance as to sort of like no go tell us more about how to repair chairs. you're an expert in repairing chairs like write the next generation book or manual or vision video or you know I don't know VR thing whatever it is um that you can have a game with many winners so you can have it so it's welcoming to come into this world right now I think a lot of people are feeling like they're they're dealt a really bad future right whether it's climate or the surveillance or the corporations or the wars it just feels awful out there. What a terrible way to run a society.

So, uh, I want a game with many winners. I want these to be an empowering decentralized systems that you don't feel like you have to bow down to some corporate god or um these governments making unreasonable demands or, you know, having people be disappeared by masked men. Um that's just not a way we want to live.

Yeah. But like the question is like imagine uh we have this beautiful wonderful society like we won right and we have this web of the future with private decentralized secure and it's like a perfect web. And the question is like how do you think that this future will look like? like what's what's going to change when the internet will be private, secure, and decentralized?

I I I'm hoping that people will feel that they're not being taken advantage of, that they're not being run over,

that they're there's not some sort of spooky things happening around the corner, that um what you see is what you get. that uh you can be upfront with people personto person, but you could be upfront with people digitally and you're not going to be being lied to by an AI.

You're not, you know, you're not going to be just being surveiled to go and train on you. to hear you.

I I there was an interview that uh at the internet archive somebody was applying or something was applying for a tech job and as best I can tell that thing was an AI and it was just it completely freaked me out. I just like this is really not right. It's just seriously bad behavior. There's there are people behind that that think that the way to succeed is to cheat. Mhm.

Um that is not a great way to go. So let's make it so that cheating right isn't encouraged by our governments and our corporations. Uh and I think we have widespread cheating going on, lying, cheating, stealing by governments, by the corporations. Um, I guess I still have real faith in people and in in gatherings of people that know and want to work together and trust each other. We're not all cynical.

Um, so let's find that and grow it. Grow our communities. And those communities might be over long distances. So can you go and move from place to place? I've been spending a couple months in Amsterdam.

What a fabulous city. And I can still do that because I interact with people that work for the Internet Archive, US and Canada, Europe, all online all the time. Can we build that that internet infrastructure so that okay, I want to be in Amsterdam for the next month or two? Let's do that. Um, can we make that really easy, smooth, private, secure?

we can encourage people to learn more, be more, be healthier. Um, with this technology, if we do it right, if not, it's going to feel like 1984, you know, just like and where there's a boot on your neck all the time forever, right? That was the line out of out of 1984. So, let's uh

let's build a better future. Yep. Okay. So, Brewster, thanks for coming. It was a pleasure.

Thank you for your time and for a talk. Thanks for watching, guys. Ciao. [Music]