Video and transcript: Fireside chat with Clem Delangue, CEO of Hugging Face
Video and transcript: Fireside chat with Clem Delangue, CEO of Hugging Face
We discuss Hugging Face, Open Source, and AI
Transcript of talk below video
ELAD GIL
Hello. Thanks everybody for coming out tonight. It's a pack house so I think we had something like 1000 people who wanted to attend and so I think people are both very excited to see Clem and then I think there's ever growing enthusiasm for AI. So thanks so much for making it. And I'd also like to quickly thank Edwin Lee, ali Pavillon, emily The Stripe AV event Security food and Catering team. Thank you so much for putting on this event tonight and hosting everybody. We're going to be talking about Clem's background and origins and so I'll keep the intro really brief, which is Clem is the CEO and co founder of Hugging Face, which is really one of the main pieces of infrastructure that everybody uses in the AI industry. He's been working on AI for about 15 years now, originally from France, has been in the US for about ten years. And so welcome and thank you so much for joining us today.
CLEM DELANGUE
Thanks for having me. Excited to be able to chat.
ELAD GIL
Okay, and so could you just tell us a little about the origins of Hugging Face and how you started working on it, what it was originally, how it morphed into what it is today, and how you got started.
CLEM DELANGUE
Yeah, absolutely. As you said, I've been working on AI for quite a while. Before it was as sexy, as hot, as popular, as mainstream as today. And I think that's what gathered our cofounders with three co founders for hugging face around this idea that it's becoming kind of like a new paradigm to build technology. And we were really excited about it. When we started the company, we wanted to work on something that was scientifically challenging, because that's the background of one of our co founders, Thomas, but at the same time, something fun. And so we actually started by building an AI tamagotchi, something like a chat GPT, but really focused on fun and entertainment. At the time, there was Siri Alexa, but we thought it was pretty boring to focus only on productivity answers, and we actually did that for almost three years. We raised the first precede seed on this idea. Some users really liked it. Actually. They exchanged a couple of billion messages with it, but kind of like organically, and I can tell the story later. We pivoted from that to what we are right now, which is the most used open platform for AI.
ELAD GIL
What got you interested in AI to begin with? I mean, you started 15 years ago working in the area, and I feel like AI has gone through different waves of popularity.
CLEM DELANGUE
Right.
ELAD GIL
We had Alex Net sparked a lot of interest there's a CNN and Rnn world. Did you start even before that, or when did you first get interested?
CLEM DELANGUE
Yes, at the time, we weren't even calling it AI or machine learning. The first startup I worked for was a company called Mood Stocks, and we were doing machine learning for computer vision on device. So we were building a technology to help you point your phone at an object and recognize it. And even at the time, it was kind of like mind blowing what you were able to do with it. I remember, I think for me, the realization of how AI could really unlock new capabilities is when I met the founders of this startup. I was working at Ebay at the time, and they told me, oh, you acquired this company called Red Laser that is doing barcode recognition for you to recognize objects and then kind of like, pull up the Ebay page. They told me, you guys suck. You should use machine learning. And instead of recognizing the barcode, you can actually recognize the object itself. I was like, you're crazy. It's impossible. You can't do that with kind of like traditional software. You can't do that with code or too many objects. Possibilities are just too broad to do that. And they were actually managing to do that with some form of machine learning at the time. So that's when I realized, wow, you can do so many new things with this new technology. Apple really led me to where I am today.
ELAD GIL
That's cool. So you then started hugging face. You're going to do like an AI tamagotchi. And I think it's funny how you used to say AI and people would sneer at you and they'd be like, no, it's machine learning. Right? And so I feel like the lingo has shifted back to AI again, given what some of these systems can do, and then what made you decide to move in a very different direction of what hugging faces?
CLEM DELANGUE
Yes. It was very organic with one of these founding moments. It's a good thing that we had stripe because I think it's Pat Kodisan who talked first about the importance of not just founding a company, but having founding moments that change the trajectory of your company. And for us, that happened thanks to Thomas Wolf, one of our co founders, who I think it was like a Friday, Friday night. It was like, I've seen this thing called Birds that was released by Google, but it kind of sucks because it's on TensorFlow. I think I'm going to spend the weekend porting that into Pytorch. And we're like, yeah, you do. You have fun. Have fun during your weekends. And on Monday he came back and it's like, okay, I'm going to release it. And he released it on GitHub, tweeted about it. And we got like 1000 likes, which for us at the time, we were like, Nobody's French. Nobodies. We're like, what's happening? Why are people liking this very specific, very niche, very kind of technical tweet about Pytorch port of bird. There's something there. So we kept kind of, like, exploring that. We joined them, started to add other models to the GitHub repository. And the community came together. People started to fix bugs for us in the repository. We're like, Why are people doing that? They started adding models. Right? They started, for example, the first GPT. They added the next models that were released, and really fast. We ended up with one of the most popular GitHub repository for AI. And that's kind of like what transitioned us from this first idea to where we are now.
ELAD GIL
Okay, and could you describe for people who I'm sure most people know, but could you describe for people what hugging faced us today and how it's used and the importance of the product and the platform and the ecosystem?
CLEM DELANGUE
Yes. Now we lucky to be the most used open platform for AI. You can think of it, as mentioned before, some sort of a GitHub for AI. So the same way GitHub is this platform where companies host code, collaborate on code, share code, test code. We're the same way, but for machine learning artifacts. So there's been more than a million repositories that have been hosted on the hugging face platform with models, most of them open source. So maybe you've heard of stable diffusion. T five Burt originally, obviously, Bloom, for example, Whisper for audio data set. There's over 20,000 open data sets that you can use on the platform. And demos. Over 100,000 demos are hosted on the platform. And more than 15,000 companies are using the platform to bring AI into their features, into their products, or into their workflows.
ELAD GIL
Yeah. Some of the most popular questions on Dory or through the Airtable forum that people asked were around the future directions, because given the centrality of where hugging face is, there's so many directions that could go in everything from like, bespoke B, two B hosting, to tooling, to other types of products or activities. What are some of the major directions that you folks are pursuing currently in terms of product?
CLEM DELANGUE
I would say there are two main directions that we're following right now. One is, like, we're seeing that AI is turning from kind of like these niche techniques, solving some problems, to the default paradigm to build all tech. And for us, that means going from text that is really used on the platform right now, audio that is also really used, and text to image, to expand to every single domain, right? So, for example, last week we've started to see the first open source text to video models, right? We started starting to see on the platform a lot of time series models, right? Like, to do financial prediction, to do, like, your ETA when you order your urban. We're also starting to see more and more biology chemistry models. So kind of like making sure that we support these broadening use cases for AI is one, and the second one is making it easier for everyone to build AI, including software engineers. Historically, our platform has been more like designed for machine learning engineers and people who are really kind of like training models, who are optimizing models, assessing models. What we're seeing now, especially with the AI APIs, is that everyone wants to do AI, right? Even complex software engineers, product managers, infrastructure engineers. So a big focus of ours and some of the things that we've released in the past few weeks and that we'll keep releasing is kind of like reducing the barrier to entry to using our platform. Because ultimately, we think every single company or every single team should be able to use open source to train their own models, right? Everyone is talking today about chat GPT. About GPT. Four. But I think in a few months or in a few years, every single company is going to build their own GPT Four and they're going to train their own GPT Four the same way today. If you think of it, every company has their own code repository, right? And there's as many code repositories as companies. We think tomorrow every single company is going to have their own models, their own machine learning capabilities, not really outsource it to someone else, but really have these capabilities that will allow them to differentiate themselves, to cater to their specific audience or their specific use cases.
ELAD GIL
It's interesting because when you talk about the future, one thing that I'm really stricken by is if I look back over the course of my career, there have been multiple or a small number of very large paradigm shifts or platform shifts. Right. So there was the Internet, which was obviously a huge transition in terms of bringing everybody online. Then a few years later, we ended up with mobile and cloud. So suddenly you could host anything anywhere and simultaneously people could access any product from anywhere in the world. Crypto, I feel, was almost like a side branch that went down the financial services route but didn't become a true platform, at least not yet in terms of compute. And then now we have AI. And it feels like with each platform shift you have three or four things that change. Right. The input and output of how you program a system shifts in some ways, or at least the types of data you deal with, user accessibility and UI shifts. How do you actually interface with something from mobile was different from the desktop? And then the size and magnitude of the implications of that shift are massive. Right, and so if we view AI as a new platform, how do you view or how do you you mentioned everybody will have their own form of GPT four. It seems like the nature of programming itself may change at some point and we can put aside the whole question around do we also create a digital species? And maybe we talk about that at the end. But how does hugging face aim to play a role in terms of this massive transition of platforms?
CLEM DELANGUE
Yeah, the way we see things is we really like Andreish Kappati analogy of Software 1.0, which is the way and the methodology that we've been building technology with for the past 15 years. And now AI is Software 2.0. Right? It's a new methodology, it's a new way of building all technology. It's a new paradigm, the new default to build all technology. And if you think of that, you need for this new paradigm better tools, more adapted tools to do that. And you need better communities, you need ways for teams to collaborate and for the whole ecosystem to collaborate. And that's what we're kind of like trying to provide like a new tooling, a new collaborative platform to build AI better. And we're also trying to build a future that we're excited about. I think a lot of people are kind of scared about AI right now and the potential and the risks associated to it and the way we think about things. If you can build a future where everyone is able to understand AI and build AI, you remove a lot of these risks because you involve more people. So you reduce, for example, the probability of very biased systems. You give the tools for regulators to actually put in place safeguards and you give companies capabilities to align the systems that they use and provide to their users and their customers with their values. Right. Which is what you want. Ultimately you want Stripe to be able to say, this is our values, so this is how we're building AI in alignment with these values. So that's also something important that we're trying to do. We say sometimes that our mission is to democratize good machine learning and we're working really hard on that because we think it's important for the world.
ELAD GIL
Yeah, it feels like Hugging Face has always been very consistent in terms of wanting to have ethical AI or ways to participate that are strong in alignment. A number of companies, like for example, Anthropic, has this approach of constitutional AI where they basically say, we almost provide a constitution as we train the model for what should govern the activities or actions of the model that results. What are the approaches that you think work best and what do you hope that people are doing more of relative to alignment?
CLEM DELANGUE
Alignment is this kind of like complicated terms because it means different things to different people. It can be taken from the ethical standpoints in terms of alignment between values and systems. A lot of people use it today as more kind of like accuracy improvement. To be honest, when they kind of do some alignment work, they actually make the models more accurate thanks to reinforcement learning with human feedback. So it's kind of like hard to debate around that. I think in general, in my opinion, you can't control, improve and align a system that you don't understand. So the main thing that we're trying to push at Hugging Face is more transparency in terms of how these systems are built, what data they are trained on, what are the limitations, what are the biases. And I think if you create more transparency around that, you can almost create a system that is more ethical at core. So that's kind of like the biggest thing that we're focusing on.
ELAD GIL
What is your biggest concern in terms of how open source AI could be misused or abused?
CLEM DELANGUE
There's a lot of things that can be dangerous with AI, however it's distributed through APIs or open source. The biggest thing is dual use when you want to kind of use it in a way that is not the right way, that Model Builders defines. And so one thing that we've been experimenting with, which is super early and probably not a solution to everything, is creating new forms of licenses for models. So we've been supporting something called Rail and Open Rail, which is responsible AI license, which is supposed to be an open license for everyone to be able to use the model, but that defines uses that are prevented from the Model Authors as a way to create kind of like legal challenges for people to use it the wrong way. That's kind of like one approach that we've taken to try to mitigate some of the dual use of AI in general.
ELAD GIL
I guess as you look at the world of open source versus closed source, one of the things that's really been happening is when, before many of the different industrial research labs, the Googles and the opening eyes of the world would publish a model, they'd actually also publish the architecture of the model. They'd publish a paper that goes in depth in terms of how the thing works. The original transformer paper was reasonably explicit, and now they're starting to curtail the amount of information that's coming out with each incremental model. Do you think that puts open source at a disadvantage or how do you think about the future, particularly on the large language model side? Because when I look at the image gen models, they tend to be reasonably inexpensive to train. They tend to be more open source heavy. And it really seems to be more along the lines of the foundation models where this could become an issue because of the ones that need massive scalability and compute. Are you concerned about the lack of publishing? That's starting to happen. And how do you think about the delta between open and closed source models for big foundation models?
CLEM DELANGUE
Yeah, it's definitely a challenge. I think it's good to remember that we got where we are today thanks to open science and open source. Every system that is around today is built stands on the shoulders of giants. Right? If there wasn't research papers for Birds, for Transformers, for 45, for GPT, maybe we would be like 50 years away from where we are today. I think that's what created this massive positive loop that made the progress of AI, I think, faster than anything you've seen before. And if we stop doing that, it's going to slow down. It's going to take more time, and we'll just kind of move slower as a field. But I think one thing that we're seeing is that life abhors vacuum. I think that's the proverb, right? So I think if some companies and some organizations just start to do less open research or less open source, what we're seeing is that other organizations will take over and actually reap the benefit of it. So, for example, we're seeing a lot of collective decentralized. Collective there's like a Luther AI that announced the nonprofit a few weeks ago. You have organizations like Allen AI in Seattle. You have organizations like Stability AI runway. ML you have academia that is coming back in the picture. Right. The original Stable Diffusion was built in German University in a group, a research group called Comvis using Stanford, doing more and more in open source and for open research. So I think ultimately that's what we're going to see. We're going to see different sets of organizations taking over and kind of like contributing to open research and open source because at the end of the day, it's not going to go anywhere, right? I mean, if you look at traditional software, there's always open source and closed source, right? And open science is not going to go anywhere because the goal of most scientists is actually to contribute to the society and not just to do something to make the company money. So I think that's what's going to happen. Maybe like the types of companies that are doing open research and open source are going to evolve, but I'm not too scared about it. One kind of like proof of that is that the number of models and open source models, number of open source data sets, number of like open demos on hugging face has been actually accelerating for the past few months. And you're right in pointing out that we're a little bit biased on text, right? That's one area where proprietary is ahead of open source, right? Large language models. But if you like look at audio, kind of like the best things are like Whisper for example, thanks to OpenAI that is open source. If you look at text to image, stable diffusion is huge and probably like bigger than any puppetry system. If you look at biology, chemistry, time series, also open source is very powerful. So I think it's always some sort of a cycle, right? Sometimes puppetry gets ahead thanks to some companies that are doing a really good job. Like OpenAI is doing an amazing job, for example, right now. But sometimes open source catches up, sometimes it's going to be ahead, sometimes it's going to be a little bit later. That's kind of like a normal technology cycle, I would say.
ELAD GIL
Yeah, I think that's true. If you look at passive technology cycles, it looks like often the really successful large open source approaches that are offsetting commercial efforts tend to actually have a large commercial backer who wants to offset the activities of others. It's almost like strategic counter positioning. So for example, in the biggest sponsor of Linux was IBM because they were trying to counter Microsoft. And then if you look at a variety of open source mobile browsers, webkit is backed by either Apple or Google depending on the branch. Who do you think or do you think somebody will emerge in terms of becoming one of the major sponsors of open source? Does Amazon do it to offset Google and the relationship between Microsoft and OpenAI in the cloud? Is it nvidia is it Oracle? Is it a conglomeration of multiple parties? Or do you think a government or somebody else may intervene in this case?
CLEM DELANGUE
Yeah, I think there are a lot of big tech companies that have kind of like good alignment with open science and open source. You mentioned some of them. Like Amazon has been really good backer of open source. NZDA has been a very good support. Microsoft has been supporting open source a lot too. So yeah, I think some of it is going to come from there. I'm also excited about more governments involvement in kind of like democratizing access to compute, which has been kind of like one challenge for large language models. So when we trained with the big science group, a model called Bloom, which at the time when we released it was the largest language model that was open source, we got support from a French supercomputer called John Z. So I'm excited to see that more because I think if you look at how public policy and kind of like governments can have a positive impact, I think providing compute to universities or, like, independent organizations, nonprofits in order to avoid concentration of power and create more transparency is a very obvious way where they can have an impact and a positive impact on society. So I'm also excited about that, about this ability for public organizations to support more open source and open research in AI.
ELAD GIL
Yeah, it makes a lot of sense. I guess if you look at the types of open source, there's going to be models of various sizes. And to your point on the large language models, if you assume the rumor of the public estimates are if Gpt-3 took $10 million at the time, although I guess now it would be $7 million to train and then GPT four, say, was 50 to 100 if you were to do it from scratch. And then maybe GPT five is $200 million and GPT Six is half a billion or whatever it is you keep scaling up cost. And so you need these sort of large sponsors to at least be at the cutting edge of all times. But then one model behind may be dramatically cheaper. And so it's interesting to ask how that world evolves relative to government intervention or corporate intervention or other things in terms of sponsoring these models, with the.
CLEM DELANGUE
Caveat that we've seen that some scaling is good. We don't really know if that's the scaling that helps the current emerging behavior, to be honest. And that's one of the challenge of the lack of transparency that's happening right now.
ELAD GIL
Actually a really interesting question. What do you think are the basis for the emergent behavior and what do you think is the biggest driver for scale going forward? Is it compute? Is it data? Is it algorithms? Is it something else?
CLEM DELANGUE
I think we're starting to realize and have a better consensus in the science community that data, and not only the quantity of data, but the quantity of data is starting to matter more than just blindly scaling the compute. But I think also something that gap like is important to remember is that training a very good large model today is still very much an art. And it's not just a simple recipe of saying like, you have good data, you have a lot of compute. You're going to get a good model. It's very much still like a very difficult, very hard to understand technical endeavor. It's almost like alchemy that a very small number of people really manage to do today, right? And maybe it's like 20 people in the world today. Maybe it's 50 people in the world today. It's a very small number. I think people sometimes don't realize that. And so I think there's a lot of progress also to be made on understanding the techniques to get to a good model almost independently of compute and data.
ELAD GIL
Why do you think it's such a small number of people?
CLEM DELANGUE
It's a billion dollar question. If if it was easy to to know, I think everyone would would be doing it. I think I think interestingly. It's a mix of technical skills, science skills, and kind of like almost projects management skills, which are kind of like unique. It's not just a matter of doing the right training, but it's kind of like knowing how much more training you want to do. It's a matter of kind of like understanding when you want to release things, when you want to keep doing kind of like optimizations before launching your training run, when you want to start the big six months, three months training run, or where you should kind of keep experimenting. So it's a mix of all of that, which makes it super hard, but super fun at the same time, right? If it was too easy, wouldn't be fun, but hopefully it gets easier and it gets more democratized so that everyone can kind of take advantage of that, reap the benefits of that, learn from that and then, as we said before, build like better systems for each organization.
ELAD GIL
Where do you think are the most exciting areas of AI research right now? Or where do you wish more people were working?
CLEM DELANGUE
I'm super excited about it's fun to do like text, right? And I'm just here for short periods of time. So I went to a couple of Acathan and there are some really cool stuff. But I think it's interesting and important to work on more technically challenging problems right now, especially in the other domains. Like, I'm super excited about biology. How do you apply AI to biology? How do you apply AI to chemistry? To kind of both have positive impact in the world, but also to differentiate yourself and kind of build a more technically challenging stack for AI. So these are some of the things I'm excited about right now.
ELAD GIL
And then how do you think about I feel like there's two views of the world and maybe neither is fully correct in terms of general purpose models versus niche models. Right? So some people are making the argument which is you just keep scaling up models, you make them more and more general and eventually they can just do anything. And then on the other side of it, people are saying, well, just do the focus small model that is targeted to the specific thing that you're trying. To do with the data set that you're trying to do. It can be highly performant and you don't need to wait for the big generalization. Where do you think we'll be in three or four years?
CLEM DELANGUE
Yeah, that's a good question. I've tried to stop doing predictions in AI because it's too hard these days. Like I say something three months, three months later, it goes completely the other way around and I look like a fool. So I won't do too many predictions, but I usually try to more like look at the past and data points. Since Chat Gpd got released, companies have uploaded to hugging face over 100,000 models, right? And I don't think companies like train models for fun if they can use something else, if they don't need the training, they would. And an interesting, other interesting data point is that if you look at all the models on the hugging face hub, the most used ones are actually models from 500 million to 5 billion parameters. And I think the reason why is that when you get kind of like more customized spatialized models, you get something that is first simpler to understand and iterate on. You get something that is faster most of the time, which sometimes can run on device on your phone or on specific hardware, something that is cheaper to run and actually gets you better accuracy for your specific use case. When you specialize it sometimes for some applications when you're doing a chat bot for customer support, where customers are asking for your last invoice, you probably don't need a chat bot to be able to tell you about the meaning of life and the weather in San Francisco. You just need it to be really good at your specific use case. And what we're saying is that having a more specialized, customized, smaller model for that usually is a better fit. But there are some use cases, like if you're being for example, and you want to do like a general search engine to be able to answer all these questions, obviously, like a large more general model makes sense. Ultimately, I think there's going to always be all sorts of different models the same way there are all sorts of code based, right? Today you don't really say oh, my code base is better than yours. You don't say like Stripe code base is better than Facebook code base, right? They just do different things, right? They answer different problems to different questions. The same for models, there's no one model that is better than others. It's more like what model makes sense for your use case and how can you kind of optimize it for your specific use case?
ELAD GIL
The last set of questions I wanted to ask before we open things up to the audience is around business models and business opportunities. And Ali, the Cofounder and CEO of databricks, has this really good framework for open source where he says with open source. You first start out with some open source software and just making that work is like hitting a grand slam in baseball. And then you put down the baseball bat and you pick up a golf club and you hit a hole in one to have a successful business. So it's almost like you need two miracles in order to build something amazing and open source, sustainable as a company as well as a product. How do you think about Monetization of Hugging Face and what are some of the directions that you all are going for that?
CLEM DELANGUE
Yeah, I don't know if I totally agree with this analogy because I think open source also gives you superpowers and things that you couldn't do without it. I know that for us, like I said, we are the kind of random French founders and if it wasn't for the community, for the contributors, for the people helping us on the open source, people sharing their models, we wouldn't be where we are today. Right? So it also creates new capabilities, not only challenges for us, the way we've approached it is that when you have kind of like an open platform like Hooking Face, the way to monetize is always some sort of freemium model or some kind of like version of a freemium model. So we have 15,000 companies using us right now and we have 3000 companies paying us to use some of our services and usually they pay for additional features like enterprise features, right? Some companies, they want security, they want user management or they pay for Compute, like they want it to run on faster hardware, they want to run the inference on the platform, they want to run the training on the platform and like that. We found kind of like a good balance where if you're a company actually contributing to the community and to the ecosystem, you're releasing your models in open source, it's always going to be free for you. And if you company more like taking advantage of the platform then you contribute in a different way, you contribute financially. Right. By helping us monetize and keep working on this. We're still early on that, but we've kind of found this kind of differentiation between the two that allows us to keep working for the community, keep doing open source, keep contributing in alignment with our values and what we want to do, but at the same time make it like a good business, a sustainable business that allows us to scale and grow our impact.
ELAD GIL
You mentioned the community a few times and I think Hugging Face is one of the most beloved products and communities in the AI world. Were there specific tactics you took to build out that community or things that you felt were especially important in the early days? Or how did you evolve something that's so powerful from a community basis perspective?
CLEM DELANGUE
I would say just the emoji, having the hugging face emoji as a logo as a name. That's all it took to get the love of the community. No, it's hard to say. We're really grateful. Some of the things that we've done that we've been happy with is that we never hired any community manager. It's a bit counterintuitive, but it led to actually every single team members, every single hugging Face team members to actually share this responsibility of contributing to the community, talking to the community, answering to the community. Instead of having a couple of team members and instead of having researchers being like, oh, I'm not going to do the community work because we have this community manager. So, for example, our Twitter account, the hugging Face Twitter account, everyone in the company can tweet from it. So if you're seeing the tweets from the hugging Face Twitter accounts, it's not from me, it's not from like a community manager. It's from any of the hugging Face team members, which was kind of like a bit scary, scary at the beginning, especially as we grow. We haven't had any problem yet. But I apologize in advance if at some point you see like, rogue tweets that might be a team member.
ELAD GIL
But yes, it's a smart approach to always be able to blame someone else for exactly.
CLEM DELANGUE
Maybe it's going to be me, actually who's going to be doing the bad tweets, but I'll be able to say it's an intern or something like that.
ELAD GIL
Yes, you must have a lot of interns just in case you kind of yes, that's smart idea. Last question for me. And then we'll open up to the audience. What do you wish more startup founders were working on? Or where do you think there are interesting opportunities for people to build right now?
CLEM DELANGUE
So I'm a bit biased on that, but I wish more startup founders were actually building AI, not just using AI systems, because I think there's a big difference. The way I see things in the early days of software, you could, you know, use an AP, you could use an API, you could use like a Weeks or Squarespace or WordPress to build a website. Right? And that's that's good. That's kind of like a good way to get something up quickly and you can do beautiful things. But I think the real power came from people actually writing code and building technology themselves. And that's how you get kind of like the power out of this thing. It's kind of like the same for AI, right? You can do something quickly. I think ultimately, if you really want to be serious about AI, you need to kind of understand how models work, how they're trained, how you can optimize them. And that's also what's going to unlock the most potential for truly great startups and great products and companies that are differentiated from incumbents, just like adding some AI features. And we're seeing a lot of these companies like Runway ML announced the release of their text to video, I think today or yesterday. That's a good example of a really kind of like AI native startup that is really actually training models, building models, really kind of like doing and building AI, not just using AI. So that's one thing that I usually recommend startups do. Or if you're just using AI, just build your company accordingly, knowing that your mode or your advantage, especially dirty stage, won't be so much on the technical capabilities, but more on getting customers or getting users or any you wrote a beautiful, very good article that I would recommend everyone to read on modes for AI. They need to then kind of take advantage of other kind of modes than, in my opinion, like technical modes.
ELAD GIL
Okay, great. Let's open it up to the audience. If there are any questions, maybe we start in the corner right there.
AUDIENCE MEMBER
Hi. You mentioned something about open sourcing thing. It's good because you give it to good actor and also bad actor can use. But good actors probably are. We have more good actors. But how do you usually respond to claims like OpenAI that they don't say any details about their models, they don't open source anything because they're afraid of AI safety.
CLEM DELANGUE
I respect everyone's approaches like different organizations have different ways of seeing the future or the current way technology is building. I have a bit of a different view. Right. For me, if you look at development of technology, usually the biggest risks come from concentration of power and the fact that some technologies are built, like behind closed door. And if you build things like in the open, you actually create a much more sustainable path in the long run for this technology to be embedded in society in general, right? For regulators to be able to create the regulatory framework for these technologies, for NGOs, for civil society to be able to weigh in. So, yeah, I think we're starting from a very different position, philosophically speaking. But that's not too much of a problem, in my opinion, for the ecosystem. You can have different organizations with different points of views and kind of like the most important thing is just that your company doing are aligned with your company values.
AUDIENCE MEMBER
I wanted to ask a question on privacy. You mentioned data is integral to improving models. A lot of end users struggle with This paradox of this paradox of how do they preserve the privacy of their data while improving the models? Is in housing open source models the solution or approaches like Federated Learning? Appreciate your thoughts.
CLEM DELANGUE
We've been working a little bit on kind of like more like distributed or decentralized training, which is still hard to do. And I think nobody has really figured it out yet. But that's when you asked me about my interest in the science progress. That's one area where I'm really excited about to see more people working. But yeah, the more like, practical answers and solutions today are on device models, or there are also some more solutions that are embedded in how you train models. So, for example, we're leading an initiative called Big Code, which is releasing some, I think it released a few weeks ago, the biggest open repository of code that people can train code models on. It's called the stack. And the interesting thing about it is that we gave the ability to opt out from these data sets before training the model. I think you've seen last week, the training of the Adobe model that also have been really good at training on good data, where users have actually opted in for the training. So these are also some, I think, important developments in the field where you want to be a bit more intentional about the data or more transparent about it. One of the challenges is that a lot of these systems today, we don't really know what they've been trained on. Right? Because there's no transparency about it. I wish there was, so that we can kind of have a better understanding of what you're capable of doing with which data and then kind of find solutions to make sure it stays like privacy preserving for people. But there's some good development and I think we're making a lot of progress there.
ELAD GIL
Do you think we're going to end up with just like, those robots text right now for search? Do you think we'll have like, AI text or something for sites to be able to opt out of use in AI data sets?
CLEM DELANGUE
Yeah, probably. We'll need to have norms around that, for sure, around consent for AI. I think that's really important. That's really important, for example, for artists, for digital artists or non digital artists, that's important for attribution and distribution of value, right? Because we want people who are contributing to be able to be rewarded for it. An interesting question that I don't think has a good solution right now. But in a world where search is only kind of like a chat interface, what rewards kind of like the underlying creators of the content, right? If I build a website and before I was leaving because I was getting traffic on this website so I can do ads, and if now the results of this website is actually shown on like a chat answer without mentioning me, as a content creator, what's my incentive to create this content? Right? So will people just stop building websites because they basically don't get the attribution or the reward for it? These are very important questions. I think we're just scratching the surface of what needs to be done for things like that to be resolved, but very important questions.
AUDIENCE MEMBER
My name is DJ. I'm the founder of Sync, where AI we use AI compliance warehouses. These foundation models, they're very expensive to trade and that's often out of the region startups. So what are the modes that can be built on top of them? There's data, there's human feedback, there's data, there's human feedback, there's maybe to some degree being good at prompting. And are there any other methods that you feel startups can build? Modes?
CLEM DELANGUE
That's a good question. That depends a lot on your skills, your background as a team, what you're excited about. I think there's a lot to be built around specializing for a domain, a specific domain, a specific use case, specific industry, specific hardware. Right. That's what I'm most excited about, right. Trying to leverage some specific expertise or specific domain specific kind of like problem that others bigger players are not going to be focusing on. As I said, like for example, biology, chemistry, time series, all these domains where you don't see as much activity. I think it's a good way to have, in a way, more time as a startup to build your differentiation and your tech stack to a point where you're not at the mercy of 100 other startups, like releasing exactly the same thing as you did and kind of like losing your edge. So that would be some of my recommendations. But again, I've told about the story of fucking Face, right? How we started with AI Tamaguchi and ended up where we are. So one of the main things is just to start working, start building, listening to what you're seeing as signals. Iterate on things and I'm sure you'll land on something that you're excited about at some point.
ELAD GIL
Okay, great. I think unfortunately we're out of time. I think for the next 55 minutes. Feel free to hang out here. Thanks to Stripe for being such a gracious host. So this is also an opportunity for you to meet other folks who are very excited and working in AI. Thank you again as well. To clem is awesome.
CLEM DELANGUE
Thank you.
MY BOOK
You can order the High Growth Handbook here. Or read it online for free.
OTHER POSTS
Firesides & Podcasts
Clem Delangue: Hugging Face, Open Source, AI
Markets:
Startup life
Co-Founders
Raising Money
Old Crypto Stuff: