LlamaIndex: Unleashing LLMs on Your Data with CEO Jerry Liu

Full episode

Matt Turck [1:29] Welcome, Jerry. You are the co-founder and CEO of LlamaIndex. LlamaIndex is a data framework for your large language model application, basically an interface between LLMs and your data. And in just a few months, LlamaIndex has become a very popular open-source project and community. So as I was prepping for this, I was looking up some stats, and this is really quite impressive. So almost 19,000 GitHub stars, 227 contributors, 25,000 Twitter followers, 8,000 Discord members. So the kind of metrics that successful projects and new communities, fast-growing communities, tend to have.

Matt Turck [1:53] So congrats on all of that. $5 million seed round led by our friends at Greylock. I'd love to start with your personal and professional journey that led you to start LlamaIndex.

Matt Turck [2:20] Definitely, yeah. And first off, thanks, Matt, for having me. It's great to be here, and I really appreciate the invite. So I've been in AI for most of my career. I started off as a machine learning engineer at Quora after graduation. Actually, around junior or senior year of college, I started getting into GANs, the initial version of generative models. And if you're familiar with what GANs were, or generative adversarial networks, they were just initial deep learning models that could generate very small pixelated images of different things.

Matt Turck [2:53] So for instance, like faces or bedroom images or a lot of other stuff. And so when I first saw that, I thought it was magic. And I remember that was really the spark that got me interested in AI and deep learning in general, because I wanted to work in a field where I could also generate cool pictures. So I started diving a little bit more into my machine learning journey as a machine learning engineer at Quora. After a year, I then spent some time as a research scientist at Uber ATG.

Matt Turck [3:21] Where, one, I met my current co-founder, but two, I spent a lot of time diving into the foundations of machine learning and really learning a little bit more about model architectures and how things work, and applying them in a self-driving computer vision setting. Afterwards, I spent two years at Robust Intelligence, which was a Series B startup focused on testing and monitoring of machine learning models, pre-production and also post-production. And so there I learned about the intersection of how do you actually take all this cool AI technology, apply it in production, and make sure that they're functioning correctly.

Matt Turck [3:49] And also in the process, Robust was like—I joined as like eighth or ninth hire, and it grew to 40 or 50 people by the time I left. And so I got to be a part of that startup experience and really understand what it means to bridge industry and AI. And so I think all that kind of culminated in a variety of ways. And the way I discovered how LLMs operate and how to build apps on top of LLMs really occurred in a spontaneous manner.

Matt Turck [4:22] Because, in the end, I think this company is really a mixture of all my previous interests, but the way I actually stumbled upon this project was relatively spontaneous. I started off playing around a bit more with large language models and DALL-E. I've kind of been keeping track of the developments of transformers and all these models over the past few years, but really started just trying to hack on some apps back in November. And I think I remember, as I was trying to build applications, I really thought, wow, these large language models are not just capable of knowledge, but they're also capable of reasoning.

Matt Turck [4:53] And how can we adequately leverage these reasoning capabilities to really kind of operate over data that it doesn't necessarily inherently know about, but over your own personal data or over your enterprise data? For instance, ChatGPT, how do you get it to know about you as a person or you as an organization? And so I started working on an initial iteration of GPT Index at the time, which is basically designed around how do you organize data that the language model doesn't know, but in some sort of structure that it can access and retrieve.

Matt Turck [5:27] And that led to the initial iteration of GPT Index, which at the time was more of a fun design project to solve a pain point. But it became clear as more people became interested in the project and it got a little bit of traction on Twitter that there really was an opportunity here, that more and more people were starting to build large language model-powered applications, and that pretty much for the majority of these use cases, everybody was trying to figure out the best practices for how do you organize, retrieve, and store data.

Matt Turck [5:48] And so that really led to the evolution of this toolkit from an initial project designed to kind of augment language model context windows to this overall comprehensive data framework for building LLM applications.

Matt Turck [6:00] Great, great. So fast forward to today, what does LlamaIndex—that data framework that you mentioned—what are the different components, and what do they do?

Matt Turck [6:26] Yeah, so the high-level mission is really to connect your language models with your data and basically unlock the capabilities of language models, whether it's reasoning, agent-like planning, or also question answering, insight extraction on top of your data. And so to that end, we offer a lot of tools to both get your data in the right format, so kind of the ingest and storage part, but also the query side. On the ingest and storage part, we offer a lot of modules to allow you to connect to your various data sources, for instance, your PDFs, your APIs, your databases, and ingest unstructured and also structured data in the right format that you can then pass to your downstream application and also define additional stuff like metadata, annotations, those types of things.

Matt Turck [6:50] We then have—

Matt Turck [6:56] While we are on that topic of ingestion, so what is the right format? What does that mean?

Matt Turck [7:22] Yeah, so I could probably spend an hour talking about this, but at a high level, the trick is, I can talk about the kind of naive thing that everybody does these days and then the kind of considerations that you might need to think about if you're trying to build something more production quality. So just a quick overview of retrieval-augmented generation as a concept. You ingest some source documents. Let's say they're a set of PDFs. Now, the current stack is basically you take these documents and then you split it up into a bunch of text chunks, and then you store the text chunks in a vector database.

Matt Turck [7:53] And these days, there's a bunch of different vector databases, from Pinecone to Chroma to Weaviate to a bunch of others. These act as a storage layer, right? And so then when you actually, say you're building a question-answering system, you ask a question. What's going to happen is you're first going to do retrieval from your storage layer. You're going to hit the query interface of the vector database and do embedding-based retrieval from the vector database, because that's what it's specialized at.

Matt Turck [8:27] Then after you get a set of retrieved contexts, you put these contexts into the language model to actually synthesize an answer. So that's the overall stack of retrieval-augmented generation, as most people understand it today. And the kind of naive way to do a lot of these things is you just do some very basic text splitting. Let's say you split after every 1,000 words or so, or every 1,000 characters. And then you just throw everything into a single collection in the vector database.

Matt Turck [8:58] So what we're really invested in is allowing you to do the easy stuff out of the box pretty easily. But we also invest a lot in thinking about what additional data considerations you need to take on when you actually build a more production-quality LLM application around your data. And so the ingest parsing side actually is quite important because, one, we have more sophisticated text splitters than, for instance, just splitting every 1,000 characters. Second, being able to inject metadata actually is quite important.

Matt Turck [9:24] To actually improve retrieval performance of the downstream application. Because let's say you're splitting up an SEC 10-K filing into a bunch of chunks. Within a single chunk, it might lack the context of how this actually relates to the rest of the document. And so being able to explicitly define that, whether in terms of relationships or additional metadata, can help a lot. At a high level, we provide a lot of these abstractions and structures so that you can define these advanced annotations on top of your data, to allow you to define the right state.

Matt Turck [9:51] The metadata is just one piece of that, kind of like the ingest side. Now, the next few steps, if you think about the overall pipeline, are: you have this set of chunks that you get from a set of documents. Now, the question is, how do you actually store this in some downstream storage abstraction? We're not building our own vector database, but we have a rich set of integrations with, like, 12 to 20 different vector databases out there these days.

Matt Turck [10:12] Yeah, I saw that, which I thought was amazing. One, because you built so fast, but two, that there would be so many vector databases in the first place.

Matt Turck [10:32] Yeah. And so we really do try to be a framework that provides the right toolkits, right? So, allow users to kind of pick and choose the storage solution that makes sense for them. But also, we provide these advanced capabilities for orchestration and additional indexing on top of the storage abstractions. And so we offer support for both vector databases as well as existing object storage providers, like, for instance, S3 or MongoDB.

Matt Turck [11:02] And we allow users to configure the storage provider that makes sense. But the high-level idea is, you take in all this data, you've split it up, you've maybe added a bunch of annotations. Now you need to store it somewhere, and you need to be a little bit mindful about how you actually store it. Like, what are, for instance, the collections that you want to store this data in? How do you want to define indexes on top of this data?

Matt Turck [11:25] These are all tools that we provide. And so our ingest and indexing stage really is about getting a lot of this data in the right format so it can end up in the storage system in a way that can fit in well with the rest of your LLM application. And so that's probably one half of our toolkit, right, is really getting your data in that right format. The second half of that toolkit is, okay, now that you've defined the right state over your LLM application, how do you define the compute side, the query side?

Matt Turck [11:56] How do you actually leverage the language model as this reasoning engine on top of this data that you've now gotten into a certain format? And so from the basic retrieval-augmented stack, that's typically what happens. Typically, what you do is some sort of top-k retrieval from a vector database, and then you take in all this context and synthesize a response from that context. But what we allow you to do is do that part pretty easily, but also define more sophisticated interactions between your language model and your data as well to satisfy more advanced query use cases.

Matt Turck [12:46] Okay, very, very good. And you started alluding to this a little bit, but to help people who may listen to this make sense of where LlamaIndex fits in that emerging LLM or generative AI infrastructure stack, people may have heard of names like LangChain or Fixie or Dust. Are those competitors? Are they partners? Is that all still moving pieces that everybody's trying to figure out?

Matt Turck [13:12] Yeah, to some extent it's all moving pieces. I think there are definitely overlaps with certain frameworks, but there's also key differences. And so let's talk about, for instance, LangChain. LangChain is a great application framework for you to get a set of building blocks for a lot of different components, for instance, from the LLM modules to prompts to some basic retrieval and vector database abstractions to agent frameworks. We have, almost from the beginning, been very focused around the data.

Matt Turck [13:35] And so really what we think about is, how do you get your data in the right format so that you can use it with the LLM? And then also, how do you get the LLM to effectively query your data? And so there are some overlaps between that and LangChain, but we're very hyper-focused on developing deep tech around that and making that really good. And so, not just doing, again, the basic naive retrieval-augmented generation stack of text-splitting your data into some form, dumping it into a vector database in a single collection, and doing top-k retrieval.

Matt Turck [14:07] We offer that, but also a rich set of advanced functionality to do additional capabilities, combining LLMs on top of your data. And so we've almost intentionally made it so that we want to provide deep tech around this space. And really, we think we provide a very nice toolkit with a nice set of abstractions for both beginner and advanced users. But we also understand that if you want to build, for instance, things that are not explicitly within our framework, at least at the moment, for instance, stuff like Auto-GPT or agent simulations.

Matt Turck [14:46] But for those, maybe data is a component of that, we make it pretty easy to integrate with other frameworks like LangChain. We also, for instance, have an integration with Microsoft Guidance, the framework that allows you to have token-level control of interleaving prompting and generation. We're looking into integrations with other toolkits like Semantic Kernel, and also in conversations with Fixie as well. And so I think the high level is, there's a lot of pieces still moving.

Matt Turck [14:59] And for us, what we really care about is being thought leaders and providing a good toolkit in the data and LLM space. Great.

Matt Turck [15:16] And as I was looking through your materials and site, I saw that there are just a couple of things. It's LlamaHub and LlamaLab. I think you already covered some pieces here, but just to bring it home, what are those?

Matt Turck [15:41] Yeah, it's a good question. So we have an entire ecosystem of different projects that are extensions of the core repo. So the idea of LlamaHub, as it is right now, is to create a central toolkit for community-driven data loading abstractions. And so what it really is, is it's supposed to be an extension of the core repo. The core repo consists of some basic abstractions around data loading, indexing, and querying. And what we really did with LlamaHub is, let's make certain parts of this purely community-driven and just encourage everybody to submit a bunch of modules for certain components.

Matt Turck [16:20] And so data loading made a lot of sense because there's a long tail of different data loaders that you might want to have. You might want to load data from your Slack, your Salesforce, your Notion, different file formats that you have. And we wanted to open that up to community contribution. And so these days, LlamaHub is a very rich repository of 100-plus different data loaders from all different services and formats. And it's growing every day.

Matt Turck [16:35] And the next thing we're thinking about, and we'll have a release very soon, is expanding LlamaHub into becoming a community-driven repository, not just of data loaders, but other aspects of the toolkit too.

Matt Turck [16:37] And what about LlamaLab?

Matt Turck [16:51] LlamaLab is almost just like a fun repo for experimental projects. And so we launched it a few months ago, mostly as, I think, this is when BabyAGI and AutoGPT came out, and it was a way for us to just play around with, here's, like, if you want to have some sample implementations of agents, here's ways that you can both build that with LlamaIndex as well as use different components of LlamaIndex within that ecosystem.

Matt Turck [17:19] It's more experimental and more of a fun showcase of what we can do, which is why the contributions there are probably a little bit more sporadic. But it's something that, as new fun project ideas come in, we'll certainly add to.

Matt Turck [18:00] Going back for a second to LlamaHub, thinking of parallels with the world of data infrastructure, as you know, there's this whole world of ETL or ELT companies on the one hand and orchestration companies. And all of those are—you know, entire companies, $100 million-plus ARR companies, are being built solely on building those connectors. Do you anticipate that it's the same level of complexity over time? And do you believe that long term, you can do it all, both all the connectors and the orchestration and the compute side?

Matt Turck [18:29] Yeah, I mean, that's a good question. I think there's the overall mission statement that we're solving. There's what the open-source project does, and there's also the eventual enterprise version. And all three things are slightly separate, but overall tie back to our core mission. If you think about what the open-source project is right now, what LlamaHub really represents is an easy way for people to get started building their LLM application because now they just have this open-source repository of loaders that represent basically boilerplate that they don't really want to write themselves, right?

Matt Turck [19:12] Like, no one wants to look through the API docs of 50 different services just to try to build an application. If there's something that works a little bit out of the box, they would just use that and plug it into their toolkit. And so it's an entry point for people building LLM apps on top of their data. And we wanted to make that entry point as low friction as possible. Of course, there's this existing world of data loaders, ETL, ELT tools, in probably a more production setting.

Matt Turck [19:32] For instance, being able to ingest data from different sources, sync it to your data warehouse, transform it in some way, get it in the right format. There's different variants of these companies. I do think, high level, conceptually, if we're building this new age of LLM-powered applications, the requirements for the type of data that you want to load, as well as how you want to extract information from that data, will be a little bit different than the existing ETL stack.

Matt Turck [20:13] The reason is LLMs have this inherent capability of just understanding unstructured information along with structured information. And you do want to be mindful about how you actually leverage the context window of LLMs. At the same time, if you just think about this idea of splitting stuff into chunks and throwing it into a vector database, that's already very different than, for instance, how you think about ETL before. And so then the question is, how do you make this new ETL pipeline very effective to build LLM-powered applications on top of?

Matt Turck [20:41] And how can we provide the tools for people to do that? So our open-source toolkit is going to be focused a lot on enabling users to get some initial value there. I do think there are going to be similarities with existing data connectors for more production-grade workloads, and this is something of consideration as we're building an enterprise product. So, for instance, whenever you're building data connectors, being able to deal with updates to the source data is a big component of that, right?

Matt Turck [21:15] Like, what happens if the source documents change? And then all of a sudden you have to propagate updates from your source data to your destination, in this case, for instance, a vector database. And then how do you, for instance, do batch refreshes? How do you update stuff in real time in a streaming setting? These are all—there are lessons there that you could probably take from existing connectors. But at the same time, I think the actual form factor of how you do ETL on this data will be a little different.

Matt Turck [21:36] Great. What are some of the most recent stuff that you've released? 0.7 release. What does that do, for example?

Matt Turck [22:05] Yeah, so if I had to paint a picture of the overall toolkit, LlamaIndex started off relatively high level. I think what people really liked about it was the fact that it provided really nice out-of-the-box tools for people to get up and running really quickly, building any sort of question-answering system over their data. Because in about three or four lines of code, you can load data, ingest it, index it, and then query it. And then you would immediately get back a response.

Matt Turck [22:27] And so I think just from a pure user experience standpoint, a lot of people did appreciate that because they didn't have to dig super deeply just to get something basic working. Of course, for beginner users or users that just want something out of the box, that works quite well. But in general, for more advanced users, people want to define custom workflows over their prompts, orchestrate different flows over their data, and actually write the core business logic, which is really the prompts, deciding what LLM to pick, and defining the wiring themselves.

Matt Turck [23:07] In 0.7, what we've been really focusing on is taking the power of LlamaIndex as a high-level framework, but really exposing, rewriting the underlying abstractions to make it way more modular so that you could peel back the layers and really customize as deep as you want to. 0.7 really is a step in that direction because we now have our own very powerful LLM modules as well as response synthesis modules that users can just use completely on their own and independently of the rest of our abstractions.

Matt Turck [23:54] And it's only if they want to kind of plug this into a more powerful query engine system and they don't want to write it themselves that they can actually use some of our out-of-the-box data retrieval and query abstractions as well. And what this overall enables is kind of a trend toward enabling bottoms-up LLM application development over your data. And it's a little bit different than top-down development, where you just get something out of the box and it basically works. What bottoms-up development allows is for more advanced users to fiddle around with different modules over their data, be able to really start off with the very core building block modules like LLMs and prompts, see what's working, see what's not, and then eventually compose them into more powerful systems.

Matt Turck [24:26] And so, just at a framework level, it's very nice to be able to have the properties of both stuff that's working nicely out of the box, but also modular enough that you can just plug in and swap in different pieces.

Matt Turck [25:01] And yeah, that's an important point, right? So you're trying to be a solution for two kinds of users, or users with two kinds of appetites: both the people that want to do the simple stuff, but also the people that go very, very deep. How are you thinking about the tension between those? Are there trade-offs? And then, as a related question, where do you feel the most pull in the current state of the market? Do people want simple stuff, or is there a growing community of people that want to go really deep?

Matt Turck [25:32] The ethos that we're continuing to build toward is this idea that was inspired by Keras, which is this idea of progressive disclosure of complexity, where in the beginning things are very high level and work well out of the box. But the more advanced you are, and if you have a higher need for customization, you can peel back the layers of the toolkit and really customize things at a very, very low level. So at a certain point, if you think about what a framework should allow, it should offer some degree of opinion because it provides a certain degree of abstractions that are nice, that are a core set of abstractions that most users generally wouldn't want to touch because otherwise they would just build everything themselves.

Matt Turck [26:12] There always is going to be some segment of users that might potentially want to literally write everything themselves, and at a certain point, maybe they just wouldn't use any framework at all. But it is important for us to capture the entry users that might want to just understand things at a very high level and get started up and running, but really capture those core advanced use cases. And so I do think our framework has made pretty big strides in being able to capture that these days.

Matt Turck [26:40] I honestly think that whatever workflow that you have over your data, you can basically do within LlamaIndex, even if it's a very custom workflow, because we provide just very light base-level abstractions that you can just plug in. And then if there are individual components that you want to use to basically abstract away the need to write some boilerplate, you're free to use that as well.

Matt Turck [26:57] What are some use cases that you've observed in your few months doing this, both maybe the ones you predicted and some that surprised you?

Matt Turck [27:21] Yeah, so I think this actually probably relates to your earlier question too, as to how people are using maybe the high-level stuff versus the lower-level stuff. I think in the beginning there is, and there still are, a decent amount of, for instance, chatbots over your data and question-answering systems over your data. These are all systems that are built on top of LlamaIndex. And the reason for that is we do make it very easy for people to get up and running just building these systems over their data.

Matt Turck [27:55] And so we've seen people build these workflows in different settings, from, for instance, hackathon projects to startups building, for instance, a ChatGPT plugin over your Slack or your Notion, all the way to bigger companies, for instance, Instabase or other later-stage companies like Uber as well, using the toolkit to build more enterprise-level applications over their data too. And so a lot of our toolkit, for what it's worth, is centered around search and retrieval and insight extraction from your data.

Matt Turck [28:33] So it's not surprising that a lot of the applications are chatbots. These days, I think it's very interesting because I think there's starting to become a broader range of different apps that we're seeing within LlamaIndex as well. For instance, I'm having a webinar this Thursday with the CEO of OpenBB, which is an open-source financial analysis tool, and they recently incorporated LlamaIndex as a natural language layer to help power their basically open-source Bloomberg bot, but OpenBB-style. Then we've talked to some other companies.

Matt Turck [29:04] They're actually using some of the more modular components of LlamaIndex, for instance, for structured data extraction to define their own custom workflows. For instance, just convert unstructured data into structured information. And then the last category that I think is actually also very, very interesting is using certain LlamaIndex constructs to not just deliver short answers over your data, but actually generate entire long-form pieces of content. And I do think generating something that's like a paragraph is pretty easy for ChatGPT to do.

Matt Turck [29:31] Generating an entire blog post or essay, especially over your data, is a pretty challenging problem. And I've talked to a few users that are using components of LlamaIndex for that. And so I think, in general, this very much is in our wheelhouse. We want to offer the best tools to build these types of experiences over your data. And we're also starting to see a bit more application developers build things potentially in the agents category too, right?

Matt Turck [29:52] Where you not only do search and retrieval over your data, but can actually act upon your data in different ways too. And I think that's something that we are actively keeping an eye on and should have a release very soon for that.

Matt Turck [30:08] Speaking of upcoming releases, so what's on the roadmap, both short-term and, I guess, more medium-term? You talked about an enterprise version. Is that something that you're actively working on just yet, or is that more in the future?

Matt Turck [30:29] Yeah, so we're actually building out and scoping out the initial version of what production-grade LlamaIndex would look like. The way we're thinking about it right now is that the open-source tool is a Python package that provides a really nice orchestration framework for you to, again, ingest, index, and query data in different ways. What we really want to do is, we've talked to a decent number of enterprise users that have used our toolkit, and they are interested in potentially—and we've asked them, what are the additional pain points or gaps that you would still want?

Matt Turck [31:04] Right, out of a service that wasn't just an open-source tool. And so what we really want to do is develop a complementary set of services around the open-source tool that really helps augment the capabilities, especially in an enterprise setting. And so there's a few initial things that we're basically building right now. One is around scalability, being able to handle larger volumes of data, especially something that a hosted service could provide that maybe a Python package itself can't.

Matt Turck [31:40] On your laptop might not have the same guarantees. Another is, especially in an organization setting, there's a big need for multi-tenancy, access control, those things, over your data. And then how do you actually provide this as a service as opposed to, again, just something that you build on your computer? And then third is, going back to this point of data connectors, we have a rich set of community-built data connectors that make it very easy for users to load in data.

Matt Turck [32:01] But what about data connectors with enterprise-grade guarantees? So, things that allow you to do loading from custom data sources with reliability guarantees, scalability guarantees, and also have access control and those things baked in.

Matt Turck [32:29] How do you almost personally, as a founder, keep track of everything that's happening in this space, where things seem to be changing every day, sometimes several times a day, both in terms of your personal information regimen and then how you translate that into the roadmap for LlamaIndex?

Matt Turck [32:45] Yeah, yeah, I think that's a great question because there's the question of where do I get my information, which is just Twitter and Hacker News, and I downloaded Threads as well. There's a small but growing community of AI people on Threads. But then there's also a very valid question of, in a space where everything's changing very rapidly, how can you plan out product roadmaps effectively?

Matt Turck [33:16] How can you actually make sure that you maintain team morale and maintain focus and vision about what's going on, but still be adaptable to the changes that are happening? I have told pretty much the team, there's going to be cases where there's going to be things that come out where this really is going to be a drop-everything moment. Just stop what you're doing. This actually is a P0.

Matt Turck [33:34] We have to figure out our strategy for this thing. But at the same time, that's probably maybe 10 to 15% of the actual day-to-day work. We still have a concrete vision of this thing that we're going to build, and we're going to make steps towards that, because that really is a north star. So, for instance, a drop-everything thing is just this past weekend, LlamaIndex ended up as a post on Hacker News, and we were going through the comments, and a lot of people said that our docs were not great because they couldn't figure out, actually, going back to our previous points, they were able to figure out the high level of what's going on.

Matt Turck [34:15] They couldn't figure out how to customize things and how to actually play around with the low-level modules. So we basically, that was very important to us because if new users are coming in and they're just churning because they can't actually figure out how to do things they want, then it meant that we wouldn't be able to have real user growth and adoption of the framework. So we basically just stopped what we were doing, spent the past three days completely rewriting the documentation, and then we just launched that yesterday.

Matt Turck [34:37] But at the same time, there are these longer-term feature releases that are going on that we have planned out and that we're launching in the next few weeks. And these things are really just bets that, hey, this would be good to have. It's good to spend at least a few weeks working on this, and let's get this out, develop it a bit more, make it a bit more usable and production-ready, and let's iterate on user feedback.

Matt Turck [35:00] So in the end, I don't think the roadmap cycles can be super long, but you should have a north star of what you roughly aim to get within like six months to a year, but iterate quickly in probably like one- or two-week increments.

Matt Turck [35:29] It sounds like you have a wonderful and very responsive team. On that note, how do you think about building a team and who to recruit for a company like LlamaIndex? Because arguably the generative AI space, depending on how you define it, is new. And there's not that many people that have experimented with those things. How do you recruit, and who do you look for?

Matt Turck [35:52] Yeah, so I think it's a great question because it's something that is very relevant to what we're thinking about. You want to hire people that, in general for startups, are passionate about what you're doing. I mean, it sounds pretty basic, but the people that are just passionate are willing to put in more hours, do more things, and really do whatever it takes to help the company get off the ground, especially in those early days, because that's so important to make sure that your company is successful.

Matt Turck [36:15] And it's something that you really know when you see it, right? Some people are just very energized about what you're doing and willing to do whatever it takes to help the company. The next part is relative to this whole fast-moving generative AI space. Yeah, you have to be both technically savvy, but also very adaptable, right? And it's not going to be for everybody because when you sit down at the beginning of every day and you check Twitter and you check Slack and you check everything that's coming in, it feels a little bit like a fire hose.

Matt Turck [36:50] There's a lot of information to parse through, and you have to be willing to both aggressively prioritize what's important and try to ignore the rest of the noise, but also not overly fixate on something for a few weeks if it means that something more urgent has come up. And so we do really try to look for those qualities. Right. And along with, we have a high bar for hiring. We want to hire smart technical AI people, backend people. Really, from a values perspective, that's what we optimize for right now.

Matt Turck [37:11] And again, it's a high bar. I think there's a lot of people that are very interested in working on a project like this, but we want to make sure that these are the qualities that are going to take the company forward. Great.

Matt Turck [37:41] So maybe as a last theme to close, zooming back out, any thoughts on where you see things going, the space going, whether that's in the next few months or in a few years from now? What gets you excited? What do you think is overrated on the flip side? And, yeah, how do you think about all of this?

Matt Turck [38:05] It's a good question. I guess I could drop my spicy takes on all of this. Let's see. I think what gets me excited is this overall vision of having knowledge workers that can do automated decision reasoning and just understand data that's given to it, but also act over that data. I think that's just almost like an Auto-GPT, but focused around data orchestration management, and just really thinking about what that new software stack is going to be composed of.

Matt Turck [38:29] Because on the one hand, you're going to want to define a new data stack to enable these LLMs and agents to operate. But these LLMs and agents can also be core components within that data stack too. They can help really automate a lot of the pieces of ETL, insight extraction, all these different things. And so defining that vision is very exciting because I think if you're actually able to achieve that, you unlock a tremendous amount of value for basically anybody that is untapped with the current state of software.

Matt Turck [39:10] The ability to understand and act upon information just becomes much better, right, than what existing software allows. So I think as we build towards that vision, we want to make sure that we're constantly providing these tools that are allowing people to build this new world of software, to build these types of knowledge workers over your data in a manner that's very reliable and performant. And I think the other part that's exciting here is that no one really understands what exactly does that mean because there's so many different choices out there these days in terms of what model to pick, what embedding model to pick, how do you actually define different types of prompts, how do you wire different agents together.

Matt Turck [39:53] And then also, the field moves very fast every day. And so really getting a handle on the best practices here and also helping to define this new software stack is something that's very exciting, right? Just from a very conceptual level, whether it is on the open-source side or kind of more on the enterprise value side.

Matt Turck [40:22] Great. Well, that feels like a wonderful place to leave it. Sounds like you are building a really exciting company in a very exciting time. So congratulations on all the success so far, and hope to catch up again maybe a year from now, or as you continue to progress and build LlamaIndex into an industry giant.

Matt Turck [40:24] Awesome. Thanks so much for your time.

Matt Turck [40:25] Thanks, Jerry.

Matt Turck [40:35] Thanks for listening to The MAD Podcast. If you liked this episode, be sure to leave us a review. FirstMarkCapital.com/events/data-driven.