Hype or Hope: Opportunities, Challenges, & Potential Impact of LLMs & Foundation Models in Medicine | LSI Europe '23

Transcription

Shabbi Khan  0:06  
Great. Well, everybody, thank you for joining. It's 5pm On the first day, so I'm surprised you're still here. Well, I shouldn't say surprising. Surprised there are so many of you here. Other than being at the beach outside, so thank you for sticking around. Before we kind of dive in, let Manish introduce himself. Manish, if you want to go ahead and introduce yourself, tell us about your fund.

Manish Kothari  0:30  
Sure. Hey, everybody, I'm Manish, run First Spark Ventures. It's a 300 plus million dollar fund, seed-A, co founded by myself and Eric Schmidt, former CEO of Google, we do hardware software, where my background is I've done three startups in the medical space to hardware two software, one hardware, myself, probably the company that you all know that I helped put the first check in, and it's part of most of your lives is is Siri for the iPhone. So after doing the medtech world, I moved into the tech world. And I've done everything from apple picking robots to space situational awareness to Siri to other medical systems. So that's my background. And that's our fund. 

Shabbi Khan  1:17  
Great. And I'm Shabbi Khan, I'm a partner at Foley and Lardner where a US based general practice law firm that represents med tech companies all the way from garage to global. We help them across all different practice areas, from financings to commercial agreements to IP, which is what I do, achieve and immigration. So to the extent that they're European companies looking to move to the US, happy to chat after. But moving on to today's topic, generative AI, you know, in ChatGPT, since November of last year, has just taken the stage. And we've kind of gone through this phase of super euphoria, excitement, to some skepticism, to just questions about ethics, and where does all of this lie. And so hopefully, we can unpack what this means for medtech. I know med techs, definitely one that's not necessarily the most innovative in terms of trying to jump on the bandwagon. And there's a rationality involved in medtech. And so let's just start with the first question maybe is, are we in a hype cycle with generative AI?

Manish Kothari  2:37  
So maybe I'll just start by asking, Can I do this? I can see how many of you have you used Chad GPT? That's okay, that says a lot. How many of you have used Midjourneym, Dall-E or any of those more? Okay, there's fewer, but still Zen. How many of us have used chatGPT, to ask a question about medicine. Okay, so, clearly, it's not purely a hype cycle, right? Like, there's a lot of people here who have tried it at least dabbled in it. Of course, there's a bit of a hype cycle, too, right? I mean, you've got a situation here where you have these solutions that are fundamentally transforming how you interact with the internet, I'm just gonna take one or two steps back. First of all, the question, this question that I asked, if I asked this question to this audience, I got a pretty high response. If I ask it to a 15 year old audience, they look at me and most of them don't raise their hands because they assume it's a given that they use generative AI in some form, or the other people are using it in ways you would never imagine. I mean, my 15 year old daughter does not use it to answer questions for school or write essays, but asks it to write four different essays and then decides which one she wants to pick on mixes and matches. I know other people use it to generate artwork that they then use as an inspiration for their own artwork. So very different ways. Maybe not necessarily in medicine, but it is being used and a 15 year old uses it as natively as we today. Use the internet. So I think you have to think there is a little bit of a generational difference here in how frequently to use. Of course, there's some hype, of course it's over. Not everybody thinks that generates a character that can be used offline. Yes, I like your kid may like Elsa, but do you really need your child to interact with Elsa via text on an ongoing basis via generative AI? Probably not.

Shabbi Khan  4:34  
I like the word generational. And when you say generational, if you think about the use of MLMs in health tech, what does the first generation look like? And where does the future go from there?

Manish Kothari  4:48  
ever met medicines interesting. So I'll give you an example. If you go to Kaiser or UCSF, the num For one use case doctors seem to be asking for the use of LLMs. For is actually to answer their emails. They're they're just sitting there going, we are spending multiple hours a day, at the end of the day, responding to emails, many times the patient has already sent us a question based on the click the labs that they've received answers through that we haven't yet seen ourselves. And we have to sit and ask emails, we often have to go to 15 to 20 different sections of an epic or Cerner, an EHR system to answer the email, if that could be done automatically. So paradoxically, I think the drive in medicine is starting to come from different sources than you would typically expect people who are tired of doing certain things like that. We're seeing a lot of energy in that area. The other thing to think about is that these systems are improving rapidly, the first generation of systems like chatGPT is actually terrible for doing that that activity that we talked about, why is it terrible? Once it's trained on data that has no value for the medical domain or limited value to every physician wants to respond in a way that's more adherent to what their practices what their beliefs are, their communication style is? So you need things like retrieval, augmentation generation, like rag, and a few other things to add to generative AI, that together allow you to do things. So what we're seeing is the driver is actually coming from there. I would make one other point that we're seeing really interesting is in medicine. The if I, as a person who does both medical tech and non medical tech, let me just tell you right at the beginning, the EHR sucks. And it sucks, for reasons, not what physicians only say, which is it doesn't, it makes their life miserable, which it does. But it sucks because it doesn't have complete information. I mean, I'm gonna bet that 100% of you here have had a medical treatment that has worked, and never told your doctor that the medical treatment worked. Right? So just think about it, you are missing the fundamental outcome research outcome data in the EHR. The idea of being able to create synthetically generated data using generative AI for time series data in medicine could be fundamentally transformative, you may be actually able to get the power of AI out of it. So while chatGPT is probably limited in its scope of what it can do. There is a huge potential in LLM or generative AI.

Lu Zhang  7:47  
Apologies for being late. But also I was impressed like everything runs on time. Thank you very much for having me. 

Shabbi Khan  7:54  
Yeah, great to spend a few minutes 10 seconds telling the audience who you are, what you do, what your fund is about?

Lu Zhang  8:02  
Oh, happy too. I'm Lu, I'm the Founder and Managing Partner of Fusion Fund, or a VC firm based in Silicon Valley, Palo Alto mainly focus on healthcare, loss of AI, healthcare, enterprise AI and the industrial automation investment. And I've been mainly focused on AI in healthcare since 2017. So lots of discussion about using different AI model to solve the hardest problem in healthcare industry. And also myself was an entrepreneur I build sold my company to Boston Scientific before I went to so-called, the dark side, and start my own VC firm back in 2015. 

Shabbi Khan  8:35  
Great, well, maybe I'll pick on you now that you're here, so, you know, the the first generation as LLMs and we I think we've we're starting to see it right in chatbots in the use of using generative AI to maybe complete Doctor notes and so forth. But what do you think happens after that? Where are the applications of LLMs? Beyond maybe just the more simplistic clinical workflow issues? 

Lu Zhang  9:03  
Yeah, so of course, you know, AI could be a super effective tool to help us solve the workflow efficiency issue out there. So there's a AAA problem was saying healthcare system. Accessibility, Affordable, Accuracy and AI could be a perfect to help us But meanwhile, there are more beyond that. For example, we have company be using large language model, generative AI for medical imaging enhancement since 2019 2020. So essentially, they could upgrade a low resolution scan from CT MRI to a high resolution scan was in minutes. And the accuracy is really good and also is much cheaper, faster, safer solution. Another thing is we talk a lot about how to apply a vertical large language model to a different application and you have to build up specific I would say industry specific model in order to make it highly professional and accurate. So we see company for concern for them. We have a company focus on Gen AI for life science and pharma, essentially, the scientists from Pfizer, Novartis could directly ask how to design clinical trial for this new medication. And it gave down so highly professional accurate right away. So there's more to a big Spore, for example, another vertical would be investing called the Digital biology, synthetic biology. And there's an opportunity for a large language model to be used in that domain, as well. But before we talk too much about a large language model, I still want to highlight one thing, that Gen ai, large language model is not the only AI, there's so many different AI model, the AI algorithm being used in healthcare sector already. And some of the application not necessarily need to apply large language model which are supposed to be very expensive as well. So we're seeing in general AI healthcare is because there are so many broader application within healthcare crucial the capability of AI and vice versa, you know, good for healthcare industry, for example, federal learning, we all care about the sensitivity of the healthcare data, federal learning is a new algorithm could really help us solve it. 

Shabbi Khan  11:06  
Yeah, one of the opportunities, but challenges is that people will look at generative AI and say, Hey, this is where all the money's flowing. How do I incorporate gen AI? Into my company into my business model? And as investors? What's your view on or advice to start off they're thinking about? Maybe generative AI, but also AI in general? Is there a value proposition that they have to have? And what are the sort of ways they should think about it?

Manish Kothari  11:36  
So yeah, so first of all, you know, around 18 months ago, every company just randomly started a adding AI to the title of their, their pitch decks, right. So, you know, I'm a medical device company with no software component, but I clearly have an AI component somehow, which is impossible to figure out. But so, you know, generative AI also is a buzz phrase, and, you know, you start seeing a lot of it added in the in the circumstance. So put those aside, right, like, I think there's real value in many, many cases in AI and in generative AI, right. So, yes, we are actively looking at companies that harness it, whenever a new component technology becomes available, there are new doors that are a lot unlocked. And to the extent that somebody can identify exactly the pathway from where this component technology unlocks a new door, a new market, a new opportunity, of course, we will take it very seriously. I think the other thing is, data still matters. And to the extent that you can generate some degree of proprietary data, or have a very clever way of integrating your proprietary data with the larger data sources that are becoming increasingly available. One of the phrases I often use is people use the phrase data as the new oil. I'm not sure I completely agree, I would say data is the new sand. Glass is really important in our life, glass is what we use, but you need to process the sand to get to the glass. So without the sand, there is no glass, but the sand itself is not sufficient. So the ability to look at that and convert it, I think becomes where we pay a lot of attention. And if somebody figures that out, it is actually a net real bonus versus a negative and we would consider it seriously.

Lu Zhang  13:28  
You know there's lots of new momentum happening is not only just a power by large language model, just the improvement of AI really open up the door for us to explore a different opportunity provide a more personalized digital therapeutics solution digital diagnostics solution, and also go back to what I mentioned early on digital biology. So last year, we all know whizzing healthcare life science industry are super exciting about Alpha Fold, which kind of opened the door for digital biology, synthetic biology. And meanwhile, now we have large language model Gen AI as another effective to work together with alpha fold. So things like that really gave us lots of hope we'll be able to really achieve the goal of the future of healthcare, which is personalization, right, highly personalized from the moment of diagnostic to therapeutics. I heard there's a new term called digital medicine. So I hope that's really could happen in the near future. Another good thing is I feel with chatGPT was the popularity of regenerative AI. The whole health care system also got the education of understanding the value of AI, not only just the physician and caregiver, but also the payers, like the insurance company regulator. They really understand now what is the value of integrity and how urgent they have to understand and also instead of being the real stopper, but also be the supporter to push this wave move forward. I still remember earlier this year, I was on this panel in JPMorgan Healthcare conference in San Francisco and across the panel. You know, me and Adam investor were complaining, oh, you know, there are so many digital therapeutics solution got approved by FDA, but why insurance company are now taking action to reimburse it. And then couple months later, I heard lots of insurance companies start thinking about giving reimbursement to digital therapeutics solution, because they got a question asked from the board. What's going on with this AI thing? Was this chatGPT. So I think that's all the good momentum and also require more effort from the founder investor side to allocate the industry better.

Manish Kothari  15:30  
Yeah, maybe I'm just gonna build on this a bit more. It's great common, first of all, you know, look, we've all everybody has wanted personalization, nobody's been able to deliver personalization in a significant enough way. And with the exception of physicians, physicians do their level best in the time they have available to deliver personalization, and they have the odds stacked against them. So any means we can provide to provide that degree of personalization, I think is absolutely critical. The second point I would make is, with many of these technologies, paradoxically enough, the consumer comes before enterprise. Even the internet, the consumers adopted way faster than enterprise adopted it. Even phones. Yes, people were using blackberries. But the moment smartphones came, the consumer adopted it maybe five to 10 years, sometimes 10 years before big corporations came up with great smartphone based solutions for themselves at least five years. It is not surprising that while AI was around for the last five years, it has been sputtering and medium successful in enterprise situations. It needed this consumer drive, as Lou sang for the board, rember boardroom members to suddenly say, Hey, what's this Chad? GPT? Yeah, this consumer push is now going to be followed by a massive enterprise push. So arguably the best 10 years to invest in this is now after everybody has accepted it and understood it and not before.

Lu Zhang  17:03  
Yeah, I really also really good fun. I want to share a little bit more about athletes who are we heard from our this large corporate, we have a sec. So network just give you more context I established in 2018 was infusion found, and we have 44, CTO from Fortune 500 company within this network, including some of the healthcare and pharma company. I remember earlier this year when we're doing our quarterly catch up, of course, we're talking about, you know, data strategy in the age of AI chat GPT generative AI, so I was asked, I was curious. Okay, what's the discussion on the board? Exactly, as you said, you know, loss of the board, push, push on the CTO XM to really bring in this digital technology AI to help them build up their data strategy. Another motivation they have is really about liability concern, because I was joking with them. I was like, Oh, don't worry about AI replace human because we need humans to take responsibility. If anything were dry, especially in this highly regulated healthcare sector. Someone needs to go to jail for that I was supposed to tell a joke. Nobody laughs when I when I tell a joke. They're like, Yes, that's true. We're the one taking the liability. So we need to integrate vertical AI solution data privacy solution sooner than later because our employee are throwing sensitive data into charge EBT, you would hold them now to do so. So there's another push on one side is efficiency improvement data strategy and other pushes? Variable liability. So when we saw our portfolio company talking with large pharma company, large insurance company, they are eager to do the integration sooner than later they have dedicated budget and increase our budget, even $2 billion level just dedicated work with startup for partnership contract language acquisition. So I hope that some useful information and encouragement for the startup founders.

Shabbi Khan  18:56  
Great, that was a lot. I want to touch on wearables for just a second, you mentioned the consumers are going to adopt this technology. First. Is it going to be in the form of wearables is it going to be and what they expect their patients or their physicians to provide to them in terms of service? Where do you see the path for adoption start?

Manish Kothari  19:16  
Lu, did you want to start, and I'll follow?

Lu Zhang  19:18  
Oh, yeah, happy to. Yeah, so I don't think is really limited to variables. We didn't invest any variable devices for the personalization. Because on one side, when you look at a market map, in terms of competition, it's really hard to compete with Apple. And another thing is also we're investing we don't touch anything consumer tech or invest in clinical level devices or healthcare solution, and this simple wearable couldn't provide accurate enough data to do that. But some new things. For example, smart pill is as for digital therapeutics, is going through FDA right now we have a company and they could basically go inside the body so not really invasive, but be able to really click that alive data for the GI system and also doing personalized therapeutics planned potentially. So I think there will be definitely smaller scale devices involved, but not necessarily on the variables. Another thing is I feel in the future, the concept of diagnostics will be slightly different, used to be concept is one point, diagnostic. But now there's an extension between clinic to the device at home to the community clinic, how we'll be able to assembling all the data together eventually contribute to a diagnostic, I think that's the future.

Manish Kothari  20:35  
So I think that there will be new forms of interaction happening here, again, mean, just abstracting out to a high level, you have a patient who is fundamentally worried about their health, or some aspect of their health, not necessarily knowledgeable, at best their knowledge is by a quick look on internet, which for some 10% really helps them the other 90 confuses them. On the other side, you have a physician who is extremely knowledgeable has limited time doesn't necessarily want to tell all variations of what can happen because most of them will not happen. So you have a fundamental mismatch in communication level, you have an emotional and limited knowledge person, you have a hyper knowledge person with limited time and trying to manage the patient as well. It's fundamentally the worst communication environment you can have. And I would say that the last 20 years, we've tried to say both from the physician's perspective and the patient's perspective, trying to fix a system that is actually fundamentally challenged by the very nature, whether it was in medicine or not in medicine. So to the extent that wearables, any other communication device can help bridge this fundamental challenge a communication system is actually net beneficial for both, I do think that there will be some groups of physicians, some of the more senior physicians who really have identified this problem and want to fix it. But a lot of physicians who are growing up now with it being very native in the system that will adopt it, I think wearables give, again, I'll go back to data as the sand, not the oil, the data from a wearable, whether it's from any group or the other, is inherently problematic, just in its raw form for anything that occurred, right, like, you have to process it in a way that addresses a key need, you can't just say, I've got this, and therefore I'm going to solve heart disease, it's not going to happen. So this idea of trying to process it in a way that addresses communication mismatches, data gaps, this is where we see opportunity, and the best companies are already focusing in these areas to to fill these needs.

Shabbi Khan  22:56  
So when you said that physicians have less time, I thought you were gonna say less patience. And so the more patience they have, the less patience they have. What does this does this, you know, does AI provide for scalability? Like what does this open the door to if we think about the opportunities that it presents itself?

Manish Kothari  23:18  
I think absolutely, yes. Is the simple answer here, right? Like, how much time do you spend replying to emails? How much time do you spend looking at insurance claims and, and dealing with insurance claims? I mean, Luke pointed out that there was earlier on there was a huge role for this on the insurance side, the healthcare tech side, and it's very easy to say we're a medical device company, and we sell medical devices. But when your the physicians you're selling it to are living in this world, where they're dealing with insurance, they're dealing with emails, they're dealing with us, that means they're spending less time dealing with you. So very simply put, it's very easy to say, this is not part of our day to day life, but it is part of your day to day life. And to the extent that you understand that and take advantage of it, you're you'll do much better, even as a pure medical device maker, you will do better. So I would say, I think it's actually fundamental. And I think that the world 15 years later will be fundamentally different because of these technologies for the physician in almost all positive ways. And relative to the internet, definitely in almost all positive ways that they have had them before. 

Shabbi Khan  24:32  
If we want to talk about maybe patient communications and biomarkers and the future of generative AI in terms of identifying new biomarkers. What does that look like? I understand maybe, you know, from imaging, you can do better diagnostics, or get better, sort of biomarkers just from imaging. But can we take the conversation further and expand people's views of what that might look like? Yeah,

Lu Zhang  24:57  
so first, I feel for loss of imaging. For example, a pathology not necessarily when you use generative AI, we could use, you know Explainable AI to find a better correlation. I think that's the beauty of AI just be able to help us process data super efficiently and quickly identify the correlation connecting all the dots. And if we were able, you know, I would say in the perfect wars that now the issue was in healthcare system is data isolation, that we're we sounds like we have lots of high quality data, but is, this is California, that's the Texas there's different states different hospitals system. But in the ideal world, if we could combine all the data together, I'd be able to have a such a big database and find all the correlation between different diseases, of course, we're gonna find the more accurate and the more personalized, more better and earlier stage diagnostic. And also, I think that's another thing. Another key questions applying AI is, we also need to find a better solution to help fund or help medical devices start out different AI a healthcare startup who have access to the data, and who owns the data is now the startup company is a large one is a house, our service provider. So that's the reason I mentioned federated learning early on that we have some solution available right now. There's more data encryption technology out there. And sooner or later that this data could be accessible by loss of the startup founder. And also go back to your previous question, I really want to add into that, you know, we've been talking about okay, AI is a new tool to one replace human by the doctor who is new to replace the doctor who doesn't use the new tool, right. And also fundamentally is also go to the startup founder. I know the same of the hour, I said, there are lots of founder focus on medical devices. My previous company was a medical device for type two diabetes. And at the time, we're doing a lot of data analysis to help us do super early stage diagnostic already. That was more than 10 years ago, I feel now since AI is so easy to integrate development cost is so low device also have a really perfect position at the entrance of the data. That's super critical. Without the control for entrance, the data, don't talk about transfer the data process the data. So if you already control the entrance of the data, why not add in another layer, as a platform play on top of it to transfer data, stores the data and further process the data and also expanded the potential access to neuro for platform play. So I feel that's kind of the beauty of AI. So to me, it's not really about whether we should is like, of course, we should integrate AI but how, how to use a factory and efficiently.

Manish Kothari  27:34  
Yeah, yeah, for sure. For sure. So the lowest hanging hanging fruit is integrating labs last year imaging data and clinical data today, the radiologist reads a report generates a report, a good chunk of that can be generated in an automated fashion. But then the physician looks at that report looks at their own clinical knowledge integrates to those could be done, you could actually do that so that the radiologists report integrates in some of the clinical stuff. Look, if I've just recently if I've got a lesion in my in an x-ray, lung X-ray, and I've recently been to India versus I've not recently been to India changes how the interpretation may be made. Why is it that that radiologist makes it in absence of that information, and then goes back. So there's multiple sources here and LLM can solve crack that today, right? But there are other new things too, like so far, if you've noticed, our biomarkers tend to fall in three categories fluids, blood, urine, saliva, imaging, and maybe some extended signals, right? Why not? There are companies now that are using transformers and large language models that are detecting depression, suicidality, Alzheimer's way more accurately than almost anything else? Right. So why wouldn't you use those techniques. So part of the adoption of this, the challenge of this is going to be people have accepted blood and fluid markers from the 1500s, the 1600s, for sure, people have accepted imaging markers from the 1900s or the 1920s. But there was resistance at the beginning. There's resistance right now to new markers that are coming out because of transformers and alarms that are not in those categories. And that's going to take a five to 20 year, hopefully faster. But pessimistically may be taking 10 years before the adoption happens. That also fundamentally changes things. So I think you've got a brand new horizon coming here in terms of the use of brand new markers that never existed before. And I think we should be pretty aggressive about adopting them. I think in terms of actual devices themselves, look MLMs are probably not going to be incorporated in your pacemaker anytime soon, and I'm not sure I see a clear need for an LLM in a pacemaker. But I see an absolute clear need for an LLM to be used in patient patient interaction, post surgery to optimize the effectiveness of a pacemaker or any other implants. So, again, if you think about the implant as just the surgery, just the implant versus the 14 days post op care or the 30 days post op care, the answer is completely different. 

Lu Zhang  30:28  
Yeah, I really want to add on that or totally agree was you're sad, especially first it was about Barker, I invest a lot of mental disease especially I personally has strong passion for dementia, but if you know the nature of dementia, it just hard harder to even find out the specific reason what trigger certain person to have dementia and why this person progressed much faster than the other. But using AI will be able to identify new biomarker is now just a simple biomarker, like a traditional biomarker, new biomarker specific parameters, individuals situation. So that's really a big kind of potential milestone for the general mental disease vertical. And another thing Yes, I think for the past many years, even sometimes without without certain biomarker and symptom and we potentially identify what is the causes and disease, still challenges are certain symptom is now directly related to what our traditional assumption, and if you go to a doctor, as a neurosurgeon, and they will think about neurology issue, you went to a different physician, they may think, okay, so heart issue is a massive issue. If you'd couldn't be able to combine all the information together, find a doctor have the domain knowledge across different specific sector, maybe you won't God, super accurate agnostic as well. But this issue could be potentially solved by large language model by just a cross check the information, fun, you know, correlation. So that's really fascinating. So I feel like for the future, I totally agree with you that we should aggressively pursue the understanding of the new biomarker, and also welcome all this biomarker, and they're using AI to help us find the most efficient way to matchmaking. Which biomarker for which CTs and take it from there.

Shabbi Khan  32:12  
So to create just a little bit of controversy? Is it too early to start thinking about ethics and regulations? Or do you think if you're investing in a company, you want them to have a clearer view of what that looks like? And what that pathway might result in?

Manish Kothari  32:30  
Why don't you start? 

Lu Zhang  32:31  
Yeah, happy to I think, definitely not too early at all. Because if the startup company are now thinking about this issue, that it's hard for them to to have conversation with a customer, because their customer concern the large pharma company out maybe whoever their partner was sharing the data, so have to think about it earlier than later. Of course, we also are hoping the regulation could come along to give certain guidance, but unfortunate, in the history of technology, innovation regulator always came the last and that we couldn't rely on government to essentially give the guidance, right. They also want the private sector to get more education. But the good news is, from what I heard from my venture partner, Rohini, she just left White House half year ago, since both party really don't want to be the showstopper for the AI. And meanwhile, very open to talk with industry leader in order to understand what is the proper regulation should we put in place in the future? And then another thing is, what is the regulation should be as a regulation for technology or regulation on data? I personally think the regulation should be focused on data, because essentially, the people, the same should be recognized that we're being processed by AI and who are going to use AI. We're gonna be processed by AI. So data who are gonna use AI is us. Yeah. So

Manish Kothari  33:47  
I'll start by saying, one of our companies in the non medical area, which is doing extremely well, we funded it a few years ago, does the following. If you're a kid who wants to work with a synthetic character, and Little Mermaid versus a person who wants to deal with a synthetic character in Friday the 13th, it automatically creates completely different guardrails and characters. If you're doing that in the consumer world, and it is the right thing to do in the consumer world. Heck, yes. It's important to think about ethics and, and policy in the medical world, right? Like if that's the discussion, I've been to DC many times talking about the ethics and I'm on one of the panels that is dealing with the AI side of things. It's, it's going to be very hard for regulators to regulate AI per se, they just the lack of knowledge, and the speed at which the field is moving is almost impossible. I would agree that in areas where the data is sensitive or data is owned or data is manipulated, there needs to be very clear guidelines on who owns it, who might who gets manipulated, who gets to make decisions on it. Who gets to you in a particular way, and I think, you know, the good news is a lot of these things are technologies, also handling some of these things. There's techniques such as federated learning right now, which is never take the data from any one place, leave it where it is, and but do all the analysis you want. So there's a lot of new technologies that are starting to handle this problem, because they know if they don't handle this problem, the rules are going to come in and they're going to be harder, you probably need both. You need the rules and the technology. But I think we're starting to converge converge there. I'll just make maybe one last comment here, which is the generational element is important today. You know, 20 years ago, you could graduate with a biology degree and do very little math. Today, there is nobody graduating with a biology degree without at least math, almost certainly Python programming. And almost certainly data sciences. At the minimum, if you didn't do programming yourself, probably both. So you got to imagine a 21 22 year old graduating from college today in biology, maybe becoming a physician maybe not, has such a fundamental different interpretation of biology than somebody who graduated 20 25 years ago. So I would say, you know, we've in the past, had great biologists who didn't do math and great physicists and mathematicians who participated as a collaborator in biology. Those two worlds are merging. And it's a beautiful thing. And it's a great thing they're merging. So I think a lot of these discussions are going to build upon the fact that there's going to be a new group of folks, which we're all going to be working closely with in our careers that are going to cheat, treat things more intuitively and inherently and make the right decisions, in many cases, more intuitively and inherently than us who have to think about should I be hitting this stroke in tennis versus not thinking about it, just hitting the right stroke? Right. So that's, that's a very big difference.

Shabbi Khan  37:01  
Right? Well, I think we're out of time. But I thank Lu for joining for sure. Definitely enhance that conversation. And Manish, thank you again for participating. And thank you, all of you for spending time with us as opposed to on the beach. So please go to the beach, if you want, yeah.

Manish Kothari  37:18  
Take care. Thanks, everybody. 

Lu Zhang  37:20  
Thank you.