What Successful Medical Device Development Actually Looks Like — Lessons from the Trenches | LSI USA '25

Mathew Lickwar  0:05  
There we go. Hi. I'm Mathew Lickwar. I'm the director of hardware product development at fresh consulting, and I have a pleasure to introduce fresh a little bit. And then my colleagues here, and we How many innovators, perhaps, are present right now? Anybody want to raise a hand? Oh, awesome. Excellent. As a former entrepreneur and co founder of a hardware software business, I understand, hopefully this is an interesting topic for you. As you decide how to develop your product or take it to market, there's always complex decisions to be made, and hopefully we can convey some some of our lessons learned from years of product development. The format today, we wanted to share some of our lessons learned, as I just mentioned. Well, we will, kind of our focus is, is converting from where you take science and then help you film product development and take that to market. So we're going to share some stories from our experiences doing that, and then we'll have some Q and A will allow you to to ask your questions, ideally. So there's an opportunity to scan this Q and R, QR code, and we can read your questions out. Or there's also a mic here, if you want to stand up and read those questions. But I wanted to start maybe with probably the overarching the secret, if you will, to product development from our eyes is, you know, especially this would resonate, I think, with a lot of entrepreneurs. Is successful product development is all about determining the critical insights to make bold decisions at the exact moment that will shape the future, but simply put, that is the right information to make the right decisions at the right time. That seems quite obvious, especially this is something I think we all face in real life quite often, is making critical decisions. As a teenager, I had a profound love of racing dirt bikes, and if you'd asked me, then I thought it would be my professional career, but an unfortunate crash turned into a knee injury and a decision for me to have a PCL replacement at the age of 21 with an hologram. This is an injury that I live with in an intervention every day, and this is actually an x ray of my left knee from last year. And it was only then that I found out, actually, through probably the 15th orthopedic surgeon I've seen, that that particular surgery and intervention was contraindicated for my age group at that time, and that had been new information to the market or to the to surgeons when I when I was 21 and the secret to successful product development shares a very similar truth to my story, in that, you know, we're looking for the information that's valuable at the right time. And in my case, that information was available but didn't make it to me. So I, you know, obviously have to live with that particular thing, and I'm facing potentially a knee replacement at some time in my 50s, unless somebody innovates some successful knee less invasive surgery. I would love to please come talk to me. That would be great clinical trial patient. The experience has made, obviously, a lifelong impact on me, but also my professional career, and so I'm fortunate to work with people that share very similar ethos. Today, you're going to hear from Melinda allian. She's a principal technical program manager at fresh she has decades of experience, specifically in medical product development. John parent, who is a director of firmware engineering for fresh has also decades of experience building the software for class 123, medical devices. And you'll see a colleague of ours, Kris, down in the corner as well. You could flag him down if you see him walking around. He loves to talk about medical innovation. We are fresh consulting. We are based in Bellevue, Washington. We are an innovation services firm that leverages strategy, Design Development and Engineering to create what's next for our clients. We are 350 people in engineering, design, strategy, software and hardware development, and we help startups and multinational brands solve really tough design engineering and product strategy problems. Our expertise is really in kind of the unique technical like really challenging, difficult problems in product development. So we tend to do not only medical device development, but we we have a divisions that also do robotics and consumer based electronics, as well as AI and ML type integration. So complex products like that. There's a interesting product here on the right hand side of video, you can see, I think it's probably playing, uh. Something. We partnered with Microsoft to develop and integrate computer vision with an AI and ML learning model around this kind of automated cable hockey device. Kind of cool. So as I said, Our expertise is really in kind of the end to end innovation. And what that includes is like in house being able to do strategy, design, software and hardware engineering. And what we do is kind of really build a product team around your team, or with your team as a kind of an outsourced potential solution provider. You can think of fresh or product development in general, as you know, these really large kind of humps of effort, work, effort that is required in like different phases. And what we do is build kind of a core team around your product and problem, and then we can kind of grow that based on different needs, different technical needs, etc, as you expand or as you kind of go through different phases with that, I'd like to introduce my colleague, Melinda. She's going to share three kind of core elements what we think is integral to successful medical product development. And there's an opportunity here for you to scan and maybe ask a question if you want. And with that, I will give it to Malinda.

Malinda Elien  6:15  
So thank you very much for joining us today. As Matt said, we're really excited to help you figure out how to get through this next journey of your of your product and of your company. So I'm going to talk just for a few minutes, and then we love, love, love to answer any questions that you have. So John, Matt and I have collectively involved, been involved with hundreds of different products development projects, both medical and non medical, with every team configuration you can imagine, sometimes we're embedded in other people's teams. Sometimes we take the entire product development with us. Sometimes we're just doing little pieces. And we were talking about, like in the projects that we've been involved in, what has made the projects the most successful to be able to get to the end of your product development and starting your market launch in a most predictable passion fashion possible. And the very first thing that we all decided was that the people who get out of the product development phase the fastest are the ones that come into it the most prepared. You're like, well, Dom Linda, like, that's totally obvious, right? Like, of course you need to prepare before you actually start on this very expensive journey, but we know that preparation is really costly, and it can feel like you're spending a lot of time, kind of spinning your wheels and not doing very much, when you know that there's a long lead time in front of you to be able to actually get to your market. So we did. We did still really believe that that's the most important thing. But we kind of were able to break it down into like three sub, sub things that are really, really, really, really important to focus on. The first one is having the right team. So so much what you have done and will be doing will hinge on judgment calls by people that you've taken advice from people who have done this before, people who you trust. And while you don't necessarily want to be super anchored to people who are like, Oh, this is the only way to do it. You can't innovate. You also need to understand that expertise has its place, and usually the people who know where the shortcuts are in your product development process are the ones that have walked it before and know which is a good path and which one is just going to lead to a dead end. So I was hack officially having lunch with one of my friends, Amy, who founded a med device startup about 10 years ago. And I asked her, like, what was the most important thing that she did in terms of making her, her products get to market and her, her response was super immediate, and it was that she needed to hire a project manager, and that was the most valuable thing that she could do. Her the project manager that she actually hired had expertise and experience and time time being a really big one to be able to work with many of the different teams helping her bring her product to market. They were she was able to take the consultant, managing the consultants, off of Amy's plate, as well as be have the expertise to help guide decision making, to be able to be the right person to like who's actually focused on what Amy needed to do. So Amy's expertise is actually in business strategy, sales and marketing, and she is unrivaled at those areas. But she was being asked to make decisions around product development and regulatory stuff that she had no idea anything about. She had no gut instinct to be able to say, Oh, I feel like this is the right thing to do, and she had no time to actually learn what the right thing was to do. So the project manager that she brought in had expertise in engineering, in manufacturing, in regulatory, quality management systems, all of those things that she didn't have any experience in at all. And so her project manager was able to take some of that burden off. Now for each of you, it might not be a project manager. That's the right person that you need, but it's someone that has a skill set that you don't already have on your team. You need to make sure that everyone on that there's someone on your team who has a gut feel for all directing all of the different consultants that you're going to be working with as part of your startup journey. The second theme that we see in successful projects are the people. I always call it, you nail your science. So I've worked with a lot of projects that that they haven't quite figured out what their science is. So and by science, I really mean science. It could be a lot of different things. So you're sitting in this room because you have an idea for a new thing, you have a new instrument, a new diagnostic tool. You want to do an improved treatment, something that is at the hardest to why you're actually starting this product development journey and starting a new company with a new idea. But before you actually need to jump into the product development part of it like, does your science work? Does the core thing about what you're actually working on actually work? Or is it a theory on a piece of paper? And you've been so distracted doing all of the other things with doing a startup company, that you're like, I'm pretty sure this works. But if someone came into my lab and asked me show it on the bench, or if I had to walk through someone who's going to ask me a bunch of technical questions, does it actually work? Work? That's the that's the second piece when I say, nail your science. So since we learn a little bit more from challenges than from successes, I'll share you a story from a recent project that we worked on that illustrates this. So we were working with this startup company. It was their very first device to market, and they had a super specialized sensor that they needed in order to actually be able to validate that their idea was the right idea. The smooth, super specialized sensor took 18 months to develop, just a really long time. I know it's a really, really long time. And so they were like, ah, we can do product development. We know what the boundary conditions for this sensor is. We know enough to be able to actually incorporate this into a device. And so they were like, we'll we'll take the risk. We're going to jump into product development with two feet. And by two feet I mean hard tooling, miniaturized electronics like the whole package deal. Their sensor finally came in and it didn't work. It kind of worked, but it didn't work quite the way that they were expecting it to work. But they were completely stalled in being able to actually do their science because they had already spent all of their money and all of their time doing their product development and doing a bunch of activities that they hadn't nailed their science enough to be able to actually start so hard pivots, which are pretty typical, and They are now working on their science before they design their second version of their product. So nail, nail your science. Because really, and by the science, it could be a bunch of things, but unless the person who's going to actually implement your science agrees with you that your science is actually done, you're probably not there yet. The third theme that we had is to really make sure that you have a solid market strategy that aligns with your product development strategy. So I guess the flip side of the nail your science is we'll definitely have people who have nailed what they think their science is, but it turns out the rest of their business strategy isn't quite there yet. Either you we really, we worked recently with a with a company who had this very elaborate business plan home health care device, and they were like, Oh, we have these specialized sensors we want to develop in order to make the home health care experience even better. But unfortunately, they knew that that was going to take a long time and a lot of money, and so their business model relied on actually taking a fitness device, getting it white labeling it, getting it through FDA, so they could actually start with a market launch on a cheap device, and then actually go off and do their sensor development in general. That's not necessarily a bad strategy, which really can work as long as you planned it out correctly. Unfortunately, they had not actually vetted their regulatory strategy, and they really believed they could take a white labeled fitness device and get it through FDA filings without doing any of the additional clinical trials, user research, risk analysis, you know, most of the things that you have to put into a product package, and they could get it through no problem. Needless to say, they found out that that wasn't going to work while they were in the midst of their FDA filing, and they ended up pivoting hard again because they needed to have their their business model no longer had enough money in it because they were solving the wrong problem. They were spending a bunch of time solving their new sensor development problem, rather than making sure their business model was tight and that the regulatory strategy supported how they were going to actually make money in the future. So really making sure that your that your entire business model nuts boats actually makes sense all the way through from all of the different people who have expertise in those areas is so critical, because any one of those can change. You up, even if your product development aspects are actually going fine. So that's all I wanted to say. In terms of those three experiences, I know that you guys might have a lot of questions. Product Development is a really, really complicated thing. You know you may be wondering like, how many rounds of prototyping do you need to do? How do you validate your value proposition? How do you make a decision about outsourcing to a company like ours versus doing internal hiring? What is that balance? When do you need to start thinking about your Quality Management System? Does your manufacturing strategy actually support your launch strategy? How much should you weigh future plans for internal international markets versus actually thinking about how to get to your domestic market as soon as possible. So there's lots and lots of questions I know that are around product development that we did not get a chance to talk to today. But the most important thing is we want to know what you want to know from us. Since we've done it a couple times, hopefully some of you have done this a couple times as well. And so please, either if you've used the QR code or please come up and the microphone and ask us some questions.

Mathew Lickwar  16:01  
Jon Melinda, we often get kind of that question around, what is like? Why? Why would somebody choose somebody, like fresh consulting over like, building your own T What? What would be? Kind of the decision making factors around that, yeah, usually

Jon Perrin  16:19  
so a lot of companies have the core expertise in their specific science and the research they do, but they don't have the engineering know how. So you would come to a company like us to help you move to the next level of actually producing that product or getting it designed or working on the initial steps. And so we provide all the different engineer disciplines that would help you achieve that goal.

Malinda Elien  16:37  
Yeah, I often think about it as to after your product is on market. What are you going to do for the next five years, if you're going to need that person for the next five years, keep them on staff. If you're not going to need that skill set or expertise for the next five years, like that's when you I would consider outsourcing. There's a question in the corner. Oh, no, you Yes.

Audience Question  16:57  
Hi, yeah. A quick question for you. So at the very end, you mentioned something about your manufacturing strategy, align into your launch strategy? Yeah. And I've seen this damage startups many, many times over. Yes. I was just wondering if you could elaborate on a couple of times you've seen things like that happen. I've got a startup right now. That's not a huge order. Can't scale, can't fulfill it. Yeah, so I'd love to just for everyone in the room, for you to share a little 

Malinda Elien  17:19  
Yeah, right. And I think that that question can go both ways, right? So, like, there's the oh my gosh, I have so much product. How am I gonna possibly get it out at the door? And then the other half is the, oh my gosh, my launch plans are amazing. I think that I'm gonna actually, like, sell 5 million of them, and all of a sudden, you don't even have order for your first two so the first half, which sounds like your problem, maybe, is more of the if you have all of the sudden this massive order, and you're like, how am I actually going to get this through? We don't have the resources, we don't have the manufacturing strategy to be able to do that. I would say I can't talk on the money side. I'm a project manager for hardware, so I can talk a lot about manufacturing. So from the manufacturing standpoint, I would say, especially in the last three or four years like there's been a lot of innovation around being able to do rapid, rapid prototyping that's actually a most production quality. There's still a couple things that are really long lead time, items, like electronic components, will always be long lead time items, if you can afford it, to risk buy on those. Risk buy. Risk buy is my favorite like term, mostly because I'm not the money person. Like, do your risk buy on the things that you have a 36 week lead time on, like, buy them as soon as you think that you might need to in terms of things that it's like, oh, tooling is going to take a really long time. Because tooling takes a really long time. There's other ways that you can actually get production equivalent parts made these days that actually can shorten that time to market for for a short period of time, the investment is more you have to do. The trade off is to like, Is my product cost actually going to be worth the additional sales that I can get now, or how quickly I can fulfill my my customer needs. I would also say that because lead times are known to be long, the manufacturing companies that we tend to deal with do have quick turn lines that as long as they're intelligently managed, don't blow your costs out of the water. Unintelligently managed ones can totally blow your costs out of water. So I think it's a combination of sort of planning for a major launch in ways that make sense for super long lead time parts, and identifying those early and then also figuring out how to do rapid prototyping on the back end. So 

Jon Perrin  19:40  
I also think hiring a consultant to help you do that is the right tee like, not necessarily, but I'm just saying that that you need help to do that, navigate that. Yeah.

Mathew Lickwar  19:50  
Jon, from a from a technical standpoint, you know, firmware often is like, how do you differentiate firmware from normal software? Software development, I guess. And like, how does that like, how should you know, someone building a medical product device? Think about that in terms of if they have a firmware on a hardware device, versus, like, the software necessary to run certain elements of that product.

Jon Perrin  20:13  
So firmware is defined as sort of between software which is mostly run on a PC or desktop or in the cloud or on a mobile phone versus on hardware. So it's kind of some of the hardware. So it's kind of firm, because it's soft and Parker, and so we run it in a microprocessor. And so it's usually takes special engineers to be able to work interface with the hardware and with the with the software that you're writing, the algorithms and things like that. So that that's what firmware is. That's how we differentiate it.

Malinda Elien  20:44  
I would say, from development standpoint, usually your firmware developments being so much more Waterfall development methodology, just because of all of the interactions with the potential hardware versus your software.

Jon Perrin  20:57  
We sit between the electrical engineers because we're interacting with the sensors, and so we're kind of like dealing with the physics and the algorithm. So we're sitting in between the electrical engineers and mechanical physics of the product and actually making it come to life. So it's a little more difficult, a little more challenging, to work on, firmware than typical software, but it also depends on the complexity of the algorithms. Things that we work on, 

Mathew Lickwar  21:20  
there's a question like, What? What do people need to consider if the product is being maybe developed in one country and sold internationally? Or, like, how does, how does that affect the product development process, especially like, product strategy, or even if you don't know, but you have, I guess, an indication that you're gonna be selling internationally, is there something to consider for 

Malinda Elien  21:44  
I mean, the FDA is always the hardest people to get through, but every country has its own specialized regulations. Really, from a product development standpoint, identifying anything that's super special about a potential market, my experience has been that they all if you meet the FDA requirements, you get 95% from a product standpoint, of the rest, not from a clinical trials and all of those, but from the actual product standpoint, and then from a safety regulation standpoint, like understanding the harmonized UL standards, the CB scheme to be able to just preemptively do that testing upfront, so you don't have to go back and do another round of testing, assuming funding works out that way.

Jon Perrin  22:29  
Just talking with a European investor, and she was saying that actually used to be the way around. Everyone would go to the TUV, first unified body, then come to the US, because it's harder. But now they're saying that the FDA is easier because they're more upfront with you. You meet with them. They tell you what need to do. You get the rules up front, and as long as you do what they say they do, you pretty much pass, whereas the TUV, your notifying bodies is you. You just basically wait till the end, submit it, wait a year and a half, and then they tell you, Oh, this is what you missed. So so now they're thinking, rethinking that, going the FDA first.

Mathew Lickwar  23:03  
There's another question that often comes up, kind of, especially that comes that we see a lot, but is, you know, What qualities should someone be looking for when they're looking for an outsourced partner? So not so much the decision about to do it, but like, what? What makes actually a good product development? Or, you know, yeah,

Malinda Elien  23:22  
partner, partner, a partner, yeah. I think about this problem a lot, kind of related to the when do you decide to outsource, versus when do you decide to keep things in house? Is All right, I've made the decision that it makes sense to outsource. Now, who's the right person that I should actually partner with in order to do that, I would say the first thing is, partner is, is absolutely key. As a product development house, we get very antsy if we can't get the attention of the people who are paying our bills. Like, if you're paying someone to do this, make sure that you have time to work with them. That is 100% the first thing I would say, regardless of all the rest of the decisions, the decisions, though, are are really dependent on every person's need. But I would say some things to look out for is, do you care about how much interaction you have with these people? Like, sometimes, like you're doing a hard product, because it really enables a different thing, and you need to spend your time, 75% of that time on the different thing. And this hard product is actually relatively straightforward and simple. And so if you don't have to interact with that team as much, because they're not going to have any decisions that they need to make that are significant, like, you may choose to go overseas, you may choose to do a different time zone. You may choose to do a different city, if you want to be like in it with them every single day, spend two days in their office. Obviously location then becomes a big deal, or at least time zones become a big deal because you want to be in it because, you know, there's a lot of things that you have in your head that haven't been translated to paper, and you want to make sure. That your product development partner is with you along that journey. So that's like, one of my my biggest things is like, if I have a company that's we're in Seattle, Bellevue area in Washington, if I have a company that's on the East Coast that's hired us, I've made assumptions about how much involvement they need from the day to day of what it is that we're doing, the decisions we're making, if I have a company that's like up in Bothell Washington, and they're hiring us like I expect, I'm having on site meetings, and we're going to touch and feel together, and that they're going to be a little bit more involved in the process. So that's a consideration. It shouldn't be your well, it may be your primary consideration only because of the secondary effect as to how developed you how tied into product development you want to be, from a technical standpoint, I would say, just make sure that, like, don't hire a prototype house that only usually does prototypes if you're expecting that you're going to get a design that actually is manufacturable, and that you have a company that helps You bring it all the way into your manufacturing facility, like, if they only usually do industrial design and and like first user studies and you're expecting them to transfer your design into your manufacturer, you will not have a good experience. Also, though, if you're are only meeting video in order to help pitch to an investor, like, don't hire someone who's expecting to take this all the way into into manufacturing, because you'll also probably not have a good experience on that end, 

Jon Perrin  26:25  
you want to, you want to treat your partner as a business partner, because you're really going into business with them for a while, and you want to feel comfortable working with them. You want to be able to, you're going to discover things along the way of development, and so you want to be able to feel comfortable that you trust them and that you can pivot together, and that you can come up with a cost effective way of actually developing a product that you actually are proud of and you want to sell. And and then once, once we arrive at the the finish line, that partner can go away or become, you know, like the old ant or something like that. And then you can keep going forward, and then you call them later on when you want to do incepts or modifications. But from the initial start of the product, you want to really get to know them and really trust you trust them in all aspects.

Mathew Lickwar  27:11  
Malinda, you had mentioned, kind of building the right team. There's there's questions often, kind of, especially for someone starting out building, turning science into a product, and this like, what, what are the key or what is the primary role people should be thinking about hiring first when it comes to especially when it's like in that initial phase, like, do you have, have you seen in your experiences with medical device, product development, particular structures or kind of organizations? Does it depend on the type of product? Is it more relevant to like the I don't know the market or like what you think is the key differentiator for building the successful team? 

Malinda Elien  27:54  
This building the successful team, I think the most important person to hire first is the person that complements your skill set. I mean, after you have your core team of your founder team, like, the next most important person is, what skills do do we already? Are we missing? And how do I find the person who has that skill set? Like, if say, John and I actually founded a company like, we would need to hire a finance and business person like, that's not my skill set. That's not John skill set. We don't necessarily want to do those roles. So like, figuring out of the major things you're going to be doing for your particular company, like, Where, where are your skills gap? And I would say that's the most important thing. The second thing that I will say is, and this is selfish, project management is really important. Having someone who know is really important. Like, we work with a lot of companies who are like, I don't need a project manager. And it's like, you actually need someone who's dedicated to making sure that everything else is moving along. And it may not be a title of a project manager, maybe it's an operations manager, it's something but you need someone whose sole job is to make sure that decisions are being made in a rapid enough fashion to be able to continue marching, that there's a point,

Jon Perrin  29:07  
not only that, that you need someone who can cross discipline. So yeah, so in a mate, in a project, the development you might have a mechanical engineer, industrial design engineers, plastic engineers, former engineers, test engineers, verification, evaluation. But and they're all, they're all marching to their own they all know what to do independently, but, but there's all these touch points where, like, you know, the the firmware needs, the needs the processor and the boards and the in the electrical engineers, from the electrical engineers, and that engineers can't finish the board till the mechanical engineers told them where the holes are and where all the what the size of the boards are. So there's all these interactions. And then the test engineers need to know when this is going to be done. So you need someone on a project manager side to actually can cover all the boundaries, understand all the touch points from all the different teams, interactions and dependencies. So that's really important to hire that person. That's probably one of your keyest people that you need. Yeah, and it may be more than one program manager,

Malinda Elien  29:56  
it may be, it may be, depending on skill sets of what you have and what you need. 

Mathew Lickwar  30:01  
there's if my end goal is being acquired by a major company, does that change how I approach product development? 

Malinda Elien  30:09  
Oh, that's a good question. Yes, yes, but I have to think of why I'm so confident in my Yes, it does, because there's some things that you're going to want to do, if you know that you're going to be doing this for the next 10 years. And then there's some things that like, if your end goal it's yes, but it's depends on your exit strategy. It depends on your exit strategy. If your exit strategy is like, do or die, I'm going to be acquired. Then there's some things that maybe will be more like international patents may be something that you want to pay a little bit more attention to, depending on what your end goal is. But from a product development standpoint, you may not invest in tooling quite in like hard tooling. By hard tooling mean like, if you're doing plastic ejection, mold and tooling, or or more metal tooling of some sort, castings or whatever you may choose not to invest in that quite as early as you would if your end goal was like, I'm gonna, I'm gonna have this product for 10 years, because it's entirely likely that if you are purchased like, there'll be a change in that they'll have more resources. It's a heavy lift to invest in hard tooling when, for many companies like you can get 10,000 run out of a soft tooling that's much more cost effective. So things like that, like, Am I really going to invest for 10 years? Like, or is this? Is this something I could invest in now, but as soon as soon as I get acquired, it's going to change anyway, so there's not a lot of value in that. Like, those are the types of decisions that I think would change depending on what your exit strategy is. With the knowledge that my friend, her exit strategy was to be acquired in three years, and it's 10 years later. So yeah,

Jon Perrin  31:59  
so it's a question of what you kind of end up investing in, right? 

Mathew Lickwar  32:03  
This is a question that is near and dear to your heart. I know Melinda, how important is, ISO on 3485, QMS to product O and then, how should somebody think about that quality management system, like, especially with, like, success and failure and product development? 

Malinda Elien  32:20  
Yeah. Quality management systems. So one of the startup companies that I was loosely affiliated with was doing personal lubricants, which is a cost to medical device, it turns out. And they really believe that, since it's so consumer readable, ready, right, you can just buy it on the shelves. I don't know why. I can't say that word, that they didn't really need to have their own quality management system, that they could use their compounders quality management system to be able to actually just market their device. And it turns out the FDA came knocking on their door one day and is like, Hey, we're doing an audit. And they're like, great, go talk to our manufacturer. And they were like, no, no, you're the manufacturer. Like, you're the manufacturer of record. I don't care about your compounder. Like, where's your quality management system? So I it's really important if you're doing large scale product development, you know that, right? Like, if you're like, Oh, I'm doing this implantable I know I need a quality management system that's going to actually, like, drive all of my device development. I think the trickier one is when you're like, Hey, I'm over here on the fringes. Like, this is sort of a medical device, but it's not a real medical device. Like, then you try to figure out the shortcuts in terms of actually having a quality management system. Let me just say the FDA doesn't care. When they come in and say, Hey, where's your complaint system if you don't have your complaint system to show them. So start early, start often. But my experience has been there's a lot of grace as long as you're trying.

Mathew Lickwar  33:58  
Maybe time for one more is there? Is there when transitioning from like science to product development, is there a particular like milestone or like, how should somebody understand that science is, like, ready or mature enough to go into product development phase and like, be something that, I mean, consistency and reliability and repeatably, can be delivered through a product. So is there a standard format for making that decision?

Malinda Elien  34:25  
I have my idea. Do you have your idea? 

Jon Perrin  34:27  
Yeah, it's, can you realize it in a in the price point that you think it's going to be at? And do you have an idea of how you're going to actually, what kind of technology you're going to use to actually make it real? I think you know where it becomes from science to producible.

Malinda Elien  34:41  
Yeah, I would put it as, can you write a requirements document? Can you write a requirements document that if someone follows that recipe every single time, then you'll get the same result? And so that, to me, it's like, is there a procedure that's standard? Is there a requirements document that can be written? Down, or is it still like? Well, we think that we need blah blah, blah, blah, blah, blah, like our bounds to run from one to 50, but we haven't really tested it at either one or 50, but usually around 35 it works like that. Seems a little bit hand wavy, which doesn't mean you can't start product development. It just means that you might have a bumpier path. Then if you

Jon Perrin  35:23  
and part of actually could write it down and going towards your development process actually is proving out to be able to make it producible, and that that's okay too, you just plan for that and just say, Well, you know, we here's how we think we're going to do it. Then you would get engaged with engineering engineers to try and actually realize that, and you and you can find out through different iterations of prototyping how you can get there. 

Malinda Elien  35:44  
Yeah,

Mathew Lickwar  35:47  
great, thank you. Yeah, I think that covers it. We're almost about out of time. So yeah,

Malinda Elien  35:52  
any last questions? All right, you wrap it. All right,

Mathew Lickwar  35:58  
great. Thank you. 

Jon Perrin  35:59  
Thank you. 

Malinda Elien  35:59  
Thank you.