AI is a Collective Effort and Not a One-person Governance

Q: Welcome to the Big Unlock Podcast, David. We’re really happy to have you here. This is the 144th episode and it’s been going on for a while and we’re happy to continue the legacy that Paddy left behind.  

David: Thank you for having me. First and foremost, I must say, I started recently listening to the past episodes and seeing the incredible lineup of guests that have been on it. I’ve been working for the Crystal Clinic for the past 15 years.  

Q. Tell us a little bit, about your journey, like how you came into healthcare and how you came to the Crystal Clinic and what has your experience been? How the healthcare field has evolved with digitization and the Affordable Care Act and how it’s impacted your work. Where do you see it going now with the trends in generative AI?

David: I’ll give you a brief story. So back when I was doing my first undergrad, I had a dream and a passion to get into healthcare, either be an anesthesiologist or a cardiologist. But then looking at my career, as it would extend and what age I would be at and when I would actually start being a real physician and getting paid that salary or that pay, it just wasn’t lining up for me. The rest of my family, almost the entire family is involved in nursing. I figured I would go that route because it does open doors for many opportunities. 

So, I got into nursing, and within the first week, in Nursing 101, they listed out all these opportunities. I got into nursing for two career paths: one was informatics, and the other was nursing law. Well, nursing law didn’t pan out. I had an opportunity developing with a woman who had a practice in town, and she was the consultant for all the major law firms in nursing law. The day I graduated from nursing school, I found out she had closed her practice. I was already working at one of the local hospitals in the ICU as a nurse tech, so I followed that path into ICU. Then, I got hurt on the job, which required surgery, and I was laid up for about 10 weeks. During that time, I started dabbling in website programming to pass the time.  

While I was sitting around, I got a call from a friend of my wife’s who was working with Cleveland Clinic to implement their EPIC system. This was about 16 or 18 years ago. I jumped at the opportunity to get into informatics. From there, I helped roll out EMAR, order entry, and worked with Children’s Hospitals and Cancer Centers, focusing on orthopedics and surgery.  

At the end of my contract, I was supporting all the ORs at the main campus of Cleveland Clinic, specifically in orthopedics. I started looking for a more stable position because I didn’t like not knowing if I had a job when the contract ended.  

Surprisingly, I got a call from Summa Health System, which Crystal Clinic used to partner with, about a clinical analyst position. I had applied for a similar position about a year earlier and heard nothing, so ghosting is not new. That was on a Friday. I went in for an interview on Monday, February 22nd, and they wanted me to start the next day. I asked for a day to think about it and started on Wednesday. By Thursday, I was neck-deep in programming the EHR, training on it, and more. Myself and one other person were responsible for it all. 

At that time, we had a deadline because Crystal Clinic Orthopedic Center is a physician-owned facility. There was a regulation, I believe from Congress, that physicians couldn’t own hospitals, with a hard stop deadline of June 30th, 2009. The project was about six months behind when I joined, but with a colleague’s help, we turned it around in three months and went live with our EHR, HMS (now Medhost), in May. That’s how my journey into informatics began. 

From there, I moved into management after a few years and, since 2015, have been working as a director. I’ve held that position ever since. Working on promoting interoperability and meaningful use allowed me to learn more about healthcare. 

Crystal Clinic is a physician-owned hospital system, with 16 clinic locations that are outpatient extensions of the hospital. We’re governed by the Joint Commission, unlike typical physician practices. We have a surgical hospital, an outpatient surgery center for orthopedic-specific urgent care, and as of last Friday, we have 51 physicians. We specialize in orthopedic and plastic reconstructive surgery, but also offer pain management, physical medicine, rehabilitation, and sports medicine, rounding it out with over 90 physical and occupational therapists. We cover the entire spectrum of orthopedic care, end-to-end. 

Q: Is this all in Ohio, David?  

David: Yes, it’s all in Ohio. We’re venturing out this year with something I’ll call “travel medicine.” It’s not exactly telehealth, but we’ve been ranked number one in Ohio and top 1% in the U.S. for orthopedic hospitals by CareChex Analytics. We have four certifications from the Joint Commission for disease-specific measures in total knee, total hip, total shoulder, and spine, making us one of only three in the nation. We do it better and cheaper.  

We decided to partner with local, regional, and now national organizations, many of them self-insured, because they find it’s cheaper to bring people here as a destination rather than have it done locally. For example, if you go to Texas for a total ankle replacement, it might cost $300k, while here, it’s less than half, including travel, room, and board. Plus, we have quicker turnaround times and better outcomes, so companies don’t lose key personnel for long periods. This is catching on with OrthoForum and others. It’s great because patients get the care they need faster with the outcomes they want. As we expand nationally, we still maintain our local presence. Rather than open facilities in other states, we bring people here. We couldn’t replicate our cost structure elsewhere like we have here in Ohio. 

Q: Yeah, that’s a really interesting model you’ve outlined because usually, when people talk about expansion, they think of opening new locations rather than bringing people to their existing ones. I think that makes digital health initiatives even more important. How do you see your vision for this, especially in terms of sharing everything back with other hospitals? Have you had to rework your systems, and what investments are you making to make this possible for Crystal Clinic? 

David: The nice thing about not expanding physically is that you avoid capital costs. If you have a platform like Zoom, you can use it no matter where you are, and it’s the same for us. We have availability with some of the new providers we’re bringing on board, and we have the capacity to fill up both our surgical and clinical schedules. 

We’re not doing anything different than if the patient were local. We have a care coordinator, or sometimes multiple ones, depending on the patient’s path. They handle all the details—booking travel and hotel stays—while we focus on the patient and the healthcare we provide. 

Q: I was asking more from the point of view of, because the patient isn’t coming to you pre-procedure, you need to get all their information into the system and make sure the pre-checks are done, maybe not locally. How does that impact your IT outlook, and how do you handle that in this model? 

David: Yeah, we manage that through the care navigator. Let’s say we have a patient in California who can’t follow the usual pathway. We have someone who coordinates with them ahead of time and arranges everything. We recently rolled out online scheduling. We’re probably a bit behind compared to larger organizations, but as a physician-owned facility, things move a little slower. We want to ensure physician buy-in because the last thing we want is to launch something that impacts them, and they don’t accept it—that’s just throwing money away. 

So, we started with a few interested providers, taking baby steps. We had them pilot the system, and from there, we went live with a beta test group. Within a few days to a week, we had over 100 people schedule online. As providers see their schedules filling up, they’re getting on board. Some providers are still a little hesitant, feeling like they’re losing control of their schedules, but we’re getting more people in here. 

The data shows these patients are scheduling in the middle of the day or even at night—times when they can’t call the office. That’s encouraging because we were missing these people before. So, that’s how people outside of the travel medicine pathway are accessing our care. We also learned this through patient feedback, seeing why calls were being dropped. Since opening up online scheduling, those numbers have declined. 

Q: So what were the major learnings? What do you think patients or consumers are looking for today in the digital health experience? What insights did you get from that, which you then used to improve the systems and make it work? 

David: That’s a good question. I’d say we weren’t really meeting the patients where they are. We weren’t providing the opportunities that newer generations—like millennials and Gen Z—are asking for. We were stuck in the mindset that our core population is Medicare. It was a bit of a misnomer on our part, thinking we shouldn’t roll out too much technology because older patients may not have it. 

We found out that almost everyone now has access to technology, or they have a caregiver who does. That realization was one of the driving factors. We also started listening more to our patients—reaching out to those who passed on scheduling or cancelled, and understanding why. Since implementing changes, we’ve seen a decline in cancellation rates in a short time. It’s not necessarily tailored medicine, but we’re definitely tailoring the experience to meet patient expectations, and we have to evolve with that. 

Q: So this leads nicely to the next question. I think it’s a valuable insight where you’re saying that you can’t just wait for things to happen. You can’t push back and say, “This group is not going to use the technology,” because it’s coming, and you have to be ready for it. That’s similar to how we feel about AI and generative AI. Every organization needs to at least recognize that it’s out there and consider the use cases within their own organization. How do you feel about that? I saw one of the questions you sent about AI said, “Just because we can, doesn’t mean we should,” but you also mentioned that you had to get on the technology bandwagon. How do you feel about something similar now applying to AI? 

David: Our short-term initiative right now is getting into ambient generative AI. We have physicians willing to participate, but we’re taking our time. It’s about starting small and not rushing, similar to how we approached online scheduling. We made sure not to rush through it or go big bang. We also had to ensure it integrates with our current EHR. 

What’s been great for us is that over the years, we’ve been able to customize and program our EHR to our specific specialties—orthopedics, plastics, and now pain management, among others. A lot of large organizations try to fit everyone into the same mold, but we’re customizing ours for our needs. 

The company we’re piloting with—though I won’t name them as we haven’t signed a contract—works well with our system. It outputs the ambient AI into specific observation terms, which go into our EHR’s fields. It’s just another means of getting data into our custom-programmed EHR. 

Q: So, this is during the patient consultation? 

David: Yes, during the patient consultation. As the patient is speaking, the ambient AI generates a note and segments it into different sections, like HPI, health history, past medical problems, and so on. The assessment and plan go into their own sections. The AI-generated text is then assigned to specific fields in the EHR. It’s almost like having a scribe, but it’s working ambiently in the background. 

What’s nice about our current system is that it’s all structured data. If we want to report on it, we can run a SQL query and pull it all out. That’s the beauty of it. Instead of having just a note or a searchable PDF, we have structured data we can run analytics on. This is where we’re at now and what we’re looking forward to—it’s a nice, easy transition. 

Some providers using scribes are excited about this. Scribing is a transitional role, often filled by those waiting for med school. With this AI, they don’t have to worry about things like a scribe calling in sick. It replaces the need for a human scribe and does all the documentation for the provider. It’s also exciting because the company we’re working with can control the EHR via voice, which makes it a great blend of both. 

Q: When we did our CHIME focus group, we found that a lot of work was happening in this area, and in our generative AI workshops, we presented ambient listening as one of the success stories. However, the CHIME survey showed that people still want a human in the loop. How does that work here? Do physicians still need to review everything done by the scribe and then click an approval button? Or does it just go into the system? How are you handling that? 

David: They do have to approve it. Nothing is final, even with a physical scribe in the room. The provider still needs to attest to what’s in the documentation. If you look at regulations, especially in Ohio, but also nationally, a scribe can’t place orders. So the physician still has to enter and sign off on documentation and orders. 

There’s also a need to handle anomalies. AI has come a long way, and it’s not just a transcription service anymore. But anomalies can still happen, and that’s why a provider must sign off on it. It’s human nature that sometimes a tired provider might just glance at the note and approve it without fully reviewing it, which can lead to mistakes. At the end of the day, no matter how the information gets into the system, the provider is responsible for it. They need to read it, agree with it, and sign it. 

Q: This leads to another question—do you think AI is contributing to the decline of critical thinking? For example, we see cases where LLMs generate care plans, and nurses go into review mode, just signing off if the plan looks okay. Do you think having everything pre-filled for you leads to a loss of core skills that professionals have spent years developing? 

David: That’s a great question. In our current setup, I would say no. We’re still having conversations with the patient, doing exams, and verbally documenting findings. We don’t yet have clinical decision support in place, and that’s where the line between AI and critical thinking can blur. 

I’ve seen this evolution in healthcare, from handwritten notes to electronic systems. Nurses in particular have lost some basic skills—like even how to sign their name—because they rely so much on the system. Some newer nurses, who have never worked with paper records, might say, “The system didn’t tell me to do this,” when it should be part of their critical thinking. 

One example from my time as a nurse involved a stroke patient. They passed all the cognitive tests, but they made an offhand comment that their loved one was in the basement of their house, which was actually the hospital floor below their room. AI probably wouldn’t flag something like that as an issue because it seems irrelevant. However, as a nurse, I alerted the attending because it was unusual for this patient. 

Unfortunately, while attending to another crashing patient, the stroke patient had a large hemorrhage, and the outcome was unfavorable. This is a case where critical thinking made a difference, and AI might not have picked up on it. 

That’s my fear with AI in healthcare. Just because we can use AI doesn’t always mean we should. AI can help with documentation, but we risk losing those small pieces of critical thinking if we rely too heavily on it. It’s a slippery slope—if the system doesn’t prompt you, does that mean you shouldn’t do something? Critical thinking should always be at the forefront. 

Q: Yeah, that’s a really interesting point of discussion. You hear that it can ace all the medical exams, learn everything, and you have these AI companions trained for mental health. But it’s paradoxical because the main problem in mental health is loneliness. If you’re substituting it with a computer that doesn’t have feelings but can talk to you, then it’s a paradox. We have to wait and see where this technology is going to take us, but it’s good to be aware of the pitfalls, limitations, and what to watch out for. That’s why we want to take it slow. We know it’s there; we don’t need to rush into it. We’re not being forced to yet. Because you’re dealing with patients and have a really interesting background coming from nursing, which is a great motivational story. Whenever we talk to people in healthcare, almost everyone has a special story about how they got here and what made them enter healthcare. 

So, from your perspective within your organization, what do you think could be a good use case for AI? Where would it directly benefit your patients and be a real win for your organization? If you had to choose a field, where do you think this implementation could be? 

David: That’s an interesting question. Honestly, I don’t know if I can answer that. I’ve never really thought about it. I don’t want to just say something specific without having a solid plan. For orthopedics, I think one area where we could potentially use AI is in radiology or imaging. AI could analyze digital images and create reports with findings, with a radiologist signing off to agree with it. We haven’t implemented that yet, but I believe it would be a huge win for us. 

It would improve accuracy in what we’re finding and help fine-tune what needs to be done. From a surgeon’s perspective, we have radiologists review our X-rays as a backup, but they may miss an anomaly that doesn’t seem right. They’re still held responsible if it shows up in the image, but with an AI layer, we could flag potential tumors or other concerns for further evaluation. I see that being ortho-specific and really helping us take better care of our patients. 

More issues are surfacing in patients, and I think we can provide more holistic care, even if we’re not directly treating those issues. We can refer them and ensure the patient is well taken care of. I also have another use case involving custom implants. We have a program that depends on MRI or CT scans to create custom implants for our patients. I see that improving and becoming more precise. For example, my mother had her knee replaced this year. After her CT scan, they sent the data to the vendor to design a custom implant based on her specific anatomy. It’s no longer a trial-and-error process. Now, it’s almost like plug and play. I believe we’ll see an increase in AI usage there, tailoring everything to each patient’s anatomy. 

Q: Personalization. Those were very good examples, David. Thank you for sharing. Regarding radiology, I have an interesting story. During a generative AI program at Harvard Chan School of Medicine, one participant, the head of NHS from England, discussed how every radiology report must have two radiologists sign off on it. This led to significant backlogs, causing people to wait a long time for reports. 

One AI solution they found was allowing AI to generate the initial report, with the second radiologist reviewing and signing off. This resulted in a significant reduction in wait times. 

So, your points about radiology and the custom implants are solid examples of AI’s potential. AI can also enhance personalized medicine. For instance, when treating a fever, the recommendation for Tylenol is based on an average weight. However, AI could tailor the dosage specifically to an individual’s body. This concept relates to digital twins, where AI could tailor treatments based on a digital model before applying them in real life. These are all interesting examples. 

This brings us to our next topic of AI governance. I’ve heard both pros and cons about having a chief AI officer. Some argue that with AI’s applications across many departments, having a dedicated officer could limit its scope. What are your thoughts? Is your organization considering hiring a chief AI officer, or is AI spread across different departments? 

David: Our system is set up as a small organization with big problems. We limit our chiefs, and I haven’t seen more than four during my time here, with each wearing multiple hats. I don’t foresee a chief AI officer; that could be both good and bad. Each department has unique needs or wants related to AI, but it ultimately ties back to IT. 

While we may not govern AI usage, we should support it. We have a physician director for IT who acts as a liaison to ensure we meet providers’ and patients’ needs and maintain appropriate communication for any system changes. He essentially fulfills the role of a chief medical information officer without the title. I don’t see a chief AI officer because we’re still early in our AI journey. Maybe a CIO could oversee it, or perhaps a chief informatics officer, depending on how we expand our leadership. 

I don’t think it should be governed by one person; it needs to be collective because AI touches all departments. A committee might be beneficial to ensure that while AI can be effective, we also consider fiscal responsibility. Just because we can implement something doesn’t mean we should if the ROI isn’t there. 

Q: So, do you have that committee in place? 

David: We don’t have an AI committee yet, but we have a technology committee that looks at these areas. 

So what do you think is your timeline for, you know, like, are you thinking of implementing this in the next six to nine months or maybe like a year? 

David: Me personally, it has to be the next six to nine months. And part of that committee  once formed is definitely we had to educate people. That’s the biggest thing I’m finding. In our organization and all of these conferences I’m going to and people I speak with,  when you say AI, everybody thinks chatGPT. 

Yes. It goes beyond chat GPT. I know we started writing AI policies and we’re really being vague. Um, but the first AI policy is it must be approved to go through these steps before we can. Start implementing it, but it was, it started out as a blanket AI policy and it’s like, well, wait a minute back when we started programming our EHR and yes, 2000, 2012, I think we sort of did AI there, you might not think it, but you look at how we programmed it, there is AI involved, granted, the algorithms and what we use are extremely small and not really large, but it was, we took point and clicks and And we created a narrative from that point and click based on everything that all the data was being fed into it. 

So that is definitely on my radar to make sure that we need to educate. I know one of the workshops this year in the spring that I was at, someone mentioned the difference between people Who have a job in the future and those who don’t are those who are AI and light. Yes, exactly. Yeah, you gotta be educated and you have to have a familiarity. 

You don’t need to know LLMs and everything else that goes along with AI and how is all there, but you need to be aware of it and we need to educate the end users. Not to be afraid of it, of replacing you, it’s going to help you do your job better and be more efficient. It’s a power tool and it’s, it should be in your, and you should be able to have that conversation. 

That’s how we budget, you know. And you should apply it to the right areas too. I mean, do the ROI on it. See how much time is being invested. And just project management is another one that I’m passionate about, that there’s a lot of stuff Where AI can really help out there and just doing the mundane task and helping up with the communication plan. 

Making sure everybody has all the information and it’s  nicely tailored and it looks pretty and the product manager didn’t have to spend the time to go through and redo all the stuff and make sure that the system does it for you. Let the system work for you where it can, but still have the checks and balances in place that need to be there so it can be, the anomalies can be addressed. 

Ritu: Yeah, we’ve seen a very good implementation of that with our client, like I was telling you about. They’ve been able to generate these comprehensive care plans, and then it goes into the nurses’ admin panel. Only once they hit publish does it go out, and that’s the beauty of it. It can read and scan a lot more data, understand, and remember everything, especially for people with multiple chronic conditions, and come up with quite a comprehensive care plan. They’ve launched it in a couple of hospitals already and have received really good feedback, but as you said, it’s all evolving and it’s very new, so we have to kind of wait and watch to see where it takes us. 

David: I just had a recent conversation at the beginning of last month, actually, it was the day I came back from holiday. I had a broken car in the middle of the highway, so I was doing a Zoom call with this woman discussing AI education. A lot of universities, like many healthcare systems, are struggling with budgets, but they know they have to have something AI-related to stay competitive. Otherwise, they risk falling behind, and students will choose organizations that offer these programs over those that do not. It’s not a choice, but it’s changing so rapidly that universities lack the financial backing or budget to keep up. They’re working with companies that specialize in AI education and courses. Ideally, it would be nice for universities to handle it all, but… 

Ritu: This was a New York Times article talking about synthetic data. I read it too. We’ve incorporated it into our workshop for the next session because it’s really interesting. Sam Altman said that billions of pieces of data are generated every day. Just like you said, these bots are crawling the web looking for new content to read and ingest, and they can’t tell the difference between human and AI-generated content. They’re reading their own content and building upon it. Once you go through several generations of this, it all starts evening out and becoming monotonic. A lot of the content that bots generate is very impersonal; you can’t add those little human stories. The example you gave about the patient won’t be there in ChatGPT-generated content. So everything starts becoming very vanilla and indistinguishable. That’s a big pitfall right now because they’ve run out of human-generated data. They’ve already read everything on Reddit, and now they’re moving to generated data, which is going to be a huge problem. Even in image generation, I saw an example, maybe in the same article or a different one, where all the faces of people start looking alike after about 10 generations of reading their own content. The differences and diversity get wiped out, and everything ends up looking the same. There are definitely a lot of concerns that need to be addressed, and regulations need to be put in place. But yeah, those are topics for another podcast. 

David: One thing I find interesting is realistic expectations. A couple of weeks ago, I spoke with a company that scrapped their AI initiative for social determinants of health. They were aiming for perfection—100% accuracy—saying, “We ingest all this data, and here is the guarantee that this patient will have no sugar for their next visit.” They found that the timing of when that data is ingested and documented affects accuracy. The algorithm runs only based on the data at that particular time. They didn’t want to sell something that might only be 75% accurate because life-changing events can happen after that initial visit, like losing a job or a family member. Well, you lose that AI data because it happened outside of when the algorithm was run. They were trying to gather everything to produce a very high percentage of accuracy but struggled to get even close to 90%. All those external factors come into play, and garbage in, garbage out—you have to document appropriately. So yeah, it’s a tough problem to solve. 

Ritu: The no-shows are a huge issue, like you said. There are so many variables and factors. In fact, we show that as one of our case studies of where things could go wrong because those no-shows are biased towards people who already have problems. That’s why they don’t show up. Maybe someone has to look after their kids or something, and when you build these algorithms, it kind of reinforces that and makes it worse for those people. It’s really a difficult problem to solve. 

David: Then you take that data and have an insurance company use it, designed specifically for one area but applied elsewhere, which it wasn’t designed for. It doesn’t support the decisions being made. It can be good and bad. I find myself sitting on the fringes, waiting to see where we’re going with this and whether it will ever truly be 100% effective. We’re heading in one direction while I’m not quite sure if that’s the way we should go or if we should just wait and see what others do and learn from their mistakes. 

Ritu: Thank you, David. It’s been great having this conversation. Any last thoughts before we conclude? 

David: Not really. I’m just excited about where AI is taking us. I’m excited about what Big Rio and Damo are doing and where this podcast has gone and is going. I thank you for having me on and allowing me to share my story and experiences. Hopefully, it benefits others out there. 

We hope you enjoyed this podcast. Subscribe to our podcast series at www.thebigunlock.com and write to us at info@damoconsulting.net

Disclaimer: This Q&A has been derived from the podcast transcript and has been edited for readability and clarity.