September 17, 2021: What are the characteristics of a high functioning healthcare system with regard to data and analytics? How do you provide data to third parties? How do you protect that data? How do you anonymize that data? How do you ensure that you’re not allowing it to just run loose in the wild? Paula Edwards is Consultant, Analytics & Data Governance Practice Leader at Himformatics, an advisory company who works with health systems to make sure they’re getting the most value out of their technology and data investments. How do you design systems that have people and technology working together? Who participates in data governance? What does an effective data literacy program look like? And what trends are we seeing with academic medical centers?
Understanding Data Governance, Analytics, and Human-Centered Design with Himformatics’ Paula Edwards
Episode 444: Transcript – September 17, 2021
This transcription is provided by artificial intelligence. We believe in technology but understand that even the smartest robots can sometimes get speech recognition wrong.
[00:00:00] Bill Russell: Today on This Week in Health IT.
[00:00:01] Paula Edwards: I think electronic health records are the perfect example of why we need more user centered design in healthcare because the early versions really did not use user centered design methods when they design their systems. And once you have poor usability baked in, it’s really hard to recover from that. [00:00:20]
[00:00:25] Bill Russell: Thanks for joining us on This Week in Health IT influence. My name is Bill Russell. I’m a [00:00:30] former CIO for a 16 hospital system and creator of This Week in health IT. A channel dedicated to keeping health it staff current and engaged.
[00:00:38] Special thanks to our influence show sponsors Sirius Healthcare and Health Lyrics for choosing to invest in our mission to develop the next generation of health IT leaders. If you want to be a part of our mission, you can become a show sponsor as well. The first step is to send an email to [email protected]
[00:00:53] A common question I get is how do we determine who comes on This Week in Health IT? To be honest, it started organically. It was just me inviting my peer [00:01:00] network. And after each show I’d ask them, is there anyone else I should talk to. The network group larger and larger and it helped us to expand our community of thought leaders and practitioners who could just share their wisdom and expertise with the community. But another way is that we receive emails from you saying hey, cover this topic, have this person on the show. And we really appreciate those submissions as well. You can go ahead and shoot an email to [email protected] We’ll take a look at it and see if there’s a good fit to bring their knowledge and wisdom to the [00:01:30] community as well.
[00:01:30] Today we are joined by Paula Edwards. Paula is who I called when I was setting up data governance for St. Joe’s back in the day. Paula Edwards is a Consultant Analytics and Data Governance Practice Leader for Himformatics and Affiliated Professor at the Rollins School of Public Health at Emory University in Atlanta. Paula welcome to the show.
[00:01:48] Paula Edwards: Thank you. I’m excited to be here. It’s great to see you again.
[00:01:51] Bill Russell: Are you really excited to be here. I mean, you just came back from Italy. Are you really excited to be here?
[00:01:56] Paula Edwards: I am. Believe it or not. I can talk about [00:02:00] analytics and data governance all day long. So this is right up my alley.
[00:02:03] Bill Russell: That’s not what you do on vacation is it?
[00:02:05] Paula Edwards: No, I don’t look at the computer when I’m on vacation.
[00:02:08] Bill Russell: Well, that’s good. It’s interesting with this work from home thing, I’ve been talking to a lot of people, cause they’re not used to this kind of environment. And some of them are talking about the fact that they just feel like they’re being run into the ground.
[00:02:18] They’re working so many hours. I’m like, but if you do this, like we’ve done this for awhile. You just have to learn to draw the barriers and live within those barriers.
[00:02:28] Paula Edwards: You do. You have to have [00:02:30] balance. And so that’s why I have my home office set up. And when I shut down at night and leave my office, I’m done with work. So that’s how I’m made it work.
[00:02:38] Bill Russell: Yeah. And I have a little sign on my door that says leaving the office when I go out. It’s just a reminder that the family’s on the other side and they expect you to be leaving the office. So that’s what I’m doing. Paula I’m really excited to have you on the show.
[00:02:53] As I said, you were the one who worked with me on data governance at St. Joe’s and we had a lot of stuff to do. [00:03:00] Back in the day, you felt like you were out in Southern California more than you were in Atlanta. You were out there a lot, but here’s where we’re going to start.
[00:03:06] I’d like to unpack your LinkedIn profile just a little. You have a couple of things, which I think are really interesting. You have Analytics Aficionado, Data Governance Coach, Healthcare Data Translator, Human Centered Design Advocate. So let’s start with your day job and then work through these. Tell us about Himformatics and the type of work that you do.
[00:03:27] Paula Edwards: So Himformatics is an advisory [00:03:30] company and we work with large health systems and community hospitals and other healthcare providers to make sure they’re getting the most value out of their technology and data investments. So we have a range of services and I lead our analytics and data governance practice.
[00:03:45] And that’s where the data governance coach comes in because that’s a lot of what I do. So I work with health systems like when we came and worked with you at St. Joe’s on analytics strategy and roadmap, data governance, strategy, and training and coaching, [00:04:00] and helping people look at their portfolio of analytics solutions and how they’ve gotten the people process side of how they provide analytic services set up to really help them get the most value out of their analytics staff as well, and really make sure they’re teed up to be able to really innovate and drive change through the organization using data.
[00:04:20] Bill Russell: When I got to St. Joe’s 20 late 2011, early 2012. We had no formal data governance. I mean, there was ad hoc data governance. I mean, [00:04:30] people were trying to figure things out. And our data people were really smart. And so they were able to sort of hold it all together.
[00:04:36] But as we grew, as we added more things, it started to fray. And that’s when we went after data governance. I’m curious what you’re seeing these days? What does data governance look like in healthcare? Is it mature? I mean, would you rate it as mature or would you rate it as still fledgling and progressing?
[00:04:54] Paula Edwards: It depends on the organization. At this point most of the places that I’ve [00:05:00] been, they’ve at least made some effort at establishing some form of data governance. They either had a data quality problem that was really. Driving them crazy. Or they were having trouble to figure out where they should be focusing their analytics resources.
[00:05:14] And so they made an effort, but then it sort of lost steam. And they’ve had a hard time really getting it to take traction and really be able to use it to provide the level of data asset management the organization really needs. Especially given how important [00:05:30] data is to running a healthcare organization these days.
[00:05:34] So a lot of times when we come in, we’re helping them look at what do you have already in place that we can build and leverage. And then where are some of the gaps that we need to help coach you and give you some ideas for how you can make things more effective?
[00:05:47] Bill Russell: I’m going to come back to some of those terms, data asset management. I keep hearing the term data supply chain come up. We’re going to go into that in depth, but I want to finish working through you have Analytics Aficionado, is that just part of the game of being in the data [00:06:00] world? You appreciate good analytics when you see some?
[00:06:02] Paula Edwards: I really do appreciate good analytics. I love a good visualization, like a really well thought out visualization that tells a story as understandable and actionable by the intended audience. And I love seeing a lot of things that are happening these day with all the AI and machine learning and the opportunity to really use that, to impact how care is delivered. To me, it’s just an exciting time to be an analytics.
[00:06:29] Bill Russell: It’s interesting. [00:06:30] I used to always try to drive that home, that we are telling stories with the data. And it’s important to know what story you are telling or not telling by the visualization that you put in there. Do you have some examples of just some of the things that you’ve seen through the pandemic? I would imagine there’s a ton of great work that’s been done through the pandemic.
[00:06:49] Paula Edwards: There’s been a lot of really great work that’s been done through the pandemic. As I think about some of the things that I’m the most thankful for from the really innovative cutting edge stuff, [00:07:00] what they’ve been able to do with AI and machine learning to really speed up the vaccine development process is just fantastic.
[00:07:08] So we wouldn’t be sitting here with vaccines in our arms if that hadn’t happened. Normally it takes years to do that. But for a broad, I guess, application of analytics in a pandemic, one of the favorite things that I went back to over and over was one of the maps of the US, I know everybody started out looking at the Johns [00:07:30] Hopkins map.
[00:07:31] But I think it’s a good example of why metrics that you choose matter because during the early days of the pandemic everybody was watching John Hopkins, so they could see how the pandemic spread. And where was it now? But because it was based on cumulative totals of incidences, you didn’t have any idea of what the transmission rate was immediately.
[00:07:52] So you knew like I live in Georgia, a lot of people in Georgia had gotten COVID, but how many were getting it actively [00:08:00] now? And so there’s a group called global epidemics that published their version of the map that focused on spread. So you could see at the county level what is the seven day moving average of new cases, total and new cases per a hundred thousand people.
[00:08:17] So you could say what’s the transmission rate where I am, and based on that, do I want to go to this event or go to this store or go to this restaurant and really make your own decisions? So it did a good job of just making it actionable for an individual.
[00:08:29] Bill Russell: It’s [00:08:30] interesting. You make that decision. That’s a great example of telling two different stories with the data. I had a bunch of people call me and say the pandemic’s bad here and I’d be like, well, look at it, because you can break the Johns Hopkins ones down and drill into it and get, get some of that detail but look over the last 30 days, you’re actually trending down and you could look at the CDC and see some of those numbers and say it’s heading in the right direction.
[00:08:53] But if you just look at the cumulative, it tells one story. Whereas if you’re looking at a time series data, it’s [00:09:00] something completely different. It’s the definitions around data. The data you choose the quality of the data, all that stuff, and the timeliness of the data, all that stuff matters.
[00:09:11] But anyway, I get ahead of myself. We’re going to go into all those things. Healthcare data translator implies the need to translate something. Where where does this skill come into play?
[00:09:22] Paula Edwards: So to me in analytics, it’s absolutely critical to have some of these health care data translators. And if you think about [00:09:30] historically in healthcare IT especially, but also in analytics 10 years ago, or even less than that, you had the technology people who spoke their language, you had the business and clinical people who spoke a different language. And so when they were trying to communicate requirements and what are we trying to do and what do we need to build?
[00:09:49] They were talking at each other, but not really communicating because they just fundamentally spoke different languages. And so translators like me knowenough subject matter [00:10:00] expertise about the business and clinical side of the operation and what they’re trying to accomplish, and what’s going to be valuable to them and know enough about the data and technology and what they’re capable of and what’s possible that we can help translate between those groups to really make an impact. So help the business visualize what is possible with the analytics tools and the data sets that we have and how can we really use it to change how we care for populations. And then work with them, with the technology people to say, okay, what do we need [00:10:30] from a platform and a data integration and a data standardization perspective to be able to deliver on that promise and get some value out there to our providers and the populations
[00:10:41] Bill Russell: Fantastic. Human centered design advocate. What is human centered design? Why does it need an advocate and where does it come into play in healthcare?
[00:10:50] Paula Edwards: So my PhD is actually in human integrated systems, which is about how do you design systems that have people and technology [00:11:00] working together to accomplish a goal? And human centered design is what a lot of the methods and tools that are used in that space to make sure that you have your end user front and center at every stage of trying to design your system so that you make sure that it’s going to work efficiently and effectively for the intended user. So a lot of what you hear now with user experience has a lot of its foundation in human center design methods. And a lot of what you see in agile [00:11:30] methods is actually very much founded in some of the user centered design methods. It’s how do you get your user and those key stakeholders involved early and often through the design life cycle so that they can provide input, tell you where you got it, right where you got it wrong really provide valuable insight into the context in which the system is going to be used. And I think your question about why is it important in healthcare? I think electronic health records are the perfect example of why we need more [00:12:00] user centered design in healthcare because the early versions, the early iterations of the current generation of electronic health records really did not use user centered design methods when they design their systems.
[00:12:13] And once you have poor usability baked in, it’s really hard to recover from that. So that’s why some of the usability is so bad there.
[00:12:21] Bill Russell: I’m going to get into data in just a minute, but I’m going to put you in a time machine. We’re going to go back and sit down with David Brailer and his team before they, they [00:12:30] put into place the meaningful use whatever it was.
[00:12:34] I was going to say doctrine. It’s not a doctrine, but meaningful use program. We’re sitting there, it’s just, it’s close knit group four or five people. And what are we going to say to them to say, look, let’s make sure that we can do something here that changes the direction of this.
[00:12:53] Otherwise, we’re going to end up with where we’re at today, which is essentially systems are getting better, they are [00:13:00] getting incrementally better and they really need to take a dynamic leap forward in terms of usability and design around the clinician workflow and those kinds of things. What would we be saying back then? Or did the problem start even before that?
[00:13:15] Paula Edwards: I think the problem started before that. I think it’s going back to those early EHR developers to say, you guys need to have doctors and nurses and patient registration clerks as part of your design team that are really [00:13:30] helping you understand how does a patient flow through our health system and what do we need to do to make it easier on the patient and make it easier for the people who are caring for.
[00:13:40] Bill Russell: Paula, I did an EHR project. Have you ever put clinicians in a room together and tried to get them to agree on what the workflow should be? It’s really hard. I mean what’s the trick. How do you drive through that?
[00:13:54] Paula Edwards: Go back to the beginning of our conversation. You need healthcare translators that can help facilitate those [00:14:00] conversations and really get them to focus in on what are the commonalities and how do we build to the standards? And there is a lot of, a lot of people in healthcare historically have wanted to do it their way. And that kind of dovetails into our data governance conversation. You have to standardize to get efficiency and quite frankly, good user experience.
[00:14:20] And so you have to be able to facilitate those conversations to get everybody to a workflow that that works well for everyone, even if it’s not necessarily their ideal [00:14:30] workflow. You got to get a little bit of give and take. Yeah.
[00:14:34] Bill Russell: Yeah, this isn’t a paid for spot for Himformatics but I’m going to give them a shout out because you guys did great work for me. And the reason I’m going to give him a shout out is cause I’m going to hit you up for some free consulting right now. We’re going to share it with a lot of people. So we’re going to talk about data definitions, getting to those standards and those kinds of things. And let’s start with what are the characteristics of a high functioning healthcare system with regard to data and analytics?[00:15:00]
[00:15:00] Paula Edwards: Well, there are a few characteristics. I think number one is culture. You have to have a data-driven culture and for me that culture starts at the top. So I include your leadership in that culture statement. You have to have leaders who are data driven and who are going to invest in analytics.
[00:15:19] And that means both capital and people’s time and analytics and they have to hold all of their direct reports and the organization as a whole [00:15:30] accountable for actually using information to inform decisions. You can’t just say, oh, we’re gonna put out a balanced scorecard. And check the box.
[00:15:38] You actually have to hold people accountable for their performance on that balance scorecard. And if things are going in the wrong direction, they have to be able to answer why is this happening and have data to back it up. So really truly having that data-driven. To me, that’s number one you can’t be high performing without that.
[00:15:56] Obviously you have to have data platforms. [00:16:00] And this, when I think about platforms, it’s the platforms to integrate the data and curate it to the degree that is necessary for how you want to use. And has a range of analytical tools on top of that to meet your target end users. So for a less mature organization, you might start out with those tools, very focused on like information visualization and dashboarding and reporting.
[00:16:25] But for more mature organizations, you’ve got to have those more advanced analytics tools, so that [00:16:30] you’re really power user analysts and data scientists. Can really explore and innovate and do some really exciting things with the new generation of machine learning tools, data, prep tools. There’s a lot of things people can do with data these days. If you give them the right tools and have the data available.
[00:16:48] Bill Russell: When we talk about healthcare, sometimes we talk about it like it’s one big homogenous thing, but you know, the academic medical center or the community health system, the large IDN, the the two hospital systems, community [00:17:00] hospital, these are very different.
[00:17:01] Obviously the analytic strategies will be different based on budget. So let’s do this. Let’s start with you know that the two hospitals system. They have a set of tools that come with the EHR and a set of tools that come with the various things. How do they get started if they’re that small of a system?
[00:17:20] Paula Edwards: I think you really do want to double down on the tools that are available through your core EHR vendor and really take advantage of that reporting [00:17:30] content and the dashboarding content that you’ve basically bought with your EHR. With the smaller hospitals you’ve got less talent. So you’re going to have less opportunities ready to do things your way.
[00:17:44] So you’re going to have to align more on industry standard metrics, industry standard definitions, to the degree that you can, because quite frankly, you just, aren’t going to be able to afford and get the talent to be able to do it your way, which in academic medical centers, they [00:18:00]always want to do it their way because their definition is better for them than whatever the industry standard.
[00:18:06] Bill Russell: Right. To a certain extent they’re driving the industry, the academic medical centers.
[00:18:10] Paula Edwards: They are. They’re thinking down the road. And quite frankly they have the staff and the academic knowledge and things like that. And they should be doing those things to kind of push the industry forward. So I think you really do have to level set based on the realities of your budget and the [00:18:30] sophistication of the organization. And just the availability of analytics talent in your area.
[00:18:34] Bill Russell: Which way are we seeing the academic medical centers go? I mean, clearly we want to consolidate, we want to standardize but there’s a lot of self-service needs across academic medical centers. There’s a lot of people doing individually funded studies and those kinds of things. Are we still pulling everything in and then creating ways for them to access that data, or are we still finding silos of data out in the academic medical centers? [00:19:00]
[00:19:00] Paula Edwards: Oh, there’s lots of silos of data out there. A lot of the academic medical centers that I work with are doing a better job of building platforms that bring the data together, but what you’re seeing now is more let’s for example, in a cloud analytics environment, let’s land it somewhere where everybody can get access to let’s provide playbooks of how you link different data sets. So they’re having this sort of either uncurated or lightly curated data [00:19:30] available more broadly because you can do that much faster and give people faster time to data.
[00:19:35] And in an academic medical center, a lot of the people who are doing a lot of the modeling and research, they want that raw data. Like they don’t want it curated. But when you get to the operational people who are taking care of patients and are really running the business of of care delivery in his academic medical centers, they need more curated data to be self-served because if [00:20:00] you’re a nurse manager on a unit, you don’t have time to go look at raw data. You need to be able to click and pull up a dashboard to see your metrics and see if something’s trending in the wrong direction that you need to take action on. So you have to have different levels of analytics for the different self-serve audiences in your organization, even if you’re an academic medical center.
[00:20:22] Bill Russell: All right, so let’s, let’s push a little deeper. Well, let’s start at the top. Let’s start in a data governance meeting. Who participates in data [00:20:30] governance and what goes on at a good data governance meeting?
[00:20:33] Paula Edwards: So my dog Lula is very excited that we’re moving to data governence. She loves it when I talk data governance.
[00:20:43] If I think about it, there’s sort of two layers of data governance that every organization needs. There’s an executive level data governance and then there’s a more tactical kind of data stewardship level of data governance. And the people who need to be at the table and what they need to be focused [00:21:00] on is different at those two levels.
[00:21:02] So at the executive level, this is where they should be making strategic decisions. What is important for us as an organization to do with analytics? How are we going to allocate our enterprise analytics resources, whether that’s capital dollars or staff in order to get the most value from our portfolio of analytics and data assets.
[00:21:26] So that means you’ve got to have executives there and you need to have a cross [00:21:30]disciplinary set of executives there. So you need somebody from quality finance. It. Compliance risk. I’m hearing a lot of organizations these days that one of their big drivers for sort of breathing and then governance is all sorts of questions are coming up about third-party access to data.
[00:21:49] Somebody wants to buy the healthcare’s data or there’s an it vendor that wants to partner to develop some cool new machine learning algorithms. [00:22:00] Do we give them access to our data? What do we expect to get in return from it? Are our patients going to be okay with this? If we give this access to this data, those are decisions that executives need to be making, not a project manager in IT who gets an extract data request.
[00:22:17] Bill Russell: I can only imagine. Actually, we could spend all day on that. Providing data to third parties and how to protect that data, how to anonymize that data, how to ensure that [00:22:30]you’re not allowing your data to just run loose in the wild. So I like that meeting. I assume clinicals part of that as well.
[00:22:36] Paula Edwards: Clinical absolutely needs to be part of it. And realistically, you need somebody to represent the acute hospital side and somebody from the physician practice side, because a lot of times the considerations are very different between the two, the different settings of care. And if you ha, if you are a large integrated delivery network that has home health or things like that, you probably want somebody who can at least talk to those needs and considerations too.[00:23:00]
[00:23:00] Bill Russell: So who’s at the tactical meeting and what goes on there?
[00:23:03] Paula Edwards: The tactical meeting is going to be more what I call the data steward level. So these are people who really know and use the data on a regular basis. And they’re either going to be Analysts people who are over the departments, they’re actually creating the data.
[00:23:20] So our director of revenue cycle and data owners potentially, it’s going to be a mix depending on exactly what you want to focus on. A lot [00:23:30] of organizations when they’re starting that data steward layer have problems with data standards 15 different definitions of length, of stay, 12 different ways to count visits.
[00:23:42] We’re getting conflicting data. We have to do about something about this. Let’s get a group together to decide how are we actually going to count these things and what are the metrics that matter to our organization? And so these are the people who can really say, this is how this group has always counted it. The other group [00:24:00] says here’s how we’ve always done it. And they have to come to consensus on how are we going to do it as an organization going forward.
[00:24:06] Bill Russell: Are you saying an organization can get to one definition of length of stay?
[00:24:09] Paula Edwards: No, they probably can not. And that’s okay. It’s okay. And my rule is always, if there is a valid business or clinical reason to have two or more definitions of length of stay, that is fine, but we need to know what they are and we need to call them something different.
[00:24:27] So you can’t call them all length of stay. [00:24:30] At one of the organizations I worked with, we ended up with length of stay, ended up being the clinical length of stay. That was actually the difference in time between when they showed up on our unit. And when they left. And that was the length of stay because the clinical operation said that’s how long we have to care for this patient.
[00:24:49] So we need to know the duration and need to staff. Finance wanted to count length of stay as heads in beds that they got to bill for. And so finance ended up having to [00:25:00] call that build patient days. And so we solved the problem by saying, okay, there’s two needs. We’re going to have two metrics, but we’re going to call them something different and this is how we’re going to do it going forward.
[00:25:10] Bill Russell: Isn’t there a regulatory definition or a CMS definition for length of stay though that you have to track?.
[00:25:16] Paula Edwards: There is. And we had that one too. I know cause it has all these weird nuances. Like if somebody comes back after they got discharged. And so there was a CMS length of stay also. So they got to,
[00:25:27] Bill Russell: But that’s what, that’s what goes on at that meeting? [00:25:30]
[00:25:30] Paula Edwards: That’s what goes on at that meeting. So that’s why you don’t want your executives in that meeting because they’ll check out and never come back. But you don’t want that group deciding whether or not you’re going to partner. Google on some AI initiative.
[00:25:45] Bill Russell: But that group could use the executive group as an escalation path if they come to an impasse.
[00:25:52] Paula Edwards: And that’s typically what that executive group is, is an escalation path for things like policies around who gets access to [00:26:00] what if the data steward group can’t get to consensus on a standard definition and they need somebody to basically be the tie breaker and tell them how the organization wants to do it, they can escalate that kind of stuff there.
[00:26:13] Bill Russell: When you’re setting up these data governance teams or these groups I we used to always have charters for these. Do you recommend a charter and is there anything specific you would put in those charters?
[00:26:24] Paula Edwards: I do recommend a charter and really, and I’m seeing some organizations getting away from charter and more [00:26:30] defining a data governance policy, but it effectively has a lot of the same stuff that a charter does. It’s just saying here are the roles that are going to be part of our data governance program. Here’s what the different people that are part of our data governance program are responsible for and accountable for. So who gets to decide on budget and resource allocation, who gets to decide on source of truth and data standards, who gets to decide on third-party release of data and [00:27:00] internal access to information. And really having those roles and responsibilities and accountabilities really clearly defined. So everybody’s basically operating from the same playbook.
[00:27:11] Bill Russell: Interesting. So a policy-based approach to it and a charter based approach to it.
[00:27:18] Our charters were pretty simple. It just said this is what the group is. This is the makeup of the group. Here’s what their their core responsibilities are as a group. That’s essentially what that said. We still had policy documents that sort of drove [00:27:30] how we approach data governance.
[00:27:33] Paula Edwards: Yeah and the governance groups are going to be responsible for that policy. And so it’s really like how does the organization want to drive their data governance program? I’m for less documents and red tape where that’s feasible. So we’ve kind of in a couple of cases, turned a data governance policy into sort of mini charter for different groups so that it’s all there together. And everybody’s got the same understanding of what the other groups are responsible for.
[00:27:59] Bill Russell: All [00:28:00] right. I want to ask you about organizing analytics. I’ve seen so many models. It’s kind of mind boggling at this point. I’ve seen a Chief Data Officer have the whole thing I’ve seen the CIO have it. I’ve seen the CIO not have it and it’s I don’t know, sort of in a, in a weird space within the organization, not necessarily to a data person per se, but just another group. It’s interesting to me how many different models I’ve seen and I haven’t really put my finger [00:28:30] on what’s the most effective. Is there something that’s emerging in terms of either principles around how we organize this that works the best?
[00:28:38] Paula Edwards: I think so. I mean, when you get into the details, there’s a very wide range. And quite frankly, what will work in an organization is very culture and context dependent. But I’ll tell you what we know doesn’t. A fully decentralized model does not work because that’s why so many organizations have ended up with silos of information and [00:29:00] conflicting definitions. It’s just very inefficient. And it’s not good for having enterprise view of data. If you go to the other end with centralized, that’s not great either, because once you centralize everything, you can never keep up with demand and everybody’s always complaining that how come you did their stuff first? And I didn’t get mine. So the emerging best practice is really a hybrid model where you’ve got certain core resources that are centralized so that you get economies of scale and you can have those [00:29:30]resources really focused on that enterprise platform build. So what’s the 20% of the data that everybody is using, that we need to be using a common language and a common platform on. And then you still have embedded analysts, especially in some of the really analytics, heavy departments, like finance, like quality, et cetera.
[00:29:50] Bill Russell: Yeah. I’m thinking, how do you take care of that rogue area? The ambulatory group is going off and doing their own thing. They’re buying a new tool because [00:30:00] essentially you have this hybrid model, you’re providing some services, but what you’re saying by that hybrid model is essentially you’re going to have head count out in all the departments.
[00:30:08] The departments are going to start hiring analysts. The analysts are going to be working with their constituents. They’re going to identify analytics, they need for a clinically integrated network or for population health or whatever it is. And then before long, all of a sudden you have this group that’s growing into its own analytics arm of the [00:30:30] organization. How do you keep that from that?
[00:30:33] Paula Edwards: It’s a constant battle. It really is a constant battle. There’s a couple of different strategies. One to your point about going out and buying their own tools. That’s where your executive level governance needs to hold people accountable because they should be the clearing house for purchase of any new analytical tools like that budget shouldn’t get approved until that group has signed off on it.
[00:30:55] And so they need to hold the various [00:31:00] departments accountable for not going rogue when it comes to technology purchases. And that’s where that whole leadership thing and part of my. This is what a high-performing analytics organization does. You’ve got to have that leadership. That’s going to say no.
[00:31:15] The other thing is you really need to concentrate in your centralized in your hub of enterprise analytics capabilities on showing them that they can be more efficient and more effective by working with [00:31:30] you then by working on their own. So being able to leverage all of the data sets that are available in the enterprise platform, all of the tools and the subject matter experts in those tools that should be in your hub.
[00:31:44] If you can say if you’ll work with us and partner with us and be engaged in the data stewardship processes and collaborate with us on projects, you can draw on all of our technical experts and our data experts. And we’re going to accelerate your timeline to insight [00:32:00] and your timeline to data.
[00:32:02] Then you can get them to be more of a partner where they’re working with you instead of just going totally off on their own. And that’s what you want cause really eventually what you want to get to is where those departments that have their own little analytics enclave are sort of a prototyping and an experimentation platform.
[00:32:21] And then when they find something, they say, Hey, this is great. We should spread it to the rest of the organization. And they put that in the pipeline for the enterprise team to be able [00:32:30]to build and spread out to other parts of the organization.
[00:32:33] Bill Russell: Is the data any cleaner than when we were working on it in 2015? Or how are we making progress or is it just, is it a system by system down in the weeds kind of thing. I know that there was always this thought that like a Google was going to come in and take all the health data, normalize it and present us this really phenomenal interface. And they’re doing something right now, along those lines but they’ve also written a paper that essentially said, no, you’ve got to clean up [00:33:00] your data. The best tool in the world still can’t make sense of your data. If it’s this messy.
[00:33:05] Paula Edwards: The data is still messy. And I think it at its very nature, healthcare data, and especially clinical data is just messy. It’s unstructured and it’s being captured by a lot of different people, about a lot of different patients that are just a highly varied population. So it’s always going to be a little messy. I think whenever I have conversations about data quality, [00:33:30] what I like to focus on is data quality that is fit for purpose. And so to the Googles and others of the world, there’s sort of two questions that I would have. One is given the quality of the data that we have.
[00:33:44] What can you do that’sof value. Like, what is it? What is the purpose it is fit for because I guarantee there’s something valuable they could do with the data.
[00:33:52] Bill Russell: Yeah, there is. And I brought that up as an example and they just came out with something. I have this story here. I forget what it is. [00:34:00] They just came out with this tool that essentially sits on top of this clinical data across a bunch of different EHRs and it provides value. I mean, it provides a single interface. That’s searchable for clinicians and it does that kind of stuff. So I’m not, I’m not saying they weren’t able to create value out of it, but I am saying they also wrote a paper.
[00:34:19] A peer review kind of thing they put in to the academic side saying essentially, Hey, here’s the problems that you have. We’ve looked at a lot of healthcare data now and [00:34:30] here’s what needs to happen.
[00:34:31] Paula Edwards: I think that’s what they should be doing is providing that feedback loop to the people who are capturing the data to say, if you want to enable these future use cases, and here’s what you need to do to address the data quality problems.
[00:34:42] But I think making the data quality in healthcare better goes back to one of the things we talked about at the beginning, which is that user centered design. To get better quality data we’ve got to make it easier for the data to be captured cleanly. Because right now, with the [00:35:00] current generation of EHR, it’s really time-consuming to capture really good quality data in an EHR, and we’ve got to make it easier.
[00:35:09] Bill Russell: When you’re designing an application, you almost have to start with who’s it being built for and what’s its purpose. And I hear this all the time, the EHR was designed to capture billing. And because it was designed to capture billing, and now we’re trying to do public health things with it.
[00:35:24] We’re trying to do population health with it. We’re trying to, we’re trying to do a lot of different things with it, but [00:35:30] that wasn’t its original purpose. So now we’re retrofitting this whole thing to capture better clinical data, more discrete data around the clinical event so that we can start to provide value that we need through a pandemic. And so is it a problem of definition of what is this thing here for, and what’s its core mission as an EHR?
[00:35:54] Paula Edwards: Well, I mean, I think that’s one way to look at it. What I’m really interested to see and excited [00:36:00] about the opportunity for, as there’s a lot going on in the world of natural language processing.
[00:36:05] And there’s a lot of talk about Amazon Alexa and other ambient listening devices capturing and doing some of the documentation in the clinic room. And to me, those, if they do them right, could just really have a good opportunity to make the data capture easier for the clinician. But capture more useful data and better quality data that can be used for all those [00:36:30] downstream uses.
[00:36:30] So do we want to spend a lot of time and money going back and trying to retrofit how we capture things in the EHR today? Or do we want to layer some of these newer technologies on to find a better way to capture more useful, cleaner data in an easier way?
[00:36:46] Bill Russell: I hear that. And I’m having flashbacks of you saying layer some things on and I’m sitting there as the CIO going yeah every time I have to put a new tool on it’s, it’s a million dollars and I have to hire five people to feed that tool. And there’s [00:37:00] another five people times you know Southern California, a hundred thousand dollars a person there’s half a million dollars to maintain it every year ad we haven’t even done maintenance or anything to that effect.
[00:37:10] Paula Edwards: The health system is the one that has to do that. I mean, could the EHR vendors start building this into their products? That’s part of what you get with their core EHR.
[00:37:20] Bill Russell: They call it modules. They increase my price. Anyway, we talked about the pandemic and you have public health is part of what you [00:37:30] teach and talk about.
[00:37:30] Let’s talk a little bit about the impact of the pandemic on the way we interact with data, the way we view data practices in healthcare, will it be fundamentally changed as a result of it?
[00:37:43] Paula Edwards: I think it should be. I think will is the key question. I think that’s going to require some investment in our public health infrastructure, and quite frankly, I think we need to rethink fundamentally how we’re capturing and exchanging data for public [00:38:00] health. It was really interesting Providence St Joe’s is where patient number one in the US got diagnosed for COVID and talking to some of their analytics folks there they said the challenge in the first few weeks was there was no ICD 10 code for COVID.
[00:38:15] So there was no way to code that diagnosis. And for the first few weeks, we didn’t really have a valid test that people trusted that would tell us if the patient was COVID positive or not. So how do you use the data that is captured [00:38:30] when it’s something new and be able to kind of have that early warning system that, Hey, we may not know the name for this yet, and be able to code it as a diagnosis, but we know something is going on here that we need to pay attention to.
[00:38:41] So I wonder if we need to be thinking about how do you look at variation in symptoms and vitals and things like that as an indicator that something is going on as opposed to waiting till we can actually name it?
[00:38:58] Bill Russell: Well, it’s, it’s interesting that you [00:39:00] bring that up because that was February 29th. First reported patient at Providence St. Joe’s. I think history will tell us that was not the first patient in the United States. But, I mean, so we’re going to look at all this data retrospectively through through Epic’s platform or through Truveta or through wherever we’re going to start pulling this data to, to do this. Are we going to be able to find that signal that said, Hey, here was the start of the pandemic in the United [00:39:30] States?
[00:39:31] Paula Edwards: I don’t know. That’s a good question. I think if you … now that they know more about the symptoms and the, like the physical presentation of COVID-19, they could go back and look at data and figure that out. But you know, it’s getting access to that data and having the right tools to be able to basically tease that out of the medical record.
[00:39:55] Bill Russell: Yeah. Cause a lot of that’s going to be in the notes, isn’t it?
[00:39:57] Paula Edwards: It would be clinical notes. So you’re going to have [00:40:00] to look at NLP and I think actually, I think Providence St. Joe’s did actually create an NLP algorithm that let them target people that they suspected were COVID patients. And then they could go and sort of validate and confirm whether or not that was the case.
[00:40:17] Bill Russell: One of the things that happens is, we just see this over our lifetime the federal takes a stronger hand during certain times, and then a little lighter touch at other times. The pandemic [00:40:30] seems to me to be and the need for better public health seems to be a case where the federal and even state will start to weigh in a little bit on this data conversation. Defining the data. What data are we going to collect on report and those kinds of things. Do you think that will be a factor? Do you think that the top-down factor will be of the federal and state coming to bear on this is going to have an impact on how we collect data at the [00:41:00] field level?
[00:41:01] Paula Edwards: I think it will. And I really hope that in addition to funding to do that work, what they really do is they bring the right stakeholders together to talk about how that looks, because what we’ve seen through meaningful use and other things like that is. You really have to think about what’s the impact that data collection is going to have on the people who are caring for patients, because if it’s putting a huge burden for the care providers to do [00:41:30] manual processes and things that are going to be time-consuming, if you’re in a pandemic or other health emergency, you’re taking them away from patient care.
[00:41:38] You’re taking them away from prevention measures and things like that. We’ve got to find a way to make it more seamless. And require little to no effort on the part of the health systems and the care providers to be able to collect and send that data. It should be collected in the background as they’re providing clinical care, instead of being like we’re going to deliver care. And [00:42:00] then we’re going to go back and documented up a bunch of stuff on the backend.
[00:42:05] Bill Russell: We are seeing it and I’m curious about this, we’re seeing a lot of these big data data storage. We’re seeing a Mayo partner with Google, Ascension partner with Google. We’re seeing Truveta come out and there’s others that have been out prior to that.
[00:42:18] I guess my question is is that a trend? Are we going to keep seeing this consolidation mass data sets or is that just for a specific use case? Is that just for [00:42:30] pharma and population health and research and those kinds of things, is that what that’s for and where we still have a significant need for the operational types of data source and data practices.
[00:42:43] Paula Edwards: I think you’re still going to have the operational data practices cause a lot of what those, those larger integrated data sets are for is that discovery. And anything that’s gonna involve AI and machine learning development the more data you have, the [00:43:00] better you can get in terms of your models.
[00:43:01] And especially if you’re trying to develop something, that’s going to generalize across populations. You’ve got to have access to data from a lot of different organizations for the model to actually work in generalize. But I think you’re always going to have your local, we’ve got to run our business and we need our operational data to manage our populations, manage our financials, manage our patient flow.
[00:43:27] And I don’t see a lot of those big [00:43:30] aggregators of data, really focusing on that. I think they do have a little bit of an opportunity there, and there’s a couple of places that, that Didas, you know, premier and others, if you can give benchmarking data, like how does my revenue cycle performance look versus my peers?
[00:43:44] Do I have an opportunity to improve? Or am I doing pretty good? And what can I learn for those people who are the top performers? So I, I don’t see those big aggregators of data focusing on some of this just operational delivery of care. [00:44:00]
[00:44:02] Bill Russell: I started asking this question to close out my interviews and it sounds like a cop out kind of question, but I’ve gotten some great answers. What question didn’t I ask that you’re surprised? What area didn’t we talk about or question didn’t I ask?
[00:44:15] Paula Edwards: Well, I think there’s two topics that didn’t come up that that I’m kind of surprised. One of them is data literacy. Because if we go back to your question about what are the characteristics of high performing, we kind of got off topic. The data [00:44:30] literacy is definitely one of those because you can have a great culture and you can have great platforms and great data governance but if you don’t have a data literate population that can put all of those things into practice, they’re never going to have your analytics really impact care delivery or organizational outcomes.
[00:44:50] So data literacy, for sure. And then the other one that I’m hearing a lot more talk in the industry about that. I’m not seeing a lot of people doing it in practice yet is [00:45:00] model governance. So as people start to put predictive models and machine learning into practice and operational processes who’s going to decide is this model performing well enough that we will put it into production and who’s going to monitor and manage that over time to make sure that if you have data drips, the model’s still performing. And we can’t roll all the models out at the same time, like who decides, which ones get implemented when. And a lot of organizations [00:45:30] aren’t thinking about that yet.
[00:45:31] Bill Russell: All right. So let’s hit data literacy real quick. What does an effective data literacy program? I mean, it almost has the connotation of it. I mean, the word literacy is learning to, learning to read. I mean, it almost seems very fundamental.
[00:45:47] And so we have to give everyone sort of a foundational level of the data, how to read it, how to understand it. And the definitions that we’re using, those kinds of things. Do you end up with a training department that does that?
[00:45:57] Paula Edwards: I think it becomes part of your professional [00:46:00] development. You’ve got training programs on clinical competencies and other professional competencies. You need to have one on data. And there’s a lot of parts to it. A lot of organizations where they fail in terms of adoption and utilization of analytics is they train people on the tools, but they don’t actually train them to understand the data and how to act when they see the data I know how to pull up the dashboard, but what do I do when the metrics going in the wrong direction? How do I drill in and figure out what’s [00:46:30] the root cause? And what are the leading indicators that I can start watching so I can intervene early. And literally data literacy is one of those things that if you’ve got a good program, I think it’s not one size fits all. The level of data literacy and the specific skills and knowledge that revenue cycle manager or a physician needs to have is going to be very different than the level of literacy a finance analyst needs to have and say, your program needs [00:47:00] to be a little bit modular and flexible to meet the needs of the different constituents in the work organization.
[00:47:05] Bill Russell: Are the colleges and potentially med schools, are they putting out people with a higher level of data literacy, just general data literacy that we can plug into?
[00:47:16] Paula Edwards: That’s a good question. I don’t know. I do feel like there’s more technical literacy and there are a lot more people coming out that have been exposed more to data and analytics through. Throughout college [00:47:30] compared to like 20 years ago. So I think progress is being made. I think the other thing just for, for people to know is you don’t have to go create a bunch of content to put together a data literacy program.
[00:47:42] There’s a ton of really good content out there already. You really just need somebody who’s going to kind of curate that content and put it together into curriculum for organizations that use LinkedIn learning. There’s tons of things out there on Tableau and Excel and how you [00:48:00] use those things and some basic concepts around data. The main content you’d have to develop internally is about understanding your data and your metrics. But you don’t want to have to build it all from scratch for sure.
[00:48:13] Bill Russell: Are there good programs out there to teach you Lotus 1, 2, 3, and Harvard graph.
[00:48:20] Paula Edwards: I don’t know. I haven’t seen any.
[00:48:22] Bill Russell: Model governance. It’s interesting to me, cause model, we used to go out and buy models, essentially. You would hear of another health system that was doing something [00:48:30]effective and it was integrated into some tool and you, you ended up either lLayering that into your EHR or layering that into another program or even some of the analytics platforms.
[00:48:40] Now you can buy some of these models and bring them across. It’s interesting to me. I understand the concept of model governance but sometimes my concern is about the black box of the model. If you’re buying the models, a lot of times people don’t want to tell you what’s really all in the model.
[00:48:58] Paula Edwards: Yeah, and there’s a lot of [00:49:00] research going on around the concept of explainable AI. And especially when you get into like clinical scenarios, it’s going to be a lot easier to get trust and in the models and adoption and use of the models, if you have explainable AI so that you can see what are the factors that drove this risk score. And I can use my experience to judge whether that makes sense. Or if I think something strange is going on. The other thing is for organizations that aren’t going to develop their own models, they still need [00:49:30] to have model governance.
[00:49:31] If they’re going to be buying and implementing models from their core EHR vendor or other vendors, because they need to be asking for data about how was this trained? What’s the efficacy? Let’s run it on our population and see if it keeps that model efficacy.
[00:49:46] There’s all these publications on we can use models to do these things, which is great but to actually put it into production and operationalize it, you really need to understand how effective is it. And was it trained on a [00:50:00] population that was comparable enough to mine that it’s actually gonna work on my population.
[00:50:04] Bill Russell: Wow. That’s really interesting. Well, thank you for coming back from your vacation to just spend time with us. We really actually, you may still be on your vacation. Cause people are, if they’re watching on on our YouTube channel, they’re probably saying that background can really be anywhere in the world now. You really don’t know where people are anymore, but you are, but you’re back in Atlanta now.
[00:50:26] Paula Edwards: Yeah. Well, it was great to talk with you today. I really enjoyed this. [00:50:30]
[00:50:30] Bill Russell: Thank you very much. Appreciate your time.
[00:50:32] What a great discussion. If you know someone that might benefit from our channel, from these kinds of discussions, please forward them a note, perhaps your team, your staff. I know if I were a CIO today, I would have every one of my team members listening to this show. It’s conference level value every week. They can subscribe on our website thisweekhealth.com or they can go wherever you listen to podcasts, Apple, Google, Overcast, which is what I use, Spotify, Stitcher. You name it. We’re out there. They can find us. [00:51:00] Go ahead. Subscribe today. Send a note to someone and have them subscribe as well. We want to thank our channel sponsors who are investing in our mission to develop the next generation of health IT leaders. Those are VMware, Hill-Rom, StarBridge Advisers, Aruba and McAfee. Thanks for listening. That’s all for now.