Strategies for Keeping Users Trained Up on Ever-Changing Applications

Anthony Guerra
Good afternoon and welcome to Strategies for Keeping Users Trained Up on Ever-Changing Applications, a healthsystem CIO Media Inc. production sponsored by 314e. Just a little housekeeping before we get started. My name is Anthony Guerra. I'm the Editor-in-Chief at healthsystem CIO, and I'll be your moderator today. We're looking forward to your participation. You can send in your questions or comments at any time in the Q&A box, and we'll take those later in the program. Just so you see how we're going to spend our time today, first, we're going to go about 35-40 minutes with our main panel discussion featuring Dr. Alistair Erskine, CIO and Chief Digital Officer with Emory HealthCare and VP for Digital Health with Emory University, Lisa Stump, SVP, Chief Information, and Digital Transformation Officer with Yale New Haven Health System, and Ryan Seratt, Director of Training and Development with 314e. And then, we will have our Q&A. So let's jump right into our discussion. Alistair, let's start with you. Can you give us an overview of your organization and your role?

Alistair Erskine
Sure. So I'm Alistair Erskine. I'm a pediatrician and internal medicine physician. I'm the Chief Information and Digital Officer for Emory HealthCare and Vice President for Digital Health for Emory University. And so my scope is all things digital transformation for the organization, ranging from Electronic Health Records, activity-based cost in, you know, ERP, and any number of different applications, data platforms, kind of the overall data architecture of the organization. I will say I'm new to the role in the past couple of weeks. I'm just putting down the fire hose for a second to do this, and then I'll pick it back up as soon as it's done.

Anthony Guerra
Well, we certainly appreciate you putting it down for us. Lisa.

Lisa Stump
Hi. Good afternoon, everyone. My name is Lisa Stump. I am the Chief Information and Digital Transformation Officer for the Yale New Haven Health System. I have been in my role for just about seven years now with our organization for a little over 26 years. Began my career as a clinical pharmacist, came to healthcare IT with a passion for improving the usability of technology to help clinicians take better care of the patients that we are privileged to serve. And I continue to be impassioned by that charge. So really pleased to be with you all today.

Anthony Guerra
I cannot believe it's been seven years, Lisa. Time flies. It's amazing. That's wonderful. Ryan.

Ryan Seratt
Hi, I'm Ryan Seratt. I'm the Director of Training and Education for 314e. 314e has been in the IT healthcare field since 2004, providing services around IT, around staffing implementation, data. And recently, we've gotten into training as well for the digital training transformation that's helping people scale and provide training in our ongoing field as things continue to change at a rapid pace. And we're starting to look at how can technology and human interaction be blended together for the best results.

Anthony Guerra
Very good, Ryan. All right, let's jump right in. Lisa, let's start with you. In a health system, who is responsible for ensuring that users have received the proper training to navigate the applications they need to do their jobs? Is this a shared responsibility between enterprise leaders such as CIOs, CMIOs, etc., and department-level operational leaders such as the users' managers? How would you describe your responsibility, because it may be different from organization to organization and role to role? And who are the key partners you have to work with to help move this forward?

Lisa Stump
Yeah, thanks, Anthony. I do believe this is an absolute shared responsibility that starts with the operational owners in the area really helping to inform, as Subject Matter Experts, the expectations of the technology, the utility of the technology that we are putting into place, and the workflow in which that technology will be used. And I'll underscore the workflow component, as our training is always grounded in that component. If we simply train people in the points and clicks of how the technology works, we've really not done our job in training them overall on how to best utilize the technology. That partnership also obviously includes our IT experts. So those that are Subject Matter Experts in the technology itself also provide a key element of informing what that training includes, what are the core features and functionalities, what are the nice-to-know features and functions. And we often frame our training in that way. We frame it as sort of the 100-level class training, what's the minimum required information to do my job safely, well, effectively. And then let's bring you back and share with you what are the other features and functions that ideally make the technology a joy to use. We try not to overwhelm people by intermingling those two too soon before they have a basic understanding of what the technology can do. My responsibility for that; we do operate a training team within our IT and digital enterprise that supports all of the EMR-related solutions, the EMR itself, as well as any of the clinical solutions that sort of touch it, as well as all of the ERP and business applications. And so my responsibility is to ensure we've got the right trainers that are able to both deliver the content but do it in a way that engages our users in a personalized way, that we are creating not just content but a program that meets the needs of our users and the organization. I've touched on it, but the key partners are clearly my fellow leaders in the organization who are responsible for setting a clear expectation around completion of training and why it's important. As I said, the key Subject Matter Experts in the areas where our technologies will be utilized, the users themselves, have a key role in this and giving us important feedback around whether our training is meeting their needs or not. I might have missed a few, but I'll leave some opportunity for others to comment. But I'll just emphasize the nature of that shared responsibility is key. Very good.

Anthony Guerra
Very good. Alistair, your thoughts?

Alistair Erskine
Yeah. So to answer the question directly, I'm responsible for the training for the digital systems applications and so forth for the organization and have a team who's dedicated to that. I guess one of the ways to answer the question is kind of, of course, with a question which is, OK, what do we mean by training? Because is training something which is taught in a classroom? Is training something that occurs as we are discussing workflows, and some people come to learn that people function in different ways across the organizations, have different skill sets, and so forth? Is training come in the form of a request for something that actually already exists, and therefore there's an opportunity to learn about that? So I think that the training piece, and I'm sure others would agree, isn't a discrete component that's delivered over a small period of time, but instead something that occurs over a longer time frame. In fact, the training is forever. The system is changing forever. Medicine is changing. The workflows, new technologies changing forever. Therefore, it's continuous training. And so, to answer the question in terms of whose responsibility is, I completely agree with Lisa. It's going to be a shared responsibility because as the organization changes, then all the various different stakeholders, whether it's operations, management, the digital, and so forth, have to change along with it. One way to kind of split it out is to say, all right, what are the sources? What are the deliveries, and what are the competencies for it? Who's responsible? Which piece? So the sources come from many different areas. Of course, the Electronic Health Record vendor, for example, will provide some canned stuff that's not specialized to your particular organization. Those who provide tier-one support will have all kinds of ideas on what the users need help with and don't know. And, of course, those who perform the training will also have a good idea as to where there are holes and so forth. Critical feedback, the ability to kind of ask a tiny subset of users what are their ideas and opinions about the system and delivering that across all users at least once a year, is another way to gather what's missing and what's needed. And the other problem from a source perspective is unfortunately training tends to be at the tail end of any sort of change product because you talk about it, you build it, you promote it to your production environments, and then the poor trainers at the very end can snap their snapshots and then try to put the training together before it actually goes live and ideally get a few folks online with some of that knowledge before it goes live or that it may be. So they always sort of tucked in the caboose of the train, trying to do their best to get in front of the users to teach them whatever new system. So that's the source. Now delivery needs to come in a number of different ways. We're talking about adult learners that learn in different ways. And like Lisa said, there's a kind of a 101 class, maybe there's a 201 class, something that you provide as basic, how do you log in, how do you navigate the system? And then later on, how do you make shortcuts, and how do you kind of become more efficient? So the timing of that content delivery is very important. And then it also depends on the role, whether you're talking about a front desk staff, revenue cycle staff, a clinician that's doing a patient-facing role, it will depend a little bit about how you actually deliver that. Now traditionally, back in the day, we used to think that bringing people into a classroom, taking them away from work, and doing all the training there was the way to do this. And what we found is people would exit that class, less than 10% of the information they would remember, and they can kind of get in front of the computer, log in, and then they were sort of done. So over time, that classroom training has come closer and closer to the front line where you have at this elbow support. And I think, again, the up-the-elbow support piece, or even virtual at-the-elbow support, meaning you call a phone number within a couple of rings, somebody is in your session, moving your mouse around, showing you how to change something, that ends up being a better strategy. And in fact, the play domains that we would give people, say, okay, go and practice, don't tend to get used the way that we would hope they'd get used; people are busy, they don't want to spend the time. When they get stuck on something, they'll learn forever when you help them at that moment. So that sort of Just-in-time training in terms of the delivery piece is really important. And then you can outsource training; you can send it elsewhere and just completely get out of the business. And then, of course, the last piece is just competency. So did you get anything out of the training? What is the way to understand that in terms of how is the system changing? And then when do you test? How do you test? Do you do that sort of hard stop if you're not competent enough and you can't get out? Is a driver's license. If you fail the test, you drive the car, and you run into a tree, maybe you shouldn't have a driver's license. And the same thing with some of these systems, which are life-saving; you want to make sure people really know. You want to take people who have already been trained elsewhere and make sure they're competent, or take people who have never been trained elsewhere and make sure they get the full suite and then provide that access control for competency. So those, I think source delivery competency are important dimensions. And that probably spread over a number of different people across the system, just given the immensity of what we're talking about.

Anthony Guerra
Very good, Ryan. I have a feeling Alistair was saying a lot of things that you understand and agree with. His philosophy there, breaking it down into those three parts, how does that sound to you?

Ryan Seratt
Yeah. So, you know, I think everything that Alistair and Lisa said are kind of spot-on on what an excellent training program looks like is that people can only learn a certain amount of information in one sitting. And once you actually reach that limit, anything else is just really just kind of lost. They have to be able to apply that knowledge to really internalize it. And Lisa was talking about she brings people back in for training. So when they're ready to accept more, they've gotten their initial checkmark. You know, the event is completed. That doesn't mean the learning is completed. And that learning is an ongoing process that as you're ready to take in more, you're ready to move to your next level of proficiency, then developing programs like that are very important. And it does take a lot of different players to really the operations needs to determine what the success factors are or what are, you know, what are the objectives of the training? How is it delivered? What are those proficiency levels and working together with technology and training, and operations to deliver that?

Anthony Guerra
Very good. All right. Lisa, let's start with you. Some of the best practices for keeping users trained on key applications discussed. So you have different times when they need training. And some of these are easier than others. So when somebody is hired, that may be fairly easy to structure training on that. We know when it's happening, they come in. When the new application is rolled out, again, it's sort of a big-picture thing happening to everybody. And perhaps the third one is the trickier one. On small changes that are continually happening, as Alistair said to applications. But talk to me about handling training at those different in those different situations.

Lisa Stump
Yeah, Anthony, I think you're spot on, you know, new hire training, we tend to have pretty well developed. a playbook for, right? Every new hire to the organization, based on their role receives that baseline training in the ERP system, in the Electronic Medical Record. And more and more, we're finding the complexities of our, I'll call them collaboration systems, right? Email's not even simple anymore, right? All of the email, whether it's Zoom or Teams, however, we're communicating and collaborating in the organization; we need to be helping people understand how best to utilize those tools. I'll focus in on the EMR and our clinical users probably most intensely. We've then, through feedback from our physicians and clinicians, heard them clearly, just as we said, that they're overwhelmed with the amount of training we throw at them in their first few days with us. And we have now scheduled regular checkpoints at 30, 60, and 90 days post that initial training to continue to reinforce what they heard in their first session but to layer in some of those additional pieces of functionality, usability, and utility in the system, right? Where we're just teaching you how to navigate the system in your first training. We can't even begin to help you understand how to get data out, right? Until you've had an opportunity to use the system as one example. So, you know, search and reporting capabilities is something we layer in at a later time in that training. Any major upgrade to the system is the other opportunity where we are finding that well-orchestrated and organized training is important. Alistair touched on the delivery methods. We have incorporated, they are virtual but instructor-led training courses. So you are live and present in an interactive session, much like this one with a live trainer. Some of the training or to complement that is also in the form of prerecorded podcasts where users can come back and look at those at any point in time, multiple times if they have questions in their own mind. And then really simple changes, we still use a, you know, very simple tips and tricks, kind of, you know, one page, two page at the most, description with a picture of some screenshots if it's a really simple change. And we make sure that we push those out through a variety of communication methods. All of that gets, I would say, augmented and supported by, we try to utilize existing meeting forums, right? So if there's a quality and safety committee meeting and a change to the system is aimed at improving data capture around one of our quality and safety targets, as an example, we'll socialize that concept in the broader organizational meeting about the topic so that it doesn't just always feel like IT is throwing these little sound bites about the screen change. When we incorporate it into the broader conversation about an initiative or an improvement, it takes on a whole different tone. And then we find people saying, well, gee, we do wanna know more about that. Can you come to my department meeting or whatever, right? To give us more information about that technology component. And so, you know, that's an area I would say we found to be a best practice is making sure it's always couched in the broader why we're making the change. It's not just because Lisa or IT thinks, you know, moving that checkbox, we had nothing better to do, and we're gonna drive our users crazy. There's, I hesitate on always. We always try to be sure there's an important reason that we're making those changes.

Anthony Guerra
Very good, Alistair.

Alistair Erskine
So I agree that the, you know, there's a playbook for new hires. In other words, there's a sort of minimum number of things that you wanna expose a new hire to. They may not remember it. It's almost like in medical school, you know, you learn about biochemistry and the Krebs cycle. It still gives me PTSD, but you're exposed to it just so that as you later learn about something, you remember kind of the framework and the structure. So I think there is value, even if you don't remember the details, to kind of be exposed at least once to the various different elements that you may run into at some point in the future. And then for new applications, you know, any rollout of a new application obviously has to come with its own kind of subset of a training compendium, part of the program of work, and then measures of adoption to understand if there are holes in that. So the end goal is that you know, whatever application is actually moving the operational needle, what was the purpose of that application, as opposed to it's live or, you know, kind of like Lisa said, it's checked the box and it's kind of, you know, people are trained, you know, it only works if it's adopted and it's moving the needle towards what the ultimate outcomes were. I think the really hard and interesting and kind of longer conversation is the maintenance training. So, you know, even with an upgrade, we'll use it as an opportunity to retrain the things that maybe weren't caught on the first few go-arounds based on what's being poorly adopted, but would still be useful for the users. Looking at the data is gonna be really important. You know, everybody's gonna be on a bell curve. So there'll be some champions that are terrific, and then there's some folks that will struggle. And I think the sort of that maintenance piece is really well supported if you have 300 super users across the system that are themselves sort of more highly trained or sort of recognized as being the folks, the go-to people across all the subspecialties that can carry a little bit more of that, you know, applying and interpreting the training to the folks that have specific key workflows. So that super, you cannot bribe super communities, super users enough to be part of that community in the sense that, you know, they're so valuable to be able to kind of provide that sort of maintenance training and give you feedback as to what's going on. I think other, you know, tip sheets are useful. Some people like to hold onto something as they kind of work into a workflow. You know, things I've seen some of the EMRs do is sort of put embedded training in the system. So, okay, it's time for me to go order something. Here's a 30-second video just to remind you how to order something. Those, and you can start thinking about modularizing training so that you can cut those up and sort of put them into the different parts of the workflow. And, of course, that sort of virtual at-the-elbow support piece is a great way to kind of offer maintenance to training to those who, again, are struggling. Maybe you can identify there are folks that are struggling, they're spending too long in pajama time, you know, they're spending too long in the in-basket. How can I create a personalization session where we can take one hour of the schedule? They may waste eight hours a week, but they have a hard time finding one hour to learn how to save four hours a week. And that's just the hurly-burly of clinical practice. And, you know, it's just hard to find that time. And so I think those are some of the key things when it comes to that maintenance piece.

Anthony Guerra
Great stuff in there. Ryan, what are your thoughts about what you're hearing?

Ryan Seratt
So I think that, you know, one thing that, in the healthcare industry, that there's more and more of an appetite for is something called just enough training. And both Lisa and Alistair kind of referred to that is what skills can I teach someone to carry them forward? And what do they really need to know to do the job? And what can they reference at a later time? And kind of categorizing that. Alistair was kind of talking about not all training is built the same. And if we really define the term, what requires training? CPR requires training, correct? I don't want someone watching a YouTube video while I'm laying on the floor. So that requires a skillset. That's actual training. But the step-by-step how to do a workflow and an EMR, I can look that up. There's no time restriction to that. I can pull that knowledge to myself with small micro-learning trainings. So that's small bits of training, whether it's delivered in person, like Lisa was talking about, during meetings, or it's a digital video, or whether it's a tip sheet or a super user actually is that kind of micro-learning that happens when I'm trying to perform the job. So we're seeing more micro-learning and needs for micro-learning. And also kind of cutting back on that formal training. We know that that is, the more time we spend there, we don't get the results back that we're investing in. So really kind of paring that down, teaching people what they absolute, their survival skills, I've heard some people call it, and then we bring them back for additional trainings, is a best practice.

Lisa Stump
Yeah. Anthony, if I could add a point, Alistair triggered a thought for me. One of the things that I think we've found to be very successful is using the data to identify those users that do seem to be struggling, or at least spending more time in given activities than their peer group. We also use those data to identify the true super users, not just the people who have volunteered to help but those who are spending the least amount of time in certain activities or appear to be very efficient. And we sort of, those positive outliers, we talk to them to say, what is it that makes you so efficient? Let's make sure our training reflects what appear to be those best practices. And then, very often, that feedback to those that are struggling is much better received from their peer who's doing it better than saying, hey, Dr. Jones, we need you to come back to training because it looks like you need remedial training. Instead, it's their colleagues saying, hey, I found this really effective way to get through morning rounds or to get through a busy day in the clinic or in the office. And so, using the data to identify both our opportunities to help people and our positive outliers has been really helpful.

Alistair Erskine
So, Lisa, you bring up a really interesting point which I hadn't thought about, which reminds me of an experience I had early in my career where it was a department chair of surgery and I won't say where, and he was such an opponent to the whole concept of going online and using Electronic Health Record and so forth. And just for years, I went on about that. And then, one day he pulls me into his office, and he says to me, look, Alistair, I'm a dinosaur. I get it. I just don't know how to use this thing. So can you just show me, get to a patient list and open a chart. And I spent a few seconds going through that. And from that point forward, he was a different person. And so I guess the point is training is also navigating the inevitable egos of sort of super Ph.D. level people that you're teaching. You're having somebody who may not have that level of education trying to train somebody who has MDs and PhDs and whatever else. And that's part of the challenge of the training backwards and forwards that we have to negotiate. And so the idea of saying like, wow, you're brilliant. You're a super user. Look at your data. This is great. Is a much easier conversation than the point of, like, yeah, remedial classes at nine o'clock in the morning on Monday.

Lisa Stump
That's right.

Anthony Guerra
Ryan, anything about navigating egos?

Ryan Seratt
Yeah, actually I just took an online class, a 12-week class. One of the main things that we talked about was motivating trainers or not trainers, but trainees. That the trainers really need to pull that motivation out of everyone. Everyone's got different motivations. And whether that's, well, you know what? My schedule is completely booked up. I don't get out of here till eight o'clock every night. And I need to find a better way to do that. Or you know what? I am a dinosaur, and I need to learn how to use this system. And but finding what motivates people, what their goals are, and helping them achieve that is the training that I find the most rewarding, but also the training that I probably spend, that's the kind of the time I spend the less time in. I do a lot more of the moving from upgrade to upgrade with clients and less on the, you know, building those skills, that next level proficiency that I really, really find rewarding and other people do as well.

Anthony Guerra
All right, very good. Good dialogue there. Alistair, let's start with you. How do you ensure vendor upgrades are worth the effort to change users, worth the effort to train users on the changes? Do you ever skip upgrades due to this issue?

Alistair Erskine
No, so I wouldn't skip an upgrade due to the issue. And the reason for that is upgrade will have in it regulatory needs, you know, other requirements that, you know, motivates us to upgrade. It will also have, you know, beneficial functionality that people have been waiting for, presumably in a good upgrade. So do everything you can to stay as close as possible, but not too close to kind of general availability of the software. So I wouldn't delay for that. In terms of would I skip training with an upgrade depending upon the size and the impact of the upgrade? Yes, or is it a black and white, everybody gets trained, or nobody gets trained? Not necessarily. If an upgrade has something that is particularly significant for let's say contact center folks, then it may be a good idea to skip training. you know, what degree of training am I going to choose? Am I going to do some tip sheets, a few videos, make available sort of like the online virtual at the elbow? You know, I there's this different degrees of training. How do I activate the super users? Can I get some people in early and give them functionality and see how they do and then use that as a means to be able to decide whether or not training is required or not. It's a good practice for an organization, I think, to just sort of expect that you have to be trained like once a year, like you do annual training for other things, like how to put out a fire and, you know, how to avoid, you know, leaking HIPAA information or whatever it may be, you know, and if you can kind of get an organization that cadence of look, the system keeps changing, but like once a year, we're going to give you the basic, you know, kind of important things that you need to know. We're not going to waste your time. We're going to; we're going to tell you the things that we really think we've learned people really benefit from and kind of make it a high-value thing. I think that's kind of how you can negotiate upgrade training, you know, kind of going forward.

Anthony Guerra
Alistair, just as a follow-up, I would imagine misreading an upgrade on your part, if you're ultimately responsible for deciding the level of training that has to go with it. If you misread that and underestimate the amount of change, that's going to be a huge problem. Do you ever bring in any of the individuals who may be affected by an upgrade and have them sort of look at it and say, how much training do you guys think is necessary for your team to absorb this? Because, like I said, you don't want to get this wrong.

Alistair Erskine
No, no, you're a hundred percent right. And, you know, the vendor will tend to do that. Kind of give you a prelim on it, kind of saying, look, we think this is going to be a major change to the users, or we think this is going to be a major change to the impact. That's step one. Step two is absolutely you bring in people, you know, ahead of time and saying, what do you think? You know, and, and these, the trouble sometimes is those people are pretty savvy because they're the kinds of people that show interest and they don't always represent the dinosaurs. And so it's also really important to bring in a couple of, you know, brontosauruses with you just to make sure that, that you get a full picture. But yes, you definitely try to get a better, and if you miss it, if you suddenly, oh, holy cow, this is actually much bigger than we realized, you need to have something you can grab right away and just put throughout the organization again, in some sort of like rapid, let me, you know, log into my system and show you how to use this really quick.

Anthony Guerra
Lisa?

Lisa Stump
Yeah. Very similarly, I would say we don't skip the upgrade. You know, they, they do build upon each other and, you know, there's ways to look at that. So certainly, there are the required elements that might enhance the underlying performance of the application, or to Alistair's point, they're meeting some other important safety regulatory patient experience requirement. Where there is optionality around individual features of the upgrade, though, we have made decisions around whether or not we turn on those discretionary elements. And there have been times we've disagreed, you know, the vendor thought they were bringing forward a great enhancement, and we didn't necessarily see it that way. And so we chose not to, you know, turn on individual features, or we recognized that we needed to take the full upgrade in a short time period. That individual change might be good and important, but it was going to take us longer to communicate and educate. And so we wait on individual features within an upgrade sometimes. The other danger in skipping the upgrades is they, they, that fact, again, that they build upon each other in a way that when you try to take a double or triple upgrade, then the magnitude of change on your users is sometimes even, it's five times, but it's five times, right? The exponential change when you put it all together. And so we do believe that those smaller increments of change are easier to manage. Not all of our clinicians always agree that they feel a little bit like they're getting pecked to death by chickens. You know, when, when the system is changing every quarter or, or every six months. And so we do have to be careful about that. You know, I think misreading the change, I was having a little bit of my own PTSD, listening to Alistair. We did misread one. And for the reason that Alistair talked about the people we asked, you know, I have a group of physician informaticists who work with me in it. Now they're incredibly tech-savvy, right? They looked at it and said, ah, it's not a big deal, right? We can just tell people that the change is happening. And it was a major, you know, source of discontent and confusion for people that we did have to immediately sort of pull a rip cord and get at-the-elbow support out there to help our physicians. Some of the most vocal people about that change are the ones I now be sure to include in the, do you see this next change as a big one or not? So that we, we have both ends of the spectrum again and how, how significant that change might be. So.

Anthony Guerra
Lisa, can you tell us a little bit, I'm just curious, can you tell us a little bit more about when you began to realize that a mistake had been made? Was it a ton of calls coming into the help desk? Was it people banging on your door? When did you go, Oh boy, something's going on here?

Lisa Stump
Yeah. You know, there probably were, there were calls going into the help desk. I know it's a major issue. Yeah. People start banging down my door and it was, you know, fairly high level people in the organization, department chairs, and leaders who were, you know, either in that five minutes before a meeting started or were directly emailing me. That was like, you know, I think, you know, we do our upgrades over a weekend that Monday, they were seeking me out to tell me whatever we did over the weekend was, was a major problem. So, you know, when they reach that level, I, I know we've, we've missed the mark on, on how we,

Alistair Erskine
Lisa has to look underneath her car for the little red blinking light.

Anthony Guerra
That was not a fun day. Was it Lisa?

Lisa Stump
No, no. I want to knock on wood. We try to avoid those.

Anthony Guerra
Right. Ryan, really interesting stuff. Right?

Ryan Seratt
Yeah, absolutely. I think, you know, for upgrades, not, not everything needs trained. As a trainer, you probably don't hear that a lot from my side of the house, but there, you know, as we look at upgrades, the best practices I've seen is people will generally categorize something as discoverable, which means that, you know, it's a new option on dropdown. People will see it. They'll know it in, you know, exactly what that means. You don't need to communicate that even because the, there's actually a signal to noise ratio as well. Right. You know, it's, I'm going to teach you everything about this upgrade in 150 easy steps, is not a good communication practice. So let the discoverable things kind of, kind of be discovered. Then there's things you're going to want to communicate that they're important enough that people need to know they're happening. And then there's the actual ones that need to be trained. Is there the patient safety issue? Is it different enough from their normal workflow where you're going to need to provide some commentary back and forth, give them a chance in a forum to ask questions. That's quite a bit different than, you know, a new option on a dropdown.

Anthony Guerra
Excellent. Well, I want to go to my favorite part and hear you guys ask each other questions. So Ryan, I'm going to let you go first. Do you have a question for one or both of your co-panelists?

Ryan Seratt
Lisa, I've got a question for you. You were, you know, as you were talking about the one-on-one training, bringing people back in, is that something that's scheduled ahead of time when you bring on a new provider that, you know, your class is going to be four hours, but in two weeks, you're going to come in for another two hours, two weeks after that, another two, what does the structure look like there?

Lisa Stump
Yeah, we give folks a bit of flexibility there. So we say, you know, it's recommended that you come back in at 30, 60, 90. Here's a schedule of classes that will fall in, you know, the week that would represent your 30-day checkback. And then, if we don't see those people register, we proactively reach out, but we give them the opportunity to have some flexibility around when they choose to come back in within that recommended window.

Ryan Seratt
Okay. And so they sign up for their own classes at that point in time?

Lisa Stump
Right, right. But we know it's the 30-day, you know, post-hire, so that sort of content is well-defined. But it is an opportunity as well for the users to help us understand how to personalize what they need at that point. So it's both some standard content delivered at each of those time points, as well as now that the users had an opportunity to use the system, what are they finding confusing or difficult or, you know, or easy. We try to get the, where did we make it easy for you as well.

Anthony Guerra
Lisa, quick follow-up. I don't know if you know this, but would you say the majority of people actually schedule that follow-up on their own, or do you think the majority need to be pinged by you before they do it?

Lisa Stump
Yeah, we're actually early in rolling that out, Anthony. I probably need a little more time. I think because it's new, people are tending to sign up for it. I'd want to give it more time to see if that holds or that sort of newness fades a bit.

Anthony Guerra
Very good. Lisa, do you have a question for one or both of your co-panelists?

Lisa Stump
Yeah, I would love to know that. The other thing we sort of struggle with a bit is how to, or if to, create incentives and mandates around training. You know, anything from, do you offer CMEs, do you offer protected time, do you mandate training around significant upgrades? You know, we're trying to, so we do mandate training for new hires. You don't get access to the EMR, for example, if you don't complete training. But beyond that, I'm curious how you might structure any programmatic incentives or mandates.

Anthony Guerra
Alistair, do you want to go first?

Alistair Erskine
Yeah, I think it's particularly relevant now because if you pull a nurse out of clinical practice when you're spending so many more dollars on traveling nurses to be able to staff hospital beds because there's such a demand, you know, how do you work out the finances for that? Does the department get compensated somehow for the costs of having to hire, you know, backfill workforce while a nurse is being trained? You know, do you pay them a certain, you know, hourly fee of some kind for them to be able to train? Those kinds of things are not so much the new hire piece, but more kind of, you know, the ongoing maintenance and training. I don't think we have a great model yet. I think it's a bit all over the place and depends a little bit upon, you know, upon the department's ability to pay, the department, you know, kind of types of users, you know, the level of education of the people that you're training and so forth. I think it's not really well, I don't think I've done, we've done a good job yet of being able to sort that out.

Lisa Stump
Yeah, thanks.

Anthony Guerra
Ryan, any thoughts there?

Ryan Seratt
Yeah, that's a great question. I've seen a lot of people struggle with that. You know, the, you know, is upgrade training mandatory is a question that we hear quite often. And it's not, you want people to take all the training, but yet you're not going to chase them down with a stick necessarily if they don't. But I think that, you know, one thing that I've seen, especially in kind of a best practice, Alistair, I think you reminded me of this when you were talking that sharing that, you know, providers sharing provider training, nurses sharing nursing training is a great way to provide best practices. And actually being able to report out on people, you know, people who took the training are seeing this difference and actually reporting out of what are those best practices that everyone's doing and being able to share that data with people that, you know, what people who didn't take this upgrade training, they actually are not performing as well as the other people and having some sort of Just-in-time training tool available so they can get caught up is something that I would say would be desirable.

Alistair Erskine
Yeah, it's really important. At the end of the day, it's really important. That's a good way to look at it. And, you know, you show data, people will argue the data, but it's true that if you can say, look, you know, first of all, NPS scores are really high. People really appreciate the training. Number two, look how much faster and more efficient they are in the system compared to how they were before. Don't you want to be like that too is a clever way of going about it. Very good.

Anthony Guerra
Alistair, do you have a question for one or both of your co-panelists?

Alistair Erskine
It will be for both. And it's kind of, you know, a high-level question, but is training a failure in human-centered design? I think about, you know, the Apple products of the world that don't come with a training manual, they're beautiful, they're well designed, you know, so forth, you know, is so that's kind of a broader, higher-level question.

Anthony Guerra
Ryan, we're going to go to you first on that.

Ryan Seratt
Oh, nice, easy, easy. Now, you know what, I think that training can oftentimes get lost in delivering content and not developing the skills or the aptitude that we need. So a lot of time, we're focused on, you know what, this is everything that everyone needs to know. And then we spend our time doing that instead of actually kind of reversing that, talking about what do people need to do, and making our training focused on that. So does it fail a lot of times? Yeah, because I think it's focused on the wrong thing. You know, once you leave college, you're actually not learning something just to learn it. You're learning something to apply it in some shape or fashion. And there's got to be that element of execution. And I think that a lot of our training doesn't address that.

Anthony Guerra
Lisa?

Lisa Stump
Yeah, I think similarly to Ryan, I would say if our training only focused on the points and clicks and, you know, which button or do I swipe left or right? I think that might be a failure. But I think the training we're delivering, if we're doing it well, is encompassing, you know, I'll go back to, it's the workflow, it's the why. It's often the policy behind, you know, the need to document or the need to, you know, complete something in whatever that system is that we're talking about. And so I think when the training is comprehensive and holistic in that way, it doesn't reflect that failure of human design. If we have to truly train you on here's where you click and, you know, then I think, yeah, we've probably got work to do in the true intuitive usability of the system.

Ryan Seratt
I'm sorry. Yeah. That's one of the opportunities we see quite often is that there's a disconnect between the technical side of the training and the actual workflow side of the training. And those two things really need to be merged, or people don't have the context to provide or to implement that knowledge that they're getting the context of the workflow has to be there.

Anthony Guerra
Ryan, I want to ask you a question. So, you know, we have the health systems. And then inside the health systems we talked about the IT department and the operational leaders having a shared responsibility for training. We talked about the vendors that are created theoretically the product that we're talking about being trained on and, and they're trying to make it as user friendly as possible and they're doing the upgrades and I'm not sure if they're providing any level of assistance to the health systems on training but we can hear about that, and then organizations like yours, which is sort of outside consulting coming in to help with this process but tell me a little bit about the typical scenario where you're brought in.

Ryan Seratt
Yeah, you know I always say that you've got to have the strategy and the tools to actually implement training. And so we're brought in quite often to review what the training ecosystem looks like. So how do you we actually communicate what needs to be communicated or train what needs to be trained. So we provide services in creating the content to deliver to students, but we also develop tools. So for example we have a just in time training tool that we also help people implement to provide that micro learning training throughout their learning cycle. So how do you get that training to them in the format they want. So we're brought in to look at the overall training department or help with certain projects and develop the training now.

Anthony Guerra
And when you come in, is there some like when you see a place that's struggling is there one thing that comes to mind that whenever they're struggling they're, they're just doing this thing or they're not doing this thing.

Ryan Seratt
Yeah, I think the probably. Yeah, the one thing that I see probably most often is not having a diverse delivery strategy and really trying to make that classroom training experience that we're all familiar with, and use it throughout all different modes. So, you know, I like to think that well so once you get your initial foundational training, you really don't need to come into the classroom, because the workflows changed a little bit, you just need to know that that is going to be in the workflow, you can adapt to that. And really just using that that tool. When it's appropriate, when they need to learn new skills, but with you don't have different approaches for different situations. It's not one size fits all. And if you try to do that, then, then you can encounter some difficulty.

Anthony Guerra
Very good. Alistair just touching on something you were talking about we talked about sort of the competitiveness, especially of physicians. And this used to come up with data quality. When they were saying you show them their quality scores, and you're going to motivate physicians are very competitive and possibly leveraging that in the training dynamic saying some of some of your co workers here are. This is how fast they are and, but you have to do that without humiliating because we talked about the egos so if you go to a physician and say hey hey you're, you're struggling here you're having trouble you could really offend that person and embarrass them and not get the result you want. So what are some of your thoughts about leveraging competitiveness without embarrassing someone.

Alistair Erskine
Yes, a lovely world full of minds. Medical Informatics. So, you know, we didn't talk about gamifying, but that's kind of a neutral way, in some ways to try to encourage people to learn. And if you think about it. One could get pretty clever with gamification, you know, if you complete a number of different tasks you get little coffee, you know, cards that you could be delivered to get local coffee, keep you well caffeinated during your, you know, your round and or whatever it may be. And so that's probably a way of thinking about that. You know what motivates people in the CIA uses a, an acronym called mice, which is money ideology collaboration and ego. So, you know, that's applicable here I mean you know money to some degree can motivate folks but ideology is like the mission the mission of health care is pretty compelling, you know, you want to do things because you need to take care of patients and save lives and so forth. And, you know, the, the collaboration or the coercion that exists, you know, kind of what you're talking about saying look everybody else is doing a great job. What about you or can you collaborate you know some good things that person knows some good things maybe we can collaborate and kind of share what we know, I think sharing of knowledge is a very common thing in medicine as a consultant comes and talks to other consultants and other consultants and they share each other's expertise. So I probably would find more in that in those terms as opposed to you don't know anything this person knows a lot, you know, come together. And then, you know, and then we talked about the ego piece and. In terms of motivating people, get them to somehow work together, understand what the common purposes, and try to reinforce the fact that this is all done to make it. Number one, better take care of patient number two but take care of yourself so you're not spending all this time in the system where you don't need to so we can get you to be more efficient.

Anthony Guerra
Right. And that goes right to burnout and things like that. Let’s have a lightning round of final thoughts. Lisa, I’m going to let you go first. If you want to just give, you know from your experiences with training, and pretend you're giving advice to someone in your position at a comparably sized health system, what's your best piece of advice for them dealing with this training puzzle.

Lisa Stump
I would say never underestimate the need for the training and support and to keep it centered on the value that it delivers and the work that we're doing, not training for the sake of training the same way we say it's never tech for the sake of tech. Technology is how we deliver our care and service, and, you know, keeping your, your ability and your acumen in those systems is is as important as keeping your, your ability and your technical area, your, your, I'm sorry clinical area of expertise and so I think it has to be always top of mind on how we're continuing to keep our users trained and capable in the systems.

Anthony Guerra
That's a great point. Alistair, you know, do you think of it as the stethoscope is what people traditionally think of with the clinician. This is another tool may be just as important a tool in the delivery of care.

Alistair Erskine
I think it's a tool in the healthcare operating system, anyways, just like Windows is for a computer. And, and, you know, and you really can't, and maybe Ryan would agree, but you can't spend enough on training, and the thing is, you know, you're always going to be able to go to the next level of personalization the next level of challenging users. And so you try to get the training out, and I think the other problem with it is a bit invisible in terms of its impact. It's a huge impact but just distributed over so many people over so many different processes. But the thing is, you pay for it in spades if you don't train appropriately at the beginning, you know, where you don't get that value that you anticipated or maybe was in your return investment, you know, performer. If you don't train appropriately. So it's, it's, you know, it's definitely worth investment; it's worth the time and thinking like we're doing today kind of trying to wrestle with the various different sets of options. It's worth the time and thinking with a team of how to optimize it with whatever constrained resources you have and spend time thinking through how to get, you know, it's not will, it's skill, a lot of times when it comes to the clinicians.

Anthony Guerra
It's great point. Ryan. We have millions and millions and millions on the tool. We don't want to go cheap on the training doing.

Ryan Seratt
No, really don't, and then, you know, kind of going back. And thanks, Alistair, and everyone should have the best training that they can afford. And, you know, I think, as you know, we're looking at as things are changing so rapidly and continue and will continue to change so rapidly. We really need to start looking outside of the classroom. And how can we support people with continuous learning. I see a lot of spin taking place for that initial training but very little investment at the tube for continuous training or Just-in-Time training. That's where a lot of a lot of a lot of value can really come out of your investment and supporting people through that learning experience as things do change, and they continue to change, the system in two years will be completely different than what they initially learned on. So you've got to help people through that journey to stay updated and current.

Anthony Guerra
Well, that's about all we had time for today regarding continuing education; you could use the final slide in this deck you get an email when the on-demand recording is ready. If you want to sponsor an event with us, you can reach out to Nancy Wilcox from our team, and you can go to our website to register for upcoming panels. With that, I want to thank our panel. This was a tremendous conversation, incredible amounts of value in there for folks who are struggling with this Dr. Alistair Erskine, Lisa Stump, and Ryan Seratt, and I want to thank the 314e for making the event possible and our attendees for joining. With that, everybody have a wonderful day. Thank you.