Welcome back, everyone. For this session, we will discuss program director and resident membership and engagement within the college. First is Melissa Ferrari. She is ACPM's vice president of membership and operations. She has extensive experience in associations from the American Bankers Association, National Council of Teachers of Mathematics, and the Society for Neuroscience, which she introduced her to the health sciences, healthcare, and academic arena. She has worked closely with volunteer leaders and boards to enhance organizational management. From systems integration to launching a new membership model to providing strategic oversight of membership programs and committees. Next we have Dr. Prasad Acharya, who is a graduate of the program at the California Department of Public Health. He is actively engaged with an ACPM serving as the vice chair of the membership committee, and a member of the advocacy committee. He's also working on one of two faculty members for a course that ACPM is developing as part of their population health curriculum. And with that, I'm going to turn the session over to you all. Thank you. Good afternoon, everyone. Or morning, I guess, depending on where you are. So, Anita, thank you for helping out with the slides. So first of all, I want to thank you for the opportunity to provide some insight on our resident and PR, PMR program director engagement with the college. As we continue the conversation, I'd like to provide some data that's going to help outline the current landscape. And our goal in sharing this data is to really give you some insight into the universe of residents and the PR, PMR program directors, and mainly the level of engagement with ACPM. Following my data overview, I'm going to hand it off to Dr. Prasad Acharya, vice chair of the ACPM membership committee, who will continue our conversation around resident engagement. A couple things I'd like to highlight for the purposes of our discussion today and the following data, we're going to focus only on the GPM PH programs, which total 42. I know earlier we heard the number 43 floating around. I also wanted to highlight that based on years of reporting, you might see some numbers that are slightly off from what was discussed earlier today. But I think at the end of the day, it still tells the story that we want to share. Next slide, please, Nina. So before we dig into the details, let's take a look at the universe of residents. According to AAMC, the total number of active residents slots in preventive medicine in 2020 totaled 288. This has remained steady from 2019, following a slight drop from 2018. So I want you to kind of keep that 288 number in mind as we go to the next slide. So as we think about the universe of the residents, let's take a look at how this plays into ACPM membership. As you can see from the charts, the one on the left reflects actual numbers of members and the one on the right, percentage of total ACPM membership. We've witnessed a slight decline from the same time last year. I am happy to report since this number of 141 at the end of March, we are closer to about 160 in terms of resident members. So that's a little over 50 percent of our total residents in preventive medicine. One of the things I would like to point out is that we've been going through a data hygiene exercise over the last six months at the college. And while we are confident in our 21 numbers and potentially our 20 numbers, we don't have substantial reporting outside of a total number to back up 2019. And I think to be able to verify that we potentially have 94 percent of total residents of members in 2019 is not something that we currently have the backup reporting to support. Next slide, please. Outside of the membership, I think another key data point for us to look at in terms of evaluating engagement with the college is the annual conference. And as we look at the registration numbers over the past few years for the preventive medicine conference, please note that our registration categories are combined. They include resident and medical students. So moving forward, we'll be working with our meetings team to separate those two so that we can get a little more finite and precise with our analytics. But in the meantime, I think you can see that compared to 2019, which was an in-person meeting, we are on par or close to for 2021. And as expected, 2020 showed a significant decline. Another engagement point, which we don't have a chart for, is our Hill Day. And we have no comparative data because we have not had residents participate in previous years. But we are happy to report that we had 10 residents participate in our April Hill Day this year. So we're really excited about that. Next slide, please. So now let's take a look at the universe of the PMR directors. And so as I mentioned earlier, and as we've heard earlier today, we have 73 programs, which includes the OCMED and the aerospace. And of those, 42 are GPM PH programs. So taking a look at those 42 program directors and ACPM membership, you'll see that about 30 or 70, 71% or 72% are current members. Six or 14% are lapsed members, meaning they were a member at some point, but they've let the membership expire. And then six or, again, another 14% have never been members or they maintain a guest account, meaning they've created an account to be able to access some resources or information, but have never actually taken that to join. Next slide, please. And, again, similar to what we did with residents, as we kind of looked at opportunities with ACPM, looking at attendance or registration at the annual conference, you'll see a significant drop in participation from our 2018 and 2019 years. And, you know, I'm new with the organization. I joined in November, but last year, I understand the workshop was held separately as well. And you can see there was a significant drop in attendance in 2020 with the workshop being held at a separate time. And then, as you can see, our numbers in May for 2021, we started to see a slight increase again. And then I also want to highlight Hill Day for our PMR directors. We actually had three PMR directors participate in Hill Day. And as Drew and Dr. Remington mentioned earlier, the focus of this year's Hill Day really was on increasing the funding for our PMR programs. Next slide, please. So the last bit of information I'd like to share is when we started to think about residents and residents and engagement with our college, we reached out to some of the other specialty medical societies and kind of did an informal poll. But I think it was very, very telling and very interesting. You'll see those who participated on the right side of the screen. But three questions we asked, I think, about 10, but three that really provided some insight included, does your organization have an expectation that all residents are members? Are funds allocated in the residency budget for membership dues or are they paid by the individual? And is there an expectation that all residents attend your annual conference? And so while you can see we had a relatively small sample, it was about 18, you'll see on this slide that 50% of respondents had no expectation of their residents being members of the organization and 22% offered free memberships. Next slide, please. So when digging into the potential for budgeted funds for membership dues, about a quarter have a mix of dues being paid by the programs and the individuals. While a little over 20% were not aware of the source of the dues funding. If you offer a fellowship program that includes dues, and again, nearly 25% offer free membership. Next slide. And in terms of our last slide here, we show the responses around expectations for attendance at the annual conference and more than 50% do not have an expectation for the residents to attend. 28% are hopeful that they'll attend and offer some level of a special rate or special programming to encourage attendance and then about 11% hold the expectation that residents should attend. So that's kind of an overview and I hope it gives you some data points to keep in mind as we kind of shift to the next phase of our conversation with Dr. Acharya and if anyone has any questions, I'm happy to take them or we can save them until we get through the full segment. Thank you. Dr. Acharya, I just want to confirm, you're just going to share your thoughts, correct? Or do you want me to share the slides? I'll just speak without the slides. Okay. Great. Thank you, Melissa. Fantastic. And thank you for having me here. Glad to be here and hear a little bit about what you all are working on. So I just wanted to share a little bit about what we're doing on the membership committee to engage residents. So in the past six months, we've started a resident engagement subcommittee, and that team has done a lot of brainstorming and come up with plans to help facilitate a college-wide approach to resident engagement. And so right now we're at the point where we would like to align our efforts with your efforts because you all have the closest contact and the best understanding of what residents need. And what that means is we want you all to connect with us in some way. What would be fantastic would be having like a brainstorming session together where we float around ideas kind of like in a design thinking style and share some of the ideas that have come before and just come up with innovative approaches to really meet residents and maximize participation in the college. Just to give a little bit of context, what we've done so far is we have come up with plans. So we've set some goals, and our main goal is to maximize resident participation, which is obviously important. And one way we want to do that is by sending a welcoming packet to all residents who are entering the field of preventive medicine with resources about the field, less about preventive medicine, about ACPM in specific. And our goal is to set up some coaching programs and basically create journeys that relate to professional planning that span the entire resident experience and attract residents into this planning process. That's part of the planning that we've been doing. And we have a number of very established preventive medicine docs who are interested in providing such coaching. So that's something we've really been interested in. But one of the main areas that we'd like to focus in on the college structurally is creating a resident lounge, a place where the residents can have a sense of a cohort with their incoming class. And so in the past, there was the resident physician section, which kind of was a little bit independent of the rest of the college. What the plan is for the resident lounge is to connect residents with each other, to just create that sense of cohort, and also create a place where residents can become aware of opportunities that are inside the college on different committees and so on. And similarly, we have other ideas like creating a new member and resident liaison to every committee, which can ensure that every committee in the college that wants to have residents, and as it's set up, we currently have not an expectation, but a desire that every committee has resident representation, that the membership committee can place residents onto these committees pretty easily. So we're trying to establish a role on each committee of a resident liaison to help facilitate that process. So I just wanted to share a couple of the ideas that we've been brainstorming, and also just extend an invitation to any of you all who want to participate in our efforts, because your participation will be critical in understanding how to approach different program directors who are less engaged with the college, how to approach preventive medicine residents who are in programs, and how to formulate programming that really meets residents' needs and get the word out to residents. So thank you so much for your attention, really appreciate it, and look forward to collaborating with you all. Thank you. I don't see any questions in the chat. Are there any raised hands? I don't see any raised hands. Let me just, but I do see some just for Dr. Acharya and Melissa. There are just some general comments about timing and when we should be aware of when to sign up, etc. Drew, are you still on the call? No, Drew just signed out because of, I think, sign-on issues that I'm having here on my end, so I'll have him sign on again. I just thought it might be helpful if he speaks to the timing and the process of Hill Day, so that folks know moving forward that when we typically start to promote it, when we would like for you to acknowledge your participation, because we have a set Hill Day, obviously doing it virtually, we made it like three Hill Days. And Dr. Remington, you might be able to speak to this as well, but we ask for folks to sign up by us within a week to two weeks, so that we then can schedule the meetings and see who has participated, so that we can group folks from the same state together, so there's some coordination with it as well. Here comes Drew. Yeah, Melissa, just quickly, I do think scheduling is a challenge. It was hard. We did a Hill visit with our residents to, I mean, the Hill visit actually was just, you know, just a Zoom call with each Senator's office. You don't talk, rarely do you talk to the Senators, you just talk to staff. But it did require, you know, some calendar calisthenics and trying to figure out, you know, when the staff can meet, and then when everybody else can, and you usually do the best job, you know, get the most people you can, but you won't get everybody. So, Drew, we're talking about sort of the tactical issues of scheduling Hill visits. And Drew, if you could speak to a little bit about the timing as well. Typically, how far out we promote, when do we promote? Yeah, so I can kind of like walk it back from basically when the meeting, the last thing to happen is the meeting will be confirmed and scheduled by the Congressional Offices, and that, you know, that's kind of where we're at right now. Typically, how far out we promote, when we would like to have individuals commit it so we can then do the scheduling? Yeah, so I can kind of like walk it back from basically when the meeting, the last thing to happen is the meeting will be confirmed and scheduled by the Congressional Offices, and that, just given that their schedules, especially around appropriation season, tend to be pretty wild, meetings are frequently not confirmed until literally the day before they happen, if not a few hours before. So I know that can be extremely difficult and frustrating to deal with. So ideally, we like to provide times to the Congressional Offices about a week to two weeks before the meeting will be, ideally in pretty large windows. The best situation of a full day, where we know we're doing is kind of a standard like fly-in Hill day where people fly in and are kind of committed to spending a full day on Capitol Hill. Otherwise, if we're doing them virtually, looking to kind of do half-day blocks, so like morning or afternoon, but really the widest timeframes that we can secure. And really having those kind of secured about two weeks out from when we want to do our Hill day. And unfortunately, just because of how late Congress will confirm things, really committing to holding those big windows open until the meeting is secured. So I guess my question to you all is, in order to book half a day or a full day for yourselves or your residents, how far back do you then need to kind of start planning around that and be aware of it? Is that a month before, three months before? I think that would really help me in terms of when I kind of open up registration for Hill day and think about planning. The other thing is, if you need to hold a half day, when does that need to get on the calendar? In past years, I think we've usually gone about three months before, around 90 days is when we try. But if it needs to be earlier, we can definitely try and kind of secure our date or range of dates. Given that we found kind of flexibility with the virtual environment where we aren't necessarily committed to a single day, and that has definitely been helpful. But yeah, so I think the more I can kind of learn from you guys on like what your schedule looks like, what your process is for getting time for residents, that would make a big impact. Okay, so I've seen mostly about three months, three to four months. Kind of at the far end is kind of when you need to know when we're looking for something to happen. Thank you. Do you think you would require the residents to be members to participate now that we're talking about membership? So that's one question. The second one is since we disbanded the resident committee or whatever it was called, I used to have that chair of that committee on my committee, but then they just disappeared so I don't have a resident now. How are you going to or how are we going to kind of choose or how are the residents going to choose what to participate in? I'm going to say I would encourage the residents to become members to take advantage of training and Hilday, but I mean, I don't know if you want to limit that to just resident members. Anyway, those are my two questions. I would say that yeah, if you're participating in Hilday, we'd certainly hope you're a member. Kind of given that you're representing the college as part of that activity, I think membership should be required. And this is Jordan. I'm the director of membership and marketing. I'll just chime in that participating in ACPM committees and other activities membership is a requirement for those types of activities. So as you encourage your residents to become involved with ACPM, the first step is membership. And I'll add and Prasad, you might be able to speak to this, but our current chair, Dr. Maria Moron-Prezon, if I get that correct, is really has worked to try to get residents on the different committees and I know she recently asked for a roster and is reviewing that so that she can identify some opportunities. And Dr. McElyde, thank you for sharing your thoughts on chat. If you want to just go ahead and share what you shared on the chat in response to questions for you, both in terms of timing for the 2022 meeting, the board review courses and yeah. Thank you very much for the floor, Anita. I'll first address the conference in June. The PM20 meeting was supposed to be held in Denver, Colorado, in the month of June. And it's my understanding that the date, the preferred date for that meeting, were based on recommendations from the board. Or based on responses to a survey that was sent out to the members prior to my arrival at ACPM. So in order for us to be released from our contract during the beginning of the COVID pandemic, we made an agreement with the hotel in Denver that we would hold the conference there in 2022. However, since my arrival here, I've been hearing from many members that June is not a good time for the preventive medicine community. And therefore, we are working to move the meeting to March. We've already committed to the omni-shorum for March of 2024. That allowed us to not have a face-to-face meeting this year for PM21, which was virtual. For 2023, however, we are just beginning to do the research necessary to look at cities, what might be available. But we're also keeping the option of having a virtual meeting in 2023 open. And we're surveying all of the PM21 participants. And we've asked that question in the survey about a preference for a virtual meeting every other year. So stay tuned on that. In terms of the board review course materials, those are available now. They're available in our LMS system, Learning Management System, OASIS, for purchase. We decided last year, because of COVID, to not have a face-to-face board review course in Baltimore. And we had a virtual course instead. And Dr. Jill Wallen, who's on this call, and our team talked about what to do for 2022. Given that the pandemic is precluding travel, we are going to make the recordings available from 2021 to participants, as well as the materials. But we're scheduling 22 Ask the Expert hours that residents and others can purchase once they've reviewed the materials to come back to the faculty to ask their specific questions. Dr. Wallen, would you like to add anything to that? No, I think you covered it well. We're looking forward to Ask the Expert sessions. We had thought about that previously, even after in-person. So that's a new twist, and we're looking forward to it. And the June issue of AJPM will include information about this possibility, and I believe it's already posted on our website. Thank you, Dr. McElyde. There's a quick question from the new program director, Dr. Nowinski-Koncheck from Cook County. I think this is regarding the membership. It would be helpful to know when the residents receive e-communication from ACPM, urging them to become members, participate in Hill Day joint committees, welcome packet, etc., so I can encourage and support this. Maybe just a quick forward to PDAPD, and also always CC the program directors and associate program directors. And so I'll turn it over to Jordan, if you're still on, in terms of the protocol for communication for membership. Hi, yes, and I was just putting the information for the board review course in the chat. So let me speak to this, and then I'll finish that. So everyone stay tuned. Stay tuned there. And Anita, I'm sorry. Now, what did you ask? I'm happy to repeat the question. So the question was about what is the timing of the resident communications, receiving e-communications about various membership-related activities, and you just forwarded to the PD listserv, and also there's an encouragement to CC the program directors and associate program directors, etc. So what is the process that ACPM engages in for resident communication? Thank you. Right, absolutely. So that's something that with the resident engagement subcommittee with Dr. Atara and the others, we, you know, we're talking about, I definitely have heard in recent discussions, the importance of reaching out, including the, including you all, the resident directors on communications with the residents. So as we're looking ahead, the thought is, is likely to reach out to the incoming residents as cohorts and copying, including you all. Go ahead. Good, checking in. I'm sorry. That's okay. And the other thing is, you know, we do need to be considerate or cautious around data privacy protocols. So in terms of just adding the incoming residents to our database, we want to make sure that they have expressly opted into receiving communications. They know who ACPM is. We'll be relying on you all kind of to be our conduit to, for that introduction and for getting them added to any of official communications going forward. We are exploring options for, this was mentioned earlier, for kind of a discount because these incoming residents are joining mid-year and our membership year is based on the calendar. So whether that looks like kind of a discount for the remainder of 2021, or whether it's an opportunity to purchase membership through 2022. And so you're paying for kind of a longer timeframe when you're dealing with your initial registration and payment kind of upfront. We're looking at both of those options and likely both will be available and you'll be able to select which kind of works best for your program and your residents. That would eliminate joining residents in July and then needing to renew them in December. The second option would, is the thought to kind of cut down on administrative work. And, oh, the other point is, yes, we'll be working on, we have an application form for joining committees that we can definitely share and it's available on the website that folks, and especially your residents can fill out and submit and then working with individuals directly based on what they are interested in, what is part of our engagement strategy going forward. And I'll pass it back to you, Anita. Thank you. I don't see any other hands raised or any other questions in the chat. So with that, Dr. Johnson, I'll turn it over to you, but I will say, I will mention that so far it was going smoothly. We do have 15 polls. Don't be scared for the next session with the SAP. Unfortunately, I'm having technical issues accessing the poll since I'm going to have to log back out and log back in with my colleague and partner in crime, Drew. So I would suggest if Dr. Tachi, if you could just not start your presentation yet, because I need to come back and record. And then in the meantime, just provide some little bit of context. So I'll come back out and log back in. So with that, so I'll turn it over to you, Dr. Johnson, for your introductions, for the introduction. Okay, well, I will do the introduction then. And that's just Murphy's law, I think, Anita. Next, we have Dr. James Tachi, who will present on Public Health General Preventive Medicine Standardized Acceptance Process with the SAP. Dr. Tachi is the Medical Director and Legal Policy Director at New York State Workers' Compensation Board. He holds a time teaching and clinical appointment at the University of Rochester Medical Center, where he serves as the Program Director for the Preventive Medicine Residency and Medical Director of the Travel Medicine Program. He is also an Attorney of Counsel at the law firm of Barclay-Damon LLP. And with that, I am going to turn it over to Dr. Tachi. Thanks, Tisha, and thanks, Anita. I am going to, so when Anita mentioned the technical difficulties they're having with the polling, rest assured, the polling is mostly just to get sort of sort of a showing of hands as I'm giving the talk. And we'll probably circulate the same or a very similar SurveyMonkey to everybody, all of these participants, all of us today, and or everyone on the Program Director listserv. So if we don't get it working together today, that's fine. I will, Anita, do you want me to go ahead and get started or still give you a few minutes to? So I think what I'm gonna, I think what would be useful for you is we'll go ahead and get started. Unfortunately, the polling is not working at this time and I think we can just ask people to type into the chat as to where they agree or not. I mean, we can go ahead and send a format. So I'm sorry about this, Dr. Tachi. No, that's okay. And, you know, it's been happening, we've been doing this for a year now, right? Struggling through different Zoom snafus. So anyway, so this is not our first slide, though, Anita, so I'm gonna back us up to the beginning. Oh, yeah, yeah, yeah, got it, of course, I'm sorry. So anyway, quickly, and I will tell people that I'm gonna move quickly through my slides and I apologize in advance for doing that, but I really wanna save as much time as we can for open discussion. So with each of the questions that we present, it looks like we might not be able to do the actual polling, but please put your responses in the chat and we're recording the session so we can sort of loosely record people's responses. We'll probably survey monkey everybody afterwards anyway. And also, I don't wanna stop with each question to have a big discussion so that we can get through all of them, but I do wanna, again, sort of move through them quickly so that we have time for just, again, some open discussion at the end of the talk. And my goal, if I can shut up and keep moving or just keep moving faster, is to save us at least 20 minutes at the end for conversation. So in terms of results of this year's SAP, they are highlighted on the far right of this slide. Fairly typical of years past, as you heard before, about 42 programs total in GPM, 33 eligible programs. Of those 33 eligible programs, only 24 or slightly over 70% participated in the SAP. There were 57 total slots, only 75 applicants, which is another issue, a bigger issue for another day. Of the spots that were available, 20 were filled in the first choice match, which was the historic way we've done the first choice match in years past. Another 12, so 32 out of the 57, another 12 were filled in the expanded first choice match. And that's where the program directors were given the opportunity. We were all told which of our applicants ranked our program their first choice. And we were able to, given the opportunity to go beyond the first choice match and extend offers to any of the folks who had named us their first choice. And then another 11 were filled essentially on SAP day. So the first day that the SAP opened up beyond the first choice match and the expanded first choice match. So 43 out of the 57 on that first day. I'm just gonna mention the cost for all of us this past year was $175, because we do have a question later on about the pricing of the SAP. Moving forward to the next slide, Amita. So the issues that warranted highlighting, as I mentioned, first of all, the fact that only about 70% of our SAP eligible programs participated as opposed to our high watermark last year, which was almost 90%. I went through the numbers in terms of this year's performance in the SAP. And then we talked a little bit about the cost. Forging ahead. By way of background, I know that there are some new program directors on the call today or relatively new program directors. The SAP standardized application process, this is sort of a quasi matching process that the way it's structured right now, it's basically a web platform. Programs get to list their names and their number of open slots. Applicants get to register. We go through that first and a choice match and extended match that we talked about before. And then beyond that, it's just a series of telephone calls and emails back and forth between program directors and applicants, but people can monitor the status of their programs that they're applying to in real time. Or in the case of the program directors, we can monitor the status of applicants that we are interested in in real time because it's all up on this electronic. Go to the next slide, please, Anita. Also, by way of background, for those of us who remember pre-SAP, and I think even the program directors who are new program directors probably applied to programs as part of the pre-SAP era. It was a free-for-all. There was no timetable in terms of when interviews, offers, acceptances could take place. Applicants could receive an offer, hold it, and then accept another offer and sort of disrupt the programs and other applicants' apple carts on just an ongoing basis. It was pretty, it was fairly chaotic and it was very unprofessional to not put our best foot forward as a specialty, that's for sure. Post-SAP, we've had, again, an agreed upon process and timetable. It's much more professional. We can point to applicants and say, this is our process. The whole thing is resolved. Not the interviewing, of course, but the whole thing is resolved, as we mentioned, within a few days to a few weeks, as opposed to offers and acceptances being bandied about over periods of months. And another nice part about it, I think, is that ACPM has asserted its leadership in terms of coordinating this, and it's an early introduction to ACPM for applicants when they first become associated or familiar with preventive medicine. Next slide, please. So also, again, by way of background, we've been doing this now for about five years. I don't know why I'm this guy right now. I've been on the GME committee and on the SAP task force, I guess, for about five to six or seven years now. And my predecessors all went on to greener pastures, so that guy was it. So I'm sort of our liaison to the SAP process. We've been doing it for five years. Invariably, we have greater than 70% of programs agreeing to participate in participating, which was the threshold that we originally set five years ago. There's been high satisfaction with the program among program directors who uniformly would like to see all eligible programs participate. And most of the program directors actually already have expressed a desire to have somewhat of a sort of a one-day SAP process. Not part of today's conversation, there's a subgroup of those who would actually like us to be part of the NRMP. Again, not part of today's discussion per se. As I mentioned, the majority of slots have been filling in that first choice process or the near first choice process. And then the majority of the remaining slots filled within the next couple of days. Move on to the next slide, please, Anita. So this past year, and I don't wanna dwell too, too heavily on this slide, but there were some negative activities, if you will, this past year. There was an increased number of eligible programs. And just to be clear, when we're talking about eligible programs, we're not talking about the military programs who have their own sort of matching process. We're not talking about combined programs who are linked with internal medicine or family medicine or some other specialty who by definition have to participate in that other program's NRMP process. There are a small number of programs who exclusively recruit internal to their institution, and they do not, Dartmouth is a good example. They do not recruit external to their institution, so they don't count. An expanded definition, if you will, of eligible programs or ineligible programs, there's sometimes, there's instances where programs just don't have enough money, right? So, and we've all been here, right? And we live with this. We talked about it earlier today. If a program says, hey, listen, yeah, normally I wouldn't be one of those ineligible programs. Normally I'd be eligible to participate in the SAP. However, we don't know that we're gonna have funding this year, so we're not going to participate in that. Then that program would also be considered sort of not eligible to participate in the SAP. So anyway, that in the past, we've had that. We will, we can show the data slide again later, but I wanna sort of try to move through quickly. I just saw that comment come in, and we'll share all of these slides with people as well. And the other thing that was interesting too, that was in years past, even non-participating programs, had the sort of a general person's agreement to not offer spots to applicants prior to the SAP day, which was not done this past year. So anyway, there were an increased number of programs that were not participating. Some of those programs were not adhering to that SAP timeline, and they were offering positions in advance of the SAP, which created an appearance. And I don't wanna make a value judgment candidly one way or another, but it created an appearance to the participating program directors that the non-participating programs were doing sort of a bird in the hand worth chewing the bush sort of thing, saying, you know, take my offer now of a residency position, and don't leave things up to chance, you know, in two weeks or two days or a month by participating in the SAP. I just got a note that you folks have lost my sound. Can you hear me? Liz, I see your picture on the- I hear you. I hear you. I can hear you. I hear you as well. All right. Somebody commented that the sound was gone. It must be on their end. So anyway, it gave the impression that folks were trying to game the system. So similarly, I don't wanna, it's not material necessarily who the programs were, but this really raised some concerns among several leading participating programs in the SAP to say that, you know, if programs are gonna do this, then why should any of us participate? And if the whole thing started to come dismantled, then it would be back to sort of that free-for-all that we were talking about before. And it would also lose, you know, it would deteriorate ACPM's leadership position in the SAP process. And then finally, more than one program director said to me, and again, I happen to be our SAP guy for lack of a better term, but I'm our SAP liaison, and I also happen to sit on our ethics committee. And it was raised, I didn't raise it, but it's certainly something that could be looked at. Does this sort of gaming of the system raise questions of ethics or professionalism by those program directors who do it? So this is the context of the sort of the concern that we've been operating in over the past two or three months based, again, primarily on this past year's SAP. And there may have been a bunch of innocent mistakes, but this sort of raised the issue to the forefront for today's discussion. So Anita, we're gonna forge ahead on the next slide. Okay, so in response to these transpirings, the GMA committee met and came up with a set of recommendations that we subsequently presented to the board. So first of all, the overwhelming consensus was just make the SAP ongoing and that we really don't need to do this annual vote every year. We've done it for five years, 70 plus percent of programs vote in favor and participate. So we really don't need to do that anymore. So I will tell you that first and foremost, the GMA committee, based on the GMA committee's input and the board's input, we're not gonna keep on voting every year. We're just gonna do the SAP or some permutation of it. The GMA committee and the board agreed that we should have universal program participation by eligible programs. ACPM is not a regulatory body in this regard, so they can't force programs to participate. But again, the GMA committee and board both agreed that a set of incentives and disincentives towards getting that 100% participation would be worthwhile to pursue. And then finally, again, both of the GMA committee and on the board, the notion of having an ACPM driven one day SAP was found to be quite favorable. Again, not only would it make everyone's lives simpler and probably be reflective of the same results that we've had for the past five years anyway, but it would also make preventive medicine and public health look more like all of the other specialties. And the more we look like the other specialties, the more applicants we will receive and the higher caliber applicants we can probably expect. And as a side note, the GMA committee met after the board of regents meeting and sort of reconfirmed all of this. Next slide, Anita. So two big issues, we're gonna sort of compartmentalize or bifurcate the discussion a bit today. One is to incentivize sort of universal SAP participation and the other is the development of a one day SAP. In terms of the incentivizing the universal SAP, again, ACPM, not a regulatory body, can't mandate anyone's participation. However, ACPM can and should perhaps create a package of incentives which would align with ACPM's goals, of course. I think, and I think most of them would agree, even if it costs ACPM, most of us would agree, maybe the finance committee and the board would agree, but even if it costs ACPM money, I think it would be a good investment, but the goal would be to try to keep it fairly budget neutral, but at the same time, making the changes meaningful to program directors. So if there are gonna be incentives, make them incentives that program directors really value so that you're actually driving that behavior. Again, so all of that was approved in concept, but we were sort of charged with coming back to the board with greater detail and that's what our poll questions, which we're not really gonna do as poll questions, but our poll questions and our discussion is going to inform later today to help inform what that process should look like to the board. The other big issue that the one day SAP, again, it would look very similar to our current first choice match and near first choice match programs and applicants would submit rank lists, just like the NRMP does, et cetera. It would not be as sophisticated. It would be done manually at first, but it would go through rank lists and instead of stopping at that near first choice match level, it would actually continue and it'd be done by ACPM staff with sort of physician participation. It would continue through that iterative process such that as long as you continue to have programs first choices ranking with applicants first choices, that applicant can be matched into that program. So it doesn't have to stop at the first choice match level or the near first choice match level. It would not be making any judgment, value judgments on behalf of programs or applicants to go beyond where those first choices don't match up. It would be binding on both participants and it would eliminate the need for the back and forth telephone calls and waiting to hear back from people and watching other applicants that you would like to make offers to evaporate into other programs while you're waiting to hear in six hours, you're waiting to hear back from them, all that stuff. So it eliminates all of that. And again, it was sort of a proven concept, but requires greater detail, which we'll be discussing today if needed and also sharing via SurveyMonkey, et cetera. All right, Anita, I'm gonna move us onto our next slide. So in terms of the incentives, we talked about things like listing a program on the ACPM website. So in order to be listed as a program that applicants could apply to on the ACPM website, would you have to participate in the SAP? And if you're not participating, then they can find you someplace else. They can find you on the ACGME site or they can find you maybe on the APM site, but they won't find you at the ACPM site. Would discounts be helpful and appreciated? Whether it's program director membership, annual meeting registrations, online educational materials or exhibitor fees. Many of us in the past have exhibitor tables and booths at the annual meeting. Participation in the program director workshop, either absolute participation, meaning you have to participate in the SAP in order to participate in this workshop, which I think most of us find really valuable. Thank you again to staff for putting it together every year or a significant discount for participating programs in the workshop registration fee. There was none this year, but there have been fees in years past. And then another incentive might be to participate in a resident poster session at the annual meeting. The program has to be sort of a fully participating program. And again, in terms of the one day SAP, manual process at first, manual, walking through the algorithm manually, if it becomes computerized in the future, great. If not, our numbers in terms of spots and applicants are small enough that it could probably always be done manually. Binding for purposes of the SAP. And it may, I think that invariably there will still be unmatched applicants and open slots and something will be done with those, whether it's another interview period, either with a repeat SAP for match day, or just go to back to the old SAP process for those few remaining slots where people can phone call and email back and forth and then notify Anita as spots get filled that those details would be filled later on. Can I forward your hand Anita? So I'm gonna very quickly move through some questions with folks. And as Anita mentioned, there's a bunch of them. There's like 15 of them. And I wanna save time for us to talk. So I'm only gonna spend about a half a minute or so on each question. This is not a formal vote on anything. We're not creating new policy. This is purely information gathering, again, sort of a virtual show of hands to let us know where people stand. And we'll repeat this if there's any trouble, registering your preference, we'll repeat this via SurveyMonkey, et cetera, in the weeks to come. Or you can email me. I know that this is not a shy group. I'm not shy either. So we're all on the program director listserv. So if you wanna weigh in on these things on the listserv, then that's fine too. Anita, I'm gonna move us into the questions. The first set of questions have to do, sorry, next slide. The first set of questions have to do with the incentives and disincentives for SAP participation. And I apologize for those of you who are survey design experts. I am not, and I'm even less so at one or two o'clock in the morning, probably when I was doing some of these. Not this morning, but a few mornings ago. But anyway, the first question, just ask the opinion. Do we think that having the single SAP process is good for us as a specialty as a whole and will result in a higher number and higher quality of applicants so that ACPM should encourage programs to participate in the SAP? And I see that folks are weighing in. So that's good in the interest of time and to preserve discussion time. Anita, we'll move to the next question as well. Thank you. And everybody, I'm gonna put in a message and to distinguish between the 15 responses, the following. That's what is great about working with Anita is I was just gonna ask her to do that. And she's like, she can read our minds and does things automatically. Okay, I'm gonna post a message for the next question. So please stop. I'm gonna give like five more seconds for people to post their responses to this question. And then I need to be able to distinguish between the responses. Thank you, everybody. The following will be for the next question. Okay, so the next question has to do with the notion of posting in our sort of a residency program application section of the ACPM website. If you want your program to be posted on such a section of the ACPM website and to have interested applicants or potential applicants know about you, should the program be participating in the SAP? And if they don't, should they be excluded from such postings? And we'll have that, we'll keep that up for about 30 seconds. So we see some differences here in terms of response. Okay, the comments seem to have slowed down on Nita, so I think we can move on to our third question. Thanks, Nita. So ACPM could or should use discounts on such things as program director memberships, conference registration fees, exhibitor fees, things like that, meaning that if you participate in the SAP, you get discounts on these items, and if you're not participating in the SAP, then you're not eligible for those discounts. For the most part, we want the SAP-eligible programs to respond. I forgot to mention that before. We don't want to exclude the non-eligible programs, and certainly we want your verbal comments when we go into the discussion, but we're most interested in the opinions of the folks who are participating or at least eligible to participate in the SAP. Thank you. Okay, so the comments are slowing down a little bit. I think, and I think Anita will probably okay to move on to our fourth question. I'm going to read them in faster. If they see this is just a question answering pricing and bundling and not to sound like a used car salesman here, but if ACPM were to bundle some of these things by saying like, you know, if you do all of these four things in a SAP conference, registration, membership, et cetera, you know, then you'll get a 10% discount across the board. Might that sort of bundling make you more inclined to purchase things that you may be having in the past, like online educational items or things like that. So if you knew that you were going to get a discount by having things bundled, not to sound like Flo, the progressive insurance person, but if you, if you bundle things, would you be more likely? I noticed that I have not been putting my comments in. So I need to, I'll have to give you my comments later. Of course, it gets slowed down a bit. It's going to wait for a few more seconds before I put in the next question. Thank you, everyone. We're moving on to the next question. All right, our fifth question, and this is valuable information, even though we're doing it in a sort of shoestring and bubblegum manner sometimes. So ACPM could or should use the discounts as part of this program director's workshop, or perhaps even the ability to participate in the program director's workshop based on SAP eligible programs participation in the SAP, which means if you want to benefit from all of this information, share, et cetera, then you're, again, for SAP eligible programs, not for non-eligible programs, then you should participate in the SAP, or there would be differential pricing. Right now it's free, so the differential is nil, but differential pricing in the old days, I think they charged like $100, $150 for this thing. And, Anita, are we? Yeah, I think it's slowed down. I'm going to, one second. All right, moving on to our next question. Great. ACPM could or should use the ability of programs to participate in designated resident research presentations at the annual meeting as an incentive for SAP participation. And I'll also say that these are not my ideas. These are sort of brainstorming sort of things that came in, and we're throwing them all out to the group as part of the discussion. And I'm, okay, people are still waiting. And I think they're slowed down, so we can go ahead to our next question. Yeah. And, Anita, if you can switch the slides while you're typing in the next question part, I can just start and throw this in. ACPM could or should charge the Committee on Ethics with looking into, and again, I'm on the Committee on Ethics, and I'm not actually sure where this would fall, and I haven't analyzed it, nor has the committee been given this. But the ACPM should charge the Committee on Ethics with looking at non-participating programs who are eligible, who appear to be gaming the system by offering offers in advance of the SAP, et cetera, as a potential ethics or professionalism issue. I do also want to say that originally when we were going to do this as voting buttons, this would have all have been anonymous. Again, I know that there are no shrinking violets in our group, so I probably don't mind having your votes linked to your name. I think that the votes have slowed down, Anita, so if you want to move to the next slide, you can type in there. Question break. So we're just going to switch gears now from the SAP and the uniform participation in the SAP to the one-day SAP. So we're just going to switch gears, and this is just my switch gear slide. Anita, you can go to the next slide, please, question-wise. One second. I'm sorry. Just one second, please. Don't vote yet. Okay. Go ahead, Dr. Tocci. Okay. So having a single one-day SAP process that looks more like the match is good for preventive medicine as a specialty and may result in either more applicants and or higher quality applicants and would likely be more efficient and convenient for both applicants and programs so that ACPM should facilitate its development. There's a lot of questions come in. Okay. I think it slowed down. I'll move on to the next question. This is a similar question, but it's just framed a little bit differently, but assuming that we have an algorithm in place that assured programs and applicants were matched at their highest rank matches, again, not subjectively putting in any judgment of ACPM staff or anything like that, where there was a link-up, people being matched with their highest matches, and the protections in place like NRMP has, although perhaps not as stringent as NRMP in terms of the rules that you have to follow, which is why we like having ACPM do it. Folks on the line or in the session would prefer efficiency versus not. And again, one is agree, and five is disagree. And I know that's bad survey form. I think it slowed down, so I'll move on to the next question. So as a PD or program, I would be willing to pay a slightly higher fee. And again, I think the fee was $175, and I put this in here as an example that was imposed actually by the board. And it would take additional staff time and effort to do this. So the notion is, would it be worth an extra $50 or $100 of a fee for the efficiency and convenience, et cetera, convenience, et cetera, of having that one-day SAP where we're not making the phone calls and emails back and forth, and basically Anita spends a weekend doing it for us. Not making the phone calls, but preparing people's rank lists. I think it's slowed down. I'm going to move on to the next question. Okay. Earlier in Liz's comments, she mentioned the token system that I believe that ENT used this past year in the ARIS process where applicants were given three to five tokens, I don't know what the number was, where they were able to give those tokens to their most interested programs that they felt they were most interested in. And then that way the programs could look at those tokens and say, these are people we want to look more closely at because we know they're interested in us. So does the group feel as though we should approach ARIS about doing some sort of token system? Again, three to five is the number that's been bandied about. I think it's slowed down. I'm moving on to the next question. Okay. I think ACPM should explore the SAP, moving the SAP to December in order to better align with the fellowship match. So that's when, as people know, residencies typically match in March, fellowships match in December. Is there interest in moving our match from this January, February timeframe that we've been in to March to match up with them? I've got this in the form of two questions. One is, if we're able to move the ARIS application release to July, like the fellowships are, which we don't think there's going to be that opportunity, but we're posing the question in both forms. So this would be if we got the applications earlier, like in July, would we want to move our match to December? And then things are slowing down. The next is the similar question, which is basically going to ask, I'm going to let Anita catch up. Okay. So this is if the ARIS applications didn't open up to us as PDs until September, like they currently do, which would greatly constrict the interview timetable right to only October and November. Would people still want to consider moving our match to December? Not surprisingly, we're seeing some lower numbered responses in that instance. And I don't know, and I don't know how hard and fast the ARIS position is on this. That's why I put both questions up. And Liz may be able to speak to that later on when we're done with the survey questions. All right. Forging ahead. I think we're slowing down. Or we're just having one or two more questions. I think ACPM should explore combining the SAP process with the application process used by occupational medicine, either in December. Historically, occupational medicine does their match in December. This past year, they did it in January because of COVID. And rumor has it they're thinking of moving to January in general because they found it less time compressed. So we've got, you know, whether it's in December or in January, February, when we do it, would we want to approach the OCMED folks about doing this in a combined fashion? We do have a fair number of overlap in the applications to the two programs, and that was the reason for this ask. I'm pretty sure that's our last question. I'm embarrassed that I don't know. Yep, that's it. All right. I'm going to stop and catch my breath. And with that, I didn't save us the 20 minutes that I had hoped to. We only have about 10 minutes. But really, I would just sort of open it up. And actually, Liz, if you want to sort of jump in with the Q&A part as well, I think that people were putting a lot of questions and comments in the comments as they were voting. But if folks have questions, Dr. Lane, I see that Dory, you have your hand up. You're muted. I just wanted to, I can't type fast enough to keep up your questions. That's why I didn't vote. But in terms of the one about the tokens, the fear I would have with our specialty is that we don't, it's from the resident side, the applicant side, because we don't have a huge number of programs, number one. I don't know how easy they would find it. And I think it's more of a burden on them actually to, it's good from our side that we would know who's interested. But I mean, it's not the same as you're applying to an orthopedic program, you know, pick your top five choices. We don't have a tremendous volume. We don't, and some of them may have geographic constraints, but I don't know. I don't feel strongly. I just wanted to bring up that perspective. I'm just kind of scanning through the group. I don't see any other hands up, but if people just want to jump, oh, there's Pat. Yeah, Jim, thanks for this presentation. I think that'll be interesting to compile the votes just to get a sense of where there's a consensus and where there isn't. And I think this is sort of, you know, the tragedy of the commons issue that, you know, the resource is applicants and, you know, do we fight over it and, you know, make the fishery extinct and have a bad process or do we govern? So I think, actually, you've sort of proposed some of the principles of governing, avoiding the tragedy of the commons. I do think one issue that we should address, maybe it's ethics or just honesty, is this issue that has to do with the last question, which is the issue of when a resident comes to you and says, I really want to come to your program. It's like, do you say, no, you're not supposed to. I'm going to rank you number one. You should say, you're not supposed to say that, but now I heard it. So I can't say what I want to say, but to you, because I'm not supposed to. So it's just how do we, in effect, that's what, you know, if you ask the residents to rank their top five programs and you see you're on their list, you know that, you know, you're one of their top programs. So I think more explicit direction with some examples would help us finesse that, because anybody who's applied to a match knows what's supposed to happen and sort of the wink and nod and says, you know, do you want to look at houses when you come through? Because you might want to do that. That isn't telling them that you're ranking them number one, but you've just told them that they're probably going to be ranked number one. So I just think more explicit direction, because what we had happen this year was actually people were just sealing the deal outside of the SAP. And that clearly was frustrating for me as a program director, because a number of my top applicants, they said, you know, Dr. Remington, I've got this offer and I'm going to take it because you haven't told me whether I'm your top two candidate. And I said, I can't. You're a competitive applicant and we think you'd do well here, but I can't make an offer. And they said somebody else did. So that was frustrating. I will add to that, just with our experience this year, two-thirds of the applicants who had engaged to have interviews with us, advised us in early January that they had received offers in December and backed out of their interviews too late for us to really get other applicants to interview. So it really negatively affected our application season. And we thought, why would we do this again? Jennifer, I had the same expression. I think I expressed that to Jim. I said, I'm not doing this next year. One hand behind my back is okay. Both hands behind my back. I'm not going to do this app again if I can't compete by making offers. Jill? At San Diego, we've also decided that we just can't do this without 100% participation because you're at a disadvantage being in the SIP. So I don't know how we do that. I didn't see all the voting, but it appeared the majority of people did feel very strongly about all in. We need to look like all other specialties and that it's not an option not to be part of it unless it's a funding issue, obviously in the military, et cetera. And I think we all agree on that. Did anybody who did not agree on that want to speak out? We do want to hear that side. Because the GMB committee is very strongly in favor of 100% in. There is no option not to participate. That's off the boards right now. Hi, this is Caroline. Can you guys hear me? Yeah. So we were really on the fence whether we wanted to participate last year. And if we hadn't have participated, that would have been 69.6%. And whether you, yeah. So whether you would have assumed that was 70% or said, hey, you know, we haven't met our 70% threshold. And it made me laugh at the end because ultimately we decided to go with the SAT process, even though it doesn't work perfectly for us, you know, as a public health agency program. And to hear Jim say that, okay, well, we're not even going to query people whether they want to do it for next year. That was a little concerning to me because we were right on the borderline and we could have made a difference maybe or maybe not depending on how you looked at 70%. And, you know, and all I would say is, you know, and actually the SAP worked very well for us this year. There were other people who communicated with us and said that they had an offer during the interviews. So for us, that's kind of like a wasted interview. It's wasted time. But I think the question that needs to be asked is, you know, sort of like a no blame, no shame type of question. Why did people feel that they had to offer the positions earlier? You know, because there's probably something behind that. And I can tell you our frustration with using the SAP process is that we interview like a lot of people, like, you know, for we have two to three slots, we interview 13 or 14. And in past years, we have matched with none of the people that we have interviewed. And so, you know, because I think our program is just very unusual, it's not academic, we're pretty strict about what we want people to do and where they are. And it's not like you get to do rotations and do what you want here and there. You're in a local health department, working as a, you know, sort of like a resident under the local health officer. And that's just not for everyone. So for us, it's kind of like, and so then what I do is, you know, you scramble and we found find really good people. But and like I said, it worked very well for us this year. But seeing those numbers, I'm like, well, maybe we'll decide not to do it this year. But so anyway, so I would maybe talk to the people who offer the positions, talk to people who it's not perfect for like us. So maybe the token, maybe for that will be the token may actually benefit you, you could see who's really interested in you. Yeah, exactly. Yeah. And, and I would say as well that, you know, and we've had this debate in our program with prior directors as well is really ACGME the best place for us, because we are our focus is not clinic, where I think the majority of ACPM program or preventive medicine programs do want to incorporate the clinical piece. And so it was very interesting to me that medical genetics actually now has no clinical requirements. And so I would like to learn more about that, and how they got that designation. I have a question about the fellowship model. Because being a fellow, I mean, I'm a fellowship model, but I'm still part of SAP. I think Dartmouth only recruits from Dartmouth. So that's different main, you're still recruiting from the same pool, I believe. So just to be, just to be clear, there is, and Dory, I know you had your hand up and out, we're getting pressed for time. But there is no such thing as a quote, fellowship model. Okay, so Dartmouth is unique in that they are an internal recruiting model in that they only accept applicants from Dartmouth, people can people can set as an admissions criteria, only admitting applicants who have completed a prior residency, as opposed to just the one year, that doesn't make you a fellowship. And if you ask ACGME, they'll tell you that you're a residency, if you ask ABPM, they'll tell you that you're a residency. So there's really no such thing as a fellowship model. So there's, there's the Dartmouth model that only and exclusively recruits internal applicants. But if you recruit anybody from outside of main Medical Center, or anybody else who may be using sort of fellowship type criteria, that doesn't make you a fellowship model that just makes sure a residency program who's eligible to participate in the SAP that has a different set of admissions criteria to your program. And Dory, I'll let the staff let me know if we have time for more. I think I think Dr. Thomas and Dr. Ettling have had their hands raised for a while. So we'll just take those two comments. And then we'll move on to the ABPM presentation. Go ahead, Dr. Thomas. Okay, I just had a question. How does the occupational medicine group get full, they they have full participation? How do they do it? I will just say, and this is me editorializing, and if there's any folks on the call, they, they have at or near full participation, but they also are much more loosely organized in terms of their process. And they also don't all adhere to the process very strictly. So I wouldn't I would not having gone through both processes, I would not hold up Ahmed as a as a pathway for us in terms of their process. I'll just say that. And just one addition, I did speak to some Ahmed people this week, who said, gee, GPM never about joining forces. They said, Oh, no one from preventive medicine ever came and brought that up before. So I said, Well, I'm bringing it up now. Their meetings in June, they're going to bring it, they thought it was, you know, anecdotally, a really good idea. They are very open to moving to January. And so then we could both do it in January, we start we get errors, end of September, we could both match in January, we could look, you know, more similar. The other thing is 100% participation in SAP is what are, you know, I think that should be our goal. If there are some programs that don't want to do it, that's really fine. You just can't get your, at least we would like some agreement that you can't swoop in early and get people on positions before the SAP deadline before all the other programs actually go through the formal process. If you want to do it the next day, not the day before, which is what was happening last year, that I think that was the concern. It's not like you have to be part of that. You just can't swoop in early. That, you know, I thank you, Dr. Gunnar. I'm sorry, we have two comments, one from Dr. Ettling. She has her hand raised. And now also from Dr. de la Vega from Boston. So Dr. Ettling, go ahead. I just think that that we are being very premature in even bringing up the discussion of whether the SAP is going to be a required all-in because of the funding situation for our specialty. There are programs that don't know from one year to the next whether they're going to be able to continue supporting the residents that they have, let alone if they're going to have funding to support another batch. And so the all-in discussion should not even begin until we as a specialty are able to stabilize our funding sources. So just to be clear, and we included programs with insufficient or uncertain funding as those who would be deemed not eligible if they had raised their hand and said we can't participate this year because we don't know that we have funding. And then, you know, the expectation would be fine. But if you get funding, you know, a month before the SAP, then you should participate in the SAP. If you get funding two months after the SAP, then, you know, fill your position. That's great. But not, you know, so the funding question really, we've tried to take that off the table. And I would just add, there's all kinds of, you know, whether it's Lincoln or Colonial, I mean, there's the whole notion of, you know, a house divided against itself can't stand or, you know, we can survive together or perish as individuals. I think that if the SAP does disintegrate, the programs have been successful over the years will continue to be very successful. The programs who have struggled will struggle more. And the profession itself will attract lower quality applicants and fewer applicants because medical students or residents don't like the uncertainty of, just the uncertainty that we used to have. That's just my editorialization. And I'll leave it up to staff if we're going to continue the discussion or move on. Yeah, I already called out Dr. de la Vega from Boston. I don't know if Dr. Bursz has already shared his concern, but Dr. de la Vega, if you still want to share your comment, go ahead, and then we'll move on to the next session. Thank you. Thank you, Anita. It was just mainly what Jonathan put in the chat. In our case, it's different because we are under the General Medicine Fellowship, not really as a fellowship model. So that makes it really hard to compete with the other fellowship programs around the area. Dr. Lane, go ahead. I know you have your hand up. I did because, you know, you said earlier that you're going to make the assumption that this year we're not going to have a vote. We're just going to have SAP. Now we have a lot of people discussing the fact that they're hesitating and they may not want to. So nobody, I don't see, I think you either have to have a vote or, you know, give up. Yeah, actually the decision was already made to not have the vote anymore. And when you looked at the poll answers and when people were putting in their answers to the question, the overwhelming majority were, you know, ones and twos in terms of not voting anymore. Because, I mean, internal medicine, family medicine, pediatrics, they don't vote every year whether or not they're going to, you know, do the NRMP. They just do it every year, right? Well, I agree that it's annoying to vote too, but you've just heard from several people that they participated last year and they're not going to do it again. But that's because they got preempted by programs. And so that's why. So we're going to make sure that there's no preempting. And so therefore, if they don't want to participate, they have to wait till after SAP. We have to agree upon that as a specialty that we have to look organized to the applicants. And we all compete for general internal medicine fellowships and general pediatric fellowship. A lot of people who come to all of us are choosing between the two specialties. I don't think that's a reason to offer early a position. They, you know, the applicant has to choose. And GIM, at least in my case where last year it happened, they were very nice and they waited till after the SAP to hear back from the applicant. I think these are ongoing issues, but we still have to look uniform. And I think we should look like other like other specialties. That's where I stand. I think we should look like everybody else. I think we should have a hundred percent participation. I agree. Yeah. Just a quick comment. Mine was conditional. I said, if nothing changes, then I won't participate. I agree with Liz. I think if we would have a hundred percent participation and some more clarity on rules of engagement, then I'm happy to participate. I was happy to participate in the past. I think, you know, delaying it this year because COVID may have contributed to this feeling of it's time to make an offer. I think moving back to our original timeline, moving maybe even a little bit earlier would help. Okay. Thank you, everyone. I know there's still a lot of questions and concerns that you want to discuss. Maybe we have to have a separate Zoom meeting just for all GP and PH programs to air their concerns. But Dr. Johnson, I'm going to turn it over back to you. I know Dr. Murray is getting prepared to present her on ABPM. So just quickly, thanks everybody. I know that was a lot that we covered in a long lot. I know there are a lot of questions and as Anita mentions, maybe we'll have to have another session to sort of review this again, but I appreciate everybody's input and participation and we'll process it for the group. Thank you, Dr. Tatchy. And unfortunately, that's all the time we have for questions. And I like Anita's suggestion of possibly having another meeting to spend more time on this. But if anyone has any additional questions on the SAP, please email Dr. Tatchy, Dr. Garland, or Anita. Next, we have Dr. Carolyn Murray, who will present updates from ABPM. She is a public health preventive medicine physician with expertise in occupational and environmental medicine. For the past five years, she has directed the community outreach and translation core for the NIEHS slash EPA Dartmouth Children's Environmental Health and Disease Prevention Research Center. As a member of the founding faculty for the Dartmouth Institute, Dartmouth-Hitchcock Medical Center Leadership Preventive Medicine Residency Program, she has mentored dozens of residents in a public health and preventive medicine training. She is board certified in general preventive medicine, public health, and occupational medicine. She is a fellow of the American College of Preventive Medicine and serves on the Board of Directors for the American Board of Preventive Medicine. Welcome, Dr. Murray. Thanks. Can you hear me okay? Yeah, I'm just going to share your slides in just a second, Dr. Murray. Okay, so you're going to put up my slides and I'll just pass them? Okay, great. Thank you. Can everybody see Dr. the ABPM slides? Can anybody give me a thumbs up? Oh, yeah, I see them. That's good. Second for a second, I didn't see them. That would be a bad thing. Okay, and we just make sure I can advance here. I know if you if you can. Okay, if you can just say next slide, I can move. Okay, you're not going to give me those options. Okay, well, thanks very much for this opportunity. And thanks for the introduction. Go ahead and advance, please. I have no disclosures. You can go ahead and advance. So here's our outline. I just have I'm sensitive to the time today. And I do want to allow at least 10 minutes or so for questions. But this is just a rough breakdown of where I'm going to be concentrating on. Go ahead, next slide. So just for those of you who are new to your residency, director's roles, just a reminder that under the umbrella of ABPM, we do have our three specialty, three key specialties, aerospace, med and obviously public health GPM. And we do under our umbrella of certification have the subspecialty areas, addiction medicine, clinical informatics, med tops, and undersea and hyperbaric medicine. Next slide. And our board of directors is represented by individuals from each of the three main specialties. Our current chair is Joe Ortega from aerospace and soon will be soon will be the chair position will be taken over by Wendy Braun from PhD PM. We also this this year want to point out that we have a public member George Roberts, we expanded our board to include a public member member, and we have provided a directorship for representing our sub boards. Next paragraph. Next slide, please. Here's the chairs of our three sub boards. And Michael Weaver is the one who represents the sub boards on the board of directors. Next slide. And this is our staff in Chicago for ABPM under the leadership of the executive director Chris Andrula. And I just want to call out that they're very interested in being receptive and your key stakeholders to the staff. And so their contact information is on our website. And I really encourage you any issues, whether it's with residency applications or other other issues that you reach out to them. And I think we've gotten very good feedback about the turnaround for any kinds of inquiries to the board. So these are really, it's a great staff, and I encourage you to seek them out as needed. Next slide. So the goals, as you well know, of certification are to assure the public by providing this voluntary standard of assessment recognizable as designating a high level of knowledge in the field. And so this primary public assurance is really a key part of board certification. And then obviously we seek to advance our profession amongst our peers and in the knowledge of practice, education, and evaluation process of preventive medicine. Next slide. So quickly some diplomat demographic data. This slide has on the top the different kinds of ABPM diplomate certificates with aerospace, med, PH, GPM on the far left. And the non-time limited and time limited are broken down. And obviously over time, the non-time limited will be phasing out. But I think the thing to point out here is that under our umbrella of preventive medicine, our subspecialty certificates are starting to drive our numbers and addiction medicine and clinical informatics. So while we have over 12,000 total diplomates, there's a growing proportion of those that are coming from our subspecialties. Next slide. In terms of our specialty certificates, PHGPM and AACMEN are the lion's share. Next slide. And then as you predict, addiction medicine is driving the pie on the subspecialty certificates as a proportion of all the certificates. Next slide. So a quick review of a really challenging ABPM cycle last year. We had to revert pretty quickly to offering sort of a home-proctored monitored exam and juggling just all the challenges we face with COVID, but the staff really did a phenomenal job. This is the registrations over time. We had a peak up in 2017 with addiction medicine and last year we had over 1600 registrations for our exams. Next slide. This is for public health GPM exam registrations. You can see by the last few years, it's been relatively flat. This year, we're obviously still in the process, but we're not sure we'll make our 20, we might be just a little bit below 2020. Next slide. Next slide. So here is a display of pass rates with the blue being the core exam and the green being the specialty exam and PHGPM. They just gave me data for public health GPM and not the other specialties. But just to reinforce that this is all exam takers and residency program graduates, which are considered our reference population or most highly prepared, generally have acceptance rates into the 90%, well into the 90% on both the core and the specialty exam. Next slide. So I just wanna review the initial board certification pathways as we are making some changes that are gonna go into effect in 2022 and 2020 at the end of this exam cycle. For all of our pathways of certification, medical licensure is an absolute requirement. It has to be a current unrestricted and valid license. Obviously, you have to have a medical degree and successful completion of an approved training program or the other requirements that I'll outline in the other pathways. Next slide. So the residency pathway as your role is to actually assure and document that residents have successfully completed the program. Just a heads up that the alternative pathway that's been in effect for quite some time and is limited to medical school graduates prior to 1984, so it's going to be a rather older cohort, that's sunsetting after this exam cycle. And here I have outlined the current complimentary pathway requirements, which included a practice requirement, five of the core content areas, graduate coursework. And I just wanna call out and I'll go review quickly that these requirements are changing after this exam cycle. So this is what's in place for this year, but starting in the next exam cycle, there will be some changes and I'll get to those in a minute. And then the special pathway, as you know, has been an option for diplomates to have a certification in one of our primary specialties that with practice experience, they can apply to take a specialty exam in another discipline under ABPM of the core specialties. And there'll be some changes to that too, that go ahead and advance the slide and I can review those. This will be important for you to be aware of. So the changes to the complimentary pathway that are gonna be in place in the 2022 cycle, the same that applicants have to have two years in a residency prior to the one-year residency in the PM specialty. Again, your role as a program director is critical here to attest to the successful completion of 12 months of residency. The one-year preventive medicine residency must include direct patient care and those are outlined here. And the complimentary pathway applicants must have an MPH. And the MPH and direct patient care requirements, which are new in this revision of the complimentary pathway are gonna eliminate the need for us to document a practice requirement that was previously in place in the prior pathway. Go ahead and advance the slide. The changes to the special pathway, and this is important for residents who you think are gonna, or even some of your graduates that may have ended up working in one of our other primary disciplines, is a special path may only be used to qualify for one additional ABPM specialty certificate. So if you did a residency in PhDPM and later found yourself working more in an occupational medicine capacity after the requisite time, you can apply to take the specialty exam for OCMED, but you can't then, if your career changes again and you end up in aerospace, do a third time. And so that is going to place in 2022. And in our application review, these applications will get reviewed by vice chairs of both the specialties, the original residency path certificate, and then the special pathway certificate vice chair. And this is just really to make sure that we're reviewing practice experience and making sure it's distinct and qualifies to be able to sit for the secondary specialty exam. Next slide. So I encourage you and your residents to take full advantage of the website. And there's an MI eligible tool that really helps guide people interested in board certification. And this will have additional information over what I've just provided. Next slide. So as you know, the specialty certifying exams have two parts, the core and the specialty exam itself. It remains computer-based and we continue in our affiliation with Prometric Testing Centers. And we will be using in-person Prometric Testing as our plan for this year's testing cycle. Next slide. The core exam is felt to be representative of the sort of the MPH curriculum that's embedded in all three of our residency programs and is considered to be the content that should be known by all preventive med specialists. It contains 150 questions and there's a three-hour time limit. Next slide. This is the current breakdown, which has not changed in quite some time in terms of the percentage of questions that comprise the exam, reflecting the importance of epi and biostats as the largest component of the exam. Next slide. I'll get to the changes that are going in core later, but the specialty exams are, obviously each cover the specific specialty, again, 150 questions and a three-hour time limit. Next slide. So for public health GPM, this is the current breakdown and again, has not changed in quite some time with 30% of health services administration, 50% environmental health, 10% biostats, 15% epi and 30% clinical preventive medicine. Next slide. The questions are outlined here. We've put a lot of effort in the past five years in particular with our affiliation with NBME has helped as assisting us in our exam, construction of our exam and really are minimizing, eliminating recall questions with a much more emphasis on interpretation and problem solving. But each question does have the format of a STEM, four or five choices and a single best answer. Next slide. The timeline is that the exam will be administered in the fall. We scored the exams in November and December and then in January, there's a, December, January, there's a key validation process where we go over the performance of each question on each exam and any question not performing is flagged by the psychometricians and we review it to ensure it's rekeyed correctly or there's no issue with the question and we may drop it based on its performance stats. And then each candidate's exam is then re-scored. Next slide. And then December, January, we review the exam results as a full board. And then these scores are then equated to it, equated to a standardized score which permits a passing standard for each exam to represent the same level of proficiency year after year so it accounts for whether one exam one year is assessed to be more difficult. So we'll bring it to the same proficiency level. Next slide. So the certificates are, the results are disseminated at the end of January, early February. You get a certificate that's dated the year, January 1st of the year after the exam and then it expires 10 years later. Next slide. Just to remember that if a candidate fails either the core or specialty exam, they need to retake only that portion that they failed at the next exam offering. And then if they are using the special pathway to take another specialty exam, if it's within seven years of their passing both the core and specialty exam for their first preventive medicine specialty, then they just need to take the specialty exam as long as it's within seven years. Next slide. So some important dates to remember, the application window opened in mid-March. Those who've already submitted their applications and been approved have been able to register for the exam because that opened on May 25th. One call out to your residents as they're registering for the exam is that a reminder that to them that they have to register, they're actually registering for two exams because they registered for the core and their specialty exam. So there's always a few residents that get caught up in that and that can delay their option to get good seats. So just a good reminder for your residents that are applying this year that they must apply for, must register for both exams, get seats at the Prometric Center. The window closes June 30th. So if they wanna avoid late charges, please encourage them to get it submitted by then. Materials needed to complete the applications are accepted through the summer. So it's really, you know, they don't have to have every piece of paper and every piece of documentation in by June 30th, but they have to have their application in. Registration closes on September 17th and the exam this year will be administered the last two weeks of October with results generally late January, early February, the latest, the next year. Next. And then some updates. So this is what I was alluding to when I was looking at the core exam breakdown of questions. I mentioned this last year, but we did engage an outside entity to help us do in a practice analysis. And this was done just before COVID. And we started with core because it touches obviously all the specialties. And this actually involved, some of you may have been aware, participated in it, pretty extensive surveying. We had volunteers to be on a task force. Some of our diplomates actually kept diaries of their work activities. The survey was really to get a sense of what people are actually doing as they practice preventive medicine. And this was administered to a proportion of diplomates across all three specialties. I think the work journal data collection was really fascinating. We had some focus groups and then this large scale validation survey. And through that, it was to sort of look at what our diplomates felt was applicable to their current work and where we needed to change the layout and blueprint of the core exam to reflect that. So next slide. So what you'll see here is we are in the process of updating our exam blueprint for the core exam to reflect the recommendations of our practice task force. And that you'll see the changes on the right. This is a process where we'll be reallocating where our questions are being written and adding topics that populate under these outlines. So clinical preventive medicine remains, is at 25%. Public health slash population health comes in then at 25%, epi biostats and informatics at 20%, environmental medicine stays at 15%. And then strategic healthcare leadership, which is components of the prior health services administration, but adding more topics that are more topical for now is at 15%. So these are, we're currently working with NBME and our item bank to reorganize and re-strategize our questions under this new blueprint specifications. So this practice analysis, our plan had been to start doing public health GPM as the first specialty to go into a practice analysis. That got sidelined as we were supposed to launch that right in spring of last year. But ACT pro exam is now has a new schedule for us. We delayed it and we will be kicking this off in the fall. And I just hope that you'll look for communications from the board because this is gonna be critical for especially our new diplomates, your new graduates are out in practice. It's gonna be so critical that they participate in this and share their practice, what they're actually doing with their preventive medicine training so we can get that data. And cause our next task is to really update the blueprint for our specialty exam. So there'll be a range of volunteer options ranging from a task force to being in a focus group or being interviewed or actually doing the job diaries. So there'll be more about this coming out in the fall but I just encourage you to participate and especially encourage your relatively recent grads because that's just a demographic that we really wanna capture. And then our plan is to have this revised blueprint delivered in 2022. Next slide. The other thing that I think is important that relates to some program directors is that we have closely looked over our policy about massive open online courses. And there have been applicants and who have wanted to use those to meet the educational requirements for the boards. Most often these are Coursera or Next Gen U or other MOOC providers. And our challenge with this is that the general MOOC framework does not provide any mechanism to actually assess learning or that the applicant actually participated actively or that they're not SEAF, the Council on Education and Public Health accredited. So our policy has been and continues to be that if a MOOC course is wanting to be used it has to be part of a university transcript or done under the auspices of an existing program. Next Gen U has some relationships with degree granting entities that they feel they can use that mechanism. But that's really just from a quality assurance standpoint where we had to make a decision. So we have to have a transcript that validates that there was an assessment and that the individual actually took the course and met the criteria. Next slide. Oh, then we're at questions. And I think, so I'm hoping there's some questions here. Hi, thank you, Dr. Murray. So the questions in the chat box were basically the same. When will these exam question changes be implemented and when does the new blueprints go into effect? And it sounds like that will be delivered in 2022. So does that mean that will be on that exam that year? So we're in the process of, I think the challenge we have right now is that we are in the process of changing our exam vendor because NVME is getting out of the certification exam business for many smaller boards. I think the plan for the core is still for 2022 but we would provide some communication update when we finalize that. Next slide. Dr. Lane, go ahead. And I know you also had a question in the chat. You may want to repeat that if you have that as well. Yeah, I just wanted to clarify something. I was a little confused between, you had a slide where you talked about application dates and registration dates. So I don't know if they're different in terms of the application. Am I correct that it's due without penalty by the end of June? Correct, correct. So June 30th. So application registration is when they register, they have to be approved, their application has to be approved and then they're permitted to register at Prometric to actually take the exam. For the exam site, okay, for the exam site. And that's the situation where they come up with having to register for both the core and the specialty exam. Right. Not in terms of the application, it's one application. So I'm sorry if I confused you. I'm glad you brought that up. I am. Okay, the other thing that I may have put in the chat, which just maybe for the new program directors or people who don't know it, the situation with regard to the complementary pathway to the exam. In the past, I've had a couple of residents who because of their prior training, how many years they had, they wanted, they asked me if they could take the complementary exam like a year before they graduated. They intended to stay in the program anyway. But I know some people do it and want to leave. But anyway, the ACGME does not count them as residents. So in other words, when the RRC is contacting the ABPM for exam scores, they only take the ones that are, they don't take the complementary ones. So here's like, they're not residents, which is, you know, that's not good for the program. For the residents, I can understand why. It's great for them if they pass early, then when they go for a job, they're already certified. They don't have to say that they will be taking the exam, you know, in a couple of months. So I wonder if there's any way to, you know, for ACGME and ABPM to get together on that. I will bring that up with Chris Andrula because he certainly meets with our counterparts. And we have done that. There has been a mechanism for us to do that in the past with the CDC, with some other training in place programs, where we worked on what we were calling them a complementary pathway versus a residency pathway. But I will ask Chris specifically, because this is, it's really an ACGME issue, not ours. For us, it does, they get put in that bucket because the criteria are different. And that's just the way we collect our data. But I think we're very willing to communicate in ways that are beneficial to programs. So it's a great example of something that we should try to figure out how to do. Yeah, I mean, initially when that pathway was created, it's almost like an anti-residency track, or it's not helpful to residency programs. It's better if people go through the program and that's the way you get certified, not to count other pieces. But I think at the time it was created, I don't know what the thinking was or what was going on. You don't intend to phase out that pathway? No, and I think because where we do see it being used and actually really successfully in this training in place program with the CDC is a great example of people who are really working in public health and are either often are board certified in a different ABMS specialty. And they are able to do usually over a halftime over two years as a supervised preventive medicine residency. They often already have MPHs. And these are great candidates. I mean, these are people who wanna be in our family because it reflects what they're doing. And so it's been a way for pulling in really qualified individuals. Okay, that makes sense to me. Yeah, Dr. Johnson, there are questions in the chat. Yes, so I think that answers the one question that those that go through the complimentary pathway that it does not count from the total passing rate for the program, correct? Correct. Okay, then the next question in here, Anita, mine's frozen. I'm sorry, I'll read the question. Does the change in the core exam questions for mental and behavioral health include more of these questions in the specialty exam? So we're in the process of kicking off the specialty exam practice analysis. So we haven't done that yet. So I can't speak to what the influence is gonna be on the breakdown on the specialty exam. Does that answer that question? I think so. And then the next question was complimentary pathway examinees take the same exam as the residency pathway examinees, correct? Correct. And then I actually had a question of my own. Earlier in the session, we had talks of we have some residents that will be having completed their capstone within the residency time period, but their degree wouldn't be conferred until after graduation. So what kind of documentation would they need to sit for the boards that they've completed all of the MPH degree requirements, but haven't been given the degree yet because they missed the graduation deadline? That's an excellent question. And you should encourage them to apply and put their application in before the deadline. And then often with a note from their program director saying everything's completed except this one item, we can review it and approve them. We put them in approved pending and then they get to September 30th, I think is a cutoff for that additional documentation. The same thing, because we have some people whose MPHs don't get conferred till the summer or they're working on their thesis. So we really work to handle that. And generally with just a document from the program director or sometimes it comes from the MPH program, if it's a thesis, we can then approve, as long as they meet all the criteria, we approve them pending and we actually allow them to, I believe, so they have till September 30th to submit that. And I think they're allowed to actually register for the exam. Thank you. And then we have a couple other questions and then we have a few other questions, but I think then we'll have to move on. For candidates in the complimentary pathway, should the one-year PM residency meet all requirements that other residents would normally do in two years? Because they already have an MPH, no, but there is outlined in the complimentary pathway under ACGME what they would need to do in their second year, but because they have an MPH, that's one factor in that, that they wouldn't have those requirements. And who will be the new vendor if NBME is no longer working with us? Oh, good question. And I'm not sure if the ink is dry, so I'm going to defer that one, but we had a number of really qualified entities apply and we were confident we have a good partner. So I can't, I'm not going to disclose it, mostly because I can't remember off the top of my head who we picked. Regarding counting a Coursera course, what is meant by the statement, it must be part of the MPH program? So there, well, there's been some effort in some programs that they will allow a, if they're MPH residents or someone enrolled in MPH program, and they allow them to take a different kind of course to meet the degree requirement. We're just saying it has to be vetted by an entity on a transcript. How that actually occurs under Coursera or NextGenU is really up to those entities. Can I just follow up on that? So somebody comes with an MPH, but when we look at it, they didn't take advanced stats. So you mean for the complimentary pathway? No, to be eligible, that's it for the boards. Yeah, no. So then in those situations, then that would be incorporated then into their, into their PhD PM year, whatever they need to meet the requirements for the board would come under the program director's purvey. And that's part of, you assess what they have coming in and what they need. Yeah, sorry. And I think, I don't see any of the hands, and we had one other question about Prometrics from Dr. Morgan Lander. Sure. Yeah, I'm sorry. So I don't know if this is only one aberration or if anybody else has had issues, but one of our 2018 graduates was taking the exam a year later in 2019 and was scheduled for the two parts on consecutive days on the last two days of testing and received a phone call on a Friday night before the Saturday deadline saying, oh, just kidding, our spots are all full and you can't come tomorrow. And then had to defer her part two. Now, she was able to, because she was paid and approved, had to defer her part two till the following year, has since taken it and passed. But I was appalled that that was able to happen. It was a Friday night. I didn't know what recourse there was. And then whenever she checked with the board, of course, the time was passed and there was no more testing available. So I didn't know if that was one single aberration, which I hope it was, or if anyone else had ever had similar experiences with Prometric, but I'm hoping that that will never occur again. No, and I would hope that too. And as I said, those are pretty isolated, but those are the things that the board, the staff would be on top of. And I think unfortunately that was, because there was those last two days, but that shouldn't have happened. And I wasn't aware. I mean, I know the board, the staff would be aware of that, but we would want to hear about those, the feedback on that for sure. And I'm sure the board staff was involved with that, with helping this person. I would hope they were, but I'm glad you brought that up and I will bring that, mention that too, that came up at our meeting today. I mentioned that to the staff. So Dr. Johnson, I think those are all the questions. We're already almost through our break and that's fine because I know that we were delayed and these were very important questions. So I'm suggesting Dr. Johnson, if it's okay with you, we can just take a quick 10 minute break and come back at 3.10, because it's a long one and a half hour session followed by another break and followed by milestones. So stick with us, we're getting there. We're getting to the end and tomorrow is Saturday and Sunday. So let's meet 10 after wherever you are for the next session on the DEI. Thank you everybody. Thanks, Anita. Thanks everybody. Of course, yeah. So Dr. Tachi, I know you're still here. Oh, okay. Oh, I'm still here. Yeah, no, I was wondering, I think it might be useful to have a separate call just on the SAP alone. It looks like there's a lot of people wanting to air a lot of concerns. And so I'm suggesting at least an hour, hour and a half meeting at a time when everybody's available, at least they can send one representative, a PD or an APD to be a part of the discussion in July, August after the new class cohort has settled down. Yeah, I think that's a good idea. I think that there was a pretty vibrant discussion. Yes. I think for the most part, folks have a common set of goals, but folks also had some concerns that I think we should discuss and work through. And ideally, the discussion will lead to a better process, right? So we can even, I'm up to if people want to do a hour and a half, two hours, even to be able to work through everything, maybe take a break in the middle, something like that. We'll pick a Friday in July or because everybody's here. Yeah, I want to give you back some of your break time. So I'll definitely coordinate with you and schedule something that'll work for all of us. Thank you. Very good, yep. Bye-bye. So, Dr. Johnson, it's 3 10. Do you want to go ahead and get started or. Yes, let's, let's go ahead and get started. So welcome back everyone. Thank you for attending the next meeting with us today for this session. Dr. Carlson would begin with a brief overview of a CPMs efforts in diversity, equity and inclusion, and broadly within health disparities. In addition to being a CPMs president and professor, associate provost, senior associate dean at Tulane University School of Medicine, Dr. Carlson would was also a former program director for Tulane's preventive medicine program. Welcome back. Next, I'm going to turn the session over to I have more introductions to do, I'm sorry. Next, also on this session we have Dr. Anita Givergese, and I'm probably butchering your name, I'm sorry Anita, and Dr. Clarissa Hoff, leading a discussion on improving diversity, equity and inclusion. Anita is the senior director for the public health preventive medicine residency program at the New York City Department of Health and Mental Hygiene. In this role she oversees the health department's medical training programs, including the residency program, fellowship and public health practice and the elective and public health program for medical students and residents. She further serves as the designated institutional official for graduate medical education for the health department, and has the department's accreditation related committees. In addition to her work in medical education, she also participates in the department's various emergency response efforts included COVID-19. Dr. Clarissa Hoff is an assistant professor and preventive medicine residency program director at Tulane University School of Medicine. Her primary care clinic is located in the area of Louisiana. She is the director of the Southern Area Patient Oriented Research Organization. Dr. Givergese is a practice based research network located in the Gulf South, that focuses on research and quality improvement in the real world clinical setting. Her research interests include making quality improvement feasible and accessible to boots on the ground physicians, health disparities and preventive care. Thank you all for joining us today and presenting. Thank you. Yes. Thank you. I thank you so much for the opportunity to talk with you all again I've been popping in and out of the of the workshop today and certainly I'm impressed with the amount of presentations that you all have had in the work that you all are doing. Certainly, you all know I know you've experienced in your own work and your own professional and personal lives, the importance that diversity brings to all that we do and all that we are. And for the college. It is also critically important that we continue to stay focused on and addressing equity, diversity and inclusion and our efforts and our initiatives, you're already aware that we've done several things. We initiated the annual lecture in memory of Dr. Dan Blumenthal focused on addressing structural racism. Dr. Stephanie Zaza immediate past president also published an article about addressing racial disparities, and how this begins upstream, which is where preventive medicine physicians, where we live. The ACPM also responded to NIH is request for comments on how to instructional racism and biomedical research, a very important national initiative. And of course the ACPM staff has also been given the opportunity and have taken advantage of the opportunity to complete certificates to improve EDI in the workplace. So there were also several sessions for those of you who attended the PM 2021 meeting on for the annual conference to plenary speakers, including our KBS lecture by Dr. from the AMA as well as a keynote speaker Dr. Wayne Frederick from Howard University gave outstanding presentations on these topics, but we also had several other sessions about advancing health equity and building social justice. There were also presented throughout the conference. Again, please remember if you did not get a chance to hear these sessions you still have an opportunity to do so, please contact ACPM for access to those recordings. There were also additional sessions on addressing health equity at PM 2020 as well as PM 2019. And so these are also available to you as, as you feel would be appropriate. With respect to courses and the CME activities ACPM through its population health curriculum also offers several opportunities. There are the population level strategies to reduce disparities and advance health equity. There are also sessions on social determinants of health focused on cardiovascular disease and stroke. Also the use of health information technology to drive innovation again to improve population health and sessions on implicit bias among physicians at the clinician and system levels. These are actually under development. These last two are education department with ACPM is also exploring the process of organizing other courses within the learning manager assessment system. And so please be on the lookout for updates and other opportunities in the future. And as far as grants, because that's also important that we have investment in funds that actually can move the needle on some of these efforts ACPM is also actively involved. We have a cooperative agreement with the CDC, the centers for state, tribal, local and territorial support. And we've also been an active participant in several demonstration projects with that have been done over multiple years to build the capacity of physicians and health systems to engage in quality improvement processes, again, to move the needle on some of the health inequities that are that are evident with respect to diabetes, as well as hypertension. And other initiatives where we can collect race and ethnicity data use community level data to test treat and prevent COVID-19. So, again, this also aligns with some of the priorities that I shared with you in the beginning that we want our evidence based strategies that are data driven in our, in our organization, and that represent the age, the race, the gender groups and other relevant groups. They help us understand what's happening in our communities and how we can best serve them. And you heard earlier with the from the advocacy updates that were given that there is a lot going on in this space with the college on with respect to policy and practice. So part of our legislative advocacy agenda includes advocate for measures to combat systemic racism and educate policymakers on the widespread negative impact on human health. There's also the advocacy committee working on developing a statement on health equity that's expected to be completed this summer. And there's also our science and translation committee that's developing a physician statement for the college on adverse childhood experiences are aces. That's it. And that's a summarizes at least our key initiatives and we're expecting to see further engagement in this activity. And in this important construct of who we are and who we want to be in in the coming in the coming months and coming years. I'll turn it back over to you. So any quick questions for Dr. Castlewood before we move forward with Dr. Vargas his presentation. Okay, Dr. Vargas, I'll begin sharing your slides in just a moment. Thank you, Dr. Castlewood for the update. We appreciate you attending the meeting with us today. Will you enlarge the slides. Okay, perfect. Okay, great. Hi, everyone. Thanks so much. As, as was mentioned, I'm Anita G Vargas. I'm with the New York City Department of Health and Mental Hygiene. And I wanted to thank ACPM, Anita and Liz Garland and Tisha for allowing us to put together this session today. This has been a tremendous opportunity for us to get to know each other. This has been a topic that's been near and dear to my heart for a while now. And so I'm really excited in this portion here to kind of share a little bit about the work that we started doing at the health department. Started it a few years ago and things really accelerated in the last year. And then also think through how we as a group of program directors can really work together to develop actionable strategies to improve inclusion and representation within our field and really secure the future of preventive medicine. Next slide. Okay, great. So a little bit about our efforts, you know, as part of our, our efforts to become an anti-racist organization, the leadership at the health department really challenged its programs to examine their own policies, procedures and activities to find areas of potential inequities. And so for several years within the residency program, we've really been thinking about how to improve representation and that includes within our resident, but also within our faculty as well. And there are some standard things that we've done, you know, through a lot of social networking, we interface with a lot of our local chapters of professional societies like the NMA or NHMA. But we didn't really have a structured program in place that include specific goals and methods to evaluate our work. And so in 2020, we tried to put a lot more effort into this work. And I set goals specifically on how we would achieve some of these goals and targets that we had. And then COVID happened and everything kind of got derailed. And so then fortunately, starting in November 2020, I was able to recruit four trainees who were participating in an elective in public health with us. And for their projects as part of the elective, we had them focus on building equity inclusion. And so what we did was we worked together to start with a literature review on best practices for diversity and inclusion within training programs. And the goal was really to develop a set of recommendations for medical specialties, including preventive medicine and at a local level to develop strategies for our residency programs specifically. We also wanted to create a plan to review our barriers to implementation of the selected strategies, and then finally have all this result in a definitive action plan for actually operationalizing these efforts. Next slide. And so our literature review started with just a PubMed search. We used terms such as recruitment, underrepresented in medicine, and graduate medical education. And we came up with quite a few studies, some old. But what we decided to do was focus on studies that explicitly named changes in policy for recruitment and retention. And then also documented measurable changes in pre versus post intervention. And what we found was that there was actually a very few number of studies that actually included an evaluation component. Next slide. Sorry, I think you went back one instead of forward. Okay, great. And so what we found looking at the larger literature was that there were a number of kind of common themes that we saw and they tended to fall in these sort of larger buckets. One was around recruitment efforts. Another was around admissions procedures and activities. And then the other bucket was strategies for retention. And so some examples of activities within each of these groups. So within recruitment, some of the literature suggested that actually building partnerships with minority serving institutions and also participating in pipeline programs were actually successful in recruiting a more diverse pool of applicants. In the admissions process, conducting a holistic review as part of the application process was really cited in a lot of studies as being an important feature, you know, eliminating standardized testing filters. And then also on the program side to really showcase the program's commitment to diversity, equity, and inclusion. It creates an atmosphere that, you know, potential applicants feel attracted to and want to be a part of. And then finally, on the retention side, really, we know that, you know, especially I work at a public health department, I mean, turnover can be a challenge at times. And so really sort of putting money where your mouth is and having a financial commitment to DEI efforts was also cited as something that was an actionable strategy. And then finally, having support from the National Specialty Association was another recommended strategy as well. Next slide. Okay, so now that we've done the actual literature review, we are working on putting these findings into our structured action plan and then examine our barriers to implementation. In addition, we're going to be continuing to use standardized tools to examine our policies and procedures to ensure equity in our work. At the health department, we actually use a few equity tools. One is the GARE tool, which is from the Government Alliance on Race and Equity. And then similarly, our own division within the health department created a tool that we use to help examine our activities. And so we'll continue to use those. And then as we did this work, we started wondering what other preventive medicine programs were doing around this type of work. And so I reached out to the program directors listserv and started to get some initial feedback. Next slide. You know, what we have found overall was that there was limited literature about diversity within the field of preventive medicine as a specialty specifically. And so we had an idea to create a survey for program directors on how they incorporate DEI work into their programs. And so I had reached out to the listserv, turned out Claire Huff was also really interested in this. And so we got to talking and we started brainstorming some ideas and we had tossed around the idea of conducting two surveys. One was a survey directed at our program directors. And then another was a survey directed specifically for our residents. And we'd asked both to weigh in on DEI efforts that are happening at the institution and program level. And there was a suggestion to actually compare the program director responses on certain items with the resident responses so that we can gauge differences in the residents actual experiences versus the perceptions of the leadership. So we started drafting these surveys and our plan was to complete our drafts, undergo our IRB review, we'd want to give ACPM the chance to review, and ideally we'd like to potentially deploy this as early as August. Next slide. So just a high level overview of some of the content areas that we were hoping to include in this survey. For the survey directed at program directors, we would include program information, we would include questions around recruitment efforts, DEI efforts, we'd include some items on beliefs around DEI efforts. And then finally, there was a section on the impact of COVID on these DEI efforts because I think that was something that, you know, we had seen sort of happening at the national level where I think some of our efforts around DEI were really impacted by COVID, just like other things had been. And then for the resident survey, similarly, we'd ask about program info, and then there would be questions regarding race, ethnicity, and SOGI data. We'd ask questions about current institutional and program level DEI practices, and then again, beliefs around DEI practices that are occurring at the program and institutional level. Next slide, Anita. Okay, so, you know, what we wanted to do was really make this a tool that is useful for us as a group of program directors to really, you know, strengthen our specialty and increase recruitment into our specialty. And so along those lines, you know, what we aim to do is really get your feedback so that we can make this survey not just an academic exercise, but one that's actually a useful tool. And so with that, I wanted to query this group to get your feedback on what general topics you would like to see included in each of these surveys, and when you think the survey should, you know, ideally be employed. I know the summer is kind of a busy time for a variety of reasons. Perhaps the fall is better. Wanted to get a sense from this group as to when you would think we would get maximum participation from both of the program leadership side, but also from the resident side as well. And so I think we were going to do a poll, but I think, Anita, we're not going to do that, right? No, we're not, but I just thought that we can still give an opportunity for the audience to share, you know, their thoughts on the other topics that they would like to see in the survey content area for both program directors and residents. And so I was just going to post two different questions and give some time for the audience to post the answers. Does that work, Dr. Jawarghese? Yeah, absolutely. That's great. One second, please. So for context, these are some of the content areas that the program director survey will include. Any other content areas that you think that the program director survey should include, please include in the chat. Thank you. Great, and while we're taking comments in the chat, if people have questions, like this to be an interactive sort of discussion. So if people have questions or thoughts on this work, I know there are probably people that have been doing this a lot longer than I have that have probably tried to engage in this work in the past. I'd love to hear about your experiences with that effort. Dr. Dory Lane, you have your hand raised. Please go ahead. Yes, speaking of people who have been involved in this for a long time, it seems appropriate for me to comment. I was just not specifically answering that question you had, but I was just reflecting that we really need a bigger pipeline full of minority candidates. I mean, I like to hear your approach of connecting with institutions that historically are minority institutions, because many of the candidates that we rank just don't, I guess, many programs also want those candidates, so there really aren't enough of them around. Just a thought. Yeah, absolutely. And to be clear, we haven't actually implemented a lot of these strategies yet. I think I'd love to be able to partner with different organizations or different schools and institutions that really have a diverse pool of trainees, but that is something that will be on our action plan and something that we'll be working on. Thank you. I think that people are... Yeah, people are sharing on the chat. I mean, it sounds like, you know, what you guys did with at the DOH is really successful and it seems like you've got a great group of residents this year, too, I may be interested in so can you do that again for next year was that sort of a one time thing because of the program that was happening at the time or. So other other programs on the call that have found specific successful strategies. Yeah, I mean, I think with our programs, I think it was just luck. I mean, you know, we do some of these efforts, but, you know, because our candidate pool is so small, it's just kind of luck who we end up sort of matching with who's a good fit for our program. You know, we're a health department, we're not an academic program. So it takes people with very specific interests, you know, to want to enroll. So it's Yeah, it's definitely a challenge. And so I feel fortunate we do have a great diverse group of candidates and we have for the last couple of years. But, you know, I want to create a program where we can sustain that. And that's the challenge. Dr. Javarghese, do you want to read out loud the topics that were entered in the chat? Sure. Okay, so let me just scroll here. Okay, so we have definitions of what it means to be underrepresented, such as Asian, Hispanic, international medical graduates. There's interest in retention of diverse candidates after graduation to build faculty, which I agree is a really important topic. Another suggestion was what efforts are made to recruit from outside the U.S.? What barriers have been reduced to recruit international graduates? How to develop pipeline programs, which is a great question. Are other underrepresented groups considered, such as those from areas with socioeconomic or geographical barriers to care and public health services, since such positions are more likely to be interested in returning to practice in those underserved areas? And then another comment was, I think one way to recruit is to have an interest group at your affiliated medical school. That's a great point, too. So can I ask for some clarifications here? I know, maybe, Pat, your comment about definitions of what it means to be underrepresented. Was that a topic to include in the survey or just like something that we include as part of the definitions we include in the survey introduction? Well, I think it's always kind of a moving target. So I think either, you know, if you're assessing sort of collecting data, be clear on the definitions because it's not always clear. So, you know, sometimes I can have in some reports, I would say 40 percent of our residents are minority. But then, you know, I use other guidelines within our institution. And, you know, a Hispanic from Venezuela might not count as a underrepresented minority at our institution. So is there national standards or institution specific standards for defining what is an underrepresented minority versus, you know, diversity, you know, based on country of origin or other factors? Yeah, it was another question, you know, we were sort of brainstorming through is sort of, you know, what we want to gain out of this survey. You know, we want to make it broad enough so that we get information on, you know, sort of the representation and diversity in terms of race and ethnicity. But then also, do we want to collect information like the SOGI information I was alluding to, you know, so that we're ensuring we're including all sorts of, you know, gender identity and all sorts of other groups, you know, into this so that we're, you know, investigating this. Yeah, I agree. I think both are important, but particularly, I think the underrepresented racial minorities are an important group to, you know, keep focused. Dr. Morgenlunder, please go ahead. Thank you so much for this work that you're doing. And it's so great to see you approaching it the way that we approach all things, which is what does the literature say? What do we know? What do we need to find out? My thought of what something that I would be interested in in the survey is really to know if the program, if the residents and the program directors feel that the programs or the institutions allow room for facilitated or grounded conversations where people can express varying viewpoints. Because I think one of the challenges, I'm curious, as you're thinking about this, in our programs, because they're small and our numbers are small, you know, obviously structural racism is huge and has been going on for a long, long time. So these are big issues. And the fact that our attention is focused is really necessary. And also, we're not going to change it overnight. But when we talk about measuring metrics, like, it seems a little too simplified to say, oh, let's just look at our numbers and see how many underrepresented minority candidates we have in our program. I mean, in this moment, you know, our program has actually luckily been diverse over the years. But I think, I guess my question and my caution is, what do we want to measure as successful outcomes? Because in a program where you have two or three residents a year, if you have two people who are an underrepresented minority, and then the next year, you don't have any, your numbers get very skewed very quickly. And so I'm just wondering if there are any questions we can ask to kind of gather more whether an environment is conducive to inclusivity versus just counting numbers and saying, okay, you have this many of this kind of person and that makes you compliant. Yeah, that's a great thought. And I wonder if, you know, the residents survey would help with that, right? Because that's where we compare the residents to the answers might be a good way to gauge that. Okay, there's a few more comments here. Okay, we should ask if their assessments perform for the presence of implicit bias amongst faculty and residents. That's a great point. And perhaps the assessments of diversity inclusion should be multi year rather than each year individually. Okay, great. So overall, it sounds like there is interest in this topic. And so I'm hoping when we deploy this survey, we will get full participation from everyone. And maybe I'll go back to the timeline question. I know there were a few comments in the chat about that. My thought was maybe to aim for September so that like sort of the summer period is past people have done all the July like new onboarding and orientation stuff and then maybe aim for September. I think there was a comment here. I can't find it right now. But I think it said to avoid Oh, here we go. Not in January, February, March, when the ACGME surveys and internal surveys come out. So I think that's, yeah, that makes perfect sense. Okay, great. All right. Well, thank you, everyone. And you know, I just want to acknowledge that obviously, I did not do this work alone. I had a whole team of students and residents who were working with me. So I just want to acknowledge them. Robert Rock, Jessica Batia, Susan Shanti Dara, and Adetayo Fawole. And then I really want to thank Clarissa for her help in kind of making, you know, my my sort of thoughts come to fruition here in this session, and obviously the ACPM as well. So thank you. Thank you. Are there any questions or hands raised? Dr. Hoff, I just want to check with you to see that I can continue sharing the slides and you'll tell me when to stop when you're offering the Slido polling, correct? Oh, let me do it from my side, because I got to do the whole thing with Slido. Okay, no problem. So I've stopped sharing if you want to go ahead and share your slides. All right. All right. Hello, everyone. Let me pull this up. I don't know if you all are familiar with Slido or not. But if you will whip out your smartphones. We're going to do a little bit of polling. This is going to be kind of an interactive session. And I've titled it ACPM DEI plus GME, a lot of acronyms. But basically, we're looking at where we are and what should the future look like when it comes to diversity, equity and inclusion. So if you go to Slido.com, you can just go on your smartphone, type in Slido.com, and then you type in this code right here. It should take you to our presentation. And when I get to the actual Slido or this actual Slido slides where you interact, it will come up. But let me just do my introduction slides here. So the goal of this kind of workshop will be over the next hour to identify about five potential areas and diversity, equity, inclusion. They'll have a high impact on preventative medicine, GME education, or graduate medical education. And we're going to get all that information together, summarize it in an easily accessible document for internal use. And then from that document, we're going to hopefully create kind of actionable next steps, whether it's creating a DEI task force, whether it's turning that document over to the GME committee within the ACPM and letting them work from that. We're going to try to keep it focused on graduate medical education, because I know specifically that, I'm going to keep the chat, sorry, I'm trying to open up the chat here. I can't see it very well. We're going to try and keep it specifically focused on like, you know, preventative medicine in the graduate medical education space, because I know American College of Preventative Medicine is doing other things outside of that as well. So, but let you know, so that, and we're all preventative medicine residency directors or work within that space as well. So we're going to do about 10 minutes of like the Slido to kind of identify our top five topics, then we'll spend one, two minutes breaking out into our breakout groups, then 25 minutes discussing our topics as a group, five minutes to kind of wrap up. You're going to pick a presenter, go back to the main room, and then we'll present, you'll have your person from each of the groups kind of present a summary of what you discussed in your breakout groups. I'll be kind of in the main room or jumping in and out of each of the breakout groups to kind of help you discuss it. And you have, please designate a secretary in each of your groups to kind of write up a summary of what you discussed in your breakout groups and send it to Anita Balan. We will also record all the data we captured through the Slido. It's just like three questions, but I'll just save it all and send it to Anita as well so that we can kind of summarize this in our kind of finalized internal document that will allow all of you to see. If this is the questions you'll be discussing in your breakout group, what are you currently doing in your program to improve diversity, equity, and inclusion? What are you currently doing to improve diversity, equity, and inclusion in your institution, and is there broader support from your institution for DEI efforts that you can leverage? And then looking at your specific diversity, equity, inclusion topic your group was assigned, do a SWOT analysis. What are the strengths, weaknesses, challenges, opportunities, and threats as it relates to the preventative medicine GME space? And then for your topic that you were assigned, what are some reasonable short-term goals of deliverables within the next year for your GME programs or GME programs in general, and what about for the American College of Preventative Medicine as it relates to GME? And for your topic, what are some reasonable medium-term goals about three years out for GME programs and for the American College of Preventative Medicine as it relates to the GME? So here's our first Slido question. Again, go to slido.com, enter in that number. Now everyone just start typing in. This is a word cloud. If a word gets repeated more than once, it will start to get bigger and bigger. But when you think of diversity, equity, and inclusion in the preventative medicine GME space, what comes to mind? yeah Good, you can add as many responses as you want. Good, good, it looks like opportunity is the one that just keeps kind of popping up the most. It is an opportunity in a lot of ways. Recruitment, training, those are another two words that keep popping up a lot. Yeah, I mean, you probably all know how word clouds work, but if a word keeps popping up more and more, that word keeps getting bigger and bigger and bigger. So, what does FMG stand for? Foreign Medical Graduates. Ah, Foreign Medical Graduates, okay. I was thinking family medicine for some reason. Like, I don't know why. There we go. Can we move on to the next question, Dr. Koff? We wanna make sure we have enough time for discussion. Okay, I'm sorry. All right, so in starting to develop our list of the five topics that we wanna go over, as Preventative Medicine Residency Program Directors, what areas of DEI do we need to focus on most? This is not a word cloud, this is literally just a list of topics that we need to put in. Thank you. I'm going to do something here real quick. I'm going to go to this next question. Sorry. I'm going to add in Oops. Kind of pre-populated it with some possible You kind of added in some more topics, but Do you feel like pipeline and med school kind of fall under the same topic? Do people agree with that? Or is that different? I think it goes further back. Wendy Lane. I think pipeline, from my perspective, was even before med school. Yeah. Yeah. Mine too, although there's some overlap. Mine was in med students, but I meant pipeline. Yeah. I was like pipeline from the beginning. Okay. All right. I'm just trying to make sure I have all my ideas, because the next one is literally, I have a bunch of ideas voted that we're going to rank. was adding in the extra ideas that you guys had going. All right, so there's a whole list. Keep scrolling through. I think I got in most of your ideas. But vote on the top five. Is the first one you picked number one or are they unranked? They're unranked. It's just the top five and the five that get the most votes are going to be the five topics that we talk about. We can't see all of them. You should be able to scroll. Are you all voting? So, Dr. Hoff, just for clarification, I think one of the things that I saw in the previous slide most commonly voted, the shared, was recruiting and retaining a diverse group of residents. And I think that we can, I don't know if that's the same as increasing diversity in PM resident population. Yeah, I was kind of hoping that fell under that. I don't know how many people are able to get on Slido. We have about 16 people that have voted before. 17. I don't know if this is going to affect, but we want to, my understanding is that we're seeking your feedback before we create the breakout groups. And so the breakout groups will be based on the top five topics. Yeah. So I just wanted to make sure that we were creating breakout groups based on the feedback we're coming, getting from the program director. So yeah. And slido.com is the website and the code number is listed in the chat box. Yes. 7 4 8 0 6 7. It's here on the side. 7 4 8 0 7 6. Yeah. Do we want to move forward, or just give one more moment? How many people responded? 17. 17? We had about 50 people on there. We're 40 people, so. Only 20 people are on Slido though. Oh, okay. Does anyone else want to put stuff in the chat if they can't get on Slido? Just in case. Okay, so it looks like pipeline, mentorship, recruiting, retaining diverse faculty, recruiting, retaining diverse residents, and supporting undergraduate medical... URM residents? Underrepresented minority. Underrepresented minority, sorry. And supporting IRM residents also are the top five topics. And Dr. Thomas, just to respond to your question, we go until 5.15. So we break at 4.30 for 15 minutes. And if not, we can go directly into the next session. The last session is milestones. So we'll have about half hour of breakout group. It'll be a little bit compressed. We'll break out into groups and maybe discuss for 15, 20 minutes, and then report back for about five minutes, and then move on to milestones. All right. So, all right, all right, all right. Okay. So we'll divide out into our five. It looks like five and six are tied. And we might be able to compress four and five into one group, supporting underrepresented minorities and increasing diversity. Yeah. Those are kind of... So we will do one group is pipeline. Second group, the second breakout group is mentorship. Third group, breakout group is recruiting diversity. Fourth breakout group is recruiting and supporting our underrepresented minority and diversity in our resident population. And our fifth breakout group is developing a DEI health disparities curriculum. Okay, great. Thank you. Can you stop sharing, please? I want to make sure that I take charge of them. Yeah. Unless there are additional slides you need to share. Nope. Okay. I'm hoping this is going to work. So please bear with me. Thank you. And the main faculty for this one, I'll be automatically breaking you out and then moving you as needed. So please, please. Yeah. I can copy the questions real quick. These are the questions. Thank you. Hi, Anita, you know, Claire and I ended up in the same break. I just moved. I just moved her to a different group. Oh, okay. So then. Can you move me instead? Or did you, uh, where did the DI disparities research? Okay. You haven't joined. So I think it's still assigned to you. Perhaps you can join. Uh, do you know how I would do that? Because I left the room at it, like before there was a pop-up that allowed me to hit join. Uh, okay. So I don't know. Okay. Yeah. Sorry. I'm just trying to figure it out. Okay, I'm just going to have to move you to one room and then move you back. So let me just. Yeah, I'm happy to go to any of the rooms. So yeah, I just didn't want to be the same. Okay, I'm just going to, yeah, I'm going to move you back to. All right, I'm going to pipeline. Yeah, and I'll move you there and then I'll move you back to the disparities because for some reason it's not even, and I want you to, there's a lot of people in pipeline and mentorship and not enough in disparities research. So. Thank you. Okay, Dr. Haff, over to you. I'll type the five groups in the chat box and perhaps I'm turning it over to you and then we can go one by one. Okay. Recruiting, retaining faculty. Actually, I got randomly assigned to that group, so I can just go ahead and present on it. I mean, as far as like DEI efforts in the institutions, there's quite a bit of efforts. Like Loma Linda is doing blinded applications for all their recruitment, except for obviously when you interview them, because you have to see their face. And other institutions are doing climate surveys. I know Tulane is doing that right now. Meharry is actually a minority serving institution, so that's kind of written in their DNA. But that has very much been on people's minds, so it's kind of something that they can easily take advantage of at this point in time. But as far as our SWOT analysis, one of our strengths is really need to get a better handle of our numbers, but preventative medicine tends to be a very diverse field. We, as public health professionals, have been doing health disparities before health disparities was a thing. And studying diversity, equity, inclusion before it kind of was cool. And I think we really need to up that and like play that, like show that as something that we really have been, like as a field, this is kind of part of who we are and what we do. To help recruit, like to help recruit diverse faculty. And it's part of what makes us unique as a field, so that's, we need to really showcase that and really advertise it really well. A weakness is we're just a small, relatively unknown field. An opportunity is we've just had a major pandemic that we need to take advantage of, like the funding and the fact that preventative medicine and public health have clearly taken a front stage and we need to showcase the fact that we are the field that can handle this. A threat, of course, is always consistent funding to fund all these situations, which hopefully we can get more from Congress on that. But as far as one year goal is one, we just really need to up our visibility and how much we do already do in these areas. And two, kind of like get current stats of diversity in our faculty so we know where we stand and we know where we need to go. And then kind of a three-year goal is like really get mentorship going for underrepresented minorities faculty because they really need, especially younger faculty, so they can really develop their career in preventative medicine. And really find a way to maintain our diversity, equity, inclusion efforts so we can keep it going once we decide what we want to do. But especially mentorship of our underrepresented minorities because oftentimes the reason they fall off is because they don't really have someone there who represents them to kind of help show them the way through. Thank you, Dr. Huff. And we're at 425, so we're going to try and assign about two minutes for the remaining four groups to kind of wrap up. So the next group is retaining and supporting residents. Who is the representative of that? If you could wrap it up, if we can wrap up your findings in about two minutes, please. Thank you. I'll take that group. We noticed that we did already have a good deal of strength within our own institutions. Many has come from institutions that already have DEI resources, so we certainly should be encouraged to use those rather than reinforce or try to recreate it on our own. But of course, it's really a long-term process of supporting our underrepresented students because they usually come with a lot of you know, they've been doing this struggle for quite a long time. It's going to be a another long time struggle, as someone called it, a marathon and not a sprint. But at the same, it's particularly about keeping the retainment of underrepresented minorities is particularly about keeping the communication lines open and allowing them to kind of say their piece and work at their own speed for their own needs rather than try to drive that. So it's particularly about mentoring and communicating. Okay, thank you. Next group is DEI Health Disparities Research. I don't know if anybody from our group was going to talk, but if not, I'm happy to kind of give the report out. So I'll just jump right into the SWOT portion. You know, we talked about our strengths as a specialty. You know, we have the ability to look at health disparities in a systematic fashion. Our peers may not think of, you know, addressing some of the inequities we see in such a systematic, from a systematic perspective like we do in preventive medicine. There are a lot of, you know, activities that are happening within our institutions now. Look at the research that our residents are getting involved with. You know, we have a big equity focus. And so those are all strengths. In terms of our weaknesses, there are some challenges, you know, not all institutions are as successful in embedding residents in the health disparities research that's going on. It's different when you have, you know, a structural, like, let's say, a health department rotation where you get involved with case investigation and outbreak response. It's a little different for health disparities work. And so thinking of opportunities to more successfully engage those partnerships can be helpful. In terms of opportunities, I think COVID-19 has highlighted the need for further work in achieving equity and reducing disparities. And I think, you know, just other topics that are potential areas of focus are environmental health, the need to address climate change. The NIH is putting a lot more effort and funding towards health disparities research. So that's a potential opportunity for us in preventive medicine. And then in terms of threats, just, you know, the idea that environmental health is being taken out of the focus, even in our MPH curriculum, and even, you know, potentially within the boards. And environmental health is just really core to our specialty, but also to the populations that we serve. And so it is a big issue in, you know, health equity. And so that's a potential threat there. And one idea we had as a potential goal was actually to measure this work. And so one sort of achievable thing we thought was possible was actually just to quantify and evaluate how many, you know, research submissions, poster submissions we're receiving for the annual conference each year around work in health disparities, and really track that and measure that and use that as a metric to help us gauge how well we're doing this work and maintaining this effort. And I think that's it. Thank you, Dr. Givarghese. I think that we have mentorship. Next group is mentorship. Yeah, so I can speak to mentorship. So strengths, people just mentioned various strengths from their own programs, and those included mentorship by minority faculty, academic supports of paying for tutoring services for residents that needed it, community-based mentors by having a community engagement requirement, efforts to retain underrepresented minority faculty, encouraging residents to participate in institutional DEI committees, having a well-being committee, and building a community because not one person can mentor on all topics. Some of the weaknesses, small size of programs, so there may not be that many mentors, institutional buy-in, and a small pipeline of candidates. We saw ACPM as a potential opportunity as a central resource. Maybe ACPM can help coordinate faculty mentorship. And another opportunity might be DEI committees at the departmental and institutional level. And typical threats, funding, time, and potentially backlash to DEI initiatives for goals looking at what other programs are doing and survey the residents and faculty in our own programs about DEI. Thank you, Dr. Lane. And I think, and finally, we have pipeline. Any representative from the pipeline group to report findings? I can try unless someone else wants to. Okay, so I'm going to make up the SWOT. We did a lot of discussion, and we have a lot of different issues going on at the five institutions that were discussed. So, strengths, it sounds like a couple of institutions have pipeline work. Emory sounds very strong with work in diversity at various levels. Rutgers has programs for high school students. This hasn't been measured as far as how many of them end up in medicine or even medicine, let alone preventive medicine. So, for science, we don't know. But these are minority high school students in an impoverished minority city. Weaknesses, it seems like for some institutions, it's geographic. So, being distant from major cities or it just seems to be a barrier for some in recruiting in general and then among the general recruitment minorities. And then, SWOT opportunities, looking at it, I think other people have spoken of the opportunities. Threats, just such a small, every program is so small. What can a tiny program do? And disparate funding is an issue. Jim or Amy, can you add to that? I think one of the common themes was that everyone's institutions either have been looking at these pipeline issues for a while or if they hadn't been, they started looking at them more closely in the past year or two. But everyone's challenges were different, whether it was some had geographic challenges, some had institutional challenges, et cetera. I think one of the things, though, is that for most of the people within our group, I think the preventive medicine programs perhaps did better in these pipeline issues than their larger institutions have. I'm not sure why that is. I'm not sure why that is. And then the other strength that was discussed was one that other people have brought up, which is that preventive medicine is where you can work on diversity and equity issues. And we need to maybe display that more, preventive medicine, public health. And that they're not new, looking at health disparities and whatnot, it's fashionable. It's what the cool kids table is talking about in the cafeteria now, but it's stuff that we've been talking about for years. Okay. So Dr. Jowargis and Dr. Hoff, I think that is it, unless anyone else has any other thoughts to add or questions to ask Dr. Jowargis or Dr. Hoff. No, I would just ask that each group kind of write up a little summary of what their group did and email it to Anita and ideally myself here, I'll put my email in the... So I will kind of between that and the data, the whatchamacallit, the Slido data, kind of create and write up an internal document that we can kind of utilize as like a jumping off point for what we want to do next. So thank you all for participating. I really appreciate it. And thank you for moving us along, Anita. I know we are slow. No, that's not a problem. Thank you, everyone. So I originally wanted to suggest to move forward with the milestone session and wrap up the day, but unfortunately, Dr. Edgar is having internet issues and she needs to call in and she hasn't called in yet. She has internet. That might be her. Yeah, I think that's her, Anita. Oh, is that you, Dr. Edgar? It is. Okay, your timing is impeccable. I think sitting is the new smoking, but at the risk of moving forward, I think we can, if it's okay with you all, we'll move straight into milestones and take the questions and wrap up the workshop. Is that okay? I think that sounds good, Anita. And I'm seeing some heads nodding. Yeah, fantastic. Go ahead, Dr. Johnson, for introduction. Well, thank you, Dr. Gepardese and Dr. Hoth for the discussion and leading the efforts on this for us. I appreciate it. So the last session of the workshop today, Dr. Edgar is the Vice President for Milestone Development. She previously served as the Executive Director for Milestone Development and the Review Committees for Medical Genetics and Genomics, Pathology, and Radiation Oncology. In her current role, she leads volunteers and staff members through the development of milestones and related materials. She has been influential in the development of milestones for more than 100 specialties. Welcome. We look forward to your discussion today. Thank you so much. And again, apologies. And of course, because I have to do this on my phone, now you're going to be able to hear my dog bark as well. So really, my purpose... My apologies. I'm still trying to pull up the slides. Just give me one moment. It'll just take a moment for me. I appreciate your patience. Bye-bye. Okay, we're ready. Dr. Edgar, go right ahead. Awesome. So again, thank you. I'm really, today I'm here to provide an update for everybody on sort of where we are in the process for 2.0. And of course, like always, I like to give some reminders of background, especially for any of our newer program directors. Next slide. I have no disclosures. Next slide. Quick review of the milestone levels themselves. You know, one of the things that happens is over time, we start to forget about what it is that the milestone levels actually mean. Now, we use the Dreyfus developmental model for our milestone. And when we think about this, we like to give an example using clinical reasoning, just because it seems to appeal to the most, to the broadest audience. When we think of level one, remember, we're talking about the novice. So this is somebody who is very rule driven, right? You have to tell them how to do everything and when to do everything. Level two is the advanced beginner. Now, the advanced beginner is somebody who's starting to do some things on their own. Quite often, you may have residents that start at the advanced beginner level, and that's not necessarily uncommon. Level three, this is where a competent learner is. So this is our learner who is now able to manage all of those complex presentations. They're able to manage those complex projects that you have them working on. Level four is our proficient learner. And remember, this is our target for graduation. It's not a requirement for graduation, but it is how we write the milestones. That's really where we're targeting. And this is now your learner who's able to do all of those things for those more complex projects, those more complex problems that they're going to be facing when they go out into practice. And then, of course, level five, the expert level. Now, at this particular level, what's important here to remember is that we're talking about the expert learner. We're not talking about the expert practitioner who's four or five years into practice. We're talking about, in this case, the expert resident. So one out of every eight, one out of every 10 residents, hopefully, are going to be able to achieve it. Next slide. It's important to remember that the levels in the PGY do not go together. Not as hard for you to remember at your specialty, but for some specialties, this becomes a real issue for some of them. Next slide. This is another way to think about the milestone levels. So not only using that advanced beginner and competent language, but also thinking about the level of autonomy that some of your learners may be using. And again, remembering that that level four is not the requirement, but a target, and the level five exceeds their peers. Next slide. One of my favorite slides to present with every group is reminding your faculty, especially, that in this case with the milestones, we are always looking at the narrative. We're not paying attention to the numbers that much. It's more important whether or not they're completing the task or demonstrating the knowledge that's under each level than it is that they're getting a particular number. Whether, you know, if you're using a resident management system that sort of takes all of your assessment tools and brings them together, it's going to spit out a number at you. We don't really care too much about that number. We want to make sure we're getting that narrative right. Next slide. So then the other piece, too, that we always have to remember is that really the milestones are not designed to be an assessment tool in and of themselves. They're really designed to be a formative assessment tool that uses all of your other assessment tools in order to be evaluated. This is really important because not every milestone can or should be evaluated, and not everything that should be evaluated is included as a milestone. So when you use the milestones themselves as the assessment tool, then you sort of lose a lot of that extra information that you're going to need. Next slide. So here's our timeline for Milestones 2.0. We hope to have it available for public comment by the end of the summer or early fall. When we send it out, we will send it directly to program directors and program coordinators, and it will also be listed in our e-com. And of course, that goes out the Tuesday of every week. And we'll also ask, of course, the program director group to share it as well. We hope to have final publication by January with implementation for the next academic year in July of 2022. Next slide. So I want to give you a little preview of where we are right now. Now, keep in mind, of course, this can change at any time. But as of right now, these are the new patient care and medical knowledge subcompetencies. So right now, there's nine down from where you were before, which was 15. So there has been a decrease. Now, some of you may remember when we talked two years ago, hard to believe it's been two years since we were all together. And I'm trying to remember, I don't remember where we were even. But working on some ideas for how to move forward with the new milestones. And I talked to you also about how it reformatted the milestones. So the new milestones now only have no more than three rows. We're using simple language all the way across. And I think that overall, you're going to see a great improvement in the way that these milestones are written. So the edits that you all provided for those harmonized milestones, those were all given to the group. And they, in fact, use those to continue making even more edits to those. So thanks to all of you for providing all of that extra work. Next slide. Now, this is where we start, I think, to get super exciting. We have a new companion tool that we're doing with all of our milestones this time. We call it the Supplemental Guide. And it's really, it's designed to give you more information about what the group was thinking when they created the document. Next slide. There are five parts to this, to the Supplemental Guide. And I think that over time, this would be another activity that you as a program director group could even sit down and think about how to do this together to expand this document even more. But to start with, we provide you with an overall intent of the subcompetency. We provide examples for everything. For every individual milestone, there's an example. We give you an idea of assessment tools. We provide a space for curriculum mapping. I'm going to talk about that a little bit more in a minute. And then we also have an area for notes and resources. Now, with each of these things, we also provide this document as a Word version. And so, that's what kind of makes this really exciting. So, next slide, please. So, this gives you an idea of what the Supplemental Guide looks like sort of as a complete part for a particular subcompetency. You'll notice, and I know it's hard to read, but next slide, you can see it a little clearer. We have the milestones listed on the left with the examples on the right. So, if you're using the Supplemental Guide, you're not going to have to flip back and forth between the two documents. You'll have all of that information right there. Next slide. So, the Supplemental Guide is also being provided to you as a Word document with the intent that you will be able to use this document with your Clinical Competency Committee or your faculty, your residents, you know, really it can be any group, to create that shared mental model about what these milestones mean in your program. Now, why is this important? Well, because every one of your programs has a slightly different focus, right? So, what you expect at various levels might look different. I anticipate the level four is going to look the same for everybody. However, how your examples for how you get there are going to change. This is also going to be impacted even for some of those level five, for some of your programs that maybe you specialize in one of those particular areas, you may expect all of your residents to get to a level five with the example the way that it's written. So, that'll be something that you can adapt and change as you're going through it. The other thing you can do is you can actually list which assessment tools you're using specifically. So, whereas we list things in a document like direct observation, multi-source feedback, project submission, things like that, you'll be able to list your exact tools. So, you'll know exactly what it is that you want to be reviewing when it comes time to think about the milestones. In terms of curriculum mapping, that's not something we provide. But again, we're providing the space for you. So, that way you can decide if you're going to use this document, you can actually put in there which rotations or which projects you're going to be expecting to evaluate those learners on. Next slide. The last piece that I really want to focus on when it comes to the milestones, before I switch gears a little bit again, is really thinking about getting ready for the new milestones. You know, when they start coming back together and you start thinking about how you're going to implement them, remember to think about the resources that you have. Thinking about the engagement and attitudes of your faculty and how you're going to work with them to be able to do this implementation. And remembering that over time, it's going to get easier. It's always that learning curve at the beginning when you're trying to do that implementation. Now, that being said, the good news is we have lots and lots of resources available for you. Next slide. And I want to spend the next couple of minutes going through some of those resources just to give you an idea what's available. I don't think everybody even is always aware of how many different things we have. Next slide. This first one isn't available yet, but it will be coming. If you're interested, there is already one for occupational medicine and the aerospace medicine. Both will soon be up. They'll actually be up this week, I think. Maybe next. They're basically just our webcast. Great faculty development opportunity. They're less than 15 minutes long, and they explain the ins and outs of the new specialty specific milestones. So, a little bit about the construct and the meaning, a little bit about the content changes that are in place, and then also how to use the actual supplemental guide. Next slide. This is just to give you an idea of where to find the milestones, the supplemental guide, and that webcast. Those are all on your specialty specific page on the ACGME website, and if you click on the milestones button on the side, that's how you'll get to this. Next slide. Now, this is getting more towards all the different resources that we have to offer. We have a lot of different guidebooks available, and keep in mind that all of these things that we're providing are available for free. These downloads include a milestones guidebook, an assessment guidebook, which is a great new resource we just developed last year that walks through different types of assessment tools and helps you know when it's appropriate to use each one. So, you know, again, it's a great way, especially if you've got some new junior faculty coming in who haven't done a lot with assessment, it's a good way to help catch them up. We have an implementation guidebook for milestones. So, that's a great document that can help you to better frame out and prepare for the implementation and also help you with the actual implementation and the post-implementation work. The popular clinical competency committee guidebook, that particular guidebook is one of the most downloaded documents on the ACGME website. It's so popular that people kept asking us if we could find a way to make it shorter because they didn't think their faculty would read this 95-page document all about clinical competency committees. So, what we did was we created an executive summary. So, there are nine executive summaries available, one for each of the major parts of that clinical competency guidebook. They're each between one and two pages long. So, again, great resource for some faculty development. Next slide. Another set of resources we have are for residents and fellows. This is on the residents page where you can find all three of these documents. There's a milestones guidebook for residents and fellows that really focuses on what they need to know. What do they need to know about assessment? What do they need to know about milestones? There is a PowerPoint, which is a great way to introduce them to the idea of what the milestones are. And then, finally, there's a milestones 2.0 guide sheet. It's a one-page, regular-sized paper, something you can post on a bulletin board. All three of these documents were created by residents and fellows. And so, again, we think that it's really valuable, and they really put their heart and soul into this, thinking about what it is that other residents and fellows need. Next slide. If you have not seen this yet, this is a supplemental edition of the JGME, which hopefully everybody realizes is open access. And this particular supplement came out in April. It is all about milestones, assessment, clinical competency committees, and really focusing closely on all of those things. It was all peer-reviewed. And, again, it's a type of issue that if you have a paper copy of it, you'll probably be flipping pages and have page marks all over it, because there's just so many valuable resources in there. There's even several articles about things like program evaluation and, you know, using your milestones to help with that. So, again, would encourage you to take a peek at that one. Next slide. There are also several – Learn It ACGME has several opportunities and different courses. We also have live educational opportunities specifically for assessment. Next slide. We have two free assessment tools available for you. Even if you think you have a great assessment tool, I would encourage you to just take a look at these to see if there's a way that you can improve your direct observation or your multi-source feedback tool. Next slide. Every year in the fall, we put out a national report. So, this can show you the national data for your specialty, you know, so you can just compare. That's all it is. It's a way to compare. No expectations that they're going to be the same, because every program looks different. You all have different rotations. But, again, it just gives you an idea of where they are. Next slide. The last resource to share with you is really – we have a milestones bibliography. Now, this bibliography is covered – or it includes either an annotated bibliography or it includes the abstract from the original article. Everything is covered in there. These articles that are good, bad, articles that really don't like the milestones – those are all included in there. So, there's a little bit for everybody. Next slide. And as I tell you every time, we are here to help you. And I know that I've talked with many of you over the years, and we really want to encourage you to reach out to us with any questions, whether it's about milestones or assessment. If me or my team don't know the answer, we will find somebody who does. We really are dedicated to that, and we take a lot of time to help all of you as much as we can. Next slide. Are there any questions? It feels strange just talking into my phone, knowing you're all virtually listening to this somewhere else. So, there were a few questions in the chat box, Dr. Edgar, and thank you for your presentation. You actually answered some of them as you were going through your presentation, but someone asked if the July 22nd implementation date was for all specialties, and that was answered in the chat. No, that varies according to specialty. And there was an ask of when will the supplemental guide be available for preventive medicine? So, that will come out at the same time as for the public comment of the milestones. So, again, I expect that to be late summer, early fall. Thank you. And then there was a question about where are these resources, and you answered that as well in the next slide on the specialty page. So, you answered the questions as they were coming up through the chat, but does anyone else have any other questions? I don't see any hands up. And just so everybody knows, I did put the link for all of those different resource pages at the bottom of the slides. So, when you get a copy of the slides, you'll be able to just copy and paste those. Over to you, Dr. Johnson. All right. Well, thank you, Dr. Edgar for your presentation. A couple of quick reminders before we adjourn today. ACPM will share the recording and the slides from this workshop on the PMR listserv. And service exam registration will be announced on Monday, June 7th on the PMR listserv. Programs can register their residents by Friday, July 16th, 2021. And finally, I want to thank all the speakers for their presentations, attendees for your enthusiastic participation today, and all of the thoughtful questions that we had in discussion. Thank you, everyone, and I hope you all have a wonderful weekend. And I hope to see everyone in person in June of 2022 at the annual conference. So that will conclude our meeting today. Stop recording.

diversity

inclusion

training programs

preventive medicine

recommendations

underrepresented groups

bias training

safe spaces

resources

support

community organizations

curriculum

data analysis

equitable workforce

public health