Are Pandemic-Related Changes Negatively Affecting the Quality of Credentials?
The word “credential” is often used as an umbrella term to describe certificates, diplomas, certifications, licenses, and various other credentials (see Figure 1).
For more than 50 years, the credentialing industry has operated under accepted standards for the development and ongoing maintenance of credentials to help ensure the validity and reliability of the examinations and the programs they represent. In fact, most of us grew up in an educational and credentialing system that placed value on high-stakes, standardized testing to verify our knowledge. For example, Figure 2 illustrates the steps in developing a certification program.
Generation Z and Centennials are some of the first generations to be born into (and grow up with) a fully-technological environment that includes instant access to information at any time, immediate feedback for almost any activity, and a technology-enabled, learner-centered world. These generations are used to the ability to access information (including credential examinations) at any time from almost any location-the days of going to the library to look up information in a paper encyclopedia or go to a test center on a specific, pre-determined day and time are seemingly over.
During the last two years in particular, many changes and sometimes shortcuts have been made in the credentialing industry, specifically in certification and licensure, in response to the 2020–2022 COVID-19 international pandemic. Changes that the industry would never have accepted and adopted previously, such as suspension of eligibility criteria or elimination of competence verification, are now becoming more and more commonplace and are fueled by the current generations and new certificants’ acceptance of these changes as signs of “progress” or improvements many have come to expect.
The question for the credentialing industry is: are the changes that have been rapidly ramped up in the last two years due to the pandemic, and as a sustainability response to the emergency the pandemic created, actually an improvement? Or are they actually affecting the quality of credentials?
Changes to the Credentialing Industry During the COVID-19 Pandemic
In order to answer these questions, one must understand what the changes are and the reasons why these changes have been made. For instance, recent changes in obtaining a credential include changes to the eligibility process, and in many cases, this involves submitting evidence that the candidate meets the eligibility or prerequisite criteria to register to take the examination(s).
During the COVID-19 pandemic, many offices were closed, which caused workers to conduct their jobs from home, often making some types of jobs more difficult. Review of eligibility requirements was one of those jobs that became difficult to do outside of the office setting since many of the application systems were not oriented to a remote, digital world, and often still used paper applications. One change to overcome this obstacle was to suspend or minimize some of the eligibility requirements, including eliminating educational requirements, reducing work experience requirements, eliminating criminal background checks, eliminating fingerprinting, and eliminating some checks of physical capabilities. However, the problem with suspending, eliminating, or reducing the eligibility requirements is that it is difficult to bring them back, and many question why they are even needed-if they were not necessary during the pandemic, are they still necessary now?
Another significant change includes the move to remote, online proctoring of examinations instead of delivery in a third-party testing location. In addition, some examinations include a practical or performance examination, which often involves performance prompts given to candidates who are using standardized tools, equipment, and materials and then evaluated by standardized graders/raters/examiners who are using scoring rubrics and checklists to minimize subjectivity.
Frequently, candidates have to travel to a central location for the performance/practical examination. Since the pandemic, many practical/performance examinations were eliminated due to the inability of candidates to travel. However, the opening up of travel has not been met with a return to in-person testing in many cases, and some performance/practical examinations are now gone for good.
Other changes include rushed or reduced check-in procedures (i.e., allowing expired IDs for use in identification, elimination of biometric checks such as fingerprinting, elimination of physical inspection of glasses, jewelry, etc.) and reduction in the length of the examination in both number of questions and time, thus reducing the seat time for an examination in a test center already experiencing limited seats due to social distancing requirements.
But among the most concerning changes (albeit temporary) was the total elimination of competence verification for both new and returning (retired) candidates, particularly in critical workforce shortage occupations such as health care. During the height of the pandemic, many regulatory bodies suspended examinations and other qualification requirements to allow upcoming nurses and other medical students to preemptively join the workforce because of a shortage of workers. Similarly, retired workers with long expired credentials were asked to come back to work. In both cases, individuals were allowed to practice without meeting the minimum competence requirements or obtain the necessary credential. It remains to be seen whether these individuals will be required to go back and meet the requirements they missed (for example take the credentialing examination) after they have already entered the workforce as a practicing professional.
In 2020, a Council on Licensure Enforcement and Regulation (CLEAR) survey of regulatory jurisdiction indicated 50% of the responding jurisdictions suspended requirements for initial licensure during the pandemic, including:
- providing emergency licenses without any assessment, particularly for professionals moving from other jurisdictions;
- providing a provisional license without an examination;
- reducing the experience requirement or allowing longer to complete a practicum;
- suspending in-person practical assessments and in-person supervision; and
- suspending criminal background checks/fingerprinting/use of copies.
Concerns With Recent Changes to the Credentialing Industry
The problem with these changes is that they were not made using psychometrically-sound measurement practices, nor was any scholarly study put into whether they affected the validity or reliability of the credential, examination, or competence verification process. What is becoming difficult to argue is why we need competence verification at all if people are allowed to enter professions and practice at times without verifying their competence. We may be on the precipice of a wave of an incompetent and unqualified workforce.
Another worrying change occurring during the pandemic was the suspension or elimination of recertification and re-licensing requirements and verification of continuing competence. Some of this was due to a lack of workers who could review the recertification and re-licensing applications, and some of it was due to critical worker shortages during the pandemic. But in any case, without re-verification of competence prior to recertification or re-licensing, the credentialing body lacks the documented evidence that the person continues to be competent. Will credibility of certification and licensure credentials be lost if it turns out that the person recertified or relicensed is no longer competent?
Changes in the development and maintenance of certification programs have also been seen in recent years, including using outdated or lapsed job/task analyses (JTA), examination forms being kept active for longer periods of time without being refreshed or newly written questions, and re-using previously used examination forms rather than creating new exam forms. These changes occurred in part because of limited ability for the needed experts to travel and discuss such activities.
Now, even though travel has opened back up, some credentialing bodies are pushing back on these traditional practices. Why not allow the JTA to stay valid for an extra year or two? Why not allow exam forms to stay live for longer periods of time, or why not re-use old exam forms? These practices have the tempting allure of fiscal savings but they may represent a degradation of the examination process and a reduction in the quality of the credential programs. Again, while these changes were made during emergency-duress times, many of them are now here to stay.
The implications the credentialing industry needs to consider is whether these recent changes made are changes that could be affecting the quality of the credentials, particularly the validity, reliability, fairness, and legal defensibility of the credentials. We have to ask whether acceptance of temporary deviations from traditional practices in an effort to survive should become more normalized.
For some organizations, returning to pre-pandemic practices may prove too challenging to satisfy what have now become new stakeholder expectations. Instead, credentialing bodies may need to focus their goals on identifying weaknesses in these new practices and correcting them through continuous improvement. Without research and documented evidence that these changes have not had an impact on the outcome of the evaluation of the knowledge and skills for competent performance by the credential holder, the industry is taking chances that could undermine confidence in credentials or one day harm consumers.
Originally published at https://blog.ansi.org on February 17, 2022.
