A couple of days ago, I received the National Association for Healthcare Quality (NAHQ) 2010 Annual Report. Most of it was quite unremarkable, but a stacked bar chart caught my attention—aggregate pass rates for the CPHQ exam in 2010.
Although this was not the first time the Healthcare Quality Certification Board (HQCB) (recently renamed the “Healthcare Quality Certification Commission” (HQCC)) has published CPHQ exam pass rates, we haven’t seen a breakdown between domestic (US-based) and international (non US-based) candidates (presented in this manner) for some time (years!).
The chart was particularly interesting to me, but for reasons you probably did not expect.
The numbers just did not add up.
Call it whatever you want—my training in statistics, my work as a healthcare quality professional and having to deal with a lot of graphs, or my attention to detail—the chart looked wrong. I did some mental math and it was still wrong. So I took out an electronic calculator and got the following numbers:
Overall 547 ÷ (547 + 264) = 67.4%
(When rounded off to a whole number, this is consistent with what was published. This is also consistent with the CPHQ exam pass rates of recent years.)
U.S. Based 486 ÷ (486 + 160) = 75.2%
International 61 ÷ (61 + 104) = 37.0%
Notice the discrepancy in the last two bolded figures? Typo errors on the part of the HQCB/HQCC? Well, that was my thought initially.
But there is another (simple) explanation.
We know that there is always going to be a failure rate among candidates, and a large proportion of those who don’t pass on their first attempt will try again. Some candidates even take the exam three times (in a year). Though possible, I would be extremely surprised if someone attempted the CPHQ exam four times in a single calendar year because candidates are permitted to take the exam only once per 90-day period. The numerator (number of candidates who passed) will remain the same, regardless of the denominator used, because CPHQs who have passed won’t take the exam again in the same year (that’s nuts!). If the denominator is the number of attempts by all candidates in 2010 (as opposed to the total number of candidates), then the quoted pass rate for US-based candidates and international candidates (but not the overall figure) in the bar chart above could actually be valid. In fact, I believe this is probably the case.
However, the lack of internal consistency in the chart above, especially in the context it was presented, is disturbing. If the denominators used for US-based candidates and international candidates were indeed the number of attempts, then we should have seen a dilution of the overall pass rate as well, i.e. a figure lower than 67%, in addition to lower (diluted) figures for both US-based and international candidates (which is the case).
Following this analysis (which took me less than three minutes), I drew several general conclusions:
- The majority of candidates (79.7%) are based in the US.
- International candidates, though a small proportion of the candidate pool in 2010, is the faster growing segment. In other words, compared to previous years, relatively more international (non US-based) healthcare professionals attempted the CPHQ exam in 2010. Unfortunately, they are encountering problems in passing—this may actually not reflect their ability in the field of healthcare quality. Several factors contribute to this apparent low pass rate, including cultural issues, language, training, and local experience. As I point out to my students (US-based and international), the CPHQ exam is US-centric, and not as “international” as suggested by the HQCB in the past (more about this in a future article).
- The overall pass rate in 2010 is similar to, or worse than, rates in recent years. With the introduction of various educational programs and other resources to prepare candidates in the last few years, we should have expected a gradual improvement in the overall pass rate. Clearly, the latter has not happened and the utility of traditional methods of exam preparation (as well as new ones) need to be (re)considered.
- The pass rates among our students (both in North America and elsewhere) are significantly higher than the corresponding average in each segment (US-based; international). This is hardly surprising (at least to us). Our students will attest that we do many things differently around here. However, for us, what matters the most is the result. If, on the whole, candidates prepare for the exam in the same way as their predecessors, their overall results will remain the same, i.e. about 67%. Changing their approach to the exam does not guarantee a better result, but gives them a better chance of doing so. If the definition of insanity is “doing the same thing year after year and expecting different results”, then a large proportion of CPHQ candidates must be insane.
- International candidates probably do not reattempt the exam after failing the first time as frequently as their US-based counterparts. (This contributes to the wide gap in pass rates between the two groups.) Depending on their individual set of circumstances, this might be prudent. On the other hand, as shown by the numbers (and provided my hypothesis that the number of attempts is used as the denominator is true), multiple attempts—at least among US-based candidates—are associated with an improvement in the overall pass rate. This should not be misinterpreted as proof that multiple attempts improve an individual’s chances of passing the exam—clearly, this is dependent on multiple factors.
- There were only 547 new CPHQs in the whole of 2010 (!), which probably makes the credential a highly sought after commodity, and therefore extremely valuable.
Back to the bar chart. The ideal chart/graph/table tells the whole story without the reader having to refer to the text. In this case, the chart does a good job in providing the number of candidates based in the US as well as internationally. However, where it falls short is in the stratified pass rates, which do not tally with their corresponding numbers, and may leave readers perplexed.