askdoc-usmle

The NBME self assessment test is another of those tools that I feel is responsible for the increasingly high scores in the USMLE. It is a great tool that allows the test-taker to assess his or her readiness for the exam. To a certain extent, if used properly, it is also a useful tool for the examinee to target a certain score, even 99’s and achieve it.

The questions for the NBME self assessment tests were taken from a pool originally designed for the shelf exams of American Medical Student. As such it covers the most important concepts that AMGs are required to know before passing to the next level. It covers the same topics that will be covered in the actual USMLE examination.

The NBME self assessment tests are very good at predicting the score you will get at the USMLE. In fact I read somewhere (unfortunately, I can’t seem to find the site now though) that at least for Step 1 the 1^st exam set has a correlation as high as + 0.89, while the others range from + 0.72 to + 0.85. If you go to most of the forums, you will find that the correlation for most examinees is very good.

Although the topics covered in both the NBME and the USMLE is more or less the same, there is a tremendous difference between them.

First, the NBME questions are much, much more straightforward than the USMLE questions. The NBME has precious little tough questions. Therefore a low score in the NBME is more indicative of lack of knowledge of the topics covered in both exams rather than a lack of mastery of those same topics. Although the latter exam sets have tougher questions than the first sets, they still are not as tough as the tougher parts of the USMLE, plus, they actually have lower correlation with USMLE scores than the first exam set.

Second, the NBME questions total less than the actual USMLE exam and therefore will cover less lower-yield stuff than the actual USMLE exam.

Third, you can have up to 4 hours to answer the examination. Part of the toughness of the USMLE is the time constraint imposed on the examinee.

Fourth, you can take the blocks at a time of your own choosing, usually during periods where you are at your peak effectiveness. The actual examination does not give you such luxury.

Fifth, you have to take the USMLE blocks, 7 to 8 at one time with minimal breaks, which can take its toll, believe me. By the fifth or sixth block, your mind tends to stop functioning. You have only 4 blocks in the NBME and you can take it at your leisure.

So despite the obvious difference between the 2, why is the NBME such a good predictor of USMLE performance? The answer lies in the fact that the NBME has taken the time to correlate the results of hundreds of thousands of AMGs with their actual performance in the USMLE. This correlation has taken into account the difference in test conditions and actual content of the examination and that this difference although apparent is not very significant as can be shown by the correlation of + 0.72 to + 0.89 between the various NBME exam sets and the USMLE.

So the practice of using the “downloaded” version of the NBME tests effectively wipes out the predictability of the exam set itself. It’s effectiveness lies in the statistical analysis they’ve done for the results and not the results themselves.

Another ineffective practice, is studying the NBME test questions to improve your score. This will be effective only if your original review is so poor that you have not covered the required high yield topics tested in the USMLE properly. Otherwise you are better off studying the questions in USMLE world and Kaplan Q Bank to understand how to answer tough questions and low yield stuff that are more likely to appear in the USMLE.

Another common error is repeating the exact same exam set. The correlation holds only if you take the exam once and only once. Repeating it will raise your score in the NBME which will not be carried over to the USMLE. Therefore, I suggest reserving at least one set to be taken only if you are ready to do the USMLE.

Used properly, the NBME tests can be an effective tool for self-review. So do yourself a favor, use the NBME self assessment tests properly as an assessment tool rather than a study tool. And take the online form rather than downloading them, which is also subject to copyright issues. The NBME self assessment site is here.

*note: Part II deals with When and How to use the NBME self-assessment tests and whether it is wise to use USMLE World self-assessment tests instead.