It’s been over a year since I first published “NBME Self-assessment Tests and USMLE Review.” Little did I know it will become the most popular of my post with over 14,000 pageviews in the past year. Since that time a lot of things have changed, hence this update.
If you have not read the previous two posts on this topic, please do so as I will not be repeating what I have said there in this post. You can access Part 1 here and Part 2 here.
How well does NBME predict your USMLE Score?
That has been the number one question asked of me since I wrote about this topic. My answer is still the same. Fairly well. Although correlation is never 100% more like 70 to 80%. However, certain development in the past few years have made the assessment tests less reliable for some people.
One of the reasons for this problem is that a lot of NBME questions, especially the earlier forms of step 1 have been discussed extensively in various forums. What’s more some of these posters did not even bother to warn people that what they are discussing are NBME form questions. Going into the NBME assessment tests knowing some of the answers already can invalidate the predictability of those assessment tests. In fact just knowing some of the questions beforehand can also invalidate the results. The reason is that knowing the questions ahead, means you’ve had time to consider the questions and possible answers before, not just the 1 minute or so that you will actually have in a real exam. That can skew your result.
It has also come to my attention that some NBME questions or variations of those questions have come out in online qbanks. That is bad news, too as it has the same effect of invalidating some of the questions in the NBME forms. Even if you did not get the answer, encountering the questions in the NBME form for the second time rather than the first time means you had more than the 1 minute allowed in the exam to think through the questions and look for the answers.
So what are the remedies for these problems that seem to have cropped up recently. Well first is to actively avoid discussing posted NBME questions in the forums. Of course since some posters do not have the courtesy of even warning people about it, avoid participating in any discussions on questions in forums unless you know for sure that they did not come from an NBME form.
Another solution is to try to use the later forms. In Step 1 this would be form 4. 5 and 6. NBME forms for Step 2CK does not suffer from the same problems as those in Step 1 as they are not discussed as often as those of Step 1.
Doing at least 2 forms instead of just 1 as I recommended before may also help. Just make sure one of the form is not the first 3.
What about the “Downloaded” NBME Self-assessment test version.
I can never understand the popularity of the so-called “downloaded” version of the NBME. IF saving a hundred dollars or so is worth scoring low or failing the USMLE altogether, then it is understandable. But ruining your long term career to save a couple of bucks is not a very intelligent move.
In earlier posts, I have said that the value in the NBME does not lie with the fact that the questions mimic the USMLE. In fact, in general, they are much easier than the USMLE. The main value of the NBME forms is that they are fairly reliable predictors of performance in the USMLE due to the correlation they’ve done with NBME results vs. actual USMLE performance.
However, recently, some posters in my blog have commented that they are able to “predict” their USMLE scores, since there are answer keys and correlation tables available with the downloaded version. So I decided to give it another look.
So what did I find out? Well, first, there is a problem with the answer keys. There were some answers that I completely disagree with. In some cases where I myself am not sure what the right answer is, I tried to verify the possible correct answer by researching them and I still cannot decide what the right answer is even after searching through textbooks and the internet. Therefore, there is a question of how accurate the raw score one is getting for each of those NBME forms are. And that is a major problem. In my case, anywhere from 2 to 8 answers in each form fall into this category and for me, an “unknown” of 4 to 16% in the raw scores completely shoots down any chance of actually knowing the exact raw score you should be getting.
A second major problem is that there is only one correlation table in existence, instead of the 6 correlation table I am expecting. That is one per form. There might be additional correlation tables out there, but the one I got is just 1. Why six tables? Because the forms are of different levels of difficulty, you expect the same person will get different number of questions right in the different forms depending on the level of difficulty. So you need a correlation table for each form to make them comparable. Having only one correlation table means we don’t even know to which form this correlation table belongs. See the problem, now.
A third really big problem is that people who used this downloaded forms has a tendency to keep on using them throughout their prep multiple times. Probably because it’s free. In fact, it was justified that since they did not try to look at the answer, they can redo the same form and expect it to still be accurate in predicting their scores. That is actually wrong. Again, one of the reasons why USMLE is hard is the time limit imposed in answering questions. When you go through the same question multiple times, you’ve had more than the 1 minute per question limit imposed by USMLE to think of the answer and therefore will tend to score higher. That skews the predictability of the NBME.
Therefore, again, do yourself a favor and use only the online NBME assessment tests and use them only when you feel you are ready for the USMLE, to confirm your readiness. The “downloaded” NBME forms may seem free, but it’s hidden costs may be greater than you are willing to pay.
USMLE World Assessment Tests
I’ve had more time to gather information about this relatively new resource. So far based on talking to my students, posters in my blogs, people who have emailed me and reading various forums, my conclusion is that the UW assessment tests is just as good as the NBME assessment tests, so far. Although there have been some observations that UW tend to be overestimate your scores in comparison to NBME, this does not seem to happen in all cases and the score difference is not too big. So all in all, I believe the UW assessment tests have enough track record by this time that we can safely say, they are fairly accurate in predicting USMLE scores. But as in all assessment tests, correlation is never 100%, therefore expect some deviation from predicted scores in the final result.
Related Posts:
-
Which NBME Self Assessment Form to Use for USMLE Step 1
-
NBME Self-Assessment Tests and USMLE Review – Part II
-
NBME Self Assessment Tests and USMLE Review
-
How to Create a Study Plan for the USMLE
-
High Yield Fast Facts for the USMLE
-
Guide to Self-Evaluation in USMLE Prep
I have my Step 1 in 2 weeks. I have only done form 7 of NBME and got a 240. I was very very upset as I was aiming for a score in the 260s. I had a first average of 78% on the UWorld question bank and a 93% average on the USMLE official questions and a 256 on the UWorld Self Assesment test. So why was my NBME score so awful? I have taken this in a good way and have made my study habits even more intense but I desperately want to get above 265. Any advice?
-Heartbroken
Hi Marjan,
Just so you can do a comparison. My NBME was a 740, so predicted score of 265. My actual score is 256. My average for Kaplan qbank (uworld qbank for Step 1 did not yet exist) is 84%. I got 98% on USMLE official questions. UWord self-assessment grossly overestimates your score. So your NBME score is not awful, just right for your other areas. My suggestion is do not obsess about a certain score. Just do the best you can. Too much obsession creates too much tension. This pressure can even cause you to perform poorly in the actual exam. Relax take things in stride. 240 is a 99 already, so not bad.
Askdoc
I took CK exam on March/2/2010, For me the exam questions is longer and more difficult than NBME. I could finish questions on time but did not have time to review most of marked questions (around 5-10 per 1 block. When I went home, I was recalling quite a lot of questions, especially that I considered much to take one of the two similar options. So far, I found at least 12 wrong answers among them couples of stupid mistakes.
My NBME 2 one month before exam 570/246
NBME 10 days before exam 540/239
I am aiming at 99 for CK(step 1 224/96 Aug/11)because I am an old IMG-graduated in 1995. But now i am not feeling secured at all although I though what i did in the exam is ok. When I did NMBE I could guess my score since the questions is straightforward but the questions in real exam is somewhat vague.
I read a lot of your replies in this forum and I try to assure myself that my worst possibility is 20 point lower than NBME, I still got 226 and that is enough for 99 at this time. I am checking many test takers in forums and found a very small percentage of them got the real score lower than 20 point or more form NBME.
Though I know everything can happen, but you are so experienced in this, please give me advice for my case, how likely i will get 99?
Thank you so much
Bill
Hi Bill,
I know its easier said than done, but I am giving you this advise although I did not really take it that well too when I was waiting for my results. What’s done is done. There is nothing you can do about it. Your results will be out in about a week. Wait for it and hope for the best. If you read my post, the exam is always more difficult than the NBME contrary to what everyone is saying. NBME is a relatively good predictor of USMLE performance but with limitations you just cited. If you did not panic during the exam, then you will probably be within the score you are expecting. Whether that is a 99 or not, I cannot really tell as the 3 digit score that corresponds to a certain 2-digit score changes a bit depending on the question set you got in the actual exam.
Also if you are able to remember your mistakes, its usually because you have so little of them that you can actually remember most of them. So it may be good news. Note the maybe. Anyway, good luck. I think you will get a good score nonetheless.
Askdoc
I did my first NBME yesterday. I decided to do it earlier because I need to know where I stand now. Since it costs lots of money I will only take 2 NBMEs (NMBE 6 and 7). So yesterday I took NBME 6 and I got 570/238 on this exam. I am not really happy with the 3 digit score because I aim for +250. It’s also not co-related with my USMLE World Qbank performance. I have done 25% of the Qbank and averaging 79% so I expect to score higher on NBME.
Do you think I have to review my lecture notes again or I just finish the remaining questions on UW qbank ? I am lil bit depressed now and loss confidence. I know 238 is a good score, but I expect to get higher. I wish to give my exam early May, what best should I do now to utilize the remaining time ?
Thanks for your assistance
Hi Harry,
You sound just like me when I was preparing for the exam. Anyway, it’s good to aim high, but bad to be obsessed about it. You need to relax. So far so good actually. You still have about 5 weeks before your exam, that will give you enough time to raise your scores. I suggest you do both. Go through lecture notes again but concentrating on high yield stuff and finish q bank. Just beware of burning out early or what we call plateauing. And being depressed or pressuring yourself too much can increase burn out. So relax, continue your review. But make sure you sit for the exam first week of May. Further delay increases the chance of burnout and you may even go into decline, where further study can cause your score to go lower, as some people found out eventually.
Askdoc
Hi Doc,
Thanks so much for sharing your insights on this topic. I have learned a lot from you. I’m glad that I read your post before taking the NBME offline version. I plan on taking all the available NBME exams (thanks to student loans!), and I have scheduled them ~5 days apart throughout my 6wks study time frame. I know that you mentioned to not use them as a screening test for my weak spots, but do you think my plan is wise since I want to take all of them? Currently, I have taken 1 UW self-assessment (before I started my 6wk plan and got a 214) and have completed 65% of the UW Qbank. I know you always mention to not obsess over the score but it was disappointing since most posters on the USMLE forums score much higher…so it’s difficult not to worry. I have 5 weeks left in my studying, so please advise!
Thanks!
hi askdoc
I have been preparing for CK for past 3 months, My first problem is , i can not finish exam on time.. i miss about 7 questions on each block..so my scores don’t come good.
i Have been taking NBME self assessment to see my progress and check my scores.. i used #1 without studying.. and then #2 in june, and #3 july(finally got near passing score), and now i am taking #4… my first concern is once i run out of #4, what can i use as assessement priort to exam…currently i am scheduled for sep 6th, but i will only take if i have quite a bit improvement from #3… do you know any other assessment test which is accurate enough? also, how can i improve my speed? i have been reading the questions first , and then,go back and read the vineett…
i have been told by my school that “Don’t take it unless you are ready, even if that mean you take the exam in end of september or october, and just apply on september 1st” I goto Ross, i am IMG, and US citizen…
Thank You for ur kind help
Hi josh,
First, you should not have used NBME Self assessment tests to check your progress. You run out of a reliable test to know if you are ready to sit for the exam. That is the first rule you have broken. Second you should not use online qbanks to practice speedbuilding. Use another qbank software to do so. Going thru questions fast is mostly a matter of habit. Therefore you need to practice to go thru questions fast. You can’t use online qbanks because you need them to practice another skill, answering tough questions. Your school is correct. Taking the exam when you are not ready will just result in you failing it. It will not help you in the match.
Askdoc
I am struggling obtaining a residency, because my step scores are borderline. I desparately hope to improve chances by performing well on the step 3. But I cannot understand what I am doing wrong. I have read the kaplan master the board for step 3, have completed the uworld q bank (avg 56%), finished the qbook step 3 (avg 73%), but an NBME of 240 (equiv around 140 or so). My exam is scheduled in 7 days!!!!!!!! how can I use the nbme to improve? It’s saying that I am weak on things like continuing care… how does one target this?
thank you for your guidance