This started out as a reply to a question asked by dr patel on my blog on how to evaluate one’s readiness for the USMLE and avoid making the mistake of taking the examination and failing it. However, it got a little bit too long, so I have decided to rewrite it into an article.
Failure to properly evaluate one’s readiness for the examination and one’s progress during the whole USMLE preparation process may be one of the most crucial reason not only for failing the USMLE but also for failure to achieve a less than stellar score.
Proper self-evaluation involves not only how to do the evaluation, using the right tool to measure the right parameter but also when to do the measuring to insure maximal effectiveness. Most failures in self-evaluation can be divided into these 2 broad categories.
Whenever, we mention evaluation, most people think of the NBME self-assessment tests primarily or the online q Banks like UW and Kaplan. These are very useful assessment and evaluation tools, but they are not the only ones and not necessarily applicable in all situations. Used the right way, they are extremely effective, but the propensity of most people to use them in all kinds of situation is not necessarily a good idea. The right tool used for the right situation is very effective. Using the right tool for the wrong situations makes it less effective.
You see people using NBME to evaluate themselves at various stages of their review rather than at the end, to gauge their final readiness for the USMLE is an example of an expensive misuse of a great tool. I wonder what you would think of your anatomy professor if after finishing upper extremities, he uses the USMLE Step 1 to test how well you are doing so far. Ridiculous and yet that is what most people are actually doing during their prep.
The MRI may be one of the biggest breakthrough in medicine in recent years, but it is a tool with a specific function. Using it to measure blood glucose level is ridiculous. So use the right tool for the right situation. We’ll talk about that more later.
The second area where most people commit a big mistake in self-evaluation is when to take it. One of the primary reasons being the myth that only online q Banks will do as evaluation tools for the USMLE which is far from the truth. A myth perpetrated by a lot of posters in forums and of course encouraged by online q Bank providers for obvious reasons. Again as stated before, proper tools for proper situations will provide excellent results. But for other situations other tools are better.
Usually you have people studying for 3 or 4 months, reading the various subjects, Anatomy, Physiology, etc. for step 1 and Internal Medicine, etc for step 2 CK and step 3. Then they enroll in the online q Bank, the first time they will actually test what they have been able to retain in their readings for the past 4 months. Then shock when they are scoring 40 to 50% followed by panic since they realized that they have 2 months left before their scheduled exam and that after 4 months of efforts, they’ve only come so far. So they searched the forum looking for answers that tell them all is not lost and reject anything that says otherwise. Most of the time, they take the suggestion which they so want to be true, that by just doing q Banks for a month or so, they will raise their scores. Yeah, right probably by 5% or less so unless they are already scoring borderline scores, doing that will not help. Their only choice really is to restart their review, this time doing them the right way. Don’t get me wrong, doing q Banks can raise your scores. However, there seems to be a pattern. The higher your starting q Banks scores, the more help doing q Banks will be to raise your scores. Below a certain level, doing q Banks will not help. People just fail.
All of this could’ve been avoided if proper self-evaluation have been instituted right from the start of their prep process. Self-evaluation is important to insure that you are prepping the proper way and that your prep is effective. That is why I’ve always advocated that people should use Subject specific q Banks right from the start. They don’t have to be USMLE type questions, since the main objective is to see if you know the subject, not in how good you are at answering tough, USMLE type questions. You need to know the subject well, before you can answer the tougher questions. You need to learn basic arithmetic before you can answer algebra or trigonometry. In my prep course, the course participants have a quiz per chapter (comprehensive quiz with 30 to 65 questions per chapter) to insure they know the topic before moving to the next one. Any problem is solved on the spot. So they don’t have to wait until 4 months later to realize something is wrong with their prep and they have to do them all over again.
Even in medical school, everyone knows you should evaluate performance at regular intervals to insure that people are learning. Imagine if your dean tells you that due to the huge success this method of evaluation is among people prepping for the USMLE. They will now only give one evaluation examination at the end of 4th year to test whether you will become an MD or not. That’s not very realistic is it and yet that is what most people do when they prep for the USMLE.
So how do you go about doing your self-evaluation? In the beginning, when revising the subjects, you should be prepared to test yourself chapter by chapter to insure that you know your stuff before moving on. When you finished each subject. Test yourself using by subject q Books. You use online q Banks after you’ve finished your first round of review, both to assess how much you know and help raise your scores. Then just before the examination, you use NBME to confirm your actual readiness for the exam and if you are going for a certain score, whether you will make it or not.
For example you are preparing for Step 1. You can use Robbin’s Review of Pathology q Book as a chapter by chapter quiz for Pathology. It is important that the chapter quiz is comprehensive and not the short ones usually found in the end of Kaplan notes chapters or BRS Pathology chapters. For an example, you can go to my prep site at http://prep.askdoc-usmle.com. Log in and enroll into the Pathdemo program to access the quiz. Compare how comprehensive the Cell Pathology quiz is to the Cell Pathology Study Notes provided or even the Cell Pathology notes in Kaplan lecture notes. That is how comprehensive the chapter quiz should be. For Microbiology and Immunology, Levinson and Jawetz have a very good chapter by chapter quiz. My prep course will also develop its own as it is finished. For Pharmacology, Trevor and Katzung also have a comprehensive chapter by chapter quiz. For the other 4 subjects, there is no comprehensive chapter by chapter quiz q Book that exists, although, the Kaplan q Book which covers by the subject may be an adequate substitute.
For Step 2 CK and Step 3, no adequate chapter by chapter quiz exists. Harrisons q Book contains too many low yield stuff and may be more helpful for ABIM rather than Step 2 CK or Step 3. Good subject exams include the Kaplan Q Book and the Blueprints q books.
Although I have chapter by chapter quizzes in my prep courses, only Review of Pathology is available and will take 6 to 9 months to finish the rest. However, good commercial chapter by chapter quizzes are available for Pathology, Microbiology and Immunology and Pharmacology as stated above and I would suggest you make use of them right from the start of your review.
If anyone knows of other chapter by chapter q Books available, I would appreciate it if you can bring it to my attention, especially for the other subjects, so I can evaluate them and make recommendations.