Upon reading the National Association of Flight Instructors’ report on how a recent change in the FAA knowledge test question banks had increased the number of failures, my initial reaction sided with the FAA. Let me explain: When I was with Flight Training magazine, the banks of test questions were still public. With the release of every new bank I compared it to the old one and shared the changes with readers.
From experience I knew that writing good multiple-guess questions isn’t easy. The question must have a clear link to the knowledge taught and, to fully assess student understanding, subtle nuances should separate the right, almost-right, and wrong answers.
After a decade of evaluating its work, I held the FAA’s question creating ability in high regard. They’d elevated the need to RTFQ and UTFA (that would be read the freakin’ question, understand the freakin’ answer) to an art form, which is why I initially sided with it. And then I called NAFI Executive Director Jason Blair to learn how this problem came to light and any new developments since the initial March 3 report.
Our conversation reversed my take on the situation, which came to light at several university aviation programs. Professional educators, they teach knowledge, not the test. On average, 13 percent of their students failed the Fundamentals of Instruction (FOI) test. January 2011 was an average month. In February, 58 percent failed. Since the teachers and the knowledge they’d presented was unchanged, the test was the problem.
While I was talking to NAFI, Logan Derby, a student in the aviation program at the University of Nebraska-Omaha, e-mailed JetWhine. A CFI candidate, he bought a 2011 FOI test prep book to supplement his classroom learning. He thought the knowledge he learned and studied was current. “Man was I wrong! It wasn’t till I had failed the FOI written that I come to find out the FAA decide to change a number of questions.”
[Read more…] about New Pilot Tests Miss Fundamental Marks