Industry, FAA seek fix for knowledge tests

The FAA, responding to an outpouring of criticism from AOPA and others for unannounced changes to the knowledge testing system, has promised better coordination with the flight training industry on future reforms.

Officials of the FAA’s Airman Testing Standards Branch met April 20 with AOPA and other industry participants to address concerns about changes that without notice altered the content of numerous knowledge test question banks, spiking failure rates.

AOPA reported March 3 on the impact of the changes that drove up some failures rates to as high as 56 percent from about 13 percent on one exam. In a joint letter with the National Association of Flight Instructors (NAFI), AOPA called on the FAA to restore the previous test questions, and expunge failures of new tests from student records. The FAA has not responded to those requests, nor has it released pass-fail statistics on many of the new tests.

Flight training industry members who attended the meeting emphasized the need for the FAA to provide guidance on training that matches the FAA’s new testing expectations. For the industry to function, providers must know the completion standards that have been set, they said.

AOPA stressed that the industry was not calling for the FAA to resume publishing all test questions, as it did until about seven years ago. Test applicants should be able to prepare with an understanding of how the FAA validates its learning expectations with the knowledge tests. 

Unfortunately, the surprise test changes--and the FAA’s apparent reluctance to reverse them--damaged the FAA’s credibility with students and flight training providers, said AOPA Manager of Regulatory Affairs Kristine Hartzell, who attended the meeting at the FAA’s Mike Monroney Aeronautical Center in Oklahoma City, Okla.

“Changing the knowledge areas tested without coordinating with instructors does nothing to advance aviation safety; it only results in more money spent by applicants for their certification, scarred permanent records, and frustrated instructors not given the tools to perform their duties effectively,” she wrote. She reiterated AOPA’s calls for a return to the test question banks in use prior to the changes.

Hartzell pointed out that for several years since the publishing of actual test questions ceased, the FAA has contradicted its own advice about training by not providing guidance about learning objectives or completion standards.

“Ironically, the FAA Instructor’s Handbook repeatedly stresses that instructors need to make a ‘determination of objectives and standards’ before ‘any important instruction can be presented,’” she wrote.

Hartzell illustrated the unfair impact of the unannounced test changes in a follow-up letter to the FAA on March 23. She cited the example of university students who scored in the upper 90 percent on practice exams after a semester studying the material, but then failed the knowledge test.

In sharp contrast to the knowledge tests, “practical test standards (PTS) are objective, and criteria are published and available to the public. The PTS publications “hold an important position in aviation training curricula because they supply the instructor with specific performance objectives based on the standards that must be met for the issuance of a particular aviation certificate or rating,” she wrote.

The FAA does not have test standards for the knowledge exam, leaving instructors and students to “speculate what learning objectives might be tested based on the extremely broad knowledge areas” provided in regulations, handbooks, advisory circulars, and other texts.

Hartzell said after the meeting that she was encouraged to hear FAA officials commit to changing the way it communicates changes in knowledge-test learning objectives to the industry.

“They will implement a process by which any knowledge areas will be first changed in training handbooks and Practical Test Standards, and be made available to the public for 90 days before any new knowledge test questions on the changed subjects will appear in a knowledge exam,” she said.

Based on requests from several people in the aviation industry, the FAA agreed to form a steering committee of industry leaders to make recommendations on system changes, for quick enactment.

“Until better coordination is achieved, it is unrealistic and unfair to expect students to meet the new learning objectives now being tested,” Hartzell said after the meeting.

Another problem was the reduced availability and higher costs of testing resulting from the FAA’s initiation of an organization designation authorization program, that forced some smaller test centers to close, she said.

The FAA also revealed that it is considering a modular testing format; test takers would be required to pass each module of a test with a score of at least 70 percent, officials said.

Industry members who attended the meeting included NAFI, the General Aviation Manufacturers Association, the Society of Aviation and Flight Educators, King Schools, Jeppesen, Aviation Supplies and Academics (ASA), Gleim, Computer Assisted Testing Service (CATS), LaserGrade, and others.