MDTP California State University, San Bernardino

Field-testing and Test Development Criteria

The critical criterion that each MDTP test must meet is that knowledge of the content of the topics and ability to answer the test’s questions are prerequisites for success in subsequent mathematics courses. Evidence that these criteria are satisfied comes from the acceptance of the tests by hundreds of teachers throughout California. Further evidence comes from the data collected during extensive field-testing of the tests. During the final stage of field-testing, the form is administered to students near the beginning of a course so that their test scores can be compared with measures of their performance at the end of the course. For example, the Pre-calculus Test has been administered at the start of a college calculus course and scores were compared with students’ grades in the course. For another example, the Algebra Readiness test has been given to students during the first few weeks of an elementary algebra course and their test scores were compared to both teacher rating of the students’ readiness for the next mathematics course at the end of the elementary algebra course and to the students’ performance on MDTP’s Elementary Algebra Diagnostic Test at the end of the elementary algebra course.

A part of the test development process is an extensive review of item statistics to ensure that each item tests appropriate knowledge and skill, that each item discriminates reasonably well between stronger and weaker students, and that the difficulty levels of the items are not too widespread. The R-Biserial is used as a measure of the consistency of an item with the rest of the test. Content validity of each item is also reviewed using the correlation of item student performance on the item with performance measures at the end of the course. Item discrimination is reviewed in two ways: one is by comparing the overall test performance of the subpopulations who choose each available response (including no response), the other is by comparing the performance of each of the five quintiles of students based on total test score with the performance of the other quintiles.

As new forms of tests are developed, efforts are made to ensure that these forms test the same knowledge domains and tasks at approximately the same level of difficulty. This is done partly by designing the new forms to meet the same predictive validity criteria as the older forms, with test developers maintaining their goal of developing tests of knowledge and skills critical for success in later mathematics courses. The test developers make only minor adjustments to test specifications. Equating methods are used to compare old and new test forms to ensure that there is no large change in overall test difficulty.

Send mail to lhartson@csusb.edu with questions or comments about this web site.

© 1998 MDTP