comparative study of item exposure control methods in computerized adaptive testing

by Shun-Wen Chang

Publisher: ACT, Inc. in Iowa City, IA

Written in English
Published: Pages: 46 Downloads: 937
Share This

Subjects:

  • Chang, Shun-Wen,
  • Twu, Bor-Yuan,
  • American College Testing Program,
  • Educational tests and measurements -- United States -- Data processing,
  • Computer adaptive testing,
  • Examinations -- Evaluation
  • Edition Notes

    Bibliography: p. 33-34

    StatementShun-Wen Chang [and] Bor-Yaun Twu
    SeriesACT research report series -- 98-3
    The Physical Object
    Paginationv, 46 p. :
    Number of Pages46
    ID Numbers
    Open LibraryOL15250713M

scoring. In addition, changes in components of a CAT, such as exposure rate control, content balancing, test length, and item pool size were found to result in different levels of comparability in test scores. Computerized adaptive testing (CAT) research has, in recent years, emphasized. Computerized Adaptive Testing (CAT) •Custom individualized assessment •Suitable for clinical use •Accuracy level chosen by researcher Prostate Cancer —Item40 —Item38 —Item34 —Item32 —Item26 —Item 22 —Item 18 —Item 16 —Item 8 —Item 2 Breast Cancer —Item 36 —Item 34 —Item 32 —Item 28 —Item 26 —Item 22 —Item.   This study provides more efficient tools for assessing how food parenting practices influence child dietary behaviours. Next steps are to use the IRM calibrated item bank and draw on computerized adaptive testing methodology to administer the item bank and provide flexibility in item . The purposes of the study are to investigate effects of missing data in computerized adaptive testing (CAT) on accuracy of item parameter estimation based on Rasch model using Northwest Evaluation Association (NWEA) and WINSTEPS calibration methods, and to provides information on the relationship between two methods. The results illustrate that.

Practical Questions in Introducing Computerized Adaptive Testing for K Assessments Introduction and-pencil testing methods. However, along with the conceptual advantages, there are also a number of to CAT item exposure control that have been documented in . book primarily addresses three aspects of CAT-ASVAB history in DoD (adaptive testing measures and strategies; CAT-ASVAB system design issues; and CAT-ASVAB evaluation). It provides reference information useful to practitioners developing a computerized testing system. Publication by APA will occur in early Computerized adaptive testing (CAT) has the potential to efficiently administer items for children across wide age spans and functional levels. Objective: The objective of this study was to examine the psychometric properties of a new item bank and simulated computerized adaptive test to assess activity level abilities in children with CP. Computerized adaptive testing (CAT) is a form of computer-based test that adapts to the examinee's ability level. For this reason, it has also been called tailored other words, it is a form of computer-administered test in which the next item or set of items selected to be administered depends on the correctness of the test taker's responses to the most recent items administered.

Weiss edited the Proceedings of the Computerized Adaptive Testing Conference. These Proceedings contained an up-to-date collection of paper on adaptive testing, one of the main practical uses of IRT. Lord and his staff at ETS made available the second edition of LOGIST. This updated computer program was faster, somewhat easier to. The PR-SE yielded the smallest amount of item overlap between tests across conditions and administered fewer items on average than SH. PR-SE obtained these results with similar, and acceptable, measurement precision compared to the other exposure control procedures while vastly improving on item pool usage. Citation. Adaptive testing relies on Item Response Theory (IRT) and modern computer technology which can adapt the test as the candidate completes it. IRT has long been used to identify ambiguous items (where stronger candidates so less well on an item) however it is now being used to adapt the test in as the candidate is sitting it using real time scoring. The defining feature of single-case research is the use of each participant (subject) as his or her own experimental control. This approach to research design arose from early work of researchers such as Burrhus F. Skinner and Werner Honig in the s and s focusing on behavior analysis, and it was codified in the seminal book Tactics of.

comparative study of item exposure control methods in computerized adaptive testing by Shun-Wen Chang Download PDF EPUB FB2

A Comparative Study of Item Exposure Control Methods in Computerized Adaptive Testing The development of computerized adaptive tests (CATs) and the research on computerized adaptive testing (CAT) have reached unprecedented levels.

With a great deal of effort over the last. Comparative study of item exposure control methods in computerized adaptive testing. Iowa City, IA: ACT, Inc., (OCoLC) Document Type: Book: All Authors / Contributors: Shun-Wen Chang; Bor-Yuan Twu; American College Testing Program.

This study compared the properties of five methods of item exposure control within the purview of estimating examinees’ abilities in a computerized adaptive testing (CAT) : Shun-Wen. Chang. A Comparison of Item Exposure Control Methods in Computerized Adaptive Testing Javier Revuelta Vicente Ponsoda Universidad Aut6noma de Madrid Two new methods for item exposure control were proposed.

In the Progressive method, as the test progresses, the influence of a random component on item. Two new methods for item exposure control were proposed. In the Progressive method, as the test progresses, the influence of a random component on item selection is reduced and the importance of item information is increasingly more prominent.

In the. Predicting Item Exposure Parameters in Computerized Adaptive Testing One practical advantage of computerized adaptive tests (CATs) is that they can be administered on a flexible schedule rather than at fixed times. The convenience and flexibility for examinees, however, may severely compromise test security if item exposure is not well controlled.

This article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels.

Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. computerized adaptive testing, item exposure control, test security, item selection There are at least two objectives to be maximized in a computerized adaptive test (CAT): the first is.

Exposure control research with polytomous item pools has determined that randomization procedures can be very effective for controlling test security in computerized adaptive testing (CAT).

The current study investigated the performance of four procedures for controlling item exposure in a CAT under the partial credit model. Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests.

The method resembles Sympson and Hetter's () method of item-exposure control in that the decisions to impose the.

This study investigated and compared the properties of five methods of item exposure control within the purview of estimating examinees' abilities in a computerized adaptive testing (CAT) context. Each of the exposure control algorithms was incorporated into the item selection procedure and the adaptive testing progressed based on the CAT design established for this study.

This study aims to investigate the effects of item exposure control methods on measurement precision and on test security under various item selection methods and item pool characteristics. In this study, the Ran-domesque (with item group sizes of 5.

Compared the properties of five methods of item exposure control in the context of estimating examinees' abilities in a computerized adaptive testing situation.

Findings show advantages to the Stocking and Lewis conditional multinomial procedure (M. Stocking and C. Lewis, ) and, to a lesser degree, the Davy and Parshall method (T. Davey and C. Parshall, ). Exposure control research with polytomous item pools has determined that randomization procedures can be very effective for controlling test security in computerized adaptive testing (CAT).

The current study investigated the performance of four procedures for controlling item exposure in a CAT under the partial credit model. In addition to a no. A Review of Item Exposure Control Strategies for Computerized Adaptive Testing Developed from to Subject: Article submitted to Journal of Technology, Learning, and Assessment Created Date: 5/7/ AM.

A Strategy for Controlling Item Exposure in Multidimensional Computerized Adaptive Testing Abstract While computerized adaptive tests have enjoyed tremendousgrowth inthe pastdecade, sat-isfactory solutions for many important problems are still unavailable.

Among these problems, a critical one is the control of item exposure rate. * This study is based on a brief summary of the doctoral dissertation entitled “Investigation of Item Exposure Control Methods in Computerized Adaptive Testing” prepared in the Educational Measurement and Evalua-tion Program, Hacettepe University.

a Corresponding author. Item exposure control for multidimensional computer adaptive testing under maximum likelihood and expected a posteriori estimation Alan R. Huebner1 & Chun Wang2 & Kari Quinlan1 & Lauren Seubert1 Published online: 20 October # Psychonomic Society, Inc.

Abstract Item bank stratification has been shown to be an. A comparative study of item exposure control methods in computerized adaptive testing. Journal of Educational Measurement, 40, 71 - Google Scholar | Crossref | ISI.

A Review of Item Exposure Control Strategies for Computerized Adaptive Testing Developed from to The Journal of Technology, Learning, and Assessment Volume 5, Number 8 May A publication of the Technology and Assessment Study Collaborative Caroline A. & Peter S. Lynch School of Education, Boston College The fourth method combines Robust CAT and the exposure control method.

Also because in practical testing situations, exposure control methods always have to be implemented to prevent that the most informative items in the pool become known (e.g.

Sympson and Hetter ; van der Linden and Veldkamp). Choosing a strategy for controlling the exposure of items to examinees has become an integral part of test development for computerized adaptive testing (CAT).

Item exposure can be controlled through the use of a variety of algorithms which modify the CAT item selection process. The effects of different specifications of practical constraints, such as content balancing and item exposure rate control, and the effects of using alternate item pools are examined.

One interesting finding from this study is that a large part of incomparability may be due to the change from number‐correct score‐based scoring to IRT. Second, the case of conditional item‐exposure control given the uncertainty of examinee's ability parameter is addressed.

Third, although rare for a well‐designed item pool, when applied in combination with the shadow‐test approach to adaptive testing the method may meet occasional infeasibility of the shadow‐test model. Computer Adaptive Testing (CAT), a form of computer-based testing that selects and administers items that match the examinee’s trait levels, can be shorter in length and maintain comparable or greater measurement precision than traditional fixed-length paper-and-pencil testing.

Computerized adaptive testing, or CAT for short, is the testing method that is used to administer the NCLEX. This method uses some of the best technology available today to decide which questions a candidate will see during their exam, and then to measure their competency based on their answers to those questions.

Computerized Multistage Testing: Theory and Applications covers the methodologies, underlying technology, and implementation aspects of this type of test design. The book discusses current scientific perspectives and practical considerations for each. Computerized Adaptive Testing: From Inquiry to Operation traces the development of Computerized Adaptive Testing (CAT) from its origins in the s to its integration with the Armed Services Vocational Aptitude Battery (ASVAB) in the s.

A paper-and-pencil version of the battery (P and P-ASVAB) has been used by the Defense Department since the s to measure the abilities of. Computerized adaptive testing (CAT) solves these problems by selecting items that directly target the observed level of ability of each student.

An effective CAT can quickly identify and administer items that are the most informative at a particular ability level, and each CAT is specifically matched to test taker. This is achieved by first. Computer adaptive testing (CAT), a contemporary method based on item response theory (IRT), 8 offers a promising approach for overcoming limitations of fixed-form balance measures and designing comprehensive, tailored, efficient, and responsive tests.

The performance of the method was examined via a simulation study and compared to existing methods of item selection and exposure control. Also, for the first time, maximum likelihood (MLE) and expected a posteriori (EAP) estimation of examinee ability were compared side by side in a multidimensional computer adaptive test.A comparative study of item exposure control methods in computerized adaptive testing.

Journal of Educational Measurement 40 71 Chen, Y., & Ankenman, R. D. (). Public Abstract. This study implemented two optimal pool design methods, the linear programming (LP) method and bin-and-union (BU) method, to evaluate the desired characteristics of item pools and compare the performances of the LP and BU pools in terms of psychometric properties under a fixed-length computerized adaptive test with the one-parameter logistic (1PL) discrete items.