Principled Assessment Design

Principled Assessment Design (PAD) is a movement in the field of the field of educational measurement that seeks to create tests that are better aligned to their purported constructs and to provide better evidence of that alignment and for the appropriate uses of tests.^[1] It is not a particular procedure or technique, but rather a set of principles that some approaches to test design and development are built upon. Individual principled design approaches do not necessarily have to subscribe to every one of the principles, but they must be compatible with all of the principles.

The different approaches developed in the late 1990s or in the 21st century. They each attempt to improve upon weaknesses of longstanding traditional test design and development practice, and are generally oriented towards large scale standardized assessment, including educational measurement, psychological measurement and even professional licensure/certification. While all of these approaches have been developed in response to the same testing landscape, same history of research and practice in assessment and over a relatively short period time, they were not developed together.

Some more recent PAD approaches build on older PAD approaches, but the use of the term principled assessment design is itself rather new. Moreover, some prefer the term principled assessment development, some prefer principled approaches to assessment design. All, however, refer to elements of both test design and development, most also include test operations (e.g., test administration, scoring). All consider test use, which The Standards for Educational and Psychological Testing sets as the focus of test validity.

The influence of PAD approaches, especially ECD, is now being seen in assessment projects that are less mindful of particular models or approaches to test design and development.^[2] While the future of any of these individual approaches is, of course, unknowable, the influence of the principles of PAD are undeniable.

Tenets of Principled Assessment Design

Clearly defined constructs and then assessment targets
Clarity of intended use
Theory of cognition Learning, thinking
Matching psychometric models to test use
Provide bases for further item development

Examples of Principled Assessment Design Approaches

Assessment Engineering (AE)

Berkeley Evaluation and Assessment Center (BEAR) Assessment System

Cognitive Design Systems (CDS)

Evidence Centered Design (ECD)

Huff, Nichols & Schneider's Principled Assessment Design (HNS PAD)

ECD-based, and most prominent in the Fifth edition of Educational Measurement^[3] ===Rigorous Test Development (RTD)=== ECD-based, focusing on content development.and Rigorous Test Development.^[4]

Huff, Nichols & Schneider's Principled Assessment Design (HNS PAD)

ECD-based, and most prominent in the Fifth edition of Educational Measurement^[5]

References

^ Ferrara, Steve; Lai, Emily; Reilly, Amy; Nichols, Paul (2016). "Principled approaches to assessment design, development, and implementation". In Rupp, Andre; Leighton, Jacqueline (eds.). The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications. Wiley-Blackwell. pp. 41–74. ISBN 978-1-118-95657-1.
^ Suárez-Álvarez, Javier; Caroline, Prendergast; Hannah, Smith; Wan, Siyu (2024). "Promoting Social Responsibility in Educational Research" (PDF). 55th Annual Conference Program. Northeast Educational Research Program. Trumbull Connecticut: NERA. Retrieved October 14, 2024.
^ Huff, Kristen; Nichols, Paul; Schneider, Christina (2024). Cook, Linda; Pitoniak, Mary (eds.). Designing and developing educational assessments. American Council on Education.
^ Wine, Marjorie; Hoffman, Alexander (2024). Rigorous Test Development: The New Core Principles and the Pillar Practices. Annual meeting of the American Educational Research Association. Philadelphia, Pennsylvania. Retrieved October 14, 2024.
^ Huff, Kristen; Nichols, Paul; Schneider, Christina (2024). Cook, Linda; Pitoniak, Mary (eds.). Designing and developing educational assessments. American Council on Education.

[1] Ferrara, Steve; Lai, Emily; Reilly, Amy; Nichols, Paul (2016). "Principled approaches to assessment design, development, and implementation". In Rupp, Andre; Leighton, Jacqueline (eds.). The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications. Wiley-Blackwell. pp. 41–74. ISBN 978-1-118-95657-1.

[2] Suárez-Álvarez, Javier; Caroline, Prendergast; Hannah, Smith; Wan, Siyu (2024). "Promoting Social Responsibility in Educational Research" (PDF). 55th Annual Conference Program. Northeast Educational Research Program. Trumbull Connecticut: NERA. Retrieved October 14, 2024.

[3] Huff, Kristen; Nichols, Paul; Schneider, Christina (2024). Cook, Linda; Pitoniak, Mary (eds.). Designing and developing educational assessments. American Council on Education.

[4] Wine, Marjorie; Hoffman, Alexander (2024). Rigorous Test Development: The New Core Principles and the Pillar Practices. Annual meeting of the American Educational Research Association. Philadelphia, Pennsylvania. Retrieved October 14, 2024.

[5] Huff, Kristen; Nichols, Paul; Schneider, Christina (2024). Cook, Linda; Pitoniak, Mary (eds.). Designing and developing educational assessments. American Council on Education.

[1]

[2]

[3]

[4]

[5]