INTERNATIONAL GUIDELINES FOR TEST USE Carlos A. Almenara, PhD Online and Offline Resources In Psychological Assessment PSY494P122 (April 29th 2015) Standards for Educational and Psychological Testing 2 ¨The Standards for Educational and Psychological Testing have been used as an authoritative source in USA since 1954. ¨ ¨ ¨These and other standards are used in countries were standards have not been established. Oakland, T. (2009). How universal are test development and use? In E. L. Grigorenko (Ed.), Multicultural psychoeducational assessment (pp. 1–40). New York, NY: Springer. Standards for Educational and Psychological Testing 3 ¨Czech translation is available in MUNI ¨ ¨ Standardy pro pedagogické a psychologické testování. (2001). (1. české vyd., 192 s.) Praha: Testcentrum. Standards for Educational and Psychological Testing 4 ¨Last edition (2014) is available in MUNI ¨ ¨ AERA, APA, & NCME (2014). Standards for educational and psychological testing. Washington DC: American Educational Research Association. 5 FURTHER INFORMATION 6 ¨ ¨ http://www.apa.org/science/programs/testing/ 7 ¨ ¨ http://www.apa.org/science/programs/testing/ 8 GUIDELINES IN DIFFERENT COUNTRIES Guidelines in different countries 9 ¨USA: The American Educational Research Association (AERA) , American Psychological Association (APA), National Council on Measurement in Education (NCME), in collaboration since 1966. ¨ ¨AUSTRALIA: The Australian Psychological Society (APS) ¨ ¨UK: The British Psychological Society (BPS) ¨ ¨CANADA: The Canadian Psychological Association (CPA) 10 INTERNATIONAL GUIDELINES FOR TEST USE (ITC, 2000) International Guidelines for Test Use 11 ¨Focus on: ¨Good test use ¨Encouraging best practice in assessment ¨ ¨Purpose: ¨Uniformity in the quality of tests adapted for use across different cultures and languages. International Test Commission (ITC) (2000). International Guidelines for Test Use. Downloaded electronically on April 22th from http://www.intestcom.org/files/guideline_test_use.pdf International Guidelines for Test Use 12 ¨REASONS: ¨Degree of control over test use and its consequences. ¨Patterns of access, in terms of the rights to purchase or use test materials. ¨Internet access (copyright violation) ¨Job applicants from other countries. ¨Distance- or remote-assessment. International Test Commission (ITC) (2000). International Guidelines for Test Use. Downloaded electronically on April 22th from http://www.intestcom.org/files/guideline_test_use.pdf International Guidelines for Test Use 13 ¨LONG TERM AIMS: ¨Competencies needed by test users, related to: • •professional and ethical standards in testing, •rights of the test taker and other parties involved in the testing process, •choice and evaluation of alternative tests, •test administration, scoring and interpretation, •report writing and feedback. International Test Commission (ITC) (2000). International Guidelines for Test Use. Downloaded electronically on April 22th from http://www.intestcom.org/files/guideline_test_use.pdf 14 DEVELOPMENT OF ITC GUIDELINES International Guidelines for Test Use 15 ¨DEVELOPMENT: ¨ ¨14 main headings from primary sources ¨ ¨Draft of the framework document ¨ ¨ITC Workshop (Dublin - July, 1997) ¨ ¨Several revisions and consultations International Test Commission (ITC) (2000). International Guidelines for Test Use. Downloaded electronically on April 22th from http://www.intestcom.org/files/guideline_test_use.pdf International Guidelines for Test Use 16 ¨CORE COMPETENCES: ¨ ¨Professional and ethical standards of good practice. ¨ ¨Knowledge, understanding and skills (process). ¨ ¨Knowledge and understanding (to inform). International Test Commission (ITC) (2000). International Guidelines for Test Use. Downloaded electronically on April 22th from http://www.intestcom.org/files/guideline_test_use.pdf International Guidelines for Test Use 17 ¨Take responsibility for ethical test use: vAct in a professional and ethical manner vEnsure they have the competence to use tests vTake responsibility for their use of tests vEnsure that test materials are kept securely vEnsure that test results are treated confidentially International Test Commission (ITC) (2000). International Guidelines for Test Use. Downloaded electronically on April 22th from http://www.intestcom.org/files/guideline_test_use.pdf International Guidelines for Test Use 18 ¨Follow good practice in the use of tests: vEvaluate the potential utility of testing in an assessment situation vChoose technically sound tests appropriate for the situation vGive due consideration to issues of fairness in testing vMake necessary preparations for the testing session vAdminister the tests properly vScore and analyze test results accurately vInterpret results appropriately vCommunicate the results clearly and accurately to relevant others vReview the appropriateness of the test and its use International Test Commission (ITC) (2000). International Guidelines for Test Use. Downloaded electronically on April 22th from http://www.intestcom.org/files/guideline_test_use.pdf 19 EUROPEAN TEST USER STANDARDS European Test User Standards 20 European Association of Work and Organizational Psychology (EAWOP) + European Federation of Psychologists’ Associations (EFPA) http://www.eawop.org/background European Test User Standards 21 The EFPA-EAWOP Working Group modified and expanded the ITC Guidelines to provide the basis for qualifications and good practices. http://www.efpa.eu/professional-development/assessment European Test User Standards 22 ¨ ¨The emphasis of these standards is for good practice. ¨ ¨They are aimed at people who use tests and testing as part of their work as a practitioner. ¨ ¨These are not standards for academic researchers or methodologists in the area of testing. http://www.efpa.eu/professional-development/assessment EFPA Test User Qualifications 23 ¨Assistant Test User - Test administrator (EFPA Level 1) v vUses specific tests in well-defined and constrained contexts vIs under the supervision of a more experienced test user vOperates within organizational policies and directives vNot able to make choices about tests to be used vNot able to provide interpretations or scores vAwareness http://www.efpa.eu/professional-development/assessment EFPA Test User Qualifications 24 ¨Test User (EFPA Level 2) v vUnderstands technical psychometric qualities for their use vCan work independently in specified, limited settings vCan interpret a limited range of specific tests vNot able to make choices or interpretations (beyond those on which is qualified) http://www.efpa.eu/professional-development/assessment EFPA Test User Qualifications 25 ¨Specialist in Tests and Testing (EFPA Level 3) v vProvides advice and consultancy on testing vTrain others in test use vTest construction vProvides expert evidence relating test use in court cases vDoesn’t require sufficient expertise in methodology to construct and develop tests, but is covered by expertise. http://www.efpa.eu/professional-development/assessment EFPA Test User Qualifications 26 ¨Assessment of knowledge, skills, and competence v vKnowledge can be assessed using conventional summative assessments (i.e. written examinations) vSkills should be assessed through observation of performance in real or role-play situations. vAssessment of competence should be carried out by a suitability qualified supervisor. http://www.efpa.eu/professional-development/assessment 27 QUESTIONS? NEXT CLASS 28 ¨READING: Sources of knowledge and perspectives (Willis, Inman, & Valenti, 2010) Knowledge synthesis (Tricco, Tetzlaff, & Moher, 2013) Optional: Knowledge translation (KT) (McKibbon & Lokker, 2013) How to read a paper (Greenhalgh, 2014)