Psychological Test Psychometric Characteristics

Psyc 3110

You will write a 5 to 7 page (plus title page and references) APA-style, evaluating the psychometric characteristics of a psychological test. To do this, you first find a public domain test – that is, as with your first workshop assignment, you will find a test that is included in a journal article. Typically, when a test is introduced in a journal article, the authors provide some information about how they came up with items, as well as evidence regarding the reliability, validity, and maybe even factor structure. Importantly, you are going to choose a test which has also been used in at least two other journal articles. Thus, not only will you report on what the original set of authors said about the test, but you will have access to some kind of subsequent information. That is, in the other articles, is there any information given about the test’s reliability? Did it predict or correlate with other study variables in a way that seems indicative of good validity?

Describe:

What construct is being measured? Why is it important to measure? How is this measure different from previous attempts to measure the construct? What is the purpose of the test (e.g., research instrument, making diagnostic decisions, screening tool, etc.)

Test construction: How did the authors generate items? Did the authors narrow down items from a larger item pool? If so, how?

Test bias and response bias: Who is the intended target group for this test (e.g., men, women, children, students? Is it a clinical tool? Research tool?)? Was the test administered to an appropriate group of participants in the validation process? For example, it might be problematic if a test is intended for community members with eating disorders but administered only to female university students. Is the test vulnerable to response bias? For example, does it include reverse-keyed items? Are people likely to misrepresent their levels of the characteristic? Explain.

Factor structure: What is the factor structure? Is it reported here or in a follow-up study?

Reliability: What kinds of reliability did the authors report? What about follow-up studies of this test?

Validity: What kinds of validity did the authors report? What about follow-up studies that include this test?

Use your critical thinking to make evaluative comments. Are the reliability and validity reasonable? Should the authors have done more to evaluate the test? Is the original participant group representative of the intended target audience? Is the factor structure appropriate? Are the items appropriate? Is the response scale appropriate? Do you see room for improvement? Does the other study (or studies) you report support the usefulness of the test? Your critical conclusion should clearly summarize these strengths and weaknesses, as well as identifying what more could be done.

Guidelines:

Both the copy you submit through SafeAssign on Blackboard and the hard copy are due at the beginning of class on Dec. 1. Late deductions are as described in the syllabus.

Five to seven typed, double-spaced pages plus title page and references.

Use APA style throughout. Make extensive use of your APA handbook (available in the Trent Durham library if you don’t own your own copy).

Clear introduction, stating what test you are evaluating and what you will conclude. I want to know exactly where  going at the end of the first paragraph.

Use subheadings.

Don’t quote except unless there is a highly compelling reason to do so. In general, it is best to paraphrase or summarize. (Note: an exception to this rule is that you will probably be providing sample items in describing the test, and that is very appropriate).

You will be citing and referencing at least three journal articles (the original test construction/validation article plus the two follow-ups) and your textbook (I’m sure you will want to refer to various guidelines and constructs from the text).

Adopt a formal, objective tone with no slang and no contractions.

Do not use personal opinion, but do use critical thinking. By the time I finish reading, I should have a very clear understanding of the strengths and weaknesses of the psychological test you are evaluating.

Remember that we refer to authors’ arguments and findings but not their “beliefs.”