Principles of Testing, Validity, Reliability and the Table of Specifications

1. Determine the purpose (diagnostic, formative, summative, selection).
2. Identify learning objectives / SLOs to be tested.
3. Prepare a table of specifications (blueprint).
4. Select item types suited to each objective (MCQ, short answer, essay, performance).
5. Write items following item-writing guidelines; prepare more than needed.
6. Review items (content experts, language, bias).
7. Assemble the test – group by item type, easy to difficult, clear instructions, marks shown.
8. Administer under standard conditions.
9. Score using a key or rubric.
10. Analyse items and interpret results; revise the item bank.

Characteristics of a good test: validity, reliability, objectivity, usability (practicality), discrimination, comprehensiveness, appropriate difficulty.

Validity is the degree to which a test measures what it is intended to measure, and the appropriateness of the interpretations made from scores. It is the most important quality of a test.

• Content validity – the test adequately samples the content and objectives taught. Ensured by a table of specifications. Judged by experts.
• Construct validity – the test measures the intended theoretical trait (e.g. creativity, reading comprehension).
• Criterion-related validity – scores correlate with an external criterion:
  • Concurrent – criterion measured at the same time (new test vs established test).
  • Predictive – criterion measured later (entry test predicting first-year GPA).
• Face validity – the test appears to measure what it should (superficial, judged by test-takers).

Threats: unclear directions, difficult vocabulary, ambiguous items, too few items, poor sampling of content, cheating, items testing only recall when objectives demand application.

Reliability is the consistency of scores – across time, forms, items or scorers. A reliability coefficient ranges from 0 to 1.

• Test–retest – same test given twice; measures stability.
• Parallel / alternate forms – two equivalent forms; measures equivalence.
• Split-half – one administration, test split into two halves (e.g. odd/even); corrected with the Spearman–Brown formula.
• Internal consistency – KR-20 / KR-21 (dichotomous items), Cronbach's alpha.
• Inter-rater (scorer) reliability – agreement between different scorers; important for essays/CRQs.

Factors: longer tests are more reliable, objective scoring, homogeneous items, a wider spread of ability, clear instructions; guessing and ambiguous items lower reliability.

Relationship: a test can be reliable without being valid, but it cannot be valid unless it is reliable. Reliability is necessary but not sufficient for validity. (A clock that is always 10 minutes fast is reliable but not valid.)

A two-way chart that shows the content areas (rows) against cognitive levels (columns, e.g. Remember, Understand, Apply, Analyze), with the number of items or marks in each cell. It ensures content validity and balance.

Steps:

1. List the units/topics.
2. Decide weight for each topic (e.g. by teaching time or importance).
3. Decide weight for each cognitive level.
4. Fix the total number of items/marks.
5. Distribute items into cells: items per cell ≈ total × topic % × level %.

Example (40 items): Topic A 50%, Topic B 30%, Topic C 20%; Remember 30%, Understand 40%, Apply 30%. Topic A = 20 items → 6 Remember, 8 Understand, 6 Apply.