IJͼ

Quality Control of IJͼ Assessments

Consistent results in high quality assessments

Policies
Share:

Quality Control Overview

The quality of IJͼ assessments is a top priority. It’s what sets IJͼ assessments apart and ensures that our tests remain the gold standard for assessing proficiency and performance. The IJͼ Assessment Program focuses on:

  • Creating high-quality assessments that fulfill test specifications,
  • Developing test items/prompts following a rigorous and iterative process, adhering to best practices and guidelines for test development,
  • Maintaining consistency and rigor throughout the test development process,
  • Reviewing assessment content for fairness, bias, and sensitivity,
  • Piloting new test prompts and field-testing new assessments to determine the extent to which they measure or elicit language as they are intended to do,
  • Ensuring that tesst remains valid and reliable throughout the duration of their use,
  • Maintaining and refreshing test content on a regular basis, and
  • Monitoring testing policies for administration and security.

IJͼ’s standards for establishing and maintaining high-quality IJͼ assessments is rigorous, ongoing, and consistently results in language assessments of the highest quality. Quality Control and Quality Assurance measures are integrated into every step of IJͼ processes-- from test development, tester and rater recruitment, certification, and calibration, to test review, item performance, and maintenance.

The Quality of IJͼ assessments is a priority. The IJͼ Assessment Program focuses on:

  • Creating high-quality assessments that fulfill test specifications
  • Developing test items/prompts following a rigorous and iterative process, adhering to best practices and guidelines for test development
  • Maintaining consistency and rigor throughout the test development process
  • Reviewing assessment content for fairness, bias, & sensitivity
  • Piloting new test prompts and field test new assessments to determine that they measure or elicit language as they are intended to do
  • Ensuring that the test remains valid and reliable throughout the duration of its use
  • Maintaining and refreshing test content
  • Monitoring testing policies for administration and security

IJͼ’s standards for establishing and maintaining high-quality IJͼ assessments is rigorous, ongoing, and consistently results in language assessments of the highest quality. Quality Control and measures are integrated into every step of IJͼ processes-- from test development, tester and rater recruitment, certification, and calibration to test review and maintenance.

Test Development

IJͼ assessments are developed according to a framework that addresses test design, its intended use, test takers, and the interpretation of test scores. IJͼ test development processes adhere to professional guidelines and best practices for test development such as the Standards for Educational and Psychological Testing (2014) and current technical and professional standards such as F2889 - 11(2020), Standard Practice for Assessing Language Proficiency.

IJͼ’s test development workflow is an intensive, multi-stage, multi-draft, and iterative process to ensure that the test content adheres to the test specifications.

IJͼ’s test development process includes:

  • Recruiting qualified individuals for test development teams: item/prompt writers, prompt selectors, reviewers, and target language experts
  • Providing test development workshops for team members
  • Maintaining standardized test development and review guidelines
  • Monitoring and supporting online test development workflow processes
  • Ensuring fairness and conducting bias & sensitivity reviews
  • Pilot testing new item types and field testing new assessments to determine that they measure or elicit language as they are intended to do.

Quality Control Areas

Quality Control (QC) of IJͼ assessments occurs throughout the lifetime of an assessment. IJͼ assessments are monitored throughout the duration of their use to ensure consistent evaluation and measurement of examinee performance and to ensure that the assessments continue to align with the validity and reliability of the original assessments. Quality Control areas focus on the following:

  • Monitoring frequency of topic and prompt exposure
  • Reviewing item performance on a regular basis
  • Conducting reliability and validation studies
  • Requiring periodic independent psychometric evaluations of assessments
  • Reviewing test taker experiences and feedback
  • Following test maintenance and refreshment plans
  • Implementing security-breach protocols and reviews
  • Collaborating with the IJͼ Quality Assurance Program to coordinate psychometric reviews of IJͼ assessments

Assessment Policies

In order to provide assessments that result in fair and valid measurements, IJͼ Assessment Policies serve as guidelines for test development, administration, scoring, and maintenance and refreshment of IJͼ assessments. These policies ensure that the assessments elicit and measure what they are intended to measure, provide equitable and fair access to the assessments, maintain test taker personal information and scores, and maintain the integrity of the proficiency-based nature of IJͼ assessments.

Accommodations Policy

IJͼ is strongly committed to facilitating equal access to its assessments and accommodations to all individuals seeking to assess world language abilities. To this end, IJͼ assessments are available with accommodations that are appropriate for each candidate’s conditions and needs. IJͼ’s Accommodations Policy serves as guidance to Language Testing International (LTI), exclusive licensee of IJͼ assessments.

Assessment Integrity Policy

Test takers are required to review a Test Taker Integrity Policy, prior to taking an IJͼ assessment. IJͼ proficiency-based assessments are designed to evaluate spontaneous, unrehearsed language. Any responses provided by the test taker are assessed by certified IJͼ raters with that in mind; likewise, the machine-scored assessments account for this in the respective scoring algorithms. IJͼ’s Assessment Integrity Policy outlines test taker actions that undermine the intended use of IJͼ’s assessments of language proficiency and outlines the consequences of such actions.

Record Retention and Test Taker Confidentiality Policy

Test taker confidentiality and secure record retention is a priority for the IJͼ Assessment Program. Based on IJͼ’s Record Retention and Test Taker Confidentiality Policy serves as guidance for Language Testing International (LTI), exclusive licensee of IJͼ, for their administration and scoring of IJͼ assessments.

Re-Test Policy

Because IJͼ assessments are proficiency-based, language use and language learning over time are integral factors for the progression of proficiency. As such, IJͼ does not generally permit test takers to retake an assessment within 90 days of a test administration. In addition, familiarization with the content of the assessment creates a test effect, which also, affects the assessment of proficiency. In order to provide an accurate measurement of language proficiency, an extended time period must pass before two successive administrations of an assessment. IJͼ’s Re-Test Policy provides information about re-testing and exceptions to the Re-Test Policy.

Test Maintenance and Refresh Policy

IJͼ’s Test Maintenance and Refresh Policy outlines the maintenance and refresh plans to ensure that the test remains valid and reliable throughout the duration of its use. Maintenance documents include the specifications for evaluation of test performance and review processes for item performance and tester/rater certification and calibration. Refreshment documents include the process and metrics for refreshing test content. Validity and reliability evidence documents are retained, and retention goes as far back as the original validity and reliability review.

Test Security Policy

Test security is high priority for IJͼ in order to protect the integrity, validity, and content of its assessments. IJͼ’s Test Security Policy serves as guidance to Language Testing International (LTI), exclusive licensee of IJͼ assessments, for security measures to safeguard IJͼ assessments. In the event of a security breach, the exposed prompt(s) or item(s) are pulled from the assessment and new items.