Envoy’s AI Rater gives you an accurate picture of a student’s English language proficiency. Sure, you’ve been able to automate the marking of multiple-choice questions for years. But now, Envoy’s AI can test productive skills and open-ended questions. This giant leap forward in language testing can give you confidence that you’re comprehensively assessing a student's overall English proficiency.

The Envoy AI Rater can assess a student's performance on written and spoken open-ended questions. We’re not talking about reading a sentence out loud. These open-ended questions simulate real-life English communication and require extended responses from students.

The Envoy AI Rater assesses a student's performance on the following criteria:

• Speaking tasks: Grammar, vocabulary, fluency, pronunciation, and cohesion.

• Writing tasks: Grammar, vocabulary, spelling, punctuation, organisation, and topic development.

All test scores align with the Common European Framework of Reference for Languages (CEFR) levels, so you can be confident that they give a well-rounded picture of a student's proficiency.

How does it align with CEFR?

AI models rely on data. Envoy’s AI Rater is trained on vast amounts of authentic, genuine speech and writing samples from various types and topics. This diverse training data means it can assess proficiency over multiple tasks and levels. CEFR rating experts tag the training data so the model can accurately evaluate performance based on the CEFR can-do statements.

Within 2 hours of test completion, you get a report detailing your student’s performance. And because of open-ended grading and alignment to CEFR, you get a complete picture of a student's English proficiency. The report is full of valuable insights for you to use when placing students in classes, checking progress during a course, or assessing skills at course completion.

High-quality data and unbiased results are the foundation of the Envoy AI Rater. Here’s how high-quality data and responsible AI best practices ensure accurate test scores.

For AI to be responsible and fair, the training data needs to be representative of the user population. So, Envoy’s algorithms are trained on a vast range of data from students worldwide. This means training data includes a variety of first languages, accents, ages, and other demographics. This diverse training data reflects the diversity of students who take the test, leading to less bias in the models.

Diversity doesn’t stop there, though. Data-tagging team members come from diverse backgrounds, too. We are committed to ensuring fairness throughout the model training process. Independent raters who score training data are blind to demographic data, and all data sets are rated by multiple groups of experts.

To further ensure fairness, our algorithms are blind to demographic data during the scoring process. And the algorithms rate each language criteria separately. The Envoy AI Rater uses separate models for each criterion to minimise the influence one aspect of performance has on another.

Finally, Envoy’s algorithms are constantly training and evolving. When new data from new demographics is created, the model is trained again. Also, human raters and expert linguists continuously assess the quality of the output for consistency. They look for issues with accuracy or fairness, recommend changes, and help refine the algorithms.

As with any test, there may be students who try to cheat. Envoy has built-in security features to ensure the integrity of the tests.

Envoy combines fairness and accuracy with AI-powered anti-cheating measures and human expertise to provide a test you can rely on. Here’s how.

Open-ended questions don’t just assess a student's full proficiency. They also make it harder to cheat on the tests. Students can trick algorithms with things like multiple repetitions, loud or fast speech, or pre-prepared answers. Because of Envoy’s natural speech training, these don’t work. The algorithm knows when they’re giving answers unrelated to the question, and it can sift through any other tricks to find the actual proficiency of your students.

Proctoring measures are native to the Envoy test platform. Audio and visual checks throughout the test mean external help or looking away from the screen are easily spotted. Envoy prevents copy and pasting and can discontinue a test if a student leaves the test tab in their browser. And if the AI flags any inappropriate or suspicious behaviour, human raters at Envoy review the test for you prior to results being released.