Goodbye to the strict
1, 2, 3, 4 QC scale!
And Hello to a sophisticated new grading spectrum with weighted scoring!
The New Review Scoring System is everything you’ve asked for and more.
We have combined new design features, weighted error types, fractional numbers, and weighted transcript length to enhance the precision of your transcript scores!
Welcome to the New Review Interface and Scoring Algorithm for MTurk Transcription HITs.
Here’s what you need to know!
- New Review Interface! New tooltips guide you through the reviewing process and help you choose the most correct error types!
- Weighted Error Types! Major, medium, minor and negligible errors are assigned different point values for smarter insight on transcript quality.
- Fractional Numbers! Scores calculated to the hundredth decimal now offer greater precision on Review Scoring.
- Weighted Transcription Length! Word count will now considered in your score for built-in leniency for longer, more difficult HITs.
The New Review Interface
Take a look!
Use new tooltips to help you choose the correct error types.
Don’t forget to leave comments to help your peers improve their performance!
The New Review Scoring Algorithm
This is where the magic happens!
The scoring algorithm takes the reviewer’s inputs and considers error types and weights, transcript length and determines the final QC score for the transcript.
Here’s an example:
- 1 missed or incorrect disfluency in 45 words =
3 QC scorenow 4.0 QC score
What does this mean for you?
The new Review Scoring has significant impact on both Transcribers and Reviewers!
- Fairer QC Scores!
Minor errors no longer tank your QC score, so you can maintain an awesome QC score for promotions and bonuses! - Review scores will now reflect true transcript quality!
Your transcript scores are no longer confined to a strict 4 point scale, now enjoy a full spectrum of fractional scores. This means that you’ll see a wide array of scores between 3 and 4 including 3.2, 3.5, 3.8 on any given HIT. - Easier Reviewing!
Improved interface design clarifies what error types should be marked. - More lenient reviews!
Weighted reviewing based on error types and transcript length ensure that Review Scores are as fair as possible!
Next steps!
We’re making the Review System even better with a brand new Transcription Review team!
That’s right, no more bad reviewers!
Over the next couple of weeks we will release new Clean Verbatim Guidelines and keep only the best, most qualified Transcription Review workers to grade your transcripts!
This is just one of several improvements coming your way! Stay tuned to the CrowdSurf Blog for Guideline and Review updates!
Log in now to see Review Scoring in action!