“Every time we went through a calibration session, we realized how different our graders were grading. In addition, calibrations were our only opportunity to meet with graders and get their thoughts about the rubric, which made it difficult to ensure alignment.”
“We take our calibrations very seriously and involve a large number of stakeholders, including members of the Customer Success team. Even then, the sessions were still not effective at helping us make our team more consistent.”