Auditor General fun

The Ontario AG's report was released Monday, and contained two sections of importance to public education in Ontario. One has received some reporting (thanks to Moira Macdonald) whereas the other may have been mentioned but received little to no coverage.
The auditor general examined, as part of a cyclical review of all areas of government, the Education Quality and Accountability Office, as well as the Ministry of Education's Literacy and Numeracy Secretariat. The former definitely received a more positive review than the latter. I was personally contacted a few times by the EQAO both in advance of and after the release of the AG's report Monday-- the opportunities to report on it in print for me are few, but this is an appropriate venue to do so. The AG's report is overall favourable to the EQAO, noting it does its job -- testing students against the established Ontario curriculum and reporting those results in a fair and efficient manner. It's also done so while reducing costs by 20% while providing the same level of service. The office has four main recommendations for the EQAO, which I've copied and pasted below without the EQAO responses. I realize this lengthens this post considerably, but it's important to have them here verbatim.

Recommendation 1: To improve the Education Quality and Accountability Office’s (EQAO’s) test development and administration process and to ensure that student assessments continue to be reliable and objective and that all students are given the opportunity to demonstrate their competence, the EQAO should:
• highlight to principals and teachers any significant changes in the compliance requirements outlined in the guides to administer EQAO testing;
• improve the process for selecting the schools visited by quality assurance monitors to ensure that all school boards and large private schools are periodically monitored;
• assess the equity of including exempt students in the overall assessment results as having not met the provincial standard; and
• identify schools and school boards where the number of exempt students appears to be relatively high and follow up to ensure that exemptions are justified.
Recommendation 2: To improve the assessment marking process to ensure that results continue to be valid, consistent, and reliable, the Education Quality and Accountability Office should:
• consider adopting on-line training for assessment markers;
• examine different methods to increase the number of validity reads for each marker, especially early in the marking process; and
• consider implementing supervisory backreading to help improve marker accuracy.
Recommendation 3: To ensure that assessment results continue to be reliable, consistent, and valid, the Education Quality and Accountability Office (EQAO) should enhance its quality assurance procedures by:
• implementing a formal complaints process to help determine if there are any trends and to identify potential actions that could prevent non-compliance with assessment guidelines or student cheating;
• considering more complete disclosure when test results at a particular school are withheld as a deterrent against non-compliance with assessment guidelines;
• outlining in its administration guides potential penalties for violating EQAO policy;
• tailoring its quality assurance processes to address unique risks associated with different assessments;
• reviewing Grade 9 applied mathematics results to assess whether incorporating EQAO results into the student’s final markis effective in motivating students and, if so, suggest a more consistent approach; and
• investigating any abnormally large variations in school assessment results from year to year and ensuring that they are justified.
Recommendation 4: To further improve its policies and processes and the procedures designed to produce accurate and reliable reports that can be used to improve student performance, the Education Quality and Accountability Office (EQAO) should:
• consider formalizing its pilot initiative to provide more open-ended questions for principals, teachers, and students to obtain better feedback on any concerns with the assessment process and ways to improve it;
• develop a more formal outreach strategy to give all schools and school boards an opportunity to gain further insight into the value of EQAO data and how it can be used to improve student learning; and
• increase the understanding of parents and the general public of how the assessment process enhances student learning.

The EQAO's response to most recommendations demonstrates either that it is making changes to meet the recommendations for pending assessments or that it will take the matter under advisement. Under the third set of recommendations I would love to see those come into play-- as an example, today one has to go to the so-called blue pages at the Ontario College of Teachers to see where some have flaunted test procedures.
Compare that to the LNS portion of the audit, where five recommendations (which I won't post here or this post would start inducing somnolence) point to a need for greater control and accountability. The AG notes both dollars going out the door that aren't adequately tracked or accounted for, as well as recommending the LNS does a better job of tying its efforts into EQ and report card assessments to ensure the impacts of its programs are effectively measured and reported. I particularly liked the recommendation stating all board and school improvement plans should be a matter of public record. These are no doubt full of eduspeak and other terms, however a competent translator (such as a parents' groups, journalists, etc.) could ensure those with interest understood what each says.
Overall, sadly, both these audits got overshadowed by the sexier sections of the report on social program spending and accountability. I lamented this with an EQAO staffer on Monday, noting they got out-sexed in the report and that few media would bother with a 'good news' story on the office's audit when there was waste and scandal to report.