PC News - November 2016
Improving education outcomes through evidence-based policy and practice
A new Commission draft report recommends improvements to the national education evidence base to enhance policy development and outcomes in school and early childhood education.
Australian governments have committed to goals emphasising the importance of excellence and equity in the education system. Like other developed nations, Australia has sought to achieve these goals through increased investment in education, and by implementing reforms focused on monitoring outcomes, performance benchmarking and reporting against national standards.
However, Australian students’ performance on national and international assessments has stalled or, in some cases, declined. Although resourcing and accountability matter, there is growing consensus that they alone are insufficient to improve outcomes.
In March 2016, the Australian Government asked the Commission to investigate improving the national education evidence base to better inform policy development and improve education outcomes in early childhood and school education.
In its draft report, released in September 2016, the Commission argues that the path to better education outcomes lies in identifying and evaluating the policies, programs and teaching practices that work best, and applying this evidence across the nation’s early childhood and school education systems.
What is a national education evidence base?
An education evidence base is more than an accumulation of data in a single collection or data ‘warehouse’ (figure 1). It should:
- meet the diverse needs of decision makers throughout the education system
- provide high-quality data and evidence to inform decisions
- drive improved student achievement through performance monitoring, evaluation of what works best, dissemination of evidence and application of that evidence
- generate benefits in excess of the costs incurred in collecting and processing data and in creating, sharing and applying evidence.
The national education evidence base should have two capabilities:
- A ‘top-down’ capability, for monitoring, benchmarking and assessing performance, as well as promoting transparency and accountability, and informing resource allocation.
- A ‘bottom-up’ capability that evaluates the effectiveness of education policies, programs and teaching practices, enabling systematic identification of ways to improve student achievement. Bottom-up evaluation includes prioritising, developing, testing, reporting and applying what works best in education.
Figure 1: An Effective Education Evidence Base
Action is required to improve evidence creation
A lot of data are collected to monitor and report on the outcomes of students, schools and early childhood education. But there is relatively little high quality evidence about which policies, programs and teaching practices work best in Australia (and few systematic processes to evaluate the initiatives that are in place). This is the largest gap in the Australian education evidence base. To address it, we need to build our bottom-up capability.
The Commission recognises that teachers have the greatest impact on student performance, after accounting for the characteristics of students themselves.
Looking within the classroom, particularly at teaching practices, can improve outcomes across the education sector. We also know there are some schools whose students perform better than expected compared with similar schools. Governments should be lifting the bonnet on those schools to find out what they are doing, and carefully evaluating whether their methods can be applied systematically across other schools.
Ideally, evaluations of what works best should use rigorous research techniques, particularly trials run in classrooms and schools that compare the outcomes for children taught using a certain program or practice with those of similar children who don’t receive that intervention.
Government leadership is needed
In Australia’s federated system, responsibility for the funding and delivery of early childhood and school education is spread between the Australian, state and territory governments.
All governments must take a shared and co-operative approach to implementing a high-quality and relevant education evidence base, particularly focussing on the bottom-up capability, by:
- putting in place a new Education Agreement (building on previous agreements) that defines the objectives of, and framework for, commissioning and applying evaluative research about what works best
- assigning an institution to be responsible for the implementation of the evaluative research framework, which is accountable to, and funded by, all governments
- specifying the assigned institution governance arrangements, functions and operations.
The institution would ensure that:
- national research priorities are developed
- high-quality research is commissioned using rigorous processes, including provision of guidelines to applicants about the nature of research that will be considered
- research quality is verified
- capacity in high-quality education research is fostered.
To distil and communicate the findings of this research, a central repository of trusted, high-quality evidence, including resources to support practitioners, is needed.
But simply creating evidence and making it available to education professionals is not enough.
Evidence only leads to improved education outcomes if it is used to inform decision making and changes the behaviour of practitioners. Research also needs to focus on how evidence can most effectively be translated into improvements in practice.
There is also scope to improve the collection and use of existing data
Better use could also be made of the data that are already collected. Access to data for research in the public interest could be improved through changes to privacy protections, and if those providing data were asked to consent to its use for research purposes when data are collected.
Smoother processes for linking information from different datasets would also help, and the use of linked data could reduce the costs of evaluations of what works best.
Schools and early childhood education face a serious burden in collecting data, which could be reduced by better use of existing data and more cost-effective collection. Use of fit-for-purpose collections is important.
While national data are needed for some monitoring and benchmarking purposes and funding, data from samples of students (which impose a lower collection burden) are sufficient for other purposes.
The Commission also recommends that data collection agencies remove duplication in data collection and processing, and that data providers are given plenty of lead time to update their systems when changes to existing collections need to be made.
Summary of key draft recommendations*
Developing the evidence base
The Australian, state and territory governments should pursue a national policy effort to develop a high-quality and relevant Australian evidence base about what works best to improve school and early childhood education outcomes, including: development of research priorities; commissioning of high-quality education research; adoption of rigorous research quality control processes; dissemination of high-quality evidence; and development of researcher capacity.
The Australian, state and territory governments should task the COAG Education Council to develop a new Education Agreement, which would build on prior agreements and define the:
- objectives, and nature of the research to be undertaken in the bottom-up evaluation of what works
- evidentiary standards or frameworks to be applied, including assessment of cost effectiveness
- requirement for translation of evidence into guidelines accessible by schools, early childhood education and care services and teachers.
Governments should also request the Education Council to assign an institution to be responsible and accountable for implementation of these functions and the evidence base. The assigned institution’s governance arrangements, functions and operations should include promoting a culture of using the evidence base by policy makers and educators.
Burden of data collection, privacy and consent
To reduce administration costs and compliance burden, agencies collecting data should only collect fit-for-purpose data (for example, sample data in situations where census data are unnecessary); remove duplication in data collection and processing; and avoid frequent changes to reporting requirements.
The Australian Government should amend the Privacy Act 1998 (Cwlth) to extend arrangements for collection, use or disclosure of personal information without consent in the area of health and medical research to cover public interest research more generally. State and territory governments should seek to have consistent legislation and to establish policies where the onus is on data custodians to share data.
The ACT Government should enact in its privacy law an exception to cover public interest research. In Western Australia and South Australia where there is not a legislated privacy regime, their privacy arrangements should reflect a similar public interest research exception.
Agencies responsible for education data collections should amend their processes for collecting personal information from parents/guardians to incorporate formal consent and notification procedures regarding the use and disclosure of personal information at the initial point of collection.
Data and data gaps
The agencies that conduct the Longitudinal Study of Australian Children and the Longitudinal Study of Indigenous Children should be funded to establish new cohorts of children at regular intervals.
In assessing whether to improve the quality of existing education data, governments should examine whether: there is a need to improve the quality of the data so it is fit for purpose; data quality improvements are feasible given the context of data collection; other options are available; and the benefits of improving data quality exceed the costs.
Australian, state and territory governments should support greater use of value-added measures of education outcomes.
* A complete list of recommendations is available in the overview of the draft report.
National Education Evidence Base
- Read the Draft Report released September 2016
- The Inquiry Report is due to be provided to the Australian Government in December 2016.