Queen Rania Foundation

Introducing the Toolkit

Our Teaching and Learning Toolkit is an accessible summary of education research. 

The Toolkit is designed to support teachers and school leaders who are making decisions about how to improve learning outcomes, particularly for disadvantaged children and young people. 

The Toolkit presents 35 approaches to improving teaching and learning, each summarised in terms of: 

  1. its average impact on attainment;
  2. its cost; 
  3. the strength of the evidence supporting it.

The Toolkit does not make definitive claims as to what will work to improve outcomes in a given school. Rather it provides high quality information about what is likely to be beneficial based on existing evidence. 

The Toolkit is a live resource that is updated on a regular basis as new findings from high-quality research become available. We welcome suggestions for new topics to be included in future updates of the Toolkits, as well as comments or questions. Please email [email protected]

Using the Toolkits

The Toolkit is designed to support teachers and school leaders who are making decisions about how to improve learning outcomes. It is based on real life data about what has happened when particular approaches have been used in schools before. 

The Toolkit does not make definitive claims as to what will work to improve outcomes in a given school. Rather they provide high quality information about what is likely to be beneficial based on existing evidence - ‘best bets’ for what might work in your own context.

Because the Toolkit does not provide definitive answers, it should not be used in isolation. Your professional judgement and expertise is also needed to move from the information in the Toolkit to an evidence-informed decision about what will work best in your school. 

Here are our top tips for getting the most out of the Toolkit

1. Look beyond the headlines. Every approach in the Toolkit has a dedicated page which describes in more detail what it is, what it costs and what impact you can expect. This extra detail can really matter. For example, the headline page shows that the average impact for Teaching Assistants is small, on average. However, the Teaching Assistants page reveals that using TAs to provide intensive, structured interventions, either one-to-one or in small groups, can deliver benefits of between three and five months’ additional progress.

2. Think about cost and evidence as well as impact. The most impactful approaches do not always provide the best value for money. For example, reducing class size has been shown to have a positive impact, on average (3 additional months’ progress), but because it requires new teachers it is fairly expensive (with a cost rating of 4 out of 5). Other approaches have had the same impact at a lower cost.

The strength of the evidence also matters. Homework in secondary school is estimated to deliver 5 months’ additional progress. However, the evidence strength rating (2 out of 5) shows that this estimate is less reliable than most of the estimates in the Toolkit.

3. Draw on your professional expertise. The Toolkit can tell you whether an approach has a good track record, but you need expertise and insight into your school’s context to make a well-informed decision about whether to introduce it for your pupils. For example, parental involvement approaches have had an average impact of 3 additional months’ progress. However, the evidence suggests that it can be difficult to get parents engaged in programmes in the first place. So specific knowledge of your school and its pupils is needed before you can decide whether a parental engagement approach would be feasible for you.

Summarising evidence in the Toolkit

For any education approach, there will be variation in the findings of different research studies, which can be lost when evidence is summarised. The Teaching and Learning Toolkit: what are the risks and how do we address them? is a recent blog which discusses the most commonly cited risks of summarising evidence as we do in the Toolkits and explains how we address them. 

Impact

Additional months' progress

The impact measure shows the number of additional months of progress made, on average, by children and young people who received the intervention, compared to similar children and young people who did not.

For example, the 'Feedback' strand shows that in the research studies included in the Toolkit, improving the quality of feedback had an average impact of 8 additional months’ progress.

This means that pupils in the classes where feedback interventions were provided made, on average, 8 months' more progress than similar pupils in other classes.

The months of additional progress estimates are based on ‘effect sizes’ reported in international research studies.

Effect sizes describe the size of the difference between two groups in a standard and comparable way. However, it can be difficult to understand what a given effect size actually means for the progress of children and young people. That is why we translate effect sizes into the months’ progress measure, as shown in the table below.

Months' Progress

Effect size from...

to...

Description

 

-0.01

0.01

Very low or no impact

1

0.02

0.09

Low impact

2

0.10

0.18

Low impact

3

0.19

0.26

Moderate impact

4

0.27

0.35

Moderate impact

5

0.36

0.44

Moderate impact

6

0.45

0.52

High impact

7

0.53

0.61

High impact

8

0.62

0.69

High impact

9

0.70

0.78

Very high impact

10

0.79

0.87

Very high impact

11

0.88

0.95

Very high impact

12

0.96

1.00

Very high impact

Notes: (i) Effect sizes are presented to the nearest 2 decimal places. (ii) For negative effect sizes, the conversion from effect size to months of additional progress is of the same size as shown in the table but negative in value. If the months' progress figure is negative, this does not necessarily imply negative progress. It means that individuals have made less progress than would be expected without the intervention. The Toolkit prioritises effect sizes derived from systematic reviews of research and meta-analyses of studies. Approaches are only included in the Toolkits if there are effect sizes available which can be used to create a months’ progress estimate. 

Evidence strength

The 'padlock' rating

The Toolkit presents a rating of the security of the evidence for each approach. This is illustrated on the Toolkit website using a padlock icon so it is sometimes referred to as the 'padlock' rating. This rating provides an overall estimate of the robustness of the evidence, to help support professional decision-making in schools. 

The 'padlock' rating for each strand is based on:

  • the number and types of research studies available;
  • the outcomes measured in those studies;
  • the quality of the studies and whether they enable researchers to draw conclusions about impact;
  • the consistency of the impact estimates in the different studies.

The following descriptions provide a brief description of what each padlock rating means:

🔒 = Very limited evidence: No evidence reviews available, only individual research studies.

🔒 🔒 = Limited evidence: At least one evidence review. Reviews include studies with relevant outcomes, and studies with methods which enable researchers to draw weak conclusions about impact.

🔒 🔒 🔒 = Moderate evidence: At least two evidence reviews. Reviews include studies with relevant outcomes, and studies with methods and analysis which enable researchers to draw moderate conclusions about impact.

🔒 🔒 🔒 🔒 = Extensive evidence: At least 3 evidence reviews. Reviews include studies with highly relevant outcomes, and studies with methods and analysis which enable researchers to draw strong conclusions about impact. Impact estimates are broadly consistent across studies.

🔒 🔒 🔒 🔒 🔒 = Very Extensive evidence: At least 5 evidence reviews. Reviews are recent, and include studies with highly relevant outcomes, and studies with methods and analysis which enable researchers to draw strong conclusions about impact. Impact estimates are consistent across studies.

Cost

The cost estimates in the Toolkits are based on the approximate costs of adopting an approach with a class of 25 pupils. 

Cost estimates can include: 

  • the cost of new resources, such as IT hardware or software;
  • the cost of training courses or professional development to support a new approach, and teacher cover to enable this to happen;
  • the cost of activities for pupils such as outdoor education programmes or music tuition.

Cost estimates do not cover resources which are needed for the approach but which schools are expected to already have, such as interactive whiteboards, or teachers (unless an additional teacher is necessary to deliver the approach). This is because we think additional expenditure is the most useful figure to present to schools leaders who are deciding how to allocate budgets.

Presenting the cost for each Toolkit strand allows schools to consider the cost-effectiveness of different approaches, as well as their impact. For example, One to one tuition and Peer tutoring both deliver 5 months of additional progress on average. However, One to one tuition is considerably more expensive on average. 

The cost ratings are based on a 5-point scale as shown below:

Cost estimate

Description

£

Very low: up to £2,000 per year per class of 25 pupils, or less than £80 per pupil per year.

££

Low: £2,001 to £5,000 per year per class of 25 pupils, or up to £200 per pupil per year

£££

Moderate: £5,001 to £18,000 per year per class of 25 pupils, or up to £720 per pupil per year.

££££

High: £18,001 to £30,000 per year per class of 25 pupils, or up to £1,200 per pupil.

£££££

Very high: over £30,000 per year per class of 25 pupils, or over £1,200 per pupil.