It has long been known that a single research study cannot resolve an issue as it will be limited not only by the sample size but a number of other factors such as confounding variables and treatment fidelity. Hence the importance of synthesizing knowledge acquired by a number of research studies to better understand a phenomenon.
The oldest and most common method for capturing what a body of literature has to offer is the literature review. However, these narrative reviews are neither scientifically sound nor adequate in extracting information from a vast body of literature. Moreover, they are biased and subjective rendering them highly unreliable. As such, systematic review methodologies started garnering more attention as a stronger alternative to narrative literature reviews.
Systematic reviews are demarcated by their adherence to a set of scientific methods with the explicit objective of limiting systematic error and bias by identifying, appraising and synthesizing all relevant studies (of different designs) in order to answer a particular question.
Generally speaking, a systematic review depends on seven important steps, namely:
- Clear define the research question to be answered by the review
- Decide on the types of studies to be identified in order to answer the question
- Conduct a comprehensive literature search to locate studies of interest
- Screen the search results and decide which ones meet the inclusion criteria and thus need more in-depth examination
- Critically evaluate the included studies
- Synthesize the findings from the studies
- Disseminate the findings of the review
In other words, a systematic review goes beyond the usual literature review approach to ensure inclusivity of documents addressing the same topic, with the least bias and subjectivity. The systematic review helps frame the questions and inform the syntheses of available literature. In addition, it provides clarity in order to know what generalizations may be drawn from the existing empirical evidence, what gaps exist, and what may be fruitful avenues for further research.
Moreover, the value of a systematic review includes being able to identify the consistencies and inconsistencies that exist within research evidence. For example, within the Arabic language learning literature, do different research findings indicate the same impact of Diglossia (or the gap between Modern Standard Arabic and dialect) on students’ learning of Arabic? Identifying similarities and differences among the identified studies and manuscripts may also help us to better understand how to mobilize knowledge in order to impact policy and practice.
Another value of a systematic review is that it supports a comprehensive understanding of the issues surrounding, say, Arabic Diglossia including its impact, applications, and the pedagogical, and socio-political factors which moderate its influence. The combination of findings from multiple studies will strengthen and qualify conclusions from any single study.
Despite the powerful advantages of doing a systematic review, it is an underused methodology in education research in the Arab speaking world. This is why we think the systematic review into Arabic Diglossia that we are carrying out with the support of the Queen Rania Foundation represents a methodological advance within a priority research field.
In carrying out this work, we will target documents in various areas including early childhood learning; elementary and secondary learning; mother tongue and native learners of Arabic and postsecondary learning. The coding of the documents will be done by a team of researchers to ensure rigour and minimal bias. After that, we will examine the primary literature more closely by considering the types of evidence reported and whether it is generated by qualitative or quantitative studies. This will then inform the actual review which will identify current themes in the research and future directions for enquiry.
This is by necessity a long, diligent process. But we think the effort is worthwhile – after all, we would be doing students a dis-service if we did not make good use of the best available methodologies to advance the teaching and learning of Arabic literacy.