Evidence-Based Practice in Social Work and Nursing

PART 1 – Reading studies with a method: terms, structure and reading routine

From finding to appraising: critical reading in the EBP cycle

Once you have built a search strategy in Module 4 and screened the hits, the real thinking begins: what does this study say, and how sure can you be of it? In the cycle of evidence-based practice this is the step of critical appraisal, which sits between searching and changing practice. “Critical” here does not mean distrustful or smug, but asking systematically how the robustness of a statement can be recognised. The Institute for Quality and Efficiency in Health Care (IQWiG) stresses that evidence-based decisions should not rest on opinions alone, but on knowledge gained with scientific methods that are as objective as possible (IQWiG, 2025).

A word on scope: “study” here means an empirical investigation that discloses its procedure – from an interview study to the randomised controlled trial that you classified in Module 3. This module concentrates on comparative studies on questions of effectiveness and on systematic reviews, because tools for assessing bias, effect measures and reporting guidelines are most fully developed for them. You know from Module 3 how to classify qualitative work: there, traceability and appropriateness to the subject are the guiding questions, not effect measures.

Figure 1 gives you the map of this module: critical reading is in the centre, and the building blocks you will get to know one after another hang on the five branches. You can come back to this overview at any time to find your bearings.

The structure of a study: five sections, five ways of looking

Most empirical journal articles follow a similar pattern, even if the names and the order vary by discipline and journal. The abstract is the short version: it tells you what the study is about and what came out of it, but it is necessarily condensed and often worded optimistically. The introduction explains why the question matters and leads to the research question. The methods section states who was studied, how the groups were formed, what was measured and how the data are analysed. The results report what came out – ideally without interpretation. In the discussion the authors interpret their findings, compare them with other studies and name limitations.

Each of these sections answers a different question, and each calls for a different reading attitude. The following table brings together guiding questions you can use to work through the sections critically. It is a reading aid and not a complete appraisal method; for systematic judgements you need the tools in the following parts.

The reading routine: methods first, then results

Why the methods first? If you read the abstract and the discussion first, you unwittingly adopt the authors’ interpretation and then read the results through those glasses. If you first know how the groups were formed and the outcomes measured, you can place the results in context before interpretation colours them. This is a working habit, not a law; what matters is that you choose your order deliberately and stick to it. Figure 2 shows a routine in five steps that you can use for term papers, presentations and the journal club (Module 8).

PART 2 – Recognising bias: systematic errors and the RoB 2 tool

What does bias mean?

Bias (German: Verzerrung) is a systematic error that shifts results in a particular direction. It must be distinguished from random fluctuation: this evens out on average and can be narrowed down with more participants. Systematic distortion, by contrast, does not disappear just because a study is large. Imagine kitchen scales that always show a little too much: if you weigh a hundred times, the average becomes more stable, but it stays wrong.

IQWiG bases the certainty of results of clinical studies on three aspects: internal validity, external validity and statistical precision (IQWiG, 2025). Bias mainly affects internal validity, that is, the question of whether a difference that has been found can be due to the intervention under study at all. You know the two validity concepts from Module 3 and do not need to learn them anew here; you will turn to statistical precision in Part 3.

RoB 2: five domains, three judgements

For randomised trials, Cochrane has developed the tool RoB 2 (Risk of Bias 2). It breaks the question “How susceptible is this result to bias?” down into five areas, which are called domains (Higgins et al., 2024). Each domain comes with guiding questions that you use to go through the study. In your own words and applied to everyday life, they read as follows:

• Randomisation process: Was the allocation to the groups really generated at random, and could nobody predict or steer it in advance? Groups that already differ clearly at the start are a warning sign.
• Deviations from the intended interventions: Did both groups receive what was planned? It becomes problematic, for example, if the comparison group also uses a similar service or if professionals change their behaviour because they know who belongs to which group.
• Missing outcome data: For how many people are there no measurements at the end, and is dropping out related to the intervention or the outcome? If people with an unfavourable course are more likely to leave the study, the result looks better than it is.
• Measurement of the outcome: Was the outcome measured in the same way in both groups, and did the assessors know who belonged to which group? People who know the group allocation sometimes judge more favourably without realising it.
• Selection of the reported result: Was what had been planned in advance reported, or only the favourable results from many possible analyses?

For each domain, RoB 2 gives one of three judgements: low risk, some concerns or high risk. The overall judgement for a study is at least as unfavourable as the most unfavourable domain; some concerns in several domains can also lead to an overall “high risk” if they substantially lower confidence in the result (Higgins et al., 2024). That sounds strict, but it is logical: a chain is only as strong as its weakest link, and a serious flaw in one place cannot be made up for by care in four others. Figure 3 shows the five domains with an invented example judgement. Note that the judgement levels are distinguished not only by colour, but also by shape and text.

You should avoid two misunderstandings. First, “high risk” does not mean that a study is worthless; it means that its result has to be read with particular caution and should carry little weight on its own. Second, RoB 2 is intended for randomised trials. It cannot simply be transferred to studies without random allocation; for those you need other tools; you saw in Module 3 which designs exist and how to classify them. And one more thing: two assessors may reach different judgements on individual domains. RoB 2 structures your thinking, it does not replace judgement.

Blinding and closeness to practice: a tension

In social work and nursing, RoB 2 reaches a practical limit. People who take part in a group programme generally know that they are taking part, and professionals who deliver a programme know who is in which group. Completely concealing group membership, as one imagines it from everyday medicine, is often not possible here. This is not a failing of the researchers but lies in the nature of the subject. It must nevertheless be reported openly and taken into account when weighing the results, for example through the domains on deviations from the intended interventions and on measurement of the outcome.

At the same time, a tension appears that plays a role in the German-language debate on evidence-based social work: positions in favour stress the strength of randomised effectiveness studies, while critical positions point to the technology deficit of social interaction and to the limited transferability of causal models. For your critical reading this means: bias tools are valuable, and they also show how hard it is to translate practice conditions into strict study designs. You can keep both in view at the same time.

PART 3 – Effects in words: risk, difference, NNT and precision

Which effect measures exist and what they are for

An effect measure is a number that describes how large the difference between two groups is. Which number makes sense depends on the type of outcome. For outcomes with two possible values (“event yes/no”, so-called dichotomous outcomes), the Cochrane Handbook names the risk ratio, the odds ratio and the risk difference; for continuous outcomes (for example scores on a scale) it names the mean difference and the standardised mean difference (Higgins et al., 2024). Added to these is the number needed to treat (NNT), which is derived from the risk difference. The following table translates all the measures into questions that you could also explain to a team.

A warning in advance: there are deliberately no fixed thresholds here for when an effect would be “small”, “medium” or “large”. How important a difference is depends on the professional question, on the outcome and on the consequences for the people affected, and such rules of thumb easily suggest a clarity that does not exist.

Worked example: from “20 out of 100” to an everyday sentence

Now we work through an example step by step before you try a task yourself. The figures are entirely invented: this is an invented study of an invented group programme, and it says nothing about the effectiveness of real programmes. The event that is to be avoided we simply call an “adverse event”. The intervention group (with the group programme) contains 100 people, of whom 20 have an adverse event. The control group (without the programme) also contains 100 people, of whom 30 do. Figure 4 records these details in a two-by-two table: two groups times two outcomes.

What the calculation does not achieve is at least as important as what it shows. It does not tell you whether the groups were formed fairly (that is the bias question from Part 2), nor how certain the estimate is given the sample (that is the precision question further below), nor whether the difference reaches a particular person. The NNT is also an average across groups and not a prediction for individuals. The same applies to the question of causation: whether a difference may be attributed to the intervention is decided by the design of the study (Module 3) and not by the type of calculation.

Relative or absolute? Why “risk halved” alone says little

The Cochrane Handbook states that relative measures can only be interpreted with the baseline risk in view, whereas absolute measures such as the risk difference are particularly suitable for weighing up options (Higgins et al., 2024). The example above shows it: “a third fewer” sounds like a lot, yet behind it are 10 percentage points. For a very rare event, the same relative reduction can stand behind a tiny absolute difference; for a common event, it stands behind a large one. Without the starting values you cannot tell the two apart. So whenever you read statements such as “risk halved” or “twice as common”, always ask: starting from what value?

The confidence interval: how precise is the estimate?

Every study examines only a sample, never all people. Every effect measure is therefore an estimate with uncertainty. The confidence interval (CI) makes this uncertainty visible: it gives a range of values that is well compatible with the observed data. A narrow interval means a precise estimate, a wide one an imprecise estimate. IQWiG lists statistical precision as a separate aspect of the certainty of results (IQWiG, 2025). Small studies tend to have wide intervals, large ones narrow intervals; a narrow interval, however, says nothing about whether the study is free of bias. Precision and freedom from bias are two different properties.

Figure 5 shows the principle schematically. The dashed line marks “no difference between the groups” (for differences this is the value 0, for ratio measures such as the relative risk the value 1). Study A has a narrow interval that lies entirely on one side of the line: the study separates a difference fairly clearly from “no difference”. Study B has a wide interval that includes the line: with these data, both a benefit and no difference are compatible. Both studies are invented, and the graphic deliberately contains no numbers.

PART 4 – Reporting guidelines as a reading aid: CONSORT 2025 and PRISMA 2020

Reporting guidelines are not a quality verdict

In journal articles you keep coming across notes such as “reported in line with CONSORT” or “according to PRISMA”. What is meant are reporting guidelines: checklists that set out what authors should transparently state in a publication. They therefore say what should be reported, not how well a study was carried out. For you as a reader this is helpful all the same: if you know what a complete report should contain, you notice more quickly when something is missing. Figure 6 places three tools side by side that are easily confused but have different tasks.

The difference between the columns is fundamental: CONSORT and PRISMA are aimed at authors who report, RoB 2 at readers and review teams who appraise. A study can be carefully reported and still have methodological weaknesses; conversely, a well-conducted study may have been reported incompletely. If an item is missing, however, you do not know what happened, and it is precisely this uncertainty that you should make visible in your judgement.

CONSORT 2025: checklist for randomised trials

CONSORT 2025 is the reporting guideline for randomised trials and has replaced CONSORT 2010 (Hopewell et al., 2025). The current checklist has 30 items. For your reading it is worth looking at some of the new features, because they touch on questions you already know from judging bias: there is an item on the handling of missing data, which refers to the area of missing outcome data from Part 2, and one on recording harms, that is, adverse effects alongside the benefit. Added to these are, among others, items on data sharing, on conflicts of interest and on the involvement of patients and the public.

PRISMA 2020: reporting guideline for systematic reviews

PRISMA 2020 applies to systematic reviews. The guideline consists of a checklist with 27 items in seven sections; it sets out what a review should report transparently, and it is expressly not a guide to conducting a review and not a tool for quality appraisal (Page et al., 2021). The flow diagram you know from Module 4 belongs in this context: it shows how many hits were screened and how many were included. When you read a review, you can use four questions as a guide: is the search documented in a traceable way? Is it clear by which criteria studies were included or excluded? Was the quality of the individual studies appraised, for example with RoB 2? And does this appraisal feed into the conclusions?

That the link between appraisal and reporting guideline is also put into practice is shown by a look ahead to Module 6: the DNQP (German Network for Quality Development in Nursing) documents its literature search with a PRISMA flow diagram and appraises the literature it finds with instruments such as AMSTAR-2, RoB 2 and AGREE II (DNQP, 2025b). Here you can see how the building blocks of this module work together, without having to master the individual instruments in detail yet.

Putting it together: a reading pass in four questions

The previous parts can be turned into a short routine that you can apply to any comparative study. First: does the question of the study fit my practice question (PICO from Module 2)? Second: how were the groups formed, who dropped out, who knew what, what was planned in advance (bias from Part 2)? Third: how large is the effect in words, absolute and relative, and how precise is the estimate (Part 3)? Fourth: is the report complete enough that I could answer the first three questions at all (Part 4)? If you cannot answer one of these questions, that is a finding in itself, which you should record in your judgement.

PART 5 – Applying and placing in context: practice, limits and outlook

Practice: reading a fictional abstract critically

You have followed the worked example, now it is your turn. The following abstract is entirely invented and does not reproduce a real study. It is only meant to make typical gaps visible.

Limits of critical reading

Part of dealing with studies scientifically is knowing the limits of your own tools. You should keep five restrictions in mind.

First, bias tools are aids to thinking, not calculating machines. Two readers can reach slightly different judgements on the same study, and the reasons should be documented. Second, the best-developed instruments are tailored to randomised trials. Many questions in social work cannot, or only with difficulty, be investigated in this way, and often no such studies exist for them; this is part of the debate on evidence-based social work, in which supporters and critics weigh their positions differently. The absence of randomised trials does not mean that there is no robust knowledge; it means that you have to classify other designs differently (Module 3).

Third, effect measures show average differences between groups. They say nothing about whether an intervention fits the values, wishes and life circumstances of a particular person; Module 7 is devoted to this question. Fourth, the quality of a report is not the quality of a study: careful reporting according to CONSORT or PRISMA makes a study transparent, but not automatically good. Fifth, criticising can become an end in itself. If you dismiss every study as useless, you have gained nothing. The aim is a reasoned judgement with reservations that is usable for decisions in the team.

Behind this lies a professional-ethical stance: the professional ethics of the German Professional Association for Social Work (DBSH) assume that the practice competence of social work rests on scientifically secured knowledge (DBSH, 2024). Anyone who reads studies critically takes this claim seriously: it prevents statements from being believed simply because they appear in a journal or because they sound good. It is also a matter of responsibility towards the service users whose living situation is affected by decisions.

Outlook: what comes next

In Module 6 you classify guidelines, expert standards and professional standards, which summarise many studies as syntheses and whose structure you read step by step there. In Module 7 you ask how evidence comes together with the preferences and the lifeworld of the people concerned, and in Module 8 you bring everything together in the journal club and the literature report. The reading routine from this module accompanies you through all three.

Self-reflection – 5 questions

Voluntary · not evaluated – your answers are not saved and not analysed by anyone. Click an answer; the explanation helps you assess your own understanding.