EBM = the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients…integrating individual clinical expertise with the best available external clinical evidence from systematic research.

*there’s an overlap here with next chapter Research – this is more understanding stats and EBM*

Medpage Guide To Medical Statistics

MeReC Bulletin Aug 2011 Making Decisions Better

Centre for Review and Dissemination York

http://youtu.be/fbFVQg6GkFs

# EBM sources

Internal Link Prescribing Information Sources

# EBM process

The practice of EBM is a process of lifelong, problem based learning in which caring for our own patients creates the need for evidence about diagnosis, prognosis, therapy and other clinical health care issues.

Convert these information needs into answerable questions

Track down (efficiently) the best evidence to answer them

Critically appraise that evidence for validity (closeness to truth) and usefulness (clinical applicability)

Apply the results in our clinical practice

Evaluate our performance.

Three principles of EBM in keeping up to date:

How to practice EBM ourselves (journal clubs etc..)

Seek out and apply examples of EBM produced by others

Combining evidence based guidelines with validated strategies (audit and feedback)

The Cochrane Collaboration reviews are usually based on the results of RCTs and have an efficient search strategy.

# Research methodologies and types of study

Meta Analysis

Systematic Review

Review

Randomised Controlled Trials RCTs

Qualitive

Quantitive

Narrative Research

Cross Sectional Studies

Case Control Studies

Cohort Studies

Prospective & Retrospective Studies

# Systematic reviews and meta analysis

**Review**

any attempt to synthesise the results and conclusions of two or more publications on a given topic.

**Systematic review**

a review that strives to comprehensively identify and track down all the literature on a topic. Searches needed of unpublished work, foreign journals, citation searches and follow up of references.

**Meta-analysis**

a specific statistical technique for assembling all the results of several studies into a single numerical estimate.

# Literature review

How to write a literature review Santa Cruz University

What is a literature review? University North Carolina

The purpose of a literature review is to find and evaluate existing research evidence on a topic.

It can be described as ‘secondary research’, and as such should set out to answer a clear question

**Aim**

Say why you chose your particular topic and what use the findings will be to your practice. Keep your question simple, clear and relevant e.g.

Do patient participation groups improve patient care?

What is the evidence for the use of antibiotics in acute otitis media?

**Method**

Which databases did you search? e.g. Medline, Embase, Cochrane. What keywords did you use? How did you select the papers to read? How did you judge a paper to be worth including? What criteria did you use to evaluate the papers?

**Results/Findings**

Describe the range of literature you identified, and critically appraise the most relevant and important papers. These papers may themselves be reviews or meta-analyses. You need to provide more than just a descriptive list of articles and books

**Discussion**

Say what the main findings of your review are, how complete the review is, and what its limitations are. How much weight do you think you can give to the evidence you present? How do your findings compare with existing guidelines or accepted practice? Your recommendations for change should be in this section e.g. recommending that there should be more young people on the patient participation group

**Conclusions**

Summarise the evidence and information you have collected, and the implications for practice including suggestions for further study.

**Literature**

You can include references chosen for your review and also those that have helped you with your method e.g. a paper on how to do a literature review or on critical appraisal of the literature

Levels of evidence | |
---|---|

A | Strong research-based evidence, at least one randomised controlled trial, drawn from high quality scientific studies coming to similar conclusions Ia RCTs Ib at least one RCT |

B | Moderate research-based evidence drawn from well controlled studies IIa, at least one well-designed controlled study without randomisation IIb, at least one other type of well-designed quasi-experimental study III well-designed non-experimental descriptive studies |

C | Limited research-based evidence drawn from expert reports IV expert committee reports or opinions and/or clinical experience of respected authorities |

D | no evidence |

Heirarchy of evidence |
---|

systematic review of 2 or more R.C.Ts |

R.C.T. randomising to groups, treated differently and results analysed |

cohort study following a well population prospectively. |

case-control study matching cases with controls and looking back for associations. |

cross-sectional study looking at actual samples at a particular time |

respected authorities, expert committees |

someone once told me |

# How to read a paper

Objectives Worthwhile? Ethical? Results valid? Clearly presented?

Design

Setting

Patients

Outcome

Results

Conclusion

Remember bias, confounding factors

Control Event Rate (CER) = C/C+D

Experiment Event Rate (EER) = A/A+B

Relative Risk = EER/CER { RR=1, no effect; RR>1,favours treat; RR<1,favours no treat } Relative Risk Reduction (RRR) = CER EER / CER …….this will be a percentage

Absolute Risk Reduction (ARR) = CER EER…………this will be a percentage

Numbers needed to treat (NNT) = 1 / ARR………this will be a figure

Control Event Odds = C/D

Experim. Event Odds = A/B

Relative Odds = Odds Ratio = OR = A/B / C/D = AD / BC

Relative risk reduction (RRR) fails to discriminate huge absolute effects and discards the underlying susceptibility (or baseline risk ). It cannot tell the difference between huge risks and benefits from small ones.

In contrast, the absolute differences in the rates clearly do discriminate between these extremes and is called absolute risk reduction (ARR). Unfortunately, this gives a percentage and the reciprocal of this will give an easier figure to handle. This is the NNT.

This significance of this figure (?high or low) will depend on clinical significance and not statistical significance.

We can then compare these figures with other interventions we are familiar with in medicine and in doing so need to add the dimension of time; i.e. NNT for ?duration to prevent one event.

Ability to convert NNT to our own local patients by dividing NNT figure by F (where F is the estimated susceptibility of our own untreated patients relative to the average control patient in the trial ).

Odds Ratio describes the odds of an experimental patient having an event / benefit relative to a control patient. If interpret odds as risk then risk would be exaggerated especially with events / benefits that are more common. The difference between OR and RR increases as event rate rises

Odds ratios interfere with clinical application as

1. Not useful at bedside

2. Not even similar to risk reduction in most trials (as common events usually studied). Treating them the same would overestimate the harm / benefit

3. Cannot be used to calculate NNT

4. League tables of OR different to RRR.

This tutorial was prepared by Dr J A Crane

# Critical reading prompts

**IMROD – Una Coales**

Introduction

Methods

Results

Discussion

Others

**TBOAR**

Title Design Tables Critical Evaluation Conflicts of Interest

Background Outcome Understandable Aims met Overall

Originality Subjects Response Rate Conclusions References

Aims Dropouts Applicability Ethics

Relevance Stats

DOS

TURDS

CACA

CORE

# Making sense of a review – ten important questions

Address a clearly focused issue?

Looked at appropriate sort of papers?

Important, relevant studies included?

Qualities assessed of these studies?

Reasonable to combine results?

What is the overall result of the review.

How precise are these?

Can the results be applied to local population?

All outcomes considered?

Are the benefits worth the harm/costs?

Oxman, AD et al. JAMA 1994;272(17) 1367-71

# EBM deceptions

Surrogate End Points

Composite End Points

Relative Risk Reduction

Absolute Risk Reduction<

Inclusion Exclusion Bias Representiveness

Reliability Validity Generability

Surrogate Endpoints BMJ Aug 2011 @ NeLM

Surrogate Endpoints BMJ Jan 2012

Treat to Target – Treat the numbers or treat the patient? NeLM

EBM / Stats Glossary | |
---|---|

Incidence | |

Prevalence | |

Absolute Risk Reduction | difference in rate of events between the two groups risk of event in control grp-risk of event in Rx group |

Relative Risk | How many times more likely is it that an event will occur in the treatment group compared to control group RR is risk in treatment group/risk in control group often preferred by drug companies |

RRR | Relative risk reduction (RRR) Tells us the reduction in the rate of the outcome in the treatment group relative to the control group. RRR = ARR/risk of outcome in control group or RRR = 1-RR |

Hazard Ratio | form of relative risk HR>1 means an event is more likely to happen in the treatment group than in the placebo group |

Odds Ratio | another way of expressing probability or relative risk –OR>1 means an event is more likely to happen in the treatment group than in the placebo group |

Sensitivity | the proportion of people with a disease who are detected by the test (true positives) Sensitivity =TP/(TP+FN) Eg. You work out the proportion of cancers detected as a proportion of all the cancers. High sensitivity – good test for cancer |

Specificity | the people who don’t have the disease (true negatives) and don’t test positive (ie. they test negative) Specificity = TN/(TN+FP) Eg. You work out the proportion of the people who haven’t got cancer and test negative for cancer as a proportion of all those without cancer. High specificity = few false positives. |

Likelihood ratios | incorporate both sensitivity and specificity. For a positive test result it is: The ratio of: the probability of a positive test result in those with the disease To: the probability of a positive test result in those who do not have the disease. Put another way the likelihood ratio of a positive test is: sensitivity/(1-specificity) The likelihood of a negative test is: (1-sensitivity)/specificity |

Pre-test probability |

# Simple statistics

**Arithmetic mean aka average**

add all the results up and divide by the number of results you had

**Mode**

most common

**Median**

line up all the numbers in order, and the median is the middle number

**Interquartile Range**

the difference between the 25th quartile and 75th quartile of data (ie. the middle 50% of data).

Incidence prevalence morbidity mortality

Basic Statistics health.state.ny.us

# Sensitivity and specificity

Sensitivity specificity med.emory.edu

Understanding sensitivity and specificity with the right side of the brain BMJ 2003

Sensitivity and Specificity medpedia.com

Clinical tests: sensitivity and specificity ceaccp.oxfordjournals.org

# Likelihood ratios

Introduction to Likelihood Ratios gim.unmc.edu

Pre-test probability

Post-test Probability

PPV

NPV

# Risk reduction

**Absolute Risk Reduction**

difference in rate of events between the two groups

risk of event in control grp-risk of event in Rx group

**Relative Risk**

How many times more likely is it that an event will occur in the treatment group compared to control group

RR is risk in treatment group/risk in control group

often preferred by drug companies!

**Relative risk reduction (RRR)**

Tells us the reduction in the rate of the outcome in the treatment group relative to the control group.

RRR = ARR/risk of outcome in control group or RRR = 1-RR

**Hazard Ratio**

form of relative risk HR>1 means an event is more likely to happen in the treatment group than in the placebo group

**Odds Ratio**

another way of expressing probability or relative risk –OR>1 means an event is more likely to happen in the treatment group than in the placebo group

# NNT and NNH

Dr Chris Cates’ EBM Website nntonline.net

**Numbers needed to treat NNTs**

how many people have to be treated for 1 person to benefit.

An ideal NNT is 1; everyone treated gets better, no one given the placebo group gets better.

**NNH numbers needed to harm**

NNT/H should, but don’t always, quote a time frame

NNT = 1/ARR (absolute risk reduction)

# Confidence intervals

Confidence intervals stattrek.com

Confidence intervals onlinestatbook.com

# null hypothesis

null-hypothesis.co.uk what is a null hypothesis

# p value

# type 1 and type 2 errors

type 1 and type 2 errors intuitor.com

# Forrest plot

# Power calculation

Power Calculation statsoft.com

dssresearch.com statistical power calculators

# Triangulation

Triangulation research methods johnnyholland.org

## Feedback/Errata