EBM / Stats

EBM = the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients…integrating individual clinical expertise with the best available external clinical evidence from systematic research.

there’s an overlap here with next chapter Research  – this is more understanding stats and EBM

Medpage Guide To Medical Statistics

MeReC Bulletin Aug 2011 Making Decisions Better

Cochrane Collaboration

Centre for Review and Dissemination York
http://youtu.be/fbFVQg6GkFs

http://youtu.be/Nbd–s2dFY0

http://youtu.be/QsIYwWwi_r4

http://youtu.be/IfZ1yzPvUQ8

http://youtu.be/AuoMvl0ueFM

http://youtu.be/I1Try6c6R5M

http://youtu.be/Og2WjVpgLFc

 

EBM sources

NHS Evidence

CKS

NICE

Sign Guidelines

Map Of Medicine

BMJ Best Practice

NeLM

Cochrane Library

Internal Link Prescribing Information Sources

 

EBM process

The practice of EBM is a process of lifelong, problem based learning in which caring for our own patients creates the need for evidence about diagnosis, prognosis, therapy and other clinical health care issues.
Convert these information needs into answerable questions
Track down (efficiently) the best evidence to answer them
Critically appraise that evidence for validity (closeness to truth) and usefulness (clinical applicability)
Apply the results in our clinical practice
Evaluate our performance.

Three principles of EBM in keeping up to date:
How to practice EBM ourselves (journal clubs etc..)
Seek out and apply examples of EBM produced by others
Combining evidence based guidelines with validated strategies (audit and feedback)
The Cochrane Collaboration reviews are usually based on the results of RCTs and have an efficient search strategy.

 

Research methodologies and types of study

Meta Analysis
Systematic Review
Review
Randomised Controlled Trials RCTs
Qualitive
Quantitive
Narrative Research
Cross Sectional Studies
Case Control Studies
Cohort Studies
Prospective & Retrospective Studies

 

Systematic reviews and meta analysis

Review
any attempt to synthesise the results and conclusions of two or more publications on a given topic.

Systematic review
a review that strives to comprehensively identify and track down all the literature on a topic. Searches needed of unpublished work, foreign journals, citation searches and follow up of references.

Meta-analysis
a specific statistical technique for assembling all the results of several studies into a single numerical estimate.

 

Literature review

How to write a literature review Santa Cruz University

What is a literature review? University North Carolina

The purpose of a literature review is to find and evaluate existing research evidence on a topic.
It can be described as ‘secondary research’, and as such should set out to answer a clear question

Aim
Say why you chose your particular topic and what use the findings will be to your practice. Keep your question simple, clear and relevant e.g.
Do patient participation groups improve patient care?
What is the evidence for the use of antibiotics in acute otitis media?

Method
Which databases did you search? e.g. Medline, Embase, Cochrane. What keywords did you use? How did you select the papers to read? How did you judge a paper to be worth including? What criteria did you use to evaluate the papers?

Results/Findings
Describe the range of literature you identified, and critically appraise the most relevant and important papers. These papers may themselves be reviews or meta-analyses. You need to provide more than just a descriptive list of articles and books

Discussion
Say what the main findings of your review are, how complete the review is, and what its limitations are. How much weight do you think you can give to the evidence you present? How do your findings compare with existing guidelines or accepted practice? Your recommendations for change should be in this section e.g. recommending that there should be more young people on the patient participation group

Conclusions
Summarise the evidence and information you have collected, and the implications for practice including suggestions for further study.

Literature
You can include references chosen for your review and also those that have helped you with your method e.g. a paper on how to do a literature review or on critical appraisal of the literature

 

Levels of evidence
A Strong research-based evidence, at least one randomised controlled trial, drawn from high quality scientific studies coming to similar conclusions
Ia RCTs
Ib at least one RCT
B Moderate research-based evidence drawn from well controlled studies
IIa, at least one well-designed controlled study without randomisation
IIb, at least one other type of well-designed quasi-experimental study
III well-designed non-experimental descriptive studies
C Limited research-based evidence drawn from expert reports
IV expert committee reports or opinions and/or clinical experience of respected authorities
D no evidence

 

Heirarchy of evidence
systematic review of 2 or more R.C.Ts
R.C.T. randomising to groups, treated differently and results analysed
cohort study following a well population prospectively.
case-control study matching cases with controls and looking back for associations.
cross-sectional study looking at actual samples at a particular time
respected authorities, expert committees
someone once told me

Levels of Evidence EE+

Levels of evidence CEBM

How to read a paper

Objectives Worthwhile? Ethical? Results valid? Clearly presented?
Design
Setting
Patients
Outcome
Results
Conclusion

Remember bias, confounding factors
Control Event Rate (CER) = C/C+D
Experiment Event Rate (EER) = A/A+B
Relative Risk = EER/CER { RR=1, no effect; RR>1,favours treat; RR<1,favours no treat } Relative Risk Reduction (RRR) = CER EER / CER …….this will be a percentage
Absolute Risk Reduction (ARR) = CER EER…………this will be a percentage
Numbers needed to treat (NNT) = 1 / ARR………this will be a figure
Control Event Odds = C/D
Experim. Event Odds = A/B
Relative Odds = Odds Ratio = OR = A/B / C/D = AD / BC

Relative risk reduction (RRR) fails to discriminate huge absolute effects and discards the underlying susceptibility (or baseline risk ). It cannot tell the difference between huge risks and benefits from small ones.

In contrast, the absolute differences in the rates clearly do discriminate between these extremes and is called absolute risk reduction (ARR). Unfortunately, this gives a percentage and the reciprocal of this will give an easier figure to handle. This is the NNT.

This significance of this figure (?high or low) will depend on clinical significance and not statistical significance.

We can then compare these figures with other interventions we are familiar with in medicine and in doing so need to add the dimension of time; i.e. NNT for ?duration to prevent one event.

Ability to convert NNT to our own local patients by dividing NNT figure by F (where F is the estimated susceptibility of our own untreated patients relative to the average control patient in the trial ).

Odds Ratio describes the odds of an experimental patient having an event / benefit relative to a control patient. If interpret odds as risk then risk would be exaggerated especially with events / benefits that are more common. The difference between OR and RR increases as event rate rises
Odds ratios interfere with clinical application as
1. Not useful at bedside
2. Not even similar to risk reduction in most trials (as common events usually studied). Treating them the same would overestimate the harm / benefit
3. Cannot be used to calculate NNT
4. League tables of OR different to RRR.
This tutorial was prepared by Dr J A Crane

 

Critical reading prompts

IMROD – Una Coales
Introduction
Methods
Results
Discussion
Others

TBOAR
Title Design Tables Critical Evaluation Conflicts of Interest
Background Outcome Understandable Aims met Overall
Originality Subjects Response Rate Conclusions References
Aims Dropouts Applicability Ethics
Relevance Stats
DOS
TURDS
CACA
CORE

 

Making sense of a review – ten important questions

Address a clearly focused issue?
Looked at appropriate sort of papers?
Important, relevant studies included?
Qualities assessed of these studies?
Reasonable to combine results?
What is the overall result of the review.
How precise are these?
Can the results be applied to local population?
All outcomes considered?
Are the benefits worth the harm/costs?

Oxman, AD et al. JAMA 1994;272(17) 1367-71

 

EBM deceptions

Surrogate End Points
Composite End Points
Relative Risk Reduction
Absolute Risk Reduction<
Inclusion Exclusion Bias Representiveness
Reliability Validity Generability

Surrogate Endpoints BMJ Aug 2011 @ NeLM

Surrogate Endpoints BMJ Jan 2012

Treat to Target – Treat the numbers or treat the patient? NeLM

 

EBM Glossary CEBM.net

EBM / Stats Glossary
Incidence
Prevalence
Absolute Risk Reduction difference in rate of events between the two groups
risk of event in control grp-risk of event in Rx group
Relative Risk How many times more likely is it that an event will occur in the treatment group compared to control group
RR is risk in treatment group/risk in control group
often preferred by drug companies
RRR Relative risk reduction (RRR)
Tells us the reduction in the rate of the outcome in the treatment group relative to the control group.
RRR = ARR/risk of outcome in control group or RRR = 1-RR
Hazard Ratio form of relative risk HR>1 means an event is more likely to happen in the treatment group than in the placebo group
Odds Ratio another way of expressing probability or relative risk –OR>1 means an event is more likely to happen in the treatment group than in the placebo group
Sensitivity the proportion of people with a disease who are detected by the test (true positives)
Sensitivity =TP/(TP+FN) Eg. You work out the proportion of cancers detected as a proportion of
all the cancers. High sensitivity – good test for cancer
Specificity the people who don’t have the disease (true negatives) and don’t test positive (ie. they test negative)
Specificity = TN/(TN+FP) Eg. You work out the proportion of the people who haven’t got cancer
and test negative for cancer as a proportion of all those without cancer. High specificity = few false positives.
Likelihood ratios incorporate both sensitivity and specificity. For a positive test result it is: The ratio of: the probability of a positive test result in those with the disease To: the probability of a positive test result in those who do not have the disease. Put another way the likelihood ratio of a positive test is: sensitivity/(1-specificity) The likelihood of a negative test is: (1-sensitivity)/specificity
Pre-test probability

 

Simple statistics

Arithmetic mean aka average
add all the results up and divide by the number of results you had
Mode
most common

Median
line up all the numbers in order, and the median is the middle number

Interquartile Range
the difference between the 25th quartile and 75th quartile of data (ie. the middle 50% of data).

http://youtu.be/uhxtUt_-GyM

Incidence prevalence morbidity mortality

Basic Statistics health.state.ny.us

 

Sensitivity and specificity

Sensitivity specificity med.emory.edu

Understanding sensitivity and specificity with the right side of the brain BMJ 2003

Sensitivity and Specificity medpedia.com

Clinical tests: sensitivity and specificity ceaccp.oxfordjournals.org

http://youtu.be/ICKaSKoDPmI

http://youtu.be/Nr2WBd_LKO0

http://youtu.be/JdCynjF-I1E

http://youtu.be/cWetUF1Nhfs

 

Likelihood ratios

Introduction to Likelihood Ratios gim.unmc.edu

Pre-test probability
Post-test Probability
PPV
NPV

 

Risk reduction

Absolute Risk Reduction
difference in rate of events between the two groups
risk of event in control grp-risk of event in Rx group

Relative Risk
How many times more likely is it that an event will occur in the treatment group compared to control group
RR is risk in treatment group/risk in control group
often preferred by drug companies!

Relative risk reduction (RRR)
Tells us the reduction in the rate of the outcome in the treatment group relative to the control group.
RRR = ARR/risk of outcome in control group or RRR = 1-RR

Hazard Ratio
form of relative risk HR>1 means an event is more likely to happen in the treatment group than in the placebo group

Odds Ratio
another way of expressing probability or relative risk –OR>1 means an event is more likely to happen in the treatment group than in the placebo group

 

NNT and NNH

Dr Chris Cates’ EBM Website nntonline.net

Numbers Needed to treat PUK

Numbers needed to treat NNTs
how many people have to be treated for 1 person to benefit.

An ideal NNT is 1; everyone treated gets better, no one given the placebo group gets better.
NNH numbers needed to harm
NNT/H should, but don’t always, quote a time frame
NNT = 1/ARR (absolute risk reduction)

 

Confidence intervals

Confidence intervals stattrek.com

Confidence intervals onlinestatbook.com

http://youtu.be/Hn6C21GC0vA

 

null hypothesis

null-hypothesis.co.uk what is a null hypothesis

http://youtu.be/5N7L1cGCL-w

 

p value

p value graphpad.com

http://youtu.be/lm_CagZXcv8

http://youtu.be/ax0tDcFkPic

http://youtu.be/NdCDg_A84zw

http://youtu.be/UBbtWUXG1Bw

 

type 1 and type 2 errors

type 1 and type 2 errors intuitor.com

http://youtu.be/taEmnrTxuzo

http://youtu.be/FHT6e_mdGoU

http://youtu.be/plmu-64iq84

 

Forrest plot

Forrest plots bmj.com

http://youtu.be/F0Wp8dyz7OY

 

Power calculation

Power Calculation statsoft.com

dssresearch.com statistical power calculators

http://youtu.be/Ki_Oy06jG2g

http://youtu.be/OyuQn-P8iQ0

http://youtu.be/4-5pFrqJz9w

http://youtu.be/HuGl50C8Q30

http://youtu.be/Mfia4nbh-zU

 

Triangulation

Triangulation research methods johnnyholland.org

 

Presenting data as natural frequencies

Communicating Data as Natural Frequencies NELM 2009

What Are Natural Frequencies Gerd Gigerenzer BMJ Oct 2011

Comments are closed.