7
EBM = the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients…integrating individual clinical expertise with the best available external clinical evidence from systematic research.
there’s an overlap here with next chapter Research – this is more understanding stats and EBM
Medpage Guide To Medical Statistics
MeReC Bulletin Aug 2011 Making Decisions Better
Centre for Review and Dissemination York
http://youtu.be/fbFVQg6GkFs
EBM sources
Internal Link Prescribing Information Sources
EBM process
The practice of EBM is a process of lifelong, problem based learning in which caring for our own patients creates the need for evidence about diagnosis, prognosis, therapy and other clinical health care issues.
Convert these information needs into answerable questions
Track down (efficiently) the best evidence to answer them
Critically appraise that evidence for validity (closeness to truth) and usefulness (clinical applicability)
Apply the results in our clinical practice
Evaluate our performance.
Three principles of EBM in keeping up to date:
How to practice EBM ourselves (journal clubs etc..)
Seek out and apply examples of EBM produced by others
Combining evidence based guidelines with validated strategies (audit and feedback)
The Cochrane Collaboration reviews are usually based on the results of RCTs and have an efficient search strategy.
Research methodologies and types of study
Meta Analysis
Systematic Review
Review
Randomised Controlled Trials RCTs
Qualitive
Quantitive
Narrative Research
Cross Sectional Studies
Case Control Studies
Cohort Studies
Prospective & Retrospective Studies
Systematic reviews and meta analysis
Review
any attempt to synthesise the results and conclusions of two or more publications on a given topic.
Systematic review
a review that strives to comprehensively identify and track down all the literature on a topic. Searches needed of unpublished work, foreign journals, citation searches and follow up of references.
Meta-analysis
a specific statistical technique for assembling all the results of several studies into a single numerical estimate.
Literature review
How to write a literature review Santa Cruz University
What is a literature review? University North Carolina
The purpose of a literature review is to find and evaluate existing research evidence on a topic.
It can be described as ‘secondary research’, and as such should set out to answer a clear question
Aim
Say why you chose your particular topic and what use the findings will be to your practice. Keep your question simple, clear and relevant e.g.
Do patient participation groups improve patient care?
What is the evidence for the use of antibiotics in acute otitis media?
Method
Which databases did you search? e.g. Medline, Embase, Cochrane. What keywords did you use? How did you select the papers to read? How did you judge a paper to be worth including? What criteria did you use to evaluate the papers?
Results/Findings
Describe the range of literature you identified, and critically appraise the most relevant and important papers. These papers may themselves be reviews or meta-analyses. You need to provide more than just a descriptive list of articles and books
Discussion
Say what the main findings of your review are, how complete the review is, and what its limitations are. How much weight do you think you can give to the evidence you present? How do your findings compare with existing guidelines or accepted practice? Your recommendations for change should be in this section e.g. recommending that there should be more young people on the patient participation group
Conclusions
Summarise the evidence and information you have collected, and the implications for practice including suggestions for further study.
Literature
You can include references chosen for your review and also those that have helped you with your method e.g. a paper on how to do a literature review or on critical appraisal of the literature
| Levels of evidence | |
|---|---|
| A | Strong research-based evidence, at least one randomised controlled trial, drawn from high quality scientific studies coming to similar conclusions Ia RCTs Ib at least one RCT |
| B | Moderate research-based evidence drawn from well controlled studies IIa, at least one well-designed controlled study without randomisation IIb, at least one other type of well-designed quasi-experimental study III well-designed non-experimental descriptive studies |
| C | Limited research-based evidence drawn from expert reports IV expert committee reports or opinions and/or clinical experience of respected authorities |
| D | no evidence |
| Heirarchy of evidence |
|---|
| systematic review of 2 or more R.C.Ts |
| R.C.T. randomising to groups, treated differently and results analysed |
| cohort study following a well population prospectively. |
| case-control study matching cases with controls and looking back for associations. |
| cross-sectional study looking at actual samples at a particular time |
| respected authorities, expert committees |
| someone once told me |
How to read a paper
Objectives Worthwhile? Ethical? Results valid? Clearly presented?
Design
Setting
Patients
Outcome
Results
Conclusion
Remember bias, confounding factors
Control Event Rate (CER) = C/C+D
Experiment Event Rate (EER) = A/A+B
Relative Risk = EER/CER { RR=1, no effect; RR>1,favours treat; RR<1,favours no treat } Relative Risk Reduction (RRR) = CER EER / CER …….this will be a percentage
Absolute Risk Reduction (ARR) = CER EER…………this will be a percentage
Numbers needed to treat (NNT) = 1 / ARR………this will be a figure
Control Event Odds = C/D
Experim. Event Odds = A/B
Relative Odds = Odds Ratio = OR = A/B / C/D = AD / BC
Relative risk reduction (RRR) fails to discriminate huge absolute effects and discards the underlying susceptibility (or baseline risk ). It cannot tell the difference between huge risks and benefits from small ones.
In contrast, the absolute differences in the rates clearly do discriminate between these extremes and is called absolute risk reduction (ARR). Unfortunately, this gives a percentage and the reciprocal of this will give an easier figure to handle. This is the NNT.
This significance of this figure (?high or low) will depend on clinical significance and not statistical significance.
We can then compare these figures with other interventions we are familiar with in medicine and in doing so need to add the dimension of time; i.e. NNT for ?duration to prevent one event.
Ability to convert NNT to our own local patients by dividing NNT figure by F (where F is the estimated susceptibility of our own untreated patients relative to the average control patient in the trial ).
Odds Ratio describes the odds of an experimental patient having an event / benefit relative to a control patient. If interpret odds as risk then risk would be exaggerated especially with events / benefits that are more common. The difference between OR and RR increases as event rate rises
Odds ratios interfere with clinical application as
1. Not useful at bedside
2. Not even similar to risk reduction in most trials (as common events usually studied). Treating them the same would overestimate the harm / benefit
3. Cannot be used to calculate NNT
4. League tables of OR different to RRR.
This tutorial was prepared by Dr J A Crane
Critical reading prompts
IMROD – Una Coales
Introduction
Methods
Results
Discussion
Others
TBOAR
Title Design Tables Critical Evaluation Conflicts of Interest
Background Outcome Understandable Aims met Overall
Originality Subjects Response Rate Conclusions References
Aims Dropouts Applicability Ethics
Relevance Stats
DOS
TURDS
CACA
CORE
Making sense of a review – ten important questions
Address a clearly focused issue?
Looked at appropriate sort of papers?
Important, relevant studies included?
Qualities assessed of these studies?
Reasonable to combine results?
What is the overall result of the review.
How precise are these?
Can the results be applied to local population?
All outcomes considered?
Are the benefits worth the harm/costs?
Oxman, AD et al. JAMA 1994;272(17) 1367-71
EBM deceptions
Surrogate End Points
Composite End Points
Relative Risk Reduction
Absolute Risk Reduction<
Inclusion Exclusion Bias Representiveness
Reliability Validity Generability
Surrogate Endpoints BMJ Aug 2011 @ NeLM
Surrogate Endpoints BMJ Jan 2012
Treat to Target – Treat the numbers or treat the patient? NeLM
| EBM / Stats Glossary | |
|---|---|
| Incidence | |
| Prevalence | |
| Absolute Risk Reduction | difference in rate of events between the two groups risk of event in control grp-risk of event in Rx group |
| Relative Risk | How many times more likely is it that an event will occur in the treatment group compared to control group RR is risk in treatment group/risk in control group often preferred by drug companies |
| RRR | Relative risk reduction (RRR) Tells us the reduction in the rate of the outcome in the treatment group relative to the control group. RRR = ARR/risk of outcome in control group or RRR = 1-RR |
| Hazard Ratio | form of relative risk HR>1 means an event is more likely to happen in the treatment group than in the placebo group |
| Odds Ratio | another way of expressing probability or relative risk –OR>1 means an event is more likely to happen in the treatment group than in the placebo group |
| Sensitivity | the proportion of people with a disease who are detected by the test (true positives) Sensitivity =TP/(TP+FN) Eg. You work out the proportion of cancers detected as a proportion of all the cancers. High sensitivity – good test for cancer |
| Specificity | the people who don’t have the disease (true negatives) and don’t test positive (ie. they test negative) Specificity = TN/(TN+FP) Eg. You work out the proportion of the people who haven’t got cancer and test negative for cancer as a proportion of all those without cancer. High specificity = few false positives. |
| Likelihood ratios | incorporate both sensitivity and specificity. For a positive test result it is: The ratio of: the probability of a positive test result in those with the disease To: the probability of a positive test result in those who do not have the disease. Put another way the likelihood ratio of a positive test is: sensitivity/(1-specificity) The likelihood of a negative test is: (1-sensitivity)/specificity |
| Pre-test probability | |
Simple statistics
Arithmetic mean aka average
add all the results up and divide by the number of results you had
Mode
most common
Median
line up all the numbers in order, and the median is the middle number
Interquartile Range
the difference between the 25th quartile and 75th quartile of data (ie. the middle 50% of data).
Incidence prevalence morbidity mortality
Basic Statistics health.state.ny.us
Sensitivity and specificity
Sensitivity specificity med.emory.edu
Understanding sensitivity and specificity with the right side of the brain BMJ 2003
Sensitivity and Specificity medpedia.com
Clinical tests: sensitivity and specificity ceaccp.oxfordjournals.org
Likelihood ratios
Introduction to Likelihood Ratios gim.unmc.edu
Pre-test probability
Post-test Probability
PPV
NPV
Risk reduction
Absolute Risk Reduction
difference in rate of events between the two groups
risk of event in control grp-risk of event in Rx group
Relative Risk
How many times more likely is it that an event will occur in the treatment group compared to control group
RR is risk in treatment group/risk in control group
often preferred by drug companies!
Relative risk reduction (RRR)
Tells us the reduction in the rate of the outcome in the treatment group relative to the control group.
RRR = ARR/risk of outcome in control group or RRR = 1-RR
Hazard Ratio
form of relative risk HR>1 means an event is more likely to happen in the treatment group than in the placebo group
Odds Ratio
another way of expressing probability or relative risk –OR>1 means an event is more likely to happen in the treatment group than in the placebo group
NNT and NNH
Dr Chris Cates’ EBM Website nntonline.net
Numbers needed to treat NNTs
how many people have to be treated for 1 person to benefit.
An ideal NNT is 1; everyone treated gets better, no one given the placebo group gets better.
NNH numbers needed to harm
NNT/H should, but don’t always, quote a time frame
NNT = 1/ARR (absolute risk reduction)
Confidence intervals
Confidence intervals stattrek.com
Confidence intervals onlinestatbook.com
null hypothesis
null-hypothesis.co.uk what is a null hypothesis
p value
type 1 and type 2 errors
type 1 and type 2 errors intuitor.com
Forrest plot
Power calculation
Power Calculation statsoft.com
dssresearch.com statistical power calculators
Triangulation
Triangulation research methods johnnyholland.org