By Beth Fields, PHD, OTR/L and Ellen Erdman, PT, DPT, HPCS
Our everyday practices are informed by our professional knowledge base and personal experiences. We blend this information to guide us and rationalize that our actions in practice are evidence-based. During our professional education, we learn that evidence-based practice (EBP) is “the integration of best research evidence (when available) with clinical expertise and patient values” (Sackett et al., 1996, p.312). In the absence of research evidence, we must rely on our clinical expertise and the values and preferences of our clients.
There is increasing evidence to support the powerful impact of incorporating hippotherapy into treatment plans for clients with a variety of conditions across the lifespan. Clinicians have a responsibility to stay informed of current evidence, which requires a certain set of critical inquiry skills. The AHA, Inc. Research Committee encourages clinicians to use the PICO format to engage in the EBP process. PICO represents components of a clinical question: Patients, Intervention, Comparison, and Outcomes. When reviewing evidence, start by noting these PICO components to help you develop a clinical question of interest that will contribute to your treatment planning and implementation with your client. Please refer to the following resource for more detailed examples of PICO framework: https://www.ncbi.nlm.nih. gov/pubmedhealth/PMHT0029906/.
The following is a guide developed to assist clinicians in making informed decisions about published literature focusing on interventions that utilize equine movement in conjunction with the larger equine environment to engage clients’ physical, sensory, emotional, and cognitive systems. The image below provides a quick hierarchical rating of investigative articles. The top of the pyramid lists types of evidence with the highest rigor; each layer below describes a design with less scientific rigor, with the bottom layer being expert opinion.
With the help of supporting literature, the AHA, Inc. Research Committee assembled the following considerations when evaluating evidence. Applying these considerations to your own evaluations will help facilitate the development of your critical inquiry skills.
SOURCE AND AUTHOR CREDENTIALS
Articles can be published in an array of forums, from medical journals to personal blogs. Readers need to understand the spectrum of evidence and be able to filter out less-credible information. The strongest peer-reviewed journals require authors to undergo extensive review and edits before publication. Many submitted articles are rejected due to flawed analysis or conclusions. Because of this vetting process, publications in peer-reviewed journals are considered to present stronger evidence. Typically, publication in a peer-reviewed journal involves the approval of an Internal Review Board from a university, which assures public safety and privacy. If an article is published in a non-peer-reviewed/medical journal, the information may be valid, but you might question why the results were published in this forum.
Is the author a licensed physical therapist, occupational therapist, or speech/language pathologist? Is the intervention described provided by a licensed physical therapist, occupational therapist, or speech/ language pathologist? If not, you might question if the intervention is truly hippotherapy (defined as therapy incorporating equine movement and the larger equine environment). If the provider is not a licensed therapist, you may still glean important information from the article, but the information should be interpreted through a lens of understanding that the intervention provided was likely not hippotherapy.
BACKGROUND/ THEORETICAL FRAMEWORK
A well-written paper will present existing evidence in the introduction of the paper. Does the paper present supporting literature and a framework or model to guide the study process and to help situate and/ or explain outcomes from study?
Look at the research question. This is typically written near the end of the introduction section of a paper. A focused investigation will clearly state the purpose of the investigation in regard to the client and intervention, often specifically stating something like, “the purpose of this study is to investigate the impact of the use of equine movement on by a therapist for clients with .” Is the client population (diagnosis, age, therapy discipline, impairments, etc.) similar to your client population or interests? Do the therapy approaches make sense to your practice? Can you generalize aspects of the findings/ results to your practice? If the study does not match your clientele/setting, it could still provide interesting reading and ideas.
What type of research is the focus of the paper: quantitative (measurements of outcomes using numerical data), or qualitative (narrative description to discern a deeper understanding)? Does the research aim to 1) describe a population or participant (descriptive study), 2) find relationships (exploratory study), or 3) establish cause and effect (experimental study)?
As mentioned previously, published information covers a range of validity from expert opinion—the least validated form of evidence, usually gained from life experience and success stories—to the strongest evidence, an analytical review of controlled studies. Within this range, there are a variety of research designs, ranging from a case study of one client’s course of care to a randomized controlled trial with large numbers of participants. Areas to consider:
- How many participants are in the study? Studies with larger numbers lead to more statistically significant results.
- If hippotherapy is incorporated, what discipline is the therapist? How often is hippotherapy used? How long is an individual session? What is the duration of the episode of care? Is a protocol provided, including information regarding type of horse, equine movement, equipment used, client positions, mounted activities, etc.?
- Is there a control group? A sound study design has a control group to better establish that any change was due to the intervention, versus another causative factor. One strategy might be to have a new client act as his/her own control. This is a common research method, but it does impact the timing of therapy. For a client to be his/her own control, baseline measurements are taken, but the experimental intervention (hippotherapy) is not implemented right away. A second baseline assessment is taken 4, 8, or more weeks later. Any change in the client during this period is not linked to hippotherapy. At this point hippotherapy can be added. Any future change can be more readily linked to the addition of hippotherapy versus general maturation/development, other therapies, or any additional factors.
- How were the participants recruited and assigned to a group? Was it a “sample of convenience”— that is, everyone on the waiting list or who expressed interest? Were participants recruited from a variety of sources? Were the participants randomly assigned to the experimental group that received the intervention or a control group (random assignment/allocation)? Were the control and experimental groups similar at the start? The more alike each group is regarding age, diagnosis, functional level, and presence of any comorbidities, the easier it is to draw conclusions that are based on the impact of the intervention versus other factors.
- Is there an obvious bias? If the author is the person to administer any testing and is also the provider of the intervention (also known as an unblinded study), there is opportunity for bias in interpretation of the results. The researcher or author may knowingly or unknowingly describe the participant in the light they want to see them in.
- Was the experimental intervention/hippotherapy the only difference between the groups? Or, were there multiple variables that could have caused a difference in results? Hippotherapy is provided in a rich environment. It is difficult to isolate equine movement from the equine environment and the social interactions. A control group may seek to recreate some of these aspects to more clearly link any change in outcome to equine movement versus the equine environment.
- Was the intervention provided repeatable? A therapy technique, intervention, or procedure gains strength in acceptance if more people use it and report positive outcomes. Clearly defining and describing the intervention provides a way for others to repeat the protocol and hopefully add to the strength of the intervention. Unfortunately, because hippotherapy is individualized to a client, it can be difficult to describe a consistent protocol.
- Were the selected outcome measures valid, reliable, and acceptable measures in your professional community? If an outcome measure is not reliable, valid, easy to use, or meaningful, its use and the information stemming from that use might be questioned.
Are the reported changes meaningful, either statistically or clinically? A study can provide clinically significant information without providing statistically significant change. Often, smaller studies with less than 50 participants produce clinically significant observed results, but because the number being analyzed is small, mathematical significance is not demonstrated. This does not mean the study or outcomes are not worthy. It may be that the limited number of participants limited the mathematical impact of the study.
This dilemma of clinically significant change without statistically significant change can potentially be avoided if the researcher conducts a power analysis prior to the start of the study. A power analysis can determine the minimum number of participants needed to yield a significant change. Unfortunately, particularly with research involving pediatric participants or individuals with neurological deficits, the number of available subjects and matched control groups are not readily available, so larger scale studies can be difficult to conduct. Because of this, many quality studies lack the number of participants needed to produce a statistically significant change. If a study with a small number of participants meets most of the criteria discussed above and does not report statistically significant change but does report clinically significant change, its results might still be relevant.
Do the results report information on all the areas tested, that is, all the outcome measures? Did the authors report any statistical analysis? Are the changes documented what you expected? Are the results reported linked to the outcome measures used? An upcoming Research Committee document will further address statistical data interpretation tips.
Do the conclusions make sense to you? Perhaps most importantly, is any change meaningful to the participant? While research studies involving body systems/impairments provide a solid base, changes in a client’s functional abilities or ability to participate in society have more impact on the successful use of a therapeutic intervention, the client, and others. The author’s or researcher’s interpretation and discussion of the changes can be just as valuable as the actual data. Does the author present a reasonable explanation for any change or lack of change in the client? Is a theory or reference to a previously published study discussed to support the conclusions?
The author will discuss potential limitations, biases, flaws, or mistakes made in the investigation or analysis. A limitation could include the small population size, no control groups, potential bias if the person providing the intervention also administered any measures, and other factors. By exploring the limitations of the study, the authors share weaknesses, biases, and possible suggestions to increase rigor.
CLINICAL UTILITY AND FEASIBILITY
After reviewing an article, you can decide if the information presented has a practical application to your practice, or whether the authors have stimulated your mind to reassess and possibly change your practice. Are the financial, time, and social costs worth the intervention? Realistically, using equine movement and the larger equine environment in a client’s plan of care can be an expensive option. If the clinician adds specialized equipment, training, or personnel, cost may increase further. These costs could be interpreted as either investments in your future or an excess that is not reasonable for your practice. Have the authors described an outcome measure or provided an intervention in a way you had not before considered? Will the intervention described enhance your client outcomes? Perhaps components of the described intervention will have a positive impact. Often aspects, pieces, or interpretations of published literature can be applied to practice. After thoughtful consideration of the validity, strength, and clinical changes, apply what is best evidence. If someone questions you about why you chose a specific intervention, you can explain, cite an article, and feel good about practicing with evidence.
The Research Committee is currently working on developing a research evaluation toolkit for AHA, Inc. members. Our goal is to help clinicians develop the skills necessary to make educated judgements about evidence on the use of hippotherapy in their practice. ◀
SUGGESTED FURTHER READING
Burns, PB, Rohrich, RJ, Chung, KC. (2011). The Levels of evidence and their role in evidence-based medicine. Plastic and Reconstructive Surgery, 128(1), 305-310. doi:10.1097/ PRS.0b013e318219c171.
Miser, WF. (1999). Critical appraisal of the literature. Journal of American Board of Family Practice, 12(4), 315-333.
Sackett, DL, Rosenburg, MCW, Muir Gray, A, Haynes, B, Richardson, SW. (1996). Evidence based medicine: What it is and what it isn’t. BMJ, 71, 312. doi: https://doi.org/10.1136/ bmj.312.7023.71