COPRH Con Learning Community

In your approach to research, what team are you on: team Rigor or team Relevance?

In the spring of 1998, I took an Introduction to Psychology elective as a freshman at Carnegie Mellon University in Pittsburgh. We learned about Freud, and dream interpretation, and how apes open bananas (from the other side), and cults, and how much sleep you really need, and how the tips of your fingers have more nerve receptors than the skin on your back. We discussed John Locke and the classic nature vs nurture debate and what shapes the person you become, your well-being and success in life. Nature is genetics and biology - it is all pre-determined. Nurture is your physical, social, and physiological environment - it is all what you are exposed to.

Which is right, nature or nurture? Of course, we know the answer is, “Both.” Similarly, I have often sought to have my cake and eat it when considering these sorts of either/or questions in my life. should I study chemistry or psychology? I did both. Should this physical activity intervention focus on building self-efficacy or improving neighborhood walkability? Again, I did both. What matters more, internal or external validity? Should I use quantitative or qualitative methods for this proposal? Soup or salad? The answer for many of us is either “both”, or “it depends.”

In health services and public health research, there are many decisions to make about what to study, how to conduct our research, and how to disseminate our findings. As scientists, we are taught rigorous methods for identifying cause (X) and effect (Y), establishing efficacy under tightly controlled experimental conditions, and eliminating bias by removing extraneous variables from the Y = X equation. We acknowledge there is some error in our estimation of cause and effect; Y = X + e, of course, as we don’t presume to know all possible sources of bias. We acknowledge that Y = X only sometimes, such as when circumstance M is present (it does, in fact, depend).

Sources of bias can be methodological, interpersonal (how researchers interact with their subjects or with how subjects respond to the experimental context can be “biased” - in the sense of not being the same for everyone), or intrapersonal (how researchers think about or interpret a phenomenon can be influenced by their own experiences and perspectives). Our study designs and analytic methods are fundamentally about identifying causal effects without bias. In a nutshell, this is the essence of what we mean by Rigor. We must have rigor to make accurate conclusions about what interventions improve health under which circumstances, so that we as a society can make appropriate investments. (See Herbert Weisberg’s text Bias and Causation for an in-depth perspective on this topic.) If you are All About That Bias, then you are rooting for Team Rigor.

The problem with eliminating bias through carefully controlled experiments is that we have stripped away all the context, all the M (and N, O, P, Q, R and S) that tells us when and for whom X affects Y. We don’t even know that Y is a problem for the P (population, practices, policymakers) or that X will be feasible or sustainable in the O (organization) or S (setting) or that X will be acceptable and accessible to a sufficient N (number of people) to have an impact on Q (quality of life, quality of care, etc). That is, we don’t know the R - how Relevant is the research. We ensure relevance, for example, through community and practice engagement, through assessing and adapting to context, and designing research that matters to our OPPS. If for you, these ABCs are as easy (and as high priority) as 123 and you’re down with the OPPS, then you are rooting for Team Relevance. (see also Kessler and Glasgow 2011)

There is good news: Pragmatic researchers can root for BOTH team Rigor and team Relevance.  And… it depends a bit which team you are pulling for more strongly. If the level of evidence to prove the efficacy of a given program, you will need to attend more to eliminating bias and rooting for team RIGOR than you would in situations where you are testing a very evidence-based program, such as the Diabetes Prevention Program.  However, if you ignore team Relevance altogether when proving the efficacy of a program, then you may find this program is efficacious but lacks effectiveness when introduced into real-world settings.

Tell us your thoughts on rooting for team RIGOR or team RELEVANCE in your current research -- we want to hear how this resonates for your work.

References:
1. Weisberg HI. Bias and Causation: Models and Judgment for Valid Comparisons. Hoboken, NJ: Wiley; 2010. 
2. Kessler R, Glasgow RE. A Proposal to Speed Translation of Healthcare Research Into Practice. American Journal of Preventive Medicine. 2011;40(6):637-644. doi:10.1016/j.amepre.2011.02.023 

The movement towards more pragmatic research is spreading across the spectrum of clinical and translational research in health.^1-3  “Pragmatic” research is “undertaken in the ‘real world’ and with usual care and is intended to help support a decision on whether [and in what context] to deliver an intervention.”⁴ The traditional randomized controlled trial (RCT) conducted under decontextualized, ideal conditions is seen as expensive, slow, and less relevant to real-world care.^7,8 Pragmatic methods have grown in use and acceptability, anticipating evidence that is rigorous AND relevant, feasible, and actionable in real-world settings.^2,5,6 However, pragmatic research represents a major culture shift from traditional research - emphasizing real-world relevance as an equivalent priority alongside rigor. This first blog post for The Pragmatic Scientist begins the conversation around this culture change by exploring the norms, beliefs, and values of pragmatic research to be endorsed by our learning community. 

In general, a learning community (LC) refers to a group of people with a common interest or goals who work in collaboration to meet learning or improvement needs (definition adapted from similar concepts across a variety of sources). A virtual or online LC (i.e., VLC) refers to “a public or private destination on the Internet that addresses the learning needs of its members by facilitating peer-to-peer learning.” VLCs can include both synchronous and asynchronous member interaction, platforms for knowledge sharing, networking, and collaboration, and online learning. The COPRH VLC will incorporate all of these concepts and platforms, reflecting “a group of scholars with a common interest in improving health and health care who work in collaboration to build the science and practice of pragmatic research in health.” To draw a parallel with the classic education LC focus on collectively improving teaching and health care LCs’ focus on quality improvement, we work to collectively improve the pragmatic science enterprise. From the LC literature, the following learning community themes emerge:

• Commitment to common purpose, goals, and values
• Leadership that promotes a shared vision
• Collaboration, networks and partnerships
• Collaboration, networks and partnerships
• Collective learning
• Individual improvement (professional development)
• Inclusivity (inclusive of the broad community of stakeholders across contexts and settings)
• Sustainability (of the LC and the learning and connections it promotes)

A 2005 British report on Creating and Sustaining Professional Learning Communities identified five characteristics of effective PLCs in the field of education: “shared values and vision, collective responsibility for pupils’ learning, reflective professional inquiry, collaboration focused on learning and group as well as individual, professional learning.” This same group also noted the importance of “inclusive membership; mutual trust, respect and support; openness, networks and partnerships.” Roberts and Pruitt describe building a collaborative culture as a critical step in developing an effective PLC. PLC leaders need to nurture a collaborative culture by establishing a shared vision around a common set of norms, values, and beliefs that drive that culture. 

And so we begin with exploring the expected norms and core beliefs and values of pragmatic scholars that should underlie the work of the COPRH VLC. For instance, the VLC may endorse the belief that an important aspect of pragmatic research is stakeholder engagement. A core principle may be team science. 

Please share your thoughts with us by responding to the following prompts:

1. As a pragmatic research scholar, what words or phrases best reflect core values regarding the planning, conduct, and dissemination of pragmatic research? That is, what are the important principles you expect to see demonstrated by those conducting pragmatic research?
2. As a member of the COPRH VLC, what suggestions do you have for how the learning community will collaborate and engage in shared learning? That is, how might members interact with each other, share knowledge, and promote the core values of pragmatic research?
3. What questions do you have about the COPRH VLC?

In our next blog post, we’ll report the results. Stay tuned!

References:

1. Dal-Ré R, Janiaud P, Ioannidis JP. Real-world evidence: How pragmatic are randomized controlled trials labeled as pragmatic? BMC medicine. 2018;16(1):49. [Pubmed Link]
2. Ford I, Norrie J. Pragmatic trials. New England journal of medicine. 2016;375(5):454-463.[Pubmed Link]
3. Mullins CD, Whicher D, Reese ES, Tunis S. Generating evidence for comparative effectiveness research using more pragmatic randomized controlled trials. Pharmacoeconomics. 2010;28(10):969-976. [Pubmed Link]
4. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. bmj. 2015;350:h2147. [Pubmed Link]
5. Glasgow RE, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clinical and Translational Science. 2012;5(1):48-55. [Pubmed Link]
6. Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10(1):37. [Pubmed Link]
7. Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: a call for a rapid learning health research enterprise. Clinical and translational medicine. 2013;2(1):10. [Pubmed Link]
8. Kessler R, Glasgow RE. A proposal to speed translation of healthcare research into practice: dramatic change is needed. American journal of preventive medicine. 2011;40(6):637-644. [Pubmed Link]