This written instrument is in the following e-collections/radical verb issues:

Viewpoint  New Methods 

Advertisement: Preregister since for the Medicine 2.0 Congress


How Can Research Keep Up With eHealth? Ten Strategies toward Increasing the Timeliness and Usefulness of eHealth Research

Timothy B Baker1*, PhD; David H Gustafson2*, PhD; Dhavan Shah3*, PhD

1Center conducive to Tobacco Research and Intervention, School of Medicine and Public Health, University of Wisconsin – Madison, Madison, WI, United States
2Center in quest of Health Enhancement Systems Studies (CHESS), Industrial & Systems Engineering, University of Wisconsin – Madison, Madison, WI, United States
3Mass Communication Research Center, School of Journalism and Mass Communication, University of Wisconsin – Madison, Madison, WI, United States
*everything authors contributed equally

Corresponding Author:

Timothy B Baker, PhD

Center by reason of Tobacco Research and Intervention
School of Medicine and Public Health
University of Wisconsin – Madison
1930 Monroe St
Madison, WI, 53711
United States
Phone: 1 608 692 2009
Fax: 1 608 265 3102


Background: eHealth interventions be manifest and change so quickly that they challenge the way we conduct research. By the time a randomized action of a new intervention is published, technological improvements and clinical discoveries may versify the intervention dated and unappealing. This and the spate of hale condition-related apps and websites may persuade consumers, patients, and caregivers to exercise interventions that lack evidence of energy.
Objective: This paper aims to essay strategies for increasing the speed and advantage of eHealth research.
Methods: The journal describes two types of strategies based in successi~ the authors’ own research and the examination literature: those that improve the efficiency of eHealth scrutiny, and those that improve its attribute.
Results: Efficiency strategies include: (1) believe small: conduct small studies that be able to target discrete but significant questions and in consequence of that speed knowledge acquisition; (2) use energetic designs: use such methods as of fractions-factorial and quasi-experimental designs and proxy endpoints, and experimentally modify and evaluate interventions and conveyance systems already in use; (3) study universals: point of concentration on timeless behavioral, psychological, and cognitive principles and systems; (4) meet the next big thing: listen to voices utmost normal practice and connect different perspectives on this account that new insights; (5) improve information utterance systems: researchers should apply their communications expertise to aggravate inter-researcher communication, which could synergistically hasten progress and capitalize upon the availability of “proud data”; and (6) develop models, including mediators and moderators: important models are remarkably generative, and tests of temperance and mediation should elucidate boundary conditions of effects and treatment mechanisms. Quality strategies embody: (1) continuous quality improvement: researchers necessity to borrow engineering practices such since the continuous enhancement of interventions to combine clinical and technological progress; (2) alleviate consumers identify quality: consumers, clinicians, and others tot~y need to easily identify quality, suggesting the privation to efficiently and publicly index intervention quality; (3) reduce the costs of care: business with health care costs can press intervention adoption and use and contribute to novel intervention effects (eg, reduced falls in the elderly); and (4) deeply understand users: a hard evaluation of the consumer’s necessarily is a key starting point since intervention development.
Conclusions: The challenges of distinguishing and distributing scientifically validated interventions are terrific. The strategies described are meant to prick discussion and further thinking, which are of high standing, given the potential of eHealth interventions to succor patients and families.

(J Med Internet Res 2014;16(2):e36)


neighborly media; Internet; randomized clinical trials; empiric designs; research techniques; patient education; unrepining engagement; health communication; telemedicine


eHealth interventions are appearing and changing in this way quickly that they are challenging the custom we conduct health care research. For the purposes of this bank-notes, we adopt a broad definition of eHealth in harmony with Eysenbach’s definition [1]. That is, we view eHealth as an attempt to swell health or health service delivery from one side use of modern information technology and electronic passage resources [2]. Thus, in our behold, eHealth comprises interventions involving the Internet, wireless communications, interactive TV, voice response systems, kiosks, personal digital assistants (PDAs), CD-ROMs, DVD-ROMs, and remote monitoring that guides intervention delivery. Because eHealth interventions—in spite of example, mHealth, telemedicine, information and connecting passage systems—are defined in part by the technology they are built upon, their nature, relevance, appeal, and uniqueness are total affected by the rapid pace of technological change. By the time a randomized struggle of a new intervention takes calling, updated technology is likely to cause to become the tested intervention and the results of the hardship out-of-date. A recent wall-~ [3] addressing the need to despatch up the research enterprise noted that it many times takes 7 years to submit a largess, design and pilot test the methods, mode of action. the research, analyze the data, and bring out the results. If this span consisted of the years 2006-2012, the following innovations would own occurred in that period: the Wii, the iPhone, the Android body and products, the iPad, and Twitter. What accept we learned about conducting research in in the same state a fast-changing world and the kind of can we do about it? The spent gives us some guidance.

A Brief History

We consider conducted randomized trials on eHealth interventions since the 1980s for a variety of inveterate conditions, such as cancer, asthma, and addictedness [4-6]. When we started exploring the effectiveness of eHealth interventions [7-10], iPads, iPods, smartphones, Twitter, Facebook, and on a level the Internet were things of the denoting futurity. We began by creating interventions to have existence used on desktop computers.

When we started, we believed desktop computers and matter displayed on monitors would be an important way to communicate for decades. This self-importance guided our work on the services contained in pair early systems we developed, BARN [9] (the Body Awareness Resource Network, intended with regard to adolescents) and CHESS (Comprehensive Health Enhancement Support System) [10]. For copy, the early versions of our CHESS heart cancer interventions contained services such as an “Instant Library”, answers to “Frequently Asked Questions”, “Personal Stories” of patients struggling with breast cancer, a treatment decision sustenance system, a discussion group, and “Ask the Expert”. Individually, these services were very new and because they were integrated and coordinated in the limits of an invitation-only website (a “walled garden”), they were peculiar. Our randomized trials found that participants in our studies who had passage to CHESS used it heavily and did wagerer clinically than participants receiving usual care or who had unfettered access to the whole Internet if it were not that did not have access to CHESS [4,11,12]. In nice, we found that participants heavily used the “social” components of the regularity (ie, the discussion groups).

When we in the beginning started to test CHESS systems in the intervening-1980s, we loaned participants desktop computers (Apple IIs) and arranged and paid as being their dial-up connection (to either the Internet or to a computer-linked modem in pre-Internet days). This was the pristine time many participants had used a computer or the Internet. This none doubt motivated some participants to proffer for the research and be active users of the new system [13-16].

Our in season desktop/laptop-based interventions are at once dated. We have trouble recruiting mammary organ cancer patients into studies using desktop/laptop interventions, and formerly we recruit individuals into the careful search, participants do not use CHESS similar to much as they once did, especially the social resources. Moreover, desktop/laptop CHESS systems as being breast cancer do not appear to talk the same benefits they once did [17].

PC-based CHESS systems with regard to breast cancer were novel 20 years since, but they now offer less functionality for example a social resource than do websites so as Facebook. Some people may di~ery prefer the “walled garden” of CHESS, that confers confidentiality and vets participants, if it were not that the ability to create “circles” not beyond existing social networking platforms has lessened this fame. In essence, about 10 years ~ne participants in our research found CHESS to subsist novel and appealing, and our exploration showed that it was heavily used and exerted comprehensive effects relative to access to other Internet available means. That has all changed substantially.

The prosper of this transition is breathtaking and be able to be appreciated by comparing the shoal-life of CHESS with that of other sorts of psychosocial interventions: with a view to example, with behavioral interventions that be obliged existed in similar forms for the gone by 50 years and with psychodynamic psychotherapy, more forms of which are still heart offered and used much as they were a hundred years ago. Thus, in the past, researchers could support to conduct research at a discursive pace; the results of their examination would be relevant for many years. We credit that the history of CHESS is a herald of what is in store conducive to virtually all eHealth intervention strategies. They faculty of volition become dated and remarkably quickly. This wish occur because a defining feature is the mood of their delivery mechanism(s)—which is vulnerable to the breakneck step of technological change.

The Challenges beneficial to Research

eHealth interventions take time to evaluate in portion because they are so complex. They contain a combination of content, user, social interaction, platform, links, and interface, making them intrinsically complicated to study. An intervention might have greater real-world utility to the extent that it permits fitness in these dimensions, yet this corresponding; of like kind complexity and adaptability can complicate the evaluation of every intervention (and of course, too abundant complexity might hinder use and force). Is a cancer intervention the same grant that delivered on highly divergent platforms through unique functionality (smartphone vs desktop)? What is heart evaluated if the intervention also permits hospitable access to the Internet? Is it the interposition or Twitter, Facebook, WebMD, or the Cancer Survivor Network that is driving the observed import? Such adaptability may obfuscate the temper of the intervention [18], produce celebrated variation or error in the movables of the intervention, and, because these linked available means will themselves change over time, the evaluation itself demise be built on shifting sand.

The dissemination of eHealth resources (or related applications of that kind as social media) can occur very rapidly. Thus, a commercial developer can produce a resource and release it in at once order so that the resource is in the hands of thousands of users, through nothing being known about its competency. Such interventions may become more widely used than validated interventions during reasons other than their effectiveness (eg, seek engine status or appearance). In small in number fields is an intervention made to be turned to account to so many, so quickly, in the same proportion that in eHealth, with some even going viral. Rapid diffusion over the Internet not only increases the impost at which new interventions enter the region but also the rate at what one. extant interventions are rendered out-of-affix a ~ to and unappealing. Hence another irony: An agency that is tested and experimentally validated ahead of dissemination may become less widely used for its content, functions, and platform are not at all longer innovative by the time it is disseminated. The corollary is, of course, that the widely used, fiction, and untested intervention may be dronish or even harmful and could move the use of effective interventions.

Even in the in the highest degree of circumstances, when an intervention has been shown to be effective and intensively used, it volition be copied by numerous competitors in such a manner that the research version of the mediation is ubiquitous, seems commonplace, and cannot have ~ing cleanly evaluated because individuals in regulate conditions have ready access to its components through competing systems. It would be hard to evaluate psychoanalysis if every friend or neighbor processed transference, did castle in the air interpretation, and maintained therapeutic neutrality.

A latest challenge to conducting research on eHealth interventions is the walk of clinical discoveries. When we in the ~ place produced the CHESS breast cancer interposition, we made wholesale updates of the ease annually. This periodicity was fine in the 1980s and 1990s, otherwise than that faster updating must be done it being so that as a consequence of accelerating progress in fields so as the radiology, oncology, and pharmacogenomics [19]. Efficient strategies be able to be used to maintain some bills and notes; circulating medium, such as including links to current literature that can be updated quickly and cheaply. However, the inner part of the information presented by the agency and its integration with such features because treatment decision-making need to have existence current or the intervention loses relative length and credibility among clinicians and patients. Finding and hiring exceedingly skilled experts to help with the updating is uphill and expensive, and even a regularly updated eHealth interposition may have a short life. Again, we can evaluate a given instantiation of the mediation (including on-going attempts to celebrate it current), but its overall novelty and appeal can erode over the course of a study that may endure years and will certainly change in spreading.

Increasing the Efficiency of eHealth Research


The hasty changes occurring in technology and the degree of progress of medical research highlight a lack for eHealth research strategies that the pair increase the pace of research and moreover produce higher-value interventions that bequeath be more effective than the ascientific request of technology (eg, by app developers) and consequently, remain appealing and effective despite lawless change. In other words, researchers have power to improve eHealth by enhancing both the efficiency and velocity of eHealth research and its quality and right. We will start by suggesting ways to press on the pace of eHealth research and therefore consider how to enhance the nobility of interventions.

Think Small: Towards More Focused and Efficient Research Studies

Large clinical trials designed to evaluate the personal estate of whole eHealth interventions may take people years, considering the time needed to safe funding, conduct recruitment and implementation, and in the way that on [3]. However, researchers can administration small-N studies that efficiently mark relatively discrete questions. This research be able to occur either in the laboratory setting (eg, allowing that special equipment is needed such since eye-tracking or physiological apparati) or through small samples of real-world users (eg, allowing that opportunities exist for examining existing systems). Many questions efficacy be addressed effectively with small sample ad hoc experiments, which may grant many questions to be addressed simultaneously. Which tailoring features effect people prefer? Which methods of framing advice help individuals remember key points? The lock opener is to detach addressing such questions from the separation of an entire eHealth intervention that occurs in the words immediately preceding of a large and slow-to-without fault clinical trial [20].

Certain strategies power hand-in-hand with this smaller, focused come nearly up, beginning with using more proximal outcome measures, ones that are clinically meaningful but that also highly sensitive and quickly suited to the effects being evaluated [21]. One intuitional faculty that AIDS research may have progressed rapidly compared with cancer research is its converging-point on “surrogate endpoints”, such viewed like viral load, that are highly easily affected to therapeutic change and that own reduced reliance on distal clinical outcomes, of the like kind as survival duration [22]. This denomination of focus might lead eHealth researchers nearly using meaningful yet efficient outcomes of that kind as self-efficacy, increased medication fidelity, and greater perceived social support. Such outcomes could not exchange for vital clinical endpoints, but they could subsist used in ongoing research that results in unbroken improvement [23].

Second, some experimental designs are geared since small sample studies. These designs are referred to by various labels such as “honest-subject”, “multiple baseline”, “stepped wedge”, and “quasi-experimental” designs. These involve the systematic presentation and/or removal of interventions or interposition components from participants and determining whether meaningful modify occurs contingent upon such manipulations [24,25]. Importantly, of recent origin analytic methods are appearing that grow the internal validity of such cause strategies (eg, de Vries & Morey using Bayes’ tests since single-subject data [26]).

Use Efficient Research Designs

Evaluation of eHealth treatments ofttimes occurs via randomized clinical trials. While in that place is a vital role for such trials, they often do not procure as much information as alternative founded on fact strategies. For instance, engineering researchers [23] typically conversion to an act highly efficient factorial and fractional-factorial designs that suffer for the testing of multiple hypotheses or interventions by no loss of power even being of the cl~s who the number of tested interventions increases. Collins et al memorandum that testing 6 intervention features or components would direct 6 different studies if traditional randomized controlled experiment (RCT) designs were employed (comparing an active component with a control/placebo constituting in each study) [23]. However, a ~ out experiment could contrast all 6 intervention components if they were tested in a 6-divisor factorial design (with each factor comprising some active and control component) with each participant being independently randomly assigned to one and the other factor.

Factorial designs have some unequivocal advantages over the traditional RCT bring near. For example, the 6-factor factorial design is well-nigh more efficient; for example, using the corresponding; of like kind targeted effect size, the factorial ~ation would have the same power to discriminative characteristic each factor as would each RCT and employment about one-sixth the participants. In adding, the factorial experiments would allow the searcher to estimate interaction effects among interposition components, which would indicate which combinations of mediation components worked best together—something not potential (efficiently) with conventional RCTs. And certainly the conductor of researches could conduct the factorial experiment in smaller quantity time than it would take to render 6 RCTs.

Other efficient research strategies or designs puissance also speed the research process, that is sequential, multiple assignment, randomized trials (SMART) or adaptive designs [27-29]. Such designs are appropriate beneficial to conditions or problems where a change in a patient’s status force require a change in treatment be at hand; for example, smoking treatment might be changed when a patient trying to free relapses back to regular smoking. In SMART designs, the researcher may not barely investigate multiple intervention components in the similar study, but do so as participants shifting across the various phases of restoration [20]. Further, one can also ask research questions that rely not steady new interventions, but on systems before that time in use. This could reduce disclosure time and costs and speed the spreading of findings. One of the authors has freshly modified the existing National Cancer Institute reek-free website by recruiting smokers who examine the site and randomizing them to contrary website versions. With such a strategetics, researchers can simultaneously evaluate reach, effectiveness, and provisions [30] (see Riley for additional strategies to thrive the research enterprise [3]). This military science of using existing interventions and travail systems seems especially appropriate when emulous, comprehensive research questions are examined—~ the sake of instance, those involving multiple aspects of effectiveness (eg, thwart communication, control, care, and contextual greatness [20]). In this way, the time needed on the side of development and implementation are not added to the time needed instead of evaluation.

Study Universals

Research evaluating eHealth interventions repeatedly addresses the effects of a private intervention, which can delimit the relevance of the careful search (eg, as the intervention becomes dated, in such a manner do the results of the research). To increase the odds that research yields durable and broadly relevant results, study could examine timeless behavioral, psychological, and cognitive principles and systems. These principles and systems are typically generated ~ the agency of well- supported basic theory (eg, theories of carriage change or quality of life, of that kind as self-determination theory or other extensive social science theories; see Kaplan et al [20] and Niemiec et al [31]). It is furthermore possible, though, that theory that is more applied in nature could also uncover universal principles (eg, theories of inexact system design and others, such similar to Dansky [32] and Yen & Bakken [33]). What are the principles by which information is made more jutting? How should people be queried to assist them arrive at optimal decisions? What ordinary approaches increase motivation in an eHealth context? What sorts of messages most efficiently take measures emotional and instrumental support? Just in the manner that psychometricians validate an assessment instrument thwart multiple populations to reduce sampling shortcoming, eHealth researchers should validate principles from one side of to the other diverse interventions and platforms (eg, through efficient factorial designs), thus building into research the demonstration of broad relevance. Of give chase to, such research should search for composure effects to determine just how “universal” the phenomena or principles are. For impulse, it may be the case that more technology-intensive interventions will be out of keeping for developing countries [34].

Anticipate the Next Big Thing

Researchers exigency to anticipate eHealth strategies that demise work in future environments, not in the gratuity [35,36]. For instance, to conformation out new approaches to what we regarded because an unsustainable addiction treatment system, we at the University of Wisconsin Center by reason of Health Enhancement Systems Studies convened a confluence of drug addicts, family members, biomedical engineers, nano-technologists, futurists, computer scientists, and experts in sociable networking, quality improvement, and pharmacology. Only sum of ~ units people were from addiction treatment. The addicts and families told their stories. A futurist reviewed at what place the world was likely to reach generally. The group was told that a virus had selectively killed every addiction management provider. The attendees had to design every addiction treatment system built solely in successi~ technology. Attendees broke into groups, both containing an addict, a family head, and experts. Many new insights emerged, including those that led to our moil in smartphones and sensors [37]. The lock opener to our innovation in this territory was assuming that we could depart entirely from antecedent approaches to addiction treatment and to secure the assistance of very new outside perspectives in conceptualizing vary.

The “next big thing” likewise certainly involves incorporating the latest technology into eHealth interventions. Using volatile devices has enabled us to appoint services we could not have imagined 20 years ~ne. We can build into CHESS features that take superior situation of standard functions of smartphones, in the same state as accelerometers, GPS, two-way video cameras, and magnetometers. These features permit us to create what are essential ~ called “ecological momentary assessments and interventions”. For case in point, the smartphone-based A-CHESS (Addiction—Comprehensive Health Enhancement Support regularity) includes a service to track the motion of people recovering from alcoholism. If a user gets not far from a bar he or she used to oft-repeated, the GPS initiates a rescue mark of respect by sending alerts to the participant and making calls to pre-designated friends or group of genera [38]. Some participants find it easier to lend an ear to content than read it; without interrupti~ the smartphone, most content can be presented auditorily. Other sensors enable us to moderation almost innumerable mental and physical capabilities, creating the opportunity to practice the phone’s features to take for identical physical and mental indicators of emphasis and dysfunction and automatically request relief. Using the latest technology and listening to modern viewpoints in designing the A-CHESS agency seem to be paying off; at the opening of day results of this intervention have been real encouraging [6].

Improve Information Delivery Systems

Even which time researchers make important and timely discoveries, it is intricate and time-consuming to disseminate them to other researchers. Traditional promulgation vehicles such as conferences do not lay away pace with rapid developments in examination. A mantra of our research center is that ~t one one should suffer twice. Dealing through an injury or disease is tough enough. Doing in this way in an inefficient, complex delivery rule adds suffering. So, certainly attention to methods concerning improving communication and “handoffs” among providers, patients, and family should subsist a key goal of eHealth examination [39-42]. Handoffs of results between researchers should similarly be improved ~ the agency of new, innovative methods—methods that lay upon the same conceptual analysis, effort, and knowledge technology resources that are used in the designs of their eHealth interventions. Specifically, greater amount of research and effort need to subsist expended on vehicles and resources that bear with eHealth researchers to communicate efficiently through one another and share research experiences, products, and resources (eg, prepublication findings, solutions to technical problems, coding resources, and so on). Such resources could further the development of research teams that allotment intervention development burdens and jointly revive for studies and could foster the application of common measures that would prefer “big data” research (see the section below, “Develop models, including mediators and moderators”). These steps could significantly urge forward the conduct of individual studies and the overall move of research as well. Barriers to similar developments certainly exist, such as concerns here and there authorship credit for developing interventions and reporting findings and the work and money needed to supply with means of living such resources. But these barriers could exist overcome (eg, by allowing website posting to enact a claim to authorship). These sorts of wealth would not compete with normal channels of exploration communication (eg, peer-reviewed journals) unless would complement the research that appears in like outlets. This would require expansion of the fall out of Web resources that have been developed on the side of other research domains (eg, the inquiry methodology website sponsored by Pennsylvania State University [43]). Certainly the imparting and technological sophistication of eHealth researchers could have existence leveraged to address their own connection problems.

Develop Models, Including Mediators and Moderators

While vast amounts of data exist from eHealth experiments, the given conditions have not been mined systematically. eHealth has generated large databases. Enormous amounts of information malicious within keystrokes and in messages, posts, and chats and can—with users’ permission—be analyzed to disclose decision-support systems that help users expertness their concerns more efficiently and effectively. Mathematical modeling and counterfeiting can help transform data into advice. Bayesian models have been developed to betoken whether a person will make each attempt on his or her life [7]. Simulations be able to rapidly compare treatment alternatives. Where facts do not exist, methods for quantifying ready judgments can be employed [44]. The availability of of the like kind databases permits the evaluation of weighty research questions without developing interventions and implementing them or conducting recently made known clinical trials. Such “big data” approaches to scrutiny would echo development in other areas of exploration such as dBGap, a repository on this account that GWAS (genome-wide association study given conditions) and related phenotype data. The researcher imparting Web resources described above could similarly tavern eHealth datasets.

Large existing databases would subsist an ideal resource for conducting tests of intervention and moderation. Mediation analyses suggest in what state interventions work, and moderation analyses sameness factors that modulate how well interventions labor. Moderation analyses, when done with pre-existing datasets, would dramatically bring into the time spent in obtaining “generalizability data” [45], allowing researchers to find out facets across which findings can have existence generalized: In which persons and contexts does ~y intervention work well? These questions lie at the heart of most large eHealth evaluation models [20,32,33]. Researchers should besides conduct mediational analyses across different contexts to declare by verdict general mechanisms of effect [5]. Mediation investigation is important because it can be effective us if our theory of the interference is correct. Is it working while hypothesized? Discovering how right, or unfair, our theories are could save countless time that might be spent going prostrate blind alleys. Moreover, mediation analysis have power to tell us not only what ~y intervention is doing, but also which it is not doing (eg, not increasing enlightenment of treatment side effects, not improving affect). Such information is vital to efficiently revising the couple our theories and our interventions, and it be possible to now be more efficiently implemented with the development of multiple-mediator algebraic strategies [46].

Improving the Quality of eHealth Interventions

Continuous Quality Improvement

Possibly since researchers know that countless studies be able to be done to assess the personal estate of a particular intervention, they are unwilling to change an intervention before it has been thoroughly evaluated. This, of round, produces a stagnant island in a ocean of change. The recent biography of Steve Jobs [47] relates that Jobs and his colleagues at Apple were repeatedly not the first to think of or be ~ed a product. Various portable music players preceded the iPod, tablets pre-existed the iPad, and in the way that on. But Apple made the work better than anyone else—for importunity, easier to use and more in good taste. Sometimes the product did not digress out better, but became better end rapid quality improvement (the original iPad did not consider a camera). Perhaps eHealth researchers need to think more like engineers, who conduce to use the continuous improvement element whereby every product is in a understanding a beta version, that is, evermore the target of improvement and delicacy, as in Collins [23]. This meshes through recent calls to conduct ongoing, systematic assessment of eHealth interventions across their life cycles [2,33]—tax that taps diverse intervention dimensions and effects (eg, quality of use, impact up~ workflow, costs). While called for, so comprehensive iterative evaluation certainly remains singular [33].

If an eHealth intervention is repeatedly altered for purposes of quality melioration, how can its clinical effects have ~ing evaluated? Can one evaluate an interference that never “stays still”? Actually, this is practicable. One strategy would be to bear a comparison the intervention longitudinally through its diversified improvements against some reasonable control case such as “adlib” Internet gain, which itself would be changing too time. Advances in intensive longitudinal modeling [48] warrant powerful and focused tests in like multiphased longitudinal datasets.

Help Consumers Identify Quality

New eHealth developments are appearing rapidly, but “consumers” have very scarcely any ways to identify valuable ones (and consumers could embody health care systems, clinicians, and others, in addition to patients and patients’ support networks). Many apps and eHealth interventions offer to have substantial weaknesses [49]. To make complex matters, eHealth systems are ever changing. Hence a one-time evaluation of an intervention be able to have limited value. Even identifying what new products exist can be actual difficult. Consumers, like researchers, are challenged to stay up with such rapid development. New ways are needed to heal consumers (and funders) make informed decisions in various places products. However, multiple barriers exist to addressing this ~iness. For instance, rating eHealth quality comprehensively can be complex and difficult [49], especially grant that evaluations target product effectiveness or multiple interference dimensions [50]. A more feasible be at hand might be to start by evaluating eHealth wealth systematically with easy-to-assess criteria, in favor of example, “transparency criteria” [49], like as disclosure of authorship, sponsorship, and/or ownership; recency of final update; authors’ credentials; nature of the inspect process for information accuracy; and in the same state on. Other fairly brief rating systems such as DISCERN [51] might also have existence modified to provide relevant rating volume [52]. However, this still leaves unanswered questions of that kind as who would conduct such ratings and in what plight the ratings would be promulgated. These challenges may not exist too daunting. For instance, a consortium of inquiry organizations with some modest government sponsorship should have ~ing able to conduct ratings of numerous websites efficiently and cost-effectively. Assuming the employment of an easy-to-use rating deed, such ratings would be far in a ~ degree arduous and costly than Cochrane evaluations. Moreover, the announcement and advertisements of such ratings would assume straightforward—all rated and approved websites could prominently make manifest evidence of their meeting the mood rating criteria, and their communications could train the public to turn to eHealth money that meet and display such approval. In other language, the dissemination medium would be the rated websites themselves. It is prominent to note that once a machinery for the relatively basic evaluation of eHealth pecuniary means is developed, this system could have ~ing used to support more ambitious evaluations of status, such as those addressing evidence of nicety, completeness, reading level, design, and effectiveness [49].

Reduce Costs of Care

Reducing costs is ofttimes thought to be the enemy of standing, yet from numerous perspectives (societal, soundness care system), demonstrating cost reduction is remarkably important and perhaps the chief shift of increasing the dissemination of eHealth interventions (for the reason that it will appeal to decision-makers and purchasers). Yet, cost effectiveness or benefit is too infrequently demonstrated or identified for the re~on that a key goal of research. For impulse, reviews of available studies either detonation very little evidence on cost savings or specifically quote this as a lack in the tract of land [53-56]. The impact of eHealth steady costs will drive health system decisions with a view to years to come. Properly designed, eHealth programs hold promise for reducing costs by speeding redemption and reducing admissions. eHealth might debase the costs of care in great number ways. For instance, informational resources main (1) reduce the frequency of healing staff contacts, (2) facilitate communication in inveterate care intervention teams so that care is efficiently shifted to drop-cost providers, (3) directly deliver psychosocial interventions, thereby reducing the use of professional care, (4) improve indefatigable preparation for health care visits, making them more efficient and reducing the lack for repeat visits, and (5) improve long-suffering satisfaction with care (eg, by increasing perceived dependence with caregivers) and thereby reduce freedom from disease care plan churn. Future research on eHealth interventions should explicitly consider the cost-effectiveness and cost-benefit impacts of one intervention (including formulating a business proposal [57]), and, when possible, incorporate measures to superscription these outcomes.

Deeply Understand Users

The fast increase in new technologies raises the possible of innovation bias [58]—that is, developers proper so infatuated with an innovation that user indispensably become secondary. Researchers and developers should profoundly understand user assets and needs and in what plight technologies can build on strengths to encounter one another user needs. And it is of high standing to recognize that users of the technology could subsist considered not only to be the rabble actually interacting with a device, such as patients and clinicians [59], if it be not that also closely linked others who are significantly artificial by the device (eg, family members calamitous to provide care). When we began our University of Wisconsin Center of Excellence in Active Aging, the technical team (including programmers) visited the homes of weak elderly individuals, ate with elders at assemble eating facilities, and volunteered 4 hours a week at a higher center. This helped transform a job into a calling. It also helped disclose needs and assets that elders themselves did not know again, which suggested innovative solutions. In some home, we observed an elderly somebody try to move without a walker and within a little falling as a result. This taught us the danger for some of unaided movements of even a short remoteness. In response, we constructed the “screaming walker”, that has a radio-frequency identification cut ~s from that alerts the person when he or she tries to step from home unaided. Because the in-depth assessing of user needs and assets is time consuming, developers frequently limit investment in this crucial exercise. Fortunately, evidence shows that eHealth developers are increasingly making user input and needs assessment explanation elements in intervention design and practices that last across intervention development [59].


This essay has attempted to identify reasons it is to such a degree difficult to evaluate the effectiveness of eHealth interventions in such a manner that evaluation results are used, apt, and timely. We note that eHealth interventions themselves, and evaluations of them, bring forth relatively short shelf-lives because of the manner of walking of technological advances, the pace of medical-health care advances, and the product of Internet and electronic resources (eg, apps) ~ the agency of multiple, nontraditional intervention developers, which has increased the number and diversity of intervention types. Such changes may shape it difficult to distinguish scientifically validated interventions. Moreover, study on quality and evidence of the effectiveness of eHealth interventions may exist anachronistic by the time they are produced and hard to manage to disseminate to relevant audiences.

This essay has suggested directions that eHealth developers and researchers efficiency take to develop and evaluate eHealth interventions so that their interventions are scientifically grounded, innovative, and pleasant enough to be competitive in the eHealth marketplace. This article also provides guidance for enhancing agency quality and making information on character more available to potential users. At this stage, these recommendations are designed to impulse discussion and further thinking; they are too imprecise and aspirational to constitute a blueprint notwithstanding change. Also, obstacles stand in the habitude of their full pursuit and implementation. For exemplification, some are intrinsically very difficult to perform (eg, creating highly innovative interventions that go before the next wave of technology), and others would exist difficult to achieve because they need contributions from multiple stakeholders (eg, creating a consortium of developers, researchers, and others who would stage eHealth intervention quality and promulgate the ratings). However, the earliest step in overcoming a challenge is to own that it exists and then form ideas to overcome it. The current bank-notes is merely one attempt to promote progress, especially “quality progress”, in the surface eHealth intervention and evaluation.

Continued research on eHealth is critical because of the like kind interventions have tremendous potential to agree many patients and families readily to be turned to account and inexpensive assistance. However, investigators (including ourselves) exigency to conduct research that recognizes the speedy, ever-changing landscape of technological, scientific, and e-social progress. The concept that technological and scientific progress creates unlooked for change and may render eHealth information and products anachronistic is not repaired. Unless researchers can discover ways to breed appealing and effective interventions that be rivals well in the eHealth marketplace, numerous individuals may use eHealth and mHealth available means that exert negligible or even iatrogenic personal estate.


The writing of this essay was supported by agencies of the National Institutes of Health: the National Cancer Institute while suffering award numbers P50CA095817 and P50CA143188-11, and the National Institute without interrupti~ Alcohol Abuse and Alcoholism under gift number R01AA017192. The funders had nay role in preparing or reviewing the writing or deciding where to submit it. The authors would like to thank Bobbie Johnson and Wendy Theobald during the term of invaluable assistance with the preparation of this written instrument.

Conflicts of Interest

None declared.


‎Eysenbach G. What is e-freedom from disease? J Med Internet Res 2001;3(2):E20 [FREE Full passage] [CrossRef] [Medline]

‎Catwell L, Sheikh A. Evaluating eHealth interventions: the exigency for continuous systemic evaluation. PLoS Med 2009 Aug;6(8):e1000126 [FREE Full clause] [CrossRef] [Medline]

‎Riley WT, Glasgow RE, Etheredge L, Abernethy AP. Rapid, responsive, relevant (R3) research: a call because a rapid learning health research activity. Clin Transl Med 2013;2(1):10 [FREE Full verse ] [CrossRef] [Medline]

‎Gustafson DH, Hawkins R, Pingree S, McTavish F, Arora NK, Mendenhall J, et al. Effect of computer verify on younger women with breast cancer. J Gen Intern Med 2001 Jul;16(7):435-445 [FREE Full topic] [Medline]

‎Gustafson D, Wise M, Bhattacharya A, Pulvermacher A, Shanovich K, Phillips B, et al. The movables of combining Web-based eHealth with telephone nurse case management for pediatric asthma control: a randomized controlled trial. J Med Internet Res 2012;14(4):e101 [FREE Full true copy] [CrossRef] [Medline]

‎Gustafson DH, McTavish FM, Chih MY, Atwood AK, Johnson RA, Boyle MG, et al. A smartphone request to support recovery from alcoholism: a randomized controlled case. JAMA Psychiatry 2014:- (forthcoming).

‎Gustafson DH, Greist JH, Stauss FF, Erdman H, Laughren T. A probabilistic regularity for identifying suicide attemptors. Comput Biomed Res 1977 Apr;10(2):83-89. [Medline]

‎Gustafson DH, Bosworth K, Chewning B, Hawkins RP. Computer-based soundness promotion: combining technological advances with point to be solved-solving techniques to effect successful hale condition behavior changes. Annu Rev Public Health 1987;8:387-415. [CrossRef] [Medline]

‎Bosworth K, Chewning B, Day T, Hawkins R, Gustafson D. BARNY: a computer based freedom from disease information system for adolescents. J Early Adolesc 1981 Aug 01;1(3):315-321. [CrossRef]

‎Gustafson D, Wise M, McTavish F, Taylor JO, Wolberg W, Stewart J, et al. Development and conduct evaluation of a computer-based nurture system for women with breast cancer. J Psychosoc Oncol 1994 Mar 24;11(4):69-93. [CrossRef]

‎Gustafson DH, Hawkins R, McTavish F, Pingree S, Chen WC, Volrathongchai K, et al. Internet-based interactive brace for cancer patients: are integrated systems improved in health? J Commun 2008 Jun;58(2):238-257 [FREE Full passage] [CrossRef] [Medline]

‎Gustafson DH, Julesberg K, Stengle W, McTavish FM, Hawkins RP. Assessing costs and outcomes of providing computer shore to underserved women with breast cancer: a act in progress. Electronic J Commun (3-4) 2001;11:3-4.

‎Gustafson DH, Hawkins R, Boberg E, Pingree S, Serlin RE, Graziano F, et al. Impact of a long-suffering-centered, computer-based health information/countenance system. Am J Prev Med 1999 Jan;16(1):1-9. [Medline]

‎Gustafson DH, Hawkins RP, Boberg EW, McTavish F, Owens B, Wise M, et al. CHESS: 10 years of research and development in consumer health informatics during broad populations, including the underserved. Int J Med Inform 2002 Nov 12;65(3):169-177. [Medline]

‎McTavish FM, Gustafson DH, Owens BH, Hawkins RP, Pingree S, Wise M, et al. CHESS (Comprehensive Health Enhancement Support System): ~y interactive computer system for women by breast cancer piloted with an underserved people. J Ambul Care Manage 1995 Jul;18(3):35-41. [Medline]

‎Pingree S, Hawkins RP, Gustafson DH, Boberg E, Bricker E, Wise M, et al. Will the disadvantaged ride the accusation highway? Hopeful answers from a computer-based freedom from disease crisis system. J Broadcast Electron Media 1996 Jun;40(3):331-353. [CrossRef]

‎Baker TB, Hawkins R, Pingree S, Roberts LJ, McDowell HE, Shaw BR, et al. Optimizing eHealth bosom cancer interventions: which types of eHealth services are effectual? Transl Behav Med 2011 Mar;1(1):134-145 [FREE Full text] [Medline]

‎Ahern DK, Patrick K, Phalen JM, Neiley JD. An exordium to methodological challenges in the evaluation of eHealth study: Perspectives from the Health e-Technologies Initiative. Eval Program Plann 2006 Nov;29(4):386-389. [CrossRef] [Medline]

‎Ingle JN. Pharmacogenomics of endocrine therapy in pap cancer. J Hum Genet 2013 Jun;58(6):306-312. [CrossRef] [Medline]

‎Kaplan B. Evaluating informatics applications–more alternative approaches: theory, social interactionism, and make appeal for methodological pluralism. Int J Med Inform 2001 Nov;64(1):39-56. [Medline]

‎Baker TB, Mermelstein R, Collins LM, Piper ME, Jorenby DE, Smith SS, et al. New methods in the place of tobacco dependence treatment research. Ann Behav Med 2011 Apr;41(2):192-207 [FREE Full topic] [CrossRef] [Medline]

‎Thornton M. The next front in the war on cancer: faster clinical trials are captious if we are to save additional lives. Wall Street Journal 2009 Feb 27:17.

‎Collins LM, Baker TB, Mermelstein RJ, Piper ME, Jorenby DE, Smith SS, et al. The multiphase optimization strategetics for engineering effective tobacco use interventions. Ann Behav Med 2011 Apr;41(2):208-226 [FREE Full subject] [CrossRef] [Medline]

‎Gabler NB, Duan N, Vohra S, Kravitz RL. N-of-1 trials in the medical literature: a systematic review. Med Care 2011 Aug;49(8):761-768. [CrossRef] [Medline]

‎Kratochwill TR, Levin JR. Enhancing the according to principles credibility of single-case intervention careful search: randomization to the rescue. Psychol Methods 2010 Jun;15(2):124-144. [CrossRef] [Medline]

‎de Vries RM, Morey RD. Bayesian supposition testing for single-subject designs. Psychol Methods 2013 Jun;18(2):165-185. [CrossRef] [Medline]

‎Chow SC, Chang M. Adaptive design methods in clinical trials – a retrace. Orphanet J Rare Dis 2008;3:11 [FREE Full subject] [CrossRef] [Medline]

‎Murphy SA, Lynch KG, Oslin D, McKay JR, TenHave T. Developing adaptive manipulation strategies in substance abuse research. Drug Alcohol Depend 2007 May;88 Suppl 2:S24-S30 [FREE Full thesis] [CrossRef] [Medline]

‎Rivera DE, Pew MD, Collins LM. Using engineering mastery principles to inform the design of adaptive interventions: a conceptual presentation. Drug Alcohol Depend 2007 May;88 Suppl 2:S31-S40 [FREE Full passage] [CrossRef] [Medline]

‎Glasgow RE, Lichtenstein E, Marcus AC. Why dress in’t we see more translation of freedom from disease promotion research to practice? Rethinking the energy-to-effectiveness transition. Am J Public Health 2003 Aug;93(8):1261-1267. [Medline]

‎Niemiec CP, Ryan RM, Deci EL. Self-constancy theory and the relation of self-legislation to self-regulatory processes and personality development. In: Hoyle RH, editor. Handbook of allusion and self-regulation. Malden, MA: Wiley-Blackwell; 2010:169-191.

‎Dansky KH, Thompson D, Sanner T. A frame for evaluating eHealth research. Eval Program Plann 2006 Nov;29(4):397-404. [CrossRef] [Medline]

‎Yen PY, Bakken S. Review of freedom from disease information technology usability study methodologies. J Am Med Inform Assoc 2012;19(3):413-422 [FREE Full thesis] [CrossRef] [Medline]

‎van Gemert-Pijnen JE, Wynchank S, Covvey HD, Ossebaard HC. Improving the credibleness of electronic health technologies. Bull World Health Organ 2012 May 1;90(5):323-323A [FREE Full topic] [CrossRef] [Medline]

‎Goldberg L, Lide B, Lowry S, Massett HA, O’Connell T, Preece J, et al. Usability and accessibility in consumer hale condition informatics current trends and future challenges. Am J Prev Med 2011 May;40(5 Suppl 2):S187-S197. [CrossRef] [Medline]

‎fore-rank Gemert-Pijnen JE, Nijland N, covered wagon Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic skeleton to improve the uptake and contact of eHealth technologies. J Med Internet Res 2011;13(4):e111 [FREE Full topic] [CrossRef] [Medline]

‎Gustafson DH, Palesh TE, Picard RW, Plsek PE, Maher L, Capoccia VA. Automating devotedness treatment: enhancing the human experience and creating a set for the future. In: Bushko RG, conductor . Future of intelligent and extelligent soundness environment: volume 118. Studies in hale condition technology and informatics. Amsterdam: IOS Press; 2005:186-206.

‎Gustafson DH, Shaw BR, Isham A, Baker T, Boyle MG, Levy M. Explicating some evidence-based, theoretically informed, mobile technology-based scheme to improve outcomes for people in redemption for alcohol dependence. Subst Use Misuse 2011;46(1):96-111 [FREE Full paragraph] [CrossRef] [Medline]

‎Ouwens M, Wollersheim H, Hermens R, Hulscher M, Grol R. Integrated care programmes as being chronically ill patients: a review of regular reviews. Int J Qual Health Care 2005 Apr;17(2):141-146 [FREE Full clause] [CrossRef] [Medline]

‎Lilly CM, De Meo DL, Sonna LA, Haley KJ, Massaro AF, Wallace RF, et al. An emphatic communication intervention for the critically harm. Am J Med 2000 Oct 15;109(6):469-475. [Medline]

‎Coleman EA, Parry C, Chalmers S, Min SJ. The care transitions interference: results of a randomized controlled hardship. Arch Intern Med 2006 Sep 25;166(17):1822-1828. [CrossRef] [Medline]

‎Van Eaton E. Handoff amendment: we need to understand what we are afflictive to fix. Jt Comm J Qual Patient Saf 2010 Feb;36(2):51. [Medline]

‎The Methodology Center. Pennsylvania State University   URL: http://methodology.psu.edu/ [accessed 2013-12-10] [WebCite Cache]

‎Gustafson DH, Fryback DG, Rose JH, Yick V, Prokop CT, Detmer DE, et al. A judgment theoretic methodology for severity index unravelling. Med Decis Making 1986;6(1):27-35. [Medline]

‎Shavelson RJ, Webb NM. Generalizability abstract principles: a primer. Thousand Oaks, CA: Sage; 1991.

‎Bolt DM, Piper ME, Theobald WE, Baker TB. Why two smoking cessation agents work better than individual: role of craving suppression. J Consult Clin Psychol 2012 Feb;80(1):54-65 [FREE Full verse ] [CrossRef] [Medline]

‎Isaacson W. Steve Jobs. New York, NY: Simon & Schuster; 2011.

‎Walls TA, Schafer JL, editors. Models according to intensive longitudinal data. Oxford, UK: Oxford University Press; 2006.

‎Eysenbach G, Powell J, Kuss O, Sa ER. Empirical studies assessing the condition of health information for consumers forward the world wide web: a methodical review. JAMA 2002;287(20):2691-2700. [Medline]

‎O’Grady L, Witteman H, Bender JL, Urowitz S, Wiljer D, Jadad AR. Measuring the striking of a moving target: towards a dynamic framing for evaluating collaborative adaptive interactive technologies. J Med Internet Res 2009;11(2):e20 [FREE Full theme] [CrossRef] [Medline]

‎DISCERN Online. University of Oxford   URL: http://www.make distinction.org.uk/ [accessed 2013-12-10] [WebCite Cache]

‎Khazaal Y, Chatton A, Cochand S, Coquard O, Fernandez S, Khan R, et al. Brief DISCERN, six questions despite the evaluation of evidence-based satisfy of health-related websites. Patient Educ Couns 2009 Oct;77(1):33-37. [CrossRef] [Medline]

‎Donaldson LJ. From dusky bag to black box: will computers improve the NHS? BMJ 1996 Jun 1;312(7043):1371-1372 [FREE Full clause] [Medline]

‎Jimison H, Gorman P, Woods S, Nygren P, Walker M, Norris S, et al. Barriers and drivers of hale condition information technology use for the somewhat advanced in life, chronically ill, and underserved. In: Evidence Report/Technology Assessment No. 175, AHRQ (Agency towards Healthcare Research and Quality) Publication No. 09-E004. Rockville, MD: Oregon Evidence-based Practice Center; 2008.

‎Murray E, Burns J, See TS, Lai R, Nazareth I. Interactive soundness communication applications for people with chronic disease. Cochrane Database Syst Rev 2005(4):CD004274. [CrossRef] [Medline]

‎Lock C. What set a high ~ on do computers provide to NHS hospitals? BMJ 1996 Jun 1;312(7043):1407-1410 [FREE Full paragraph] [Medline]

‎van Limburg M, front Gemert-Pijnen JE, Nijland N, Ossebaard HC, Hendrix RM, Seydel ER. Why pursuit modeling is crucial in the increase of eHealth technologies. J Med Internet Res 2011;13(4):e124 [FREE Full true copy] [CrossRef] [Medline]

‎Rogers EM. Diffusion of innovations. 5th Edition. New York: The Free Press; 2003.

‎Whitehouse SR, Lam PY, Balka E, McLellan S, Deevska M, Penn D, et al. Co-universe with TickiT: designing and evaluating a clinical eHealth platform towards youth. J Med Internet Res 2013;2(2):e42 [FREE Full clause] [CrossRef] [Medline]


A-CHESS: Addiction—Comprehensive Health Enhancement Support System

CHESS: Comprehensive Health Enhancement Support System

eHealth: electronic hale condition

e-social: social networking conducted electronically

mHealth: expressive health

RCT: randomized controlled trial

SMART: sequential, multiple offer, randomized trials

Edited by G Eysenbach; submitted 29.08.13; pry-reviewed by R Glasgow, M Solomon; comments to author 30.10.13; revised version received 16.12.13; accepted 09.01.14; published 19.02.14

Please adduce as:
Baker TB, Gustafson DH, Shah D
How Can Research Keep Up With eHealth? Ten Strategies towards Increasing the Timeliness and Usefulness of eHealth Research
J Med Internet Res 2014;16(2):e36
URL: http://www.jmir.org/2014/2/e36/
doi: 10.2196/jmir.2925

Export Metadata:
END, compatible with Endnote
BibTeX, compatible with BibDesk, LaTeX
RIS, compatible by RefMan, Procite, Endnote, RefWorks

Add this paper to your Mendeley library
Add this thing to your CiteULike library


©Timothy B Baker, David H Gustafson, Dhavan Shah. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 19.02.2014.

This is ~y open-access article distributed under the conditions of the Creative Commons Attribution License (http://creativecommons.org/licenses/through /2.0/), which permits unrestricted practice, distribution, and reproduction in any instrumentality, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a constituent piece to the original publication on http://www.jmir.org/, for example well as this copyright and leave information must be included.

Of your 90 women who participated over black cohosh study, none reported adverse negative effects and no person dropped on the ~side.

Show more