INTRODUCTION
Environmental exposures, such as tobacco smoke, radiation, and endocrine disrupting chemicals (EDCs), are an area of concern because of their association with increased systemic effects and risk of disease. To highlight, researchers have found that high concentrations of EDCs in the body are linked to numerous health impacts including organ dysfunction, hormonal imbalances, and increased risk of developing cancer1-3. Furthermore, exposure early in an individual’s life has the potential for increasing the risk of non-communicable diseases, such as cancer, cardiovascular disease, and chronic lung illnesses4-6. The usage of personal care products (PCPs) is increasing, as highlighted by Swei et al.7 who concluded that men used an average of 11 PCPs daily, which has increased from six PCPs in 2004. This exposure amounts to an average of 105 different chemicals every day7. Women, on the other hand, used an average of 13 different PCPs daily: six products for body care, three for skin care, two for cosmetics, one for hair care, and one for baby care7. This exposure amounts to an average of 114 different chemicals everyday through PCPs7. These include heavy metals, carcinogens, synthetic preservatives, and fragrances. While the intent of these products is to enhance hygiene, beauty, and overall well-being, they may also present significant health concerns.
The increase in PCP use over the years presents a significant concern, as the use of parabens in PCPs also continues to increase8. According to the FDA’s Voluntary Cosmetic Registration Program database from 2019, methylparaben was reported to be used in 11739 formulations (9347 of which are leave-on products), while in 2006, a total of 8786 uses were reported8. This represents a 34% increase in 13 years. Propylparaben had the next highest number of reported uses at 9034 (7520 uses in leave-on products), which was an increase from 7118 uses in 2006. Despite important health concerns associated with parabens, there are no restrictions on the use of parabens or their concentrations in PCPs in Canada9. While the Government of Canada has acknowledged the potential health concerns due to parabens by completing a chemical risk assessment in 2020, no changes have been made to restrict or eliminate the use of parabens in PCPs in general, except in the case of Natural Health Products10,11. Due to the lack of Canadian regulations to limit individuals’ exposure to parabens in PCPs, consumers are required to take personal behavioral action.
While there are safer alternative PCPs available, most individuals do not have the knowledge or means to access them, as highlighted by several studies. For example, Buchmüller et al.12 investigated consumers’ risk perceptions of various chemical household products, including laundry detergent and essential oils, through an online survey. They found that consumers were only aware of the possible risks associated with these products after being prompted to consider them, rather than intuitively identifying them. Similarly, Llanos et al.13 investigated PCP purchasing and usage behaviors and perceptions among US adults and observed that participants shared moderate concern about being exposed to toxic chemicals. Furthermore, due to low-risk perceptions, research has underscored that both pregnant and non-pregnant women either did not change or intend to change their PCP use during pregnancy14. The low levels of knowledge surrounding PCPs are further exacerbated among those with lower levels of education and income15. Additionally, availability and accessibility of EDC-free products remain a concern, as lower income and marginalized communities face barriers to accessing PCPs16 A common misconception is that EDC-free products are more expensive, therefore, consumers may think that they are even more inaccessible16. These data suggest that consumers have low knowledge and risk perceptions, as well as a lack of access to information. However, while consumers have low knowledge and are unable to access information, there is minimal focus on increasing knowledge and bridging this gap.
An educational toolkit that provides individuals with information about the harmful effects of EDCs and where to seek safer alternatives may help to reduce exposure. The effectiveness of educational toolkits as a form of knowledge translation for behavior change was investigated in a systematic review17. Several of the included studies were either partially or mostly effective in changing behavior to inform, share knowledge, build awareness, change practice, or clinical outcomes in healthcare settings, inform policy, or to commercialize an innovation17. Additionally, a scoping review by Barac et al.18 explored the use of educational toolkits in health and healthcare. While information regarding toolkit evaluation was only available in 31 of the 38 included toolkits, they reported that 21 of these studies found that the toolkit was satisfactory, useful, or resulted in an intention to change practice18. For instance, the toolkit by Sawyer et al.19 consisted of an information letter and brochure, reference cards, fact sheets, posters and a video and the authors observed a satisfaction rate of nearly 100%. Additionally, following utilization of the toolkit developed by Kopp and Hornberger20, 92% of nurses planned to incorporate it into their practice.
While research suggests that characteristics of toolkits may be effective, few studies have developed a toolkit that specifically targets environmental health and EDC exposure reduction. Limited knowledge exists regarding the characteristics of existing environmental health literacy toolkits and the data collection methods to evaluate their effectiveness. Based on this information, the objectives of this systematic review are:
METHODS
Systematic review
A systematic review was conducted as it allowed for the identification, appraisal, and synthesis of the evidence regarding characteristics of environmental health toolkits and how to collect data for their evaluation. This report was structured according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines21 (Supplemental file Material 1). In this review, educational toolkits are defined as a package of resources for knowledge translation and facilitating behavior change17. Changes in behavior, or avoidance behavior, involves adopting new health-related behaviors because of heightened knowledge, risk perceptions, and health beliefs22. Knowledge translation activity is defined as the synthesis and dissemination of knowledge to improve health and provide more effective health services and products23.
Search strategy
A comprehensive systematic search was conducted for peer-reviewed published articles from three databases: MEDLINE via EBSCOhost, CINAHL, and Web of Science from inception of the databases until 12 May 2024, using the search terms listed in Table 1. In consultation with a health sciences librarian a search strategy was developed in MEDLINE via EBSCOhost and translated into other databases, including CINAHL (EBSCOhost), and Web of Science (Supplemental file Material 2). For our search, we did not have any language restrictions. Search terms incorporated subject headings and free-text terms for educational toolkits, behavior, and personal care products. Non-English articles were translated into English using DeepL Translate.
Table 1
Database search terms (MeSH terms were also used where applicable)
Grey literature (i.e. information produced or obtained outside usual publishing and distribution channels) was also explored through keyword searches on Google Scholar and Google Search from inception until 6 June 2024. The grey literature search was performed using the following predefined key terms: educational toolkits, behavior, and personal care products. Boolean operators were applied in the following manner: (educational toolkits) AND behavior AND (personal care products).
Inclusion and exclusion criteria
There were no restrictions on the population; studies involving participants of any age, gender, socioeconomic background, or geographical location were eligible. Interventions of interest were environmental health-related educational toolkits that aimed to promote behavior change and included a knowledge translation component. This review included studies that: 1) were randomized controlled trials to evaluate environmental health toolkits; 2) featured toolkits that had knowledge translation activity for behavior change; 3) included outcome measures for behavior change; and 4) were in the English language or translatable (Table 2). Records were excluded based on the following criteria: 1) did not include a randomized controlled trial for an environmental health-related educational toolkit; 2) did not include knowledge translation activity for behavior change; and 3) did not report outcome measures for behavior change (Table 2).
Table 2
Eligibility criteria for selecting studies
Screening
Screening was conducted by two independent reviewers (GD and RD), both of whom underwent training for screening and risk of bias assessment, in two stages: 1) title and abstract, and 2) full text using EndNote 21.4 for reference management and tracking of the screening process. EndNote 21.4 was also used to automatically remove duplicate references before the independent reviewers started screening. In the first stage of screening, titles and abstracts were reviewed and classified as relevant or irrelevant according to the inclusion criteria. To establish inter-rater reliability, each reviewer pilot tested 10 studies using the inclusion criteria. There was 90% agreement on the selected articles. During the second phase of screening, the full texts of the relevant studies were reviewed for eligibility. Any disagreements during the screening processes were discussed by the two reviewers to achieve consensus.
Risk of bias assessment
Included articles were critically appraised by the two independent reviewers (GD and RD) for overall quality using the Risk of Bias tool for randomized trials (RoB 2)24. For cluster-randomized trials and random crossover trials, the appropriate RoB 2 checklists were used24. The RoB 2 assessed for appropriate methodology and study design, reporting of results, and risk of bias by assigning quality scores to studies. This systematic review categorized the validity of each study as either low, moderate or high risk of bias. Following the RoB 2 tool’s methodology for determining the level of bias, studies were considered to have a high risk of bias if reviewers concluded that the methodological quality was affected by biases and flaws in the methodology. Any disagreements during the appraisal process were discussed by the two reviewers to achieve consensus.
Data extraction, management, and synthesis
During the data extraction stage, two researchers (GD and RD) worked independently to extract the intervention elements of the included studies using a modified version of the template for intervention description and replication (TIDieR) checklist25. The TIDieR checklist has twelve items, including the name of the intervention, the rationale, the materials and procedures, who provided the intervention, the mode of delivery, the location where the intervention occurred, the intervention’s frequency, intensity, and duration, if it was adapted or tailored, any modifications, and fidelity. Extracted data included the aims of the study, design, geographical location, frequency, intensity, and duration of intervention, data collection methods, as well as outcomes associated with the use of the intervention. All available data for each outcome were recorded for each study, as well as all methods of measurement and analysis. The data extracted were synthesized using qualitative content analysis to identify themes associated with environmental health educational toolkits, including the characteristics of these toolkits and their data collection methods. Study characteristics and intervention details were tabulated using summary tables, which were developed to organize common themes related to environmental health educational toolkits, their characteristics, and data collection approaches. Studies with high risk of bias were excluded from the qualitative content analysis.
RESULTS
PRISMA diagram
Our search strategy retrieved 3619 articles from CINAHL, 600 from MEDLINE via EBSCOhost, and 129 from Web of Science (Figure 1). In all, 220 duplicate articles were removed, and we screened 4128 based on their title and abstract. In terms of grey literature, 15 articles were found from Google Scholar and 6 articles from Google Search. Of the 67 records that were to undergo full-text screening, 3 could not be retrieved, 30 were excluded because they were not a randomized controlled trial that evaluated the effectiveness of an environmental health-related educational toolkit, and 7 were excluded because they did not include knowledge translation activity for behavior change (Supplemental file Material 3). In total, 27 studies were included in the systematic review following full-text screening. Of the 27 included studies, 1 was rated as having low risk of bias, 16 had moderate risk of bias, and 10 were deemed inadmissible because of high risk of bias (Supplemental file Material 4). Differences between studies rated as admissible and inadmissible were largely due to biases related to deviations from intended intervention, biases in measurement of the outcome, and bias in selection of the reported result.
Characteristics of included studies
The included studies were published from 1994 to 2020 (Table 3). In terms of study location, 13 of the included studies were conducted in the USA and Canada, one was from Vietnam, one from Laos, one from Iran, and one from Turkey. The 17 included studies featured a variety of environmental health topics, including EDC exposures from PCP, sun protection and cancer risk reduction, handwashing behavior and hygiene, and smoking (Table 3). Fourteen studies focused on behavior change related to sun safety, protection, and cancer risk reduction26-39. One study evaluated the effect of educational toolkits related to behavior change and hygiene40. Finally, one study focused specifically on behavior change in relation to EDC exposures from PCPs and household products41.
Table 3
Characteristics of included studies, including study design, year of publication, geographical setting, sample size, population, methods, and findings (N=17)
| Program name Authors Year | Aims Rationale | Country/Region Population | Location Intervention | Data collection methods and analysis | Evaluation of findings |
|---|---|---|---|---|---|
| Not reported Armstrong et al.26 2011 | To develop an educational vehicle that not only improves sunscreen behavioral outcome and sunscreen application knowledge but also has the potential to be convenient, inexpensive, and easily distributed among the general population. Interventions including community programs, school programs, and physician recommendation have been successful at improving sunscreen use, particularly among parents and their children. However, these interventions require more resources to implement than the traditional pamphlet, limiting their widespread use. | USA General population English-speaking, at least 18 years old, have Internet access, and be able to view videos online. n=94 | Not reported After pre-test 1 session For 2 hours Addressed how sunscreens work to protect skin, different types of sunscreens, importance of sunscreen use, and proper application. Participants were instructed to view the material at least once during the study, and were allowed to view the material as often as desired. | Sunscreen application behavior was assessed before and after the study intervention using standard questions from the NHANES Baseline knowledge regarding proper sunscreen use and the effects of ultraviolet light was assessed with a pre- and post-study questionnaire; satisfaction survey; unpaired t-tests for between-group comparisons; paired t-tests for within group comparisons; chisquared tests for categorical variables | Inter-group analysis showed a significant difference in the improvement of post-study scores of the online-video group (2.0±1.5) compared to the pamphlet group (1.2±1.0), p=0.003; Inter-group analysis revealed that the improvement in sunscreen application behavior was greater in the video group (1.9±2.3 increase in days per week) than the pamphlet group (0.2±0.5) with p<0.001. |
| Not reported Aygun and Karayağız Muslu et al.27 2021 | To discover whether a schoolbased sun protection program (SPP) developed in line with the transtheoretical model (TTM) had an impact on students in the intervention group in the stages of change in terms of developing sun protection behavior, their perceptions of pros and cons and their self-efficacy, as compared to the controls. Previous studies emphasize that sun protection programs should be implemented in the schools and be made more widespread in an effort to protect and improve school health | Turkey Children Students from two schools n=452 | School After pre-test Weeks 1–8: weekly sessions; weeks 9–24: biweekly reminders For 6 months Researchers created a 12-page student guidebook, 72 posters, 4-word puzzles, a UV-sensitive Frisbee game, and educational videos; Reminders about sun protection were prepared according to the TTM stages of change using written or multimedia content and achievement certificates | Prior to the study, students were administered the descriptive information form, SPBS, SPSOC, DBS and SES measurement tools; repeated forms 6 months & 1 year later Two-way repeated measures analysis of variance was used to assess the effect of the sun protection program on the students in the intervention and control groups; the Bonferroni Test was employed in the comparison of the two groups and in the intragroup comparison | The mean scores of the intervention group on the sun protection behavior scale and its subscales of sun avoidance, sunscreen use and hat use were significantly higher than those of the control group on the post-test (p<0.001) and follow-up test (p <0.001); the mean scores of the intervention group on the sun protection self-efficacy scale and its subscales of sun avoidance, sunscreen use and hat use were significantly higher than those of the control group on the posttest (p<0.001) and follow-up test (p<0.001) |
| Sunny Days, Healthy Way Buller et al.28 2006 | To evaluate a version of SDHW for middle schools – grades 6, 7 and 8 – that contained content and instructional strategies based on social cognitive theory. The school environment is a potentially effective venue for delivering sun safety interventions, but it is uncommon in the United States. | USA Children in grades 6–8 n=2038 | School After pre-test 6 sessions; 50 minutes each For 6 weeks Lessons intended to increase perceived personal risk for skin damage and skin cancer, positive outcome expectations about sun protection to reduce personal risk, and self-efficacy expectations for performing sun protection in a variety of situations; key prevention skills | Self-reported sun-protection behavior using frequency ratings and diary. Linear mixed-models were used to test the main hypothesis on the effect of SDHW taking into account the clustering within school.; Associations were tested between each outcome measure, separately, and potential covariates in bivariate mixed models; Multivariate analyses were then performed including significant covariates; general linear mixed model for continuous and Likert scale outcomes; generalized mixed-model for binary outcomes | Compared to control schools, children receiving the curriculum reported more frequent sun protection (p=0.0035), and a greater proportion wore long-sleeved shirts during recess (p<0.0001) and applied sunscreen (p<0.0001). Exposure to the curriculum improved knowledge (p<0.0001), decreased perceived barriers to using sunscreen (p=0.0046), enhanced self-efficacy expectations (p=0.0577) about sun safety, and reduced favorable attitudes toward sun tanning (p=0.0026 to <0.0001). |
| Sunshine and Skin Health Buller et al.29 1994 | To determine the feasibility of administering Sunshine and Skin Health. The earlier the age at which children are educated and encouraged to practice sun protection, the greater is their opportunity to avoid skin cancer. | USA 2 elementary schools; children in grades 4–6 n=139 | School After pre-test 1 session; 50 minutes long For 5 weeks Intervention included five multi-disciplinary units that synthesize material from science, history, social studies, health, and geography into a comprehensive cause-and-consequence presentation about man’s relationship with the sun; each unit contains lesson material, in-class activities, take-home activities, a glossary of key terms, a quick review, and a student/parent newsletter | Data were collected using the Sunshine and Your Skin questionnaire one week prior to the curriculum (pre-test), at the completion of the curriculum (post-test 1), and eight weeks after completion of the curriculum (post-test 2) Comparisons were made using analysis of covariance and correlational techniques; Additional information about the impact of the curriculum was obtained through open-ended interviews with teachers | The curriculum substantially increased students’ knowledge of the relationship between exposure to sunlight, skin cancer, and sun-protection behavior immediately following completion of the curriculum compared with the students in the control group [F (1, 126)=132.95, p<0.05]; At the end of the intervention, the students receiving the curriculum were less likely to agree that a tan made them feel and look better [F (1, 117)=4.69, p<0.05] and that having a tan is in style today [F(1, 111)=4.89, p<0.05) than students in the control group |
| Not reported Campbell et al.41 2011 | To test the efficacy of educational and environmental interventions in a cohort of urban newborns and their families during the first year of life Few studies have looked at intensive interventions during the newborn age, when, presumably, a child’s BLLs have not yet been elevated from postnatal lead exposure | USA Children (newborns) From urban outpatient practices located in low-income neighborhoods of Philadelphia n=314 | Home After pre-test Frequency/intensity not reported For 12 months All families were administered pre- and post-tests with both standard lead and maintenance questions on the first home visit (before and after the education was given), and the standard lead test was repeated at six and 12 months; intervention included a combination of education, home evaluation, and lead remediation | Shortened version of the Chicago Lead Knowledge Test to evaluate parental knowledge regarding lead exposure prevention; blood lead levels in newborns Compared geometric mean BLLs (due to non-normal distribution) of the study children and a comparison group using a two-tailed t-test; chi-squared tests to compare sociodemographics of homes meeting criteria for remediation; predictors of BLL by using bivariate tests such as Mann-Whitney U tests and t-tests analysis of variance, and correlation coefficients; two-tailed Wilcoxon tests to determine changes from baseline within groups and Mann-Whitney U tests to compare groups at each visit; a Spearman correlation was used to test for association between parental knowledge at 12 months and 12-month BLLs | Geometric mean BLLs were 2.6 and 2.7 for the LSH cohort and comparison group and not significantly different (p=0.477); the combined groups’ standard education test scores rose during the baseline visit from a pretest mean of 6 to a post-test mean of 12 (p<0.001) |
| Project SCAPE Glanz et al.30 2010 | To evaluate the impact of tailored interventions on skin cancer prevention and skin self-examination among adults at high and moderate risk for skin cancer in a randomized trial. Tailored, mailed communications hold great promise for skin cancer prevention and detection among individuals at increased risk, but these interventions have not been widely tested | USA General population Residents of Honolulu or Long Island aged 20–65 years, and moderate or high risk for skin cancer n=1371 | Home After pre-test 3 packages every 2 weeks For 6 weeks Participants completed a baseline survey and sun exposure and protection diary, a telephone interview at midsummer, and a second survey and diary in late summer; Along with personalized risk feedback and recommendations, the experimental group received UV self-monitoring aids, skin self-examination instructions and practice tools, and skin cancer prevention and detection information | Baseline Sun Habits Survey asked participants about demographic characteristics, knowledge about skin cancer, habitual sun exposure and sun protection, and whether they had ever and recently (in the past 3 months) conducted a thorough skin self-examination; the sun exposure diary is a 4-day record of sun exposure and protective behaviors; the interviews asked about sun exposure and sun protection habits for the preceding weekend, and receipt of and reactions to the intervention materials Chi-squared test and the t-test on survey baseline responses to check that randomization resulted in comparable groups, to determine whether any characteristics were associated with attrition or missing diary data, and to document differences between the 2 study locations; multivariate analyses | The sun protection habits index showed a greater increase over time for participants in the intervention arm (effect size=0.13); The intervention arm in both locations showed significant improvement, but the treatment effect was attenuated for Honolulu (effect size=0.04), and Long Island (effect size=0.23); compared with the control group, the treatment group showed significant improvement on the sun protection habits composite (effect size=0.39), sunscreen use, and hat wearing |
| Sun Protection is Fun! (SPF) Gritz et al.31 2007 | To increase the sunscreen and sun-avoidance practices of preschool staff to protect children from sun exposure Interventions to increase the sun protection of young children must include educational components for child care providers, as young children are dependent on these caregivers for sun protection | USA General population Preschool staff 22 preschools n=245 | Preschool After pre-test 3 sessions; 1–2.5 hours For 2 years Intervention was designed to promote sun-screen and sunavoidance behaviors among preschool staff; after baseline, staff participated in a 2-hour training on how to protect children from sun exposure and teach the SPF curriculum; in the spring of 1997, staff participated in a 2.5-hour training that covered sun protection for children, the SPF curriculum, and suggested strategies for supporting sun protection at preschools such as policy development and adding shade structures; sessions included guided practice activities that provided opportunities to observe the actions and reinforcement of fellow staff members to develop skills and build self-efficacy, norms, and expectancies for teaching the curriculum and engaging in sun protection; In the spring of 1998, research staff conducted training sessions using a 1-hour training module consisting of a video, worksheets, and answer pages; four issues of newsletter contained sun safe facts, a physician’s column, and role-modeling stories developed from interviews with preschool staff using the intervention; curriculum had seven units with five activities per unit | Five items assessed staff sunscreen use on their students, four items assessed students’ use of protective clothing, and one item assessed staff’s use of canopies and umbrellas; Sun-protection knowledge items measured knowledge about sunscreen use, limiting sun exposure during midday, and sun-reflective surfaces; Higher scores on behavioral items represented more frequent sun-protection practices, Higher scores on psychosocial survey items represented higher levels of self-efficacy, expectancies, norms, and ACI and lower levels of impediments Bivariate analyses; multilevel analysis; the group estimate indicates the magnitude of the intervention effect with positive group estimates indicating effects in the desired direction; | At the 12-month and 24-month assessments, the intervention group means were higher for both sunscreen use and sun avoidance; All intervention effects on staff behavior were in the desired direction; At the 12-month and 24-month assessments, the intervention group means were higher for most variables |
| Capacity building for community leaders in a healthy living environment Hien et al.40 2008 | To promote the healthy living environment and to mobilize human resources in rural communities. Community-based health educational programs are effective for specific target groups, including health professionals such as physicians | Vietnam General population Individuals living in rural communities 14 communes n=160 | Meetings After pre-test Frequency/intensity not reported From 5 days Each intervention commune hosted a 5-day CCH educational program to enable community leaders to improve their knowledge and skills regarding a healthy living environment, to understand potential health risks of unsanitary conditions, to provide counseling and promote a healthy living environment and to apply effective health communication skill; materials were designed to provide simple but useful messages for the community leaders; participatory style was employed in the educational program with a participant-centered solution-based approach with activities such as small group discussions, brainstorming, role-playing, demonstrations, games, case studies and field visits; participants in the educational program were community leaders who were representatives of various sectors and mass organizations in the village | The pre-assessment questionnaire was used to assess the baseline knowledge of community leaders in both the intervention and control groups prior to the beginning of the educational program, 25 questions originally developed to assess four required competency areas for community leaders The total and HPC scores in the pre- and post-assessment tests were used to assess the community leaders’ improvement in knowledge regarding a healthy living environment; pre- and post-test totals and HPC scores in the intervention and control groups were compared by the paired t-test; analysis of variance was performed to compare score means between different categories of demographic factors in each group; the chi-squared test was applied to examine the differences between demographic factors of the two groups and the one-sample t-test was used to evaluate the educational program; linear regression analysis was used to obtain the regression coefficients and to determine the strengths of associations between the effects of demographic factors and study outcomes | The intervention group showed a significant improvement from the pre-test score of 32.0±11.9 to the post-test score of 75.8±14.4 (p<0.001), whereas no statistical change was observed in the control group; The participants rated the educational program highly with regard to educational objectives (4.8±0.3, p<0.001), methods (4.80±0.3, p<0.001), learning atmosphere (4.9±0.2, p<0.001), the facilitators (5.0±0.2, p<0.001), time management (4.7±0.4, p<0.001) and training materials (4.8±0.3, p<0.001) |
| Not reported Hoffman et al.32 1999 | To examine the effects of a multicomponent intervention on children’s sun protection knowledge, beliefs, attitudes, intentions, and behaviors Few studies have described the implementation and evaluation of sun protection interventions for children. | USA Students in 5th grade 2 schools n=181 | School After pre-test 50-minute class daily For 3 days Intervention provided information directed at changing children’s attitudes toward sunscreen usage and providing activities aimed at influencing the subjective norms associated with practicing sun protective behaviors; follow-up | A 12-item knowledge scale to assess participants’ knowledge of skin cancer and sun protection; the 13-item attitudes and behavior scale assessed participants’ perceptions of sun exposure risks, ability to practice sunsafe behaviors, and perceived costs and benefits of engaging in sun protective behaviors; the 6-item intentions scale measured participants’ reported plans to use sunscreen in several situations; the self-reported behavior scale quantified sunscreen use during outdoor activities over 2 weeks Group × Time multivariate analysis of variance; univariate analysis; multiple regression analysis | Significant Group × Time of assessment interaction effect [F(4, 151)=11.58, p<0.001; intervention group scored higher on the knowledge scale [F(1, 171)=34.47, p<0.0001], attitudes and beliefs scale [F(1, 160)= 3.46, p<0.06], intentions scale [F(1, 171) =13.99, p<0.001], and self-reported behavior [F(1, 169)=13.59, p<0.001] |
| Not reported Jeihooni and Rakhshani et al.43 2019 | To determine the effect of educational intervention based on the health belief model and social support on the promotion of skin cancer preventive behaviors among the farmers Skin cancer is one of the most common cancers in Iran. Farmers are exposed to the sun’s ultraviolet radiation due to their job and are susceptible to skin cancer | Iran General population Farmers n=200 | Health clinic After pre-test 8 sessions; 55–60 minutes each For 8 weeks baseline; lectures, group discussions, questions, and answers, as well as the use of educational posters and pamphlets, showing films, and PowerPoint presentations, to maintain and enhance the subjects in the experimental group, they were sent a weekly text message on skin cancer, and a telegram group was set up to exchange information, an educational session was also held once a month to follow up their activities; To determine the effectiveness of the intervention, the questionnaire was completed by both experimental and control groups in before the intervention, 3 months later, and 6 months after the educational intervention. | A questionnaire consisting of demographic information, knowledge, HBM constructs (perceived susceptibility, severity, benefits, barriers, self-efficacy and cues to action), and social support was used to measure skin cancer preventive behaviors before, 3 months after the intervention, and 6 months later Chi-squared test, Mann-Whitney U test, and repeated measures ANOVA test | 3 months after the intervention and 6 months later, the experimental group showed a significant increase in each construct compared to the control group, except for the perceived barriers, and had a significant decrease in the perceived barriers compared to the control group. |
| Project SUNWISE Mayer et al.34 2007 | To develop an intervention for increasing specific sun safety behaviors among this population In a descriptive study, US Postal Service (USPS) letter carriers reported working out- doors an average of 5 hours per day, but only 14% reported that they always wore sunscreen, and only 33% reported that they always wore a hat. | USA General population US Postal Service letter carriers 70 postal stations n=2662 | Workplace Within 2 weeks of pre-test 6 sessions; 5–10 minutes each For 2 years Baseline measurement; the intervention package included 1) increased access to protective hats, 2) increased access to sunscreen, 3) visual cues that prompted use of solar protective strategies, and 4) delivery of brief educational sun safety messages; large flip charts with colorful graphics that were tailored to letter carriers; primary outcomes were self-reported occupational use of sunscreen with an SPF of 15 or higher and wide-brim hats (≥2.5 inches wide) | The primary outcomes were self-reported occupational use of sunscreen with an SPF of 15 or higher and wide-brim hats (≥2.5 inches wide). These behaviors were measured onsite with a paper-and-pencil questionnaire, which asked how often respondents had used each item while carrying mail during their past 5 workdays Intraclass correlations were estimated for all outcomes to determine the extent of postal station clustering, the correlations suggested that clustering must be taken into account in the analyses; For the primary outcome measures, we analyzed trends over 2 years with generalized linear mixed models that treated 3 months, 1 year, and 2 years as a set of repeated measures on each participant; multilevel model; All analyses were based on intent to treat; For each analysis, we tested the time-by-group interaction to determine if the intervention effect remained constant over time and the group main effect; The secondary outcomes (colorimeter data) were analyzed with mixed-effects regression models that treated 3 months, 1 year, and 2 years as a set of repeated measures on each participant and that adjusted for participants nested in postal stations within a multilevel model | Comparisons at baseline were made using a mixed-effects model that compared the intervention group with the control group after adjusting for the clustering effects of postal stations; the intraclass correlation was 0.014 for sunscreen and 0.091 for wide-brim hat; There was a significant interaction for sunscreen use, which was a result of the difference between intervention and control groups diminishing in size over time, At 3 months, the estimated odds ratio (OR) for sunscreen use was 2.8 times higher among the intervention group compared with the control group; at 2 years, the OR was 2.0 times higher among the intervention group; the group main effect was significant, with the intervention group that used wide-brim hats having a higher rate (OR=2.88; 95% CI: 2.31–3.61) |
| Survivor Health and Resilience Education (SHARE) Program Mays et al.33 2011 | To examine the efficacy of the SHARE Program intervention for improving short-term sun safety behaviors among adolescent survivors of pediatric cancer Evidence for effective interventions promoting sun safety behaviors within this population remains scarce | USA Adolescent survivors of childhood cancer Survivors of childhood cancer, aged 11–21 years n=75 | Health clinic After baseline 1 half-day workshop Duration not reported Consented participants completed a detailed baseline assessment via two telephone calls lasting approximately 30–40 minutes each. Participants were also asked to maintain a behavioral record for several days as part of the baseline assessment and were provided with instructions to do so; half-day, group-based interactive workshop that included health promotion content addressing sun safety and other relevant health behaviors for adolescent survivors of childhood cancer, Aspects of the intervention that focused on promoting sun safety practices were included with content that targeted other health behaviors relevant to childhood cancer survivors, such as diet and physical activity, didactic presentations of sun exposure and sun protection, demonstrations of sun safety practices, and reviewing action plans regarding sun safety and other health-promoting behaviors, administered by a masters-level facilitator who was trained by a multi-disciplinary research team, including experts in pediatric oncology, nutrition, and behavioral sciences; Participants completed an outcome assessment via telephone approximately 1-month post-intervention | Sun safety behaviors were assessed using a scale adapted from previous research; responses were summed to reflect an overall sun safety score, with higher values indicating more frequent sun safety behaviors Differences between intervention and control groups based on demographic characteristics, medical information, and baseline sun safety behaviors were assessed with bivariate statistical tests (i.e. χ2 tests, t-tests); linear regression model was then used to examine whether there was a difference between study groups in sun safety behaviors at 1-month post-intervention, after controlling for baseline sun safety practices, participants’ gender, and season of intervention (fall/winter vs spring/summer) | This model suggests that, after taking into account baseline sun safety, gender, and seasonal influences, participants in the intervention group reported significantly greater sun safety behaviors at 1- month post-intervention (mean= 26.8, SD=5.7) compared with participants in the control group (mean=23.8, SD=4.4) (B=2.64; 95% CI: 1.02–4.27, p= 0.002). |
| Not reported Norman et al.35 2007 | To evaluate a multicomponent primary care-based intervention to increase sun protection behaviors among adolescents. Insufficient evidence that clinician counseling was effective at changing patient behaviors related to skin cancer prevention, as well as insufficient evidence for the effectiveness of interventions in secondary schools targeting adolescents. | USA Adolescents aged 11–15 years n=800 | Primary care physician offices and participant homes After pre-test 6 sessions For 2 years At the study onset & 12 month follow-up, the adolescents engaged in an office-based expert system assessment of sun protection behaviors followed by brief stage-based counseling from the primary care provider; intervention incorporated the Sun Smart sun protection expert system computer program; The expert system was implemented as 1) an interactive tailored computer session in the primary care office at the start of the intervention and at 12 months and 2) a means of generating printed tailored feedback reports between office visits; The expert system included an audio component that mirrored all text appearing on the computer screen along with color graphics and music; brief 2- to 3-minute counseling sessions with the adolescent to further prompt for questions and reinforce the messages presented in the expert system session; At the 3-, 6-, 15-, and 18-month interventions, adolescents were contacted by a health counselor by telephone and completed the expert system assessments during the telephone interview | Adolescents self-reported their sun protection behaviors with a 7-item scale; Vulnerability to sun exposure was determined from a previously validated instrument; primary outcome was the sun protection score measured at the baseline and 6-, 12-, and 24-month assessments Mixed-model repeated-measures analysis was conducted to compare the effect in the Sun Smart intervention group with that in the control group during the 2-year intervention period; Logistic regression models were specified to test the effect of the intervention on the movement from precaution stages of change at the start of the study to the action or the maintenance stage of change at the 6-, 12-, and 24-month assessments; A multiple linear regression model was specified with the sun protection score as the de- pendent variable and the baseline sun protection score, sex, sun sensitivity, and the number of intervention sessions completed (range: 2–6) as the independent variables | The Sun Smart intervention group had a greater increase in sun protection scores compared with the control group, with the trajectory of scores flattening between 12 and 24 months; Five of the 7 items were statistically significant, with higher sun protection scores for the adolescents in the Sun Smart intervention group compared with those in the control group; A multiple linear regression model for the subsample of adolescents in the Sun Smart intervention group who completed assessments at 24 months (n=315) indicated that sun protection behavior was positively associated with completing more intervention sessions (standardized B=0.16; p=0.002), after controlling for the baseline sun protection score, sex, and sun sensitivity; Overall satisfaction with the Sun Smart program was high, with 79.4% of participants indicating they liked the program ‘some’ or ‘a lot’. |
| Not reported Reynolds et al.36 2008 | To test an intervention that included tailored and nontailored print communications delivered by mail to adolescents (age 11 to 15 years) and their parents who were also participating in an evaluation of an in-school intervention Few family-based interventions to increase sun safe behavior among adolescents have been evaluated. | USA Families Adolescents and parents n=599 | Home After pre-test 4 newsletters sent out at equal interviews throughout the summer May to August 2022 Pretest assessment for evaluation of the summer intervention was completed in the schools for students and by telephone survey for parents; summer intervention program was delivered to adolescents and parents using cover letters, Sun Scoop newsletters for parents, Summer Raze newsletters for adolescents, and small gifts to enable adolescents to practice the recommended sun protection, parent = three main components: an introductory cover letter, a series of newsletters, and a booklet, student = introductory cover letter and tailored newsletters; post-test assessment | Surveys contained self-report measures on sun protection; sun protection diary items described above also were completed by the parents, yielding estimates of time outside, mostly in sun/shade, wearing a head covering, wearing clothing that covered shoulders/arms, wearing clothing that covered legs, and wearing sunscreen In order to test the effects of the summer intervention on the adolescents, analyses were performed on each of the behavioral and psychosocial outcomes at the end of the summer, using the pre-summer value of each outcome as a covariate along with gender, age, and skin sensitivity to burning as additional covariates; linear fixed-effects model; logistic model | For students, treatment group (i.e. summer program vs no summer program) had no effect on any of the sun protection outcomes tested; By contrast, the summer program did have a positive effect on parents’ knowledge (F=5.52, p<0.05) and the propensity to have their child wear sunglasses (F=4.07, p<0.05); Greater program exposure/engagement led to enhanced sun protection behavior (e.g. fewer sunburns) and psychosocial factors among adolescents and parents; Greater exposure /engagement led to improvements in family interaction and home environment (e.g. shade audit completed) |
| Girls’ Integrated Health Care Program Shin et al.37 2020 | To evaluate the effect of an integrated health care educational program on several behavioral changes related to environmental health among Laotian students. Investment in Laotian child and adolescent capital development, particularly in the areas of education, health, and participation in decision making, will ensure that every young person’s potential is fulfilled. | Laos Girl students in grades 5 and 6 4 schools n=1275 | School After pre-test 24 sessions For 8 weeks Baseline questionnaire; The Girls’ Integrated Health Care Education consisted of seven health-related subjects, namely, health behavior, social relationships, environmental health, nutrition, physical activity, gender and sexuality, and sexual and reproductive health; 24 sessions held over eight weeks; three of the 24 sessions (over two weeks) focused on environmental health; environmental health education textbook regarding air pollution and chemical use; activities such as games and quizzes about air pollution and chemical use to make the students understand hazardous air pollution and chemical exposure easily and teach them how to keep their homes and classrooms clean from these exposures; 100 girls from the experimental group participated in the Peer Girls’ Special Group Activity in the same period, This activity, which was based on jumping rope, also contained seven health-related practices and an in-depth discussion, Two sessions of this activity were conducted per week, for a total of 16 sessions over eight weeks, One session’s discussion topic during one week was environmental pollution; followup survey | Four items were designed to assess risk perception, three to assess self-efficacy, and three to assess informationseeking behavior Analyses of covariance with each group as the between-subjects factor to examine immediate pre-to-post-intervention effects; Changes in the scores were analyzed within groups by paired t-tests stratified by the students’ gender and drinking and smoking habits, given that gender and juvenile delinquency have been found to modify the effect of education in previous studies; To assess the effect of Peer Girls’ Special Group Activity on girls, we compared the post-test scores between them after different educational activities | ANCOVA results showed a significant difference between the risk perception scores of the two groups after the intervention (control group mean=22.57, SD=3.25) (experimental group mean=23.38, SD=3.68) after adjusting for the baseline pre-test scores, gender, school grade, drinking habits, and smoking habits; comparison of the post-test scores between the groups revealed that the score of information-seeking behavior significantly increased in the experimental group after the program (control group mean=12.74, SD=3.21) (experimental group mean=13.83, SD=3.42). |
| Not reported Tuong and Armstrong38 2014 | To compare the effectiveness of appearance-based video education with that of health-based video education on improving sunscreen use and knowledge. Appearance-based education shows promise in promoting sunscreen use although resource-intensive methods used in prior studies preclude wide dissemination. Appearance-based video education can be made easily accessible. | USA Students in grade 11 n=50 | School After pre-test 1 video; 5 minutes long Duration not reported Baseline questionnaire; videos were approximately 5 minutes, health-based video discussed the growing incidence of melanoma among young people and the link between skin cancer and UV radiation, appearance-based video discussed the contribution of UV light to premature cutaneous aging and how sunscreen use could help delay signs of skin aging; follow-up 6 weeks post-intervention | Sunscreen application behavior was assessed at baseline and at 6 weeks postintervention using standard questions from the National Health and Nutrition Examination Survey on sun-protective behaviors Unpaired Student’s t-test; paired Student’s t- test was used for within-group comparisons; chi square tests for categorical variables | The health-based group had a non-statistically significant increase in sunscreen use (0.9±1.9 d/wk, p=0.096), whereas the appearance-based group demonstrated a statistically significant increase in sunscreen use (2.8±2.2, p<0.001); Between-group comparisons revealed that the appearance-based group applied sunscreen at significantly greater frequencies compared with the health-based group (2.2±1.4 vs 0.2±0.6, p<0.001); Knowledge scores significantly improved in both study groups; The difference in knowledge scores between the study groups was not significant. |
| Go Sun Smart Walkosz et al.39 2007 | To evaluate and test the hypothesis that implementation of the program would increase sun protection of children as reported by their parents. Unprotected exposure to UVR during the wintertime can result in both sunburns and cumulative exposure to UVR | USA and Canada General population Parents of children enrolled in ski and snowboard schools at high altitude resorts in western North America n=357 | Ski resorts After pre-test 6 units For 4 months Baseline interviews; sun safety program that utilized written, electronic, visual, and interpersonal channels of communication to promote sun safe practices to employees and guests at the ski resort; posters; brochures; puzzles; follow-up interviews | A series of questions about sun protection practices for the child were then posed, including whether the child was wearing sunscreen on her or his skin that day, a sunscreen lip balm, sunglasses or goggles, and a hat or a helmet; Respondents were also asked if their child’s skin had ever been sunburned while skiing, snowboarding, or playing outside at a ski resort; The final set of questions related to the dissemination of sun safety information by the ski and snowboard school staff and exposure to sun safety messages. Tests of association were calculated between potential predictors and our outcome variables; As the study was a pair-matched group randomized design, generalized linear models were used to account for the clustering of responses within resort. | Using generalized linear models with resort as a random variable to account for any intra-class correlation between respondents within a resort, this difference was not statistically significant (OR= 2.37; 95% CI: 0.93–5.99); In the Northwest region, significantly more parents at the intervention resorts reported that their child was wearing sunscreen than at the control resorts (OR=2.72; 95% CI: 1.24–5.95). By contrast, there was no effect on the intervention group in the Central and Southwest region (OR=1.48; 95% CI: 0.69–3.18) |
Characteristics of environmental health toolkits
Through the qualitative content analysis, 7 characteristics were identified from the 17 included studies, which include 2 elements and 5 features. The elements were identified as core, structural aspects that form the backbone of the toolkit and they include: 1) having simple, consistent, and frequent messaging; and 2) both health-based and appearance-based motivators. The features were specific attributes that make the educational toolkit more engaging and appealing, such: as 1) being interactive, 2) multimedia, 3) personalized, 4) incentive-based, and 5) accessible.
Including simple, consistent, and frequent messaging was an element of the environmental health-related educational toolkits (Table 4). In this systematic review, this theme was found in 3 of the 17 included studies27,37,40. Many of the studies shared frequent reminders with participants, either written or through text messages. Furthermore, the educational material was written to be age-appropriate and easy to understand no matter their education level.
Table 4
Extracted characteristics of educational toolkits for behavior change in environmental health (N=17)
| Authors Year | Elements | Features | |||||
|---|---|---|---|---|---|---|---|
| Simple, consistent and frequent messaging | Motivators based on health and appearance | Interactive | Multimedia | Personalized | Incentive-based | Accessible | |
| Armstrong et al.26 2011 | X | ||||||
| Aygun and Karayağız Muslu et al.27 2021 | X | X | X | X | |||
| Buller et al.28 2006 | X | X | |||||
| Buller et al.29 1994 | X | X | |||||
| Campbell et al.41 2011 | X | ||||||
| Glanz et al.30 2010 | X | ||||||
| Gritz et al.31 2007 | X | X | X | ||||
| Hien et al.40 2008 | X | X | |||||
| Hoffman et al.32 1999 | X | X | |||||
| Jeihooni and Rakhshani43 2019 | X | X | |||||
| Mayer et al.34 2007 | X | X | |||||
| Mays et al.33 2011 | X | X | |||||
| Norman et al.35 2007 | X | X | |||||
| Reynolds et al.36 2008 | X | ||||||
| Shin et al.37 2020 | X | X | X | ||||
| Tuong and Armstrong38 2014 | X | ||||||
| Walkosz et al.39 2007 | X | X | |||||
Furthermore, one study included appearance-based motivators in addition to the health-based motivators typically used38 (Table 4). The appearance-based intervention focused on premature aging due to increased exposure to ultraviolet (UV) light and how sunscreen use can delay the signs of skin aging, while the health-based intervention focused on the association between skin cancer and UV light exposure. In the study by Tuong and Armstrong38, participants in the appearance-based group showing a statistically significant increase in sunscreen use (mean ± SD: 2.8 ± 2.2 days/week, p<0.001). Furthermore, between-group comparisons showed that sunscreen use in the appearance-based group was significantly greater than the health-based group (2.2 ± 1.4 vs 0.2 ± 0.6 days/week, p<0.001). While effect sizes were not reported, these differences may be important for public health because it provides evidence that appearance-based messaging, in the form of a video, could influence behavior.
One feature commonly included in the studied educational toolkits for behavior change and environmental health was having interactive components (Table 4). In this review, 8 of the 17 admissible studies used some form of interactivity in the design of their educational toolkit, such as games, in-class activities, small group discussion, role-playing, and question-and-answer periods27-29,31-33,37,40.
Another feature commonly observed in the review of 7 of the 17 included studies was building an educational toolkit that had various multimedia tools27,31-35,39 (Table 4). These educational toolkits included workbooks, PowerPoint presentations, text message reminders, educational videos, and online games.
Personalization of the educational toolkit was also included in 3 of the 17 studies in this review as a key feature30,31,33 (Table 4). Some key aspects of personalization included having personalized risk feedback and recommendations and a wide variety of methods for learning.
Incentive-based learning also appeared in one study27 (Table 4). The authors provided participants with achievement certificates following completion of the material included in the educational toolkit, motivating participants to work through the educational material27.
Accessibility of the toolkit was another very common feature identified in this review (Table 4). Accessibility appeared in 7 of the 17 studies26,28,29,34,36,37,41. For example, Armstrong et al.26 provided participants with a video that addressed how sunscreens work to protect skin, the different types of sunscreens, importance of use, and proper application. While they asked participants to view the video at least once during the intervention period, participants had access to the material whenever they wished to view it. By allowing participants to view the educational material as often as they would like, in different locations, and increasing awareness on how they can access protective products such as sunscreen, this increased the accessibility of the educational toolkit26. On the other hand, in the study by Shin et al.37, participants were provided with access to the educational toolkit during classroom hours. This provided participants with the ability to review topics, such as air pollution and chemical use, during dedicated sessions, rather than putting the onus on them to review.
Data collection methods used to evaluate environmental health toolkits
Understanding how to measure behavior change in terms of environmental health-related educational toolkits is important for evaluating whether they are effective. In all, 13 studies reported values (i.e. means) and p-values, while four reported effect sizes (i.e. standardized B, odds ratio, etc.) and 95% confidence intervals (Table 3). From the 17 studies included in this systematic review, five different data collection methods were identified as means for evaluating the effectiveness of these toolkits.
First, knowledge questionnaires appeared in 10 out of 17 studies28-33,36,37,40,41 (Table 5). In terms of knowledge, questionnaires discussed whether participants had knowledge or awareness of disease, risk factors, and prevention, as well as participants’ information seeking behavior. Five of the studies tested for validity or reliability of the knowledge questionnaires; however, only five studies reported the results31-33,37,40. In the study by Buller et al.28, the overall kappa value estimating two-week test-retest reliability was 0.39 for knowledge. Additionally, Buller et al.29 evaluated knowledge by asking questions regarding key term recognition (second post-test reliability analysis, α=0.76) and skin cancer (second posttest reliability analysis, α=0.73). Furthermore, Gritz et al.31 used a modified version of the Awareness Concerns Interest (ACI) Scale, which was specific to sun protection (α=0.73). A similar study by Hoffman et al.32 used a 12-item knowledge scale that contained multiple choice and true or false questions (α=0.63, 2-week test-retest, r=0.61). Reynolds et al.36 developed questionnaires for both the adolescents and parents to complete as part of the study. The 10-item knowledge questionnaire had 2-week test-retest reliability 0.39 for adolescents. Additionally, Campbell et al.41 used a shortened version of the Chicago Lead Knowledge Test. While the full-length questionnaire has a test-retest reliability of 0.91, it was not evaluated for the shortened version.
Table 5
Extracted data collection methods for behavior change in environmental health (N=17)
| Authors Year | Knowledge | Health beliefs | Health risk perceptions | Behaviors and practices | Biological measures |
|---|---|---|---|---|---|
| Armstrong et al.26 2011 | X | ||||
| Aygun and Karayağız Muslu27 2021 | X | X | X | ||
| Buller et al.28 2006 | X | X | X | ||
| Buller et al.29 1994 | X | X | |||
| Campbell et al.41 2011 | X | X | |||
| Glanz et al.30 2010 | X | X | |||
| Gritz et al.31 2007 | X | X | |||
| Hien et al.40 2008 | X | ||||
| Hoffman et al.32 1999 | X | X | X | X | |
| Jeihooni and Rakhshani43 2019 | X | X | X | X | |
| Mayer et al.34 2007 | X | ||||
| Mays et al.33 2011 | X | ||||
| Norman et al.35 2007 | X | ||||
| Reynolds et al.36 2008 | X | X | X | ||
| Shin et al.37 2020 | X | X | X | ||
| Tuong and Armstrong38 2014 | X | ||||
| Walkosz et al.39 2007 | X |
Measurement of health beliefs was also commonly used to study the effectiveness of the educational toolkits on changing behavior42. Of the 17 studies included in this review, seven used questionnaires that evaluated the participants health beliefs and were tested for validity and reliability27-29,32,33,36,37 (Table 5). Aygun and Karayağız Muslu27 used the Sun Protection Self-Efficacy Scale (SES) to assess participants’ confidence or belief in their ability to reduce their exposure to the sun and use sunscreen and a hat in various situations. Validity and reliability testing for the Turkish version of this questionnaire found that it was valid and reliable (SES α= 0.86, sun avoidance α=0.65, sunscreen use α=0.84, and hat use α=0.69). In the study by Buller et al.28, the overall kappa value estimating two-week test-retest reliability was 0.67 for attitudes and 0.54 for self-efficacy (p<0.0001). Buller et al.29 used three two-item factors for attitude and health beliefs, which reported Cronbach alpha values from 0.54 to 0.72 for the second post-test reliability analysis. Additionally, Hoffman et al.32 investigated participant health beliefs by looking at their perceived costs and benefits of engaging in sun protective behaviors (α=0.74, 2-week test-retest, r=0.64). For Jeihooni and Rakhshani43, reliability measures of perceived benefits, perceived barriers, and self-efficacy were 0.78, 0.82, and 0.79, respectively. The attitude and self-efficacy questionnaires developed by Reynolds et al.36 had a 2-week test-retest reliability of 0.67 and 0.54, respectively. Furthermore, Shin et al.37 did not report exact values for their test of internal consistency of the self-efficacy questionnaire; however, it was reported to be above 0.70.
Using measurements of health risk perceptions was another form of evaluating toolkit effectiveness. In this review, 4 of the 17 included studies incorporated the evaluation of health risk perceptions in their questionnaires27,32,33,37 (Table 5). Measuring health risk perceptions included questions about perceptions of exposure risks and perceived pros and cons of adopting new health behaviors. Aygun and Karayağız Muslu27 used the Sun Protection Decisional Balance Scale (DBS) to evaluate the perceived pros and cons of sun protection behaviors. Validity and reliability testing was carried out for the Turkish version of the DBS, with results indicating that the questionnaire is valid and reliable (pros α=0.76, cons α=0.71). Hoffman et al.32 evaluate health risk perceptions using a validated intentions scale, which examined participants intended use of sun protection based on their perceived health risks (α=0.74, 2-week test-retest, r=0.68). Additionally, for Jeihooni and Rakhshani43, reliability measures of perceived susceptibility and perceived severity were 0.84 and 0.77, respectively. Shin et al.37 did not report exact values for their test of internal consistency of the risk perception questionnaire; however, it was reported to be above 0.70.
Direct measures of behavior change and practices were used to evaluate toolkit effectiveness in 12 out of the 17 included studies26-28,30-36,38,39 (Table 5). Additionally, 12 of these studies used validated questionnaires to assess behaviors, while the study by Walkosz et al.39 did not complete any validity or reliability testing. These studies included questions about product use before and after intervention, the frequency of protection behaviors, as well as intended behaviors and practices. Both Armstrong et al.26 and Tuong and Armstrong38 used questions about sun protective behaviors from the National Health and Nutrition Examination Survey (NHANES), which is a validated questionnaire. Aygun and Karayağız Muslu27 developed the Sun Protection Behavior Scale (SPBS) to measure sun protection behaviors. The validity and reliability of the Turkish version of the SPBS was evaluated and found to be valid and reliable (SPBS α=0.74, sun avoidance α=0.63, sunscreen use α=0.89, hat use α=0.71). In the study by Buller et al.28, behavior was measured by collecting a sun protection diary of the frequency of behaviors, which was validated in a previous pilot study. Similarly, Glanz et al.30 collected data from a sun exposure diary in addition to the Sun Habit Survey. The main outcomes of the diary had testretest reliability scores of 0.69 to 0.73. Gritz et al.31 measured sun protective behaviors using a five-item questionnaire, with items summed to create aggregate scores for sunscreen use (α=0.87) and sun avoidance (α=0.56). Self-reported behavior was also measured by Hoffman et al.32, in which a visual analog scale was used to quantify sunscreen use (testretest, r=0.59–0.85). Jeihooni and Rakhshani43 measured self-reported behaviors against skin cancer, which included 10 ‘yes or no’ questions (score range: 0–10). While the questionnaire was validated, the exact measure was not reported (α >0.70). Frequency of sun protective behaviors were reported by Mayer et al.34; however, the validation of the instrument was not discussed. Mays et al.33 measured sun safety behaviors using a scale adapted from previous research, which was demonstrated to be valid and reliable. The scale demonstrated acceptable internal consistency within their sample (baseline α=0.63, follow-up α=0.80). A sun protection behavior scale was also used by Norman et al.35, which was found to have a one-week test-retest reliability of 0.70 and a Cronbach α of 0.78). Interestingly, Reynolds et al.36 used sun protection diaries that had been validated in a previous pilot study and were to be completed by both the adolescents and parents in the study.
Finally, one study included in this review used biological markers as a method for evaluating the effectiveness of the toolkit41 (Table 5). The authors collected participant’s blood to measure their blood lead levels (BLLs) at baseline, 6 months, and 12 months. Differences in baseline and follow-up BLLs between the control and intervention group were used to evaluate the effectiveness of their educational toolkit at reducing lead exposure in newborns.
DISCUSSION
Summary of findings
This systematic review identified seven characteristics that are commonly found in educational toolkits for environmental health and behavior change including 2 elements and 5 features. On the one hand, the elements include having simple, consistent, and frequent messaging and including both health-based and appearance-based motivators. On the other hand, the features include being interactive, multimedia, personalized, incentive-based, and accessible. These findings are important as they can be used to guide the future development of toolkits for environmental health education. Furthermore, this review has also found appropriate and commonly used methods for collecting data to evaluate the effectiveness of these educational toolkits. These methods include questionnaires that evaluate knowledge, health beliefs, health risk perceptions, behaviors and practices, attitudes, and biological measures. Additionally, most of the studies used many of the elements and data collection methods in unison to develop the most effective toolkits and reliable methods of evaluation.
Characteristics of environmental health literacy toolkits
This systematic review identified seven key characteristics of environmental health literacy toolkits. Accessibility and offering simple, consistent and frequent messaging were important characteristics of the educational toolkits, which can be accomplished using features such as interactivity and multimedia tools. These findings are consistent with a number of existing review articles on educational toolkits for health education and behavior change18,42. Additionally, a scoping review of educational toolkits used in health and healthcare also observed the importance of including multimedia characteristics. In this review, eight of the 83 included educational toolkits contained both written and audiovisual components, facilitating increased knowledge through multimedia tools18. Furthermore, a systematic review on digital tools in health education found that they can influence active learning, the development of skills, and promote critical thinking42. This supports the use of multimedia tools in the development of environmental health literacy toolkits. Having simple, consistent, and frequent messaging was another one of the key characteristics of the environmental health literacy toolkits. These findings are consistent with a previous scoping review, which found that simplification, in the form of simplified label information and clear breakdowns of product ingredients, may help consumers make informed decisions.
Personalization by having personalized risk feedback and having a wide variety of methods for learning, as well as including incentives and self-rewards, has been identified as an important feature. These characteristics are consistent with previous literature44,45. To highlight, tailoring educational content to individual needs and providing personalized recommendations, shopping tips, and content were important for educational toolkits. Patient self-assessment tools, checklists, activity journals and diaries, links to online resources for patients, educational videos, and peak flow meters for patients are all aspects that can assist personal user learning and have been utilized by previous health-related educational toolkits45. Furthermore, a systematic review on behavior change tools used by family doctors observed that the most common behavior change strategies included goals and planning, feedback and monitoring, social support, shaping knowledge, and repetition and substitution44.
Data collection methods for evaluating effectiveness of toolkits
Methods of data collection to evaluate effectiveness were also identified by this systematic review. These methods include questionnaires that assess knowledge, health beliefs and attitudes, health risk perceptions, behaviors and practices, and biological measures. These outcome measures have been used in previous research and are consistent with the existing evidence base17,44,46. To highlight, in the scoping review by Yamada et al.17, patient-reported or perceived self-efficacy, self-management behaviors, PCP behaviors, and health-related behaviors were commonly used outcome measures. Additionally, biological measures such as the means number of lymph nodes present and A1c levels were also reported. These outcome measures were also identified in a study that aimed to validate the Environmental Health Literacy Index (EHLI), a tool to evaluate environmental health literacy in university students46. The validated EHLI covers three domains of health literacy: 1) functional (actions to fight against pollution); 2) interactive (how much the user agrees with some statements on the link between environmental pollution and health); and 3) critical (the perception of health risks).
Strengths and limitations
This systematic review provided a qualitative content analysis of the current literature surrounding the characteristics and data collection methods of educational toolkits that support behavior change in environmental health. Screening, critical appraisal, and data extraction were all completed by two independent reviewers, with the aim of reducing bias as much as possible.
There are also some limitations that may be identified within the systematic review. First, it is possible that all relevant studies were not retrieved because our search strategy was in English. Second, our search strategy was not reviewed by a second librarian, which is recommended by the Peer Review of Electronic Search Strategies (PRESS) checklist. Thirdly, although this systematic review was not a meta-analysis, another limitation may arise due to publication bias. There may be studies or grey literature that were not included in this review because they had negative results and were, therefore, not published. Additionally, the majority of the studies included in this review were based in the United States and Canada, two high-income countries. This presents a limitation with the generalizability and applicability of the results.
Implications and future directions
The findings from this systematic review present the need to develop and evaluate the effectiveness of an environmental health literacy toolkit to increase knowledge translation and facilitate behavior change in relation to parabens in PCPs. The majority of the included studies featured in this review evaluated the effectiveness of the educational resources and toolkits in the United States and Canada. This presents an area of concern, but also of future research, because these toolkits were evaluated in a high-income country. Future research should include evaluate the effectiveness of an environmental health literacy toolkits in other settings as well, especially low-income countries as they are exposed to products and waste that were considered to be too contaminated from higher income countries47.
The findings from this review informed the development of ‘Paraben-Free & Me’, a digital educational toolkit designed to support behavior change in environmental health, particularly around EDCs. The app targets women’s perceptions of susceptibility and severity regarding parabens by providing accessible, multimedia content such as blog posts, videos, podcasts, and quizzes. It also addresses barriers to behavior change by offering practical resources for finding paraben-free products and includes interactive features like forums to foster community learning. The toolkit is freely accessible on smart devices via the Spaces by Wix app.
The findings from this systematic review may significantly contribute to recommendations for future educational toolkits and their evaluation. First, include the core, structure elements of having simple, consistent, and frequent messaging, putting accessibility at the forefront, and include both health- and appearance-based motivators. Second, the features that make toolkits appealing and engaging include being interactive, multimedia, personalized, and incentive-based. Finally, self-reported outcome measures such as knowledge, health beliefs and attitudes, health risk perceptions, behaviors and practices, and biological measures remain critical methods for evaluating the effectiveness of environmental health literacy toolkits.
CONCLUSIONS
This systematic review highlighted the important characteristics that are commonly used in environmental health literacy educational toolkits to promote behavior change. Furthermore, this review also identified the methods of data collection that evaluate the effectiveness of these educational toolkits. By disseminating the evidence of why EDCs are harmful and what to look for in a format that is accessible and easy to understand, women will be provided with more knowledge and better awareness about the health impacts. Not only will they be able to reduce their personal exposure to EDCs, but their family’s exposure as well, resulting in physical and environmental health promotion. By reducing exposures to EDCs, it encourages health improvements and enhanced quality of life. While these findings can be used to begin addressing personal EDC exposure, more research is needed before widespread implementation in practice.
