Psychosocial Intervention Psychosocial Intervention
Psychosocial Intervention 26 (2017) 117-24 - Vol. 26 No.2 DOI: 10.1016/j.psi.2017.01.001
An illustration of how program implementers can use population-specific analyses to facilitate the selection of evidence-based home visiting programs
Ejemplo de cómo los directores de programas pueden emplear análisis específicos para grupos de población con el fin de facilitar la selección de programas de visitas domiciliarias basados en la evidencia
Ayesha C. Sujana,, , John Eckenrodeb
a Indiana University, USA
b Cornell University, USA
Received 30 October 2016, Accepted 23 January 2017
Abstract

Given that effective home visiting (HV) programs targeting at-risk families impact different outcomes and associations between risk factors and outcomes may vary across populations, program implementers should evaluate population-specific risk-outcome associations in order to select interventions that are most likely to benefit families in target communities. We used data collected in a rural community in upstate New York (i.e., Elmira) and three standard statistical methods (i.e., bivariate, multivariate, and cumulative risk analyses) to assess associations between maternal socio-demographic risk factors and outcomes typically targeted with HV interventions. With the results, we illustrated how program implementers could use population-specific analyses of data collected prior to the implementation of HV interventions to select interventions that may be most likely to benefit families in a target community. For example, our multivariate results suggested that lower socioeconomic families in Elmira were particularly at-risk for child maltreatment, poor family economic self-sufficiency, and poor child academic achievement, indicating that it may be particularly beneficial to implement HV programs that have been shown to affect these outcomes (e.g., Nurse Family Partnership and Parents as Teachers) in Elmira. We encourage program implementers to conduct similar population-specific analyses to help select evidence-based HV interventions for their target communities.

Keywords
  • Home visiting interventions
  • Maternal and child health
  • Targeting interventions
  • Disseminating interventions
Resumen

Puesto que el impacto de los programas de visitas domiciliarias (VD) centrados en las familias en situación de riesgo obtiene resultados diferentes y las asociaciones entre factores de riesgo y resultados pueden variar en función de los grupos poblacionales, los encargados de los programas deberían evaluar las asociaciones entre los riesgos y los resultados en poblaciones específicas con el fin de seleccionar las intervenciones que más beneficio reportarán a las familias de las comunidades seleccionadas. Se recogieron datos de una comunidad rural del norte del estado de Nueva York (p. ej., Elmira) y tres métodos estadísticos estándar (p. ej., análisis de riesgo bivariante, multivariante y acumulativos) para valorar las asociaciones entre los factores de riesgo materno sociodemográficos y los resultados típicamente buscados en las intervenciones de VD. Con los resultados ilustramos cómo los directores de los programas pueden utilizar análisis de datos específicos de una población que se han recopilado de manera previa a las intervenciones de VD para seleccionar las intervenciones que tendrían más probabilidad de beneficiar a las familias de las comunidades seleccionadas. Por ejemplo, nuestros resultados del análisis multivariante sugerían que las familias de Elmira con un nivel socioeconómico más bajo presentaban un riesgo especial de maltrato infantil, una pobre autosubsistencia económica familiar, y un rendimiento académico infantil bajo, lo que indica que podría ser especialmente beneficioso para esa localidad el poner en marcha programas de VD que hayan demostrado un efecto sobre estos resultados (p. ej., Nurse Family Partnership y Parents as Teachers). Se anima a los encargados de los programas a dirigir análisis similares específicos para grupos poblacionales que ayuden a la selección de las intervenciones de VD basadas en la evidencia para las comunidades seleccionadas.

Keywords
  • Intervenciones de visitas domiciliarias
  • Salud materno-infantil
  • Intervenciones dirigidas
  • Intervenciones diseminadas

Providing effective home visiting (HV) interventions to at-risk pregnant women and at-risk families with young children is a public health priority. The Patient Protection and Affordable Care Act established the Maternal, Infant, and Early Childhood Home Visiting Program (MIECHV) to provide $1.5 billion dollars to evidence-based HV programs (US Department of Health & Human Services, 2013). In order to determine what programs are effective, in 2009, the U.S. Department of Health and Human Services (DHHS) contracted with Mathematica Policy Research to conduct the Home Visiting Evidence of Effectiveness (HomVEE) review. Mathematical Policy Research considered the following eight maternal and child outcomes: (1) child health, (2) child development and school readiness, (3) reductions in juvenile delinquency, family violence and crime, (4) maternal health, (5) positive parenting practices, (6) reductions in child maltreatment, (7) family economic self-sufficiency, and (8) linkage and referrals to community resources and supports. In order for a program to meet criteria for evidence of effectiveness, either (a) at least one high- or moderate-quality study had to have found that the program had favorable, statistically significant impacts in two or more of the eight outcome domains or (b) at least two high- or moderate-quality studies with non-overlapping samples had to have found that the program had one or more favorable, statistically significant impact in the same outcome domain. Thus, depending on how the original evaluation studies were conducted, a program was considered effective if it had demonstrated favorable, statistically significant impacts on either one or two of eight specified maternal and child outcomes. As such, different HV programs were deemed effective based on their demonstrated impact on a variety of maternal and child health outcomes.

State administered MIECHV programs typically target high-need communities based on a profile of socio-demographic risk factors. Likewise, individual HV programs tend to select families for services based on socio-demographic risk factors, such as young maternal age, single parent families, and low family income. A public health model focusing on preventing the development of adverse outcomes would argue for an understanding of the statistical relationships between prevalent adverse outcomes and the risk factors predictive of those outcomes in the population of interest. It cannot be assumed, however, that every individual risk factor (e.g., young maternal age, single mothers, or low family income) used to target families for HV services is necessarily predictive of all of the outcomes specified by DHHS as criteria for designating a HV program as evidence-based. Indeed, some of the commonly used socio-demographic risk factors used for targeting families for such programs, such as teenage childbearing, have been found to poorly predict future child outcomes, compared to other individual risk factors or combinations of risk factors (Chittleborough, Searle, Smithers, Brinkman, & Lynch, 2016). Also, it may be useful to consider that the total number of risk factors present in a family's life may be an important predictor of some outcomes. Finally, it cannot be assumed that risk factor-outcome associations reported in the literature from observational studies of a given population at a given time will generalize to other populations and times.

In this paper we propose that prior to implementing a HV intervention in a target community, as part of a needs assessment or feasibility study, program planners and implementers should use population-specific data to identify maternal and child health outcomes that are associated with risk factors in that community. Although to illustrate how this can be done, we used data from the control group of a randomized intervention, data can come from other less resource intense sources, such as retrospective surveys or administrative data (e.g., hospital and school records). Then, based on the results of the analyses, specific evidence-based HV programs should be selected for families in that community. This decision-making should take into account the specific outcomes different HV programs on the approved MIECHV list have been shown to impact in evaluation studies. To illustrate how results from population-specific analyses could be used to select HV programs for specific communities, we conducted population-specific analyses using data collected in a rural community in upstate New York (i.e., Elmira, New York).

Approaches to testing population-specific associations between risk factors and outcomes

Our illustrations assumes that program implementers have access to either (a) a well-designed observational study (e.g. representative community survey with good response rates) using well-measured constructs or (b) administrative data with good coverage of the population to be served by the HV program. In our illustration, we use statistical approaches likely to be familiar to local program planners and evaluators (rather than “cutting edge” approaches typically favored by university-based researchers).

Bivariate associations

Bivariate associations (e.g., correlations or t-tests) are a simple and common way to assess associations between individual risk factors and individual outcomes. However, there are major limitations to bivariate analyses. First, they cannot definitively assess whether a risk factor other than the one of interest may be responsible for an observed association. For example, young maternal age may correlate with a child health outcome. But it may be that the reason why this association is observed is because women who have children at younger age are more likely to be poor, with poverty as the real “driver” of this association. So, in this case, targeting poor women may be a more effective strategy than selecting communities or families based on maternal age. Second, bivariate associations cannot evaluate the potential impact of the number of risk factors present. Given the high rate of co-occurrence of risk factors in families, it may be that the number of maternal or family risk factors is more predictive of an adverse outcome than any specific risk factor.

Multivariate regression models

Multivariate regression models allow for the examination of the predictive power of each risk factor independent of other measured risk factors. For example, the association between maternal age and child health could be assessed while accounting for poverty status. There are also limitations of multivariate regression. For example, if risk factors are highly correlated, multivariate regression models may yield biased parameter estimates and reduced statistical significance for individual risk factors (Cohen, Cohen, West, & Aiken, 2003; Myers & Wells, 2003).

Cumulative risk models

These models predict outcomes from a count of the number of risk factors present and yield an estimate of the overall effects of multiple interrelated risk factors in a simple and comprehensive way without requiring large sample sizes to have adequate statistical power (Evans, Li, & Whipple, 2013; Lee & Harris, 2009). However, there are limitations to these models, including the assumption that all included risk factors are of equal importance (Lee & Harris, 2009). Additionally, for simple counts, they require dichotomizing outcomes to a present/absent status, which results in a loss of information regarding the severity of each risk factor and reduced statistical power (Cohen, 1983; Royston, Altman, & Sauerbrei, 2006).

Recommendations for testing population-specific associations between risk factors and outcomes

Given that there are strengths and weaknesses to bivariate, multivariate, and cumulative risk analyses, we recommend that program implementers use all three methods and compare results from the methods to evaluate population-specific associations between risk factors and targeted outcomes.

Methods to evaluate program outcomes

Given that DHHS stipulated that HV programs must have shown impacts on only one or two (of eight) outcomes, it is important to conduct analyses to assess associations between risk factors and individual outcomes. However, some HV programs impact many of the outcome domains targeted by DHHS (US Department of Health & Human Services, 2013), and many states and communities are interested in improving several maternal and child health outcomes with their HV programs, not just one or two. As such, it would also be helpful to assess associations of a risk factor with several outcomes simultaneously since community-level and family-level risk factors might be chosen that have the most broad-based impact on desired outcomes, and programs might be selected based on their impact on multiple outcomes. In this case, the focus can shift from individual outcome measures to composite outcome scores.

There are also some statistical advantages to composite outcome scores. When assessing associations between several risk factors and several outcomes, it is possible to find statistically significant associations simply by chance. Statistical adjustments can correct for these potential errors, such as the Bonferroni adjustment; however, these adjustments tend to yield overly conservative estimates, especially when outcomes are highly correlated (Tyler, Normand, & Horton, 2011; Yoon et al., 2011). Combining outcomes into a composite measure addresses problems associated with multiple testing without requiring statistical adjustments (Freemantle, Wood, Eastaugh, & Calvert, 2003). However, it is important to note that there are also limitations to composite measures. They are extremely sensitive to missing data (Yoon et al., 2011), especially if a complete case analysis is adopted (Teixeira-Pinto, Siddique, Gibbons, & Normand, 2009). Also, it may be incorrect to assume that all outcomes making up the composite score are of equal importance (Drake et al., 2013; Prochaska, Velicer, Nigg, & Prochaska, 2008), or that a significant association between a risk factor and a composite score means that the risk factor equally affects all the outcomes that make up the composite (Freemantle et al., 2003).

Given that there are strengths and weaknesses to using individual outcomes and strengths and weaknesses to using composite outcome scores, we recommend that program implementers assess associations between risk factors and individual outcomes, as well as associations between risk factors and composite outcome scores.

Objectives

The purpose of the present study was to conduct a set of analyses (i.e., bivariate, multivariate, and cumulative risk analyses) assessing risk-outcome associations on a community sample of at-risk families to illustrate how results from population-specific analyses can facilitate the selection of evidence-based HV interventions that are particularly likely to benefit families in a target community.

MethodsParticipants

Participants were from the control group of the Nurse Family Partnership (NFP) program randomized clinical trial conducted in and around Elmira, a small semi-rural town in upstate New York. Between 1978 and 1980, pregnant women who had no previous live births were recruited from health clinics and doctors’ offices. Five hundred eligible women were invited to participate and 400 enrolled. The 400 women who enrolled were randomly assigned to the intervention group or the control group. Women completed informed consent at baseline interviews. The institutional review board at Cornell University in Ithaca, New York granted approval. All procedures were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. See previous publications for additional details about the original Elmira NFP study (e.g., Olds et al., 1997, 1998). Given that the intervention had several effects on maternal and child outcomes (Olds et al., 1997, 1998) and that the purpose of the present investigation was to examine associations of risk factors with maternal and child outcomes, rather than assess intervention effects, analyses were limited to data from 141 control group families.

Procedure

Prior to randomization women completed informed consent and baseline intake assessments. At a 15-year follow-up, both women and children completed assessments. See earlier publications for details about intake assessments (Olds, Henderson, Chamberlin, & Tatelbaum, 1986; Olds, Henderson, & Kitzman, 1994; Olds, Henderson, Tatelbaum, & Chamberlin, 1988) and follow-up assessments (Olds et al., 1997, 1998).

MeasuresSocio-demographic risk factors

During the intake interviews women reported socio-demographic information, including age, highest grade attended, and marital status. Women also completed the Hollingshead Four-Factor index to estimate household SES (Hollingshead, 2011).

Cumulative risk score

We computed a cumulative risk score by dichotomizing each risk factor into a 0/1 variable (maternal age under 18 years=1, less than a high school education=1, unmarried=1, below the mean Hollingshead score=1) and summing all dichotomized risk variables.

Child anxiety and depression

At the follow-up assessment women completed the Child Behavior Checklist, which is a measure that assesses child behavior problems occurring in the last 12 months and includes an anxiety/depression subscale (Achenbach & Edelbrock, 1981; Ivanova et al., 2007).

Child academic achievement

We converted seventh through ninth grade English, Science, Math, and Social Studies letter grades to a five-point numeric scale and averaged them (range=0–4, with higher values indicating higher grades).

Child school suspensions

We obtained information on the total number of times a child received a school suspension before the 15-year assessment.

Child early onset behavior problems

To create a total early onset behavior problem score, we summed dichotomized early onset behavior problems variables. Early onset behavior problems included smoking cigarettes before age 12; having sexual intercourse before age 13; smoking marijuana before age 14 and using inhalants, hallucinogens, cocaine, or heroin; having a person in need of supervision report filed; being arrested; being convicted of a crime, and binge drinking before age 15. Cutoff ages were based on national data or frequency distributions of the behaviors in the NFP control group sample. See Eckenrode et al. (2001) for additional details on the early onset behavior problems measure.

Composite poor child outcome score

Composite poor child outcome scores were computed by reverse coding academic achievement, standardizing all child outcome variables, and then summing the standardized child outcome variables.

Maternal mental health

At the 15-year follow-up, women completed the Mental Health Inventory, which yields an overall mental health score reflecting anxiety, depression, behavior and emotional control, general positive effect, and emotional ties (Veit & Ware, 1983). A low score is indicative of poor mental health.

Maternal drug and alcohol impairment

At the 15-year follow-up, women completed an adapted questionnaire from the National Comorbidity Survey (Kessler, 1995) to assess the impact of substance use on the following life domains: missing work, experiencing trouble at work, having a motor vehicle crash or traffic violation, compromising child care, and receiving treatment. We used an overall score which summarized impact across all life domains assessed.

Child maltreatment

After receiving consent from the women, research staff obtained information from state Child Protective Service records on the total number of maltreatments reports involving the women and children up to the children's 15th birthdays.

Family instability

A family instability measure was based on maternal reports of number of intimate partners, work hour changes and residence changes, and number of school transfers documented in the child's school records. See Marcynyszyn, Evans, and Eckenrode (2008) for additional information about this measure.

Family economic self-sufficiency

Variables used to create this measure were collected at the 15-year follow-up and included total number of months a woman was employed since the birth of her first child, number of months a family was on Aid to Families with Dependent Children since the birth of the first child, highest grade attended by the mother, and family economic hardship. Family economic hardship was assessed with the Economic Hardship Questionnaire, a 12-item measure of changes in a family's style of living during the past six months (Lempers, Clarklempers, & Simons, 1989). Poor family economic self-sufficiency scores were computed by reverse coding months of maternal employment and highest maternal education, standardizing all family economic self-sufficiency variables, and summing all the variables.

Number of subsequent children

At the 15-year follow-up, women reported the number of subsequent children they had.

Composite poor maternal outcome score

We computed composite poor maternal outcome scores by reverse coding maternal mental health, standardizing all maternal outcome variables, and then summing the standardized maternal outcome variables.

Analyses

We used three statistical analyses to assess associations between individual outcomes and composite poor outcome scores, and we standardized all predictors and outcomes so we could compare the magnitude of the associations. First, we examined bivariate associations by assessing correlations between each risk factor and outcome. Second, we conducted multivariate regression analyses, which included all risk factors as predictors in each model. Third, we conducted regression analyses using cumulative risk scores to predict each outcome.

Results

See Table 1 for descriptive statistics and information about missing data. All risk factors were significantly, but moderately, correlated with the other risk factors and the cumulative risk score (results not shown).

Table 1.

Descriptive statistics.

  n (%aMean  SD  Range 
Risk factors
Maternal marital statusb  141 (100)  –  –  – 
Maternal age (in years)  141 (100)  19.33  2.89  14–31 
Maternal education (in years)  141 (100)  11.23  1.46  7–16 
Household SES  141 (100)  54.45  18.49  22–126 
Cumulative risk  141 (100)  1.88  1.32  0–4 
Child outcomes
Anxiety/depression  136 (96)  3.61  4.31  0–20 
Academic achievement  129 (91)  2.13  0.85  0.58–4 
Number of school suspensions  141 (100)  6.99  21.00  0–190 
Number of early onset behavior problems  139 (99)  1.27  1.71  0–8 
Composite poor child outcome score  125 (89)  −0.11  2.63  −4.02 to 13.6 
Maternal outcomes
Mental health  141 (100)  3.74  0.59  2.39–4.92 
Drug/alcohol impairment  141 (100)  0.60  1.59  0–10 
Number of child maltreatment reports  137 (97)  0.91  2.33  0–14 
Family instability  140 (99)  −0.02  2.69  −4.07 to 8.2 
Poor family economic self-sufficiency  128 (91)  2.79  −6.26 to 7.93 
Number of subsequent children  140 (99)  1.68  1.11  0–6 
Composite poor maternal outcome score  127 (90)  −0.22  3.06  −5.18 to 10.97 
a

Percent of sample.

b

52 (37%) of mothers were married.

Bivariate models

See Table 2 for correlations between risk factors and outcomes. All risk factors were associated with multiple individual outcomes, such that greater levels of a risk factor predicted poorer outcomes. Younger maternal age and less maternal education were associated with poorer composite child outcome scores, and all four risk factors predicted poorer composite maternal outcome scores. Overall, marital status and household SES were less robust predictors of child outcomes, compared to maternal age and education.

Table 2.

Bivariate associations: correlations between baseline risk factors and 15-year outcomes.

  Maternal age  Maternal education  Maternal marital statusa  Household SES 
Child outcomes
Anxiety/depression  −0.06  −0.03  −0.10  −0.06 
Academic achievement  0.17*  0.24*  0.10  0.30* 
School suspensions  −0.01  0.02  0.06  −0.04 
Early onset behavior problems  −0.14  −0.19*  0.13  −0.06 
Composite poor child outcome score  −0.16  −0.15  −0.13  −0.11 
Maternal outcomes
Mental health  0.02  0.05  0.20*  −0.04 
Drug/alcohol impairment  −0.21*  −0.16  −0.21*  −0.07 
Child maltreatment reports  −0.01  −0.18*  −0.06  −0.23* 
Family instability  −0.29*  −0.21*  −0.30*  −0.15 
Poor family economic self-sufficiency  −0.14  −0.49*  −0.39*  −0.42* 
Number of subsequent children  −0.28*  −0.24*  −0.05  −0.08 
Composite poor maternal outcome score  −0.25*  −0.38*  −0.41*  −0.30* 
*

p<0.05.

p<0.10.

a

Unmarried=0, married=1.

Multivariate models

See Table 3 for multivariate results. Compared to the bivariate models, there were fewer statistically significant results in the multivariate models, which assessed the impact of each risk factor while controlling for other risk factors. This suggests that results from multivariate models gave a more accurate estimate of the independent association between the risk factor and outcome. In all but one case, greater levels of a risk factor predicted poorer outcomes in the multivariate models. The results from the multivariate models showed that younger maternal age was associated with more family instability and subsequent children; lower levels of maternal education was associated with more child maltreatment reports and poorer family economic self-sufficiency; mothers being unmarried was associated with poorer maternal health, higher levels of maternal drug/alcohol impairment, higher levels of family instability, and poorer family economic self-sufficiency; and lower household SES was associated with lower levels of child academic achievement, more child maltreatment reports, and poorer family economic self-sufficiency. The only instance where a risk factor predicted lower levels of a poor outcome was younger maternal age at intake predicted better family economic self-sufficiency scores.

Table 3.

Multivariate results.

  Maternal age  Maternal education  Maternal marital statusa  Household SES 
Child outcomes
Anxiety/depression  0.05  0.00  −0.10  0.09 
Academic achievement  −0.07  0.09  0.00  0.25* 
School suspensions  −0.06  0.06  0.07  −0.07 
Early onset behavior problems  −0.02  −0.17  −0.07  0.02 
Composite poor child outcome score  −0.10  −0.03  −0.08  −0.07 
Maternal outcomes
Mental health  −0.05  0.04  0.22*  −0.09 
Drug/alcohol impairment  −0.15  −0.01  −0.16  −0.01 
Child maltreatment reports  0.14  −0.19  −0.01  −0.18* 
Family instability  −0.24*  0.05  −0.23*  −0.09 
Poor family economic self-sufficiency  0.28*  −0.46*  −0.29*  −0.22* 
Number of subsequent children  −0.23*  −0.12  0.07  −0.00 
Composite poor maternal outcome score  −0.00  −0.19  −0.32*  −0.16 
*

p<0.05.

p<0.10.

a

Unmarried=0, married=1; all reported estimates are standardized coefficients.

None of the risk factors predicted composite poor child outcome scores. However, lower levels of maternal education, being unmarried at intake, and lower levels of household SES predicted less favorable composite maternal outcome scores.

Cumulative risk models

The number of families at each level of cumulative risk was similar (see Fig. 1). See Table 4 for cumulative risk regression results. Cumulative risk significantly predicted the majority of the outcomes and the composite child and maternal poor outcome scores. In general, there was an increase in negative outcomes as the number of risk factors present increased (see Fig. 2 for examples).

Fig. 1.
(0.05MB).

Cumulative risk score distribution.

Table 4.

Cumulative risk results.

  Cumulative risk 
Child outcomes
Anxiety/depression  0.04 
Academic achievement  −0.24* 
School suspensions  0.01 
Early onset behavior problems  0.19* 
Composite poor child outcome score  0.17 
Maternal outcomes
Mental health  −0.12 
Drug/alcohol impairment  0.28* 
Child maltreatment reports  0.13 
Family instability  0.29* 
Poor family economic self-sufficiency  0.49* 
Number of subsequent children  0.22* 
Composite poor maternal outcome score  0.45* 
*

p<0.05.

p<0.10.

Fig. 2.
(0.29MB).

Examples of cumulative risk results.

Discussion

We used data from the control group of the Elmira randomized trial of the NFP program to illustrate how analyses of population-specific data can help program implementers better target HV programs to families most likely to benefit from them. We used three types of analyses, specifically bivariate, multivariate, and cumulative risk analyses, to look at the association of risk factors with individual outcomes and child and maternal composite outcome scores.

A comparison of results from the bivariate models and multivariate models showed that there were several risk factors that statistically significantly predicted outcomes in the bivariate models, but no longer predicted outcomes once other risk factors were accounted for in the multivariate models. This suggests that, if possible, program implementers should rely on results from multivariate analyses, rather than bivariate analyses, to obtain a more accurate depiction of what maternal or child health outcome should be targeted for at-risk families.

Once these outcomes have been identified, program implementers can select a home-visiting program that has been shown to effectively address those outcomes (see the HomVEE Review for a summary of the outcomes affected by particular HV programs). For example, our multivariate results suggested that lower SES families in Elmira, New York were particularly at-risk for child maltreatment, poor family economic self-sufficiency, and poor child academic achievement. Based on the HomVEE review there were two programs (i.e., NFP and Parents as Teachers) that target all three of these outcomes. Similarly, because our results indicated that maternal age (and not education, marital status, or SES) predicted number of subsequent pregnancies, if the policy concern is reducing unplanned pregnancies, our analyses suggest that the best strategy may be targeting mothers in Elmira based on age with a program shown to impact the number of subsequent pregnancies (e.g. NFP).

Targeting families with a risk factor that has been shown to be associated with an adverse outcome increases the probability of delivering an intervention to a family with a mother or child that otherwise may have developed an adverse outcome. It is important to note that in any given community there will be women and children from families without specified risk factors, or have lower levels of a risk factor, that do develop adverse outcomes (e.g., children of older mothers may show adverse outcomes). However, from a statistical perspective, targeting an at-risk family, rather than a family that does not have risk factors that have been shown to be associated with adverse outcomes, has a greater chance of having an impact. Thus, given that resources for HV programs are quite limited, it may be prudent to focus efforts on targeting families with risk factors that have been shown to be associated with adverse outcomes. The “prevention paradox” as popularized by the epidemiologist Geoffrey Rose (1985) is that “a large number of people at a small risk may give rise to more cases of disease than the small number who are at high risk.” However, offering expensive programs like home visitation to low risk families is beyond the capacity of most states and communities, so targeting high-risk families is still the preferred approach by many policy makers.

In addition to the bivariate and multivariate analyses, which assessed associations between individual risk factors and maternal and child outcomes, we also conducted cumulative risk analyses to assess the association between the number of risk factors present in families’ lives and maternal and child outcomes. We found that, in contrast to the estimated impact of individual risk factors in the multivariate models, cumulative risk scores predicted the majority of the individual outcomes, as well as the maternal and child composite outcome scores. This suggests that, if the policy goal is to impact multiple outcomes, several risk factors should be utilized in selecting families and interventions should be chosen that have been shown to effectively impact a number of outcomes, such as NFP or Healthy Families America. The challenge with this approach, however, is to evaluate the costs and benefits of using cumulative risk scores in practice, since many community or state agencies may be uncomfortable limiting services to the small number of families at the very highest level of risk (e.g. only 13.5% of families in our Elmira sample had all four risk factors).

Study limitations and future directions

There were several limitations to the present study. First, the sample size was relatively small, which may have limited the statistical power to detect statistically significant results. Second, we also only examined a limited number of risk factors, and there are other important socio-demographic risk factors, such as parental employment status, that should also be considered when targeting families for HV programs. Third, the baseline risk factors were assessed in the late 1970s and early 1980s, which may limit the generalizability of our findings to the current social and political climate (e.g. in a post welfare-reform era) given that variables collected then may not have the same meaning as they do now. However, the purpose of our analyses was not to evaluate how HV programs should be disseminated in the population assessed in our analyses, but rather to demonstrate how population-specific analyses can be conducted to facilitate effective targeting of HV programs. Fourth, we provide an illustration of a method to target existing home-visiting programs, prior to implementation, to families most likely to benefit from them. Future research should also evaluate how home-visiting programs can more precisely target specific services to specific families within their own existing models once programs are adopted in a community. For example, a recent clinical trial, within the Nurse Family Partnership program, demonstrated that allowing nurses to tailor the frequency of visits and content of the program, depending on the needs and preferences of the parent participants, led to greater retention of parents in the program (Olds et al., 2015). Of course, modifications of program protocols should be done in the context of a rigorously researched program of quality improvement to assure that core program elements are not sacrificed in the name of personalizing program content.

Conclusions

In sum, our analyses illustrate the utility of population-specific examinations of associations between socio-demographic risk factors used for recruiting families into HV programs and the outcomes that are the intended targets of these interventions. The DHHS criteria specified that in order for a home visiting program to be considered effective, it had to show evidence for effectiveness for only one or two outcomes. However, in practice, program implementers often operate under the assumption that evidence-based home visiting programs should be made available to families representing a variety of risk profiles and that they will be effective for improving multiple maternal and child outcomes. Our results demonstrated how conducting analyses using a sample from a particular population can help program implementers more systematically target HV programs to the at-risk families most likely to benefit from the interventions.

Funding

Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health (Grant No. T32MH103213). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. This work was also supported by a National Science Foundation Graduate Research Fellowship (Grand No. 1342962).

Conflict of interest

The authors declare that they have no conflicts of interest.

Acknowledgements

We are very grateful to David Olds, Brian D’Onofrio, and an anonymous reviewer for their valuable feedback on this manuscript.

References
Achenbach and Edelbrock, 1981
T.M. Achenbach,C.S. Edelbrock
Behavioral problems and competencies reported by parents of normal and disturbed children aged 4 through 16
Monographs of the Society for Research in Child Development, 46 (1981), pp. 1-82 http://dx.doi.org/10.2307/1165983
Chittleborough et al., 2016
C.R. Chittleborough,A.K. Searle,L.G. Smithers,S. Brinkman,J.W. Lynch
How well can poor child development be predicted from early life characteristics? A whole-of-population data linkage study
Early Childhood Research Quarterly, 35 (2016), pp. 19-30 http://dx.doi.org/10.1016/j.ecresq.2015.10.006
Cohen, 1983
J. Cohen
The cost of dichotomization
Applied Psychological Measurement, 7 (1983), pp. 249-253 http://dx.doi.org/10.1177/014662168300700301
Cohen et al., 2003
J. Cohen,P. Cohen,S.G. West,L.S. Aiken
Applied multiple regression/correlation analysis for the behavioral sciences
3rd ed., Erlbaum, (2003)
Drake et al., 2013
B.F. Drake,L.M. Quintiliani,Y. Li,A.E. Harley,A.L. Sapp,K.M. Emmons,G. Sorensen
Comparing strategies to assess multiple behavior change in behavioral intervention studies
Translational Behavioral Medicine, 3 (2013), pp. 114-121 http://dx.doi.org/10.1007/s13142-013-0195-2
Eckenrode et al., 2001
J. Eckenrode,D. Zielinski,E. Smith,L.A. Marcynyszyn,C.R. Henderson,H. Kitzman,…,D. Olds
Child maltreatment and the early onset of problem behaviors: Can a program of nurse home visitation break the link?
Development and Psychopathology, 13 (2001), pp. 873-890
Evans et al., 2013
G.W. Evans,D.P. Li,S.S. Whipple
Cumulative risk and child development
Psychological Bulletin, 139 (2013), pp. 1342-1396 http://dx.doi.org/10.1037/A0031808
Freemantle et al., 2003
N. Freemantle,J. Wood,J. Eastaugh,M. Calvert
Composite outcomes in randomised trials: Greater precision but with greater uncertainty?
Controlled Clinical Trials, 24 (2003), pp. 98S
Hollingshead, 2011
A. Hollingshead
Four factor index of social status
Yale Journal of Sociology, 8 (2011), pp. 21-51
Ivanova et al., 2007
M.Y. Ivanova,T.M. Achenbach,L. Dumenci,F. Almqvist,S. Weintraub,N. Bilenberg,…,F.C. Verhulst
Testing the 8-syndrome structure of the child behavior checklist in 30 societies
Journal of Clinical Child and Adolescent Psychology, 36 (2007), pp. 405-417 http://dx.doi.org/10.1080/15374410701444363
Kessler, 1995
R.C. Kessler
The national comorbidity survey: Preliminary results and future directions
International Journal of Methods in Psychiatric Research, 5 (1995), pp. 139-151
Lee and Harris, 2009
H. Lee,K.M. Harris
Race, ethnicity and the social context of disadvantage and its links to obesity and physical activity in adolescence and the transition to adulthood
University of North Carolina at Chapel Hill, (2009)
Unpublished doctoral dissertation
Lempers et al., 1989
J.D. Lempers,D. Clarklempers,R.L. Simons
Economic hardship, parenting, and distress in adolescence
Child Development, 60 (1989), pp. 25-39 http://dx.doi.org/10.1111/J.1467-8624.1989.Tb02692.X
Marcynyszyn et al., 2008
L.A. Marcynyszyn,G.W. Evans,J. Eckenrode
Family instability during early and middle adolescence
Journal of Applied Developmental Psychology, 29 (2008), pp. 380-392 http://dx.doi.org/10.1016/J.Appdev.2008.06.001
Myers and Wells, 2003
J.L. Myers,A.D. Wells
Research design and statistical analysis
Erlbaum, (2003)
Olds et al., 2015
D. Olds,P. Baca,M. McClatchey,E.M. Ingoldsby,D.W. Luckey,M.D. Knudtson,…,M. Ramsey
Cluster randomized controlled trial of intervention to increase participant retention and completed home visits in the nurse-family partnership
Prevention Science, 16 (2015), pp. 778-788 http://dx.doi.org/10.1007/s11121-015-0563-x
Olds et al., 1997
D. Olds,J. Eckenrode,C.R. Henderson,H. Kitzman,J. Powers,R. Cole,…,D. Luckey
Long-term effects of home visitation on maternal life course and child abuse and neglect: Fifteen-year follow-up of a randomized trial
JAMA-Journal of the American Medical Association, 278 (1997), pp. 637-643 http://dx.doi.org/10.1001/Jama.278.8.637
Olds et al., 1986
D. Olds,C.R. Henderson,R. Chamberlin,R. Tatelbaum
Preventing child-abuse and neglect: A randomized trial of nurse home visitation
Pediatrics, 78 (1986), pp. 65-78
Olds et al., 1998
D. Olds,C.R. Henderson,R. Cole,J. Eckenrode,H. Kitzman,D. Luckey,…,J. Powers
Long-term effects of nurse home visitation on children's criminal and antisocial behavior: 15-year follow-up of a randomized controlled trial
JAMA-Journal of the American Medical Association, 280 (1998), pp. 1238-1244 http://dx.doi.org/10.1001/Jama.280.14.1238
Olds et al., 1994
D. Olds,C.R. Henderson,H. Kitzman
Does prenatal and infancy nurse home visitation have enduring effects on qualities of parental caregiving and child health at 25 to 50 months of life
Pediatrics, 93 (1994), pp. 89-98
Olds et al., 1988
D. Olds,C.R. Henderson,R. Tatelbaum,R. Chamberlin
Improving the life-course development of socially disadvantaged mothers: A randomized trial of nurse home visitations
American Journal of Public Health, 78 (1988), pp. 1436-1445 http://dx.doi.org/10.2105/Ajph.78.11.1436
Prochaska et al., 2008
J.J. Prochaska,W.F. Velicer,C.R. Nigg,J.O. Prochaska
Methods of quantifying change in multiple risk factor interventions
Preventive Medicine, 46 (2008), pp. 260-265 http://dx.doi.org/10.1016/J.Ypmed.2007.07.035
Rose, 1985
G. Rose
Sick Individuals and sick populations
International Journal of Epidemiology, 14 (1985), pp. 32-38 http://dx.doi.org/10.1093/ije/14.1.32
Royston et al., 2006
P. Royston,D.G. Altman,W. Sauerbrei
Dichotomizing continuous predictors in multiple regression: A bad idea
Statistics in Medicine, 25 (2006), pp. 127-141 http://dx.doi.org/10.1002/sim.2331
Teixeira-Pinto et al., 2009
A. Teixeira-Pinto,J. Siddique,R. Gibbons,S.L. Normand
Statistical approaches to modeling multiple outcomes in psychiatric studies
Psychiatric Annals, 39 (2009), pp. 729-735 http://dx.doi.org/10.3928/00485713-20090625-08
Tyler et al., 2011
K.M. Tyler,S.L.T. Normand,N.J. Horton
The use and abuse of multiple outcomes in randomized controlled depression trials
Contemporary Clinical Trials, 32 (2011), pp. 299-304 http://dx.doi.org/10.1016/J.Cct.2010.12.007
US Department of Health and Human Services, 2013
US Department of Health and Human Services
Home visiting evidence of effectiveness review: Executive summary
(2013)
Retrieved from http://homvee.acf.hhs.gov
Veit and Ware, 1983
C.T. Veit,J.E. Ware
The structure of psychological distress and well-being in general populations
Journal of Consulting and Clinical Psychology, 51 (1983), pp. 730-742 http://dx.doi.org/10.1037/0022-006x.51.5.730
Yoon et al., 2011
F.B. Yoon,G.M. Fitzmaurice,S.R. Lipsitz,N.J. Horton,N.M. Laird,S.L.T. Normand
Alternative methods for testing treatment effects on the basis of multiple outcomes: Simulation and case study
Statistics in Medicine, 30 (2011), pp. 1917-1932 http://dx.doi.org/10.1002/Sim.4262

( ! ) Warning: Invalid argument supplied for foreach() in /var/www/html/includes_ws/librerias/html/item.php on line 1203
Call Stack
#TimeMemoryFunctionLocation
10.0003243808{main}( ).../index.php:0
20.42343798744include( '/var/www/html/includes_ws/plantilla/template.php' ).../index.php:54
30.42753834224include( '/var/www/html/psyint/contenidos/item.php' ).../template.php:20
40.46383954808getAutorCorrespondenciaHTML( ).../item.php:168
Copyright © 2017. Colegio Oficial de Psicólogos de Madrid
Psychosocial Intervention 26 (2017) 117-24 - Vol. 26 No.2 DOI: 10.1016/j.psi.2017.01.001