Can You Use Pre- Post- in Multiple Regression

J Biom Biostat. Author manuscript; available in PMC 2022 Dec 12.

Published in final edited form every bit:

PMCID: PMC6290914

NIHMSID: NIHMS992909

Methods for Analysis of Pre-Post Data in Clinical Enquiry: A Comparison of Five Common Methods

Abstract

Ofttimes repeated measures data are summarized into pre-postal service-treatment measurements. Various methods exist in the literature for estimating and testing treatment effect, including ANOVA, analysis of covariance (ANCOVA), and linear mixed modeling (LMM). Under the beginning two methods, outcomes tin can either be modeled as the post treatment measurement (ANOVA-Mail service or ANCOVA-Postal service), or a alter score betwixt pre and mail measurements (ANOVA-Change, ANCOVA-CHANGE). In LMM, the upshot is modeled as a vector of responses with or without Kenward-Rogers adjustment. We consider five methods mutual in the literature, and hash out them in terms of supporting simulations and theoretical derivations of variance. Consistent with existing literature, our results demonstrate that each method leads to unbiased treatment effect estimates, and based on precision of estimates, 95% coverage probability, and power, ANCOVA modeling of either change scores or postal service-treatment score equally the upshot, show to be the near effective. We farther demonstrate each method in terms of a real data case to exemplify comparisons in real clinical context.

Keywords: Analysis of variance, Analysis of covariance, Linear mixed model, Pre-postal service, Rrandomized trial, Repeated measures

Introduction

In clinical research, it is common to record repeated measurements for subject area responses across multiple occasions. In that location exist a variety of assay methods, including repeated measures analysis of variance (RANOVA), multivariate ANOVA (MANOVA) and linear mixed modeling (LMM). Researchers often simplify repeated measures information past using summary data to quantify pre-and mail service-handling outcomes, allowing for a more intuitive and easier interpretation of handling comparisons [1]. When accordingly applied, simplifying repeated mensurate outcomes to only two fourth dimension measurements in the blueprint phase improves efficiency and toll effectiveness, particularly in circumstances when responses are expensive to measure. In many instances this is washed because the response of patients at a certain final fourth dimension point is more clinically relevant than trends over time. Rather than comparing trends over time within each handling group, the pre-post treatment summary method also simplifies data analysis to standard t-test procedures.

Decades of literature exists exploring and comparing methods for pre-post analysis, in both theory and application. The goal of this paper is not in developing new methods of analysis, simply to review and succinctly tie together existing literature into a cohesive comparison of common methods frequently discussed and employed. We revisit and review the basic methods of pre-post information analysis discussed in the literature, and then exemplify the results through simulation and existent data examples to corroborate existing cognition. The residue of the paper is structured as follows: we showtime provide a review of key literature in pre-mail analysis. We then outline the models to exist compared and set upwardly a simple simulation study to demonstrate the comparing of methods, and discuss simulated results in conjunction with the theoretical expectations of variance and related implicit measures. A existent data example is used to exemplify the difference in methods in practice, and highlight the importance of a proper assay method. We finish with a discussion of results and farther present ideas for future avenues of research in the area of pre-post data analysis.

Review of Literature

Frison and Pocock [two] discuss three methods for analyzing data from pre-post designs: a) ANOVA with the post measurement as the response variable (ANOVA-Postal service), b) ANOVA with the modify from pre-treatment to mail service-treatment every bit the response variable (ANOVA-Alter), and c) ANCOVA with the postal service measurement as the response variable (ANCOVA-POST), adjusting for the pre-treatment measurement. Brogan and Kutner [3] compare the use of ANOVA-CHANGE with RANOVA. However, Huck and McLean [4] criticize the latter method due to its frequent misinterpretation in practice. Furthermore, they note the F-test in an RANOVA interaction is equivalent to the F-exam in change score analysis. RANOVA provides the same conclusion as ANOVA-CHANGE, but use of ANOVA-CHANGE is simpler and more accurately interpreted compared to RANOVA. These conclusions are defended by Jennings [5], who asserts RANOVA is not recommended for pre-mail analysis given the simpler alternatives presented.

Among the methods, ANCOVA-POST is by and large regarded as the preferred approach, given that it typically leads to unbiased treatment effect estimate with the everyman variance relative to ANOVA-POST or ANOVA-CHANGE [one-6], All the same, ANCOVA has been criticized as being biased in the case of diff pre-treatment mean measurements between groups [7,8]. This conundrum, known equally "Lord's Paradox," was first documented in 1967 Lord [9], and has been discussed in the literature extensively. Amidst a detailed examination of various methods of repeated measures data assay for pre-mail outcomes, Liang and Zeger [ten] note in the simple case with only ii responses (i.east. pre- and post-handling measurements), ANCOVA-Mail service produces an unbiased estimate only in the instance of equal pretreatment measurements, whereas ANOVA-Alter leads to unbiased estimates that are only slightly less efficient than ANCOVA-POST. Senn [xi] discusses these criticisms at length, providing diverse weather for which these claims do not concur, ultimately concluding ANCOVA should be used with circumspection in the example of unequal pretreatment measurements, only ANOVA-CHANGE is non impervious to bias either.

A recent simulation written report by Egbewale, Lewis, and Sim [12] over varying degrees of pre-handling imbalance and pre-mail treatment correlations, demonstrates that a comparison of the methods is not straightforward in the presence of diff pre-treatment measures between groups. They recommend ANCOVA when pre-handling measurements are equal in expectation across groups, as should be the instance in properly designed randomized trials [12]. ANOVA-Mail service has a larger variance because it allows for possible random baseline imbalance for which it cannot adjust. ANCOVA allows adjustment for baseline differences and thus has a smaller variance than ANOVA. In farther support for ANCOVA, Vickers and Altman [xiii] note that ANCOVA achieves the greatest power relative to ANOVA-Alter or ANOVA-Postal service, but the power of ANOVA-CHANGE approaches ANCOVA as correlation between pre-post measures approaches one.

Combining analysis of change scores with adjustments for pre-treatment measures, Laird [fourteen] offers a slight modification to ANCOVA, in which the change score is incorporated as the outcome and pre-treatment measures as covariate. Compared with traditional ANCOVA, this ANCOVA-CHANGE leads to equal results in terms of variance of treatment event, although Laird [14] asserts the latter method allows one to assess whether change occurred in individual handling groups. Despite this possible reward, this appears less frequently used or discussed in the literature.

In the rest of the paper, we will discuss and compare results between the five common methods ANOVA and ANCOVA modeling both the postal service-treatment response only and the change score, along with a linear mixed model (LMM) modeling the pre-post treatment response vector, Y_ij . We use simulations over a range of sample sizes and pre-post measurement correlations to corroborate the comparison of methods with the existing literature and theoretical expectations of variance.

Methods

To set up the modeling framework, let Y_i exist the continuous response variable from a randomized trial, for i=1,…,n patient responses from samples due north ₁ and n ₂ from each handling grouping. Let the group assignment be designated past the indicator, X_i, such that for the i ^thursday patient, X_i =1 for the active treatment and X_i =0 in the control/placebo group. Responses for each treatment are each sampled from a Gaussian distribution with mean μ _x and variance σ², where μ _x=β ₀+β ₁ X_i . To distinguish between post-treatment and change score measures as outcomes, permit $Y_{i}^{[p]}$ correspond the post handling response and $Y_{i}^{[c]}$ represent the change from pre-handling to mail service-treatment measurements. In the case of ANCOVA, let Y_0i be the pre-handling measurement for which the model is adjusted for. Lastly, let ε_i stand for the random error terms for each of the models. Maximum likelihood is used to gauge the parameters corresponding to each model except for the variance in LMM which are estimated using restricted maximum likelihood (REML). Derivations of variance for each of the estimators from the different methods are given in the Appendix.

Method ane: ANOVA-POST

Method 1 uses linear regression to compare treatment effects. Formally, the model is as follows:

$Y_{i}^{[p]} = β_{0}^{[p]} + β_{1}^{[p]} X_{i} + ε_{i}^{[p]} \cdot$

It is assumed that ε_i are independently and identically normally distributed with mean 0 and variance σ². $β_{ane}^{[p]}$ is interpreted equally the difference in the post-handling mean between treatment groups. The variance of the estimated treatment effect is given by:

Method 2: ANOVA-CHANGE

Similar to ANOVA-Mail, ANOVA-CHANGE employs a uncomplicated ANOVA framework, but instead models the outcome, $Y_{i}^{[c]}$ without adjustment for pre-treatment values. Formally, the model with ε_i assumed to exist independently and identically unremarkably distributed with mean 0 and variance σⁱⁱ is given every bit follows:

$Y_{i}^{[c]} = β_{0}^{[c 1]} + β_{1}^{[c ane]} X_{i} + ε_{i}^{[c one]} \cdot$

Here $β_{1}^{[c 1]}$ is interpreted as the divergence in the modify score mean of the treatment groups. Nether an unstructured covariance structure assumption, the variance of ${\hat{β}}_{i}^{[c 1]}$ is given past:

$five a r ({\hat{β}}_{1}^{[c 1]}) = (σ_{p r e}^{2} + σ_{p o due south t}^{2} - ii ρ σ_{p r due east} σ_{p o s t}) (\frac{1}{{northward}_{1}} + \frac{ane}{{north}_{ii}}) .$

Under a compound symmetry assumption, where pre- and post-treatment variance is assumed to exist equal, the variance is given by:

$five a r ({\hat{β}}_{1}^{[c ane]}) = 2 (i - ρ) (\frac{1}{n_{i}} + \frac{1}{{north}_{2}}) σ^{2} .$

Method 3: ANCOVA-CHANGE

Method 3 employs an ANCOVA model to analyze the change score as an outcome, adjusting for the pre-treatment values. Essentially, ANCOVA-Alter is equivalent to ANOVA-Alter, with an added aligning for the pre-treatment measurement for every patient. Formally, the model is as follows:

$Y_{i}^{[c]} = β_{0}^{[c ii]} + β_{i}^{[c 2]} X_{i} + β_{2}^{[c 2]} Y_{0 i} + ε_{i}^{[c 2]} \cdot$

It is assumed that ε_i are independently and identically normally distributed with hateful 0 and variance σⁱⁱ. $β_{1}^{[c 2]}$ is interpreted as the divergence in the modify score mean of the treatment groups, given the pre-treatment measurement and the variance of its estimator is given by

$var ({\bar{Y}}_{T .}^{cov} - {\bar{Y}}_{P .}^{cov}) = \frac{n_{1} + n_{2} - ii}{n_{i} + n_{2} - iii} σ_{p o s t}^{2} (1 - ρ^{2}) [\frac{i}{{northward}_{1}} + \frac{1}{{due north}_{2}} + \frac{({\bar{Y}}_{T 0 .} - {\bar{Y}}_{P 0 .})^{2}}{({northward}_{1} + n_{2} - 2) σ_{p r e}^{2}}]$

Which, as the sample size increases, simplifies to:

$five a r (\overset{[]}{β}) = (1 - ρ^{2}) (\frac{1}{{due north}_{i}} + \frac{1}{n_{2}}) σ_{p o southward t}^{ii} \cdot$

Method 4: ANCOVA-POST

Method iv employs an ANCOVA model to clarify the postal service-treatment measurements every bit the outcome, adjusting for the pre-treatment values. In the context of previous methods, ANCOVA-POST is essentially ANOVA-Mail (method 1) with pre-treatment measurement included as a covariate. Formally, the model is as follows:

$Y_{i} = β_{0}^{[a]} + β_{1}^{[a]} 10_{i} + β_{ii}^{[a]} Y_{0 i} + ε_{i}^{[a]} \cdot$

It is assumed that ε_i are independently and identically normally distributed with hateful 0 and variance σ². $β_{1}^{[a]}$ is interpreted equally the difference in the mail handling score mean of the handling groups, given the pre-treatment measurement. Since this method is equivalent to method 3 Laird [14], results including the variance of the estimated handling outcome $β_{1}^{[a]}$ is the same.

Method 5: LMM

Method 5 consists of employing a linear mixed model (LMM) to clarify a vector of the pre-and post-measurements equally the outcome. Y_ij denotes the j^th measure of the i^th subject. Formally, the model is equally follows:

$Y_{i j} = β_{0}^{[b]} + β_{ane}^{[b]} X_{i} + β_{2}^{[b]} t_{i j} + β_{three}^{[b]} t_{i j} X_{i} + ε_{i j}^{[b]},$

Where t_ij is an indicator for pre-treatment measurement (coded 0) or post-handling measurement (coded ane). In LMM, information technology is assumed that ε_ij are bivariate normally distributed with means 0 and heterogeneous compound symmetric (HCS) covariance matrix and correlation, ρ. The term $β_{one}^{[b]} + β_{3}^{[b]}$ is interpreted every bit the mean difference between handling groups post-treatment, and $β_{1}^{[b]}$ is the mean difference betwixt handling groups pre-treatment. LMM allows for pre-treatment mean differences between the groups.

The variance of ${\hat{β}}_{1} + {\hat{β}}_{3}$ under HCS covariance matrix for the fault term is given by:

$Var ({\hat{β}}_{ane} + {\hat{β}}_{3}) = Var ({\hat{β}}_{1}) = (\frac{1}{n_{1}} + \frac{ane}{n_{2}}) {\hat{σ}}_{p o south t}^{2} = (\frac{one}{n_{1}} + \frac{ane}{n_{2}}) σ^{2} .$

If assuming compound symmetry (CS) is causeless, the variance is:

$Var ({\hat{β}}_{i} + {\hat{β}}_{3}) = (1 - \frac{1}{2} ρ) (\frac{1}{{due north}_{ane}} + \frac{1}{n_{2}}) σ^{2}$

The LMM was evaluated nether REML interpretation in PROC MIXED (SAS ix.3, SAS Institute Inc, Cary, NC). Acknowledging that many caste of freedom adjustments may exist employed in mixed models, nosotros choose to evaluate this approach with the bourgeois, and widely used Kenward and Rogers (KR) adjustment, as well every bit an unadjusted model. The KR adjustment Kenward and Roger [15]; Schaalje, McBride and Fellingham [16]; Senn [17] appropriately inflates the variance-covariance matrix, along with an adjusted degree of liberty judge (KR degree of freedom adjustment) when making inference on fixed effects which rely on asymptotic distributions that can lead to biased variance estimates when sample sizes are small. No adjustment, according to Senn [17], leads to negligible difference with the first scenario as sample size grows reasonably large (due east.g. n>40).

Simulation Study

Information are false using SAS nine.3. Simulations are designed to stand for a variety of situations which are plausible in pre-post studies. Using 1000 repetitions, we compare models under three sample sizes (n=fifty, 100, and 200), under three pre-postal service correlations (ρ=0.1, 0.5, and 0.8), and six β _i coefficients for treatment effect (β ₁=−ane.5, −0.1, −1.0, 0.one, 1.0, and one.5). Covariates are generated bold 10~uniform (0,1), and Y ₀ ~N(0,1); the post handling response Y _i is generated using: Y _one=10+1.5*{10 ≥ 0.5} +1.5* Y _g +ε, such that {Ten ≥ 0.5} represents treatment 1 (i.east. Ten_i =1) and {Ten<0.5} represents control/placebo (i.due east. Ten_i =0). To generate correlation between pre- (Y₀ ) and post- (Y ₁) treatment measures, we apply the human relationship between correlation and gradient: $ρ = β \frac{σ_{y 0}}{σ_{y 1}}$ where σ_y0 and σ _y1 are the standard deviations for pre- and postal service-treatment responses, respectively. β is fixed at i.5 and σ _y1, is calculated for each combination of σ _y0 and ρ. Random errors are generated such that ε ~North(0 σ² ). The corresponding residue variance is calculated using the relationship between σ _y1 and the variance of ε for unlike β ₁ coefficients: $σ_{ε}^{2} = σ_{y 1}^{2} - β_{1}^{2} \times 0.25 - 2.25 \times σ_{y 0}^{2}$ . There are a full of 108 faux scenarios among the combinations of n, ρ, and β _one. Estimates for the parameter ( $\hat{β 1}$ ), its variance, bias, power, and nominal 95% coverage probability are computed for each simulation scenario, and the results are compared beyond the five methods. Code used to implement this may be plant in supplementary material bachelor online.

Simulation Results

In this department, findings from the simulations are discussed and compared to expected theoretical results. Treatment result parameter estimates and associated standard deviations are reported in Table ane, while bias and power are presented in Tables 2 and 3 shows the 95% coverage probability. The results, consistent with theoretical expectations, show that all methods produce equally unbiased estimates of the handling effect beyond equivalent combinations of ρ, β _one, and n, with accuracy of the estimates improving with sample size. In full general, as the number of observations increases, the bias converges to zero for each of the methods across all of the fake scenarios. When ρ increases, the bias decreases, regardless of sample size or true _β ₁.

Tabular array 1:

Mean Parameter and standard deviation estimates across 1000 simulations for positive β ₁ values.

Simulated β _one Values across methods		ρ=0.1			ρ=0.five			ρ=0.8
Simulated β _one Values across methods		northward=l	northward=100	northward=200	n=50	n=100	northward=200	n=50	north=100	due north=200
0.1	ANOVA-POST	0.095(iv.240)	0.013(3.065)	0.206(2.121)	0.114(0.856)	0.117(0.601)	0.110(0.447)	0.110(0.532)	0.096(0.386)	0.102(0.268)
	ANOVA-CHANGE	0.095(iv.220)	0.018(iii.055)	0.198(2.115)	0.107(0.768)	0.110(0.537)	0.105(0.396)	0.104(0.343)	0.096(0.246)	0.101(0.177)
	ANCOVA-CHANGE	0.110(four.261)	0.022(3.075)	0.201(2.119)	0.106(0.767)	0.106(0.535)	0.102(0.387)	0.101(0.312)	0.095(0.222)	0.100(0.159)
	ANCOVA-POST	0.110(4.261)	0.022(3.075)	0.201(2.119)	0.106(0.767)	0.106(0.535)	0.102(0.387)	0.101(0.312)	0.095(0.222)	0.100(0.159)
	LMM	0.095(4.240)	0.013(three.065)	0.206(2.121)	0.114(0.856)	0.117(0.601)	0.110(0.447)	0.110(0.532)	0.096(0.386)	0.102(0.268)
1.0	ANOVA-Mail service	ane.104(iv.424)	0.997(3.197)	0.912(two.075)	ane.038(0.833)	i.007(0.598)	1.006(0.424)	1.028(0.504)	0.992(0.351)	1.006(0.256)
	ANOVA-Alter	1.105(4.411)	1.008(three.178)	0.917(ii.066)	i.031(0.723)	ane.000(0.525)	1.005(0.379)	1.010(0.311)	0.988(0.219)	1.002(0.159)
	ANCOVA-CHANGE	1.098(4.448)	1.022(3.188)	0.913(2.059)	ane.025(0.712)	0.997(0.519)	1.004(0.373)	0.999(0.278)	0.986(0.199)	0.999(0.143)
	ANCOVA-Mail service	ane.098(iv.448)	1.022(3.188)	0.913(2.059)	1.025(0.712)	0.997(0.519)	1.004(0.373)	0.999(0.278)	0.986(0.199)	0.999(0.143)
	LMM	1.104(four.424)	0.997(3.197)	0.912(2.075)	1.038(0.833)	1.007(0.598)	1.006(0.424)	i.028(0.504)	0.992(0.351)	1.006(0.256)
1.v	ANOVA-Postal service	ane.557(iv.295)	1.668(3.048)	ane.577(two.094)	1.525(0.831)	1.520(0.578)	1.522(0.412)	1.503(0.504)	ane.492(0.347)	1.487(0.242)
	ANOVA-Change	i.567(iv.277)	one.650(3.039)	1.579(2.083)	i.526(0.724)	1.518(0.508)	1.526(0.361)	1.502(0.290)	one.494(0.199)	1.489(0.140)
	ANCOVA-Change	1.556(four.313)	1.638(three.057)	one.586(2.086)	1.528(0.718)	one.516(0.502)	1.530(0.354)	1.500(0.250)	ane.496(0.170)	1.491(0.119)
	ANCOVA-Post	one.556(4.313)	i.638(3.057)	1.586(two.086)	1.528(0.718)	1.516(0.502)	1.530(0.354)	1.500(0.250)	1.496(0.170)	1.491(0.119)
	LMM	1.557(four.295)	1.668(three.048)	i.577(ii.094)	ane.525(0.831)	1.520(0.578)	1.522(0.412)	1.503(0.504)	one.492(0.347)	one.487(0.242)

Table 2:

Power and bias estimates from 1000 simulations for true values of β ₁.

False β _ane Values across methods		ρ=0.1			ρ=0.five			ρ=0.viii
False β _ane Values across methods		n=fifty	north=100	n=200	n=50	n=100	n=200	due north=50	n=100	n=200
0.one	ANOVA-Mail	−0.005	−0.087	0.106	0.014	0.017	0.010	0.010	−0.004	0.002
	ANOVA-Mail	0.056	0.062	0.049	0.053	0.060	0.070	0.056	0.060	0.069
	ANOVA-Change	−0.005	−0.082	0.098	0.007	0.010	0.005	0.004	−0.004	0.001
	ANOVA-Change	0.056	0.061	0.049	0.051	0.061	0.066	0.058	0.064	0.091
	ANCOVA-Modify	0.010	−0.078	0.101	0.006	0.006	0.002	0.001	−0.005	0.000
	ANCOVA-Modify	0.057	0.062	0.045	0.054	0.060	0.060	0.071	0.068	0.107
	ANCOVA-Mail service	0.010	−0.078	0.101	0.006	0.006	0.002	0.001	−0.005	0.000
	ANCOVA-Mail service	0.057	0.062	0.045	0.054	0.060	0.060	0.071	0.068	0.107
	LMM	−0.005	−0.087	0.106	0.014	0.017	0.010	0.010	−0.004	0.002
	LMM	0.056	0.062	0.049	0.053	0.060	0.070	0.056	0.060	0.069
one.0	ANOVA-POST	0.104	−0.003	−0.088	0.038	0.007	0.006	0.028	−0.008	0.006
	ANOVA-POST	0.058	0.067	0.088	0.211	0.382	0.644	0.490	0.794	0.976
	ANOVA-Modify	0.105	0.008	−0.083	0.031	0.000	0.005	0.010	−0.012	0.002
	ANOVA-Modify	0.059	0.066	0.086	0.267	0.470	0.756	0.856	0.988	1.000
	ANCOVA-Alter	0.098	0.022	−0.087	0.025	−0.003	0.004	−0.001	−0.014	−0.001
	ANCOVA-Alter	0.060	0.065	0.085	0.275	0.479	0.772	0.920	0.999	1.000
	ANCOVA-Postal service	0.098	0.022	−0.087	0.025	−0.003	0.004	−0.001	−0.014	−0.001
	ANCOVA-Postal service	0.060	0.065	0.085	0.275	0.479	0.772	0.920	0.999	1.000
	LMM	0.104	−0.003	−0.088	0.038	0.007	0.006	0.028	−0.008	0.006
	LMM	0.058	0.067	0.088	0.211	0.382	0.644	0.490	0.794	0.976
1.v	ANOVA-POST	0.057	0.168	0.077	0.025	0.020	0.022	0.003	−0.008	−0.013
	ANOVA-POST	0.058	0.094	0.102	0.412	0.713	0.949	0.848	0.987	i.000
	ANOVA-Change	0.067	0.150	0.079	0.026	0.018	0.026	0.002	−0.006	−0.011
	ANOVA-Change	0.058	0.094	0.105	0.522	0.825	0.985	ane.000	i.000	one.000
	ANCOVA-Modify	0.056	0.138	0.086	0.028	0.016	0.030	0.000	−0.004	−0.009
	ANCOVA-Modify	0.057	0.088	0.107	0.527	0.836	0.987	one.000	1.000	1.000
	ANCOVA-POST	0.056	0.138	0.086	0.028	0.016	0.030	0.000	−0.004	−0.009
	ANCOVA-POST	0.057	0.088	0.107	0.527	0.836	0.987	1.000	1.000	one.000
	LMM	0.057	0.168	0.077	0.025	0.020	0.022	0.003	−0.008	−0.013
	LMM	0.058	0.094	0.102	0.412	0.713	0.949	0.848	0.987	1.000

Table three:

95% confidence interval coverage probabilities from 1000 simulations.

Simulated β _one Values across methods		ρ=0.ane			ρ=0.5			ρ=0.8
Simulated β _one Values across methods		n=50	n=100	n=200	n=50	n=100	north=200	n=50	n=100	north=200
0.1	ANOVA-Mail	0.939	0.949	0.952	0.957	0.960	0.939	0.959	0.943	0.953
	ANOVA-Change	0.936	0.950	0.950	0.949	0.947	0.946	0.940	0.936	0.956
	ANCOVA-Modify	0.935	0.950	0.949	0.947	0.950	0.952	0.944	0.942	0.948
	ANCOVA-Post	0.935	0.950	0.949	0.947	0.950	0.952	0.944	0.942	0.948
	LMM	0.939	0.949	0.952	0.957	0.960	0.939	0.959	0.943	0.953
1.0	ANOVA-Mail	0.937	0.951	0.945	0.944	0.951	0.955	0.954	0.941	0.960
	ANOVA-Modify	0.941	0.952	0.947	0.950	0.954	0.951	0.962	0.952	0.959
	ANCOVA-CHANGE	0.945	0.952	0.947	0.943	0.950	0.951	0.955	0.947	0.956
	ANCOVA-POST	0.945	0.952	0.947	0.943	0.950	0.951	0.955	0.947	0.956
	LMM	0.937	0.951	0.945	0.944	0.951	0.955	0.954	0.941	0.96
1.5	ANOVA-Post	0.954	0.947	0.951	0.957	0.957	0.951	0.964	0.951	0.948
	ANOVA-CHANGE	0.953	0.948	0.952	0.954	0.953	0.954	0.961	0.952	0.950
	ANCOVA-Change	0.950	0.942	0.953	0.953	0.947	0.949	0.960	0.954	0.952
	ANCOVA-POST	0.950	0.942	0.953	0.953	0.947	0.949	0.960	0.954	0.952
	LMM	0.954	0.947	0.951	0.957	0.957	0.951	0.964	0.951	0.948
−0.1	ANOVA-Mail	0.948	0.954	0.959	0.952	0.961	0.952	0.964	0.949	0.944
	ANOVA-Alter	0.947	0.953	0.962	0.951	0.961	0.950	0.965	0.952	0.945
	ANCOVA-Alter	0.950	0.952	0.959	0.953	0.954	0.950	0.959	0.955	0.944
	ANCOVA-Mail	0.950	0.952	0.959	0.953	0.954	0.950	0.959	0.955	0.944
	LMM	0.948	0.954	0.959	0.952	0.961	0.952	0.964	0.949	0.944
−1.0	ANOVA-Postal service	0.953	0.954	0.957	0.955	0.959	0.953	0.957	0.959	0.955
	ANOVA-CHANGE	0.953	0.953	0.956	0.955	0.959	0.948	0.957	0.948	0.955
	ANCOVA-CHANGE	0.954	0.951	0.956	0.952	0.948	0.95	0.962	0.950	0.949
	ANCOVA-POST	0.954	0.951	0.956	0.952	0.948	0.95	0.962	0.950	0.949
	LMM	0.953	0.954	0.957	0.955	0.959	0.953	0.957	0.959	0.955
−i.v	ANOVA-POST	0.958	0.948	0.946	0.956	0.950	0.950	0.954	0.954	0.948
	ANOVA-CHANGE	0.958	0.947	0.944	0.957	0.949	0.952	0.957	0.953	0.950
	ANCOVA-Alter	0.958	0.951	0.942	0.96	0.949	0.947	0.955	0.950	0.951
	ANCOVA-Postal service	0.958	0.951	0.942	0.96	0.949	0.947	0.955	0.950	0.951
	LMM	0.958	0.948	0.946	0.956	0.950	0.950	0.954	0.954	0.948

Since all methods result in unbiased estimates for the treatment effect, we utilise variance and other implicit measures of the estimates to compare the five methods (Table one and Figure 1). In full general, when the correlation between pre-treatment and mail-treatment values is high, the variance of the estimates are relatively pocket-size, regardless of method and the value of β _i. However, for higher correlation values, differences in variability betwixt methods become more than credible when the sample size and value of β _i are fixed. For case, in the scenario when β ₁=0.1 and north=200, the difference between the largest and smallest standard deviations among the 5 methods with depression correlation (ρ=0.ane), is 0.006. However, under the same scenario with loftier correlation (ρ=0.8), the difference between the largest and smallest standard deviations is 0.109. While the individual standard deviation estimates are greater in low correlation scenarios, the difference between estimates of differing methods are much more pronounced in scenarios with high correlation, i.e. the greatest variability in β _one occurrs when ρ is shut to 0 (i.e. ρ=0.i). Furthermore, as sample size increases, variability decreases as expected theoretically. To summarize in terms of correlation and sample size, variability of β ₁ estimates are greatest when northward=50 and ρ=0.1, and they are lowest when due north=200 and ρ=0.8.

An external file that holds a picture, illustration, etc. Object name is nihms-992909-f0001.jpg

Distribution of treatment effects estimates varied by correlation, sample size and truthful positive β _ane values under Y₀~N(0,1).

Boxplots for parameter estimates for the 1000 simulations for the combinations of β1, n, and ρ are displayed in Effigy 1 Consistent with the data tables, all parameter estimates are unbiased, and the boxplots highlight differences in variability for the models. In general, variance was much larger for small values of ρ and small north. ANCOVA models have smaller variances compared to ANOVA and LMM, though differences are quite small-scale.

Comparing the two ANOVA methods specifically, ANOVA-Alter produces approximately equal or less variability compared to ANOVA-POST, the difference of which increases as ρ approaches one. Intuitively, information technology follows that ignoring pre-treatment observations in ANOVA-POST causes a loss of information which leads to an increment in variance estimates when pre-handling and post-treatment values are correlated. Yet, when correlation is low, results are dependent the on pre-post covariance structure. These discrepancies are addressed and explained in detail in post-obit paragraphs. Table 1 displays the difference between variance estimates which grows with increasing correlation, property sample size and true β _ane parameters abiding.

The two scenarios of ANCOVA (ANCOVA-POST and ANCOVA-Alter) give identical measures of variability, regardless of differing combinations of ρ, β _ane and north. These results can be seen numerically in Tabular array 1 and visually assessed in Effigy 1, and are further supported by theoretical derivation of variance. Lastly, the LMM analysis results are equivalent to the ANOVA-Mail in terms of estimates and their standard deviations, regardless of the causeless within-subject covariance structure, R_i . The LMM arroyo is evaluated with and without a KR aligning, ultimately showing no divergence. These results are consistent with the literature, given the smallest simulated sample size was n=50.

Across all methods, ANCOVA models consistently performed best compared to the other methods, regardless of ρ, β ₁ and north, as has been demonstrated in existing literature [xviii]. The ANCOVA methods (ANCOVA-CHANGE and ANCOVA-Post) are compared with ANOVA-POST, ANOVA-CHANGE and LMM in terms of variance of the estimate of β ₁ (Figure i). Except at the everyman combination of ρ and n values where discrepancies between differing method variances is negligible, ANCOVA had the lowest variability. Similarly, the 95% coverage intervals are smallest under ANCOVA, and widest in ANOVA-POST (Table 4). Compared to ANOVA-Change, ANCOVA models have less variation. These results are also like when ANCOVA is compared with LMM, except when the correlation is close to zero.

Table four:

Estimates of toothbrush effect on bacterial plaque index in 1^st and 4^th sessions.

Method	$\hat{β}$	95% Conviction Interval		SE( $\hat{β}$ )	P-value
ane^st Session
ANOVA-POST	0.sixteen	−0.09	0.41	0.123	0.192
ANOVA-CHANGE	0.xiv	0.05	0.24	0.047	0.005
ANCOVA-CHANGE	0.14	0.05	0.24	0.046	0.004
ANCOVA-POST	0.14	0.05	0.24	0.046	0.004
LMM	0.16	−0.09	0.41	0.123	0.191
4^th Session
ANOVA-Mail	−0.08	−0.28	0.12	0.099	0.409
ANOVA-CHANGE	0.09	−0.02	0.21	0.056	0.107
ANCOVA-Modify	0.07	−0.05	0.19	0.059	0.243
ANCOVA-Mail	0.07	−0.05	0.19	0.059	0.243
LMM	−0.08	−0.28	0.12	0.099	0.409

It is of interest to compare the simulations to theoretical expectations for the variance of the treatment effect estimates. As the correlation between pretreatment and mail-treatment observations approaches zero, the variance for ANOVA-POST, ANCOVA-Post, ANCOVA-Change, and LMM should be approximately equivalent, just the variance for ANOVA-CHANGE is two times that of the others. When ρ=0.1 and assuming equal variance, the variance of ANOVA-Change should theoretically approach 2(one-ρ=0.1), or 1.viii times that of ANOVA-Postal service, and ANCOVA methods should approach (1-ρ ⁱⁱ), or 99 of ANOVA-Mail. Similarly, for ρ = 0.i, the ratio of ANOVA-Mail service to ANOVA-CHANGE should exist i and ANOVA-POST to ANCOVA should be 0.75. Finally, at ρ=0.8, ANOVA-Alter and both ANCOVA models should exist at 0.4 and 0.36 times ANOVA-Post respectively. However, permitting pre and post measurements to have dissimilar variances leads to differing results, particularly apparent when correlation is depression, such that the variance of ANOVA-CHANGE approaches the variance of ANOVA-POST as ρ approaches zero and the variance of mail-measures becomes increasingly greater than premeasures. Presented simulations are for information simulated assuming this unstructured covariance matrix. For example, the variance of ANOVA-Modify and ANOVA-Mail service are approximately equal when ρ=0.1; had we assumed equalvariance, ANOVA-Change would be almost twice that of ANOVA-Postal service. These are made apparent given the following ratio of variances:

As a result, the variance of ANOVA-CHANGE approaches the variance of ANOVA-POST as ρ approaches zero and β ₁>0.v Nether the given method of simulating pre and post treatment variances where σ_pre = i, β ₁ = one.5, and σ _mail=β _one σ _pre/ρ, the ratio of the variances is given past:

$\frac{(σ_{p r east}^{ii} + σ_{p o s t}^{two} - 2 ρ σ_{p r e} σ_{p o s t}) (\frac{1}{{north}_{1}} + \frac{ane}{n_{2}})}{(\frac{one}{n_{1}} + \frac{1}{n_{ii}}) σ_{p o s t}^{ii}} \approx 0.99, for ρ = 0.i \cdot$

Since the HCS structure is more conservative in its assumptions (permitting the pre- and post-treatment effects to have different variances), it is used to report the main simulation results of the study. Equally correlation increases beyond 0.5, results become less sensitive to the pre-mail mensurate covariance structure.

Power for testing β ₁ = 0 was assessed for the methods nether the simulated conditions (Tabular array two). Nether the chief simulation method assuming Y ₀ ~N(0,ane), power across methods did not vary by a big degree. Observing power beyond simulated scenarios at the lowest sample size (due north=fifty), there is marginally higher power in ANCOVA and ANOVA-Change methods over ANOVA-POST and LMM. Overall, ANCOVA methods achieve the greatest ability, intuitively so given ANCOVA leads to the lowest variability. As correlation between pre-and post-measurements increment, the difference in power between ANCOVA and ANOVA-CHANGE compared to ANOVA-POST and LMM, grows appreciably, while ANOVA-CHANGE nears that of ANCOVA as correlation approaches one. Finally, increases in sample size leads to increased power for detecting a significant treatment effect similarly across methods, pregnant that an increase in sample size does not appear to touch on whatever single methods statistical power more than other methods. Boosted results from simulations with an increased variance (assuming Y ₀ ~Northward(0,9)), are reported.

Data Example

To illustrate the awarding of the five methods, we consider data from a dental hygiene study characterized by modest sample size [nineteen]. There were a total of 32 subjects, randomized to two treatment groups based on blazon of toothbrush, and effectiveness was measured by reduction in bacterial plaque index over time. Of the original four contained sessions, we analyzed data from the first and last sessions, comparing pre and mail treatment outcomes. The results for the kickoff and final session are summarized in Table 4. In the first session, the pre-treatment mean (standard error) for treatment grouping 1 and treatment grouping 2 are 1.31 (0.35) and 1.33 (0.38), respectively, and similarly for the concluding session, one.54 (0.26) and one.36 (0.27), respectively. The pretreatment measures betwixt groups bear witness no meaning difference for either of the sessions. The correlations in the pre and mail service treatment measures are 0.91 and 0.82 for the first and final sessions, respectively. In line with simulations and theoretical expectations, it follows that nether the beginning session, ANCOVA-Change and ANCOVA-POST performed equally well amid methods, presenting the lowest standard error for handling effect. However, in the last session, ANOVA-Change actually presented a slightly lower standard mistake compared to ANCOVA methods (0.0564 compared to 0.0592). Given the loftier correlation between pre and post measurements in this data ready, it follows theoretical expectations that ANOVA-Change is extremely close to that of ANCOVA models. In the instance where ρ=0.91, ANOVA-CHANGE should theoretically produce variance that is 1.05 times ANCOVA. In both sessions, ANOVA and LMM showroom larger variance than ANCOVA methods. The variance estimates from ANOVA and LMM are 2.seven times greater in the starting time session and 1.seven times greater in the second session than ANCOVA.

Give-and-take

This newspaper compares four traditional approaches (ANOVA-POST, ANOVA-CHANGE, ANCOVA-Post, and ANCOVA-Alter) and a more than modern arroyo (LMM) used in the analysis of pre-post data. These five methods are compared via simulated data from a typical clinical trials setting, where pre-treatment groups are assumed as allocated through proper randomization, and the chief interest is to examine estimates of treatment result. Comparisons of these methods have been investigated in theoretical framework (Brogan and Kutner [3]; Dimitrov and Rumrill [4]; Frison and Pocock [2]; Huck and McLean [four]; Laird [14]), and in a similar manner as Egbewale et al. [12] we review these methods and discuss them in terms of several false circumstances, as well as a real data application.

Overall, all of the five methods in the simulated scenarios yield equally unbiased treatment effect estimate. Nevertheless, their functioning (in terms of variance, type-I fault and 95% CI coverage) varies with pretreatment group differences as indicated in previous literature [7,xi,12]. For example, LMM is establish to be more conservative compared to the ANCOVA methods. Consistent with previous literature, ANCOVA models accept the smallest variance, highest power, and nominal 95% confidence interval coverage compared to ANOVA-POST, ANOVA-Modify, and LMM. Similar to conclusions reached by Vickers and Altman [thirteen], in our simulation report as correlation between pre-and post-handling measures increase, ANOVA-Change approaches ANCOVA in both variance and power. However, in all simply the near extreme cases (i.east. when ρ ≈ 0 or 1), ANCOVA methods are the virtually optimal, achieving the greatest power and lowest variability. Thus, in the case of balanced pre-treatment data, our results are consistent with well-nigh existing literature, in that ANCOVA is a preferred method. This may non hold in situations with some degree of imbalance between handling groups at baseline and different levels of pre-post correlation [12].

Nosotros also examine the robustness of these methods when the pre-mail measures are simulated nether CS versus HCS covariance structures. The data imitation under HCS produces the greatest upshot on the ANOVA-Change results, where handling effect variances are particularly influenced at lower correlations. When pre and post treatment measures accept equal variance and depression correlation, ANOVA-POST outperforms ANOVA-CHANGE, simply as the imbalance betwixt pre-post variance grows, the 2 methods perform similarly. In practical applications, when i does not have control over pre- and post- treatment variances, results demonstrate that one could reasonably expect ANOVA-Modify to consistently perform better than ANOVA-POST when ρ ≤ 0.five regardless of equality of variances. When ρ ≥ 0.5, the all-time performing method volition depend on the degree of equality of the variances in pre-and post-measurements. Nevertheless, both methods are still consistently outperformed by ANCOVA.

In our simulation report, the LMM approach performs only besides as ANOVA-POST. Withal, these simulations assume no missing data. In clinical trials evaluating patients over time, on the other hand, missing information are common (i.due east. some patients are lost to follow-up and post-treatment measurements are never recorded). In such cases, the mechanism of missing data as divers past Petty and Rubin [20], along with method of assay are important in reaching unbiased results. When data are missing completely at random (MCAR), the method of assay makes little difference, i.e. the generalized least squares (GLS) based methods and LMM should provide equivalent results [21]. Withal, in the case when information are missing at random (MAR), GLS tin lead to biased inference on effects, whereas the LMM approach which relies on likelihood provides unbiased results when the within-subject field covariance matrix, R_i , is specified correctly [21]. When information are missing not at random (MNAR), all 5 methods may atomic number 82 to biased results. Thus, conclusions made in this report with regards to the LMM approach compared to the other approaches concur merely when data are MCAR or not missing. In the example of MAR data, LMM may be more optimal.

Conclusion

Despite decades of long scrutiny of this topic and our extensive simulation study under a wide range of scenarios, there however remain several avenues of future work. For instance, in the clinical trials setting, there are often many more important covariates which are included to bargain with baseline covariate imbalance. Additionally, this study assumed that the treatment assignment was random, which is usually the case in clinical trials, resulting in equal pre-handling values among treatment groups. As it was discussed in the introduction, results may differ when pre-treatment measures are diff, particularly affecting the bias of ANCOVA-POST. Hereafter piece of work is needed to appraise how ANCOVA-Change as presented by Laird [14], performs under varying degrees of pre-treatment imbalance every bit in the simulation study performed by Egbewale [11]. Finally, these methods could exist explored under the generalized linear mixed model framework with non-Gaussian pre-post data.

OMICS International: Open Admission Publication Benefits & Features

Unique features:

Increased global visibility of articles through worldwide distribution and indexing
Showcasing recent research output in a timely and updated style
Special issues on the current trends of scientific research

Special features:

700+ Open up Admission Journals
l,000+ editorial team
Rapid review process
Quality and quick editorial, review and publication processing
Indexing at major indexing services
Sharing Option: Social Networking Enabled
Authors, Reviewers and Editors rewarded with online Scientific Credits
Better disbelieve for your subsequent articles

Submit your manuscript at: http://world wide web.editorialmanager.com/biobiogroup/

Supplementary Material

appendix

Acknowledgement

The written report concept and design and conquering of data is past MG. Assay and interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual content and final approval of manuscript is made by all co-authors. The authors give thanks (names omitted to maintain integrity of the review process) for helpful comments. The manuscript represents the views of the authors. Information technology is a byproduct of group project work past graduate students in an advanced statistical methods course.

Abbreviations:

ANCOVA	Assay of covariance
ANCOVA-CHANGE	Assay of covariance using change score as the outcome
ANCOVA-Mail	Analysis of covariance using post-treatment equally the effect
ANOVA	Analysis of variance
ANOVA-CHANGE	Analysis of variance using modify score as the outcome
ANOVA-Mail service	Analysis of variance using post-handling as the issue
CI	Conviction interval
HCS	Heterogeneous chemical compound symmetry
KR	Kenward-Rogers
LMM	Linear mixed model
MAR	Missing at random
MCAR	Missing completely at random
MNAR	Missing non at random
REML	Restricted maximum likelihood
SD	Standard deviation

References

ane. Matthews J, Altman DG, Campbell Thousand, Royston P (1990) Analysis of serial measurements in medical research. BMJ: British Medical Periodical 300: 230. [PMC free article] [PubMed] [Google Scholar]

two. Frison L, Pocock SJ (1992) Repeated measures in clinical trials: assay using hateful summary statistics and its implications for blueprint. Statistics in medicine 11: 1685–1704. [PubMed] [Google Scholar]

3. Brogan DR, Kutner MH (1980) Comparative analyses of pretest-posttest research designs. The American Statistician 34: 229–232. [Google Scholar]

four. Huck SW, McLean RA (1975) Using a repeated measures ANOVA to analyze the data from a pretest-posttest pattern: A potentially confusing task. Psychological Bulletin 82: 511. [Google Scholar]

5. Jennings E (1988) Models for Pretest-Posttest Data: Repeated Measures ANOVA Revisited. Periodical of Educational and Behavioral Statistics 13: 273–280. [Google Scholar]

six. Dimitrov DM, Rumrill J, Phillip D (2003) Pretest-posttest designs and measurement of change. Work: A Journal of Prevention, Assessment and Rehabilitation 20: 159–165. [PubMed] [Google Scholar]

vii. Samuels ML (1986) Use of analysis of covariance in clinical trials: a clarification. Controlled clinical trials 7: 325–329. [PubMed] [Google Scholar]

8. Van Breukelen GJ (2006) ANCOVA versus modify from baseline had more than power in randomized studies and more bias in nonrandomized studies. Journal of clinical epidemiology 59: 920–925. [PubMed] [Google Scholar]

nine. Lord FM (1967) A paradox in the interpretation of group comparisons. Psychological Bulletin 68: 304–305. [PubMed] [Google Scholar]

10. Liang KY, Zeger SL (2000) Longitudinal information analysis of continuous and detached responses for pre-post designs. Sankhya: The Indian Journal of Statistics Series B: 134–148. [Google Scholar]

11. Senn South (2006) Change from baseline and analysis of covariance revisited. Statistics in medicine 25: 4334–4344. [PubMed] [Google Scholar]

12. Egbewale Be, Lewis M, Sim J (2014) Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study. BMC medical research methodology xiv: 49. [PMC free article] [PubMed] [Google Scholar]

xiii. Vickers AJ, Altman DG (2001) Statistics notes: analysing controlled trials with baseline and follow upwardly measurements. BMJ: British Medical Journal 323:1123. [PMC free article] [PubMed] [Google Scholar]

14. Laird N (1983) Further comparative analyses of pretest-posttest research designs. The American Statistician 37: 329–330. [Google Scholar]

15. Kenward MG, Roger JH (1997) Small Sample Inference for Fixed Effects from Restricted Maximum Likelihood. Biometrics 53: 983–997. [PubMed] [Google Scholar]

sixteen. Schaalje GB, McBride JB, Fellingham GW (2002) Adequacy of Approximations to Distributions of Test Statistics in Complex Mixed Linear Models. Periodical of Agronomical, Biological, and Ecology Statistics 7: 512–524. [Google Scholar]

17. Senn S (2014) Various varying variances: The challenge of nuisance parameters to the practising biostatistician. Stat Methods Med Res [PubMed] [Google Scholar]

eighteen. Betticher DC, Delmore G, Breitenstein U, et al. (2013) Efficacy and tolerability of two scalp cooling systems for the prevention of alopecia associated with docetaxel handling. Back up Intendance Cancer 21: 2565–2573. [PubMed] [Google Scholar]

19. Parizotto SPC, Rodrigues CRMD, Singer JM, Sef HC (2003) Effectiveness of low cost toothbrushes, with or without dentifrice, in the removal of bacterial plaque in deciduous teeth. Pesquisa Odontologica Brasileira 17: 17–23. [PubMed] [Google Scholar]

20. Picayune RJA, Rubin DB (2002) Statistical Assay with Missing Data: Wiley. [Google Scholar]

21. Fitzmaurice GM, Laird NM, Ware JH (2012) Applied longitudinal assay: John Wiley & Sons. [Google Scholar]

mynattrallown.blogspot.com

Source: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6290914/