ASYMPTOTIC PROPERTIES OF SIMPLE LINEAR MEASUREMENT ERROR MODELS

The main objective of this paper is to study estimators of regres- sion models on the independent variable X which is not directly observed for some reasons. In such a situation, a substitute variable W is observed instead. This substitution complicates the statistical analysis of the observed data when the purpose of the analysis is inference about a model deﬁned in terms of X . The substitution causes a inconsistent estimator; this is deﬁned as a measurement error problem. To correct this problem, the conditional score and corrected score methods are proposed by Stefanski&Carroll (1985) and Nakamura (1990), respectively. In this study, large sample distribution the- ory for both the conditional score and corrected score estimators are derived and the performance of the estimators and the adequacy of the large sample distribution theory are obtained via Monte Carlo simulation.


I
The regression analysis is a statistical methodology for studying the functional relationship between two or more quantitative variables so that one can be explained from the other variables. Y named as the response variable is a dependent variable whose variation can be explained by an explanatory or named independent variable X. The independent variable in the regression analysis cannot be observed for some reasons either because it is too expensive to obtain, unavailable, or mismeasured. In this kind of situations, a substitute variable W is observed instead of the true variable X. Therefore, the statistical inference of regression coefficient involves the additive measurement error model, W = X + U, where U is a measurement error with 0 mean and a constant variance. The effect of measurement error on fitting a regression model causes inconsistent parameter estimation and also its statistical inferences. The statistical analysis of inaccurately measured data or data measured with a substitute variable causes a common problem that is called attenuation, in other words measurement error problem. When measurement error is in presence, the statistical models and methods for analyzing the such data is studied by Fuller and Hidiroglou (1978), Moran (1971) and recently Prentice (1982), Fuller (1982a, 1982b), Carroll et al.(1984), Stefanski (1985) and Stefanski and Carroll (1985). The model fitting has been studied for functional case in which the X is regarded as an unknown fixed constants, and structural case in which the X is regarded as a random variable. For fixed X, several approaches are given to correct bias due to measurement error. One is regression calibration method studied by Carroll and Stefanski (1990) and Glesjer (1990)). The conditional score function suggested by Stefanski and Carroll (1987) and another one is corrected score function given by Stefanski (1989) and Nakamura (1990). Also, the conditional-score and corrected-score are defined and derived for the interactions between error-free and error-prone regressors in the models by Dagalp (2001). The SIMEX (simulation extrapolation) method is proposed by Cook and Stefanski (1994). Actually, the analysis of data with measurement error is well described by Fuller (1987) and Carroll et al. (1995). Hanfelt and Liang (1997) focused more directly on hypothesis test which is an alternative to Wald's test for the regression parameters. In this article, basic theory of the conditional-score and the corrected-score methods are explained. Asymptotic properties of the conditional-score and corrected-score estimators are derived and results of a simulation studied are presented.
Consider the usual simple linear regression model .., n where i represents experimental error, and the additive measurement error model for the observed W i = X i + U i , where U i represents measurement error. In this article, the simple linear regression model is regarded as a form of the exponential family given in McCullagh and Nelder (1989, Chap. 2). Given a covariate X = x, the response variable Y has density function as a generalized linear model in canonical form with respect to σ-finite measure m(·). where η = α + β x x is called the natural parameter, b(η) = η 2 2 , c(y, σ 2 ) = − y 2 2σ 2 − log( √ 2πσ 2 ), and θ = (α, β x , σ 2 ) T is the unknown parameter. The mean and variance of Y given X are b (η) and σ 2 b (η), where b and b are the first and second derivatives of b(η) with respect to η, respectively. When the measurement error U is distributed as a normal random variable with mean zero and variance σ 2 u , the density of W given X = x is (1. 2) The models in (1.1) and (1.2) together define a generalized linear measurement error model, but in this paper, simple linear model case is taken into consideration. In the simple linear case, it is emphasized that the parameters of interest (α, β x ) appear in regression model for b (η) = E(Y | X) = η, which depends on the unobserved true variable X. For some cases, the variable W is observed instead of X with a measurement error variable. Hence, the second component of the model relates the observed substitute variable W to X. If the model fits the observed data, prediction of the parameter of interest results in biased estimators. If X is directly observed, then the (α, β x , σ 2 ) is estimated by solving the normal equations given in matrix form as where the natural parameter, η i = α + β x X i and p is the number of parameters in the model. Note that the estimating equations (1.3) and (1.4) result in maximum likelihood estimators when n − p is replaced by n.The equations in (1.3) yield the ordinary least squares estimate of slope on the true data given by and substituting W for X in (1.3) yields to the so-called naive slope estimator that is given by where S XX is the sample variance of X 1 , ..., X n , S UU is the sample variance of U 1 , ..., U n , S XU is the sample covariance of (X i , U i ) and S Y U is the sample covariance of (Y i , U i ) , i = 1, 2, ..., n. By the Law of Large Numbers, both S Y U and S XU converge in probability to zero, S XX P −→ σ 2 X , and S UU P −→ σ 2 U , as n −→ ∞. Thus, by Slutky's theorem, so-called the attenuation factor (Fuller, 1987) is a real number in the range [0, 1] for σ 2 X positive and finite. The extreme case λ = 1 is obtained when there is no measurement error σ 2 U = 0 . The other extreme case is approached only in the limit as σ 2 U −→ ∞ for fixed σ 2 X < ∞. The naive slope estimator β Y |W is biased towards zero because of 0 ≤ λ ≤ 1.
The purpose is to obtain sufficient and consistent estimators of the parameters on the observed data instead of the true data. Some methods are suggested to correct this attenuation to get statistical inference for the parameters of interest. The conditional-score and corrected-score are defined in the following sections for measurement error problem.

C -S E
Consider the simple linear regression model of Y on X Y = α + β X X + , and the additive measurement error model where the error-free regressor X ∼ N μ X , σ 2 X , the experimental error ∼ N 0, σ 2 , and the measurement error U ∼ N 0, σ 2 U are independent variables. A common strategy for handling models with unobserved X 1 , . . . , X n , so-called nuisance parameters, is to base inference on conditional likelihoods since functional maximum likelihood estimation neither is computationally achievable nor the estimators are generally consistent due to the large number of nuisance parameters (Neyman and Scott, 1948;Stefanski and Carroll, 1987). Stefanski and Carroll (1987) showed how to derive conditional estimating equations for generalized linear measurement error models. When X is given, the conditional density of Y and the conditional density of W given as To find the estimator for error-free unobserved regressor X, functional maximum likelihood estimation maximizes the likelihood as a function of θ and the unobserved predictors X 1 , . . . , X n , i.e., Consider the joint density in (2.1) and define Ω = The joint density of (Y, W ) given X = x can be written as When X is regarded as a parameter and all other parameters as known in the density of (Y, W |X), the statistic is complete and sufficient for X by the Factorization Theorem (Casella and Berger 1990, p.250). Thus, the distribution of Y |Δ depends only on Y , W and θ, but not on the unobserved true regressor X. From the joint density function of (Y, Δ), the conditional density of Y given Δ is and S(·, ·, ·) is defined as The moments of Y given Δ = δ can be computed from the partial derivatives of S(ϕ, β X , σ 2 ) with respect to ϕ because (2.3) is an exponential family density in ϕ and Y is the natural sufficient statistic. The conditional distribution of Y given Δ = δ is an exponential family with respect to the σ-finite measure m(·) which does not depend on θ. Thus From this equation, consistent estimating equations for θ can be derived to estimate the regression parameters of interest. An alternative derivation of the conditionalscore from the conditional distribution of Y given Δ = δ is suggested by Stefanski and Carroll (1985) as defined and derived clearly by Dagalp (2001). The conditional-score function given by Stefanski and Carroll (1987) is Using the fact that the σ-algebra generated by Δ is a sub-σ-algebra generated by Y and W , thus combining the results in (2.5) and (2.6), the conditional-score function is driven as (2.7) It follows from (2.5) and (2.7) that and from (2.8) , the conditional-score estimators can be obtained as solutions of named as a consistent estimating equations for θ.

C -S E
The corrected-score method is proposed and defined by Stefanski (1989) and Nakamura (1990) and later studied by Novick (2000). It is a natural competitor to the conditional-score estimator and a technique for eliminating asymptotic bias caused by measurement error for the generalized measurement error models. In this paper, the corrected-score method is defined for simple linear measurement error models by using unbiased estimating equations for the parameters of interest. Assume that there exists an unbiased score function Ψ for the true data X as an error-free predictor. The unknown parameter θ in the absence of measurement error is estimated as the solution of the consistent estimating equations and is called the true estimator.
Suppose that Ψ(Y, X; θ) is a score function from the model for the true data such that the estimatorθ T rue , solving Suppose that there exists a certain smooth Ψ functions such that where Ψ M (Y, X; θ) is a corrected-score function of the observed data. It follows from this property that The problem is that, the corrected-score function satisfying (3.2) does not always exist, and when it exists, it is not easily find. The corrected-score functions for some models have been studied and derived by Stefanski (1989) who provides a mean of determining corrected-score functions for a large class of models. Let Z ∼ N (0, 1) be independent of (Y, W ) and i = √ −1. For the existence of a certain smooth Ψ functions a corrected-score can be found as When it exists, the corrected-score function in (3.3) can sometimes be found mathematically and can always be computed by Monte Carlo simulation (Novick, 2000).
Consider the linear regression model Y = α + β X X + and the measurement error model W = X + U where the error-free regressor X ∼ N μ X , σ 2 X , the experimental error ∼ N 0, σ 2 , and the measurement error U ∼ N 0, σ 2 U are independent variables, the true-data likelihood score function is where η = α + β X X, and θ = (α, β X , σ 2 ) T . To find the corrected-score function in (3.3) from certain smooth Ψ functions in (3.4), result shows that the corrected-score function Ψ M (·, ·, ·, ·) is given as The corrected-score estimating equations based on the observed data are then An estimator satisfying the equations in (3.6) is called a corrected-score estimator and Ψ M (·, ·, ·, ·) in ( 3.5) is called the corrected-score function for linear regression.
Corrected-score and conditional-score defined in this paper provide M-estimators which are consistent in the absence of measurement error. These estimators are determined by solving the unbiased estimating equations of the form in (3.1). The conditional-score and corrected-score estimators are obtained by replacing Ψ function by Ψ C in (2.7) and Ψ M in (3.5), respectively. Provided that If it is known that the large sample distribution of M-estimators is multivariate normal with mean 0 and a covariance matrix that depends on Ψ and Ψ = ∂/∂θ T Ψ under some regularity and moments conditions (Stefanski, 1985). Asymptotic distribution of M-estimators are reviewed and applied to conditional-score and corrected-score estimators by a Taylor-series expansion, The asymptotic normality follows from the Central Limit Theorem. Thus, the M-estimator θ is asymptotically normally distributed with mean θ and covariance matrix For the case of independent and identically distributed variables {Y i , W i }, i = 1, . . . , n, A and B are defined as For the conditional-score function in (2.7), the matrices A and B are designated A C and B C , that are also given by the expressions, respectively; For the corrected-score function in (3.5), the A and B matrices are designated A M and B M that can be written as Replacing each term in the form X j , j = 1, 2 by E{(W + iσ U Z) j |W } in a M (X; θ) results a * M (W ; θ) having the property that and replacing the covariate In this section, two methods of estimating the covariance matrix of θ are described. The first is often called the sandwich variance estimator which uses empirical expectation to estimate the A and B matrices appearing in the sandwich variance matrix. In this case, the estimators of A n (θ) and B n (θ) for the conditionalscore and the corrected-score functions are Thus, the sandwich variance estimator of the asymptotic covariance matrix of θ is The second method is called the conditional model-based expectation method.
For the conditional-score function, we have the distribution of Y given the sufficient statistic Δ for X in (2.2) and the observed covariate. It follows from (4.1) that are consistent estimators of A C and B C , respectively. When the true predictor X is measured without error, Ψ is the likelihood score function. Thus, the estimator of θ that satisfies the M-estimating equations in (3.1) is the maximum likelihood estimator (MLE) and denoted θ T rue . In this case, the matrices A n (θ) and B n (θ) are equal to the Fisher Information matrix, denoted by I(θ),

T R M C S
In this section, results from a Monte Carlo simulation study are reported. The SAS PROC IML software was used to perform all simulations and analysis of the output for both the conditional-score and the corrected-score functions. Each simulation consisted of generating B = 100 data sets of size n. The data {X j , j , U j } n j=1 were generated from the normal distribution ⎛ For each data set, the response variable Y j is normally distributed with mean θ T X j , where X j = (1, X j , ) T . The measured data W j was generated as W j = X j + U j , j = 1, . . . , n.
For all of the estimators given above, Newton-Raphson iterations with an initial guess, sayθ where B is the iteration number, and ).
The iteration stops when two successive iterates differ by less than a specified tolerance or when the number of iterations exceeds an allowed maximum. The true, naive, conditional-score, corrected-score estimators of α, β X and σ 2 were compared and Monte Carlo means are reported in Table 1 for sample size 50, 100, 200, 500 and 1000 when measurement error variance σ 2 U takes 0.5 and 0.75 values. The silent features of Table 1 include the attenuation in the naive estimators of β X for two different values of σ 2 U . When the sample size increases, the naive estimator cannot obtain better estimate. The conditional-score and corrected-score estimates of slope using W as the measurement of X produce unbiased estimators under the assumption that the error variance is known and equal to 0.5 and 0.75. Both the conditional-score estimator and the corrected-score estimator of the slope completely correct for the attenuation due to measurement error, even if the sample size is small. When measurement error variance is 0.5, the naive estimator is drastically biased even though the sample is large. If the conditional-score and the corrected-score methods are compared for the slope estimator, both methods correct the attenuation and yield much better estimators for σ 2 U = 0.5. When the true and naive slope estimators are obtained by the conditional-score and the corrrected-score methods, both estimators are the same. Also noteworthy is the fact that the conditional-score and corrected-score methods obtain the same true and naive estimator, therefore it shows both methods work skillfully for error-free and error-prone regressor at all sample sizes.
Simulation study results of the true, naive, conditionalscore and corrected-score estimators for the simple linear regression model with parameters θ = (α, β X , σ 2 ) T = (0, 0.5, 0.5) T and measurement error variance σ 2 U = {0.5, 0.75}.  Two methods, the conditional score model-based expectation method and the corrected score model-based expectation method, for estimating the covariance matrix of θ are given in Table 2 via Monte Carlo simulation for {100, 1000, 10000, 100000} . The conditional-score yields smaller variance estimates then the corrected-score. ARE shows that the corrected-score produces smaller variances then conditionalscore for 100 and 1000 Monte Carlo runs when σ 2 U = 0.5. If the measurement error variance is equal or greater than 0.75, the variance of the conditional-score estimator is obtained smaller than the corrected score's.