Complete Case Analysis

This approach deletes cases with missing data in any of the variables used in the analysis (Baraldi & Enders, 2010; Hill, 2004). The traditional propensity score estimation method of using logistic regression implements complete case analysis by default. Therefore, this method is commonly used in applied research. The data that remains after deleting cases with missing data are assumed to be a simple random sample of the full data (Baraldi & Enders, 2010). Missingness is not related to any study variables nor to the hypothetically complete values of itself (Equations and ). According to Hill (2004), the assumption underlying complete case analysis is that the joint distributions of \(X_{obs}\) and \(X_{mis}\) are same across the two treatment conditions: \[\begin{equation} X_{obs}, X_{mis} \perp\!\!\!\perp D \end{equation}\] Therefore, an unbiased causal effect estimate can be retrieved after deleting cases with missing data. Such an assumption is very stringent and unlikely to be met in the types of data required for propensity score analyses (Baraldi & Enders, 2010; Hill, 2004). As mentioned above, deleting cases can also result in loss of power. Additionally, whether \(X_{mis}\) is balanced between the treatment groups cannot be confirmed.

Multiple Imputation

Multiple imputation (MI) generates multiple sets of data with the missing values drawn from an imputation model (Mitra & Reiter, 2016; Rubin, 1987). MI will create \(m > 1\) imputed datasets that contain different imputed values (Murray, 2018; van Buuren, 2018). Analyses can be performed on each of the datasets and results from each dataset can be aggregated across to derive a final estimate, standard error, degrees of freedom, and test result. Thus, MI involves two stages: (1) imputation and creation of the \(m\) imputed datasets, and (2) analysis and pooling of estimates across the datasets (Murray, 2018; van Buuren, 2018).

There are two approaches for imputing multivariate missing data: (1) joint modeling, JM, and (2) fully conditional specification, FCS, also called multivariate imputation by chained equations, MICE (Murray, 2018; van Buuren, 2018; van Buuren & Groothuis-Oudshoorn, 2011). JM entails jointly modeling variables with missingness by drawing from a multivariate distribution (Murray, 2018; van Buuren, 2018; van Buuren & Groothuis-Oudshoorn, 2011). FCS entails univariate conditional imputation models of variables with missing data that iteratively condition on all other variables using Monte Carlo Markov chain methods (van Buuren, 2018; van Buuren & Groothuis-Oudshoorn, 2011). JM imputes all variables simultaneously whereas FCS imputes one variable at a time (van Buuren, 2018). Because JM requires specification of a joint distribution for all the variables, it may not be as flexible as FCS when dealing with a large number of covariates with missing data (Akande, Li, & Reiter, 2017). However, FCS is computationally more intensive than JM (van Buuren, 2018). FCS also has been shown to outperform JM for categorical variables and is more robust under mis-specification of imputation model (van Buuren, 2018). Therefore, van Buuren (2018) recommended to use FCS over JM.

If the missingness mechanism is MAR or MCAR and if assumptions underlying the imputation model are correct, MI will yield unbiased results, as it uses the information available in \(X_{obs}\) to impute missing values (Murray, 2018). In the causal inference context, Hill (2004) argued that MI relies on the assumption of latent ignorability, a concept introduced by Frangakis & Rubin (1999). The assumption requires that the treatment assignment mechanism is ignorable given complete covariate data including the values that are latent or missing. These missing values are derived from MI. Below, let \(e_{MI}(X)\) denote propensity scores derived after multiple imputation: \[\begin{equation} X_{obs}, X_{mis} \perp\!\!\!\perp D| e_{MI}(X) \end{equation}\] \[\begin{equation} Y(1), Y(0) \perp\!\!\!\perp D | e_{MI}(X) \end{equation}\] Hill (2004) proposed two different ways to combine propensity scores estimated in each of the m datasets:

Generalized Propensity Scores

Rosenbaum & Rubin (1984) proposed the use of generalized propensity scores (GPS) as a way to address missing covariate data. The GPS represents the probability of treatment given observed covariates and missingness indicators (Rosenbaum & Rubin, 1984): \[\begin{equation} e^*(X) = P(D = 1|X_{obs}, R) \end{equation}\] Conditioning on \(e^*(X)\) will balance the treatment groups in terms of the observed covariates and missingness patterns (Rosenbaum & Rubin, 1984). The observed part of \(X\) and the missingness pattern indicators, \(R\), will be independent of treatment assignment given the GPS (Rosenbaum & Rubin, 1984): \[\begin{equation} X_{obs}, R \perp\!\!\!\perp D| e^*(X) \end{equation}\] However, conditioning on GPS will not balance the groups in terms of the unobserved values of \(X\) (Rosenbaum & Rubin, 1984): \[\begin{equation} X_{mis} \not\!\perp\!\!\!\perp D| e^*(X) \end{equation}\] Although this technique of treating missing data is not generally recommended for other types of missing data analyses, it has been recommended for use in propensity score analysis literature (Rosenbaum & Rubin, 1984; Stuart, 2010). In the context of propensity score analysis, this approach does not assume latent ignorability of treatment assignment because legitimate values for missing data are never derived. The assumption underlying this method is that balancing the treatment and control groups on \(X_{obs}\) and \(R\) is a sufficient condition to satisfy ignorability. With the GPS, the treatment and control groups are possibly not going to be balanced in terms of \(X_{mis}\).

For large studies with few missing data patterns, Rosenbaum & Rubin (1984) suggested estimating separate logit models for each missingness pattern. In practice, it is common to encounter many patterns of missing data. For these scenarios, Rosenbaum & Rubin (1984) suggested creating an additional category indicating missingness for categorical variables. For continuous variables, Stuart (2010) recommended imputing missing data with a single arbitrary value, such as the overall mean of the covariate, and then creating a missingness indicator variable. In general missing data analysis context, van Buuren (2018) noted that this method of combining arbitrary (mean) imputation along with missingness indicators can underestimate the standard error of the estimate of interest.

The CART algorithms treat missing data natively as they split missingness as a category itself. In this manner, this approach is similar to the GPS which uses missingness pattern indicators when estimating propensity scores. The missingness categories are used to estimate propensity scores and conditioning on the propensity scores should balance the treatment and control condition in terms of the patterns. However, splitting does not actually impute the missing data so it is plausible to assume that like GPS, scores derived using the splitting method will not balance the groups in terms of the latent missing data. In addition, unlike MI, there are no imputed complete datasets saved to analyze for the outcome model. Therefore, splitting would need to be combined with some other technique for outcome modeling.

Other Methods

The following methods have been discussed theoretically in literature examining missing data methods in propensity score analysis. However, these methods are not commonly used in applied literature.