I don’t disagree with anything you are saying: There is nothing in the propensity score estimation method itself that makes the results less prone to bias, compared to other methods
I should have been more specific, my point was rather that the specific implementation of propensity score matching which Rubin recommends in the paper I linked to, allows the investigator to blind themselves to the outcome while assessing whether they have been able to create balanced groups. It is possible that you can do something similar with other estimation methods, but I haven’t heard anyone talk much about it, and it is not immediately obvious to me how you would go about it
Thanks for the quick clarification! I guess I am not following you. If you want to blind yourself, you can just do it—you don’t need to modify the estimator in any way, just write the computer program implementing your estimator in such a way that you don’t see the answer. This issue seems to be completely orthogonal to both causal inference and estimation. (???)
If you use the propensity score matching method, you begin by estimating the propensity score, then you match on the propensity score to create exposed and unexposed groups within levels of the propensity score. After you create those groups, there is a step where you can look at the matched groups without the outcome data, and assess whether you have achieved balance on the baseline covariates. If I understand Rubin’s students correctly, they see this as a major advantage of the estimation method.
You can obviously blind yourself to the outcome using any estimation method, but I am not sure if there is a step in the process where you look at the data without the outcome to evaluate how confident you are in your work.
In order for the estimand of the propensity score method to be unbiased, the following is sufficient:
(a) SUTVA (this is untestable)
(b) Conditional ignorability (this is testable in principle, but only if we randomize the exposure A)
(c) The treatment assignment probability model (that is the model for p(A | C), where A is exposure and C is baseline covariates) must be correct.
It may be that the “balance property” tests a part of (b), but surely not all of it! That is, the arms might look balanced, but conditional ignorability might still not hold. We cannot test all the assumptions we need to draw causal conclusions from observational data using only observational data. Causal assumptions have to enter in somewhere!
I think I might be confused about why checking for balance without working out the effect is an advantage—but I will think about it, because I am not an expert on propensity score methods, so there is probably something I am missing.
I don’t disagree with anything you are saying: There is nothing in the propensity score estimation method itself that makes the results less prone to bias, compared to other methods
I should have been more specific, my point was rather that the specific implementation of propensity score matching which Rubin recommends in the paper I linked to, allows the investigator to blind themselves to the outcome while assessing whether they have been able to create balanced groups. It is possible that you can do something similar with other estimation methods, but I haven’t heard anyone talk much about it, and it is not immediately obvious to me how you would go about it
Thanks for the quick clarification! I guess I am not following you. If you want to blind yourself, you can just do it—you don’t need to modify the estimator in any way, just write the computer program implementing your estimator in such a way that you don’t see the answer. This issue seems to be completely orthogonal to both causal inference and estimation. (???)
Am I missing something?
If you use the propensity score matching method, you begin by estimating the propensity score, then you match on the propensity score to create exposed and unexposed groups within levels of the propensity score. After you create those groups, there is a step where you can look at the matched groups without the outcome data, and assess whether you have achieved balance on the baseline covariates. If I understand Rubin’s students correctly, they see this as a major advantage of the estimation method.
You can obviously blind yourself to the outcome using any estimation method, but I am not sure if there is a step in the process where you look at the data without the outcome to evaluate how confident you are in your work.
In order for the estimand of the propensity score method to be unbiased, the following is sufficient:
(a) SUTVA (this is untestable)
(b) Conditional ignorability (this is testable in principle, but only if we randomize the exposure A)
(c) The treatment assignment probability model (that is the model for p(A | C), where A is exposure and C is baseline covariates) must be correct.
It may be that the “balance property” tests a part of (b), but surely not all of it! That is, the arms might look balanced, but conditional ignorability might still not hold. We cannot test all the assumptions we need to draw causal conclusions from observational data using only observational data. Causal assumptions have to enter in somewhere!
I think I might be confused about why checking for balance without working out the effect is an advantage—but I will think about it, because I am not an expert on propensity score methods, so there is probably something I am missing.