Article Summary
Introduction
What do we measure when we measure partisan presidential approval ratings? Partisan approval ratings are often taken as a signal of strength within one’s own party; high approval ratings insulate presidents from criticism and vindicate potentially unpopular policy actions. They are also often used as signals of partisan disagreement, with large differences between groups indicating greater polarization. Our knowledge of presidential approval comes from surveys, where respondents self-report their partisanship. In a short article by Montagnes, Peskowitz, and McCrain, the authors draw attention to the fact that the composition of public opinion polls vis-a-vis partisanship is endogenous to Presidential approval. Put simply, a more generally unpopular president may cause co-partisans to either not identify as co-partisans on surveys or not even appear in the surveys, artificially inflating their partisan approval rating.
In their short article, the authors address this compositional problem by calculating upper and lower bounds for partisan approval ratings for president. In doing so, they provide more methodologically rigorous measurements of partisan approval, and emphasize apparent divisions in partisan approval may be more illusory than real.
Analytical Approach
The authors use data from weekly Gallup approval polls from 2009 to 2017, covering all of the Obama presidency and the first year of the Trump presidency. They begin by setting a benchmark partisan approval rating from a survey they believe to more accurately capture the proportion of Democrats and Republicans: the last poll available before the previous election.
There are two possibilities about how the proportion of copartisans may differ between the benchmark and subsequent polls. First, there may be a deficit of copartisans: fewer copartisans exist in the survey population than in the benchmark survey. In this case, the authors assume all the “missing” copartisans either approve or disapprove of the president, thereby constructing upper and lower bounds for partisan approval. In the case where there is a surplus of copartisans, the authors construct the same upper and lower bounds by assuming the “extra” copartisans either all approve or disapprove.
Standard errors are calculated by bootstrapping the upper and lower bounds over 200 replications and calculating the standard deviation. Bounds are calculated for every week of presidential approval ratings.
Main Findings
The authors find the deficit of presidential partisans during the Trump presidency is much higher than during the Obama presidency, leading to a much more dramatic lower bound for partisan presidential approval relative to the mean estimate. The maximum deficit reached by Trump just 1 year into his presidency exceeds the maximum deficit reached by Obama over 8 years in office. This deficit grows for both presidents over the course of their presidency, but attenuates for Obama toward the end of his terms.
In the first 150 days of each Presidents’ term in office, the upper bound of Trump’s partisan approval was only rarely greater than the lower bound of Obama’s partisan approval. The estimates grow closer in subsequent weeks, and converge more closely in Obama’s second term in office. In absolute terms, the lower bound of Trump’s partisan approval is less than 80% in 40 of the 49 weeks of data.
Implications
From a measurement perspective, the results indicate Trump’s partisan approval rating is likely artificially inflated due to respondents who would typically identify as Republicans either not completing the survey or not identifying themselves as Republicans. Americans are likely much more unified in condemnation of Trump than partisan approval ratings suggest. More generally, the paper has significant implications for the measurement of public opinion when self-reported group identity is endogenous to the opinion being measured. Bounding the unknown compositionally correct quantity of interest is one way to gauge uncertainty around such a quantity.
Substantively, the paper should also raise reasonable skepticism of Presidents claiming to have the support of their party when enacting generally unpopular policies. It is precisely these generally unpopular actions that may reduce the willingness of survey respondents to identify as a presidential copartisan, raising questions for how presidents can be kept accountable by their own parties.
Questions left unanswered
The paper is short and concise, defining a purposefully narrow question and executing on it precisely. One question that arises from the article is how one is meant to use the upper and lower bounds in subsequent analyses. By definition, these are the extreme ends of the estimates, so the truth is likely somewhere in the middle. How should we interpret models using such bounds as dependent variables in future analyses?
Additionally, there is a broader theoretical question about which subgroup identifications we should expect to be endogenous to attitudes. Partisanship seems like an easy candidate for such analysis, as partisan identity is made up of both social identity and policy positions. Other group identities, such as race/ethnicity, may also be somewhat endogenous to attitudes, but additional theorizing needs to be done as to where we expect these relationships to emerge. This is admittedly beyond the scope of a short article, but perhaps an interesting avenue of future research nonetheless.
Methods and Analysis
Was the study and its analyses pre-registered?: No
Did the study rely on proxy variables to measure polarization?: No
Were standard p-value thresholds used (p<.05 or 95% Confidence Intervals that don’t overlap zero)?: Yes
- Largest p-value presented as significant: 0.05
Were correlational results interpreted with causal language?: No
Limitations / Weaknesses
The article makes a compelling theoretical point that self-reported group identification is likely endogenous to attitudes, a finding the authors corroborate in the supplementary appendix. Of course, this means that any survey would be subject to such endogeneity. This makes the selection of the baseline survey critical, and this appears somewhat understated in the paper. The amount of “missing” or “excess” copartisans is always calculated relative to some baseline that is itself subject to the same endogeneity concerns as the subsequent surveys. It is maybe the case the last poll prior to the previous election is subject to less endogeneity than subsequent polls, but that claim has to be tested. The authors suggest a robustness check using average values across three weekly polls as a baseline, but could still induce bias if there was consistent over- or under-reporting of copartisan status across all sampled weeks. There is also a theoretical question that comes from the analysis: at what point are the “missing” copartisans simply no longer copartisans? In Trump’s case, the deficit is consistent and large. Should we still be treating these “missing” copartisans as members of the Republican party? Self-reported party identification is certainly endogenous to attitudes, but so is the true composition of the party.
Open Data & Analyses
Does the article make the replication data publicly available?: Yes
Does the article make the replication analysis scripts publicly available?: Yes
Article Citation
Montagnes, B. P., Peskowitz, Z., & McCrain, J. (2019). Bounding partisan approval rates under endogenous partisanship: Why high presidential partisan approval may not be what it seems. The Journal of Politics, 81(1), 321–326. https://doi.org/10.1086/700572
Bibtex
1
2
3
4
5
6
7
8
9
10
11
12
13
@article{doi:10.1086/700572,
author = {Montagnes, B. Pablo and Peskowitz, Zachary and McCrain, Joshua},
title = {Bounding Partisan Approval Rates under Endogenous Partisanship: Why High Presidential Partisan Approval May Not Be What It Seems},
journal = {The Journal of Politics},
volume = {81},
number = {1},
pages = {321-326},
year = {2019},
doi = {10.1086/700572},
URL = { https://doi.org/10.1086/700572},
eprint = { https://doi.org/10.1086/700572}
}