In“Lowering the Threshold of Statistical Signiﬁcance to p *< *0.005 to Encourage Enriched Theories of Politics,” I claimed that:

When

K-many statistically independent tests are performed on pre-speciﬁed hypotheses that must be jointly conﬁrmed in order to support a theory, the chance of simultaneously rejecting them all by chance isαwhere^{K}p < αis the critical condition for statistical signiﬁcance in an individual test. AsKincreases, theαvalue for each individual study can fall and the overall power of the study often (though not always) increases.

This argument is oﬀered to support the conclusion that “moving the threshold for statistical signiﬁcance from *α *= 0*.*05 to *α *= 0*.*005 would beneﬁt political science if we adapt to this reform by developing richer, more robust theories that admit multiple predictions.”

Similarly, in “Questions and Answers: Reproducibility and a Stricter Threshold for Statistical Signiﬁcance,” I claimed that:

Another measure to lower Type I error (and the one that I discuss in my article in

The Political Methodologist) is to pre-specify a larger number of diﬀerent hypotheses from a theory and to jointly test these hypotheses. Because the probability of simultaneously conﬁrming multiple disparate predictions by chance is (almost always) lower than the probability of singly conﬁrming one of them, the size of each individual test can be larger than the overall size of the test, allowing for the possibility that the overall test is substantially more powerful at a given size.

This reasoning, which is similar to reasoning oﬀered in Esarey and Sumner (2018*b*), is incorrect; it would only be true when all predicted parameters were equal to zero. When the alternative hypothesis is that multiple directional predictions for parameters, for example *β*_{i}*> *0 for *i **∈ *1*…K*, separate *t*-tests rejecting each individual null (*β*_{i}*≤ *0) separately using *t*-tests with size *α *will jointly reject all the null hypotheses at most *α *proportion of the time. The key insight is that the joint null hypothesis space includes the possibility that some *β** _{i}* parameters match the predictions while others do not; if (for example)

The upshot is that my argument for making additional theoretical predictions in order to facilitate lowering the threshold for statistical signiﬁcance to *α *= 0*.*005 is based on faulty reasoning and incorrect.

I plan to post this correction as an addendum to both of the print editions featuring these articles.

Berger, Roger L. 1982. “Multiparameter Hypothesis Testing and Acceptance Sampling.” *Technometrics *24(4):295–300.

Berger, Roger L. 1997. Likelihood ratio tests and intersection-union tests. In *Advances **in statistical decision theory and applications*, ed. Subramanian Panchapakesan and Narayanaswamy Balakrishnan. Boston: Birkhäuser pp. 225–237.

Casella, George and Roger L.. Berger. 2002. *Statistical Inference, Second Edition*. Belmont,

CA: Brooks/Cole.

Cohen, Arthur, Constantine Gatsonis and John I. Marden. 1983. “Hypothesis testing for marginal probabilities in a 2 x 2 x 2 contingency table with conditional independence.” *Journal of the American Statistical Association *78(384):920–929.

Esarey, Justin and Jane Lawrence Sumner. 2018*a*. “Corrigendum to Marginal Eﬀects in Interaction Models: Determining and Controlling the False Positive Rate.” Online. URL: http://justinesarey.com/interaction-overconfidence-corrigendum.pdf.

Esarey, Justin and Jane Lawrence Sumner. 2018*b*. “Marginal Eﬀects in Interaction Mod- els: Determining and Controlling the False Positive Rate.” *Comparative* *Political Studies *51(9):1144–1176. DOI: https://doi.org/10.1177/0010414017730080.

Lehmann, Erich L. 1952. “Testing multiparameter hypotheses.” *The Annals of Mathematical Statistics *pp. 541–552.

Silvapulle, Mervyn J. and Pranab K. Sen. 2005. *Constrained* *Statistical Inference: Inequality, Order, and Shape Restrictions*. Hoboken, NJ: Wiley.

Kristin Bryant, Rice University

**Synopsis**

A recent symposium in *Political Analysis*, anchored around Dion, Sumner and Mitchell (2018), discusses their finding that articles authored by women are more likely to cite at least one paper authored by women. Our contribution to this symposium (Esarey and Bryant, 2018) noted that articles in the Dion, Sumner, and Mitchell (2018) data set with at least one female author are cited no more or less often than male-authored articles once we control for the publishing journal and the number of authors. In this paper, we present additional findings that place the results of our original paper into a broader context. This context is important to fully understand how scholarship by women is utilized by the discipline, how scholars’ careers are impacted as a result of this utilization, and how we might achieve greater gender parity in the field.

When looking at the the unadjusted data set, articles with at least one woman author *are* in fact cited fewer times on average. It is plausible that this citation gap *does* represent a substantively meaningful barrier to the advancement of women in the discipline. As we reported in *Political Analysis*, papers with women authors are no more or less likely to be cited once the number of authors and the publishing journal are controlled for via linear regression. However, simply controlling for author count is insufficient to eliminate the gender disparity in citations: controlling for the publishing journal is crucial. An implication is that women may be systematically disadvantaged in the field, but that this disadvantage is not a function of discrimination against women when articles are chosen to be cited. Instead, consistent with the findings of Teele and Thelen (2017), we find that articles in the most-cited journals of the discipline are less likely to have women authors. The etiology of *that* relationship (and the citation gender gap that it creates among political scientists) is difficult to unravel.

**Full Text**

Replication File

A replication file for this paper is available at https://doi.org/10.7910/DVN/XC76G3.

]]>The International Methods Colloquium (IMC) is a weekly seminar series of methodology-related talks and roundtable discussions focusing on political methodology; the series is supported by Wake Forest University and was previously supported by Rice University and a grant from the National Science Foundation. The IMC is free to attend from anywhere around the world using a PC or Mac, a broadband internet connection, and our free software. You can find out more about the IMC at our website:

http://www.methods-colloquium.com/

where you can join a talk in progress using the “Watch Now!” link. You can also watch archived talks from previous IMC seasons at this site. Registration in advance for a talk is encouraged, but not required.

Note that **all talks begin at 12:00p Eastern Time** and last precisely one hour.

Here is our schedule of presenters (and a link to our Google Calendar):

- Oct 12: Matthew Blackwell, Harvard [register to attend]
- Oct 19: Masha Krupenkin, Stanford [register to attend]
- Nov 2: Roundtable on Gender, Citations, and the Methodology Community with Michelle Dion (McMaster), Sara Mitchell (Iowa), Dave Peterson (Iowa State), and Barbara Walter (UCSD) [register to attend]
- Nov 9: Luke Keele, University of Pennsylvania [register to attend]
- Nov 16: Kevin Munger, Princeton/Penn State [register to attend]
- Nov 30: Pablo Barbera, London School of Economics [register to attend]

- Feb 1: Michelle Torres, Rice [register to attend]
- Feb 8: Marcel Neunhoeffer, Mannheim [register to attend]
- Feb 15: Winston Chou, Princeton [register to attend]
- Feb 22: Erin Rossiter, WUSTL [register to attend]
- Mar 1: Matthew Tyler/Christian Fong, Stanford [register to attend]
- Mar 8: Rob Carroll, Florida State [register to attend]
- March 29: Carlos Carvalho, University of Texas Statistics [register to attend]

Additional information for each talk (including a title and a link to a relevant paper) will be released closer to its date.

Please contact me if you need any more information; I hope to see many of you there!

Peter John, Department of Political Economy, King’s College London

**Abstract: **We argue that sequence analysis, mainly used in sociology, may be effectively deployed to investigate political careers inside legislatures. Career progression is a classic topic in political science, but political scientists have mainly examined access to legislatures. Although data reduction methods, for instance, can provide insight, we argue that sequence analysis can be used to understand better the career patterns inside parliaments. In this paper, we explain the method. Then we show how it can describe steps in political careers and map different patterns of advancement. We apply sequence analysis to a case study of MPs in the UK House of Commons from 1997 to 2015. We describe the variety of career paths and carry out regression analysis on the determinants of MP career progression.

**Full Article**

**Online Appendix**

**Replication File**

Replication files for this paper are located at: https://doi.org/10.7910/DVN/I8YHPT.

]]>— begin announcement —

**MZES Open Social Science Conference 2019: Practicing New Standards in Transparency and Reproducibility**

This conference is a forum for practicing and discussing credibility, transparency and replicability in the social sciences.

About a decade ago, John Ioannidis claimed that “most published research findings are false”. While seeming outrageous at the time, a growing body of meta-science research in the behavioral and social sciences substantiated this claim, causing uncertainty about the trustworthiness of published scientific findings. We believe that threats to the validity of published findings in the social sciences are widespread and systemic. Therefore, this conference promotes introspection about the current state of social science research and exchange on the opportunities for institutional and methodological improvement in the future.

The conference is supported by the *Berkeley Initiative for Transparency in the Social Sciences* (BITSS) and will take place from **25-27 January 2019** in **Mannheim**, Germany.

**Keynote speakers:**

- Jeremy Freese (Stanford University)
- Thomas König (APSR, University of Mannheim)
- Arthur Lupia (OSF, University of Michigan)
- Julia Rohrer (100% CI, Leipzig University)

**Participate in the conference:**

**Give a talk**: We call for researchers to advance discussion, debate, literature synthesis, or methods in open social science. We welcome methodological advances, e.g., p-curve analysis, systematic reviews, pre-analysis planning, and replication. We welcome general research findings that apply best practices of open science while conducting the research – Abstract submission DL: 22 August 2018 Read more**MZES-GESIS Pre-Registration Challenge**: We call for researchers to participate in a competition to win funding or survey time for the most innovative and rigorous pre-registration plan for a social science study. – Abstract submission DL: 22 August 2018 Read more**OSSC19 Crowdsourced Replication Initiative:**We call for researchers to replicate and expand a previously published cross-national macro-comparative study. The goal is to explore and develop crowdsourcing methods and generate research surpassing what a single researcher could achieve. The replication comes from the field of immigration and social policy, but we encourage social science researchers of all disciplines and levels to participate. All full participants will be co-authors on the final paper. – Registration DL: 27 July 2018 Read more- Participate
**as a guest**in Mannheim during the conference or during the subsequent Open Science Workshop, offered in collaboration with the*Berkeley Initiative for Transparency in the Social Sciences*(BITSS). Or use the live stream online.

**Organizing Committee**

Nate Breznau (MZES, University of Mannheim)

Eike Mark Rinke (MZES, University of Mannheim)

Alexander Wuttke (MZES, University of Mannheim)

Conference website: http://www.open-socsci.org/

Twitter: @opensocsci

#ossc19

On January 11 and 12, 2018, the fifth Asian Political Methodology Meeting was held at Seoul National University, Republic of Korea. The meeting was co-sponsored by the Department of Political Science and International Relations at Seoul National University and Program for Quantitative and Analytical Political Science of Princeton University.

This year’s program, available at https://asiapolmeth.princeton.edu/online-program, had eight sessions including two keynote speeches and two poster sessions. In total, 14 papers and 30 posters were presented. In total, 95 registered participants (25 foreign and 70 local) attended the entire conference and many unregistered participants also joined the conference. Nationalities of participants were from Australia, China, Germany, Hong Kong, Ireland, Japan, Republic of Korea, Singapore, United Kingdom, and United States.

The invited keynote speaker was Prof. Michael D. Ward from Duke University. Prof. Michael D. Ward gave a talk about how to analyze relational (network) data using statistical methods. Another keynote speaker was from KAIST, Republic of Korea: Prof. Meeyoung Cha. Prof. Meeyoung Cha presented a method of detecting fake news in online social media using machine learning techniques. After the keynote speech by Prof. Ward, the conference moved to a session of “Big Data in Political Methodology,” “Experimental Methods,” and “Bayesian Analysis.” The first day of the conference ended with the first poster session, consisting of faculty and post-doc participants.

The second day of the conference started with a theme of “Political Methodology for Lobby,” and then moved to a session of “Statistical Methods for Representation.” After the second poster session by graduate students, Prof. Cha gave a keynote speech and the conference finalized the program with a session of “Analyzing Congress using Statistical Methods.”

To make this conference successful, six graduate students at Seoul National University voluntarily contributed their time and resources for two months. Their names are Soonhong Cho, Suji Kang, Doeun Kim, Sunyoung Park, Sooahn Shin, Hyein Yang in alphabetical order. On behalf of the program committee, we sincerely appreciated their help and contribution.

* *The program committee for this conference included Jong Hee Park (committee chair and local host: Seoul National University, Republic of Korea), Fang-Yi Chiou (Academia Sinica, Taiwan), Kentaro Fukumoto (Gakushuin University, Japan), Benjamin Goldsmith (University of Sydney, Australia), Kosuke Imai (Princeton University, USA), and Xun Pang (Tsinghua University, China).

The 2019 Annual Meeting will be held in Kyoto, Japan. We look forward to seeing you in Kyoto next year!

*Jong Hee Park is Professor,* *Department of Political Science and International Relations,** **Seoul National University, Republic of Korea.*

]]>

Political Science Research & Methods (PSRM) is the first Press journal adopting Code Ocean, an extension of the journal’s existing policy that requires authors to deposit data necessary to reproduce the results in their articles. A PSRM article with the Code Ocean widget embedded on Cambridge Core, the Press’s publishing platform, can be seen here. The widget enables readers to view and run the code without leaving Cambridge Core.

The release also indicates that similar adoptions might follow at other CUP journals.

The “information for contributors” instructions at *PSRM* have not been updated to reflect this change, but Butler, Karpowitz, and Pope (2017) linked to in the press release indicates how this policy might change the integration of replication code into articles.

The full press release is available at this link.

]]>

Since Brambor, Clark and Golder’s (2006) article in *Political Analysis* (hereafter BCG), our understanding of interaction models has improved significantly and most empirical scholars have now integrated the tools to execute and interpret interaction models properly. In particular, one of the main recommendations of BCG was to include all constitutive terms of the interaction in the model specification. However, BCG acknowledge (in the text surrounding equation 7 of their paper) that there is a mathematically equivalent model specification that allows researchers to exclude certain constitutive terms from an interaction model when one of the modifying variables is discrete. A recent review experience made me realize that this exception is not as widely recognized as BCG’s core advice to include all constitutive terms, suggesting therefore, that a brief note to the scholarly community might be important in publicizing this exception. In the next section, I show the equivalency of BCG standard specification and this alternative specification. I then provide a brief example of both approaches when applied in a substantive case — Adams et al. (2006) study “Are Niche Parties Fundamentally Different from Mainstream Parties?” — and show that we get the same results either using BCG’s approach or the alternative approach.

Overall, I show that while the two model specifications are equivalent, each has some advantages in terms of the interpretation of the regression results. On the one hand, the advantage of the standard specification is to present directly in the regression results whether the *difference in the marginal effects *of *X *on *Y *between the categories of the modifying variable *Z *is statistically significant. On the other hand, the main benefit of the alternative approach is to present directly in the regression results the *marginal effects *of *X *under each category of the modifying variable *Z*. Researchers may thus choose between the two equivalent specifications depending on the results they want to present and emphasize.

**1 Equivalency of the Standard and Alternative Specifications**

In order to show the equivalency of BCG standard specification and the alternative specification when one of the modifying variables is discrete, I take as an example a dependent variable *Y *which is a function of an interaction effect between a continuous variable *X *and a dummy variable *D*. BCG standard approach to interaction models indicates that we must multiply the variables *X *and *D *and include this interaction term as well as the constitutive terms *X *and *D*, respectively, in the regression model. Specifically, the standard specification is the following:

*Y *= *b*_{0} + *b*_{1}*D *+ *b*_{2}*X *+ *b*_{3}*XD *+ *ϵ *(1)

where X is continuous and D is a dummy variable (0,1). The marginal effect of *X *when *D *= 0 is given by *b*_{2} while the marginal effect of *X *when *D *= 1 is given by *b*_{2} + *b*_{3}.²

The alternative approach explained briefly in BCG (see equation 7) and Wright (1976) consists in treating the dummy variable *D *as two dummy variables: *D*, the original variable, which equals 0 and 1 and *D*_{0}, the inverse of *D*, which equals 1 when *D *= 0 and 0 when *D *= 1. For example, if *D *is a dummy variable where 1 represents democratic countries and 0 authoritarian countries *D*_{0} would simply be the inverse dummy variable where 1 represents authoritarian countries and 0 democratic countries. Consequently, *D *+ *D*_{0} = 1 *and D*_{0} = 1 *− D*. The alternative approach consists in multiplying *X *respectively with *D *and *D*_{0}, including all constitutive terms in the regression model except *X *and one of the dummy variables, *D *or *D*_{0}. The reason for including only *D *or *D*_{0} is that these variables are perfectly collinear. It is not possible to include *X *neither because of perfect multicollinearity with *XD *and *XD*_{0}.

The alternative specification is thus the following:

*Y *= *a*_{0} + *a*_{1}*D *+ *a*_{2}*XD *+ *a*_{3}*XD*_{0} + *ϵ *(2)

Equation 2 could be rewritten as

*Y *= *a*_{0} + *a*_{1}*D *+ *a*_{2}*XD *+ *a*_{3}*X*(1 *− D*) + *ϵ *(3)

Equation 3 highlights explicitly that we do not necessarily need to create *D*_{0} but only to multiply *X *by (1 *− D*). In equations 2 and 3, the marginal effect of *X *when *D *= 0 (i.e. when *D*_{0} = 1) is given by *a*_{3} while the marginal effect of *X *when *D *= 1 is given by *a*_{2}.³

The main advantage of this alternative specification is that for each category of the discrete modifying variable *D *(0 and 1 in this case) the marginal effect of *X *and its associated standard error are provided directly from the regression results. This is not the case in the standard approach where only one of these results is directly provided (i.e. *b*_{2}, the effect of *X *when *D*=0). Consequently, we need to add up *b*_{2} and *b*_{3} to obtain the effect of *X *when *D*=1. This is easy to do in Stata with the command *lincom *(*lincom **_b*[*coef *] + *_b*[*coef *]).

A disadvantage of the alternative specification is that the regression results do not indicate whether the difference between the marginal effects of *X *when *D*=0 and when *D*=1 is statistically significant. This is the advantage of the standard approach which provides this information in the regression results . If the coefficient *b*_{3} is statistically significant in equation 1, this indicates that the marginal effect of *X *when *D*=1 is statistically different than the marginal effect of *X *when *D*=0. To answer this question with the alternative approach, we must test the equality of *a*_{2} and *a*_{3}. This is also straightforward in Stata with the command *test *(*test _b*[*coef *] = *_b*[*coef *]) or *lincom *(*lincom _b*[*coef *] *− _b*[*coef *]).

The specification of the alternative approach could be easily generalized to discrete variables with multiple categories whether the discrete variable is nominal or ordinal. The procedure is the same. We need first to create a dummy variable for each category of the discrete modifying variable. We then multiply *X *with each of these dummy variables and include all constitutive terms in the equation (except *X *and one of the dummy variables). This specification will also allow researchers to evaluate directly the magnitude of the substantive effect of *X *across the different values of the discrete modifying variables without *including all constitutive terms of the interaction explicitly*.

**2 Replication of “Are Niche Parties Fundamentally Different From Mainstream Parties? ”**

In this section, I compare the results of the standard and alternative approaches to interaction models in replicating Adams et al. (2006) study “Are Niche Parties Fundamentally Different from Mainstream Parties?” published in the *American Journal of Political Science*. Two main research questions are examined in this article. First, the authors examined whether mainstream parties are more responsive than niche parties to shift in public opinion in adjusting their policy programs. Second, and building on this prediction, they examined whether niche parties are more penalized electorally than mainstream parties when they moderate their policy positions. Here, I only replicate their model associated with the first question.

Adams et al. (2006) tested these hypotheses in seven advanced democracies (Italy, Britain, Greece, Luxembourg, Denmark, Netherlands, and Spain) over the 1976-1998 period. They measure parties’ policy position on the left-right ideological scale with data from the Comparative Manifesto Project (CMP). Surveys from the Eurobarometer are used to locate respondents on the corresponding left-right ideological dimension. Public opinion is measured as the average of all respondents’ self-placement. Finally, the authors coded Communist, Green, and Nationalist parties as niche parties with a dummy variable.

In table 1, I examine party responsiveness to public opinion and present the results of the standard and alternative approaches. Adams et al. (2006) use the standard approach and interact the variable *public opinion shift *with the dummy variable *niche party*. The dependent variable is the change in a party’s left- right position. Adams et al. (2006) thus specified a dynamic model where they assess whether a change in public opinion influences a change in party positions between two elections. The models include fixed effects for countries and a number of control variables (see the original study for the justifications). The specification of the standard approach in column (1) is the following:

∆*party position *= *b*0 + *b*1∆*public opinion *+ *b*2*niche party *+ *b*3(∆*public opinionXniche party*) + *controls*

In column (1) of Table 1, I display the same results as those published in Table 1 of Adams et al. (2006) article. The results in column (1) support the authors’ argument that niche parties are less responsive than mainstream parties to change in public opinion. The coefficient of *public opinion shift *(0.97) is positive and statistically significant indicating that when public opinion is moving to the left (right) mainstream parties (niche party=0) adjust their policy positions accordingly to the left (right). The coefficient of *public opinion shift X niche party *indicates that niche parties are less responsive than mainstream parties to shift in public opinion by -1.52 points on the left-right scale and the difference is statistically significant (p<0.01).

In column (2) of Table 1, I display the results of the alternative approach. The specification is now the following:

∆*party position *= *b*0 +*b*1*niche party*+*b*2(∆*public opinionXniche party*)+*b*3(∆*public opinionXmainstream party*)+*controls*

* *where *mainstream party *equals (1 *− niche party*).

It is important to highlight that the results in columns (1) and (2) are mathematically equivalent. For example, the coefficients of the control variables are exactly the same in both columns. There are some differences, however, in terms of the interpretation of the interaction effect. In column (2), the coefficient of *public opinion shift – mainstream party *(0.97) equals the coefficient of *public opinion shift *in column (1). This is because *public opinion shift – mainstream party *in column (2) indicates the impact of a change in public opinion on the positions of mainstream parties as it is for *public opinion shift *in column (1). On the other hand, the coefficient of *public opinion shift – niche party *in column (2) equals -0.55 and is statistically significant at the 0.05 level. This indicates that when public opinion is moving to the left (right) niche parties adjust their policy positions in the opposite direction to the right (left). This result is not explicitly displayed in column (1) when using the standard approach. The coefficient of *public opinion shift – niche party *in column (2) equals actually the sum of the coefficients of *public opinion shift *and *public opinion shift X niche party *in column (1) — i.e. 0.97 + -1.52 = -0.55. In column (2), a Wald-test indicates that the difference of the effects of *public opinion shift – niche party *and *public opinion shift – mainstream party *is statistically significant at the 0.01 level, exactly as indicated by the coefficient of *public opinion shift X niche party *in column (1).

Overall, researchers may choose between two equivalent specifications when one of the modifying variables is discrete in an interaction model: BCG specification which includes all constitutive terms of the interaction and an alternative specification that does not include all constitutive terms of the interaction *explicitly*. Each specification has its advantages in terms of the interpretation of the interaction effect. The advantage of the alternative approach is to present directly the *marginal effects *of an independent variable X on Y for each category of the discrete modifying variable Z. On the other hand, the advantage of BCG approach is to present directly whether the *difference in the marginal effects *of X on Y between the categories of Z is statistically significant. In both specifications, researchers then need to perform an additional test to verify whether the difference in the marginal effects is statistically significant (in the alternative specification) or to calculate the substantive marginal effects under each category of the discrete modifying variable (in the standard specification).

**Notes**

¹Assistant Professor, School of Political Studies, University of Ottawa, 120 University, Ottawa, ON, K1N 6N5, Canada (bferland@uottawa.ca). I thank James Adams, Michael Clark, Lawrence Ezrow, and Garrett Glasgow for sharing their data. I also thank Justin Esarey for his helpful comments on the paper.

² The marginal effect of X in equation 1 is given by *b*_{2} + *b*_{3}*D*.

³ The marginal effect of X in equations 2 and 3 is calculated by *a*_{2}*D *+ *a*_{3}*D*_{0}.

Note also that equation 1 on the left-hand side equals either equation 2 or 3 on the right-hand side:

*b*_{0} + *b*_{1}*D *+ *b*_{2}*X *+ *b*_{3}*XD *+ *ϵ*=*a*_{0} + *a*_{1}*D *+ *a*_{2}*XD *+ *a*_{3}*X*(1 *− D*) + *ϵ *

*b*_{0} + *b*_{1}*D *+ *b*_{2}*X *+ *b*_{3}*XD *+ *ϵ*=*a*_{0} + *a*_{1}*D *+ *a*_{2}*XD *+ *a*_{3}*X − a*_{3}*XD *+ *ϵ*

It is possible then to isolate *XD *on the right-hand side:

*b*_{0} + *b*_{1}*D *+ *b*_{2}*X *+ *b*_{3}*XD *+ *ϵ*=*a*_{0} + *a*_{1}*D *+ *a*_{3}*X *+ (*a*_{2} *− a*_{3})*XD *+ *ϵ*

Assuming that the models on the left-hand side and right-hand side are estimated with the same data *b*_{0} would equal *a*_{0}, *b*_{1} would equal *a*_{1}, *b*_{2} would equal *a*_{3} (i.e. the estimated parameter of X(1-D)), and *b*_{3} would equal (*a*_{2} *− a*_{3}).

**References**

Adams, James, Michael Clark, Lawrence Ezrow and Garrett Glasgow. 2006. “Are Niche Parties Fundamentally Different from Mainstream Parties? The Causes and the Electoral Consequences of Western European Parties’ Policy Shifts, 1976-1998.” *American Journal of Political Science *50(3):513–529.

Brambor, Thomas, William Roberts Clark and Matt Golder. 2006. “Understanding Interaction Models: Improving Empirical Analyses.” *Political Analysis *14:63–82.

Wright, Gerald C. 1976. “Linear Models for Evaluating Conditional Relationships.” *American Journal of Political Science *2:349–373.

“Redefine statistical significance,” a paper recently published in *Nature Human Behavior* (Benjamin et al., 2017) generated a substantial amount of discussion in methodological circles. This paper proposes to lower the threshold for statistical significance from the conventional level of to a new, more stringent level of and to apply this threshold specifically to newly discovered relationships (i.e., relationships that have not yet been demonstrated in multiple studies). This proposal touched off a debate about the effect null hypothesis significance testing (NHST) has on published work in the social and behavioral sciences in which many statisticians and social scientists have participated. Some have proposed alternative reforms that they believe will be more effective at improving the replicability of published results.

To facilitate further discussion of these proposals—and perhaps to begin to develop an actionable plan for reform—the International Methods Colloquium (IMC) hosted a panel discussion on “reproducibility and a stricter threshold for statistical significance” on October 27, 2017. The one-hour discussion included six panelists and over 240 attendees, with each panelist giving a brief initial statement concerning the proposal to “redefine statistical significance” and the remainder of the time being devoted to questions and answers from the audience. The event was recorded and can be viewed online for free at the International Methods Colloquium website.

Unfortunately, the IMC’s time limit of one hour prevented many audience members from asking their questions and having a chance to hear our panelists respond. Panelists and audience members alike agreed that the time limit was not adequate to fully explore all the issues raised by Benjamin et al. (2017). Consequently, questions that were not answered during the presentation were forwarded to all panelists, who were given a chance to respond.

The questions and answers, both minimally edited for clarity, are presented in this article. The full series of questions and answers (and this introduction) are embedded in the PDF below.

]]>