Thursday, March 5, 2015

Just Tell Us What Is Proper and Appropriate: Thoughts on the Upcoming ASA Statement


These are great times to be a methodologist.  There is a crisis in reproducibility; there is a loss of confidence across several domains; and standard are being debated and changing.  It is a time of flux, innovation, and chaos.  None of us is quite sure how it is going to shake out.  

The most recent installment in this drama is the banning of significance tests in a social-psychology journal.  This banning has received much attention, and the comments that has caught my eye are those from American Statistical Association's (ASA) Ronald Wasserstein.  In it, Wasserstein reports that a group of more two dozen experts is developing an official ASA response.  Until then, ASA "encourages journal editors and others... [to] not discard the proper and appropriate use of statistical inference." (emphasis added)  ASA's statement provided me an opportunity to more closely examine the relationship between scientists and statisticians.  Here is my message to ASA:

We psychologists tend to put you statisticians on a pedestal as experts who tell us what to and what not to do.  You statisticians should not welcome being put on this pedestal for there is a dark side.  What we psychologists tend to do is cleave off analysis from the other parts of the research process.  Whereas we see our research overall as perhaps creative and insightful, many of us see analysis as procedural, formulaic, hard, and done to meet sanctioned standards.  In this view, we psychologists are shifting our responsibilities to you statisticians.  We ask you to sanction our methods, to declare our calculations kosher, to absolve from being thoughtful, judicious, transparent, and accountable in analysis.  My sense is that this transfer of responsibility is as rampant as it is problematic.  It results in a less transparent, more manufactured, more arbitrary psychological science. 

When ASA weighs in, it should be mindful of the current pedestal/responsibility dynamic.  ASA should encourage researchers to be responsible for their analysis---that is, to justify them without recourse to blind authority or mere convention.  ASA should encourage thoughfulness rather than adherence.  Telling us which options are *proper and appropriate* won't do.  Promoting repsonsiblity and thoughtfulness, however, seems easier said than done.  Good luck.  Have fun.  Knock yourselves out.

Perhaps my experience is helpful.  Statisticians add immeasurable value by helping me instantiate competing theories as formal models.  These models imply competing constraints on data.  We work as a team in developing a  computationally-convenient analysis plan for assessing the evidence for the models from the data.  And if we do a good job instantiating theories as models, then we may interpret our statistical inference as inference about the theories.  In the end we share responsibility as a team for understanding how the constraint in the models is theoretically relevant, and how patterns in the data may be interpreted in theoretical terms. 

2 comments:

pg1dpu said...

Maybe they should tell us that if we use statistics for our research we should know that we are doing research on statistics. This is, our methods and statistical models are part of our research responsibility

Tendaxes Hobisons said...

Typically the manufacturing connected with an electric utility sit back and watch turned a possible chance on account of the actual miniaturization for computer pieces as well as the benefits associated with modest electric battery. Different watches at the same time attempted to are the initially promote. That keep an eye on discharged the initial you inside fresh fashion. Nevertheless it really had been harried to plug as a consequence of frights about devices publishing one before them. Accordingly, it turned out affected utilizing excellence issues in the first place, which in turn destroyed its repute. Source ended up being using the electricity liaisons which shortly was wearing away and additionally have been hard automobile. A see obtained quite a few global components of which created these attractive to the public. Given that they are driven utilizing a tuning branch, some people hummed in place of ticked not to mention was the initial of having sweeping moments fingers. Devices styles confirmed the actual highly special together with unconventional tuning division mechanism throughout. Nonetheless of course, having the capacity to promise to generally be quite possibly the most legitimate timepiece obtainable www.refinedwatch.com, seemed to be an enormous online marketing benefits on their behalf. By means of 1970 however, this producing has been at the wall membrane for the designer watches. The running watches would hope to cling over a little lengthier with the development belonging to the design which unfortunately merged tuning fork technology having quartz manipulate. These kind of innovations infected the correct way quartz watches grown and also the betterment associated with precision products. The earliest view was initially costly, pricing comparable as a check out when, plus are not extremely trustworthy with a large number of searching for come to be were recalled. Although the quartz genie seemed to be out from the wine bottle, and additionally there was very little returning to college. Nowadays, a good number of different watches really are quartz and tend to be probably the most cost-effective and also genuine looks after obtainable.