Thursday, April 30, 2015

Brainstorming: What Is A Good Data-Sharing Policy?

This blog post is coauthored with Richard Morey.

Many talented and generous people are working hard to change the culture of psychological science for the better.  Key in this change is the call for transparency and openness in the production, transmission, and consumption of knowledge in our field.  And change is happening---the field is becoming more transparent.   It is delightful to see.

To keep up with these changes, I have recently reconfigured how data are curated in my lab.  In that process I examined APA's statement on the ethical obligation of researchers to share data:

8.14 Sharing Research Data for Verification
(a) After research results are published, psychologists do not withhold the data on which their conclusions are based from other competent professionals who seek to verify the substantive claims through reanalysis and who intend to use such data only for that purpose, provided that the confidentiality of the participants can be protected and unless legal rights concerning proprietary data preclude their release. This does not preclude psychologists from requiring that such individuals or groups be responsible for costs associated with the provision of such information. 
(b) Psychologists who request data from other psychologists to verify the substantive claims through reanalysis may use shared data only for the declared purpose. Requesting psychologists obtain prior written agreement for all other uses of the data.

More commentary is provided in APA's 6th Edition Publication Manual (p 12-13):

To avoid misunderstanding, it is important for the researcher requesting the data and the researcher providing the data to come to a written agreement about the conditions under which the data are to be shared.  Such an agreement must specify how the shared data may be used (e.g., for verification of already published results, for inclusion in meta-analytic studies, for secondary analysis).  The written agreement should also include a formal statement about limits on the distribution of shared data....  Furthermore, the agreement should specify limits on the dissemination .... of the results of analyses performed on the data and authorship expectations.

What's Good About This Policy?

The above policy shines a spotlight on verification.  Verification is needed for a healthy psychological science.  Researchers are prone to mistakes, and allowing others to see the reasoning from data to conclusion provides an added layer of integrity to the process.  It also provides at least some expectation that data may be shared, albeit perhaps with stringent limits.

Understanding The Difference Between "Data" and "Interpretation"?

To act ethically, we need to differentiate data from the interpretation of data.  

Data serve as facts, and like facts, are interpreted in the context of theories and explanations.  The data themselves are incontrovertible in as much as the actual values stand on their own.  The interpretation of these facts, whether and how they support certain theories, explanations, and accounts, is not incontrovertible.  These interpretations are just that.  They are creative, insightful, varied, judged, negotiated, personal, etc.   

Who is qualified to interpret these data?  We all feel like we are uniquely qualified to interpret our own data, but we are not.  On the contrary, we should respect the abilities of others to interpret the data, and we should respect that their interpretations may differ from ours.  Our interpretations should derive no special authority or consideration from the fact that we collected the data.  When we collect data, we gain the right to interpret them first, but not last.  

What's Wrong With The Policy

Jelte Wicherts and Marjan Baaker have recently pointed out the flaws in APAs policies.  Their unpublished paper and  commentary in Nature are important reads in understanding the possible ramifications.  

The APA policy does not insure others' rights to independently interpret the data.  The PI explicitly retains sanctioned control of subsequent interpretations.  They may exercise this control by refusing to share without being granted authorship, limiting the scope of use of data, or limiting where the results may appear.  The APA policy serves to privilege the data collectors' interpretation where no such privilege should ethically exist.  

What Does A Good Policy Look Like?

Let's brainstorm this as a community.  As a starting point, we propose the following statement:  Ethical psychologists endeavor to insure that others may independently interpret their data without constraint.  How that should play out?  We welcome your ideas.


Unknown said...

It had been outrageous once an important stainless steel sport activity keep an eye on of which sold with the same selling price to be a sit back and watch as a result of other brandnames The particular parallel concerning the family car and watch is usually amusing, additionally, the phone numbers are generally just about the exact same for each and every. Sad to say, that exquisite interpretation on the see which has been blisteringly rapid, along with especially magnificent, satisfied their collapse. This looks after without doubt would not! It absolutely was therefore beautiful and even sophisticated. This became a wrist watch destined in becoming a iconic representation of the product. While everybody believed your see ended up being attractive, sole a handful of most people extremely realized the benefit for it has the design, and the extraordinary supplier in back of the software. Rapid send to be able to 2015. all the different watches come in vogue use in your prior to when along with ceramic will be the best cloth around. The marketplace is full of running watches produced from plastic-type, precious metal or even manipulated material. Your pieces for seems to have experimented with distinct fabrics as well as such as others adores ceramic for numerous job applications however completely ceramic watches are still exceptional. Because the guide of the extremely first see, they need developed regarding very first system generating a number of models as well as is done who extends his or her accomplish within the market place whilst left over correct for their unique vision- an extravagance sport activity watch for a truly figuring out purchaser. The fundamental keep an eye on has been grown to contain a tourbillion, a chronograph, a new never ending diary, some grand unwanted effect, and even more!. That may purchase a view fully blanketed inside precious stones in case which is any point. Be it for the taste or even in no way, that observe is definitely ready consider a little something brand new, dazzling not to mention audacious. These are dimensions, I really like the actual enjoy that will is placed 48mm great, though alas Ariel by now examined the software, i really los angeles not even. Moreover, I can merely that is amazing your pendant provided a separate within the typic from ideas available on the market, with regard to active anklet bracelets out there. In addition to, when you have received the pendant regulated thoroughly, it should be pleasant in any kind of arm.

Evan said...

Brainstorming isa idea generating technique. Mind maps are a tool used to visualize and organize information. Capturing your free flow of ideas using a mind map during brainstorming will help you quickly make sense of the relationships between the information you come up with. They are also a great way to break down an idea and analyze it. You can draw mind maps with Creately mind mapping software