Thursday, 21 May 2015

Smack you in the eye test

There was some debate in comments yesterday about the use of the word "trend".

To a clinician it may mean " Maybe I can infer something positive from the study as its a near miss , to give a hint of finding something useful out of a negative study".

To a basic scientist it should mean "this data says nothing really and I should have done a bigger more definitive study and I need to do it again". 

To an MSer it probably means nothing useful will come of this as the regulators aren't going to be impressed. 

Statistics can be used to show many things but when you look at any form of data first try the: Smack you in the eye test...does the data jump out at you and tell you something or or do you have to squint to see anything. Are the deviations in the graph massive so there are big overlaps  telling you there is not much real difference. When they not shown is this because the deviations were massive. 

So if you are having to squint (look at the data is rose-coloured glasses) to see something, then maybe think the smack you in the eye test is telling you that the story will turn out to be a pile of pants:-) 

P.S. Having good clear negative results is positive data as it still informs, just as a good clear positive results informs

3 comments:

  1. I always find this little piece helpful, particularly when applied to clinical/pharma data ;-)

    New Definitions for Neuroscience: Data-fracking

    Data-fracking is the fracturing of a dataset by the application of a pressurized search through as many datapoints as is necessary to find one that has the appearance of being positive. Typically, data is combined with cortisol and caffeine, and the mixture is injected at high pressure into statistical analyses to illuminate illusory correlations along which the appearance of success may migrate to a press release. The products of data-fracking tend to be more gaseous than substantive.

    http://neurogram.wordpress.com/2013/08/27/new-definitions-for-neuroscience-data-fracking/

    ReplyDelete

Please note that all comments are moderated and any personal or marketing-related submissions will not be shown.