how to response reviewer's comments about the low threshold of Z > 2.3

Submitted by yaqiong on Tue, 07/07/2015 - 13:56

Dear experts,

I have an urgent question to ask you for help. In our recent study, we investigated the relationship between resting-state functional connectivity and behavior performance, and the results were corrected for multiple comparisons using Gaussian Random Field theory (Z > 2.3, cluster-wize p < 0.05, two-tailed). However, the reviewer commented 'It is inappropriate to use extremely low primary thresholds (i.e., Z > 2.3, p < 0.01 for one-tailed tests) for a cluster-wise correction. The initial threshold for cluster-wise correction should be p < 0.001 (Woo et al., 2014, NeuroImage), which corresponds to Z > 3.3 for two-tailed tests. ' 'The justification of using the low thresholds is still needed. Previous papers with such low thresholds do not answer this critical issue.'

I noticed that Shen et al. (2015) cited Woo et al., (2014) for the correlation analysis, but I think the threshold is still p < 0.05 rather than p < 0.001 based on the statement in the paper (‘The Alpha-Sim procedure was used to account for multiple comparison issues by combining a height P < 0.001 and an extent P < 0.05 [Woo et al., 2014].’), isn't it?

Could you please give me some suggestion on how to reply this comment properly? Thanks so much.

Best,
Yaqiong

Forums:

YAN Chao-Gan

Wed, 07/08/2015 - 06:54

Hi Yaqiong,

1. "by combining a height P < 0.001 and an extent P < 0.05" means they set individual p at 0.001, e.g., z > 3.3.

2. You can argue with the reviewer that you admit setting individual p < 0.001 could increase spatial specificity as compared to p < 0.01. However, the thresholded clusters are more scattered and impacted by spatial discrepancy across subjects (e.g., imperfect registration). When repeating the analysis on another separate group, the clusters are less likely to be overlapped (even they are significant after correction). By ensuring p < 0.05 after correction, setting individual p < 0.01 is not necessary looser than setting individual p < 0.001, as the extent threshold increases accordingly. The two strategies have different emphases (spatial specificity or repeatability).

Hope this helps.

Best,

Chao-Gan

yaqiong

Fri, 07/10/2015 - 07:38

Hi Chao-Gan,

Thanks so much.  It is greatly helpful.

Best,

Yaqiong