Discussion on the meeting on ‘Signs and sizes:understanding and replicating statistical findings’

Research output: Contribution to journalComment/debateResearch

Abstract

Two‐sided tests are often used by ‘experts’ as well as ‘non‐experts’ to verify a certain hypothesis. I congratulate Rice, Bonnett and Krakauer for rejuvenating this topic by giving an alternative decision theoretic approach by viewing the signs of an underlying parameter and making a decision based on whether the parameter is positive or negative. The whole procedure looks very complicated and it seems that plenty of subjectivity is involved, which is a little difficult to interpret. For example, how do we set an optimal value of a which minimizes the risk in testing the significance? Killeen (2006) proposed a decision‐theory‐based approach for hypothesis testing which calculates the expected utility of an effect on the basis of the probability of replicating it. However, it seems like it was only a theoretical contribution.

Rice and his colleagues have not given any examples to show how the results may be different from conventional statistical tests or a p‐value approach. Conventional statistical tests look more transparent and can lead to further analysis like confidence intervals, which are lacking in this approach.
Original languageEnglish
Pages (from-to)455
Number of pages1
JournalJournal of the Royal Statistical Society. Series A: Statistics in Society
Volume183
Issue number2
Early online date29 Dec 2019
DOIs
Publication statusPublished - Feb 2020

Fingerprint

statistical test
Statistical test
hypothesis testing
Expected Utility
Hypothesis Testing
subjectivity
Confidence interval
confidence
expert
Verify
Minimise
Calculate
Testing
Alternatives
Rice
Statistical tests

Cite this

@article{429d16ddbe2f483e94c2f391715b0405,
title = "Discussion on the meeting on ‘Signs and sizes:understanding and replicating statistical findings’",
abstract = "Two‐sided tests are often used by ‘experts’ as well as ‘non‐experts’ to verify a certain hypothesis. I congratulate Rice, Bonnett and Krakauer for rejuvenating this topic by giving an alternative decision theoretic approach by viewing the signs of an underlying parameter and making a decision based on whether the parameter is positive or negative. The whole procedure looks very complicated and it seems that plenty of subjectivity is involved, which is a little difficult to interpret. For example, how do we set an optimal value of a which minimizes the risk in testing the significance? Killeen (2006) proposed a decision‐theory‐based approach for hypothesis testing which calculates the expected utility of an effect on the basis of the probability of replicating it. However, it seems like it was only a theoretical contribution.Rice and his colleagues have not given any examples to show how the results may be different from conventional statistical tests or a p‐value approach. Conventional statistical tests look more transparent and can lead to further analysis like confidence intervals, which are lacking in this approach.",
author = "Kuldeep Kumar",
year = "2020",
month = "2",
doi = "10.1111/rssa.12544",
language = "English",
volume = "183",
pages = "455",
journal = "Journal of the Royal Statistical Society. Series A: Statistics in Society",
issn = "0964-1998",
publisher = "Wiley-Blackwell",
number = "2",

}

TY - JOUR

T1 - Discussion on the meeting on ‘Signs and sizes:understanding and replicating statistical findings’

AU - Kumar, Kuldeep

PY - 2020/2

Y1 - 2020/2

N2 - Two‐sided tests are often used by ‘experts’ as well as ‘non‐experts’ to verify a certain hypothesis. I congratulate Rice, Bonnett and Krakauer for rejuvenating this topic by giving an alternative decision theoretic approach by viewing the signs of an underlying parameter and making a decision based on whether the parameter is positive or negative. The whole procedure looks very complicated and it seems that plenty of subjectivity is involved, which is a little difficult to interpret. For example, how do we set an optimal value of a which minimizes the risk in testing the significance? Killeen (2006) proposed a decision‐theory‐based approach for hypothesis testing which calculates the expected utility of an effect on the basis of the probability of replicating it. However, it seems like it was only a theoretical contribution.Rice and his colleagues have not given any examples to show how the results may be different from conventional statistical tests or a p‐value approach. Conventional statistical tests look more transparent and can lead to further analysis like confidence intervals, which are lacking in this approach.

AB - Two‐sided tests are often used by ‘experts’ as well as ‘non‐experts’ to verify a certain hypothesis. I congratulate Rice, Bonnett and Krakauer for rejuvenating this topic by giving an alternative decision theoretic approach by viewing the signs of an underlying parameter and making a decision based on whether the parameter is positive or negative. The whole procedure looks very complicated and it seems that plenty of subjectivity is involved, which is a little difficult to interpret. For example, how do we set an optimal value of a which minimizes the risk in testing the significance? Killeen (2006) proposed a decision‐theory‐based approach for hypothesis testing which calculates the expected utility of an effect on the basis of the probability of replicating it. However, it seems like it was only a theoretical contribution.Rice and his colleagues have not given any examples to show how the results may be different from conventional statistical tests or a p‐value approach. Conventional statistical tests look more transparent and can lead to further analysis like confidence intervals, which are lacking in this approach.

U2 - 10.1111/rssa.12544

DO - 10.1111/rssa.12544

M3 - Comment/debate

VL - 183

SP - 455

JO - Journal of the Royal Statistical Society. Series A: Statistics in Society

JF - Journal of the Royal Statistical Society. Series A: Statistics in Society

SN - 0964-1998

IS - 2

ER -