Psychology

Claims and Evidence: A Joke

The other day a friend posted the following old joke about the level of rigor that mathematicians usually require. (Disclaimer: if you take the joke as a serious claim about the standards of quality in the other fields referenced in the joke, it is an obviously unfair characterization of both astronomy and physics.) A Mathematician, […]

A Variant on "Statistically Controlling for Confounding Constructs is Harder than you Think"

A Variant on “Statistically Controlling for Confounding Constructs is Harder than you Think”

Yesterday, a coworker pointed me to a new paper by Jacob Westfall and Tal Yarkoni called “Statistically controlling for confounding constructs is harder than you think”. I quite like the paper, which describes some problems that arise when drawing conclusions about the relationships between theoretical constructs using only measurements of observables that are, at best, […]

Falsifiability versus Rationalization

Here are two hypothetical conversations about psychological research. I’ll leave it to others to decide whether these conversation could ever take place. Theories are just directional assertions about effects Person A: And, just as I predicted, I found in my early studies that the correlation between X and Y is 0.4. Person B: What do […]

What is Economics Studying?

Having spent all five of my years as a graduate student trying to get psychologists and economists to agree on basic ideas about decision-making, I think the following two pieces complement one another perfectly: Cosma Shalizi’s comments on rereading Blanchard and Fischer’s “Lectures on Macroeconomics”: Blanchard and Fischer is about “modern” macro, models based on […]

A Cheap Criticism of p-Values

One of these days I am going to finish my series on problems with how NHST is issued in the social sciences. Until then, I came up with a cheap criticism of p-values today. To make sense of my complaint, you’ll want to head over to Andy Gelman’s blog and read the comments on his […]

The Social Dynamics of the R Core Team

The Social Dynamics of the R Core Team

Recently a few members of R Core have indicated that part of what slows down the development of R as a language is that it has become increasingly difficult over the years to achieve consensus among the core developers of the language. Inspired by these claims, I decided to look into this issue quantitatively by […]

Criticism 5 of NHST: p-Values Measure Effort, Not Truth

Criticism 5 of NHST: p-Values Measure Effort, Not Truth

Introduction In the third installment of my series of criticisms of NHST, I focused on the notion that a p-value is nothing more than a one-dimensional representation of a two-dimensional space in which (1) the measured size of an effect and (2) the precision of this measurement have been combined in such a way that […]

Criticism 3 of NHST: Essential Information is Lost When Transforming 2D Data into a 1D Measure

Criticism 3 of NHST: Essential Information is Lost When Transforming 2D Data into a 1D Measure

Introduction Continuing on with my series on the weaknesses of NHST, I’d like to focus on an issue that’s not specific to NHST, but rather one that’s relevant to all quantitative analysis: the destruction caused by an inappropriate reduction of dimensionality. In our case, we’ll be concerned with the loss of essential information caused by […]

Criticism 2 of NHST: NHST Conflates Rare Events with Evidence Against the Null Hypothesis

Introduction This is my second post in a series describing the weaknesses of the NHST paradigm. In the first post, I argued that NHST is a dangerous tool for a community of researchers because p-values cannot be interpreted properly without perfect knowledge of the research practices of other scientists — knowledge that we cannot hope […]

Criticism 1 of NHST: Good Tools for Individual Researchers are not Good Tools for Research Communities

Introduction Over my years as a graduate student, I have built up a long list of complaints about the use of Null Hypothesis Significance Testing (NHST) in the empirical sciences. In the next few weeks, I’m planning to publish a series of blog posts, each of which will articulate one specific weakness of NHST. The […]