This semester I’m TA’ing a course on the Psychology of Music taught by Phil Johnson-Laird. It’s been a great course to teach because (i) so much of the material is new to me and (ii) because the study of the psychology of music brings together so many of the intellectual tools I enjoy, including music theory, psychophysics and Fourier analysis.

One topic this semester that was completely new to me was the theory of tuning: I had known about the invention of the well-tempered system of tuning, but had never heard of Pythagorean tuning or just tuning — and certainly was not aware that the well-tempered system Bach celebrated was not identical to our current equal-tempered system of tuning.

As a way of consolidating some of the knowledge I’ve gained, I decided I’d write a blog entry after several months of neglecting this blog. (For that neglect, I’ll blame a combination of grant writing, book writing, ongoing research projects and personal life developments.) In what follows, I’ll give a brief overview of the theory of tuning at a theoretical level that should be accessible to anyone who’s familiar with the names of intervals and feels comfortable thinking quantitatively.

After surveying the field, I’ll turn to a discussion of some code I’ve written in R that implements these ideas using the ‘tuneR’ package, which is one of my favorite hidden gems from CRAN. Along the way, I’ll introduce some of the simplest tools from the ‘tuneR’ package that can be used for generating computer music.

Tuning Systems: Pythagorean, Just and 12-Tet

It’s worth noting right at the start that tuning is a misleading name for the topic we’ll be discussing: we’re not talking about how one tunes a fixed instrument so that it sounds in tune, but rather we’re interested in how one defines the very notes that the instrument should be able to produce when it’s perfectly in tune.

To make that clear, let’s assume that we’ve accepted as a given that a frequency of 440 Hz will be called A. Our problem then becomes one of deciding which of the infinitely many frequencies we could produce actually deserves the label of A#, B, C, C#, and so on.

Pythagorean Tuning

The simplest solution to this problem I know of is the Pythagorean tuning system. It’s based on constructing all of the possible notes using a series of perfect fifths. If you remember the Circle of Fifths, you’ll remember that you can reach every chromatic note by ascending fifths: if you start at A, you’ll proceed through E, B, F# and so on.

The Pythagorean system implements the Circle of Fifths directly using repeated multiplication of a base frequency. To do this, you first declare that a perfect fifth is at a frequency 3/2 above your base frequency. For example, this definition implies that the perfect fifth above the A at 440 Hz has to be at a frequency of 3/2 * 440 = 660 Hz. Once you do this, you’ve defined the frequency we’ll call E.

And following on with this logic, you produce a B at 990 Hz. Of course, this B occurs an octave above the base A at 440 Hz, so you transpose it down an octave to produce the B you’ll actually use. To do this, you need to assume that an octave is at a frequency 2 times the base frequency. Since we’ve accepted that 990 Hz is a B, we divide 990 by 2 and conclude that 495 Hz should be B.

With these three notes defined, we have the following table of frequency/note pairs:

If we continue on with this logic and calculate many more multiplications by 3/2 and divisions by 2, we will eventually produce a complete table for all of the notes in the chromatic scale that looks like the following:

One thing about this table might strike you as odd if you’re mathematically savvy: the octave, which we’ve defined by fiat as a ratio of 2:1, could never have been produced by successive multiplication by 3/2, since no power of 3 will be evenly divisible by a power of 2. This is the one flub in the Pythagorean system: you can’t really produce the entire chromatic scale using only multiples of 3/2. Here we’ve solved that problem by replacing the note we would have called A with a true octave generated using multiplication by 2. Because the exact octave produced by Pythagorean tuning is slightly out of tune with our preferred definition of an octave, you may hear people refer to this discrepancy as the the Pythagorean comma.

Just Tuning

Given that we had to cheat a bit to create a proper octave using the Pythagorean tuning system based on multiples of 3/2, it makes sense to ask why we shouldn’t just allow ourselves to use other multipliers than 3/2. Looking at the Pythagoren tuning table, we see some pretty ugly fractions like 729/512. What if we forced these fractions to be simpler by employing ratios like 4/3 and 5/4 to build up the whole system?

The result of allowing ourselves several fractions beyond just those derived from 3/2 is called the just tuning system. Here we assume that perfect fifths occur at a frequency ratio of 3/2 and that perfect fourths occur at a frequency ratio of 4/3. Continuing on with this process, we eventually end up with the following tuning table:

This is the tuning that early Classical music was written in. Looking at the table you con immediately appreciate the theoretical assertion that the relative dissonance of an interval is determined by the simplicity of the ratio of frequencies between the two notes: perfect fifths are 3/2 and major thirds are 5/4, while minor seconds are 16/15 and major sevenths are 15/8. This is one of the things I most enjoy about the theory of harmony: there’s a match between the aesthetics of fractions and the aesthetics of sounds that, for me, helps to justify my sense that certain fractions are more beautiful than others.

12 Tet / Equal-Temperament

Now, if you know the history of Bach’s Well-Tempered Clavier, you know that there is a problem with the just tuning system: it sounds great in the key you used as the base (here A), but it sounds a bit out of tune in other keys. The modern 12-tet system is the most recent approach to solving this problem: you assume the gap between two semitones (e.g. A to A# or A# to B) is always the exact same multiple. Since you’ll repeat this multiplication 12 times before reaching an octave, you can conclude that two notes that are a semitone apart must be separated by the 12th root of 2. Building a tuning system using that ratio alone gives us our modern system of tuning, which is shown in the table above using the decimal expansion of the ratios instead of their representation as powers of the 12th root of 2:

Listening to the Results

We’ve just described three ways to define the notes used in Western music. But how different do they sound? To answer that, I decided to produce a series of simple sine wave audio samples that were tuned using each of the three tuning systems. To produce those audio samples, I used the ‘tuneR’ package, which I’ll describe now. Before you read on, you should install it from CRAN using the standard install.packages('tuneR') invocation.

A tuneR Tutorial

The tuneR package is an extremely convenient tool for generating audio files from R based on a numeric description of the audio stream. For the purposes of this discussion of tuning systems, we simply need to produce basic sine waves. Thankfully, that’s very easy to do with tuneR. Here’s an example:

1
2
3
4
5

library('tuneR')
 
sound <- sine(440, bit = 16)
 
writeWave(sound, '440.wav')

Here we’ve loaded the tuneR package, created a 1s snippet of sine wave audio at 16 bits resolution using the sine function, and then written out the audio to a WAV file using writeWave. If you look at your current directory and listen to this file, you’ll hear a sine wave at 440 Hz.

If you want to explore the use of sine, you can easily play with the duration of the sound by changing the duration parameter. If you want to, you can also change the sample rate and the bit rate, but I don’t see any reason to do that while exploring ideas about tuning.

More important is knowing that you can superimpose two sine waves using the `+` operator and that you can concatenate them using the bind function. To show off producing octaves, for example, you might use the following code to hear an A at 440 Hz, then an A an octave above it, and finally the harmony they produce together:

1
2
3
4
5
6
7

library('tuneR')
 
sound <- bind(sine(440, bit = 16),
              sine(880, bit = 16),
              sine(440, bit = 16) + sine(880, bit = 16))
 
writeWave(sound, 'octaves.wav')

Unfortunately, this sample code produces an error because of the naive addition we’ve implemented using the `+` operator. Adding two sine waves directly together overfills the bit rate we’re using. To safely perform addition of two sine waves, we need to normalize the results of our summation using the normalize function. This gives us just one more line of code:

1
2
3
4
5
6
7
8
9

library('tuneR')
 
sound <- bind(sine(440, bit = 16),
              sine(880, bit = 16),
              sine(440, bit = 16) + sine(880, bit = 16))
 
sound <- normalize(sound, unit = '16')
 
writeWave(sound, 'octaves.wav')

For reasons that are not clear to me, you have to specify the bit rate to normalize using the unit parameter rather than the bit parameter.

Demoing Tuning Systems

Our little octave demo is cute, but we really want to know what more interesting harmonies like major thirds and minor seconds sound like in the various tuning systems we described. To do that, I first wrote a function called interval that spits out the multiplier you need to use to produce a given interval for any of the three tuning systems. That function is in a GitHub repository I’ve set up with code for making these demos. If you download that repository, you could load my interval function using a simple call to source like the one seen below. And using this interval function, we can generate demos of various intervals as follows:

1
2
3
4
5
6
7
8
9
10
11

library('tuneR')
source('interval.R')
 
base <- 440
 
sound <- sine(base) + sine(interval('minor-second',
                                    tuning = 'pythagorean') * base)
 
sound <- normalize(sound, unit = '16')
 
writeWave(sound, 'minor_second_pythagorean.wav')

On GitHub there’s a file called test_intervals.R that will go through and generate all of the intervals in all three tuning systems. If you run that file, you’ll generate a lot of audio files you can listen to as demos of the three tuning systems we’ve described. For me, these tuning systems all produce intervals that sound surprisingly similar, though at high volumes I find it moderately easy to hear slight differences between the tuning systems. That said, I very much doubt I would pick up on them in a normal musical context.

That’s the end of my little introduction to tuning systems and the use of the tuneR package to explore them. If you’re interested in thinking computationally about music, I highly recommend playing around with tuneR until you feel like you can produce interesting results. I’m already working on trying to build up some interesting timbres to work with.

Show that every number of the form ABC,ABC is divisible by 13.

The puzzle comes up in Baron’s book as an example of an “insight problem” in which one goes from not knowing the answer at all to knowing the complete answering in a sudden moment of insight.

Several people replied to my tweet with solutions: I especially like Will Townes’s solution. In particular, if you’re familiar with modular arithmetic, I like the logic of Will’s answer because it gives a simple generalization. First, represent ABC,ABC as ABC * 1000 + ABC * 1 rather than as ABC * 1001. Then notice that

1. 1 = 1 mod 13
2. 1000 = -1 mod 13

Thus ABC,ABC = ABC * -1 + ABC * 1 = 0 mod 13. This logic can be easily extended to show that (ABC,ABC,)*ABC,ABC = 0 mod 13 no matter how many times you repeat the ABC,ABC pattern.

In the process, I’ve started to learn something about how to write code that runs quickly in R. In this post, I’ll try to share some of that knowledge by describing three ways of performing maximum likelihood estimation in R whose runtimes differ by two orders of magnitude. The differences seem to depend upon two factors: (1) how I access the entries of a data frame and (2) whether I use loops or vectorized operations to perform basic arithmetic.

To simplify things, I’ll present a model that should be familiar to people with a background in economics: the exponentially discounted utility model. To implement it in R, we define the discounted value of x dollars at time t as:

1
2
3
4

discounted.value <- function(x, t, delta)
{
  return(x * delta ^ t)
}

In addition to the discounted utility model, we assume that choices originate from a stochastic choice model with logistic noise. To invert this noise during inference, we’ll use the inverse logit transform:

1
2
3
4

invlogit <- function(z)
{
  return(1 / (1 + exp(-z)))
}

To test my inference routine, I need to generate “stochastic” data of the sort you would expect to see from an exponentially discounting agent that’s indifferent between having $1 at time t = 0 and $3 at time t = 1. I’ll refer to the first good as (X1, T1) and the second good as (X2, T2). If the agent chooses (X2, T2), I’ll write that as C == 1; if they choose (X1, T1), I’ll write that as C == 0. With those conventions, the sample data is generated as:

1
2
3
4
5
6
7

n <- 100
 
choices <- data.frame(X1 = rep(1, each = n),
                      T1 = rep(0, each = n),
                      X2 = rep(3, each = n),
                      T2 = rep(1, each = n),
                      C = rep(c(0, 1), by = n / 2))

To fit the exponential model to this data set, we’ll use the optim function to minimize the negative log likelihood of the data by setting two parameters: a, the variance of the noise in the utility function; and delta, the discount factor in the discounted utility model. The three implementations of this model that I’ll show only differ in the definition of the log likelihood function, so the final call to optim to perform maximum likelihood estimation is constant across all examples:

1
2
3
4
5
6

logit.estimator <- function(choices)
{ 
  wrapper <- function(x) {-log.likelihood(choices, x[1], x[2])}
  optimization.results <- optim(c(1, 1), wrapper, method = 'L-BFGS-B', lower = c(0, 0), upper = c(Inf, 1))
  return(optimization.results$par)
}

Here, I had to specify bounds for the parameters, a and delta, because it’s assumed that a must be positive and that delta must lie in the interval [0, 1]. To deal with these bounds, one has to use the L-BFGS-B method in optim.

The first implementation I’ll show is the one I find most natural to write, even though it turns out to be the least efficient by far:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

log.likelihood <- function(choices, a, delta)
{
  ll <- 0
 
  for (i in 1:nrow(choices))
  {
    u2 <- discounted.value(choices[i, 'X2'], choices[i, 'T2'], delta)
    u1 <- discounted.value(choices[i, 'X1'], choices[i, 'T1'], delta)
 
    p <- invlogit(a * (u2 - u1))
 
    if (choices[i, 'C'] == 1)
    {
      ll <- ll + log(p)
    }
    else
    {
      ll <- ll + log(1 - p)
    }
  }
 
  return(ll)
}

In the second implementation, I define a row level likelihood function, so that the summing and logarithmic transform are vectorized.

1
2
3
4
5
6
7
8
9
10
11
12
13

rowwise.likelihood <- function(row, a, delta)
{
  u2 <- discounted.value(row['X2'], row['T2'], delta)
  u1 <- discounted.value(row['X1'], row['T1'], delta)
  p <- invlogit(a * (u2 - u1))
  return(ifelse(row['C'] == 1, p, 1 - p))
}
 
log.likelihood <- function(choices, a, delta)
{
  likelihoods <- apply(choices, 1, function (row) {rowwise.likelihood(row, a, delta)})
  return(sum(log(likelihoods)))
}

In the third implementation, I define a fully vectorized log likelihood function that avoids any explicit iteration and therefore removes most of the data frame indexing operations:

1
2
3
4
5
6
7
8

log.likelihood <- function(choices, a, delta)
{
  u2 <- discounted.value(choices$X2, choices$T2, delta)
  u1 <- discounted.value(choices$X1, choices$T1, delta)
  p <- invlogit(a * (u2 - u1))
  likelihoods <- ifelse(choices$C == 1, p, 1 - p)
  return(sum(log(likelihoods)))
}

The code I used to call all of these implementations and compare them is up on GitHub for those interested. The results, which strike me as remarkable, are below:

1. On my laptop, implementation 1 takes ~1.0 second to run.
2. On my laptop, implementation 2 takes ~0.25 seconds to run.
3. On my laptop, implementation 3 takes ~0.01 seconds to run.

In short, the third implementation is 100x faster than the first implementation with only minor changes to the code I originally wrote. Hopefully this example will help inspire others who have R code they’d like to speed up, but aren’t sure where to start.

But modeling devices that make sense for an unbiased decisionmaker may not make sense for a biased one. For example, why would individuals have priors and posteriors if they are destined to apply Bayes’ law incorrectly?¹

A question I often ask myself.

1. Wolfgang Pesendorfer : Behavioral Economics Comes of Age: A Review Essay on Advances in Behavioral Economics

1. Probabilists and Statisticians: “I think that’s an interesting case, but it’s in a set with measure zero.”
2. Economists: “X group’s behavior is clearly rent-seeking.”

Do any readers have good examples from other fields?

To convince non-psychologists that response times are a quantity worth measuring, I’ll give an example from my own work that I suspect is widely generalizable. I consistently find that the quality of data I can extract from Mechanical Turk HIT’s grows considerably if I simply weight each person’s responses by a function of their time to complete the HIT. For one recent task, the histogram of task completion times looked like the following:

As you can obviously see, there are two noticeable clusters of participants for this HIT: a substantial chunk who spent shockingly little time on the task and a large majority of people who spent a longer and more variable period of time performing the task. In this case, completely excluding those who completed this specific task within ten seconds improved the results of my later analyses considerably at virtually no cost to myself. Other approaches, like using k-means clustering, mixture modeling or simple quantile cut-offs, can be equally useful. In general, response times are a useful proxy for the amount of effort a person puts into a task.

Hopefully that example convinces you that reaction times are worth studying carefully. If you’re interested, I strongly recommend Luce’s book as an introduction to the use of response times to infer information about the mental state of specific human beings and about the global architecture of the human mind in general. It’s quite long, so many people would probably gain the most from only reading the first few chapters.

In the rest of this post, I’m going to discuss a detail about the measurement of response times that I found particularly beautiful and insightful. Specifically, I was struck by a point that Luce raises in the second chapter of his book: the proper measurement of response times requires the use of an experimental design that employs the exponential distribution to produce a Poisson process of stimuli. That’s probably opaque to most readers, so let me unpack it:

1. Our Objective: We want to estimate the speed with which a normal person can respond to a stimulus.
2. Our High-Level Method: We present subjects with a train of simple sensory stimuli, like 10 dB tones, and ask them to respond as quickly as possible to each by pressing a button. We record the time from the onset of the tone to the moment when they press the button.
3. Our Challenge: The length of time before a stimulus is presented must be randomly distributed, so that subjects cannot adopt a fixed strategy for timing their responses. More importantly, we must insure that the mere act of waiting for a stimulus is not itself informative: the subject should not be able to notice that the arrival of a stimulus is becoming progressively more or less likely as the time they’ve been waiting grows. This means that a naive sampling distribution for waiting times before presenting a stimulus, like the uniform distribution, is simply not acceptable.
4. Our Solution: We calculate a measure of the informativeness of each moment of waiting: the measure we use is called the hazard rate and can be calculated for any probability distribution. Using this metric of informativeness, we simply need to find a probability distribution that has a constant hazard rate. The distribution we arrive at is the exponential distribution. A train of stimuli separated by exponentially distributed waiting periods is called a Poisson process. It allows us to treat all of the instants within a single trial’s waiting period as comparable.

For me, this was a revelatory example that finally made the exponential distribution’s relevance clear. Understanding this point made me understood intuitively, rather than simply algebraically, why Luce says this in the first chapter:

[We] see that the answer is [...] the exponential distribution. This may come as a bit of a surprise since, after all, the largest value of the exponential value is at 0 — instantaneous repetition — which must tend to produce clustering if the process is repeated in time. This is, indeed, the case, as can either be seen, as in Figure 1.3, or heard if such a process is made audible in time — it seems to sputter. To neither the eye nor the ear does it seem uniform in time, but that is an illusion, confusing what one sees or hears with the underlying rate of these events. The tendency for the event to occur is constant. Such a process is called Poisson — more of it in Chapters 4 and 9. It is the basic concept of constant randomness in time — that is, the unbounded, ordered analogue of a uniform density on a finite, unordered interval.

First, let’s assume the following things about our idealized human being:

1. They possess cyclical preferences where A > B, B > C and C > A.
2. Their derived pleasure from experiencing A, B or C is based on an adaptive scale, i.e. they derive pleasure from having something better than their local average consumption.

Under these assumptions, the best possible consumption stream is an ordered walk through the cycle of their preferences, such as (A, B, C, A, B, C, …). Assuming that their adaption is sufficiently local, they will able to continually derive pleasure from moving along the preference chain, despite the hedonic treadmill slowly eroding the gains made from any single movement along the preference graph.

The big idea here is that cyclical preferences, which are generally assumed to lower welfare, can be exploited to increase welfare for agents that suffer from a hedonic treadmill. Since there is some reason to believe in both violations of classical theory, this is potentially one trick for improving one’s own well-being.

Professional bookmakers rarely accept bets from individuals who directly control the outcome of the bet. We analyse a unique exception to this rule and a potential policy innovation in the battle against obesity: a weight loss betting market. If obese individuals have time-inconsistent preferences then commitment mechanisms, such as personal gambles, should help them restrain their short-term impulses and lose weight. Correspondence with the bettors confirms that this is their primary motivation. However, it appears that the bettors in our sample are not particularly skilled at choosing effective commitment mechanisms. Despite payoffs of as high as $7350, approximately 80% of people who spend money to bet on their own behaviour end up losing their bets.¹

HT Cheap Talk. Reading this makes me glad to have finished my last Master’s degree requirement today by turning in a review of the literature on intertemporal choice.

1. Nicholas Burger and John Lynham : Betting on weight loss . . . and losing: personal gambles as commitment mechanisms

In my mind, Paul Meehl was one of the greatest psychologists of the 20th century. Few other people in our field were more level-headed or insightful. Every paper written by Meehl is a delight to read, so this is a veritable treasure trove of material.

Of course, there is “believe,” and then there is believe. The same people who don’t believe in intelligence are proud of their GRE scores, convinced that Republicans and religious people have lower IQ’s, and outraged when the mentally deficient, as measured on IQ tests, are executed. This probably reflects some mental modularity. People might say they don’t believe in IQ, but the decisions they make are to some extent informed by the assumption that intelligence exists, and individuals vary.¹

While I frequently insist that measuring intelligence must, at least in principle, be a meaningful activity because all we do it implicitly in using words like intelligent or stupid to describe people — about which we are all in surprisingly good agreement –, I’d never thought of these very clear examples of hypocrisy in rejecting IQ only when it doesn’t serve one’s interests and prejudices.

1. Razib : Gene Expression : IQ and “conventional wisdom”