Signal Detection Analysis

Signal Detection Analysis

Purpose

This skill encodes expert methodological knowledge for applying Signal Detection Theory (SDT) to behavioral and cognitive science data. SDT separates an observer's perceptual sensitivity from their decision criterion -- a distinction that raw accuracy conflates. A competent programmer without cognitive science training would typically compute percent correct, missing the critical insight that two observers with identical accuracy can differ drastically in their ability to detect signals vs. their willingness to say "yes."

When to Use SDT (Not Simple Accuracy)

Use SDT whenever:

• Stimuli belong to two classes (signal vs. noise, old vs. new, present vs. absent) and the observer makes a binary classification
• You need to distinguish how well someone can discriminate (sensitivity) from how willing they are to respond in a particular way (bias/criterion)
• Response bias may differ across conditions, groups, or time points, making raw accuracy misleading
• You want a measure that is independent of base rates and payoff structures

Do not use standard SDT when:

• There are more than two stimulus classes (use multi-class extensions or confusion matrices)
• Responses are continuous rather than categorical (use regression-based approaches)
• The task has no noise distribution (e.g., simple threshold detection with catch trials absent)

Research Planning Protocol

Before executing the domain-specific steps below, you MUST:

1. State the research question — What sensitivity or bias question is this SDT analysis addressing?
2. Justify the method choice — Why SDT (not simple accuracy, logistic regression, etc.)? What alternatives were considered?
3. Declare expected outcomes — Do you expect sensitivity differences, bias differences, or both?
4. Note assumptions and limitations — What does SDT assume (e.g., Gaussian distributions, equal variance)? Where could it mislead?
5. Present the plan to the user and WAIT for confirmation before proceeding.

For detailed methodology guidance, see the research-literacy skill.

⚠️ Verification Notice

This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please open an issue.

Core Concepts

The 2x2 Response Matrix

Every SDT analysis begins with classifying each trial into one of four categories:

From these four cells, compute two rates:

• Hit Rate: H / (H + M) = proportion of signal trials correctly identified
• False Alarm Rate: FA / (FA + CR) = proportion of noise trials incorrectly called "signal"

Sensitivity: d' (d-prime)

d' measures the distance between the signal and noise distributions in standard deviation units, assuming equal-variance Gaussian distributions (Green & Swets, 1966, Ch. 1):

d' = z(Hit Rate) - z(False Alarm Rate)

where z() is the inverse of the standard normal CDF (the z-transform).

What d' values mean in practice (Macmillan & Creelman, 2005; Table 1.1):

The typical experimental range avoiding floor/ceiling effects is d' = 0.5 to 2.5 (Macmillan & Creelman, 2005).

Bias Measures: c, beta, and c'

SDT provides three interchangeable bias measures. The choice matters when d' varies across conditions.

Criterion location c (Macmillan & Creelman, 2005, Ch. 2):

c = -0.5 x [z(Hit Rate) + z(False Alarm Rate)]

• c = 0: unbiased (optimal for equal base rates and symmetric payoffs)
• c > 0: conservative (tendency to say "no" / fewer false alarms, fewer hits)
• c < 0: liberal (tendency to say "yes" / more hits, more false alarms)

Likelihood ratio beta (Green & Swets, 1966, Ch. 1):

ln(beta) = d' x c

• beta = 1: unbiased
• beta > 1: conservative
• beta < 1: liberal

Relative criterion c' (Macmillan & Creelman, 2005, Ch. 2):

c' = c / d'

Normalizes criterion placement by sensitivity; useful when comparing bias across conditions with different d' values.

Which bias measure to use (Macmillan & Creelman, 2005, Ch. 2):

• Use c as the default -- it is statistically independent of d', defined when d' = 0, and symmetric around chance
• Use beta when testing whether observers approximate an optimal likelihood-ratio decision rule (e.g., recognition memory; Stretch & Wixted, 1998)
• Use c' when you need to compare bias across conditions where d' changes substantially

Decision Logic: Choosing a Sensitivity Measure

Is the task a single-interval (yes/no) design?
|
+-- YES --> Are assumptions of equal-variance Gaussian distributions met?
| |
| +-- YES --> Use d' = z(H) - z(FA) (Green & Swets, 1966)
| |
| +-- NO, distributions have unequal variance
| | --> Use da with estimated variance ratio
| | (Macmillan & Creelman, 2005, Ch. 3)
| |
| +-- NO, distributions are non-Gaussian or unknown
| --> Use Az (area under the ROC curve)
| (Swets, 1986; Macmillan & Creelman, 2005, Ch. 3)
|
+-- NO --> Is it a two-interval forced choice (2AFC/2IFC)?
 |
 +-- YES --> d'(2AFC) = z(proportion correct) x sqrt(2)
 | (Green & Swets, 1966, Ch. 6; Macmillan & Creelman, 2005, Ch. 5)
 |
 +-- NO --> Is it same-different or ABX?
 |
 +-- YES --> Use paradigm-specific formulas
 | (see references/sdt-formulas.md)
 |
 +-- NO --> Is it a rating-scale (confidence) design?
 |
 +-- YES --> Construct ROC from rating data;
 use Az or fit parametric model
 (Macmillan & Creelman, 2005, Ch. 3)

When to Use Az Instead of d'

Use the area under the ROC curve (Az) when:

1. You have rating-scale data (multiple confidence levels) and can construct a full ROC
2. The equal-variance assumption is violated (common in recognition memory, where the zROC slope is typically ~0.80 rather than 1.0; Mickes, Wixted, & Wais, 2007; Ratcliff, Sheu, & Gronlund, 1992)
3. You want a distribution-free sensitivity measure that does not assume Gaussian internals (Swets, 1986)

AUC benchmarks (Swets, Dawes, & Monahan, 2000):

Common Paradigms

Yes/No Detection

The canonical SDT paradigm. On each trial, either a signal or noise is presented; the observer responds "yes" (signal present) or "no" (signal absent). Yields H and FA rates directly.

• d' = z(H) - z(FA)
• Bias c = -0.5 x [z(H) + z(FA)]

Two-Alternative Forced Choice (2AFC / 2IFC)

Two intervals are presented (one signal, one noise); the observer selects the signal interval. Only proportion correct is measured; there is no independent FA rate, and no bias measure can be computed.

• d'(2AFC) = z(proportion correct) x sqrt(2) (Green & Swets, 1966, Ch. 6)
• d'(2AFC) = d'(yes/no) x sqrt(2) (Macmillan & Creelman, 2005, Ch. 5)

Critical domain pitfall: A task where the observer chooses between two labels (e.g., "left" or "right") on a single stimulus is not a 2AFC -- it is a yes/no task in disguise (Macmillan & Creelman, 2005). True 2AFC requires two temporal or spatial intervals.

Rating Scale (Confidence Ratings)

Observers make a detection judgment plus a confidence rating (e.g., 1-6 scale from "sure noise" to "sure signal"). Each confidence boundary yields a separate (H, FA) pair, constructing a multi-point ROC.

• Fit with parametric (Gaussian) or nonparametric methods
• Compute Az from the fitted ROC
• The zROC slope estimates the variance ratio of the two distributions

Same-Different

Two stimuli are presented; the observer judges "same" or "different." Two observer models exist (Macmillan & Creelman, 2005, Ch. 6):

• Independent observations model: observer compares each stimulus to an internal criterion
• Differencing model: observer computes the difference between the two percepts

These yield different d' formulas; see references/sdt-formulas.md.

ABX (Oddity / AX)

Stimulus A, then B, then X (which matches A or B); the observer identifies X. Sensitivity depends on assumed observer strategy (Macmillan & Creelman, 2005, Ch. 6). See references/sdt-formulas.md.

Handling Extreme Hit/False Alarm Rates

When H = 1.0 or FA = 0.0, z-scores become infinite and d' is undefined. This is a common computational pitfall that requires correction.

Correction Methods

1. The 1/(2N) rule (Macmillan & Kaplan, 1985):

• Replace 0 with 0.5/N
• Replace 1 with (N - 0.5)/N
• Where N = number of signal trials (for H) or noise trials (for FA)
• Applied only to extreme values

2. The log-linear rule (Hautus, 1995) -- recommended:

• Add 0.5 to every cell in the 2x2 matrix (hits, misses, FA, CR) before computing rates
• Applied to all cells, regardless of whether extremes are present
• Produces less biased estimates that consistently underestimate true d' (Hautus, 1995)

Which to use: The log-linear rule is preferred because it produces less biased d' estimates and avoids the asymmetric bias of the 1/(2N) rule, which can either over- or underestimate d' (Hautus, 1995). Apply the log-linear correction routinely, not just when extremes occur, for consistency across participants and conditions.