Jmgmt Methods

Research Design & Methods (jmgmt-methods)

When to trigger

• The design may not match the theory's level, timing, or causal claim
• Data are single-source, single-wave, self-reported (common-method bias risk)
• The theory is causal but the design is cross-sectional/correlational
• Constructs lack established, validated measures
• A meta-analysis needs a defensible coding protocol and artifact-correction plan
• A reviewer says "the design cannot test this hypothesis" or "endogeneity is unaddressed"

Match the design to the claim

JOM welcomes all empirical methods — survey, experiment, archival panel, multilevel field study, qualitative, and meta-analysis — and judges on fit and rigor, not a preferred method. JOM's research-methods identity (it explicitly covers research methods and runs methods-focused reviews) means design choices are scrutinized closely.

A two-study design (field study for generalizability + experiment for the causal mechanism) is a recognized JOM strength — it buys internal and external validity at once.

Designing against the threats JOM referees punish

• Common-method bias (CMB): separate the sources of predictor and outcome; separate them temporally across waves; use objective/archival outcomes where possible. Procedural remedies beat statistical fixes (the Podsakoff et al. guidance is the field reference). Plan this before collecting data; a Harman single-factor test alone will not satisfy a JOM reviewer.
• Endogeneity (archival/macro): anticipate omitted variables, reverse causality, and selection. Specify an identification strategy — instrument, natural experiment, panel fixed effects, difference-in-differences, Heckman/2SLS, propensity matching — and state the assumptions each requires.
• Measurement / construct validity: use validated multi-item scales; pilot new measures; plan a confirmatory factor analysis (CFA) with fit indices and a discriminant-validity test (AVE vs. shared variance, or the HTMT ratio). State the level at which each construct is measured.
• Multilevel discipline: if data are nested, justify aggregation with ICC(1), ICC(2), and r_wg; model the nesting (random effects/HLM). Theorizing at the team level but running OLS on disaggregated individuals is a standard rejection trigger.
• Sampling & power: justify the frame, response rate, and statistical power — especially for interactions, which JOM reviewers know are underpowered when authors present null moderation as a "boundary condition."

Meta-analysis design

• Pre-specify inclusion/exclusion criteria and a transparent search; report a PRISMA-style flow.
• Double-code a subset; report inter-coder agreement.
• Choose the artifact-correction model (Hunter–Schmidt psychometric meta-analysis vs. Hedges–Olkin random-effects) and justify it; correct for sampling error and, where defensible, measurement unreliability and range restriction.
• Plan moderator/meta-regression analyses that map to competing theories, plus publication-bias diagnostics.

Referee pushback mapped to the design fix

• "This is single-source, single-wave — common-method bias is unaddressed." → Add temporal/source separation or an objective outcome; a Harman test alone will not close it.
• "Your archival regressor is endogenous." → Specify and defend an identification strategy (IV/NE/FE/DiD/matching) and report first-stage strength.
• "The new scale's discriminant validity is unestablished." → Report a CFA with AVE vs. shared variance or an HTMT ratio, plus an alternative-model comparison.
• "You theorize at the team level but test individuals." → Justify aggregation (ICC, r_wg) and model the nesting, or re-pitch the theory at the individual level.
• "The interaction is underpowered." → Report power for the interaction specifically; if it is a true null, theorize the boundary rather than presenting an underpowered null as a finding.