method note
Common method variance: preventing, testing, correcting
What is common method variance and how is it controlled?
Common method variance is spurious covariance produced when predictor and outcome are collected from the same source, in the same format, at the same time. The effective fix is at design stage: separate source, time, or format. After collection, Harman's single-factor test is weak evidence; unmeasured or measured latent method factor approaches are stronger.
Why it matters
If one questionnaire asks the same person for both the attitude and the outcome, part of the resulting correlation comes from the method rather than the relationship. This inflates effects and can manufacture ones that do not exist. A substantial part of the organisational behaviour literature rests on data collected under this risk.
What can be done at design stage
Collect predictor and outcome from different sources (employee and supervisor), separate them in time (two waves), vary the response format (Likert and forced choice), counterbalance item order, guarantee anonymity. All of these beat any post-hoc correction.
The problem with Harman's test
Harman's single-factor test runs all items through an unrotated factor analysis and asks whether the first factor explains less than half the variance. It is almost always passed; it does not demonstrate the absence of CMV, only catches its crudest form. It nonetheless remains the most common basis for the sentence 'CMV was controlled for'.
The work behind this note
- A Metadata-Only Feature-Augmented Method Factor for Ex-Post Correction and Attribution of Common Method Variance2025 · Preprint
- Factor Analysis and Validity in Social Sciences: Application of Exploratory and Confirmatory Factor Analyses2017 · 2,435 citations
- Structural Equation Modeling (SEM) for Social and Behavioral Sciences Studies: Steps Sequence and Explanation2024 · 44 citations
Other method notes
- Exploratory or confirmatory? Which factor analysis, and whenShould I use exploratory or confirmatory factor analysis?
- Which fit index to report, and whenWhich fit indices should I report in structural equation modelling?
- Validity and reliability: what to report, and howHow are validity and reliability reported, and how do they differ?
Written for researchers and graduate students. Conventions described here are conventions, not rules — check them against your own design before you report.