Population and Sample in Case Study Research for Software Engineering: Framework and Practical Guide

Abstract

Research in computer science and software engineering frequently relies on case study design, yet the way population and sample are declared and justified in these works is often imprecise or outright absent, undermining credibility with reviewers trained in quantitative methodology. This review article examines the epistemological foundation of case study research in software engineering, analyzes how the concepts of population and sample are reformulated in this context, and proposes an operational framework for declaring, justifying, and delimiting both the population and the selected case in this type of research. The canonical methodological frameworks of the field are reviewed and their principles are articulated into a five-step guide applicable to original work in software engineering, systems architecture, security, and tool development. The article concludes that rigorous population specification and explicit sampling justification are necessary conditions for a case study in engineering to produce generalizable knowledge, and that this generalization operates analytically —not statistically—, with direct consequences for how methodology and discussion sections are written. Research in computer science and software engineering frequently relies on case study design, yet the way population and sample are declared and justified in these works is often imprecise or outright absent, undermining credibility with reviewers trained in quantitative methodology. This review article examines the epistemological foundation of case study research in software engineering, analyzes how the concepts of population and sample are reformulated in this context, and proposes an operational framework for declaring, justifying, and delimiting both the population and the selected case in this type of research. The canonical methodological frameworks of the field are reviewed and their principles are articulated into a five-step guide applicable to original work in software engineering, systems architecture, security, and tool development. The article concludes that rigorous population specification and explicit sampling justification are necessary conditions for a case study in engineering to produce generalizable knowledge, and that this generalization operates analytically —not statistically—, with direct consequences for how methodology and discussion sections are written. Research in computer science and software engineering frequently relies on case study design, yet the way population and sample are declared and justified in these works is often imprecise or outright absent, undermining credibility with reviewers trained in quantitative methodology. This review article examines the epistemological foundation of case study research in software engineering, analyzes how the concepts of population and sample are reformulated in this context, and proposes an operational framework for declaring, justifying, and delimiting both the population and the selected case in this type of research. The canonical methodological frameworks of the field are reviewed and their principles are articulated into a five-step guide applicable to original work in software engineering, systems architecture, security, and tool development. The article concludes that rigorous population specification and explicit sampling justification are necessary conditions for a case study in engineering to produce generalizable knowledge, and that this generalization operates analytically —not statistically—, with direct consequences for how methodology and discussion sections are written.