4 June 2024 in ACM Transactions on Software Engineering and Methodology
Reflecting on the limitations of a study is a crucial part of the research process. In software engineering studies, this reflection is typically conveyed through discussions of study limitations or threats to validity. In current practice, such discussions seldom provide sufficient insight to understand the rationale for decisions taken before and during the study, and their implications. We revisit the practice of discussing study limitations and threats to validity and identify its weaknesses. We propose to refocus this practice of self-reflection to a discussion centered on the notion of trade-offs . We argue that documenting trade-offs allows researchers to clarify how the benefits of their study design decisions outweigh the costs of possible alternatives. We present guidelines for reporting trade-offs in a way that promotes a fair and dispassionate assessment of researchers’ work.
24 December 2022 in Empirical Software Engineering
21 June 2021 in IEEE Software
We investigate three tutorials on Android development to elicit design choices related to their organization and content. We describe decision points and tradeoffs and propose guidelines to assist tutorial authors in making explicit and informed decisions about the design of tutorials
26 August 2020 in Empirical Software Engineering
17 November 2023 in IEEE Transactions on Software Engineering
A large number of tutorials for popular software development technologies are available online, and those about the same technology vary widely in their presentation. We studied the design of tutorials in the software documentation landscape for five popular programming languages: Java, C#, Python, Javascript, and Typescript. We investigated the extent to which tutorial pages, i.e. resources , differ and report statistics of variations in resource properties. We developed a framework for characterizing resources based on their distinguishing attributes , i.e. properties that vary widely for the resource, relative to other resources. Additionally, we propose that a resource can be represented by its resource style , i.e. the combination of its distinguishing attributes. We discuss three techniques for characterizing resources based on our framework, to capture notable and relevant content and presentation properties of tutorial pages. We apply these techniques on a data set of 2551 resources to validate that our framework identifies valid and interpretable styles. We contribute this framework for reasoning about the design of resources in the online software documentation landscape.