Zusammenfassung

Context: Software Product Line (SPL) development requires at least concepts for variability implementation and variability modeling for deriving products from a product line. These variability implementation concepts are not required for the development of single systems and, thus, are not considered in traditional software engineering. Metrics are well established in traditional software engineering, but existing metrics are typically not applicable to SPLs as they do not address variability management. Over time, various specialized product line metrics have been described in literature, but no systematic description of these metrics and their characteristics is currently available.

Objective: This paper describes and analyzes variability-aware metrics, designed for the needs of software product lines. More precisely we restrict the scope of our study explicitly to metrics designed for variability models, code artifacts, and metrics taking both kinds of artifacts into account. Further, we categorize the purpose for which these metrics were developed. We also  analyze to what extent these metrics were evaluated to provide a basis for researchers for selecting adequate metrics. Method: We conducted a systematic literature review to identify variability-aware implementation metrics. We discovered 42 relevant papers reporting metrics intended to measure aspects of variability models or code artifacts. Results: We identified 57 variability model metrics, 34 annotation-based code metrics, 46 code metrics specific to composition-based implementation techniques, and 10 metrics integrating information from variability model and code artifacts. For only 31 metrics, an evaluation was performed assessing their suitability to draw any qualitative conclusions.

Conclusions: We observed several problematic issues regarding the definition and the use of the metrics. Researchers and practitioners benefit from the catalog of variability-aware metrics, which is the first of its kind. Also, the research community benefits from the identified observations in order to avoid those problems when defining new metrics.

Zusammenfassung
Systematic exploration of hypotheses is a major part of any empirical research. In software engineering, we often produce unique tools for experiments and evaluate them independently on different data sets. In this paper, we present KernelHaven as an experimentation workbench supporting a significant number of experiments in the domain of static product line analysis and verification. It addresses the need for extracting information from a variety of artifacts in this domain by means of an open plug-in infrastructure. Available plug-ins encapsulate existing tools, which can now be combined efficiently to yield new analyses. As an experimentation workbench, it provides configuration-based definitions of experiments, their documentation, and technical services, like parallelization and caching. Hence, researchers can abstract from technical details and focus on the algorithmic core of their research problem. KernelHaven supports different types of analyses, like correctness checks, metrics, etc., in its specific domain. The concepts presented in this paper can also be transferred to support researchers of other software engineering domains. The infrastructure is available under Apache 2.0: github.com/KernelHaven. The plug-ins are available under their individual licenses. Video: youtu.be/IbNc-H1NoZU

Zusammenfassung
Development of Software Product Lines (SPLs) requires additional implementation concepts to manage variability and to facilitate the derivation of individual products based on a common platform. These variability implementation concepts are not required for the development of single systems and, thus, are not considered in traditional software engineering. Metrics are well established in traditional software engineering, but are typically not applicable to SPLs as they do not address variability management. Over time, a number of specialized product line metrics have been described in literature. However, no systematic description of the characteristics of these metrics is currently available. We conducted a systematic literature review to identify variability-aware implementation metrics, designed for the needs of SPLs. We list these metrics according to the measured artifact types: variability models, code artifacts, and combined metrics measuring both artifact types. Further, we analyze to what extent these metrics were evaluated as a basis for qualitative conclusions.

Zusammenfassung

KernelHaven is an open infrastructure for Software Product Line (SPL) analysis. It is intended both as a production-quality analysis tool set as well as a research support tool. Its design follows the principle of an experimentation workbench [13] to support researchers in systematically exploring research hypothesis. For flexibility and ease of experimentation KernelHaven components are plug-ins for extracting certain information from SPL artifacts and processing this information, e.g., to check the correctness and consistency of variability information or to apply metrics. A configuration-based setup along with automatic documentation functionality allows different experiments and supports their easy reproduction.

Here, we describe KernelHaven as a product line analysis research tool and highlight its basic approach as well as its fundamental capabilities. In particular, we describe available information extraction and processing plug-ins and how to combine them. On this basis, researchers and interested professional users can rapidly conduct a first set of experiments. Further, we describe the concepts for extending KernelHaven by new plug-ins, which reduces development effort when realizing new experiments.

Zusammenfassung

EASy-Producer is an open-source research toolset for engineering product lines, variability-rich software ecosystems, and dynamic software product lines. In this tutorial, we will focus on its (textual) variability modeling capabilities as well as its configuration and validation functionality. Further, we will provide an outlook on how EASy-Producer can be applied to variability instantiation.

Zusammenfassung

EASy-Producer is an open-source research toolset for engineering product lines, variability-rich software ecosystems, and dynamic software product lines. In this tutorial, we will focus on its model-based development and implementation capabilities, which are realized by the Variability Instantiation Language (VIL) and the Variability Template Language (VTL). Further, we will provide a basic introduction into the Integrated Variability Modeling Language (IVML) in order to use the provided information of IVML variability models and configurations during instantiation defined with VIL and as part of the templates created with VTL.

Zusammenfassung

The evolution of a Software Product Line (SPL) typically affects a variety of artifact types. The intensity (the frequency and the amount) in which developers change variability information in these different types of artifacts is currently unknown. In this paper, we present a fine-grained approach for the variability-centric extraction and analysis of changes to code, build, and variability model artifacts introduced by commits. This approach complements existing work that is typically based on a feature-perspective and, thus, abstracts from this level of detail. Further, it provides a detailed understanding of the intensity of changes affecting variability information in these types of artifacts. We apply our approach to the Linux kernel revealing that changes to variability information occur infrequently and only affect small parts of the analyzed artifacts. Further, we outline how these results may improve certain analysis and verification tasks during SPL evolution.

Zusammenfassung

Ideally, a variability model is a correct and complete representation of product line features and constraints among them. Together with a mapping between features and code, this ensures that only valid products can be configured and derived. However, in practice the modeled constraints might be neither complete nor correct, which causes problems in the configuration and product derivation phases. This paper presents an approach to reverse engineer variability constraints from the implementation, and thus improve the correctness and completeness of variability models.

We extended the concept of feature effect synthesis [18] to extract domain knowledge from code artifacts of the Bosch PS-EC product line. We present an application of the approach to a large-scale industrial product line and discuss its required modifications to obtain meaningful results in an industrial case.

Zusammenfassung
Development of Software Product Lines (SPLs) requires additional implementation concepts to manage variability and to facilitate the derivation of individual products based on a common platform. These variability implementation concepts are not required for the development of single systems and, thus, are not considered in traditional software engineering. Metrics are well established in traditional software engineering, but are typically not applicable to SPLs as they do not address variability management. Over time, a number of specialized product line metrics have been described in literature. However, no systematic description of the characteristics of these metrics is currently available. We conducted a systematic literature review to identify variability-aware implementation metrics, designed for the needs of SPLs. We list these metrics according to the measured artifact types: variability models, code artifacts, and combined metrics measuring both artifact types. Further, we analyze to what extent these metrics were evaluated as a basis for qualitative conclusions.

Zusammenfassung
Software Product Line (SPL) evolution affects a variety of artifact types, each containing their artifact-specific information as well as variability information. While the former information type defines the basic content of an artifact, like the general program definition in code artifacts, the latter supports the customization of these artifacts for different products of the SPL. Existing work that aims at characterizing the state and evolution of a product line identifies general SPL evolution scenarios or evaluates the (co-)evolution of variability information in different types of artifacts. However, these results are typically based on a feature perspective, which abstracts from the SPL evolution at large. We argue that an artifact-based analysis of product line evolution will complement existing work by analyzing the intensity of changes to different artifact and information types. In this report, we therefore present an approach for and the results of the extraction and analysis of changes introduced to artifact-specific and variability information in code, build and variability model artifacts. This approach has been developed for and applied to the Linux kernel. In order to broaden our analysis, we also apply the approach to the Coreboot firmware. The results reveal the intensity of changes over the evolution history of both SPLs. Further, we compare the intensity of changes across both SPLs with respect to the size of the changes as well as over time.

Zusammenfassung
Ideally the variability of a product line is represented completely and correctly by its variability model. However, in practice additional variability is often represented on the level of the build system or in the code. Such a situation may lead to inconsistencies, where the actually realized variability does not fully correspond to the one described by the variability model. In this paper we focus on configuration mismatches, i.e., cases where the effective variability differs from the variability as it is represented by the variability model. While previous research has already shown that these situations still exist even today in well-analyzed product lines like Linux, so far it was unclear under what circumstances such issues occur in reality. In particular, it is open what types of configuration mismatches occur and how severe they are. Here, our contribution is to close this gap by presenting a detailed manual analysis of 80 configuration mismatches in the Linux 4.4.1 kernel and assess their criticality. We identify various categories of configuration issues and show that about two-thirds of the configuration mismatches may actually lead to kernel misconfigurations.

Zusammenfassung
In contrast to traditional software systems, the evolution of a Software Product Line (SPL) affects not only artifacts like source code or requirements, but also variability information, which supports the customization of these artifacts across different products of the SPL. While some work exists that aims at characterizing the state and evolution of a product line from a feature perspective, this abstracts away the details of code evolution, hence, ignoring aspects like the difference in size of features. In this paper, we present an approach for the extraction and analysis of changes introduced to code, build and variability model artifacts. This approach has been developed for and applied to the Linux product line.