abstract

Context: Software metrics, as one form of static analyses, is a commonly used approach in software engineering in order to understand the state of a software system, in particular to identify potential areas prone to defects. Family-based techniques extract variability information from code artifacts in Software Product Lines (SPLs) to perform static analysis for all available variants. Many different types of metrics with numerous variants have been defined in literature. When counting all metrics including such variants, easily thousands of metrics can be defined. Computing all of them for large product lines can be an extremely expensive process in terms of performance and resource consumption.

Objective: We address these performance and resource challenges while supporting customizable metric suites, which allow running both, single system and variability-aware code metrics.

Method: In this paper, we introduce a partial parsing approach used for the efficient measurement of more than 42,000 code metric variations. The approach covers variability information and restricts parsing to the relevant parts of the Abstract Syntax Tree (AST).

Conclusions: This partial parsing approach is designed to cover all relevant information to compute a broad variety of variability-aware code metrics on code artifacts containing annotation-based variability, e.g., realized with C-preprocessor statements. It allows for the flexible combination of single system and variability-aware metrics, which is not supported by existing tools. This is achieved by a novel representation of partially parsed product line code artifacts, which is tailored to the computation of the metrics. Our approach consumes considerably less resources, especially when computing many metric variants in parallel.

abstract

Variability-aware metrics are designed to measure qualitative aspects of software product lines. As we identified in a prior SLR [6], there exist already many metrics that address code or variability separately, while the combination of both has been less researched. MetricHaven fills this gap, as it extensively supports combining information from code files and variability models. Further,we also enable the combination of well established single system metrics with novel variability-aware metrics, going beyond existing variabilityaware metrics. Our tool supports most prominent single system and variability-aware code metrics. We provide configuration support for already implemented metrics, resulting in 23,342 metric variations. Further, we present an abstract syntax tree developed for MetricHaven, that allows the realization of additional code metrics.

abstract

A number of SAT-based analysis concepts and tools for software product lines exist, that extract code dependencies in propositional logic from the source code assets of the product line. On these extracted conditions, SAT-solvers are used to reason about the variability. However, in practice, a lot of software product lines use integer-based variability. The variability variables hold integer values, and integer operators are used in the conditions. Most existing analysis tools can not handle this kind of variability; they expect pure Boolean conditions.

This paper introduces an approach to convert integer-based variability conditions to propositional logic. Running this approach as a preparation on an integer-based product line allows the existing SAT-based analyses to work without any modiications. The pure Boolean formulas, that our approach builds as a replacement for the integer-based conditions, are mostly equivalent to the original conditions with respect to satisfiability. Our approach was motivated by and implemented in the context of a real-world industrial case-study, where such a preparation was necessary to analyze the variability.

Our contribution is an approach to convert conditions, that use integer variables, into propositional formulas, to enable easy usage of SAT-solvers on the result. It works well on restricted variables (i.e. variables with a small range of allowed values); unrestricted integer variables are handled less exact, but still retain useful variability information.

abstract

Ideally, a variability model is a correct and complete representation of product line features and constraints among them. Together with a mapping between features and code, this ensures that only valid products can be configured and derived. However, in practice the modeled constraints might be neither complete nor correct, which causes problems in the configuration and product derivation phases. This paper presents an approach to reverse engineer variability constraints from the implementation, and thus improve the correctness and completeness of variability models.

We extended the concept of feature effect synthesis [18] to extract domain knowledge from code artifacts of the Bosch PS-EC product line. We present an application of the approach to a large-scale industrial product line and discuss its required modifications to obtain meaningful results in an industrial case.

abstract
Ideally the variability of a product line is represented completely and correctly by its variability model. However, in practice additional variability is often represented on the level of the build system or in the code. Such a situation may lead to inconsistencies, where the actually realized variability does not fully correspond to the one described by the variability model. In this paper we focus on configuration mismatches, i.e., cases where the effective variability differs from the variability as it is represented by the variability model. While previous research has already shown that these situations still exist even today in well-analyzed product lines like Linux, so far it was unclear under what circumstances such issues occur in reality. In particular, it is open what types of configuration mismatches occur and how severe they are. Here, our contribution is to close this gap by presenting a detailed manual analysis of 80 configuration mismatches in the Linux 4.4.1 kernel and assess their criticality. We identify various categories of configuration issues and show that about two-thirds of the configuration mismatches may actually lead to kernel misconfigurations.

abstract
Context. Software product line engineering has been established to minimize costs and efforts, while maximizing the quality of products in a family of software products. Software product lines typically contain a variability model, which supports the derivation of permissible variants. These variability models may contain expert/domain knowledge in the form of constraints. These constraints are used during the configuration process to avoid the selection of unsupported product variants. Problem. Developers must encode their knowledge about supported product variants and restrictions, otherwise the variability model becomes ineffective or even incorrect. The initial development of the variability model as well as the evolution of the product line implementation bear the risk that model and implementation drift apart. In this report, we introduce the notion of mismatched configuration information to describe the situation if the variability model does not react the dependencies of the implementation. This may indicate an incomplete variability model or undesired dependencies between code artifacts. Solution. We discuss the impact of mismatched configuration information and show how to detect this conceptually. Subsequently, we focus on mismatched hierarchical configuration information and present an effective heuristic for their detection. These results serve as an input to complete variability models or a code review to remove undesired implementation dependencies. We discuss the application of our approach on a Linux case study. The analysis of the x86 architecture of the Linux kernel takes only around 30 minutes and revealed mismatched configuration information, which was not treated by prior work.

abstract
The Linux Kernel is often used as real world use case to demonstrate novel Software Product Line Engineering techniques. The large open source repository facilitates the analysis of the variability model, the instantiation process, the instantiable artefacts, and the evolution of all of them. This report focusses on the analysis of undocumented KConfig functionalities. These functions have to be considered while applying any variability management technique to the Linux Kernel. Hence, this report will contribute to a better understanding how variability is handled in KConfig files. Further, we analyse existing work, which also analysed KConfig. Based on the weak documentation of KConfig, these works contain errors. These errors threat the validity of many existing analysis of the Linux Kernel.

abstract
The Linux kernel is often used as a real world case study to demonstrate novel Software Product Line Engineering research methods. An important point in this is often the analysis of the Kconfig semantics. However, we detected that the semantics of Kconfig is rather unclear and has many special cases, which are not documented in its short specification. We performed a systematic analysis to uncover the correct behaviour of Kconfig and present the results, which are necessary for applying semantically correct analyses. Further, we analyse existing analysis tools of the research community whether they are aware of the correct semantics of Kconfig. These analyses can be used for improving existing analysis tools as well as decision support for selecting an appropriate tool for a specific analysis. In summary we contribute to a better understanding of Kconfig in the research community to improve the validity of evaluations based on Linux.