Kontakt:Telefon: +49 5121 - 883 40348
Fax: +49 5121 - 883 40349
Raum: C 141 Spl - Gebäude C (Samelson-Campus)
Homepage: https://sse.uni-hildesheim.de/mitglieder/cui-qin/ Homepage
Holger Eichelberger, Cui Qin und Klaus Schmid
Experiences with the Model-based Generation of Big Data Applications
Lecture Notes in Informatics (LNI) - Datenbanksysteme für Business, Technologie und Web (BTW '17) - Workshopband
Developing Big Data applications implies a lot of schematic or complex structural tasks, which can easily lead to implementation errors and incorrect analysis results. In this paper, we present a model-based approach that supports the automatic generation of code to handle these repetitive tasks, enabling data engineers to focus on the functional aspects without being distracted by technical issues. In order to identify a solution, we analyzed different Big Data stream-processing frameworks, extracted a common graph-based model for Big Data streaming applications and developed a tool to graphically design and generate such applications in a model-based fashion (in this work for Apache Storm). Here, we discuss the concepts of the approach, the tooling and, in particular, experiences with the approach based on feedback of our partners.
Holger Eichelberger, Cui Qin und Klaus Schmid
From Resource Monitoring to Requirements-based Adaptation: An Integrated Approach
Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion (ICPE '17)
In large and complex systems there is a need to monitor resources as it is critical for system operation to ensure sufficient availability of resources and to adapt the system as needed. While there are various (resource)-monitoring solutions, these typically do not include an analysis part that takes care of analyzing violations and responding to them. In this paper we report on experiences, challenges and lessons learned in creating a solution for performing requirements-monitoring for resource constraints and using this as a basis for adaptation to optimize the resource behavior. Our approach rests on reusing two previous solutions (one for resource monitoring and one for requirements-based adaptation) that were built in our group.
Cui Qin und Holger Eichelberger
Impact-minimizing Runtime Switching of Distributed Stream Processing Algorithms
Proceedings of the Workshops of the EDBT/ICDT 2016 Joint Conference
Stream processing is a popular paradigm to process huge amounts of data. During processing, the actual characteristics of the analyzed data streams may vary, e.g., in terms of volume or velocity. To provide a steady quality of the analysis results, runtime adaptation of the data processing is desirable. While several techniques for changing data stream processing at runtime do exist, one specific challenge is to minimize the impact of runtime adaptation on the data processing, in particular for real-time data analytics. In this paper, we focus on the runtime switching among alternative distributed algorithms as a means for adapting complex data stream processing tasks. We present an approach, which combines stream re-routing with buffering and stream synchronization to reduce the impact on the data streams. Finally, we analyze and discuss our approach in terms of a quantitative evaluation.
Impact-minimizing Runtime Adaptation in Cloud-based Data Stream Processing
Proceedings of the PhD Symposium at the 5th European Conference on Service-Oriented and Cloud Computing (ESOCC '16)
Recently, cloud-based data stream processing has been rapidly emerged. In particular, it has become competitive in high scalability for processing huge amounts of data. During such processing, the actual characteristics of data streams may vary, e.g., in terms of volume or velocity. To provide a steady quality of the analysis results, runtime adaptation of the data processing is desirable. While several techniques for changing data stream processing at runtime do exist, one specific challenge is to minimize the impact of runtime adaptation on the data processing, in particular for real-time data analytics. In this research work, we aim at performing a runtime adaptation in cloud-based data stream processing, namely, dynamically switching alternative distributed algorithms which has similar functionality but operate at different characteristics. The goal of this work is to provide a generic approach which can automatically determine the algorithm switch with quality guarantees on the data processing. To minimize the impact on the data streams as well as optimize resource usage during adaptation, we combine stream re-routing with buffering and stream synchronization along with a support of dynamic deployment of alternative stream processing algorithms into the cloud.
Holger Eichelberger, Cui Qin, Roman Sizonenko und Klaus Schmid
Using IVML to Model the Topology of Big Data Processing Pipelines
Proceedings of the 20th International Systems and Software Product Line Conference
S. 204 - 208.
Creating product lines of Big Data stream processing applications introduces a number of novel challenges to variability modeling. In this paper, we discuss these challenges and demonstrate how advanced variability modeling capabilities can be used to directly model the topology of processing pipelines as well as their variability. We also show how such processing pipelines can be modeled, configured and validated using the Integrated Variability Modeling Language (IVML).
Holger Eichelberger, Cui Qin, Klaus Schmid und Claudia Niederée
Adaptive Application Performance Management for Big Data Stream Processing
In: Softwaretechnik-Trends, 35 (3): 35-37.
Big data applications with their high-volume and dynamically changing data streams impose new challenges to application performance management. Efficient and effective solutions must balance performance versus result precision and cope with dramatic changes in real-time load and needs without over-provisioning resources. Moreover, a developer should not be burdened too much with addressing performance management issues, so he can focus on the functional perspective of the system For addressing these challenges, we present a novel comprehensive approach, which combines software configuration, model-based development, application performance management and runtime adaptation.