QualiMaster - A configurable real-time date processing infrastructure mastering autonomous quality adaptation

QualiMaster - A configurable real-time date processing infrastructure mastering autonomous quality adaptation

The recent global financial crisis has shown, that even today, where large parts of the private and working life are supported and facilitated by information technology (IT), an early detection of cross-market risk situations is difficult. Especially in areas such as the financial sector, where in the stock market enormous amounts of data are processed immediately every day, IT faces there limits when it comes to the analyzes of big data. For example, up to 250 gigabytes of data (approximately 54 DVDs) are generated by current trading data and daily currency exchange rates in Europe and America. Country and cross-market risk analyzes, which are necessary for early intervention by central banks, require real-time analysis of this data and the ability to focus on individual phenomena, i.e. to process more data in a more detailed way in risk situations. This also includes data from social networks as, for example, the demise of Lehman-Brothers began with a rumor that the bank could not obtain their daily capital.

Today’s associated IT systems are designed for the maximum case, in which both the maximum amount of data flow and at the same time the maximum processing capacity is required. However, this is neither effective nor cost-effective since the capacity remains unused at times of lower data streams, instead of performing additional detailed analysis. The vision is the automatic and dynamic adaptation of such systems to the individual situation so that existing capacities are optimally utilized.

The group "Software Systems Engineering" by Prof. Dr. Klaus Schmid is working on methods and techniques for efficient software adaptation and how this adaptation can be carried out by software autonomously. With this and other work, the group has made a nationwide name and is therefore (in addition to the L3S Research Center Hanover, the Telecommunication Systems Institute of the Technical University of Crete and the companies Maxeler Technologies Ltd. in London and Spring Techno in Bremen) a partner in the research project "QualiMaster", which is funded by the EU with about 2.9 million Euros over three years. In this EU-project, the researchers of the working group focus in particular on the automatic configuration and adaptation of mechanisms for processing big data with respect to different quality characteristics.

With nearly 100 million messages per second at the stock market (Europe and America) the time factor is an important quality characteristic. Thus, the processing and analysis of these messages must not delay the receipt of results under agreed quality criteria for each recipient. The challenge of considering the relevant quality characteristics and performing an automatic adaptation of processes based on the weighting of these characteristics and the current amount of data will be part of the work of the SSE group in the project.

The researchers support the development of a configurable real-time data processing system for autonomous quality adjustment. In the long term, predictions about the development in the financial market (the analysis and prediction of systemic risk in the project) will be done using such a system. Further, this system can be used in other areas with large amounts of data, like macro-economic analyzes, weather analyzes, the analyzes of social networks, or large scientific experiments, for effectively and efficiently prediction mechanisms.

Further information:

 

Duration: 3 years

Contact: Dr. Holger Eichelberger, eichelberger(at)sse.uni-hildesheim.de

The QualiMaster-project is funded by the European Commission Grant 619525 , Area Scalable Data Analytics in the 7th Framework Programme. The EU is funding the project with approximately 2.9 million Euros.

Public Deliverables

NumberName
D1.1Initial Use Cases and Requirements
D1.2Full Use Cases and Requirements
D2.1Approach for Scalable, Quality-aware Data Processing
D2.2Scalable, Quality-aware Data Processing Algorithms V1
D2.3Scalable, Quality-aware Data Processing Algorithms V2
D2.4Final report on Scalable, Quality-aware Data Processing Methods
D3.1Translation of Data Processing Algorithms to Hardware
D3.2Hardware-based Data Processing Algorithms V1
D3.3Hardware-based Data Processing Algorithms V2
D3.4Optimized Translation of Data Processing Algorithms to Hardware
D4.1Quality-aware Processing Pipeline Modelling
D4.2Quality-aware Processing Pipeline Adaptation V1
D4.3Quality-aware Processing Pipeline Adaptation V2
D4.4Quality-aware Processing Pipeline Modelling and Adaptation
D5.1QualiMaster Infrastructure Set-up
D5.2Basic QualiMaster Infrastructure
D5.3QualiMaster Infrastructure V1
D5.4QualiMaster Infrastructure V2
D6.1QualiMaster Applications V1 (internal)
D6.2Intermediary Evaluation Report
D6.3QualiMaster Applications V2 (internal)
D6.4Final Evaluation Report
D7.1Initial Project Fact Sheet
D7.2Project Presentation and Project Web Site
D7.3Dissemination Plan

 

 

The research leading to these results has received funding from the European Union Seventh Framework Programme [FP7/2007-2013] under grant agreement n° 619525.

Publications

Author:  
All :: E, H, K, N, Q, S 
  
 
Preferences: 
References per page: Show keywords Show abstracts
References

2017

13.
Eichelberger, Holger, Qin, Cui and Schmid, Klaus
Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering Companion (ICPE '17) , page 91-96.
Publisher: ACM,
2017
12.
Eichelberger, Holger, Qin, Cui and Schmid, Klaus
Lecture Notes in Informatics (LNI) - Datenbanksysteme für Business, Technologie und Web (BTW '17) - Workshopband , page 49-56.
2017

2016

11.
Heinrich, Robert, Eichelberger, Holger and Schmid, Klaus
Proceedings of the 3rd International Workshop on Interplay of Model-Driven and Component-Based Software Engineering (ModComp '16) , page 37-38.
2016
10.
Eichelberger, Holger
2016 IEEE 1st International Workshops on Foundations and Applications of Self-* Systems, Proceedings of the 9th International Workshop on Dynamic Software Product Lines (DSPL '16) , page 12-15.
Publisher: IEEE,
2016
9.
Eichelberger, Holger, Qin, Cui, Sizonenko, Roman and Schmid, Klaus
Proceedings of the 20th International Systems and Software Product Line Conference , page 204 - 208.
Publisher: ACM,
2016
8.
Qin, Cui and Eichelberger, Holger
Proceedings of the Workshops of the EDBT/ICDT 2016 Joint Conference
Publisher: CEUR-WS.org,
2016

2015

7.
Eichelberger, Holger, Qin, Cui, Schmid, Klaus and Niederée, Claudia
Softwaretechnik-Trends, 35(3):35-37
2015
6.
Eichelberger, Holger and Schmid, Klaus
International Journal of Software Tools for Technology Transfer, 17(5):559-584
2015
5.
Schmid, Klaus and Eichelberger, Holger
Proceedings of the 19th International Conference on Software Product Line , page 390-391.
Publisher: ACM,
2015
4.
Eichelberger, Holger and Schmid, Klaus
Proceedings of the 19th International Conference on Software Product Line , page 365-369.
Publisher: ACM,
2015
Page:  
Previous | 1, 2 | Next
Export as:
BibTeX, XML