Contract Information

Programme:	SEVENTH FRAMEWORK PROGRAMME
Programme Acronym:	FP7-FET OPEN
Contract Type:	SPECIFIC TARGETED RESEARCH PROJECT
Start Date:	2010-10-01
End Date:	2013-09-30
Contract No:	255957
Role for SoftNet:	contractor
Funding for SoftNet:	482.500 Euros
Principal Investigator for SoftNet:	Minos Garofalakis

Project Information

Official Web Site: http://lift-eu.org/ 

As the scale of today’s networked techno-social systems continues to increase, the analysis of their global phenomena becomes increasingly difficult, due to the continuous production of streams of data scattered among distributed, possibly resource-constrained nodes, and requiring reliable resolution in (near) real-time.

The goal of LIFT is to enable the local detection of global phenomena and the efficient and effective detection of phase changes in very large data streams, where it is impossible or ineffective to accumulate all data into a single place. In addition, this will give rise to new methods for analyzing privacy-sensitive data, where it is not desirable to move data away from the point where it is collected. This will be facilitated by developing a theory based on the novel Safe-Zone-Approach and related methodologies.

We will explore a novel approach for realising sophisticated, large-scale distributed data-stream analysis systems, relying on processing local data in situ. Our key insight is that, for a wide range of distributed data analysis tasks, we can employ novel geometric techniques for intelligently decomposing the monitoring of complex holistic conditions and functions into safe, local constraints that can be tracked independently at each node (without communication), while guaranteeing correctness for the global-monitoring operation. While some solutions exist for the limited case of linear functions of the data, it is hard to deal with general, non-linear functions: in this case, a node’s local function value essentially tells us absolutely nothing about the global function value. Our fundamental idea is to design novel algorithmic tools that monitor the input domain of the global function rather than its range. Each node can then be assigned a safe zone (SZ) for its local values that can offer guarantees for the value of the global function over the entire collection of nodes. This represents a dramatic shift in conventional thinking and the state-of-the-art. We aim to reduce the amount of communication and data collection across nodes to a minimum, requiring nodes to communicate only when their local constraints are violated.

To demonstrate the potential of the approach, LIFT has selected four scenarios in challenging, high-impact areas of research, where communication bottlenecks currently are severe limiting factors. These scenarios are (1) the analysis of traffic and network data, as exemplified in the scenario of mobility analysis (2) the analysis of high-speed, high-volume data in an internet-scale distributed querying scenario (3) monitoring in sensor network as an example of devices allowing only few communication, applied to the real-time analysis of natural disasters, and (4) real-time log monitoring in clouds and large data centres as a scenario where already today high volume of data is severely limiting the optimisation and monitoring of IT systems.