Objectives of the Project
Network traffic monitoring and measurement is increasingly regarded as an essential function for understanding and improving the performance and security of our cyberinfrastructure. With networking technologies and services evolving rapidly, as witnessed by the explosive growth of the World-Wide Web, peer-to-peer networks, and the GRID, accurate network traffic monitoring is required to ensure the security and optimize the efficiency of our cyberspace.
To make an analogy from the physical sciences, network traffic monitoring systems are to the networking scientists as necessary as telescopes are to astronomers. Indeed, in the same way that a telescope monitors the outer space and records all interesting astronomic events, a traffic monitoring sensor monitors the cyberspace and records all interesting networking events. It is difficult to imagine where astronomy would be today if Galileo hadn't turned his telescope up towards the distant skies. On the contrary, it is easy to realize how this single monitoring instrument, i.e. the telescope, enhanced our understanding of the outer space, and eventually improved our life on earth. It is our belief that advanced network traffic monitoring sensors can have a similar impact to our understanding of the cyberspace, and can result in a significant improvement of the quality of our everyday lives.
The main goal of the project is to deploy a pilot advanced European Internet Traffic Monitoring Infrastructure based on passive monitoring sensors at speeds starting from 2.5 Gbps, and possibly up to 10 Gbps. Such an infrastructure will serve as a catalyst that will boost our understanding of the Internet and will lead to its better use in the long-run. Passive monitoring at such high speeds stresses significantly the computational, communication, and storage capabilities of the underlying monitoring sensor and poses several interesting research challenges. Fortunately, the FP5 IST SCAMPI project successfully met several of these challenges by designing and developing an advanced Internet passive monitoring system at 10 Gbps that combines novel hardware and software components. Having met the research challenges posed by passive monitoring at 10Gbps, it is proposed to deploy a network of such passive sensors in several key nodes setting the foundations to create a European Passive Internet Traffic Monitoring Infrastructure.
The objectives of the LOBSTER project are therefore to:- Deploy an pilot Internet Traffic Monitoring Infrastructure across Europe.
Based on passive monitoring, and capitalizing on the experience gained in the SCAMPI IST FP5 project, this infrastructure will be unique in Europe and among the only two similar infrastructures that exist in the world today. The passive monitoring infrastructure will be installed at NRNs and possibly ISPs. Some of them, being partners of this project will pioneer such installations during the first phase of the project. Once the pilot core monitoring infrastructure has been installed during the first phase of LOBSTER, the second phase will start, during which, more NRNs and ISPs will be able to join the infrastructure through the installation of passive monitoring sensors. - Organize stakeholders in the area of advanced Internet traffic monitoring.
The virtual network will consist of major stakeholders in the area including NRNs, ISPs, research organizations, and network equipment manufacturers. This network will deal with (i) the operation of the monitoring infrastructure, (ii) the expansion of the infrastructure through the inclusion of new member nodes, (iii) the support of the new member nodes through transfer of know-how, (iv) the training of personnel in monitoring technologies, and (v) the establishment of policies necessary to share and collaboratively use the monitoring infrastructure. - Realize the appropriate data anonymizing tools that will prohibit unauthorized tampering with the original traffic data.
To avoid any unauthorized use of network traffic data, a set of tools for encryption and anonymization of the original information contained in the monitored traffic will be realised. At the lowest layer, this infrastructure may consist of code running on the packet capture card which will encrypt and sanitize the data before they get the chance to reach the host computer. At the higher level, this toolset will provide application-specific anonymization through a Scripting Sanitization Language (SiSaL). SiSaL will enable authorized users to anonymize the data in application-specific ways so that both the anonymity of users is protected and the necessary information is provided to the monitoring application. Develop novel applications enabled by the availability of the passive network traffic monitoring infrastructure.
Within LOBSTER, novel applications that were not possible to be developed on top of traditional monitoring systems will be developed. Such applications may be:Accurate traffic characterization for programs using dynamic ports
This application will provide accurate distribution of traffic to applications and will work even for applications that use dynamic ports to communicate, such as peer-to-peer systems. In contrast to our application, traditional traffic characterization methods, such as netflow and IPFIX, are based on static ports and thus are not able to categorize packets belonging to applications that use dynamically generated ports.Spread of zero-day worms
Based on traffic captures at several different sensors, this security-related application will focus on finding worms as soon as they start to spread on the Internet, so as to provide an early-warning system.European Internet measurement service
Virtually any kind of network statistics can be measured at a European scale. This would include information about the kind of services that are most-used by citizens, network security information, quality-of-service, social-cultural and behavioral information, and the use of encrypted channels. The information could benefit member states policy-development, various areas of research, as well as network planning within Europe.- Provide anonymized data traffic information on a regular basis.
Once the monitoring infrastructure is in place, it is planned to provide periodic summaries of anonymized traffic data at regular intervals, possibly on a daily basis. These data, strictly anonymized to protect the privacy of the original Internet users, will be used to detect of Internet trends, to calibrate models of the Internet, and in general to support Internet-related research. Disseminate project results.
Dissemination in this project will have an important and multi-dimensional role:Dissemination of traffic data to interested network researchers
The installed infrastructure will collect traffic data, which will be made available to interested network researchers for further analysis. Such a resource currently exists only in the United States (http://pma.nlanr.net/PMA/), where it has given significant boost to the research and development in the area of network monitoring.Dissemination of project results to ISPs and ASPs
The partners of this project will form the initial core of the monitoring infrastructure, which will be extended by more ISPs and ASPs that will join during the second phase of the project.Disseminate of project results to security analysts
Security experts will be able to use the infrastructure to spot and contain the spread of worms and various forms of cyberattacks.