Instituto de Investigacións Tecnolóxicas Development, deployment and validation of an oceanographic virtual laboratory based on Grid computing David Mera Pérez Brno, Czech Republic, October 21th 2013 Index 1. Context and motivation 2. Main goals 3. Retelab project - Virtual laboratory development 4. Sentinazos project - Virtual laboratory validation 5. On going work Context Objectives Laboratory development Testbed Future Index 1. Context and motivation 2. Main goals 3. Retelab project - Virtual laboratory development 4. Sentinazos project - Virtual laboratory validation 5. On going work Context Objectives Laboratory development Testbed Future Context and Motivation • The oceanographic research community has access to huge amount of available datasets • Researchers must deal with new issues such as how to store, process and make the best of available datasets • The study of the ocean requires multidisciplinary teams • Several levels of computer skills Context Objectives Laboratory development Testbed Future Context and Motivation • Distributed computing: Grid computing – Resource sharing via Internet – Low cost – Security – Open Standars – Virtual Organizations management Context Objectives Laboratory development Testbed Future Index 1. Context and motivation 2. Main goals 3. Retelab project - Virtual laboratory development 4. Sentinazos project - Virtual laboratory validation 5. On going work Context Objectives Laboratory development Testbed Future Main goals • To develop a user-friendly distributed computational environment based on Grid computing. • To develop an oceanographic application to test the Grid environment. – An oil spill automatic detection system based on the analysis of satellite Synthetic Aperture Radar imaging. Context Objectives Laboratory development Testbed Future Index 1. Context and motivation 2. Main goals 3. Retelab project - Virtual laboratory development 4. Sentinazos project - Virtual laboratory validation 5. On going work Context Objectives Laboratory development Testbed Future Retelab – Easy and accesible tool – Computer skills should be minimum or even unnnecesary – Open source – Security – Distributed storage and computational capacity Basis Context Objectives Laboratory development Testbed Future Retelab Architecture Context Objectives Laboratory development Testbed Future • Command line interface • Digital certificates managed by users • Advanced computer skills are required • System user registration • Complex user account management • Web interface based on portlets • Credential managed by the system • Minimum computer skill are required • Single Sign-On registration system • Role base access system VS User access and registration system Retelab Traditional Grid systems Context Objectives Laboratory development Testbed Future Retelab User access and registration system • Arquitectura Context Objectives Laboratory development Testbed Future Retelab Distributed storage system • Web interface integration • Based on Metadata - ISO 19115, Geographic Information • Integration of visualization tools Context Objectives Laboratory development Testbed Future • Visualization tools Retelab Distributed storage system Integrated Data Viewer Live Access Server Context Objectives Laboratory development Testbed Future Retelab • Traditional Grid systems – User interaction was required – Complex submission systems • – Grid metasqueduler • Web integration • To make decisions on behalf of users • To facilitate the optimal utilizations of available resources • It undertakes the tasks for resource discovery, job scheduling, executing, monitoring and output retrieval. Job submission and monitoring system Context Objectives Laboratory development Testbed Future Retelab Job submission and monitoring system Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Retelab Use case Context Objectives Laboratory development Testbed Future Index 1. Context and motivation 2. Main goals 3. Retelab project - Virtual laboratory development 4. Sentinazos project - Virtual laboratory validation 5. On going work Context Objectives Laboratory development Testbed Future Sentinazos • The international trade is mainly supported by maritime transport • The intensive traffic sails along the Exclusive Economic Zones (EEZ) of countries and generates important pollution problems. • Only the 7% of oil spills come from catastrophes like tanker and oil platform accidents. Introduction Context Objectives Laboratory development Testbed Future Sentinazos Introduction Context Objectives Laboratory development Testbed Future Sentinazos Introduction Context Objectives Laboratory development Testbed Future Sentinazos Introduction Context Objectives Laboratory development Testbed Future Sentinazos • Synthetic Aperture Radar Introducción Context Objectives Laboratory development Testbed Future Sentinazos • Synthetic Aperture Radar - Examples Introduction Prestige catastrophe (Envisat, 17/11/2002) Galician coast (Envisat, 13/10/2008) Context Objectives Laboratory development Testbed Future Sentinazos Goal Hypothesis 1. Wind data could be used to improve the SAR image segmentation 2. Oil spill shape could be relevant to correctly classify them Goal • To develop an automatic oil spill detection system based on SAR images and focused on the galician coast. This system should take advantages of win data as well as candidate shape features to improve detection results Context Objectives Laboratory development Testbed Future Sentinazos • Dataset- 47 SAR images from the Envisat Methodology Context Objectives Laboratory development Testbed Future Sentinazos • Dataset- 47 SAR images from the Envisat Methodology Context Objectives Laboratory development Testbed Future Sentinazos • System architecture Methodology Context Objectives Laboratory development Testbed Future Sentinazos Methodology - Segmentation Wind intensity data. CMOD5 modelSAR image. Envisat, 01/06/2007 Context Objectives Laboratory development Testbed Future X Y Wind I.A. Intensity 2003 1212 5,4 41,3 0,021 1233 5298 3,2 23,74 0,5421 6832 4523 3,8 27,21 0,00234 X Y Wind I.A. Intensity 2003 1212 5,4 41,3 0,021 1233 5298 3,2 23,74 0,5421 6832 4523 3,8 27,21 0,00234 Sentinazos Methodology - Segmentation Wind intensity data. CMOD5 modelSAR image. Envisat, 01/06/2007 Context Objectives Laboratory development Testbed Future X Y Wind I.A. Intensity 2003 1212 5,4 41,3 0,021 1233 5298 3,2 23,74 0,5421 6832 4523 3,8 27,21 0,00234 X Y Wind I.A. Intensity 2003 1212 5,4 41,3 0,021 1233 5298 3,2 23,74 0,5421 6832 4523 3,8 27,21 0,00234 Sentinazos Methodology - Segmentation Wind intensity data. CMOD5 modelSAR image. Envisat, 01/06/2007 Context Objectives Laboratory development Testbed Future X Y Wind I.A. Intensity 2003 1212 5,4 41,3 0,021 1233 5298 3,2 23,74 0,5421 6832 4523 3,8 27,21 0,00234 X Y Wind I.A. Intensity 2003 1212 5,4 41,3 0,021 1233 5298 3,2 23,74 0,5421 6832 4523 3,8 27,21 0,00234 Sentinazos Methodology - Segmentation Wind intensity data. CMOD5 modelSAR image. Envisat, 01/06/2007 Context Objectives Laboratory development Testbed Future Sentinazos Context Objectives Laboratory development Testbed Future Methodology - Characterization Sentinazos • 17 shape characteristics • 2 physical characteristics • 2 contextual characteristics PCA Context Objectives Laboratory development Testbed Future Methodology - Characterization Sentinazos • Classification – Clustering of oil spills and look alikes. – Evaluation of the characteristics vector. – Developed classifiers • Artificial Neural Network • Decision Tree Methodology - Classification Context Objectives Laboratory development Testbed Future Sentinazos Methodology - Postprocessing Context Objectives Laboratory development Testbed Future Sentinazos Validation subset Test subset Sentinazos False positives Sentinazos False positives ANN 85,7% 85,2% 92,9% 96,3% Decision tree 92,9% 85,2% 92,9% 92,6% Parallel version HW characteristics Segmented pixels Segmented candidates Processing time Improvement OpenMP + TBB Intel Core 2 Duo Processor E6400 (2.13GHz), 3GB RAM 210064 670 45,33 sg. 27,07% 36366 108 18,57 sg. 23,89% 27211 65 15,29 sg. 26,81% Results Context Objectives Laboratory development Testbed Future Sentinazos Results • Desktop application Context Objectives Laboratory development Testbed Future Sentinazos Results • Portlet Grid Context Objectives Laboratory development Testbed Future Sentinazos Results Context Objectives Laboratory development Testbed Future Future • Sentinazos – New SAR data sources • Sentinel-1. Currently, the Envisat is not operative – Integration of new remote sensing sources • Buoys • Optical sensors, • Aircraft missiones - SLAR sensor • … – Algorithm improvement • Contextual data • New wind model • Multi classifier sytems Context Objectives Laboratory development Testbed Future Instituto de Investigacións Tecnolóxicas Thank you Development, deployment and validation of an oceanographic virtual laboratory based on Grid computing