Spacecraft constantly scans the Earth, creating hundreds of gigabytes of new data products daily. Working with this ever-growing mass of information has made ESA’s Earth Observation Directorate a pioneer user of powerful Grid computing. Now ESA is participating in an ambitious project that applies this same technique to information distribution and retrieval, with the aim of creating ‘Digital Libraries’ for global scientific collaboration.
The Grid concept is named after the electricity grid, and works in the same way: a geographically dispersed (or ‘distributed’) network of computers provides users with access to advanced computing services, processing power and memory, enabling the solution of complex tasks beyond the capabilities of a single machine or local network. Within the field of Earth Observation, use of a Grid enables researchers to access large amounts of spatial and temporal data and perform high-level processing as well as complex applications such as data fusion and modelling, and all within previously impossible timescales. Applying a Grid, tasks that might otherwise take days to accomplish are reduced to a few hours, minutes or even seconds.
New mosaics of the world can be put together as soon as fresh satellite imagery becomes available, as can up dated atmospheric maps, vegetation indexes or ocean chlorophyll products. Grid computing shows changing Mediterranean chlorophyll levels
And rather than having to directly download vast amounts of data in order to locally evaluate new processing algorithms, researchers can directly combine their algorithm with vast amounts of data within the virtual Grid system to test and tweak it based on outputs that are now made available in near-real time, and with much larger geographical or temporal coverage than previously feasible.