Fairfax, US: The National Geospatial-Intelligence Agency (NGA) needs new efficient data storage management technologies such as data deduplication, tiered storage and data archiving to manage exabytes of data with shrinking budgets, said David Jones, Director of data centre services for the NGA in his keynote address at the Storage Decisions National System for Geospatial Intelligence (NSG) Summit 2011.
Jones observed that storage systems and technologies can help the intelligence community meet the challenges it faces from unprecedented data growth in the face of budget cuts. “Information technology is a key enabler to the NGA’s mission and storage may be the most important component,” he added. “We face a daunting increase in the amount of data we have to store and process to make the intelligence products we do today at the NGA.”
According to Jones, the intelligence community’s data is rapidly approaching exabyte level. It has to keep critical data for 30 years, and its information-gathering sensors generate a great deal of data that must be tagged and frequently searched. Like other US government agencies, the NGA is expected to reduce the footprint and power requirements of its storage and data centre.
To manage and make the most of the data, the NGA needs to deduplicate to slow growth, tier data to keep high-value data on high-performing storage and move the rest to cheaper tiers, and use metadata tagging to determine its value for retention.
“We’re going to be transforming the structure of our data,” Jones stated. “We will look at how data is discovered and accessed across cloud environments, how we will be able to share data across the community, how capacities are managed virtually and we will look at global namespace. Bringing storage bits down and reducing size and power is another priority.”
Cloud storage might help, he said, but it needs to be more clearly defined.
“There is a particular type of cloud that’s called fog, and fog tends to obscure what you’re really seeing,” he said. “So we have to be careful about the way we describe the cloud, and really understand the features it provides, so we can make decisions that are appropriate to our needs.”