Real-time Insights Require Specialised Hardware

Real-time Insights Require Specialised Hardware

Bob Braham, CMO, SGI
Bob Braham

Delivering actionable intelligence in real-time from the Big Data is a major challenge before defence agencies today. In an interview to GeoIntelligence, Bob Braham, CMO, SGI, tells us how his company is addressing this critical need of the defence

Can you tell us about your offerings for Big Data Analytics?
SGI offers a range of big data solutions designed to fit the needs of customers working at a variety of stages in their big data deployment process. Its three main big data systems are the SGI InfiniteData cluster, SGI DataRaptor, and SGI UV family.

SGI’s InfiniteData is a cluster computing platform which brings exceptional scale-out performance to a highly flexible storage system for big data insights. The InfiniteData Clusters can adapt to the data demands of its users, growing to encompass multiple racks with thousands of nodes and linear scalability. The clusters are factory integrated with the Cloudera Hadoop system and run on a Linux OS to offer a highly versatile system that can tackle a range of big data problems. InfiniteData Cluster delivers 40 2U cluster nodes and 1.9PB in a single rack, enabling companies to scale-out to meet massive data demands with ease.

The SGI DataRaptor with Mark- Logic Database for NoSQL data analysis offers companies a turnkey operational database solution that comes pre-configured to allow companies to quickly deploy a scalable big data system. The system starts at five nodes with 80 cores, but can expand further to meet a company’s big data needs.

The SGI UV system is a highly scalable in-memory system designed to overcome exceptionally large data-intensive applications and quickly provide insights to its users. Offering up to 64 terabytes of memory, the UV system is one of the largest in-memory systems available and can scale a Single System Image (SSI) to a maximum of 2,048 cores (4,096 threads) because of its innovative NUMAlink® interconnect. The UV system’s huge capacity makes it ideal for data-intensive workloads and is designed to lower IT burden per compute core by consolidating complete workflows in a single system. The UV system also supports GPU-accelerated technology allowing for massively scalable parallel processing systems.

We are now talking about real-time intelligence. Delivering actionable insights in real-time from the massive amounts of structured and unstructured data. What are the challenges involved in this and how do you overcome this?
Real-time insights require specialised hardware that are not typically delivered by the commodity hardware in use today. The typical cluster has too much latency because of a less capable interconnect to provide a rapid stepthrough of lots of data. An answer that is too late, is just no longer of value. Fortunately, SGI provides hardware that is specialised in that — it can store massive amounts of data in RAM memory (up to 64 TB) in a low-latency flat memory address space (on an average a hundred times faster than even flash memory) but at the same time, it is based around commodity components such as the Linux operating system and x86 instruction set processors. This is what is in use by INSCOM today.

Can you explain in detail how SGI supports Activity-based Intelligence (ABI)?
ABI is a broad category of work which SGI does for geospatial intelligence. There is a real-time component to it, which is addressed by our in-memory platform, and a software layer provided by our software partner. There is also a non real-time component to it and this is addressed by Hadoop on top of our Rackable and InfiniteData Cluster platform with customer-created applications (not third party like the above). And then there is a virtualisation component. This gives the government flexibility to build a software layer that is extensible (that is, you can change the software or hardware layer independently). This is our Rackable hardware with various virtualisation offerings both open source and proprietary. Last but not the least is a storage component. This is addressed by our data management solutions such as InfiniteStorage hardware and DMF software.

You have recently partnered with GIS Federal and NVIDIA for a project of United States Army Intelligence and Security Command? Can you tell us about it?
The SGI/GIS Federal searchable geospatial database system runs on a 10 terabyte (TB) SGI UV 2000 system, incorporating 16 NVIDIA Tesla K20x GPUS for GPU-accelerated parallel computing.

The deployment runs a GAIA distributed database, used by the Army INSCOM, to rapidly render complicated geospatial features and heat maps to more accurately pinpoint potentially dangerous activity at a given point. The SGI UV highly scalable shared-memory architecture allows GAIA to analyse and render a high volume of data streams at a lightning fast pace to provide insights in a matter of seconds. The INSCOM GAIA system is the only system with 16 GPUs in such a large single-system image.

What, according to you, are the potential geographies/ markets for SGI in defence sector?
SGI sells to US, and our allies in Europe, Asian, and Australia.

What are the key trends as far as Big Data is concerned?
The adoption of in-memory technologies has been a leading trend in the Big Data market as organisations continue to accumulate massive stores of data that need to be accessed quickly to provide insight. In-memory systems enable fast access to information, allowing businesses to pull real-time insights that can be applied to real world situations.

We are also seeing a more general trend of organisations realising the value of big data systems, adopting technology that was traditionally used solely for high performance computing workloads.