Multi-int intelligence: Effective Multi-Sensor Data Fusion

Multi-int intelligence: Effective Multi-Sensor Data Fusion


<< Although experienced military analysts can develop tactical situation awareness by combining multisensory-derived information fragments, the automation of this process presents significant challenges… >>

The advent of new information technologies has enabled commanders to know more about the enemy, plan faster, make decisions faster and synchronise sensors and shooters to create desired effects on the battlefield. Network Centric Warfare (NCW) focuses on the combat power that can be generated from the effective linking or networking of the war fighting enterprise.

NCW is executed by means of three vertically linked grids: the sensor, shooter and information grids. All three are interconnected – actions flow from sensors through decision makers to shooters. Tactical data fusion supports effective operational planning and its execution by attempting to provide battlefield commanders with complete and timely situation awareness. Traditionally, much of the sensor derived information has been analysed by humans. As a result of the ever increasing volume of sensor data and the continually shrinking decision making timelines, increased automation of the situation development process is required that involves development of strong algorithms catering to the tactical requirements.

There is, thus, a need to have a requirements based view of tactical situation development and identify information system with algorithm development principles that support effective data fusion system design.

Tactical Battlefield Characteristics
Ideally, tactical situation development systems should be designed using a top down requirements-driven approach. The overall information requirements would first be identified; next, the optimal sensor mix meeting those information requirements would be determined and necessary enhancements to existing systems would be selected. However, due to a wide range of doctrinal, financial and practical considerations, most data fusion systems are actually developed using a more bottom-up approach that focuses primarily on information combination methodologies.

Tactical battlefield characteristics can be classified under two main heads namely, static analysis requirements and dynamic analysis requirements.

Static Analysis Requirements

Sensor Performance Limitations: All practical sensor systems possess performance limitations, including most notably limited range sensitivity and resolution. The use of multiple single-source sensors provides at least three means for overcoming individual sensor limitations.

  • Overall system performance can often be improved by using multiple sensors
  • Each sensor with overlapping coverage can enhance collection system reliability
  • Use of multiple cooperating sensors can often extend the inherent capability of a sensor class

For instance, although SIGNT sensors fundamentally provide azimuth – only target information, multiple cooperating SIGNT collectors using triangulation or Time Difference of Arrival techniques permit range determination, as well as enhanced azimuth resolution. By the same token, target and signal masking by terrain and foliage can potentially be reduced by employing multiple single source sensors operating from significantly different vantage points.

Sensor Capability Limitations: All sensor classes suffer limitations in terms of their ability to measure target attributes. If mapping between target attributes and sensor capabilities for a number of sensor classes is carried out, the two observations are apparent. First, no sensor class provides complete target characteristics. Second, sensor classes tend to measure overlapping subsets of target attributes. The combinations of information derived from relatively independent measurements potentially generate a more complete target characterisation than if the measurements were highly correlated. Thus, sensor classes that measure relatively independent attributes tend to be the most complementary. For example, since MTI radars measure features associated with the object’s size and motion, SIGNT sensors provide information about a target’s onboard radar and or radios, and imagery (satellite/ UAV) characterises object’s physical appearance, the three sensors provide highly complementary information.

Object Organisations: In addition to detailed targeting level, information against individual tactical objects, effective situation understanding requires a more global evaluation, including the identification and attribution of military units at multiple echelons. Since most sensors provide information about individual physical objects (radar, tanks, command posts, aircraft), multiple object organisations must be recognised based on observed or inferred relationships among objects. Multiple object templating typically relies heavily on historical and/ or doctrinal knowledge.

Physical World Constraints: As the terrain can impose significant constraints on mobility, observability and vulnerability of both individual entities as well as organisations of entities, effective situation development requires that sensor data analysis be sensitive to relevant domain constraints. The sensitivity of fusion tasks to domain context can vary greatly. Strictly, statistical-based algorithms tend to be appropriate for tracking targets that obey wellunderstood physical motion models like tracking a ballistic projectile or those which are not highly constrained by the environment like tracking ships in the open ocean. Although a ship’s motion is constrained by performance bounds such as its maximum velocity, acceleration and turning rate – such parameters provide very weak constraints on target motion. Ground-based vehicles, on the other hand, exhibit complex behaviours that can be highly constrained by domain features such as roads, rivers and terrain. Consequently, for ground target applications, strictly statistical target tracking algorithms tend to be under constrained.

Real-Time Analysis Requirements: In general, data fusion algorithms need to operate within a time scale appropriate to the applications. Domain sensitive target tracking algorithms for air defence applications, for instance, might require analysis to be completed with less than 1 sec latency, resulting in potentially very demanding algorithm performance requirements. The performance requirements for many higher level fusion tasks, on the other hand, operate on a much longer decision cycle. The identification of a river bridging operation is an example of a considerably less time critical analysis task. Latency associated with both the local processing and distribution of sensor-derived information imposes additional constraints on the timeliness of intelligence generation. Since the computational resources in a tactical environment tend to be limited, achieving real-time performance across all applications can present a significant challenge.

Dynamic Analysis Requirements

Dynamic, Time Varying Situations: In a battlefield, targets may be moving at one instant of time and stationary at another, communicating during one interval and silent during another. The four mutually exclusive target states can be defined as:-

  • moving/ non-emitting
  • moving/ emitting
  • Non-moving/ non-emitting
  • Non-moving/ emitting

Since many entities will change between two or more of these four states over time, the situation awareness product must be continuously maintained. As a result, data fusion algorithms require a recursive analysis element. Table I shows mapping between these four target states and a wide range of sensor classes. As can be observed, the ability to track entities through these state changes effectively demands multiple source data.

Complex Behaviour of Individual Objects: In general, individual targets exhibit complex patterns of behaviour that can help discriminate object classes and identify activities of interest. Consider the scenario of the movement of a missile transporter/ erector/ launcher (TEL) vehicle between the two time intervals, that is, time t0 and time t6 as under:-

  • At t0, the vehicle is in a location that makes it difficult to detect.
  • A t1, the vehicle is moving along a dirt road at velocity V1.
  • At t2, the vehicle is moving and begins communicating with its support elements.
  • At t3, the vehicle is travelling off-road at velocity V3 along a minimum terrain gradient path.
  • At t4, the target has stopped moving and is beginning to erect its launcher.
  • At t5, just prior to launch, radar emissions begin.
  • At t6, the vehicle is travelling to a new hide location at velocity v6.

Table 2 identifies sensor classes that could contribute to the detection and identification of the various target states. At the lowest level of abstraction, observed behaviour can be interpreted with respect to a highly local perspective as indicated in column 6. By assuming that the object is executing a higher level behaviour progressively more global interpretation can be developed as listed in columns 7 and 8.

Coordinated Behaviour of Groups of Objects: Since individual battle space objects are typically organised into operational or functional-level units, observed behaviour among groups of objects can be analysed to generate higher level situation awareness products.

Limited Sensor Assets and Sensor Availability: Due to sensor resource limitation, available collection assets must be effectively and efficiently managed. Even as the generation of unnecessary or redundant information can overwhelm the data analysis and information dissemination process, ineffective collection management may prevent more important information from being collected and utilised. Once adequate information about a particular target has been obtained, the utility of continued collection may be low. Thus, collection requirements must be constantly re-evaluated based on both the commander’s guidance and the current situation awareness product short falls. Because only a limited number of the supporting assets might be locally controllable and only a subset of these assets might be available for re-tasking, collection management will necessary be sub-optimal. Further, complicating the collection management process are latencies associated with onboard sensor analysis and reporting.

Deception: In general, the use of extensive problem domain knowledge and the fusion of multiple source data can minimise the impact of intentional deception on the situation development process. Deep problem knowledge permits the identification and exploitation of inconsistencies in sensor-derived measurements. Although single sensor deception is relatively easy to perpetrate, effective multisource deception can be considerably difficult. For example, while inflatable rubber tanks might be indistinguishable from real ones when based strictly on overhead imagery, the consideration of SAR and/ or FLIR imagery may reveal that the targets do not process either the electromagnetic signature or thermal properties of operational tanks. Thus, complementary sensors can play a key role in identifying deliberate deception.

Situation Awareness Requirements
Based on the above tactical battle space characteristics, three high-level situation awareness development requirements can be identified. Each of these requirements is discussed in the succeeding paras.

System Level Requirements: Three classes of system level requirements exist – adequate input information; adequate communication bandwidth and computational capability, and effective collection management. Performance and capability limitations of individual sensors can be at least partially offset by employing multiple single-source sensors as well as multiple-source sensors. High performance hardware and adequate communication bandwidths are required to support real-time operational requirements of situation awareness development. Based on the commander’s guidance, collection plans must be developed for all organic assets that optimally support the information needs of the situation development process. In addition to identifying the optimal spatial coverage of the sensor suite as a function of time, effective control requires the selection of individual sensor parameters such as mode, frequency band and resolution. Due to the dynamic nature of the battle space, dynamic re-planning is an integral part of the collection management process.

Algorithm Requirements: Based on the aforesaid discussion, four primary algorithm requirements can be identified which are robustness, context-sensitivity, efficiency and recursive update potential. In general, achieving robust performance requires modelbased reasoning that employs deep problem domain knowledge. This ability to generate multiple level of abstraction characterisations of the battle space is a critical aspect of effective situation awareness development. In many real-world applications, the use of non-context sensitive fusion algorithms tends to generate ineffective, inaccurate and inconsistent products. Strictly statistical based tracking of ground vehicles might exhibit extensive track association and fragmentation errors. On the other hand, the consideration of non-sensor derived context such as local, natural and cultural features, provide constraints that can focus the reasoning process as well as specialise the fusion product. The potentially valuable, context-sensitive reasoning can require the maintenance of extensive domain databases and support for sophisticated spatial reasoning. Due to real-time performance requirements and the limited computational resources available in a tactical environment, the computational efficiency of fusion algorithms is an important consideration. Problem-solving approaches that possessed large state space representations and rely on global optimisation, for instance, tend to generate high computational requirements. When fusion tasks do not lend themselves to efficient closed form solutions, algorithm efficiency can often be enhanced by problem decomposition, the use of powerful heuristics and/ or top down reasoning. Top down reasoning, in turn relies on multiple level of abstraction and multiple resolution representations that support the determination and hierarchical refinement of global solutions.

Due to the time varying nature of tactical situation and latencies associated with both the analysis and dissemination process, dynamic truth maintenance is an underlying requirement in many data fusion applications. As additional information becomes available, refinements or even significant corrections in interpretation may be required. Viewed from the perspective, truth maintenance is really a special form of temporal reasoning.

Support Function Requirements:
Data fusion automation requires two general classes of infrastructure support, database management system and generic functional software. Database requirements include support to algorithm development and efficient access and manipulation of potentially extensive non-sensor derived domain knowledge bases. At a minimum, fusion algorithms can require support for spatial, temporal and hierarchical reasoning. The database management system, in turn must provide efficient support to these reasoning classes.

An effort has been made to present a brief view on the requirements of the tactical situation development process. Based on this, a matrix depicting relationship between the characterisation of tactical problem domain and fusion requirement can be drawn which can lead to effective multi-sensor data fusion.