Home Articles Virtual Image Generation from the Linear Array Image

Virtual Image Generation from the Linear Array Image

Ali Azizi
Centre of Excellence for Geomatics Engineering and Disaster Management,
Department of Surveying Engineering,
Faculty of Engineering, University of Tehran

Mohammad Saadatseresht
Centre of Excellence for Geomatics Engineering and Disaster Management,
Department of Surveying Engineering,
Faculty of Engineering, University of Tehran

The so called push broom images are generated by the linear array technology by which successive lines on the ground are systematically scanned. Consequently, these images have somehow a dynamic geometry in the sense that each image line has its own exterior orientation parameters leading effectively to a multi-projection image. This dynamism in geometry complicates the space resection intersection operations with rigorous mathematical models and necessitates the incorporation of the observed values for the satellite trajectory. Alternatively, the generic approach may be utilized. However, this approach has also its own shortcomings regarding the requirement of large number of the GCPs and resulted instability of the mathematical solution. In this paper we propose a pre-processing stage through which a virtual single projection image is generated from the linear array image by intersecting each pixel position in the scan lines with an approximate digital elevation model (DEM) by a forward intersection approach. This is then followed by an inverse projection to the virtual image plane and gray shade interpolation. The generated virtual image more or less satisfies the geometry of a real single projection image and hence can be treated with conventional mathematical models used in photogrammetry. The errors in the generated virtual image are partly due to the approximations of the DEM and partly due to the attitude and altitude variations of the scan lines. Assuming that the DEM error is negligible, the orientation and position variations of the scan lines may be regarded as a sort of systematic image displacement which can be handled by a rigorous self calibration or similar mathematical models. The potential of the proposed approach, as far as the impact of the relief displacement is concerned, is investigated using a simulation strategy.

Typically, two different approaches of rigorous and generic mathematical models are utilized for the geometric correction of the linear array images. The former approach to be implemented inevitably takes the form of a multi-projection model whereas with the latter method multi-projection assumption is not necessary. Thus, the main drawback of the rigorous sensor model is its complex mathematical model which requires some approximations for the exterior orientation parameters of each scan line. With the generic sensor model, although much simpler mathematical model is used, the number and distribution of GCPs are crucial. In this paper an alternative approach is proposed by which a virtual image having a single projection centre is first generated using the linear array scan lines. The virtual image is then regarded as a conventional camera generated image. Space intersection is then performed on the stereo-virtual images to generate the object space coordinates. In the section that follows, the adopted procedures for the generation of the virtual image are outlined.

Virtual image generation
The geometry of the single and multi-projection imaging devices is different (Fig.1.). The difference lies in the fact that with the single projection images the relief and tilt displacements are radial from the nadir point and iso-centre respectively. These points are unique in a single-projection image.

Fig.1. Multi and single-projection images.

With the linear array images, on the other hand, each scan line has its own nadir point and iso-centre. Therefore, there are as many nadir points and iso-centres as the number of the scan lines. Fig.2, visualizes different geometry of these two classes of images.

Fig.2. (a) Relief displacement in a single-projection image,
(b) Relief displacement in a multi-projection image.

With a flat terrain, in the absence of tilt displacement, the geometry of single and multi-projection images is identical, since the relief displacements of the pixels are equal. However, in a non-flat terrain as indicated in Fig.2., the two classes of images have different displacement patterns. To generate a virtual single-projection image, relief variation on the ground must be taken into consideration, without which the single-projection image can not be generated. Fig.3. depicts graphically the simplified relationship of the virtual single-projection image and the linear array image.

Fig.3. Virtual image generated from the scan lines using a digital elevation model.

Assuming that the approximate satellite trajectory is known, pixels of the scan lines can be transferred to the object space via collinearity condition equations. This is performed by a forward intersection of the collinearity equation with the DEM surface using the following relations:

where XA, YA, ZA are the object coordinates of the pixels; R1,…,R3 are the direction cosines of the orientation angles of the scan lines; x is a vector containing the 3d scan line coordinates of each pixel. XC, YC and ZC are the 3d position of the projection centre of each scan line estimated via the approximate satellite trajectory. To evaluate equation 1, the rotation matrix is considered to be an identity matrix and the height values (i.e. ZA), is extracted from the available DEM. Having transferred the pixels to the object space, an inverse transformation given by equation 2 is employed to transfer the points in the object space into the virtual single projection plane:

where x, y, f, are the 3d coordinates of the pixels with respect to the virtual camera projection centre coordinate system; M is the rotation matrix of the virtual camera; and X is given by:

where Xc, …, Zc are the 3d coordinate of the virtual camera projection centre with respect to the object space coordinate system. The values for the exterior and interior orientation parameters of the virtual camera can be assigned arbitrarily. However, to acquire a virtual image with the same resolution as the linear array image, the virtual camera focal length should be equal to the focal length of the linear array imaging camera. Each transferred pixel to the virtual plane has its own grey value. An off-line grey value interpolation is then carried out to generate the final virtual image. To avoid the off-line grey shade interpolation, the whole process described above can be performed in a reverse order. That is, a 2d out put array is constructed in the virtual image plane. By reverse transformation these pixels are first transferred to the object space and then to the linear array scan lines and simultaneously a density value is resampled and assigned to the out put pixels. This latter approach is implemented in the present project. Fig.4. shows a simulated stereo-virtual image generated from an ETM+ satellite image and a DEM. Note that in this simulated stereo-pair, the satellite trajectory and the off-nadir viewing angle for the stereo-pair is assigned arbitrarily.

Having generated the stereo-virtual images, conventional space intersection may be applied to generate the object space. Since the interior orientation parameters of the virtual image are already established, the mathematical model for the space intersection can be a rigorous collinearity model. However, as stated above, the virtual image has inherited several systematic errors, the most important of which are the neglected attitude parameters of each scan line. These image distortions can be handled via a self calibration collinearity equation. Alternatively, a much simpler approach can also be adopted using a direct linear transformation model which corrects the scale affinities in the image space for the virtual camera:

where L1,…, L11 are the DLT transformation parameters. The DLT model can be further elaborated with the inclusion of the higher terms for the corrections of the higher order distortions.

To evaluate the proposed method and for the purpose of thorough analysis of the influence of the DEM on relief displacements of the push-broom data, a DEM was generated by a known mathematical model. Fig. 5 gives the isometric view of the simulated DEM.

Two stereo push-broom images where also generated in ideal situation, i.e. without scan lines attitude and altitude variations. Based on these data, the following virtual images were generated:

  1. virtual images obtained by a perfect DEM,
  2. virtual images obtained by different approximations of the DEM.

The approximate DEMs were generated by successive smoothing the original DEM leading to the DEMs with 75%, 50%, 25% and 12.5% roughness factors respectively.

Space intersection were then carried out for a dense network of image points on the virtual images and their corresponding mesh on the object space were produced using space intersection by both rigorous and DLT mathematical models. The accuracies were then evaluated on the check points. To have better understanding of the success of the proposed method, a simple DLT space intersection was also carried out on the original linear array data. The results for all datasets are presented in Table 1.

Table 1. accuracy on check points for the stereo-virtual image and the linear array images.

In this paper a study is conducted on the nature of the relief displacements as occurred on the push-broom images. The results of the simulated data presented in Table 1. indicate that relief displacement in linear array images is particularly problematic during the rectification process. The effect of relief displacement can be dramatically reduced by a virtual single-projection image if a DEM of the imaging area is available. In particular it was demonstrated that the available DEM need not be accurate and even a general trend of the DEM can well reduce the effect of relief displacement. This trend can easily be estimated by a surface fitted to the ground control points. The preliminary conclusion to be drawn from this study is that the DLT approach for the space resection and intersection of the linear push-broom images (Almanadili and Novak,…) can be greatly improved by a virtual image which is generated simply using the general trend of the DEM.

These results, however, should be validated by the real data. The following tasks should be the subjects for further investigation:

  • evaluating the relief displacement on the real push-broom images,
  • investigating the influence of the scan line attitude and altitude variation on the accuracy of the generated virtual image,
  • incorporation of a self calibration space resection intersection and/or inclusion of the higher order terms in the DLT model on the stereo-virtual images to reduce the influence of the attitude and altitude variation,
  • investigating the utilization of the satellite ephemeris for the virtual image generation,
  • investigating the possibility of epipolar resampling of virtual images by conventional methods.


  • Almanadili and Novak ………….