Prof. Dr. Armin Gruen
AInstitute of Geodesy and Photogrammetry, ETH Zurich
The modeling of large sites has received much attention in recent years.
This was triggered on the one side by the increased interest of UNESCO and other supranational and national organizations and on the other side by the new technologies available for recording, processing, administration and visualization of the data. As can be see from the UNESCO World Heritage List (whc.unesco.org), many recent additions can actually be classified as „Large Sites“, both in terms of culture and nature. In a press release (No. 2002-77: „For UNESCO, Space Technologies should be Harnessed for Sustainable Development“) UNESCO has stressed the use of satellite imagery for monitoring World Heritage sites. Lately many conferences are devoted to this issue (e.g. the International Symposium on „Conserving Cultural and Biological Diversity: The Role of Sacred Natural Sites and Cultural Landscapes“, 30 May – 2 June in Aichi, Japan, on occasion of the World Exhibition 2005). Conservation and management of these sites rely heavy on the availability and timeliness of data.
On the technology side we have nowadays at our disposal a vast array of relevant and efficient data acquisition tools: Highresolution satellite images/middleeast/2006/may-june, large format digital aerial cameras, hyperspectral sensors with several hundreds of channels, interferometric radar from space and aerial platforms, laserscanners of aerial and terrestrial type, partially with integrated cameras, model helicopters with off-the-shelf digital cameras, panoramic cameras and a large number of diverse customer still video cameras and camcorders. This is augmented by GPS/INS systems for precise navigation and positioning.
Automated and semi-automated algorithms allow us to process the data more efficiently than ever before and Spatial Information System technology provides for data administration, analysis and other functions of interest. Finally, also visualization and animation software is becoming affordable at better functionality and lower costs. This sets the scene for a totally new evaluation of the tools and techniques for large sites cultural heritage recording and modeling.
We have conducted in the past a number of projects that have shown the potential, but also the limitations of some of these new technologies. Among those are Mount Everest, Ayers Rock/Australia, Kunming/China, Bamiyan/Afghanistan, Geoglyphs of Nasca/Peru, Tucume/Peru, Inka settlement Pinchango/Peru, Machu Picchu/Peru (in work), Xochicalco/Mexico.
In this paper we will report about our experiences with large site modeling and also address some of the new algorithmic approaches and software modules for the 3D processing of digital satellite and aerial images/middleeast/2006/may-june, that we have developed in the course of these projects.
Here we will limit ourselves to a brief description of the Pinchango Alto and Tucume projects, both in Peru.
2. Photogrammetry and remote sensing
Photogrammetry and remote sensing are image-based techniques for the extraction of metric and semantic information from images/middleeast/2006/may-june. Originally terrestrial photogrammetry, aerial photogrammetry and satellite remote sensing developed along separate lines, both in terms of types of sensors used and processing methodology and tools. Today, within an almost totally digital environment, we see a strong trend towards convergence. This opens the path for a much more cost-efficient use of a variety of different sensor data and processing tools.
We will concentrate in this paper on the processing of satellite and aerial images/middleeast/2006/may-june, because they are most relevant when it comes to large site modeling. Figure 1 shows the workflow of the photogrammetric techniques used to turn images/middleeast/2006/may-june into hybrid 3D models („hybrid“ meaning mixed models including geometry and texture).
Figure 1: Workflow and products of the photogrammetric/remote sensing process
3. Relevant satellite sensors and new aerial digital cameras
The development and increased availability of highresolution, multispectral and stereo-capable satellite sensors and of a new generation of digital large format aerial cameras is very crucial for the efficient modeling of large sites.
Table 1 shows an overview of highresolution satellite sensors (including medium resolution ASTER because of its good availablity and low costs), which might be useful in cultural heritage applications. There is a great variety of image products available in terms of geometrical resolution (footprint), spectral resolution (number of spectral channels) and costs. All images/middleeast/2006/may-june of Table 1 are acquired with digital sensors, using Linear Array CCD camera technology. For precise processing this requires a particular sensor model and the related special software There are and have been also a number of film-based photographic satellite cameras in use (Jacobsen et al., 1999). This includes the US Corona satellite (2-3 m footprint, B/W, stereo, US$ 24 for a scanned image). The availability of images/middleeast/2006/may-june and the costs can be checked through a number of image providers over the Internet (see Table 1). It is very important to select the right product for a particular task.
Table 1: Main characteristics of high and medium resolution pushbroom sensors carried on satellites. L = along-track; C = across-track, PAN = panchromatic, MS = multispectral
We are witnessing right now a move of photogrammetry towards the use of large format digital aerial cameras. A number of manufacturers are offering their products since 2000 and these cameras have found their way already to many customers. Table 2 gives an overview of the current status in digital aerial large format camera development. We define „large format“ as having more than 10 000 pixels in one image format direction. There are many consumer-type still video cameras on the market, some of them with up to 16 Mpixel image format, but non of them will even closely match the format of these professional cameras.
Table 2: Large format digital aerial cameras (MS = Multispectral, PAN = Panchromatic)
4. New methods for digital photogrammetric processing
The new generation of sensors have a number of particular properties which require new approaches in processing, if the inherent accuracy potential shall be used. images/middleeast/2006/may-june from CCD sensors do have a much larger dynaminc range than film-based images/middleeast/2006/may-june, so there is more detailed radiometric information present in those images/middleeast/2006/may-june. This is important in particular in areas of shadows and areas close to saturation. Linear Array sensor do have almost parallel projection in flight direction, which leads to less occlusions and gives better orthoimage products. Linear Array imagery, if acquired in multi-image mode, e.g. by Three-Line-Scanners or Multi-Line-Scanners, has 100% overlap for all strip images/middleeast/2006/may-june over the same area. This delivers better precision and reliability of results. Finally, Linear Array imaging systems are using GPS/INS sensors for position and attitude determination of the imaging sensor, which can be used advantageously at different stages of the processing chain.
Taking into consideration these facts and other parameters and constraints, we have developed some new methods and the related software packages for the high accuracy processing of aerial and satellite Linear Array images/middleeast/2006/may-june. For the aerial case we have developed a complete software system in cooperation with Starlabo Inc., Tokyo, the manufacturer of the Three-Line Scanner STARIMGER, consisting of the following modules:
+ TLS-SMS: Userinterface
Image measurement in mono and stereo
3-ray forward intersection (point positioning)
Image and shadow enhancement
+ TLS-IRS: Quasi-epipolar rectification to plane or via DTM/DSM Orthoimage generation
+ TLS-LAB: Sensor/trajectory modeling, georeferencing/triangulation
Automatic and semi-automatic tiepoint generation
+ TLS-IMS: Image matching for DSM/DTM generation
DSM/DTM modeling and interpolation
+ Feature/object extraction, e.g. city modeling: CC-Modeler, CC-TLSAutotext
This software can also be used for other Linear Array aerial camera systems and for single farme systems. For instance we have applied it already to ADS40, DMC and Vexcel UltraCam images/middleeast/2006/may-june. For the satellite image case we have developed a modified version of this software, called SAT-PP (Satellite Image Precision Processing), with similar functionality as described before. The key difference to the aerial case is the use of other sensor and trajectory models.
In recent years we have done a number of experiments and tests with TLS/STARIMAGER aerial images/middleeast/2006/may-june (Gruen and Zhang, 2003a, Zhang and Gruen, 2004) and with satellite stereo images/middleeast/2006/may-june from SPOT (Poli et al., 2004), IKONOS and Quickbird (Gruen and Zhang, 2003b, Eisenbeis et al., 2004, Gruen, et al., 2005) with respect to georeferencing (orientation), measurement accuracy (point positioning), Digital Surface Model (DSM) determination and orthoimage generation. These investigations have shown that with the proper methodology and software one can achieve extraordinary results. Both with aerial and satellite images/middleeast/2006/may-june one can get a georeferencing accuracy of better than 1 pixel. In automated DSM generation one can achieve a height accuracy of 1 to 5 pixels, depending one many factors like surface roughness (flat and smooth or mountainous areas), landuse parameters (forest, desert, urban areas), local texture (sand, snow), time and month of image taking, etc.
Accurate DSM/DTM data is not only an important product in its own right but is also necessary for the derivation of good quality orthoimages/middleeast/2006/may-june.
5. Status of automated processing
The automation of photogrammetric processing is obviously an important factor when it comes to efficiency and costs of data processing. The success of automation in image analysis depends on many factors and is a hot topic in research. Progress is slow and the acceptance of results depend on the quality specifications of the user. Also, the image scale plays an important role in automation. Potentially, the smaller the scale the more successful automation will be. Therefore it is a bit difficult to make firm statements which would be valid in all cases. However, in general one can state that
- orientation and georeferencing can be done in parts automatically
- DSM generation can be done automatically, but may need substantial postediting
- orthoimage generation is a fully automatic process
- object extraction and modeling is possible in a semi-automated mode at best
Since object extraction and modeling constitute very important elements in cultural heritage applications we will give some specific comments on that in the following.
Object extraction and modeling
In commercially available digital photogrammetric software, object extraction functionality is restricted to manual or semi-automated measurements together with the capability of attribute data acquisition. The main applications are 3D modeling of city and industrial areas. Commercial systems assist the human operator in measuring 3D objects in combination with registration of attribute data in a semi-automated mode, e.g. Leica Photogrammetry Suite, Z/I Image Station or Virtuozo IGS Digitize. These systems provide libraries containing objects, e.g. buildings or streets, which allow for object modeling according to certain rules concerning object topology. However, there is no functionality available that would consider the specific requirements of cultural heritage modeling.
For the 3D modeling of buildings and other man-made objects we have developed and tested a methodology called CyberCity Modeler (CC-Modeler). This is a semi-automated technique, where the operator measures manually in the stereomodel a weakly structured pointcloud, which describes the key points of an object. The software then turns this pointcloud automatically into a structured 3D model, which is compatible with CAD, visualization and GIS software. Texture can be added to the geometry to generate a hybrid model. A DTM can also be integrated. An example using CyberCity Modeler for 3D modeling of terrain and buildings in an archaeological application was conducted for the pre-hispanic site of Xochicalco, Mexico, where an urban center was reconstructed photogrammetrically from aerial images/middleeast/2006/may-june (Gruen and Wang, 2002), see Fig. 2.
Figure 2: Partially textured 3D model of Xochicalco, derived semi-automatically from a stereopair of aerial images/middleeast/2006/may-june using CyberCity Modeler
6. A remotely controlled model helicopter over Pinchango Alto, Peru
Model helicopters belong to the class of UAVs (Unmanned Autonomous Vehicles). These vehicles are used in a great variety of applications. Lately we have applied such a system to Cultural Heritage modeling.
In the vicinity of Palpa, the prehispanic site of Pinchango Alto is an attractive, yet difficult target for archaeological research. On the one hand, its stone architecture, abundant surface finds, and richly furnished graves dating to the Late Intermediate Period (AD 1000-1400) offer many opportunities to study this still poorly understood pre-Inkaic period. On the other hand, access to and working on the site is rather difficult. The recording of the preserved surface remains therefore requires a highly mobile and flexible documentation system. In a 2004 field campaign we used a model helicopter carrying a CMOS camera to acquire a series of vertical aerial images/middleeast/2006/may-june for photogrammetric recording and 3D modeling of the site and the surrounding terrain. The system used in Pinchango Alto is based on a commercial low cost model helicopter. It features an integrated GPS/INS based stabilizer. While the GPS/INS unit enables semi-automated navigation along a predefined flight path, the stabilizer ensures a stable flight attitude and thus highly reliable image acquisition. The processing and analysis of the acquired images/middleeast/2006/may-june encompassed image pre-processing, semi-automatic triangulation and automated DTM generation. A 3D model of the site was produced and visualized. The results were analyzed concerning in particular the potential of DSM generation from model helicopter images/middleeast/2006/may-june as compared to terrestrial laserscan data. For details of the whole mission (data acquisition and processing) see Eisenbeiss et al., 2005. Figure 3 shows the model helicopter in action over Pinchango Alto and the userinterface of the remote control system.
Figure 3: Left: Model helicopter over Pinchango Alto
Right: User interface showing the flightplan and the control panel
Figure 4: Left: Pinchango Alto (marked with red box) in an aerial image of 1997 (1:7000) Right: Snapshot of a virtual flight over the hybrid model, generated automatically from helicopter images/middleeast/2006/may-june
7. 3D reconstruction of adobe architecture at Tucume, Peru
In recent years we have modeled a number of large Natural and Cultural Heritage sites, e.g. Mount Everest (Gruen and Murai, 2002), Ayers Rock/Australia, Xochicalco/Mexico (Gruen and Wang, 2002), Geoglyphs of Nasca/Peru (Lambers and Gruen, 2003, Lambers and Sauerbier, 2003, Lambers et al., 2004), Pinchango/Peru (see previous section and Eisenbeiss et al., 2005) and Bamiyan/Afghanistan (Gruen et al., 2004a,b, 2005).
Currently we work on Machu Picchu/Peru.
In the region of Túcume in northern Peru, nearby the cities of Chiclayo and Trujillo, the so-called “Pyramids of Túcume” represent a unique example of adobe architecture built during different periods of pre-hispanic cultures. About 3000 years ago, people started to construct various buildings until they were completed during the 13th century A.D. in the period of Sicán, and later also used by the Incas. From the Cerro La Raya, a characteristic hill in the centre of the site, 26 adobe buildings are visible, the largest one, Huaca Larga, with a length of 545m, 110m in width and 21m in height. On top of Huaca Larga, the Incas constructed a stone building. During excavations in the last years, tombs, reliefs and coloured wall drawings were found. Besides the pyramids, the complex contains platforms, citadels, residential areas and cemeteries. The fact that Túcume has been an urban settlement area for the cultures of Lambayeque, Chimú and Inca consecutively makes it one of the most important Cultural Heritage sites of the ancient Peru.
As the adobe structures are heavily affected by wind erosion, the architecture should be modelled as well as possible in an unaffected state. For this reason, aerial imagery from the years 1949 and 1983 were acquired from the Peruvian SAN (Servicio Aerofotográfico Nacional, Lima), which show the adobe complex in two different states. As no control points existed for the 1949 images/middleeast/2006/may-june, two maps and the 1983 imagery had to be used for the orientation. The orientation of the 1983 images/middleeast/2006/may-june was accomplished on an analytical plotter WILD S9, while for the orientation of the 1949 images/middleeast/2006/may-june both, the analytical plotter and a digital photogrammetric workstation Virtuozo 3.1, were used. The photogrammetric products derived from the oriented 1949 images/middleeast/2006/may-june are a manually measured DTM, an automatically generated DSM, an orthomosaic and a photorealistic 3D model. The hybrid model was visualized with the software packages Skyline Terra Builder / Explorer Pro and ERDAS Imagine Virtual GIS (see Figure 5). The 3D model can now serve archaeologists and other scientists as a means for documentation, analysis and presentation of the Cultural Heritage site of Túcume in a state of preservation as of 1949.
Figure 5: View onto the 3D model of the Tucume adobe complex, produced with Skyline Terra Explorer Pro. Overlaid is the texture from the 1949 aerial images/middleeast/2006/may-june. To the left is Huaca Larga, a huge adobe building of 545 m length, with an Inka stone building on top.
We have shown here how highresolution satellite images/middleeast/2006/may-june, aerial and terrestrial images/middleeast/2006/may-june can be used in order to generate hybrid 3D models for archaeological and Cultural Heritage applications with photogrammetric techniques. The digital nature of many of those images/middleeast/2006/may-june and the progress in automatic photogrammetric processing allows for very efficient procedures and for new kinds of results. Additional options for recording and processing are available through the use of aerial and terrestrial laserscanners, panoramic cameras and combined systems. Of particular interest is a UAV (Unmanned Aerial Vehicle) – a model helicopter, which works in an autonomous mode, based on integrated GPS/INS, stabilizer platform and digital cameras, and which can be used to get images/middleeast/2006/may-june from otherwise hardly accessable areas. This system, together with advanced software for automated processing will allow us in the near future to generate at least an initial model of the object fully automatically on-line in the field or immediately after data collection in the campaign office.
All these presented technologies, together with Spatial Information Systems, 3D modeling, visualization and animation software are still in a dynamic state of development, with even better application prospects for the near future.
I would like to thank my cooperators H. Eisenbeiss, M. Sauerbier and Zhang Li for their very valuable contributions to this paper.