Home Articles Time for neo-photogrammetry?

Time for neo-photogrammetry?

Franz W. Leberl
Franz W. Leberl
Institute for Computer Graphics and Vision
Graz University of Technology, Austria
Email: [email protected]

In January 2008, a keynote was delivered at the Consumer Electronics Show in Las Vegas by Microsoft's Bill Gates. He argued that the world had just entered into its second digital decade. He defined the first digital decade by:

  • The install base of personal computers at well beyond 1 billion;
  • Cellular telephony available to more than 40% of humanity;
  • Broadband connectivity having gone from nothing to 250 million users;
  • Imaging by the "power of software" in lieu of film.

Of course, Bill Gates connected this to the Internet which predated the first digital decade. What strikes one in this statement is that "digital imaging" is one of the four defining elements of our digital age. Well stated, given that a year later, in 2009, the Nobel-price for physics was awarded to the inventors of the digital camera. Anyone dealing with "photogrammetry" will thus pay attention. An alert observer will note today's creative Internet-juices enabling dramatic changes in how we all live, work, read, interact, travel and drive. Some say that we have hardly scratched the surface of what is feasible. We do snap digital images with a smart telephone, upload them via a broadband connection and publish them on a medium like Flickr, and this is happening at a rate of 1 million images per month, with 3 billion images already being available today. 20% of all Internet traffic is taken up by YouTube movies.

How is this related to geodata? The major source of fresh geodata is from imagery and their analysis by photogrammetric and photo-interpretation technology. One might ask how a field so heavily dependent on images will change during the second digital decade. Let us try and find out.


Search in the Internet
'Search', as an Internet-phenomenon, is best understood by the rapid growth of one company, Google. By 2005, search started to become supported by geodata, first in the form of 2D street maps, identical to the data used in car navigation. Not much later, the 3rd dimension was introduced in the form of a topographic model, a DEM and the density of information was increased by the use of aerial or satellite orthophotos. Today's search providers all need to offer a 'location-aware internet'.

Navigation and location based services
Massive digital street map repositories have been assembled for navigation, and were entirely decoupled from the Internet. Today, it is the location-based search function that overwhelms the commercial relevance. Adding the GPS enhances the travel planner to a navigation system so that navigation function is now available on every smart phone equipped with a GPS-receiver. As a result we now observe a vast transition of geodata from everyone's car to everyone's pocket. "Everyone" is us all, but also includes experts in location-based services or engineers using augmented reality to combine camera images from a smart phone with computed 3D geoinformation received via a broadband connection to the Internet.

From the Internet-of-things to ambient intelligence
Objects in museums or parts of a car are valuable, do of course get digitally cataloged and are being oftentimes digitally modeled and closely monitored. However, with the advent of the Internet-of-Things, all objects, even living beings do become of interest to cataloguing, monitoring, tracking.

At the root of the development is RFID in the form of tiny markers as small as 0.5 x 0.5 mm2. These get embedded in all objects as they are being manufactured, as well as in many living beings, and even in electronic markers in credit cards, car keys, phones etc. One can see the RFID as the successor of the bar code.

It seems fairly obvious to combine RFID-marked objects and beings with a wireless network and the Internet. If we assign to each marker an URL, we have the Internet-of- Things. An essential idea of this development is the location of each mark and associated object. In the Internetof- Things, location will need to become known at a human-scale, naturally in 3 dimensions and at accuracies in the ±10 cm range.

Now consider an extension of the Internet-of-Things with hundreds of sensors in each home measuring everything one can think of, from the temperature of each wall, the moisture in each room to the status of the milk supply in the refrigerator and associated computing and display facilities. We find ourselves in a world of ubiquitous sensing, computing, connecting and displaying that will extend to all areas of the human presence. One denotes this as ambient intelligence.

Table 1: Expected image resolution for a 3D World Model.

Support for an Internet-search is thus but an initial factor in developing a model of the world at a human scale so that we can experience the world via its model in some realism.

Table 2: The input images needed to create a 3D World model are expected to be at geometric resolutions shown in Table 1. To fully automate the image analysis, a redundancy factor of 10 is postulated. This will add up to more than 1 Exabyte of input data. We assume a structure to exist per 4 people. For example, France has a population of 60 million and 15 million structures, as reported by the Institut Géographique National (IGN) in Paris.
The Internet-of-Things and Ambient Intelligence may come to mind when reflecting on such ambitious visions. The human scale of such data leads one to a specification for the geometric resolutions, as shown in Table 1. Street canyons need to be sufficiently resolved to be able to read street signs and text on facades. The interior of buildings needs to be resolved to model geometric detail.

We now convert this into an estimate of Terabytes of the image sources. Table 2 summarises that we will deal with data in excess of 1 exabyte to develop such a model once, and additional data will be needed for upkeep.

Considering the need to use street-level imagery and images of the interiors of important buildings or shopping malls, one will quickly expect that such images will come from us, the universe of Internet users. We are already feeding images at a rate of 1 million per day into FLICKR and similar sites. In anticipation of such developments, Microsoft has introduced its Photosynth solution, and has begun to integrate this with its BING/Maps location aware search engine.

While this is an ongoing effort and far from being completely integrated with the 3D world model, it already shows how users can submit a set of overlapping images from street sides or from the interior of buildings. The uploaded photographs then get processed by the website into a triangulated block and a navigation tool helps one to navigate the 3D space using the rectified 2D photographs. Figures 3 and 4 explain.

Traditional photogrammetric mapping has been in the realm of industrial organisations collecting imagery from the air and developing maps and perhaps 3D information of urban spaces from these images.

The 150 sq km surface area of the city of Graz has traditionally been imaged from the air onto about 150 film images. Scanning of these images may have produced a pixel size in the 25 cm range. Lately, a digital aerial coverage has been collected, but now on 3000 images, simply because the lack of variable image costs results in selecting a smaller pixel size at 8 cm, and because one collects images at a much higher overlap, namely at 80% forward and 60% sideward overlaps: less occlusions, more automation.

But what if one augments the coverage by street side imagery? Graz has about 70,000 structures. If each one of these gets photographed onto 40 images, either by the owners or by interested amateurs, one will obtain almost 3 million photographs. If one had those street-side images, do we still need aerial imagery? We may get surprised by emerging fully automated photogrammetric solutions, such as those implemented inside Photosynth, and seeing these executed on GPU-accelerated computers capable of passing through 3 million photographs within a single day.

We envision a geo-data-future dominated by community- contributed images and Internet-resident automated photogrammetry solutions. One may well find an entirely new mapping paradigm, away from industrial solutions and defined by the global community of Internet-users, each one a potential neo-photogrammetrist.