The real-world basemap is moving towards 3D

The real-world basemap is moving towards 3D

Stephen Lawler
Stephen Lawler
Chief Technology Officer, Microsoft Bing Maps

A champion of the ‘where’ dimension, Stephen Lawler, Chief Technology Officer, Microsoft Bing Maps, speaks about the IT giant’s mapping business and the future of location technology

Microsoft has been in the mapping business for some time now. Please tell us about your work on Bing Maps.

Microsoft has a long history with mapping. We have been working over two decades now on improving consumer and enterprise experiences through mapping and location technology.

Historically, the conventional use of mapping focused on GIS analysts. Their job was to understand the spatial relations and meanings through visual references in the layers of a mash-up of the real-world and business datasets. There was limited semantic and computational help, and GIS specialists had to wield the system, derive insight, and apply it to business logic.

At Microsoft, we want to empower a broader reach of individuals in the field of mapping, and work with the data from a non-GIS standpoint. We want everyone to benefit from location intelligence; whether you are using it simply to enhance your everyday living through familiar apps on your phone or you are an operations researcher pivoting on location dimensions.

We want the data to become more intelligent and semantically organised along the lines of a ‘where’ dimension so that there is an intersection between people’s personal data, business data and geodata. Today, most geo datasets are in structured databases which do not benefit from the insight of people and data from the Web. We are working on how to relate and intersect crowdsourced data, blogs, and websites with large enterprise datasets through big data graphs to unlock large-scale learnings and reasoning. Simply put, the work involves around how we can apply big data and machine learning to create intelligent location graphs.

As a champion of the ‘where’ dimension,what are your views about the growing consumer interest in location?

A little over a decade ago, the very question that people had was ‘what’ and their search engine tried to address that ‘what’ question. Then seven years ago, a strong user interest started to develop around ‘who’, which led to the rise of Facebook, LinkedIn etc. It was intense personalisation of commerce with Amazon too. But today, everyone seems to be looking at the ‘where’ dimension.

Today, we can answer questions by looking at the tags in photos on Flick or the content in Twitter streams. Take a simple question like ‘where is the best place to see July 4 fireworks in Seattle’. We could try to answer this by looking at the 3D model of digital surface and terrain, but a simple analysis doesn’t provide the real answer. The GPS tags in the photos and tweets help us surmise the crowdsourced knowledge of the actual best locations. In order to do this, entities have to be extracted from local and hyper-local information on the Web and organised in a way to bind structured geospatial data with unstructured data from the Web. Additionally, there must be a strong conversational understanding of the system beyond the current search norm of keyword matching.

Geospatial data is highly structured, but when we talk about something amorphous like the social Web, the data can’t be structured. So how do you do the indexing?

We are looking at relating structured datasets with the unstructured data on the Web. We have to understand unstructured data, and extract entities and relations from it. Then we need to relate that to a similar entity inside the structured data. That relationship enables us to garner the knowledge from the Web and crowdsourced unstructured data associated with that entity, and relate that to enterprise or personal data. We can combine all that to analyse and derive consumer benefits. We envision a graph with many relationships — intent graphs, user graphs or interest graphs —pivoting on a powerful ‘where’ dimension deriving spatial insight and reasoning. Depending upon what data is effectively mashed into a graph, you can have a unique insight or generalised understanding.

How does Microsoft visualise a revenue stream out of such an effort?

It is still developing. But Microsoft sees location technology as an important component of devices and services, and not as an independent business. Of course, there are traditional monetising efforts like advertising, transaction and subscription. Moving forward, this data-enriched experience will not have different monetisation efforts but be part of a broader offering. If things are subscription-oriented today, it will make those notions a lot stronger on the consumer side. If they are developer-oriented, it would end up being consistent with developer models. If it’s a must-have for mobile devices, then it is a supporting component of our devices’ offering. We are not looking at a direct RoI kind of return always. We need to make sure to meet the evolving location needs of our consumers. Further, we look at how location is an important piece of the overall mobile offering. Our outlook is a little different from GIS experts: for us, location is an important horizontal element.

What are your latest innovations to address the evolving consumer and professional needs?

We are always focussed on the needs of our customers, helping them with highvalue tasks that make their lives easier. For example, our Local Scout feature on Windows Phone uses location to help people find fun, interesting, relevant things to do, including personalised recommendations based on what their friends like and what is popular. One good example of our “everyday” innovation is our tips in driving directions.

We also take on some of the biggest challenges in the industry. Bing has always been known for beautiful imagery. Recently, we did a large-scale mapping project, where a specially designed widearea camera was used to capture all of Western Europe and the United States totalling 12.4 million sq km to create a consistent ortho between 15 cm and 30 cm in resolution. This imagery, done in partnership with DigitalGlobe on the B2B side, is a game changer because there hasn’t been a good dataset at this resolution. This is an example where Microsoft is leading the way to create bestin- class imagery for both rural and urban situations. We are looking at it market by market since worldwide coverage is an expensive task.

Another recent innovation is our newly designed UltraCam Osprey, which is the combination of a photogrammetric nadir camera with RGB oblique image collection capability. The unique sensor design makes Osprey the leading camera system for 3D city model applications and was specifically designed for our upcoming release of 3D cities in Windows 8.1.

What are your plans for taking these innovative products to other markets?

With the launch of Windows 8, we have taken mapping to every market that Windows ships in, with 35 localised variants of our Bing Maps app. We have also recently rolled out a new, localised maps experience at in many markets, including Australia, Brazil, Canada, China, France, Germany, India, Italy, Japan, UK, USA, Mexico, Austria, Hong Kong, the Netherlands, Norway, Poland, Russia, Spain, Sweden, Taiwan, Turkey, Belgium and Switzerland.

If you look at monetisation, construction is at the top followed by agriculture, while consumerism is way down. Why aren’t you looking at the higher end of the value chain which can give you better returns?

We have a generalised platform where people can add value on top of our maps platform. At the same time, we have an open platform for developers. Microsoft generally looks at the larger industry to do value addition and provide domain-specific solutions. So, experts or knowledge workers can be the ones to decide how to apply this technology for their businesses. Developers can now use our newly launched Windows map SDK beta for Windows 8 and 8.1 in their projects.

However, even with an ecosystem of developers, we need to listen to businesses and enterprises to understand their specific needs. Microsoft focuses on making sure the fundamentals of our platform are high quality, precise and reliable. We specifically focus on empowering our own devices and experiences with the best geospatial technology and building a great platform for developers.

What is your take on indoor location?

More and more everyday consumers are telling the story of our changing world through the lens of their cameras, which are a valuable source for filling out the 3D map. While professional acquisition may produce a high-quality consistent outcome, the reality remains the same. We need to cast a wider net to capture the entire interesting human-scale spaces — whether they are outdoor enthusiast spaces, event-based urban settings, the countless interior spaces or the extents and reaches that people live and play that simply will not be captured professionally and programmatically. While it is true that the algorithms and models will need to deal with all of this uncertainty, there is definitely a rich source of metadata and pixels to be potentially mined.

The more you zoom in on the world from a satellite picture to aerial to streetside/ outdoor to interior, the higher is the rate of change. Keeping a fresh copy of the digital world will need the help of many. Our Photosynth team is constantly evolving our technology to create 3D scenes from consumer photography.

They talk of mapping where the human being himself becomes a map. The ‘Internet of Things’ may lead to a situation like ubiquitous mapping. Your comments?

The last decade was all about understanding you… search engines looked at where you spent time and interpreted that, browsers understood what you were looking at and user graphs like Facebook looked into you interests.

The next important phase is understanding your physical user path and how that bridges with your digital user path. As you authorise applications to use your GPS or mobile phone signal etc, the sensors start constructing your physical user path. When people use Foursquare to check-in, they give a physical user signal to your digital world. Through proper privacy and authorisation, you can decide how and when those physical paths are shared and intersected with the digital user path. Some very forward thinking innovations and disruptions can come out of that.

Today, there is not enough value returned to the user but that is starting to change. These signals could be more relevant when they have offers and personalised value associated. Soon, even objects will start to broadcast their state and position. The spatiotemporal aspects of these entities and their physical trajectories will make it a “live” 3D map evolving at the world’s course and speed.

What do you see as the next disruptive technology in mapping?

I think the user experience will take a huge leap forward and things will get more immersive with 3D. Companies like Microsoft, Google and others have been increasing imagery coverage through efforts like aerial or streetside. I think the move is towards 3D. The real-world basemap is moving to 3D to provide visually real and symbolic digital experiences, enrich the realworld view with augmented reality and enhance the underlying data and relations.

We will see a lot of augmented reality coming in different forms. You already have things like Google Glasses. The whole visual field of view will become enriched with relevant and personal data. Ultimately, the convergence of technology between the understanding of intent, new natural user interfaces, the location semantic organisation of data, a 3D model and ontology and adaptive 3D contextual rendering will set the stage for our future digital 3D world.