Spatial data infrastructure: So close, yet so far?

Spatial data infrastructure: So close, yet so far?


Prof Arup Dasgupta
Managing Editor


The need for an SDI in information society is well acknowledged. And it is in principle a wonderful concept to create a data sharing platform. What are the main ingredients for a successful SDI? Derek Clarke, Chief Director: NGI/Surveys & Mapping Department of Rural Development & Land Affairs South Africa, listed four key factors. According to him, the first factor is institutional support – We need people in influential positions in all the stakeholder organisations to understand the need for and to fully support the establishment of SDI; political support – a strong political champion who can drive the SDI; accessibility – the relevant geospatial information must be available and accessible timely to the user; and last, but not least, usability – users must be able to use the geospatial information. James Ikkers, Senior Advisor, Natural Resources Canada listed in addition, community involvement in the development of the elements of the SDI; broad communication of the benefits and the implications of the SDI; accessible and implementable standards and people and policies to enable the discovery, access, use and sharing of spatial data. These, he felt, required openness, transparency and cooperation among organisations. He further stated that it should be a self-organising, self-sustaining system which evolves as it progresses depending on new user needs. Thus it has to be user driven and include organisations which are primary data producers. Timeliness and security are important cornerstones though. According to Dato' Prof Sr Dr Abdul Kadir Bin Taib, Director General, Department of Survey and Mapping (JUPEM), Malaysia, integration is the key to the success of any SDI because there are many services and applications that rely on integrated datasets. At the same time, other elements such as data integrity, interoperability and commitment of stakeholders, are equally important.

Dorine Burmanje, Chair Executive Board of Dutch Cadastre, Land Registry and Mapping Agency, also highlighted the organisational cooperation aspect by stressing that the most essential ingredients for a successful SDI are the motivation from different organisations to work together and the raising awareness of benefits that such collaboration brings to all parties. Echoing Derek Clarke, she avers that there must also be one legally accepted central government in place that appoints a governmental body to be responsible for geoinformation and to define SDI regulations. A legal framework for the adoption of (inter)national geo-standards and a clear strategy for their implementation creates strong conditions for a successful SDI. In many European countries, the national mapping and cadastre agency (NMCA) plays a central strategic role in the creation of e-government strategies and plans. The successful INSPIRE directive is an excellent example of dialogue between networks of policy makers and various European and member State geospatial communities. In many of these countries, the availability of accurate and well-maintained land administration data, cadastral data, small scale and large scale topographic datasets provided by the NMCAs, is the reason for the excellent SDI development and implementation processes.

Putting building blocks together into a functioning SDI can be tricky. Ron Lake, Chairman and CEO of Galdos Inc, Canada believes that one of the problems with SDI's thus far is that they have been largely top down activities based on existing data of national governments. This is upside down in two ways. The most valuable data is at urban scales, and the most important thing is to enable multidirectional exchange of data and collaboration across the spectrum of urban development activities including design, construction, management and operations. SDI should be seen more like Enterprise Application Integration (EAI) except that it is pan-enterprise.

Abbas Rajabifard, President GSDI cautions that while SDI is being developed by many countries as an enabling platform to improve access, sharing and integration of spatial data and services, there are many issues and challenges, which need to be overcome in order to have a fully functioning SDI. In particular, there are a range of key factors and ingredients for development of a successful SDI, including raising awareness of spatial data and SDIs and their benefits, cooperation between the various key players and users, the involvement of politicians, knowledge about availability of data and services accessibility to these resources and usability and use of data and services. He is in consonance with Derek Clarke when he says that accessibility of data and services is a very important aspect to the creation of a successful SDI as the measure of success of the SDI will be the widespread use that is made of it and an appreciation by its users that it is providing the promised benefits which were the justification for establishing the SDI. He adds that SDI is a longterm process and therefore collaboration and partnerships is another vital area for the success. The ability to implement an SDI requires a range of activities and processes to be created across all jurisdictional levels and this is where the need for partnership and cooperation among all those involved in the development and use of spatial information and "place", to work together. We need innovation and research – undertaken by academia and the private sector; creation and delivery of SDI tools and systems – undertaken by the private sector and government; and the development of a policy framework for an SDI to operate.

In spite of this enthusiasm, belief and commitment, we see that even in countries where NSDI activity started couple of decades back or more, there still are several issues around sharing of data. According to Derek Clarke, Internet band-width capable of transmitting large geospatial datasets and poor quality of electric power are limitations in developing countries. Institutions continue to operate in silos with little or no cooperation or communication among key stakeholders. Institutional and individual protectiveness/jealousy tends to limit data and information sharing and there is no open knowledge of available geospatial datasets. Lack of clear policies and no means to monitor compliance also acts as a hindrance. Semantic interoperability, particularly in multicultural and multi-disciplinary environment, the lack of geospatial information that meets the requirements of users and lack of knowledge of use of geospatial information all add to failure. These views are echoed by James Ikkers when he says that challenges and limitations include lack of long-term commitment to building a community of cooperation and trust; restrictive sub-set of data types and failure to confirm availability; lack of mandated authority to act and identification of champions; lack of on-going funding to cover costs; vertical accountabilities that do not support horizontal initiatives; policy challenges regarding privacy; security and intellectual property and challenges of maintaining data types and ability to adapt to changing user requirements.

According to Dorine Burmanje, the limitations that continue to exist in spite of strong organisational and institutional arrangements among key players are often due to technical and financial processes. Creating high quality spatial data is a costly process. Sometimes government bodies on different levels produce similar reference data with different specifications according to their own needs, which influences the budget that is available in a country. Also, different software systems and exchange formats of geo data limit the exchange of data among different organisations. In several countries, the business models of data producing agencies are based on cost recovery, which is often not understood by the users.

Ron Lake and Dato Kadir observe that there are ongoing challenges respecting the sharing of data in many jurisdictions ranging from cost sharing issues to issues of legal responsibility and national security. Many of these issues are resolvable if the SDI can provide easy-to-use access control so that sensitive information can be restricted as required by local laws and policies. Cultural issues on data sharing range from fear of misuse to fear of exposing data quality limitations. These need to be dealt with by education, noting that in a networked environment, everyone's data quality tends to improve because of the mutual internal feedbacks. Again, taking the view of SDI as systems, integration can help overcome some of these data centric issues.

According to Abbas Rajabifard, developing a successful SDI initiative depends at least as much upon issues such as political support, clarifying the business objectives which the SDI is expected to achieve, sustaining a culture of sharing, maintaining reliable financial support and enlisting the cooperation of all members of the community, as upon technical issues relating to spatial data access, networking and standards. Therefore, developing a successful SDI within a jurisdictional level must be seen as a socio-technical, rather than a purely technical exercise. This view is echoed by Santiago Borrero, Secretary General, Pan American Institute of Geography and History. Having talked about social aspects, there are also a number of other issues and challenges, which have been reflected in global trends and are impacting on the development of SDI models, including expanding technology market demand, changing business models, e-government and participatory democracy and sustainable development.

Theme-based SDI
If starting small is a good idea then segregating SDIs by application should also be a workable idea. For example, global SDI for weather and climate analysis, national for country based applications and a local SDI for self governance could be one approach. This the stakeholders felt was a bad idea. At the regional and global level, SDIs can become more sectoral but there is still a lot in common, particularly with key policies and fundamental geospatial datasets.

Most often, these datasets are multi-purpose and should therefore be the same to ensure consistency across applications. SDIs are not about applications – but about providing access to multitude of data at multiple scales that can be analysed and visualised using applications. Each 'thematic' type (climate, land use, water source, etc.) rely on other data types to enhance the information depicted. There are some domains that are more global (i.e. climate change) than local (i.e. spatial planning) so there are reasons to build different applications for those domains. But they all need harmonised, standardised and interoperable geographical reference data.

This data should be collected once and distributed in the most effective way to be used many times. Thus these different domains have to work together and exchange data which profits all producers and users. Dato Kadir felt that any segregation should be at usage level and not at the SDI realisation phase.

While global and national level SDIs are moving at a slow pace, we see that local SDIs involving local government or a small enterprise or SDIs of small countries are taking off quickly and effectively. Borrero, Ikkers, Dato Kadir and Clarke aver that at the local level, it is easier and more convenient to communicate because local SDIs are less complex, use simpler governance models, have better defined goals, and greater decision making. Local level SDI effort involves fewer organisations, and these generally share a common goal, which is close to 'home'. They generally involve persons at a lower, more operational level. However, SDIs should be built at least at the national level to ensure sharing of geospatial information across national and local applications. SDIs should be designed to be scalable between all levels by using common standards – thereby ensuring compatibility across global, national and local levels.

At the regional and global level, the SDI can become more sectoral. However, there should still be a common base set of data. Burmanje feels that SDIs should start at a local or national level; otherwise they cannot link with regional and global level. The current attention on global SDIs (i.e. GSDI) and at the regional level (i.e. INSPIRE) helped to build awareness at the local and national levels. Good governance and data techniques must function well at the local level before data can be shared at the regional and global level. Ron Lake takes a similar view when he says that SDIs can be deployed at any level. However, it is clear that collaboration and sharing are of greatest importance at the urban level. The success seen at the local level so far is more about the greater simplicity of the politics than anything to do with technology. Emphasising local levels will also focus on the right technology for all levels – (i.e. transparent, unobtrusive, automated, secure sharing ("I get what I need"), rather than discovery and then access). This view is also reflected by Rajabifard and Dato Kadir.

Many countries believe they can benefit both economically and environmentally from better management of their spatial information by taking a perspective that starts at a local level and proceeds through the State, national and regional levels to a global level. This is happening at the local level more quickly, as local government is a rich source of accurate and detailed spatial information, which is utilised not only at the local level but increasingly at other levels of government. Furthermore, the technical and physical capacity of the smaller jurisdictions can impact on their ability to participate with larger and usually better resources jurisdictions. He also adds that it is important the benefits achieved at the local level are utilised and built on by higher levels of government. One of the major roles of national level governments is in helping set the framework, at a national level, so that the efforts being made at the local level can be broadened out to the national level. If there is no overarching framework to work in, it becomes increasingly difficult to build on the effective tools and developments made at the local level.

As we can see there is a great emphasis on standards. But the ISO 19000 series and OGC are just too complicated for implementation by the average analyst and requires the services of a knowledgeable information technologist who is also well versed in geography and mathematics. This is quite a unique combination. Dato Kadir feels that there is no getting around standards as without standards data becomes unusable for others. They may be complicated but not impossible to implement, says Clarke. Many of these standards have to be generic in nature to cover a wide range of possible events. This then could make them difficult to implement. These standards must be taken down to a specific implementation level, and this often requires specialist knowledge. The community can come together to agree on the manner in which a standard will be implemented. According to Ikkers, standards are the backbone to data sharing and interoperability, and essential to realising an SDI. Tools to implement standards should be built to facilitate buy-in from community. He quotes the example of Canadian Geospatial Data Infrastructure (CGDI) adoption of KML as a standard which has resulted in increased participation of user community. Burmanje agrees that although standards are rather complicated, it is the only way to reach interoperability of the data for exchange and comprehension. In general, it is important that standards are adopted internationally. Software companies have a responsibility to support standards in their GIS software products, so that the end users do not encounter problems. Lake disagrees with the opening contention. He says this is too broad a statement to be meaningful since some ISO and OGC standards are very simple. The core OGC standards such as WMS/FPS, WCS, GML, KML, SWE, CSW-ebRIM are all implementable and together with various standards from OASIS and the W3C can be used to build a practical SDI. Galdos recently completed a successful Phase II SDI project in the City of Riyadh, Kingdom of Saudi Arabia, which makes use of all of these standards but SWE which meets the requirements of near transparent, automated, secure and unobtrusive data sharing. He adds that there are lighter and less capable schemes. One can put all data on an FTP site and then just pass on the site address but that doesn't accomplish much. His contention with OGC is that it is not looking at SDI in terms of systems integration, and is not providing the standards stability that one needs to build on. Many ISO 19000 standards are abstract standards and provide useful support to the OGC implementation standards. Standards need to be used in a pragmatic fashion and will fail if we try to be too dogmatic about it.

We have often heard that the government should only set the SDI framework and base data and leave the rest to industry and other users. Disagreeing with this view Clarke asks, "What use is the base data on its own for the ordinary citizen? Particularly in developing nations there has to be significant government involvement to foster the empowerment of the people and hence the democracy of their country." Ikkers seems to accept the idea when he says that national government is responsible for providing framework and common-use (multiple users) data layers as the basis for industry to enhance with user specific needs. Both public and private sectors data should be accessible through the SDI. Burmanje on the other hand states that about 80% of all the data that government uses has some geo-reference in it. Such data must be reliable and exchangeable, and privacy should be guaranteed where needed. Government is responsible for setting up the framework and basedata and take care of the distribution. Industry can produce the data according to specifications and quality as defined by the government. The government takes care for consistencies and completeness to serve the citizens. If industry would be the only responsible player, then applications like security that are not profitable may suffer. Lake feels that this depends on the country. Governments should start by specifying the standards that an SDI must adhere to, and then enforce and encourage this by funding and legal mechanisms. It is then up to the private sector to respond with the technology. Dato Kadir is outspoken when he says that SDI framework and base data is essential and so are other datasets. We need to know the quality and the integrity of the datasets. If we leave the rest of the data to the industry or other users, we will not have a successful SDI, because it is difficult to share data. Each industry has its own standards, quality, interoperability etc. Therefore, the government need to step in and make clear guidelines, rules and standards for data sharing purposes.

As the above discussion brings out, private industry and public participation are essential for the success of an SDI. Yet industry nurses a distinct feeling of being let down and not finding a role in the SDI other than as a hardware and software supplier. This situation has to be improved so that the potential of the industry can be fully harnessed for the SDI. According to Clarke, private industry has the flexibility for innovation and can provide advanced services that are beyond the resources of governments. The participation of the public is mainly to bring forward the local knowledge of their environment and to express the needs in appropriate ways. Private industry is already involved but at times lacks the resources to implement advanced applications. Regular open fora with the scope for open and honest debate should be encouraged. Burmanje states that the private industry has their dedicated applications serving the public like navigation. Working together and using the knowhow and technology is fruitful for both parties. Good agreements about co-producing and sharing data are needed. These rather equivocal responses are also echoed by Dato Kadir and leave an impression that there is really no clear vision about industry involvement. Lake has a more forthright view though. Government should not build anything. They should provide the legal and standards framework to make it possible. For example, a city could require submission of all drawings for new buildings/roads using a particular IFC-BIM specification. That would be huge. Building things should be left to the private sector.

Data sources on Internet
One section of the geospatial community holds a radical view and feels, "We do not need SDI, just open the data sources on the Internet." According to Clarke, this may be true for the specialist in a niche private sector market but of no significance to the ordinary citizen. Having data sources on the Internet will not ensure that there is interoperability through data standards. There will be no direction given for the collection and sharing of geospatial information. What guarantees will there be that the data available is complete, and if not what will be done about it. There has to be government involvement, particularly in developing countries. Ikkers and Dato Kadir hold similar views. Standards provided by an SDI ensure the compatibility of multiple geographic data sets resulting in enhanced interoperability. Open online data sources do not guarantee compatibility across common data types. SDI provides the infrastructure of leadership and partners that define and organise elements needed to realise an SDI. Delivery of the SDI can be through the Internet. Burmanje is also of the same opinion.

The Internet, whilst it houses an immeasurable amount of data and information, cannot replace a spatial data infrastructure. There is no quality control and data is by no means deemed reliable. An SDI gives structure to data and links the producers and users together. Lake also agrees when he says that there are issues of security and law. Furthermore, without mechanisms to enable secure, automated, transparent and unobtrusive sharing there is no SDI at all. Making all of the data sources/sinks available as standard Web services would, however, be a step in the right direction but nowhere near to meeting the requirements that SDI is expected to fulfil.

The public is more often than not treated as a dumb recipient of 'benefits' from an SDI. No one has ever asked the public what they really want and more importantly what they can contribute. The public can become a 'sensor' in the SDI network. One of the ways of involving the public actively is to channelise volunteer information. Efforts like Open Street Maps (OSM) and Google Map Maker are early efforts in this direction. OSM has gained a degree of legitimacy as it is now being incorporated in 'authoritative' SDIs. There was general agreement that volunteered information provides the local knowledge that government generally does not have. As such it is a good source for updating information that can be used to enhance existing data. There must, however, be strong controls in place to assess the quality of the volunteered information, particularly the completeness and bias of the data. Volunteered information gives local communities the opportunity to take ownership of the information of their community. This will foster greater use of the information. However, volunteered information is not uniformly collected over a region. Rural areas may be sparsely covered as compared to urban areas. Nevertheless, SDIs should develop protocols to incorporate volunteered information. Burmanje is more circumspect when she says that the link, relationship and exchange of voluntary collected data in communities with professional organisations are a growing market. According to her, OSM is increasingly being used by governmental users, because it is free and easily accessible. The problem is that the content is not harmonised and quality is not guaranteed. For that reason, it cannot be used for many official governmental applications. It's known that OSM also uses data from NMCAs and that from private companies to fill their database. It is therefore a handy application for recreational purposes. Lake feels that all information should be encouraged and adds input from anyone must be viewed as an observation and pretty much all observations should be allowed. At the same time, there are custodians of particular kinds of data. Custodians have agreed to a certain community responsibility for certain types of data. In some cases, this is a legal matter like parcel boundaries at issue. In a given community one would generally give more weight to information from a custodian than an observer. For example, the location of a water main coming from the water company might be given more weight than the same information from the gas company. Nonetheless one would expect that the water company would pay attention to such an observation from the gas company, investigate it and if found to be correct update their data and disseminate to the parties involved. Open Street Maps is a kind of custodian for basic street and other urban features.

As the discussion shows information propriety and security are major issues in SDIs. These issues have to be tackled without compromising on development and security needs. Commenting on the possible legal framework, Clarke feels that access to geospatial information should not be restricted as it will detract users from the use of that information. He votes for democracy to be fostered where possible. It may be necessary to restrict certain information in order to protect the individual or community or the existence of certain species. Such data could be presented at a higher, aggregated level which serves the security requirement. Unfortunately, criminals and evil people are always on the prowl and it requires a legal regime to deal with. In a similar vein, Ikkers states that there are a variety of approaches that can be used to regulate SDIs. The objective should be to encourage innovation, implementation and use while balancing the protection of users and providers. There are social, economic, security and liability issues. There are also jurisdictional issues in large countries. For example, Canada has 14 jurisdictions that have legislative authority on most of the groupings. Consequently, for a truly national SDI legal framework, there should be a requirement to ensure harmonisation amongst all jurisdictional legislation, regulations, protocols and policies. Protection of privacy and confidential data is also the concern voiced by Burmanje. Lake agrees that there are requirements for data privacy and security and these should be enforced to the extent possible by the SDI and there is no need to compromise on these objectives.

It is important that the SDI provide access control, identification and authentication services for its community – and that the participants can then specify the access control policies appropriate for their data. Privacy issues are more complex, since data may be released outside the SDI and there are then no mechanisms that the SDI can apply to further constrain it. Standards like SAML/XACML offer a suitable framework for access control and authentication in a wide area environment and these should be supported within the SDI.

This article began quoting from the GSDI 12 closing session and will end with some more insights from the conference. Greg Scott of Geoscience Australia felt that a one size fits all approach will not work and there is a need to look at specific requirements of different groups. This is more or less the view of all the respondents. While it is not advisable to segregate SDIs by size or by application, they must cater to a variety of users who will use common data sets provided by the SDI. Whatever that may be, the need is to start small and start fast according to Brian Chew, Vice President, Singapore Land Authority. Therefore, starting with local and building up to national seems to be the agreed approach. However, the framework and standards have to be unique and well defined. Abbas Rajabifard, President GSDI opined that volunteer information and cloud computing seem to be the trends of the future. This is also borne out by the experts. However, the role of industry is still vaguely defined although their technological prowess is recognised. This is an area that the industry and the government need to sit together and discuss.