Supporting Resources

for Lecture 18: Digitizing and How to Avoid it

Index of Resources:


Sites on Digitizing

  • Sidwell GIS Conversion services
  • ETAK Geocoding Services; map database;
  • Infotech conversion shop in Hydrabad, India (a growing company with a software direction too) Caution, some of their pages lock up some versions of browsers... [not responding today, sorry]
  • ESRI database services
  • ArcData Guide; List of Arc Data providers
  • ESRI "Business Partners"
  • USGS GIS pages
  • Delorme Mapping, selling map productsv; topo maps with 3D views (based on Digital Raster Graphics?)
  • Digital Raster Graphics (generated by USGS, hence public domain- full res example, low res full sheet, ) generating an industry: "More than DRG"; (telling us how they did it...)
  • GIS Datadepot (Free GIS data downloads...) Washington State selection;

  • Data Marketplace

    Data Availability

    Status maps present the process of data creation and conversion: Soils Survey of Spokane County, National Wetlands Inventory, ...

    World coverage:

    World Data Bank I & II (CIA 1970s) public domain (reproduction cost)

    Defense Mapping Agency (now National Imagery and Mapping Agency): Digital Chart of the World (CD-ROM) 1:1,000,000 (cheap); lots of world coverage; used as the prototype for later Vector Product Format materials from NIMA.

    lots of world coverage in Global Change Database; Global Land Information Network


    Data for sale covering the whole US:

    USGS DEM's :

    USGS Digital Line Graphs :

    Orthophoto Quads: rectified photography 1 meter cell size (Map Library has CDs as they arrive)

    TIGER
    replaces DIME (those files are still available for urban areas); TIGER available at $175/state + $15/county (at CSSCR) total US coverage $57,785 major emphasis on roads for census blocks

    Lots of commercial versions of street networks developed from TIGER [eg. Delorme Mapping, selling map products ,ETAK]


    In State of Washington (for example)

    Washington DNR holds many layers under its Public Trust doctrine

    DNR sells Soils, PLSS, and certain other layers for $50 per township (plus sales tax...) under a license.

    The Washington Clearinghouse is a part of the NSDI clearinghouse network.
    The Olympic Natural Resources Center operates a "subnode".


    Local governments across US are torn between cost recovery and open information. (Witness recent case Drummond versus City of Bellevue)

    The Imagery Market

    EOSAT, the US attempt to commercialize a federal program. LANDSAT scenes $600-3000 depending on size, resolution etc. (market based?)

    SPOT Image, a French govt. owned commercial operation, 10m (20 m) for sale 10meter monochrome maybe $700, color (20m) $1400 up.


    Eventually, the custodians of specific records - those with explicit mandates to update them- should become the sources. The central database may be a dinosaur.
    There may be a lot of data out there, but how do you know if it is the right stuff?

    Data Quality perspective....


    Transfer Standards

    To transfer the information from one system to another, there must be some mechanism to encode information so that some other software can understand it. In addition, the different models must often be transformed.

    The original thinking was dominated by the technology: tape drives and arms-length transactions. A network of clients and servers may render all this obsolete...

    The case for standards (at ISO)


    The old way: System specific formats

    Raster formats are not about representation of each cell: that is almost always binary. (one byte, etc.)

    Issue is organization:

    GIS example	Remote sensing	details
    MAP II (GRASS)	Band Sequential	all cells for each layer
    IMGRID	Band Interleaved	all values for each cell

    TIFF (Tagged interchange format), PICT, MPEG, GIF, lots of industry methods

    Attempts to coordinate: The Universal Format to End All Universal Formats

    Raster data compression: run-length encoding, scan line differences, quadtrees...


    Vector formats: more complex

    Different amounts of topological information: CAD typically has none, GIS does.

    Different graphic primitives - beyond point, line, area

    CAD offers curve generation (spline, conics), GIS tends to have only straight lines

    Cartographic "features" (thingies) versus coverages (whole exhaustive surfaces); big issue of reuse of the "same" line


    Model Transformations:

    Convert one data model (measurement framework) to another.

    Raster to vector/ vector to raster translation: lots of options, none very satisfying... Raster linework [representation of object boundaries in a raster representation] (different from raster areas [raster representation of an spatial control framework])

    Creation of topology from spaghetti, translation of object definitions (not easy)


    Current state of transfer formats in current use

    CAD world has de facto standards due to dominance of specific software packages:

    Intergraph Standard Interchange File (SIF) at high end (internal IGDF)

    (Intergraph has recently tried to make their formats open)

    Auto-CAD DXF (an ascii format) at low end. [neither have a lot of structure]

    USGS has Digital Line Graphs (DLG) as a topological format.

    Defense Mapping Agency (now part of NIMA) has a lot of standard products.

    ARC/INFO has an export format (E00), but the internal structures are proprietary. ArcView has 'shapefiles', another simpler format.


    Contrast: file formats versus self-describing files

    early approach: ODYSSEY (ca. 1978) had a "Global" description file which preceeded any actual data file. The globals contains the descriptions of file content and the "information about the information" (data quality, coordinate reference, etc.)

    Had simple primitives: files of records, fields of int, real, char, one var. length

    SDTS (US National Standard: FIPS 173) Spatial Data Transfer Standard generated by National Committee for Digital Cartographic Data Standards for USGS Working Group on Data Organization chaired by T. Nyerges, Federal Register Dec 90 Uses ISO 8211 (a self-describing interchange encoding scheme) as a vehicle. Includes NCDCDS Data Quality Standard (see Data Quality lectures)
    also adopted as an ANSI standard.

    DIGEST the military (NATO) similar to SDTS but just different enough. adopted by French, UK. Includes a feature based standard and the Vector Relational Format (originally called VPF) plus a raster/grid format.

    VPF
    Vector Product Format: designed for DMA Digital Chart of the World. Not an exchange standard (though incorporated as Part C of DIGEST), but a format for direct use of database from CD-ROM. Original application: Digital Chart of the World , now used for many DoD products and applications (eg. VMAP)

    Open GIS: a consortium dedicated to interoperability
    membership is mixed...
    Emphasis on Interoperability

    International Standards Organization (ISO) is trying to generate a worldwide standard: Technical Committee 211 TC211
    European Coordination Group EUROGI, Australian Coordination Group


    Metadata Standards

    Before anyone transfers data, they need information describing that data. Metadata is 'data about data'. [Implements a 'fitness for use' situation where the user makes the judgement.]

    SDTS includes some issues about metadata, the Data Quality Report was meant to be available separately to act as metadata. Not really implemented that way by anyone....

    Federal Geographic Data Committee has published Content Standard for Geospatial Metadata which provides a guideline for metadata content. Various states have developed their own variant or simply adopted the federal guideline. [eg. Montana GIS site]

    NASA has massive data system projects, such as EOSDIS, now ESDIS, with attention to metadata as a core concept.

    Alternative: German ATKIS standard specifies everything about the database, conformance to expectation.


    Digital Libraries: a mechanism for future data sharing?


    There have been some massive investments in research on "Digital Libraries". A fair proportion of this money has gone to spatial data.

  • Alexandria Project (UC Santa Barbara, and cooperators)
  • Sequoia 2000 (UC Berkeley)
  • Illinois (various projects, including a Server designed for NASA EOSDIS)
  • USDAC (Universal (!) Spatial Data Access Consortium; Rutgers, Bellcore, NASA, etc.) an implementation (GeoLens) of the OpenGIS specification

  • Client Server Futures

    Data transfer (as considered in the earlier systems) shipped a client a whole database, usually in the format of the provider. This role may switch so that the user sends out a request for specific 'objects' in a specific format, and the server extracts them and ships them.

  • NSDI Clearinghouse is a beginning, but still developing.
  • GeoWeb a web resource that is a bit dated now, stilll interesting.
  • GeoLens and GeoHarness use OpenGIS to operate a client-server model. Using software developed at Illinois
  • ESRI has placed strong emphasis on SDE (Spatial Database Engine) as their client-server model. Not certain yet how interoperable this will be.

  • From here: Back to Lecture 18 | Class Resources | Lectures | Exercises and Discussions | How to contact us

    Version of 18 November 2002