Meeting Notes

Marine Data Model Workshop #2

June 7-8, ESRI Redlands

 

FRIDAY, June 7 -  9:00-5:00

 

9:00

Welcome and Introductions of Attendees

Core Working Group

- Joe Breman, ESRI, jbreman@esri.com

- Steve Grisé, ESRI, sgrise@esri.com

- Simon Evans, ESRI, sevans@esri.com

- Pat Halpin, Duke, phalpin@duke.edu

- Eric Treml, Duke, eat4@duke.edu

- Dawn Wright, OrSt, dawn@dusk.geo.orst.edu

 

Review Team

- Jan Benson, NOAA Alaska Fisheries Science Center, Jan.Benson@noaa.gov

- Michael Blongewicz, Danish Hydrologic Institute, DENMARK, mjb@dhi.dk

- Peter Etnoyer, Marine Conservation Biology Institute, petnoyer@earthlink.net

- Phil Henderson, GIS Solutions, Inc., phenderson@gis-solutions.com (for Chris Friel)

- Pat Iampietro, Cal State-Monterey Bay, pat_iampietro@csumb.edu (for Rikk Kvitek)

- Chris Jenkins, Institute of Arctic & Alpine Research (INSTAAR),

University of Colorado at Boulder,  chris.jenkins@colorado.edu

- Nazila Merati, NOAA Pacific Marine Env. Lab and NOAA NESDIS, merati@pmel.noaa.gov

- Deidre Sullivan, Marine Advanced Technology Education Center, deirdres@marinetech.org

- Tiffany Vance, NOAA Alaska Fisheries Science Center and OrSt, Tiffany.C.Vance@noaa.gov

Other Interested Participants

- Jim Ciarrocca, ESRI, jciarrocca@esri.com

- Tanya Haddad, Oregon Coastal Management Program, Tanya.Haddad@state.or.us (1st day only)

- Sue Heinz, NASA/JPL Physical Oceanography DAAC, sue@seastar.jpl.nasa.gov (1st day only)

- Mira Park, California Department of Fish and Game Marine Region GIS Lab (for Nancy Wright)

- Jason Willison, ESRI "Team Raster", jwillison@esri.com (1st day only)

 

9:15

Handouts:

- printout from "interested parties" database to this point - 144 people from all over the world

- comments on data model from Halcow Ltd., UK

- notes from initial marine data model workshop in October 4-5, 2001

 

Presentation: ArcGIS Marine Data Model, Project Introduction, Conceptual Framework ­ Dawn Wright

¨     Goals for meeting

¨     Background and overview (from conceptual framework)

¨     Inform review team about ArcGIS and what ArcGIS plus a data model really means

 

Handouts and PPT file from this presentation downloadable from dusk.geo.orst.edu/djl/arcgis/docs/

 

Discussion:

Tiffany Vance - Atmospheric models? Should be helpful when trying to tackle 3- and 4-D challenges for marine data model

 

Jan Benson ­ cartographic perspective, difficult to model ­ lines w/depth, volumes in ocean ­ how does this fit mathematically ­  trying to apply a model that will never fit ­ point, line polygon will never fit the ocean

 

Steve Grisé- Marine DATA model vs. marine object model  - hard to define behavior for ocean, real world is sooo much more complex ­ in GIS we just try to capture some representation in a computer of whatıs going on ­ behavior is simulated ­ in CS object-orientation is for bank transactions (easy stuff) but behavior weıre trying to model is so much more complex ­ people have very different perspectives on what the behavior is ­ hard to get agreement on behavior ­ so behavior just means rules for good quality data to make sure relationships are there (e.g., if I add one feature it needs to be contained in something else)

 

Simon Evans - IHO S57 e.g., lighthouse point has to be on land, marine mammal point has to be in ocean

 

 

9:50

Presentation: Marine GIS Data Model Example Data Problems for Marine Analysis - Pat Halpin, Eric Treml

 

PPT file from this presentation downloadable from dusk.geo.orst.edu/djl/arcgis/ docs/

 

Notes/Discussion:

What makes marine applications unique

-       we have tons of instruments

-       we have an incredibly dynamic environment

 

central issue -  how do we do a better job of analyzing 4D with a 2D tool

we have more people involved in process now ­ we may want to throw everything open and re-examine model completely

georelational to geodatabase transition

using old ESRI feature classes ­ basic building blocks of GIS that are problematic to marine applications

Gulf of Maine ­ lots of different data, similar to Monterey

-       sounding and elevation data, different kinds

-       different resolution rasters and grids for bathy

-       hydrographic circulation models ­ TIN hydrographic

-       SST fronts, SeaWifs, AVHRR

-       Biological ­ fisheries, fish stocks

 

Start with POINTS and variations on them, then lines, areas, rasters

Fixed xyz ­ MMS data, bathymetry where depth is an attribute

Instantaneous point, an observation in time ­ harbor porpoise ­ tons of observations tied to a single moment in time

Time duration point ­ buoy, xyz is stationary but it is recorded over several times

Time series point ­ telemetry data for harbor porpoises, 5 animals tracked as instantaneous points

 

LINES

 traditional lines

Route ­ tracklines

Time duration vector ­ trawl data, prey for harbor porpoises, data are collected with single attribute, how many fish did you collect ­ volume of net through water ­ catch per unit effort (volume of water within which the fish were caught) ­ how to define what the trawl actually is ­ trawl is going through volume ­ but attribute may be just fish count

Cruise survey, effort data, time duration effort ­ how fast was vessel moving, where did it go (track), what were conditions (sea state), all impt. For interpreting the data collected

 

AREAS

Basic area polygons ­ a marine sanctuary has a boundary

Region ­ multiple polygons ­ patches or MPAs within a sanctuary

Time duration areas ­ area features that persists only for a certain area of time ­ this area off-limits for fishing from Sept to Oct., so that t-value is turned on or off

 

RASTERS

Bathymetry raster grid

SSTs, calculating gradients or fronts, then calculating distances from those fronts ­ mining calculations on the fly

 

Feedback on errors of omission to commission

May have to rethink the jargon to take us from old Arc to new geodatabase

 

Existing data model defines an EXTREMELY generic, conceptually designed framework

 

What TYPES of data are missing? Should be included?

What relationships should be included?

What should be optional vs. required

 

Should temporal components be treated as optional attributes within a geodatabase or as required variables of new ³feature classes²?

 

Should there a more generic metadata model that atmospheric, hydrological, people come under

 

***Should there a single ³marine data model² or more field specific models: benthic mapping, navigation, pelagic-animal tracking, oceanographic, bottom mappingŠ?

There is no terrestrial data model, why should be there a broad marine data model?

 

 

Eric - need for dynamic features is key ­ build on that nugget to make submodels, navigation, seafloor, biodiv might be extensions to core model

 

Chris ­ we are really talking about data, and how data visualizations are done

 

Pat - Building in z-values and t-values into structure so that map display can switch from xyz to xyt and other switches of variable ­ go from map to profile to time series

 

Chris ­ missing is reliability structure, uncertainty has to be expressed ­ we need provisions for this ­ weight of confidence of a certain line can be added to attribute table ­ but need to go further than just adding another attribute

Can use metadata to document and provide info.

Data set may not work unless you provide the uncertainty that should be required?

 

Michael B. ­ spent a lot of time on Hydro data model ­ used to think that data model (DM) should be as inclusive as possible ­ but come around to agree that DM should be a CORE DM ­ applications are immense ­ cannot develop a DM that can accommodate all ­ people can add their own submodels

Simon agrees ­ lightweight for performance issues too ­ guidelines good

 

Steve ­ technology and organizational ­ FGDC marine boundary identifiers could be identified or marine buoys, data infrastructure or framework to hang things off of ­ can look at different examples and everybody is different ­ USGS

 

Joe and Eric ­ this model may be more of dynamic data model rather than a marine data model ­ keep it focused at addressing those questions ­ what a specific tools that we want to develop

Tech support perspective ­ enhancement requests ­ the more specific and targeted they are, the more likely they are to be carried out

DM serves as framework to port data into, see where weıre limited and take things to next level

 

Simon ­ that 80% effort of getting data into GIS applies to the geodatabase data structure too

 

Steve - Good size knowledge gap between how people are using the software and what can we use it for

ArcGIS 9 is coming - geodatabase will grow in power quickly ­ faster we can learn it and ramp up with it the better - coverages and shapefiles will not be supported forever

 

10:45 to 11:00 ­ BREAK

 

11:00

Handout - old marine data types figure from first workshop - attendees added missing data types to the following:

 

Chris - animated maps are commonplace data outputs from models so there should be some way to georegister them so that other data can be overlain - example of animated maps ("benthic movies") linked to the ArcGIS marine data model site at http://dusk.geo.orst.edu/djl/arcgis/links.html

 

Tiffany - area data types expanded to volumes

Additional time duration vector example: algal bloom trawl

Additional data types: model output (e.g., atmospheric model or circulation model), water properties (e.g., the cold pool in the Bering Sea)

 

Michael - additional data type: transect lines in profile -

polyline zm

ID

x

y

z

m

No time association

z represents depth

m represents measure across a transect

then expand to cube (Nazi suggested this also) from x,y,z to xn,yn,zn

x,y, (z or t), m

x1,y1,z1

xn,yn,zn

n cols

n rows

n layers

nodata

 

Additional data type: time series points

ID

x, y

z

s

t1 ... t2

m1 ... m2

 

Nazi - additional instantaneous point examples: fish densities (bongoes, ichthyo zooplankton at a point), tide gauge, bird or mammal sighting that then might be used to calculate overal abundances in trawls

Additional time duration point examples: mooring, current meter

Addtional time duration vector examples: ADCP tracks with z, ARGO drifter

Addtion to interpolated surfaces --> images (bil)?, additional examples: climatology, analysis products

 

Emerging Developments ­ Joe Breman

¨     3D visualization and analysis with Israeli and Monterey data

¨     animation capabilities in ArcGIS with Monterey data

 

Handout - "3D" (2.5-D) Functionality in ArcGIS 8.2

·       "3D" effects toolbar: interactive transparency, front/back face culling, drawing priority (z conflict resolution)

·       "3D" navigation tools: explicit observer, target, and fly tools for "3D" GIS data perspective and navigation

·       "3D" visualization

·       ActiveX viewer control for custom applications

·       Animation export functionality (as AVI)

·       ArcCatalog - new 3D Preview

·       ArcMap - new 3D Analyst toolbar containing functionality below

·       ArcToolbox - added TIN functionality

·       Conversion: tin to polygons, raster to TIN, raster to features,TIN to raster

·       Creation of elevation profile graphs from line features/elements

·       Cut/Fill command

·       Direct access to OpenGL API on draw events for custom "3D" viewing

·       Feature interpolation from surfaces

·       Gestures: rotate/spin either direction with adjustable speed

·       Interactive contour, line of slight, and steepest path tools

·       Interactive light source position

·       Multiple scene viewers

·       On-the-fly raster and TIN projection

·       Orthographic view support

·       Picture fill symbols for textures

·       Quality control over high resolution imagery, including while draped on elevation models

·       Raster interpolation (natural neighbor, IDW, spline, kriging)

·       Raster reclass and vectorization functions

·       Scene animation

·       Support for "3D" CAD drawings

·       Support for "3D" propertis of layers in ArcScene - base heights, extrusion of features based on attribute or constant, etc.

·       Support for image catalogs

·       Support of creation of VRML world files with geoVRML extension

·       Surface analysis (shared with Spatial Analyst): Contour, Aspect, Slope, Hillshade, Viewshed, Cut/Fill)

·       TIN creation commands

·       TIN element enumerators, selected set operators for TIN editing

·       TIN node deletion

·       View setting dialog: interactive perspective control

·      Volume and area calculations for a TIN if you specify the height of a plane for the top or bottom boundary. This is accessed from the Area and Volume Statistics dialg of 3D Analyst, and from ISurface::Get Volume and ISurface::GetSurfaceArea

 

Tip: Build a raster of the difference between seafloor and sea level and flip the raster by multiplying by -1, then convert to TIN and use a plane of ) to find the volume

 

 

Discussion of  3D and 4D issues and continued lack of true 3D and 4D in ESRI products ­ Nazi, Jan, Tiffany

PMEL is like JPL ­ sophisticated users and programmers who will write the C++, Java code theyıll themselves to build a tool rather than use a commercial product

Vis5D

Fear of leaving ESRI fold to go into 3D and 4D ­ issues of sticking with just one tool

 

Jim - ESRI has received requests for 3D but they have not been well justified - "we need 3D so we can fly through our data" - "we need 3D for eye candy"

Joe's recommendation to apply continued pressure on ESRI development teams - not enough just to say that "we need 3D or 4D" - must add specific, targeted functionality to list above - give development team case study examples that they can test in order to show them why current functionality is not sufficient - we need to defend WHY 3D and 4D are needed beyond just fly-through

 

In the meantime we need to come up to speed with new functionality in ArcGIS as above - Joe's demo quite stunning when considering how limited ArcInfo was with this at version 7

 

Dawn's poll of group - only 4 or 5 attendees have really used geodatabase and understand it

 

Email participants to have their names added to the enhancement request (support@esri.com), name, user #,  and each idea as an individual request

 

Demonstration of Marine Data Model in Action ­ Joe Breman

¨     Use ArcMap and real data to explain what we are trying to accomplish/support with the design

¨     Build an initial model in UML and generate the schema in ArcCatalog to show group how this is done with a model they understand (and assuming that they will NOT be having to deal with UML)

¨     Load small data set into the model

¨     Discuss implications for model

 

 

Can we carry vertical datum with a data set? No, not with feature data set but figure out difference between those and use values in fields with multiplier and offset ­ yes ESRI is working on it (great for tsunami modelers and marine mammal folks)

 

 

Joeıs demos

ArcCatalog ­ engine

Geodatabase (GD) ­ composed of feature data set as drawer

 

DM is file organization mechanism but it has limitations ­ ArcScene for instance just uses DM as place to store data to bring it in

Can put rasters in GD only with SDE

 

Can do everything right now w/o MDM but distill concepts from that ­ bring a set of concepts and ideas up to higher level so that we can communicate with others and have patterns, standards for how we do things ­

 

How toıs of getting data into DM - Joe working on document

 

12:05 - Lunch

 

1:00 ­ Jack Dangermond himself brings in 3D Analyst development team to discuss 3-D and 4-D needs with the group!!

 

isosurface draped with results of analysis ­ just a surface, not a volumetric model ­ might consider converters to let people use other rendering packages

Nazi - still a lot of work for people used to using ArcGIS and now they want to look at data in true 3D ­ they donıt want to have to write a program in VBA or learn a whole other rendering program to look at their data

 

other packages that read and write data in 3D

Matlab

Fledermaus

Vis5D

AVS

 

VRML export is critical

Volume output from AVS

 

Magic bullet is to allow everything to be done on the desktop, which may not reachable

 

Voxels, different data type

Water column that has lots of data

ESRI team - Business picture ­ niche market ­ resources to develop this - need justification that marine applications is a vertical market (not just horizontal in relating to or drawing from hydro, transporation, biodiversity, petroleum, etc.)

 

Competing priorities, Jack is interested in many things

 

Jan ­ as a user what can I do to help you adjust to my needs ­ what needs to be pushed, pursued

ESRI wants to improve relations with vendors

 

Dawn ­ create interface between 2 packages that is transparent to user (e.g., between ArcGIS and AVS)

 

Instead of all this import/export ­ shapefile format being open reduces that to some degree

If other vendors could be write to geodatabase or ArcObjects ­ so that translation between 2 packages ­ their software would benefit from considering ESRI spatial indexing and mgmt tools

 

Michael ­ just being able to get at proprietary GB format would be useful

 

Sam's demo ­ layer transparency animation in a nutshell

Ozone TOMs data in ArcScene

Automating a technique that can be done  manually

Notion of ³group layers² ­ animation turns only one layer on at a time

Earth probe NASA satellite ­ thickness of the ozone layer ­ thinner ozone layer in red, thicker is blue

This was gridded from satellite, not an image

Nazi ­ interesting ­ all their gridded data are in NetCDF, HDF for satellite data, JPL has converter for HDF?

 

Using ArcGIS 3D analyst pdf ­ file to distribute to group

 

Send public domain extensions to Steve Cope or Jason Willison for incorporation in next revision so that ESRI developers donıt have to bear full burden of development (or maybe they will if our code is so kludgy! :- )

 

California ozone concentration animation ­ made a simple AVI from ArcScene in 8.2

SST changes over time, sea surface topography or SST for El Nino, bottom erosion

 

Can we reproject to spherical coordinate system? Right now have to run a script to get them to display in ³3D² on a globe

 

ArcGlobe Demo

 

Lists of ideas, case studies, enhancement requests and associated data ­ conversion tools handed in too

Joeıs facilitation with 3D Analyst team

Working with 3rd party vendors ­ Steve Grise will work with Steve Cope

 

DM will not solve all worldıs problems ­ itıs a piece of the thing ­ ESRI takes concepts, ideas, and things people are trying to do and figure out a reasonable database to help people do that ­ confusing with issues of scale, how much data you want to use

Compact, flexible data structure or all sorts of extensions or multiple data models, is that the right way to solve the problem

 

**ESRI experience - manmade applications have lots of feature classes and behavior is simple ­ with natural world there are only a few feature classes and behavior is v. complex

The more complex things we are representing the data structures are more compact and simple

 

Michael ­ hydrological and marine process modeling perspective ­ itıs important to have a DM that can be used by a variety of apps ­ hydro uses DM as a repository ­ they know how to do animation they just need source of data ­ ESRI simple structure can be extended to accommodate their own modeling software ­ ArcObjects and COM lets them incorporate their animations inside of ArcMap.

 

Stress again on simple DM

 

Some people are end users, some are developers, and some consulting and academic (ideas from an R&D perspective, system integration, basic science)

 

Common structure, technique, approach = helps people to get their next rev.

 

2:15

Emerging Developments ­ Steve Grise

general science data models

4 somewhat related, recent efforts:

Geology (boreholes)

Petroleum DM (wells and boreholes that are spatially enabled)

Groundwater DM

Marine DM

 

Steve ­ overview of discussion of ESRIıs science data models are going

 

Then will go through the boxes in case we want to break into groups and look at them

1)    discussion of Hydro Data Model from ArcOnline in relation to ours

 

analysis diagrams ­ let ESRI deal with UML and us deal with problems

brought up diagram in Visio (like forcing people to know C++ before being able to use ArcScene)

-       they are stepping stone for DB designer, between ideas and physical DB

 

Network part of Hydro DM ­ simple idea, DEM or landscape or drainage view of the world from raster side and then a cartographic side (hydrography) ­ have been 2 separate things in the past but can we come up with a common approach ­ regardless of where data comes from we can build them into a hydro network

Work put into TS box of DM has been limited and objects there are fixed

 

Massage, format, reconstruct, getting data into a form they can use it in ­ give them a template to give them a starting point then work with data providers to help them output something that people want ­ they donıt know how people are actually USING their data and users havenıt really talk to providers to tell them exactly what it is that they really need

 

Names specific things and implement them as physical tables in database ­ didnıt add too many attributes

Generating DB from repository - can load data into it at that point ­ or apply schema onto some existing data sets and keep all attributes they already had

 

Go now to science-oriented data model

Geology data model is different from Hydro ­ target for this group is national geologic mapping agencies for DB structure to hold national inventory of geologic info rather than just project level of hydro DM

 

2 tables for concepts and occurrences ­ those are the 2 tables in this DB

concept ­ is an idea, a formation, a rock, name of a map, could have something to do with a species, a scientific concept

occurrences instances of these things in real world ­ make linkages betw concepts and occurrences

occ can be represented with 0 or 1 or point, line, poly ­ represents 1 thing but 1 polygon could actually have multiple occurrences

a concept can strat age, lithology, drillholecollar, drillholesegment

 

this is a different style, structure ­ in hydro, every concept has a set of physical tables to go along but here there is a small DB structure than can contain any info. ­ over time we get more and more info. and donıt have to keep changing DB structure but this is also hard to navigate, hard to browse through and recognize that this polygon might be part of a particular rock formation

 

practical templates for researchers or how to create national and global data sets for servers??

 

Chris ­ big blob of data (like core log) pass algorithm over it that meets criteria and only then do you create relational table ­ instrumental data that is well controlled comes out of vendors standards, bank info, goes into DB easily ­ but all bio and geol descriptions that are a dogıs breakfast, v complex ­ another aspect is how to deal with non-numeric data ­ well organized instrumental data ­ geological data is 85% or so of the non-numeric observations ­ can be a lot of work to pull that off

 

Data mining approach might work for us in the marine world ­ mines through glob and pulls out template and only then go to relational table

 

Petroleum side of it ­ GD rep. called PPDM ­ not spatially enabled just textural descriptions ­ focusing on wells, well tests, may publish some ideas in 3-4 months ­ not a very advanced effort ­ 1500 pg conceptual framework document!!!

 

Groundwater DM ­ ideas popping around but not much happening yet ­ subsurface, natural water

 

Pat ­ nested DM ­ generic at top but nested underneath could be benthic mapping vs. oceanographic data, etc. but all could be compatible ­ can use one without the other ­ sub model inherits from core, generic model

 

ESRI publishes core model and then lets users specialize ­ DHI is working on hydro data model extension

 

3:00-3:30 - Break

 

Steveıs overview of analysis diagram and DM

 

Conceptual Framework -à  High level

Cruises ­ Feature is point, line poly ­ object is just a table

Set of attributes associated (relationship classes are pre-related)

Vehicle type is attribute is relationship between those points and vehicle associate with it

Measurement and TS

Canıt distinguish instantaneous point and time series vector yet ­ can only do this, figure it out, with real data

 

Observations (or could have information like from a collection at a museum)

An individual is a group of one, etc.

 

Cartographic ­ has some features like points and areas ­ use this as a geographic context for a lot of data (shoreline, marine boundary), FGDC

 

Types of Conclusions

Raw data, interpretation, may predict and here is how people may represent them as time duration areas, interpolated surfaces, or volumes changing over time

¨     footnotes - notes to integrate in tables or shape, z value or t-values so that we can plug tools in there later

 

Measurements ­ from data types diagram

 

Tutorial on creating initial schema

 

UML ­ convert to repository ­ then go to ArcCatalog

Repository DB

Create new personal GB ­ creates MS Access file

Customize ­ commands ­ Case tools ­ schema wizard

Select mdb file ­ start schema wizard

Chose repository DB

Open repository ­ repo.mdb (in this case)

(Figures out whatıs inside of dB)

Use default values

Here are tables itıs going to create ­ can look at properties

Will have to change spatial reference to tailor model to spatial extent of your data ­ change spatial ref properties at feature data set level (as opposed to feature class)

Geodatabase is like a project database ­ same repository, same DM

 

Can take GB created AFTER these steps and tweak it to fit your data too

 

Now have tables in an MS Access dB

In Marine Features think of Shapefiles in one DB

Group things in feature data sets and also has the relationships in it ­ look at relationship class properties

 

Simple Data loader

 

Now where data comes from ­ data exploration in ArcMap

Look at data source tab in ArcMap

Can look at relationships in Identify Results

 

Case studies ­ people go off and try some ideas and present back to group

 

Going from real field data to DM is tricky ­ users can give feedback to make crosswalks easier ­

 

Load data into model

Pts in notepad used to populate TS events, xyz measurement value

1 ­ open up file in MS access ­ get external data, import, import text wizard ­ rename to new table, TS2

if you have data open in one application, donıt try to manipulate it in another

donıt put spaces in file names

TS2 in Access ­ seems to have best set of tools for manipulating, massaging

 

2 ­ ArcCatalog, Times series ­ Load data, open GB ­ use simple data loader to match your data fields with DM fields


SATURDAY, June 8 - 9:00-4:30

 

9:15-10:00

Feedback on marine data types (see previous pages 6-7)

 

Chris - missing are images for animated maps, hot link to it or put some inheritance to it ­ when you get to a point that you know you can animate the data then you can activate the hotlink ­ an extra table or column to activate it

 

-       variation on time duration point ­ stick diagram, click on point to run another process to create that chart, click on instrument and you get a diagram showing what temp recorder did over time ­ a chart tool issue ­ an extension from the measurement of that data to display of data from the instrument ­ work that in to the display functionality of that feature ­ click on CTD location and show me where the mixed layer depth is

 

time duration vectors ­ plankton data but bongos in and it goes from bottom up and takes 25 min to cover half a mile ­ should also be calculated as an instantaneous point ­ just as in old model with a coverage that may have point, line, poly topology  in one coverage, we should be able to ³mix and match² these data types in same GB

 

Culture clash discussion

Mike ­ does the DM provide the data in such a way that you can do these diff functionalities,

 

If people are used to looking at things a certain way, hard to get them to change (non GIS users) ­ pessimistic that physical oceanographers will ever take up GIS ­ climate modelers are coming around  - climate people need volumes

 

An additional data type ­ to get climate modeling and global modeling community

 

Skeptics need to see how a GIS will allow them to do something that they cannot already do ­ phys oceanographers think in time domain rather than space domain

Bunch of CTDS on a line and make a transect plot ­ requires volume tools to make a depth transect

 

EPIC tools perhaps translated to Arc8 ­ EPIC is in C++ and Fortran

 

Time series split for phys oceanographer

 

Another problem is multi-platform - Data server, browsing multiplatform - desktop editing (Windows only)

 

10:00

Hands on - Tutorial on loading data ­ hands on how to get data into DM, dealing with obstacles on the fly, Gulf of Maine data from Pat, dbSEABED data from Chris

 

1 ­ ArcMap of all Gulf of Maine data showed in Patıs PPT yesterday, cov, shapefiles, grids, point data came in txt files from people who donıt use GIS, so rep. of each of Noahıs Ark of data types from Patıs data figure (which couches things in old terminology to help us make transition to new GB) , also an ASCII file of Chrisı data from dbSEABED

 

data are in geographic DD in NAD83

- shoreline (arc)

- baseline points, MMS survey points for calculating MLLW shoreline ­ fixed pt

- bathym points ­ pts from bathy survey (from NOS raw survey data) in a harbor area ­ simple xy with attribute of depth (we can make the depth a z attribute when plunked into DM) ­ fixed pt

- buoy pt ­ sea hts, wind speeds, traditional recording buoys ­ time duration pt ­ data had over 2400 fields of data ­ raw data

 

-       get repository DB (RTBLin file name suffix) ­ build DS from that

-       made Maine.mdb ­ got schema wizard

-       run schema wizard, select Db, find repository

 

 

- TS points ­ one animal tracked through time ­ can be turned into a vector if you assume they swam in somewhat of a straightline

(where daylight matters ­ stick that in somewhere before Cruise boundary

time area data ­ ozone demo (rasters), fishing area closure (poly)

time duration area series?

 

Chris - animated maps demo while Steve works on getting some data imported:

11 km gridded, 1 hour ts wave heights, wave periods, from Bass Strait (Australia)

map can georegistered ­ can see storms come in and retreat back to s. ocean

wanted to find out benthic creature weather was like ­ calc bottom velocity from bathy convolved with wave info ­ waves donıt influence center of strait

3rd movie ­ engineer, cable layers, mine burial, habitat, people index of movement of bottom ­ surprising result ­ lee and sheltered areas most disturbed

 

can see potential for putting this into GIS and have other data overlain on top, other TS like whale sightings ­ this is a feature type that is not on matrix ­ how to maintain the animation and load that in ­ this is really a stack of rasters that can be manipulated ­ play them each in step or run an equation on them ­ but series of input files, processing them ­ x,y,fixedz,t  - an extension of time duration raster ­ storage of features is impt and how to assoc them to TS is more impt than if ArcMap can display it ­ what we really want is a pt connected to a diff row and column of values and that one pt changes over time or find all pts at a given time and show their values ­ developing a struct where pt feature represents the cells and assoc time and values with it ­ a lot of data ­ hundreds of files (gifs) for Chrisı TS of a weekıs worth of hourly data ­ right now everyone stores these in flat file instead of DB ­ we can store it in DB in the DM and we would want to superimpose other features on it

 

reliability and uncertainty issue ­ visualize uncertainty ­ as just another attribute it will cripple size of dB ­ glyphs might be a good way to define uncertainty ­ build a glyph or a multicache in 3D Analyst ­ transparent glyph can be shown ­ to build in 3D analyst is not trivial (Avenue and VB) ­ how to deal with propagation of errors

 

Pat - errors of omission and commission common in env. Apps of GIS ­ receiver operator curves that engineers use to tune gain on a radio ­ tradeoff ­ can overclassify RS image ­ balance error to get optimum amt of error ­ trying to get intersection any correcting error in one classification can cause more errors in another

 

 

Running into quite a few glitches with data input ­ good to iron these out now

 

Can create custom repository to keep settings of spatial reference

Cancel wizard and start again if you run into problems with spatial reference

Loaded into schema ok now

One table called TimeSeries but weıre going to  - try to stuff all data into that one table or make multiple sets of TS tables or features classes and come back to some structure afterwards ­ donıt worry about jamming everything into that one table

 

Baseline data set first now ­ fit into marine features, marinePoint

Load data à baseline.shp ­ for each target field select the source field that should be loaded into it

 

Add fields later or have all fields in there and delete fields you donıt need ­ xyzmt,uniqueID as a feature class then based on that ID have another table with ID and extra attributes ­ data can change but feature stays put ­ empowers data ­ keeps things simplistic

Doubles in database canıt load into int ­ data types didnıt match so data didnıt load SO

Create a new personal GB and call it load then format it ­ so import shapefile again, and change MMSBASE data type from double to

Load.mdb ­

Massage databases before you load it in

2 Dbs trying to load ­ can we pop up a window ­ which do you want to resolve? ­ can we implement this fix

MarinePoint feature class is going to be changed ­ no zıs, no mıs, double (maximum number of sig digits, so better than float and donıt have to worry about file storage in todayıs modern computers) long I can be 16, short I value can be up to 32000

 

Ok ­ he finally got bathym_pts in !

 

Z values and where itıs going to be stored in the DM

 

Look at more data

Discuss focus of this effort

Then breakout into groups ­ kinds of data that can fit into thematic groups of model

 

Lunch and informal discussions 12:00-1:30

 

Cruise_obs point ­ instantaneous pt.

ArcObjects VBA script lets you generate a line out of latlong numbers that are in the same field

 

Metadata template that goes along with features and classes so that things like abstract, and whose terms we are using will just be in metadata

 

ESRI are not the right people to drive and initiate a standards effort

 

2:00

Follow-up tasks

 

¨     Publishing model on ArcOnline and/or our data model web site

 

¨     Setting up broader review period

 

Plans for final publication as a book?

 

Planning for ESRI User Conference - Marine SIG Meeting, Tuesday, July 9th, 5:30-7:00 p.m.

Time duration and content for Joe's panel session, focus, scope

Brief demo of model at this session?

 

Last yearıs SIG had 40 people and list is 1350 people ­ donıt know how many people will come to SIG

- Joe pep talk to marine GIS community and demo of some current capabilities

- Presentations by Dawn to introduce model conceptually and Pat to introduce data types and model (blow up Patıs figures and let people write on it)

- Panel of core group and others in audience questions/answers feedback

 

-       why DM is important, why it matters, why we need a DM, hereıs another way of looking at my data and here also is an educational process for ESRI to expose them to new data types

-       bear DM in mind when collecting data and then when they get home they can do the more advanced analysis with ArcInfo or collaborate with colleagues who do have AI for the more advanced analysis

-       amount of effort and time ESRI is putting into GB ­ GB is not going away and is going to be continually modified ­ industry groups such as Utilities has been in GB for longer time ­ we need to catch up

-       need to be consistent and strong on discussing the DM, NOT getting too much into why we donıt have 3-D and 4-D yet, visualization is NOT the biggest priority of this model

-       here are limitations that we are dealing with ­ and these are areas that we donıt want to discuss are viz, migration of covs and shapefiles, AML into VB, etc.

-  we are still an emerging industry at ESRI

-  print out data types with glossary Letıs put together a glossary NOW so that we donıt get hung out to dry at the SIG where some people arenıt going to recognize the terms

**semantics issue of objects and features, attributes ­ feel free to write on poster

-       what special sessions might be helpful to learn about migrating to ArcGIS

-       be up front about needing AI in order to get full benefit of any of the ESRI custom models

 

***not excluding ArcView 8 users ­ how to deal with this because you need ArcInfo in order to fully use the DM ­

AV or ArcReader can view the databases produced by DM ­ viewing feature classes and some of the analysis

** 2 kinds of users ­ power users of DM and readers of DM

-       call regional office to inquire about getting AI

DVDs or CDs example ­ some people can make cool movies and burn them on DVD and some can only read DVDs, some can only read CDs and need to upgrade their drive to CD/DVD ­ such is with the DM

-       Joeıs demo ­ if you have a set of pts with a z column that has values (an attribute),

-       3D analyst ­ convert features to 3D ­ run that, base it on something (source of heights)

-       need 3D Analyst, raster layer is not part of GB that is in DM ­ need ArcSDE for that

 

-       rasters in 8.1 release ­ no raster catalog or images in ArcSDE ­  this is now in 8.2 -image can be stored in table and additional columns to add more info to that image ­ raster catalog can be another table in the system in 8.2 and can be used in a GB as part of a DM

-       **canıt bring rasters into a personal GB in 8.2 but they are a map document, something that we can use, part of DM

 

data import will be biggest stumbling block ­ have tutorial ready ­ lead in to Patıs talk

 

Will be a data model wall in sail area at UC so MDM poster can be put up there

Zeiler is working on Modeling our World 2 which will have chapters on several of the custom environmental data model efforts, which may include 40-50 pages on MDM but we will still be doing a separate DM reference book

 

Chris ­ another data structure? whale communication distances ­ Great Australian Bight, southern right whales, how compatible is park with potential oil development area ­ combined seabed type dB with world ocean atlas oceanography and whole stack of acoustic parameters and used software from British aerospace and pretended whales were submarines ­ put variables into that software and got set of sections, a cut, one part of compass rose through water section, showing acoustic propagation predicted from model, another section in diff direction from same source, diff acoustic propagation from whatever direction - can make a rosette, rose diagram from this in AV and classify probability of detection (by one whale of another whaleıs call) ­ introduce platform or seismic noise, changes probabilities - can get stack of rosettes, 100 Hz, wavelength of 15-20 m in sediment ­ rosettes are like a jukebox stack ­ can dummy it up to put into Gulf of Maine area - Can we compute rosette as whale moves around? ­ is this a new data structure? Like 2D autocorrelograms says Pat

 

Get questions that we stumbled on with regard to z-values ­ how are we addressing z value ­ as shape or as attributes ­ Michael ­ every shape can have a z value and should take advantage of that, make it an inherent property of shape rather than as an attribute ­ option to have z brought in as a shape ­

Simon ­ easiest to manage and get visible, tangible and shown in model in demo and put an asterisk

Mike ­ z-enable everything

Pat - if z is an attribute in your data than z in shape defaults to NaN(?), and not populated in that shape property ­ PUT THIS IN DOCUMENTATION ­ wait for someone to break it and then deal with it then -

If DM supports z-value it shouldnıt break anything ­ as long as data can get in

Eric ­ would be nice to have AV explicitly ask what do you want to set as your X, Y, AND Z

If we do this we need to make a note that users need to convert their xy data w/z as an attribute with 3D Analyst to put zıs into shapefile, and THEN load into the DM

 

Joeıs demo ­ if you have a set of pts with a z column that has values (an attribute),

3D analyst ­ convert features to 3D ­ run that, base it on something (source of heights)

need 3D Analyst, raster layer is not part of GB that is in DM ­ need ArcSDE for that

 

rasters in 8.1 release ­ no raster catalog or images in ArcSDE ­  this is now in 8.2 -image can be stored in table and additional columns to add more info to that image ­ raster catalog can be another table in the system in 8.2 and can be used in a GB as part of a DM

**canıt bring rasters into a personal GB in 8.2 but they are a map document, something that we can use, part of DM

 

rough template with sample tools vs. data structure that incorporates many variations

one set of tables only vs.

modify data and fields will definitely be a necessity so why not simplify data structure with not so many fields ­ why not just take in attributes you have and say this is a ³time duration vector² or whatever

work from that, produce several data sets we can flag diff things like a time duration buoy table or temperature table that might be needed ­ or we have 7 diff measuring devices that we use all the time and we might

first identify measuring device, will they be the same tables(?), what kind of time period

from bottom up maybe we can see patterns better rather than top down ­ what key things do we really want to focus on ­ we diversity and huge quantity of data ­ a couple of sample data sets help us see any patterns or similarities ­ go to indiv tables or feature classes for TS pts of diff kinds for example ­ geometry and xyz should be in all tables perhaps ­ so if its fixed itıs a feature class, if itıs not fixed then itıs not

 

definition queries to spatially limit data set will be helpful too ­ ³let me look at only top 100 m depth of my data²

 

 

Never got to the following ­ took long time to get data formatted for input to DM:

Conceptual - Breakout Groups to Review Portions of Model?

¨     broad conceptual, time series, conservation/biodiversity, cartographic framework, etc.

 

Adjourn - 5:05 p.m.