GRASS 5.0.x and PostgreSQL - First Steps
[$Id: index.html,v 1.14 2003/02/05 09:06:12 markus Exp $]
Recommended reading:
http://www.PostgreSQL.org/docs/index.html
GRASS 5.0.x/PostgreSQL interface modules
- g.select.pg - select a database to use in GRASS 5.0.x.
- g.table.pg - list tables in currently selected DB.
- g.columns.pg - list columns information in a table of selected DB.
- g.stats.pg - calculate means, min, max, frequencies of specified
numeric column.
- d.site.pg - display and optionally copy/reclass sites selected by
DB query.
- d.vect.pg - display and optionally extract lines/areas selected
by DB query.
- d.rast.pg - display(optionally) and reclass raster maps through
DB query.
- d.what.s.pg - query with mouse sites on graphics monitor. Option
to dump the fields in comma-separated lists for other programs.
- d.what.v.pg - query with mouse lines/areas. May be used by UPDATE
command in PostgreSQL input through SQL command file. Consider piping
anything you grabbed on screen through various filters.
- d.what.r.pg - the same as above on raster maps.
- v.reclass.pg - reclass vector maps, rules based on any SELECT
filter.
- v.in.shape.pg - DEPRECATED. Use v.in.shape + pg.in.dbf instead.
- v.in.arc.pg - DEPRECATED. Use v.in.arc + pg.in.dbf instead.
- pg.in.dbf - program that inputs DBF file and dumps all columns to
PostgreSQL.
- v.to.pg - this program is used to export GRASS 5.0.x vectors
(lines or polygons) to PostgreSQL native "line" or "polygon" formats.
- NVIZ2.2 - PostgreSQL interface. I have added new entries field in
the "What's here" panel; PostgreSQL queries may be disabled in
"Attributes" checkbox.
Please see these modules manuals for explanations and examples.
NB: Tcl-Tk modules may have input "where" clause restricted to one rule
typed without "whitespace" while in terminal input the number of
sub-clause is not limited by this.
Bugs and other possible caveats reports are welcome.
Alex Shevlakov,
sixote@yahoo.com
==================================================================
Short introduction to GRASS 5.0.x/PostgreSQL interface Markus Neteler
and Alex Shevlakov
The following text shall introduce you to interface usage. This text is
subject to change...
Let's start.
Enter GRASS 5.0.x
Of course you need to have a location defined and some data you want to
import here.
------------------------------------------------------------------------
A) First see, if PostgreSQL is working - check for errors
------------------------------------------------------------------------
We begin with looking at the list of existing databases:
g.select.pg -l
Error: select PostgreSQL:connectDB() failed: Is the postmaster running
and accepting TCP/IP(with -i) connections at '130.77.22.66' on port
'5432'? See for the log file:
cat /var/log/PostgreSQL.log-> No data directory -- can't proceed.
/usr/lib/pgsql/bin/postmaster does not find the database system.
Expected to find it in the PGDATA directory "/var/lib/pgsql/data", but
unable to open file with pathname
"/var/lib/pgsql/data/base/template1/pg_class".In this case we have to
install further packages (here: SuSe Linux, names may be different in
your installation):
- pg_datab.rpm needs to be installed
- ps_ifa.rpm, maybe pg_iface.rpm Restart the "postmaster" (the daemon
listening for db-queries):
cd /sbin/init.d/rc2.d/
./S25PostgreSQL start
ps -ax |grep postmaster
Is postmaster there? You should see it in the process list. Check
again, if it working now:
g.select.pg -l
Error: select PostgreSQL: User authentication failed
You need to setup the PostgreSQL user's list. Otherwise jump to letter
B in the text.
In case of error, enter:
su
Set a password for user "PostgreSQL" (this is needed first time only!):
passwd PostgreSQL
[set the password]
exit Now login as user "PostgreSQL":
su - PostgreSQL
and add yourself as PostgreSQL-user:
createuser neteler
Enter user's PostgreSQL ID or RETURN to use unix user ID: 601 ->
<return>
Is user "neteler" allowed to create databases (y/n) y
Is user "neteler" allowed to add users? (y/n) n
createuser: neteler was successfully added
exit
Now you are again back in GRASS 5.0.x environment: Again we try:
g.select.pg -l
The following databases are in the Unix catalogue:
template1
This indicates that you PostgreSQL environment is o.k.
Fine. Now we can proceed and start using the interface.
------------------------------------------------------------------------
B) Using GRASS 5.0.x/PostgreSQL: creating a database and importing table
------------------------------------------------------------------------
Say, you have a SHAPE-file set: humus.shp, humus.shx and humus.dbf.
The file *.shp will be imported into GRASS 5.0.x, the file *.dbf into
PostgreSQL.
Now you have to use "createdb" do create database tables. First we
create an empty database
createdb humus
This new table we select in GRASS 5.0.x:
g.select.pg database=humus To destroy a database use: "destroydb humus"
(PostgreSQL 6.x) or "dropdb humus" (PostgreSQL 7.x).
In this first step we import the plain attribute table only without
importing geographical features.
To import the Dbase-table into PostgreSQL enter:
pg.in.dbf in=humus.dbf
Executing create table humus (AREA float4,PERIMETER float4,G2_UEB09_
int8,G2_UEB09_I int8,STONR int4,BOTYP text,HORIZ text,BODART text,HUMUS
float4,SKELETT text)
You will be asked: Additionally dump to ASCII file (enter full Unix
name or hit <return> for none):
Enter "ENTER" if you don't need an additional ASCII file in your local
directory.
The table is imported into PostgreSQL now. Note: The vectors are not
yetr imported!
------------------------------------------------------------------------
C) Getting simple table statistics
------------------------------------------------------------------------
Now we want to get some information: For teaching real life we start
with an error...
g.stats.pg table=humus col=BOTYP
Error: connect PostgreSQL:ERROR: No such function 'min' with the
specified attributes The reason is that BOTYP is a text field (yes, see
above in the output of pg.in.dbf!). Of course we cannot calculate
statistics from letters.
So we try another field:
g.stats.pg table=humus col=HUMUS
Min, Max, Mean
-------------------------------------
2.81, 1.72372
Well, quite nice.
It tells us about humus contents of soil patches in percent.
To proceed we remove this database:
destroydb humus # on PostgreSQL 6.x
dropdb humus # on PostgreSQL 7.x
------------------------------------------------------------------------
D) Importing a SHAPE file with DBASE table (*.dbf)
------------------------------------------------------------------------
Deprecated. Use v.in.shape + pg.in.dbf
------------------------------------------------------------------------
E) Getting table information
------------------------------------------------------------------------
Here we can use
g.column.pg -v table=humus
| columnname | type | length |
----------------------------------------------------
| area | float4 4 |
| perimeter | float4 4 |
| g2_ueb09_ | int8 8 |
| g2_ueb09_i | int8 8 |
| stonr | int4 4 |
| botyp | text var |
| horiz | text var |
| bodart | text var |
| humus | float4 4 |
| skelett | text var |
As you can see you get information about the field types within the
PostgreSQL table.
------------------------------------------------------------------------
F) Selecting table entries and displaying on the map
------------------------------------------------------------------------
To select map features and display the the selected areas/lines in
GRASS 5.0.x Monitor, you can use
d.vect.pg A SQL-statement is required to select the features of
interest.
Create an ASCII file "sql.query" (using an text editor) with contents:
select stonr from humus where HUMUS > 1.2
[change "humus" to your table name and "HUMUS" to a column existing in
your table]
Now query:
d.vect.pg -f -s sql.query map=humus color=red
Alternate method: Write query into command line:
d.vect.pg -f key=HUMUS tab=humus where='HUMUS>1.2' map=humus col=red
Executing
SELECT Distinct HUMUS from humus where HUMUS>1.2 and HUMUS is not
null;
16 Rows
key is the column name, tab the table, and where the statement.
"-f" fills the selected areas, col defines their colors.
Now you see the selected areas in the GRASS 5.0.x Monitor.
------------------------------------------------------------------------
G) Query example
------------------------------------------------------------------------
There's a map of digitized forest stands called "terney_id" (say three
hundred in the region); each plot has unique rec_id in database table
info_terney.
Now we'd like to calculate correlations between hights and diameters of
trees in plots with main species Pinus koraensis (type_id <21), with
age more than 100 years, growing in southern slopes and lying along
specific routes.
(Remember: If you forgot the column names of your table, use
g.column.pg - see above)
First thing we highlight red all plots that satisfy these conditions.
We write a sql-statement ASCII file "query1.sql":
select rec_id from info_terney where type_id <21 and age > 100
and expo ~ 's' Then we use the query command:
d.vect.pg -f -s query1.sql map=terney_id color=red
Then, we pick these plots in d.what.v.pg, following along our routes:
d.what.v.pg -f map=terney_id tab=info_terney col=rec_id color=green
fillcolor=gray hv=h
Recommendation: Use the TclTkGRASS with PostgreSQL support. Then you
can easily copy-paste results.
After this module is done, we Control-C the results from tcltkgrass
output window and paste them to spreadsheet.
Now we can calculate correlations (and other stats).
------------------------------------------------------------------------
H) Import of ARC ungenerate files
------------------------------------------------------------------------
Deprecated. Use v.in.[shape|arc] + pg.in.dbf
------------------------------------------------------------------------
I) Reclass of vector map
------------------------------------------------------------------------
1. Reclass to vector map of quartiles from forest stands map
(kuruma_id).
v.reclass.pg -s -d sql=reclass.sql input=kuruma_id output=kuruma_quart
type=area
and reclass.sql is:
select rec_id, quartnum from info_kuruma
2. Reclass to vector map of forest types (kuruma_oak) from map of
forest plots (kuruma_id) taking only oak (types 32-37).
v.reclass.pg -d kuruma_id key=rec_id col=type_id tab=info_kuruma
where='type_id > 31 and type_id < 38' output=kuruma_oak type=area
------------------------------------------------------------------------
J) Problems importing DBF-tables into PostgreSQL?
------------------------------------------------------------------------
The most common problem ( as i run into it too often) while converting
*.dbf files to PostgreSQL with pg.in.dbf is format dismatch - pg_atoi()
ERROR - saying there's something in the field declared as int (or
float) that does not seem like number, such as "***", "NO", "infrared
spectrum", etc.
Alas, PostgreSQL is very restrictive (unlike we people who type in this
stuff). My approach to this (as i still want to import the .shp from
Arcview that my collegue sent me from ArcviewLand) - exit the program -
it would already have done so-, and rerun the module saying "no" to
dbf-to-PostgreSQL dump.okay, after we have imported the coverage ,
let's use pg.in.dbf and say "yes" to question "Do you want additionally
DUMP to ASCII?".
Now, find the line(s) that spoils your breakfast (hopethere are not
many), kill'em all and then - use psql and COPY TABLE from
'/home/user/user.stuff'. That's it. Besides, dumping to ascii file and
doing things about it before importing to PostgreSQL is a must when
there are some weird encoding chars in text fields.
--alex
-------------------------
Please send tutorial improvements to
Markus Neteler
<neteler@itc.it>