I'm still getting up to speed with SmartOS. Here are some notes I took while trying to get PostgreSQL on SmartOS to parse OpenStreetMap (osm) files. My intent is to use Mapbox Studio to do some custom styling for OSM as we develop some "single purpose" maps.
Here's the summary I wrote at LAX when I came home from MMWCON down at UCLA in September 2014.
I was ultimately successful in getting data parsed in the VM and can use Mapbox Studio to connect and style the maps. Now to find some time to finish that little project...
SmartOS, PostgreSQL, PostGis is a Joyent SDC image and works well out of the box.
I had wanted to run osm2pgsql to import OSM files, but could not get a good compile and no native package is available for SmartOS. Probably should ask the community to help get the proper compile dependencies together so that it can ship with the Postgres instance.
Cliff was able to get Imposm 2.5 compiled and I was too. I was unable to get Imposm to read my sample OSM or PBF files correctly. Tried many alternative command line switches. See the firewall and pg_hba notes below. Imposm uses an approach whereby it reads an XML (.osm) or (PBF) file and processes by writing out local tables that are later read into the PostgreSQL db.
I then downloaded Osmosis and Gradle. Gradle is used to build Osmosis. It’s sitting in /opt/local/share. Osmosis seems to be finicky about absolute path names.
Some of the problem in running Osmosis seem to be getting a database connection. That requires configuring
Along the way, I did create an OSM user. Most recipes have an osm database and user. We may use that later.
I also found a script called osm2postgresql that utilizes osmosis. It appeared to read my file, but I couldn’t see any data loaded in tables, so there’s more investigating to do here.
The Joyent image has the postgres user set and most activities can take place in that shell.
Firewall and PostgreSQL Config
I also opened up a firewall rule for 5432 to allow traffic in from the VPN subnet. This allows me to launch VPN and then a OS X copy of pgAdminIII and make a successful connection to the server.
the pg_hba.conf file at a per-database config for a trusted connection to localhost.
Creating the DB
It seems that Imposm and Osmosis create different table spaces than what I’d been using in osm2pgsql. Most of the recipes I’ve been reading involve running the create extension postgis command as well as other schema building sql files simple_schema, simple_load are examples.
Loading Data
One tip I looked at indicated that I need to pull the first line of xml definition out of the OSM file. I did that but can not confirm that I’ve got a successful data load yet. I was still messing with firewall and pg_hba configs.
Things to try:
- it appears that you need to give Osmosis the correct schema command line directive when loading data. Here’s an example that uses the pg_simple schema. It also presumes you’ve got an osm user and password.
/home/disk2/OSM/osmosis/trunk/bin/osmosis \
--read-xml file="/home/disk2/OSM/OSM/uk-090610.osm.bz2" \
--write-pgsimp user="<username>" database="osm" password=“osm”
/opt/local/share/osmosis/package/bin/osmosis --read-xml file=/home/admin/ucsc/ucscmap_091514.osm —write-pgsimp host=localhost database="ucsc_091514" password='~at:G-y)cD'
Are there other Osmosis write or output options?
We could try again with Imposm. It had been failing with a missing python or java library. I looked in google for solutions, but couldn’t find any after many attempts.
It looks like using postgis extension, then hstore extension should be good first steps for setting up the db.
It also looks like from theosm2postgresql.sh script that it creates different table names. Still looks like using osm2pgsql is the easiest way to go with existing recipes