[Tim Osborn]

A couple of month ago in this thread (http://forums.topogr...p?showtopic=941) I showed how a user could make his own map tiles for use in ExpertGPS from aerial photos and topo maps available on the Internet, and provided a couple of simple utilities to aide in that process. But the process was exacting and somewhat limited. You could only produce tiles from maps/photos that were already projected in UTM NAD83. Also you had to be very deliberate to crop the image (usually using MrSid viewer) to specific coordinates and pixel sizes and you could only produce one scale of tiles at a time (e.g. 1 meter/pixel, 4 meters/pixel, etc). Imagemagick was used to bust the cropped image into 200x200 pixel tiles....and since Imagemagick loads the whole image into memory, you couldn't use a very large image.

Since then I have written a much more sophisticated utility that overcomes all these limitations. I call it "Tiler for ExpertGPS". Here are some of its features:

* Permits creation of map tiles (aerial or topographic) to augment or replace the map tiles that ExpertGPS downloads from Terraserver-USA. User does not need to insure that bounding coordinates of source image fall exactly on the Terraserver tile grid, that source image pixel dimensions are an even multiple of 200, or that source image is projected in UTM NAD83.

* Reads most geographic image file formats (e.g. tif, geotiff, jpeg, jpeg2000, ecw, MrSid).

* Permits creation of 1 meter/pixel, 4 meter/pixel, 16 meter/pixel, 64 meter/pixel, and/or 256 meter/pixel tiles from just one source image file of any scale.

* Permits creation of tiles from really large source images (multi-gigabyte) because the whole source image is not loaded into memory at one time.

* Permits saving tiles in an existing ExpertGPS map folder or a separate non-map folder both with or without overwriting existing tiles. If tiles are saved to a separate folder, Tiler will create an appropriate ExpertGPS map folder hierarchy under that folder.

* If the source image file has embedded projection and georeference information as in a GeoTiff file, if needed Tiler will automatically reproject and rescale a temporary copy of the image (warp the image) to UTM NAD83 in the correct UTM Zone consistent with Terraserver/ExpertGPS tiles. This may result in narrow black triangles (no data areas) on border tiles that have to be reprojected.

* If the source image has only an accompanying world file with georeference information (this is the minimum requirement), Tiler will automatically use that information. The user can then manually specify the projection (Lat-Long, UTM, or State Plane), the Datum (NAD27, NAD83, or WGS84) and the linear units (meters, Intl Feet, US Survey Feet, or degrees) of the source image. Tiler will then reproject and rescale as needed to produce tiles.

* If the source image spans a UTM Zone boundary (e.g. part of the image lies in UTM Zone 14 and part in UTM Zone 15), Tiler will automatically deal with each part separately and produce correctly projected ExpertGPS tiles in both Zones.

* If it is necessary to warp the image, Tiler will check if the computer has multiple cpu's or multiple cores and if so will use multiple threading to process chunks of image and perform input/output operation simultaneously, speeding the warping process.

* User may specifiy "crop-in" on the source image so that tiles have a minimum of "no-data" borders, or "pad-out" on the source image so that none of the edge information in the source image is lost in the tile making process. "Crop-in" will most often be used when tiles are being created to replace downloaded Terraserver tiles. "Pad-out" would be appropriate when the user wants to create a separate ExpertGPS map folder for a special project and wants all of the source image to be included at all map scales. However, at 64 pixel/meter and 256 pixel/meter scales, the source image will likely just be a small area of map detail located inside a large black tile(s).

* Provides a running status report on what the program is doing at any give time.

Here is a look at the interface:



Tiler for ExpertGPS uses the GDAL/OGR library of spatial tools developed by Frank Warmerdam, and which is now an open-source project. As such, in order to use Tiler for ExpertGPS, you must download FWTools version 1.3.1 or later from http://fwtools.maptools.org/ and install it in your Program Files folder. You must also have Microsoft .NET Framework version 2 or later installed on your computer. Tiler for ExpertGPS does not use Imagemagick.

The first time you run Tiler for ExpertGPS is will check that you have FWTools 1.3.1 or later installed, and will add the location of the FWTools bin folder to your environment path variable. It will also create 4 additional environment variables. These are the only changes that Tiler for ExpertGPS will make to your system. You can place the folder containing Tiler for ExpertGPS anywhere you want on your harddrive. In addition to the executable Tiler.exe, there are 4 dll files in the folder that must also be present.

You can download a zip file containing Tiler for ExpertGPS from this LINK. Take a look at the "read me" pdf file in the folder after you unzip the downloaded file.

To give you some idea of how long the tiling process takes, I recently processed a 1GB 24-bit RGB tiff file into 8700 tiles at 1 meter, 4 meter, and 16 meter scales in about 26 minutes. A similar 2GB file that resulted in 17,700 tiles took 77 minutes. Small files can be tiled in a shorter time. While the utility is able to read MrSid and ECW files directly, extracting tiles from these formats is extremely slow, so Tiler for ExpertGPS will expand these filetypes into temporary Geotiff files in the Temp folder on your harddrive. However, a 500MB MrSid file can easily turn into a 25GB Geotiff file....so be careful! In such a case it would be better to export a smaller area of interest using MrSid Viewer and then convert that into tiles for ExpertGPS.

I have converted many USGS DRG's, a few Canadian Topos, and portions of many U.S. aerial photos (mostly NAIP images), with Tiler for ExpertGPS and have eliminated the bugs I have found. However, it's likely that some remain. If you run into one, let me know here and I will try to fix it. I developed and tested Tiler for ExpertGPS under Windows XP. I have no idea if it plays well with Vista. As with all free software, use at your own risk and I hope you find it helpful.

Tim Osborn

[Chris Bolstad]

Thanks Tim

Chris Bolstad

[Dan Foster]

On Windows Vista, the first time you run the program, you should right-click Tiler.exe and Run as Administrator. Otherwise, you'll get an error when the program tries to set some environment variables.

[Dan Anderson]

Tim Osborn, on Jun 3 2007, 07:48 PM, said:

* If the source image has only an accompanying world file with georeference information (this is the minimum requirement), Tiler will automatically use that information. The user can then manually specify the projection (Lat-Long, UTM, or State Plane), the Datum (NAD27, NAD83, or WGS84) and the linear units (meters, Intl Feet, US Survey Feet, or degrees) of the source image. Tiler will then reproject and rescale as needed to produce tiles.

Thanks! for your work Tim.

You might consider supporting a modification that Karen of GPSy has made to the world files. She added the UTM zone and datum information to the world files. See...

http://www.gpsy.com/faq/maps.html#tfw

[Dan Foster]

Any chance you'll extend this to support the new 1/4 meter Urban Imagery in ExpertGPS?

[Tim Osborn]

Sure. I'll need to know how you are going to implement it in ExpertGPS (I don't think you have yet...right?). On Terraserver it looks like urban area imagery is its own map type (T=4), and is available at scales 8-19.

[Dan Foster]

I snuck Urban imagery into ExpertGPS a few weeks ago - click Show Urban Photo Map on the Map menu (or U on the keyboard).

ExpertGPS uses the even terraserver scales (8, 10, 12...)


I spent all morning trying to convert two 8000x8000 MrSID files to a scanned map. I finally gave up around lunchtime, and used your Tiler to create the imagery I needed in about 5 minutes. Thanks again for creating this!!!

[shura_k]

I've convert some file with my region for tiles for ExpertGPS (Zone 34, Type 1)
Load my track and try to view it on map.
Program says me that "Some map data lies outside the United State and cannot be shown."
What is it? Why i cant work with my region even without Internet map retrieval?

[Miguel]

shura_k, on Mar 18 2008, 11:34 AM, said:

I've convert some file with my region for tiles for ExpertGPS (Zone 34, Type 1)
Load my track and try to view it on map.
Program says me that "Some map data lies outside the United State and cannot be shown."
What is it? Why i cant work with my region even without Internet map retrieval?

Hello I have the same problem, I convert a geotiff file and the following folder was added to the type1 existing folder: zone16, but when I want to go to a waypoint in the region it said that the data is outside US, I think is a boundary condition of the expertGPS program, no the tiler, is there a way to overcome that, and let the program go outside the US boundary?

Thaks

[DonK]

I downloaded and installed your utility, but so far I have not had success in getting it to work the way I would like.

What I want to do is combine several 4km x 4km .tif files of high quality scans of USGS topo available from the MassGIS. Each of these 4km x 4km tiles is available with it State Plane tfw world file. I would like to make a larger tile out of them which I can import into ExpertGPS (I recently emailed Dan F. on this topic)

Is this something the utility is designed for? If so, can you either give me or point me to instructions for doing this?

When I run the utlity, I end up with a series of .png and .xml files. How do I import these as map tiles into ExpertGPS?

Thanks for any help,

Don

[Tim Osborn]

The utility was designed for taking one georeferenced image and constructing from it the tiles necessary to display those tiles in ExpertGPS the same way ExpertGPS displays topographic map tiles and aerial photo tiles from Terraserver USA. It wasn't designed to combine separate georeferenced images. You could use Tiler to break down the separate images into tiles, but there would be a strong probability that you would loose a certain amount of the images around their edges. That's why it's better to feed Tiler one image covering the entire area you are interested in.

You can use the gdalwarp utility that comes with FWTools to combine multiple tif files into one large tiff file. The command-line utility in this case would be

gdalwarp -s_srs EPSG:26986 -t_srs EPSG:26986 sourcefile1.tif sourcefile2.tif ... destinationfile.tif

where sourcefile1.tif sourcefile2.tif ... are two or more of your source files from Mass GIS and destinationfile.tif is the name you want your destination file to have. The destination file will be a geotiff file with the georeference information stored inside the file. Tiler will get the necessary information from the file so no world file is needed.

I don't know why you are getting .png files. Tiler should only produce .jpg files. The .xml files are the result of a recent change in FWTools, which Tiler calls on. ExpertGPS doesn't use these .xml files so you can delete them.

Additional:

I just downloaded one of the 4km x 4km topographic images from Mass GIS. I was able to convert it into tiles without any problems using Tiler and I was able to view those tiles in ExpertGPS. The tiles from Mass GIS are in the lower right portion of the attached image.

[DonK]

Tim Osborn, on Nov 4 2008, 10:18 PM, said:

It wasn't designed to combine separate georeferenced images.

That is what I want to do: Combine several 4km x 4km .tif tiles into a single tile so I can view it in ExpertGPS.

If I understand you correctly, that is what the gdalwarp utility does?

I tried entering the following in the command line:

gdalwarp -s_srs EPSG:26986 -t_srs EPSG:26986 C:\Documents and Settings\Office\Favorites\Travel\ExpertGPS\q165922.tif C:\Documents and Settings\Office\Favorites\Travel\ExpertGPS\q165926.tif ... C:\Documents and Settings\Office\Favorites\Travel\ExpertGPS\OtterRiver.tif

I get the following error: "ERROR 4: `C:\Documents' does not exist in the file system, and is not recognised as a supported dataset name."


This looks like a path problem? Where should the .tif files (and .tfw files) be located? I have to admit my experience working with command line applications is limited, so I don't want bother you with too many questions.

The ".png" extension was my error. You are correct, they are .jpg files along with the.xml files which are generated.

Thanks

Don

[Tim Osborn]

I'm pretty sure that the command line interpreter in Windows is getting hung-up on the space character in the "Documents and Settings" part of the path. It's not seeing the rest of the path after the first space.

It would be best for you to create a new folder off a root drive with a simple name like "MERGEMAP". Copy all your input tif files and their associated tfw files into that folder. Then start a command-line session, navigate to the MERGEMAP folder using the DOS CD command, and then issue your GDALWARP command like this:

gdalwarp -s_srs EPSG:26986 -t_srs EPSG:26986 q165922.tif q165926.tif ... OtterRiver.tif [this command has been corrected]

If you just want to view the combined maps in ExpertGPS using the scanned map feature and not split the map up into tiles in order to view the map in ExpertGPS in the same way that Terraserver tiles are viewed, you should use the following GDALWARP command line. It will create a tfw file which ExpertGPS will need.

gdalwarp -s_srs EPSG:26986 -t_srs EPSG:26986 -of GTiff -co "TFW=YES" q165922.tif q165926.tif ... OtterRiver.tif [this command has been corrected]

[DonK]

Thanks, Tim, for working with me.

I tried your both of your command suggestions and here is what I get:

C:\Mergemap>gdalwarp s_srs EPSG:26986 -t_srs EPSG:26986 -of GTiff -co "TFW=YES"
q165922.tif q165926.tif ... OtterRiver.tif
ERROR 4: `s_srs' does not exist in the file system,
and is not recognised as a supported dataset name.

C:\Mergemap>

I also ran setfw.bat as suggested in the readme notes.

Don

[Dan Anderson]

DonK, on Nov 6 2008, 08:28 AM, said:

[font="Courier New"]C:\Mergemap>gdalwarp s_srs EPSG:26986 -t_srs EPSG:26986 -of GTiff -co "TFW=YES"
q165922.tif q165926.tif ... OtterRiver.tif
ERROR 4: `s_srs' does not exist in the file system,
and is not recognised as a supported dataset name.

My guess: try putting a "-" in front of "s_srs".

So it will start...

gdalwarp -s_srs EPSG:26986 -t_srs

[Tim Osborn]

Dan's got it right. I omitted the dash before the s_srs in my previous post.

It should be:

gdalwarp -s_srs EPSG:26986 -t_srs EPSG:26986 -of GTiff -co "TFW=YES" q165922.tif q165926.tif ... OtterRiver.tif

[DonK]

Tim Osborn, on Nov 6 2008, 01:02 PM, said:

Dan's got it right. I omitted the dash before the s_srs in my previous post.

It should be:

gdalwarp -s_srs EPSG:26986 -t_srs EPSG:26986 -of GTiff -co "TFW=YES" q165922.tif q165926.tif ... OtterRiver.tif

Well, we're making progress. I ran the above command. An error 4 was still generated, however, this time it didn't like ' ... ', so I removed the two spaces either side of "..." and the command ran without an error.

Here's what it generated: A new file called "q165926.tif...OtterRiver.tif". However, this file does not contain a merged version of q165922.tif AND q165926.tif, but only the single tile q165922.tif.

Also, there seems to be an issue with the new .tfw which is generated. I only see a file called ".tfw" which was generated at the same time as the q165926.tif...OtterRiver.tif file.

I'll try to get back an look at this issue later today.

Ideas?


Don

[Tim Osborn]

Don:

The ... was meant to indicate that you could add two or more input files by continuing to list them in the same way as the first two. The last filename entered will be the output file. If you were only going to mosaic two images the command would be:

gdalwarp -s_srs EPSG:26986 -t_srs EPSG:26986 -of GTiff -co "TFW=YES" q165922.tif q165926.tif OtterRiver.tif


Documentation for gdalwarp is at http://www.gdal.org/gdalwarp.html

[DonK]

Thanks Tim,

I was able to merge four .tif tiles into one with no problems. Thanks for sticking with me. It's been a while since I did anything with the Windows command prompt interface!

Yes, I discovered the documentation pages this afternoon. Now that I see that it can do what I want, I'll spend some time looking over the documentation.

Thanks again for your help and the work you put in constructing the GDAL Library.

Don

[Tim Osborn]

Don:

I'm glad you were able to get things to work. BTW...I didn't have anything to do with constructing the GDAL library. I'm just a grateful user that has incorporated parts of it in Tiler.