regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
We have released some code which we hope will be of
interest to those working with Region KML.
For more information please visit:
http://code.google.com/p/regionator/
Here is a direct link to the project's README:
http://regionator.googlecode.com/svn/trunk/README
Edited by ink_polaroid (08/10/06 05:06 PM)
|
bplewe
Tourist
Reged: 03/31/05
Posts: 14
Loc: Spanish Fork, Utah
|
|
A non-programmer having some trouble getting regionator to work: We're running under windows (not CygWin), which may be part of the trouble, and python 2.4, GDAL, and numpy seem to be installed and working. The tests in the test folder all worked without a hitch. [ImageMagick was only needed for the image tests, not for regionator in general, right?]
However, whenever we try to run a script that opens an image (like superoverlay.py or testextractor.py), we get the following:
self.__in_ds = gdal.Open(imgfile)
AtrributeError: 'module' object has no attribute 'Open'
My guess is that this is an installation snafu, not a bug in the program. Any ideas?
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
Sounds like a problem with the installation of gdal or the gdal python bindings.
A quick first test of just gdal's python installation might be to start python
interactively and enter the following lines manually. This presumes 'screeno01.jpg'
exists (yes, ImageMagick is only to create these files for other tests and is not
needed for creating a superoverlay). Obviously if you use a different image file
the x,y values may be different.
>>> import gdal
>>> ds = gdal.Open('screeno01.jpg')
>>> print ds.RasterXSize
32
>>> print print ds.RasterYSize
32
|
bplewe
Tourist
Reged: 03/31/05
Posts: 14
Loc: Spanish Fork, Utah
|
|
Yes, it was the installation of GDAL. "import gdal" was interpreted as the gdal folder, not gdal.py, which confused everything. We ended up having to change the folder to "gdal1", then add that folder explicitly to the pythonpath. Not a clean workaround, and not sure what is different from your system, but it works. Works great, in fact. Thanks for the tool!
|
vestigialars
Tourist
Reged: 10/05/06
Posts: 6
|
|
Hi All,
I am having similar problems on Windows, I'm keen to try regionator. Anyone willing to help me? I am a newbie on Python and GDAL, all my imagery is on a Novell network and my sysman only allows logins from our Windows machines, so Windows is my only useful option for this work. I've got FWTools installed, which I think means GDAL is installed somehow as well, and have also put GDAL 1.3.2 in a couple other places, all of which are indicated on PYTHONPATH.
However, when I try and run any of the image based test scripts or the superoverlay.py script I am told:
C:\temp\subversion\build\scripts-2.5>superoverlay.py
Traceback (most recent call last):
File "C:\temp\subversion\build\scripts-2.5\superoverlay.py", line 28, in <module>
import kml.superoverlay
File "C:\Python25\Lib\site-packages\kml\superoverlay.py", line 27, in <module>
import kml.image
File "C:\Python25\Lib\site-packages\kml\image.py", line 25, in <module>
import gdal
File "C:\gdal-1.3.2\pymod\gdal.py", line 191, in <module>
import _gdal
ImportError: No module named _gdal
Anyone know what to do? Got any help? Anything at all?
Thanks,
Lars
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
Python isn't finding gdal. To more directly debug this start python
interactively and enter "import gdal". Once that works the regionator code
which uses gdal will also work.
Also fwtools includes its own fairly old python so watch you might want
to be mindful of mixing different python installations.
|
vestigialars
Tourist
Reged: 10/05/06
Posts: 6
|
|
Okay, gave up on the windows version. My gdal says it is fine but the _gdal is throwing everything off. I am probably missing something simple but can't figure it out right now.
Happily, I have it running on a linux box and it seems to be working. So far so good, thanks!
Lars
|
vestigialars
Tourist
Reged: 10/05/06
Posts: 6
|
|
Hi All, me again.
Any advice on how to improve the output image quality from regionator? I am using it on large high-res imagery showing small villages in Darfur. The output JPEGs are downsampled to a certain extent and I lose some quality overall. Not a lot, but enough that I notice. Any quick fix?
Also, at some point I ran regionator and it produced PNGs. Why did that happen, and do I have control over it?
Thanks for any tips...
|
pawill79
Tourist
Reged: 02/03/05
Posts: 14
|
|
I have been trying to get the "regionator" to output my files as PNG's only. I have found the image.py and extractor.py as possible culprits for automatically choosing my output files but have not successfully edited those files to do what I want.
Has anybody changed these files so that only PNG's are produced, it seems I am only able to create GTiff output files.
|
pawill79
Tourist
Reged: 02/03/05
Posts: 14
|
|
Ok, I figured this one out. Forgot to recompile the regionator after making changes to the extractor.py and image.py files.
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
when running superoverlay.py or testextractor.py I am getting the following error:
ERROR 1: TIFFOpen:/tmp/file.gtiff: No such file or directory
Traceback (most recent call last):
File "C:\regionator\tests\testextractor.py", line 36, in ?
ex.Extract(0,0,256,256,'0')
File "C:\regionator\kml\extractor.py", line 100, in Extract
o_ds.WriteRaster(0,0,twid,tht,i_data)
AttributeError: 'NoneType' object has no attribute 'WriteRaster'
can anyone help?
--------------------
-Pattt
http://adventuresforthecure.com
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
oh nevermind. I simply had to create a C:\tmp directory on my system.
--------------------
-Pattt
http://adventuresforthecure.com
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
Tnx for reporting this.
I just updated the code to use a safer and platform independent temp file method.
happy regionating...
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
Awesome! Thanks Regionator... I just checked out the new code and it works great.
I am new to python (started using it 2 days ago when I checked out regionator).
when running the regionator code I am getting the following errors:
version 81
original dimensions (880, 880)
maxdepth 4
count 24
root region (28.828125, 28.125, -80.15625, -80.859375)
llb-3\4.kml
llb-3\5.kml
llb-3\6.kml
llb-3\7.kml
llb-3\3.kml
llb-3\9.kml
llb-3\10.kml
llb-3\11.kml
llb-3\12.kml
llb-3\8.kml
llb-3\2.kml
llb-3\15.kml
llb-3\16.kml
llb-3\14.kml
llb-3\13.kml
llb-3\18.kml
llb-3\20.kml
llb-3\21.kml
llb-3\22.kml
llb-3\23.kml
llb-3\19.kml
llb-3\17.kml
llb-3\24.kml
llb-3\1.kml
Extractor NASA_KSC.jpg 3 bands
1/24 llb-3\1.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpx-lf88GTiff failed.
2/24 llb-3\2.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmp__uo3bGTiff failed.
3/24 llb-3\3.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpudegm5GTiff failed.
4/24 llb-3\4.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmp78vghwGTiff failed.
5/24 llb-3\5.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpctoubgGTiff failed.
6/24 llb-3\6.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpl8odn2GTiff failed.
7/24 llb-3\7.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpungpbyGTiff failed.
8/24 llb-3\8.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpzaxovtGTiff failed.
9/24 llb-3\9.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpisjsccGTiff failed.
10/24 llb-3\10.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpan4qtcGTiff failed.
11/24 llb-3\11.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmp4qcwkyGTiff failed.
12/24 llb-3\12.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpo-ppxvGTiff failed.
13/24 llb-3\13.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpef8lb1GTiff failed.
14/24 llb-3\14.JPEG
15/24 llb-3\15.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpp7swwtGTiff failed.
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpcdunluGTiff failed.
16/24 llb-3\16.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpxvz7h5GTiff failed.
17/24 llb-3\17.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpsf4jhcGTiff failed.
18/24 llb-3\18.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpffdsakGTiff failed.
19/24 llb-3\19.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpq5c5s3GTiff failed.
20/24 llb-3\20.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpcchkdmGTiff failed.
21/24 llb-3\21.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmppeoxfeGTiff failed.
22/24 llb-3\22.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpf3wcv4GTiff failed.
23/24 llb-3\23.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmpiqfgotGTiff failed.
24/24 llb-3\24.JPEG
ERROR 1: GTiff: Attempt to unlink c:\docume~1\admini~1\locals~1\temp\tmptfzq5sGTiff failed.
What does that mean and where is it comming from? The error message is not very descriptive.
Thanks!
--------------------
-Pattt
http://adventuresforthecure.com
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
I had to do the same thing on my Windows system. Has anyone figured out a better way to do this?
--------------------
-Pattt
http://adventuresforthecure.com
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
Here are the steps that I followed to get it working on my windows machine.
1. Go to Google Earth Regionator Project Home and checkout the latest version of the code using svn (I used TortuseSVN )
2. Go to Python 2.4 and download Python 2.4
3. Go to UrbanSim's GDAL installation page and download/install GDAL (I followed the instructions for Quick Installation )
4. Go to NumPy and download/install NumPy from it’s Sourceforge download site (you should download numpy-1.0.win32-py2.4.exe)
Put all those pieces together in the same order as above and everything should work great (on windows).
For fun I also downloaded PyDev which is a super awesome extension to Eclipse 3.2 for python development!!
--------------------
-Pattt
http://adventuresforthecure.com
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
So,
I have a ridiculously big jpg image ~ 20,000px x 20,000px. I tried running regionator on it and it quickly creates ~ 5,500.kml files. Then it starts making the .jpeg files. The first few take almos 1 minute each to create but it gets faster after that. I have not delved into the code yet and I am not saying that it runs too slow, but I am wondering if there is any easy way to make it faster? Or could someone at least point me in the right direction or have any ideas about this?
Thanks!
--------------------
-Pattt
http://adventuresforthecure.com
|
Forkboy2
Cartographer
Reged: 07/16/05
Posts: 445
|
|
I have a feeling you're going to start running into problems with a file that large even if you do get it to work. The more levels of nested KMLs you have, the more GE seems to choke trying to process all of them as you zoom in. Anything more than a 32x32 grid of tiles and things really start to slow down. At least that's been my experience. Might not be so bad if you're not planning on putting the files online.
Matt
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
20k x 20k is entirely reasonable (and pretty much the whole point :-)
and we've done significantly larger with this exact software.
Extracting 5000+ tiles takes time indeed. Some will be faster than
others as the code is extracting and resampling the original at
each level of hierarchy. I would guess 10's of minutes, but it
obviously depends on your hardware and likely very much on
how much RAM you have.
We've created SuperOverlays upwards of 400,000 tiles and pretty
deep hierarchies.
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
Thats good news.
Say I am running on a multiprocessor system does anyone have any suggestions on how I could better thread the program to make it run faster? Would adding threading even make a difference?
Or is the only solution to this problem (kinda slow with huge files) to throw more hardware at it?
--------------------
-Pattt
http://adventuresforthecure.com
|
denisew
Tourist
Reged: 10/24/06
Posts: 2
|
|
Colégio Inovação II
|
ink_polaroid
Sysop
Reged: 01/01/05
Posts: 1868
Loc: SF Bay Area, CA
|
|
Internally, GDAL (the heart of the regionator) will work with TIFF files, no matter what you throw at it.
In the case of a very large JPEG, it will decompress that file and convert to TIFF many, many times. This is the source of the slow operations.
You should convert any large source file to TIFF before regionating it...
The brain-off method is to use gdal_translate:
gda_translate img.jpg img.tif
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
A 2006 vintage computer with a couple of gig of ram and a ghz or 2 of cpu
completes a 20k x 20k SuperOverlay in less than 15 minutes. Some tiles will
take far longer than others as the process involves downsampling all to most
of the entire image for the coarsest levels in the hierarchy. I most commonly
use an Apple PowerBook G4 (yes, not even an Intel MacBook) with 2gb of ram
and a mere 1.67 ghz PowerPC cpu. This generally presumes TIFF input for
some reasons Ink mentioned previously.
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
Anyone creating SuperOverlay tools might be interested in this:
http://code.google.com/p/regionator/issues/detail?id=1&can=2&q=
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
Thanks Ink, Regionator. Converting to a .tiff and using a Windows Server with a 3.80GHz processor and 3.5 GB of RAM solved all my problems!
Converted a 20,000px x 20,000px in 11mins
--------------------
-Pattt
http://adventuresforthecure.com
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
I modified the code in extractor.py starting at line 99.. so that only 1 temporary
file is created. Since on Windows there is a bug in deleting a temporary file using
Python, this change will create only 1 temporary file that will always be used each
time the program is run.
There may be a problem with this change if you run more than one instance of
regionator simultaneously or if the code is modified to be multithreaded.
Any suggestions or corrections would be greatly appreciated
The code in extractor.py after line 99 was changed to the following:
# Have to Create out to GTiff first (?)
tmpfilePath = tempfile.gettempdir()+"\\tmpfile"
tmpfile = open(tmpfilePath, "w")
o_ds = self.__gtiff_driver.Create(tmpfilePath,twid,tht,bands=self.__bands)
o_ds.WriteRaster(0,0,twid,tht,i_data)
# Save off using the specified driver
filename = '%s.%s' % (basename,self.__fmt)
self.__o_driver.CreateCopy(filename,o_ds)
if self.__verbose:
print filename
# Delete the tmp file
tmpfile.close()
# Does not delete the file if running on Windows.
# Will produce the following:
# [Errno 13] Permission denied:
# 'c:\\docume~1\\admini~1\\locals~1\\temp\\tmpfile'
self.__gtiff_driver.Delete(tmpfilePath)
--------------------
-Pattt
http://adventuresforthecure.com
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
What exact version of Python and Windows are you have this problem?
I happen to have a colleague who knows alot about Python :-)
(The original extractor code actually did what you did to your copy,
but as you point out this is unsafe).
Other Python on Windows hackers are free to chime in with their
tmp file creation experiences.
Also, just to make your life difficult kml/extractor.py was just modified
to explicitely set a higher quality level on the JPEG output driver.
The default is "75" which is just plain too low most of the time.
extractor.py now sets it to "90".
|
PatrickB
Tourist
Reged: 07/28/05
Posts: 17
|
|
I am using Python 2.4 on Windows XP and Windows 2003 Server (both give the same error). I will try using Python 2.5 instead!
--------------------
-Pattt
http://adventuresforthecure.com
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
The latest release of GE4 (241X) supports GroundOverlays at altitude and
the Regionator code has been updated to support this as well.
superoverlay.py now takes a groundoverlay.kml as input and
parses out LatLonBox, altitude, drawOrder as well as time and
reflects such into the superoverlay.
Here is an example:
http://regionator.googlecode.com/svn/trunk/tests/ksc-llb-a.kml
An update to the KML 2.1 Reference will appear soon...
|
pawill79
Tourist
Reged: 02/03/05
Posts: 14
|
|
To make the regionator more dynamic for your webserver... You can edit /usr/local/lib/python2.4/site-packages/kml/regionator.py (or your path to python site packages) and change...
link.href = '%s/1.kml' % dir
to...
link.href = 'http://www.yourwebsite.com/%s/1.kml' % dir
or even
link.href = 'http://www.yourwebsite.com:8080/%s/1.kml' % dir
if you have to run your webserver on a different port.
This prevents you from having to edit the root.kml after running the superoverlay.py
|
regionator
The Crusader
Reged: 01/24/06
Posts: 63
|
|
I'm not following why it's easier to change every KML file
within the hierarchy than it is to change just the root.kml file?
(This is the whole point of root.kml).
But, if you for some reason wish to bake absolute <href>'s
into your KML that will work most certainly. You might
also wish to do that with the <href>'s of the <GroundOverlay>
<Icon>'s as well?
It's certainly possible to bake in <href>'s to multiple
servers as well. Put kml on one server, and imagery on another.
Or, put different parts or levels of the overlay on different
servers. But, even then I suspect that load balancing
technologies at the webserver might be a better idea.
It's a very powerful concept that all <href>'s are relative
within the KML hierarchy. It means the exact same hierarchy
can be simultanously served over http or accessed as
local files (as I do pretty much all the time: I create the
overlay and verify it on my local file system and then
simply move the whole thing to a directory my favorite
http server can see and now everyone else is one
"Add Networklink" away from seeing it too).
Anyways... tnx for your interest in KML and the Regionator
and let me know if there's something I'm not understanding
about what you are trying to accomplish.
|
pawill79
Tourist
Reged: 02/03/05
Posts: 14
|
|
Making the change I stated previously only changes the "href" for the root.kml. The other associated kml's still have relative urls for its hierarchy. The main reason that the root.kml has to be changed for the href is because if it does not have a absolute http address then it tries to find the associated kml's on the local file system.
Here is an excerpt from the superoverlay.py in the svn trunk...
"The root file is a small NetworkLink KML file which points
to the top of the SuperOverlay. The href to 1.kml should be
changed to be the URL of the directory. This file is
distributed to users of the SuperOverlay. Note that the
NetworkLink within is a Region-based load and that opening
this file in Google Earth neither loads any imagery nor
flies to the SuperOverlay. When the user flies to the
vicinity of the SuperOverlay this first NetworkLink is loaded
which triggers further NetworkLink loads within the SuperOverlay
as appropriate for the users viewpoint."
Since this seems that the root.kml has to be manually changed, I wanted just to show that you could make the change more dynamic by changing some of the python code.
|
jsundqui
Tourist
Reged: 01/06/07
Posts: 5
|
|
Quote:
Sounds like a problem with the installation of gdal or the gdal python bindings.
A quick first test of just gdal's python installation might be to start python
interactively and enter the following lines manually.
--SNIP--
>>> import gdal
--SNIP--
I have the problem of python not seeing gdal.
>>> import gdal
just gives me
ImportError: No module named gdal.
I am running Kubuntu Edgy, and I've installed python-gdal, which puts the modules in /usr/share/pycentral/python-gdal/site-packages
This is the same place that Debian testing puts this, although I noticed that Debian stable put the gdal python bindings in /usr/lib/python2.3/site-packages/
I suppose I could do a brute force kludge with sym-links if python is looking at /usr/lib/python2.*, but is there a better way?
Or is the /usr/share/pycentral/* approach correct, but I have other problems? Seems like there are plenty of other modules in /usr/share/pycentral/* For example, I built numeric before installing regiona |
Previous
Index
Next
Threaded
Edit
Reply
Quote
