Tuesday, September 15, 2009

Getting Data Into OpenJUMP

Over the last year or two I’ve written a couple of plug-ins designed to get data into OpenJUMP. The first of these plug-in imported GPX files, and the second plug-in imported survey point data from a delimited text file. I was allowing the user to access these plug-ins through a menu item under the “SurveyOS” top-level menu, but I recently decided it would be good if they could be located under the “File” top-level menu.

The problem with this is OpenJUMP automatically put the menu items for my plug-ins at the bottom of the File menu, which looks a little awkward.

So I decided to see how difficult it would be to allow access to the plug-ins using the built-in system for opening files to get data into OpenJUMP.

When OpenJUMP was originally written there was only a single menu item that could be used to import files. This menu item opened a file selection dialog, and the user selected the type of file from a list box at the bottom of the dialog box.

A couple years ago, we ripped out this rather simple system and replaced it with another one. I didn’t realize how much we complicated things with this replacement until I started looking at the code today. The current system uses a wizard framework to open different files in OpenJUMP. It implements a system to support “import options” or parameters supplied by the user. It seems to automatically build dialogs that allow the user to enter these parameters based on information from the plug-in that actually imports the data.

Here are the things I don’t like about the current way we open files in OpenJUMP:

(1) A large majority of the files we OpenJUMP don’t require any parameters or options. We just want to suck in the contents of the file. ESRI Shapefiles are an example of this type of file. I’ll bet the majority of the files opened with OpenJUMP are ESRI shapefiles.
(2) The current system limits the steps and validation of parameters that can be used to open a file. I can’t have my plug-in walk the user through a complicated (and custom) wizard, and I can’t really validate the parameters as they are collected from the user. So we are only serving a limited slice of the plug-ins that need to collect parameters from the user before we import data from a file. Only very simple parameters can be collected in our current system.
(3) The plug-in programmer has little or no control over the GUI that is automatically constructed to collect the parameters. In my experience automated GUIs are ugly and hard to work with. They are appropriate for some situations, but I don’t know if this is one of them.
(4) Our current system doesn’t seem to support different plug-ins that import data from files with the same file extension. For example: You might use my recent plug-in to import survey point data from a delimited text files, but there might be several other plug-ins that import entirely different data from delimited text files.
(5) The current system is more complicated than it needs to be. We’ve got at least 24 classes in three (3) different packages that are needed to open files. Should it be that hard?

There are some things I like about the system we currently have for opening files. It keeps track of recent files, which can be handy. But I bet we can do the same thing with something like the old system, and it won’t be nearly as complicated. The new system also allows plug-in programmers to get a GUI for collection parameters needed to open a file without too much work, but I bet if you are writing a plug-in for data import you can handle whipping up a simple GUI.

This is what I will work on implementing in BizzJUMP, and what I would like to see implemented in OpenJUMP:

Move back to a simpler system for importing data from files. This would be something similar to the old system. I’d like to see 2 menu items for each supported file format. One menu item for each file format will be used to import data, and will be found under File>Open File. One menu item for each file format will be used to export data, and will be found under File>Export File (or save file). Each plug-in programmer will place his own menu items for file import and export, just like other plug-ins place their own menu items.

I will create plug-ins for the standard file formats that OpenJUMP already supports:
ESRI Shapefiles
OGC WKT
JML
GML
JPG/PNG/TIFF

I think this will make our API easier for plug-in programmers that want to create support for new/custom file formats, and it will make our core less complex.

The Sunburned Surveyor
Posted on 12:24 PM | Categories:

Friday, July 03, 2009

JSR-275 (javax.measure) Jar

Some of my buddies from Geotools have done work on the JSR that provides a standard way to work with measurements and measurement units. That is definitely something a surveyor (and most GIS junkies) would be interested in. I had a hard time locating a JAR of the JSR-275 implementation, but with some help from Simone at Geotools I found its location in a maven repository.

You can download it here:
http://download.java.net/maven/2/javax/measure/jsr-275/1.0.0/

The source code and javadocs are there as well.

Here is the home page for the JSR itself:
https://jsr-275.dev.java.net/

Here is a link to the PDF of the spec for the JSR:
https://jsr-275.dev.java.net/files/documents/4333/34956/jsr-275.pdf

The Sunburned Surveyor
Posted on 9:13 AM | Categories:

Tuesday, April 14, 2009

Your unit tests will lie to you.

I've become a very big proponent of unit tests. I don't practice strict test-driven development, in which you write the test and then the code that meets the test. However, I do write JUnit tests for almost all of my classes. (High-level OpenJUMP classes in the core are an exception to this. They can be very hard to write unit tests for.) I find unit tests are especially worth the pay-off when writing low-level libraries and parsers, which I seem to do a lot.

Did you know that your unit tests can lie to you?

It is the truth. Consider the following method of a Java class, which compares two make-believe CustomColor objects:

public boolean customColorsAreEqual(CustomColor argCompareTo)
{
// Method implementation goes here.
}

Now consider a unit test that tests this method. It creates two CustomColor objects that should be equal, and then uses the above method to compare them. If the boolean value returned by the method is true, the test passes, if the method is false, it fails. Our test method might look like this:

public void testCustomColorsAreEqual(CustomColor argCompareTo)
{
CustomColor myGreen1 = new CustomColor("My Green");
CustomColor myGreen2 = new CustomColor("My Green");

boolean areEquals = myGreen1.customColorsAreEqual(myGreen2);

if(areEquals == false)
{
fail("The method did not recognize two CustomColor objects that are equal.");
}
}

The programmer runs the unit test, which passes, and assumes the method is bug-free. But the unit test lied.

The programmer neglected to have his test repeat a comparision with two CustomColor objects the programmer knew to be unequal. He should modify his test to add this comparision as well. If he doesn't, a bug could result in the method under test indicating that two unequal CustomColor objects are equal.

Most tools I know of to examine JUnit test coverage deal with the program as a whole. The programmer also needs to think about the test coverage of indivdual methods under test. You should test the method for all likely scenarios, not just the first scenario that pops in your head. This often means running the method under test multiple times from the same test method, but each time with different arguments or program state.

I'll post some more comments on unit testing in Java later.

The Sunburned Surveyor
Posted on 8:43 PM | Categories:

Tuesday, April 07, 2009

Good Summary of Proprietary Versus Open Source

From Paul Ramsey's Blog comes a link to an arcticle entitles Open Source and Software Procurement.

Paul like one part of the article, but I like it for another part. The part I like explained the difference between the up-front license cost of software versus the implementation/customization cost.

Here is the part I liked:

One thing he said that suddenly made the bulb above my head light was to do with procurement. At Netsight, we pretty much gave up on responding to blind tenders we were sent many years ago due to the overhead of responding to them. Not just that but we always thought that the tenders generally asked 'the wrong questions' which made it very hard for us, as an Open Source implementation company, to really give a decent answer. Many of you, no doubt, will be familiar with this scenario as an example: The tender has a cost table you have to fill in the blanks in. You are not allowed to deviate from the format of the table (because the tenderer wants to try and compare apples with apples). So you fill the table in, but your numbers just don't quite fit in the boxes right. We've even had feedback from tenders we've lost saying things like 'Well your solution costs 0 for licensing and 50K for implementation; the solution we chose cost 40K in licensing and 30K in implementation, so must be the more appropriate solution for our needs'.

Now I can see their thinking here. Our solution cost 50K and 100% of the cost is in implementation/customisation so obviously wasn't the correct solution to start with. The one they chose cost 70K but only 42% of that was implementation/customisation. What they don't realise that (especially in the CMS market) any large software procurement is only going to give you a fraction of your requirements out of the box. That is because every business is different, and so no one piece of software can do everything you need. Open Source might give you slightly less out of the box than a commercial offering, but that is generally because Open Source software focusses on delivering a lowest common denominator out of the box, whereas commercial software has loads of bells and whistles (many of which you will never need), but more on that later. You are still going to need professional services to tailor the software to your specific business and its requirements. The problem is looking at ratio of costs and the perception of these ratios.


The Sunburned Surveyor
Posted on 2:09 PM | Categories:

172 Plug-Ins For OpenJUMP

I made a list of all the OpenJUMP plug-ins packages as part of the core. This doesn't count plug-ins distributed separately from the plug-in. I did this by examining the class names in the Javadoc for the program.

How many Plug-Ins did I find?

172

That is a lot of plug-ins. I was suprised at how much of OpenJUMP's functionality is acutally added via plug-in, which increases the flexibility of the program.

I also found a lot of plug-ins that seemed to duplicate functionality, or to offer "junk" functionality. I think there might be some house cleaning to do. Here are some examples of a plug-in names that made me raise my eyebrows:

AddWMSDemoBoxEasterEggPlugIn
MicroscopePlugIn

This list of plug-ins should make it easier for OpenJUMP programmers to thin down OpenJUMP's functionality, or to locate the plug-in that contributes behavior to OpenJUMP.

Here is the full list of plug-ins for OpenJUMP that I compiled:

AboutPlugIn
AbstractAddDatastoreLayerPlugIn
AbstractLoadDatasetPlugIn
AbstractLoadSaveDatasetPlugIn
AbstractPlugIn
AbstractSaveDatasetAsPlugIn
AbstractSaveProjectPlugIn
AddDatastoreLayerPlugIn
AddImageLayerPlugIn
AddNewCategoryPlugIn
AddNewFeaturesPlugIn
AddNewLayerPlugIn
AddSIDLayerPlugIn
AddWMSDemoBoxEasterEggPlugIn
AddWMSQueryPlugIn
AffineTransformationPlugIn
AffineTransformPlugIn
AttributeQueryPlugIn
BeanShellPlugIn
BeanToolsPlugIn
BoundaryMatchDataPlugIn
BufferPlugIn
CalculateAreasAndLengthsPlugIn
ChangeCoordinateSystemPlugIn
ChangeLayerableNamePlugIn
ChangeSRIDPlugIn
ChangeStylesPlugIn
ClearSelectionPlugIn
CloneWindowPlugIn
CombineSelectedFeaturesPlugIn
ConnectionManagerToolboxPlugIn
ConstrainedMoveVertexPlugIn
ConvexHullPlugIn
CopyImagePlugIn
CopySelectedItemsPlugIn
CopySelectedLayersPlugIn
CopySelectedLayersToWarpingVectorsPlugIn
CopyStylesPlugIn
CopyThisCoordinatePlugIn
CreateThiessenPolygonsPlugIn
CustomFillPatternExamplePlugIn
CutPolygonPlugIn
CutSelectedItemsPlugIn
CutSelectedLayersPlugIn
DeeChangeStylesPlugIn
DeleteAllFeaturesPlugIn
DeleteEmptyGeometriesPlugIn
DeleteSelectedItemsPlugIn
DiffGeometryPlugIn
DiffSegmentsPlugIn
DrawCircleWithGivenRadiusPlugIn
DrawConstrainedArcPlugIn
DrawConstrainedCirclePlugIn
CrawConstrainedLineStringPlugIn
DrawConstrainedPolygonPlugIn
EditablePlugIn
EditAttributeByFormulaPlugIn
EditingPlugIn
EditSelectedFeaturePlugIn
EditSelectedSidePlugIn
EditWMSQueryPlugIn
EnsureAllLayersHaveSRIDStylePlugIn
ExplodeSelectedFeaturesPlugIn
ExportImagePlugIn
ExtensionManagerPlugIn
ExtractSegmentsPlugIn
FeatureInfoPlugIn
FeatureStatisticsPlugIn
FirstTaskFramePlugIn
GenerateLogPlugIn
GeometryFunctionPlugIn
ImageLayerManagerPlugIn
InstallGridPlugIn
InstallKeyPanPlugIn
InstallReferencedImageFactoriesPlugIn
InstallRendererPlugIn
InstallScaleBarPlugIn
InstallShowScalePlugIn
InstallSkinsPlugIn
InstallStandardDataSourceQueryChoosersPlugIn
InstallStandardFeatureTextWritersPlugIn
InstallZoomBarPlugIn
JoinAttributesSpatiallyPlugIn
JoinTablePlugIn
JoinWithArcPlugIn
LayerableClipboardPlugIn
LayerStatisticsPlugIn
Layer2SLDPlugIn
LineNoderPlugIn
LoadDatasetPlugIn
MacroPlugIn
MainButtonPlugIn
MapToolTipPlugIn
MapToolTipsPlugIn
MeasureM_FPlugIn
MergeTwoSelectedPolygonsPlugIn
MicroscopePlugIn
MoveAlongAnglePlugIn
MoveLayerablePlugIn
NewTaskPlugIn
OpenProjectPlugIn
OptionsPlugIn
OutputWindowPlugIn
OverlayPlugIn
PasteItemsPlugIn
PasteLayersPlugIn
PasteStylesPlugIn
PersistentBlackboardPlugIn
PlanarGraphPlugIn
PolygonizerPlugIn
PrecisionReducerPlugIn
ProgressReportingPlugIn
ProjectionPlugIn
RandomArrowsPlugIn
RandomTrianglesPlugIn
RedoPlugIn
ReducePointsISAPlugIn
RemoveSelectedCategoriesPlugIn
RemoveSelectedLayersPlugIn
ReplaceValuePlugIn
ReplicateSelectedItemsPlugIn
RotatePlugIn
RotateSeletedItemPlugIn
RunDatastoreQueryPlugIn
SaveDatasetAsPlugIn
SaveDatasetsPlugIn
SaveImageAsPlugIn
SaveImageAsSVGPlugIn
SaveLegendPlugIn
SaveProjectAsPlugIn
SaveProjectPlugIn
ScaleBarPlugIn
ScaleSelectedItemsPlugIn
SelectAllLayerItemsPlugIn
SelectByTypePlugIn
SelectFeaturesInFencePlugIn
SelectItemsByCircleFromSelectedLayersPlugIn
SelecteItemsByFenceFromSelectedLayersPlugIn
SelectOneItemPlugIn
ShortcutKeysPlugIn
ShowFullPathPlugIn
ShowScalePlugIn
ShowTriangulationPlugIn
SimpleQueryPlugIn
SpatialJoinPlugIn
SpatialQueryPlugIn
StandardPirolPlugIn
TestColorThemingPlugIn
ThreadedBasePlugIn
ThreadedPlugIn
ToggleVisibilityPlugIn
ToolboxPlugIn
UndoPlugIn
UnionByAttributePlugIn
UnionPlugIn
ValidateSelectedLayersPlugIn
VerticesInFencePlugIn
ViewAttributesPlugIn
ViewSchemaPlugIn
WarpingPlugIn
WKTPlugIn
ZoomBarPlugIn
ZoomNextPlugIn
ZoomPreviousPlugIn
ZoomToClickPlugIn
ZoomToCoordinatePlugIn
ZoomToFencePlugIn
ZoomToFullExtentPlugIn
ZoomToLayerPlugIn
ZoomToScalePlugIn
ZoomToSelectedItemsPlugIn
ZoomToWMSPlugIn

The Sunburned Surveyor
Posted on 12:43 PM | Categories:

Blog Post On User Interface Design Tools

I liked this blog bost on the use of interface design tools. In my own experience I've found the tools are troublesome at best. It can be very hard to make sense of the code they produce, and the zillion inner classes they create for event handling really annoy me.

At any rate, I think the blog author makes a good argument:

http://java.sys-con.com/node/898537

The Sunburned Surveyor
Posted on 7:57 AM | Categories:

Wednesday, April 01, 2009

Tuesday, March 24, 2009

Historical Map Collection Under The Creative Commons

A post from the Geodata mailing list at the OSGeo revealed what could be an interesting source of historical GIS data. It is the David Rumsey Historical Map Collection. The collection is apparently released under one of the Creative Commons licenses. You can find the collection online here:

http://www.davidrumsey.com/

I didn't figure out a way to download high resolution copies of the maps, but I think there is a way to do it.

The Sunburned Surveyor
Posted on 9:10 AM | Categories:

Thursday, March 12, 2009

Spatial Data Accuracy and Georeferencing Standards

The Arizona Professional Land Surveyors Association has teamed with to produce a "Spatial Data Accuracy and Georeferencing Standards". I think this is a great example of land surveyors and GIS professionals working together for the betterment of both professions and larger society. You can download the standards here.

The Sunburned Surveyor

Posted on 1:27 PM | Categories:

Revoking the Name Change

I decided to revoke my recent name change. It didn't make sense to rename the blog, but not change the URL. I didn't want to change the URL, because that would break a bunch of links, and people might not find the blog at the new URL.

So the "OpenJUMP Blog" it will be.

The Sunburned Surveyor
Posted on 9:16 AM | Categories:

Monday, March 09, 2009

Name Change

I've changed the name of this blog from the "OpenJUMP Blog" to the "Sunburned Surveyor's GIS Blog". Stefan suggested this name change long ago, and I'm just now getting around to it.

I'll still be talking about OpenJUMP quite a bit, but since I talk about a lot of other programming and GIS related topics (and not just OpenJUMP) the name change seemed appropriate.

The Sunburned Surveyor
Posted on 1:51 PM | Categories:

WKT for curves?

The original spec for WKT (Well Known Text) geometries did not include (to my knowledge) support for 2D curveys, like circular arcs. I didn't realize that had changed, and the WKT spec now includes some limited support for 2D curves. You can see this paper for more information:

http://doesen0.informatik.uni-leipzig.de/proceedings/paper/68.pdf

This is cool, because I will need support for circular arcs in WKT if I am ever get around to working on a linear referencing (route stationing) application for flood control levees. I don't think will be seeing support for these curves in JTS anytime soon (because of the complexities that it introduces), but I wonder if we might find room for it in Geotools?

The Sunburned Surveyor
Posted on 1:42 PM | Categories:

Summary of Important "Open GIS Data" Court Decision

The California First Amendment Coalition has a summary of a very important court decision in California. The decision involves requiring Santa Clara County to give the public reasonable access to the parcel GIS data it maintains. This is a great court decision and I hope we see more decisions like it.

Here is the link to the summary:

http://www.cfac.org/content/index.php/cfac-news/legal_development/

The Sunburned Surveyor
Posted on 11:52 AM | Categories:

Wednesday, March 04, 2009

Rand Org Report on Assessing Homeland Security Implications of Publicly Available Geospatial Information

From the geowanking mailing list and the Curious Judith blog comes this link to an interseting report from the RAND Corporation:

http://www.rand.org/pubs/monographs/2004/RAND_MG142.pdf

The report is entitled "Assessing the Homeland Security Implications of Publicly Available Geospatial Information".

Here is a summary of the report from the preface:

This report assesses the homeland security implications of publicly
available geospatial data and information. Specifically, it seeks to
frame the analytical issues concerning whether and how this type of
data and information that is available from U.S. government sources
can be exploited by terrorists and other adversaries seeking to attack
U.S. critical infrastructure and other key homeland locations. We
give particular attention to surveying and characterizing these federal
data and information within the broader context of diverse publicand
private-sector producers of potentially relevant geospatial information.

This is a big report...but I think it touches on a critical issue for GIS enthusiasts in all parts of the world and I hope to read it at some point.

The Sunburned Surveyor
Posted on 4:40 PM | Categories:

Tuesday, March 03, 2009

Citing OpenJUMP In Publications

Stefan (always the academic) posted a proposed citation for OpenJUMP that can be used in articles and publications. I wanted to share it with everyone.

The JUMP Pilot Project (2008): OpenJUMP GIS - The free and open sourceJava-based desktop GIS. Available from http://www.openjump.org/. (AccessedJanuary 2009)

Thanks Stefan.

The Sunburned Surveyor
Posted on 8:31 AM | Categories:

Monday, March 02, 2009

Searching For Place Names With The USGS

The USGS maintains a web page that can be used to search a database of place names, both for the United States and places around the world. The web page is one part of the Geographic Name Information System (GNIS). The USGS also maintains a database for the GNIS. You can download the data from this database for all 50 of the states in the United States as a single zip file, or for just a single state.

The home page for the GNIS is here.

The web page that lets you search for place names is here.

The wikipedia page for the GNIS is here.

You can even use the site to propose a new place name, which is pretty cool.

I wonder if other countries maintain a database of place names like the GNIS.

The Sunburned Surveyor
Posted on 5:46 PM | Categories:

Watermarking Vector GIS Data

A recent posting on the gislist about watermarking vector GIS data led to the link for this interseting article on the topic:

http://www.directionsmag.com/article.php?article_id=195&trv=1

I just skimmed the article, but I plan on reading it in more detail. This is an interesting topic. A common method of watermarking vector GIS data is adding "fake" features, like road segments that don't really exist. But I have often thought that there has got to be better ways of placing a watermark. The tricky part is embedding a watermark that can resist translation, rotation, and scaling. (Or even reprojection to a new coordinate system.) This rules out using any coordinate geometry to record the watermark.

Perhaps using the topology of features that remain intact despite translation, rotation, and scaling would be a valid way to watermark vector GIS data without "polluting" the dataset with fake features.

I'll have to chew on this some more. Might be cool to add a vector GIS data watermarking plug-in to OpenJUMP.

The Sunburned Surveyor
Posted on 9:23 AM | Categories:

Thursday, February 26, 2009

Tuesday, February 24, 2009

Using XML To Describe GIS Data Models

ESRI has been the main creator of widely distributed GIS data models. See this page for examples.

An ESRI data model is usually distributed as some type of poster, but other documentation is sometimes included. The data model is usually tuned for ESRI GIS software, although it is possible to extract some of the basic database design from the model.

I'd like to work on some of my own GIS data models some day. A data model for flood control infrastrucutre, survey control, the Public Land Survey System, and for Open Street Map are some that come to mind. I'm not very good at producing posters, but I was thinking there might be a better way to share data models.

What if we had an XML file format that could be used to document and share a data model? That would be a lot more useful than a poster. You could write a little desktop application that could be used to browse and modify the different parts of a model. You could cut and paste parts of a model from on model to another. You could even write a tool that autmatically creates a graphical representation of the model in something like SVG, or make a tool that generates a relational database from the information in the model.

I took a crude stab at an XML file format that describes a data model for Open Street Map:

http://www.redefinedhorizons.com/shared_files/samples/

This is something I would like to explore more in the future. Of course, the XML file is kind of ugly. Ir would need to be accompanied by a cool JAva API and desktop app to be really useful. :]

For now, I need to finish my Feature Photo Manager.

The Sunburned Surveyor
Posted on 12:47 PM | Categories:

Java for desktop applications?

It seems the whole world is developing web applications. Sometimes I wonder if I'm the only guy left developing cool applications (like OpenJUMP) for the desktop. Guess I'm not the only one. Here is a short blog post about growing revenues for Java desktop development:

http://java.dzone.com/news/growing-revenue-desktop-java

The Sunburned Surveyor
Posted on 11:51 AM | Categories:

Thursday, January 15, 2009

Google Programmer Explains Why Static Methods Are Bad For Java Unit Testing

This article made me question my frequent use of static utility methods:

http://misko.hevery.com/2008/12/15/static-methods-are-death-to-testability/

Here are a couple of the excerpts I liked from the article:

The basic issue with static methods is they are procedural code. I have no idea how to unit-test procedural code. Unit-testing assumes that I can instantiate a piece of my application in isolation. During the instantiation I wire the dependencies with mocks/friendlies which replace the real dependencies. With procedural programing there is nothing to “wire” since there are no objects, the code and data are separate.

Lets do a mental exercise. Suppose your application has nothing but static methods. (Yes, code like that is possible to write, it is called procedural programming.) Now imagine the call graph of that application. If you try to execute a leaf method, you will have no issue setting up its state, and asserting all of the corner cases. The reason is that a leaf method makes no further calls. As you move further away from the leaves and closer to the root main() method it will be harder and harder to set up the state in your test and harder to assert things. Many things will become impossible to assert. Your tests will get progressively larger. Once you reach the main() method you no longer have a unit-test (as your unit is the whole application) you now have a scenario test. Imagine that the application you are trying to test is a word processor. There is not much you can assert from the main method.

The Sunburned Surveyor
Posted on 1:44 PM | Categories:

Best "General" Java Programming Book

I wanted to task some time to mention what I feel is the best "general" Java programming book I've ever read. I'm not talking about a book on a specific Java topic, like Swing or threads, but a book that gives a good overview of the language.

I think the best book of this type is O'Reilly's Learning Java. (I've got the third edition.) The book is written by Patrick Niemeyer and Jonathan Knudsen. Patrick is also the creator of Beanshell, the really cool scripting language that is best buddies with Java (and included in OpenJUMP). The book encourages the use of Beanshell to learn Java.

I wish I would have had this book when I first started to learn Java. I find it still gives me better coverage on some topics than other programming books. For example, Learning Java has a topic on creating classes with generics that is much better than any coverage of the topic in my book that is dedicated to Java generics.

If you are a serious Java programmer, this book should be in your library.

The Sunburned Surveyor
Posted on 12:48 PM | Categories: