Getting Data Into OpenJUMP

Over the last year or two I’ve written a couple of plug-ins designed to get data into OpenJUMP. The first of these plug-in imported GPX files, and the second plug-in imported survey point data from a delimited text file. I was allowing the user to access these plug-ins through a menu item under the “SurveyOS” top-level menu, but I recently decided it would be good if they could be located under the “File” top-level menu.

The problem with this is OpenJUMP automatically put the menu items for my plug-ins at the bottom of the File menu, which looks a little awkward.

So I decided to see how difficult it would be to allow access to the plug-ins using the built-in system for opening files to get data into OpenJUMP.

When OpenJUMP was originally written there was only a single menu item that could be used to import files. This menu item opened a file selection dialog, and the user selected the type of file from a list box at the bottom of the dialog box.

A couple years ago, we ripped out this rather simple system and replaced it with another one. I didn’t realize how much we complicated things with this replacement until I started looking at the code today. The current system uses a wizard framework to open different files in OpenJUMP. It implements a system to support “import options” or parameters supplied by the user. It seems to automatically build dialogs that allow the user to enter these parameters based on information from the plug-in that actually imports the data.

Here are the things I don’t like about the current way we open files in OpenJUMP:

(1) A large majority of the files we OpenJUMP don’t require any parameters or options. We just want to suck in the contents of the file. ESRI Shapefiles are an example of this type of file. I’ll bet the majority of the files opened with OpenJUMP are ESRI shapefiles.
(2) The current system limits the steps and validation of parameters that can be used to open a file. I can’t have my plug-in walk the user through a complicated (and custom) wizard, and I can’t really validate the parameters as they are collected from the user. So we are only serving a limited slice of the plug-ins that need to collect parameters from the user before we import data from a file. Only very simple parameters can be collected in our current system.
(3) The plug-in programmer has little or no control over the GUI that is automatically constructed to collect the parameters. In my experience automated GUIs are ugly and hard to work with. They are appropriate for some situations, but I don’t know if this is one of them.
(4) Our current system doesn’t seem to support different plug-ins that import data from files with the same file extension. For example: You might use my recent plug-in to import survey point data from a delimited text files, but there might be several other plug-ins that import entirely different data from delimited text files.
(5) The current system is more complicated than it needs to be. We’ve got at least 24 classes in three (3) different packages that are needed to open files. Should it be that hard?

There are some things I like about the system we currently have for opening files. It keeps track of recent files, which can be handy. But I bet we can do the same thing with something like the old system, and it won’t be nearly as complicated. The new system also allows plug-in programmers to get a GUI for collection parameters needed to open a file without too much work, but I bet if you are writing a plug-in for data import you can handle whipping up a simple GUI.

This is what I will work on implementing in BizzJUMP, and what I would like to see implemented in OpenJUMP:

Move back to a simpler system for importing data from files. This would be something similar to the old system. I’d like to see 2 menu items for each supported file format. One menu item for each file format will be used to import data, and will be found under File>Open File. One menu item for each file format will be used to export data, and will be found under File>Export File (or save file). Each plug-in programmer will place his own menu items for file import and export, just like other plug-ins place their own menu items.

I will create plug-ins for the standard file formats that OpenJUMP already supports:
ESRI Shapefiles
OGC WKT
JML
GML
JPG/PNG/TIFF

I think this will make our API easier for plug-in programmers that want to create support for new/custom file formats, and it will make our core less complex.

The Sunburned Surveyor

Update on My OpenJUMP Work

Its been a while since I’ve posted anything meaningful. I want to provide a quick update on what I’ve been working on for OpenJUMP.

I’ve got a plug-in completed that allows the OpenJUMP user to import point data that is stored in a delimited text file. The point data format the plug-in currently supports is point number, northing, easting, elevation and description. (It can be modified to support additional formats.) The plug-in is built to work with point data form land surveyors (although it will work with data from any source in the specified format) so I set it up to separate out a feature code and linework code from the description. This means, in the future, additional tools could be created that automatically draw linework from layers created by this plug-in. You should also be able to separate the imported points into different layers based on each point’s feature code.

I ran into some trouble finishing the plug-in. The trouble is related to the use of threads while performing the import. I am currently looking into this. In the meantime, I may release the plug-in without the use of the threaded code. This means I won’t be able to show the user a progress dialog during the import, but we can probably live with that until I get the threads issue resolved.

I’ve also finished a web page on my personal web site that talks about OpenJUMP. It also discusses some of the future challenges to OpenJUMP development. You can read the web page here.

Christopher, our Google Summer of Code student, is wrapping up his summer project. I hope he will have some code nicely packaged that I can use to access an H2 database from OpenJUMP. I hope to write a plug-in that will support non-spaital entities and temporal events in OpenJUMP using an H2 database. That is the very next plug-in I will begin work on.

You can view Christopher’s code here:
http://jump-pilot.svn.sourceforge.net/viewvc/jump-pilot/summer_of_code/H2-OJ/

I’m also working on a distribution of BizzJUMP. It will hopefully include a couple of my recent plug-ins and the integration of the InfoNode docking windows framework.


The Sunburned Surveyor

I’ve decided to narrow the focus of my OpenJUMP blog. (We’ll see how successful this effort is.) To assist in this narrowing of focus I have created a separate blog that will deal with general programming and Java programming topics that do not relate directly to OpenJUMP. You can read it here.

I have also created a separate blog to deal with general GIS topics not related to OpenJUMP.

There will certainly be some cases with a topic could apply to more than one of my blogs. I’ll do the best I can to choose the most appropriate blog for a topic with consistency. I hope this will make things easier for the readers of my blog.

From this point forward, I would like the OpenJUMP blog to deal topics that relate directly the programming and promotion of OpenJUMP.

The Sunburned Surveyor

JSR-275 (javax.measure) Jar

Some of my buddies from Geotools have done work on the JSR that provides a standard way to work with measurements and measurement units. That is definitely something a surveyor (and most GIS junkies) would be interested in. I had a hard time locating a JAR of the JSR-275 implementation, but with some help from Simone at Geotools I found its location in a maven repository.

You can download it here:
http://download.java.net/maven/2/javax/measure/jsr-275/1.0.0/

The source code and javadocs are there as well.

Here is the home page for the JSR itself:
https://jsr-275.dev.java.net/

Here is a link to the PDF of the spec for the JSR:
https://jsr-275.dev.java.net/files/documents/4333/34956/jsr-275.pdf

The Sunburned Surveyor

Summary of Oracle's Buyout of Sun

Directions Magazine has an article with a good summary of Oracle's buyout of Sun Microsystems. It makes you wonder what will happen with technology like Java and MySQL.

Here is a link to the article:

http://www.directionsmag.com/article.php?article_id=3131

The Sunburned Surveyor

Your unit tests will lie to you.

I've become a very big proponent of unit tests. I don't practice strict test-driven development, in which you write the test and then the code that meets the test. However, I do write JUnit tests for almost all of my classes. (High-level OpenJUMP classes in the core are an exception to this. They can be very hard to write unit tests for.) I find unit tests are especially worth the pay-off when writing low-level libraries and parsers, which I seem to do a lot.

Did you know that your unit tests can lie to you?

It is the truth. Consider the following method of a Java class, which compares two make-believe CustomColor objects:

public boolean customColorsAreEqual(CustomColor argCompareTo)
{
// Method implementation goes here.
}

Now consider a unit test that tests this method. It creates two CustomColor objects that should be equal, and then uses the above method to compare them. If the boolean value returned by the method is true, the test passes, if the method is false, it fails. Our test method might look like this:

public void testCustomColorsAreEqual(CustomColor argCompareTo)
{
CustomColor myGreen1 = new CustomColor("My Green");
CustomColor myGreen2 = new CustomColor("My Green");

boolean areEquals = myGreen1.customColorsAreEqual(myGreen2);

if(areEquals == false)
{
fail("The method did not recognize two CustomColor objects that are equal.");
}
}

The programmer runs the unit test, which passes, and assumes the method is bug-free. But the unit test lied.

The programmer neglected to have his test repeat a comparision with two CustomColor objects the programmer knew to be unequal. He should modify his test to add this comparision as well. If he doesn't, a bug could result in the method under test indicating that two unequal CustomColor objects are equal.

Most tools I know of to examine JUnit test coverage deal with the program as a whole. The programmer also needs to think about the test coverage of indivdual methods under test. You should test the method for all likely scenarios, not just the first scenario that pops in your head. This often means running the method under test multiple times from the same test method, but each time with different arguments or program state.

I'll post some more comments on unit testing in Java later.

The Sunburned Surveyor

Good Summary of Proprietary Versus Open Source

From Paul Ramsey's Blog comes a link to an arcticle entitles Open Source and Software Procurement.

Paul like one part of the article, but I like it for another part. The part I like explained the difference between the up-front license cost of software versus the implementation/customization cost.

Here is the part I liked:

One thing he said that suddenly made the bulb above my head light was to do with procurement. At Netsight, we pretty much gave up on responding to blind tenders we were sent many years ago due to the overhead of responding to them. Not just that but we always thought that the tenders generally asked 'the wrong questions' which made it very hard for us, as an Open Source implementation company, to really give a decent answer. Many of you, no doubt, will be familiar with this scenario as an example: The tender has a cost table you have to fill in the blanks in. You are not allowed to deviate from the format of the table (because the tenderer wants to try and compare apples with apples). So you fill the table in, but your numbers just don't quite fit in the boxes right. We've even had feedback from tenders we've lost saying things like 'Well your solution costs 0 for licensing and 50K for implementation; the solution we chose cost 40K in licensing and 30K in implementation, so must be the more appropriate solution for our needs'.

Now I can see their thinking here. Our solution cost 50K and 100% of the cost is in implementation/customisation so obviously wasn't the correct solution to start with. The one they chose cost 70K but only 42% of that was implementation/customisation. What they don't realise that (especially in the CMS market) any large software procurement is only going to give you a fraction of your requirements out of the box. That is because every business is different, and so no one piece of software can do everything you need. Open Source might give you slightly less out of the box than a commercial offering, but that is generally because Open Source software focusses on delivering a lowest common denominator out of the box, whereas commercial software has loads of bells and whistles (many of which you will never need), but more on that later. You are still going to need professional services to tailor the software to your specific business and its requirements. The problem is looking at ratio of costs and the perception of these ratios.

The Sunburned Surveyor