JD-Eclipse Realign - Usability

Background

In my previous post I talked about the problems caused by the original JD-Eclipse Realign approach of using a special editor to open decompiled source for a class. The key problem was that opening a class file is something which you only expect to involve the selected class but the resulting decompiled source is attached to the entire jar. Worse, the decompiled source attachment was something which could then be used by the standard Class File Viewer resulting in some weird behaviour.

I previously fixed this by switching to a model where decompiled source was enabled/disabled through a context menu that was intended to make the similarity with source attachments much more explicit.

This solution was technically sound but the usability was frankly awful. Any time you wanted to enable/disable decompiled source you were forced to use the Package/Project Explorer context menu which has loads of entries and is therefore always quite unpleasant to use.

Toolbar Buttons (V1.1.0)

The first step to improve this situation was to take the ability to attach/detach decompiled source and move it into the top level toolbar.

This pair of buttons expose two commands - Attach Decompiled Source, Detach Decompiled Source. The buttons are dynamically displayed in the toolbar when either a Class File Viewer editor is active or when at least one class, package in a jar or entire jar is selected in the Package/Project Explorer.

Pressing one of these buttons applies the action to all relevent selected objects in the active "Part". This means that if an editor is active the action applies to the class in that editor and if the Package/Project Explorer view is active the action applies to all selected classes, packages in jars or entire jars.

Decompiled By Default (V1.1.1)

Having added the toolbar buttons I realised that it was still not easy enough to jump straight into decompiled source. If you open a class without source you have to deal with a tiny moment of disappointment/frustration before reaching for the toolbar button to enabled decompiled source.

The solution to this problem is to restore the JD-Eclipse Realign editor (JDE-R Editor) with subtely different functionality.

The original decompiled source editor was styled with a different icon and always attached decompiled source when opening a class file. In the restored JDE-R Editor the behaviour has been changed so that decompiled source is only attached when there isn't already a normal source attachment. The icon of the JDE-R Editor has also been changed to be the same as the standard Class File Viewer.

The overall intention is that the JDE-R Editor should behave and look exactly the same as the built in Class File Viewer except that opening a class in a jar without a source attachment will attach decompiled source.

Once you have a class file opened with either normal or decompiled source in either the standard Clas File Viewer or JDE-R Editor you can use the toolbar buttons to quickly switch between normal and decompiled source.

File Associations

With the JDE-R Editor restored in V1.1.1 I have also restored the code which automatically attempts to set up the "correct" file associations on startup. These are as follows:

Pre-Juno:

• .class file - Class File Viewer [Decompiled]

Juno:

• .class file - Class File Viewer
• .class file without source - Class File Viewer [Decompiled]

"Class File Viewer [Decompiled]" is the name I have given to the JD-Eclipse Realign editor.

Getting Started

To try out JD-Eclipse Realign you should use the update site listed on the JD-Eclipse Realign project page.

Eclipse Tips - Navigation

Eclipse is a really powerful tool but it's really easy to miss some of the most powerful features. I was therefore interested to come across the following link on /r/java: My Top 10 Tips on how to be more productive with the Eclipse IDE.

I liked these tips but I think there are some other important ones which are worth pointing out.

Navigation Tips

Eclipse has some fantastic functionality to enable fast navigation to find whatever you are looking for.

1) Open Type (Ctrl-Shift-T)

The Open Type window is the right way to open Java classes. If you find yourself looking for classes by manually looking around in the Package Explorer you are almost certainly wasting your own time unless you have no idea what the class you are looking for is called.

Open Type supports wildcards (e.g. H*Map matches HashMap) and camelcase (e.g. LHM matches LinkedHashMap).

2) Open Resource (Ctrl-Shift-R)

The Open Resource window is the right way to open any file which isn't a Java class e.g. a properties file or an xml file etc.

Open Resources works very similarly to Open Types with support for wildcards and camelcase. However there is also support for specifying Folder Prefixes (e.g. */ui/*.html matches all html files within a folder called ui).

3) Quick Open (Ctrl-O within a Java file)

Quick Open is the right way to jump to particular elements within a Java file. If you use the Outline View you are wasting your time as you are forced to visually search for what you are looking for.

Quick Open supports wildcards and camel case (as of Eclipse Juno).

Other Tips

There are three more tips which I think are particularly worth noting.

4) Quick Access (Ctrl-3)

Eclipse functionality is exposed through a large number of Views, the menus include a huge number of Commands and there are lots of Preferences which control all of this. Finding all of these elements is extremely tricky.

Quick Access supports wildcards and can be used to load any of these elements. This is by far the fastest way to open things like Views.

5) Local History

Have you ever made a series of changes to a file and then wished that you could go back to a previous version? It's easy if you are using a version control system and have checked in a previous version but what if you didn't?

Eclipse has got a feature to help you out. The Local History feature keeps track of a limited number of previous versions of every file which you save through Eclipse. To revert to one of these previous versions you just have to use the correct menu. Right click the file within Package Explorer and select Replace/Compare With -> Local History...

6) Eclipse Help

It is absolutely worth noting that the Eclipse Help is really excellent. For example, try looking at the “Open Type” or “Open Resource” articles for full details of the pattern matching which you can do.

Java Libraries Which Use Annotations

I recently added a command line user interface to my Java Thread Dump Analyser tool (Issue #2). As part of this work I tried out a new library that leveraged Java annotations in a nice way. This was the third library which I have encountered that uses annotations so I thought it might be useful to write a post about annotations and point out which Java libraries are putting them to good use.

Background

What are annotations? Annotations are user defined markers which can be used in Java source code to encode any kind of metadata. Here is a contrived example:

@RequestForEnhancement(
    id       = 2868724,
    synopsis = "Enable time-travel",
    date     = "4/1/3007"
)
public static void travelThroughTime(Date destination) { ... }

More information is available in the excellent Java documentation. It is interesting to note that Annotations get a better constructor syntax than the rest of the Java language including named parameters and default values.

(http://docs.oracle.com/javase/6/docs/technotes/guides/language/annotations.html)

JDO

My first contact with a library which makes extensive use of annotations was the use of JDO within Google App Engine. Once again I'll dive straight into an example:

@PersistenceCapable
public class Employee {
    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Key key;

    @Persistent
    private String name;

} 

What I like about this code is that the annotations provide a very concise way of marking which fields need to be stored and how they should be stored. The actual mechanics of storing and retrieving the data are then entirely generic and driven by the annotations which are used.

(https://developers.google.com/appengine/docs/java/datastore/jdo/dataclasses)

GSON

GSON is an excellent library for handling JSON serialization. By default it doesn't actually use Java annotations but the ExclusionStrategy API allows the user to define arbitrary logic for picking which fields to serialize. The docs include an example of how to use this API to select fields based on an annotation:

(https://sites.google.com/site/gson/gson-user-guide#TOC-User-Defined-Exclusion-Strategies)

JCommander

This command line argument parsing library uses annotations to simplify command line argument parsing. Here is a simple example:

public class JCommanderExample {
  @Parameter(names = { "-log", "-verbose" }, description = "Level of verbosity")
  private Integer verbose = 1;
 
  @Parameter(names = "-groups", description = "Comma-separated list of group names to be run")
  private String groups;
}

The online docs provide plenty of examples to get you started:

(http://jcommander.org/)

Eclipse 4.2 (Juno) - Bugs

For the last couple of releases the Eclipse project has been working on two streams of development. 3.x uses the legacy workbench and has been the primary release train in past years. However, since Eclipse Helios (3.8) there has also been a second 4.x release train working on a new workbench framework. This has been delivering releases at the same time as 3.x (e.g. Helios (3.6/4.0), Indigo (3.7/4.1)).

This year Eclipse Juno (3.8/4.2) was released and for the first time the 4.x version was chosen to be the primary release and 3.8 is in theory the last 3.x release. Unfortunately my testing has quickly revealed some glaring bugs - basic elements of interacting with the workbench don't quite work correctly. I have raised five bugs and found one existing bug covering the key issues which I hit:

• 362420 - Make "Quick access" optional and hidden by default
• 386804 - The target area to DND Views and Editors into existing stacks is too small
• 386803 - Quick Access mishandles Home and End
• 386802 - DND of Views and Editors not working properly
• 386806 - New Eclipse Icon not used by Eclipse 4.2 on Windows 7
• 386817 - Customize Perspective Tool Bar Visibility appears broken in 4.2

Eclipse 4.2 also includes a new default theme (new and noteworthy, bug covering implementation) which I really dislike. A classic theme is included which makes Eclipse look much more like the previous 3.x releases but tediously you have to restart to fully apply a theme change despite Eclipse updating most elements dynamically and not warning that a restart is needed to fully apply the change (362522).

Eclipse 3.8 workbench:

 

Eclipse 4.2 workbench:

As a result of all of these issues I will be sticking to Eclipse 3.8 this year but I will keep an eye on future 4.x releases. The Eclipse 4.2 downloads are easy to find but there is only one 3.8 package available:

• 3.8 - http://download.eclipse.org/eclipse/downloads/drops/R-3.8-201206081200/
• 4.2 - http://www.eclipse.org/downloads/

JD-Eclipse Realign - Source Lifetime

In my initial post about my fork of JD-Eclipse Realign I mentioned the difficulties I encountered in trying to allow a user to switch quickly between decompiled source and attached source. Since that post I have realised that my original fix was insufficient.

Opening a class file in the JD-Eclipse Realign editor involves the following steps:

1. Check whether a source mapper is already installed for the container jar.
2. If one already exists use that to get the source for this class file.
3. If one does not exist create a new one, register it with the container jar and then use it to get the source for this class file.

The JD-Eclipse editor subclasses the built in Class File Viewer (which already implements steps 1. and 2.) and provides an implementation of 3. that always returns a source mapper that generates decompiled source code. This has the unfortunate side effect that once you have loaded a single class in the JD-Eclipse editor (which registers a decompiling source mapper with the jar) you can open the standard Class File Viewer and get decompiled source for any other class in the same jar.

My original JD-Eclipse Realign editor changes fixed this by de-registering the source mapper when you close the editor. This still meant that while a single JD-Eclipse Realign editor was open you could still get decompiled source in the standard Class File Viewer. I tried to address this once and for all by removing the source mapper as soon as the JD-Eclipse Realign editor was opened. Unfortunately this had unforeseen consequences. It turns out that navigating to a particular element (e.g. from the Outline view) relies on a source mapper being available.

In light of all of this I concluded that having a special JD-Eclipse editor was actually the wrong approach. A better solution is to treat enabling class decompilation the same way as attaching real source code. To enable this I reworked the Open Class With menu which I had already added to work slightly differently.

• When there is no source attached and decompilation is disabled:

• When there is no source attached and decompilation is enabled:

• When there is source attached:

In this last case the menu doesn't allow decompilation to be directly enabled for dull implementation reasons. In all cases when the source attachment/decompilation state changes all open class file viewers are updated.

With the addition of these new menu options I have also removed the JD-Eclipse Realign editor. My code is packaged as a fragment that extends the base JD Eclipse plugin so I can't remove the vanilla JD Eclipse editor. However, on every startup my fragment will check whether the JD Eclipse editor has been associated with class files (or class files without source) and revert any such associations to use the standard Class File Viewer.

An updated version of JD Eclipse Realign is available from my update site now: http://mchr3k-eclipse.appspot.com/

If you have any issues please do raise it here.

Java Packaging

jar (http://ant.apache.org/manual/Tasks/jar.html)

The basic mechanism for packaging Java code is within a jar file. This can be achieved in ant using the jar task. The jar task is more powerful than it first appears as you can use a nested zipfileset to package multiple source jars into a single output jar.

However, the danger with this mechanism is that you may end up packaging a version of a class which conflicts with the version required by another dependency.

jarjar (http://code.google.com/p/jarjar/)

jarjar extends the default jar ant task with a new "rule" child element which specifies mappings of package names. This rewrites all the classes within the output jar to move the classes within the specified packages and update all the references to these classes.

For example org.apache.commons.io could be moved to com.product.internal.commons.io. This is one way to avoid the classic jar hell.

jarinjarloader (no site)

The jarinjar classloader is an internal part of the Eclipse JDT project. However, you can get the classes by opening org.eclipse.jdt.ui_*version_number*.jar with an archiver and extracting the file jar-in-jar-loader.zip.

With the aid of these classes it is possible to package a jar within another jar and still load the contained classes and resources. For an example of what this looks like in practice you can look at getSWTClassloader() in SWTLoader.java.

Another project which claims to do the same is One-Jar (http://one-jar.sourceforge.net/). However, I haven't used this myself.

swtjar (http://mchr3k.github.com/swtjar/)

The standard approach to packaging SWT applications is to create a bundle per platform (32/64 bit and Win/Linux/OSX). This is because SWT requires the application to load the correct jar for the platform being used.

However, with the help of the jarinjar loader classes it is possible to include multiple SWT jars within an application jar and then use standard platform detection code to pick the correct one to programmatically add to the class path. My swtjar project packages this solution as an ant task.

jarbundler (http://informagen.com/JarBundler/)

OSX has its own package format to allow an application to provide an icon and control startup options. This is particularly useful as SWT on OSX requires an extra command line argument to work. The jarbundler project is an ant task that helps you to build OSX application bundles.

For details about using this tool you should refer to this page: http://mchr3k.github.com/swtjar/osxapp.html

Posted with Blogsy

Reverse Literate Programming - Code Trails

Background

Literate Programming is a technique which was introduced by Donald Knuth in 1984:

I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: "Literate Programming."

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reinforce each other.

The key idea here is that the programmer is tasked with writing documentation that compiles into code rather than writing the code and attempting to document it separately.

At the other end of the spectrum is the imaginatively named Reverse Literate Programming. This tasks the programmer with writing code that is directly compiled into documentation. This is likely to sound quite familiar as systems like Javadoc enable API documentation to be produced by compiling program source code. However, API documentation alone is not enough as it is focused on the minutia of source code rather than any high level details. Package level documentation can be added but this isn't very accessible as it is stored in a separate file.

Code Trails

I think IDEs need to do a better job of making it easier to provide simple high level source code documentation.

The use of markers within code comments allow key locations within code to be highlighted. The IDE collects together these markers into Trails that are displayed together. These Trails provide simple high level documentation and every marker can be double clicked to quickly examine the code at the location of the marker.

What does this look like in Eclipse? Here is some example code:

And the corresponding Trail view:

Installation

If you are interested in trying out the Code Trails plugin you can install it from my update site:

http://mchr3k-eclipse.appspot.com/

When you first open the Trails view you will get the following warning about needing to run a full workspace rebuild.