tag:blogger.com,1999:blog-40616582888191077062024-03-05T04:48:15.247+00:00Mchr3k - CodingA blog about my open source projects and other coding subjects.mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comBlogger22125tag:blogger.com,1999:blog-4061658288819107706.post-15486898826110380342013-10-01T00:41:00.001+01:002013-10-01T00:41:10.928+01:00Picasa Web UpdateGood news! I can now download my photos from Picasa Web which were <a href="http://mchr3k-coding.blogspot.com/2013/09/why-you-shouldnt-trust-picasa-to-store.html">previously being truncated</a>!<br />
<br />
I am a little concerned that I seem to be getting back some smaller files than I expected. In one case I was able to download a 2.6MB file using Web Albums on my iPad but only get a 1.46MB file from Picasa Web. However, I don't actually mind much because I can't see a visual difference and I'm glad to at least have a file which contains the whole image.<br />
<br />
So now I can go back to using Picasa Web right? In a previous post I mentioned that I was sorting out my pictures locally and then intending to re-upload everything. I heard that it was possible to store photos in Google Drive and have
them show up in Google+ Photos. This worked great
to view the photos in Google+ but they didn't show up in
Web Albums on iOS.<br />
<br />
<br />
So I tried uploading an album using Picasa 3 and this seems to work fine :) Now I have to decide whether to delete everything from Picasa Web and re-upload everything or just try and identify the albums which I have actually changed.mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-62723722390892944512013-09-21T23:15:00.001+01:002013-09-21T23:25:08.084+01:00Storing Photos In The Cloud And OfflineThese days it is easier than ever to take lots of photos. The tricky part is how to sensibly manage all of these photos. I have an existing collection of ~20GB of photos and I take new photos on my iPhone.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHTstrOQiE2pEVXsLD7Yx8nWYczEPmZBIJE23mshp_fQ-ms9WLu8MpEBrDg6yN529oj81KHTlUtCaRNu-fpIP-l_hpPGKpYL0idSvjh9ahxVUNnEymHygtObnhmR39hLGyVVbEuLs2-Gk/s1600/IMAGE_768668D2-DABD-4225-B546-66068A6004F8.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="239" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiHTstrOQiE2pEVXsLD7Yx8nWYczEPmZBIJE23mshp_fQ-ms9WLu8MpEBrDg6yN529oj81KHTlUtCaRNu-fpIP-l_hpPGKpYL0idSvjh9ahxVUNnEymHygtObnhmR39hLGyVVbEuLs2-Gk/s320/IMAGE_768668D2-DABD-4225-B546-66068A6004F8.JPG" width="320" /></a></div>
<br />
I want to be able to:<br />
<ul>
<li>Manually organize my photos into albums. </li>
<li>Access all my photos from my iOS devices (iPhone/iPad) and on the web - and the cloud storage shouldn't cost too much!</li>
<li>Store selected albums offline on my devices for fast access without waiting for each picture to download.</li>
<li>Easily sync photos/albums from the cloud down to my PC for the purposes of local backup.</li>
</ul>
Up until recently I was managing this by using <a href="http://picasaweb.google.com/">Picasa Web</a> as my cloud store and the excellent <a href="http://pixiteapps.com/">Pixite</a> app "<a href="http://webalbumsapp.com/">Web Albums</a>" on my iPhone/iPad to handle uploads and local album caching. This worked great up until recently when a <a href="http://mchr3k-coding.blogspot.com/2013/09/why-you-shouldnt-trust-picasa-to-store.html">problem</a> with getting my photos back out of Picasa Web caused me lose my trust in the service. This also made me realise that my current approach didn't include an easy way to sync all of my photos to my PC to protect me from cloud service issues.<br />
<br />
<span style="font-size: large;">Looking At The Alternatives</span><br />
I have spent some time browsing through existing reviews of the alternatives and so far I haven't managed to find anything which quite fits my requirements. <br />
<br />
<span style="font-size: large;">Offline Caching</span><br />
The easiest way to filter out much of the competition is to look at the support for offline caching within their iOS apps.<br />
<ul>
<li>Pixite - developers of similar iOS apps for several cloud services which all support offline caching:</li>
<ul>
<li>Picasa Web</li>
<li>Dropbox</li>
<li>Flickr</li>
</ul>
<li>Everpix - automatically syncs some photos locally. No control over exactly which photos.</li>
</ul>
I also tried the following apps which didn't meet my requirements:<br />
<ul>
<li>Google Cloud Drive - allows individual files to be cached for offline access. This is useless when I want to make whole albums available offline.</li>
<li>Amazon Cloud Photos - no support for organizing photos into albums. </li>
<li>No offline caching support:</li>
<ul>
<li>SkyDrive</li>
<li>SugarSync</li>
<li>Picturelife</li>
<li>Loom</li>
</ul>
</ul>
So the candidates at this point are (ignoring Picasa Web which I am trying to move away from):<br />
<ul>
<li>Dropbox</li>
<li>Flickr</li>
<li>Everpix</li>
</ul>
<span style="font-size: large;">Uploading Photos From My P<span style="font-size: large;">C</span></span><br />
I currently prefer to organize my photos into folders on my PC. Any service which I choose to use needs to make it easy to upload my existing collection of photos in folders.<br />
<br />
Flickr quickly fails this test as the desktop uploader wants to upload a flat stream of photos rather than folders of photos. The closest Flickr equivalent of a folder is a "photoset" but these have to be created manually during upload - not exactly the easy sync I was hoping for! There are independently developed tools available but none of them seemed to offer exactly the kind of sync which I was looking for. I was also put off the Flickr service which seemed complex to setup for storing photos privately - it feels very similar to Facebook in that all of the defaults want you to share your photos!<br />
<br />
Dropbox ticks the box for ease of upload as it has an excellent desktop sync client that will simply upload my existing folders of photos.<br />
<br />
Everpix provides a desktop uploading client and allows manual re-download of photos. However photos can't be manually organized into albums. I also found that it took a few minutes for each uploaded photo to become accessible. Together these limitations make Everpix unusable for me.<br />
<br />
<span style="font-size: large;">Affordable Storage</span><br />
Dropbox charges $100/year for 100GB.<br />
<br />
I currently have a grandfathered Google storage plan - $5/year for 20GB. However if I want more space the closest normally available plan on Google is $5/month ($60/year) for 100GB.<br />
<br />
It is worth mentioning that Amazon Cloud Drive actually has low prices at various capacities - 20GB for $10/year, 50GB for $26/year and 100GB for $52/year (all these prices are approximate as I have converted from GBP). What a pity that the Amazon Cloud Drive Photos iOS app is so poor.<br />
<br />
<span style="font-size: large;">What About iCloud?</span><br />
As far as I'm concerned iCloud is a total failure for photos. iCloud is not a cloud photo store. iCloud Photo Stream is a cache of recent photos which makes it simple to push photos from the device where you take them to all of your other Apple devices. <br />
<br />
<span style="font-size: large;">Verdict</span><br />
I already have a total of 36GB of storage with Google at a price I am very happy with. They even have a service (Picasa - becoming Google+ Photos) dedicated to photo management and a 3rd party app (Web Albums by Pixite) which provides offline caching. Unfortunately I just don't trust them to reliably store my photos. I am also frustrated to see Picasa Web redirecting to Google+ Photos which lacks simple features like sorting my albums by their names.<br />
<br />
Google Drive would be a great alternative that uses the same storage but is run by a different team in Google that might do a better job with my files. Unfortunately there is currently no way for me to cache entire folders stored in Google Drive offline on my iPhone.<br />
<br />
Dropbox seems to be the clear winner amongst the competition as it also provides an easy to use file syncing desktop app and allows me to continue to use a Pixite app on my iPhone to manage my uploads and local caching.<br />
<br />
I'll give it a few weeks before I actually move all of my stuff. If Picasa starts working again in that time I might consider sticking with that service a bit longer and just manually download newer albums periodically to backup locally. It is also possible that Google Drive will add folder level caching at some point.mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-36904015132200604172013-09-17T23:49:00.000+01:002013-10-01T00:41:47.599+01:00Why You Shouldn't Trust Picasa To Store Your Photos<span style="font-size: large;">tl;dr</span><br />
A while ago I uploaded this picture to Picasa:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXFnawuKZOQwa46QD5T7sY6Tk-ywwvX1bPLuEnkP_iYKSNTheMnWw1DSA6ZOLyGe_fYGLaqg8mltj8Z3S0vcANA0t8IFMWUasf863BV69yOMhXiAOnxsE9jq_w7lCcyOyP9O97ihJXElM/s1600/plant_ios.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXFnawuKZOQwa46QD5T7sY6Tk-ywwvX1bPLuEnkP_iYKSNTheMnWw1DSA6ZOLyGe_fYGLaqg8mltj8Z3S0vcANA0t8IFMWUasf863BV69yOMhXiAOnxsE9jq_w7lCcyOyP9O97ihJXElM/s320/plant_ios.JPG" width="240" /></a></div>
<br />
Recently I tried to download this photo and got this back from Picasa:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5lJce1wfQFHBRXj4DkljFWi0cmvlIzkTOCTiVbXDsoPQWtlk5pbc0Kja2aTh7yGjJ4_aCH5nI-uPCI_F_4PGlvmeIjuCnv7QwKFQJAJbyyGNqtSZy6bwKys9aPOLk_llI5F16_kCCLCY/s1600/plant_picasa.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5lJce1wfQFHBRXj4DkljFWi0cmvlIzkTOCTiVbXDsoPQWtlk5pbc0Kja2aTh7yGjJ4_aCH5nI-uPCI_F_4PGlvmeIjuCnv7QwKFQJAJbyyGNqtSZy6bwKys9aPOLk_llI5F16_kCCLCY/s320/plant_picasa.JPG" width="240" /></a></div>
<br />
WTF!<br />
<br />
<span style="font-size: large;">Update: I Found A Workaround!</span><br />
It turns out that <a href="https://www.google.com/takeout/">Google Takeaway</a>
provides the ability to download individual albums/all photos from
Picasa Web. I tried downloading one of my albums which appeared
corrupted through Picasa Web/Picasa for Windows and it worked! All of
the photos in the album were downloaded correctly! I will probably use this to extract all of my photos and move to another cloud photos service.<br />
<br />
Update 2: I managed to extract all my photos using Google Takeaway. 17 photos failed to download and Google Takeaway generated an error report which included links that I could use to manually download the missing photos.<br />
<br />
Update 3: This bug appears to be <a href="http://mchr3k-coding.blogspot.com/2013/10/picasa-web-update.html">fixed now</a>.<br />
<br />
<span style="font-size: large;">Background</span><br />
For the last two and a half years I have been uploading all of my photos to Picasa. I also have four years of even older photos which I never got round to uploading to Picasa. I always felt guilty about this as I knew I was taking a risk - that my hard drive could fail and I would lose loads of photos. I actually have multiple local hard drives with copies of the photos but what if my computer was stolen or my house burnt down?<br />
<br />
I decided to finally sort this out and get all of my photos into the cloud. I loaded up Picasa for Windows and got it to index all of my local photos. At the end of the process it offered to tag all the faces it had detected in my photos. This sounded like it could be useful but I figured it would be even better to download my more recent photos and tag everything at the same time.<br />
<br />
So I ran an Import to start downloading my cloud albums. The first album which downloaded was very large with 316 photos and it was immediately obvious in Picasa that some of the photos weren't loaded properly. I took a look on disk and found that 37/316 photos had been corrupted in the way shown above.<br />
<br />
<b>That's ~12% corruption in a single album!</b><br />
<br />
<span style="font-size: large;">Investigating The Issue</span><br />
I earn a living as a software engineer so I immediately set to work trying to come up with a meaningful set of symptoms to report to Picasa. This is what I found:<br />
<ul>
<li>The corruption is probably just a truncation:</li>
<ul>
<li>Every corrupted file was very close to (but not exactly) 2MB.</li>
<li>The missing data was always at the "bottom" of the image.</li>
</ul>
<li>The original file data is probably not lost:</li>
<ul>
<li>I can actually download uncorrupted individual files using Web Albums on my iPhone. I don't think these are coming from a local cache but I'm not certain.</li>
</ul>
<li>The corruption isn't random:</li>
<ul>
<li>The same files in an album are always truncated when downloaded.</li>
</ul>
<li>It isn't related to the image contents:</li>
<ul>
<li>If I re-upload an uncorrupted image (extracted using my iPhone) then it can be downloaded again without corruption.</li>
</ul>
<li>Picasa might be doing some kind of backend migration:</li>
<ul>
<li>I saw two filename formats within my albums:</li>
<ul>
<li>IMAGE_[0-9]+.jpeg (e.g. IMAGE_141.jpeg)</li>
<li>IMAGE_[0-9A-Z]+-[0-9A-Z]+-[0-9A-Z]+-[0-9A-Z]+-[0-9A-Z]+.JPG (e.g. IMAGE_520FFAF1-2FF7-4199-82F4-5A88F5BA8076.JPG)</li>
</ul>
<li>All of the corrupted images have the second filename format.</li>
</ul>
<li>The corruption occurs regardless of whether I download using Picasa for Windows or Picasa Web (tested in IE/Chrome/Firefox). </li>
</ul>
<span style="font-size: large;">Reporting The Issue</span><br />
Time to report the issue! I headed over to <a href="https://support.google.com/picasa/">Picasa support</a> and took a look at my options which turn out to be pretty limited:<br />
<ul>
<li><a href="http://productforums.google.com/forum/?hl=en#!categories/picasa/picasa-for-windows">Picasa Google Groups</a> - user-to-user support</li>
<li><a href="https://support.google.com/plus/answer/1678224">Google+ Feedback</a> - "Although we're not able to personally reply to you, we'll use your
feedback to help us continue to improve the Google+ experience."</li>
</ul>
In other words there is no real way to raise an issue and get a reply.<br />
<br />
My issue is with Picasa and not Google+ so I decided to try out the Picasa Google Group but got <a href="http://productforums.google.com/forum/?hl=en#!category-topic/picasa/picasa-for-windows/jkNt2eDB_JI">no reply</a>.<br />
<br />
<span style="font-size: large;">What Now?</span><br />
Frankly, I am hoping that writing this blog post might get the issue some attention. I am also curious to find out whether anyone else has hit this issue. Maybe I'm the only one?<br />
<br />
It seems to me that the Google support model only works when every issue affects enough users to show up in some kind of top level metrics. If Google have really lost a load of my photos I will be really upset but maybe Google doesn't care if there really is only one of me.<br />
<br />
<span style="font-size: large;">The Competition</span><br />
This issue has made me revisit my decision to use Picasa. The world of cloud photos has moved on in the last few years and maybe I would be better served by another service.<br />
<br />
I currently pay Google $5/year on a grandfathered storage plan to get an extra 20GB/year for a total of 36GB. The cheapest up to date plan seems to be 100GB for $5/month. <br />
<br />
I took a look at Flickr today and their service offering sounds pretty good - 1TB of storage for free in exchange for seeing some ads on their site sounds great. If I don't want ads I have to pay $50/year which is still cheaper than the cheapest Google plan. They even <a href="http://www.flickr.com/help/with/photos/">have a form</a> where you can submit a message and they say they will actually send you an answer!mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-69376517191033161252013-04-23T23:52:00.001+01:002013-04-23T23:52:56.741+01:00An Update On Finally BlocksAs I mentioned in my <a href="http://mchr3k-coding.blogspot.com/2013/01/java-code-coverage-filtering-out-noise.html">last</a> <a href="http://mchr3k-coding.blogspot.com/2013/01/java-bytecode-finally-blocks.html">three</a> <a href="http://mchr3k-coding.blogspot.com/2013/01/improving-finally-block-handling.html">posts</a> I have spent some time working on adding filters to the JaCoCo project. I've been meaning to provide a quick update on my progress for some time now.<br />
<br />
Back in February I actually <a href="https://github.com/mchr3k/jacoco/commit/f2748e13b69547fc49f1c2efb16297b86dd073fa">completed a rewrite</a> of my finally dedup code. This rewrite dropped the dependency on line number information and instead relies on spotting duplicate bytecode that coincides with try/catch blocks. Unfortunately there turned out to be quite a few edge cases so the resulting code was fairly ugly even though it was in theory using a more sensible approach. I never got round to releasing this rewrite <br />
<br />
Marchof, the original author of JaCoCo, started a <a href="https://github.com/marchof/jacoco-playground-filters">playground project</a> in February to invstigate how to structure filtering code in a neater way. This was <a href="https://groups.google.com/d/msg/jacoco-dev/unorHxSDEvY/YJ67-wtPWxMJ">mentioned</a> in the JaCoCo developer mailing list.<br />
<br />
I don't think anything further has happened on this since February but I am hoping to find some time to take another look at how to produce a maintainable finally block dedup implementation.<br />
<br />
There certainly appears to be some interest in this work as I have been seeing ~60-80 downloads/week of the version of EclEmma which I publish on my own Eclipse update site ever since I first released it in January.mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-85332475338672123422013-01-14T00:58:00.000+00:002013-01-14T00:58:08.397+00:00Improving Finally Block Handling<link href="http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.css" rel="stylesheet" type="text/css"></link>
<script language="javascript" src="http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.js" type="text/javascript"></script>
<script language="javascript" src="http://google-code-prettify.googlecode.com/svn/trunk/src/lang-css.js" type="text/javascript"></script>
<script type="text/javascript">
document.addEventListener('DOMContentLoaded',function() {
prettyPrint();
});
</script>
In my <a href="http://mchr3k-coding.blogspot.com/2013/01/java-bytecode-finally-blocks.html">last post</a> I described a heuristic for de-duplicating finally blocks within Java bytecode. Since then I have done some more work on this which is worth writing up.<br />
<br />
<span style="font-size: large;">Cleaning Up My Code</span><br />
<br />
My initial implementation of the heuristic I described can be seen <a href="https://github.com/mchr3k/jacoco/blob/b9f35a0814443b4d53ec27b45e17f0ed5f70fc8f/org.jacoco.core/src/org/jacoco/core/internal/analysis/MethodAnalyzer.java#L299">here</a>. This code isn't very clear at all. My initial instinct was to improve this by adding better comments. However, I recently watched a talk which was based on a book called <a href="http://www.amazon.co.uk/dp/0132350882">Clean Code</a> which made a very interesting point:<br />
<br />
<a href="http://www.kblaney.com/software-blog/2012/6/29/comments-are-a-failure.html">Every Comment is a Failure</a> (This isn't quite a hard and fast rule - comments are sometimes appropriate to express high level design decisions)<br />
<br />
In light of this I decided to avoid comments and instead focus on improving the names of types, names of variables and names of functions. Here are some of the examples of changes which I made:<br />
<pre class="line"><span class="x">List<Instruction> => </span><span class="x">Sequence</span>
<span class="x">List<List<Instruction>> => Cluster</span>
<span class="x">List<List<Instruction>> cluster = findCluster(clusters, p); </span>
<span class="x">=> Cluster cluster = findClusterContainingInsn(clusters, p);</span></pre>
The full diff can be seen <a href="https://github.com/mchr3k/jacoco/commit/697ecaf7442aeb6d5bc8e73e9144ef87bdcae33e">here</a>. I think the result is <a href="https://github.com/mchr3k/jacoco/blob/697ecaf7442aeb6d5bc8e73e9144ef87bdcae33e/org.jacoco.core/src/org/jacoco/core/internal/analysis/MethodAnalyzer.java#L299">much easier to read</a> but I still need to improve or ideally strip out some of my existing comments. In some places I am still trying to use a comment to describe the following block of code which suggests I should actually be extracting out more methods.<br />
<br />
<span style="font-size: large;">Improving Finally Block Dedup Heuristic</span><br />
<br />
My initial heuristic wasn't quite good enough. Consider the following simple Java code:<br />
<pre class="prettyprint lang-java">
try {
System.out.println("A");
} catch (RuntimeException ex) {
System.out.println("B");
} finally {
System.out.println("C");
}
</pre>
This compiles to the following bytecode (with catch blocks and coverage probes annotated).<br />
<pre class="prettyprint lang-java">
0 getstatic #16 <java/lang/System.out>
3 ldc #22 <A>
5 invokevirtual #24 <java/io/PrintStream.println>
8 goto 42 (+34)
catch (RuntimeException) => 11
11 astore_1
12 getstatic #16 <java/lang/System.out>
15 ldc #30 <B>
17 invokevirtual #24 <java/io/PrintStream.println>
catch (Anything) => 31
20 getstatic #16 <java/lang/System.out>
23 ldc #32 <C>
25 invokevirtual #24 <java/io/PrintStream.println>
coverageProbe[0] = true
28 goto 50 (+22)
31 astore_2
32 getstatic #16 <java/lang/System.out>
35 ldc #32 <C>
37 invokevirtual #24 <java/io/PrintStream.println>
40 aload_2
coverageProbe[1] = true
41 athrow
42 getstatic #16 <java/lang/System.out>
45 ldc #32 <C>
47 invokevirtual #24 <java/io/PrintStream.println>
coverageProbe[2] = true
50 return
</pre>
Assuming no exception was thrown coverageProbe[2] would be hit naturally. The finally block dedup heuristic would also mark coverageProbe[1] and coverageProbe[0] as hit. Working backwards from these probes would lead the RuntimeException block to be marked as covered even though no RuntimeException has been thrown.<br />
<br />
My solution is to treat copied probes differently. For these probes coverage is only propagated up until a line where the number of Sequence duplicates changes. This prevents coverage from a finally block leaking outside the finally block. This appears to achieve the desired result.mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-52804821765046312082013-01-09T23:38:00.002+00:002013-01-12T14:25:30.765+00:00Java Bytecode - Finally Blocks<link href="http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.css" rel="stylesheet" type="text/css" />
<script type="text/javascript" language="javascript" src="http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.js"></script>
<script type="text/javascript" language="javascript" src="http://google-code-prettify.googlecode.com/svn/trunk/src/lang-css.js"></script>
<script type="text/javascript">
document.addEventListener('DOMContentLoaded',function() {
prettyPrint();
});
</script>
In my <a href="http://mchr3k-coding.blogspot.com/2013/01/java-code-coverage-filtering-out-noise.html">last post</a> I talked about
some new code coverage filters which I have added to my <a href="http://mchr3k.github.com/jacoco/">fork of the JaCoCo project</a>. These filters are used at coverage analysis time when
the recorded coverage data is analysed against the corresponding class
bytecode. In the process of writing these filters I came across the grim truth behind Java finally blocks.<br />
<br /><div>
<span style="font-size: large;">Finally Blocks</span></div>
<div>
<br /></div>
<div>
It turns out there is no explicit support for this Java language
feature so in bytecode it's use has unfortunate consequences. Consider the
following Java code:</div>
<pre class="prettyprint lang-java">
try {
try {
System.out.println("A");
} catch (RuntimeException ex) {
System.out.println("B");
} finally {
System.out.println("C");
}
} catch (RuntimeException ex) {
System.out.println("D");
} finally {
System.out.println("E");
}
</pre>
<div>
This will end up producing the following bytecode (annotated with line numbers and exception handlers):</div>
<pre class="prettyprint lang-java">
try{
try{
try{
try{
L3:
0 getstatic <java/lang/System.out>
3 ldc <A>
5 invokevirtual <java/io/PrintStream.println>
8 goto 42 (+34)
} catch (RuntimeException) => 11
L4:
11 astore_1
L5:
12 getstatic <java/lang/System.out>
15 ldc <B>
17 invokevirtual <java/io/PrintStream.println>
} catch (Anything) => 31
L7:
20 getstatic <java/lang/System.out>
23 ldc <C>
25 invokevirtual <java/io/PrintStream.println>
28 goto 84 (+56)
L6:
31 astore_2
L7:
32 getstatic <java/lang/System.out>
35 ldc <C>
37 invokevirtual <java/io/PrintStream.println>
L8:
40 aload_2
41 athrow
L7:
42 getstatic <java/lang/System.out>
45 ldc <C>
47 invokevirtual <java/io/PrintStream.println>
L8:
50 goto 84 (+34)
} catch (RuntimeException) => 53
L9:
53 astore_1
L10:
54 getstatic <java/lang/System.out>
57 ldc <D>
59 invokevirtual <java/io/PrintStream.println>
L12:
62 getstatic <java/lang/System.out>
} catch (Anything) => 73
65 ldc <E>
67 invokevirtual <java/io/PrintStream.println>
70 goto 92 (+22)
L11:
73 astore_3
L12:
74 getstatic <java/lang/System.out>
77 ldc <E>
79 invokevirtual <java/io/PrintStream.println>
L13:
82 aload_3
83 athrow
L12:
84 getstatic <java/lang/System.out>
87 ldc <E>
89 invokevirtual <java/io/PrintStream.println>
L14:
92 return
</pre>
<div>
From
a code coverage point of view this is a complete nightmare. For any
branch within a finally block the number of actual branches in the
bytecode is multiplied by the number of times that the finally block
is duplicated (at least two). </div>
<div>
<br /></div>
<div>
Technically this
is correct as to cover all the possible paths through the bytecode you
must consider all of the possible exception paths. However, in practice
the developer is likely only concerned with ensuring that all the
branches in their finally blocks are covered at least once and don't
want to worry about the possible paths into the finally block. </div>
<div>
<br /></div>
<div>
Implementing
the filtering required to deduplicate coverage of finally blocks is
made tricky by the fact that the start/end of the duplicate blocks are not explicitly
annotated in any way in the bytecode. The closest I could get was to
write a heuristic which works during the analysis of a method:</div>
<div>
<ul>
<li>When a new line number marker is seen a new list of instructions is started.</li>
<li>Instructions are added to this list as they are seen. The instruction arguments are ignored.</li>
<li>After all the instructions in a method have been visited:</li>
<ul>
<li>For each line number which was seen:</li>
<ul>
<li>The instruction sequences seen with that line number are "clustered". Sequences which start with the same instruction are grouped together and assumed to be copies caused by the line being within a finally block.</li>
</ul>
<li>For each covered probe:</li>
<ul>
<li>Find the instruction sequence which contains the probe.</li>
<li>Mark the corresponding probe in every other sequence in the cluster as covered.</li>
</ul>
<li>Run the existing probe coverage propagation.</li>
<li>For each cluster:</li>
<ul>
<li>If the last instruction of any sequence in the cluster is covered, mark all other sequences in the cluster as covered.</li>
<li>Disable coverage reporting for all but one sequence.</li>
</ul>
</ul>
</ul>
This (relatively) simple heuristic is good enough to work in most cases. However it is straightforward to come up with code which will trip this up:</div>
<pre class="prettyprint lang-java">
int i = 0;
for (i++; i < 10; i++) {
System.out.println("A");
}
</pre>
<div>
This produces the following bytecode:</div>
<pre class="prettyprint lang-java">
L1:
0 iconst_0
1 istore_1
L2:
2 iinc 1 by 1
5 goto 19 (+14)
L3:
8 getstatic #16 <java/lang/System.out>
11 ldc #22 <A>
13 invokevirtual #24 <java/io/PrintStream.println>
L2:
16 iinc 1 by 1
19 iload_1
20 bipush 10
22 if_icmplt 8 (-14)
L5:
25 return
</pre>
<div>
There are two blocks of code tagged as being on line 2 and they both start with the same instruction. With my heuristic in place one of these blocks will be ignored as they are assumed to be duplicates. This is a shame but in practice I have yet to think of an example of realistic code which would be matched by my heuristic. </div>
<div>
<br /></div>
<div>
<span style="font-size: large;">Conclusions</span></div>
<div>
<br /></div>
<div>
I'm pleased that my heuristic works as well as it does but I would love to come up with a better mechanism. If you have any suggestions I would love to hear them!</div>
mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-58461353696605361972013-01-09T00:41:00.001+00:002013-01-09T00:41:38.217+00:00Java Code Coverage: Filtering out the noiseCode Coverage is a measure of how much of your code has been executed. This is usually measured while running unit tests as a way of tracking how much production code is being exercised. It is important to note that high code coverage doesn't give any guarantee that the behaviour of the covered code is actually being asserted on by your unit tests but it is a useful indicative measure.<br />
<br />
However, there are a couple of serious drawbacks with most simple code coverage tools.<br />
<br />
<span style="font-size: large;">Implicit Code</span><br />
<br />
Java source code is compiled to bytecode before being executed. The most common mechanism for tracking Java code coverage is to instrument this bytecode. The problem with this approach is that it can lead to reporting of coverage (or lack of) for elements which are not present in the actual Java source code. This is frustrating as it either lowers the overall code coverage figure or forces a developer to write pointless extra tests.<br />
<br />
Examples:<br />
<ul>
<li>Classes without an explicit constructor have an implicit no-args constructor.</li>
<li>Enum classes have implicit values() and valueOf(String) methods.</li>
<li>Synchronized(object) {} blocks are implemented in bytecode by duplicating the instructions for releasing the lock inside and outside of a catch block. This would naively require an exception to be thrown from within every synchronized block to achieve full coverage.</li>
<li>All finally {} blocks are implemented in bytecode by duplicating blocks of instructions possibly several times. This would naively require every branch within the finally block to be exercised for every duplicate.</li>
</ul>
<span style="font-size: large;">Uncovered Code</span><br />
<br />
It is commonly acknowledged that 100% code coverage is an expensive goal to attempt to reach. This is usually because getting the final 5-10% code coverage involves writing pointless extra tests that add little real value and can require production code to be distorted to support the pointless extra tests.<br />
<br />
<span style="font-size: large;">JaCoCo</span><br />
<br />
<a href="http://www.eclemma.org/jacoco/">JaCoCo</a> is an open source Java Code Coverage tool which is in active development. It is a mature project and since late 2011 has been the library behind the popular Eclipse plugin EclEmma.<br />
<br />
Over the last few months I have been working to add some key features to this library to address the concerns discussed above. <br />
<br />
I have updated JaCoCo to automatically filter out coverage of all of the following:<br />
<ul>
<li>Implicit no-args constructors.</li>
<li>Implicit enum methods.</li>
<li>Synchronization exit handling.</li>
<li>Finally block code duplication.</li>
</ul>
I have also added support for source directives that allow blocks of code to be explicitly excluded from code coverage reports. This addresses the issue of uncovered code and allows developers to explicitly mark blocks of code which are deliberately not being covered by unit tests.<br />
<br />
All of these changes are ready to try out now by downloading the release of my JaCoCo fork from the link below.<br />
<br />
I am working with the JaCoCo developers to get my changes accepted back into the core JaCoCo project. <br />
<br />
<span style="font-size: large;">More Information </span><br />
<br />
<a href="http://mchr3k.github.com/jacoco/">http://mchr3k.github.com/jacoco/</a>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-87498808268197618912012-11-11T17:18:00.002+00:002012-11-11T17:22:14.910+00:00JD-Eclipse Realign - Usability<span style="font-size: large;">Background</span><br />
<br />
In my <a href="http://mchr3k-coding.blogspot.co.uk/2012/08/jd-eclipse-realign-source-lifetime.html">previous post</a> I talked about the problems caused by the original <a href="http://mchr3k.github.com/jdeclipse-realign/">JD-Eclipse Realign</a> approach of using a special editor to open decompiled source for a class. The key problem was that opening a class file is something which you only expect to involve the selected class but the resulting decompiled source is attached to the entire jar. Worse, the decompiled source attachment was something which could then be used by the standard Class File Viewer resulting in some weird behaviour.<br />
<br />
I previously fixed this by switching to a model where decompiled source was enabled/disabled through a context menu that was intended to make the similarity with source attachments much more explicit.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEis9M8aCba-LLXP6wD4c3wB9HJqNJb1a6H6uyOpz5BAFeEMTU6foXKBi56D9kC2w5-ykoW9K3LiVqzLoPF-H_pASFqD53r-xtbfyWqujcchgzuxG-zRHQvNscnd9NmXrPdlLdtMYfhPgOY/s1600/jde_enabled.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="86" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEis9M8aCba-LLXP6wD4c3wB9HJqNJb1a6H6uyOpz5BAFeEMTU6foXKBi56D9kC2w5-ykoW9K3LiVqzLoPF-H_pASFqD53r-xtbfyWqujcchgzuxG-zRHQvNscnd9NmXrPdlLdtMYfhPgOY/s320/jde_enabled.png" width="320" /></a></div>
<br />
This solution was technically sound but the usability was frankly awful. Any time you wanted to enable/disable decompiled source you were forced to use the Package/Project Explorer context menu which has loads of entries and is therefore always quite unpleasant to use.<br />
<br />
<span style="font-size: large;">Toolbar Buttons (V1.1.0)</span><br />
<br />
The first step to improve this situation was to take the ability to attach/detach decompiled source and move it into the top level toolbar.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="http://mchr3k.github.com/jdeclipse-realign/img/jde_buttons.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://mchr3k.github.com/jdeclipse-realign/img/jde_buttons.png" /></a></div>
<a href="http://mchr3k.github.com/jdeclipse-realign/img/jde_buttons.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a><br />
<div style="text-align: left;">
This pair of buttons expose two commands - Attach Decompiled Source, Detach Decompiled Source. The buttons are dynamically displayed in the toolbar when either a Class File Viewer editor is active or when at least one class, package in a jar or entire jar is selected in the Package/Project Explorer.</div>
<br />
Pressing one of these buttons applies the action to all relevent selected objects in the active "Part". This means that if an editor is active the action applies to the class in that editor and if the Package/Project Explorer view is active the action applies to all selected classes, packages in jars or entire jars.<br />
<br />
<span style="font-size: large;">Decompiled By Default (V1.1.1)</span><br />
<br />
Having added the toolbar buttons I realised that it was still not easy enough to jump straight into decompiled source. If you open a class without source you have to deal with a tiny moment of disappointment/frustration before reaching for the toolbar button to enabled decompiled source.<br />
<br />
The solution to this problem is to restore the JD-Eclipse Realign editor (JDE-R Editor) with subtely different functionality. <br />
<br />
The original decompiled source editor was styled with a different icon and always attached decompiled source when opening a class file. In the restored JDE-R Editor the behaviour has been changed so that decompiled source is only attached when there isn't already a normal source attachment. The icon of the JDE-R Editor has also been changed to be the same as the standard Class File Viewer. <br />
<br />
The overall intention is that the JDE-R Editor should behave and look exactly the same as the built in Class File Viewer except that opening a class in a jar without a source attachment will attach decompiled source.<br />
<br />
Once you have a class file opened with either normal or decompiled source in either the standard Clas File Viewer or JDE-R Editor you can use the toolbar buttons to quickly switch between normal and decompiled source.<br />
<br />
<span style="font-size: large;">File Associations</span><br />
<br />
With the JDE-R Editor restored in V1.1.1 I have also restored the code which automatically attempts to set up the "correct" file associations on startup. These are as follows:<br />
<br />
Pre-Juno:<br />
<ul>
<li>.class file - Class File Viewer [Decompiled]</li>
</ul>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXgYuhTEaf_eXQnpywBJ8td5cFAplhiK4izaEn-4T_Yzy7IEfbfGDiVduEUi7PpcULycNWPeS8CpNJy-Vlqc8L6sOfjhcASP9R7UwQLByJGTr35bxbChpPchf-iEiBpZZxXh-fqSC3u9c/s1600/jde_filetypes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="234" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgXgYuhTEaf_eXQnpywBJ8td5cFAplhiK4izaEn-4T_Yzy7IEfbfGDiVduEUi7PpcULycNWPeS8CpNJy-Vlqc8L6sOfjhcASP9R7UwQLByJGTr35bxbChpPchf-iEiBpZZxXh-fqSC3u9c/s320/jde_filetypes.png" width="320" /></a></div>
<ul>
</ul>
Juno:<br />
<ul>
<li>.class file - Class File Viewer</li>
<li>.class file without source - Class File Viewer [Decompiled]</li>
</ul>
"Class File Viewer [Decompiled]" is the name I have given to the JD-Eclipse Realign editor.<br />
<br />
<span style="font-size: large;">Getting Started</span><br />
<br />
To try out JD-Eclipse Realign you should use the update site listed on the <a href="http://mchr3k.github.com/jdeclipse-realign/">JD-Eclipse Realign project page</a>.mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-57331795892892546412012-10-30T23:25:00.000+00:002012-10-30T23:26:41.455+00:00Eclipse Tips - NavigationEclipse is a really powerful tool but it's really easy to miss some of the most powerful features. I was therefore interested to come across the following link on <a href="http://www.reddit.com/r/java">/r/java</a>: <a class="title " href="http://eclipsesource.com/blogs/2012/10/26/top-10-tips-how-to-be-more-productive-with-the-best-ide/">My Top 10 Tips on how to be more productive with the Eclipse IDE.</a><br />
<br />
I liked these tips but I think there are some other important ones which are worth pointing out.<br />
<br />
<span style="font-size: large;">Navigation Tips</span><br />
<br />
Eclipse has some fantastic functionality to enable fast navigation to find whatever you are looking for.<br />
<br />
<span style="font-size: large;">1) Open Type (Ctrl-Shift-T)</span><br />
<br />
The Open Type window is the right way to open Java classes. If you find yourself looking for classes by manually looking around in the Package Explorer you are almost certainly wasting your own time unless you have no idea what the class you are looking for is called.<br />
<br />
Open Type supports wildcards (e.g. H*Map matches HashMap) and camelcase (e.g. LHM matches LinkedHashMap).<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLJBGzzj3yvqj2LOehZS1j7e2_L80lxjQgdnFFxgYk-QZC7fPJco_s3ScxFKnimTaM4YP9LI5lwmU9tOMtjdRjryJgxAPP4JRM1aaOSHB1PXnRzh0eA7Idu8U9OBpD2O9dnzn795SQQ_Y/s1600/eclipse_tips1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgLJBGzzj3yvqj2LOehZS1j7e2_L80lxjQgdnFFxgYk-QZC7fPJco_s3ScxFKnimTaM4YP9LI5lwmU9tOMtjdRjryJgxAPP4JRM1aaOSHB1PXnRzh0eA7Idu8U9OBpD2O9dnzn795SQQ_Y/s320/eclipse_tips1.png" width="320" /></a></div>
<br />
<span style="font-size: large;">2) Open Resource (Ctrl-Shift-R)</span><br />
<br />
The Open Resource window is the right way to open any file which isn't a Java class e.g. a properties file or an xml file etc.<br />
<br />
Open Resources works very similarly to Open Types with support for wildcards and camelcase. However there is also support for specifying Folder Prefixes (e.g. */ui/*.html matches all html files within a folder called ui).<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaYxFIMWdHaOWgdF0BfC-l0qXbR8zbiUyodwSL6cKIdTsBxJ1V4EbSFkAioJ-mVzDx2NNG7Ved8DXSVJjxUEBwS0bO41KH1_csKWVlmbU1nyJYDbbG2GPYEzmFALGFElT9yGem6O3dvf0/s1600/eclipse_tips2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgaYxFIMWdHaOWgdF0BfC-l0qXbR8zbiUyodwSL6cKIdTsBxJ1V4EbSFkAioJ-mVzDx2NNG7Ved8DXSVJjxUEBwS0bO41KH1_csKWVlmbU1nyJYDbbG2GPYEzmFALGFElT9yGem6O3dvf0/s320/eclipse_tips2.png" width="320" /></a></div>
<br />
<span style="font-size: large;">3) Quick Open (Ctrl-O within a Java file)</span><br />
<br />
Quick Open is the right way to jump to particular elements within a Java file. If you use the Outline View you are wasting your time as you are forced to visually search for what you are looking for.<br />
<br />
Quick Open supports wildcards and camel case (as of Eclipse Juno). <br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiviMTQZ0EOEJu8Cnny22YqBcaL-SxK1fnts2mSXzkhwttE7gCFmZcpdFxDmqJdFQ86NE35F-yphTR9h1JO9zYqJHpcfZpp3y8tGkBz2zeNU4jSuCVK_4Y4v_0n2IbpxWn5EV4KuHaIf-o/s1600/eclipse_tips3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="269" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiviMTQZ0EOEJu8Cnny22YqBcaL-SxK1fnts2mSXzkhwttE7gCFmZcpdFxDmqJdFQ86NE35F-yphTR9h1JO9zYqJHpcfZpp3y8tGkBz2zeNU4jSuCVK_4Y4v_0n2IbpxWn5EV4KuHaIf-o/s320/eclipse_tips3.png" width="320" /></a></div>
<br />
<span style="font-size: large;">Other Tips</span><br />
<br />
There are three more tips which I think are particularly worth noting.<br />
<br />
<span style="font-size: large;">4) Quick Access (Ctrl-3)</span><br />
<br />
Eclipse functionality is exposed through a large number of Views, the menus include a huge number of Commands and there are lots of Preferences which control all of this. Finding all of these elements is extremely tricky.<br />
<br />
Quick Access supports wildcards and can be used to load any of these elements. This is by far the fastest way to open things like Views.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGcQpkwrW_76b0HPspUnAUYt9XHkHGcC22JfIfFr1uF-rGVQSmsM-ow4z5EUXpONioyodWPQF6Zp7T2_s-HsMp6GGQeCIxuhHWNMs0njF0UD4yePlR0qr_ZaeD_D_CLLWpf3b5xFla-8w/s1600/eclipse_tips4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="123" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhGcQpkwrW_76b0HPspUnAUYt9XHkHGcC22JfIfFr1uF-rGVQSmsM-ow4z5EUXpONioyodWPQF6Zp7T2_s-HsMp6GGQeCIxuhHWNMs0njF0UD4yePlR0qr_ZaeD_D_CLLWpf3b5xFla-8w/s320/eclipse_tips4.png" width="320" /></a></div>
<br />
<span style="font-size: large;">5) Local History</span><br />
<br />
Have you ever made a series of changes to a file and then wished that you could go back to a previous version? It's easy if you are using a version control system and have checked in a previous version but what if you didn't?<br />
<br />
Eclipse has got a feature to help you out. The Local History feature keeps track of a limited number of previous versions of every file which you save through Eclipse. To revert to one of these previous versions you just have to use the correct menu. Right click the file within Package Explorer and select Replace/Compare With -> Local History...<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIpB5bLcOLrpXu1Eozt6_MvXW3SicxOy6E4K77OmyYQxqFGsr8d7LHl1mCbDYvRugl7CtmNYe01wTwCJ95WVaCJd3Y9L5RCRe5pRLHAQJCz5Efq5CETUn68xBejV9H0mrbI96CLF6XiRk/s1600/eclipse_tips5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="187" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIpB5bLcOLrpXu1Eozt6_MvXW3SicxOy6E4K77OmyYQxqFGsr8d7LHl1mCbDYvRugl7CtmNYe01wTwCJ95WVaCJd3Y9L5RCRe5pRLHAQJCz5Efq5CETUn68xBejV9H0mrbI96CLF6XiRk/s320/eclipse_tips5.png" width="320" /></a></div>
<br />
<span style="font-size: large;">6) Eclipse Help</span> <br />
<br />
It is absolutely worth noting that the
Eclipse Help is really excellent. For example, try looking at the “Open
Type” or “Open Resource” articles for full details of the pattern
matching which you can do.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1N0mh0mlpvr6TMK7_J4iZ4QBg4U89FCatp1vchiElDM8aYm1pjOJjXSQ3YibIbZcchxYUBYtRebb2lo5J9oiTvAW1qUK3iGnx3_YB6fCIeGRw54JxTsV-rmgUWMJNKF-pXhrOKYf3Gbw/s1600/eclipse_tips6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="287" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1N0mh0mlpvr6TMK7_J4iZ4QBg4U89FCatp1vchiElDM8aYm1pjOJjXSQ3YibIbZcchxYUBYtRebb2lo5J9oiTvAW1qUK3iGnx3_YB6fCIeGRw54JxTsV-rmgUWMJNKF-pXhrOKYf3Gbw/s320/eclipse_tips6.png" width="320" /></a></div>
mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-3240421229075357052012-10-29T00:09:00.000+00:002012-10-29T10:21:15.884+00:00Java Libraries Which Use AnnotationsI recently added a command line user interface to my <a href="http://mchr3k.github.com/javathreaddumpanalyser/">Java Thread Dump Analyser</a> tool (<a href="https://github.com/mchr3k/javathreaddumpanalyser/issues/2">Issue #2</a>). As part of this work I tried out a new library that leveraged Java annotations in a nice way. This was the third library which I have encountered that uses annotations so I thought it might be useful to write a post about annotations and point out which Java libraries are putting them to good use.<br />
<br />
<span style="font-size: large;">
Background</span><br />
<br />
What are annotations? Annotations are user defined markers which can be used in Java source code to encode any kind of metadata. Here is a contrived example:<br />
<br />
<pre>@RequestForEnhancement(
id = 2868724,
synopsis = "Enable time-travel",
date = "4/1/3007"
)
public static void travelThroughTime(Date destination) { ... }</pre>
<br />
More information is available in the excellent Java documentation. It is interesting to note that Annotations get a better constructor syntax than the rest of the Java language including named parameters and default values.<br />
<br />
(<a href="http://docs.oracle.com/javase/6/docs/technotes/guides/language/annotations.html">http://docs.oracle.com/javase/6/docs/technotes/guides/language/annotations.html</a>)<br />
<br />
<span style="font-size: large;">
JDO</span><br />
<br />
My first contact with a library which makes extensive use of annotations was the use of JDO within Google App Engine. Once again I'll dive straight into an example:<br />
<br />
<pre class="prettyprint"><span class="lit">@PersistenceCapable</span><span class="pln">
</span><span class="kwd">public</span><span class="pln"> </span><span class="kwd">class</span><span class="pln"> </span><span class="typ">Employee</span><span class="pln"> </span><span class="pun">{</span><span class="pln">
</span><span class="lit">@PrimaryKey</span><span class="pln">
</span><span class="lit">@Persistent</span><span class="pun">(</span><span class="pln">valueStrategy </span><span class="pun">=</span><span class="pln"> </span><span class="typ">IdGeneratorStrategy</span><span class="pun">.</span><span class="pln">IDENTITY</span><span class="pun">)</span><span class="pln">
</span><span class="kwd">private</span><span class="pln"> </span><span class="typ">Key</span><span class="pln"> key</span><span class="pun">;</span><span class="pln">
</span><span class="lit">@Persistent</span><span class="pln">
</span><span class="kwd">private</span><span class="pln"> </span><span class="typ">String</span><span class="pln"> name</span><span class="pun">;</span><span class="pln"></span><span class="pln"></span><span class="pun"></span></pre>
<pre class="prettyprint"><span class="pun">} </span><span class="pln">
</span></pre>
<br />
What I like about this code is that the annotations provide a very concise way of marking which fields need to be stored and how they should be stored. The actual mechanics of storing and retrieving the data are then entirely generic and driven by the annotations which are used.<br />
<br />
(<a href="https://developers.google.com/appengine/docs/java/datastore/jdo/dataclasses">https://developers.google.com/appengine/docs/java/datastore/jdo/dataclasses</a>)<br />
<br />
<span style="font-size: large;">
GSON</span><br />
<br />
GSON is an excellent library for handling JSON serialization. By default it doesn't actually use Java annotations but the ExclusionStrategy API allows the user to define arbitrary logic for picking which fields to serialize. The docs include an example of how to use this API to select fields based on an annotation:<br />
<br />
(<a href="https://sites.google.com/site/gson/gson-user-guide#TOC-User-Defined-Exclusion-Strategies">https://sites.google.com/site/gson/gson-user-guide#TOC-User-Defined-Exclusion-Strategies</a>)<br />
<br />
<span style="font-size: large;">
JCommander</span><br />
<br />
This command line argument parsing library uses annotations to simplify command line argument parsing. Here is a simple example:<br />
<br />
<pre>public class JCommanderExample {<string><string>
@Parameter(names = { "-log", "-verbose" }, description = "Level of verbosity")
private Integer verbose = 1;
@Parameter(names = "-groups", description = "Comma-separated list of group names to be run")
private String groups;
}
</string></string></pre>
<br />
The online docs provide plenty of examples to get you started:<br />
<br />
(<a href="http://jcommander.org/">http://jcommander.org/</a>) mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-36799318697585340922012-08-09T22:10:00.001+01:002012-08-09T22:10:27.928+01:00Eclipse 4.2 (Juno) - BugsFor the last couple of releases the Eclipse project has been working on two streams of development. 3.x uses the legacy workbench and has been the primary release train in past years. However, since Eclipse Helios (3.8) there has also been a second 4.x release train working on a new workbench framework. This has been delivering releases at the same time as 3.x (e.g. Helios (3.6/4.0), Indigo (3.7/4.1)).<br />
<br />
This year Eclipse Juno (3.8/4.2) was released and for the first time the 4.x version was chosen to be the primary release and 3.8 is in theory the last 3.x release. Unfortunately my testing has quickly revealed some glaring bugs - basic elements of interacting with the workbench don't quite work correctly. I have raised five bugs and found one existing bug covering the key issues which I hit:<br />
<ul>
<li><a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=362420">362420</a> - Make "Quick access" optional and hidden by default </li>
<li><a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=386804">386804</a> - The target area to DND Views and Editors into existing stacks is too small</li>
<li><a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=386803">386803</a> - Quick Access mishandles Home and End</li>
<li><a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=386802">386802</a> - DND of Views and Editors not working properly</li>
<li><a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=386806">386806</a> - New Eclipse Icon not used by Eclipse 4.2 on Windows 7</li>
<li><a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=386817">386817</a> - Customize Perspective Tool Bar Visibility appears broken in 4.2</li>
</ul>
Eclipse 4.2 also includes a new default theme (<a class="jive-link-external-small" href="http://download.eclipse.org/eclipse/downloads/drops4/R-4.2-201206081400/news/eclipse-news-part1.html">new and noteworthy</a>, <a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=293481">bug covering implementation</a>) which I really dislike. A classic theme is included which makes Eclipse look much more like the previous 3.x releases but tediously you have to restart to fully apply a theme change despite Eclipse updating most elements dynamically and not warning that a restart is needed to fully apply the change (<a class="jive-link-external-small" href="https://bugs.eclipse.org/bugs/show_bug.cgi?id=362522">362522</a>). <br />
<ul></ul>
Eclipse 3.8 workbench:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMw3Tq-_m80PTLBbIA3poR_1LFtkBI5fCZ2ivF-Rj45YypaVClEwtDtEc5INhfMlmyvangtkKfmlrqF-ZVH2Uh7ljBRlCDGZuLl_gCD2DFBRP_RjI2uolfgJCyhh8tGoIBDKmzApMZerc/s1600/3.8.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="189" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiMw3Tq-_m80PTLBbIA3poR_1LFtkBI5fCZ2ivF-Rj45YypaVClEwtDtEc5INhfMlmyvangtkKfmlrqF-ZVH2Uh7ljBRlCDGZuLl_gCD2DFBRP_RjI2uolfgJCyhh8tGoIBDKmzApMZerc/s320/3.8.png" width="320" /> </a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
Eclipse 4.2 workbench:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4RLqq4zWw-U1T1KaP-Zw2OcJ8rJvzq5fwe9pBzKXLjDuShRfmucXJMDQNlp3Tf5bq6VktMhl5ZlHhUhT-nTGbHrOY9KBKsGn2hRaNjxky1rXCZMfp9Ay5tVb1iQ0PW34XeqfaKDK_1po/s1600/4.2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="189" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj4RLqq4zWw-U1T1KaP-Zw2OcJ8rJvzq5fwe9pBzKXLjDuShRfmucXJMDQNlp3Tf5bq6VktMhl5ZlHhUhT-nTGbHrOY9KBKsGn2hRaNjxky1rXCZMfp9Ay5tVb1iQ0PW34XeqfaKDK_1po/s320/4.2.png" width="320" /></a></div>
<br />
As a result of all of these issues I will be sticking to Eclipse 3.8 this year but I will keep an eye on future 4.x releases. The Eclipse 4.2 downloads are easy to find but there is only one 3.8 package available:<br />
<ul>
<li>3.8 - <a href="http://download.eclipse.org/eclipse/downloads/drops/R-3.8-201206081200/">http://download.eclipse.org/eclipse/downloads/drops/R-3.8-201206081200/</a></li>
<li>4.2 - <a href="http://www.eclipse.org/downloads/">http://www.eclipse.org/downloads/</a></li>
</ul>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-37501922039031675192012-08-08T23:58:00.001+01:002012-11-11T15:44:03.502+00:00JD-Eclipse Realign - Source LifetimeIn my <a href="http://mchr3k-coding.blogspot.com/2012/07/realignment-for-jd-eclipse.html">initial post</a> about my fork of JD-Eclipse Realign I mentioned the difficulties I encountered in trying to allow a user to switch quickly between decompiled source and attached source. Since that post I have realised that my original fix was insufficient.<br />
<br />
Opening a class file in the JD-Eclipse Realign editor involves the following steps:<br />
<ol>
<li>Check whether a source mapper is already installed for the container jar. </li>
<li>If one already exists use that to get the source for this class file.</li>
<li>If one does not exist create a new one, register it with the container jar and then use it to get the source for this class file.</li>
</ol>
The JD-Eclipse editor subclasses the built in Class File Viewer (which already implements steps 1. and 2.) and provides an implementation of 3. that always returns a source mapper that generates decompiled source code. This has the unfortunate side effect that once you have loaded a single class in the JD-Eclipse editor (which registers a decompiling source mapper with the jar) you can open the standard Class File Viewer and get decompiled source for any other class in the same jar. <br />
<br />
My original JD-Eclipse Realign editor changes fixed this by de-registering the source mapper when you close the editor. This still meant that while a single JD-Eclipse Realign editor was open you could still get decompiled source in the standard Class File Viewer. I tried to address this once and for all by removing the source mapper as soon as the JD-Eclipse Realign editor was opened. Unfortunately this had unforeseen consequences. It turns out that navigating to a particular element (e.g. from the Outline view) relies on a source mapper being available. <br />
<br />
In light of all of this I concluded that having a special JD-Eclipse editor was actually the wrong approach. A better solution is to treat enabling class decompilation the same way as attaching real source code. To enable this I reworked the Open Class With menu which I had already added to work slightly differently. <br />
<ul>
<li>When there is no source attached and decompilation is disabled:</li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdYwwnUBMWTs3N6TSjPML1VV2z3kQ6iJK8PnCt_VnLncb__czsOinqCTVASw5OO7czT1HvicJNpmUoT5JtARKXij9_m3N8QC8KRcq5Ly1a8vmr2D-MuxKJ9MY2GL7fE8DOSjLaWcdZT4o/s1600/jde_disabled.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" height="90" id="blogsy-1344445051156.0305" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdYwwnUBMWTs3N6TSjPML1VV2z3kQ6iJK8PnCt_VnLncb__czsOinqCTVASw5OO7czT1HvicJNpmUoT5JtARKXij9_m3N8QC8KRcq5Ly1a8vmr2D-MuxKJ9MY2GL7fE8DOSjLaWcdZT4o/s320/jde_disabled.png" width="320" /></a><br />
<ul>
<li>When there is no source attached and decompilation is enabled:</li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEis9M8aCba-LLXP6wD4c3wB9HJqNJb1a6H6uyOpz5BAFeEMTU6foXKBi56D9kC2w5-ykoW9K3LiVqzLoPF-H_pASFqD53r-xtbfyWqujcchgzuxG-zRHQvNscnd9NmXrPdlLdtMYfhPgOY/s1600/jde_enabled.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" height="86" id="blogsy-1344445051168.5393" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEis9M8aCba-LLXP6wD4c3wB9HJqNJb1a6H6uyOpz5BAFeEMTU6foXKBi56D9kC2w5-ykoW9K3LiVqzLoPF-H_pASFqD53r-xtbfyWqujcchgzuxG-zRHQvNscnd9NmXrPdlLdtMYfhPgOY/s320/jde_enabled.png" width="320" /></a><br />
<ul>
<li>When there is source attached:</li>
</ul>
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTGJUv77f1kJVtfvqfUiFfZYPT8tNjxEYhqNs16N-hudfFcfUImE1CSqbCbR1WNmPANZDXnpG2HKfZA4Lh4_jj_FLdsUyiwfzi3Qd3CFkbYelP9PEllU1izAAOiWqBZtsQU4bZmpoa2A4/s1600/jde_attached.png" style="margin-left: 1em; margin-right: 1em;"><img alt="" height="73" id="blogsy-1344445051164.046" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgTGJUv77f1kJVtfvqfUiFfZYPT8tNjxEYhqNs16N-hudfFcfUImE1CSqbCbR1WNmPANZDXnpG2HKfZA4Lh4_jj_FLdsUyiwfzi3Qd3CFkbYelP9PEllU1izAAOiWqBZtsQU4bZmpoa2A4/s320/jde_attached.png" width="320" /></a><br />
<br />
In this last case the menu doesn't allow decompilation to be directly enabled for dull implementation reasons. In all cases when the source attachment/decompilation state changes all open class file viewers are updated. <br />
<br />
With the addition of these new menu options I have also removed the JD-Eclipse Realign editor. My code is packaged as a fragment that extends the base JD Eclipse plugin so I can't remove the vanilla JD Eclipse editor. However, on every startup my fragment will check whether the JD Eclipse editor has been associated with class files (or class files without source) and revert any such associations to use the standard Class File Viewer. <br />
<br />
An updated version of JD Eclipse Realign is available from my update site now: http://mchr3k-eclipse.appspot.com/<br />
<br />
If you have any issues please do <a href="https://github.com/mchr3k/jdeclipse-realign/issues">raise it here</a>. <br />
<br />
<span class="Apple-style-span" style="font-size: x-small;"><br /></span>
mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-74231762789792000092012-08-07T00:26:00.001+01:002012-08-07T00:26:52.156+01:00Java Packaging<font class="Apple-style-span" size="5"><strong>jar</strong> (<a href="http://ant.apache.org/manual/Tasks/jar.html" target="_self" title="">http://ant.apache.org/manual/Tasks/jar.html</a></font><font class="Apple-style-span" size="5">)</font><div><br></div>The basic mechanism for packaging Java code is within a jar file. This can be achieved in ant using the jar task. The jar task is more powerful than it first appears as you can use a nested zipfileset to package multiple source jars into a single output jar. <br/><br/>However, the danger with this mechanism is that you may end up packaging a version of a class which conflicts with the version required by another dependency.<br/><br/><font class="Apple-style-span" size="5"><strong>jarjar</strong> (<a href="http://code.google.com/p/jarjar/" target="_self" title="">http://code.google.com/p/jarjar/</a>)</font><br/><br/> jarjar extends the default jar ant task with a new "rule" child element which specifies mappings of package names. This rewrites all the classes within the output jar to move the classes within the specified packages and update all the references to these classes. <br/><br/>For example org.apache.commons.io could be moved to com.product.internal.commons.io. This is one way to avoid the classic <a href="http://en.wikipedia.org/wiki/Java_Classloader#JAR_hell" target="_self" title="">jar hell</a>. <br/><br/><font class="Apple-style-span" size="5"><strong>jarinjarloader</strong> (no site)</font><br/><br/>The jarinjar classloader is an internal part of the Eclipse JDT project. However, you can get the classes by opening <code>org.eclipse.jdt.ui_*version_number*.jar</code> with an archiver and extracting the file <code>jar-in-jar-loader.zip</code>.<br/><br/>With the aid of these classes it is possible to package a jar within another jar and still load the contained classes and resources. For an example of what this looks like in practice you can look at getSWTClassloader() in <a href="https://github.com/mchr3k/swtjar/blob/master/src/org/swtjar/SWTLoader.java" target="_self" title="">SWTLoader.java</a>.<br/><br/>Another project which claims to do the same is One-Jar (<a href="http://one-jar.sourceforge.net/" target="_self" title="">http://one-jar.sourceforge.net/</a>). However, I haven't used this myself.<br/><br/><font class="Apple-style-span" size="5"><strong>swtjar</strong> (<a href="http://mchr3k.github.com/swtjar/" target="_self" title="">http://mchr3k.github.com/swtjar/</a>)</font><br/><br/>The standard approach to packaging SWT applications is to create a bundle per platform (32/64 bit and Win/Linux/OSX). This is because SWT requires the application to load the correct jar for the platform being used. <br/><br/>However, with the help of the jarinjar loader classes it is possible to include multiple SWT jars within an application jar and then use standard platform detection code to pick the correct one to programmatically add to the class path. My swtjar project packages this solution as an ant task. <br/><br/><font class="Apple-style-span" size="5"><strong>jarbundler</strong> (<a href="http://informagen.com/JarBundler/" target="_self" title="">http://informagen.com/JarBundler</a>/)</font><br/><br/>OSX has its own package format to allow an application to provide an icon and control startup options. This is particularly useful as SWT on OSX requires an extra command line argument to work. The jarbundler project is an ant task that helps you to build OSX application bundles.<br/><br/>For details about using this tool you should refer to this page: <a href="http://mchr3k.github.com/swtjar/osxapp.html" target="_self" title="">http://mchr3k.github.com/swtjar/osxapp.html</a><p> </p><div style="text-align: right; font-size: small; clear: both;" id="blogsy_footer"><a href="http://blogsyapp.com" target="_blank"><img src="http://blogsyapp.com/images/blogsy_footer_icon.png" alt="Posted with Blogsy" style="vertical-align: middle; margin-right: 5px;" width="20" height="20" />Posted with Blogsy</a></div>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-24308671545122388192012-07-26T20:27:00.000+01:002012-07-26T20:41:06.104+01:00Reverse Literate Programming - Code Trails<b><span style="font-size: large;">
Background </span></b><br />
<br />
Literate Programming is a technique which was introduced by Donald Knuth in 1984:<br />
<br />
<i>I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: "Literate Programming."<br /><br />Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.<br /><br />The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reinforce each other.</i><br />
<br />
The key idea here is that the programmer is tasked with writing documentation that compiles into code rather than writing the code and attempting to document it separately.<br />
<br />
At the other end of the spectrum is the imaginatively named Reverse Literate Programming. This tasks the programmer with writing code that is directly compiled into documentation. This is likely to sound quite familiar as systems like Javadoc enable API documentation to be produced by compiling program source code. However, API documentation alone is not enough as it is focused on the minutia of source code rather than any high level details. Package level documentation can be added but this isn't very accessible as it is stored in a separate file.<br />
<br />
<b><span style="font-size: large;">Code Trails</span></b><br />
<br />
I think IDEs need to do a better job of making it easier to provide simple high level source code documentation.<br />
<br />
The use of markers within code comments allow key locations within code to be highlighted. The IDE collects together these markers into Trails that are displayed together. These Trails provide simple high level documentation and every marker can be double clicked to quickly examine the code at the location of the marker.<br />
<br />
What does this look like in Eclipse? Here is some example code:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPV3-n2o8l8ptHG2RefRbWqSR28_HAey0Uv34iUn1MxdbJa5G1ejl645mh2ZDmGtSRE3ZcJewjt9t-Z1WtXKMjcYiF_HXTHCL06T-_JdrCj3KjDm_aD2bVG6FB2oG9PtTeu-OmipEuGNA/s1600/trailcode.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="85" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjPV3-n2o8l8ptHG2RefRbWqSR28_HAey0Uv34iUn1MxdbJa5G1ejl645mh2ZDmGtSRE3ZcJewjt9t-Z1WtXKMjcYiF_HXTHCL06T-_JdrCj3KjDm_aD2bVG6FB2oG9PtTeu-OmipEuGNA/s320/trailcode.png" width="320" /></a></div>
<br />
And the corresponding Trail view:<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_HPE6ZqApjqHox6vPFyiwHzS9CzD6LJBrlkggb2_ijPkSdkSz74CGRq_8jtCTx07Rq2MinhGdfbgzyZFvuV7rHatkrlks8TJONqfvVw1G110MfSsp4sav1Nr3P7wH1T2J_AtW2O3WxZk/s1600/trailview.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="124" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_HPE6ZqApjqHox6vPFyiwHzS9CzD6LJBrlkggb2_ijPkSdkSz74CGRq_8jtCTx07Rq2MinhGdfbgzyZFvuV7rHatkrlks8TJONqfvVw1G110MfSsp4sav1Nr3P7wH1T2J_AtW2O3WxZk/s320/trailview.png" width="320" /></a></div>
<br />
<b><span style="font-size: large;">Installation</span></b><br />
<br />
If you are interested in trying out the Code Trails plugin you can install it from my update site:<br />
<br />
http://mchr3k-eclipse.appspot.com/<br />
<br />
When you first open the Trails view you will get the following warning about needing to run a full workspace rebuild.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj54WCznI8d5gdl98TcgXwPvPqifJVo1qaFkJJ3tOhjLwzGKJHWN0IJfBWjEb4hICbMvaBkMoS5mLlhndxHAzyIC9tzsa-kKerp4URSW27sk_DGimrjsnVvYdQSyKBiaoMIvPT0af5h5aQ/s1600/trailwarn.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="101" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj54WCznI8d5gdl98TcgXwPvPqifJVo1qaFkJJ3tOhjLwzGKJHWN0IJfBWjEb4hICbMvaBkMoS5mLlhndxHAzyIC9tzsa-kKerp4URSW27sk_DGimrjsnVvYdQSyKBiaoMIvPT0af5h5aQ/s320/trailwarn.png" width="320" /></a></div>
<br />mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-65641443264396930162012-07-09T23:17:00.000+01:002012-08-09T21:32:17.115+01:00Realignment for JD-Eclipse<b>Update:</b> I have posted details about a new release of my plugin <a href="http://mchr3k-coding.blogspot.com/2012/08/jd-eclipse-realign-source-lifetime.html">here</a>. <br />
<br />
In Eclipse, when debugging a Java program or simply trying to explore some code, it is very frustrating when you hit a class without source.<br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9q4HYHGfqsPaCh6wRn_nkX-keNE94PQ_OFLRYMF_repv21reaxgG1yeeaKrL6wGK_QXpw17RInjNDl7niTgi8amf0pQQ0bbiT8Do2WGkD0ewqRXn63qnD8oMSb8OdhufYFz7smsMhWP0/s1600/classview.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" class="" height="301" id="blogsy-1344238630085.612" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj9q4HYHGfqsPaCh6wRn_nkX-keNE94PQ_OFLRYMF_repv21reaxgG1yeeaKrL6wGK_QXpw17RInjNDl7niTgi8amf0pQQ0bbiT8Do2WGkD0ewqRXn63qnD8oMSb8OdhufYFz7smsMhWP0/s320/classview.png" width="320" /></a></div>
<a href="http://metacom/servlet/JiveServlet/showImage/38-3463-13492/nsource.png"></a><br />
<br />
There are a small number of open source Java decompilers which promise to improve this by generating some source code based on the class file. The best one which I have tried is called <a href="http://java.decompiler.free.fr/?q=jdeclipse">JD-Eclipse</a>. However, JD-Eclipse has a couple of key limitations:<br />
<br />
<br />
<ol>
<li>The generated source code doesn’t line up with the debug line numbers because the generated code can’t include the original comments.</li>
<li>If you choose to associate the “*.class” file type within Eclipse with the “Class File Editor” then you always get decompiled output even if you have real source attached.</li>
</ol>
I recently came across a project <a href="http://sourceforge.net/projects/realignmentjd/">Realignment for JD-Eclipse</a> which partially fixes (1) by properly lining up methods and lines within methods. I have <a href="https://github.com/mchr3k/jdeclipse-realign">forked this project</a> and made some further enhancements to fix both (1) and (2). In particular, I have added the following:<br />
<br />
<br />
<ul style="list-style-type: disc;">
<li>Enhance the re-alignment process to handle enums and fields.</li>
<li>Automatically attempt to setup the “correct” file associations.</li>
<ul>
<li>On first start:</li>
<ul>
<li>On Eclipse Juno there is a new “class without source” file type, register to decompile this and restore the built in Class Viewer for classes with source so that attached source can be viewed.</li>
<li>On Eclipse Indigo and below register to handle the “class” file type. This can be undone in Window -> Preferences -> General -> Editors -> File Associations.</li>
</ul>
<li>On every other start:</li>
<ul>
<li>If the default “JD Eclipse” editor is registered to handle “class” or “class without source” update the mapping to use the realignment enabled JD Eclipse editor.</li>
</ul>
</ul>
<li>Add a context menu for choosing which editor to open a class with (see picture below).</li>
<li>Add entry to this context menu to allow the attached source to be changed.</li>
</ul>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizDS2mkMkefAn0yq5pjlExLSYFCH4h4B-OZ7r0eM6-y6lmZWpn29V94k8czQwhg4PPGBw4vL9-acfLLVkA8wK3T4GUkJxu2I2YSUunU88kYmJC1ZPjmg90-S9kQu0UrasIE3u23XwM0ns/s1600/openclasswith.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img alt="" class="" height="140" id="blogsy-1344238630098.4607" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEizDS2mkMkefAn0yq5pjlExLSYFCH4h4B-OZ7r0eM6-y6lmZWpn29V94k8czQwhg4PPGBw4vL9-acfLLVkA8wK3T4GUkJxu2I2YSUunU88kYmJC1ZPjmg90-S9kQu0UrasIE3u23XwM0ns/s400/openclasswith.png" width="400" /></a></div>
<a href="http://metacom/servlet/JiveServlet/showImage/38-3463-13493/openwith.png"></a><br />
<br />
If you are interested in trying this out you can download both JD-Eclipse along with my fork of Realignment for JD-Eclipse from my update site: <a class="jive-link-external-small" href="http://mchr3k-eclipse.appspot.com/">http://mchr3k-eclipse.appspot.com/</a><br />
<br />
<b>Obligatory warning:</b> This code has had very little testing so you please do report any weird behavior to me or raise an issue (<a class="jive-link-external-small" href="https://github.com/mchr3k/jdeclipse-realign/issues">https://github.com/mchr3k/jdeclipse-realign/issues</a>).<br />
<br />
<h1>
Implementation Wrinkle</h1>
One of the trickiest parts of this project so far has been trying to make it possible to switch quickly between the built in Class Viewer (to view attached source) and the JD Eclipse Class Viewer (to view decompiled source). It turns out that there are two mechanisms which make this tricky when you open a Class in the JD Eclipse Class Viewer.<br />
<br />
Firstly the IClassFile is associated with the decompiled source in the BufferManager. With this mapping in place this means that even the built in Class Viewer will display decompiled source. However, the BufferManager is a 60 element LRU cache so if you open enough other files the decompiled source will drop out of the cache.<br />
<br />
Secondly the SourceMapper which is used to generate the decompiled source is registered with the IClassFile's parent PackageFragmentRoot. This means that even when the decompiled source is not cached in the BufferManager the built in Class Viewer will still get hold of the cached SourceMapper and generate and then display decompiled source!<br />
<br />
To solve both of these issues I added some code to the dispose() method of the Realigned JD Eclipse Class Viewer to both remove the entry from the BufferManager and remove the SourceMapper.<br />
<br />
Why does this matter? One of the frustrations with using a decompiler is that once you have one installed you can only view decompiled source even if you have real source attached. I added an "Open Class With" menu to allow a class file to be quickly opened in the regular class viewer to view attached source. <br />
<br />
Eclipse Juno makes this much better as it introduces a "class without source" file type so that you can associate your decompiler with only those classes for which you don't have real source.mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-36499320458444332492012-05-02T21:53:00.001+01:002012-07-09T14:43:23.331+01:00Talk at JUGS<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Today I gave a talk at Java User Group Scotland. The talk covered the material in my last three posts along with the Java Live Thread Analyser. I demoed InTrace, InMemProfiler and JLTA and talked about how they worked.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
The slides from my talk are <a href="http://dl.dropbox.com/u/16837235/Instrumentation.ppt">available here</a>.</div>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-18313303358710158502012-05-02T21:48:00.002+01:002012-07-09T14:43:36.871+01:00InMemProfiler<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
The next instrumentation tool which I wrote (also started in 2010) was motivated by a session using the excellent VisualVM memory profiling to investigate why an application was performing profligate memory allocation. The results in VisualVM are presented as a tree of allocation stack traces. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
This was frustrating in some cases. For example a string constructed from several parts results in stack traces through StringBuilder.append() as well as through StringBuilder.toString(). In practice I would prefer to see both of these allocations presented together as the methods are called from the same site. I also wanted to get a summary of the lifetime of the objects being allocated grouped together into buckets. This is something which VisualVM doesn't offer at all - the closest thing available is an indication of the number of GCs which an object has survived. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
I managed both of these by using the JVM TI API to write an agent that added an InMemProfiler method call into the java.lang.Object constructor and after every array allocation (as these don't use the Object constructor). This instrumentation is done with the native JVM TI API.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
When profiling object lifetimes the profiler code creates a Java weak reference for each allocated object to detect when it has been collected. The tracking of collection could be done much more efficiently using the native JVM TI object tagging mechanism but I wanted to do as much as possible using Java. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
For further information you should refer to my GitHub page: <a href="http://mchr3k.github.com/org.inmemprofiler/" target="_self" title="">http://mchr3k.github.com/org.inmemprofiler/</a></div>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-71806757187358752102012-05-02T21:48:00.001+01:002012-07-09T14:43:49.891+01:00InTrace<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
In my last post I introduced the low level APIs available in the JVM. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
The first tool which I wrote to use the <a href="http://docs.oracle.com/javase/6/docs/api/java/lang/instrument/package-summary.html">java.lang.instrument API</a> was InTrace which I started in the summer of 2010. This tool was written to allow the execution of any Java program to be traced - method entry, exit, method args, thrown exceptions, caught exceptions and execution path within methods. All of this is made possible by inserting InTrace method calls into classes. This can be done at any time thanks to the JVM's support for redefining already loaded classes.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
There are two parts to InTrace. The InTrace Agent allows for the dynamic instrumentation of Java classes to add trace calls. The InTrace UI allows for the remote control of the InTrace Agent and collection of trace output.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
To get started with InTrace you can use the Agent and UI directly. However, I have also written an Eclipse plugin which automates the process of launching a program with the agent and connecting to the launched program with the UI.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="http://mchr3k.github.com/org.intrace/img/eclipse_active.jpg" target="_blank"><img alt="" class="alignnone" height="786" id="blogsy-1335730327840.2156" src="http://mchr3k.github.com/org.intrace/img/eclipse_active.jpg" style="border-bottom-width: 0px; border-color: initial; border-left-width: 0px; border-right-width: 0px; border-style: initial; border-top-width: 0px; padding-bottom: 6px; padding-left: 0px; padding-right: 6px; padding-top: 0px;" width="549" /></a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</div>
<ul style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<li><a href="http://mchr3k.github.com/org.intrace/" target="_self" title="">http://mchr3k.github.com/org.intrace/</a></li>
<li><a href="http://mchr3k.github.com/org.intrace/eclipse.html" target="_self" title="">http://mchr3k.github.com/org.intrace/eclipse.html</a></li>
</ul>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-40751828060799951202012-05-02T21:46:00.000+01:002012-07-09T14:43:58.203+01:00Java Instrumentation APIs<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Historically there were two low level APIs within the JVM:</div>
<ul style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<li><a href="http://docs.oracle.com/javase/1.5.0/docs/guide/jpda/jvmdi-spec.html" target="_self" title="">JVM Debug Interface</a></li>
<li><a href="http://docs.oracle.com/javase/1.5.0/docs/guide/jvmpi/jvmpi.html" target="_self" title="">JVM Profiler Interface</a></li>
</ul>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Both of these were deprecated in Java 5 and removed in Java 6. The replacement introduced in Java 6 was the <a href="http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html" target="_self" title="">JVM Tools Interface</a>. This is a native C API but there is an equivalent available in two parts in pure Java. The <a href="http://docs.oracle.com/javase/6/docs/jdk/api/jpda/jdi/" target="_self" title="">Java Debug Interface</a> and the <a href="http://docs.oracle.com/javase/6/docs/api/java/lang/instrument/package-summary.html" target="_self" title="">java.lang.instrument package</a>. One of the most noticeable changes made in the switch from JVMPI to JVMTI was the reliance on <a href="http://docs.oracle.com/javase/6/docs/platform/jvmti/jvmti.html#bci" target="_self" title="">bytecode instrumentation</a> for tracking method entry/exit and object allocation. The justification for this was to allow the profiling code to be optimised along with the code being profiled to minimise the overall performance impact. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
In my opinion the most interesting part of all of this is that the current JDI and java.lang.instrument APIs allow for interesting debugging and profiling tools to be written in pure Java. In other words, if you are an experienced Java developer, it is relatively easy for you to write great tools that interact with the internals of the JVM and the running of Java program's.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-88838916540392068132012-04-18T23:58:00.001+01:002012-07-09T14:44:25.759+01:00Exciting Java Tools<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
I wanted to blog about some of the most exciting Java tools which I have come across. Sadly none of these tools are free but the technology which they implement is really impressive.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div class="separator" style="clear: both; font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1EkqFyQAn1JvisfW_vKCaTcr8fic3flzdzQBlOxxuBLTODN9IPsl17AKvMPVKR37F7msqgk31KaxP-z1O4VJPHAgHIefq-8QFf7H80_h1I6TE1N9k1k7-eFr3pJPy9f3JF7yzHvP3rxw/" target="_blank"><img alt="" class="alignnone" height="55" id="blogsy-1334874611706.182" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1EkqFyQAn1JvisfW_vKCaTcr8fic3flzdzQBlOxxuBLTODN9IPsl17AKvMPVKR37F7msqgk31KaxP-z1O4VJPHAgHIefq-8QFf7H80_h1I6TE1N9k1k7-eFr3pJPy9f3JF7yzHvP3rxw/s500/chronon-logo2.png" width="244" /></a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="http://www.chrononsystems.com/" target="_self" title="">http://www.chrononsystems.com/</a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Chronon records a complete trace of the execution of a Java program. There are a couple of reasons why this is really cool. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Firstly, the actual logging mechanism is claimed to have a low enough impact that you could realistically run with Chronon all the time. The theory here is that when you hit a hard to reproduce bug you will always have enough information to investigate and fix the bug. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Secondly, Chronon have produced some excellent UI to get the most out of the collected trace files. The <a href="http://www.chrononsystems.com/products/chronon-time-travelling-debugger" target="_self" title="">time travelling debugger</a> allows you to step forwards and backwards through a trace file as if you were stepping through a live program. Even cooler is the <a href="http://www.chrononsystems.com/post-execution-logging" target="_self" title="">post execution logging</a> which allows the developer to add logging statements and have them be executed as if they were present when the trace file was captured. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div class="separator" style="clear: both; font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvfntK5a8BdoaEYDN5jICvMdL71IhlKdQnH2iV5dxa1qj4he1Hr7BiAxeFLix7eK8csW6ZDrJ2qv3cVqvquchbnldcj4KB0ZRr0ww-_ijnuOJvcnWWDeMHfL_DTei3OLo_SV2RT8nwf7I/" target="_blank"><img alt="" class="alignnone" height="43" id="blogsy-1334874611709.7026" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvfntK5a8BdoaEYDN5jICvMdL71IhlKdQnH2iV5dxa1qj4he1Hr7BiAxeFLix7eK8csW6ZDrJ2qv3cVqvquchbnldcj4KB0ZRr0ww-_ijnuOJvcnWWDeMHfL_DTei3OLo_SV2RT8nwf7I/s500/zero.gif" width="315" /></a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="http://zeroturnaround.com/" target="_self" title="">http://zeroturnaround.com/</a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
The Sun Hotspot JVM includes support for a cool feature called Hotswap. This allows the body of Java methods to be updated on the fly while the JVM is running. The feature is nice but limited as a lot of things are not allowed - methods can't be added or removed, method signatures can't be changed, fields can't be added or removed etc. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
ZeroTurnaround have managed to produce a system which runs within the standard Hotspot JVM and removes all of these limitations. This is really cool!</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
This technology has been packaged in two ways. <a href="http://zeroturnaround.com/jrebel/" target="_self" title="">JRebel</a> allows developers to work on applications and change the code without restarting the JVM. <a href="http://zeroturnaround.com/liverebel/" target="_self" title="">LiveRebel</a> does the same but is aimed at live upgrade of webapps without downtime. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div class="separator" style="clear: both; font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi36HHqhFfEwha6DUSXBXpD9jCkqeYIvV3RSUmOEM-gxMwPMAxO-uduvNnd4Q5BKQOQ9GBAfow1Vppj5maFtInWO9kGYuhEkqr3-n7tnB0crKCPvCse56nwTlg3_GxrWemvy-vGHl28LHo/" target="_blank"><img alt="" class="alignnone" height="39" id="blogsy-1334874611661.402" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi36HHqhFfEwha6DUSXBXpD9jCkqeYIvV3RSUmOEM-gxMwPMAxO-uduvNnd4Q5BKQOQ9GBAfow1Vppj5maFtInWO9kGYuhEkqr3-n7tnB0crKCPvCse56nwTlg3_GxrWemvy-vGHl28LHo/s500/azul_logo.png" width="163" /></a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="http://www.azulsystems.com/" target="_self" title="">http://www.azulsystems.com/</a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Azul Systems have created a new garbage collector (C4) that claims to put an end to stop the world garbage collections. The promise is that this allows a single JVM to use 10s of GB of heap without having to endure periodic stop the world GCs that last several seconds. </div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
The big side benefit of this work is that Azul have published lots of <a href="http://www.azulsystems.com/resources/presentations" target="_self" title="">great information</a> about the <a href="http://www.azulsystems.com/sites/www.azulsystems.com/SpringOne_UnderstandingGC2.pdf" target="_self" title="">details</a> of garbage collection. </div>
<h1 style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
One more thing</h1>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
All the tools I have spoken about so far aren't free but there is one more tool which I should mention. </div>
<div class="separator" style="clear: both; font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiPsdxJdJSHHnQb0JGhhsYhjNrLGw0l60LBMjmR3jdIqb7yNyZ9ks2y7odbvmRp8qKeK27XxyQBZSyVoKEpQyRGxnyMZFEeEmJs8uKsa8oFHAo6bWWQh0aoPzxT_0KWKd4YfDWUlgHIr4/" target="_blank"><img alt="" class="alignnone" height="91" id="blogsy-1334874611709.0962" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiPsdxJdJSHHnQb0JGhhsYhjNrLGw0l60LBMjmR3jdIqb7yNyZ9ks2y7odbvmRp8qKeK27XxyQBZSyVoKEpQyRGxnyMZFEeEmJs8uKsa8oFHAo6bWWQh0aoPzxT_0KWKd4YfDWUlgHIr4/s500/eclipse.png" width="171" /></a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
Eclipse is both free and excellent and is definitely the tool that makes the most difference to me every day that I work on Java code. </div>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-11901930786018256492012-04-10T14:44:00.000+01:002012-07-09T14:51:18.931+01:00Java Live Thread Analyser<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
One of my most recent projects was motivated by a session using the excellent <a href="http://visualvm.java.net/">Visual VM</a> tool. I was monitoring the startup of a Tomcat based application and watching the number of threads as a rough metric for how far along the startup was. The application I was monitoring has completely started when there are around 200 live threads out of around 300 started in total.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
I was somewhat taken aback at the number of live threads which were being used and also the number of short lived threads which had been started. Who allocated all these threads? What are they all doing? Unfortunately many of the threads had not been named. A thread dump gives some clues about what live threads are doing but many threads were blocked in generic methods (e.g. Timers). What I really want is to capture the stack trace of all Thread allocations and track the lifetime of all threads.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
This objective prompted me to write a tool which I've called the Java Live Thread Analyser (JLTA).</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div class="separator" style="clear: both; font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</div>
<div class="separator" style="clear: both; font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
</div>
<div class="separator" style="clear: both; font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigmOgfLnYZAestSFuTdFI5Di51DSzFpBx5VbRATHX8lw2jkwvBHuDBeGAYBEC6D83GqPDMEoNAnZ9kK37bhlr239lwfPSg6Jbb8ywayz0YLO1PqgT0u2PqzSFcVnLJExnnhrEYWlUKVG8/" target="_blank"><img alt="" class="alignnone" height="450" id="blogsy-1333984113324.6306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEigmOgfLnYZAestSFuTdFI5Di51DSzFpBx5VbRATHX8lw2jkwvBHuDBeGAYBEC6D83GqPDMEoNAnZ9kK37bhlr239lwfPSg6Jbb8ywayz0YLO1PqgT0u2PqzSFcVnLJExnnhrEYWlUKVG8/s500/IMAGE_97BCC528-0929-449B-97B6-0A3967E1FACA.PNG" width="500" /></a></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
There are two main parts to JLTA.</div>
<ol style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<li>A Java Agent allows me to collect the raw data and allow it to be downloaded over a TCP socket. </li>
<li>A Java SWT GUI to connect to the agent, download the raw data and summarise the results in a useful way. </li>
</ol>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
The Agent collects data by making two transformations to the classes within the JVM. The Thread class is transformed to add a call to my tracking code from the end of all the Thread constructors. The Thread class and all subclasses are transformed to add a call to ky tracking code to the start and end of all run() methods. A Boolean thread local is used to make sure the run() method entry and exit are only handled once. This takes care of the case where a subclass calls into a superclass run() method. The tracking code attempts to detect when a Thread has been allocated by a Tomcat webapp by examining the context classloader of each Thread.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
<br /></div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
The GUI filters the collected data by grouping thread allocations by stack trace. Threads can be filtered based on their thread state, context and whether they have been given a name. The number of stack frames being used for grouping threads together can be limited.</div>
<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
I have published a <a href="http://mchr3k.github.com/jlta/" target="_self" title="">static site</a> describing this tool and linking to the appropriate files to download and use it. </div>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.comtag:blogger.com,1999:blog-4061658288819107706.post-15638406154505371152012-04-09T10:49:00.000+01:002012-07-09T14:45:55.150+01:00Introduction<div style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">
For the last couple of years I have been writing software in my spare time to solve problems I encounter and because I frankly enjoy coding.
For the last few months I have been enjoying getting to grips with Arduino hardware and <a href="http://mchr3k-arduino.blogspot.com/">blogging about it</a>. Given this experience I thought it was time to start this new blog about my coding projects.
In this blog I plan to talk both about new projects and some of my old projects. I have an <a href="http://mchr3k.github.com/">existing static site</a> which documents some of my projects. However, I thought that starting this blog would give me the opportunity to go into more of the technical details behind my projects. </div>mchrhttp://www.blogger.com/profile/17440521866442603326noreply@blogger.com