One of my most recent projects was motivated by a session using the excellent Visual VM tool. I was monitoring the startup of a Tomcat based application and watching the number of threads as a rough metric for how far along the startup was. The application I was monitoring has completely started when there are around 200 live threads out of around 300 started in total.
I was somewhat taken aback at the number of live threads which were being used and also the number of short lived threads which had been started. Who allocated all these threads? What are they all doing? Unfortunately many of the threads had not been named. A thread dump gives some clues about what live threads are doing but many threads were blocked in generic methods (e.g. Timers). What I really want is to capture the stack trace of all Thread allocations and track the lifetime of all threads.
This objective prompted me to write a tool which I've called the Java Live Thread Analyser (JLTA).
There are two main parts to JLTA.
- A Java Agent allows me to collect the raw data and allow it to be downloaded over a TCP socket.
- A Java SWT GUI to connect to the agent, download the raw data and summarise the results in a useful way.
The Agent collects data by making two transformations to the classes within the JVM. The Thread class is transformed to add a call to my tracking code from the end of all the Thread constructors. The Thread class and all subclasses are transformed to add a call to ky tracking code to the start and end of all run() methods. A Boolean thread local is used to make sure the run() method entry and exit are only handled once. This takes care of the case where a subclass calls into a superclass run() method. The tracking code attempts to detect when a Thread has been allocated by a Tomcat webapp by examining the context classloader of each Thread.
The GUI filters the collected data by grouping thread allocations by stack trace. Threads can be filtered based on their thread state, context and whether they have been given a name. The number of stack frames being used for grouping threads together can be limited.
I have published a static site describing this tool and linking to the appropriate files to download and use it.
