Chaotic Java

One of the latest discovered bugs in Yielder, originally thought to be linked to Eclipse, is apparently related to the debugging information usually supplied by compilers. The lack of debugging information made promoting local variables to class members impossible with the current way the yielder works, and new approaches are much more complex.

I’ll try to explain the bug and add some detail about some of the mechanics of compiled code, and some more information about what the yielder does behind the scenes. First, let’s mention the stack frame for a minute. We all know the term from the second most famous method Java has, Throwable.printStackTrace, but what is it really, this “frame”?

Java frames, slots array and debugging information

Well, in essence, the frame contains two essential sections: The code section, where the code pointer and exception handler tables reside in, and the data section where the operand stack, parameters and local variables’ data is held in. The data section vary in size, and is determined at compile time to be as minimal as possible. That means, that all the bytecode calls are being examined to see how deep a stack needs to be in order to perform all of them, and how many slots in the local variables array are needed to capture all the data local variables will provide when the method is being run. This slot array also contains the parameters of the method, including the this pointer in non-static methods.

It is also interesting to note that the slots for local variables are generic: There is no “type” attached to them, as each of them is 4 bytes long and can be used interchangeably between all the primitive types except for long or double (which require two consecutive slots), and as a pointer for any Java object. The compiler takes advantage of this and reuses slots for local variables declared in separate scopes. Thus, smarter compilers will also move the declaration of a local variable as deep as they can inside a scope, to reduce the amount of slots a frame will require. It gets more complicated since after compilation, there is no real mention in code for the local variables: They are just sets of bytes stored in the slots array, used with compile-time calculations’ precision.

So, if local variables disappear after compilation, how can the debugger tell what value each local variable has during a debugging session? This is exactly where the debugging information comes to the rescue, by keeping a table which maps between a slot number and scope pair to a local variable name and its type. For example, the following code (Bear with me - it’s only an example code!):

L1:
int[][] mat = ...;
int total = 0;
int mul = 1;
L2:
for (int i = 0; i < mat.length; i++) {
  for (int j = 0; j < mat[i].length; j++) {
    total += mat[i][j];
  }
}
L3:
for (i = 0; i < mat.length; i++) {
  for (int k = 0; k < mat[i].length; k++) {
    mul *= mat[i][j];
  }
}
L4:

Might produce a slots array of 5 slots, with the following debug information (suppose L1-4 are scope identifiers):

[0, L1, L4] -> mat, int[][]
[1, L1, L4] -> total, int
[2, L1, L4] -> mul, int
[3, L1, L4] -> i, int
[4, L2, L3] -> j, int
[4, L3, L4] -> k, int

With me so far? Great. Now let’s get back to the yielder.

The yielder bug with debugging information disabled

In order to create the members of the yielding class (need a memory refresh?), the yielder (in the current, 0.1.1 version) is looking at the local variables map, and creates a member called variable-name$promoted for each local variable, with the type of the local variable. Matching between a slot access (for read or write, using the xLOAD or xSTORE bytecode operators) to a local variable is done using the scope mapping, also found in the debugging information. However, when debugging information is disabled, this information is unavailable, and thus the yielder should work differently.

The simplest solution that comes to mind is to give up on trying to match a local variable to a promoted mebmer, and just promote the slots themselves. But then, there are two problems: First, since the slot is interchangeable, it can contain any type of information - primitives and Java objects alike, something that a class member cannot do. Second, even if there was a way to create a 4-byte member which contained all types of primitives and Java objects, what about the double and long primitives, taking two slots and not just one?

Well, I have a solution, even though I’m not completely happy about it. What I came up with at the moment is to promote each slot into an Object; after that, each primitive used in the slot will be boxed and unboxed as required, using the new valueOf methods. This might seem like a bit of a performance problem but the fact is that in the simple cases (which are the majority, I hope) the flyweight implementation of valueOf, together with automatic inlining of methods by the JVM, will remove any overheads created by this procedure.

As always, I’m looking for feedback, so: go ahead!

Other posts of interest

• How does garbage collection work?
• First Blood on the IDE Wars
• Drag and Drop in GWT - The How-To
• I guess PriorityBlockingDeque wasn’t high on the priority list..
• LinkedList vs. ArrayList