Implementation details for Java Yielder

To recap the last post: I've tried to mimic the really helpful dotNet/C# feature "yield", which allows you to quickly write iterators without worrying about preserving state. Let me just make one thing clear: The implementation does not just collect all the yielded values into a list, and eventually returns the list. It actually returns the items one item at a time, saving both calculation time and allocation space. The implementation had to deal with two issues: First, it had to keep the state of all local variables. Second, it had to allow the method to return to the previous location the next time it is invoked. The first thing that comes to mind with these method-changing procedures is AOP - Aspect Oriented Programming. That option was not taken eventually, and I'll explain why. First, let's look at what the two issues mean, and what were my solutions for them. Keeping the state for local variables has a simple concept: Simply "promote" all of them to become class members. That way, no matter how they're used inside the method, their state is saved for the next time the method is invoked. For example, a Yielder implementation such as:

class MyYielder extends Yielder {
  private int[] arr = ...;
  public void yieldNextCore() {
    int total = 0;
    for (int i = 0; i < arr.length; i++) {
      yieldReturn(arr[i]);
    }
    yieldReturn(total);
  }
}

Would become:

class MyYielder extends Yielder {
  private int[] arr = ...;
  private int i;
  private int total;
  public void yieldNextCore() {
    total = 0;
    for (i = 0; i < arr.length; i++) {
      yieldReturn(arr[i]); 
    }
    yieldReturn(total);
  }
}

The second problem is a bit more tricky, but has a simple solution: in order to return to the previous location in a method, one had to create a simple state machine: keep a state member in memory, holding an integer. Mark each yieldReturn with an integer, and after each call make sure the member is set to that state and that the method exists immediately after it. After each of these, a label is placed. When all of that is done, a switch statement needs to be placed in the beginning of the method over the new state member, and jump to the corresponding label. To go back to the previous example, after the state machine elements are in place we get:

class MyYielder extends Yielder {
  private int[] arr = ...;
  private int i;
  private total;
  private byte state;
  public void yieldNextCore() {
    switch (state) {
      case 1: goto L1;
      case 2: goto L2;
    }
    total = 0;
    for (i = 0; i < arr.length; i++) {
      yieldReturn(arr[i]); 
      state = 1;
      return;
      L1:
    }
    yieldReturn(total);
    state = 2;
    return;
    L2:
  }
}

Some people might raise an eyebrow and say, "but Java doesn't have a goto statement!". These people would be both right and wrong. Java does have a goto statement; it's just not in use by compilers. Meaning, the previous code will not compile on anything. Eventually, that's the reason I chose not to go with the AOP solutions available today when I implemented this feature: I couldn't see a way to do it without making the state machine overly complex. To be honest, I don't know enough about the AOP solutions available, so if anyone cares to contradict me I'll be happy to learn from them. So, not having AOP as an option I was forced to go a bit lower, to "manual" bytecode manipulations. Since I'm still not BC guru, I decided to use the tools available today. The only tool I knew when I started this was Jakarta's BCEL - the ByteCode Engineering Library. I had a working version using BCEL, but eventually I switched the entire thing to ObjectWeb's ASM (which doesn't stand for anything), and I'm quite happy with that decision. In a later post I will compare the two and explain my decision to stop working with one and start working with the other. So far, any questions?