Chaotic Java

Recently we switched from Log4J to the java.util.logger package (for this entry it will be called the “Java Logger”). Why, you might wonder, and I don’t have a good reason to give other than the illusion the Java Logger package gives as being more standard. I would like to stress the word illusion for a minute here.

Since the title ruins any chance for a surprise ending, I will be brief, but give a few points about the tools I used. The starting point was that the entire back-end system slowed down significantly; no exceptions found in the logs, nothing stopped working, everything just got 20-40 times slower. So, what can be done?

One of the most classic patterns in software is the producer-consumer pattern. There is a module producing data, and a module reading it for further processing. Moreover, in order to achieve better performance, usually there are many consumer modules running on many different threads while the producer (or several producers) run on its own thread. This allows to distribute processing work between threads, and in the multi-core, multi-processor environment of today, between physical processors as well. Concurrent frameworks (such as java.util.concurrent) provide out-of-the-box solutions for these kind of problems.

Same pattern, multiple machines

This pattern also works well when wanting to distribute tasks between different physical machines as well. The producer machine somehow creates information, and offloads the task of processing that information to other machines. Since we want to distribute the work between different machines without coupling issues, the data is written to a physical disk and using some sort of a distribution system (e.g. JMS queues), a notification containing the full path to the data is sent. An available consumer receiving the notification can then read the data from the physical disk and process it. The following diagram illustrates this simple idea:

Continue reading »

After the two posts about garbage collection (basic and advanced) I started receiving questions regarding the works of the GC, and tips on how to use it properly. This post will be devoted to that subject, with some of the questions asked brought into highlight and answered using a lot of information coming from a great presentation Sun gave on the subject and with some knowledge acquired from the previous posts’ research material.

Continue reading »