My previous article on java references introduced the java referencing model with the nice features that it can include. One of them the soft-reference was as I explained very useful for cache and buffers in a VM.
In this article I will discuss the particular effect that using soft-references can have on a java cloud or data-grid. We will particularly focus on GigaSpaces as it is the currently more active solution as a Cloud-Computing end-to-end application server and build in java.
When a developer uses soft reference, he allows the JVM to take control on a part of the heap and manage it just the way she likes. The developer in that case really has no control at all. Even if the Garbage Collection cycles are controlled by JVM the developer can try to invoke GC (unless the -XX:+DisableExplicitGC is on but we really not consider this as a good practise). As all of us know calling an explicit GC is not a hundred percent guaranty of having a real full GC cycle but certainly help to reduce the heap in different scenarios. However this will for sure have no impact at all on the soft-reference usage.
So “why can this be bad?” After all the goal of soft-reference is too be a cache, use the heap but still guarantee the security of the JVM (no risk of OutOfMemoryException). Basically if there is no direct impact on most of application we can wonder what is the impact of the soft-reference on heap monitoring. Indeed the objects held by soft-reference are here but are not consuming heap in a “risky way”. Basically they cannot be considered as responsible for any memory leak or even “real usage”.
However a monitoring of the heap may make people think that the heap is really used and that basically they have a risk for their application. If people can be educated and can find out that memory is related to soft-reference, there is no statistics in the JVM that clearly shows it.
The problem is even more relevant when an application uses a self-monitoring to take actions at given SLA. This is basically what will and should happen in many cases on a java cloud. We can this way enter in SLA being triggered without any good reason.
This is why soft-reference usage must be used with caution in those environments.
If we look at the example of GigaSpaces:
-As a cloud provider GigaSpaces help you to monitor and to define SLAs based on the heap and memory usage. As we already saw this can be really dangerous (depending on your SLA action) if many soft references are used. We can run in scenarios where some part of your application will not be relocated on a VM because of these soft-references.– I want to really emphasis that such scenarios will mostly be bad usage of soft-reference in the application code or sometimes some really extreme situation that I will detail later.
-As a data-grid vendor, GigaSpaces has a very nice memory protection feature that allows the Spaces to protect themselves from OutOfMemory by cancelling write operations performed on it. These protections are based on memory SLAs (see http://www.gigaspaces.com/wiki/display/XAP66/Memory+Management+Facility#MemoryManagementFacility-MemoryUsage) and implies the same impact when the memory is consumed by soft-references.
Again the scenarios that will push to run in such situations are rare and quite extreme. But let details one of them that can happen with GigaSpaces.
The GigaSpaces software uses soft-reference in his communication protocol LRMI to manage buffers. In most situations this will never cause any high heap consumption. However when many client connect to the space to request concurrently big object or many small objects we may run in a situation where many buffers are created and will consume memory. This kind of scenario may happen typically in Grid Computing scenarios, when the whole Grid (we encounter mostly DataSynapse or Platform systems) tries to connect and perform request on a single data-grid node.
The question that comes next is: “so what can I do in such a case?”
The truth is that there is no easy answer to this question. The parameter -XX:SoftRefLRUPolicyMSPerMB help to tune the soft-reference garbaging in SUN JVM. However this parameter is not part of the java specification. It does indeed not exist in other JVM.
I will therefore recommend using in such use-case a SUN JVM to be able to perform the tuning that my application will require.
I don’t know if any of you have been involved in similar situations, and I wonder how you managed to handle this.