Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

How does the Gigaspaces LRU eviction strategy cope with the G1 garbage collector?

Hi - if there is too much memory usage in a GSC then we want to unload data from a GigaSpace incrementally but in blocks of data that we choose, based on some criteria. LRU won't work for us because : a) You don't know what was evicted b) You end up with the data for a logical group being partially present c) You get to the threshold and Gigaspaces chooses the data to be evicted, not the app

So the plan is to have a background thread that monitors the heap usage percentage.
There will be a few memory usage thresholds (exact numbers configurable) 1. <50% Take no action 2. >50% Passive eviction (e.g. no users currently logged into that data segment) 3. >70% Aggressive eviction 4. >85% stop fast-moving data producers

We wish to use G1 because of the more predictable pause times. We have seen >30 second GC pauses in the past, that results in a split brain problem. We would also like to use -XX:+UseStringDeduplication (or String.intern), because there can be quite high levels of duplication.

However is probably going to cause a few problems that I'm guessing the GS LRU cache eviction must also suffer from, and we would like to know how Gigaspaces deals with this situation (perhaps with some better GC params?).

I presume (from the docs here: http://docs.gigaspaces.com/xap102adm/... ) that the LRU evictor is also working from a simple heap usage percentage. The doc talks about the use of space-config.engine.memory_usage.explicit-gc which is to get around that fact that the garbage collector might not be proactive enough, e.g. doesn't do a collection until 90% heap, by which time the LRU would have been fully evicted. So (I presume) you lose performance (risk significant pauses) from triggering GC manually using space-config.engine.memory_usage.explicit-gc.

I see that the recommended GC settings for LRU are to use CMS incremental mode, but this is deprecated in Java 8 with no replacement ( http://docs.oracle.com/javase/8/docs/... ) LRU JVM settings: -Xmx2g -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:ParallelGCThreads=8 -XX:+UseParNewGC -XX:+CMSIncrementalPacing -XX:MaxGCPauseMillis=1000

The Gigaspaces suggested Garbage collector is G1 here: http://docs.gigaspaces.com/sbp/moving...

So, after all that, I guess my basic questions are: How well does Gigaspaces LRU cache work well with G1? Does it sill need explicit-gc in order to not clear the cache unnecessarily? Does explicit-gc on G1 cause pause problems? Do explicit gc calls still try to keep to the G1 pause target? (-XX:MaxGCPauseMillis) or do you get STW problems with explicit GC calls?

Gigaspaces must have had to consider all these for LRU to work.

Thanks in advance

Paul.

asked 2016-02-19 07:25:30 -0500

Paul Hilliar gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

Paul,

You should consider the approach described here: http://docs.gigaspaces.com/sbp/custom...

This will give you good control what is evicted and when.

Another option is described here: http://docs.gigaspaces.com/xap102adm/...

In both cases evicting large number of objects too fast will cause the JVM to deal with plenty of memory that need to be reclaimed. This will require fast CPU and multi-core machine. Both CMS and G1 need free CPU and memory to function properly. In general to avoid long pauses you should always have about 30% of the heap free with 4 cores per GSC , one space instance per GSC, Avoid multiple space instances (primary , backup) running within the sale GSC.

2GB heap is way too small. You should consider 10GB when using CMP or G1.

space-config.engine.memory_usage.gc-before-shortage should be set to false.

See also space-config.engine.memory_usage.synchronous_eviction_watermark_percentage value. This parameter may cause pauses with wrong values.

To completely avoid pauses and data reload from the database on cache miss you should consider using the native persistence. In this case the space using heap and file (HDD / SSD) to store its data. See: http://docs.gigaspaces.com/xap102adm/...

If you need additional help please contact me directly: shay at gigaspaces.com

Shay

answered 2016-02-21 19:52:34 -0500

shay hassidim gravatar image
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2016-02-19 07:25:30 -0500

Seen: 362 times

Last updated: Feb 21 '16