Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

Async persistanci speed

I have been playing with space and I have tried to load huge number of data into async persistent data grid

my feeder feeds space with writemultiple method in batch of 100 objects (and I was about to create 6M of oobjetcs)

space is configured as sync2backup witjh mirror persistency and LRU

I have setup LRU according to guide id documentation

<os-core:space id="space" url="/./space" schema="persistent"
    mirror="true" external-data-source="hibernateDataSource">
    <os-core:properties>
        <props>
            <prop key="space-config.engine.cache_policy">0</prop>
            <prop key="space-config.engine.cache_size">80000</prop>
            <prop key="space-config.engine.memory_usage.enabled">true</prop>
            <prop key="space-config.engine.memory_usage.high_watermark_percentage">70</prop>
            <prop key="space-config.engine.memory_usage.write_only_block_percentage">68</prop>
            <prop key="space-config.engine.memory_usage.write_only_check_percentage">65</prop>
            <prop key="space-config.engine.memory_usage.low_watermark_percentage">60</prop>
            <prop key="space-config.engine.memory_usage.eviction_batch_size">2000</prop>
            <prop key="space-config.engine.memory_usage.retry_count">100</prop>
            <prop key="space-config.engine.memory_usage.explicit-gc">false</prop>
            <prop key="space-config.engine.memory_usage.retry_yield_time">4000</prop>
            <prop key="space-config.external-data-source.usage">read-only</prop>
            <prop key="cluster-config.cache-loader.external-data-source">true</prop>
            <prop key="cluster-config.cache-loader.central-data-source">true</prop>
        </props>
    </os-core:properties>
</os-core:space>

But when I reach about 1M of objetcs written writinnig get on hold and I have found from memory manager that it was evicting (and emptying memory) slowly (5mb each minuute or so) (max memory 1GB)

I have looked into my DB and tehre was only a few rows (tens of thousands insetad of hundreds of them)

With show sql turned on I have seen that thet there were allaway Selects followed by inserts, but there were allway lag then - nothing was happening. I left my PC for night in the morning everything was persisted.

everythink else was on default configuration. I think there is some problem with that, what do I have to change to achieve faster evictiona and faster persistance?

Edited by: Milos Broulik on May 28, 2009 5:40 AM

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=3109]{quote}

asked 2009-05-28 05:39:41 -0600

hurda gravatar image

updated 2013-08-08 09:52:00 -0600

jaissefsfex gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

Take a look on :
http://www.gigaspaces.com/wiki/display/XAP66/AsyncPersistency-Mirror

You might want to use some of the optimization tips described as part of the Optimizing the Mirror Activity section.

Your space-config.engine.memoryusage.retryyield_time is too large. Make it 2000 ms.

XAP 7.0 eviction mechanism has been improved and works faster than XAP 6.6..

Shay

answered 2009-05-28 07:27:05 -0600

shay hassidim gravatar image
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2009-05-28 05:39:41 -0600

Seen: 49 times

Last updated: May 28 '09