Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

Problem with reading from External Data Source

Hi,

I have custom made External Data Source that implements DataProvider and BulkDatePersister. Everything works fine, until I switch to LRU. On start I load only half of the data, then I read data that is in the database, but not in the cache, this returns _null_. However if I do write on the same data, it is found,extracted from the database and then updated. I check and the iterator method is not called at all during my tests!!! Any ideas what could be the reason? If you have seen such behavior, how did you fix it?

Thanks, Tomo

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=3981]{quote}

asked 2012-12-06 10:29:57 -0500

tomosimeonov gravatar image

updated 2013-08-08 09:52:00 -0500

jaissefsfex gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

The EDS has a different behavior when moving from ALL-IN-Cache to LRU. See:
http://wiki.gigaspaces.com/wiki/display/XAP91/ExternalDataSourceAPI#ExternalDataSourceAPI-APIMatrix

The space-config.engine.initialload control the amount of objects (percentage of space-config.engine.cachesize) to be loaded into the space when running in LRU, See:
http://wiki.gigaspaces.com/wiki/display/XAP91/LRU-CachePolicy#LRU-CachePolicy-ReloadingData

Which API you use to read data from the Space?
Do you have @SpaceID autogenerate true or false?

Shay

answered 2012-12-06 12:03:34 -0500

shay hassidim gravatar image
edit flag offensive delete link more

Comments

I am using Java API and my EDS is written in Scala.
The objects are with "autoGenerate=false".

My EDS uses SpaceDocuments, but the read method is called successfully, thus I don't think this is a problem.

Tomo

tomosimeonov gravatar imagetomosimeonov ( 2012-12-06 13:52:21 -0500 )edit

It might be related to missing meta data. How do you specify the Document meta data ? via the pu.xml?
Can you post your pu.xml?

shay hassidim gravatar imageshay hassidim ( 2012-12-06 16:01:39 -0500 )edit

I cannot provide the full pu.xml only the space configuration. Also I am using Pojos to query the Space(template with setted one field).
 This is how my space is configured:
<os-core:space id="space" url="/./mySpace"
        external-data-source="EDS" schema="persistent" mirror="true">
        <os-core:properties>
            <props>
                <prop key="space-config.external-data-source.data-class">com.gigaspaces.document.SpaceDocument</prop>
                <prop key="space-config.external-data-source.supports-inheritance">false</prop>
                <prop key="space-config.external-data-source.shared-iterator.enabled">false</prop>
                <prop key="cluster-config.cache-loader.external-data-source">true</prop>
                <prop key="cluster-config.cache-loader.central-data-source">true</prop>
                <prop key="cluster-config.mirror-service.bulk-size">2000</prop>
                <prop key="cluster-config.mirror-service.interval-millis">10000</prop>
                <prop key="cluster-config.mirror-service.interval-opers">10000</prop>
                <prop key="space-config.engine.memory_usage.enabled">true</prop>
                <prop key="space-config.engine.cache_policy">0</prop>
                <prop key="space-config.engine.initial_load">1</prop>
                <prop key="space-config.engine.cache_size">500</prop>
                <prop key="space-config.external-data-source.usage">read-only</prop>
                <prop key="cluster-config.mirror-service.url">
                    jini://*/mirror-service_container/mirror-service
                </prop>
            </props>
        </os-core:properties>
</os-core:space>

tomosimeonov gravatar imagetomosimeonov ( 2012-12-07 04:45:42 -0500 )edit

Tomo,

Can you please specify the Space Document meta data using the pu.xml? See below example:

<os-core:space id="space" url="/./space" >
      <os-core:space-type type-name="Product" >
        <os-core:id property="CatalogNumber"/>
        <os-core:routing property="Category"/>
        <os-core:basic-index path="Name"/>
        <os-core:extended-index path="Price"/>
      </os-core:space-type>
</os-core:space>

Shay

shay hassidim gravatar imageshay hassidim ( 2012-12-07 07:41:51 -0500 )edit

Shay,

I am using only pojos in the space with appropriate attributes for the automatic generation of SpaceTypeDescriptor for the Pojo, thus I do not have to configure Space Document meta data in the pu.xml
My EDS uses only SpaceDocuments, but according to http://wiki.gigaspaces.com/wiki/display/XAP9/Document-POJO+Interoperability it won't be a problem.

Tomo

tomosimeonov gravatar imagetomosimeonov ( 2012-12-07 07:48:25 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2012-12-06 10:29:57 -0500

Seen: 124 times

Last updated: Dec 06 '12