Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

polling against remote space

Hi,

I have small issue. I have partitioned space:

<os-sla:sla cluster-schema="partitioned-sync2backup" number-of-instances="2" number-of-backups="0" max-instances-per-vm="4"/>

<os-core:giga-space-context/>

<os-events:annotation-support/>

<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
    <property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
    ....
</bean>

<bean id="sessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
    <property name="dataSource" ref="dataSource"/>
    <property name="annotatedClasses">
        <list>
    .....
        </list>
    </property>
    <property name="hibernateProperties">
        <props>
            <prop key="hibernate.dialect">org.hibernate.dialect.Oracle10gDialect</prop>
            <prop key="hibernate.cache.provider_class">org.hibernate.cache.NoCacheProvider</prop>
            <prop key="hibernate.cache.use_second_level_cache">false</prop>
            <prop key="hibernate.cache.use_query_cache">false</prop>
        </props>
    </property>
</bean>

<!-- Previously was StatelessHibernateExternalDataSource 
     org.openspaces.persistency.hibernate.DefaultHibernateExternalDataSource-->
<bean id="hibernateDataSource" class="eee.processor.CustomExternalDataSource">
    <property name="sessionFactory" ref="sessionFactory"/>
    <property name="fetchSize" value="1000"/>
    <property name="queryBlacklist">
        <list>
            <value>spaceId</value>
                    <value>state</value>
        </list>
    </property>
    <property name="initialLoadEntries">
        <list>
            .....
        </list>
    </property>
</bean>

<os-core:space id="space" url="/./space" mirror="true" external-data-source="hibernateDataSource" >
    <os-core:properties>
        <props>
            <!-- Use ALL IN CACHE = 1, LRU = 0  -->
            <prop key="space-config.engine.cache_policy">0</prop>
            <prop key="space-config.engine.initial_load">50</prop>
            <prop key="space-config.engine.cache_size">250000</prop>
            <prop key="cluster-config.cache-loader.external-data-source">true</prop>
            <prop key="space-config.external-data-source.usage">read-only</prop> 
            <prop key="cluster-config.cache-loader.central-data-source">true</prop> 
            <prop key="space-config.mirror-service.bulk-size">2000</prop>
            <prop key="space-config.mirror-service.interval-millis">5000</prop>
            <prop key="space-config.mirror-service.interval-opers">2000</prop>
        </props>
    </os-core:properties> 
</os-core:space>

<os-core:local-tx-manager id="transactionManager" space="space"/>

 <os-core:giga-space id="gigaSpace" space="space" tx-manager="transactionManager"/>

</beans>

and here is service:

<os-core:giga-space-context/>
<os-events:annotation-support />

<os-core:space id="space" url="jini://*/*/space"/>

<os-core:distributed-tx-manager id="transactionManager"/>

<os-core:giga-space id="gigaSpace" space="space" tx-manager="transactionManager"/>

<bean id="mitecStatusListener" class="ee.embeddedservices.procedures.OlapReplMitecStatus" depends-on="gigaSpace"/>

<os-events:polling-container id="TxMasterStatusPollingEventContainer" giga-space="gigaSpace" concurrent-consumers="6" max-concurrent-consumers="12">
    <os-events:receive-operation-handler>
        <bean class="org.openspaces.events.polling.receive.MultiTakeReceiveOperationHandler">
            <property name="nonBlocking" value="true" />
            <property name="nonBlockingFactor" value="10" />
        </bean>
    </os-events:receive-operation-handler>
    <!-- <os-core:sql-query where="state=0" class="ee.mappings.tables.staging.TxMasterStatus"/> -->
    <os-events:listener>
        <os-events:annotation-adapter>
            <os-events:delegate ref="mitecStatusListener"/>
        </os-events:annotation-adapter>
    </os-events:listener>
</os-events:polling-container>

</beans>

as you can see I removed query from polling container. Instead of it I create template in Service class:

@Override @EventTemplate public TxMasterStatus unprocessedData() { TxMasterStatus template = new TxMasterStatus(); template.setState(Status.UNPROCESSED); template.setTemplate(true); return template; }

You can notice line: template.setTemplate(true);. We use this construction to tell object that it will be template. this important because of methods getSpaceRouting() and getSpaceId() in Domain Object ee.mappings.tables.staging.TxMasterStatus

@Override
public String getSpaceId() {
    if (isTemplate) {
        return null;
    }
    return validateSpaceId(serialNo, timestamp2 != null ? timestamp2.toString() : null);
}

@Override
public String getSpaceRouting() {
    if (isTemplate) {
        return null;
    }
    if (serialNo == null) {
        return Long.toString(getSerialVersionUID());
    }
    return serialNo;
}

Not sure if I explained everything clear.

Problem is that now polling works just on GSC where service exists. I can of course deploy service to all GSCs but I wonder what about queries from inside service? Will they work as well just against local space?

Using line:

<os-core:sql-query where="state=0" class="ee.mappings.tables.staging.TxMasterStatus"/>

as we had before worked against all GSCs but we changed it due to fact that polling with template matching is expected to be faster (as I found somewhere on wiki).

Best regards, Mateusz

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=2979]{quote}

asked 2009-03-27 02:26:54 -0500

mnemos gravatar image

updated 2013-08-08 09:52:00 -0500

jaissefsfex gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

I guess you have seen this post when decided to use batch operations with your polling container:
http://blog.gigaspaces.com/2008/10/03/how-to-implement-my-processor-the-polling-container-benchmark/

Still , there are some issues with your configuration:
- Your polling container running remotely. This means you will be using distributed transactions that will impact the performance.
- You are running in LRU + Mirror - This means that if you query the space you might have short window of time where there in case there would not be a matching object in the space (since it was deleted) , but the data was not deleted yet from the database , you will have data returned back to the client which wrong behavior.
- You are running in LRU + EDS + readMultiple/takeMultiple - This means it will be very hard avoiding querying the database each time. You will not be able to use data in memory with such config. See details here: //question/3495/eds-with-lru/?comment=3498#comment-3498. To avoid this you should use SingleTakeReceiveOperationHandler.

I strongly suggesting moving to ALLINCACHE or build a Query service that will have some business logic that will avoid accessing the database with each query.

To answer your question - as long as you use remote space URL in clustered mode , the queries will be conducted against all partitions. It does not matter if the polling container and the space running within the same GSC or different GSCs.
Only when using embedded space URL in none clustered mode your queries will be conducted against the local space.

Shay

answered 2009-03-27 07:30:58 -0500

shay hassidim gravatar image
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2009-03-27 02:26:54 -0500

Seen: 116 times

Last updated: Mar 27 '09