Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

Multi-Cluster Architecture over WAN using a Gateway Processing Unit

I have read he paragraph "Multi-Cluster Architecture over WAN using a Gateway Processing Unit" at http://www.gigaspaces.com/wiki/displa...

I don't understand what gateway PU does. Does the gateway PU notify the the change of Object in the embedded cluster and then does the same change the the Object on the other side? How does the performance reduce when compared with using replication group using static cluster configure file?

I think that this approve will cause resonance which this mentioned later in the same page. update object in site A -> notified by PU in A -> update object in site B -> notified by PU in B -> update object in site A -> ... How should we solve this problem?

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=2966]{quote}

asked 2009-03-23 03:20:44 -0500

cwtsui gravatar image

updated 2013-08-08 09:52:00 -0500

jaissefsfex gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted
0

The preferred approach in many cases would be to use the Mirror Gateway approach. This is more robust pattern.
Resonance is avoided by having an origin field within each object that is set by the Mirror prior transferring the object to the remote site. This allows the other Mirror at the remote site to filter out objects which are “mistakenly” sent to their origin site.

The Mirror Gateway approach used when:
- There are asymmetric clusters at each site – i.e. 2 partitions at one site and 3 partitions at the remote one
- There is a need to have local backup of the data within the site
- There are special requirements how to transfer the data such as special filtering or transformation before the data is sent to the remote site
- You don’t want to de-couple multiple sites via one single cluster , but would like each one to have its own cluster and space settings.
- When you would like to provide high quality data transformation over the WAN. For example use content routers such as the Solace device which can transfer data in very efficient manner over the WAN. You can read about this here:
http://www.gigaspaces.com/node/1138
http://www.gigaspaces.com/wan
http://www.solacesystems.com/solutions/cross-industry/distributed-space-bridging

Shay

answered 2009-03-24 01:16:44 -0500

shay hassidim gravatar image
edit flag offensive delete link more

Comments

Thank you for your attention.

In fact, I have a project about replicating data via LAN. I only need to replicate data from a Master spaces to service spaces. Service spaces will be deployed at the beginning of the day. At this time, I needed to replicate the part of the data to the service spaces.

In the paragraph "Multi-Cluster Architecture Over the WAN using a Mirror Gateway", I see Mirror Gateway that you mentioned. Is the mirror gateway equal to mirror used in persistence schema? If yes, what is the set up in pu.xml? Do we need to implement a new External Data Source?

Edited by: Tsui Chi wing on Mar 24, 2009 9:29 PM

cwtsui gravatar imagecwtsui ( 2009-03-24 20:45:45 -0500 )edit

You will need to implement the BulkDataPersister and the ManagedDataSource with your "Mirror Gateway". This will be getting replication events from the primary spaces and send these to the other site.
The GigaSpaces-Solace solution provides such implementation out of the box.
Shay

shay hassidim gravatar imageshay hassidim ( 2009-03-25 10:26:16 -0500 )edit

Maybe my last post is not clear enough.

I have a non-stop Master data-grid. Each day the Service data-grid will be started and then it will replicate part of the data to their own data-grid. At the end of the day, the Service data-grid will be undeployed.

BulkDataPersister seems will update the data from Service data-grid to Master data-grid. This is not what I want. I should use something like DataProvider but the mirror PU will receive too many objects (I mean not enough memory to hold them) if I use "new Data()" to get all needed data at once. How to get the object part by part without receiving repeated objects or missing any objects?

Can a mirror do such thing? How should I implement it?

Edited by: Tsui Chi wing on Mar 26, 2009 11:05 PM

cwtsui gravatar imagecwtsui ( 2009-03-26 22:56:46 -0500 )edit

See below very basic skeleton for mirror pu.xml and Mirror implementation you should use as a bases for the Mirror gateway. You will need to implement some filtering business logic that will make sure you will not have resonance affect (in case you run Mirror gateway at each site) , implement some booting mechanism that will sync newly started site , get proxy to the other site , and perform the relevant space operations against the remote site from the Mirror implementation , etc.If you need more details how to implement the above please contact me offline.

{code:title=mirror pu.xml} <bean id="myMirror" class="com.mymirror.MyMirror"/> <os-core:space id="mirror" url="/./mirror-service" schema="mirror" external-data-source="myMirror"/> {code}

{code:title=mirror skeleton} public class MyMirror implements ManagedDataSource , BulkDataPersister{

@Override
public void init(Properties arg0) throws DataSourceException {
    System.out.println(" --------- My Mirror Started ---------");
}

@Override
public DataIterator initialLoad() throws DataSourceException {
    return null;
}

@Override
public void shutdown() throws DataSourceException {
    // TODO Auto-generated method stub
}

@Override
public void executeBulk(List<BulkItem> bulkItems) throws DataSourceException {
    try {
        for (BulkItem bulkItem : bulkItems) {
            Object entry = bulkItem.getItem();
            switch (bulkItem.getOperation()) {
                case BulkItem.REMOVE:
                     // Send Remove command to the remote sites
                    System.out.println("REMOVE CALLED");
                    break;

                case BulkItem.WRITE:
                    // Send Write command to the remote sites
                    System.out.println("WRITE CALLED");
                    break;

                case BulkItem.UPDATE:
                    // Send Update command to the remote sites
                    System.out.println("UPDATE CALLED");
                    break;
                default:
                    break;
            }
        }
    } catch (Exception e) {
        throw new DataSourceException("Failed to execute bulk operation\n\n"+ e) ;
    } finally {
        // Close Session…
    }
}

}

{code}

shay hassidim gravatar imageshay hassidim ( 2009-03-27 00:38:11 -0500 )edit

Thank you for the source code.

I get the UID from the space filter but I dont know want it means. How to get the original container?

ClassName : cwtsui.common.Data UID : 294542918^18^4--0^0^0

ClassName : cwtsui.feeder.TestObject UID : 122913581^24^E^1238490384743^5

Edited by: Tsui Chi wing on Mar 31, 2009 4:08 AM

cwtsui gravatar imagecwtsui ( 2009-03-31 00:45:53 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2009-03-23 03:20:44 -0500

Seen: 78 times

Last updated: Mar 24 '09