Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

help with linear scaling in XAP

Hi, I am solving a problem how to linear scale my OLTP application. Payment orders are generated in webservices or an application logic called from web forms. On my ordinary PC I can process a few orders per second. What if I need 1000 per second? I came accross XAP and read some documentation and samples. I conclude that I can easily put the PaymentProcessor in a "Processing Unit" which can be loadbalanced by XAP runtime. But I assume that it implies distributed transaction and it further implies central transaction manager (JINI) -> a potential bottleneck and expensive RPC calls too. Does it also imply that I should mark account's name getter with SpaceRouting annotation? Is there any other way how to linear scale such kind of application e.g. without expensive distributed transactions? I'm a newbie here, maybe I miss some important feature of XAP or EDG...

In a nutshell here is a demonstrative "pseudocode":

class Account { String name; int balance; }

class PaymentOrder { String receptorName; String donorName; int amount; }

class PaymentProcessor { // autowired // works over hibernate AccountDao accountDao;

@Transaction
void processPayment(PaymentOrder order) {
  Account donor = accountDao.getByName(order.donor, READ_EXCLUSIVE);
  Account receptor = accountdAo.getByName(order.donor, READ_EXCLUSIVE);
  if (donor.balance - order.amount < MIN_GLOBAL_BALANCE) throw new InsufficientMoneyException(); 
  if (receptor.balance + order.amount > MAX_GLOBAL_BALANCE) throw new AccountOverflowExecption();

  donor.balance = donor.balance + amount;
  receptor.balance = receptor.balance + amount; 

  accountDao.update(donor);
  accountDao.update(receptor);

}  // end of transaction

}

Thank you very much for your help and hints.

PS. Sorry for my english.

Edited by: Martin Smid on Aug 21, 2008 10:44 AM

Edited by: Martin Smid on Aug 21, 2008 10:46 AM

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=2547]{quote}

asked 2008-08-21 10:39:00 -0500

smid gravatar image

updated 2013-08-08 09:52:00 -0500

jaissefsfex gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

To avoid distributed transaction usage you should have the same routing field value for the donor and the receptor objects.
This how the transaction will be done with a single space and not multiple spaces when running in partitioned space topology.

Shay

answered 2008-08-21 14:55:50 -0500

shay hassidim gravatar image
edit flag offensive delete link more

Comments

Thank you for your reply.

I guess that it means that all acounts must reside in the single space (on single HW) and PaymentProcessors could run on many computers. I assume that under the cover of XAP all reading/locking of accounts will be routed to the single space (e.g. via RPC). So the single space could be potential bottleneck like a database system because it finally must handle all transactions, isn't it?

So I have probably three choices: - let the database handle the transactions - let the distributed transaction manager handle transactions and employ asynchronous persistency to avoid waiting to a database - host all acounts in a single space and maybe also use asynchronous persistency

The first choice is probably the worst (IO operations is killer) and expensive. I had experience with Oracle 8i and the performance of a similar app was very poor. Definitely not a linear scaling architecture

The second one depends in an overhead of the manager. It might be a good way but I have no real experience with distributed transactions. The last solution relays on the performance of the single space. Is it more efficient than distributed manager for my case? What architecture/configuratioin would you suggest to solve this?

Unfortunately I couldn't find a true linear or near linear scaling architecture of my problem. Probably such issue can't be decantralized. I mean that there must be always a single authority (database, space, transaction manager) which must be involved in transactions. I wish I am wrong... .

Thank you again for your help.

smid gravatar imagesmid ( 2008-08-21 17:03:58 -0500 )edit

Martin,

That's an impotent topic. I've tried to give you a broader perspective here. So it will be a bit long post :-)

The whole idea with scaling the system is to have plenty of PaymentProcessor instances running across multiple machines , each of these having a collocated space storing a subset of the data where the data is partitioned across these collocated spaces based on some routing field value.

If you have a transaction which need multiple objects for its processing you should have the same routing value placed into these objects. This how they will land in the same partition avoiding the need to perform distributed transaction when updating these , or to perform remote call from another partition when reading these.

See full example here:
http://www.gigaspaces.com/wiki/display/OLH/ParallelProcessingTutorial-OrderManagement

Still , in some cases , there is a need to have a "shared data" that need to be accessed by all partitions. In this case a remote call will be used.
In such a case your PaymentProcessor will hold a space proxy to its collocated space and another one to the remote clustered space. In some cases you could have a local cache to store the shared data. This will avoid the extra remote read when fetching the same object again.

In other cases there is no logic to place the business logic in the same process where the data is located , but have it running in a separate process. Some users doing that , but having these processes accessing only the partition running on the same machine.

If you end up with a need to use distributed transaction remember these are much faster and lighted with in memory data grid than with a database. Since there is no disk involved these are relatively fast. So I would not recommend to perform the transaction against the database. In all cases the transactions should be delegated to the database in async manner via the Mirror service.

I suggest you to review the following when considering to collocate or not collocate your data with the business logic.
Before actually collocating the required state and business logic, you should take the following into consideration:

1. Is your business logic designed to process incoming data events without accessing remote data located in other partitions?
2. Is your data model designed to support stickiness - to be routed to the same logical partition based on its content?
3. Is your processing designed to cope with a self-healing mechanism that will restart the failed instance somewhere else on the network, allowing the system to continue and function without disruption as long as there are machines available to run the application?
4. What is the amount of work involved to process the incoming transaction? Does it involve lots of I/O operations accessing many different remote resources?

To help you make the right decision when deploying your application see below some guidelines that correlate to the above ...(more)

shay hassidim gravatar imageshay hassidim ( 2008-08-22 08:22:02 -0500 )edit

Shay, I am very appreciate your comprehensive advices. Thank you.

smid gravatar imagesmid ( 2008-08-22 16:29:23 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2008-08-21 10:39:00 -0500

Seen: 42 times

Last updated: Aug 21 '08