Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

Evolving Space Schema

We need a solution for online space schema evolution, and the details on this are murky:

We need to be able to migrate a space to a new version while keeping downtime to a minimum.

I have read this page http://www.gigaspaces.com/wiki/displa... and it is still unclear to me whether it will work for what we need. Since it is impossible to predict what new fields will be added to the model, we would not be able to employ the option of using placeholder fields. Since we expect to add fields for matching, we are not able to employ a payload field. Looking at externalizable implementation, I think it may work, but the limitations are not immediately apparent.

As I understand it, we can have two versions of a model object, and each space client may be compiled with a different version of the model object. The space would then need to be compiled with version 2 in order for this to work correctly, assuming version 2 is backwardly compatible. However, it is not clear whether it is possible to migrate a space using this technique. For example, one procedure would be to restart all the backup members with the new version. Then replication should migrate the model. This is assuming replication works with externalizable.

The data migration technique appears to allow for data inconsistencies to occur during the migration period due to changes to the space while it is online and this does not work for us.

Larry

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=2650]{quote}

asked 2008-10-10 15:04:18 -0600

larrychu gravatar image

updated 2013-08-08 09:52:00 -0600

jaissefsfex gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by » oldest newest most voted
0

Larry,

Migrating old version of the data to a new structure takes place in most cases under some window of time (planned downtime period). It involves in some cases database tables schema changes and business logic update.

During this period of time , the data within the IMDG transforms its shape to includes additional fields , merge several fields into a single one or remove fields. In some extreme cases the class hierarchy can be modified during this transformation.

A popular technique to deal with the above would be to dump the data into a CSV/XML file and later load it into new version of the class. This requires relevant business logic to construct the objects before written into the space.

Can’t you have such short downtime period to make the upgrade simple?

Addressing your comments about on-line data evolution: - Replication does not use the Externalizable methods of the object. The object values are replicated using a wrapper object (sync packet). - If you want to use Externalizable methods to deal with the data transformation , note that starting with 6.5 , the space will load the Class byte code from the client in case it does not have it within the classpath. In this case the Externalizable implementation should be able to cope with extra fields or missing fields. The extra data should be placed within some collection payload field that will allow you to construct newer versions of the class without loosing data. - If you have business logic and space collocated the Externalizable methods will not be called. This means you should call these explicitly or have a Base class for your classes that will include the conversions methods. Each class will have the relevant knowledge how to deal with extra or missing fields having the first version of the class as the baseline (Each version of the class will override the conversion methods). These conversions methods should be called before using the object data (after it was read) and before writing the object into the space (convert the object to its base line version).

Shay

answered 2008-10-11 15:53:12 -0600

shay hassidim gravatar image
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2008-10-10 15:04:18 -0600

Seen: 37 times

Last updated: Oct 11 '08