Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

FIFO Grouping Question

Hello,

I'm trying to use FIFO grouping, and I'm facing an issue, which might be related to how I'm doing it.

I have an abstract class Reservation:

@SpaceClass public abstract class Reservation {

private String flightID;
private Boolean processed;

@SpaceProperty
@SpaceFifoGroupingIndex
@SpaceIndex(type = SpaceIndexType.BASIC)
public Boolean getProcessed() {
    return processed;
}

@SpaceProperty
@SpaceFifoGroupingProperty
@SpaceFifoGroupingIndex
public String getFlightID() {
    return flightID;
}

...

}

which is extended by other classes, StandardReservation,SpecialReservation and GovtReservation.

I setup a polling container as follows:

@EventDriven @Polling (gigaSpace="myCluster",receiveTimeout=0,concurrentConsumers = 3, maxConcurrentConsumers = 5) @TransactionalEvent public class Processor implements EventExceptionHandler<reservation> {

private static Logger log = LoggerFactory.getLogger(Processor.class);

@ReceiveHandler
ReceiveOperationHandler receiveHandler() {
    SingleTakeReceiveOperationHandler handler = new SingleTakeReceiveOperationHandler();
    handler.setUseFifoGrouping(true);
    return handler;
}

public void onSuccess(Reservation reservation, GigaSpace gigaSpace,
                      TransactionStatus txStatus, Object source) throws RuntimeException {
    // process success
    log.info("Reservation " + Reservation + "processed correctly");
    txStatus.flush();
}

public void onException(ListenerExecutionFailedException exception, Reservation reservation,
                        GigaSpace gigaSpace, TransactionStatus txStatus, Object source) throws RuntimeException {
    // process failure
    log.error("error processing Reservation "+Reservation, exception);
    if(!txStatus.isRollbackOnly())
        txStatus.setRollbackOnly();
}

@ExceptionHandler
public EventExceptionHandler exceptionHandler() {
    return this;
}

@EventTemplate
SQLQuery<Reservation> unprocessedData() {
    SQLQuery<Reservation> template = new SQLQuery<Reservation>(Reservation.class, "processed = false");
    return template;
}

@SpaceDataEvent
public void processReservation(Reservation myReservation, GigaSpace gigaSpace){
    log.info("processing Reservation: " + myReservation);
}

@SpaceDataEvent
public Flight processStandardReservation(StandardReservation myReservation, GigaSpace gigaSpace){
    log.info("processing Standard Reservation: " + myReservation);
    Flight myFlight = takeFlight(myReservation, gigaSpace);
    myFlight.addStandardReservation(myReservation);
    myFlight.checkStatus();
    return myFlight;
}

@SpaceDataEvent
public Flight processSpecialReservation(SpecialReservation myReservation, GigaSpace gigaSpace){
    log.info("processing Special Reservation: " + myReservation);
    Flight myOtherObject = takeFlight(myObject, gigaSpace);
    myFlight.addSpecialReservation(myReservation);
    myFlight.checkStatus();
    return myFlight;
}

@SpaceDataEvent
public Flight processGovtReservation(GovtReservation myReservation, GigaSpace gigaSpace){
    log.info("processing Govt Reservation: " + myReservation);
    Flight myOtherObject = takeFlight(myObject, gigaSpace);
    myFlight.addGovtReservation(myReservation);
    myFlight.checkStatus();
    return myFlight;
}

public Flight takeFlight(Reservation myReservation, GigaSpace gigaSpace){
    SQLQuery<Flight> myQuery = new SQLQuery<Flight>(Flight.class,"flightID=? and status ='available'").setParameter(1,myReservation.getFlightID());
    Flight myXObject = gigaSpace.take(myQuery);
    if(myXObject == null)
        myXObject = new Flight();
    return myXObject;
}

}

in the pu.xml I have: <os-core:space id="space" url="/./mySpace" >="" <os-core:all-in-cache-policy=""/> </os-core:space>

<os-core:space id="cluster" url="jini://*/*/mySpace" lookup-groups="${lookup.groups}" lookup-locators="${lookup.locators}" no-write-lease="true"/>

<os-core:giga-space id="localSpace" space="space" tx-manager="transactionManager"/> <os-core:giga-space id="myCluster" space="cluster" tx-manager="transactionManager"/>

<tx:annotation-driven transaction-manager="transactionManager"/> <os-core:distributed-tx-manager id="transactionManager"/>

The PU is deployed in a partitioned-sync2backup schema. Object instances can arrive to any of the cluster members

If I deploy the PU with more than one partition, I see objects being processed in a different order than the order they were inserted in the grid (according to the logs), and I get more than one Flight object, each with some reservations inside, where I should have only one complete Flight.

If I deploy the PU with with only 1 partition, and set the polling container to have only 1 concurrent consumer, the behavior is what I expect (one Flight object containing all three StandardReservation, SpecialReservation and GovtReservation), .

We are using gigaspaces 9.0.2

is this setup correct to have FIFO grouping for processing all Reservation classes based on the Flight ID?

thank you for your help.

Kind Regards,

Raul

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=4044]{quote}

asked 2013-03-06 07:12:35 -0500

rdavidovich gravatar image

updated 2013-08-08 09:52:00 -0500

jaissefsfex gravatar image
edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
0

FIFO grouping work only with objects from the same class. The fact the class extends some other abstract class does not matter.
You should make sure objects are also partitioned correctly - for example have all objects from a specific type routed to the same partition. If you won't do that , each partition will consume the objects in FIFO , based based on their arrival to the collocated partition ignoring the other partitions.
Shay

answered 2013-03-07 16:30:26 -0500

shay hassidim gravatar image
edit flag offensive delete link more
0

Hello,

to simplify things up, my question is: does FIFO grouping work only with objects of the same class (ex1), or can it group instances of subclasses with the same grouping property (ex2)?

ex 1:
fifo group A:
StandardReservation {flightID : "flight1", seat 1}
StandardReservation {flightID : "flight1", seat 2}
StandardReservation {flightID : "flight1", seat 3}

fifo group B:
SpecialReservation {flightID : "flight1", seat 4}
SpecialReservation {flightID : "flight1", seat 5}
SpecialReservation {flightID : "flight1", seat 6}

fifo group C:
StandardReservation {flightID : "flight2", seat 1}
StandardReservation {flightID : "flight2", seat 2}
StandardReservation {flightID : "flight2", seat 3}

fifo group D:
SpecialReservation {flightID : "flight2", seat 4}
SpecialReservation {flightID : "flight2", seat 5}
SpecialReservation {flightID : "flight2", seat 6}

ex2:
fifo group A:
StandardReservation {flightID : "flight1", seat 1}
StandardReservation {flightID : "flight1", seat 2}
StandardReservation {flightID : "flight1", seat 3}
SpecialReservation {flightID : "flight1", seat 4}
SpecialReservation {flightID : "flight1", seat 5}
SpecialReservation {flightID : "flight1", seat 6}

fifo group B:
StandardReservation {flightID : "flight2", seat 1}
StandardReservation {flightID : "flight2", seat 2}
StandardReservation {flightID : "flight2", seat 3}
SpecialReservation {flightID : "flight2", seat 4}
SpecialReservation {flightID : "flight2", seat 5}
SpecialReservation {flightID : "flight2", seat 6}

Thank you very much for your help.

Kind Regards,

Raul

answered 2013-03-07 08:41:04 -0500

rdavidovich gravatar image
edit flag offensive delete link more

Comments

Hi Shay,

thank you for your reply. It's clear now.

I was thinking in that case to do a simple container object which has a routing and fifo grouping property, and contains the objects I actually want to process.

Something like:

@SpaceClass
public class ReservationContainer {

private String flightID;
    private Reservation reservation; //can be StandardReservation, SpecialReservation, etc
private Boolean processed;

@SpaceFifoGroupingProperty
    @SpaceFifoGroupingIndex
    @SpaceRouting
    public String getFlightID() {
        return flightID;
}

public Reservation getReservation() {
        return reservation;
}

public Boolean getProcessed() {
        return processed;
    }
}

@EventDriven
@Polling(receiveTimeout=0,concurrentConsumers = 3, maxConcurrentConsumers = 5)
@TransactionalEvent
public class ReservationProcessor implements EventExceptionHandler<ReservationContainer> {

@ReceiveHandler
    ReceiveOperationHandler receiveHandler() {
        SingleTakeReceiveOperationHandler handler = new SingleTakeReceiveOperationHandler();
        handler.setUseFifoGrouping(true);
        return handler;
}

public void onSuccess(ReservationContainer reservation, GigaSpace gigaSpace,
                          TransactionStatus txStatus, Object source) throws RuntimeException {
        // process success

        txStatus.flush();
}

public void onException(ListenerExecutionFailedException exception, ReservationContainer reservation,
                            GigaSpace gigaSpace, TransactionStatus txStatus, Object source) throws RuntimeException {
        // process failure

        if(!txStatus.isRollbackOnly())
            txStatus.setRollbackOnly();
}

@ExceptionHandler
    public EventExceptionHandler exceptionHandler() {
        return this;
}

@EventTemplate
    SQLQuery<ReservationContainer> unprocessedData() {
        SQLQuery<ReservationContainer> template = new SQLQuery<ReservationContainer>(ReservationContainer, "processed = false");
        return template;
}

@SpaceDataEvent
    public Reservation processReservation(ReservationContainer reservation, GigaSpace gigaSpace){
        //process reservation
        reservation.setProcessed(true);
        return reservation.getReservation();
    }
}

could this be a good solution?

Thank you very much for your help.

Raul

rdavidovich gravatar imagerdavidovich ( 2013-03-08 03:58:43 -0500 )edit

Conceptually , having a container class (ReservationContainer) should work.

shay hassidim gravatar imageshay hassidim ( 2013-03-08 06:19:47 -0500 )edit

works like a charm. Thanks!

rdavidovich gravatar imagerdavidovich ( 2013-03-08 09:26:36 -0500 )edit

Hi
I am using the exact copy of the codes in this thread to try to understand how to do FIFO grouping. Here is the issue I am having:
If I keep the following codes:

@ReceiveHandler
ReceiveOperationHandler receiveHandler() {
        SingleTakeReceiveOperationHandler handler = new SingleTakeReceiveOperationHandler();
        handler.setUseFifoGrouping(true);
        return handler;
}

in Processor.java and I'll get following exception from the IntegratedProcessingUnitContainer

Processing unit(s) started successfully
processor Setup of event listener invoker failed - trying to recover
java.lang.IllegalArgumentException: fifo grouping specified but no fifo grouping property defined
    at com.gigaspaces.internal.client.spaceproxy.actioninfo.ReadTakeProxyActionInfo.verifyFifoGroupsCallParams(ReadTakeProxyActionInfo.java:136)gs-runtime.jar:
    at com.gigaspaces.internal.client.spaceproxy.actioninfo.ReadTakeProxyActionInfo.<init>(ReadTakeProxyActionInfo.java:46)gs-runtime.jar:
    at com.gigaspaces.internal.client.spaceproxy.actions.AbstractSpaceProxyActionManager.take(AbstractSpaceProxyActionManager.java:362)gs-runtime.jar:
    at com.gigaspaces.internal.client.spaceproxy.AbstractSpaceProxy.take(AbstractSpaceProxy.java:325)gs-runtime.jar:
    at org.openspaces.core.DefaultGigaSpace.take(DefaultGigaSpace.java:855)gs-openspaces.jar:9.5.0-RELEASE
at org.openspaces.events.polling.receive.SingleTakeReceiveOperationHandler.doReceiveBlocking(SingleTakeReceiveOperationHandler.java:42)gs-openspaces.jar:9.5.0-RELEASE
    at org.openspaces.events.polling.receive.AbstractNonBlockingReceiveOperationHandler.receive(AbstractNonBlockingReceiveOperationHandler.java:60)gs-openspaces.jar:9.5.0-RELEASE
    at org.openspaces.events.polling.AbstractPollingEventListenerContainer.receiveEvent(AbstractPollingEventListenerContainer.java:356)gs-openspaces.jar:9.5.0-RELEASE
    at org.openspaces.events.polling.AbstractPollingEventListenerContainer.doReceiveAndExecute(AbstractPollingEventListenerContainer.java:278)gs-openspaces.jar:9.5.0-RELEASE
    at org.openspaces.events.polling.AbstractPollingEventListenerContainer.receiveAndExecute(AbstractPollingEventListenerContainer.java:255)gs-openspaces.jar:9.5.0-RELEASE
    at org.openspaces.events.polling.SimplePollingEventListenerContainer$AsyncEventListenerInvoker.invokeListener(SimplePollingEventListenerContainer.java:721)gs-openspaces.jar:9.5.0-RELEASE
    at org.openspaces.events.polling.SimplePollingEventListenerContainer$AsyncEventListenerInvoker.run(SimplePollingEventListenerContainer.java:662)gs-openspaces.jar:9.5.0-RELEASE
at java.lang.Thread.run(Unknown Source)17">:1.7.017

But if I comment out the lines of codes, the polling container can pull the Reservation objects from the space but, of course, the order of the objects is not FIFO.

Any suggestions? Your help will be greatly appreciated!

minshengshen gravatar imageminshengshen ( 2013-04-09 16:53:17 -0500 )edit

One more thing:
In the ReceiverHandler, if I set useFifoGrouping to false:

handler.setUseFifoGrouping(false);

Then the polling container can pull the message but again the order is not FIFO.

I tested it using XAP 9.0 and the lastest XAP 9.5.0. They behaive in the same way.

Thanks for any suggestion you may have!

minshengshen gravatar imageminshengshen ( 2013-04-09 16:57:26 -0500 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2013-03-06 07:12:35 -0500

Seen: 233 times

Last updated: Mar 07 '13