Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

Delays in polling

We noticed some delays in processing messages in a polling container (with GS 9.0.2) and I made the following code in an attempt to pinpoint where it happens:

@TriggerHandler
TriggerOperationHandler triggerHandler() {
    ReadTriggerOperationHandler triggerHandler = new ReadTriggerOperationHandler() {
        @Override
        public Object triggerReceive(Object template, GigaSpace gigaSpace, long receiveTimeout) throws DataAccessException {
            final Object o = super.triggerReceive(template, gigaSpace, receiveTimeout);
            log.info("Triggered "+o);
            return o;
        }
    };
    return triggerHandler;
}

@ReceiveHandler
ReceiveOperationHandler receiveHandler() {
    return new SingleReadReceiveOperationHandler() {
        @Override
        protected Object doReceiveBlocking(Object template, GigaSpace gigaSpace, long receiveTimeout) throws DataAccessException {
            Object o = super.doReceiveBlocking(template, gigaSpace, receiveTimeout);
            log.info("ReceivedB "+o);
            return o;
        }

        @Override
        protected Object doReceiveNonBlocking(Object template, GigaSpace gigaSpace) throws DataAccessException {
            final Object o = super.doReceiveNonBlocking(template, gigaSpace);
            log.info("ReceivedN "+o);
            return o;
        }
    };
}
@SpaceDataEvent
public MessageWrapper processMessage(final MessageWrapper message, final GigaSpace gigaSpace) {
    log.info("Entered "+(++i)+" processing " + message);

and got the following after running for a while:

05 Nov 2015 16:40:31,854 JST INFO  [GS-orderMessageProcessor-1] [OrderMessageProcessor]  Triggered MessageWrapper{7004.T}
05 Nov 2015 16:40:31,915 JST INFO  [GS-orderMessageProcessor-1] [OrderMessageProcessor]  ReceivedB MessageWrapper{7004.T}
05 Nov 2015 16:40:31,915 JST INFO  [GS-orderMessageProcessor-1] [OrderMessageProcessor]  Entered 234247 processing MessageWrapper{7004.T}

We were using the @ReceiveHandler before in order to use Read but we had delays even before when we were using the default polling receive handler. MessageWrapper is processed FIFO mode with a basic index on some field that has no relation with timing - a random message id revived on inbound that we use in a different query. I added the @TriggerHandler only for this logging experiment.

Questions:

  • is this test relevant anyway?

  • why is there the delay between Triggerd and Received increasing with time/number of processed messages?

  • what can we do about it? (without switching approach to passArrayAsIs or concurrentConsummers or anything that different)

Additional question: can we use MultiReadReceiveOperationHandler while still processing with @SpaceDataEvent in a one by one manner? Any hope for latency improvement by doing that?

Thanks

asked 2015-11-05 02:13:01 -0500

lukeh gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0
  • The Trigger Receive Operation attempts to bypass the processing that is involved with starting and rolling back a transaction for each unsuccessful receive operation. It checks to see if there are valid items outside of transaction before going into polling/event processing. If the delays you are seeing are not due to delays with rolling back unsuccessful receive operations, then this may not be relevant.

  • Without having the details it is hard to determine why there are delays. With regards to the polling container, please consider the following which affect the polling container performance:

Feeder speed, Feeder write batch size (in case you are using batch write), Feeder message size, Feeder message complexity; more fields = more serialization and marshaling involved, Scope of transactions used by feeder and or processor, The type and number of destructive operations used by the processor polling container, Processor polling container ReceiveOperationHandler size, Network and CPU speed

  • In addition, please check the following: Health of the system, cpu, memory, jvm properly tuned, is any garbage collection collection being done, etc.

  • Switching to PassArrayAsIs will allow you to process the data in batch and may improve overall throughput rate. If a MultiReadReceiveOperationHandle is used and @SpaceDataEvent still processes one by one, you may not be getting the full benefits of running in batch. Without having clear idea of where the bottleneck is, it is difficult to say what approach will help.

If you are still having problems with this and have a support account, please consider opening a support ticket.

Resources: http://docs.gigaspaces.com/xap102/pol...

http://test.4hilton.com/how-to-implem...

HTH,

Dixson

answered 2015-11-05 10:22:16 -0500

Dixson Huie gravatar image
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2015-11-05 02:13:01 -0500

Seen: 362 times

Last updated: Nov 05 '15