Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

We get the below error in our environment repeatedly

Hi,

We get a lot of below exception in our environment. The stack trace doesn't say from where it is being thrown and whats the issue.

Can someone please help?

03092009 17:35:09.333 SEVERE [com.gigaspaces.lrmi]: Failed to invoke one-way method: public abstract void net.jini.core.event.RemoteEventListener.notify(net. jini.core.event.RemoteEvent) throws net.jini.core.event.UnknownEventException,java.rmi.RemoteException java.rmi.NoSuchObjectException: Object 1251911743948894852 not found in object registry at com.gigaspaces.lrmi.LRMIRuntime.invoked(LRMIRuntime.java:361) at com.gigaspaces.lrmi.nio.Pivot.consumeAndHandleRequest(Pivot.java:371) at com.gigaspaces.lrmi.nio.Pivot.handleRequest(Pivot.java:459) at com.gigaspaces.lrmi.nio.Pivot$ChannelEntryTask.run(Pivot.java:149) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269) at java.util.concurrent.FutureTask.run(FutureTask.java:123) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) at java.lang.Thread.run(Thread.java:595)

regards, Tarun Naithani

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=3223]{quote}

asked 2009-09-03 04:45:13 -0600

tarun_naithani gravatar image

updated 2013-08-08 09:52:00 -0600

jaissefsfex gravatar image
edit retag flag offensive close merge delete

2 Answers

Sort by ยป oldest newest most voted
0

You might want to do the following:
- increase the file descriptors limit to 32000 (ulimit -n 32000)
- use batch notifications
- use 2 way notification mode

Shay

answered 2009-09-05 04:31:32 -0600

shay hassidim gravatar image
edit flag offensive delete link more

Comments

Hi Shay,

The file handlers can be increased, however it seems like they are being used up quickly. Our current limit is more than 8000. We can change notification if only we knew which one was causing the issue as there are many notifiers registered for that space.

I think the main reason we are getting stuck is because we are unable to understand the source of exception ot any other details which can help us pin point the problem. As of now we are running diagonsis by taking down one component at a time and studying the behaviour.

My theory as of now is space is trying to call an event listener which has gone missing and thus we get this exceptions. In any case space should close sockets in case of such error to avoid taking up file handlers.

It would be very helpful if you could give more information about the exception and circumstances in which it occurs.

regards, Tarun Naithani

tarun_naithani gravatar imagetarun_naithani ( 2009-09-07 00:29:40 -0600 )edit

Yes - if you can reproduce the problem you might need to strip down the application to pin-point the exact piece that cause the "leak". Still, having file descriptors as 8000 might be too small. A value of 32,000 is the suggested value when having activities such as notifications that opens up large amount of connections (remember the space using multiple threads to send the notifications back to the client side in case of single notifications).

If you have a "slow client" that performs heavy duty activity when the event is triggered , you might want to fork the activity into another worker thread. The open spaces notify container support this.

If the client is not there and died abnormally or can't handle the load due-to high CPU utilization, the notification registration will be canceled automatically after some time. This might take some time and might exhaust the notification thread pool and connection resources (file descriptors). In such a case , you might need to turn on the slow consumer option. See:
http://www.gigaspaces.com/wiki/display/XAP7/SlowConsumer

Shay

shay hassidim gravatar imageshay hassidim ( 2009-09-08 14:25:00 -0600 )edit

Hi Shay,

Thanks for the inputs. I will make the changes and investigate further.

regards,
Tarun Naithani

tarun_naithani gravatar imagetarun_naithani ( 2009-09-08 23:58:02 -0600 )edit
0

Even we used to get those messages a lot when we moved to 6.6.x. Gigaspace people confirmed us that they are b'cause of some unwanted stub code hanging in the classes. So not to worry, those are just a ignorable message.

Thanks Venkat

answered 2009-09-03 07:50:34 -0600

venkatg gravatar image
edit flag offensive delete link more

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2009-09-03 04:45:13 -0600

Seen: 111 times

Last updated: Sep 05 '09