Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

java.nio.channels.ClosedChannelException

I have an application that is suddenly behaving strangely, which used to work just fine... However, now, it hangs up when writing a simple entry to the space as shown below... the space, however, is still running and seems to be fine. No error msgs or stack traces show up in the space log, but the following shows up on the client side...

Can you give me an idea of what may have caused this ClosedChannelException? I'm not getting any clues in the GS log.

I'm running GS 6.6 on linux - the client in on Windows XP.

thanks,

 -dave

com.co.space.SpaceException: java.rmi.ConnectException: null; nested exception is: java.nio.channels.ClosedChannelException . . . ... 32 more Caused by: java.rmi.ConnectException: null; nested exception is: java.nio.channels.ClosedChannelException at com.gigaspaces.lrmi.nio.CPeer.invoke(CPeer.java:450) at com.gigaspaces.lrmi.ConnPoolInvocationHandler.invoke(ConnPoolInvocationHandler.java:64) at com.gigaspaces.lrmi.DynamicSmartStub.invokeRemote(DynamicSmartStub.java:381) at com.gigaspaces.lrmi.DynamicSmartStub.invoke(DynamicSmartStub.java:369) at $Proxy9.write(Unknown Source) at com.j_spaces.core.lrmi.LRMIRemoteSpaceImpl.write(LRMIRemoteSpaceImpl.java:111) at com.j_spaces.core.cluster.action.ClusterSpaceWriteActionListener.internalWrite(ClusterSpaceWriteActionListener.java:134) at com.j_spaces.core.cluster.action.ClusterSpaceWriteActionListener.write(ClusterSpaceWriteActionListener.java:87) at com.j_spaces.core.cluster.action.ClusterSpaceWriteActionListener.onAction(ClusterSpaceWriteActionListener.java:68) at com.j_spaces.core.cluster.action.ClusterSpaceWriteActionListener.onAction(ClusterSpaceWriteActionListener.java:24) at com.j_spaces.core.client.AbstractSpaceProxy.write(AbstractSpaceProxy.java:455) at com.j_spaces.core.client.AbstractSpaceProxy.write(AbstractSpaceProxy.java:435) . .

... 34 more

Caused by: java.nio.channels.ClosedChannelException at com.gigaspaces.lrmi.nio.Reader.throwCloseConnection(Reader.java:369) at com.gigaspaces.lrmi.nio.Reader.readBytesFromChannelBlocking(Reader.java:151) at com.gigaspaces.lrmi.nio.Reader.readBytesBlocking(Reader.java:464) at com.gigaspaces.lrmi.nio.Reader.bytesToPacket(Reader.java:450) at com.gigaspaces.lrmi.nio.Reader.readReply(Reader.java:110) at com.gigaspaces.lrmi.nio.CPeer.invoke(CPeer.java:377) ... 46 more Caused by: java.io.IOException: Connection has been closed by peer at com.gigaspaces.lrmi.nio.Reader.throwCloseConnection(Reader.java:370) ... 51 more

{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=2715]{quote}

asked 2008-11-17 21:38:02 -0600

jazzbutcher gravatar image

updated 2013-08-08 09:52:00 -0600

jaissefsfex gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

Can you check how many open connections you have?

Seems you maxed out nofile or nproc.

Set in /etc/security/limits.conf (root permissions required):
    * soft nofile 16384
    * hard nofile 65536
    * soft nproc 2047
* hard nproc 32768

Nofile is relevant for amount of connection/sockets and nproc is relevant for amount of threads.

Shay

answered 2008-11-17 22:07:28 -0600

shay hassidim gravatar image
edit flag offensive delete link more

Comments

When you say "number of connections", what do you mean? Check this using what tool?

I have one windows client connecting to one space server - no other processes or clients involved.

I will check the configs you suggested.

thanks,

 -dave
jazzbutcher gravatar imagejazzbutcher ( 2008-11-17 22:35:19 -0600 )edit

Shay,

I changed the settings as you suggested and rebooted.

No change in errors... error still only on the client side too...

What do you suggest I look at?

thanks,

-dave

Edited by: jazzbutcher on Nov 18, 2008 8:01 AM

jazzbutcher gravatar imagejazzbutcher ( 2008-11-17 22:45:58 -0600 )edit

Any idea what you have recently changed within the application that might impact remote calls activity? When running "netstat -ab" on the windows box , how many connection you have? How many space connections you see within the UI? Is this system running in production or you are in dev phase? Did you had the spaces or client application running for long time? Do you see this exception as a result of specific space operation or is it thrown randomly in sporadic manner? Can you try and reboot the windows box and try?

Shay

shay hassidim gravatar imageshay hassidim ( 2008-11-18 09:47:08 -0600 )edit

Shay,

The GS UI shows 2 connections between the GS server on linux and the client application on Windows.

Windows seems to have dozens of connections open for various services related to email, web, network shares, etc... My client software on Windows seems to have about 10 connections open to the GS server on linux (port 46894). The client also seems to have 10-20 connections to itself (localhost) in the 1600-1700 port range.

We are in development phase, although about to go to QA -- this part of the system has been behaving fine for several months and we've never seen an error like this before. The lack of detail in the stack trace and lack of exception on the server side is frustrating, and we're not sure where to look.

The client behaves ok when reading and writing to the space when run initially. Then if you quit, and re-launch the client, and attempt to write a type of entry that has not already been put in the space, you get the ClosedChannelException...

I have rebooted the windows box and the linux box numerous times and stopped and re-started the space (which is transient right now) many times. Thus the client and the server are both only running for minutes or tens of minutes at a time (no long running server, etc).

Can you think of anything else that could cause this type of error and this strange behavior? We are feeling very stuck at the moment.

thanks,

-dave
jazzbutcher gravatar imagejazzbutcher ( 2008-11-18 10:47:54 -0600 )edit

Lets have com.gigaspaces.lrmi.level = ALL at the client side. This will log every remote operation going on. I suggest also monitoring the LRMI threads at the client side. If everything looks good at the client side , we might have an issue at the space side. This means we need to repeat the above for the space side. I suggest to use 6.6.1 in case you are using older version. If you are using 6.6.1 I suggest to report this to support. Shay

shay hassidim gravatar imageshay hassidim ( 2008-11-18 11:08:11 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2008-11-17 21:38:02 -0600

Seen: 222 times

Last updated: Nov 17 '08