Concurrent take() is a problem?
Hi all, In my GigaSpaces app, i need to know when the data processing is ended in order to do some notifications.
As suggested here, i've developed a new object called GigaCounter, which encapsulates the number of objects processed by the PUs. The feeder knows how many objects have to be processed, and keeps polling the space to read all the GigaCounter objects until it finds that the counters sum is equal to the expected total number.
Basically, every PU calls this method to process any object, so i count how many times this method is called:
public void store (...) {
... some hibernate related operations ...
GigaCounter template = new GigaCounter();
GigaCounter counter = gigaSpace.take(template);
if (counter == null) {
counter = template;
counter.setCount(0);
}
counter.increment();
gigaSpace.write(counter);
}
This is the feeder "polling" part:
public void run() {
GigaCounter clearTemplate = new GigaCounter();
gigaSpace.clear(clearTemplate);
Integer count = 0;
try {
while(! count.equals(employeesCount) ) {
count = 0;
GigaCounter template = new GigaCounter();
GigaCounter[] counters = gigaSpace.readMultiple(template, 1000);
for (GigaCounter c : counters)
count+=c.getCount();
System.out.println("Employees processed: "+count+" of "+employeesCount+".");
if ( ! count.equals(employeesCount) )
Thread.sleep(5000);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Process finished");
And this is the GigaCounter object:
@SpaceClass(persist=false) public class GigaCounter implements Serializable {
private Integer count = null;
public GigaCounter() {
}
/*
public GigaCounter(Integer id) {
this.id = id;
this.count = 0;
}
public Integer getId() {
return this.id;
}
/*
public void setId(Integer id) {
this.id = id;
} */
/**
* @return the count
*/
@SpaceRouting
@SpaceId(autoGenerate=false)
public Integer getCount() {
return this.count;
}
/**
* @param count the count to set
*/
@SpaceId(autoGenerate=false)
public void setCount(Integer count) {
this.count = count;
}
public void increment() {
count++;
}
}
Let's call X the total expected and C the value calculated with the GigaCounter objects. The problem is that, at the end of the data processing, the counting process is correct with low data loading (X == C, the sum is equal to the total expected), but when the data rate, operations and loads is high, the count is wrong: it's about the 80% of the total expected (C == 0,8*X) , even if i can see that the method is called X times (because i see that X rows are written in the database via Hibernate).
So, what's the matter here? I just can think about some problems when different PUs ask to take the same GigaCounter object simultaneously...
{quote}This thread was imported from the previous forum. For your reference, the original is [available here|http://forum.openspaces.org/thread.jspa?threadID=2981]{quote}
