Welcome to the new Gigaspaces XAP forum. To recover your account, please follow these instructions.

Ask Your Question
0

Subqueries in Polling Container's @EventTemplate annotated method

Hi, I have 3 types of jobs in space separated by @SpaceProperty priority, Values are HIGH, MEDIUM and LOW. I want to write a query where it will pull 16 HIGH priority tasks, 8 MEDIUM and 4 LOW priority task in 1 single query. As I have 28 concurrent consumers hence total 28 tasks will be consumed by 28 threads.

Will below SQL query work in PollingContainer's @EventTemplate annotated method ?

@EventTemplate SQLQuery<jobevent> matchingJobs() { return new SQLQuery<jobevent>( JobEvent.class, " id in ( select job.id from JobEvent job where job.priority='HIGH' and rownum <= 16) "+ " or id in (select job.id from JobEvent job where job.priority = 'MEDIUM' and rownum <= 8) "+ " or id in (select job.id from JobEvent job where job.priority = 'LOW' and rownum <= 4) " ); }</jobevent></jobevent>

Also bigger question is First time this query will select 28 jobs, 28 concurrent consumers will start working on them. say 10 consumers finishes quickly and are available to consume further jobs, In that case polling container will poll again 28 jobs but only 10 consumers are available, Can I then say that out of 16 HIGH, 8 MEDIUM and 4 LOW jobs any 10 will be consumed by these available 10 consumers ? Next question is say 5 more consumers got free from first batch of jobs.... Will Polling container again execute above query OR 28-10 = 18 jobs are still available from previous query execution will be given to these 5 consumers ?

Please share is above will work or not. Here I'm trying my Polling container to consumer more HIGH priority batches, less MEDIUM and few LOW priority batches.

asked 2014-03-04 15:59:41 -0600

Atul gravatar image
edit retag flag offensive close merge delete

1 Answer

Sort by ยป oldest newest most voted
0

It might be easier to configure your 28 consumers to have 16 polling for HIGH priority jobs, 8 polling for MEDIUM priority, and 4 polling for LOW priority. If you have a steady, well-distributed stream of jobs then all of your consumers should stay busy.

If you find some consumers are not busy, you could add a worker to monitor them and bump up the priority of the longest waiting LOW and MEDIUM priority jobs so that the HIGH priority consumers will pick them up.

answered 2014-03-05 10:45:24 -0600

Patrick gravatar image
edit flag offensive delete link more

Comments

You can fifo grouping or priority based queue pattern.

shay hassidim gravatar imageshay hassidim ( 2014-03-05 10:55:46 -0600 )edit

Patrick thanks for response, I have 1 GSC which has 1 polling container with 28 concurrent consumers. Can you give me example how I will be able to configure 16 consumers to consume HIGH priority jobs, 8 consumer to consume MEDIUM and 4 to consume LOW Jobs?

To me If I create 3 GSCs where 1 GSC=16 consumers, 2nd GSC=8 consumers and 3rd GSC=4 consumers, in this case I will be able to configure Listners in all 3 GSCs differently to poll HIGH, MEDIUM and LOW priority jobs, this will solve my problem But I don't want to spawn different GSCs for this.

Atul gravatar imageAtul ( 2014-03-05 12:22:46 -0600 )edit

Shay, Thanks for suggestions, however below are constraints for not using FIFO groups and Priority Base Queue pattern 1. FIFO groups processes 1 job at a time for a gigen FIFO group in my case I will be having 3 FIFO groups 1. HIGH - This group will process 1 event/job at a time. 2. MEDIUM - This group will process 1 event/job at a time. 3. LOW - This group will process 1 event/job at a time. My Requirement is I want to assign more threads to HIGH FIFO group, less to MEDIUM FIFO group and few to LOW FIFO group, but at the same time I cannot afford to have FIFO group using only 1 thread at a time.

2 Priority Base Queue pattern is aplicable when my processing is out of GSC mainly remote processing of events. In my case GSC hosts polling container which is responsible for processing jobs. Hence I want to write a query which can select fixed number of HIGH, MEDIUM and LOW jobs.

Atul gravatar imageAtul ( 2014-03-05 12:40:46 -0600 )edit

I haven't tried this, but simply using three polling containers with the appropriate concurrent consumers set should work:

  <os-events:polling-container id="high-priority-worker"
                               giga-space="gigaSpace"
                               concurrent-consumers="16">

    <os-core:template>
      <bean class="com.foo.bar.Job">
        <property name="priority" value="HIGH"/>
      </bean>
    </os-core:template>

    <os-events:listener>
      <os-events:method-adapter method-name="processJob">
        <os-events:delegate>
          <bean class="com.foo.bar.Worker"/>
        </os-events:delegate>
      </os-events:method-adapter>
    </os-events:listener>

  </os-events:polling-container>
Patrick gravatar imagePatrick ( 2014-03-05 12:53:48 -0600 )edit

Thanks Patrick,

This will definitely solve my problem.

Atul gravatar imageAtul ( 2014-03-05 12:58:43 -0600 )edit

Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

Question Tools

1 follower

Stats

Asked: 2014-03-04 15:59:41 -0600

Seen: 265 times

Last updated: Mar 05 '14