Using Real-Time Scheduling Principles in Web Service Clusters to Achieve Predictability of Execution

Using Real-Time Scheduling Principles in Web
Service Clusters to Achieve Predictability of
Service Execution

Vidura Gamini Abhaya
Prof. Zahir Tari and Assoc. Prof. Peter Bertok

Distributed Systems and Networking Group
School of Computer Science and IT
RMIT University
Melbourne, Australia

December 8, 2010

Intro. Prev. Work Solution Results Conclusion

Presentation Structure

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion

V. Gamini Abhaya et. al. ICSOC 2010 - pages 197-212 in proceedings

Intro. Prev. Work Solution Results Conclusion Motivation

Motivation

Web Services enable communication between Distributed
Heterogeneous Systems.



Motivation

Web Services enable communication between Distributed
Heterogeneous Systems. It is the foundation for...



Motivation



Motivation

Scalability Issues...
Quality of Service ?
Performance ?
Predictability ?

Intro. Prev. Work Solution Results Conclusion Single Host

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



Our Previous Work [5,6,*]...

Achievement
Achieve Predictability and Service Diﬀerentiation in WS execution
(in a single server)

* Building Web Service Middleware with Predictable Service Execution, WISE 2010, LNCS, vol. 6488 pp. 23-37.



Our Previous Work [5,6,*]...

Achievement
Achieve Predictability and Service Diﬀerentiation in WS execution
(in a single server)

Highlights...
Introduced the notion of a deadline
Use Real-time scheduling principles
Select tasks with achievable deadlines, reject the rest
Selected tasks scheduled in the order of increasing deadlines
Implementation based on Apache Axis2
Tested for various request size, arrival rate combinations

* Building Web Service Middleware with Predictable Service Execution, WISE 2010, LNCS, vol. 6488 pp. 23-37.



Task Execution with a Deadline



Task Execution with a Deadline

Laxity of a Task
Deadline
Laxity =
ExecutionTime



Single Host Implementation

Task selection based on a ‘Schedulability Check’
Deadline based scheduling
Implementation on a single server



Deadline based Task Schedule

EDF based task schedule
Larger laxities allow requests to be executed in staggered
manner
Enable timely completion of requests with earlier deadlines


Intro. Prev. Work Solution Results Conclusion Overview Dispatching Algo Details

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



Solution Overview



Solution Overview
Sun Real-time Java Speciﬁcation used as
development platform
Sun Solaris 10 08/05 used as RT-OS



Solution Overview

Apache Synapse based dispatcher implementation
Apache Axis2 based executor implementation



Solution Overview

Apache Synapse based dispatcher implementation
Apache Axis2 based executor implementation
All Thread-pools replaced with custom-designed
Real-time Thread pools
Introduction of Real-time scheduler component



Cluster Design

Request processing at the dispatcher
Requests are dispatched based on algorithm used
Requests are ‘mostly’ queued at dispatcher
Results are sent back to clients via dispatcher



Dispatching Algorithms

Objectives
*Ensures deadline requirement of tasks could be met
*Distribute requests evenly among executors based on a condition
*Avoids executors going into overload conditions

4 algorithms used
RT-RoundRobin
RT-Sequential
RT-ClassBased
RT-LaxityBased



RT-RoundRobin

Details
Requests assignment cycles
through all executors in RR
fashion
Sched. check is done only once
per request
If sched. check fails on selected
server → request rejected



RT-RoundRobin

Details
Requests assignment cycles
through all executors in RR
fashion
Sched. check is done only once
per request
If sched. check fails on selected
server → request rejected

Hightlights
POC for how a simple dispatching algorithm could be made real-time ready
Processing overhead is minimum



RT-Sequential

Details
Requests are assigned in a
sequential manner
Req’s sent to one executor till
sched. check fails, then assigned
to the second and so on
Sched. check happens against
multiple executors till a request
can be assigned



RT-Sequential

Details
Requests are assigned in a
sequential manner
Req’s sent to one executor till
sched. check fails, then assigned
to the second and so on
Sched. check happens against
multiple executors till a request
can be assigned

Highlights
If it’s possible to schedule a job within the cluster, it will be guaranteed



RT-ClassBased

Details
Requests are divided into classes
based on a priority scheme
Mapping of requests to executors
is pre-determined and deﬁned
oﬄine
Sched. check is only considered
with the assigned executor
Reference implementation uses
priorities based on task sizes



RT-ClassBased

Details
Requests are divided into classes
based on a priority scheme
Mapping of requests to executors
is pre-determined and defined
offline
Sched. check is only considered
with the assigned executor
Reference implementation uses
priorities based on task sizes

Highlights
Results in the reduction of variance of task sizes at each executor
Pre-defined mapping of request sizes to executors could be done using
pre-profiled execution times or execution time history



RT-LaxityBased

Details
Laxity = Ability of delaying a
request whilst still meeting its
deadline
Higher the laxity the more
requests an executor could service
Sched. check is done only against
the assigned executor



RT-LaxityBased

Details
Laxity = Ability of delaying a
request whilst still meeting its
deadline
Higher the laxity the more
requests an executor could service
Sched. check is done only against
the assigned executor

Highlights
Distribution of requests results in a broad range of laxities at each executor
Keeps track of the last two laxities assigned and prevents them being assigned
for the same executor consecutively


Intro. Prev. Work Solution Results Conclusion RT-RR RT-CB Others

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



RT-RoundRobin vs Round-Robin

Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates
100
RR - 2 Execs.
RR - 3 Execs.
90 RR - 4 Execs.
RT-RR - 2 Execs.
RT-RR - 3 Execs.
80 RT-RR - 4 Execs.
Percentage of deadlines met

70

60

50

40

30

20

10

0.05s 0.1s 0.1s 0.25s 0.1s - 0.5s 0.25s - 1s
Inter-arrival times

RT-RR accepts between 20.5% (2 Exec. - fastest) and 99.9% (4 Exec. -
slowest) of the requests.



RT-RoundRobin vs Round-Robin

Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) roundrobin 3 exec)
12000
Executor 1
Executor 2
Executor 3

10000

8000
Execution Time (ms)

6000

4000

Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates
100
RR - 2 Execs.
RR - 3 Execs.
90 RR - 4 Execs.

80
RT-RR - 2 Execs.
RT-RR - 3 Execs.
RT-RR - 4 Execs.
2000

70

60

50

40

30

20
0
10
0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06
0.05s 0.1s 0.1s 0.25s
Inter-arrival times
0.1s - 0.5s 0.25s - 1s
Task Size



RT-ClassBased vs Class Based

Comparison of Class-based vs. RT-CB - Deadline Achievement Rates
100
CB - 2 Execs.
CB - 3 Execs.
90 CB - 4 Execs.
RT-CB - 2 Execs.
RT-CB - 3 Execs.
80 RT-CB - 4 Execs.

70

60

50

40

30

20

10

0.05 - 0.1 0.1 - 0.25 0.1 - 0.5 0.25 - 1
Inter-arrival times

RT-CB accepts between 28.6% (2 Exec. - fastest) and 100% (4 Exec. - slowest)
of the requests.



RT-ClassBased vs Class Based

Comparison of Class-based vs. RT-CB - Deadline Achievement Rates
100
CB - 2 Execs.
CB - 3 Execs.
90 CB - 4 Execs.
RT-CB - 2 Execs.
RT-CB - 3 Execs.
80 RT-CB - 4 Execs.

70

60

50

40

30

20

10

0.05 - 0.1 0.1 - 0.25 0.1 - 0.5 0.25 - 1
Inter-arrival times

CPU utilisation at each Executor - (1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform) classbased 3 exec) Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) classbased 3 exec)
90 16000
Executor 1 Executor 1
Executor 2 Executor 2
80 Executor 3 Executor 3
14000

70
12000

60

Execution Time (ms)
10000
CPU Utilization %

50
8000
40

6000
30

4000
20

10 2000

0 0
0 50 100 150 200 250 300 350 400 0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06
Sample # Task Size



RT-Sequential and RT-Laxity

Comparison Deadline Achievement Rates - Averaged by Algorithm
100

90

80
Averaged percentage of deadlines met

70

60

50

40

30

20
RT-RR
10 RT-CB
RT-Seq
RT-Lax
0.05s 0.1s 0.1s 0.25s 0.1s - 0.5s 0.25s - 1s
Inter-arrival times

Highest acceptance rate for any algorithm for 2 executors with highest arrival
rate is 38.5% for RT-LaxityBased



RT-Sequential and RT-Laxity
CPU utilisation at each Executor - (1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform) laxitybased 3 exec)
60
Executor 1
Executor 2
Executor 3

50

40
CPU Utilization %

30

20

Comparison Deadline Achievement Rates - Averaged by Algorithm
100

90

80
10
Averaged percentage of deadlines met

70

60

50

40

30

20
0
10
RT-RR
RT-CB
RT-Seq
0 200 400 600 800 1000 1200
RT-Lax
0.05s 0.1s 0.1s 0.25s
Inter-arrival times
0.1s - 0.5s 0.25s - 1s
Sample #


Intro. Prev. Work Solution Results Conclusion Future Work Recap

1 Introduction

2 Previous Work

3 Solution

4 Empirical Results

5 Conclusion



Future Work

Further analysis of cluster behaviour with mix of tasks
Advanced dispatching and re-dispatching models
Predictability in the networking layer



Conclusion

Previous work - We achieved predictability of execution on a
single server
We extend it to achieve predicability in a cluster of servers
Presented four task dispatching algorithms
Evaluated their performance empirically



Thank You !
Questions or Comments ?


Using Real-Time Scheduling Principles in Web Service Clusters to Achieve Predictability of Execution

Recommended

Recommended

More Related Content

Similar to Using Real-Time Scheduling Principles in Web Service Clusters to Achieve Predictability of Execution

Similar to Using Real-Time Scheduling Principles in Web Service Clusters to Achieve Predictability of Execution (20)

Using Real-Time Scheduling Principles in Web Service Clusters to Achieve Predictability of Execution