Building Web Services Middleware with Predictable Service Execution

Building Web Services Middleware with
Predictable Service Execution

Vidura Gamini Abhaya
Prof. Zahir Tari and Assoc. Prof. Peter Bertok

Distributed Systems and Networking Group
School of Computer Science and IT
RMIT University
Melbourne, Australia

December 13, 2010

[WISE 2010 - pages 23-37 in proceedings]

Intro. Guidelines Case Study Results Conclusion

Presentation Structure

1 Introduction

2 Guidelines for Predictability

3 Case Study

4 Empirical Results

5 Conclusion

V. Gamini Abhaya et. al. WISE 2010 - pages 23-37 in proceedings

Intro. Guidelines Case Study Results Conclusion Motive. QoS in the Cloud ThreadPools WS Stack

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Motivation

Web Services enable communication between Distributed
Heterogeneous Systems.



Motivation

Web Services enable communication between Distributed
Heterogeneous Systems. It is the foundation for...



Motivation - QoS in the Cloud



Motivation - QoS in the Cloud

Increased the importance of QoS
Lack of attempts in achieving Execution-time QoS
On achieving consistent and predictable Execution times


Web Services Middleware

WS-Middleware are optimised for throughput by design
Requests are accepted unconditionally
Uses thread-pools to execute them in a best-eﬀort manner
Processes multiple requests in-parallel in a processor sharing
manner



ThreadPool Pattern

Increases throughput
At the increase of the mean execution time of a web service



ThreadPool Pattern

Increases throughput
At the increase of the mean execution time of a web service

No. of requests served in-parallel ∝ Mean time to service a request



Web Services Software Stack

The execution of a web service request goes through multiple
layers of software



Web Services Software Stack

Predictability at the upper layers can only be achieved if it is
supported at the lower levels
Thread priority level mismatches between dev platform and OS
Service invocations can be interrupted by a process/thread with higher priority
Delays in resource reservations at OS level


Intro. Guidelines Case Study Results Conclusion Guidelines

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Guidelines for Predictability in Service Execution

- Enable Web Service Middleware to achieve Predictability in Service Execution.
- Can be followed in building new middleware or to enhance existing.
- Applicable for both SOAP and REST based services



Guidelines for predictability in service execution

G1. Dev platform and OS used must guarantee predictability of execution




G2. Schedule requests based on their deadlines




G3. Select requests based on their laxity




G4. Introduce ﬁne-grain diﬀerentiation in request processing




G4. Introduce ﬁne-grain diﬀerentiation in request processing
G5. Minimise priority inversions


Intro. Guidelines Case Study Results Conclusion Overview Infra. Commons Standalone Cluster PI

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Case Study - Overview

Infrastructure
Common enhancements made
Schedulability Check
Priority Model
Real-time Scheduler
Stand-alone web service engine
Apache Axis2
Web service cluster
Dispatcher - Apache Synapse
Executors - Apache Axis2



Infrastructure

Dev Platform and OS
Following G1,
Solaris 10 08/05 is used as the Real-time OS
Sun Real-time Java Speciﬁcation is used as the Dev Platform



Schedulability Check
Following G3, every incoming request is subject
to a Schedulability Check (Ref: Algorithm 1 on
paper and mathematical model in [5]).



Priority Model

Following G4, three priority levels are introduced (used by Real-time scheduler
following Algorithm 3 on paper)
High - Cannot be interrupted by GC
Mid - Can be interrupted by the GC
Low - Can be interrupted by the GC, Highest priority level available on Standard
Java



Real-time Scheduler

Schedules requests based on their deadlines
Follows Earliest Deadline First (EDF) technique
Uses the priority model on threads to decide which request is
executed at what time
..and follows G2 in doing so.



Enhancements made to Axis2

Details
Thread-pools have been replaced with RT-Thread
pools
A Real-time scheduler has been introduced
Multiple execution lanes are used



Enhancements made to Synapse

Details
RT-Axis2 instances used as Executors
Enhanced version of Synapse used for dispatching
Thread-pools have been replaced with RT-Thread pools
A Real-time scheduler has been introduced
Multiple execution lanes are used
Request processing happens at Dispatcher, Service Invocation at Executors



Minimising Priority Inversions

Can be caused by, I/O activities such as reading/writing to files and sockets.

To prevent priority inversions (following G5),
Avoid outputs resulting in on-screen messages or log file writes
Use in-memory logging and delayed writes
Use offline debugging techniques/tools
e.g. Oracle Thread Scheduling Visualizer (TSV) [16]



Minimising Priority Inversions - TSV


Intro. Guidelines Case Study Results Conclusion Stand Alone Cluster

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Stand-alone Engine performance

Comparison of Unmod. Axis2 vs RT-Axis2 - Deadline Achievement Rates
100

90
Percentage of deadlines met off percentage accepted

80

70

60

50

40

30

20

10
Unmod. Axis2
RT-Axis2
0.1s - 0.25s 0.1s - 0.5s 0.25s - 1s 0.25s - 2s
Inter-arrival times

RT-Axis2 accepts between 18.1% (fastest) and 96.7% (slowest) of the requests.



Stand-alone Engine performance

1 - 5000000 (Uniform); 0.25sec - 1sec (Uniform) 1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform)
180000 250000
Real-tme Axis2 Real-tme Axis2
Unmodified Axis2 Unmodified Axis2
160000

200000
140000

120000
Execution Time (ms)

Execution Time (ms)
150000
100000

80000
100000
60000

40000
50000

20000

0 0
0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06 0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06
Task Size Task Size



RT-RoundRobin vs Round-Robin

Comparison of Round-robin vs. RT-RoundRobin - Deadline Achievement Rates
100
RR - 2 Execs.
RR - 3 Execs.
90 RR - 4 Execs.
RT-RR - 2 Execs.
RT-RR - 3 Execs.
80 RT-RR - 4 Execs.
Percentage of deadlines met

70

60

50

40

30

20

10

0.05s 0.1s 0.1s 0.25s 0.1s - 0.5s 0.25s - 1s
Inter-arrival times

RT-RR accepts between 20.5% (2 Exec. - fastest) and 99.9% (4 Exec. -
slowest) of the requests.



RT-RoundRobin vs Round-Robin

Comparison of execution times (1 - 5000000 (Uniform); 0.1sec - 0.25sec (Uniform) roundrobin 3 exec) Task Size distribution between 3 Executors (1 - 5000000 (Uniform); 0.1sec - 0.5sec (Uniform) roundrobin 3 exec)
120000 12000
RT-RoundRobin Executor 1
Simple Round-Robin Executor 2
Executor 3

100000 10000

80000 8000
Execution Time (ms)

Execution Time (ms)
60000 6000

40000 4000

20000 2000

0 0
0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06 0 500000 1e+06 1.5e+06 2e+06 2.5e+06 3e+06 3.5e+06 4e+06 4.5e+06 5e+06
Task Size Task Size


Intro. Guidelines Case Study Results Conclusion Future Work Recap

1 Introduction


3 Case Study

4 Empirical Results

5 Conclusion



Future Work

Further analysis of cluster behaviour with mix of tasks
Advanced dispatching and re-dispatching models
Predictability in the networking layer



Conclusion

With the growing popularity of Cloud Computing and it’s
service oﬀerings, Execution level QoS has become an
important aspect
Existing WS-Middleware are optimized for throughput thereby
lacks predictability of execution
Presented Guidelines for WS-Middleware to achieve
predictability of execution
Discussed 2 case studies as examples
Presented empirical data, on evaluating the two
implementations



Thank You !
&
Questions or Comments ?


Building Web Services Middleware with Predictable Service Execution

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Building Web Services Middleware with Predictable Service Execution

Similar to Building Web Services Middleware with Predictable Service Execution (20)

Recently uploaded

Recently uploaded (20)

Building Web Services Middleware with Predictable Service Execution