A benchmark for web service frameworks

Considering the facts that existing benchmarks to measure the performance of Web service frameworks simulate only theoretical scenarios such as streaming homogeneous data structures and the computer industry has an established culture of developing performance benchmarks imitating real world scenarios, an effort was made to come up with a benchmark that closely represent the real world business services. The paper concludes that the benchmark represents an unbiased subset of actual scenarios because the ranking and performance patterns of the leading Web services frameworks used in the experiment are consistent with Industry wide experiences. Additionally the paper introduces a performance model to analyze Web service frameworks and identifies complexity of the SOAP messages and size of the payloads they carry as two major factors that affect the RTT of the SOAP messages and reveals that a framework that is good at handling complex SOAP messages may not deal with messages that carry larger payloads equally well. © 2005 IEEE.