PaaS Performance Evaluation Methodology


Marcin Jarzab
Krzysztof Zielinski
Slawomir Zielinski
Karol Grzegorczyk
Marek Piascik


This paper proposes a PaaS performance assessment methodology which is oriented towards cloud computing attributes such as resource pooling, rapid elasticity and measured service. PaaS evaluation procedure requirements and taxonomy have been specified in this context. The innovative concept exploited by the proposed methodology is its completeness, evidenced by stress, dynamicity, and stability testing scenarios. A theoretical analysis of PaaS under test load conditions has been provided for the each proposed testing scenario. This allows interpretation of the experimental study performed using private and public cloud environments. Results validate the proposed methodology and establish it as a useful tool for benchmarking PaaS deployments.


How to Cite
Marcin Jarzab, Krzysztof Zielinski, Slawomir Zielinski, Karol Grzegorczyk, & Marek Piascik. (2015). PaaS Performance Evaluation Methodology. International Journal of Next-Generation Computing, 6(1), 01–24.


  1. Stegun Abramowitz. 1964. Handbook of Mathematical Functions With Formulas, Graphs, and Mathematical Tables. National Bureau of Standards Applied Mathematics Series - 55. 1020 pages.
  2. Nishant Agnihotri and Aman Kumar Sharma. 2014. Evaluating PaaS Scalability and Improving Performance Using Scalability Improvement Systems. IJRET: International Journal of Research in Engineering and Technology 03, 03 (2014).
  3. Abel Avram. 2010. Measuring and Comparing the Performance of 5 Cloud Platforms. (2010). Kees Blokland. 2013. Testing cloud services. How to test SaaS, PaaS and IaaS. (2013). wp-content/uploads/2013/11/2013_EuroSTAR-Ebook.pdf B. Cohen. 2013. PaaS: New Opportunities for Cloud Application Development. Computer 46, 9 (2013), 97–100.
  4. Adam L Davis. 2014. Modern Java: Java 7 and Polyglot Programming on the JVM Paperback. CreateSpace Independent Publishing Platform.
  5. D.Jayasinghe, S.Malkowski,, Q.Wang, Z.Wang, and Calton Pu. 2013. Variation In Performance and Scalability: An Experimental Study in IaaS Clouds using Multi - Tier Workloads. IEEE Transactions on Services Computing (2013), 1–14.
  6. Bayo Erinle. 2013. Performance Testing With JMeter 2.9. Packt Publishing.
  7. Ganglia. 2014. Ganglia Monitoring System. (2014). Mark Geene. 2012. 5 Steps To Selecting a Platform-as-a-Service (PaaS). (2012). com/considerations-in-selecting-a-platform-as-a-service-paas/ Mark Geene. 2013. PaaS First; IaaS Second: Five Reasons to Select Your PaaS First. (2013). http://blog. Neil J Gunther. 2007. Guerrilla Capacity Planning. Springer.
  8. Shigeru Hosono, Jiafu He, Xuemei Liu, Lin Li, He Huang, and Shuichi Yoshino. 2011. Fast development platforms and methods for cloud applications. In Services Computing Conference (APSCC), 2011 IEEE Asia-Pacific. IEEE, 94–101.
  9. JMX. 2006. Java Management Extensions Specification - version 1.4. technotes/guides/jmx/JMX_1_4_specification.pdf J. Kosinski, R. Szymacha, T. Szydlo, K. Zielinnski, J. Kosinska, and M. Jarzab. 2012. Adaptive SOA Solution Stack. IEEE Transactions on Services Computing 5, 2 (2012), 149–163.
  10. Peter Mell and Tim Grance. 2009. The NIST definition of cloud computing. National Institute of Standards and Technology 53, 6 (2009), 50.
  11. Atif Farid Mohammad and Hamid Mcheick. 2012. Cloud Service Testing: An Understanding. In The 2nd International Conference on Ambient Systems, Network and Technologies. 513–520.
  12. Dan Orlando. 2011. Cloud computing service models, Part 2: Platform as a Service. (2011).
  13. Rajesh Ramchandani. 2012. Choosing and implementing suitable enterprise PaaS. (2012). Alois Reitbauer, Andreas Grabner, and Michael Kopp. 2011. Java Enterprise Performance. Entwickler. Press.
  14. Spec. 2012. Report on Cloud Computing to the OSG Steering Committee. (2012).
  15. VMware. 2011. VMware vFabric PaaS Planning Service. (2011). vmware-vfabric-paas-planning-service.pdf
  16. Jim R. Wilson. 2013. Node.js the Right Way: Practical, Server-Side JavaScript That Scales. Pragmatic Bookshelf.
  17. Wenbo Zhang, Xiang Huang, Ningjiang Chen, Wei Wang, and Hua Zhong. 2012. PaaS-Oriented Performance Modeling for Cloud Computing. In Computer Software and Applications Conference (COMPSAC), 2012 IEEE 36th Annual. 395–404.