Development of a new command line tool based on a plain shell/batch script instead of Apache ant, introducing a new configuration file management enabling sharing a CLIF runtime among an arbitrary number of test projects with different configurations.
Development of a new configuration facility to transparently choose the right network addresses for the CLIF infrastructure.
Enhanced "deep load" feature in HttpInjector: now includes CSS and JS, and gives the choice of what kind of element to load.
New primitives for FileReader, enabling loading files and clearing contents
New TcpInjector, an enriched version of the SocketInjector
Progress on the mavenization branch:
creation of artifact clif-api, containing API definitions moved from clif-core;
Decision and actions to make the ProActive branch the reference for the maven build chain;
Prepare for moving the CLIF plug-in for Jenkins to the Jenkins project official repository;
Merge with trunk to get all new features and further reorganization.
CLIF in a container: successful experiments to deploy CLIF nodes and ProActive CLIF nodes as Docker containers.
talk at Paris Open Source Summit about CLIF as a Service for performance testing in continuous integration over OpenStack
Results for year 2014
New production version 2.2.1, with binary distributions for MacOSX, Windows and Linux
the system probes implementation, based on the LeWYS project, has been fully replaced with a brand new implementation based on the SIGAR open source project. This change is absolutely transparent to CLIF users, who are still able to use their existing test plans with CPU, memory, disk and network probes, but now covering many more OS and architectures (refer to SIGAR documentation from Hyperic support)
further efforts to improve user experience with the main (Eclipse-based) CLIF console, including contextual help in the test plan editor, streamlined error reporting and an important patch solving a big instability problem
new command-line facilities to get help with probe parameters
first release of the embedded performance analysis and reporting tool
introducing .bat scripts for Windows to be used as a replacement to some commands of the ant-based command-line interface (to be extended and generalized)
work on ISAC plug-ins: improved CommandLineInjector, Counter and FileWriter plug-ins
Work on the Jenkins plug-in: fixed error rate in performance reports and improved properties management for ProActive CLIF
From a source code maintenance perspective:
maintenance of the ProActive branch
progress on the mavenization branch (still not fully functional)
Resulting from our involvement in the OpenCloudware collaborative project, a CLIF-as-a-Service architecture was built and demonstrated to automate performance testing, possibly in a continuous integration process.
CLIFaaS includes the deployment and configuration of the distributed load injection system, and possibly the deployment and configuration of the distributed application under test.
See the presentation slides and the demonstration video.
Results for year 2013
production version 2.1.1 was released at the beginning of the year, with binary distributions for MacOSX, Windows and Linux
work on ISAC plug-ins:
added new features to plug-ins Common, Counter and HttpInjector
new FileWriter plug-in to write arbitrary contents to arbitrary files during test execution
minor fixes and improvements on CLIF core, mostly on the "quick statistical reports"
miscellaneous fixes and improvements to the CLIF plug-in for Jenkins
from a source code maintenance perspective:
maintenance of the ProActive branch
creation of a "mavenization" branch dedicated to a full reorganization of the code base to support a maven-based build process (to be completed)
communication on "CLIF meets Jenkins" at OW2con'2013, dealing with the benefits of using CLIF with a continuous integration server
real-scale test-bed of CLIF as a Service on a community cloud (injecting traffic from 20 PCs of coworkers distributed through a whole country)
Results for year 2012
enhancement of the CLIF plug-in for Jenkins continuous integration server
wizard for importing a full CLIF test project with a single click!
improved automatic reporting
brand new, full-fledged, DNS load injector, including DNSSEC support and a dedicated data set provider
new "Quick Stats" feature: per-request type statistics, aggregated through all load injectors, generated as a single CSV-formatted file once measures are collected
debug and improvement of the interactive analysis and reporting tool
new SelfBench tool box: controller components providing smart benchmarking features (for advanced users)
automatic saturation detection (SaturationController) according to threshold policies on response times or resource usage
self-driven load injection (InjectionController) according to a queuing model-based regulation algorithm
further work on the SVN branch based on the ProActive middleware, applied to community cloud-based load testing with CLIF.
This work was carried out by Orange Labs (CLIF project) and ActiveEon (ProActive project), and was presented at OW2's Annual Conference in November.
beta version of the new interactive analysis and reporting tool available from the SVN trunk
new production release 2.0.7, coming with:
extended Synchro ISAC plug-in for synchronization between distributed test scenarios
optimized and enhanced deployment to support large scale test plans: more than 1000 distributed load injectors (with an average deployment time of 100ms per load injector), support for partial deployment, deployment time-out
new development branch based on the ProActive middleware to bypass some networking issues (routing, firewalls...) and provide CLIF as a service on a community cloud