We have been invited to teach a lecture on performance testing at Faculty of Informatics, Masaryk University. The choice of the performance testing tool to use during the practical part was clear (PerfCake if it wasn't obvious). We already knew about a couple of things that were broken in 4.0. So for the demo to be as smooth as possible we have decided to create a dedicated University release, numbered 4.1.
It has five bug fixes, three enhancements and we also updated our web pages. All our friends and users are strongly recommended to update to version 4.1 to avoid any possible inconveniences.
Did the lecture went all that smooth in the end? Of course not. What was the deal breaker? The students were supposed to run PerfCake on a commercial operating system by Microsoft. This has not been tested thoroughly by us. We found out that the main script in the binary release works only when executed from the main installation directory. Plus some more minor things that we were able to overcome. Some of them are already fixed in the devel branch but we are unlikely to create another release now. Possibly later in the summer as we are still hoping to be able to show PerfCake in public again.
pondělí 27. dubna 2015
neděle 19. dubna 2015
PerfCake has a cool new logo
As we already announced on Twitter, PerfCake has a cool new logo! It is also available in SVG format with its variants.
The previous logo we used to use was fine. We needed something for a beginning project and Pavel created the original one quickly. Btw. did you know that it depicted an inner side of a cake cut from a close lookup? :-)
As PerfCake matured, we knew that we are going to need something that resonates more with us. We have a stable core so it really deserved something nice. We also tweaked our web page a little bit. But more changes are likely to come to make it even more simple.
We are especially excited about the new logo because it enables us to further develop the brand. We created a nice presentation template already. Come to a public event to see it in action! We can now create a boxed version of PerfCake, we can create cool stuff like t-shirts... The possibilities are now unlimited!
Any feedback on the new logo is highly appreciated!
The previous logo we used to use was fine. We needed something for a beginning project and Pavel created the original one quickly. Btw. did you know that it depicted an inner side of a cake cut from a close lookup? :-)
As PerfCake matured, we knew that we are going to need something that resonates more with us. We have a stable core so it really deserved something nice. We also tweaked our web page a little bit. But more changes are likely to come to make it even more simple.
We are especially excited about the new logo because it enables us to further develop the brand. We created a nice presentation template already. Come to a public event to see it in action! We can now create a boxed version of PerfCake, we can create cool stuff like t-shirts... The possibilities are now unlimited!
Any feedback on the new logo is highly appreciated!
pondělí 23. března 2015
From 4.0 to 4.1
As usually, we celebrated spring equinox with an even numbered release of PerfCake. We managed to release a couple of days earlier. The main reason for that was that we had a planned vacation for the release period.
This release has some really cool features that makes PerfCake even easier for usage. Actually there are 26 of enhancements and new features in this release. Just have a look on the list to see if you could make a use of them.
Except for these cool features there is one thing we are especially proud of in this release. This is its stability. We finally managed to get rid of all the raise conditions appearing now and then and bothering us constantly. I do not want to claim that there aren't any now. But there aren't any we are aware of. And there is a fair chance that nobody will discover a new one (until we change the code ;-)).
Unfortunately, bad things happen and there will be version 4.1 soon. Especially due to a bug that prevents smooth shell script usage randomly.
We also decided to fix several other issues. This is mainly because we want to get ready for a possibility to present PerfCake publicly soon again. All information on that will be shared once we have an official confirmation. Stay tuned!
This release has some really cool features that makes PerfCake even easier for usage. Actually there are 26 of enhancements and new features in this release. Just have a look on the list to see if you could make a use of them.
Except for these cool features there is one thing we are especially proud of in this release. This is its stability. We finally managed to get rid of all the raise conditions appearing now and then and bothering us constantly. I do not want to claim that there aren't any now. But there aren't any we are aware of. And there is a fair chance that nobody will discover a new one (until we change the code ;-)).
Unfortunately, bad things happen and there will be version 4.1 soon. Especially due to a bug that prevents smooth shell script usage randomly.
We also decided to fix several other issues. This is mainly because we want to get ready for a possibility to present PerfCake publicly soon again. All information on that will be shared once we have an official confirmation. Stay tuned!
neděle 8. února 2015
DevConf 2015
This year, we were accepted with our talk at DevConf as well. We always try to show something new. So we worked hard to be able to show major cool features.
The first is the ability to automatically generate chart reports using Google Charts. This is provided in the form of a ChartDestination. PerfCake also automatically combines recent results with any previously existing in the output directory which gives you an automatic comparison.
We have demonstrated this with several HTTP Reverse Echo clients, each of which was implemented in a different language. We reused the same scenario to show the user friendliness of PerfCake and the fact that PerfCake does not care about the language the measured application is developed in.
The second major feature was automatic memory leak detection. This works only for Java currently, however, we were able to detect all the leaks.
The way we do presentations is that we try to evolve over time. We try hard to create something new for anybody visiting all of our presentation still have some fun. The thing is that we did not see many people coming back and listening to new things. After this last presentation we are likely to reevaluate our approach and make sure we always presents the basics. If you could provide your opinions on that, we would love to take them into account.
The first is the ability to automatically generate chart reports using Google Charts. This is provided in the form of a ChartDestination. PerfCake also automatically combines recent results with any previously existing in the output directory which gives you an automatic comparison.
We have demonstrated this with several HTTP Reverse Echo clients, each of which was implemented in a different language. We reused the same scenario to show the user friendliness of PerfCake and the fact that PerfCake does not care about the language the measured application is developed in.
The second major feature was automatic memory leak detection. This works only for Java currently, however, we were able to detect all the leaks.
The way we do presentations is that we try to evolve over time. We try hard to create something new for anybody visiting all of our presentation still have some fun. The thing is that we did not see many people coming back and listening to new things. After this last presentation we are likely to reevaluate our approach and make sure we always presents the basics. If you could provide your opinions on that, we would love to take them into account.
úterý 21. října 2014
How fast can PerfCake be?
For a long time, we were wondering what are the limiting abilities of PerfCake. Mainly, what is the impact of PerfCake itself on the CPU. We created a simple scenario with the DefaultMessageGenerator and DummySender (that does nothing). We did not use validation for now as you can see in the scenario below. The aim of this exercise was to investigate thread management and proper data structures usage in the PerfCake core. On an Intel Xeon (2x 8 core CPU) we achieved over 75 billions of requests per second using 4096 concurrent threads/clients. Is your system that fast?
There is one more parameter to the test - the queue size. This is the size of an internal buffer used to prepare requests. If the queue was too short or too slow, it would block the maximal throughput. You can happily ignore this as the information is mainly for the developers. The colorful lines just make the charts look more joyful ;-)
We also wondered, how PerfCake can perform on a limited device based on an ARM CPU. Could you use your mobile phone as a load generator for example? So we run the same scenario on Cubieboard 2. The numbers are lower and the queue size now plays more important role. For our Fedora 19 installation on the device, there was a limit on the maximum number of threads for a system process. This is why the chart ends around 512 threads.
Following is the scenario.
Technical preview of DSL is in 3.3
As we promised, PerfCake 3.3 is out. The main reason for this release is a feature we originally planned for 3.0. We wanted to present this at GeeCON 2014 in Prague, so here is the first "technical preview". We also fixed some minor bugs - as we found them, there was no reason for postponing the fix. We hope to collect some feedback on the DSL implementation and possibly improve it.
We also look forward to see current and find some new users at GeeCON. Meanwhile, we also presented PerfCake in an IT company and discovered some new feature requests. Therefore, we started speaking about receivers, timers, correlators... You might possibly see some of the revolutionary concepts in release 4.0.
pátek 10. října 2014
PerfCake 3.0 is out, so is 3.1 and 3.2...
For us, Release 3.0 is a major achievement and a significant success. We managed to fix multiple kids' diseases that were hidden in the code, we reworked problematic parts, automated many steps, revised the whole project infrastructure including CI, web site, release procedure... Especially the later one still needs some tweaking as explained later.
You can have a look on the list of closed issues for this release. The user experience should be now better. We prepared automated scenario migration to the last version (running this is however still little bit clumsy), we fixed all the issues with reporting (early or late results were not recorded), we did some PoC with Google charts output (and decided to solve that differently in version 4.0), we have some progress on the tooling (Eclipse plugin is almost done), we extended our tests to the level of advanced integration testing (running in-container testing of JMS sender for instance, special credit goes to an university student who implemented it), and many other changes that might be hardly visible from the outside but were important inside.
There is still one component in PerfCake that is not easy to extend or to implement on your own - the generators. They do not have an interface but an abstract parent class. This is something we would like to change for programmers' convenience.
From the users perspective, our priority is the tooling, documentation, migration from previous versions (and possibly migration from other tools), execution in a clustered environment, ability to directly represent results graphically (using charts).
We also have a plan to finally provide another way to specify the scenario - a DSL language. This feature is almost in a technical preview state.
So, what went wrong during the release and why we ended up with version 3.2? The reason is quite silly and it is a lesson learned for the next release. During the release procedure, we prepare everything on Github first. The master branch has a new version and the devel branch is set to continue on the next release. Then we prepare the artifacts and upload them to Sonatype for them to appear in the public Maven repository. Then our Continuous Integration environment automatically consumes the sources from master and runs all the tests. It also creates downloadable artifacts for automatic web pages refresh.
So after releasing version 3.0, we realized that the tests do not work well in the CI environment. We decided this must be caused by a different environment setup, implemented a fix, executed tests locally and released version 3.1.
But this did not help! Still the same errors appeared in the CI. After a more thorough inspection we realized that there is a problem with the javadoc Maven plugin (the recent version was backward incompatible). So we eventually fixed that and released version 3.2.
Next time, we must run our CI stuff before the upload to Sonatype.
And one more thing - there will be another release, version 3.3, shortly before GeeCON in Prague 2014. We want to present some cool new features there during the Lightning talks.
You can have a look on the list of closed issues for this release. The user experience should be now better. We prepared automated scenario migration to the last version (running this is however still little bit clumsy), we fixed all the issues with reporting (early or late results were not recorded), we did some PoC with Google charts output (and decided to solve that differently in version 4.0), we have some progress on the tooling (Eclipse plugin is almost done), we extended our tests to the level of advanced integration testing (running in-container testing of JMS sender for instance, special credit goes to an university student who implemented it), and many other changes that might be hardly visible from the outside but were important inside.
There is still one component in PerfCake that is not easy to extend or to implement on your own - the generators. They do not have an interface but an abstract parent class. This is something we would like to change for programmers' convenience.
From the users perspective, our priority is the tooling, documentation, migration from previous versions (and possibly migration from other tools), execution in a clustered environment, ability to directly represent results graphically (using charts).
We also have a plan to finally provide another way to specify the scenario - a DSL language. This feature is almost in a technical preview state.
So, what went wrong during the release and why we ended up with version 3.2? The reason is quite silly and it is a lesson learned for the next release. During the release procedure, we prepare everything on Github first. The master branch has a new version and the devel branch is set to continue on the next release. Then we prepare the artifacts and upload them to Sonatype for them to appear in the public Maven repository. Then our Continuous Integration environment automatically consumes the sources from master and runs all the tests. It also creates downloadable artifacts for automatic web pages refresh.
So after releasing version 3.0, we realized that the tests do not work well in the CI environment. We decided this must be caused by a different environment setup, implemented a fix, executed tests locally and released version 3.1.
But this did not help! Still the same errors appeared in the CI. After a more thorough inspection we realized that there is a problem with the javadoc Maven plugin (the recent version was backward incompatible). So we eventually fixed that and released version 3.2.
Next time, we must run our CI stuff before the upload to Sonatype.
And one more thing - there will be another release, version 3.3, shortly before GeeCON in Prague 2014. We want to present some cool new features there during the Lightning talks.
Přihlásit se k odběru:
Příspěvky (Atom)