CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Blog site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

News & Views

Posted by Barbie
on 13th July 2011

Back in May there was some discussion on the CPAN Testers Discussion list regarding the statistics available on the CPAN Testers Statistics website. As I was already in the process of cleaning up several pages, some of the ideas looked worthy of including in the next release. Two have made it, and are now available for those interested in the reporting statistics.

The first is a change to the way the leaderboards are created. We still have the master Tester Leaderboard, but now in addition to that we have leaderboards for each Operating System. It was highlighted that new testers might find it hard to get motivated to be a high volume tester as reaching the dizzy heights of the all time top ten takes a lot of effort. However, when looking on a per OS basis, it becomes a lot easier to see firstly where a tester can make a significant impact, and secondly what OSes we desperately need reports for. On the Monthly Operating System page it is reasonbly easy to see which OSes are getting the most attention, but previously we didn't highlight who was making the contributions or help the less popular OSes gain more reports. Hopefully with the new leaderboards, we can highlight the efforts of some of these testers.

Initially we did think that the new leaderboards might hightlight the need for more Windows and Mac OS X testers, but as those leaderboards show we do already have a steady stream of reports, with the likes of Serguei Trouchelle and Florian Helmberger providing some great coverage of those OSes, although we're happy to have more testers contributing too. However, looking at some of the more specialised OSes, we could do with a few more reports for AIX, Dragonfly BSD, HP-UX, Haiku, VMS (which includes OpenVMS). Although Chris Williams does have Dragonfly BSD well in his sights, it would be good to get variety of platform environments submiting reports. So if you're looking for an OS to make an impact with, please take at look the new leaderboards and see what you can help with.

Our second change has been the inclusion of a No Reports page. This page lists all the distributions that have been uploaded to CPAN, where the latest version has not received any reports. There are several reasons why a distribution may not have generated any reports, with the most common reason being that it appears on one or more ignore lists. We have listed some of the ignore lists from our high-profile testers on the page, so if one of your distributions is listed, take a look and see whether its been added to an ignore list. If you believe it shouldn't be on a tester's ignore list, perhaps because you've now resolved the issue that prevented it being tested, please post to the CPAN Testers Discussion list and let the testers know they can now test your distribution.

The list of distributions on the No Reports page is currently being refined, as some distributions do not apply. Distributions that have been repackaged under new names, or are a collection of scripts or data files, will eventually be removed. If you spot any distribution that shouldn't really be there, please let me know.

However, several distributions do deserve to be highlighted on the list, as they are often badly packaged. Although some have been left on the shelf and should perhaps have been deleted from CPAN, many have been uploaded without test files, meta files or suitable make/build files. In some cases the directory structure could do with some attention too. If you're looking for a short project, many of these distributions could do with some TLC. As an aside you may also wish to look at one of the Failures lists to help distributions with too many fail reports.

My thanks particularly to David Golden and Gabor Szabo for the ideas and suggestions for more statistics. There are more statistics to be added, but if you have any more ideas that we could use, please let me know. Patches (as always) are welcome :)

June proved to be an eventful month for CPAN Testers. We had several updates and passed a couple of significant milestones.

Firstly, the updates. As well as the blog and wiki updates earlier in the month, David Golden also released a new version of CPAN-Reporter. This release is notable for two reasons. It now defaults to using the Metabase, rather than the email transport method of old, and also attempts to automated as much of the Metabase ID creation as possible. In order to encourage new testers, simplifying the setup process is a must. David made significant changes to regards auto-configuration, and he's incorporated the same ideas into CPAN-Reporter. Ultimately we'd like to see a common client, which uses the APIs of CPAN-Reporter and CPANPLUS-YACSmoke, and can abstract away much of the configuration and processing of smoke testing. The aim being to encourage more casual testers to send us reports from the real world, as distributions are installed or upgraded.

And so to secondly, the milestones. Our first milestone was passing 13 million test reports. I haven't made a big deal of this for reasons I will come to shortly. Our second milestone, which I should have also included in last month's summary, was the number of test reports we've had submitted this month. In May we had 670243 reports submitted, which at the time was the highest number of reports we'd ever had submitted in a single month. That subsequently got blown away for June having seen 744831 reports submitted in a single month. We have seen some new testers join, but it seems in general testers have just been contributing more. All good news.

Having so many reports submitted does mean that the Reports website builder is now running at full capacity. For the past few weeks we've seen request volumes increase, both generated and processed, and while the builder can cope, it is currently running 3-4 days behind for the oldest request. I have a new mechanism that is being worked on to help improve the bottleneck, which I'll hopefully implement sometime this month. It does mean that the plans to release the code for the Report site has been put on hold for the moment until all the improvements have been implemented.

Gabor Szabo became a regular tester last month, and suggested some ideas for ways to promote CPAN Testers. Following a blog post here and on Perl Blogs, we had several suggestions for things we could do. Andy Lester suggested some ideas for articles, and I plan to work on some of these over the coming months. While these summaries are primarily for the CPAN Testers echo chamber, it would be useful to have some articles and blog posts that could reach out further than the CPAN Testers or Perl community. We have lots of good things to promote, we just need to promote the project in the right way. The numbers might be interesting for analysts, but to the regular user it's more important to promote why having CPAN Testers is a great thing for CPAN and Perl.

If you have any stories where testers have helped you, I'd be very interested to hear from you. If you post an entry in your own blog, please let me know and I'll include a link to it here too. As an idea, David Golden posted his experience some time ago. How have CPAN Testers helped you?

Which brings me back to the 13 million reports milestone. Following the discussion surrounding promotion of CPAN Testers, while these sorts of milestones are obviously of interest to CPAN Testers, with the volumes of reports we are now seeing submitted, the million milestones are now too frequent to make an impact, especially if we start seeing them almost on a monthly basis. As such I'll be making more of a splash about the 15 millionth and 20 millionth report submitted. You can still keep an eye on the milestones we do pass on the Interesting Statistics page.

Which brings me to another update. Several suggestions, includng ones from Gabor Szabo and David Golden, prompted me to look at adding them to the Statistics site, as part of a major rework I had already begun. Some of the pages on the Statistics sites are viewed very infrequently, but take a significant amount of time to construct. The site is built roughly every 4 hours, however, the rework now allows these lesser used pages to be generated just once a day. I'll have a more indepth explanation of the changes in another post soon.

And finally...looking forward to next month we have YAPC::Europe, where both Léon and myself will be presenting CPAN Testers talks. While I will look at the bigger picture of why we have CPAN Testers and how to get involved, Léon will be looking at a specific example where CPAN Testers helped him to develop his code.

We've had several suggestions for new sites, pages, articles, images and code recently, so look out for some news and updates very soon. until next time, happy testing :)

<< August 2011 (4) June 2011 (4) >>