CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Blog site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

News & Views

Posted by Barbie
on 12th March 2012

Due to the vastness that are CPAN Testers reports, David Cantrell has had to move the CPAN Dependencies site to a new server. In addition the cpXXXan service that David provides has also moved.

Both CPAN and CPAN Testers have grown at such a rate that providing these services in virtual hosts is no longer manageable. As such both have now move to their own dedicated servers. While there should be no visible difference to the sites, and all domains shoiuld resolve to the new servers, please David know if you experience any problems at all.

Thanks as always to David for providing these services to the Perl community.

With 20 million reports, CPAN Testers is very definitely one of the biggest online repositories of test reports for both programming languages and software applications. While other languages and applications may have larger communities than CPAN Testers, the Perl community's commitment to testing has uniquely enabled CPAN Testers to build on the benefits from many areas of the test community. The TAP protocol has now been incorporated into several other language and application testing frameworks, and stand-alone applications, such as Smolder, have been able to harness the output to present results in a way which best highlights problem areas. CPAN Testers too has been striving to improve the way we present and provide analysis of reports.

With regards to the analysis CPAN Testers provides, a notable example has been the work by Andreas König with the CPAN Testers Analysis site. By taking a selection of test reports, his tools have the ability to find common areas which help authors to pin-point unusual problems. One aspect this has recently helped with, is identifying a problem with development releases, which in themselves don't necessarily fail their tests (possibly because the don't include tests which test all scenarios), but when used within other modules may highlight faults in the prerequisite. The problem with this is that the report is associated with the calling distribution, not the prerequisite which has the fault. Andreas' tools recently were able to identify a prerequisite that was causing concern for a number of distributions, even though the fault lay elsewhere. This in turn led to a lengthy discussion on the CPAN Testers Discussion List, of how we could best approach this. The difficulty though is writing parsers that would automatically determine what was a fault. With many of the test reports it requires a human to determine this for single reports, while Andreas' tools can only highlight where further human investigation is needed, and needs many reports to determine if there might be a cause that may need investigating.

In the short term it is possible to re-associate reports with other distributions, by requesting to me, but in the longer term it would be better to enable a automated process. Which is where the Admin site comes in again. Although that's been on hold for a while, I think this is another feature which could simplify the process of correcting mis-filed reports. As such, I hope to continue on that site soon.

The discussion on the mailing list asked whether we could more appropriately filter these types of reports, but didn't lead a conclusive answer. However, it will be a discussion topic for the forthcoming QA Hackathon. At the end of the month, a number of testing hackers will be assembling in Paris to further Perl's testing infrastructure, processes and code. It will be a good opportunity to have a number of different views discussed in one room with all those likely to be actioned with writing the solutions. This QA Hackathon looks to be the biggest so far with roughly 40 people attending. There are a diverse set of projects being planned for the 3 days, so expect lots of output afterwards.

Gabor Szabo highlighted an issue running CPAN-Reporter with Net-HTTPS-6.02 last month on the mailing list. Thanks to Manoj Kumar, if you experience a "fact submission failed" error, try rolling back to Net-HTTPS-6.01. Tim Bunce also asked about testing on Windows. Although Windows is still a regularly tested platform, the volumes are considerably less than for other more Unix based OSes. David Golden put in a lot working getting CPAN-Reporter and Strawberry Perl working on Windows, so if you have the spare capacity and want to get involved with CPAN Testers, Windows is certainly an ideal platform to make a difference. And if you're running Windows, Cygwin would welcome some more regular reporting.

Lastly, for now, the problems with the SQLite Database have surfaced again. Although I can't definitely say why there is a problem, it would seem to be related to the sheer size of the data CPAN Testers stores. The uncompressed version is now over 8GB. To reduce the bandwidth, and better provide data in a way that consumers can store and use it, an API is being written to supply a record set, rather than the full data set. Expect further work on this during the QA Hackathon.

Posted by Barbie
on 7th March 2012

I am rather delighted to be able to announce that on 22nd February 2012, CPAN Testers reached a very significant milestone in our history. We reached and have now surpassed 20 million test reports, as can be seen via the Interesting Stats page. That's approximately 14 million via the HTTP API.

For the past few months this has been nearly 1 million reports each month. Expectations are now that we could reach 30 million by the end of the year. This is a phenomenal number of reports, and I did notice a little while ago, that a certain other test report repository have now removed their claim of having the biggest database of test reports. It's going to be a long while before any other code repository has anything like the number of test reports available for authors and users to analyse. In part that is also thanks to the 24588 distributions now on CPAN and the 5576 active CPAN authors, but is also thanks to the 1,000+ testers (even those who've only submitted one report), who keep making CPAN Testers a worthwhile project to be involved with.

Congratulations to Andreas J. König for posting the 20 millionth report. It was an UNKNOWN report for Makerelease-0.1.

<< April 2012 (4) February 2012 (1) >>