CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Blog site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

News & Views

Posted by Barbie
on 19th November 2008

One of the problems with the CPAN Testers website resources, is that where an author listing of distributions, or the list of versions for a distribution, is required, a lot of backend trawling is done. This is due to the current backends having to refer to 3 sources to get those lists. Even then the resulting lists aren't quite correct, as the version sorting can be slightly weird when you have to take into account every author has slightly different perception of versioning. Sort::Versions goes a long way, but it isn't 100% accurate. The only really accurate way of sorting is on the release date of a distribution, which until now hasn't existed in a single form.

For a couple of months now, CPAN Testers has had it's own BACKPAN and CPAN mirrors. Of the 3 sources these are represented by Parse::BACKPAN::Packages and Parse::CPAN::Distributions. and the 2 index files they use. These can take a long time to parse, and as they don't parse and return any release date for distributions and their version, using Sort::Versions is a reasonable alternative. However, there is a third source and that is the CPAN Uploads that are announced by PAUSE. Due to the time lag of the mirrors, very often a release can be made and not be available to CPAN for several hours, so while no CPAN Testers reports might exist, it's still important to know the latest version.

Previously the last source is the only one that contained any release date information, which prompted me to think about doing it for the other sources. Surprisingly quickly, using the local CPAN Testers copies of BACKPAN and CPAN, I was able to build a basic database of upload data, and tag each with 'backpan', 'cpan' or 'upload', to indicate in what state the release was currently at. Queries now take fractions of seconds instead of several seconds. But, and perhaps more importantly, the sorting of distributions actually makes more sense!

The new database is being integrated into the backend code at the moment, but for those that might wish to have this information available for their own uses, the complete database is publicly available at the following locations:

These will now be updated daily, and once everything else is in place will eventually updated hourly.

At the beginning of the last month, my automated mailer for authors and testers went live. For the most part many will never see the mails it produces, which is good, as it will mean that authors are uploading packages nicely formatted, and testers aren't sending out mangled reports. There have been an average of 3 or 4 authors a month who upload distributions without an archive extension accepted by CPAN/CPANPLUS or missing a version number. With testers it varies depending whether a bogus upload has filtered through the system, but again there are at least a few a month. This check now runs daily, as opposed to when I ran it manually roughly once a month. As a consequence it will hopefully help authors more quickly to know when they have uploaded a package that wasn't quite packaged correctly, and help testers spot when their test queue has got a bad package. If you do receive a mail from the system and don't believe your package/report should have been included, please let me know. It should be noted that authors can upload any archive format they choose, or any file format for that matter. However, to be most useful to end users, particularly those using the automated tools such as CPAN.pm and CPANPLUS, it is recommended that the following archive formats are used: .tar.gz, tgz, .tar.bz2 and .zip. In addition the format for a reliable upload is <code>-.</code>.

Also at the beginning of the month, we had a slight hiccup with the update to the Reports site. Unfortunately Léon was in the process of moving his server and the file used by Parse-BACKPAN-Packages became unavailable. Leon quickly sent me a copy of the file and I pointed the code at a local copy of the file instead of the remote one. However, it did prompt me to think about possibly running a mirror BACKPAN server, and thus creating the file myself. At least that way if anything happens to Leon's server, there is an alternative location from which the file can be grabbed from. Once I've got it working to my satisfaction, I'll shall implement a patch to Parse-BACKPAN-Packages so that it knows about alternative locations. As a consequence, this new BACKPAN mirror is now available publicly via http://backpan.cpantesters.org.

The CPAN Testers Wiki got an update this month too. Following some XHTML fixes and a little bit of a code clean up, I also added syndication feeds for the Recent Changes, as requested by David Golden. The fixes to the XHTML templates, CSS and RSS have hopefully helped to provide a cleaner interface.

Between 13th and 19th October there were some signification changes to the infrastructure of CPAN Testers. The fact that nobody appears to have noticed was a good thing. Expect an official announce soon :)

Over the course of the month, JJ came up with an alternative method of displaying the reports information on the Reports website. Many authors have complained (loudly), that due to the large number of reports their distributions have amassed, it gets difficult to see the important data from the irrelavent (to them). As a first attempt to minimise this the default selections within the Javascript ensured that only CPAN available distributions, tested against unpatched versions of Perl, were seen on each page. However, this also has the problem that for large listings the Javascript can still take a large amount of time to render the page correctly. For some visitors it can also be a bit disconcerting when the page initially locks while it runs the javascript, then half of the page suddenly disappears. This was always a short term measure, and following discussions with JJ, he has come up with a solution that uses a JSON data file, which loads in the browser with the predefined defaults. The new default will be to ONLY render the current distribution, with the ability to "Expand All" should you wish to see the other distribution versions. This will then mean no disappearing report data and no long running javascript. My thanks to JJ for persisting with this and figuring out the best way to render everything. The changes aren't live yet, but expect an update at some point this month.

Other website developments have continued at quite a pace and while some work has mostly been to improve XHTML compatibility and fixes cosmetic bugs, the new sites for Report Administration and Author Reporting Preferences are coming along very well. More news on those in the coming weeks. The CPAN Testers git repository has now been live for several weeks and will be expanding once more of the backend tools and code are cleaned up and packaged.

Again we had 126 testers submitting reports last month, our second highest by month, with 16 new addresses mapped, of which we had 6 new testers identified.

Work continues on for the websites and mailing systems along with numerous other changes within the CPAN Testers ecosystem. More news soon.

<< December 2008 (3) October 2008 (1) >>