CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Blog site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

News & Views

CPAN Testers Reports had a quiet month in April. Much the work to upgrade the Generator code has brought the Metabase feed back into line, and the site builder has been coping much better. The Generator code changes have meant an upgrade to the lastest Metabase API, which has improved the throughput, but also the gaps between search times has reduced, meaning that the blocks we get back from SimpleDB are much more likely to be the ones we want. Also the logic to track back and find the gaps has been improved. All in all, its meant we haven't been missing reports.

There were two discussion on the mailing list, that highlighting two problems CPAN Testers currently has. In the first Sisyphus asked about a problem he'd spotted with failure reports being created, where prerequisites were unmet. The first cause for confusion is when two version of the same distribution (including the version number) are uploaded to CPAN. This can only happen when the author is different. Sisyphus, who maintains the official version of the module, had uploaded a development version, unaware that an unauthorized version had been uploaded. Technically the unauthorised version shouldn't really be tested, but there isn't anything in any of the CPAN Testers clients to prevent this, or at least issue a warning. As a consequence it was initially unclear which distribution the reports related to. As it turned out they were for the official version, which helped to further understand the problem originally raised, and which was eventually resolved by uploading a fixed META.yml file.

In the second discussion, Karen Etheridge asked about unmet 'configure_requires' prerequisites. The problem was due to old installer tools being used, which raise the interesting question of what should be an acceptable minimum version of the smoker tools used to smoke CPAN? This is a complex question to answer, as not only should we consider the perl version, but also the platform version. I have suggested we capture this on the CPAN Testers Wiki, so we can improve it over time. Ron Savage made an interesting suggestion of a new meta key of 'smoking_requires'. Although a nice idea, it would mean a significant change to all the installers, and I suspect that is unlikely to happen soon, if at all, as it would still require upgrading the installers to a minimum version first. Sometimes it can be useful to see how older installers cope with installing a recent upload, and in many cases there may be no problem at all. However, having a set of suggestion minimum requirements for smoking, would at least highlight what versions to upgrade to, should smokers hit problems where they frequent submit erroneous reports. It will be something we will be looking at more in thr future, but if anyone has some suggestions as a starting point, please feel free to start a wiki page or two.

This question also highlighted why it can be very useful to look at the CPAN Testers Analysis site, run by Andreas, which analyses a number of reports to see where differences and similarities are. Looking at a single report may not be enough to pin-point the fault, but understanding a broader range of reports, particularly across perls and platforms, may give a better picture to help dig deeper.

A. Sinan Unur posted about a slight change to the tests he'd made for Crypt::SSLeay. It highlighted that potentially some smokers had machines running smoke tests that could be vulnerable to the Heartbleed Bug. Although it was acknowledged that the test were incomplete, it did re-iterate the need for smokers to ensure their machines were appropriately upgraded if they were using OpenSSL. As a side note, all the CPAN Testers servers were upgrade within a day or two of the original bug being announced.

Bruce from NebCon Inc, posted about a situation I'd already flagged up with Chris Williams. On one of Chris' smokers reported a strange fault was happening that produced 'Argument "2.07_02" isn't numeric in subroutine entry'. It was an odd one because the error was related to File::Temp, which requested a minimum version of File::Path, but one that wasn't a developer version, '2.06'. David Golden suspected the way the version is evaluated may be the problem in UNIVERSAL::VERSION, which could be cured by installing a more recent of version.pm. A newer version of File::Path is available, so in te short term it may also be just as simple to require that.

David Golden has asked everyone to upgrade CPAN::Reporter, if you smoke with CPAN+CPAN::Reporter. CPAN 2.05 now reports optional prerequisites and David has updated CPAN::Reporter to be able to deal with these. This in turn raised a note from Reni Urban, where he had notice differences between Task and Bundle installs. His particular use case does show a difference in the way installers handle the Bundles, as opposed to Tasks, but for the particular Task::CPAN::Reporter suggested for this upgrade, it shouldn't be an issue.

We do have some more news in the pipeline, and have a few things to annouce before the end of May, so look out for those.

So the big news for Spetember was the CPAN Testers Server upgrade. I posted progress on the mailing list, as well as via a Questhub quest, and was hoping no-one would be caught unawares, sadly some were. Despite a few initial hiccups the migration from old server to new server went very well. A few interesting lessons were learned, particularly with MySQL and a reminder of how bad SimpleDB is ... again. On the old server I had been using Metabase v0.16, but last year Ricardo released a newer version, with some changes to the API. Not realising this the initial install for downloading metabase reports failed. Thankfully the changes were minor, and fit better with DBIx::Class style of queries. Unfortunately the result was still a problem for SimpleDB. I have reverted back to v0.16 for the time-being, but will be upgrading to the latest version again soon. A big thank you to Chris from Bytemark Hosting, again for helping getting us upgraded.

The biggest difference, now we're on the new server, has been the speed at which pages are built and loaded. With the website pages sitting on a separate SSD disk from the db and all other files, the response times have been significantly improved. This has also meant that the builder is running almost constantly now, and is keeping up very well. The report generator is also running much better, and is getting blocked by disk I/O considerably less, such that we are now roughly 1-2 hours behind the report submissions to the Metabase. Okay so its not realtime (a much wielded stick waved at CPAN Testers), but its not far off it.

Another benefit to the speed improvements, have been to remove all the restrictions on spiders and crawlers. For the last week this has been monitored carefully, and thankfully no-one has been abusing it. One thing that has been interesting to see, is that Google's RSS Feed-Fetcher tells you how many subscribers it has for a particular RSS feed. In most cases it's just 1, typically the distribution author I suspect, but it would be great if other RSS aggregators did the same, and there are several by the looks of things, as it would be an interesting stat to have on the respective pages.

The Admin site has been put on hold while I worked on the server upgrade, but I'll be back on it very soon. Most of the site is complete (see the Questhub quest for details), so it shouldn't be too long now before its live. There are some changes happening in the database, so we can finally link the tester relationships (email addresses, PAUSE and names) between all the reports and websites. This will finally mean that the Tester Leaderboard on the Statistics site can be built with very little duplication. It will also mean that testers can manage their own appearance on the leaderboard too, saving me having to clean up entries manually every few months, or on request from testers.

Ben Bullock created a quest on Questhub to add text search of reports to CPAN Testers. I originally posted that this was already planned. However, after further consideration, I think it would be worth having fresh eyes looking at building a search site. However, the new reports are not text based. We present them as text based on the Reports website, and in emails, but that is not how they are recorded in the Metabase. Reports are now structured data, and any search site should take advantage of that. As such, I've asked Ben if he can look at writing a search site that can incorporate specific searches that can take advantage of this structured data. Ben's original target of December 2014 should definitely be possible, although I suspect Ben will be able to have some results long before this. It would also be an ideal project for next year's QA Hackathon.

Although happening in October, I did want to thank Alceu Rodrigues de Freitas Junior, for adding the page SmokeTesting on the Wiki. If you have any notes that would help others, whether that involves setting up a smoke environment, or hints and tips for authors or testers, it is very much appreciated. The Wiki is open to all to update, so please feel free to add or update entries to keep the site current.

On the mailing list, Ron Savage posted a query about Tests fail but strings identical in a test report. The problem is specific to Win32, and highlights the need to understand how Win32 I/O layers. Tony Cook posted a response explaining how to solve this.

When I firsted announced the server upgrade, Alex Balhatchet asked who pays for the servers. For the CPAN Testers Server, this is currently Birmingham Perl Mongers, but this will likely only be for another 2 years, after which we'll have to start looking for either a dedicated sponsor or dip into the CPAN Testers Fund. It's probably a good opportunity to remind people about the Fund, as although we do have a lot of support for CPAN Testers, we would welcome more to help us stay vibrant for many years to come. If you or your company want to show your appreciation of CPAN Testers, monetary donations are always very welcome. If you think you can help in other ways, please get in touch.

That's all for now, but expect plenty of news next month too.

This coming weekend, 12th-14th April will be the 6th annual QA Hackathaon. Once again returning to the UK, and will be hosted by the folks at NorthWestEngland.pm, in Lancaster. There will be several CPAN Testers in attendance, and hopefully several CPAN Testers related projects that will feature some attention. However, the event is much bigger than CPAN Testers, and there are plenty of projects involving testing, CPAN, the installation toolchain and others. If you're not attending, but would still like to help over the weekend, join the #perlqa IRC channel and let people know your available. With so many projects being worked on, additional coding help from remote hands are often very welcome.

March ended up being a slightly quieter month on the mailing list. Bo Johansson answered his own beginners questions, which I aim to put on the wiki at some point, unless anyone else gets their first. Chris Marshall asked if we were able to speed up the report feedback for PDL. Unfortunately, this is something that is dependent on a number of things, not least the speed of processing all the reports coming in. There were some issues last month, which have been (or are being) fixed, but the general speed of processing could do with improving. Disk I/O is now the bottleneck for most operations, so I'm looking at ways we can improve that. However, I would like to emphasise that the current processes run automated with very little manual intervention. Fast tracking reports for a particular distribution is not something the system is designed to do. If the process is running too slow for you, then by all means have a look at the code and suggest ways to improve it further. Alceu Rodrigues de Freitas Junior asked about distroprefs, which adds to Bo Johansson's beginners questions, and was answered by Steffen Schwigon, which again I'll be looking to add those to the wiki.

We're fast approaching the 30 million mark for number of reports submitted, but I suspect we'll have to wait until after the QA Hackathon for when it's submitted.

We have some news in the pipeline for CPAN Testers, and once we've had a chance to discuss the details at the QA Hackathon, we'll let you folks know about it all. If you can't make the QA Hackathon, remember you can always join the IRC channel, or even spend the weekend writing code. More next month.

Posted by Barbie
on 19th February 2013

This month's summary is rather late, as sadly I have been busy with other endeavours. As such it'll be a little brief.

The mailing list was rather vibrant last month and several posts generated much discussion. Ricardo's post regarding testing development versions of Perl continued, but there were several new discussions. Bo Johansson posed some Beginners questions. All very good questions, and ones we should look to adding to the Wiki. Unfortunately, Bo hasn't had a response as yet, but I will look to providing some answers, which others can expand on.  David Golden announced he was looking for a maintainer for CPAN::Reporter. Breno (GARU) stepped up and offered his services, fresh from his exploits with CPAN::Testers::Common::Client, which hopefully will form the foundation of all CPAN Testers clients.

The last post to highlight is from Karen Etheridge, who asked about Ignore lists for a specific PAUSE id. This stemmed from a misunderstanding about ignore lists. Ignore lists are compiled by individual testers and are used to skip testing distributions that are known to be problematic. Some just won't work due to external libraries that are not installed, others hang and some are just too troublesome for testers to test. These lists are sometimes made public, so that authors can see why their distributions may not have test reports, and to help other testers to create their own ignore lists. While author can ask specific testers about their ignore lists, there is no central list that can be updated for all, as every test environment is different, what works for one one testers may not for another.

However, although Karen's post started from a question about irgnore lists, it was really asking about a particular set of failure reports from one tester. As several investigated further, it became clear that the environment was using NFS, which appears to be the cause of the problem. There are fixes for File::Temp and IPC3::Run, so if you encounter similar problems, it may be worth including prerequisites for these modules, which include the appropriate patches when they're available. The NFS environment is a valid one, but unfortunately not so easy to determine. We may well have to see if we can add that sort of metadata to test reports in the future.

That's all for now, and as the discussions continue into this month, I shall be wrapping them up in the next summary.

File Under: summary / wiki
NO COMMENTS
Posted by Barbie
on 19th August 2011

Recently Jon 'JJ' Allen sent me some updated graphics with the official new design of the onion for all the CPAN Testers websites. As such I'm now in the process of updating the sites. In most cases you will likely not notice a difference, however, I've taken this opportunity to push out the changes I've been working on for the Blog site. If you haven't already, you may need to refresh your cache to see the latest changes.