CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Blog site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

News & Views

Posted by Barbie
on 24th April 2009

The CPAN Testers Preferences site has for sometime now only offered the ability to allow authors to receive the Daily Summary Report. This was a reaction to the authors who did not wish to receive the deluge of reports that they had started to receive, with the increased volume of testing and reporting done by CPAN Testers. However, there have been several authors who have asked to still receive the individual reports, while others are happier to receive less frequent reporting.

After several test runs over the last couple of weeks, the site can now offer the following reporting options:

  • Daily Summary Report
  • Weekly Summary Report (choose your prefered day)
  • Monthly Summary Report
  • Individual Reports

The Daily Summary Report remains as is, and has received no changes. The Weekly Summary Report and Monthly Summary Report simply provide the same style of reporting as the Daily Summary report, just over a longer period. Note that you can choose which day you wish to receive the Weekly Summary Report too.

Lastly we have the Individual Reports. Essentially this allows the author to get the reports direct from the tester again. However, it comes with several caveats. Firstly the process behind the scenes to generate the mails is initially only going to be run once a day. Once any issues that surface over the next few weeks are resolved, this will become more frequent. Secondly, the mailer process does not receive the reports directly from the tester, it is not a mail relay in the traditional sense. The latter therefore could cause problems.

The process for Individual Reports is to read the reports from the CPAN Testers database and regenerate the original email. Except it isn't the original email, as it isn't sent from the testers original IP. This is where problems may arise. The mails are sent from the CPAN Testers server, and any receiving server that implements SPF checking, will undoubtably reject the mails. This is perhaps most likely to affect reports from testers that use GMail or Hotmail accounts. Some authors have a challenge/request email setup, which could prove troublesome unless you're able to whitelist the CPAN Testers server in some way. However, if you've actively selected to receive Individual Reports, then I would hope that you are prepared for this. Lastly, we have the issue of dubious email addresses used by testers. While 'root@localhost' can get through some systems, it may get rejected by a few.

Over the next few weeks I shall attempt to watch for any issues (as above or otherwise), and will review how the mails are sent out. If specific domains or addresses prove too troublesome, then I may have to implement a blacklist to prevent some reports from being sent. If you wish to receive the Individual Reports and spot any issues client side, please let me know and I will look into it. If you have suggestions for improving the reporting process further, then please let me know too. Though always bear in mind this system currently operates for almost 4,000 authors :)

File Under: prefs
NO COMMENTS

Okay so there weren't exactly a hundred million, but Chris Williams did pass the 1 million reports submitted mark on 10th March 2009. Andreas König also passed the 500,000 reports submitted mark. Together with Slaven Rezić, David Golden and Dave Cantrell, these 5 CPAN testers now account for 2/3rds of all the reports now submitted to the CPAN Testers database. It was fantastic to have 4 of the 5 guys all meet up in Birmingham recently. I did mean to get a group photo during the hackathon, but sadly forgot :( Thanks to all the testers, for helping to make the CPAN one of the most admired code repositories in the world.

As mentioned above, we had a few stars meet up in Birmingham at the end of March for the 2009 QA Hackathon, all of whom made the event extremely worthwhile. The event featured 3 days of brainstorming, coding and testing. It did also feature some eating, drinking and a little bit of sleeping along the way too. The full results and thoughts of the hackathon will no doubt appear on blogs and in write-ups in the coming weeks, but I for one am really please to have been a part of it. CPAN Testers was just one of the featured projects, which saw several ideas come together. CT2.0 is firmly within our sights and we'll be looking to unleash it to testers very soon.

With the hackathon over, and the fact that we've now had two very successful QA Hackathons, it would be nice to continue the event next year. There is a possibility that another European location might be able to host the event next year, but there is no reason why there can't be a US one too. I'm sure there are several US developers that would love to take part, but just can't afford the airfare to Europe. As there isn't a formal committee to decide a location, it's pretty much up to a local group to decide they would like to host the event. Perhaps you work for a company willing to provide a venue and wifi, or have funding to find somewhere equally suitable. Either way attendees from past hackathons would be delighted to hear from you. If you do want to host a QA Hackathon, please get in touch with JJ, as he holds the DNS for the qa-hackathon.org domain, and would be only too pleased to point '2010.qa-hackathon.org' in the direction of your web server.

During the hackathon the Metabase, that David Golden and Ricardo Signes originally devised in Oslo, got a wider audience, which meant it also got some meat and flesh on its bones, as well as giving everyone the chance to pick holes in some of the design decisions. Thankfully David and Ricardo did a very good job in the first place, so it was mostly ensuring that the edge cases were covered, and that we had an agreed method to migrate existing testers and the testing infrastructure. Looking at David and Ricardo's work so far, I think we have something very exciting to look forward to. There was a lot of CPAN Testers related effort over the weekend, with Chris and Jos working together on CPANPLUS related tasks, pulling in David Golden to review how to abstract the reporting code into a common distribution for CPANPLUS and CPAN.pm. Tux and Andreas looked at the structured reports, specifically looking for better ways to parse the environment to provide much more detailed reporting, even more than we have previously managed with CPAN Testers. Rich Dawe, working from David and Ricardo's design, wrote the transporter that will now takes the structured data and feeds it into the Metabase. The Metabase itself got a name change, as during the discussions, Ricardo realised that it could eaily be used for much more than just CPAN. It still needs work, but I think it's been a very productive few days, and I looking forward to getting the Metabase finally online. If only for the fact that we will finally be able to switch off the SMTP and NNTP parts of the current system.

A couple of updates I made behind the scenes during the hackathon, included the switch to using PAUSE for the direct mirror and adding in more reporting capabilities to the Authors Notification system, that currently produces the Daily Summary Reports. With the former, Andreas' script now updates the local CPAN Testers CPAN mirror every 60 seconds, which means the database on the backend of the CPAN Testers infrastructure is now very up to date. Unfortunately due to the time taken to update all the web files, you don't get to see the benefits as quickly as I'd like, but it's on my TODO list to improve the processing. For the author reporting, shortly you should be able choose weekly and monthly summary reports, as well as the ability to switch on individual reports again. The individual reports were switched off by the testing tools, due to the number of authors that were unhappy receiving them. However, there are still several authors that would like them. If all goes well with the final round of testing, expect an announcement next week.

We topped 143 testers submitting reports last month, so thank you once again to everyone involved. The mappings this month included 18 total addresses mapped, of which 10 were for newly identified testers.

Posted by Barbie
on 2nd April 2009

Last night Slaven emailed me to let me know that the CPAN Testers Matrix that he runs had lost its domain. Thankfully he still has the site running on a new IP. As such from here on in, if you have the old URL bookmarked or referenced in documentation or on websites, you'll now need to use the sub-domain previously set up under the cpantesters.org domain.

File Under: matrix
NO COMMENTS
<< May 2009 (3) March 2009 (1) >>