CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Blog site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

News & Views

So last month saw the London Perl Workshop take place. Several talks related to testing, and plenty of interest in CPAN Testers. Mark Keating and crew were videoing many of the talks. Sadly the microphone position meant the volume is quite low in places, but they are still worth watching. You can see testing talks from myself (slides), DrForr, and additionally my lightning talk (slides). If you missed the event, there are plenty of great talks you can now catch on the Shadowcat channel.

Also during last month there have been several fixes to the Reports Builder. Also following some conversations at LPW, I came away with a few ideas to improve the performance of the Feed and Builder components so that we can get almost real time production of reports. Over the coming months and likely during the 2015 QA Hackathon, I aim to work more on this and hopoefully have a solution working around the hackathon. Karen Etheridge also help to spot a fault with the Admin site, in that if a tester hasn't allocated a contact email address, any report that has been flagged isn't allocated to them, but to the Admin user. It does mean there is a bit of extra work for me at the moment, but ultimately I need to rework the way testers are notified, particularly in the event of the system using an old email address. If you've flagged any reports and haven't seen them removed as yet, please give me a nudge and I'll approve their removal.

One person I would to single out for making a great effort to promote testing in general, is Sinan Unur, who has been writing some great blog posts. TDD is all well and good, but who's testing the tests?, You've gotta quotemeta! and Tests should not fail due to EOL differences across platforms to mention a few. He has some great in depth Perl posts, and all are well worth reading.

A colleague of mine, Matt Wheatley, recently had a blog post published about how he works in development. Except the post isn't so much about how he works, but more about the value of a QA Department. Our QA team is awesome, and we perhaps rely on them a bit too much sometimes, but I imagine there are many awesome QA teams out there, and this is just one tribute to them all. I especially liked the reference to Bill Sempf's tweat.

We had another Metabase overflow last month, which means we've filled another bucket in AWS. David Golden has created a new one, but it does mean we need to think about moving to our newer architechure sooner rather than later. David, Neil Bowers and I have been in discussions recently, and I hope we will get some movement towards this in the coming year. There will be a lot of changes needed, both to code and routing, so it won't be a simple change. However, it will be a change that will help us grow more productively in the future.

More new coming next month. And my New Year's Resolution is to get these summaries out a bit more on time!

Sorry October's summary is a bit late, as I had rather a busy few months recently, and finding time to finish this one prove more tricky than normal.

However, first off, a thanks to Gabor Szabo, as one of his screencast featured a brief explanation of how CPAN Testers are intergrated into MetaCPAN. Although anyone familiar with the CPAN Testers Report site, will probably have understood what the numbers mean, the screencast is a handy help to anyone new to MetaCPAN or CPAN to learn more about using MetaCPAN. If anyone else has similar screencasts about CPAN Testers, or wishes to create one, please let me know.

My thanks to Pete Houston for notifying me early in the month that the feed to CPAN Testers had got stuck. Another reason why we really want to move away from Amazon and SimpleDB. This is still being investigated, and hopefully will be a project during the QA Hackathon next year. It also gave me pause to think of better ways the feed and builder could be processing. Thet are both mostly serial process, although there are now 2 builders (distros and authors). However, having worked a lot with MessageQueues over the last year, I'm looking at rewriting both the feed and builders, and again is likely to be my main project during the QA Hackathon in 2015. The ultimate aim, as has been asked on many many occassions, is to try and get as close to real time as we possibly can. On a good day we are less than an hour behind, but being a minute of two behind would be even better.

Karen Etheridge highlighted some problems with the CPAN Testers Admin site. Firstly the login had broken again. I've now permanently fixed this, and uploaded the code to CPAN so the old version doesn't overwrite the new version again. However, Karen also spotted a few other problems. Next up was the find a report by GUID. I'm not quite sure what I tested before, but I suspect I overwrote the working code again :( Again this is now fixed and the code is on CPAN :)

The current CPAN Admin problem is figuring out why, when marking reports for removal, they aren't getting alerted to the tester. They were getting added to the Admin user though, so they weren't lost, but it has highlighted a few cross-reference points that are not being made. Connecting a report to tester, or more imporantly to the tester's current email address is not always straight forward, so the fallback is to email the Admin user. I'm working on the code to reconcile these cross-references better, and in the coming weeks I should get that working properly again. In the meantime if you have marked reports, please bear with me, and I'll get to them as soon as I can. In the process of digging through the report removal process, I also discovered that the Builder mechanism for this was flawed, at least from an Author perspective. I've since corrected this too.

Andreas König noted that several reports from one tester were essentially the same. Breno G. de Oliveira helped to identify the problem, and the reasoning behind it. The reporter for cpanminus, doesn't currently record history as the other smokers, and consequently doesn't filter out repeated attempts by the author to install the same distribution. Breno is planning to work more on the Common Client, and integrate this into the cpanminus Reporter. So if you get similar repeated reports, please be gentle with the tester :)

Subramanian Karunanithi was the casualty of version numbering with CGI-Test. Thanks to some investigation work from Jonas B. Nielsen, he highlighted the latest version was 0.52, but the installed version was 0.104. The problem with version numbering within the installers is that they see 0.104 as Zero Point One Hundred and Four, which is greater than Zero Point Fifty Two. Karen posted a bug report, and Alex Tokarev has now released version 1.000, so all good. However, it does bear repeating that Version Numbers Should be Boring.

Next month we'll have a few notes about LPW, more fixes to the toolchain and some ideas for the future.

After many years of promising to get the CPAN Testers Admin site online, I'm pleased to say it's finally here. You can visit the site at admin.cpantesters.org, where you can login as a tester or an author, select reports and mark them to be deleted and submit them to the respective tester to action.

If you are a tester please review the requests and if the report truly is dubious please delete. If you have any questions as to why an author might have requested the report be deleted, please contact them for more clarification. If you are aware of your smoker going off the rails and submitting bogus reports, this site will enable you to delete them before the author highlights them.

If you are an author, please don't request deletions for reports that you just dont like. This service is meant to help clear out reports that are submitted where the smoker or environment is faulty in some way, and the report it produces does not truly represent the report grade. Requesting reports to be delete simply because you don't want to see any red failures in your pages, will be rejected.

In truth, reports are not actually deleted, they are merely set to be ignored, and the build system will extract them from the reports site so they do not appear in pages or JSON files.

However, the Admin site is more than just a report deletion service. It also allows testers to claim emails (particularly many of their old emails), and set their Name and PAUSE identities in the Statistics Leaderboards. In this regard, please note that the Leaderboard is currently only built once a day, so it may take 24 hours before you see any changes.

In addition to this, the tester can also include a preferred contact address. This may not be one used for testing, but is the one they can receive emails for. As older emails become defunct, it is important that testers keep this email address relevant, to ensure anyone wishing to contact them can do so. Note that this email address will be publicly visable via the Find A Tester service, and will be shown on "raw" reports via the Reports website. If you are a CPAN Author, you may wish this to be your pause email address.

A few testers/authors have already been invited to take a look and hopefully we've ironed out the kinks, but if you do spot any perculiarities, please let me know.

In time there may be other features added to the site, so if you have suggestions for improvements, feel free to file issues on GitHub or in RT.

Last month had a bit of focus on London Perl Workshop. Aside from my own talk, there were a few other testing or CPAN related talks that were well worth attending. Neil Bowers gave us some background to his Adopt a CPAN Module list, and some of the plans to include more metrics. He also detailed some of the changes in his blog post Including CPAN Testers results in the adoption list . It was good to catch up with Neil, and several other more recently recruited CPAN Testers.

For my own talk, 'The Future of CPAN', I looked at some of the work that has been done over the past year, and took a look at the projects planned for 2014 and beyond. Some of the projects are already in progress, and some we hope to have released early in 2014. The future is looking good for CPAN Testers, and there are lots of ideas about how to make it better. There are several tweaks and improvements planned to the existing sites, and hopefully we'll have many more suggestions from you folks over the coming year. I spoke with Andreas König at LPW and we discussed promoting areas of CPAN Testers better. I'm going to look at that over the next year, so expect a slightly different focus each month, as we encourage others to get involved, or make better use of the testing resources we have.

One interesting aspect, included in my talk, was the fact that now we have the new server and are able to server the regular website from a separate disk, the response times have been dramatically improved. As a result, I've enable all the crawlers and spiders. I didn't announce it, as I wanted to see how quickly the spiders would notice, and second to see whether we could cope. The results were impressive. Google noticed almost immediately, although some of their bots have been allowed back in for a little while already, and then Microsoft noticed, followed by plenty of others. It was noticeable when a particular crawler bot spotted they could hit the site, as can be seen with the spikes in the requests. However, what is perhaps even more impressive is that the builder simply shrugged and dealt with it, resolving the requests within a few hours.

Splitting the disks for web and backend files has had such an effect, that we are now frequently less than an hour behind. The only time we aren't is when Amazon can't get their searches right. Thankfully, it looks like the code I've written to compensate for this is working well, and over the past few months the check with the tail log hasn't missed any reports. Once we move to the new Metabase servers, we should be able to rid ourselves of much of this checking code, and have even more reliability.

The biggest project over the last month for CPAN Testers, has still been the Admin site. The new testers database has been created, and the scripts are all in place updating the various tables, including the leaderboard table. The code to use the new leaderboard isn't live yet, but should be going live soon. The site is close to completion, and is undergoing some last minute testing to ensure I have everything covered. Over the Christmas period I hope to invite a few people to test the site and feedback any bugs they spot. All being well, I'd like to launch the site (finally) in the first week of January 2014.

The mailing list has been a little quite last month, although Gabor did post to promote my talk video from last year's YAPC::Europe in Frankfurt. David Wheeler asked about the RSS feeds that are posted from the reports site. As some authors/distributions can have many thousands of reports, is there a cut-off that should be employed. Perhaps the latest 1000 reports, or only those posted in the last 6 months or in the last year? Do you use the RSS feeds (PASS and No PASSes), if so what would your preference be? You can post on the list or email me, and I'll review in the new year the best limit to impose, although provisionally I'll be looking at the last years worth of reports.

The Statistics site will be getting a few extra metrics soon. David Golden suggested some metrics regarding CPAN and BACKPAN, that would be interesting to see. They're on my TODO list, and I hope to address them over the next month. If you have any ideas for more metrics, whether for CPAN or CPAN Testers, that you'd like to see on the Statistic site, please let me know.

That's all for the moment, but expect to see plenty of activity in the New Year. Happy holidays all

The server upgrade is now complete. The final pieces of the puzzle were to implement the mailers for reports. After a lot of head scratching, Robert figured out the problem was the filtering for mails coming from the new server. Many thanks to Robert and Ask for helping out here, even if Robert thinks I should be using Postfix ;) Mails are flowing now, and many thanks for everyone's patience while we got them sorted. I'm also seeing all the bouncebacks again, which means I'll need to update the preferences for those authors soon. If you've been missing your mails from cpantesters, please check the address you have set up in PAUSE. If it is old and no longer valid (including those that are hidden from public view), then the perl.org mail server is going to issue a bounceback for them. If you've changed jobs, please make sure that you update your PAUSE email if necessary too, as there is at least one person who is unwittingly sending "no longer works for this company" emails for their PAUSE email.

New SSL certificates are now in place for both the Preferences site and the forthcoming Admin site. Our last provider may have provided them for free, but the hassle in getting them, really wasn't worth it. Thankfully the new ones are now valid for 5 years at a very nice discount.

At the Birmingham Perl Mongers October Technical Meeting, I gave a presentation entitled 'The Future of CPAN Testers'. The talk went down well, and I recieved some good feedback. As such, I have submitted it for the London Perl Workshop on 30th November. The talk will cover some of the changes that have happened this year, and look at some of the projects planned for the near and far future. If you're interested in getting involved in CPAN Testers as a developer, rather than a tester, this talk will be an ideal insight.

Speaking of CPAN Testers talks, many thanks to Gabor Szabo. Firstly for setting up Perl TV, and secondly for promoting the talk I gave at YAPC::Europe 2012 in Frankfurt. 'The Eco-System of CPAN Testers' was my attempt to explain how all the different processes used by CPAN Testers all fit together, detailing the path from report creation to appearing on the Reports website and beyond. Hopefully my latest CPAN Testers talk will be a suitable follow-on companion to last year's talk.

And speaking of the future of CPAN Testers, the Admin site is steaming ahead again. After getting my head round all the email stores, a new script to populate the new tables is off and running. It'll take a while to get through the 36 million reports, but most of the codebase is now ready to go. There are some further changes to the Statistics site and the address maintenance scripts to do, but I'm hoping to have a release date in the next summary.

Hopefully some readers will be at the London Perl Workshop at the end of the month, so please say hello if you're a tester, or want to get more involved in CPAN Testers.