CPAN Testers is only made possible with the support of our sponsors.
For more information on sponsoring, please visit the I CPAN Testers website.

Upgrade Notice

The CPAN Testers Blog site has been upgraded since you last accessed the site. Please press the F5 key or CTRL-R to refresh your browser cache to use the latest javascript and CSS files.

News & Views

So last month saw the London Perl Workshop take place. Several talks related to testing, and plenty of interest in CPAN Testers. Mark Keating and crew were videoing many of the talks. Sadly the microphone position meant the volume is quite low in places, but they are still worth watching. You can see testing talks from myself (slides), DrForr, and additionally my lightning talk (slides). If you missed the event, there are plenty of great talks you can now catch on the Shadowcat channel.

Also during last month there have been several fixes to the Reports Builder. Also following some conversations at LPW, I came away with a few ideas to improve the performance of the Feed and Builder components so that we can get almost real time production of reports. Over the coming months and likely during the 2015 QA Hackathon, I aim to work more on this and hopoefully have a solution working around the hackathon. Karen Etheridge also help to spot a fault with the Admin site, in that if a tester hasn't allocated a contact email address, any report that has been flagged isn't allocated to them, but to the Admin user. It does mean there is a bit of extra work for me at the moment, but ultimately I need to rework the way testers are notified, particularly in the event of the system using an old email address. If you've flagged any reports and haven't seen them removed as yet, please give me a nudge and I'll approve their removal.

One person I would to single out for making a great effort to promote testing in general, is Sinan Unur, who has been writing some great blog posts. TDD is all well and good, but who's testing the tests?, You've gotta quotemeta! and Tests should not fail due to EOL differences across platforms to mention a few. He has some great in depth Perl posts, and all are well worth reading.

A colleague of mine, Matt Wheatley, recently had a blog post published about how he works in development. Except the post isn't so much about how he works, but more about the value of a QA Department. Our QA team is awesome, and we perhaps rely on them a bit too much sometimes, but I imagine there are many awesome QA teams out there, and this is just one tribute to them all. I especially liked the reference to Bill Sempf's tweat.

We had another Metabase overflow last month, which means we've filled another bucket in AWS. David Golden has created a new one, but it does mean we need to think about moving to our newer architechure sooner rather than later. David, Neil Bowers and I have been in discussions recently, and I hope we will get some movement towards this in the coming year. There will be a lot of changes needed, both to code and routing, so it won't be a simple change. However, it will be a change that will help us grow more productively in the future.

More new coming next month. And my New Year's Resolution is to get these summaries out a bit more on time!

The server upgrade is now complete. The final pieces of the puzzle were to implement the mailers for reports. After a lot of head scratching, Robert figured out the problem was the filtering for mails coming from the new server. Many thanks to Robert and Ask for helping out here, even if Robert thinks I should be using Postfix ;) Mails are flowing now, and many thanks for everyone's patience while we got them sorted. I'm also seeing all the bouncebacks again, which means I'll need to update the preferences for those authors soon. If you've been missing your mails from cpantesters, please check the address you have set up in PAUSE. If it is old and no longer valid (including those that are hidden from public view), then the perl.org mail server is going to issue a bounceback for them. If you've changed jobs, please make sure that you update your PAUSE email if necessary too, as there is at least one person who is unwittingly sending "no longer works for this company" emails for their PAUSE email.

New SSL certificates are now in place for both the Preferences site and the forthcoming Admin site. Our last provider may have provided them for free, but the hassle in getting them, really wasn't worth it. Thankfully the new ones are now valid for 5 years at a very nice discount.

At the Birmingham Perl Mongers October Technical Meeting, I gave a presentation entitled 'The Future of CPAN Testers'. The talk went down well, and I recieved some good feedback. As such, I have submitted it for the London Perl Workshop on 30th November. The talk will cover some of the changes that have happened this year, and look at some of the projects planned for the near and far future. If you're interested in getting involved in CPAN Testers as a developer, rather than a tester, this talk will be an ideal insight.

Speaking of CPAN Testers talks, many thanks to Gabor Szabo. Firstly for setting up Perl TV, and secondly for promoting the talk I gave at YAPC::Europe 2012 in Frankfurt. 'The Eco-System of CPAN Testers' was my attempt to explain how all the different processes used by CPAN Testers all fit together, detailing the path from report creation to appearing on the Reports website and beyond. Hopefully my latest CPAN Testers talk will be a suitable follow-on companion to last year's talk.

And speaking of the future of CPAN Testers, the Admin site is steaming ahead again. After getting my head round all the email stores, a new script to populate the new tables is off and running. It'll take a while to get through the 36 million reports, but most of the codebase is now ready to go. There are some further changes to the Statistics site and the address maintenance scripts to do, but I'm hoping to have a release date in the next summary.

Hopefully some readers will be at the London Perl Workshop at the end of the month, so please say hello if you're a tester, or want to get more involved in CPAN Testers.

If you haven't come across QuestHub, you might want to take a look at the Perl Realm. It's essentially a TODO list of tasks and projects, which can earn you points, if you like to game these sorts of things. There are several stencils to help get you started, and in particular there are now several relating to CPAN Testers. If you're new to the Perl or CPAN Testers community, and want to contribute in some way, the stencils are an ideal starting point. Some are technical, others are more social, but all are to the benefit of Perl. Neil Bowers pointed me to the site in the first place, thanks to his post Make me do some work, via Questhub!, which in turn drew inspiration from a previous CPAN Testers Summary post.

Last month the CPAN::Changes Kwalitee Service saw 40% of CPAN meet the CPAN::Changes::Spec. In the past month, and thanks to Neil and his stencils on QuestHub, that figure has risen to 41%, and is continuing to improve. While the specification is no way necessary to upload your distribution to CPAN, it does help to have some form of standard format for Changes files. As a consequence, Neil Bowers submitted a ticket to include a suggestion in the Perl documentation, which has now been approved and patched.

Reini Urban posted Smoking CPAN in one line on blogs.perl. Although we would recommend more conventional ways of smoking CPAN, this one-liner certainly does the job. Reini had a specific test he was looking, at, but the outcome has been to raise tickets on several distributions, many of which have already been fixed. Testing for specific bugs is not usually what CPAN Testers is about, but in this instance it proved a very productive effort.

After reading several posts and emails over the past few years, and particularly one in the past few months, I felt a post was needed to provide a bit of clarity. There are many ways to get involved with CPAN Testers, aside from being a tester. If you wish to help improve the tools and websites, feel free to take a look at the repos and see how you can help. Let us know your ideas on the mailing list, and by all means ask for guidance and advice there too. CPAN Testers is continually evolving, but it does take time.

The Admin site is coming along. The original framework has been in place for sometime, as it was originally designed before we had CPAN Testers 2.0 in place, but I've had to redesign some of the work with accessing and indexing tester emails. You can follow my progress on QuestHub, as I complete various aspects of coding and testing. A more public repo will also appear soon.

On the mailing list last month, Serguei Trouchelle highlighted some unusual behavior with a tester using CPANPLUS. Chris picked this up and it seems some of the instructions on the Wiki for testing CPANPLUS might not be as complete as they need to be. Hopefully we can clean these up soon, so new testers can ensure their test environments are working correctly. A few people emailed me to say the CPAN Testers Reports site was down. Every so often several backend process are all running at once, and with a few extra Apache processes on top, the disk I/O can be a bit traumatic. As such the websites are usually the first things to suffer. I'm looking at moving some of the websites around soon, to hopefully reduce the impact.

Olivier Mengué posted that he thought the YAML files that are retrieved from the Reports site were invalid. We use YAML::XS, and the real file it reads from is actually a JSON file. The probalem seems to be with the YAML module, as it requires a final newline character at the end of the file. The results of YAML::XS are sent straight out in the HTTP response, and are not written to disk, and don't include a final newline character. I'm not aware of this being a requirement in the YAML spec, so would recommend people switching to YAML::XS or YAML::Syck if they wish to read the YAML files from the Reports site. Or you could just request the JSON file directly.

This week is YAPC::Europe 2013 in Kiev. Sadly I'm not there, but I hope everyone who is making the most of the event. Four talks specifically about testing are being presented, so please check them out if you can.

We're now up to 33 million reports submitted, which is great. But it does means some spring clean is still on going to keep the disk space down. The databases combined now take up around 610GB of storage, and the websites on the main server (including CPAN and BACKPAN) take up another 150GB. I'll be looking at upgrading our disk allocation soon, but it never hurts to do a bit of spring cleaning. The compression of reports is still underway and over 16 million of them are now compressed. More updates for you next month.

Posted by Barbie
on 10th September 2012

August was quite an exhausting month, with several news worthy items. The YAPC::Europe 2012 conference, as mentioned last month, featured a few talks relating to testing, including my own looking specifically at the main components of CPAN Testers. I have often been asked about how the infrastructure of CPAN Testers works, and hopefully my talk gave some insight into that. The process diagram featured in the talk will be added to the CPAN-Testers distribution soon, and will be expanded at a later date to try and capture all the parts of the current CPAN Testers environments. CPAN Testers has grown substantially from its early beginnings, and its much easier to get involved. With so many parts to the whole system, it would be great to get more and more new contributors to the code base. See the Development site, which also needs updating now, for further links. It also intrigued me to see Test::Reporter::Transport::Metabase in the (xvii) MetaCPAN favourites weekly report post. Hopefully whoever up-voted it, has been looking at contributing too :)

Kenichi Ishigaki was also at YAPC::Europe, and he previewed some of the forthcoming changes to CPANTS. He's done a lot of work behind the scenes getting the site back online. Working alongside his CPANAuthors site, the CPANTS site now looks a lot more modern, and hopefully will get used a little more again. While watching a twitter feed during the conference, I also happened to notice that Brian Cassidy posted about reaching the next goal for his CPAN::Changes Kwalitee Service. As of writing this, he needs another 12 distributions to pass the tests to reach 10,000 passes. While it's not a necessity to have your Changes files in a machine readable format, it is useful to have a convention that works for other uses. It highlights Kwalitee rather than Quality :)

In the last few weeks I've been cleaning up some of the code behind the CPAN Testers Reports site, including some requests posted in RT. As such, the distribution CPAN-Testers-WWW-Reports got a major update and now has better crawler detection, now displays the report counter headers for each distribution release and fixes the summary displays. The backend code has also had a bit of a clean up to improve some of the performance. In the coming weeks as promised some of the crawlers will be let back into the site again. Another big change has been to match the convention used elsewhere so that TRIAL distributions are now marked as development releases. I have also worked through old reports and changed the settings to reflect this too.

Proof, if proof were needed, why we need to move away from Amazon's SimpleDB, is the recent catch-up I started running recently, to collect all the missing reports from the last 3 months, that SimpleDB failed to extract from the Metabase. It's a constant source of confusion from testers as to why their reports are not appearing. The catch-up added 245989 reports to cpanstats database. That's over 80,000 reports that are currently being lost each month. For the money they charge every month, that's exceptionally poor service as far as I'm concerned.

Special thanks this month are reserved for Steffen Schwigon, aka renormalist, who has made a significant personal contribution to the CPAN Testers Fund. Steffen was one of the people who pushed us to get a CPAN Testers Fund set-up a few years ago, so we are delighted to see him come good on his promise to donate. If you'd like to make a personal contribution, please visit the donation page and contribute.

The CPAN Testers Fund is growing, and while we are grateful for every contribution, big or small, we'd love to see more, so we can help pay for all the running costs every year. Over the last month, Mark Keating had been throwing some ideas my way of how he could promote CPAN Testers, and at YAPC::Europe the light-bulb was most definitely on. Mark launched a corporate donation programme for CPAN Testers, whereby companies could contribute a regular monthly donation, rather than a big lump sum. Mark's idea is that if we could get 20 companies all contributing monthly about £20, that would be enough to cover our current expenses for a year. Taking the first step, ShadowCat are the first company to now make a regular donation to CPAN Testers. You and your company can too by simply visiting the CPAN Testers Fund page and setting up a regular recurring payment.

On a final note, I'd like to thank chromatic for mentioning CPAN Testers in a post about why he uses Perl testing. As chromatic notes, the Perl community's attitude to testing, and they have produced to support testing, has made Perl the success it is.

July was a relatively quiet month for CPAN Testers. Although reports have been flowing, our attentions have largely been elsewhere. Development work behind the scenes is still continuing, but nothing major to report just yet.

Ben Bullock asked on the mailing list, whether he could search other people's test reports? The problem currently with this, is that we don't really expose the reports themselves, except via the CPAN Testers Reports website, when you specifically ask for the report. The reasons for this have largely been because the search of the Metabase still needs to be written. The demand on the current Metabase is expensive, and until we are able to move to the new backend system, we can't afford to expose the results. For the time being the Analysis site covers some of the demands, but Ben's specific needs aren't covered.

Ben also noted that because several web crawlers are blocked from the reports site, this restricts the ability to search the reports for common error strings via other sources. The blocks were implemented due to some crawlers being rather heavy-handed. I specifically don't like the way Google allow website owners to restrict their crawlers for a set period, and then blast their way through the site when the period ends. It's counter-productive as they get banned from sites. This wasn't the only reason Google were blocked, but the tactics aren't appreciated. Now that some of the code has been reworked, it is a little easier to restrict the side effects the crawlers can now have on the site. As such, I'm looking at unblocking some of the crawlers to see how they and the site performs. If all goes well, more crawlers will be allowed to access the site. I will be monitoring the crawlers quite closely, and any abuse will result in those crawlers being blocked again. The end result will mean that searching for specific strings, will be possible via search engines.

The next major event in the Perl conference calendar will be YAPC::Europe 2012 in Frankfurt am Main, Germany. I'll be presenting "The Eco-System of CPAN Testers", which aims to show you the process flows of CPAN Testers, highlighting the code used, and where you can help contribute code. Other talks also featuring testing are Eric Johnson with "Selenium testing with Perl" and Kenichi Ishigaki (charsbar) with "CPANTS: Kwalitative website and its tools". There are several talks that look at ways to investigate broken code and improve your approach to code too. The current schedule looks very worthwhile, so if you're still undecided take a look at what you could take away from the conference, and book your ticket now. Hope to see you there.

If you're a CPAN Tester, and will be at YAPC::Europe, please come find me and say hello during the conference. It's always great to meet testers who feature in the leaderboard, but I only know by name. If you have any suggestions for future posts, questions about CPAN Testers, or ideas for collaborations with CPAN Testers, as I'll not be arranging a BOF in Frankfurt, feel free to let me know during the conference or during the evening sessions. If you're not attending YAPC::Europe, please join the mailing list, and post your suggestions, questions and idea there. Happy testing.