Welcome to ManageWP.org

Register to share, discuss and vote for the best WordPress stories every day, find new ideas and inspiration for your business and network with other members of the WordPress community. Join the #1 WordPress news community!

×

Automated compatibility tester for WordPress.org plugins

Development | Apr. 4, 2016

Here it is again, the dreaded email announcing new WordPress version and the need to manually review and up the compatibility version on all 28 plugins I contributed. It is a battle that I can not possibly win long term.

Unless.. we can do something about it. What would it take for the WP plugin repository to have an automated compatibility suite of tests, like Mozilla Firefox for example has been utilizing for years? It seems that spinning an instance of WordPress, running the plugin in question in debug mode and logging output would be fairly straightforward to do with current tools available to devops. If someone in the community developed such a thing, what's the path for it to be adopted to wordpress.org repo?

And here are the two emails that I get from WordPress.org and Mozilla, it is easy to see which one I like much better.



WordPress.org email
------------------------

Hello, Vladimir Prelovac!

WordPress 4.5 is scheduled to be released on April 12. Are your plugins ready?

After testing your plugins and ensuring compatibility, it only takes a few moments to change the readme "Tested up to:" value to 4.5. This information provides peace of mind to users and helps encourage them to update to the latest version.

Here are the current "tested" values for each of your plugins:

* https://wordpress.org/plugins/broken-link-checker/ (tested up to 4.4.1)
* https://wordpress.org/plugins/category-search/ (tested up to 4.2)
* https://wordpress.org/plugins/cleveritics-for-wordpress/ (tested up to 4.2)
* https://wordpress.org/plugins/footer-javascript/ (tested up to 4.2)
* https://wordpress.org/plugins/insights/ (tested up to 4.2)
* https://wordpress.org/plugins/live-blogroll/ (tested up to 4.2)
* https://wordpress.org/plugins/navbar/ (tested up to 3.3)
* https://wordpress.org/plugins/no-frames/ (tested up to 4.2)
* https://wordpress.org/plugins/plimus-for-wordpress/ (tested up to 4.2)
* https://wordpress.org/plugins/plugin-central/ (tested up to 4.3.1)
* https://wordpress.org/plugins/pretty-theme-files/ (tested up to 4.2)
* https://wordpress.org/plugins/seo-automatic-links/ (tested up to 4.1)
* https://wordpress.org/plugins/seo-image/ (tested up to 4.1)
* https://wordpress.org/plugins/seo-super-comments/ (tested up to 4.2)
* https://wordpress.org/plugins/shellshock-check/ (tested up to 4.3)
* https://wordpress.org/plugins/show-me-options/ (tested up to 4.2)
* https://wordpress.org/plugins/smart-youtube/ (tested up to 4.1)
* https://wordpress.org/plugins/snazzy-archives/ (tested up to 4.2)
* https://wordpress.org/plugins/theme-test-drive/ (tested up to 4.3.1)
* https://wordpress.org/plugins/what-should-we-write-about-next/ (tested up to 4.2)
* https://wordpress.org/plugins/worker/ (tested up to 4.4)
* https://wordpress.org/plugins/wp-digg-this/ (tested up to 4.2)
* https://wordpress.org/plugins/wp-figlet/ (tested up to 4.2)
* https://wordpress.org/plugins/wp-nofollow-categories/ (tested up to 4.2)
* https://wordpress.org/plugins/wp-quick-deploy/ (tested up to 4.2)
* https://wordpress.org/plugins/wp-wall/ (tested up to 4.2)
* https://wordpress.org/plugins/yahoo-news-feed/ (tested up to 2.9)

For each plugin that is compatible, you don't need to release a new version -- just change the stable version's readme value.

Looking to get more familiar with 4.5? Check out this roundup post on the core development blog: https://make.wordpress.org/core/2016/03/30/wordpress-4-5-field-guide/

Thank you for all you do for the WordPress community, and we hope you will enjoy 4.5 as much as we do.


WordPress core contributors


Mozilla add-on message
-----------------------------

Dear add-on author,

Good news! Our automated tests did not detect any compatibility issues with your add-on and Firefox 46.*. We've updated your add-on's compatibility to work with Firefox 46.* so that our beta and release users can begin using your add-on.

We encourage you to view the results of the compatibility test, as some compatibility issues may have been detected but without enough certainty to declare the add-on incompatible:


• SEO Doctor:
▶ https://addons.mozilla.org/developers/addon/seo-doctor/validation-result/521518

This compatibility bump is server-side and we did not modify your add-on package in any way.

Have you considered putting some of those up for adoption? wordpress.org/plugins/tags/adopt-me

Maybe someone else would be happy to give them some attention, and have the time for things like this. Not that I disagree with the concept of making the process easier. I'm all for that. But I know from experience that some projects would be better off in someone else's hands when I don't have time to devote to them.

via Donna Cavalier

Yup, it is easy to see which email you prefer.

How would that work? All I can think of off the bat is that you copy the plugin into a fresh copy of the release in question and activate it ... then check for errors. Some errors appear on use, so that would only catch the first wave. How could you automate a test beyond that? There may be some search in the plugin code that happens when you first submit it, and I guess that could be run again.

via David McCan

Not each project needs continuous development. Some are working fine as they are, just need to test if its fine, and list as compatible. And I don't think Vlad wants to give up his plugin up for adoption, he loves all of those!

via M Asif Rahman

Remember this service? Details here - addendio.com/announcing-wordpress-1-click-demo-links/
This could be good tool to get started with.

via M Asif Rahman

Probably the most appropriate place to voice such a proposal would be in WordPress Slack in the #meta channel. After getting some feedback from folks there in real time, the next step would be to open a ticket here: meta.trac.wordpress.org/query?status=accepted&status=assigned&status=new&status=reopened&status=reviewing&component=Plugin+Directory&col=id&col=summary&col=component&col=status&col=owner&col=type&col=priority&col=keywords&order=priority

That's where it would get official acknowledgement and support if it were to actually be created. Cool idea though!

via Kyle Maurer

Luca here, the guy from Addendio.com

I did not think of Addendio that way, but I share the pain of updating the Readme.txt like everyone else.

question is: how would you define a automated generic test for each plugin?

a simple success/fail upon installation of the plugin with the new version of WP?

Vladimir, can you share a screenshot of what Mozilla shows in their test?

via Luca Fracassi

Orion Wins My Heart

Development | Apr. 8, 2016

Had a client ask me to test some functionality today that involved importing demo data onto a live site. I was all like, "Dude, hold up there. That sounds like a great way to make a huge mess of things. How about I clone the site to a subdomain, import the demo data there, and we can muck around in the resulting mess and see just how bad (or good) the result might be".

I then whipped out my handy-dandy magic tool - in other words - ManageWP Orions Clone feature, and after creating the subdomain and database, filling out the credentials form, and waiting breathlessly for a couple of minutes...voila, there was our test site. I then imported demo data into the clone site, and madness reigned! But the madness was confined to the test site and not the live site, so all was well.

Thanks Orion!

Now, I'm not sure if cloning back to the live site would work well or not. If anyone has experienced going back the other way, after playing around with the test clone, would love to hear about it. (Assuming of course, no changes were made to the live site while playing with the test site).