Welcome to ManageWP.org

Register to share, discuss and vote for the best WordPress stories every day, find new ideas and inspiration for your business and network with other members of the WordPress community. Join the #1 WordPress news community!

Register Learn more
×

ManageWP.org

We need Automatic Code Compatibility checks!

Dec. 29, 2014

Last week I went through mini-hell, updating 28 of my plugins for the latest version WordPress compatibility.

managewp.org/articles/8538/how-the-new-plugin-discovery-tool-made-me-update-my-28-plugins

It took me two days of the time that I do not have (job and family, and just before holidays too).

At the same time I received this email from Mozilla with the subject "Add-on compatibility with Firefox 35.*"

---
Dear add-on author,

Good news! Our automated tests did not detect any compatibility issues with your add-on and Firefox 35.*. We've updated your add-on's compatibility to work with Firefox 35.* so that our beta and release users can begin using your add-on.

We encourage you to view the results of the compatibility test, as some compatibility issues may have been detected but without enough certainty to declare the add-on incompatible:

• SEO Doctor:
▶ addons.mozilla.org/developers/addon/seo-doctor/validation-result/362642

This compatibility bump is server-side and we did not modify your add-on package in any way.

For more information about Firefox 35 compatibility, please refer to this blog post: blog.mozilla.org/addons/2014/12/18/compatibility-for-firefox-35/

Thank you,
Firefox Add-ons team
---

Ive been receiving these for about two years since I made a Firefox add-on. It is such a huge relief that somebody wrote automated tests and takes care of the version compatibility for me.

We need this for WordPress, and we need it badly! It is not that hard to do, 80% of functionality can be achieved with 20% time, and even if you badge something with WordPress 4.1 compatible (auto test) the users can still report problems etc. For developers it is such a huge relief as honestly I do not see my self going through that manual process again.

Not only it would benefit the huge community of plugin (and theme) developers but it would also show a level of engineering excellence and care of the WordPress.org (repository) team.

Comment
24 votes   Flag
Pascal Birchler

In the most simple form an automated test could install the plugin on the latest WP version and see if it breaks.

Problems:

* Which PHP version should be used? Some plugins require PHP 5.3 or higher, but WP requires 5.2. These plugins would be marked as incompatible.
* There's so much more to test, like UI compatibility

Reply | Flag
Reply
Tim Nash

So my gut reaction is to say yes, automated testing is a good thing!
My brain says this is going to get very complicated very quickly, it's also taking emphasis away from the developer and putting it on w.org

The big issue is unlike Firefox, WordPress does not have any real frameworks to build on and therefore nothing to test against. The brilliant thing about WordPress is a plugin can just about do anything, the dangerous thing about WordPress....

How do you create a set of common tests in such circumstances?

Take Pascal example PHP compatibility, sure you can test to see if a plugin uses a function that is not 5.2 compatible but that doesn't mean it's not a brilliant plugin and it is compatible with WordPress just not with PHP5.2. It doesn't make sense to put a negative mark to something 90% of users have no issue with, and those that do should be upgrading their hosts.

What if I use a different coding style, should that be marked?

It's just a slippery slope, however where I do think people should be putting their attention is educating developers to take responsibility for their own testing.

Reply | Flag
Reply
Vladimir Prelovac

In reality, in most cases, plugins are not compatible with the latest version because they use a deprecated function, or because a structure of an object has changed. I think this can be tested reasonably well with code analysis and a database of new/deprecated functions. Checking if it breaks the latest WordPress is relatively easy. A fun thing and bonus thing to do would be to run all repo plugins as two plugins in a pair and see which combination breaks WordPress.

Reply | Flag
Reply
We use cookies to offer you a better browsing experience, analyze site traffic, personalize content, and serve targeted advertisements. Read about how we use cookies and how you can control them by clicking "Cookie Settings". If you continue to use this site, you consent to our use of cookies.