This post is extremely late to publish, but I thought I would see publish it anyone, as someone might find it useful. Here are a list of important tickets in the WordPress 5.1 release.
So this is a post that I have been meaning to write for a long time. I think that the fields API and how core handles meta is without doubt the most important problem that is facing the WordPress project. But looking the work done to date on the fields api, it can get a little confusing, the project has changed owners and scope a number of times. In this post, I hope to clear up a little of the history and explain my ideas for the fields API going forward.
So what is the fields api anyway and not having it is such a problem?
The fields API is a bit of a different name for what it really is. It is in short, a library to build custom meta boxes and UI to interact with meta data. The first reference to field api goes all the way back to 2011, in ticket 18179 – WP_Meta_Box. Basically the original idea what to make the need for plugins like advanced custom fields, fields manager and CMB 2 mute by making it easy for developers to register meta boxes and UI for post meta data in core. This would make the job of plugin / theme developers much easier, as it would make adding new fields the post screen extremely easy and would mean that testing on WordPress core updates, much simpler. Plugin / theme developers have long wanted this, as building out custom meta, is a key part of the developing anything custom for WordPress. I believe it true to say, that it is the first thing that most WordPress developers do when building a project, is to set up either ACF or CMB 2. This setup is normally a manual process, as currently there is no way to require interdependencies with plugins (if this plugin /theme is installed, then install this library / plugin). This gives developers two choices. Either hard code the meta boxes / fields that you want and build out their own libraries or bundle a meta box library in with their plugin. Either solution is not great. First, building your own meta boxes, takes time and testing. It means that whenever WordPress core updates itself, that plugins require retesting. Option 2, means having a whole library in the plugin, that adds overhead. If any other plugin is including the same library, it may mean that having both plugins activate at once will break your site.
So having the a fields api in core seems to make sense, from both a user and developer level. It makes developers lives easier and makes WordPress a more usable piece of software for users. But the fields api should not just be limited to post meta, users, terms and comments all to have meta in core. Also multisite too, will have some meta tables, with some work that I am currently working on, by adding site (blog) and network ( site ) meta. So my previous blog post on the subject on site (bl0g) meta and check on the tickets for site and network meta. It is possible to make the fields api the interface for all meta data in core. However, I have also seen people discussing the idea of using the fields api, as a general api for drawing UI for any data stored in the database. That would mean it could also be used in on the core objects like Posts, Users and Options. Where on the face of it, this may seem like a good idea, I am a strongly against this. For me, this muddies the water of the fields api and makes the scope of the project limitless. It makes the project a silver bullet for the project and nearly impossible to ship. The fields api project has been going since 2013 and has had a number different versions and teams working on it. For some reason, this project hasn’t taken hold or got the support it should have. What is worse, is with Matt’s three focuses of customiser, editor and API that this and many other projects are being sidelined.
An API driven future
The reason for limited meta support in the core endpoints is explained in Ryan Mccue’s post The (Complex) State of Meta in the WordPress REST API. In short, meta data isn’t added core endpoints for two reasons.
- Protected data
Core developers have no idea of knowing what data is private / sensitive and should not be displayed publicly.
- Serialised data
WordPress can store serialised objects in meta data and these can be a security problem. There is no way of knowing data type
There is much more to Ryan’s post, do check it out. But it comes down to one problem, meta is not registered anywhere, so there is no way to know, if it should be in the api or not, it’s data type, what are valid values and permissions to read / edit / delete are not defined.
A new hope
In WordPress 4.6 Jeremy Felt made some changes to a little known function called register meta. These changes were designed to allow developers to register some details about the meta data. It allows for some important information to be registered, such as.
- Authorization callback
Adds a permissions call to each meta key. Allow for secure meta data
- Sanitize callback
Allows for meta data to formatted on save, meaning data meta can be trusted to be the correct format / data type.
Does type hinting, so gives core an idea of what data type of data stored.
Allows developers to flag weather or not this meta data in the api at all.
As you might understand, this is a big step forward for meta data. For the first time, meta data is much more control on it’s processing, handling and saving.
It also makes improvement to multisite a lot easier. Multisite has been missing a settings api for a long time. As under the hood, network settings are just (site) meta data, the fields api could replace the need for a network settings api.
This doesn’t even mean that plugins like ACF would die as a plugin. I would still have a place, support fields that core didn’t like repeaters and google maps. But under the hood ACF could be converted to use the fields api, converting many sites already using this plugin, a more structured data set in the database.
With the fields api in core, we can work on having the user experience an enjoyable and consistent user interface across all platforms. With form fields being generated from one place, we can make the process of translation and accessibility much easier, making aWordPress and much more user friendly piece of software. It would allow those using the WordPress via apps or third party connected software, have a much consent experience and hopefully mean less breakable in the future.
But this gets down to the core issue, I have with the WordPress project. The user comes first. All the work that is currently being done on core, is about making end user experience better. I understand this desire, but it forgets one simple thing, that developers are users too. That a part like having a fields api, is a problem for both developers and users, and WordPress would be better if we solved this problem ahead of other things. The fields api, makes things cleaner, faster and more reliable for custom development. It means that developers, can save time and add value to the projects they are working on in other ways.
I would like to thank some people for their tireless work to date.
So as part of the Nijmegen WordCamp, I run the core team at the contributor day. Even through this took up a lot of my time, I still had a little time to work on networks endpoint for core. There was an existing ticket for this and it is on the Multisite roadmap that we have been working on. But of the two require endpoints for full support of the multisite is core (sites and networks), networks may seem like the less important of the two. After all, networks are created rarely (in most cases only once) compared to sites, where sites can be created all the time. However, there is one important feature of multisite that doesn’t have api rest endpoint that really needs it and that is network options. Adding a networks endpoint would add an api not only for networks but also it’s options (meta) as well.
Network options, a history
Network options (or site options as they are sometimes known) different from site level options and are not a feature of WordPress that many people know about it. Network level options are used to core network level configuration in multisite, however not many plugins bother with network level settings. This is partly because many developers do not bother to test their plugins with multisite. The other reason is there isn’t currently a developer api to add network pages and settings easily. For single site WordPress, there is a settings api that makes it easy for developers to build out there own settings panels in the admin terminal. However, there is nothing like this for multisite. Currently the process adding a network options is manual, adding to manual posts requests and manually saving values in the database. This means that developers have to do all their own data sanitation and escaping themselves. There is a ticket to add a similar network settings api written by one Joost de Valk the creator of WordPress SEO however this ticket at time of writing is over 7 years old. With a new endpoint for networks in core, we have an opportunity to improve the whole network options interface. Once there is api for networks, developers can use the CRUD (create, read, update and delete) methods to interact with options and not need to have to do all the data sanitation themselves. We can can extend this more, by creating a network settings interface, that allows developers to register settings to network settings pages. There network settings pages, then become a single page application written in React or similar. This could result in some really nice user interface and make network settings a better than site level settings.
Blocker to this bright future
Even through there is now a patch up for a networks endpoint, there is a lot of work to be done. Currently the first patch, only has read methods. This is because in core there are not currently create, edit and delete network functions. To add this functionality to the api, the functions must be added to core. There is a ticket for this that are based on some of the code in WP Multi Network plugin. The reason these functions have no been added before, is because it would then mean that WordPress core officially support multi network setups. Core has unofficially support multi network for a while, with many commits (#40486, #40590, #40591) going in the last couple of releases. Adding these functions to add a maintenance cost as well, the multisite codebase is large and only has a couple of maintainers work on it at any given time.
Another blocker are the current CRUD functions for network options. Currently there are following functions
I worked on these functions as part of the 4.2 release and were introduced in #34777 . However, these functions were just a simple replacement for the existing *_site_options functions. I wanted to go further with these functions and use it as a chance to leverage the meta api. Under the hood, network options are stored in a sitemeta table. This means that data is structured like meta in the database. This means that the meta api used by post / user / comment / term meta could be used here. It adds a number of key benefits, including
- Improved caching profile
- New filters and actions
- Enables use of register_meta
- All improves to meta api, filter down to network options.
- Possible integration with fields api
I have been working on a ticket for a while now, however this is not a small change. It fundamentally change network options and that has effect on every piece of code that uses them. An example of the effect is if code is saving network options for id 0. It is impossible to have a network with id 0, however some plugins have been using 0 as a way to store global settings. This would break if we use the meta api doesn’t allow for 0 or negative numbers.
As you can see, there is a lot of work to done to make this a networks endpoint happen, but I think it is really worthwhile. Once this work is done, it will make developing for multisite much easier.
How you can help
The key tickets that need feedback and code review are
- Introduce a REST API endpoint for networks / network options
- Use metadata api in *_network_options
- Add multi-network helper functionality
Any help with these tickets is more than welcome. You can also join in on a weekly multisite meetings on slack in the #core-multisite channel every Tuesday at 4pm UTC.
This is the first in a series of posts about what I am working on in the open source community. I am currently in a maintainer on WordPress core and have a number of open source projects on the go at any given time. I thought I would start to highlight some of the code / tickets I am working on lately.
Use metadata api in *_network_options (WordPress Core)
This is a ticket that has been in the works for a while. Basically under the hood, the network options for WordPress multisite are stored in a meta table. This is standardised format for storing meta data on objects and is used elsewhere in core. The rest of the meta functions (such as get_post_meta) used a defined meta api. However the network options functions do not. This patch seeks to convert the network options over to use the meta api for CRUD functions.
Replace $wpdb->siteid with get_current_network_id() (WordPress Core)
The helper function get_current_network_id() was introduced in 4.6. It is already used in a number of places, however there was still references to $wpdb->siteid throughout the core. Accessing the siteid from the wpdb class, is an outdated way of accessing current site.
Site meta (WordPress Core)
Site meta is the idea of adding a new table to WordPress multisite that would allow you to easily extend the WP_Site object. This global store for multisite, as many uses, including replacing domain mapping and blog versions tables.
Add caching to get_adjacent_post (WordPress Core)
The get_adjacent_post functions are used in for the next and previous posts functions in core. Currently this function has no caching by default. It can be an expensive query, as it is doing a date based sql query.
Add caching to WP_Query (WordPress Core)
Current the main query class WP_Query doesn’t have any caching on it. It does have a number of filters and actions, which make it easier to do as a plugin. I worked on a plugin while at Time inc called enhanced post cache. However, I believe core should have this functionality built in. This is a hard ticket to work on, as it effects nearly everything in core. However, I think it is worth while.
Version 1.2 (Echo js Lazy load)
- Updated Echo.js to 1.7.3 from 1.7.
- Use data url instead of 1px gif saving another request.
- Don’t do replacement in REST API.
- Formatting for tests.
WP Concatenator ( wp-concatenator )
There are other things I am always working on, but these things have been my focus of late.
Greeting all and welcome to the Spacedmonkey blog. This blog will discuss all things WordPress and share tips, tricks and code snippets of some of the more advanced things we are working on here in Spacedmonkey.
We are big fans of open source here and all the code we write is open sourced when possible.
We also take requests, so if there is anything you want us to talk about, just in touch in the coments below.