Blog

REST API Blocks plugin update

The REST API blocks plugin, has been available to download via github, since June 2019. You can checkout the original blog post where I discuss the use of this plugin here.

But I got a request from a user to add the plugin to the WordPress.org plugin repo. Normally, I don’t submit my plugins to the plugin repo, as they are developer focused, but in this case, it seems to makes sense to submit this one. So if you want to download this plugin, it can be found here. If you are using composer, I have also submitted it to packagist, so it can also be installed via composer.

Along with the plugin now easier to get than ever, I also released version 0.3.0. This update includes.

  • Improved handling of table blocks.
  • Improved translations.
  • Improved error handling, if composer install is not run.

The future of the plugin

I am currently actively maintaining the plugin and adding new features. If you have any feature request, please feel free to add them in as a github issue.

As of WordPress 5.8, there is a new block based widget editor. There is a pull request to a block data to the new widget rest api. I hope this functionality merged and get a version 0.4.0 out very soon.

I want to get to the point, there the unit test coverage is a such that it is a plugin that can be used widely. It is useful for others to use this plugin and help me find bugs and test the code. I hope in the future, that this plugin could be used to create a template for a WordPress core patch, meaning this functionality would be part of WordPress. But while blocks are actively changing and with full site editing around the corner, but feels like this functionality should live as a plugin, for the near future.

I hope you joy the plugin, if you have an feedback, please create a github issue. If you like my work and want to see more of it, I opened sponsorship via my github sponsor page.

Detect if video has audio

While working on the Web Stories for WordPress plugin and working on functionality to mute videos, there was a need to detect if a video already had audio or not. But doing some quick googling, I noticed there none of the answers I found online works well or at all. Worse yet, the answers were not written using modern javascript, either not using promises or using jQuery. So today, I am sharing this gist, in hopes that the next person who comes across this problem, has a workabout solution.

function hasAudio(video) {
return (
video.mozHasAudio ||
Boolean(video.webkitAudioDecodedByteCount) ||
Boolean(video.audioTracks?.length)
);
}
function hasVideoGotAudio(src) {
const video = document.createElement('video');
video.muted = true;
video.crossOrigin = 'anonymous';
video.preload = 'auto';
return new Promise((resolve, reject) => {
video.addEventListener('error', reject);
video.addEventListener(
'canplay',
() => {
video.currentTime = 0.99;
},
{ once: true } // Important because 'canplay' can be fired hundreds of times.
);
video.addEventListener('seeked', () => resolve(hasAudio(video)), {
once: true,
});
video.src = src;
});
}
export default hasVideoGotAudio;
view raw hasVideoGotAudio.js hosted with ❤ by GitHub

Open for sponsorship

I have been contributing to WordPress since 2015. In my 6 years of contributing I have worked on nearly all parts of core, spreading my contribution to the parts of WordPress I felt that needed me the most. In my time I have help land, some features I am really proud of. These include but are not limited too

  • White screen of death protection.
  • Default meta values
  • WP_Site_Query and WP_Network_Query.
  • Block types REST API
  • Lazy loading images.
  • WebP support.
  • And many more.

Today I announce that I’ve started accepting sponsorship via the GitHub Sponsors program. I am asking for sponsorship, so that I can continue to work on open source and dedicate more time to and not be restricted by limited time between client projects. This ensures that going forward, I continue to contribute and keep making the open web better.

I want to make it clear, even without sponsorship, I will continue to work on open source, but support will help me maintain my plugins and tools and keep them free for everyone.

What have I been working on in WordPress 5.5

WordPress 5.5 is fast coming around the corner and in my opinion, it is going to be a massive release. There is a lot in this release. With all the extra time I have on my hands, now that I am unable to travel, I decided to focus my energies on open source. This gives me something to do and hopefully helps the wider community. I thought I would highlight some of the tickets, I am especially proud to have worked on and will likely end up in the WordPress 5.5 release.

Menu / Menu item / Menu location REST API endpoints
This ticket has been over year in the making. The original first patch to the feature plugin was in June 2019. But it was then merged to the gutenberg plugin. These endpoints, will allow developers to get menus data. The plan is use to rebuild the menus screens in WordPress core. It will also help with developers that wish to use menu data in a headless WordPress front end.

Block Type REST API endpoint
The block type REST API goes along with another piece of work that the gutenberg team has been working to register all core blocks in PHP. The hope is to someday use this endpoint in the WordPress mobile app, to allow custom blocks to be supported on mobile.

Plugins REST API endpoint
This REST API endpoint, allows developers to install, activate and deactivate on single and multisite. This work was done for the block directory project. But my personal hope is that, it will be used for remote management tools.

Scripts and Style REST API endpoint
This REST API endpoint is again part of the block directory project. But has a lot of useful applications. One of which will be allowing for the gutenberg team to lazy load scripts and styles in core, enabling better performance.

Return all themes into the themes REST API
This is a simple extension to the themes endpoint, so all themes are received. This is a step towards, having all data for themes in the REST API.

Add default value to register meta
This has been in the works for nearly 2 years. Meta data, like post meta, does not support default values. This makes it a little outlier in core, as options and network options do support a default value. This change adds a new filter for default values and leverages, the register meta function.

Introduce wp_cache_get_multi()
This ticket is something I have been thinking about for 8 years. It is has been a lot of work to get this one into core. But it is massive possible performance benefits for WordPress core. It means that object-cache drop-ins can now, get multiple values in one request. The hope is that receiving values from cache will be much faster.

There are many other thing I have been working on, but these are my favours. Thanks to XWP for giving me time to work on these tickets. Here is hoping that all of them make into WordPress 5.5.

WordCamp US 2019 Talk – The REST API and Authentication

Here is the video from the talk that I did from WordCamp US 2019. In this talk, I discuss the different types and solutions for the authentication in the REST API.

Since this presentation, there is a new core project for authentication. It is still in the research stages, but well worth following on the github repo.

Plugins

More reading

Getting gutenberg block data in the REST API

WordPress core added the block editor in version 5.0.0, changing fundamentally how WordPress stores and displays elements in post (page) content. The block editor, named gutenberg, is written using react and uses the WordPress REST API to interact with data in WordPress. So you maybe forgiven to believe that it would then be easy to interact with blocks and block data (attributes) via the REST API. However it is not, but to understand why, first you must understand how block data is stored. 

Take the following example. 

<!-- wp:button {"className":"is-style-outline"} -->
<div class="wp-block-button is-style-outline"><a class="wp-block-button__link" href="#">test</a></div>
<!-- /wp:button -->

This example shows how a the core/button block is stored in post content in the database. Blocks are represented as mixture of html comments (with a json object) and html tags. A block is defined with the starting comment 

<!– wp: . Blocks can either have an opening and closing tag (like the button block above) or a single line like the following search block example. 

<!-- wp:search /-->

Block data is used to define how the block behaves and renders. Block data can be stored in one of two ways. First, as a json blob in the html comment, see the className in the core button in the above example. Handling this data, once parsed, is pretty simple in javascript or in other systems that can modify json. The second way of storing data is as part of the html markup inside the block. Take for example how the button block is defined. 

{
    "name": "core/button",
    "category": "layout",
    "attributes": {
        "url": {
            "type": "string",
            "source": "attribute",
            "selector": "a",
            "attribute": "href"
        },
        "title": {
            "type": "string",
            "source": "attribute",
            "selector": "a",
            "attribute": "title"
        },
        "text": {
            "type": "string",
            "source": "html",
            "selector": "a"
        },
        "backgroundColor": {
            "type": "string"
        },
        "textColor": {
            "type": "string"
        },
        "customBackgroundColor": {
            "type": "string"
        },
        "customTextColor": {
            "type": "string"
        }
    }
}

As you can see from this block definition, attributes like url, title and text are all stored in the html of the block. Data can be stored in the contains of the html tags, meaning everything that appears after the close of the opening tag and before the end of the tag. Or data can also be stored in attributes of the tag, such as the href or title. This can make working with block data extremely difficult. 

Processing block data

When rendering blocks in WordPress in the front end (theme) on for example a post page, the PHP will parse the html that contains blocks (post content). Core does this, as blocks can have registered script associated with them, which are defined using register_block_type. The function parse_blocks uses regex, to find the block definition and return an array of blocks. From there, it loops around each block, and enqueues the registered javascript libraries. Each element in the array, as a field called attributes, this contains all the fields and data defined in the html comment (json blob). However, what is missing from these attributes is the data found in the html attribute / tags. At the time of writing (in WP 5.2), core doesn’t parse these fields and return them as usable data. To be able to extract data from html, PHP would have to parse the html of the block and navigate the doc tree generated using css selectors, to get the tag and field where the data is stored. This would be a resource intensive process and result in false positives. Even if this process wasn’t resource intensive, at time of writing it is  not possible to process all blocks. Many blocks both core and ones defined in plugins are only defined in the javascript, meaning the PHP is completely unaware of the block and associated block data. 

Defining block data 

As part of an going project to be able to use gutenberg in the WordPress mobile app, the gutenberg team have been working on a RFC to define block structure. This project hopes to define the structure of a block as a platform independent json file. This file can then be used by both the javascript and PHP, to define each block and make so the both the server and front end are aware of all blocks. Once the blocks are defined, the core team plan to make two new REST APIs. These new apis are documented in this post

Fetching the available block types through REST APIs.
Fetching block objects from posts through REST APIs.

Once these apis exists, headerless applications like the WordPress mobile app or a headerless frontend written in react or a similiar framework, could much easier use block data to render. 

It is likely that it take some time to get all existing blocks converted to this new style and will require efforts from developers to do so. It is likely that functions required to register the block in PHP, will not be merged until 5.3 at the earliest. 

Accessing block data now

What if you need access to this block data now and you can’t wait? Well, you are in luck, there is a proof of concept plugin, called wp-rest-blocks written to expose this data. Once installed and activated, this plugin adds two field to the post / page REST API endpoints. These fields are ‘has_blocks’ and ‘blocks’. Has block is a boolean, the denote, if the post content contains blocks and the blocks field, is an array of blocks. An example of the output is displayed below. 

{
    "blockName": "core/heading",
    "attrs": [
       
    ],
    "innerBlocks": [
       
    ],
    "innerHTML": "\n<h2><strong>Flight Deals to Tianjin</strong></h2>\n",
    "innerContent": [
      "\n<h2><strong>Flight Deals to Tianjin</strong></h2>\n"
    ],
    "rendered": "\n<h2><strong>Flight Deals to Tianjin</strong></h2>\n"
  },

As detailed above, it is not possible to get all block data currently. But this plugin does get existing block data found in the html comment which in many cases would be enough. But there is also a “rendered” field, that has the fully rendered block. This is specially useful for dynamic blocks. If the block uses lots of html attibributes to store block data, the innerContent or rendered fields, can be used to extract this data. The api, even supports nested inner blocks, returning an array of the inner blocks, in each block recursively. 

I have also made an effort to try and extract block data of core blocks. Current I have mapped data of the following core blocks. 

  • Image
  • Gallery
  • Heading
  • Paragraph
  • Button
  • Video 
  • Audio
  • File

There is even some unit test coverage, in case there are some breaking changes in core in the future. This plugin is in the very early stages, but maybe a good starting point for anyone wishing to build a front end using headerless WordPress. 

I was a guest on the WordPress weekly podcast

I had the honour of being invited to talk on the hosts of the WordPress weekly podcast this week (episode 356). I have been listening to the podcast for years and been friends with JJJ (the co host) for a while.

I talked about on many topics and spoke a little bit about how I got into the industry. In the podcast I mention the RFCs the gutenberg team have released. There is an “Add the block registration RFC” and “Add Blocks in Widget Areas RFC“. The widget area RFC, is one where I wished more developers had commented and fedback.

What I learn at WCEU 2018

I was lucky enough to attend WordCamp Europe 2018 and it was by far the biggest WordCamp that I have ever been too. Lots of the things I learn at WordCamps are not from the sessions or talks, but from the hallaway track and  chat with people in the WordPress community. So here are some of the interesting fidbits that I learn while talking to people on the ground.

Automattic are enabling developers on there VIP GO platform, to empower themselves with a growing toolkit. The VIP GO team have adding some new tools. First of which is the ability to allow you to use tools like Circle CI and travis, to build / compile your code. Read more at there official blog post here. This will enable more javascript / scss heavy projects, to compile and minify there code. This will enable more faster workflows using Frameworks like React or Vue.js. Automattic announced CLI tools, written in node. This tool uses github to authenticate and enables developers to sync data from different environments and I am sure functionality will only ever grow.

I saw an interesting talk from the Human made team, discussing there use of react and the rest api, with the redesign of  Tech crunch. Hosted on VIP GO, the site has a react based theme. With the site being a WordPress theme, they also adding none javascript fallback to pages and allowed react to replace content that appears on the site. This means that the site fade in gracefully and there isn’t such a jump in. They said, that they noticed no lose in SEO, from moving to javascript. There also mention that is was very hard to integrate tracking libraries, like google analytics and nicely load ads from there custom platform.

There are some interesting talk about javascript apis, documented in the slides. All new functionality for javascript is being put on NPM and made more generic. This means, that other members of the javascript community can jump and use them libraries and start contributing back.

WordPress in 2019, was an interesting talk about, the current state of WordPress. It was a little bleak and mentions that growth will not continue if the experience in the admin terminal and on site doesn’t improve. Noel mentions many AI and machine learning to improve user experience.

Matt Mullenweg keynote was brief. He talked about the state of the gutenberg project and the plans for the rest of the year. The plan is to get everything merged by September. But that automattic has provided a plugin called  gutenberg ramp. This plugin, disables gutenberg, and will allow you to selectively turn this functionality on or off. I am sure this will be welcome to many smaller hosts and agencies that are dreading when the new editor drops into core. There are plans to put blocks in every part of the site. The plan is to make everything blocks on WordPress. Get rid of menus, widgets and headers. Also removing custom meta boxes by using custom post types and fixed templates. These fixed templates looks pretty clean and I become a believer as seeing it in action. The Q&A was a bit of a mixed bag for me. Many of the questions were weird or pointless. The only useful question to me was the move away from SVN / Trac for issue management and version control. A question I was going to ask myself until someone beat me to it. He said, that he believed they were better tools out there and he wanted to move to them someday, but it was low down the list of stuff they want to get done.

On the contributor day, I sat with the REST api team for most of the day. This team, are extremely smart and hard working and they don’t get the credit they deserve. The rest api team having being doing a lot of work on the register meta function. This means there is lots more work being done on make the data stored in WordPress much more structured. See this ticket for more information, this has since been merged id which amazing, as it has been in the works for over 2 years. 😀

One of the biggest missing pieces of the rest api, is a way in core to authenticate requests with the REST API. I pressed the team of this and got some interesting information. At the moment, the most popular methods are Oauth1, Oauth2 and JWT tokens. Regarding these solutions, they said.

  • Regarding JWT tokens, they are novel solution to the problem, but not workable for core.
  • Oauth1 plugin is just a proof of concept and is not be actively worked on, as the workflow is too hard for many applications.
  • Oauth2 plugin is currently the recommended plugin to auth the rest api.

The Rest team are also looking into X-Auth, an auth method that Twitter first implemented and have since removed support for.

I know there is more that I am missing, so let me know what you learnt and remember to follow me on twitter for more twitter news.