Scaling WP Queries with Elasticsearch

WordPress.com is big. Currently, there are:

  • 18.6 billion pages views/month
  • 409 million unique visitors/month
  • 56 million new posts/month
  • 68 million new comments/month

All of this happens on a dynamic platform that undergoes 5,000 deploys a month.

To try and surface better content to users, there are taxonomies and meta queries that normally don’t get used anywhere else. It can get difficult to scale these queries, which may more often than not result in the error message: MySQL server has gone away.

Xiao Yu, an Automattician, presents a solution: Elasticsearch. He states that Elasticsearch:

  • Optimizes for search not indexing
  • Is near realtime
  • Is denormalized
  • Uses inverted indices
  • Uses Bitsets

“Elasticsearch can represent a >200x improvement for queries against cold caches.” – Nick Daugherty, WordPress.com VIP

The es-wp-query plugin made by Matt Boynes at Alley Interactive does all of the transpiling of queries on the fly and can be found here:

https://github.com/alleyinteractive/es-wp-query

You can browse his slides below:

This embed is invalid

See the presentations from previous Big Media & Enterprise WordPress Meetups. For Big Media & Enterprise WordPress Meetup groups in other cities, see the full list on VIP Events and join your local group.

Want more information about WordPress services for media or enterprise sites? Get in touch.

 

Ready to get started?

Tell us about your needs

Let us lead the way. We’ll help you select a top tier development partner. We’ll train your developers, operations, infrastructure, and editorial teams. We’ll coarchitect your deployment processes. We will provide live support for peak events. We’ll help your people avoid dark alleys and blind corners, and reduce wasted cycles.

%d bloggers like this: