Site Merges

As part of a migration either to WordPress.com VIP or between two WordPress.com VIP sites, you might need to perform a site merge. Doing so is potentially time-consuming and can produce issues that do not normally occur during a site migration.

What Is a Site Merge?

A site merge is any time you are migrating more than one WordPress site’s content into a single WordPress site. This could be:

  • Two (or more) sites you are intending to collapse into one new site in the course of a migration to WordPress.com VIP
  • One site on WordPress.com VIP you are closing and “folding” that content into a parent site on WordPress.com VIP

If you are migrating content and the sources come from more than one posts table, it’s a site merge.

What Makes Site Merges Different from Other Imports?

When you merge more than one WordPress site into a new or existing single site, you will almost definitely encounter post ID overlap in the two datasets. When the WordPress importer then pulls this data into the eventual target site, the post IDs will be necessarily changed as you cannot have more than one post with the same ID.

Because of this, any connections between posts or solutions that have been created to use post IDs as linking data will need to have that data reassigned or rewritten. Anything that uses post IDs will need to take this into consideration, including:

  • Parent/child relationships between posts
  • Featured image assignments for posts
  • Serialized post ID data stored in postmeta
  • Any post ID information used in shortcodes (e.g. the gallery shortcode)

While post IDs are usually the primary concern, this ID confusion can also apply to IDs in other tables, such as term and meta IDs.

Considerations

Compared to an import that is single-site to single-site, you’ll have additional things to consider before the import begins and as you create your timeline for the migration. Keep these things in mind as you work with the WordPress.com VIP team to create achievable and realistic target dates and expectations. Our advice is to keep things as simple as possible. The more complicated your merge, the more time you’ll need for troubleshooting and QA.

Timeline

  • What is your migration timeline?
    • Are you properly allotting enough time to both implement and QA post ID changes that might be necessary due to the merge?
  • How many posts and images are you importing overall?
    • The greater the number of changes, the longer the additional work for a merge will take to complete.
  • Where will you be staging your content? For WordPress.com VIP, it needs to be publicly accessible for us to download images.

Authors

  • Are you using Co-Authors-Plus on one or both sites?
  • Have you looked at author slugs and profiles between the two sites to see how much (if any) overlap you will have?
    • You may need to do bulk reassignment of guest author terms to unify the two sets of guest author profiles and attributions.
  • If you are merging from a site without Co-Authors-Plus to a site using it (or vice versa), how are you going to integrate that data and make sure your authorship attribution is properly carried forward?

Post Relationships and Custom Features

  • Are you utilizing any custom metadata relationships between posts that would encounter trouble if those post IDs were to change?
    • A good example of this is custom photo gallery implementations or even the built-in WordPress galleries feature, which depends on post IDs.
  • You will need a scripting plan to alter the post metadata as needed to match the new post ID assignments. Remember that you can use the _original_post_id meta key to determine which posts have changed and their original IDs.

Publishing Schedules and Site Activity

  • With what frequency are posts created on both sites? This will determine the merge timing and how much data will need to be brought forward using a large initial import followed by smaller delta imports. Content fixing will need to be done at each step and will change the timing available. It’s usually best to move as much as possible in the first migrations and then delta-import smaller chunks of content immediately prior to launch.

Quality Assurance

  • Migrations that involve merges will need additional time and effort dedicated to QA. Because these relationships can be altered, you’ll need to come up with a plan for determining a representative sample of posts or other content to check for consistency. It’s best to do this with the initial large import prior to launch to discover and flag and fix any potential data issues.

Who Should Perform the Merge, and When?

Site merges can and will be performed in different ways depending on the combination of source sites and target sites. Depending on the requirements of the site, it may make more sense for you to perform the merge before the site is moved to WordPress.com VIP at all.

The merge should happen completely prior to moving to WordPress.com VIP if:

  • You are merging more than one site to WordPress.com VIP and the merge can be completed prior to your launch on WordPress.com VIP, or
  • You are merging more than one site that is no longer active into a site that is also no longer active, whether it is on WordPress.com VIP already or not.

The merge should happen entirely on WordPress.com VIP if:

  • You are merging any site into a site that is already live on WordPress.com VIP.

There may be situations where a combination of the two situations is best. Please discuss the merge process with WordPress.com VIP Support for more details.

Starting the Site Merge Process

As with any import to WordPress.com VIP, please contact WordPress.com VIP Support as much in advance as possible so we are aware of and can schedule resources to assist you with the merge process. Make sure you inform us that the import is going to have merged content.

We request that you prepare a local development environment on which you can partially test the merge to look for potential problems and identify scripting needs you’ll need to address. We’ll happily provide you with partial exports for the sites you plan to merge so you can do a test run and identify problems before they occur during an actual import. Remember that imports to a live site are much more difficult to reverse; the more we can help you identify problems ahead of time, the more efficiently and error-free the eventual import and merge will be.

Making Site Merges Successful

We’ll need you and your developers to contribute code for many types of merges, usually in the form of wp-cli commands. When you contact us, we’ll send you a questionnaire to make sure we’re considering all of the factors necessary to ensure your merge is successful.

When you merge to a site on WordPress.com VIP, we’ll take care of:

  • Fixing featured images,
  • Remapping image URLs in post content,
  • Providing useful import origin meta to help you in fixing problems after the fact,
  • Fixing post parent/child relationships, and
  • Fixing duplicate Guest Authors in Co-Authors Plus that might have been created.

You and your development team will need to:

  • Identify any postmeta or shortcode usages in your site content that will be disrupted by a site merge,
  • Create and test any CLI scripts that will be needed to repair these relationships,
  • Commit any needed CLI scripts to your theme, inform us of them and their purpose, and provide us with instructions for executing them at the proper time,
  • Create any scripts needed for term mapping (or perform that prior to the import itself), and
  • Test any merges with sample data to identify potential issues prior to scheduling the site merge import(s).

If you have any questions, please don’t hesitate to contact WordPress.com VIP Support.

VIP Go Local Development

Because developing sites for VIP Go is different than developing for WordPress.com hosted VIP sites, you need a different development environment.

Your VIP Go site runs three codebases: WordPress core (tracking the most current version), the VIP Go mu-plugins, and the codebase from your specific site repo. Because of this, a variety of WordPress local development environments can be suitably configured for VIP Go development purposes.

Here we describe using a VVV-based local development environment. Other options may include: Chassis, Docker-WordPress, Laravel Valet, etc.

VVV for VIP Go Development

Note: These instructions assume a familiarity with command line tools an macOS, Linux, or similar Operating System.

Prerequisite: all git operations referenced in this guide assume you have an ssh keypair registered with GitHub and are using ssh (vs. https) protocols. Using https protocols may lead to unexpected errors and is not supported.

Step 1: Setting up VVV

The basic instructions for installing VVV are in their documentation. Complete setup per their instructions before continuing below.

Step 2: Add your site code

Follow the instructions to add a new site here, and reprovision. Then, find and remove the entire wp-content folder at {VVV FOLDER}/www/{site name}/public_html/wp-content. Replace {VVV FOLDER} with the path to your VVV folder (i.e. the folder you installed VVV into), and {site name} with the name of your site.

Git clone your VIP Go site repo in place of it, using the following command. Replace {CLONE URL} with the GitHub clone URL for your VIP Go GitHub repository.

git clone {CLONE URL} {VVV FOLDER}/www/{site name}/public_html/wp-content

Note: VIP Go sites must use the wp-content folder structure from https://github.com/automattic/vip-skeleton. If you do not yet have a VIP Go site repo hosted with us, please use this vip-skeleton repo for the “CLONE URL” above and place your codebase (theme & plugins) within it for testing. Once your VIP Go site repo has been provisioned, you’ll most likely use that repo instead.

Step 3: Add the VIP Go MU plugins

Git clone the VIP Go MU plugins repo at wp-content/mu-plugins/; using the following command. Replace {VVV FOLDER} with the path to your VVV folder (i.e. the folder you installed VVV into).

git clone git@github.com:Automattic/vip-go-mu-plugins.git --recursive {VVV FOLDER}/www/{site name}/public_html/wp-content/mu-plugins/

Note: the vip-go-mu-plugins repository is using SSH protocol for submodules, and as GitHub does not allow anonymous SSH connections, you’ll have to set up SSH key for your GitHub account and use it for interaction with the repository (cloning and submodule updates).

Periodically pull changes down from this repository to ensure you have the latest code… we suggest checking for and pulling changes before the start of development on any given day:

$ cd {VVV FOLDER}/www/{site name}/public_html/wp-content/mu-plugins/
$ git pull origin master
$ git submodule update --init --recursive

Note: Do not commit the mu-plugins/ directory to your VIP Go site’s repository.

Step 4: Include the VIP config file

Add the following code to your wp-config.php just above this line: /* That's all, stop editing! Happy blogging. */:

if ( file_exists( __DIR__ . '/wp-content/vip-config/vip-config.php' ) ) {
    require_once( __DIR__ . '/wp-content/vip-config/vip-config.php' );
}

That’s it! Your development environment is ready to “Go” and you should see a “VIP” menu in wp-admin.

Disabling unused features and assets

WordPress.com comes with a bunch of useful features turned on by default, and most VIP sites make use of them every day. We also know that sites may want to disable some of these for various reasons, e.g. if you’re focused on performance and minimizing the number of assets loaded, you may want to disable any CSS/JS that would be loaded for features not in use.

We offer a number of helper functions to facilitate this:

  • wpcom_vip_disable_custom_customizer – disables the WordPress.com-specific Customizer and Custom Design
  • wpcom_vip_disable_devicepx_js – devicepx.js loads retina/HiDPI versions of certain files (Gravatars, etc) for devices that run at a higher resolution (such as smartphones). Using this function disables it.
  • wpcom_vip_disable_global_terms – remove your taxonomy from the global taxonomy array
  • wpcom_vip_disable_hovercards – if you are not using Gravatars and Blavatars and need to disable the loading of related Javascript and CSS resources
  • wpcom_vip_disable_likes – disables Likes for Posts and Custom Post Types. Sharing can also be disabled from the Dashboard (Settings > Sharing).
  • wpcom_vip_remove_opensearch – disables rendering of the OpenSearch description documents at /osd.xml and /opensearch.xml
  • wpcom_vip_disable_sharing – disables Sharing in Posts and Pages
  • wpcom_vip_disable_sharing_resources – disables the CSS/JS involved in Sharing functions, powering things like smart buttons and share counts displayed alongside the buttons.
  • wpcom_vip_remove_playlist_styles – disables enqueuing of wp-mediaelement.css which is necessary for sites using Playlist shortcode
  • wpcom_vip_remove_mp6_styles – disables enqueuing f mp6-hacks.css stylesheet which adds backward compatibility for legacy .mp6 body classes
  • wpcom_vip_load_geolocation_styles_only_when_needed – Conditionally dequeues Geo Location stylesheets in case they are not needed. IE.: post is not actually using Geo Location
  • wpcom_vip_remove_bbpress2_staff_css – disables enqueueing of wpcom-bbpress-premium-themes.css stylesheet

Check out all of the helper functions available.

Rolling Back / Reverting Changes to Your Theme Using Subversion

You can access your theme repository version history and, when necessary, quickly roll back your deployed theme to a previous state.

The Time Machine

Every time you commit a change to your theme to your VIP theme repository, Subversion remembers by creating a version. Each of these versions is assigned a unique number and certain other meta, namely: the username of the person who made the version, the date and time of the version, the number of lines of code changed in the version relative to the previous one and a commit message indicating the nature of the changes in the version.

You can inspect your theme’s version history on the command line using the svn log command (documentation: clitortoiseSVN.) Here’s some sample svn log output for Automattic’s test theme on VIP:

vip/test-theme$ svn log .
------------------------------------------------------------------------
r69251 | viper007bond | 2012-06-20 21:09:52 +0000 (Wed, 20 Jun 2012) | 1 line

Remove testing code
------------------------------------------------------------------------
r69209 | batmoo | 2012-06-20 13:59:49 +0000 (Wed, 20 Jun 2012) | 2 lines

auto-deploy: rm image in a subfolder
------------------------------------------------------------------------
r69197 | batmoo | 2012-06-20 13:40:55 +0000 (Wed, 20 Jun 2012) | 2 lines

Auto-deploy: testing a php + css commit
------------------------------------------------------------------------
r66930 | tottdev | 2012-05-27 11:19:52 +0000 (Sun, 27 May 2012) | 1 line

reverting error code

To inspect a commit in more detail (such as seeing the changes it introduced), svn provides the svn diff command (docs: clitortoiseSVN.) Here’s an example:

vip/test-theme$ svn diff -c 17546 .

Index: header.php
===================================================================
--- header.php	(revision 17545)
+++ header.php	(revision 17546)
@@ -6,7 +6,6 @@
 
  
 
-<meta name="generator" content="WordPress "/> 
 
 <link rel="stylesheet" href="?2" type="text/css" media="screen" />
 

The -c flag shows a change set resulting from a single revision, but there are many other options including -r n:m (show differences between revision n and m) and more — see the svn diff documentation for full details.

Reverting

If you need to roll back code you’ve committed it to the deploy queue, the quickest way to do this is to revert and commit the code yourself. Here’s how to prepare and push a roll-back:

1. Update your theme’s working copy to the latest revision (use

svn up your-theme

or tortoiseSVN.)

2. Use svn log to find out how far you’d like to roll back. Suppose your log reveals something like:

r4 -- change paint color
r3 -- introduce clunk engine
r2 -- modify blee tag
r1 -- add snorkle

3. Use svn merge to roll back the code to the revision before things went ‘wrong.’ For example, a typical cli command for this would be:

vip/my-theme$ svn merge -c -4 -c -3 .

This “un-does” the commits with version numbers 3 and 4, and rolls the code back to its previous state as of revision 2. As with svn diff, the merge command admits lots of different syntax combinations. You can read about some of those here and here. You can also roll back easily using clients like tortoiseSVN.

4. The merge you just performed has produced a changeset in your working copy. Commit your roll-back like any other change using svn commit.

Make sure that you include an explanatory commit message such as “roll-back of r3 and r4 due to JS error”. Your message should indicate which versions are being rolled back, and for what reason. This will help WordPress.com VIP deploy engineers quickly deploy your changeset.

Pure roll-backs usually require little or no review, and can be deployed almost immediately, so make sure your change only contains the roll-back, and nothing else.

In an emergency situation, you can open an urgent ticket requesting a deploy of your roll-back. Generally this is not necessary, however, and we request that you reserve urgent tickets for when your site is down or for other serious end-user issues.

Best practices:

1. Think in terms of “rolling back” code instead of reverting individual commits. It’s possible to revert things out of sequence. For instance in the previous example we could have reverted r3 but not r4. However, to do so ignores the possibility that something in r4 may depend on something in r3. In almost all cases, it’s safer to roll back all commits in sequence, especially if doing so quickly.

2. Data dependencies. Before a major roll-back it’s good to take a second to consider whether the underlying data has changed in a way that would cause problems with the old code. Have the content or options been modified in any way, including by CLI scripts? This is not usually an issue, but can be in certain cases.

3. Confusingly, subversion also features a command called svn revert whose purpose is not to take your code to a previous state, but only to un-do local un-committed modifications. Therefore we use the term “revert” both in the sense of “roll-back” and the sense of svn revert.

4. Finally, There is an extremely large number of Subversion clients which make all of this very easy. Cornerstone is a great (though somewhat expensive) client for OSX.

Anatomy of a VIP Theme

This page explains the anatomy and structure of an example theme that might be used on WordPress.com VIP. It complements the Theme Handbook that apply to regular WordPress theme development. While every client’s themes will have different requirements and constraints, following this general structure will help ensure faster theme reviews and more efficient long term maintenance of your site.

Theme Naming

Themes in the VIP environment live in WP_CONTENT_DIR . '/themes/vip/'. So if your organization is called Acme Kite Co., for example, your theme path might be /wp-content/themes/vip/acmekites/.

If you anticipate setting up multiple themes for your site or if you are a developer working on different themes for different sites, you might need a naming scheme that further distinguishes themes, e.g. /wp-content/themes/vip/acmekites-main/ and /wp-content/themes/vip/acmekites-seasonal/.

Plugin Locations

Each theme should have a dedicated directory for custom plugins:

/wp-content/themes/vip/acmekites-main/plugins/

You can then load a plugin from this directory with:

wpcom_vip_load_plugin( 'plugin-name', 'theme' );

If you are developing plugins that might be used across multiple sites/themes, you can put them in a plugins directory not associated with a specific theme:

/wp-content/themes/vip/acmekites-plugins/

Plugins managed this way will live in a separate SVN repo and loaded using wpcom_vip_load_plugin:

wpcom_vip_load_plugin( ‘plugin-name’, ‘acmekites-plugins’ ); // note this requires a specific naming structure: /plugin-name/plugin-name.php

There is also a directory for plugins shared across all VIP sites, make sure you avoid this directory namespace in your development structure:

/wp-content/themes/vip/plugins/

Plugins available to all VIP sites can be loaded using wpcom_vip_load_plugin as well:

wpcom_vip_load_plugin( 'plugin-name' );

or

wpcom_vip_load_plugin( 'plugin-name', 'plugins', 'version number' );

Child Themes

WordPress.com VIP sites can create and use themes that are child themes of the themes available on WordPress.com. This allows you to build on the great themes already developed and tested for WordPress while incorporating your own custom features and styles. When the parent theme is updated or expanded, your child theme can take advantage of those changes without having to re-incorporate your customizations. Child theming works the same in this case as child theming on self-hosted WordPress sites. Each child theme is managed in its own SVN repository.

functions.php

The functions.php file in your theme lays the groundwork for customizing the functionality of your theme.  You can read more about it in Functions File Explained. On WordPress.com VIP sites, the functions.php file must include the vip-init functions.

It is critically important that `vip-init.php` is included very early in `functions.php` – before any VIP constants or functions are used.

Here’s a sample VIP functions file for Acme Kites:

<?php

// Set a theme constant for later use
define('AK_DIR',__DIR__);

/**
* Standard VIP configuration
*/
require_once WP_CONTENT_DIR . '/themes/vip/plugins/vip-init.php';

// Site-specific Config
require_once(AK_DIR . '/functions/config.php');

// Custom widgets and sidebars for this site.
require_once(AK_DIR . '/functions/widgets.php');

Use Underscores for a head start

The Underscores project will generate a starter theme that can be used to get a head start on your theme development. It includes lean, well-commented, modern, HTML5 templates, minimal CSS that’s ready for you to build on, and a variety of tools to help you work efficiently in customizing your theme.

See Also

Related Posts

You can take advantage of the WordPress.com Related Posts functionality to return related posts quickly. Enable it by turning on Related Posts in Settings > Reading and you’re done!

If you’d like to programmatically retrieve and display posts within your theme code, you can do so like this:

$posts = Jetpack_RelatedPosts::init_raw()->get_for_post_id( $post_id, $args );

// Returns an associative array in the form of id => $post_id:
/*
array(
    array ('id' => 12345),
    array ('id' => 54321),
)
*/

Related Posts indexing will work regardless of whether your site is marked as “Private.” For self-hosted sites, Related Posts will currently only work for standard post-types; we are working on enabling it for custom post types.

For more information about ways to customize related posts, see http://jetpack.me/support/customize-related-posts/

High Traffic Alert

On WordPress.com, we serve 100+ million pageviews per day without blinking. However, sometimes you may be expecting a high traffic event where you may want to notify the VIP team for additional assistance.

To do so, please open a ticket with us with the following information:

  • Date and time of event:
  • Traffic expected:
  • Heavily trafficked pages:
  • Are there any external services, dependencies, or applications that use or rely on the site (e.g. mobile apps)? If so, how do these services interact with the site?

On VIP, we review every line of code to ensure that your site will perform under high traffic events. When you send in a high traffic alert, we will check through the theme again to ensure that it is performance-ready. Please notify us of the high traffic event as far in advance as possible so that there is enough time to resolve any potential issues.

As always, should you need our team immediately, please open a ticket with the word “urgent” in the subject line.

Creating Good Changesets

Changesets are the heart of any version control system, and making good changesets is vitally important to the maintainability of your code. As all code on WordPress.com VIP is reviewed by a real person, it’s even more important all changesets are well crafted.

Remember always code (and commit) for the maintainer.

A Good Changeset:

Represents one logical change

What comprises a ‘logical change’ is up for interpretation, but only directly related changes are included. Generally, the smaller the changeset, the better.

Good Example: Adding the CSS, JS, HTML, and PHP code for a new UI button.

Bad Example: Adding the new UI button, fixing whitespacing, and tweaking copy in the footer.

Bundles related changes together

It’s much easier to trace refactorings and other changes if related changes are grouped together. Rather than splitting a logical change into many separate commits, related changes should be combined.

Good Example: Refactoring code into a new plugin by moving it to a new file and including that file.

Bad Example: Refactoring code into a new plugin by putting the code removal, addition, and include into separate commits.

Is Atomic

An atomic commit means that the system is always left in a consistent state after the changeset is committed. No one commit would cause the codebase to be in an invalid state. The commit is easily rolled back to a previous valid state, including all related changes, without the need to analyze the potential interdependencies of neighboring commits.

Good Example: Adding a new feature to the homepage by committing the HTML / PHP changes alongside the required CSS / JS changes, so there is never an incomplete state (HTML elements without styling) in the codebase.

Bad Example: Committing the HTML changes and requisite CSS / JS separately. The first commit represents an inconsistent state, as the feature can exist in the DOM without being properly styled.

Is Properly Described

Accurately describing the changes is very important for others (and future you) looking at your code. A good commit message describes the what and why of a change. Please see Writing Good Commit Messages for more information.

User_meta vs User_attributes

User Meta

As you read in our what we look for documentation, we don’t allow the use of user_meta. The reason for this is that WordPress.com is an incredibly large multisite and user_meta is all stored in one cache key. Therefore anything stored in user_meta will be loaded on every page load. Another reason is that as the user_meta grows to over 1MB it can no longer be stored in memcache.

User Attributes

User_attributes are a drop in replacement for user_meta. They perform in exactly the same way with the small exception that you cannot have multiple values for the same user_attribute meta_key. You can find the function declarations in vip/plugins/vip-do-not-include-on-wpcom/wpcom-functions.php. These are functions that imitate the behavior of user_attributes on your local dev environement. This means you can code against them without needing to have the cache or table schema that runs on WordPress.com.

While the code you will see in that file looks just like a layer on top of user_meta on WordPress.com, user_attributes are stored in the cache on an individual basis and not globally. This means that requests to user_attributes by meta_key and user_id are cached individually and not grouped by user_id and loaded all at once per page load.

You can use user_attributes as a drop in replacement for user_meta. That means:

update_user_attribute( $user_id, $meta_key, $meta_value )

get_user_attribute( $user_id, $meta_key )

delete_user_attribute( $user_id, $meta_key, $meta_value = ” )

will all work as expected. Just remember, the meta_key are shared among all of WordPress.com so prefix them. As with user_meta you shouldn’t store large quantities of data (several KB), and lastly, since just like user_meta meta_key or meta_value on user_attributes isn’t indexed you shouldn’t query against them.

Custom Error Pages

We have error pages that are standard and served for all sites on WordPress.com, and as a VIP customer you can override these defaults and specify your own custom error pages instead.

Setting it up is simple: all you need to do is add a custom error page (a simple HTML file) to your theme.

The file needs to be located in a folder called “error-docs” and named “error-page.html”. Note that the file can only contain HTML and basic inline or remote-hosted CSS and Javascript.

I.e., from the root of your theme: ~/error-docs/error-page.html

Once you’ve committed the file to your theme, you’ll need to open a ticket with our team to notify us, so we can make some associated configuration changes on our end.

Ready to get started?

Drop us a note.

No matter where you are in the planning process, we’re happy to help, and we’re actual humans here on the other side of the form. 👋 We’re here to discuss your challenges and plans, evaluate your existing resources or a potential partner, or even make some initial recommendations. And, of course, we’re here to help any time you’re in the market for some robust WordPress awesomeness.