URL constants and moving databases across stages

[EvanK]

Relative URL constants have strange side effects ranging from non-fatal warnings in the admin to breaking poorly written plugins. The trade off was to be able to up/down databases between stages without worrying about hardcoded domains in the db data.

As a workaround, I've set the URL constants as absolute urls in wp-config, using the $_SERVER[HTTP_HOST] $_SERVER[SERVER_NAME] variable, and then added some perl regex filtering wp-cli string replacing of the dumped sql before after importing on db down/up operations.

Fixes #35 (I hope)

[EvanK]

@ericrasch @germanny Do y'all remember what wordpress features & plugins were breaking due to relative urls? I'd like to test this comprehensively before I consider it fixed

[germanny]

The BWP Google Sitemaps plugin - https://wordpress.org/plugins/bwp-google-xml-sitemaps/ - did not work with Evolution, and we believed that it was due to the site_url option having /wp there instead of the full URL. I can't remember exactly but I believe the urls out putted by the plugin were incorrect.

I am a little concerned what kind of SEO issues we're going to have with relative. We rely on Yoast SEO plugin to create canonical links for us, for example. Aren't these going to turn up as relative links now? According to advice from Google and Yoast, we want these and all the links on our sites to be absolute links, not relative. Canonical links need to be absolute. And given the number of pages we have on some of our sites that are super super similar (ie, ./masters/, ./history/masters/, ./accounting/, ./accounting/careers/, ./careers/ - these types of very similar links and more existed at one time on sites like OEDb.org and certainly could return in the future), this is going to turn into a rabbit hole of relative urls that the content team isn't going to be able to manage easily nor are we going to be able to keep track of well. A Yoast article calls them "spider traps," and I'm a little worried that with the way we do business that could be a real issue.

I feel like there were a couple other issues but I can't find documentation on them so can't say what they were. Personally, I don't think it's nearly as big of a problem to do a couple very simple SQL queries or use a plugin or Screaming Frog to to identify and update the DB as needed with non-local or -staging urls. It's something that should be part of the dev's job and it literally takes 5 minutes to do.

Just my two cents.

[ericclemmons]

Wait, URLs have to be absolute? This reminds me of SEO debates we had @ WhiteFence nearly 10 years ago =/

It's not that the paths are relative, but they omit the hostname, since that's what's dynamic across environments.

That's why most application frameworks & the like generate URLs that are paths (e.g. /path/to/resource?id=123), not absolute URLs including hostname.

Having a <link rel="canonical"> in the dynamically set in satisfies the "SEO" correct?

https://support.google.com/webmasters/answer/139066?hl=en

Genesis currently hacks the absolute URLs (via ob_start / ob_flush) with their correct version. Doing a DB find/replace is also a hack.

I always felt the best solution (which is yet to be discovered in WordPress) is one that ensures resource URIs use paths, unless they're external.

But that's the thing, I'm trying to apply how applications are built to WordPress, which like trying to herd cats. My patience for WordPress being suitable for this sort of workflow is strained at this point, so I'm cool to just back away & let ride whatever works with the "architecture" we're stuck with...

[EvanK]

I'll test the sitemaps and yoast plugins, but I suspect the issue really was a relative url in the runtime constants rather than urls saved in the database.

[EvanK]

Looks like there are still issues (plugin related) with relative urls in the database. To address this, I'm replacing the stage subdomains via the regex filter...instead of converting them entirely to relative urls.

For example, on production evolve:db:up, instead of this:
http://local.example.com/?p=1 -> /?p=1

We'll get this:
http://local.example.com/?p=1 -> http://production.example.com/?p=1

[EvanK]

I think this latest couple commits will be a good fix! Here's what I've done:

• Apache is configured with a default canonical url of whatever bare domain the project is generated with (example.com in our test project). This means that regardless of whether the client sends a Host: header or hits the server directly by ip, we will always have a ServerName set
• The Evolution php class is generated with the bare domain set as a constant, meaning we can match any stage subdomain (or none) anchored to the domain we know to look for...and we're matching against SERVER_NAME instead of HTTP_HOST -- read more on why that's a sound security practice
• The wp config is now setting absolute urls based on the environment and domain in the Evolution php class, so we end up with roughly homogenous absolute urls:
  • Host header of local.example.com has a siteurl of http://local.example.com
  • Host feature-branch.staging.example.com has http://staging.example.com
  • Host production.example.com has http://example.com
  • Host www.example.com has http://example.com
  • Host example.com has http://example.com

[germanny]

So will http://example.com be stored in the wpdb instead of http://www.example.com?

[EvanK]

@germanny Yes, the urls that are saved in the database come from the WP_HOME and WP_SITEURL constants.

By standardizing those, we can ensure that any production activity will end up saving example.com, and that any staging activity (even on a crazy vhost aliased branch) will save staging.example.com.

[germanny]

Slick! I'm think about 301s. If the site is meant to have www. in the link, but it's in the wpdb without, I wonder if that will come up as a 301?

[ericrasch]

We have only a handful of sites that don't use www.. Most do include the www. and is also specified via Google Webmaster Tools to prefer the www. versions. Will this affect that (with 301s)?

Also was wondering about the staging sites and if we had multiple staging branches, how would those be affected? One branch might have a plugin installed that's not on the main staging branch. Will that break the site?

[EvanK]

I thought the bare domain (non-www) was preferred in most cases...but I can add a setting in the generator to set the canonical production domain way or the other.

As for 301s, it shouldn't be redirecting the user at all (except when logging into the wp admin), but rather just serve as the requested hostname. That said, the on-page links will all be using the canonical stage-specific domain. I'd suppose that, for SEO purposes, a bare->www (or vice versa) redirect should exist in the root htaccess?

[EvanK]

Support for multiple staging branches is always going to be shaky, as all staging branches share the same database. I believe Wordpress handles this as gracefully as possible though, deactivating plugins that are registered in the db but don't exist on the filesystem

[EvanK]

Okay, www vs non-www is configurable:

Added a rewrite rule to enforce it either way, supports https too. In the process, added support for preserving htaccess customizations

[ericclemmons]

That's pretty legit!

[EvanK]

Yep! The only lingering issue here is:

• Requests to production.example.com or www.example.com still serve example.com links
• Requests to branch.staging.example.com still serve staging.example.com links

I'm not sure how to resolve this without output buffering...which I really don't want to do...

[ericclemmons]

Hmmm... let me think on this one a bit...

[EvanK]

@ericclemmons Thinking about merging this, and opening a separate issue for the URL disparities on production.* and *.staging.*