Separating Frontend and Backend App Engine Services into Separate Subdomains

I’m in the middle of re-implementing my bigoquiz.com website into separate frontend (client) and backend (server) projects, which I’m deploying to separate App Engine “services”. Previously both the front end HTML and JavaScript, and the backend Java, were all in one (GWT) project, deployed together as one default App Engine service. I chose to separate the two services into their own subdomains. So the frontend is currently served from beta.bigoquiz.com and its JavaScript code uses the REST API provided by the backend, served from betaapi.bigoquiz.com. This will later be bigoquiz.com and api.bigoquiz.com.

This seemed wise at the time and I thought it was necessary but in hindsight it was an unnecessary detour. I would have been better off just mapping different paths to the services. I didn’t see how to do this at first with Google AppEngine, but it’s very easy. You just provide a dispatch.yaml file (also mentioned below) to map URL prefixes to the different services. For instance, /api/* would map to the backend service, and everything else would map to the frontend service. Larger systems might do this with a service gateway such as Netflix’s Zuul.

Nevertheless, this post mentions some of the issues you’ll have if you choose to split your frontend and backend into separate subdomains. It mentions App Engine, Go, and Angular (TypeScript), but the ideas are generally useful.

App Engine Service Names

Each service that you deploy to App Engine has an app.yaml file. This can have a line to name the service, which would otherwise just be called “default”. For instance, the first line of my frontend’s app.yaml file currently looks like this:

service: beta

The first line of my backend’s app.yaml file currently looks like this:

service: betaapi

When I deploy these services, via “gcloud app deploy” on the command line, I can then see them listed in the Google Cloud console, in the App Engine section, in the Services section.

Mapping Subdomains to App Engine Services

In the Google Cloud console, in the App Engine section, in the settings section, in the “custom domains” tab, you should add each subdomain. You’ll need to verify your ownership of each subdomain by adding DNS entries for Google to check.

App Engine very recently added the “Managed Security” feature, which automatically creates and manages (LetsEncrypt) SSL certificates, letting you serve content via HTTPS. Using this currently makes using subdomains more complicated, because it doesn’t yet support wildard SSL certificates. That’s likely to be possible soon when LetsEncrypt starts providing wildcards SSL certificates, so this section might become outdated.

Without Managed Security

If you aren’t using Managed Security yet, mapping subdomains to services is quite simple. Google’s documentation suggests that you just add a wildcard CNAME entry to the DNS for your domain, like so:

Record: *
 Type: CNAME
 Value: ghs.googlehosted.com

All subdomains will then be served by google. App Engine will try to map a subdomain to a service of the same name. So foo.example.com will map to a service named foo.

With Managed Security

However, if you are using Managed Security, you’ll need to tell App Engine about each individual subdomain so each subdomain can have its own SSL certificate. Unfortunately, you’ll then need to add the same 8 A and AAAA records to your DNS for your subdomain too.

Although App Engine does automatic subdomain-to-service mapping for wildcard domains, this doesn’t happen with specifically-listed subdomains. You’ll need to specify the mapping manually using a dispatch.yaml file, like so:

dispatch:
  - url: "beta.bigoquiz.com/*"
    service: beta
  - url: "betaapi.bigoquiz.com/*"
    service: betaapi

You can then deploy the dispatch.yaml file from the command line like so:

$ gcloud app deploy dispatch.yaml

I wish there was some way to split these dispatch rules up into separate files, so I could associate them with the separate codebases for the separate services. For now, I’ve put the dispatch.yaml file for all the services in the repository for my frontend code and I manually deploy it.

CORS (Cross Origin Resource Sharing)

By default, modern browsers will not allow Javascript that came from www.example.com (or example.com) to make HTTP requests to another domain or subdomain, such as api.example.com. This same-origin policy prevents malicious pages from accessing data on other sites, possibly via your authentication token.

If you try to access a subdomain from JavaScript served from a different subdomain, you’ll see error messages about this in the browser console, such as:

No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'beta.example.com' is therefore not allowed access.

For instance, JavaScript served from foo.example.com cannot normally access content at bar.example.com.

However, you can allow calls to a subdomain by using the CORS system. The browser will attempt the call, but will only provide the response back to the JavaScript if the response has the appropriate Allowed* headers. The browser may first attempt a separate CORS “pre-flight” request before actually issuing the request, depending on the request details.

If you configure your server to reply to a CORS requests with appropriate AllowedOrigins and AllowedMethods HTTP headers, you can tell the browser to allow the JavaScript to make the HTTP requests and receive the responses.

For instance, in Go, I’ve used rs/cors to respond to CORS requests, like so, passing in the original julienschmidt/httprouter that I’m using.

c := cors.New(cors.Options{
	AllowedOrigins: []string{"example.com},
	AllowedMethods: []string{"GET", "POST", "OPTIONS"},	
})

handler := c.Handler(router)
http.Handle("/", handler)

I also did this in my original Java code by adding a ContainerResponseFilter, annotated with @Provider.

Cookies: CORS

Even when the server responds to CORS requests with AllowedOrigins and AllowedMethods headers, by default the browser will not allow Javascript to send cookies  when it sends HTTP requests to other domains (or subdomains). But you can allow this by adding an AllowCredentials header to the server’s CORS response. For instance, I added the AllowCredentials header in Go on the server side, like so:

c := cors.New(cors.Options{
  	...
	AllowCredentials: true,})

You might need to specify this on the client-side too, because the underlying XMLHttpRequest defaults to not sending cookies with cross-site requests. For instance, I specify withCredentials in Angular (Typescript) calls to http.get(), like so:

this.http.get(url, {withCredentials: true})

Note Angular’s awful documentation for the withCredentials option, though Mozilla’s documentation for the XMLHttpRequest withCredentials option is clearer.

Cookies: Setting the Domain

To use a cookie across subdomains, for instance to send a cookie to a domain other than the one that provided the cookie, you may need to set the cookie’s domain, which makes the cookie available to all subdomains in the domain. Otherwise, the cookie will be available only to the specific subdomain that set it.

I didn’t need to do this because I had just one service on one subdomain. This subdomain sets the cookie in responses, the cookie is then stored by the browser, and the browser provides the cookie in subsequent requests to the same subdomain.

OAuth2 Callback

If your subdomain implements endpoints for oauth2 login and callback, you’ll need to tell App Engine about the subdomain. In the Google Cloud console, in the “APIs & Services” section, go to the Credentials section. Here you should enter the subdomain for your web page under “Authorized JavaScript origins”, and enter the subdomain for your oauth2 login and callback subdomain under “Authorized redirect URIs”.

The subdomain will then be listed appropriately in the configuration file that you download via the “Download JSON” link, which you can parse in your code, so that the oauth2 request specifies your callback URL. For instance, I parse the downloaded config .json file in Go using google.ConfigFromJSON() from the golang.org/x/oauth2/google package, like so:

func GenerateGoogleOAuthConfig(r *http.Request) *oauth2.Config {
	c := appengine.NewContext(r)

	b, err := ioutil.ReadFile(configCredentialsFilename)
	if err != nil {
		log.Errorf(c, "Unable to read client secret file (%s): %v", configCredentialsFilename, err)
		return nil
	}

	config, err := google.ConfigFromJSON(b, credentialsScopeProfile, credentialsScopeEmail)
	if err != nil {
		log.Errorf(c, "Unable to parse client secret file (%) to config: %v", configCredentialsFilename, err)
		return nil
	}

	return config
}