OxDEAD Unicornz

Have you ever seen so many?

Travis CI and Deploying Golang Apps to GAE

Disclamer – this post covers the situation with Travis CI and GAE as of January 2017, APIs may be changed and issues may be resolved soon.

Travis CI provides possibility to deploy go applications to GAE after a successful build. There are quite severe limitations if you’re using plain App Engine, not Managed VMs.

First of all there are two ways Golang application can be deployed to GAE. You can use either gcloud command

1
gcloud app deploy app.yaml

or goapp command

1
goapp deploy -application [YOUR_PROJECT_ID] -version [YOUR_VERSION_ID] app.yaml

The difference is that if use goapp an application is being built locally and then sent to application engine, if you use gcloud source code is being sent to Google Cloud storage and built out of GCS bucket. Travis CI uses gcloud under the hood and it appears gcloud is going to be the default way in the future.

There are two major issues with deploying via gcloud – first service account you use to deploy should have both ‘App Engine Deployer’ and ‘Storage Object Admin’ roles assigned. If it does not have ‘Storage Object Admin’ role upload to storage fails and you see something like this is Travis log:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
ERROR: gcloud crashed (UploadError): Error uploading files: HttpError accessing <https://www.googleapis.com/storage/
v1/b/staging.onetimer-deadunicornz.appspot.com/o?alt=json>: response: <{'status': '403', 'content-length': '350', 'e
xpires': 'Tue, 31 Jan 2017 01:33:10 GMT', 'vary': 'Origin, X-Origin', 'server': 'UploadServer', 'x-guploader-uploadi
d': 'AEnB2UqrVklJmrHPY8MyPu3K0sSOcJZVWscw', 'cache-control': 'private, max-age=0', 'date': 'Tue, 31 Jan 2017 01:33:1
0 GMT', 'alt-svc': 'quic=":443"; ma=2592000; v="35,34"', 'content-type': 'application/json; charset=UTF-8'}>, conten
t <{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "forbidden",
    "message": "Caller does not have storage.objects.list access to bucket staging.onetimer-deadunicornz.appspot.com
."
   }
  ],
  "code": 403,
  "message": "Caller does not have storage.objects.list access to bucket staging.onetimer-deadunicornz.appspot.com."
 }
}

the second issue is dependencies. gcloud does not pull any prior to building an application. In Travis log you see following import errors:

1
2
3
4
5
OperationError: Error Response: [9] Deployment contains files that cannot be compiled: Compile failed:
2017/01/30 18:14:32 go-app-builder: build timing: 2√ócompile (115ms total), 0√ólink (0s total)
2017/01/30 18:14:32 go-app-builder: failed running compile: exit status 1
onetimergolib/onetimergolib.go:19: can't find import: "golang.org/x/net/context"
onetimergo.go:20: can't find import: "github.com/gorilla/mux"

names of problem packages may vary, of course. Vendoring package dependencies along with your application is tricky as GAE prohibits using certain functionality. For example one of packages you depend on imports syscall package. You’ll get the following compilation error:

1
2
ERROR: (gcloud.app.deploy) Error Response: [9] Deployment contains files that cannot be compiled: Compile failed:
2017/01/31 00:13:02 go-app-builder: Failed parsing input: parser: bad import "syscall" in vendor/golang.org/x/net/internal/nettest/helper_posix.go

I get this error even if I add vendor to nobuild_files as suggested here. It’s also possible `nobuild_files option is deprecated as it’s not listed in Google documentation.

At this point I gave up. Maybe if I play more with packages in vendor directory I could fix build but the whole pipeline would be too flimsy anyway. I’ll be deploying manually until something better shows up on a horizon. Travis may work better with Golang apps deployed to Managed VMs as there are no restrictions on imports.

Few links: