OxDEAD Unicornz

Have you ever seen so many?

Few Notes on Golang Application Packaging for GAE

Once I decided to refactor onetimergo application introducing separate methods for Memcache and Data storage interaction I walked into package initiation problem. Approach I was using with a single onetimergo package didn’t work.

Everything was good till I tried to import onetimergo into my onetimergo-sync application.

1
2
3
4
5
6
7
8
9
10
11
...
import (
    "google.golang.org/appengine"
    "google.golang.org/appengine/log"
    "google.golang.org/appengine/memcache"
    "google.golang.org/appengine/datastore"
+    "github.com/sheva-serg/onetimergo"
    "net/http"
    "time"
)
...

Complication of onetimergo-sync started to fail:

1
2
3
4
5
6
7
8
9
serg@ubuntu-dev:~/onetimergo-sync$ dev_appserver.py --host=0.0.0.0 --admin_host=0.0.0.0 app.yaml
INFO     2017-01-22 17:12:59,720 sdk_update_checker.py:231] Checking for updates to the SDK.
INFO     2017-01-22 17:13:00,253 sdk_update_checker.py:275] This SDK release is newer than the advertised release.
INFO     2017-01-22 17:13:00,470 api_server.py:268] Starting API server at: http://localhost:40513
INFO     2017-01-22 17:13:00,473 dispatcher.py:199] Starting module "sync" running at: http://0.0.0.0:8080
INFO     2017-01-22 17:13:00,474 admin_server.py:116] Starting admin server at: http://0.0.0.0:8000
WARNING  2017-01-22 17:13:00,474 devappserver2.py:836] No default module found. Ignoring.
ERROR    2017-01-22 17:13:06,659 http_runtime.py:396] bad runtime process port ['']
panic: http: multiple registrations for /

Short sidenote: folks from Google suggest to use dev_appserver.py instead of goapp serve if you’re using Golang SDK because dev_appserver.py provides more options to play with as of January 2017. For example goapp serve knows nothing about admin_host option so accessing a dev server running remotely can be quite a tricky task.

the reason of failure is init() function(s)

1
2
3
func init() {
    http.HandleFunc("/", onetimergo.Throttle(responder, max_concurrency))
}

same in both onetimergo and onetimergo-sync applications. When onetimergo-sync app imports onetimergo it runs init() from onetimergo, then own init(). In Python this would not be an issue as there is a

1
2
if __name__ == '__main__':
...

clause allowing to reuse same script for both running program and imports but this is not the case for Golang. Golang forces you to write import only packages and move all execution logic somwhere else.

Please also note GAE does not allow to follow common approach to move all execution logic to the package main as it already uses main package to wrap up your handler and set up a server listening on specific ports and responding to healthchecks.

From the documentation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
package hello

import (
    "fmt"
    "net/http"
)

func init() {
    http.HandleFunc("/", handler)
}

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprint(w, "Hello, world!")
}

Note: When writing a stand-alone Go program we would place this code in package main. The Go App Engine Runtime provides a special main package, so you should put HTTP handler code in a package of your choice (in this case, hello)

Well, you can use main if you want this bad but this is not trivial.

What I did to resolve the problem

I’ve split my onetimergo application into two packages, onetimergolib exporting methods and onetimergo containing init() function and handling web requests.

1
2
3
4
5
6
7
8
9
10
├── app.yaml
├── index.html
├── LICENSE.txt
├── onetimergo.go
├── onetimergolib
│   └── onetimergolib.go
├── README.md
└── static
    ├── favicon.ico
    └── robots.txt

I’m able to use onetimergolib from onetimergo-sync application now:

1
2
3
4
5
6
7
8
9
10
11
...
import (
    "google.golang.org/appengine"
    "google.golang.org/appengine/log"
    "google.golang.org/appengine/memcache"
    "google.golang.org/appengine/datastore"
    "github.com/sheva-serg/onetimergo/onetimergolib"
    "net/http"
    "time"
)
...

application compiles and runs properly now.