Escape Analysis in Go: Stack vs. Heap

What do you think the following C code will do?

Copy 
1#include <stdio.h>
2
3int* f() {
4    int a;
5    a = 42;
6    return &a;
7}
8
9void main()
10{
11    printf("%d", *f());
12}

If you’re a Go programmer, you may be surprised by the answer:

Copy 
1main.c:6:12: warning: function returns address of local variable [-Wreturn-local-addr]                                                 
2Segmentation fault (core dumped)

We get a seg fault - which the compiler warned us about.

Here’s an equivalent program in Go. As expected, this program will print 42 to stdout.

Copy 
1package main
2
3import (
4    "fmt"
5)
6
7func f() *int {
8    a := 42
9    return &a
10}
11
12func main() {
13    fmt.Printf("%d", *f())
14}

Stack vs. Heap

The difference between these two programs comes down to where the compiler determines a should be stored in memory: on the stack or on the heap.

In C, the variable lives in the stack frame of f(). When the function returns, the stack is popped and the memory addresses of any variables in that frame become invalid. Accessing them leads to a segmentation fault.

Similarly in Go, the compiler will also try to allocate a variable to the local stack frame of the function in which it is declared. However, it is also able to perform escape analysis: if it cannot prove that a variable is not referenced after the function returns, then it allocates it on the heap instead.

We can see this in action by compiling the above program with the following flags:

Copy 
1> go build -o ./main -gcflags=-m ./main.go
2# command-line-arguments
3./main.go:7:6: can inline f
4./main.go:13:21: inlining call to f
5./main.go:13:12: inlining call to fmt.Printf
6./main.go:8:2: moved to heap: a
7./main.go:13:19: *(*int)(~r0) escapes to heap
8./main.go:13:12: []interface {} literal does not escape
9<autogenerated>:1: .this does not escape

The compiler moved a to the heap. (Additionally, we can also see that the compiler applied an optimisation to inline the call to f()).

Garbage Collection

It’s important to note that, unlike the stack, the heap does not clean up after itself. As the heap grows, the Garbage Collector (GC) must tidy it up by removing variables that will no longer be used by your program.

To understand this more deeply, let’s create a trace of a very similar program, using Dave Cheney’s excellent pkg/profile:

Copy 
1package main
2
3import (
4    "fmt"
5
6    "github.com/pkg/profile"
7)
8
9func f() *int {
10    a := 42
11    return &a
12}
13
14func main() {
15    defer profile.Start(profile.TraceProfile, profile.ProfilePath(".")).Stop()
16    for i := 0; i < 1000000; i++ {
17        fmt.Printf("%d", *f())
18    }
19}

We can inspect the results by running go tool trace trace.out:

You can see the heap footprint increasing with time. At a certain point, the GC runs and cleans up the heap, before it starts growing again.

If we rewrite the program so that there is no reference to a outside of the stack frame, then the variable doesn’t escape to the heap.

Copy 
1package main
2
3import (
4    "github.com/pkg/profile"
5)
6
7func f() int {
8    a := 42
9    return a
10}
11
12func main() {
13    defer profile.Start(profile.TraceProfile, profile.ProfilePath(".")).Stop()
14    for i := 0; i < 1000000; i++ {
15        f()
16    }
17}
18
19// go build -o ./main -gcflags=-m ./main.go
20// # command-line-arguments
21// ./main.go:7:6: can inline f
22// ./main.go:15:4: inlining call to f
23// ./main.go:13:21: ... argument does not escape

Now we can see that the heap does not grow:

Goroutine Stacks

Consider this program. We can see that a does not escape to the heap, since it is not modified “above” f’s stack frame, (only “below”, in g’s).

Copy 
1package main
2
3func g(a *int) {
4    *a++
5}
6
7func f() int {
8    a := 42
9    g(&a)
10    return a
11}
12
13func main() {
14    for i := 0; i < 1000000; i++ {
15        f()
16    }
17}
18
19// go build -o ./main -gcflags=-m ./main.go
20// # command-line-arguments
21// ./main.go:9:6: can inline g
22// ./main.go:13:6: can inline f
23// ./main.go:15:3: inlining call to g
24// ./main.go:22:4: inlining call to f
25// ./main.go:22:4: inlining call to g
26// ./main.go:9:8: a does not escape

What happens if we execute g in a goroutine? We find that a escapes to the heap.

Copy 
1package main
2
3func g(a *int) {
4    *a++
5}
6
7func f() int {
8    a := 42
9    go g(&a) // yes, this is a race condition, but this is just a demo :)
10    return a
11}
12
13func main() {
14    for i := 0; i < 1000000; i++ {
15        f()
16    }
17}
18
19// go build -o ./main -gcflags=-m ./main.go
20// # command-line-arguments
21// ./main.go:3:6: can inline g
22// ./main.go:3:8: a does not escape
23// ./main.go:8:2: moved to heap: a

This is because each goroutine has its own stack. As a result, the compiler cannot guarantee that f’s stack hasn’t been popped (invalidating a) when g accesses it. Therefore, the variable must live on the heap.

The stack allocated to a goroutine is initially very small, which helps keep goroutines very lightweight. The stack will then grow and shrink in size as required. As of Go 1.3, a goroutine’s stack is a contiguous block of memory. To grow the stack, it allocates a segment that is twice the size of the old stack, before copying over the old stack data to the new. As a result, the underlying memory addresses of your variables will change. The way this is implemented is very interesting - I recommend this excellent post from Cloudfare to understand this more deeply. As a demonstration, consider the following program:

Copy 
1package main
2
3const size = 1024
4
5func main() {
6    s := "foobar"
7    stackCopy(&s, 0, [size]int{})
8}
9
10func stackCopy(s *string, c int, a [size]int) {
11    println(c, s)
12    c++
13    if c == 24 {
14        return
15    }
16    stackCopy(s, c, a)
17}

Here we recursively call stackCopy to grow the stack, up to 24 stack frames deep, printing the underlying memory address of the string in each frame.

Copy 
1go run ./main.go
20 0xc000107f68
31 0xc000107f68
42 0xc000117f68 // change!
53 0xc000117f68
64 0xc000117f68
75 0xc000117f68
86 0xc000137f68 // change!
97 0xc000137f68
108 0xc000137f68
119 0xc000137f68
1210 0xc000137f68
1311 0xc000137f68
1412 0xc000137f68
1513 0xc000137f68
1614 0xc000177f68 // change!
1715 0xc000177f68
1816 0xc000177f68
1917 0xc000177f68
2018 0xc000177f68
2119 0xc000177f68
2220 0xc000177f68
2321 0xc000177f68
2422 0xc000177f68
2523 0xc000177f68
2624 0xc000177f68
2725 0xc000177f68
2826 0xc000177f68
2927 0xc000177f68
3028 0xc000177f68
3129 0xc000177f68
3230 0xc0001f7f68 // change!
33...

We can see that the memory address s changes for every time the stack depth doubles.

For further reading, I recommend this excellent post from Ardan Labs. For some limitations on escape analysis in Go, checkout this post.

Conclusion

The way a variable is used - not declared - determines whether it lives on the stack or the heap. Garbage collection in Go is an extremely useful property of the language. It reduces the likelihood of memory leaks and frees up your time so you can focus on the business logic of your application. Still, remember that this luxury does come at a cost.

I would like to note that in my experience, there are often other bottlenecks in my programs that can be addressed before optimising heap usage. The GC is highly optimised. As always, use profiling to understand your program before blindly applying optimisations.

If you wish to control the frequency with which the GC runs, you have a couple of options. The GOGC environment variable can be used to control the ratio of heap growth that triggers the GC. Alternatively, you may decide to use a ballast.

As always, if you’ve got comments, questions or feedback you can hit me up on Twitter ✌️