Skip to content

Commit

Permalink
fix in repo docs
Browse files Browse the repository at this point in the history
  • Loading branch information
1pkg committed Dec 7, 2020
1 parent 8fe9345 commit 3d6d71c
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 50 deletions.
21 changes: 3 additions & 18 deletions README.MD
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

## Introduction

Gotcha seamlessly patches go runtime to provides a way to track amount of allocated bytes, objects, calls per goroutine.
Gotcha seamlessly patches go runtime to provides a convenient way to trace amount of allocated bytes, objects, calls per goroutine.

```go
package main
Expand Down Expand Up @@ -50,24 +50,9 @@ func main() {

## Internals

Gotcha exposes function `Track` that tracks memory allocations for provided `Tracer` function. All traced allocations are attacehd to the single parameter of this tracer function `Context` object. Gotcha context fully implements `context.Context` interface and could be used to cancel execution if provided limits were exceeded. Gotcha supports nested treacking by providing gotcha context as parent context for derived `Tracer` function then gotcha context `Tracker` methods `Add`, `Remains` and `Exceeded` will also target parent context as well which is useful if nested tracking is required.
Note that in order to work gotcha uses [bou.ke/monkey](https://github.com/bouk/monkey) and [modern-go/gls](https://github.com/modern-go/gls) packages to patch existing runtime allocator entrypoints and track per goroutine context limts, next entrypoints are patched:
Gotcha exposes function `Track` that tracks memory allocations for provided `Tracer` function. All traced allocations are attached to the single parameter of this tracer function `Context` object. Gotcha context fully implements `context.Context` interface and could be used to cancel execution if provided limits were exceeded. Gotcha supports nested tracing by providing gotcha context as the parent context for derived `Tracer`; then gotcha tracing context methods will also be targeting parent context as well as derived context.

- direct objects allocation
- arrays allocation
- slice allocation
- map allocation (solved by arrays allocation)
- chan allocation
- strings/bytes/runes allocation

This makes gotcha inherits the same list of restrictions as `modern-go/gls` and `bou.ke/monkey` [has](https://github.com/bouk/monkey#notes).

Note that some patches with monkey patch are causing loops, for e.g. `newarray`, `growslice`.
So for them implementation either slightly changed - newarray or not patched at all - `growslice`.
For `growslice` it still possible to make the same workaround as done for `newarray`, but it will require to copy and support great amount of code from runtime which is not correlating with the goal of this project, so `growslice` is skipped for now. Note that some function from `interface.conv` family are not patched neither which will cause untracked allocation for code like `vt, ok := var.(type)`.
Note that `runtime.gobytes` is not patched as well as it seems it's only used by go compiler itself.
Note that only functions from `mallocgc` family are patched, but runtime has much more allocation tricks
that won't be traced by gotcha, like direct `malloc` sys calls, etc.
Note that in order to work gotcha uses [1pkg/gomonkey](https://github.com/1pkg/gomonkey) based on [bou.ke/monkey](https://github.com/bouk/monkey) and [1pkg/golocal](https://github.com/1pkg/golocal) based on [modern-go/gls](https://github.com/modern-go/gls) packages to patch runtime `mallocgc` allocator entrypoint and trace per goroutine context limts. This makes gotcha inherits the same list of restrictions as `modern-go/gls` and `bou.ke/monkey` [has](https://github.com/bouk/monkey#notes).

## Licence

Expand Down
2 changes: 1 addition & 1 deletion context.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ type gotchactx struct {
// which is useful if nested tracking is required.
func NewContext(parent context.Context, opts ...ContextOpt) Context {
ctx := &gotchactx{parent: parent}
// need to do type assertion here to avoid allocations in malloc
// need to do type assertion here to avoid allocations in malloc.
if ptrack, ok := parent.(Tracker); ok {
ctx.ptrack = ptrack
}
Expand Down
20 changes: 3 additions & 17 deletions malloc.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,23 +18,9 @@ type tp struct {
//go:linkname mallocgc runtime.mallocgc
func mallocgc(size uintptr, tp *tp, needzero bool) unsafe.Pointer

// init patches some existing memory allocation runtime entrypoints
// - direct objects allocation
// - arrays allocation
// - slice allocation
// - map allocation (solved by arrays allocation)
// - chan allocation
// - strings/bytes/runes allocation
// Note that some patches with monkey patch are causing loops, for e.g. `newarray`, `growslice`.
// So for them implementation either slightly changed - newarray or not patched at all - `growslice`.
// For `growslice` it still possible to make the same workaround as done for `newarray`,
// but it will require to copy and support great amount of code from runtime
// which is not correlating with the goal of this project, so `growslice` is skipped for now.
// Note that some function from `interface.conv` family are not patched neither which will cause
// untracked allocation for code like `vt, ok := var.(type)`.
// Note that `runtime.gobytes` is not patched as well as it seems it's only used by go compiler itself.
// Note that only functions from `mallocgc` family are patched, but runtime has much more allocation tricks
// that won't be traced by gotcha, like direct `malloc` sys calls, etc.
// init patches main mallocgc allocation runtime entrypoint
// it also sets goroutine local storage tracers capacity from env var
// note that pacthing will only work on amd64 arch.
func init() {
// set up local store for malloc
maxTracers := int64(golocal.DefaultCapacity)
Expand Down
28 changes: 14 additions & 14 deletions malloc_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ import (
)

// epsAlloc defines alloc error delta for test
// as it's tough to carefully track all possible small
// as it's tough to carefully trace all possible small
// underlying allocations - defining relative deviation buffer simplifies it
const epsAlloc = 0.66

func TestTraceTypes(t *testing.T) {
t.Run("track no object alloc", func(t *testing.T) {
t.Run("trace no object alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
for i := 0; i < 1000; i++ {
_ = i - 1
Expand All @@ -26,7 +26,7 @@ func TestTraceTypes(t *testing.T) {
require.Equal(t, int64(0), c)
})
})
t.Run("track new object alloc", func(t *testing.T) {
t.Run("trace new object alloc", func(t *testing.T) {
type sobj struct {
a, b int64
}
Expand All @@ -43,7 +43,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(1000))
})
})
t.Run("track new object alloc &", func(t *testing.T) {
t.Run("trace new object alloc &", func(t *testing.T) {
type sobj struct {
a, b int64
}
Expand All @@ -60,7 +60,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(1000))
})
})
t.Run("track new object alloc reflect", func(t *testing.T) {
t.Run("trace new object alloc reflect", func(t *testing.T) {
type sobj struct {
a, b int64
}
Expand All @@ -79,7 +79,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(100))
})
})
t.Run("track make slice alloc", func(t *testing.T) {
t.Run("trace make slice alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
var v []int64
for i := 0; i < 1000; i++ {
Expand All @@ -92,7 +92,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(1000))
})
})
t.Run("track make slice copy alloc", func(t *testing.T) {
t.Run("trace make slice copy alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
var v []int64
vc := []int64{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}
Expand All @@ -107,7 +107,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(1000))
})
})
t.Run("track make slice append alloc", func(t *testing.T) {
t.Run("trace make slice append alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
var v []int64
for i := 0; i < 1000; i++ {
Expand All @@ -121,7 +121,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(2000))
})
})
t.Run("track make map alloc", func(t *testing.T) {
t.Run("trace make map alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
var v map[string]int32
for i := 0; i < 100; i++ {
Expand All @@ -134,7 +134,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(100))
})
})
t.Run("track make chan alloc", func(t *testing.T) {
t.Run("trace make chan alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
var v chan [12]byte
for i := 0; i < 100; i++ {
Expand All @@ -147,7 +147,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(100))
})
})
t.Run("track string bytes alloc", func(t *testing.T) {
t.Run("trace string bytes alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
cs := "foo | | bar"
var v []byte
Expand All @@ -161,7 +161,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(1000))
})
})
t.Run("track string runes alloc", func(t *testing.T) {
t.Run("trace string runes alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
cs := "foo | | bar"
var v []rune
Expand All @@ -175,7 +175,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(1000))
})
})
t.Run("track bytes string alloc", func(t *testing.T) {
t.Run("trace bytes string alloc", func(t *testing.T) {
Trace(context.Background(), func(ctx Context) {
cb := []byte("foo | | bar")
var v string
Expand All @@ -189,7 +189,7 @@ func TestTraceTypes(t *testing.T) {
require.GreaterOrEqual(t, c, int64(1000))
})
})
t.Run("track new object alloc complex", func(t *testing.T) {
t.Run("trace new object alloc complex", func(t *testing.T) {
type cobj struct {
mp map[string][]string
self *cobj
Expand Down

0 comments on commit 3d6d71c

Please sign in to comment.