replace dep with go modules (#16017)
- guide shamelessly stolen from prometheus/prometheus - updates local interface of oauth exchange - updates local impl of hclogger - bump jaeger client version closes #16088
This commit is contained in:
+1
@@ -0,0 +1 @@
|
||||
.idea*
|
||||
+148
@@ -0,0 +1,148 @@
|
||||
# go-hclog
|
||||
|
||||
[][godocs]
|
||||
|
||||
[godocs]: https://godoc.org/github.com/hashicorp/go-hclog
|
||||
|
||||
`go-hclog` is a package for Go that provides a simple key/value logging
|
||||
interface for use in development and production environments.
|
||||
|
||||
It provides logging levels that provide decreased output based upon the
|
||||
desired amount of output, unlike the standard library `log` package.
|
||||
|
||||
It provides `Printf` style logging of values via `hclog.Fmt()`.
|
||||
|
||||
It provides a human readable output mode for use in development as well as
|
||||
JSON output mode for production.
|
||||
|
||||
## Stability Note
|
||||
|
||||
While this library is fully open source and HashiCorp will be maintaining it
|
||||
(since we are and will be making extensive use of it), the API and output
|
||||
format is subject to minor changes as we fully bake and vet it in our projects.
|
||||
This notice will be removed once it's fully integrated into our major projects
|
||||
and no further changes are anticipated.
|
||||
|
||||
## Installation and Docs
|
||||
|
||||
Install using `go get github.com/hashicorp/go-hclog`.
|
||||
|
||||
Full documentation is available at
|
||||
http://godoc.org/github.com/hashicorp/go-hclog
|
||||
|
||||
## Usage
|
||||
|
||||
### Use the global logger
|
||||
|
||||
```go
|
||||
hclog.Default().Info("hello world")
|
||||
```
|
||||
|
||||
```text
|
||||
2017-07-05T16:15:55.167-0700 [INFO ] hello world
|
||||
```
|
||||
|
||||
(Note timestamps are removed in future examples for brevity.)
|
||||
|
||||
### Create a new logger
|
||||
|
||||
```go
|
||||
appLogger := hclog.New(&hclog.LoggerOptions{
|
||||
Name: "my-app",
|
||||
Level: hclog.LevelFromString("DEBUG"),
|
||||
})
|
||||
```
|
||||
|
||||
### Emit an Info level message with 2 key/value pairs
|
||||
|
||||
```go
|
||||
input := "5.5"
|
||||
_, err := strconv.ParseInt(input, 10, 32)
|
||||
if err != nil {
|
||||
appLogger.Info("Invalid input for ParseInt", "input", input, "error", err)
|
||||
}
|
||||
```
|
||||
|
||||
```text
|
||||
... [INFO ] my-app: Invalid input for ParseInt: input=5.5 error="strconv.ParseInt: parsing "5.5": invalid syntax"
|
||||
```
|
||||
|
||||
### Create a new Logger for a major subsystem
|
||||
|
||||
```go
|
||||
subsystemLogger := appLogger.Named("transport")
|
||||
subsystemLogger.Info("we are transporting something")
|
||||
```
|
||||
|
||||
```text
|
||||
... [INFO ] my-app.transport: we are transporting something
|
||||
```
|
||||
|
||||
Notice that logs emitted by `subsystemLogger` contain `my-app.transport`,
|
||||
reflecting both the application and subsystem names.
|
||||
|
||||
### Create a new Logger with fixed key/value pairs
|
||||
|
||||
Using `With()` will include a specific key-value pair in all messages emitted
|
||||
by that logger.
|
||||
|
||||
```go
|
||||
requestID := "5fb446b6-6eba-821d-df1b-cd7501b6a363"
|
||||
requestLogger := subsystemLogger.With("request", requestID)
|
||||
requestLogger.Info("we are transporting a request")
|
||||
```
|
||||
|
||||
```text
|
||||
... [INFO ] my-app.transport: we are transporting a request: request=5fb446b6-6eba-821d-df1b-cd7501b6a363
|
||||
```
|
||||
|
||||
This allows sub Loggers to be context specific without having to thread that
|
||||
into all the callers.
|
||||
|
||||
### Using `hclog.Fmt()`
|
||||
|
||||
```go
|
||||
var int totalBandwidth = 200
|
||||
appLogger.Info("total bandwidth exceeded", "bandwidth", hclog.Fmt("%d GB/s", totalBandwidth))
|
||||
```
|
||||
|
||||
```text
|
||||
... [INFO ] my-app: total bandwidth exceeded: bandwidth="200 GB/s"
|
||||
```
|
||||
|
||||
### Use this with code that uses the standard library logger
|
||||
|
||||
If you want to use the standard library's `log.Logger` interface you can wrap
|
||||
`hclog.Logger` by calling the `StandardLogger()` method. This allows you to use
|
||||
it with the familiar `Println()`, `Printf()`, etc. For example:
|
||||
|
||||
```go
|
||||
stdLogger := appLogger.StandardLogger(&hclog.StandardLoggerOptions{
|
||||
InferLevels: true,
|
||||
})
|
||||
// Printf() is provided by stdlib log.Logger interface, not hclog.Logger
|
||||
stdLogger.Printf("[DEBUG] %+v", stdLogger)
|
||||
```
|
||||
|
||||
```text
|
||||
... [DEBUG] my-app: &{mu:{state:0 sema:0} prefix: flag:0 out:0xc42000a0a0 buf:[]}
|
||||
```
|
||||
|
||||
Alternatively, you may configure the system-wide logger:
|
||||
|
||||
```go
|
||||
// log the standard logger from 'import "log"'
|
||||
log.SetOutput(appLogger.Writer(&hclog.StandardLoggerOptions{InferLevels: true}))
|
||||
log.SetPrefix("")
|
||||
log.SetFlags(0)
|
||||
|
||||
log.Printf("[DEBUG] %d", 42)
|
||||
```
|
||||
|
||||
```text
|
||||
... [DEBUG] my-app: 42
|
||||
```
|
||||
|
||||
Notice that if `appLogger` is initialized with the `INFO` log level _and_ you
|
||||
specify `InferLevels: true`, you will not see any output here. You must change
|
||||
`appLogger` to `DEBUG` to see output. See the docs for more information.
|
||||
+5
-5
@@ -8,16 +8,16 @@ var (
|
||||
protect sync.Once
|
||||
def Logger
|
||||
|
||||
// The options used to create the Default logger. These are
|
||||
// read only when the Default logger is created, so set them
|
||||
// as soon as the process starts.
|
||||
// DefaultOptions is used to create the Default logger. These are read
|
||||
// only when the Default logger is created, so set them as soon as the
|
||||
// process starts.
|
||||
DefaultOptions = &LoggerOptions{
|
||||
Level: DefaultLevel,
|
||||
Output: DefaultOutput,
|
||||
}
|
||||
)
|
||||
|
||||
// Return a logger that is held globally. This can be a good starting
|
||||
// Default returns a globally held logger. This can be a good starting
|
||||
// place, and then you can use .With() and .Name() to create sub-loggers
|
||||
// to be used in more specific contexts.
|
||||
func Default() Logger {
|
||||
@@ -28,7 +28,7 @@ func Default() Logger {
|
||||
return def
|
||||
}
|
||||
|
||||
// A short alias for Default()
|
||||
// L is a short alias for Default().
|
||||
func L() Logger {
|
||||
return Default()
|
||||
}
|
||||
|
||||
+7
@@ -0,0 +1,7 @@
|
||||
module github.com/hashicorp/go-hclog
|
||||
|
||||
require (
|
||||
github.com/davecgh/go-spew v1.1.1 // indirect
|
||||
github.com/pmezard/go-difflib v1.0.0 // indirect
|
||||
github.com/stretchr/testify v1.2.2
|
||||
)
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
|
||||
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
|
||||
Generated
Vendored
+214
-123
@@ -1,30 +1,57 @@
|
||||
package hclog
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"encoding"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"reflect"
|
||||
"runtime"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
// TimeFormat to use for logging. This is a version of RFC3339 that contains
|
||||
// contains millisecond precision
|
||||
const TimeFormat = "2006-01-02T15:04:05.000Z0700"
|
||||
|
||||
var (
|
||||
_levelToBracket = map[Level]string{
|
||||
Debug: "[DEBUG]",
|
||||
Trace: "[TRACE]",
|
||||
Info: "[INFO ]",
|
||||
Warn: "[WARN ]",
|
||||
Info: "[INFO] ",
|
||||
Warn: "[WARN] ",
|
||||
Error: "[ERROR]",
|
||||
}
|
||||
)
|
||||
|
||||
// Given the options (nil for defaults), create a new Logger
|
||||
// Make sure that intLogger is a Logger
|
||||
var _ Logger = &intLogger{}
|
||||
|
||||
// intLogger is an internal logger implementation. Internal in that it is
|
||||
// defined entirely by this package.
|
||||
type intLogger struct {
|
||||
json bool
|
||||
caller bool
|
||||
name string
|
||||
timeFormat string
|
||||
|
||||
// This is a pointer so that it's shared by any derived loggers, since
|
||||
// those derived loggers share the bufio.Writer as well.
|
||||
mutex *sync.Mutex
|
||||
writer *writer
|
||||
level *int32
|
||||
|
||||
implied []interface{}
|
||||
}
|
||||
|
||||
// New returns a configured logger.
|
||||
func New(opts *LoggerOptions) Logger {
|
||||
if opts == nil {
|
||||
opts = &LoggerOptions{}
|
||||
@@ -32,7 +59,7 @@ func New(opts *LoggerOptions) Logger {
|
||||
|
||||
output := opts.Output
|
||||
if output == nil {
|
||||
output = os.Stderr
|
||||
output = DefaultOutput
|
||||
}
|
||||
|
||||
level := opts.Level
|
||||
@@ -40,69 +67,49 @@ func New(opts *LoggerOptions) Logger {
|
||||
level = DefaultLevel
|
||||
}
|
||||
|
||||
mtx := opts.Mutex
|
||||
if mtx == nil {
|
||||
mtx = new(sync.Mutex)
|
||||
mutex := opts.Mutex
|
||||
if mutex == nil {
|
||||
mutex = new(sync.Mutex)
|
||||
}
|
||||
|
||||
ret := &intLogger{
|
||||
m: mtx,
|
||||
l := &intLogger{
|
||||
json: opts.JSONFormat,
|
||||
caller: opts.IncludeLocation,
|
||||
name: opts.Name,
|
||||
timeFormat: TimeFormat,
|
||||
w: bufio.NewWriter(output),
|
||||
level: level,
|
||||
mutex: mutex,
|
||||
writer: newWriter(output),
|
||||
level: new(int32),
|
||||
}
|
||||
|
||||
if opts.TimeFormat != "" {
|
||||
ret.timeFormat = opts.TimeFormat
|
||||
l.timeFormat = opts.TimeFormat
|
||||
}
|
||||
return ret
|
||||
|
||||
atomic.StoreInt32(l.level, int32(level))
|
||||
|
||||
return l
|
||||
}
|
||||
|
||||
// The internal logger implementation. Internal in that it is defined entirely
|
||||
// by this package.
|
||||
type intLogger struct {
|
||||
json bool
|
||||
caller bool
|
||||
name string
|
||||
timeFormat string
|
||||
|
||||
// this is a pointer so that it's shared by any derived loggers, since
|
||||
// those derived loggers share the bufio.Writer as well.
|
||||
m *sync.Mutex
|
||||
w *bufio.Writer
|
||||
level Level
|
||||
|
||||
implied []interface{}
|
||||
}
|
||||
|
||||
// Make sure that intLogger is a Logger
|
||||
var _ Logger = &intLogger{}
|
||||
|
||||
// The time format to use for logging. This is a version of RFC3339 that
|
||||
// contains millisecond precision
|
||||
const TimeFormat = "2006-01-02T15:04:05.000Z0700"
|
||||
|
||||
// Log a message and a set of key/value pairs if the given level is at
|
||||
// or more severe that the threshold configured in the Logger.
|
||||
func (z *intLogger) Log(level Level, msg string, args ...interface{}) {
|
||||
if level < z.level {
|
||||
func (l *intLogger) Log(level Level, msg string, args ...interface{}) {
|
||||
if level < Level(atomic.LoadInt32(l.level)) {
|
||||
return
|
||||
}
|
||||
|
||||
t := time.Now()
|
||||
|
||||
z.m.Lock()
|
||||
defer z.m.Unlock()
|
||||
l.mutex.Lock()
|
||||
defer l.mutex.Unlock()
|
||||
|
||||
if z.json {
|
||||
z.logJson(t, level, msg, args...)
|
||||
if l.json {
|
||||
l.logJSON(t, level, msg, args...)
|
||||
} else {
|
||||
z.log(t, level, msg, args...)
|
||||
l.log(t, level, msg, args...)
|
||||
}
|
||||
|
||||
z.w.Flush()
|
||||
l.writer.Flush(level)
|
||||
}
|
||||
|
||||
// Cleanup a path by returning the last 2 segments of the path only.
|
||||
@@ -118,10 +125,8 @@ func trimCallerPath(path string) string {
|
||||
// and https://github.com/golang/go/issues/18151
|
||||
//
|
||||
// for discussion on the issue on Go side.
|
||||
//
|
||||
|
||||
// Find the last separator.
|
||||
//
|
||||
idx := strings.LastIndexByte(path, '/')
|
||||
if idx == -1 {
|
||||
return path
|
||||
@@ -137,37 +142,37 @@ func trimCallerPath(path string) string {
|
||||
}
|
||||
|
||||
// Non-JSON logging format function
|
||||
func (z *intLogger) log(t time.Time, level Level, msg string, args ...interface{}) {
|
||||
z.w.WriteString(t.Format(z.timeFormat))
|
||||
z.w.WriteByte(' ')
|
||||
func (l *intLogger) log(t time.Time, level Level, msg string, args ...interface{}) {
|
||||
l.writer.WriteString(t.Format(l.timeFormat))
|
||||
l.writer.WriteByte(' ')
|
||||
|
||||
s, ok := _levelToBracket[level]
|
||||
if ok {
|
||||
z.w.WriteString(s)
|
||||
l.writer.WriteString(s)
|
||||
} else {
|
||||
z.w.WriteString("[UNKN ]")
|
||||
l.writer.WriteString("[?????]")
|
||||
}
|
||||
|
||||
if z.caller {
|
||||
if l.caller {
|
||||
if _, file, line, ok := runtime.Caller(3); ok {
|
||||
z.w.WriteByte(' ')
|
||||
z.w.WriteString(trimCallerPath(file))
|
||||
z.w.WriteByte(':')
|
||||
z.w.WriteString(strconv.Itoa(line))
|
||||
z.w.WriteByte(':')
|
||||
l.writer.WriteByte(' ')
|
||||
l.writer.WriteString(trimCallerPath(file))
|
||||
l.writer.WriteByte(':')
|
||||
l.writer.WriteString(strconv.Itoa(line))
|
||||
l.writer.WriteByte(':')
|
||||
}
|
||||
}
|
||||
|
||||
z.w.WriteByte(' ')
|
||||
l.writer.WriteByte(' ')
|
||||
|
||||
if z.name != "" {
|
||||
z.w.WriteString(z.name)
|
||||
z.w.WriteString(": ")
|
||||
if l.name != "" {
|
||||
l.writer.WriteString(l.name)
|
||||
l.writer.WriteString(": ")
|
||||
}
|
||||
|
||||
z.w.WriteString(msg)
|
||||
l.writer.WriteString(msg)
|
||||
|
||||
args = append(z.implied, args...)
|
||||
args = append(l.implied, args...)
|
||||
|
||||
var stacktrace CapturedStacktrace
|
||||
|
||||
@@ -182,11 +187,14 @@ func (z *intLogger) log(t time.Time, level Level, msg string, args ...interface{
|
||||
}
|
||||
}
|
||||
|
||||
z.w.WriteByte(':')
|
||||
l.writer.WriteByte(':')
|
||||
|
||||
FOR:
|
||||
for i := 0; i < len(args); i = i + 2 {
|
||||
var val string
|
||||
var (
|
||||
val string
|
||||
raw bool
|
||||
)
|
||||
|
||||
switch st := args[i+1].(type) {
|
||||
case string:
|
||||
@@ -217,32 +225,77 @@ func (z *intLogger) log(t time.Time, level Level, msg string, args ...interface{
|
||||
case Format:
|
||||
val = fmt.Sprintf(st[0].(string), st[1:]...)
|
||||
default:
|
||||
val = fmt.Sprintf("%v", st)
|
||||
v := reflect.ValueOf(st)
|
||||
if v.Kind() == reflect.Slice {
|
||||
val = l.renderSlice(v)
|
||||
raw = true
|
||||
} else {
|
||||
val = fmt.Sprintf("%v", st)
|
||||
}
|
||||
}
|
||||
|
||||
z.w.WriteByte(' ')
|
||||
z.w.WriteString(args[i].(string))
|
||||
z.w.WriteByte('=')
|
||||
l.writer.WriteByte(' ')
|
||||
l.writer.WriteString(args[i].(string))
|
||||
l.writer.WriteByte('=')
|
||||
|
||||
if strings.ContainsAny(val, " \t\n\r") {
|
||||
z.w.WriteByte('"')
|
||||
z.w.WriteString(val)
|
||||
z.w.WriteByte('"')
|
||||
if !raw && strings.ContainsAny(val, " \t\n\r") {
|
||||
l.writer.WriteByte('"')
|
||||
l.writer.WriteString(val)
|
||||
l.writer.WriteByte('"')
|
||||
} else {
|
||||
z.w.WriteString(val)
|
||||
l.writer.WriteString(val)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
z.w.WriteString("\n")
|
||||
l.writer.WriteString("\n")
|
||||
|
||||
if stacktrace != "" {
|
||||
z.w.WriteString(string(stacktrace))
|
||||
l.writer.WriteString(string(stacktrace))
|
||||
}
|
||||
}
|
||||
|
||||
func (l *intLogger) renderSlice(v reflect.Value) string {
|
||||
var buf bytes.Buffer
|
||||
|
||||
buf.WriteRune('[')
|
||||
|
||||
for i := 0; i < v.Len(); i++ {
|
||||
if i > 0 {
|
||||
buf.WriteString(", ")
|
||||
}
|
||||
|
||||
sv := v.Index(i)
|
||||
|
||||
var val string
|
||||
|
||||
switch sv.Kind() {
|
||||
case reflect.String:
|
||||
val = sv.String()
|
||||
case reflect.Int, reflect.Int16, reflect.Int32, reflect.Int64:
|
||||
val = strconv.FormatInt(sv.Int(), 10)
|
||||
case reflect.Uint, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
val = strconv.FormatUint(sv.Uint(), 10)
|
||||
default:
|
||||
val = fmt.Sprintf("%v", sv.Interface())
|
||||
}
|
||||
|
||||
if strings.ContainsAny(val, " \t\n\r") {
|
||||
buf.WriteByte('"')
|
||||
buf.WriteString(val)
|
||||
buf.WriteByte('"')
|
||||
} else {
|
||||
buf.WriteString(val)
|
||||
}
|
||||
}
|
||||
|
||||
buf.WriteRune(']')
|
||||
|
||||
return buf.String()
|
||||
}
|
||||
|
||||
// JSON logging function
|
||||
func (z *intLogger) logJson(t time.Time, level Level, msg string, args ...interface{}) {
|
||||
func (l *intLogger) logJSON(t time.Time, level Level, msg string, args ...interface{}) {
|
||||
vals := map[string]interface{}{
|
||||
"@message": msg,
|
||||
"@timestamp": t.Format("2006-01-02T15:04:05.000000Z07:00"),
|
||||
@@ -266,17 +319,17 @@ func (z *intLogger) logJson(t time.Time, level Level, msg string, args ...interf
|
||||
|
||||
vals["@level"] = levelStr
|
||||
|
||||
if z.name != "" {
|
||||
vals["@module"] = z.name
|
||||
if l.name != "" {
|
||||
vals["@module"] = l.name
|
||||
}
|
||||
|
||||
if z.caller {
|
||||
if l.caller {
|
||||
if _, file, line, ok := runtime.Caller(3); ok {
|
||||
vals["@caller"] = fmt.Sprintf("%s:%d", file, line)
|
||||
}
|
||||
}
|
||||
|
||||
args = append(z.implied, args...)
|
||||
args = append(l.implied, args...)
|
||||
|
||||
if args != nil && len(args) > 0 {
|
||||
if len(args)%2 != 0 {
|
||||
@@ -314,107 +367,145 @@ func (z *intLogger) logJson(t time.Time, level Level, msg string, args ...interf
|
||||
}
|
||||
}
|
||||
|
||||
err := json.NewEncoder(z.w).Encode(vals)
|
||||
err := json.NewEncoder(l.writer).Encode(vals)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
|
||||
// Emit the message and args at DEBUG level
|
||||
func (z *intLogger) Debug(msg string, args ...interface{}) {
|
||||
z.Log(Debug, msg, args...)
|
||||
func (l *intLogger) Debug(msg string, args ...interface{}) {
|
||||
l.Log(Debug, msg, args...)
|
||||
}
|
||||
|
||||
// Emit the message and args at TRACE level
|
||||
func (z *intLogger) Trace(msg string, args ...interface{}) {
|
||||
z.Log(Trace, msg, args...)
|
||||
func (l *intLogger) Trace(msg string, args ...interface{}) {
|
||||
l.Log(Trace, msg, args...)
|
||||
}
|
||||
|
||||
// Emit the message and args at INFO level
|
||||
func (z *intLogger) Info(msg string, args ...interface{}) {
|
||||
z.Log(Info, msg, args...)
|
||||
func (l *intLogger) Info(msg string, args ...interface{}) {
|
||||
l.Log(Info, msg, args...)
|
||||
}
|
||||
|
||||
// Emit the message and args at WARN level
|
||||
func (z *intLogger) Warn(msg string, args ...interface{}) {
|
||||
z.Log(Warn, msg, args...)
|
||||
func (l *intLogger) Warn(msg string, args ...interface{}) {
|
||||
l.Log(Warn, msg, args...)
|
||||
}
|
||||
|
||||
// Emit the message and args at ERROR level
|
||||
func (z *intLogger) Error(msg string, args ...interface{}) {
|
||||
z.Log(Error, msg, args...)
|
||||
func (l *intLogger) Error(msg string, args ...interface{}) {
|
||||
l.Log(Error, msg, args...)
|
||||
}
|
||||
|
||||
// Indicate that the logger would emit TRACE level logs
|
||||
func (z *intLogger) IsTrace() bool {
|
||||
return z.level == Trace
|
||||
func (l *intLogger) IsTrace() bool {
|
||||
return Level(atomic.LoadInt32(l.level)) == Trace
|
||||
}
|
||||
|
||||
// Indicate that the logger would emit DEBUG level logs
|
||||
func (z *intLogger) IsDebug() bool {
|
||||
return z.level <= Debug
|
||||
func (l *intLogger) IsDebug() bool {
|
||||
return Level(atomic.LoadInt32(l.level)) <= Debug
|
||||
}
|
||||
|
||||
// Indicate that the logger would emit INFO level logs
|
||||
func (z *intLogger) IsInfo() bool {
|
||||
return z.level <= Info
|
||||
func (l *intLogger) IsInfo() bool {
|
||||
return Level(atomic.LoadInt32(l.level)) <= Info
|
||||
}
|
||||
|
||||
// Indicate that the logger would emit WARN level logs
|
||||
func (z *intLogger) IsWarn() bool {
|
||||
return z.level <= Warn
|
||||
func (l *intLogger) IsWarn() bool {
|
||||
return Level(atomic.LoadInt32(l.level)) <= Warn
|
||||
}
|
||||
|
||||
// Indicate that the logger would emit ERROR level logs
|
||||
func (z *intLogger) IsError() bool {
|
||||
return z.level <= Error
|
||||
func (l *intLogger) IsError() bool {
|
||||
return Level(atomic.LoadInt32(l.level)) <= Error
|
||||
}
|
||||
|
||||
// Return a sub-Logger for which every emitted log message will contain
|
||||
// the given key/value pairs. This is used to create a context specific
|
||||
// Logger.
|
||||
func (z *intLogger) With(args ...interface{}) Logger {
|
||||
var nz intLogger = *z
|
||||
func (l *intLogger) With(args ...interface{}) Logger {
|
||||
if len(args)%2 != 0 {
|
||||
panic("With() call requires paired arguments")
|
||||
}
|
||||
|
||||
nz.implied = make([]interface{}, 0, len(z.implied)+len(args))
|
||||
nz.implied = append(nz.implied, z.implied...)
|
||||
nz.implied = append(nz.implied, args...)
|
||||
sl := *l
|
||||
|
||||
return &nz
|
||||
result := make(map[string]interface{}, len(l.implied)+len(args))
|
||||
keys := make([]string, 0, len(l.implied)+len(args))
|
||||
|
||||
// Read existing args, store map and key for consistent sorting
|
||||
for i := 0; i < len(l.implied); i += 2 {
|
||||
key := l.implied[i].(string)
|
||||
keys = append(keys, key)
|
||||
result[key] = l.implied[i+1]
|
||||
}
|
||||
// Read new args, store map and key for consistent sorting
|
||||
for i := 0; i < len(args); i += 2 {
|
||||
key := args[i].(string)
|
||||
_, exists := result[key]
|
||||
if !exists {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
result[key] = args[i+1]
|
||||
}
|
||||
|
||||
// Sort keys to be consistent
|
||||
sort.Strings(keys)
|
||||
|
||||
sl.implied = make([]interface{}, 0, len(l.implied)+len(args))
|
||||
for _, k := range keys {
|
||||
sl.implied = append(sl.implied, k)
|
||||
sl.implied = append(sl.implied, result[k])
|
||||
}
|
||||
|
||||
return &sl
|
||||
}
|
||||
|
||||
// Create a new sub-Logger that a name decending from the current name.
|
||||
// This is used to create a subsystem specific Logger.
|
||||
func (z *intLogger) Named(name string) Logger {
|
||||
var nz intLogger = *z
|
||||
func (l *intLogger) Named(name string) Logger {
|
||||
sl := *l
|
||||
|
||||
if nz.name != "" {
|
||||
nz.name = nz.name + "." + name
|
||||
if sl.name != "" {
|
||||
sl.name = sl.name + "." + name
|
||||
} else {
|
||||
nz.name = name
|
||||
sl.name = name
|
||||
}
|
||||
|
||||
return &nz
|
||||
return &sl
|
||||
}
|
||||
|
||||
// Create a new sub-Logger with an explicit name. This ignores the current
|
||||
// name. This is used to create a standalone logger that doesn't fall
|
||||
// within the normal hierarchy.
|
||||
func (z *intLogger) ResetNamed(name string) Logger {
|
||||
var nz intLogger = *z
|
||||
func (l *intLogger) ResetNamed(name string) Logger {
|
||||
sl := *l
|
||||
|
||||
nz.name = name
|
||||
sl.name = name
|
||||
|
||||
return &nz
|
||||
return &sl
|
||||
}
|
||||
|
||||
// Update the logging level on-the-fly. This will affect all subloggers as
|
||||
// well.
|
||||
func (l *intLogger) SetLevel(level Level) {
|
||||
atomic.StoreInt32(l.level, int32(level))
|
||||
}
|
||||
|
||||
// Create a *log.Logger that will send it's data through this Logger. This
|
||||
// allows packages that expect to be using the standard library log to actually
|
||||
// use this logger.
|
||||
func (z *intLogger) StandardLogger(opts *StandardLoggerOptions) *log.Logger {
|
||||
func (l *intLogger) StandardLogger(opts *StandardLoggerOptions) *log.Logger {
|
||||
if opts == nil {
|
||||
opts = &StandardLoggerOptions{}
|
||||
}
|
||||
|
||||
return log.New(&stdlogAdapter{z, opts.InferLevels}, "", 0)
|
||||
return log.New(l.StandardWriter(opts), "", 0)
|
||||
}
|
||||
|
||||
func (l *intLogger) StandardWriter(opts *StandardLoggerOptions) io.Writer {
|
||||
return &stdlogAdapter{l, opts.InferLevels}
|
||||
}
|
||||
Generated
Vendored
+30
-17
@@ -9,38 +9,42 @@ import (
|
||||
)
|
||||
|
||||
var (
|
||||
DefaultOutput = os.Stderr
|
||||
DefaultLevel = Info
|
||||
//DefaultOutput is used as the default log output.
|
||||
DefaultOutput io.Writer = os.Stderr
|
||||
|
||||
// DefaultLevel is used as the default log level.
|
||||
DefaultLevel = Info
|
||||
)
|
||||
|
||||
type Level int
|
||||
// Level represents a log level.
|
||||
type Level int32
|
||||
|
||||
const (
|
||||
// This is a special level used to indicate that no level has been
|
||||
// NoLevel is a special level used to indicate that no level has been
|
||||
// set and allow for a default to be used.
|
||||
NoLevel Level = 0
|
||||
|
||||
// The most verbose level. Intended to be used for the tracing of actions
|
||||
// in code, such as function enters/exits, etc.
|
||||
// Trace is the most verbose level. Intended to be used for the tracing
|
||||
// of actions in code, such as function enters/exits, etc.
|
||||
Trace Level = 1
|
||||
|
||||
// For programmer lowlevel analysis.
|
||||
// Debug information for programmer lowlevel analysis.
|
||||
Debug Level = 2
|
||||
|
||||
// For information about steady state operations.
|
||||
// Info information about steady state operations.
|
||||
Info Level = 3
|
||||
|
||||
// For information about rare but handled events.
|
||||
// Warn information about rare but handled events.
|
||||
Warn Level = 4
|
||||
|
||||
// For information about unrecoverable events.
|
||||
// Error information about unrecoverable events.
|
||||
Error Level = 5
|
||||
)
|
||||
|
||||
// When processing a value of this type, the logger automatically treats the first
|
||||
// argument as a Printf formatting string and passes the rest as the values to be
|
||||
// formatted. For example: L.Info(Fmt{"%d beans/day", beans}). This is a simple
|
||||
// convience type for when formatting is required.
|
||||
// Format is a simple convience type for when formatting is required. When
|
||||
// processing a value of this type, the logger automatically treats the first
|
||||
// argument as a Printf formatting string and passes the rest as the values
|
||||
// to be formatted. For example: L.Info(Fmt{"%d beans/day", beans}).
|
||||
type Format []interface{}
|
||||
|
||||
// Fmt returns a Format type. This is a convience function for creating a Format
|
||||
@@ -53,7 +57,7 @@ func Fmt(str string, args ...interface{}) Format {
|
||||
// the level string is invalid. This facilitates setting the log level via
|
||||
// config or environment variable by name in a predictable way.
|
||||
func LevelFromString(levelStr string) Level {
|
||||
// We don't care about case. Accept "INFO" or "info"
|
||||
// We don't care about case. Accept both "INFO" and "info".
|
||||
levelStr = strings.ToLower(strings.TrimSpace(levelStr))
|
||||
switch levelStr {
|
||||
case "trace":
|
||||
@@ -71,7 +75,7 @@ func LevelFromString(levelStr string) Level {
|
||||
}
|
||||
}
|
||||
|
||||
// The main Logger interface. All code should code against this interface only.
|
||||
// Logger describes the interface that must be implemeted by all loggers.
|
||||
type Logger interface {
|
||||
// Args are alternating key, val pairs
|
||||
// keys must be strings
|
||||
@@ -121,10 +125,18 @@ type Logger interface {
|
||||
// the current name as well.
|
||||
ResetNamed(name string) Logger
|
||||
|
||||
// Updates the level. This should affect all sub-loggers as well. If an
|
||||
// implementation cannot update the level on the fly, it should no-op.
|
||||
SetLevel(level Level)
|
||||
|
||||
// Return a value that conforms to the stdlib log.Logger interface
|
||||
StandardLogger(opts *StandardLoggerOptions) *log.Logger
|
||||
|
||||
// Return a value that conforms to io.Writer, which can be passed into log.SetOutput()
|
||||
StandardWriter(opts *StandardLoggerOptions) io.Writer
|
||||
}
|
||||
|
||||
// StandardLoggerOptions can be used to configure a new standard logger.
|
||||
type StandardLoggerOptions struct {
|
||||
// Indicate that some minimal parsing should be done on strings to try
|
||||
// and detect their level and re-emit them.
|
||||
@@ -133,6 +145,7 @@ type StandardLoggerOptions struct {
|
||||
InferLevels bool
|
||||
}
|
||||
|
||||
// LoggerOptions can be used to configure a new logger.
|
||||
type LoggerOptions struct {
|
||||
// Name of the subsystem to prefix logs with
|
||||
Name string
|
||||
@@ -140,7 +153,7 @@ type LoggerOptions struct {
|
||||
// The threshold for the logger. Anything less severe is supressed
|
||||
Level Level
|
||||
|
||||
// Where to write the logs to. Defaults to os.Stdout if nil
|
||||
// Where to write the logs to. Defaults to os.Stderr if nil
|
||||
Output io.Writer
|
||||
|
||||
// An optional mutex pointer in case Output is shared
|
||||
+10
-3
@@ -1,8 +1,9 @@
|
||||
package hclog
|
||||
|
||||
import (
|
||||
"log"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
)
|
||||
|
||||
// NewNullLogger instantiates a Logger for which all calls
|
||||
@@ -40,6 +41,12 @@ func (l *nullLogger) Named(name string) Logger { return l }
|
||||
|
||||
func (l *nullLogger) ResetNamed(name string) Logger { return l }
|
||||
|
||||
func (l *nullLogger) SetLevel(level Level) {}
|
||||
|
||||
func (l *nullLogger) StandardLogger(opts *StandardLoggerOptions) *log.Logger {
|
||||
return log.New(ioutil.Discard, "", log.LstdFlags)
|
||||
}
|
||||
return log.New(l.StandardWriter(opts), "", log.LstdFlags)
|
||||
}
|
||||
|
||||
func (l *nullLogger) StandardWriter(opts *StandardLoggerOptions) io.Writer {
|
||||
return ioutil.Discard
|
||||
}
|
||||
|
||||
+5
-4
@@ -40,12 +40,13 @@ var (
|
||||
}
|
||||
)
|
||||
|
||||
// A stacktrace gathered by a previous call to log.Stacktrace. If passed
|
||||
// to a logging function, the stacktrace will be appended.
|
||||
// CapturedStacktrace represents a stacktrace captured by a previous call
|
||||
// to log.Stacktrace. If passed to a logging function, the stacktrace
|
||||
// will be appended.
|
||||
type CapturedStacktrace string
|
||||
|
||||
// Gather a stacktrace of the current goroutine and return it to be passed
|
||||
// to a logging function.
|
||||
// Stacktrace captures a stacktrace of the current goroutine and returns
|
||||
// it to be passed to a logging function.
|
||||
func Stacktrace() CapturedStacktrace {
|
||||
return CapturedStacktrace(takeStacktrace())
|
||||
}
|
||||
|
||||
+10
-10
@@ -9,12 +9,12 @@ import (
|
||||
// and back into our Logger. This is basically the only way to
|
||||
// build upon *log.Logger.
|
||||
type stdlogAdapter struct {
|
||||
hl Logger
|
||||
log Logger
|
||||
inferLevels bool
|
||||
}
|
||||
|
||||
// Take the data, infer the levels if configured, and send it through
|
||||
// a regular Logger
|
||||
// a regular Logger.
|
||||
func (s *stdlogAdapter) Write(data []byte) (int, error) {
|
||||
str := string(bytes.TrimRight(data, " \t\n"))
|
||||
|
||||
@@ -22,26 +22,26 @@ func (s *stdlogAdapter) Write(data []byte) (int, error) {
|
||||
level, str := s.pickLevel(str)
|
||||
switch level {
|
||||
case Trace:
|
||||
s.hl.Trace(str)
|
||||
s.log.Trace(str)
|
||||
case Debug:
|
||||
s.hl.Debug(str)
|
||||
s.log.Debug(str)
|
||||
case Info:
|
||||
s.hl.Info(str)
|
||||
s.log.Info(str)
|
||||
case Warn:
|
||||
s.hl.Warn(str)
|
||||
s.log.Warn(str)
|
||||
case Error:
|
||||
s.hl.Error(str)
|
||||
s.log.Error(str)
|
||||
default:
|
||||
s.hl.Info(str)
|
||||
s.log.Info(str)
|
||||
}
|
||||
} else {
|
||||
s.hl.Info(str)
|
||||
s.log.Info(str)
|
||||
}
|
||||
|
||||
return len(data), nil
|
||||
}
|
||||
|
||||
// Detect, based on conventions, what log level this is
|
||||
// Detect, based on conventions, what log level this is.
|
||||
func (s *stdlogAdapter) pickLevel(str string) (Level, string) {
|
||||
switch {
|
||||
case strings.HasPrefix(str, "[DEBUG]"):
|
||||
|
||||
+74
@@ -0,0 +1,74 @@
|
||||
package hclog
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
)
|
||||
|
||||
type writer struct {
|
||||
b bytes.Buffer
|
||||
w io.Writer
|
||||
}
|
||||
|
||||
func newWriter(w io.Writer) *writer {
|
||||
return &writer{w: w}
|
||||
}
|
||||
|
||||
func (w *writer) Flush(level Level) (err error) {
|
||||
if lw, ok := w.w.(LevelWriter); ok {
|
||||
_, err = lw.LevelWrite(level, w.b.Bytes())
|
||||
} else {
|
||||
_, err = w.w.Write(w.b.Bytes())
|
||||
}
|
||||
w.b.Reset()
|
||||
return err
|
||||
}
|
||||
|
||||
func (w *writer) Write(p []byte) (int, error) {
|
||||
return w.b.Write(p)
|
||||
}
|
||||
|
||||
func (w *writer) WriteByte(c byte) error {
|
||||
return w.b.WriteByte(c)
|
||||
}
|
||||
|
||||
func (w *writer) WriteString(s string) (int, error) {
|
||||
return w.b.WriteString(s)
|
||||
}
|
||||
|
||||
// LevelWriter is the interface that wraps the LevelWrite method.
|
||||
type LevelWriter interface {
|
||||
LevelWrite(level Level, p []byte) (n int, err error)
|
||||
}
|
||||
|
||||
// LeveledWriter writes all log messages to the standard writer,
|
||||
// except for log levels that are defined in the overrides map.
|
||||
type LeveledWriter struct {
|
||||
standard io.Writer
|
||||
overrides map[Level]io.Writer
|
||||
}
|
||||
|
||||
// NewLeveledWriter returns an initialized LeveledWriter.
|
||||
//
|
||||
// standard will be used as the default writer for all log levels,
|
||||
// except for log levels that are defined in the overrides map.
|
||||
func NewLeveledWriter(standard io.Writer, overrides map[Level]io.Writer) *LeveledWriter {
|
||||
return &LeveledWriter{
|
||||
standard: standard,
|
||||
overrides: overrides,
|
||||
}
|
||||
}
|
||||
|
||||
// Write implements io.Writer.
|
||||
func (lw *LeveledWriter) Write(p []byte) (int, error) {
|
||||
return lw.standard.Write(p)
|
||||
}
|
||||
|
||||
// LevelWrite implements LevelWriter.
|
||||
func (lw *LeveledWriter) LevelWrite(level Level, p []byte) (int, error) {
|
||||
w, ok := lw.overrides[level]
|
||||
if !ok {
|
||||
w = lw.standard
|
||||
}
|
||||
return w.Write(p)
|
||||
}
|
||||
@@ -0,0 +1 @@
|
||||
.DS_Store
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
# Go Plugin System over RPC
|
||||
|
||||
`go-plugin` is a Go (golang) plugin system over RPC. It is the plugin system
|
||||
that has been in use by HashiCorp tooling for over 4 years. While initially
|
||||
created for [Packer](https://www.packer.io), it is additionally in use by
|
||||
[Terraform](https://www.terraform.io), [Nomad](https://www.nomadproject.io), and
|
||||
[Vault](https://www.vaultproject.io).
|
||||
|
||||
While the plugin system is over RPC, it is currently only designed to work
|
||||
over a local [reliable] network. Plugins over a real network are not supported
|
||||
and will lead to unexpected behavior.
|
||||
|
||||
This plugin system has been used on millions of machines across many different
|
||||
projects and has proven to be battle hardened and ready for production use.
|
||||
|
||||
## Features
|
||||
|
||||
The HashiCorp plugin system supports a number of features:
|
||||
|
||||
**Plugins are Go interface implementations.** This makes writing and consuming
|
||||
plugins feel very natural. To a plugin author: you just implement an
|
||||
interface as if it were going to run in the same process. For a plugin user:
|
||||
you just use and call functions on an interface as if it were in the same
|
||||
process. This plugin system handles the communication in between.
|
||||
|
||||
**Cross-language support.** Plugins can be written (and consumed) by
|
||||
almost every major language. This library supports serving plugins via
|
||||
[gRPC](http://www.grpc.io). gRPC-based plugins enable plugins to be written
|
||||
in any language.
|
||||
|
||||
**Complex arguments and return values are supported.** This library
|
||||
provides APIs for handling complex arguments and return values such
|
||||
as interfaces, `io.Reader/Writer`, etc. We do this by giving you a library
|
||||
(`MuxBroker`) for creating new connections between the client/server to
|
||||
serve additional interfaces or transfer raw data.
|
||||
|
||||
**Bidirectional communication.** Because the plugin system supports
|
||||
complex arguments, the host process can send it interface implementations
|
||||
and the plugin can call back into the host process.
|
||||
|
||||
**Built-in Logging.** Any plugins that use the `log` standard library
|
||||
will have log data automatically sent to the host process. The host
|
||||
process will mirror this output prefixed with the path to the plugin
|
||||
binary. This makes debugging with plugins simple. If the host system
|
||||
uses [hclog](https://github.com/hashicorp/go-hclog) then the log data
|
||||
will be structured. If the plugin also uses hclog, logs from the plugin
|
||||
will be sent to the host hclog and be structured.
|
||||
|
||||
**Protocol Versioning.** A very basic "protocol version" is supported that
|
||||
can be incremented to invalidate any previous plugins. This is useful when
|
||||
interface signatures are changing, protocol level changes are necessary,
|
||||
etc. When a protocol version is incompatible, a human friendly error
|
||||
message is shown to the end user.
|
||||
|
||||
**Stdout/Stderr Syncing.** While plugins are subprocesses, they can continue
|
||||
to use stdout/stderr as usual and the output will get mirrored back to
|
||||
the host process. The host process can control what `io.Writer` these
|
||||
streams go to to prevent this from happening.
|
||||
|
||||
**TTY Preservation.** Plugin subprocesses are connected to the identical
|
||||
stdin file descriptor as the host process, allowing software that requires
|
||||
a TTY to work. For example, a plugin can execute `ssh` and even though there
|
||||
are multiple subprocesses and RPC happening, it will look and act perfectly
|
||||
to the end user.
|
||||
|
||||
**Host upgrade while a plugin is running.** Plugins can be "reattached"
|
||||
so that the host process can be upgraded while the plugin is still running.
|
||||
This requires the host/plugin to know this is possible and daemonize
|
||||
properly. `NewClient` takes a `ReattachConfig` to determine if and how to
|
||||
reattach.
|
||||
|
||||
**Cryptographically Secure Plugins.** Plugins can be verified with an expected
|
||||
checksum and RPC communications can be configured to use TLS. The host process
|
||||
must be properly secured to protect this configuration.
|
||||
|
||||
## Architecture
|
||||
|
||||
The HashiCorp plugin system works by launching subprocesses and communicating
|
||||
over RPC (using standard `net/rpc` or [gRPC](http://www.grpc.io)). A single
|
||||
connection is made between any plugin and the host process. For net/rpc-based
|
||||
plugins, we use a [connection multiplexing](https://github.com/hashicorp/yamux)
|
||||
library to multiplex any other connections on top. For gRPC-based plugins,
|
||||
the HTTP2 protocol handles multiplexing.
|
||||
|
||||
This architecture has a number of benefits:
|
||||
|
||||
* Plugins can't crash your host process: A panic in a plugin doesn't
|
||||
panic the plugin user.
|
||||
|
||||
* Plugins are very easy to write: just write a Go application and `go build`.
|
||||
Or use any other language to write a gRPC server with a tiny amount of
|
||||
boilerplate to support go-plugin.
|
||||
|
||||
* Plugins are very easy to install: just put the binary in a location where
|
||||
the host will find it (depends on the host but this library also provides
|
||||
helpers), and the plugin host handles the rest.
|
||||
|
||||
* Plugins can be relatively secure: The plugin only has access to the
|
||||
interfaces and args given to it, not to the entire memory space of the
|
||||
process. Additionally, go-plugin can communicate with the plugin over
|
||||
TLS.
|
||||
|
||||
## Usage
|
||||
|
||||
To use the plugin system, you must take the following steps. These are
|
||||
high-level steps that must be done. Examples are available in the
|
||||
`examples/` directory.
|
||||
|
||||
1. Choose the interface(s) you want to expose for plugins.
|
||||
|
||||
2. For each interface, implement an implementation of that interface
|
||||
that communicates over a `net/rpc` connection or over a
|
||||
[gRPC](http://www.grpc.io) connection or both. You'll have to implement
|
||||
both a client and server implementation.
|
||||
|
||||
3. Create a `Plugin` implementation that knows how to create the RPC
|
||||
client/server for a given plugin type.
|
||||
|
||||
4. Plugin authors call `plugin.Serve` to serve a plugin from the
|
||||
`main` function.
|
||||
|
||||
5. Plugin users use `plugin.Client` to launch a subprocess and request
|
||||
an interface implementation over RPC.
|
||||
|
||||
That's it! In practice, step 2 is the most tedious and time consuming step.
|
||||
Even so, it isn't very difficult and you can see examples in the `examples/`
|
||||
directory as well as throughout our various open source projects.
|
||||
|
||||
For complete API documentation, see [GoDoc](https://godoc.org/github.com/hashicorp/go-plugin).
|
||||
|
||||
## Roadmap
|
||||
|
||||
Our plugin system is constantly evolving. As we use the plugin system for
|
||||
new projects or for new features in existing projects, we constantly find
|
||||
improvements we can make.
|
||||
|
||||
At this point in time, the roadmap for the plugin system is:
|
||||
|
||||
**Semantic Versioning.** Plugins will be able to implement a semantic version.
|
||||
This plugin system will give host processes a system for constraining
|
||||
versions. This is in addition to the protocol versioning already present
|
||||
which is more for larger underlying changes.
|
||||
|
||||
**Plugin fetching.** We will integrate with [go-getter](https://github.com/hashicorp/go-getter)
|
||||
to support automatic download + install of plugins. Paired with cryptographically
|
||||
secure plugins (above), we can make this a safe operation for an amazing
|
||||
user experience.
|
||||
|
||||
## What About Shared Libraries?
|
||||
|
||||
When we started using plugins (late 2012, early 2013), plugins over RPC
|
||||
were the only option since Go didn't support dynamic library loading. Today,
|
||||
Go supports the [plugin](https://golang.org/pkg/plugin/) standard library with
|
||||
a number of limitations. Since 2012, our plugin system has stabilized
|
||||
from tens of millions of users using it, and has many benefits we've come to
|
||||
value greatly.
|
||||
|
||||
For example, we use this plugin system in
|
||||
[Vault](https://www.vaultproject.io) where dynamic library loading is
|
||||
not acceptable for security reasons. That is an extreme
|
||||
example, but we believe our library system has more upsides than downsides
|
||||
over dynamic library loading and since we've had it built and tested for years,
|
||||
we'll continue to use it.
|
||||
|
||||
Shared libraries have one major advantage over our system which is much
|
||||
higher performance. In real world scenarios across our various tools,
|
||||
we've never required any more performance out of our plugin system and it
|
||||
has seen very high throughput, so this isn't a concern for us at the moment.
|
||||
+350
-147
@@ -5,6 +5,8 @@ import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
"hash"
|
||||
@@ -19,7 +21,6 @@ import (
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
hclog "github.com/hashicorp/go-hclog"
|
||||
)
|
||||
@@ -70,16 +71,31 @@ var (
|
||||
//
|
||||
// See NewClient and ClientConfig for using a Client.
|
||||
type Client struct {
|
||||
config *ClientConfig
|
||||
exited bool
|
||||
doneLogging chan struct{}
|
||||
l sync.Mutex
|
||||
address net.Addr
|
||||
process *os.Process
|
||||
client ClientProtocol
|
||||
protocol Protocol
|
||||
logger hclog.Logger
|
||||
doneCtx context.Context
|
||||
config *ClientConfig
|
||||
exited bool
|
||||
l sync.Mutex
|
||||
address net.Addr
|
||||
process *os.Process
|
||||
client ClientProtocol
|
||||
protocol Protocol
|
||||
logger hclog.Logger
|
||||
doneCtx context.Context
|
||||
ctxCancel context.CancelFunc
|
||||
negotiatedVersion int
|
||||
|
||||
// clientWaitGroup is used to manage the lifecycle of the plugin management
|
||||
// goroutines.
|
||||
clientWaitGroup sync.WaitGroup
|
||||
|
||||
// processKilled is used for testing only, to flag when the process was
|
||||
// forcefully killed.
|
||||
processKilled bool
|
||||
}
|
||||
|
||||
// NegotiatedVersion returns the protocol version negotiated with the server.
|
||||
// This is only valid after Start() is called.
|
||||
func (c *Client) NegotiatedVersion() int {
|
||||
return c.negotiatedVersion
|
||||
}
|
||||
|
||||
// ClientConfig is the configuration used to initialize a new
|
||||
@@ -90,7 +106,13 @@ type ClientConfig struct {
|
||||
HandshakeConfig
|
||||
|
||||
// Plugins are the plugins that can be consumed.
|
||||
Plugins map[string]Plugin
|
||||
// The implied version of this PluginSet is the Handshake.ProtocolVersion.
|
||||
Plugins PluginSet
|
||||
|
||||
// VersionedPlugins is a map of PluginSets for specific protocol versions.
|
||||
// These can be used to negotiate a compatible version between client and
|
||||
// server. If this is set, Handshake.ProtocolVersion is not required.
|
||||
VersionedPlugins map[int]PluginSet
|
||||
|
||||
// One of the following must be set, but not both.
|
||||
//
|
||||
@@ -157,6 +179,29 @@ type ClientConfig struct {
|
||||
// Logger is the logger that the client will used. If none is provided,
|
||||
// it will default to hclog's default logger.
|
||||
Logger hclog.Logger
|
||||
|
||||
// AutoMTLS has the client and server automatically negotiate mTLS for
|
||||
// transport authentication. This ensures that only the original client will
|
||||
// be allowed to connect to the server, and all other connections will be
|
||||
// rejected. The client will also refuse to connect to any server that isn't
|
||||
// the original instance started by the client.
|
||||
//
|
||||
// In this mode of operation, the client generates a one-time use tls
|
||||
// certificate, sends the public x.509 certificate to the new server, and
|
||||
// the server generates a one-time use tls certificate, and sends the public
|
||||
// x.509 certificate back to the client. These are used to authenticate all
|
||||
// rpc connections between the client and server.
|
||||
//
|
||||
// Setting AutoMTLS to true implies that the server must support the
|
||||
// protocol, and correctly negotiate the tls certificates, or a connection
|
||||
// failure will result.
|
||||
//
|
||||
// The client should not set TLSConfig, nor should the server set a
|
||||
// TLSProvider, because AutoMTLS implies that a new certificate and tls
|
||||
// configuration will be generated at startup.
|
||||
//
|
||||
// You cannot Reattach to a server with this option enabled.
|
||||
AutoMTLS bool
|
||||
}
|
||||
|
||||
// ReattachConfig is used to configure a client to reattach to an
|
||||
@@ -331,6 +376,14 @@ func (c *Client) Exited() bool {
|
||||
return c.exited
|
||||
}
|
||||
|
||||
// killed is used in tests to check if a process failed to exit gracefully, and
|
||||
// needed to be killed.
|
||||
func (c *Client) killed() bool {
|
||||
c.l.Lock()
|
||||
defer c.l.Unlock()
|
||||
return c.processKilled
|
||||
}
|
||||
|
||||
// End the executing subprocess (if it is running) and perform any cleanup
|
||||
// tasks necessary such as capturing any remaining logs and so on.
|
||||
//
|
||||
@@ -342,14 +395,24 @@ func (c *Client) Kill() {
|
||||
c.l.Lock()
|
||||
process := c.process
|
||||
addr := c.address
|
||||
doneCh := c.doneLogging
|
||||
c.l.Unlock()
|
||||
|
||||
// If there is no process, we never started anything. Nothing to kill.
|
||||
// If there is no process, there is nothing to kill.
|
||||
if process == nil {
|
||||
return
|
||||
}
|
||||
|
||||
defer func() {
|
||||
// Wait for the all client goroutines to finish.
|
||||
c.clientWaitGroup.Wait()
|
||||
|
||||
// Make sure there is no reference to the old process after it has been
|
||||
// killed.
|
||||
c.l.Lock()
|
||||
c.process = nil
|
||||
c.l.Unlock()
|
||||
}()
|
||||
|
||||
// We need to check for address here. It is possible that the plugin
|
||||
// started (process != nil) but has no address (addr == nil) if the
|
||||
// plugin failed at startup. If we do have an address, we need to close
|
||||
@@ -370,6 +433,8 @@ func (c *Client) Kill() {
|
||||
// kill in a moment anyways.
|
||||
c.logger.Warn("error closing client during Kill", "err", err)
|
||||
}
|
||||
} else {
|
||||
c.logger.Error("client", "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -378,17 +443,20 @@ func (c *Client) Kill() {
|
||||
// doneCh which would be closed if the process exits.
|
||||
if graceful {
|
||||
select {
|
||||
case <-doneCh:
|
||||
case <-c.doneCtx.Done():
|
||||
c.logger.Debug("plugin exited")
|
||||
return
|
||||
case <-time.After(250 * time.Millisecond):
|
||||
case <-time.After(2 * time.Second):
|
||||
}
|
||||
}
|
||||
|
||||
// If graceful exiting failed, just kill it
|
||||
c.logger.Warn("plugin failed to exit gracefully")
|
||||
process.Kill()
|
||||
|
||||
// Wait for the client to finish logging so we have a complete log
|
||||
<-doneCh
|
||||
c.l.Lock()
|
||||
c.processKilled = true
|
||||
c.l.Unlock()
|
||||
}
|
||||
|
||||
// Starts the underlying subprocess, communicating with it to negotiate
|
||||
@@ -407,7 +475,7 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
|
||||
// If one of cmd or reattach isn't set, then it is an error. We wrap
|
||||
// this in a {} for scoping reasons, and hopeful that the escape
|
||||
// analysis will pop the stock here.
|
||||
// analysis will pop the stack here.
|
||||
{
|
||||
cmdSet := c.config.Cmd != nil
|
||||
attachSet := c.config.Reattach != nil
|
||||
@@ -421,77 +489,49 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
}
|
||||
}
|
||||
|
||||
// Create the logging channel for when we kill
|
||||
c.doneLogging = make(chan struct{})
|
||||
// Create a context for when we kill
|
||||
var ctxCancel context.CancelFunc
|
||||
c.doneCtx, ctxCancel = context.WithCancel(context.Background())
|
||||
|
||||
if c.config.Reattach != nil {
|
||||
// Verify the process still exists. If not, then it is an error
|
||||
p, err := os.FindProcess(c.config.Reattach.Pid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c.reattach()
|
||||
}
|
||||
|
||||
// Attempt to connect to the addr since on Unix systems FindProcess
|
||||
// doesn't actually return an error if it can't find the process.
|
||||
conn, err := net.Dial(
|
||||
c.config.Reattach.Addr.Network(),
|
||||
c.config.Reattach.Addr.String())
|
||||
if err != nil {
|
||||
p.Kill()
|
||||
return nil, ErrProcessNotFound
|
||||
}
|
||||
conn.Close()
|
||||
if c.config.VersionedPlugins == nil {
|
||||
c.config.VersionedPlugins = make(map[int]PluginSet)
|
||||
}
|
||||
|
||||
// Goroutine to mark exit status
|
||||
go func(pid int) {
|
||||
// Wait for the process to die
|
||||
pidWait(pid)
|
||||
// handle all plugins as versioned, using the handshake config as the default.
|
||||
version := int(c.config.ProtocolVersion)
|
||||
|
||||
// Log so we can see it
|
||||
c.logger.Debug("reattached plugin process exited")
|
||||
// Make sure we're not overwriting a real version 0. If ProtocolVersion was
|
||||
// non-zero, then we have to just assume the user made sure that
|
||||
// VersionedPlugins doesn't conflict.
|
||||
if _, ok := c.config.VersionedPlugins[version]; !ok && c.config.Plugins != nil {
|
||||
c.config.VersionedPlugins[version] = c.config.Plugins
|
||||
}
|
||||
|
||||
// Mark it
|
||||
c.l.Lock()
|
||||
defer c.l.Unlock()
|
||||
c.exited = true
|
||||
|
||||
// Close the logging channel since that doesn't work on reattach
|
||||
close(c.doneLogging)
|
||||
|
||||
// Cancel the context
|
||||
ctxCancel()
|
||||
}(p.Pid)
|
||||
|
||||
// Set the address and process
|
||||
c.address = c.config.Reattach.Addr
|
||||
c.process = p
|
||||
c.protocol = c.config.Reattach.Protocol
|
||||
if c.protocol == "" {
|
||||
// Default the protocol to net/rpc for backwards compatibility
|
||||
c.protocol = ProtocolNetRPC
|
||||
}
|
||||
|
||||
return c.address, nil
|
||||
var versionStrings []string
|
||||
for v := range c.config.VersionedPlugins {
|
||||
versionStrings = append(versionStrings, strconv.Itoa(v))
|
||||
}
|
||||
|
||||
env := []string{
|
||||
fmt.Sprintf("%s=%s", c.config.MagicCookieKey, c.config.MagicCookieValue),
|
||||
fmt.Sprintf("PLUGIN_MIN_PORT=%d", c.config.MinPort),
|
||||
fmt.Sprintf("PLUGIN_MAX_PORT=%d", c.config.MaxPort),
|
||||
fmt.Sprintf("PLUGIN_PROTOCOL_VERSIONS=%s", strings.Join(versionStrings, ",")),
|
||||
}
|
||||
|
||||
stdout_r, stdout_w := io.Pipe()
|
||||
stderr_r, stderr_w := io.Pipe()
|
||||
|
||||
cmd := c.config.Cmd
|
||||
cmd.Env = append(cmd.Env, os.Environ()...)
|
||||
cmd.Env = append(cmd.Env, env...)
|
||||
cmd.Stdin = os.Stdin
|
||||
cmd.Stderr = stderr_w
|
||||
cmd.Stdout = stdout_w
|
||||
|
||||
cmdStdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
cmdStderr, err := cmd.StderrPipe()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
if c.config.SecureConfig != nil {
|
||||
if ok, err := c.config.SecureConfig.Check(cmd.Path); err != nil {
|
||||
@@ -501,6 +541,29 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
}
|
||||
}
|
||||
|
||||
// Setup a temporary certificate for client/server mtls, and send the public
|
||||
// certificate to the plugin.
|
||||
if c.config.AutoMTLS {
|
||||
c.logger.Info("configuring client automatic mTLS")
|
||||
certPEM, keyPEM, err := generateCert()
|
||||
if err != nil {
|
||||
c.logger.Error("failed to generate client certificate", "error", err)
|
||||
return nil, err
|
||||
}
|
||||
cert, err := tls.X509KeyPair(certPEM, keyPEM)
|
||||
if err != nil {
|
||||
c.logger.Error("failed to parse client certificate", "error", err)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
cmd.Env = append(cmd.Env, fmt.Sprintf("PLUGIN_CLIENT_CERT=%s", certPEM))
|
||||
|
||||
c.config.TLSConfig = &tls.Config{
|
||||
Certificates: []tls.Certificate{cert},
|
||||
ServerName: "localhost",
|
||||
}
|
||||
}
|
||||
|
||||
c.logger.Debug("starting plugin", "path", cmd.Path, "args", cmd.Args)
|
||||
err = cmd.Start()
|
||||
if err != nil {
|
||||
@@ -509,6 +572,7 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
|
||||
// Set the process
|
||||
c.process = cmd.Process
|
||||
c.logger.Debug("plugin started", "path", cmd.Path, "pid", c.process.Pid)
|
||||
|
||||
// Make sure the command is properly cleaned up if there is an error
|
||||
defer func() {
|
||||
@@ -523,27 +587,37 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
}
|
||||
}()
|
||||
|
||||
// Start goroutine to wait for process to exit
|
||||
exitCh := make(chan struct{})
|
||||
// Create a context for when we kill
|
||||
c.doneCtx, c.ctxCancel = context.WithCancel(context.Background())
|
||||
|
||||
c.clientWaitGroup.Add(1)
|
||||
go func() {
|
||||
// Make sure we close the write end of our stderr/stdout so
|
||||
// that the readers send EOF properly.
|
||||
defer stderr_w.Close()
|
||||
defer stdout_w.Close()
|
||||
// ensure the context is cancelled when we're done
|
||||
defer c.ctxCancel()
|
||||
|
||||
defer c.clientWaitGroup.Done()
|
||||
|
||||
// get the cmd info early, since the process information will be removed
|
||||
// in Kill.
|
||||
pid := c.process.Pid
|
||||
path := cmd.Path
|
||||
|
||||
// Wait for the command to end.
|
||||
cmd.Wait()
|
||||
err := cmd.Wait()
|
||||
|
||||
debugMsgArgs := []interface{}{
|
||||
"path", path,
|
||||
"pid", pid,
|
||||
}
|
||||
if err != nil {
|
||||
debugMsgArgs = append(debugMsgArgs,
|
||||
[]interface{}{"error", err.Error()}...)
|
||||
}
|
||||
|
||||
// Log and make sure to flush the logs write away
|
||||
c.logger.Debug("plugin process exited", "path", cmd.Path)
|
||||
c.logger.Debug("plugin process exited", debugMsgArgs...)
|
||||
os.Stderr.Sync()
|
||||
|
||||
// Mark that we exited
|
||||
close(exitCh)
|
||||
|
||||
// Cancel the context, marking that we exited
|
||||
ctxCancel()
|
||||
|
||||
// Set that we exited, which takes a lock
|
||||
c.l.Lock()
|
||||
defer c.l.Unlock()
|
||||
@@ -551,32 +625,33 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
}()
|
||||
|
||||
// Start goroutine that logs the stderr
|
||||
go c.logStderr(stderr_r)
|
||||
c.clientWaitGroup.Add(1)
|
||||
// logStderr calls Done()
|
||||
go c.logStderr(cmdStderr)
|
||||
|
||||
// Start a goroutine that is going to be reading the lines
|
||||
// out of stdout
|
||||
linesCh := make(chan []byte)
|
||||
linesCh := make(chan string)
|
||||
c.clientWaitGroup.Add(1)
|
||||
go func() {
|
||||
defer c.clientWaitGroup.Done()
|
||||
defer close(linesCh)
|
||||
|
||||
buf := bufio.NewReader(stdout_r)
|
||||
for {
|
||||
line, err := buf.ReadBytes('\n')
|
||||
if line != nil {
|
||||
linesCh <- line
|
||||
}
|
||||
|
||||
if err == io.EOF {
|
||||
return
|
||||
}
|
||||
scanner := bufio.NewScanner(cmdStdout)
|
||||
for scanner.Scan() {
|
||||
linesCh <- scanner.Text()
|
||||
}
|
||||
}()
|
||||
|
||||
// Make sure after we exit we read the lines from stdout forever
|
||||
// so they don't block since it is an io.Pipe
|
||||
// so they don't block since it is a pipe.
|
||||
// The scanner goroutine above will close this, but track it with a wait
|
||||
// group for completeness.
|
||||
c.clientWaitGroup.Add(1)
|
||||
defer func() {
|
||||
go func() {
|
||||
for _ = range linesCh {
|
||||
defer c.clientWaitGroup.Done()
|
||||
for range linesCh {
|
||||
}
|
||||
}()
|
||||
}()
|
||||
@@ -589,12 +664,12 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
select {
|
||||
case <-timeout:
|
||||
err = errors.New("timeout while waiting for plugin to start")
|
||||
case <-exitCh:
|
||||
case <-c.doneCtx.Done():
|
||||
err = errors.New("plugin exited before we could connect")
|
||||
case lineBytes := <-linesCh:
|
||||
case line := <-linesCh:
|
||||
// Trim the line and split by "|" in order to get the parts of
|
||||
// the output.
|
||||
line := strings.TrimSpace(string(lineBytes))
|
||||
line = strings.TrimSpace(line)
|
||||
parts := strings.SplitN(line, "|", 6)
|
||||
if len(parts) < 4 {
|
||||
err = fmt.Errorf(
|
||||
@@ -622,20 +697,18 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
}
|
||||
}
|
||||
|
||||
// Parse the protocol version
|
||||
var protocol int64
|
||||
protocol, err = strconv.ParseInt(parts[1], 10, 0)
|
||||
// Test the API version
|
||||
version, pluginSet, err := c.checkProtoVersion(parts[1])
|
||||
if err != nil {
|
||||
err = fmt.Errorf("Error parsing protocol version: %s", err)
|
||||
return
|
||||
return addr, err
|
||||
}
|
||||
|
||||
// Test the API version
|
||||
if uint(protocol) != c.config.ProtocolVersion {
|
||||
err = fmt.Errorf("Incompatible API version with plugin. "+
|
||||
"Plugin version: %s, Core version: %d", parts[1], c.config.ProtocolVersion)
|
||||
return
|
||||
}
|
||||
// set the Plugins value to the compatible set, so the version
|
||||
// doesn't need to be passed through to the ClientProtocol
|
||||
// implementation.
|
||||
c.config.Plugins = pluginSet
|
||||
c.negotiatedVersion = version
|
||||
c.logger.Debug("using plugin", "version", version)
|
||||
|
||||
switch parts[2] {
|
||||
case "tcp":
|
||||
@@ -663,15 +736,125 @@ func (c *Client) Start() (addr net.Addr, err error) {
|
||||
if !found {
|
||||
err = fmt.Errorf("Unsupported plugin protocol %q. Supported: %v",
|
||||
c.protocol, c.config.AllowedProtocols)
|
||||
return
|
||||
return addr, err
|
||||
}
|
||||
|
||||
// See if we have a TLS certificate from the server.
|
||||
// Checking if the length is > 50 rules out catching the unused "extra"
|
||||
// data returned from some older implementations.
|
||||
if len(parts) >= 6 && len(parts[5]) > 50 {
|
||||
err := c.loadServerCert(parts[5])
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("error parsing server cert: %s", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
c.address = addr
|
||||
return
|
||||
}
|
||||
|
||||
// loadServerCert is used by AutoMTLS to read an x.509 cert returned by the
|
||||
// server, and load it as the RootCA for the client TLSConfig.
|
||||
func (c *Client) loadServerCert(cert string) error {
|
||||
certPool := x509.NewCertPool()
|
||||
|
||||
asn1, err := base64.RawStdEncoding.DecodeString(cert)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
x509Cert, err := x509.ParseCertificate([]byte(asn1))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
certPool.AddCert(x509Cert)
|
||||
|
||||
c.config.TLSConfig.RootCAs = certPool
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Client) reattach() (net.Addr, error) {
|
||||
// Verify the process still exists. If not, then it is an error
|
||||
p, err := os.FindProcess(c.config.Reattach.Pid)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Attempt to connect to the addr since on Unix systems FindProcess
|
||||
// doesn't actually return an error if it can't find the process.
|
||||
conn, err := net.Dial(
|
||||
c.config.Reattach.Addr.Network(),
|
||||
c.config.Reattach.Addr.String())
|
||||
if err != nil {
|
||||
p.Kill()
|
||||
return nil, ErrProcessNotFound
|
||||
}
|
||||
conn.Close()
|
||||
|
||||
// Create a context for when we kill
|
||||
c.doneCtx, c.ctxCancel = context.WithCancel(context.Background())
|
||||
|
||||
c.clientWaitGroup.Add(1)
|
||||
// Goroutine to mark exit status
|
||||
go func(pid int) {
|
||||
defer c.clientWaitGroup.Done()
|
||||
|
||||
// ensure the context is cancelled when we're done
|
||||
defer c.ctxCancel()
|
||||
|
||||
// Wait for the process to die
|
||||
pidWait(pid)
|
||||
|
||||
// Log so we can see it
|
||||
c.logger.Debug("reattached plugin process exited")
|
||||
|
||||
// Mark it
|
||||
c.l.Lock()
|
||||
defer c.l.Unlock()
|
||||
c.exited = true
|
||||
}(p.Pid)
|
||||
|
||||
// Set the address and process
|
||||
c.address = c.config.Reattach.Addr
|
||||
c.process = p
|
||||
c.protocol = c.config.Reattach.Protocol
|
||||
if c.protocol == "" {
|
||||
// Default the protocol to net/rpc for backwards compatibility
|
||||
c.protocol = ProtocolNetRPC
|
||||
}
|
||||
|
||||
return c.address, nil
|
||||
}
|
||||
|
||||
// checkProtoVersion returns the negotiated version and PluginSet.
|
||||
// This returns an error if the server returned an incompatible protocol
|
||||
// version, or an invalid handshake response.
|
||||
func (c *Client) checkProtoVersion(protoVersion string) (int, PluginSet, error) {
|
||||
serverVersion, err := strconv.Atoi(protoVersion)
|
||||
if err != nil {
|
||||
return 0, nil, fmt.Errorf("Error parsing protocol version %q: %s", protoVersion, err)
|
||||
}
|
||||
|
||||
// record these for the error message
|
||||
var clientVersions []int
|
||||
|
||||
// all versions, including the legacy ProtocolVersion have been added to
|
||||
// the versions set
|
||||
for version, plugins := range c.config.VersionedPlugins {
|
||||
clientVersions = append(clientVersions, version)
|
||||
|
||||
if serverVersion != version {
|
||||
continue
|
||||
}
|
||||
return version, plugins, nil
|
||||
}
|
||||
|
||||
return 0, nil, fmt.Errorf("Incompatible API version with plugin. "+
|
||||
"Plugin version: %d, Client versions: %d", serverVersion, clientVersions)
|
||||
}
|
||||
|
||||
// ReattachConfig returns the information that must be provided to NewClient
|
||||
// to reattach to the plugin process that this client started. This is
|
||||
// useful for plugins that detach from their parent process.
|
||||
@@ -749,44 +932,64 @@ func (c *Client) dialer(_ string, timeout time.Duration) (net.Conn, error) {
|
||||
return conn, nil
|
||||
}
|
||||
|
||||
var stdErrBufferSize = 64 * 1024
|
||||
|
||||
func (c *Client) logStderr(r io.Reader) {
|
||||
bufR := bufio.NewReader(r)
|
||||
defer c.clientWaitGroup.Done()
|
||||
l := c.logger.Named(filepath.Base(c.config.Cmd.Path))
|
||||
|
||||
reader := bufio.NewReaderSize(r, stdErrBufferSize)
|
||||
// continuation indicates the previous line was a prefix
|
||||
continuation := false
|
||||
|
||||
for {
|
||||
line, err := bufR.ReadString('\n')
|
||||
if line != "" {
|
||||
c.config.Stderr.Write([]byte(line))
|
||||
line = strings.TrimRightFunc(line, unicode.IsSpace)
|
||||
line, isPrefix, err := reader.ReadLine()
|
||||
switch {
|
||||
case err == io.EOF:
|
||||
return
|
||||
case err != nil:
|
||||
l.Error("reading plugin stderr", "error", err)
|
||||
return
|
||||
}
|
||||
|
||||
entry, err := parseJSON(line)
|
||||
// If output is not JSON format, print directly to Debug
|
||||
if err != nil {
|
||||
l.Debug(line)
|
||||
} else {
|
||||
out := flattenKVPairs(entry.KVPairs)
|
||||
c.config.Stderr.Write(line)
|
||||
|
||||
out = append(out, "timestamp", entry.Timestamp.Format(hclog.TimeFormat))
|
||||
switch hclog.LevelFromString(entry.Level) {
|
||||
case hclog.Trace:
|
||||
l.Trace(entry.Message, out...)
|
||||
case hclog.Debug:
|
||||
l.Debug(entry.Message, out...)
|
||||
case hclog.Info:
|
||||
l.Info(entry.Message, out...)
|
||||
case hclog.Warn:
|
||||
l.Warn(entry.Message, out...)
|
||||
case hclog.Error:
|
||||
l.Error(entry.Message, out...)
|
||||
}
|
||||
// The line was longer than our max token size, so it's likely
|
||||
// incomplete and won't unmarshal.
|
||||
if isPrefix || continuation {
|
||||
l.Debug(string(line))
|
||||
|
||||
// if we're finishing a continued line, add the newline back in
|
||||
if !isPrefix {
|
||||
c.config.Stderr.Write([]byte{'\n'})
|
||||
}
|
||||
|
||||
continuation = isPrefix
|
||||
continue
|
||||
}
|
||||
|
||||
c.config.Stderr.Write([]byte{'\n'})
|
||||
|
||||
entry, err := parseJSON(line)
|
||||
// If output is not JSON format, print directly to Debug
|
||||
if err != nil {
|
||||
l.Debug(string(line))
|
||||
} else {
|
||||
out := flattenKVPairs(entry.KVPairs)
|
||||
|
||||
out = append(out, "timestamp", entry.Timestamp.Format(hclog.TimeFormat))
|
||||
switch hclog.LevelFromString(entry.Level) {
|
||||
case hclog.Trace:
|
||||
l.Trace(entry.Message, out...)
|
||||
case hclog.Debug:
|
||||
l.Debug(entry.Message, out...)
|
||||
case hclog.Info:
|
||||
l.Info(entry.Message, out...)
|
||||
case hclog.Warn:
|
||||
l.Warn(entry.Message, out...)
|
||||
case hclog.Error:
|
||||
l.Error(entry.Message, out...)
|
||||
}
|
||||
}
|
||||
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
// Flag that we've completed logging for others
|
||||
close(c.doneLogging)
|
||||
}
|
||||
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
module github.com/hashicorp/go-plugin
|
||||
|
||||
require (
|
||||
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b // indirect
|
||||
github.com/golang/protobuf v1.2.0
|
||||
github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd
|
||||
github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb
|
||||
github.com/mitchellh/go-testing-interface v0.0.0-20171004221916-a61a99592b77
|
||||
github.com/oklog/run v1.0.0
|
||||
github.com/stretchr/testify v1.3.0 // indirect
|
||||
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d
|
||||
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 // indirect
|
||||
golang.org/x/sys v0.0.0-20190129075346-302c3dd5f1cc // indirect
|
||||
golang.org/x/text v0.3.0 // indirect
|
||||
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8 // indirect
|
||||
google.golang.org/grpc v1.14.0
|
||||
)
|
||||
+31
@@ -0,0 +1,31 @@
|
||||
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=
|
||||
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
|
||||
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
|
||||
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||
github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd h1:rNuUHR+CvK1IS89MMtcF0EpcVMZtjKfPRp4MEmt/aTs=
|
||||
github.com/hashicorp/go-hclog v0.0.0-20180709165350-ff2cf002a8dd/go.mod h1:9bjs9uLqI8l75knNv3lV1kA55veR+WUPSiKIWcQHudI=
|
||||
github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb h1:b5rjCoWHc7eqmAS4/qyk21ZsHyb6Mxv/jykxvNTkU4M=
|
||||
github.com/hashicorp/yamux v0.0.0-20180604194846-3520598351bb/go.mod h1:+NfK9FKeTrX5uv1uIXGdwYDTeHna2qgaIlx54MXqjAM=
|
||||
github.com/mitchellh/go-testing-interface v0.0.0-20171004221916-a61a99592b77 h1:7GoSOOW2jpsfkntVKaS2rAr1TJqfcxotyaUcuxoZSzg=
|
||||
github.com/mitchellh/go-testing-interface v0.0.0-20171004221916-a61a99592b77/go.mod h1:kRemZodwjscx+RGhAo8eIhFbs2+BFgRtFPeD/KE+zxI=
|
||||
github.com/oklog/run v1.0.0 h1:Ru7dDtJNOyC66gQ5dQmaCa0qIsAUFY3sFpK1Xk8igrw=
|
||||
github.com/oklog/run v1.0.0/go.mod h1:dlhp/R75TPv97u0XWUtDeV/lRKWPKSdTuV0TZvrmrQA=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.3.0 h1:TivCn/peBQ7UY8ooIcPgZFpTNSz0Q2U6UrFlUfqbe0Q=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d h1:g9qWBGx4puODJTMVyoPrpoxPFgVGd+z1DZwjfRu4d0I=
|
||||
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
|
||||
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 h1:YUO/7uOKsKeq9UokNS62b8FYywz3ker1l1vDZRCRefw=
|
||||
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sys v0.0.0-20190129075346-302c3dd5f1cc h1:WiYx1rIFmx8c0mXAFtv5D/mHyKe1+jmuP7PViuwqwuQ=
|
||||
golang.org/x/sys v0.0.0-20190129075346-302c3dd5f1cc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8 h1:Nw54tB0rB7hY/N0NQvRW8DG4Yk3Q6T9cu9RcFQDu1tc=
|
||||
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
|
||||
google.golang.org/grpc v1.14.0 h1:ArxJuB1NWfPY6r9Gp9gqwplT0Ge7nqv9msgu03lHLmo=
|
||||
google.golang.org/grpc v1.14.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmEdcZw=
|
||||
+20
-18
@@ -11,6 +11,8 @@ import (
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/hashicorp/go-plugin/internal/plugin"
|
||||
|
||||
"github.com/oklog/run"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials"
|
||||
@@ -19,14 +21,14 @@ import (
|
||||
// streamer interface is used in the broker to send/receive connection
|
||||
// information.
|
||||
type streamer interface {
|
||||
Send(*ConnInfo) error
|
||||
Recv() (*ConnInfo, error)
|
||||
Send(*plugin.ConnInfo) error
|
||||
Recv() (*plugin.ConnInfo, error)
|
||||
Close()
|
||||
}
|
||||
|
||||
// sendErr is used to pass errors back during a send.
|
||||
type sendErr struct {
|
||||
i *ConnInfo
|
||||
i *plugin.ConnInfo
|
||||
ch chan error
|
||||
}
|
||||
|
||||
@@ -38,7 +40,7 @@ type gRPCBrokerServer struct {
|
||||
send chan *sendErr
|
||||
|
||||
// recv is used to receive connection info from the gRPC stream.
|
||||
recv chan *ConnInfo
|
||||
recv chan *plugin.ConnInfo
|
||||
|
||||
// quit closes down the stream.
|
||||
quit chan struct{}
|
||||
@@ -50,7 +52,7 @@ type gRPCBrokerServer struct {
|
||||
func newGRPCBrokerServer() *gRPCBrokerServer {
|
||||
return &gRPCBrokerServer{
|
||||
send: make(chan *sendErr),
|
||||
recv: make(chan *ConnInfo),
|
||||
recv: make(chan *plugin.ConnInfo),
|
||||
quit: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
@@ -58,7 +60,7 @@ func newGRPCBrokerServer() *gRPCBrokerServer {
|
||||
// StartStream implements the GRPCBrokerServer interface and will block until
|
||||
// the quit channel is closed or the context reports Done. The stream will pass
|
||||
// connection information to/from the client.
|
||||
func (s *gRPCBrokerServer) StartStream(stream GRPCBroker_StartStreamServer) error {
|
||||
func (s *gRPCBrokerServer) StartStream(stream plugin.GRPCBroker_StartStreamServer) error {
|
||||
doneCh := stream.Context().Done()
|
||||
defer s.Close()
|
||||
|
||||
@@ -97,7 +99,7 @@ func (s *gRPCBrokerServer) StartStream(stream GRPCBroker_StartStreamServer) erro
|
||||
|
||||
// Send is used by the GRPCBroker to pass connection information into the stream
|
||||
// to the client.
|
||||
func (s *gRPCBrokerServer) Send(i *ConnInfo) error {
|
||||
func (s *gRPCBrokerServer) Send(i *plugin.ConnInfo) error {
|
||||
ch := make(chan error)
|
||||
defer close(ch)
|
||||
|
||||
@@ -115,7 +117,7 @@ func (s *gRPCBrokerServer) Send(i *ConnInfo) error {
|
||||
|
||||
// Recv is used by the GRPCBroker to pass connection information that has been
|
||||
// sent from the client from the stream to the broker.
|
||||
func (s *gRPCBrokerServer) Recv() (*ConnInfo, error) {
|
||||
func (s *gRPCBrokerServer) Recv() (*plugin.ConnInfo, error) {
|
||||
select {
|
||||
case <-s.quit:
|
||||
return nil, errors.New("broker closed")
|
||||
@@ -136,13 +138,13 @@ func (s *gRPCBrokerServer) Close() {
|
||||
// streamer interfaces.
|
||||
type gRPCBrokerClientImpl struct {
|
||||
// client is the underlying GRPC client used to make calls to the server.
|
||||
client GRPCBrokerClient
|
||||
client plugin.GRPCBrokerClient
|
||||
|
||||
// send is used to send connection info to the gRPC stream.
|
||||
send chan *sendErr
|
||||
|
||||
// recv is used to receive connection info from the gRPC stream.
|
||||
recv chan *ConnInfo
|
||||
recv chan *plugin.ConnInfo
|
||||
|
||||
// quit closes down the stream.
|
||||
quit chan struct{}
|
||||
@@ -153,9 +155,9 @@ type gRPCBrokerClientImpl struct {
|
||||
|
||||
func newGRPCBrokerClient(conn *grpc.ClientConn) *gRPCBrokerClientImpl {
|
||||
return &gRPCBrokerClientImpl{
|
||||
client: NewGRPCBrokerClient(conn),
|
||||
client: plugin.NewGRPCBrokerClient(conn),
|
||||
send: make(chan *sendErr),
|
||||
recv: make(chan *ConnInfo),
|
||||
recv: make(chan *plugin.ConnInfo),
|
||||
quit: make(chan struct{}),
|
||||
}
|
||||
}
|
||||
@@ -207,7 +209,7 @@ func (s *gRPCBrokerClientImpl) StartStream() error {
|
||||
|
||||
// Send is used by the GRPCBroker to pass connection information into the stream
|
||||
// to the plugin.
|
||||
func (s *gRPCBrokerClientImpl) Send(i *ConnInfo) error {
|
||||
func (s *gRPCBrokerClientImpl) Send(i *plugin.ConnInfo) error {
|
||||
ch := make(chan error)
|
||||
defer close(ch)
|
||||
|
||||
@@ -225,7 +227,7 @@ func (s *gRPCBrokerClientImpl) Send(i *ConnInfo) error {
|
||||
|
||||
// Recv is used by the GRPCBroker to pass connection information that has been
|
||||
// sent from the plugin to the broker.
|
||||
func (s *gRPCBrokerClientImpl) Recv() (*ConnInfo, error) {
|
||||
func (s *gRPCBrokerClientImpl) Recv() (*plugin.ConnInfo, error) {
|
||||
select {
|
||||
case <-s.quit:
|
||||
return nil, errors.New("broker closed")
|
||||
@@ -266,7 +268,7 @@ type GRPCBroker struct {
|
||||
}
|
||||
|
||||
type gRPCBrokerPending struct {
|
||||
ch chan *ConnInfo
|
||||
ch chan *plugin.ConnInfo
|
||||
doneCh chan struct{}
|
||||
}
|
||||
|
||||
@@ -288,7 +290,7 @@ func (b *GRPCBroker) Accept(id uint32) (net.Listener, error) {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
err = b.streamer.Send(&ConnInfo{
|
||||
err = b.streamer.Send(&plugin.ConnInfo{
|
||||
ServiceId: id,
|
||||
Network: listener.Addr().Network(),
|
||||
Address: listener.Addr().String(),
|
||||
@@ -363,7 +365,7 @@ func (b *GRPCBroker) Close() error {
|
||||
|
||||
// Dial opens a connection by ID.
|
||||
func (b *GRPCBroker) Dial(id uint32) (conn *grpc.ClientConn, err error) {
|
||||
var c *ConnInfo
|
||||
var c *plugin.ConnInfo
|
||||
|
||||
// Open the stream
|
||||
p := b.getStream(id)
|
||||
@@ -433,7 +435,7 @@ func (m *GRPCBroker) getStream(id uint32) *gRPCBrokerPending {
|
||||
}
|
||||
|
||||
m.streams[id] = &gRPCBrokerPending{
|
||||
ch: make(chan *ConnInfo, 1),
|
||||
ch: make(chan *plugin.ConnInfo, 1),
|
||||
doneCh: make(chan struct{}),
|
||||
}
|
||||
return m.streams[id]
|
||||
|
||||
+14
-10
@@ -6,6 +6,7 @@ import (
|
||||
"net"
|
||||
"time"
|
||||
|
||||
"github.com/hashicorp/go-plugin/internal/plugin"
|
||||
"golang.org/x/net/context"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials"
|
||||
@@ -16,12 +17,9 @@ func dialGRPCConn(tls *tls.Config, dialer func(string, time.Duration) (net.Conn,
|
||||
// Build dialing options.
|
||||
opts := make([]grpc.DialOption, 0, 5)
|
||||
|
||||
// We use a custom dialer so that we can connect over unix domain sockets
|
||||
// We use a custom dialer so that we can connect over unix domain sockets.
|
||||
opts = append(opts, grpc.WithDialer(dialer))
|
||||
|
||||
// go-plugin expects to block the connection
|
||||
opts = append(opts, grpc.WithBlock())
|
||||
|
||||
// Fail right away
|
||||
opts = append(opts, grpc.FailOnNonTempDialError(true))
|
||||
|
||||
@@ -58,12 +56,15 @@ func newGRPCClient(doneCtx context.Context, c *Client) (*GRPCClient, error) {
|
||||
go broker.Run()
|
||||
go brokerGRPCClient.StartStream()
|
||||
|
||||
return &GRPCClient{
|
||||
Conn: conn,
|
||||
Plugins: c.config.Plugins,
|
||||
doneCtx: doneCtx,
|
||||
broker: broker,
|
||||
}, nil
|
||||
cl := &GRPCClient{
|
||||
Conn: conn,
|
||||
Plugins: c.config.Plugins,
|
||||
doneCtx: doneCtx,
|
||||
broker: broker,
|
||||
controller: plugin.NewGRPCControllerClient(conn),
|
||||
}
|
||||
|
||||
return cl, nil
|
||||
}
|
||||
|
||||
// GRPCClient connects to a GRPCServer over gRPC to dispense plugin types.
|
||||
@@ -73,11 +74,14 @@ type GRPCClient struct {
|
||||
|
||||
doneCtx context.Context
|
||||
broker *GRPCBroker
|
||||
|
||||
controller plugin.GRPCControllerClient
|
||||
}
|
||||
|
||||
// ClientProtocol impl.
|
||||
func (c *GRPCClient) Close() error {
|
||||
c.broker.Close()
|
||||
c.controller.Shutdown(c.doneCtx, &plugin.Empty{})
|
||||
return c.Conn.Close()
|
||||
}
|
||||
|
||||
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
package plugin
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hashicorp/go-plugin/internal/plugin"
|
||||
)
|
||||
|
||||
// GRPCControllerServer handles shutdown calls to terminate the server when the
|
||||
// plugin client is closed.
|
||||
type grpcControllerServer struct {
|
||||
server *GRPCServer
|
||||
}
|
||||
|
||||
// Shutdown stops the grpc server. It first will attempt a graceful stop, then a
|
||||
// full stop on the server.
|
||||
func (s *grpcControllerServer) Shutdown(ctx context.Context, _ *plugin.Empty) (*plugin.Empty, error) {
|
||||
resp := &plugin.Empty{}
|
||||
|
||||
// TODO: figure out why GracefullStop doesn't work.
|
||||
s.server.Stop()
|
||||
return resp, nil
|
||||
}
|
||||
+17
-7
@@ -8,6 +8,8 @@ import (
|
||||
"io"
|
||||
"net"
|
||||
|
||||
hclog "github.com/hashicorp/go-hclog"
|
||||
"github.com/hashicorp/go-plugin/internal/plugin"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials"
|
||||
"google.golang.org/grpc/health"
|
||||
@@ -52,6 +54,8 @@ type GRPCServer struct {
|
||||
config GRPCServerConfig
|
||||
server *grpc.Server
|
||||
broker *GRPCBroker
|
||||
|
||||
logger hclog.Logger
|
||||
}
|
||||
|
||||
// ServerProtocol impl.
|
||||
@@ -71,10 +75,16 @@ func (s *GRPCServer) Init() error {
|
||||
|
||||
// Register the broker service
|
||||
brokerServer := newGRPCBrokerServer()
|
||||
RegisterGRPCBrokerServer(s.server, brokerServer)
|
||||
plugin.RegisterGRPCBrokerServer(s.server, brokerServer)
|
||||
s.broker = newGRPCBroker(brokerServer, s.TLS)
|
||||
go s.broker.Run()
|
||||
|
||||
// Register the controller
|
||||
controllerServer := &grpcControllerServer{
|
||||
server: s,
|
||||
}
|
||||
plugin.RegisterGRPCControllerServer(s.server, controllerServer)
|
||||
|
||||
// Register all our plugins onto the gRPC server.
|
||||
for k, raw := range s.Plugins {
|
||||
p, ok := raw.(GRPCPlugin)
|
||||
@@ -83,7 +93,7 @@ func (s *GRPCServer) Init() error {
|
||||
}
|
||||
|
||||
if err := p.GRPCServer(s.broker, s.server); err != nil {
|
||||
return fmt.Errorf("error registring %q: %s", k, err)
|
||||
return fmt.Errorf("error registering %q: %s", k, err)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -117,11 +127,11 @@ func (s *GRPCServer) Config() string {
|
||||
}
|
||||
|
||||
func (s *GRPCServer) Serve(lis net.Listener) {
|
||||
// Start serving in a goroutine
|
||||
go s.server.Serve(lis)
|
||||
|
||||
// Wait until graceful completion
|
||||
<-s.DoneCh
|
||||
defer close(s.DoneCh)
|
||||
err := s.server.Serve(lis)
|
||||
if err != nil {
|
||||
s.logger.Error("grpc server", "error", err)
|
||||
}
|
||||
}
|
||||
|
||||
// GRPCServerConfig is the extra configuration passed along for consumers
|
||||
|
||||
+3
@@ -0,0 +1,3 @@
|
||||
//go:generate protoc -I ./ ./grpc_broker.proto ./grpc_controller.proto --go_out=plugins=grpc:.
|
||||
|
||||
package plugin
|
||||
Generated
Vendored
+55
-42
@@ -1,24 +1,14 @@
|
||||
// Code generated by protoc-gen-go. DO NOT EDIT.
|
||||
// source: grpc_broker.proto
|
||||
|
||||
/*
|
||||
Package plugin is a generated protocol buffer package.
|
||||
|
||||
It is generated from these files:
|
||||
grpc_broker.proto
|
||||
|
||||
It has these top-level messages:
|
||||
ConnInfo
|
||||
*/
|
||||
package plugin
|
||||
|
||||
import proto "github.com/golang/protobuf/proto"
|
||||
import fmt "fmt"
|
||||
import math "math"
|
||||
|
||||
import (
|
||||
fmt "fmt"
|
||||
proto "github.com/golang/protobuf/proto"
|
||||
context "golang.org/x/net/context"
|
||||
grpc "google.golang.org/grpc"
|
||||
math "math"
|
||||
)
|
||||
|
||||
// Reference imports to suppress errors if they are not otherwise used.
|
||||
@@ -33,15 +23,38 @@ var _ = math.Inf
|
||||
const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
|
||||
|
||||
type ConnInfo struct {
|
||||
ServiceId uint32 `protobuf:"varint,1,opt,name=service_id,json=serviceId" json:"service_id,omitempty"`
|
||||
Network string `protobuf:"bytes,2,opt,name=network" json:"network,omitempty"`
|
||||
Address string `protobuf:"bytes,3,opt,name=address" json:"address,omitempty"`
|
||||
ServiceId uint32 `protobuf:"varint,1,opt,name=service_id,json=serviceId,proto3" json:"service_id,omitempty"`
|
||||
Network string `protobuf:"bytes,2,opt,name=network,proto3" json:"network,omitempty"`
|
||||
Address string `protobuf:"bytes,3,opt,name=address,proto3" json:"address,omitempty"`
|
||||
XXX_NoUnkeyedLiteral struct{} `json:"-"`
|
||||
XXX_unrecognized []byte `json:"-"`
|
||||
XXX_sizecache int32 `json:"-"`
|
||||
}
|
||||
|
||||
func (m *ConnInfo) Reset() { *m = ConnInfo{} }
|
||||
func (m *ConnInfo) String() string { return proto.CompactTextString(m) }
|
||||
func (*ConnInfo) ProtoMessage() {}
|
||||
func (*ConnInfo) Descriptor() ([]byte, []int) { return fileDescriptor0, []int{0} }
|
||||
func (m *ConnInfo) Reset() { *m = ConnInfo{} }
|
||||
func (m *ConnInfo) String() string { return proto.CompactTextString(m) }
|
||||
func (*ConnInfo) ProtoMessage() {}
|
||||
func (*ConnInfo) Descriptor() ([]byte, []int) {
|
||||
return fileDescriptor_802e9beed3ec3b28, []int{0}
|
||||
}
|
||||
|
||||
func (m *ConnInfo) XXX_Unmarshal(b []byte) error {
|
||||
return xxx_messageInfo_ConnInfo.Unmarshal(m, b)
|
||||
}
|
||||
func (m *ConnInfo) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
|
||||
return xxx_messageInfo_ConnInfo.Marshal(b, m, deterministic)
|
||||
}
|
||||
func (m *ConnInfo) XXX_Merge(src proto.Message) {
|
||||
xxx_messageInfo_ConnInfo.Merge(m, src)
|
||||
}
|
||||
func (m *ConnInfo) XXX_Size() int {
|
||||
return xxx_messageInfo_ConnInfo.Size(m)
|
||||
}
|
||||
func (m *ConnInfo) XXX_DiscardUnknown() {
|
||||
xxx_messageInfo_ConnInfo.DiscardUnknown(m)
|
||||
}
|
||||
|
||||
var xxx_messageInfo_ConnInfo proto.InternalMessageInfo
|
||||
|
||||
func (m *ConnInfo) GetServiceId() uint32 {
|
||||
if m != nil {
|
||||
@@ -68,6 +81,23 @@ func init() {
|
||||
proto.RegisterType((*ConnInfo)(nil), "plugin.ConnInfo")
|
||||
}
|
||||
|
||||
func init() { proto.RegisterFile("grpc_broker.proto", fileDescriptor_802e9beed3ec3b28) }
|
||||
|
||||
var fileDescriptor_802e9beed3ec3b28 = []byte{
|
||||
// 175 bytes of a gzipped FileDescriptorProto
|
||||
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x12, 0x4c, 0x2f, 0x2a, 0x48,
|
||||
0x8e, 0x4f, 0x2a, 0xca, 0xcf, 0x4e, 0x2d, 0xd2, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x2b,
|
||||
0xc8, 0x29, 0x4d, 0xcf, 0xcc, 0x53, 0x8a, 0xe5, 0xe2, 0x70, 0xce, 0xcf, 0xcb, 0xf3, 0xcc, 0x4b,
|
||||
0xcb, 0x17, 0x92, 0xe5, 0xe2, 0x2a, 0x4e, 0x2d, 0x2a, 0xcb, 0x4c, 0x4e, 0x8d, 0xcf, 0x4c, 0x91,
|
||||
0x60, 0x54, 0x60, 0xd4, 0xe0, 0x0d, 0xe2, 0x84, 0x8a, 0x78, 0xa6, 0x08, 0x49, 0x70, 0xb1, 0xe7,
|
||||
0xa5, 0x96, 0x94, 0xe7, 0x17, 0x65, 0x4b, 0x30, 0x29, 0x30, 0x6a, 0x70, 0x06, 0xc1, 0xb8, 0x20,
|
||||
0x99, 0xc4, 0x94, 0x94, 0xa2, 0xd4, 0xe2, 0x62, 0x09, 0x66, 0x88, 0x0c, 0x94, 0x6b, 0xe4, 0xcc,
|
||||
0xc5, 0xe5, 0x1e, 0x14, 0xe0, 0xec, 0x04, 0xb6, 0x5a, 0xc8, 0x94, 0x8b, 0x3b, 0xb8, 0x24, 0xb1,
|
||||
0xa8, 0x24, 0xb8, 0xa4, 0x28, 0x35, 0x31, 0x57, 0x48, 0x40, 0x0f, 0xe2, 0x08, 0x3d, 0x98, 0x0b,
|
||||
0xa4, 0x30, 0x44, 0x34, 0x18, 0x0d, 0x18, 0x9d, 0x38, 0xa2, 0xa0, 0xae, 0x4d, 0x62, 0x03, 0x3b,
|
||||
0xde, 0x18, 0x10, 0x00, 0x00, 0xff, 0xff, 0x10, 0x15, 0x39, 0x47, 0xd1, 0x00, 0x00, 0x00,
|
||||
}
|
||||
|
||||
// Reference imports to suppress errors if they are not otherwise used.
|
||||
var _ context.Context
|
||||
var _ grpc.ClientConn
|
||||
@@ -76,8 +106,9 @@ var _ grpc.ClientConn
|
||||
// is compatible with the grpc package it is being compiled against.
|
||||
const _ = grpc.SupportPackageIsVersion4
|
||||
|
||||
// Client API for GRPCBroker service
|
||||
|
||||
// GRPCBrokerClient is the client API for GRPCBroker service.
|
||||
//
|
||||
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.
|
||||
type GRPCBrokerClient interface {
|
||||
StartStream(ctx context.Context, opts ...grpc.CallOption) (GRPCBroker_StartStreamClient, error)
|
||||
}
|
||||
@@ -91,7 +122,7 @@ func NewGRPCBrokerClient(cc *grpc.ClientConn) GRPCBrokerClient {
|
||||
}
|
||||
|
||||
func (c *gRPCBrokerClient) StartStream(ctx context.Context, opts ...grpc.CallOption) (GRPCBroker_StartStreamClient, error) {
|
||||
stream, err := grpc.NewClientStream(ctx, &_GRPCBroker_serviceDesc.Streams[0], c.cc, "/plugin.GRPCBroker/StartStream", opts...)
|
||||
stream, err := c.cc.NewStream(ctx, &_GRPCBroker_serviceDesc.Streams[0], "/plugin.GRPCBroker/StartStream", opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -121,8 +152,7 @@ func (x *gRPCBrokerStartStreamClient) Recv() (*ConnInfo, error) {
|
||||
return m, nil
|
||||
}
|
||||
|
||||
// Server API for GRPCBroker service
|
||||
|
||||
// GRPCBrokerServer is the server API for GRPCBroker service.
|
||||
type GRPCBrokerServer interface {
|
||||
StartStream(GRPCBroker_StartStreamServer) error
|
||||
}
|
||||
@@ -171,20 +201,3 @@ var _GRPCBroker_serviceDesc = grpc.ServiceDesc{
|
||||
},
|
||||
Metadata: "grpc_broker.proto",
|
||||
}
|
||||
|
||||
func init() { proto.RegisterFile("grpc_broker.proto", fileDescriptor0) }
|
||||
|
||||
var fileDescriptor0 = []byte{
|
||||
// 170 bytes of a gzipped FileDescriptorProto
|
||||
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x12, 0x4c, 0x2f, 0x2a, 0x48,
|
||||
0x8e, 0x4f, 0x2a, 0xca, 0xcf, 0x4e, 0x2d, 0xd2, 0x2b, 0x28, 0xca, 0x2f, 0xc9, 0x17, 0x62, 0x2b,
|
||||
0xc8, 0x29, 0x4d, 0xcf, 0xcc, 0x53, 0x8a, 0xe5, 0xe2, 0x70, 0xce, 0xcf, 0xcb, 0xf3, 0xcc, 0x4b,
|
||||
0xcb, 0x17, 0x92, 0xe5, 0xe2, 0x2a, 0x4e, 0x2d, 0x2a, 0xcb, 0x4c, 0x4e, 0x8d, 0xcf, 0x4c, 0x91,
|
||||
0x60, 0x54, 0x60, 0xd4, 0xe0, 0x0d, 0xe2, 0x84, 0x8a, 0x78, 0xa6, 0x08, 0x49, 0x70, 0xb1, 0xe7,
|
||||
0xa5, 0x96, 0x94, 0xe7, 0x17, 0x65, 0x4b, 0x30, 0x29, 0x30, 0x6a, 0x70, 0x06, 0xc1, 0xb8, 0x20,
|
||||
0x99, 0xc4, 0x94, 0x94, 0xa2, 0xd4, 0xe2, 0x62, 0x09, 0x66, 0x88, 0x0c, 0x94, 0x6b, 0xe4, 0xcc,
|
||||
0xc5, 0xe5, 0x1e, 0x14, 0xe0, 0xec, 0x04, 0xb6, 0x5a, 0xc8, 0x94, 0x8b, 0x3b, 0xb8, 0x24, 0xb1,
|
||||
0xa8, 0x24, 0xb8, 0xa4, 0x28, 0x35, 0x31, 0x57, 0x48, 0x40, 0x0f, 0xe2, 0x08, 0x3d, 0x98, 0x0b,
|
||||
0xa4, 0x30, 0x44, 0x34, 0x18, 0x0d, 0x18, 0x93, 0xd8, 0xc0, 0x4e, 0x36, 0x06, 0x04, 0x00, 0x00,
|
||||
0xff, 0xff, 0x7b, 0x5d, 0xfb, 0xe1, 0xc7, 0x00, 0x00, 0x00,
|
||||
}
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
syntax = "proto3";
|
||||
package plugin;
|
||||
option go_package = "plugin";
|
||||
|
||||
message ConnInfo {
|
||||
uint32 service_id = 1;
|
||||
string network = 2;
|
||||
string address = 3;
|
||||
}
|
||||
|
||||
service GRPCBroker {
|
||||
rpc StartStream(stream ConnInfo) returns (stream ConnInfo);
|
||||
}
|
||||
|
||||
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
// Code generated by protoc-gen-go. DO NOT EDIT.
|
||||
// source: grpc_controller.proto
|
||||
|
||||
package plugin
|
||||
|
||||
import (
|
||||
fmt "fmt"
|
||||
proto "github.com/golang/protobuf/proto"
|
||||
context "golang.org/x/net/context"
|
||||
grpc "google.golang.org/grpc"
|
||||
math "math"
|
||||
)
|
||||
|
||||
// Reference imports to suppress errors if they are not otherwise used.
|
||||
var _ = proto.Marshal
|
||||
var _ = fmt.Errorf
|
||||
var _ = math.Inf
|
||||
|
||||
// This is a compile-time assertion to ensure that this generated file
|
||||
// is compatible with the proto package it is being compiled against.
|
||||
// A compilation error at this line likely means your copy of the
|
||||
// proto package needs to be updated.
|
||||
const _ = proto.ProtoPackageIsVersion2 // please upgrade the proto package
|
||||
|
||||
type Empty struct {
|
||||
XXX_NoUnkeyedLiteral struct{} `json:"-"`
|
||||
XXX_unrecognized []byte `json:"-"`
|
||||
XXX_sizecache int32 `json:"-"`
|
||||
}
|
||||
|
||||
func (m *Empty) Reset() { *m = Empty{} }
|
||||
func (m *Empty) String() string { return proto.CompactTextString(m) }
|
||||
func (*Empty) ProtoMessage() {}
|
||||
func (*Empty) Descriptor() ([]byte, []int) {
|
||||
return fileDescriptor_23c2c7e42feab570, []int{0}
|
||||
}
|
||||
|
||||
func (m *Empty) XXX_Unmarshal(b []byte) error {
|
||||
return xxx_messageInfo_Empty.Unmarshal(m, b)
|
||||
}
|
||||
func (m *Empty) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
|
||||
return xxx_messageInfo_Empty.Marshal(b, m, deterministic)
|
||||
}
|
||||
func (m *Empty) XXX_Merge(src proto.Message) {
|
||||
xxx_messageInfo_Empty.Merge(m, src)
|
||||
}
|
||||
func (m *Empty) XXX_Size() int {
|
||||
return xxx_messageInfo_Empty.Size(m)
|
||||
}
|
||||
func (m *Empty) XXX_DiscardUnknown() {
|
||||
xxx_messageInfo_Empty.DiscardUnknown(m)
|
||||
}
|
||||
|
||||
var xxx_messageInfo_Empty proto.InternalMessageInfo
|
||||
|
||||
func init() {
|
||||
proto.RegisterType((*Empty)(nil), "plugin.Empty")
|
||||
}
|
||||
|
||||
func init() { proto.RegisterFile("grpc_controller.proto", fileDescriptor_23c2c7e42feab570) }
|
||||
|
||||
var fileDescriptor_23c2c7e42feab570 = []byte{
|
||||
// 108 bytes of a gzipped FileDescriptorProto
|
||||
0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xe2, 0x12, 0x4d, 0x2f, 0x2a, 0x48,
|
||||
0x8e, 0x4f, 0xce, 0xcf, 0x2b, 0x29, 0xca, 0xcf, 0xc9, 0x49, 0x2d, 0xd2, 0x2b, 0x28, 0xca, 0x2f,
|
||||
0xc9, 0x17, 0x62, 0x2b, 0xc8, 0x29, 0x4d, 0xcf, 0xcc, 0x53, 0x62, 0xe7, 0x62, 0x75, 0xcd, 0x2d,
|
||||
0x28, 0xa9, 0x34, 0xb2, 0xe2, 0xe2, 0x73, 0x0f, 0x0a, 0x70, 0x76, 0x86, 0x2b, 0x14, 0xd2, 0xe0,
|
||||
0xe2, 0x08, 0xce, 0x28, 0x2d, 0x49, 0xc9, 0x2f, 0xcf, 0x13, 0xe2, 0xd5, 0x83, 0xa8, 0xd7, 0x03,
|
||||
0x2b, 0x96, 0x42, 0xe5, 0x3a, 0x71, 0x44, 0x41, 0x8d, 0x4b, 0x62, 0x03, 0x9b, 0x6e, 0x0c, 0x08,
|
||||
0x00, 0x00, 0xff, 0xff, 0xab, 0x7c, 0x27, 0xe5, 0x76, 0x00, 0x00, 0x00,
|
||||
}
|
||||
|
||||
// Reference imports to suppress errors if they are not otherwise used.
|
||||
var _ context.Context
|
||||
var _ grpc.ClientConn
|
||||
|
||||
// This is a compile-time assertion to ensure that this generated file
|
||||
// is compatible with the grpc package it is being compiled against.
|
||||
const _ = grpc.SupportPackageIsVersion4
|
||||
|
||||
// GRPCControllerClient is the client API for GRPCController service.
|
||||
//
|
||||
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://godoc.org/google.golang.org/grpc#ClientConn.NewStream.
|
||||
type GRPCControllerClient interface {
|
||||
Shutdown(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*Empty, error)
|
||||
}
|
||||
|
||||
type gRPCControllerClient struct {
|
||||
cc *grpc.ClientConn
|
||||
}
|
||||
|
||||
func NewGRPCControllerClient(cc *grpc.ClientConn) GRPCControllerClient {
|
||||
return &gRPCControllerClient{cc}
|
||||
}
|
||||
|
||||
func (c *gRPCControllerClient) Shutdown(ctx context.Context, in *Empty, opts ...grpc.CallOption) (*Empty, error) {
|
||||
out := new(Empty)
|
||||
err := c.cc.Invoke(ctx, "/plugin.GRPCController/Shutdown", in, out, opts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// GRPCControllerServer is the server API for GRPCController service.
|
||||
type GRPCControllerServer interface {
|
||||
Shutdown(context.Context, *Empty) (*Empty, error)
|
||||
}
|
||||
|
||||
func RegisterGRPCControllerServer(s *grpc.Server, srv GRPCControllerServer) {
|
||||
s.RegisterService(&_GRPCController_serviceDesc, srv)
|
||||
}
|
||||
|
||||
func _GRPCController_Shutdown_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(Empty)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(GRPCControllerServer).Shutdown(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: "/plugin.GRPCController/Shutdown",
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(GRPCControllerServer).Shutdown(ctx, req.(*Empty))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
var _GRPCController_serviceDesc = grpc.ServiceDesc{
|
||||
ServiceName: "plugin.GRPCController",
|
||||
HandlerType: (*GRPCControllerServer)(nil),
|
||||
Methods: []grpc.MethodDesc{
|
||||
{
|
||||
MethodName: "Shutdown",
|
||||
Handler: _GRPCController_Shutdown_Handler,
|
||||
},
|
||||
},
|
||||
Streams: []grpc.StreamDesc{},
|
||||
Metadata: "grpc_controller.proto",
|
||||
}
|
||||
+11
@@ -0,0 +1,11 @@
|
||||
syntax = "proto3";
|
||||
package plugin;
|
||||
option go_package = "plugin";
|
||||
|
||||
message Empty {
|
||||
}
|
||||
|
||||
// The GRPCController is responsible for telling the plugin server to shutdown.
|
||||
service GRPCController {
|
||||
rpc Shutdown(Empty) returns (Empty);
|
||||
}
|
||||
+2
-2
@@ -32,11 +32,11 @@ func flattenKVPairs(kvs []*logEntryKV) []interface{} {
|
||||
}
|
||||
|
||||
// parseJSON handles parsing JSON output
|
||||
func parseJSON(input string) (*logEntry, error) {
|
||||
func parseJSON(input []byte) (*logEntry, error) {
|
||||
var raw map[string]interface{}
|
||||
entry := &logEntry{}
|
||||
|
||||
err := json.Unmarshal([]byte(input), &raw)
|
||||
err := json.Unmarshal(input, &raw)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
+73
@@ -0,0 +1,73 @@
|
||||
package plugin
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/x509"
|
||||
"crypto/x509/pkix"
|
||||
"encoding/pem"
|
||||
"math/big"
|
||||
"time"
|
||||
)
|
||||
|
||||
// generateCert generates a temporary certificate for plugin authentication. The
|
||||
// certificate and private key are returns in PEM format.
|
||||
func generateCert() (cert []byte, privateKey []byte, err error) {
|
||||
key, err := ecdsa.GenerateKey(elliptic.P521(), rand.Reader)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
serialNumberLimit := new(big.Int).Lsh(big.NewInt(1), 128)
|
||||
sn, err := rand.Int(rand.Reader, serialNumberLimit)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
host := "localhost"
|
||||
|
||||
template := &x509.Certificate{
|
||||
Subject: pkix.Name{
|
||||
CommonName: host,
|
||||
Organization: []string{"HashiCorp"},
|
||||
},
|
||||
DNSNames: []string{host},
|
||||
ExtKeyUsage: []x509.ExtKeyUsage{
|
||||
x509.ExtKeyUsageClientAuth,
|
||||
x509.ExtKeyUsageServerAuth,
|
||||
},
|
||||
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageKeyEncipherment | x509.KeyUsageKeyAgreement | x509.KeyUsageCertSign,
|
||||
BasicConstraintsValid: true,
|
||||
SerialNumber: sn,
|
||||
NotBefore: time.Now().Add(-30 * time.Second),
|
||||
NotAfter: time.Now().Add(262980 * time.Hour),
|
||||
IsCA: true,
|
||||
}
|
||||
|
||||
der, err := x509.CreateCertificate(rand.Reader, template, template, key.Public(), key)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
var certOut bytes.Buffer
|
||||
if err := pem.Encode(&certOut, &pem.Block{Type: "CERTIFICATE", Bytes: der}); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
keyBytes, err := x509.MarshalECPrivateKey(key)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
var keyOut bytes.Buffer
|
||||
if err := pem.Encode(&keyOut, &pem.Block{Type: "EC PRIVATE KEY", Bytes: keyBytes}); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
|
||||
cert = certOut.Bytes()
|
||||
privateKey = keyOut.Bytes()
|
||||
|
||||
return cert, privateKey, nil
|
||||
}
|
||||
+140
-25
@@ -2,6 +2,7 @@ package plugin
|
||||
|
||||
import (
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
@@ -11,7 +12,9 @@ import (
|
||||
"os"
|
||||
"os/signal"
|
||||
"runtime"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
|
||||
"github.com/hashicorp/go-hclog"
|
||||
@@ -36,6 +39,8 @@ type HandshakeConfig struct {
|
||||
// ProtocolVersion is the version that clients must match on to
|
||||
// agree they can communicate. This should match the ProtocolVersion
|
||||
// set on ClientConfig when using a plugin.
|
||||
// This field is not required if VersionedPlugins are being used in the
|
||||
// Client or Server configurations.
|
||||
ProtocolVersion uint
|
||||
|
||||
// MagicCookieKey and value are used as a very basic verification
|
||||
@@ -46,6 +51,10 @@ type HandshakeConfig struct {
|
||||
MagicCookieValue string
|
||||
}
|
||||
|
||||
// PluginSet is a set of plugins provided to be registered in the plugin
|
||||
// server.
|
||||
type PluginSet map[string]Plugin
|
||||
|
||||
// ServeConfig configures what sorts of plugins are served.
|
||||
type ServeConfig struct {
|
||||
// HandshakeConfig is the configuration that must match clients.
|
||||
@@ -55,7 +64,13 @@ type ServeConfig struct {
|
||||
TLSProvider func() (*tls.Config, error)
|
||||
|
||||
// Plugins are the plugins that are served.
|
||||
Plugins map[string]Plugin
|
||||
// The implied version of this PluginSet is the Handshake.ProtocolVersion.
|
||||
Plugins PluginSet
|
||||
|
||||
// VersionedPlugins is a map of PluginSets for specific protocol versions.
|
||||
// These can be used to negotiate a compatible version between client and
|
||||
// server. If this is set, Handshake.ProtocolVersion is not required.
|
||||
VersionedPlugins map[int]PluginSet
|
||||
|
||||
// GRPCServer should be non-nil to enable serving the plugins over
|
||||
// gRPC. This is a function to create the server when needed with the
|
||||
@@ -72,14 +87,83 @@ type ServeConfig struct {
|
||||
Logger hclog.Logger
|
||||
}
|
||||
|
||||
// Protocol returns the protocol that this server should speak.
|
||||
func (c *ServeConfig) Protocol() Protocol {
|
||||
result := ProtocolNetRPC
|
||||
if c.GRPCServer != nil {
|
||||
result = ProtocolGRPC
|
||||
// protocolVersion determines the protocol version and plugin set to be used by
|
||||
// the server. In the event that there is no suitable version, the last version
|
||||
// in the config is returned leaving the client to report the incompatibility.
|
||||
func protocolVersion(opts *ServeConfig) (int, Protocol, PluginSet) {
|
||||
protoVersion := int(opts.ProtocolVersion)
|
||||
pluginSet := opts.Plugins
|
||||
protoType := ProtocolNetRPC
|
||||
// Check if the client sent a list of acceptable versions
|
||||
var clientVersions []int
|
||||
if vs := os.Getenv("PLUGIN_PROTOCOL_VERSIONS"); vs != "" {
|
||||
for _, s := range strings.Split(vs, ",") {
|
||||
v, err := strconv.Atoi(s)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "server sent invalid plugin version %q", s)
|
||||
continue
|
||||
}
|
||||
clientVersions = append(clientVersions, v)
|
||||
}
|
||||
}
|
||||
|
||||
return result
|
||||
// We want to iterate in reverse order, to ensure we match the newest
|
||||
// compatible plugin version.
|
||||
sort.Sort(sort.Reverse(sort.IntSlice(clientVersions)))
|
||||
|
||||
// set the old un-versioned fields as if they were versioned plugins
|
||||
if opts.VersionedPlugins == nil {
|
||||
opts.VersionedPlugins = make(map[int]PluginSet)
|
||||
}
|
||||
|
||||
if pluginSet != nil {
|
||||
opts.VersionedPlugins[protoVersion] = pluginSet
|
||||
}
|
||||
|
||||
// Sort the version to make sure we match the latest first
|
||||
var versions []int
|
||||
for v := range opts.VersionedPlugins {
|
||||
versions = append(versions, v)
|
||||
}
|
||||
|
||||
sort.Sort(sort.Reverse(sort.IntSlice(versions)))
|
||||
|
||||
// See if we have multiple versions of Plugins to choose from
|
||||
for _, version := range versions {
|
||||
// Record each version, since we guarantee that this returns valid
|
||||
// values even if they are not a protocol match.
|
||||
protoVersion = version
|
||||
pluginSet = opts.VersionedPlugins[version]
|
||||
|
||||
// If we have a configured gRPC server we should select a protocol
|
||||
if opts.GRPCServer != nil {
|
||||
// All plugins in a set must use the same transport, so check the first
|
||||
// for the protocol type
|
||||
for _, p := range pluginSet {
|
||||
switch p.(type) {
|
||||
case GRPCPlugin:
|
||||
protoType = ProtocolGRPC
|
||||
default:
|
||||
protoType = ProtocolNetRPC
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
for _, clientVersion := range clientVersions {
|
||||
if clientVersion == protoVersion {
|
||||
return protoVersion, protoType, pluginSet
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Return the lowest version as the fallback.
|
||||
// Since we iterated over all the versions in reverse order above, these
|
||||
// values are from the lowest version number plugins (which may be from
|
||||
// a combination of the Handshake.ProtocolVersion and ServeConfig.Plugins
|
||||
// fields). This allows serving the oldest version of our plugins to a
|
||||
// legacy client that did not send a PLUGIN_PROTOCOL_VERSIONS list.
|
||||
return protoVersion, protoType, pluginSet
|
||||
}
|
||||
|
||||
// Serve serves the plugins given by ServeConfig.
|
||||
@@ -107,6 +191,10 @@ func Serve(opts *ServeConfig) {
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
// negotiate the version and plugins
|
||||
// start with default version in the handshake config
|
||||
protoVersion, protoType, pluginSet := protocolVersion(opts)
|
||||
|
||||
// Logging goes to the original stderr
|
||||
log.SetOutput(os.Stderr)
|
||||
|
||||
@@ -155,12 +243,47 @@ func Serve(opts *ServeConfig) {
|
||||
}
|
||||
}
|
||||
|
||||
var serverCert string
|
||||
clientCert := os.Getenv("PLUGIN_CLIENT_CERT")
|
||||
// If the client is configured using AutoMTLS, the certificate will be here,
|
||||
// and we need to generate our own in response.
|
||||
if tlsConfig == nil && clientCert != "" {
|
||||
logger.Info("configuring server automatic mTLS")
|
||||
clientCertPool := x509.NewCertPool()
|
||||
if !clientCertPool.AppendCertsFromPEM([]byte(clientCert)) {
|
||||
logger.Error("client cert provided but failed to parse", "cert", clientCert)
|
||||
}
|
||||
|
||||
certPEM, keyPEM, err := generateCert()
|
||||
if err != nil {
|
||||
logger.Error("failed to generate client certificate", "error", err)
|
||||
panic(err)
|
||||
}
|
||||
|
||||
cert, err := tls.X509KeyPair(certPEM, keyPEM)
|
||||
if err != nil {
|
||||
logger.Error("failed to parse client certificate", "error", err)
|
||||
panic(err)
|
||||
}
|
||||
|
||||
tlsConfig = &tls.Config{
|
||||
Certificates: []tls.Certificate{cert},
|
||||
ClientAuth: tls.RequireAndVerifyClientCert,
|
||||
ClientCAs: clientCertPool,
|
||||
MinVersion: tls.VersionTLS12,
|
||||
}
|
||||
|
||||
// We send back the raw leaf cert data for the client rather than the
|
||||
// PEM, since the protocol can't handle newlines.
|
||||
serverCert = base64.RawStdEncoding.EncodeToString(cert.Certificate[0])
|
||||
}
|
||||
|
||||
// Create the channel to tell us when we're done
|
||||
doneCh := make(chan struct{})
|
||||
|
||||
// Build the server type
|
||||
var server ServerProtocol
|
||||
switch opts.Protocol() {
|
||||
switch protoType {
|
||||
case ProtocolNetRPC:
|
||||
// If we have a TLS configuration then we wrap the listener
|
||||
// ourselves and do it at that level.
|
||||
@@ -170,7 +293,7 @@ func Serve(opts *ServeConfig) {
|
||||
|
||||
// Create the RPC server to dispense
|
||||
server = &RPCServer{
|
||||
Plugins: opts.Plugins,
|
||||
Plugins: pluginSet,
|
||||
Stdout: stdout_r,
|
||||
Stderr: stderr_r,
|
||||
DoneCh: doneCh,
|
||||
@@ -179,16 +302,17 @@ func Serve(opts *ServeConfig) {
|
||||
case ProtocolGRPC:
|
||||
// Create the gRPC server
|
||||
server = &GRPCServer{
|
||||
Plugins: opts.Plugins,
|
||||
Plugins: pluginSet,
|
||||
Server: opts.GRPCServer,
|
||||
TLS: tlsConfig,
|
||||
Stdout: stdout_r,
|
||||
Stderr: stderr_r,
|
||||
DoneCh: doneCh,
|
||||
logger: logger,
|
||||
}
|
||||
|
||||
default:
|
||||
panic("unknown server protocol: " + opts.Protocol())
|
||||
panic("unknown server protocol: " + protoType)
|
||||
}
|
||||
|
||||
// Initialize the servers
|
||||
@@ -197,25 +321,16 @@ func Serve(opts *ServeConfig) {
|
||||
return
|
||||
}
|
||||
|
||||
// Build the extra configuration
|
||||
extra := ""
|
||||
if v := server.Config(); v != "" {
|
||||
extra = base64.StdEncoding.EncodeToString([]byte(v))
|
||||
}
|
||||
if extra != "" {
|
||||
extra = "|" + extra
|
||||
}
|
||||
|
||||
logger.Debug("plugin address", "network", listener.Addr().Network(), "address", listener.Addr().String())
|
||||
|
||||
// Output the address and service name to stdout so that core can bring it up.
|
||||
fmt.Printf("%d|%d|%s|%s|%s%s\n",
|
||||
// Output the address and service name to stdout so that the client can bring it up.
|
||||
fmt.Printf("%d|%d|%s|%s|%s|%s\n",
|
||||
CoreProtocolVersion,
|
||||
opts.ProtocolVersion,
|
||||
protoVersion,
|
||||
listener.Addr().Network(),
|
||||
listener.Addr().String(),
|
||||
opts.Protocol(),
|
||||
extra)
|
||||
protoType,
|
||||
serverCert)
|
||||
os.Stdout.Sync()
|
||||
|
||||
// Eat the interrupts
|
||||
|
||||
+9
-4
@@ -8,6 +8,8 @@ import (
|
||||
"net/rpc"
|
||||
|
||||
"github.com/mitchellh/go-testing-interface"
|
||||
hclog "github.com/hashicorp/go-hclog"
|
||||
"github.com/hashicorp/go-plugin/internal/plugin"
|
||||
"google.golang.org/grpc"
|
||||
)
|
||||
|
||||
@@ -140,9 +142,11 @@ func TestPluginGRPCConn(t testing.T, ps map[string]Plugin) (*GRPCClient, *GRPCSe
|
||||
// Start up the server
|
||||
server := &GRPCServer{
|
||||
Plugins: ps,
|
||||
DoneCh: make(chan struct{}),
|
||||
Server: DefaultGRPCServer,
|
||||
Stdout: new(bytes.Buffer),
|
||||
Stderr: new(bytes.Buffer),
|
||||
logger: hclog.Default(),
|
||||
}
|
||||
if err := server.Init(); err != nil {
|
||||
t.Fatalf("err: %s", err)
|
||||
@@ -165,10 +169,11 @@ func TestPluginGRPCConn(t testing.T, ps map[string]Plugin) (*GRPCClient, *GRPCSe
|
||||
|
||||
// Create the client
|
||||
client := &GRPCClient{
|
||||
Conn: conn,
|
||||
Plugins: ps,
|
||||
broker: broker,
|
||||
doneCtx: context.Background(),
|
||||
Conn: conn,
|
||||
Plugins: ps,
|
||||
broker: broker,
|
||||
doneCtx: context.Background(),
|
||||
controller: plugin.NewGRPCControllerClient(conn),
|
||||
}
|
||||
|
||||
return client, server
|
||||
|
||||
+65
@@ -0,0 +1,65 @@
|
||||
# Versioning Library for Go
|
||||
[](https://travis-ci.org/hashicorp/go-version)
|
||||
|
||||
go-version is a library for parsing versions and version constraints,
|
||||
and verifying versions against a set of constraints. go-version
|
||||
can sort a collection of versions properly, handles prerelease/beta
|
||||
versions, can increment versions, etc.
|
||||
|
||||
Versions used with go-version must follow [SemVer](http://semver.org/).
|
||||
|
||||
## Installation and Usage
|
||||
|
||||
Package documentation can be found on
|
||||
[GoDoc](http://godoc.org/github.com/hashicorp/go-version).
|
||||
|
||||
Installation can be done with a normal `go get`:
|
||||
|
||||
```
|
||||
$ go get github.com/hashicorp/go-version
|
||||
```
|
||||
|
||||
#### Version Parsing and Comparison
|
||||
|
||||
```go
|
||||
v1, err := version.NewVersion("1.2")
|
||||
v2, err := version.NewVersion("1.5+metadata")
|
||||
|
||||
// Comparison example. There is also GreaterThan, Equal, and just
|
||||
// a simple Compare that returns an int allowing easy >=, <=, etc.
|
||||
if v1.LessThan(v2) {
|
||||
fmt.Printf("%s is less than %s", v1, v2)
|
||||
}
|
||||
```
|
||||
|
||||
#### Version Constraints
|
||||
|
||||
```go
|
||||
v1, err := version.NewVersion("1.2")
|
||||
|
||||
// Constraints example.
|
||||
constraints, err := version.NewConstraint(">= 1.0, < 1.4")
|
||||
if constraints.Check(v1) {
|
||||
fmt.Printf("%s satisfies constraints %s", v1, constraints)
|
||||
}
|
||||
```
|
||||
|
||||
#### Version Sorting
|
||||
|
||||
```go
|
||||
versionsRaw := []string{"1.1", "0.7.1", "1.4-beta", "1.4", "2"}
|
||||
versions := make([]*version.Version, len(versionsRaw))
|
||||
for i, raw := range versionsRaw {
|
||||
v, _ := version.NewVersion(raw)
|
||||
versions[i] = v
|
||||
}
|
||||
|
||||
// After this, the versions are properly sorted
|
||||
sort.Sort(version.Collection(versions))
|
||||
```
|
||||
|
||||
## Issues and Contributing
|
||||
|
||||
If you find an issue with this library, please report an issue. If you'd
|
||||
like, we welcome any contributions. Fork this library and submit a pull
|
||||
request.
|
||||
+1
@@ -0,0 +1 @@
|
||||
module github.com/hashicorp/go-version
|
||||
+53
-9
@@ -10,14 +10,25 @@ import (
|
||||
)
|
||||
|
||||
// The compiled regular expression used to test the validity of a version.
|
||||
var versionRegexp *regexp.Regexp
|
||||
var (
|
||||
versionRegexp *regexp.Regexp
|
||||
semverRegexp *regexp.Regexp
|
||||
)
|
||||
|
||||
// The raw regular expression string used for testing the validity
|
||||
// of a version.
|
||||
const VersionRegexpRaw string = `v?([0-9]+(\.[0-9]+)*?)` +
|
||||
`(-?([0-9A-Za-z\-~]+(\.[0-9A-Za-z\-~]+)*))?` +
|
||||
`(\+([0-9A-Za-z\-~]+(\.[0-9A-Za-z\-~]+)*))?` +
|
||||
`?`
|
||||
const (
|
||||
VersionRegexpRaw string = `v?([0-9]+(\.[0-9]+)*?)` +
|
||||
`(-([0-9]+[0-9A-Za-z\-~]*(\.[0-9A-Za-z\-~]+)*)|(-?([A-Za-z\-~]+[0-9A-Za-z\-~]*(\.[0-9A-Za-z\-~]+)*)))?` +
|
||||
`(\+([0-9A-Za-z\-~]+(\.[0-9A-Za-z\-~]+)*))?` +
|
||||
`?`
|
||||
|
||||
// SemverRegexpRaw requires a separator between version and prerelease
|
||||
SemverRegexpRaw string = `v?([0-9]+(\.[0-9]+)*?)` +
|
||||
`(-([0-9]+[0-9A-Za-z\-~]*(\.[0-9A-Za-z\-~]+)*)|(-([A-Za-z\-~]+[0-9A-Za-z\-~]*(\.[0-9A-Za-z\-~]+)*)))?` +
|
||||
`(\+([0-9A-Za-z\-~]+(\.[0-9A-Za-z\-~]+)*))?` +
|
||||
`?`
|
||||
)
|
||||
|
||||
// Version represents a single version.
|
||||
type Version struct {
|
||||
@@ -25,16 +36,29 @@ type Version struct {
|
||||
pre string
|
||||
segments []int64
|
||||
si int
|
||||
original string
|
||||
}
|
||||
|
||||
func init() {
|
||||
versionRegexp = regexp.MustCompile("^" + VersionRegexpRaw + "$")
|
||||
semverRegexp = regexp.MustCompile("^" + SemverRegexpRaw + "$")
|
||||
}
|
||||
|
||||
// NewVersion parses the given version and returns a new
|
||||
// Version.
|
||||
func NewVersion(v string) (*Version, error) {
|
||||
matches := versionRegexp.FindStringSubmatch(v)
|
||||
return newVersion(v, versionRegexp)
|
||||
}
|
||||
|
||||
// NewSemver parses the given version and returns a new
|
||||
// Version that adheres strictly to SemVer specs
|
||||
// https://semver.org/
|
||||
func NewSemver(v string) (*Version, error) {
|
||||
return newVersion(v, semverRegexp)
|
||||
}
|
||||
|
||||
func newVersion(v string, pattern *regexp.Regexp) (*Version, error) {
|
||||
matches := pattern.FindStringSubmatch(v)
|
||||
if matches == nil {
|
||||
return nil, fmt.Errorf("Malformed version: %s", v)
|
||||
}
|
||||
@@ -59,11 +83,17 @@ func NewVersion(v string) (*Version, error) {
|
||||
segments = append(segments, 0)
|
||||
}
|
||||
|
||||
pre := matches[7]
|
||||
if pre == "" {
|
||||
pre = matches[4]
|
||||
}
|
||||
|
||||
return &Version{
|
||||
metadata: matches[7],
|
||||
pre: matches[4],
|
||||
metadata: matches[10],
|
||||
pre: pre,
|
||||
segments: segments,
|
||||
si: si,
|
||||
original: v,
|
||||
}, nil
|
||||
}
|
||||
|
||||
@@ -301,11 +331,19 @@ func (v *Version) Segments() []int {
|
||||
// for a version "1.2.3-beta", segments will return a slice of
|
||||
// 1, 2, 3.
|
||||
func (v *Version) Segments64() []int64 {
|
||||
return v.segments
|
||||
result := make([]int64, len(v.segments))
|
||||
copy(result, v.segments)
|
||||
return result
|
||||
}
|
||||
|
||||
// String returns the full version string included pre-release
|
||||
// and metadata information.
|
||||
//
|
||||
// This value is rebuilt according to the parsed segments and other
|
||||
// information. Therefore, ambiguities in the version string such as
|
||||
// prefixed zeroes (1.04.0 => 1.4.0), `v` prefix (v1.0.0 => 1.0.0), and
|
||||
// missing parts (1.0 => 1.0.0) will be made into a canonicalized form
|
||||
// as shown in the parenthesized examples.
|
||||
func (v *Version) String() string {
|
||||
var buf bytes.Buffer
|
||||
fmtParts := make([]string, len(v.segments))
|
||||
@@ -324,3 +362,9 @@ func (v *Version) String() string {
|
||||
|
||||
return buf.String()
|
||||
}
|
||||
|
||||
// Original returns the original parsed version as-is, including any
|
||||
// potential whitespace, `v` prefix, etc.
|
||||
func (v *Version) Original() string {
|
||||
return v.original
|
||||
}
|
||||
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
# Compiled Object files, Static and Dynamic libs (Shared Objects)
|
||||
*.o
|
||||
*.a
|
||||
*.so
|
||||
|
||||
# Folders
|
||||
_obj
|
||||
_test
|
||||
|
||||
# Architecture specific extensions/prefixes
|
||||
*.[568vq]
|
||||
[568vq].out
|
||||
|
||||
*.cgo1.go
|
||||
*.cgo2.c
|
||||
_cgo_defun.c
|
||||
_cgo_gotypes.go
|
||||
_cgo_export.*
|
||||
|
||||
_testmain.go
|
||||
|
||||
*.exe
|
||||
*.test
|
||||
+86
@@ -0,0 +1,86 @@
|
||||
# Yamux
|
||||
|
||||
Yamux (Yet another Multiplexer) is a multiplexing library for Golang.
|
||||
It relies on an underlying connection to provide reliability
|
||||
and ordering, such as TCP or Unix domain sockets, and provides
|
||||
stream-oriented multiplexing. It is inspired by SPDY but is not
|
||||
interoperable with it.
|
||||
|
||||
Yamux features include:
|
||||
|
||||
* Bi-directional streams
|
||||
* Streams can be opened by either client or server
|
||||
* Useful for NAT traversal
|
||||
* Server-side push support
|
||||
* Flow control
|
||||
* Avoid starvation
|
||||
* Back-pressure to prevent overwhelming a receiver
|
||||
* Keep Alives
|
||||
* Enables persistent connections over a load balancer
|
||||
* Efficient
|
||||
* Enables thousands of logical streams with low overhead
|
||||
|
||||
## Documentation
|
||||
|
||||
For complete documentation, see the associated [Godoc](http://godoc.org/github.com/hashicorp/yamux).
|
||||
|
||||
## Specification
|
||||
|
||||
The full specification for Yamux is provided in the `spec.md` file.
|
||||
It can be used as a guide to implementors of interoperable libraries.
|
||||
|
||||
## Usage
|
||||
|
||||
Using Yamux is remarkably simple:
|
||||
|
||||
```go
|
||||
|
||||
func client() {
|
||||
// Get a TCP connection
|
||||
conn, err := net.Dial(...)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Setup client side of yamux
|
||||
session, err := yamux.Client(conn, nil)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Open a new stream
|
||||
stream, err := session.Open()
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Stream implements net.Conn
|
||||
stream.Write([]byte("ping"))
|
||||
}
|
||||
|
||||
func server() {
|
||||
// Accept a TCP connection
|
||||
conn, err := listener.Accept()
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Setup server side of yamux
|
||||
session, err := yamux.Server(conn, nil)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Accept a stream
|
||||
stream, err := session.Accept()
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
// Listen for a message
|
||||
buf := make([]byte, 4)
|
||||
stream.Read(buf)
|
||||
}
|
||||
|
||||
```
|
||||
|
||||
+1
-12
@@ -3,7 +3,6 @@ package yamux
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"os"
|
||||
"time"
|
||||
)
|
||||
@@ -31,13 +30,8 @@ type Config struct {
|
||||
// window size that we allow for a stream.
|
||||
MaxStreamWindowSize uint32
|
||||
|
||||
// LogOutput is used to control the log destination. Either Logger or
|
||||
// LogOutput can be set, not both.
|
||||
// LogOutput is used to control the log destination
|
||||
LogOutput io.Writer
|
||||
|
||||
// Logger is used to pass in the logger to be used. Either Logger or
|
||||
// LogOutput can be set, not both.
|
||||
Logger *log.Logger
|
||||
}
|
||||
|
||||
// DefaultConfig is used to return a default configuration
|
||||
@@ -63,11 +57,6 @@ func VerifyConfig(config *Config) error {
|
||||
if config.MaxStreamWindowSize < initialStreamWindow {
|
||||
return fmt.Errorf("MaxStreamWindowSize must be larger than %d", initialStreamWindow)
|
||||
}
|
||||
if config.LogOutput != nil && config.Logger != nil {
|
||||
return fmt.Errorf("both Logger and LogOutput may not be set, select one")
|
||||
} else if config.LogOutput == nil && config.Logger == nil {
|
||||
return fmt.Errorf("one of Logger or LogOutput must be set, select one")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
+1
-6
@@ -86,14 +86,9 @@ type sendReady struct {
|
||||
|
||||
// newSession is used to construct a new session
|
||||
func newSession(config *Config, conn io.ReadWriteCloser, client bool) *Session {
|
||||
logger := config.Logger
|
||||
if logger == nil {
|
||||
logger = log.New(config.LogOutput, "", log.LstdFlags)
|
||||
}
|
||||
|
||||
s := &Session{
|
||||
config: config,
|
||||
logger: logger,
|
||||
logger: log.New(config.LogOutput, "", log.LstdFlags),
|
||||
conn: conn,
|
||||
bufRead: bufio.NewReader(conn),
|
||||
pings: make(map[uint32]chan struct{}),
|
||||
|
||||
+140
@@ -0,0 +1,140 @@
|
||||
# Specification
|
||||
|
||||
We use this document to detail the internal specification of Yamux.
|
||||
This is used both as a guide for implementing Yamux, but also for
|
||||
alternative interoperable libraries to be built.
|
||||
|
||||
# Framing
|
||||
|
||||
Yamux uses a streaming connection underneath, but imposes a message
|
||||
framing so that it can be shared between many logical streams. Each
|
||||
frame contains a header like:
|
||||
|
||||
* Version (8 bits)
|
||||
* Type (8 bits)
|
||||
* Flags (16 bits)
|
||||
* StreamID (32 bits)
|
||||
* Length (32 bits)
|
||||
|
||||
This means that each header has a 12 byte overhead.
|
||||
All fields are encoded in network order (big endian).
|
||||
Each field is described below:
|
||||
|
||||
## Version Field
|
||||
|
||||
The version field is used for future backward compatibility. At the
|
||||
current time, the field is always set to 0, to indicate the initial
|
||||
version.
|
||||
|
||||
## Type Field
|
||||
|
||||
The type field is used to switch the frame message type. The following
|
||||
message types are supported:
|
||||
|
||||
* 0x0 Data - Used to transmit data. May transmit zero length payloads
|
||||
depending on the flags.
|
||||
|
||||
* 0x1 Window Update - Used to updated the senders receive window size.
|
||||
This is used to implement per-session flow control.
|
||||
|
||||
* 0x2 Ping - Used to measure RTT. It can also be used to heart-beat
|
||||
and do keep-alives over TCP.
|
||||
|
||||
* 0x3 Go Away - Used to close a session.
|
||||
|
||||
## Flag Field
|
||||
|
||||
The flags field is used to provide additional information related
|
||||
to the message type. The following flags are supported:
|
||||
|
||||
* 0x1 SYN - Signals the start of a new stream. May be sent with a data or
|
||||
window update message. Also sent with a ping to indicate outbound.
|
||||
|
||||
* 0x2 ACK - Acknowledges the start of a new stream. May be sent with a data
|
||||
or window update message. Also sent with a ping to indicate response.
|
||||
|
||||
* 0x4 FIN - Performs a half-close of a stream. May be sent with a data
|
||||
message or window update.
|
||||
|
||||
* 0x8 RST - Reset a stream immediately. May be sent with a data or
|
||||
window update message.
|
||||
|
||||
## StreamID Field
|
||||
|
||||
The StreamID field is used to identify the logical stream the frame
|
||||
is addressing. The client side should use odd ID's, and the server even.
|
||||
This prevents any collisions. Additionally, the 0 ID is reserved to represent
|
||||
the session.
|
||||
|
||||
Both Ping and Go Away messages should always use the 0 StreamID.
|
||||
|
||||
## Length Field
|
||||
|
||||
The meaning of the length field depends on the message type:
|
||||
|
||||
* Data - provides the length of bytes following the header
|
||||
* Window update - provides a delta update to the window size
|
||||
* Ping - Contains an opaque value, echoed back
|
||||
* Go Away - Contains an error code
|
||||
|
||||
# Message Flow
|
||||
|
||||
There is no explicit connection setup, as Yamux relies on an underlying
|
||||
transport to be provided. However, there is a distinction between client
|
||||
and server side of the connection.
|
||||
|
||||
## Opening a stream
|
||||
|
||||
To open a stream, an initial data or window update frame is sent
|
||||
with a new StreamID. The SYN flag should be set to signal a new stream.
|
||||
|
||||
The receiver must then reply with either a data or window update frame
|
||||
with the StreamID along with the ACK flag to accept the stream or with
|
||||
the RST flag to reject the stream.
|
||||
|
||||
Because we are relying on the reliable stream underneath, a connection
|
||||
can begin sending data once the SYN flag is sent. The corresponding
|
||||
ACK does not need to be received. This is particularly well suited
|
||||
for an RPC system where a client wants to open a stream and immediately
|
||||
fire a request without waiting for the RTT of the ACK.
|
||||
|
||||
This does introduce the possibility of a connection being rejected
|
||||
after data has been sent already. This is a slight semantic difference
|
||||
from TCP, where the conection cannot be refused after it is opened.
|
||||
Clients should be prepared to handle this by checking for an error
|
||||
that indicates a RST was received.
|
||||
|
||||
## Closing a stream
|
||||
|
||||
To close a stream, either side sends a data or window update frame
|
||||
along with the FIN flag. This does a half-close indicating the sender
|
||||
will send no further data.
|
||||
|
||||
Once both sides have closed the connection, the stream is closed.
|
||||
|
||||
Alternatively, if an error occurs, the RST flag can be used to
|
||||
hard close a stream immediately.
|
||||
|
||||
## Flow Control
|
||||
|
||||
When Yamux is initially starts each stream with a 256KB window size.
|
||||
There is no window size for the session.
|
||||
|
||||
To prevent the streams from stalling, window update frames should be
|
||||
sent regularly. Yamux can be configured to provide a larger limit for
|
||||
windows sizes. Both sides assume the initial 256KB window, but can
|
||||
immediately send a window update as part of the SYN/ACK indicating a
|
||||
larger window.
|
||||
|
||||
Both sides should track the number of bytes sent in Data frames
|
||||
only, as only they are tracked as part of the window size.
|
||||
|
||||
## Session termination
|
||||
|
||||
When a session is being terminated, the Go Away message should
|
||||
be sent. The Length should be set to one of the following to
|
||||
provide an error code:
|
||||
|
||||
* 0x0 Normal termination
|
||||
* 0x1 Protocol error
|
||||
* 0x2 Internal error
|
||||
Reference in New Issue
Block a user