Richard Hillmann
9 years ago
328 changed files with 145253 additions and 7 deletions
@ -0,0 +1,41 @@ |
|||||||
|
Challenge: Exporter for Prometheus |
||||||
|
================================== |
||||||
|
This is an prometheus metrics exporter for the soundcloud challenge. |
||||||
|
|
||||||
|
Requirements |
||||||
|
------------ |
||||||
|
|
||||||
|
* Docker, of course ;-) |
||||||
|
* Working go 1.8 (older should work as well) |
||||||
|
|
||||||
|
Build |
||||||
|
-------------------- |
||||||
|
Build binary `exporter`: |
||||||
|
```bash |
||||||
|
make |
||||||
|
``` |
||||||
|
|
||||||
|
Build docker image with tag `challenge/exporter`: |
||||||
|
```bash |
||||||
|
make docker |
||||||
|
``` |
||||||
|
|
||||||
|
Run |
||||||
|
--- |
||||||
|
Start locally in docker: |
||||||
|
|
||||||
|
```bash |
||||||
|
docker run -d -p 8080:8080 --name challenge beorn7/syseng-challenge |
||||||
|
docker run -d -p 8081:8080 --link challenge --name challenge_exporter challenge/exporter --endpoint http://challenge:8080/stats |
||||||
|
|
||||||
|
docker stop challenge_exporter |
||||||
|
docker rm challenge_exporter |
||||||
|
docker stop challenge |
||||||
|
docker rm challenge |
||||||
|
``` |
||||||
|
or with docker-compose |
||||||
|
```bash |
||||||
|
docker-compose up |
||||||
|
``` |
||||||
|
|
||||||
|
Metrics should now be availabe on `http://localhost:8081/metrics` |
||||||
@ -0,0 +1,11 @@ |
|||||||
|
exporter: |
||||||
|
build: ./ |
||||||
|
command: -endpoint=http://app:8080/stats -listen :8081 --timeout 200 |
||||||
|
links: |
||||||
|
- app |
||||||
|
ports: |
||||||
|
- 8081:8081 |
||||||
|
app: |
||||||
|
image: beorn7/syseng-challenge |
||||||
|
ports: |
||||||
|
- 8080:8080 |
||||||
@ -0,0 +1,90 @@ |
|||||||
|
# 0.11.4 |
||||||
|
|
||||||
|
* bug: fix undefined variable on solaris (#493) |
||||||
|
|
||||||
|
# 0.11.3 |
||||||
|
|
||||||
|
* formatter: configure quoting of empty values (#484) |
||||||
|
* formatter: configure quoting character (default is `"`) (#484) |
||||||
|
* bug: fix not importing io correctly in non-linux environments (#481) |
||||||
|
|
||||||
|
# 0.11.2 |
||||||
|
|
||||||
|
* bug: fix windows terminal detection (#476) |
||||||
|
|
||||||
|
# 0.11.1 |
||||||
|
|
||||||
|
* bug: fix tty detection with custom out (#471) |
||||||
|
|
||||||
|
# 0.11.0 |
||||||
|
|
||||||
|
* performance: Use bufferpool to allocate (#370) |
||||||
|
* terminal: terminal detection for app-engine (#343) |
||||||
|
* feature: exit handler (#375) |
||||||
|
|
||||||
|
# 0.10.0 |
||||||
|
|
||||||
|
* feature: Add a test hook (#180) |
||||||
|
* feature: `ParseLevel` is now case-insensitive (#326) |
||||||
|
* feature: `FieldLogger` interface that generalizes `Logger` and `Entry` (#308) |
||||||
|
* performance: avoid re-allocations on `WithFields` (#335) |
||||||
|
|
||||||
|
# 0.9.0 |
||||||
|
|
||||||
|
* logrus/text_formatter: don't emit empty msg |
||||||
|
* logrus/hooks/airbrake: move out of main repository |
||||||
|
* logrus/hooks/sentry: move out of main repository |
||||||
|
* logrus/hooks/papertrail: move out of main repository |
||||||
|
* logrus/hooks/bugsnag: move out of main repository |
||||||
|
* logrus/core: run tests with `-race` |
||||||
|
* logrus/core: detect TTY based on `stderr` |
||||||
|
* logrus/core: support `WithError` on logger |
||||||
|
* logrus/core: Solaris support |
||||||
|
|
||||||
|
# 0.8.7 |
||||||
|
|
||||||
|
* logrus/core: fix possible race (#216) |
||||||
|
* logrus/doc: small typo fixes and doc improvements |
||||||
|
|
||||||
|
|
||||||
|
# 0.8.6 |
||||||
|
|
||||||
|
* hooks/raven: allow passing an initialized client |
||||||
|
|
||||||
|
# 0.8.5 |
||||||
|
|
||||||
|
* logrus/core: revert #208 |
||||||
|
|
||||||
|
# 0.8.4 |
||||||
|
|
||||||
|
* formatter/text: fix data race (#218) |
||||||
|
|
||||||
|
# 0.8.3 |
||||||
|
|
||||||
|
* logrus/core: fix entry log level (#208) |
||||||
|
* logrus/core: improve performance of text formatter by 40% |
||||||
|
* logrus/core: expose `LevelHooks` type |
||||||
|
* logrus/core: add support for DragonflyBSD and NetBSD |
||||||
|
* formatter/text: print structs more verbosely |
||||||
|
|
||||||
|
# 0.8.2 |
||||||
|
|
||||||
|
* logrus: fix more Fatal family functions |
||||||
|
|
||||||
|
# 0.8.1 |
||||||
|
|
||||||
|
* logrus: fix not exiting on `Fatalf` and `Fatalln` |
||||||
|
|
||||||
|
# 0.8.0 |
||||||
|
|
||||||
|
* logrus: defaults to stderr instead of stdout |
||||||
|
* hooks/sentry: add special field for `*http.Request` |
||||||
|
* formatter/text: ignore Windows for colors |
||||||
|
|
||||||
|
# 0.7.3 |
||||||
|
|
||||||
|
* formatter/\*: allow configuration of timestamp layout |
||||||
|
|
||||||
|
# 0.7.2 |
||||||
|
|
||||||
|
* formatter/text: Add configuration option for time format (#158) |
||||||
@ -0,0 +1,21 @@ |
|||||||
|
The MIT License (MIT) |
||||||
|
|
||||||
|
Copyright (c) 2014 Simon Eskildsen |
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy |
||||||
|
of this software and associated documentation files (the "Software"), to deal |
||||||
|
in the Software without restriction, including without limitation the rights |
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell |
||||||
|
copies of the Software, and to permit persons to whom the Software is |
||||||
|
furnished to do so, subject to the following conditions: |
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in |
||||||
|
all copies or substantial portions of the Software. |
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR |
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, |
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE |
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER |
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, |
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN |
||||||
|
THE SOFTWARE. |
||||||
@ -0,0 +1,64 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
// The following code was sourced and modified from the
|
||||||
|
// https://bitbucket.org/tebeka/atexit package governed by the following license:
|
||||||
|
//
|
||||||
|
// Copyright (c) 2012 Miki Tebeka <miki.tebeka@gmail.com>.
|
||||||
|
//
|
||||||
|
// Permission is hereby granted, free of charge, to any person obtaining a copy of
|
||||||
|
// this software and associated documentation files (the "Software"), to deal in
|
||||||
|
// the Software without restriction, including without limitation the rights to
|
||||||
|
// use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
|
||||||
|
// the Software, and to permit persons to whom the Software is furnished to do so,
|
||||||
|
// subject to the following conditions:
|
||||||
|
//
|
||||||
|
// The above copyright notice and this permission notice shall be included in all
|
||||||
|
// copies or substantial portions of the Software.
|
||||||
|
//
|
||||||
|
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
|
||||||
|
// FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
|
||||||
|
// COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
||||||
|
// IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
||||||
|
// CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"os" |
||||||
|
) |
||||||
|
|
||||||
|
var handlers = []func(){} |
||||||
|
|
||||||
|
func runHandler(handler func()) { |
||||||
|
defer func() { |
||||||
|
if err := recover(); err != nil { |
||||||
|
fmt.Fprintln(os.Stderr, "Error: Logrus exit handler error:", err) |
||||||
|
} |
||||||
|
}() |
||||||
|
|
||||||
|
handler() |
||||||
|
} |
||||||
|
|
||||||
|
func runHandlers() { |
||||||
|
for _, handler := range handlers { |
||||||
|
runHandler(handler) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Exit runs all the Logrus atexit handlers and then terminates the program using os.Exit(code)
|
||||||
|
func Exit(code int) { |
||||||
|
runHandlers() |
||||||
|
os.Exit(code) |
||||||
|
} |
||||||
|
|
||||||
|
// RegisterExitHandler adds a Logrus Exit handler, call logrus.Exit to invoke
|
||||||
|
// all handlers. The handlers will also be invoked when any Fatal log entry is
|
||||||
|
// made.
|
||||||
|
//
|
||||||
|
// This method is useful when a caller wishes to use logrus to log a fatal
|
||||||
|
// message but also needs to gracefully shutdown. An example usecase could be
|
||||||
|
// closing database connections, or sending a alert that the application is
|
||||||
|
// closing.
|
||||||
|
func RegisterExitHandler(handler func()) { |
||||||
|
handlers = append(handlers, handler) |
||||||
|
} |
||||||
@ -0,0 +1,26 @@ |
|||||||
|
/* |
||||||
|
Package logrus is a structured logger for Go, completely API compatible with the standard library logger. |
||||||
|
|
||||||
|
|
||||||
|
The simplest way to use Logrus is simply the package-level exported logger: |
||||||
|
|
||||||
|
package main |
||||||
|
|
||||||
|
import ( |
||||||
|
log "github.com/Sirupsen/logrus" |
||||||
|
) |
||||||
|
|
||||||
|
func main() { |
||||||
|
log.WithFields(log.Fields{ |
||||||
|
"animal": "walrus", |
||||||
|
"number": 1, |
||||||
|
"size": 10, |
||||||
|
}).Info("A walrus appears") |
||||||
|
} |
||||||
|
|
||||||
|
Output: |
||||||
|
time="2015-09-07T08:48:33Z" level=info msg="A walrus appears" animal=walrus number=1 size=10 |
||||||
|
|
||||||
|
For a full guide visit https://github.com/Sirupsen/logrus
|
||||||
|
*/ |
||||||
|
package logrus |
||||||
@ -0,0 +1,275 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"fmt" |
||||||
|
"os" |
||||||
|
"sync" |
||||||
|
"time" |
||||||
|
) |
||||||
|
|
||||||
|
var bufferPool *sync.Pool |
||||||
|
|
||||||
|
func init() { |
||||||
|
bufferPool = &sync.Pool{ |
||||||
|
New: func() interface{} { |
||||||
|
return new(bytes.Buffer) |
||||||
|
}, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Defines the key when adding errors using WithError.
|
||||||
|
var ErrorKey = "error" |
||||||
|
|
||||||
|
// An entry is the final or intermediate Logrus logging entry. It contains all
|
||||||
|
// the fields passed with WithField{,s}. It's finally logged when Debug, Info,
|
||||||
|
// Warn, Error, Fatal or Panic is called on it. These objects can be reused and
|
||||||
|
// passed around as much as you wish to avoid field duplication.
|
||||||
|
type Entry struct { |
||||||
|
Logger *Logger |
||||||
|
|
||||||
|
// Contains all the fields set by the user.
|
||||||
|
Data Fields |
||||||
|
|
||||||
|
// Time at which the log entry was created
|
||||||
|
Time time.Time |
||||||
|
|
||||||
|
// Level the log entry was logged at: Debug, Info, Warn, Error, Fatal or Panic
|
||||||
|
Level Level |
||||||
|
|
||||||
|
// Message passed to Debug, Info, Warn, Error, Fatal or Panic
|
||||||
|
Message string |
||||||
|
|
||||||
|
// When formatter is called in entry.log(), an Buffer may be set to entry
|
||||||
|
Buffer *bytes.Buffer |
||||||
|
} |
||||||
|
|
||||||
|
func NewEntry(logger *Logger) *Entry { |
||||||
|
return &Entry{ |
||||||
|
Logger: logger, |
||||||
|
// Default is three fields, give a little extra room
|
||||||
|
Data: make(Fields, 5), |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Returns the string representation from the reader and ultimately the
|
||||||
|
// formatter.
|
||||||
|
func (entry *Entry) String() (string, error) { |
||||||
|
serialized, err := entry.Logger.Formatter.Format(entry) |
||||||
|
if err != nil { |
||||||
|
return "", err |
||||||
|
} |
||||||
|
str := string(serialized) |
||||||
|
return str, nil |
||||||
|
} |
||||||
|
|
||||||
|
// Add an error as single field (using the key defined in ErrorKey) to the Entry.
|
||||||
|
func (entry *Entry) WithError(err error) *Entry { |
||||||
|
return entry.WithField(ErrorKey, err) |
||||||
|
} |
||||||
|
|
||||||
|
// Add a single field to the Entry.
|
||||||
|
func (entry *Entry) WithField(key string, value interface{}) *Entry { |
||||||
|
return entry.WithFields(Fields{key: value}) |
||||||
|
} |
||||||
|
|
||||||
|
// Add a map of fields to the Entry.
|
||||||
|
func (entry *Entry) WithFields(fields Fields) *Entry { |
||||||
|
data := make(Fields, len(entry.Data)+len(fields)) |
||||||
|
for k, v := range entry.Data { |
||||||
|
data[k] = v |
||||||
|
} |
||||||
|
for k, v := range fields { |
||||||
|
data[k] = v |
||||||
|
} |
||||||
|
return &Entry{Logger: entry.Logger, Data: data} |
||||||
|
} |
||||||
|
|
||||||
|
// This function is not declared with a pointer value because otherwise
|
||||||
|
// race conditions will occur when using multiple goroutines
|
||||||
|
func (entry Entry) log(level Level, msg string) { |
||||||
|
var buffer *bytes.Buffer |
||||||
|
entry.Time = time.Now() |
||||||
|
entry.Level = level |
||||||
|
entry.Message = msg |
||||||
|
|
||||||
|
if err := entry.Logger.Hooks.Fire(level, &entry); err != nil { |
||||||
|
entry.Logger.mu.Lock() |
||||||
|
fmt.Fprintf(os.Stderr, "Failed to fire hook: %v\n", err) |
||||||
|
entry.Logger.mu.Unlock() |
||||||
|
} |
||||||
|
buffer = bufferPool.Get().(*bytes.Buffer) |
||||||
|
buffer.Reset() |
||||||
|
defer bufferPool.Put(buffer) |
||||||
|
entry.Buffer = buffer |
||||||
|
serialized, err := entry.Logger.Formatter.Format(&entry) |
||||||
|
entry.Buffer = nil |
||||||
|
if err != nil { |
||||||
|
entry.Logger.mu.Lock() |
||||||
|
fmt.Fprintf(os.Stderr, "Failed to obtain reader, %v\n", err) |
||||||
|
entry.Logger.mu.Unlock() |
||||||
|
} else { |
||||||
|
entry.Logger.mu.Lock() |
||||||
|
_, err = entry.Logger.Out.Write(serialized) |
||||||
|
if err != nil { |
||||||
|
fmt.Fprintf(os.Stderr, "Failed to write to log, %v\n", err) |
||||||
|
} |
||||||
|
entry.Logger.mu.Unlock() |
||||||
|
} |
||||||
|
|
||||||
|
// To avoid Entry#log() returning a value that only would make sense for
|
||||||
|
// panic() to use in Entry#Panic(), we avoid the allocation by checking
|
||||||
|
// directly here.
|
||||||
|
if level <= PanicLevel { |
||||||
|
panic(&entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Debug(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= DebugLevel { |
||||||
|
entry.log(DebugLevel, fmt.Sprint(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Print(args ...interface{}) { |
||||||
|
entry.Info(args...) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Info(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= InfoLevel { |
||||||
|
entry.log(InfoLevel, fmt.Sprint(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Warn(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= WarnLevel { |
||||||
|
entry.log(WarnLevel, fmt.Sprint(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Warning(args ...interface{}) { |
||||||
|
entry.Warn(args...) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Error(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= ErrorLevel { |
||||||
|
entry.log(ErrorLevel, fmt.Sprint(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Fatal(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= FatalLevel { |
||||||
|
entry.log(FatalLevel, fmt.Sprint(args...)) |
||||||
|
} |
||||||
|
Exit(1) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Panic(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= PanicLevel { |
||||||
|
entry.log(PanicLevel, fmt.Sprint(args...)) |
||||||
|
} |
||||||
|
panic(fmt.Sprint(args...)) |
||||||
|
} |
||||||
|
|
||||||
|
// Entry Printf family functions
|
||||||
|
|
||||||
|
func (entry *Entry) Debugf(format string, args ...interface{}) { |
||||||
|
if entry.Logger.Level >= DebugLevel { |
||||||
|
entry.Debug(fmt.Sprintf(format, args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Infof(format string, args ...interface{}) { |
||||||
|
if entry.Logger.Level >= InfoLevel { |
||||||
|
entry.Info(fmt.Sprintf(format, args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Printf(format string, args ...interface{}) { |
||||||
|
entry.Infof(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Warnf(format string, args ...interface{}) { |
||||||
|
if entry.Logger.Level >= WarnLevel { |
||||||
|
entry.Warn(fmt.Sprintf(format, args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Warningf(format string, args ...interface{}) { |
||||||
|
entry.Warnf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Errorf(format string, args ...interface{}) { |
||||||
|
if entry.Logger.Level >= ErrorLevel { |
||||||
|
entry.Error(fmt.Sprintf(format, args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Fatalf(format string, args ...interface{}) { |
||||||
|
if entry.Logger.Level >= FatalLevel { |
||||||
|
entry.Fatal(fmt.Sprintf(format, args...)) |
||||||
|
} |
||||||
|
Exit(1) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Panicf(format string, args ...interface{}) { |
||||||
|
if entry.Logger.Level >= PanicLevel { |
||||||
|
entry.Panic(fmt.Sprintf(format, args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Entry Println family functions
|
||||||
|
|
||||||
|
func (entry *Entry) Debugln(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= DebugLevel { |
||||||
|
entry.Debug(entry.sprintlnn(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Infoln(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= InfoLevel { |
||||||
|
entry.Info(entry.sprintlnn(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Println(args ...interface{}) { |
||||||
|
entry.Infoln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Warnln(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= WarnLevel { |
||||||
|
entry.Warn(entry.sprintlnn(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Warningln(args ...interface{}) { |
||||||
|
entry.Warnln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Errorln(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= ErrorLevel { |
||||||
|
entry.Error(entry.sprintlnn(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Fatalln(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= FatalLevel { |
||||||
|
entry.Fatal(entry.sprintlnn(args...)) |
||||||
|
} |
||||||
|
Exit(1) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Panicln(args ...interface{}) { |
||||||
|
if entry.Logger.Level >= PanicLevel { |
||||||
|
entry.Panic(entry.sprintlnn(args...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Sprintlnn => Sprint no newline. This is to get the behavior of how
|
||||||
|
// fmt.Sprintln where spaces are always added between operands, regardless of
|
||||||
|
// their type. Instead of vendoring the Sprintln implementation to spare a
|
||||||
|
// string allocation, we do the simplest thing.
|
||||||
|
func (entry *Entry) sprintlnn(args ...interface{}) string { |
||||||
|
msg := fmt.Sprintln(args...) |
||||||
|
return msg[:len(msg)-1] |
||||||
|
} |
||||||
@ -0,0 +1,193 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"io" |
||||||
|
) |
||||||
|
|
||||||
|
var ( |
||||||
|
// std is the name of the standard logger in stdlib `log`
|
||||||
|
std = New() |
||||||
|
) |
||||||
|
|
||||||
|
func StandardLogger() *Logger { |
||||||
|
return std |
||||||
|
} |
||||||
|
|
||||||
|
// SetOutput sets the standard logger output.
|
||||||
|
func SetOutput(out io.Writer) { |
||||||
|
std.mu.Lock() |
||||||
|
defer std.mu.Unlock() |
||||||
|
std.Out = out |
||||||
|
} |
||||||
|
|
||||||
|
// SetFormatter sets the standard logger formatter.
|
||||||
|
func SetFormatter(formatter Formatter) { |
||||||
|
std.mu.Lock() |
||||||
|
defer std.mu.Unlock() |
||||||
|
std.Formatter = formatter |
||||||
|
} |
||||||
|
|
||||||
|
// SetLevel sets the standard logger level.
|
||||||
|
func SetLevel(level Level) { |
||||||
|
std.mu.Lock() |
||||||
|
defer std.mu.Unlock() |
||||||
|
std.Level = level |
||||||
|
} |
||||||
|
|
||||||
|
// GetLevel returns the standard logger level.
|
||||||
|
func GetLevel() Level { |
||||||
|
std.mu.Lock() |
||||||
|
defer std.mu.Unlock() |
||||||
|
return std.Level |
||||||
|
} |
||||||
|
|
||||||
|
// AddHook adds a hook to the standard logger hooks.
|
||||||
|
func AddHook(hook Hook) { |
||||||
|
std.mu.Lock() |
||||||
|
defer std.mu.Unlock() |
||||||
|
std.Hooks.Add(hook) |
||||||
|
} |
||||||
|
|
||||||
|
// WithError creates an entry from the standard logger and adds an error to it, using the value defined in ErrorKey as key.
|
||||||
|
func WithError(err error) *Entry { |
||||||
|
return std.WithField(ErrorKey, err) |
||||||
|
} |
||||||
|
|
||||||
|
// WithField creates an entry from the standard logger and adds a field to
|
||||||
|
// it. If you want multiple fields, use `WithFields`.
|
||||||
|
//
|
||||||
|
// Note that it doesn't log until you call Debug, Print, Info, Warn, Fatal
|
||||||
|
// or Panic on the Entry it returns.
|
||||||
|
func WithField(key string, value interface{}) *Entry { |
||||||
|
return std.WithField(key, value) |
||||||
|
} |
||||||
|
|
||||||
|
// WithFields creates an entry from the standard logger and adds multiple
|
||||||
|
// fields to it. This is simply a helper for `WithField`, invoking it
|
||||||
|
// once for each field.
|
||||||
|
//
|
||||||
|
// Note that it doesn't log until you call Debug, Print, Info, Warn, Fatal
|
||||||
|
// or Panic on the Entry it returns.
|
||||||
|
func WithFields(fields Fields) *Entry { |
||||||
|
return std.WithFields(fields) |
||||||
|
} |
||||||
|
|
||||||
|
// Debug logs a message at level Debug on the standard logger.
|
||||||
|
func Debug(args ...interface{}) { |
||||||
|
std.Debug(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Print logs a message at level Info on the standard logger.
|
||||||
|
func Print(args ...interface{}) { |
||||||
|
std.Print(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Info logs a message at level Info on the standard logger.
|
||||||
|
func Info(args ...interface{}) { |
||||||
|
std.Info(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warn logs a message at level Warn on the standard logger.
|
||||||
|
func Warn(args ...interface{}) { |
||||||
|
std.Warn(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warning logs a message at level Warn on the standard logger.
|
||||||
|
func Warning(args ...interface{}) { |
||||||
|
std.Warning(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Error logs a message at level Error on the standard logger.
|
||||||
|
func Error(args ...interface{}) { |
||||||
|
std.Error(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Panic logs a message at level Panic on the standard logger.
|
||||||
|
func Panic(args ...interface{}) { |
||||||
|
std.Panic(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatal logs a message at level Fatal on the standard logger.
|
||||||
|
func Fatal(args ...interface{}) { |
||||||
|
std.Fatal(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Debugf logs a message at level Debug on the standard logger.
|
||||||
|
func Debugf(format string, args ...interface{}) { |
||||||
|
std.Debugf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Printf logs a message at level Info on the standard logger.
|
||||||
|
func Printf(format string, args ...interface{}) { |
||||||
|
std.Printf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Infof logs a message at level Info on the standard logger.
|
||||||
|
func Infof(format string, args ...interface{}) { |
||||||
|
std.Infof(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warnf logs a message at level Warn on the standard logger.
|
||||||
|
func Warnf(format string, args ...interface{}) { |
||||||
|
std.Warnf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warningf logs a message at level Warn on the standard logger.
|
||||||
|
func Warningf(format string, args ...interface{}) { |
||||||
|
std.Warningf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Errorf logs a message at level Error on the standard logger.
|
||||||
|
func Errorf(format string, args ...interface{}) { |
||||||
|
std.Errorf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Panicf logs a message at level Panic on the standard logger.
|
||||||
|
func Panicf(format string, args ...interface{}) { |
||||||
|
std.Panicf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatalf logs a message at level Fatal on the standard logger.
|
||||||
|
func Fatalf(format string, args ...interface{}) { |
||||||
|
std.Fatalf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Debugln logs a message at level Debug on the standard logger.
|
||||||
|
func Debugln(args ...interface{}) { |
||||||
|
std.Debugln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Println logs a message at level Info on the standard logger.
|
||||||
|
func Println(args ...interface{}) { |
||||||
|
std.Println(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Infoln logs a message at level Info on the standard logger.
|
||||||
|
func Infoln(args ...interface{}) { |
||||||
|
std.Infoln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warnln logs a message at level Warn on the standard logger.
|
||||||
|
func Warnln(args ...interface{}) { |
||||||
|
std.Warnln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warningln logs a message at level Warn on the standard logger.
|
||||||
|
func Warningln(args ...interface{}) { |
||||||
|
std.Warningln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Errorln logs a message at level Error on the standard logger.
|
||||||
|
func Errorln(args ...interface{}) { |
||||||
|
std.Errorln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Panicln logs a message at level Panic on the standard logger.
|
||||||
|
func Panicln(args ...interface{}) { |
||||||
|
std.Panicln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatalln logs a message at level Fatal on the standard logger.
|
||||||
|
func Fatalln(args ...interface{}) { |
||||||
|
std.Fatalln(args...) |
||||||
|
} |
||||||
@ -0,0 +1,45 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import "time" |
||||||
|
|
||||||
|
const DefaultTimestampFormat = time.RFC3339 |
||||||
|
|
||||||
|
// The Formatter interface is used to implement a custom Formatter. It takes an
|
||||||
|
// `Entry`. It exposes all the fields, including the default ones:
|
||||||
|
//
|
||||||
|
// * `entry.Data["msg"]`. The message passed from Info, Warn, Error ..
|
||||||
|
// * `entry.Data["time"]`. The timestamp.
|
||||||
|
// * `entry.Data["level"]. The level the entry was logged at.
|
||||||
|
//
|
||||||
|
// Any additional fields added with `WithField` or `WithFields` are also in
|
||||||
|
// `entry.Data`. Format is expected to return an array of bytes which are then
|
||||||
|
// logged to `logger.Out`.
|
||||||
|
type Formatter interface { |
||||||
|
Format(*Entry) ([]byte, error) |
||||||
|
} |
||||||
|
|
||||||
|
// This is to not silently overwrite `time`, `msg` and `level` fields when
|
||||||
|
// dumping it. If this code wasn't there doing:
|
||||||
|
//
|
||||||
|
// logrus.WithField("level", 1).Info("hello")
|
||||||
|
//
|
||||||
|
// Would just silently drop the user provided level. Instead with this code
|
||||||
|
// it'll logged as:
|
||||||
|
//
|
||||||
|
// {"level": "info", "fields.level": 1, "msg": "hello", "time": "..."}
|
||||||
|
//
|
||||||
|
// It's not exported because it's still using Data in an opinionated way. It's to
|
||||||
|
// avoid code duplication between the two default formatters.
|
||||||
|
func prefixFieldClashes(data Fields) { |
||||||
|
if t, ok := data["time"]; ok { |
||||||
|
data["fields.time"] = t |
||||||
|
} |
||||||
|
|
||||||
|
if m, ok := data["msg"]; ok { |
||||||
|
data["fields.msg"] = m |
||||||
|
} |
||||||
|
|
||||||
|
if l, ok := data["level"]; ok { |
||||||
|
data["fields.level"] = l |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,34 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
// A hook to be fired when logging on the logging levels returned from
|
||||||
|
// `Levels()` on your implementation of the interface. Note that this is not
|
||||||
|
// fired in a goroutine or a channel with workers, you should handle such
|
||||||
|
// functionality yourself if your call is non-blocking and you don't wish for
|
||||||
|
// the logging calls for levels returned from `Levels()` to block.
|
||||||
|
type Hook interface { |
||||||
|
Levels() []Level |
||||||
|
Fire(*Entry) error |
||||||
|
} |
||||||
|
|
||||||
|
// Internal type for storing the hooks on a logger instance.
|
||||||
|
type LevelHooks map[Level][]Hook |
||||||
|
|
||||||
|
// Add a hook to an instance of logger. This is called with
|
||||||
|
// `log.Hooks.Add(new(MyHook))` where `MyHook` implements the `Hook` interface.
|
||||||
|
func (hooks LevelHooks) Add(hook Hook) { |
||||||
|
for _, level := range hook.Levels() { |
||||||
|
hooks[level] = append(hooks[level], hook) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Fire all the hooks for the passed level. Used by `entry.log` to fire
|
||||||
|
// appropriate hooks for a log entry.
|
||||||
|
func (hooks LevelHooks) Fire(level Level, entry *Entry) error { |
||||||
|
for _, hook := range hooks[level] { |
||||||
|
if err := hook.Fire(entry); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
@ -0,0 +1,74 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"fmt" |
||||||
|
) |
||||||
|
|
||||||
|
type fieldKey string |
||||||
|
type FieldMap map[fieldKey]string |
||||||
|
|
||||||
|
const ( |
||||||
|
FieldKeyMsg = "msg" |
||||||
|
FieldKeyLevel = "level" |
||||||
|
FieldKeyTime = "time" |
||||||
|
) |
||||||
|
|
||||||
|
func (f FieldMap) resolve(key fieldKey) string { |
||||||
|
if k, ok := f[key]; ok { |
||||||
|
return k |
||||||
|
} |
||||||
|
|
||||||
|
return string(key) |
||||||
|
} |
||||||
|
|
||||||
|
type JSONFormatter struct { |
||||||
|
// TimestampFormat sets the format used for marshaling timestamps.
|
||||||
|
TimestampFormat string |
||||||
|
|
||||||
|
// DisableTimestamp allows disabling automatic timestamps in output
|
||||||
|
DisableTimestamp bool |
||||||
|
|
||||||
|
// FieldMap allows users to customize the names of keys for various fields.
|
||||||
|
// As an example:
|
||||||
|
// formatter := &JSONFormatter{
|
||||||
|
// FieldMap: FieldMap{
|
||||||
|
// FieldKeyTime: "@timestamp",
|
||||||
|
// FieldKeyLevel: "@level",
|
||||||
|
// FieldKeyLevel: "@message",
|
||||||
|
// },
|
||||||
|
// }
|
||||||
|
FieldMap FieldMap |
||||||
|
} |
||||||
|
|
||||||
|
func (f *JSONFormatter) Format(entry *Entry) ([]byte, error) { |
||||||
|
data := make(Fields, len(entry.Data)+3) |
||||||
|
for k, v := range entry.Data { |
||||||
|
switch v := v.(type) { |
||||||
|
case error: |
||||||
|
// Otherwise errors are ignored by `encoding/json`
|
||||||
|
// https://github.com/Sirupsen/logrus/issues/137
|
||||||
|
data[k] = v.Error() |
||||||
|
default: |
||||||
|
data[k] = v |
||||||
|
} |
||||||
|
} |
||||||
|
prefixFieldClashes(data) |
||||||
|
|
||||||
|
timestampFormat := f.TimestampFormat |
||||||
|
if timestampFormat == "" { |
||||||
|
timestampFormat = DefaultTimestampFormat |
||||||
|
} |
||||||
|
|
||||||
|
if !f.DisableTimestamp { |
||||||
|
data[f.FieldMap.resolve(FieldKeyTime)] = entry.Time.Format(timestampFormat) |
||||||
|
} |
||||||
|
data[f.FieldMap.resolve(FieldKeyMsg)] = entry.Message |
||||||
|
data[f.FieldMap.resolve(FieldKeyLevel)] = entry.Level.String() |
||||||
|
|
||||||
|
serialized, err := json.Marshal(data) |
||||||
|
if err != nil { |
||||||
|
return nil, fmt.Errorf("Failed to marshal fields to JSON, %v", err) |
||||||
|
} |
||||||
|
return append(serialized, '\n'), nil |
||||||
|
} |
||||||
@ -0,0 +1,308 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"io" |
||||||
|
"os" |
||||||
|
"sync" |
||||||
|
) |
||||||
|
|
||||||
|
type Logger struct { |
||||||
|
// The logs are `io.Copy`'d to this in a mutex. It's common to set this to a
|
||||||
|
// file, or leave it default which is `os.Stderr`. You can also set this to
|
||||||
|
// something more adventorous, such as logging to Kafka.
|
||||||
|
Out io.Writer |
||||||
|
// Hooks for the logger instance. These allow firing events based on logging
|
||||||
|
// levels and log entries. For example, to send errors to an error tracking
|
||||||
|
// service, log to StatsD or dump the core on fatal errors.
|
||||||
|
Hooks LevelHooks |
||||||
|
// All log entries pass through the formatter before logged to Out. The
|
||||||
|
// included formatters are `TextFormatter` and `JSONFormatter` for which
|
||||||
|
// TextFormatter is the default. In development (when a TTY is attached) it
|
||||||
|
// logs with colors, but to a file it wouldn't. You can easily implement your
|
||||||
|
// own that implements the `Formatter` interface, see the `README` or included
|
||||||
|
// formatters for examples.
|
||||||
|
Formatter Formatter |
||||||
|
// The logging level the logger should log at. This is typically (and defaults
|
||||||
|
// to) `logrus.Info`, which allows Info(), Warn(), Error() and Fatal() to be
|
||||||
|
// logged. `logrus.Debug` is useful in
|
||||||
|
Level Level |
||||||
|
// Used to sync writing to the log. Locking is enabled by Default
|
||||||
|
mu MutexWrap |
||||||
|
// Reusable empty entry
|
||||||
|
entryPool sync.Pool |
||||||
|
} |
||||||
|
|
||||||
|
type MutexWrap struct { |
||||||
|
lock sync.Mutex |
||||||
|
disabled bool |
||||||
|
} |
||||||
|
|
||||||
|
func (mw *MutexWrap) Lock() { |
||||||
|
if !mw.disabled { |
||||||
|
mw.lock.Lock() |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (mw *MutexWrap) Unlock() { |
||||||
|
if !mw.disabled { |
||||||
|
mw.lock.Unlock() |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (mw *MutexWrap) Disable() { |
||||||
|
mw.disabled = true |
||||||
|
} |
||||||
|
|
||||||
|
// Creates a new logger. Configuration should be set by changing `Formatter`,
|
||||||
|
// `Out` and `Hooks` directly on the default logger instance. You can also just
|
||||||
|
// instantiate your own:
|
||||||
|
//
|
||||||
|
// var log = &Logger{
|
||||||
|
// Out: os.Stderr,
|
||||||
|
// Formatter: new(JSONFormatter),
|
||||||
|
// Hooks: make(LevelHooks),
|
||||||
|
// Level: logrus.DebugLevel,
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// It's recommended to make this a global instance called `log`.
|
||||||
|
func New() *Logger { |
||||||
|
return &Logger{ |
||||||
|
Out: os.Stderr, |
||||||
|
Formatter: new(TextFormatter), |
||||||
|
Hooks: make(LevelHooks), |
||||||
|
Level: InfoLevel, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) newEntry() *Entry { |
||||||
|
entry, ok := logger.entryPool.Get().(*Entry) |
||||||
|
if ok { |
||||||
|
return entry |
||||||
|
} |
||||||
|
return NewEntry(logger) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) releaseEntry(entry *Entry) { |
||||||
|
logger.entryPool.Put(entry) |
||||||
|
} |
||||||
|
|
||||||
|
// Adds a field to the log entry, note that it doesn't log until you call
|
||||||
|
// Debug, Print, Info, Warn, Fatal or Panic. It only creates a log entry.
|
||||||
|
// If you want multiple fields, use `WithFields`.
|
||||||
|
func (logger *Logger) WithField(key string, value interface{}) *Entry { |
||||||
|
entry := logger.newEntry() |
||||||
|
defer logger.releaseEntry(entry) |
||||||
|
return entry.WithField(key, value) |
||||||
|
} |
||||||
|
|
||||||
|
// Adds a struct of fields to the log entry. All it does is call `WithField` for
|
||||||
|
// each `Field`.
|
||||||
|
func (logger *Logger) WithFields(fields Fields) *Entry { |
||||||
|
entry := logger.newEntry() |
||||||
|
defer logger.releaseEntry(entry) |
||||||
|
return entry.WithFields(fields) |
||||||
|
} |
||||||
|
|
||||||
|
// Add an error as single field to the log entry. All it does is call
|
||||||
|
// `WithError` for the given `error`.
|
||||||
|
func (logger *Logger) WithError(err error) *Entry { |
||||||
|
entry := logger.newEntry() |
||||||
|
defer logger.releaseEntry(entry) |
||||||
|
return entry.WithError(err) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Debugf(format string, args ...interface{}) { |
||||||
|
if logger.Level >= DebugLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Debugf(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Infof(format string, args ...interface{}) { |
||||||
|
if logger.Level >= InfoLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Infof(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Printf(format string, args ...interface{}) { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Printf(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Warnf(format string, args ...interface{}) { |
||||||
|
if logger.Level >= WarnLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Warnf(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Warningf(format string, args ...interface{}) { |
||||||
|
if logger.Level >= WarnLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Warnf(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Errorf(format string, args ...interface{}) { |
||||||
|
if logger.Level >= ErrorLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Errorf(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Fatalf(format string, args ...interface{}) { |
||||||
|
if logger.Level >= FatalLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Fatalf(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
Exit(1) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Panicf(format string, args ...interface{}) { |
||||||
|
if logger.Level >= PanicLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Panicf(format, args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Debug(args ...interface{}) { |
||||||
|
if logger.Level >= DebugLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Debug(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Info(args ...interface{}) { |
||||||
|
if logger.Level >= InfoLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Info(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Print(args ...interface{}) { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Info(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Warn(args ...interface{}) { |
||||||
|
if logger.Level >= WarnLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Warn(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Warning(args ...interface{}) { |
||||||
|
if logger.Level >= WarnLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Warn(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Error(args ...interface{}) { |
||||||
|
if logger.Level >= ErrorLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Error(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Fatal(args ...interface{}) { |
||||||
|
if logger.Level >= FatalLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Fatal(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
Exit(1) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Panic(args ...interface{}) { |
||||||
|
if logger.Level >= PanicLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Panic(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Debugln(args ...interface{}) { |
||||||
|
if logger.Level >= DebugLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Debugln(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Infoln(args ...interface{}) { |
||||||
|
if logger.Level >= InfoLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Infoln(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Println(args ...interface{}) { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Println(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Warnln(args ...interface{}) { |
||||||
|
if logger.Level >= WarnLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Warnln(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Warningln(args ...interface{}) { |
||||||
|
if logger.Level >= WarnLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Warnln(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Errorln(args ...interface{}) { |
||||||
|
if logger.Level >= ErrorLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Errorln(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Fatalln(args ...interface{}) { |
||||||
|
if logger.Level >= FatalLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Fatalln(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
Exit(1) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) Panicln(args ...interface{}) { |
||||||
|
if logger.Level >= PanicLevel { |
||||||
|
entry := logger.newEntry() |
||||||
|
entry.Panicln(args...) |
||||||
|
logger.releaseEntry(entry) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
//When file is opened with appending mode, it's safe to
|
||||||
|
//write concurrently to a file (within 4k message on Linux).
|
||||||
|
//In these cases user can choose to disable the lock.
|
||||||
|
func (logger *Logger) SetNoLock() { |
||||||
|
logger.mu.Disable() |
||||||
|
} |
||||||
@ -0,0 +1,143 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"log" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
// Fields type, used to pass to `WithFields`.
|
||||||
|
type Fields map[string]interface{} |
||||||
|
|
||||||
|
// Level type
|
||||||
|
type Level uint8 |
||||||
|
|
||||||
|
// Convert the Level to a string. E.g. PanicLevel becomes "panic".
|
||||||
|
func (level Level) String() string { |
||||||
|
switch level { |
||||||
|
case DebugLevel: |
||||||
|
return "debug" |
||||||
|
case InfoLevel: |
||||||
|
return "info" |
||||||
|
case WarnLevel: |
||||||
|
return "warning" |
||||||
|
case ErrorLevel: |
||||||
|
return "error" |
||||||
|
case FatalLevel: |
||||||
|
return "fatal" |
||||||
|
case PanicLevel: |
||||||
|
return "panic" |
||||||
|
} |
||||||
|
|
||||||
|
return "unknown" |
||||||
|
} |
||||||
|
|
||||||
|
// ParseLevel takes a string level and returns the Logrus log level constant.
|
||||||
|
func ParseLevel(lvl string) (Level, error) { |
||||||
|
switch strings.ToLower(lvl) { |
||||||
|
case "panic": |
||||||
|
return PanicLevel, nil |
||||||
|
case "fatal": |
||||||
|
return FatalLevel, nil |
||||||
|
case "error": |
||||||
|
return ErrorLevel, nil |
||||||
|
case "warn", "warning": |
||||||
|
return WarnLevel, nil |
||||||
|
case "info": |
||||||
|
return InfoLevel, nil |
||||||
|
case "debug": |
||||||
|
return DebugLevel, nil |
||||||
|
} |
||||||
|
|
||||||
|
var l Level |
||||||
|
return l, fmt.Errorf("not a valid logrus Level: %q", lvl) |
||||||
|
} |
||||||
|
|
||||||
|
// A constant exposing all logging levels
|
||||||
|
var AllLevels = []Level{ |
||||||
|
PanicLevel, |
||||||
|
FatalLevel, |
||||||
|
ErrorLevel, |
||||||
|
WarnLevel, |
||||||
|
InfoLevel, |
||||||
|
DebugLevel, |
||||||
|
} |
||||||
|
|
||||||
|
// These are the different logging levels. You can set the logging level to log
|
||||||
|
// on your instance of logger, obtained with `logrus.New()`.
|
||||||
|
const ( |
||||||
|
// PanicLevel level, highest level of severity. Logs and then calls panic with the
|
||||||
|
// message passed to Debug, Info, ...
|
||||||
|
PanicLevel Level = iota |
||||||
|
// FatalLevel level. Logs and then calls `os.Exit(1)`. It will exit even if the
|
||||||
|
// logging level is set to Panic.
|
||||||
|
FatalLevel |
||||||
|
// ErrorLevel level. Logs. Used for errors that should definitely be noted.
|
||||||
|
// Commonly used for hooks to send errors to an error tracking service.
|
||||||
|
ErrorLevel |
||||||
|
// WarnLevel level. Non-critical entries that deserve eyes.
|
||||||
|
WarnLevel |
||||||
|
// InfoLevel level. General operational entries about what's going on inside the
|
||||||
|
// application.
|
||||||
|
InfoLevel |
||||||
|
// DebugLevel level. Usually only enabled when debugging. Very verbose logging.
|
||||||
|
DebugLevel |
||||||
|
) |
||||||
|
|
||||||
|
// Won't compile if StdLogger can't be realized by a log.Logger
|
||||||
|
var ( |
||||||
|
_ StdLogger = &log.Logger{} |
||||||
|
_ StdLogger = &Entry{} |
||||||
|
_ StdLogger = &Logger{} |
||||||
|
) |
||||||
|
|
||||||
|
// StdLogger is what your logrus-enabled library should take, that way
|
||||||
|
// it'll accept a stdlib logger and a logrus logger. There's no standard
|
||||||
|
// interface, this is the closest we get, unfortunately.
|
||||||
|
type StdLogger interface { |
||||||
|
Print(...interface{}) |
||||||
|
Printf(string, ...interface{}) |
||||||
|
Println(...interface{}) |
||||||
|
|
||||||
|
Fatal(...interface{}) |
||||||
|
Fatalf(string, ...interface{}) |
||||||
|
Fatalln(...interface{}) |
||||||
|
|
||||||
|
Panic(...interface{}) |
||||||
|
Panicf(string, ...interface{}) |
||||||
|
Panicln(...interface{}) |
||||||
|
} |
||||||
|
|
||||||
|
// The FieldLogger interface generalizes the Entry and Logger types
|
||||||
|
type FieldLogger interface { |
||||||
|
WithField(key string, value interface{}) *Entry |
||||||
|
WithFields(fields Fields) *Entry |
||||||
|
WithError(err error) *Entry |
||||||
|
|
||||||
|
Debugf(format string, args ...interface{}) |
||||||
|
Infof(format string, args ...interface{}) |
||||||
|
Printf(format string, args ...interface{}) |
||||||
|
Warnf(format string, args ...interface{}) |
||||||
|
Warningf(format string, args ...interface{}) |
||||||
|
Errorf(format string, args ...interface{}) |
||||||
|
Fatalf(format string, args ...interface{}) |
||||||
|
Panicf(format string, args ...interface{}) |
||||||
|
|
||||||
|
Debug(args ...interface{}) |
||||||
|
Info(args ...interface{}) |
||||||
|
Print(args ...interface{}) |
||||||
|
Warn(args ...interface{}) |
||||||
|
Warning(args ...interface{}) |
||||||
|
Error(args ...interface{}) |
||||||
|
Fatal(args ...interface{}) |
||||||
|
Panic(args ...interface{}) |
||||||
|
|
||||||
|
Debugln(args ...interface{}) |
||||||
|
Infoln(args ...interface{}) |
||||||
|
Println(args ...interface{}) |
||||||
|
Warnln(args ...interface{}) |
||||||
|
Warningln(args ...interface{}) |
||||||
|
Errorln(args ...interface{}) |
||||||
|
Fatalln(args ...interface{}) |
||||||
|
Panicln(args ...interface{}) |
||||||
|
} |
||||||
@ -0,0 +1,10 @@ |
|||||||
|
// +build appengine
|
||||||
|
|
||||||
|
package logrus |
||||||
|
|
||||||
|
import "io" |
||||||
|
|
||||||
|
// IsTerminal returns true if stderr's file descriptor is a terminal.
|
||||||
|
func IsTerminal(f io.Writer) bool { |
||||||
|
return true |
||||||
|
} |
||||||
@ -0,0 +1,10 @@ |
|||||||
|
// +build darwin freebsd openbsd netbsd dragonfly
|
||||||
|
// +build !appengine
|
||||||
|
|
||||||
|
package logrus |
||||||
|
|
||||||
|
import "syscall" |
||||||
|
|
||||||
|
const ioctlReadTermios = syscall.TIOCGETA |
||||||
|
|
||||||
|
type Termios syscall.Termios |
||||||
@ -0,0 +1,14 @@ |
|||||||
|
// Based on ssh/terminal:
|
||||||
|
// Copyright 2013 The Go Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a BSD-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
// +build !appengine
|
||||||
|
|
||||||
|
package logrus |
||||||
|
|
||||||
|
import "syscall" |
||||||
|
|
||||||
|
const ioctlReadTermios = syscall.TCGETS |
||||||
|
|
||||||
|
type Termios syscall.Termios |
||||||
@ -0,0 +1,28 @@ |
|||||||
|
// Based on ssh/terminal:
|
||||||
|
// Copyright 2011 The Go Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a BSD-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
// +build linux darwin freebsd openbsd netbsd dragonfly
|
||||||
|
// +build !appengine
|
||||||
|
|
||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"io" |
||||||
|
"os" |
||||||
|
"syscall" |
||||||
|
"unsafe" |
||||||
|
) |
||||||
|
|
||||||
|
// IsTerminal returns true if stderr's file descriptor is a terminal.
|
||||||
|
func IsTerminal(f io.Writer) bool { |
||||||
|
var termios Termios |
||||||
|
switch v := f.(type) { |
||||||
|
case *os.File: |
||||||
|
_, _, err := syscall.Syscall6(syscall.SYS_IOCTL, uintptr(v.Fd()), ioctlReadTermios, uintptr(unsafe.Pointer(&termios)), 0, 0, 0) |
||||||
|
return err == 0 |
||||||
|
default: |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,21 @@ |
|||||||
|
// +build solaris,!appengine
|
||||||
|
|
||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"io" |
||||||
|
"os" |
||||||
|
|
||||||
|
"golang.org/x/sys/unix" |
||||||
|
) |
||||||
|
|
||||||
|
// IsTerminal returns true if the given file descriptor is a terminal.
|
||||||
|
func IsTerminal(f io.Writer) bool { |
||||||
|
switch v := f.(type) { |
||||||
|
case *os.File: |
||||||
|
_, err := unix.IoctlGetTermios(int(v.Fd()), unix.TCGETA) |
||||||
|
return err == nil |
||||||
|
default: |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,33 @@ |
|||||||
|
// Based on ssh/terminal:
|
||||||
|
// Copyright 2011 The Go Authors. All rights reserved.
|
||||||
|
// Use of this source code is governed by a BSD-style
|
||||||
|
// license that can be found in the LICENSE file.
|
||||||
|
|
||||||
|
// +build windows,!appengine
|
||||||
|
|
||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"io" |
||||||
|
"os" |
||||||
|
"syscall" |
||||||
|
"unsafe" |
||||||
|
) |
||||||
|
|
||||||
|
var kernel32 = syscall.NewLazyDLL("kernel32.dll") |
||||||
|
|
||||||
|
var ( |
||||||
|
procGetConsoleMode = kernel32.NewProc("GetConsoleMode") |
||||||
|
) |
||||||
|
|
||||||
|
// IsTerminal returns true if stderr's file descriptor is a terminal.
|
||||||
|
func IsTerminal(f io.Writer) bool { |
||||||
|
switch v := f.(type) { |
||||||
|
case *os.File: |
||||||
|
var st uint32 |
||||||
|
r, _, e := syscall.Syscall(procGetConsoleMode.Addr(), 2, uintptr(v.Fd()), uintptr(unsafe.Pointer(&st)), 0) |
||||||
|
return r != 0 && e == 0 |
||||||
|
default: |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,189 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"fmt" |
||||||
|
"sort" |
||||||
|
"strings" |
||||||
|
"sync" |
||||||
|
"time" |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
nocolor = 0 |
||||||
|
red = 31 |
||||||
|
green = 32 |
||||||
|
yellow = 33 |
||||||
|
blue = 34 |
||||||
|
gray = 37 |
||||||
|
) |
||||||
|
|
||||||
|
var ( |
||||||
|
baseTimestamp time.Time |
||||||
|
) |
||||||
|
|
||||||
|
func init() { |
||||||
|
baseTimestamp = time.Now() |
||||||
|
} |
||||||
|
|
||||||
|
type TextFormatter struct { |
||||||
|
// Set to true to bypass checking for a TTY before outputting colors.
|
||||||
|
ForceColors bool |
||||||
|
|
||||||
|
// Force disabling colors.
|
||||||
|
DisableColors bool |
||||||
|
|
||||||
|
// Disable timestamp logging. useful when output is redirected to logging
|
||||||
|
// system that already adds timestamps.
|
||||||
|
DisableTimestamp bool |
||||||
|
|
||||||
|
// Enable logging the full timestamp when a TTY is attached instead of just
|
||||||
|
// the time passed since beginning of execution.
|
||||||
|
FullTimestamp bool |
||||||
|
|
||||||
|
// TimestampFormat to use for display when a full timestamp is printed
|
||||||
|
TimestampFormat string |
||||||
|
|
||||||
|
// The fields are sorted by default for a consistent output. For applications
|
||||||
|
// that log extremely frequently and don't use the JSON formatter this may not
|
||||||
|
// be desired.
|
||||||
|
DisableSorting bool |
||||||
|
|
||||||
|
// QuoteEmptyFields will wrap empty fields in quotes if true
|
||||||
|
QuoteEmptyFields bool |
||||||
|
|
||||||
|
// QuoteCharacter can be set to the override the default quoting character "
|
||||||
|
// with something else. For example: ', or `.
|
||||||
|
QuoteCharacter string |
||||||
|
|
||||||
|
// Whether the logger's out is to a terminal
|
||||||
|
isTerminal bool |
||||||
|
|
||||||
|
sync.Once |
||||||
|
} |
||||||
|
|
||||||
|
func (f *TextFormatter) init(entry *Entry) { |
||||||
|
if len(f.QuoteCharacter) == 0 { |
||||||
|
f.QuoteCharacter = "\"" |
||||||
|
} |
||||||
|
if entry.Logger != nil { |
||||||
|
f.isTerminal = IsTerminal(entry.Logger.Out) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (f *TextFormatter) Format(entry *Entry) ([]byte, error) { |
||||||
|
var b *bytes.Buffer |
||||||
|
keys := make([]string, 0, len(entry.Data)) |
||||||
|
for k := range entry.Data { |
||||||
|
keys = append(keys, k) |
||||||
|
} |
||||||
|
|
||||||
|
if !f.DisableSorting { |
||||||
|
sort.Strings(keys) |
||||||
|
} |
||||||
|
if entry.Buffer != nil { |
||||||
|
b = entry.Buffer |
||||||
|
} else { |
||||||
|
b = &bytes.Buffer{} |
||||||
|
} |
||||||
|
|
||||||
|
prefixFieldClashes(entry.Data) |
||||||
|
|
||||||
|
f.Do(func() { f.init(entry) }) |
||||||
|
|
||||||
|
isColored := (f.ForceColors || f.isTerminal) && !f.DisableColors |
||||||
|
|
||||||
|
timestampFormat := f.TimestampFormat |
||||||
|
if timestampFormat == "" { |
||||||
|
timestampFormat = DefaultTimestampFormat |
||||||
|
} |
||||||
|
if isColored { |
||||||
|
f.printColored(b, entry, keys, timestampFormat) |
||||||
|
} else { |
||||||
|
if !f.DisableTimestamp { |
||||||
|
f.appendKeyValue(b, "time", entry.Time.Format(timestampFormat)) |
||||||
|
} |
||||||
|
f.appendKeyValue(b, "level", entry.Level.String()) |
||||||
|
if entry.Message != "" { |
||||||
|
f.appendKeyValue(b, "msg", entry.Message) |
||||||
|
} |
||||||
|
for _, key := range keys { |
||||||
|
f.appendKeyValue(b, key, entry.Data[key]) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
b.WriteByte('\n') |
||||||
|
return b.Bytes(), nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f *TextFormatter) printColored(b *bytes.Buffer, entry *Entry, keys []string, timestampFormat string) { |
||||||
|
var levelColor int |
||||||
|
switch entry.Level { |
||||||
|
case DebugLevel: |
||||||
|
levelColor = gray |
||||||
|
case WarnLevel: |
||||||
|
levelColor = yellow |
||||||
|
case ErrorLevel, FatalLevel, PanicLevel: |
||||||
|
levelColor = red |
||||||
|
default: |
||||||
|
levelColor = blue |
||||||
|
} |
||||||
|
|
||||||
|
levelText := strings.ToUpper(entry.Level.String())[0:4] |
||||||
|
|
||||||
|
if f.DisableTimestamp { |
||||||
|
fmt.Fprintf(b, "\x1b[%dm%s\x1b[0m %-44s ", levelColor, levelText, entry.Message) |
||||||
|
} else if !f.FullTimestamp { |
||||||
|
fmt.Fprintf(b, "\x1b[%dm%s\x1b[0m[%04d] %-44s ", levelColor, levelText, int(entry.Time.Sub(baseTimestamp)/time.Second), entry.Message) |
||||||
|
} else { |
||||||
|
fmt.Fprintf(b, "\x1b[%dm%s\x1b[0m[%s] %-44s ", levelColor, levelText, entry.Time.Format(timestampFormat), entry.Message) |
||||||
|
} |
||||||
|
for _, k := range keys { |
||||||
|
v := entry.Data[k] |
||||||
|
fmt.Fprintf(b, " \x1b[%dm%s\x1b[0m=", levelColor, k) |
||||||
|
f.appendValue(b, v) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (f *TextFormatter) needsQuoting(text string) bool { |
||||||
|
if f.QuoteEmptyFields && len(text) == 0 { |
||||||
|
return true |
||||||
|
} |
||||||
|
for _, ch := range text { |
||||||
|
if !((ch >= 'a' && ch <= 'z') || |
||||||
|
(ch >= 'A' && ch <= 'Z') || |
||||||
|
(ch >= '0' && ch <= '9') || |
||||||
|
ch == '-' || ch == '.') { |
||||||
|
return true |
||||||
|
} |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
func (f *TextFormatter) appendKeyValue(b *bytes.Buffer, key string, value interface{}) { |
||||||
|
|
||||||
|
b.WriteString(key) |
||||||
|
b.WriteByte('=') |
||||||
|
f.appendValue(b, value) |
||||||
|
b.WriteByte(' ') |
||||||
|
} |
||||||
|
|
||||||
|
func (f *TextFormatter) appendValue(b *bytes.Buffer, value interface{}) { |
||||||
|
switch value := value.(type) { |
||||||
|
case string: |
||||||
|
if !f.needsQuoting(value) { |
||||||
|
b.WriteString(value) |
||||||
|
} else { |
||||||
|
fmt.Fprintf(b, "%s%v%s", f.QuoteCharacter, value, f.QuoteCharacter) |
||||||
|
} |
||||||
|
case error: |
||||||
|
errmsg := value.Error() |
||||||
|
if !f.needsQuoting(errmsg) { |
||||||
|
b.WriteString(errmsg) |
||||||
|
} else { |
||||||
|
fmt.Fprintf(b, "%s%v%s", f.QuoteCharacter, errmsg, f.QuoteCharacter) |
||||||
|
} |
||||||
|
default: |
||||||
|
fmt.Fprint(b, value) |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,62 @@ |
|||||||
|
package logrus |
||||||
|
|
||||||
|
import ( |
||||||
|
"bufio" |
||||||
|
"io" |
||||||
|
"runtime" |
||||||
|
) |
||||||
|
|
||||||
|
func (logger *Logger) Writer() *io.PipeWriter { |
||||||
|
return logger.WriterLevel(InfoLevel) |
||||||
|
} |
||||||
|
|
||||||
|
func (logger *Logger) WriterLevel(level Level) *io.PipeWriter { |
||||||
|
return NewEntry(logger).WriterLevel(level) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) Writer() *io.PipeWriter { |
||||||
|
return entry.WriterLevel(InfoLevel) |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) WriterLevel(level Level) *io.PipeWriter { |
||||||
|
reader, writer := io.Pipe() |
||||||
|
|
||||||
|
var printFunc func(args ...interface{}) |
||||||
|
|
||||||
|
switch level { |
||||||
|
case DebugLevel: |
||||||
|
printFunc = entry.Debug |
||||||
|
case InfoLevel: |
||||||
|
printFunc = entry.Info |
||||||
|
case WarnLevel: |
||||||
|
printFunc = entry.Warn |
||||||
|
case ErrorLevel: |
||||||
|
printFunc = entry.Error |
||||||
|
case FatalLevel: |
||||||
|
printFunc = entry.Fatal |
||||||
|
case PanicLevel: |
||||||
|
printFunc = entry.Panic |
||||||
|
default: |
||||||
|
printFunc = entry.Print |
||||||
|
} |
||||||
|
|
||||||
|
go entry.writerScanner(reader, printFunc) |
||||||
|
runtime.SetFinalizer(writer, writerFinalizer) |
||||||
|
|
||||||
|
return writer |
||||||
|
} |
||||||
|
|
||||||
|
func (entry *Entry) writerScanner(reader *io.PipeReader, printFunc func(args ...interface{})) { |
||||||
|
scanner := bufio.NewScanner(reader) |
||||||
|
for scanner.Scan() { |
||||||
|
printFunc(scanner.Text()) |
||||||
|
} |
||||||
|
if err := scanner.Err(); err != nil { |
||||||
|
entry.Errorf("Error while reading from Writer: %s", err) |
||||||
|
} |
||||||
|
reader.Close() |
||||||
|
} |
||||||
|
|
||||||
|
func writerFinalizer(writer *io.PipeWriter) { |
||||||
|
writer.Close() |
||||||
|
} |
||||||
@ -0,0 +1,20 @@ |
|||||||
|
Copyright (C) 2013 Blake Mizerany |
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining |
||||||
|
a copy of this software and associated documentation files (the |
||||||
|
"Software"), to deal in the Software without restriction, including |
||||||
|
without limitation the rights to use, copy, modify, merge, publish, |
||||||
|
distribute, sublicense, and/or sell copies of the Software, and to |
||||||
|
permit persons to whom the Software is furnished to do so, subject to |
||||||
|
the following conditions: |
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be |
||||||
|
included in all copies or substantial portions of the Software. |
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, |
||||||
|
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF |
||||||
|
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND |
||||||
|
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE |
||||||
|
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION |
||||||
|
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION |
||||||
|
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
||||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,292 @@ |
|||||||
|
// Package quantile computes approximate quantiles over an unbounded data
|
||||||
|
// stream within low memory and CPU bounds.
|
||||||
|
//
|
||||||
|
// A small amount of accuracy is traded to achieve the above properties.
|
||||||
|
//
|
||||||
|
// Multiple streams can be merged before calling Query to generate a single set
|
||||||
|
// of results. This is meaningful when the streams represent the same type of
|
||||||
|
// data. See Merge and Samples.
|
||||||
|
//
|
||||||
|
// For more detailed information about the algorithm used, see:
|
||||||
|
//
|
||||||
|
// Effective Computation of Biased Quantiles over Data Streams
|
||||||
|
//
|
||||||
|
// http://www.cs.rutgers.edu/~muthu/bquant.pdf
|
||||||
|
package quantile |
||||||
|
|
||||||
|
import ( |
||||||
|
"math" |
||||||
|
"sort" |
||||||
|
) |
||||||
|
|
||||||
|
// Sample holds an observed value and meta information for compression. JSON
|
||||||
|
// tags have been added for convenience.
|
||||||
|
type Sample struct { |
||||||
|
Value float64 `json:",string"` |
||||||
|
Width float64 `json:",string"` |
||||||
|
Delta float64 `json:",string"` |
||||||
|
} |
||||||
|
|
||||||
|
// Samples represents a slice of samples. It implements sort.Interface.
|
||||||
|
type Samples []Sample |
||||||
|
|
||||||
|
func (a Samples) Len() int { return len(a) } |
||||||
|
func (a Samples) Less(i, j int) bool { return a[i].Value < a[j].Value } |
||||||
|
func (a Samples) Swap(i, j int) { a[i], a[j] = a[j], a[i] } |
||||||
|
|
||||||
|
type invariant func(s *stream, r float64) float64 |
||||||
|
|
||||||
|
// NewLowBiased returns an initialized Stream for low-biased quantiles
|
||||||
|
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||||
|
// error guarantees can still be given even for the lower ranks of the data
|
||||||
|
// distribution.
|
||||||
|
//
|
||||||
|
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||||
|
// returned by a query is guaranteed to be within (1±Epsilon)*Quantile.
|
||||||
|
//
|
||||||
|
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||||
|
// properties.
|
||||||
|
func NewLowBiased(epsilon float64) *Stream { |
||||||
|
ƒ := func(s *stream, r float64) float64 { |
||||||
|
return 2 * epsilon * r |
||||||
|
} |
||||||
|
return newStream(ƒ) |
||||||
|
} |
||||||
|
|
||||||
|
// NewHighBiased returns an initialized Stream for high-biased quantiles
|
||||||
|
// (e.g. 0.01, 0.1, 0.5) where the needed quantiles are not known a priori, but
|
||||||
|
// error guarantees can still be given even for the higher ranks of the data
|
||||||
|
// distribution.
|
||||||
|
//
|
||||||
|
// The provided epsilon is a relative error, i.e. the true quantile of a value
|
||||||
|
// returned by a query is guaranteed to be within 1-(1±Epsilon)*(1-Quantile).
|
||||||
|
//
|
||||||
|
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error
|
||||||
|
// properties.
|
||||||
|
func NewHighBiased(epsilon float64) *Stream { |
||||||
|
ƒ := func(s *stream, r float64) float64 { |
||||||
|
return 2 * epsilon * (s.n - r) |
||||||
|
} |
||||||
|
return newStream(ƒ) |
||||||
|
} |
||||||
|
|
||||||
|
// NewTargeted returns an initialized Stream concerned with a particular set of
|
||||||
|
// quantile values that are supplied a priori. Knowing these a priori reduces
|
||||||
|
// space and computation time. The targets map maps the desired quantiles to
|
||||||
|
// their absolute errors, i.e. the true quantile of a value returned by a query
|
||||||
|
// is guaranteed to be within (Quantile±Epsilon).
|
||||||
|
//
|
||||||
|
// See http://www.cs.rutgers.edu/~muthu/bquant.pdf for time, space, and error properties.
|
||||||
|
func NewTargeted(targets map[float64]float64) *Stream { |
||||||
|
ƒ := func(s *stream, r float64) float64 { |
||||||
|
var m = math.MaxFloat64 |
||||||
|
var f float64 |
||||||
|
for quantile, epsilon := range targets { |
||||||
|
if quantile*s.n <= r { |
||||||
|
f = (2 * epsilon * r) / quantile |
||||||
|
} else { |
||||||
|
f = (2 * epsilon * (s.n - r)) / (1 - quantile) |
||||||
|
} |
||||||
|
if f < m { |
||||||
|
m = f |
||||||
|
} |
||||||
|
} |
||||||
|
return m |
||||||
|
} |
||||||
|
return newStream(ƒ) |
||||||
|
} |
||||||
|
|
||||||
|
// Stream computes quantiles for a stream of float64s. It is not thread-safe by
|
||||||
|
// design. Take care when using across multiple goroutines.
|
||||||
|
type Stream struct { |
||||||
|
*stream |
||||||
|
b Samples |
||||||
|
sorted bool |
||||||
|
} |
||||||
|
|
||||||
|
func newStream(ƒ invariant) *Stream { |
||||||
|
x := &stream{ƒ: ƒ} |
||||||
|
return &Stream{x, make(Samples, 0, 500), true} |
||||||
|
} |
||||||
|
|
||||||
|
// Insert inserts v into the stream.
|
||||||
|
func (s *Stream) Insert(v float64) { |
||||||
|
s.insert(Sample{Value: v, Width: 1}) |
||||||
|
} |
||||||
|
|
||||||
|
func (s *Stream) insert(sample Sample) { |
||||||
|
s.b = append(s.b, sample) |
||||||
|
s.sorted = false |
||||||
|
if len(s.b) == cap(s.b) { |
||||||
|
s.flush() |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Query returns the computed qth percentiles value. If s was created with
|
||||||
|
// NewTargeted, and q is not in the set of quantiles provided a priori, Query
|
||||||
|
// will return an unspecified result.
|
||||||
|
func (s *Stream) Query(q float64) float64 { |
||||||
|
if !s.flushed() { |
||||||
|
// Fast path when there hasn't been enough data for a flush;
|
||||||
|
// this also yields better accuracy for small sets of data.
|
||||||
|
l := len(s.b) |
||||||
|
if l == 0 { |
||||||
|
return 0 |
||||||
|
} |
||||||
|
i := int(math.Ceil(float64(l) * q)) |
||||||
|
if i > 0 { |
||||||
|
i -= 1 |
||||||
|
} |
||||||
|
s.maybeSort() |
||||||
|
return s.b[i].Value |
||||||
|
} |
||||||
|
s.flush() |
||||||
|
return s.stream.query(q) |
||||||
|
} |
||||||
|
|
||||||
|
// Merge merges samples into the underlying streams samples. This is handy when
|
||||||
|
// merging multiple streams from separate threads, database shards, etc.
|
||||||
|
//
|
||||||
|
// ATTENTION: This method is broken and does not yield correct results. The
|
||||||
|
// underlying algorithm is not capable of merging streams correctly.
|
||||||
|
func (s *Stream) Merge(samples Samples) { |
||||||
|
sort.Sort(samples) |
||||||
|
s.stream.merge(samples) |
||||||
|
} |
||||||
|
|
||||||
|
// Reset reinitializes and clears the list reusing the samples buffer memory.
|
||||||
|
func (s *Stream) Reset() { |
||||||
|
s.stream.reset() |
||||||
|
s.b = s.b[:0] |
||||||
|
} |
||||||
|
|
||||||
|
// Samples returns stream samples held by s.
|
||||||
|
func (s *Stream) Samples() Samples { |
||||||
|
if !s.flushed() { |
||||||
|
return s.b |
||||||
|
} |
||||||
|
s.flush() |
||||||
|
return s.stream.samples() |
||||||
|
} |
||||||
|
|
||||||
|
// Count returns the total number of samples observed in the stream
|
||||||
|
// since initialization.
|
||||||
|
func (s *Stream) Count() int { |
||||||
|
return len(s.b) + s.stream.count() |
||||||
|
} |
||||||
|
|
||||||
|
func (s *Stream) flush() { |
||||||
|
s.maybeSort() |
||||||
|
s.stream.merge(s.b) |
||||||
|
s.b = s.b[:0] |
||||||
|
} |
||||||
|
|
||||||
|
func (s *Stream) maybeSort() { |
||||||
|
if !s.sorted { |
||||||
|
s.sorted = true |
||||||
|
sort.Sort(s.b) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (s *Stream) flushed() bool { |
||||||
|
return len(s.stream.l) > 0 |
||||||
|
} |
||||||
|
|
||||||
|
type stream struct { |
||||||
|
n float64 |
||||||
|
l []Sample |
||||||
|
ƒ invariant |
||||||
|
} |
||||||
|
|
||||||
|
func (s *stream) reset() { |
||||||
|
s.l = s.l[:0] |
||||||
|
s.n = 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (s *stream) insert(v float64) { |
||||||
|
s.merge(Samples{{v, 1, 0}}) |
||||||
|
} |
||||||
|
|
||||||
|
func (s *stream) merge(samples Samples) { |
||||||
|
// TODO(beorn7): This tries to merge not only individual samples, but
|
||||||
|
// whole summaries. The paper doesn't mention merging summaries at
|
||||||
|
// all. Unittests show that the merging is inaccurate. Find out how to
|
||||||
|
// do merges properly.
|
||||||
|
var r float64 |
||||||
|
i := 0 |
||||||
|
for _, sample := range samples { |
||||||
|
for ; i < len(s.l); i++ { |
||||||
|
c := s.l[i] |
||||||
|
if c.Value > sample.Value { |
||||||
|
// Insert at position i.
|
||||||
|
s.l = append(s.l, Sample{}) |
||||||
|
copy(s.l[i+1:], s.l[i:]) |
||||||
|
s.l[i] = Sample{ |
||||||
|
sample.Value, |
||||||
|
sample.Width, |
||||||
|
math.Max(sample.Delta, math.Floor(s.ƒ(s, r))-1), |
||||||
|
// TODO(beorn7): How to calculate delta correctly?
|
||||||
|
} |
||||||
|
i++ |
||||||
|
goto inserted |
||||||
|
} |
||||||
|
r += c.Width |
||||||
|
} |
||||||
|
s.l = append(s.l, Sample{sample.Value, sample.Width, 0}) |
||||||
|
i++ |
||||||
|
inserted: |
||||||
|
s.n += sample.Width |
||||||
|
r += sample.Width |
||||||
|
} |
||||||
|
s.compress() |
||||||
|
} |
||||||
|
|
||||||
|
func (s *stream) count() int { |
||||||
|
return int(s.n) |
||||||
|
} |
||||||
|
|
||||||
|
func (s *stream) query(q float64) float64 { |
||||||
|
t := math.Ceil(q * s.n) |
||||||
|
t += math.Ceil(s.ƒ(s, t) / 2) |
||||||
|
p := s.l[0] |
||||||
|
var r float64 |
||||||
|
for _, c := range s.l[1:] { |
||||||
|
r += p.Width |
||||||
|
if r+c.Width+c.Delta > t { |
||||||
|
return p.Value |
||||||
|
} |
||||||
|
p = c |
||||||
|
} |
||||||
|
return p.Value |
||||||
|
} |
||||||
|
|
||||||
|
func (s *stream) compress() { |
||||||
|
if len(s.l) < 2 { |
||||||
|
return |
||||||
|
} |
||||||
|
x := s.l[len(s.l)-1] |
||||||
|
xi := len(s.l) - 1 |
||||||
|
r := s.n - 1 - x.Width |
||||||
|
|
||||||
|
for i := len(s.l) - 2; i >= 0; i-- { |
||||||
|
c := s.l[i] |
||||||
|
if c.Width+x.Width+x.Delta <= s.ƒ(s, r) { |
||||||
|
x.Width += c.Width |
||||||
|
s.l[xi] = x |
||||||
|
// Remove element at i.
|
||||||
|
copy(s.l[i:], s.l[i+1:]) |
||||||
|
s.l = s.l[:len(s.l)-1] |
||||||
|
xi -= 1 |
||||||
|
} else { |
||||||
|
x = c |
||||||
|
xi = i |
||||||
|
} |
||||||
|
r -= c.Width |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (s *stream) samples() Samples { |
||||||
|
samples := make(Samples, len(s.l)) |
||||||
|
copy(samples, s.l) |
||||||
|
return samples |
||||||
|
} |
||||||
@ -0,0 +1,31 @@ |
|||||||
|
Go support for Protocol Buffers - Google's data interchange format |
||||||
|
|
||||||
|
Copyright 2010 The Go Authors. All rights reserved. |
||||||
|
https://github.com/golang/protobuf |
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without |
||||||
|
modification, are permitted provided that the following conditions are |
||||||
|
met: |
||||||
|
|
||||||
|
* Redistributions of source code must retain the above copyright |
||||||
|
notice, this list of conditions and the following disclaimer. |
||||||
|
* Redistributions in binary form must reproduce the above |
||||||
|
copyright notice, this list of conditions and the following disclaimer |
||||||
|
in the documentation and/or other materials provided with the |
||||||
|
distribution. |
||||||
|
* Neither the name of Google Inc. nor the names of its |
||||||
|
contributors may be used to endorse or promote products derived from |
||||||
|
this software without specific prior written permission. |
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||||
|
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||||
|
|
||||||
@ -0,0 +1,43 @@ |
|||||||
|
# Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
#
|
||||||
|
# Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
# https://github.com/golang/protobuf
|
||||||
|
#
|
||||||
|
# Redistribution and use in source and binary forms, with or without
|
||||||
|
# modification, are permitted provided that the following conditions are
|
||||||
|
# met:
|
||||||
|
#
|
||||||
|
# * Redistributions of source code must retain the above copyright
|
||||||
|
# notice, this list of conditions and the following disclaimer.
|
||||||
|
# * Redistributions in binary form must reproduce the above
|
||||||
|
# copyright notice, this list of conditions and the following disclaimer
|
||||||
|
# in the documentation and/or other materials provided with the
|
||||||
|
# distribution.
|
||||||
|
# * Neither the name of Google Inc. nor the names of its
|
||||||
|
# contributors may be used to endorse or promote products derived from
|
||||||
|
# this software without specific prior written permission.
|
||||||
|
#
|
||||||
|
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
install: |
||||||
|
go install
|
||||||
|
|
||||||
|
test: install generate-test-pbs |
||||||
|
go test
|
||||||
|
|
||||||
|
|
||||||
|
generate-test-pbs: |
||||||
|
make install
|
||||||
|
make -C testdata
|
||||||
|
protoc --go_out=Mtestdata/test.proto=github.com/golang/protobuf/proto/testdata,Mgoogle/protobuf/any.proto=github.com/golang/protobuf/ptypes/any:. proto3_proto/proto3.proto
|
||||||
|
make
|
||||||
@ -0,0 +1,229 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2011 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
// Protocol buffer deep copy and merge.
|
||||||
|
// TODO: RawMessage.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
import ( |
||||||
|
"log" |
||||||
|
"reflect" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
// Clone returns a deep copy of a protocol buffer.
|
||||||
|
func Clone(pb Message) Message { |
||||||
|
in := reflect.ValueOf(pb) |
||||||
|
if in.IsNil() { |
||||||
|
return pb |
||||||
|
} |
||||||
|
|
||||||
|
out := reflect.New(in.Type().Elem()) |
||||||
|
// out is empty so a merge is a deep copy.
|
||||||
|
mergeStruct(out.Elem(), in.Elem()) |
||||||
|
return out.Interface().(Message) |
||||||
|
} |
||||||
|
|
||||||
|
// Merge merges src into dst.
|
||||||
|
// Required and optional fields that are set in src will be set to that value in dst.
|
||||||
|
// Elements of repeated fields will be appended.
|
||||||
|
// Merge panics if src and dst are not the same type, or if dst is nil.
|
||||||
|
func Merge(dst, src Message) { |
||||||
|
in := reflect.ValueOf(src) |
||||||
|
out := reflect.ValueOf(dst) |
||||||
|
if out.IsNil() { |
||||||
|
panic("proto: nil destination") |
||||||
|
} |
||||||
|
if in.Type() != out.Type() { |
||||||
|
// Explicit test prior to mergeStruct so that mistyped nils will fail
|
||||||
|
panic("proto: type mismatch") |
||||||
|
} |
||||||
|
if in.IsNil() { |
||||||
|
// Merging nil into non-nil is a quiet no-op
|
||||||
|
return |
||||||
|
} |
||||||
|
mergeStruct(out.Elem(), in.Elem()) |
||||||
|
} |
||||||
|
|
||||||
|
func mergeStruct(out, in reflect.Value) { |
||||||
|
sprop := GetProperties(in.Type()) |
||||||
|
for i := 0; i < in.NumField(); i++ { |
||||||
|
f := in.Type().Field(i) |
||||||
|
if strings.HasPrefix(f.Name, "XXX_") { |
||||||
|
continue |
||||||
|
} |
||||||
|
mergeAny(out.Field(i), in.Field(i), false, sprop.Prop[i]) |
||||||
|
} |
||||||
|
|
||||||
|
if emIn, ok := extendable(in.Addr().Interface()); ok { |
||||||
|
emOut, _ := extendable(out.Addr().Interface()) |
||||||
|
mIn, muIn := emIn.extensionsRead() |
||||||
|
if mIn != nil { |
||||||
|
mOut := emOut.extensionsWrite() |
||||||
|
muIn.Lock() |
||||||
|
mergeExtension(mOut, mIn) |
||||||
|
muIn.Unlock() |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
uf := in.FieldByName("XXX_unrecognized") |
||||||
|
if !uf.IsValid() { |
||||||
|
return |
||||||
|
} |
||||||
|
uin := uf.Bytes() |
||||||
|
if len(uin) > 0 { |
||||||
|
out.FieldByName("XXX_unrecognized").SetBytes(append([]byte(nil), uin...)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// mergeAny performs a merge between two values of the same type.
|
||||||
|
// viaPtr indicates whether the values were indirected through a pointer (implying proto2).
|
||||||
|
// prop is set if this is a struct field (it may be nil).
|
||||||
|
func mergeAny(out, in reflect.Value, viaPtr bool, prop *Properties) { |
||||||
|
if in.Type() == protoMessageType { |
||||||
|
if !in.IsNil() { |
||||||
|
if out.IsNil() { |
||||||
|
out.Set(reflect.ValueOf(Clone(in.Interface().(Message)))) |
||||||
|
} else { |
||||||
|
Merge(out.Interface().(Message), in.Interface().(Message)) |
||||||
|
} |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
switch in.Kind() { |
||||||
|
case reflect.Bool, reflect.Float32, reflect.Float64, reflect.Int32, reflect.Int64, |
||||||
|
reflect.String, reflect.Uint32, reflect.Uint64: |
||||||
|
if !viaPtr && isProto3Zero(in) { |
||||||
|
return |
||||||
|
} |
||||||
|
out.Set(in) |
||||||
|
case reflect.Interface: |
||||||
|
// Probably a oneof field; copy non-nil values.
|
||||||
|
if in.IsNil() { |
||||||
|
return |
||||||
|
} |
||||||
|
// Allocate destination if it is not set, or set to a different type.
|
||||||
|
// Otherwise we will merge as normal.
|
||||||
|
if out.IsNil() || out.Elem().Type() != in.Elem().Type() { |
||||||
|
out.Set(reflect.New(in.Elem().Elem().Type())) // interface -> *T -> T -> new(T)
|
||||||
|
} |
||||||
|
mergeAny(out.Elem(), in.Elem(), false, nil) |
||||||
|
case reflect.Map: |
||||||
|
if in.Len() == 0 { |
||||||
|
return |
||||||
|
} |
||||||
|
if out.IsNil() { |
||||||
|
out.Set(reflect.MakeMap(in.Type())) |
||||||
|
} |
||||||
|
// For maps with value types of *T or []byte we need to deep copy each value.
|
||||||
|
elemKind := in.Type().Elem().Kind() |
||||||
|
for _, key := range in.MapKeys() { |
||||||
|
var val reflect.Value |
||||||
|
switch elemKind { |
||||||
|
case reflect.Ptr: |
||||||
|
val = reflect.New(in.Type().Elem().Elem()) |
||||||
|
mergeAny(val, in.MapIndex(key), false, nil) |
||||||
|
case reflect.Slice: |
||||||
|
val = in.MapIndex(key) |
||||||
|
val = reflect.ValueOf(append([]byte{}, val.Bytes()...)) |
||||||
|
default: |
||||||
|
val = in.MapIndex(key) |
||||||
|
} |
||||||
|
out.SetMapIndex(key, val) |
||||||
|
} |
||||||
|
case reflect.Ptr: |
||||||
|
if in.IsNil() { |
||||||
|
return |
||||||
|
} |
||||||
|
if out.IsNil() { |
||||||
|
out.Set(reflect.New(in.Elem().Type())) |
||||||
|
} |
||||||
|
mergeAny(out.Elem(), in.Elem(), true, nil) |
||||||
|
case reflect.Slice: |
||||||
|
if in.IsNil() { |
||||||
|
return |
||||||
|
} |
||||||
|
if in.Type().Elem().Kind() == reflect.Uint8 { |
||||||
|
// []byte is a scalar bytes field, not a repeated field.
|
||||||
|
|
||||||
|
// Edge case: if this is in a proto3 message, a zero length
|
||||||
|
// bytes field is considered the zero value, and should not
|
||||||
|
// be merged.
|
||||||
|
if prop != nil && prop.proto3 && in.Len() == 0 { |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// Make a deep copy.
|
||||||
|
// Append to []byte{} instead of []byte(nil) so that we never end up
|
||||||
|
// with a nil result.
|
||||||
|
out.SetBytes(append([]byte{}, in.Bytes()...)) |
||||||
|
return |
||||||
|
} |
||||||
|
n := in.Len() |
||||||
|
if out.IsNil() { |
||||||
|
out.Set(reflect.MakeSlice(in.Type(), 0, n)) |
||||||
|
} |
||||||
|
switch in.Type().Elem().Kind() { |
||||||
|
case reflect.Bool, reflect.Float32, reflect.Float64, reflect.Int32, reflect.Int64, |
||||||
|
reflect.String, reflect.Uint32, reflect.Uint64: |
||||||
|
out.Set(reflect.AppendSlice(out, in)) |
||||||
|
default: |
||||||
|
for i := 0; i < n; i++ { |
||||||
|
x := reflect.Indirect(reflect.New(in.Type().Elem())) |
||||||
|
mergeAny(x, in.Index(i), false, nil) |
||||||
|
out.Set(reflect.Append(out, x)) |
||||||
|
} |
||||||
|
} |
||||||
|
case reflect.Struct: |
||||||
|
mergeStruct(out, in) |
||||||
|
default: |
||||||
|
// unknown type, so not a protocol buffer
|
||||||
|
log.Printf("proto: don't know how to copy %v", in) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func mergeExtension(out, in map[int32]Extension) { |
||||||
|
for extNum, eIn := range in { |
||||||
|
eOut := Extension{desc: eIn.desc} |
||||||
|
if eIn.value != nil { |
||||||
|
v := reflect.New(reflect.TypeOf(eIn.value)).Elem() |
||||||
|
mergeAny(v, reflect.ValueOf(eIn.value), false, nil) |
||||||
|
eOut.value = v.Interface() |
||||||
|
} |
||||||
|
if eIn.enc != nil { |
||||||
|
eOut.enc = make([]byte, len(eIn.enc)) |
||||||
|
copy(eOut.enc, eIn.enc) |
||||||
|
} |
||||||
|
|
||||||
|
out[extNum] = eOut |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,970 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
/* |
||||||
|
* Routines for decoding protocol buffer data to construct in-memory representations. |
||||||
|
*/ |
||||||
|
|
||||||
|
import ( |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"os" |
||||||
|
"reflect" |
||||||
|
) |
||||||
|
|
||||||
|
// errOverflow is returned when an integer is too large to be represented.
|
||||||
|
var errOverflow = errors.New("proto: integer overflow") |
||||||
|
|
||||||
|
// ErrInternalBadWireType is returned by generated code when an incorrect
|
||||||
|
// wire type is encountered. It does not get returned to user code.
|
||||||
|
var ErrInternalBadWireType = errors.New("proto: internal error: bad wiretype for oneof") |
||||||
|
|
||||||
|
// The fundamental decoders that interpret bytes on the wire.
|
||||||
|
// Those that take integer types all return uint64 and are
|
||||||
|
// therefore of type valueDecoder.
|
||||||
|
|
||||||
|
// DecodeVarint reads a varint-encoded integer from the slice.
|
||||||
|
// It returns the integer and the number of bytes consumed, or
|
||||||
|
// zero if there is not enough.
|
||||||
|
// This is the format for the
|
||||||
|
// int32, int64, uint32, uint64, bool, and enum
|
||||||
|
// protocol buffer types.
|
||||||
|
func DecodeVarint(buf []byte) (x uint64, n int) { |
||||||
|
for shift := uint(0); shift < 64; shift += 7 { |
||||||
|
if n >= len(buf) { |
||||||
|
return 0, 0 |
||||||
|
} |
||||||
|
b := uint64(buf[n]) |
||||||
|
n++ |
||||||
|
x |= (b & 0x7F) << shift |
||||||
|
if (b & 0x80) == 0 { |
||||||
|
return x, n |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// The number is too large to represent in a 64-bit value.
|
||||||
|
return 0, 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (p *Buffer) decodeVarintSlow() (x uint64, err error) { |
||||||
|
i := p.index |
||||||
|
l := len(p.buf) |
||||||
|
|
||||||
|
for shift := uint(0); shift < 64; shift += 7 { |
||||||
|
if i >= l { |
||||||
|
err = io.ErrUnexpectedEOF |
||||||
|
return |
||||||
|
} |
||||||
|
b := p.buf[i] |
||||||
|
i++ |
||||||
|
x |= (uint64(b) & 0x7F) << shift |
||||||
|
if b < 0x80 { |
||||||
|
p.index = i |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// The number is too large to represent in a 64-bit value.
|
||||||
|
err = errOverflow |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeVarint reads a varint-encoded integer from the Buffer.
|
||||||
|
// This is the format for the
|
||||||
|
// int32, int64, uint32, uint64, bool, and enum
|
||||||
|
// protocol buffer types.
|
||||||
|
func (p *Buffer) DecodeVarint() (x uint64, err error) { |
||||||
|
i := p.index |
||||||
|
buf := p.buf |
||||||
|
|
||||||
|
if i >= len(buf) { |
||||||
|
return 0, io.ErrUnexpectedEOF |
||||||
|
} else if buf[i] < 0x80 { |
||||||
|
p.index++ |
||||||
|
return uint64(buf[i]), nil |
||||||
|
} else if len(buf)-i < 10 { |
||||||
|
return p.decodeVarintSlow() |
||||||
|
} |
||||||
|
|
||||||
|
var b uint64 |
||||||
|
// we already checked the first byte
|
||||||
|
x = uint64(buf[i]) - 0x80 |
||||||
|
i++ |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 7 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 7 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 14 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 14 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 21 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 21 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 28 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 28 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 35 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 35 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 42 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 42 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 49 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 49 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 56 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
x -= 0x80 << 56 |
||||||
|
|
||||||
|
b = uint64(buf[i]) |
||||||
|
i++ |
||||||
|
x += b << 63 |
||||||
|
if b&0x80 == 0 { |
||||||
|
goto done |
||||||
|
} |
||||||
|
// x -= 0x80 << 63 // Always zero.
|
||||||
|
|
||||||
|
return 0, errOverflow |
||||||
|
|
||||||
|
done: |
||||||
|
p.index = i |
||||||
|
return x, nil |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeFixed64 reads a 64-bit integer from the Buffer.
|
||||||
|
// This is the format for the
|
||||||
|
// fixed64, sfixed64, and double protocol buffer types.
|
||||||
|
func (p *Buffer) DecodeFixed64() (x uint64, err error) { |
||||||
|
// x, err already 0
|
||||||
|
i := p.index + 8 |
||||||
|
if i < 0 || i > len(p.buf) { |
||||||
|
err = io.ErrUnexpectedEOF |
||||||
|
return |
||||||
|
} |
||||||
|
p.index = i |
||||||
|
|
||||||
|
x = uint64(p.buf[i-8]) |
||||||
|
x |= uint64(p.buf[i-7]) << 8 |
||||||
|
x |= uint64(p.buf[i-6]) << 16 |
||||||
|
x |= uint64(p.buf[i-5]) << 24 |
||||||
|
x |= uint64(p.buf[i-4]) << 32 |
||||||
|
x |= uint64(p.buf[i-3]) << 40 |
||||||
|
x |= uint64(p.buf[i-2]) << 48 |
||||||
|
x |= uint64(p.buf[i-1]) << 56 |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeFixed32 reads a 32-bit integer from the Buffer.
|
||||||
|
// This is the format for the
|
||||||
|
// fixed32, sfixed32, and float protocol buffer types.
|
||||||
|
func (p *Buffer) DecodeFixed32() (x uint64, err error) { |
||||||
|
// x, err already 0
|
||||||
|
i := p.index + 4 |
||||||
|
if i < 0 || i > len(p.buf) { |
||||||
|
err = io.ErrUnexpectedEOF |
||||||
|
return |
||||||
|
} |
||||||
|
p.index = i |
||||||
|
|
||||||
|
x = uint64(p.buf[i-4]) |
||||||
|
x |= uint64(p.buf[i-3]) << 8 |
||||||
|
x |= uint64(p.buf[i-2]) << 16 |
||||||
|
x |= uint64(p.buf[i-1]) << 24 |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeZigzag64 reads a zigzag-encoded 64-bit integer
|
||||||
|
// from the Buffer.
|
||||||
|
// This is the format used for the sint64 protocol buffer type.
|
||||||
|
func (p *Buffer) DecodeZigzag64() (x uint64, err error) { |
||||||
|
x, err = p.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
x = (x >> 1) ^ uint64((int64(x&1)<<63)>>63) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeZigzag32 reads a zigzag-encoded 32-bit integer
|
||||||
|
// from the Buffer.
|
||||||
|
// This is the format used for the sint32 protocol buffer type.
|
||||||
|
func (p *Buffer) DecodeZigzag32() (x uint64, err error) { |
||||||
|
x, err = p.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
x = uint64((uint32(x) >> 1) ^ uint32((int32(x&1)<<31)>>31)) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// These are not ValueDecoders: they produce an array of bytes or a string.
|
||||||
|
// bytes, embedded messages
|
||||||
|
|
||||||
|
// DecodeRawBytes reads a count-delimited byte buffer from the Buffer.
|
||||||
|
// This is the format used for the bytes protocol buffer
|
||||||
|
// type and for embedded messages.
|
||||||
|
func (p *Buffer) DecodeRawBytes(alloc bool) (buf []byte, err error) { |
||||||
|
n, err := p.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
nb := int(n) |
||||||
|
if nb < 0 { |
||||||
|
return nil, fmt.Errorf("proto: bad byte length %d", nb) |
||||||
|
} |
||||||
|
end := p.index + nb |
||||||
|
if end < p.index || end > len(p.buf) { |
||||||
|
return nil, io.ErrUnexpectedEOF |
||||||
|
} |
||||||
|
|
||||||
|
if !alloc { |
||||||
|
// todo: check if can get more uses of alloc=false
|
||||||
|
buf = p.buf[p.index:end] |
||||||
|
p.index += nb |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
buf = make([]byte, nb) |
||||||
|
copy(buf, p.buf[p.index:]) |
||||||
|
p.index += nb |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeStringBytes reads an encoded string from the Buffer.
|
||||||
|
// This is the format used for the proto2 string type.
|
||||||
|
func (p *Buffer) DecodeStringBytes() (s string, err error) { |
||||||
|
buf, err := p.DecodeRawBytes(false) |
||||||
|
if err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
return string(buf), nil |
||||||
|
} |
||||||
|
|
||||||
|
// Skip the next item in the buffer. Its wire type is decoded and presented as an argument.
|
||||||
|
// If the protocol buffer has extensions, and the field matches, add it as an extension.
|
||||||
|
// Otherwise, if the XXX_unrecognized field exists, append the skipped data there.
|
||||||
|
func (o *Buffer) skipAndSave(t reflect.Type, tag, wire int, base structPointer, unrecField field) error { |
||||||
|
oi := o.index |
||||||
|
|
||||||
|
err := o.skip(t, tag, wire) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if !unrecField.IsValid() { |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
ptr := structPointer_Bytes(base, unrecField) |
||||||
|
|
||||||
|
// Add the skipped field to struct field
|
||||||
|
obuf := o.buf |
||||||
|
|
||||||
|
o.buf = *ptr |
||||||
|
o.EncodeVarint(uint64(tag<<3 | wire)) |
||||||
|
*ptr = append(o.buf, obuf[oi:o.index]...) |
||||||
|
|
||||||
|
o.buf = obuf |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Skip the next item in the buffer. Its wire type is decoded and presented as an argument.
|
||||||
|
func (o *Buffer) skip(t reflect.Type, tag, wire int) error { |
||||||
|
|
||||||
|
var u uint64 |
||||||
|
var err error |
||||||
|
|
||||||
|
switch wire { |
||||||
|
case WireVarint: |
||||||
|
_, err = o.DecodeVarint() |
||||||
|
case WireFixed64: |
||||||
|
_, err = o.DecodeFixed64() |
||||||
|
case WireBytes: |
||||||
|
_, err = o.DecodeRawBytes(false) |
||||||
|
case WireFixed32: |
||||||
|
_, err = o.DecodeFixed32() |
||||||
|
case WireStartGroup: |
||||||
|
for { |
||||||
|
u, err = o.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
break |
||||||
|
} |
||||||
|
fwire := int(u & 0x7) |
||||||
|
if fwire == WireEndGroup { |
||||||
|
break |
||||||
|
} |
||||||
|
ftag := int(u >> 3) |
||||||
|
err = o.skip(t, ftag, fwire) |
||||||
|
if err != nil { |
||||||
|
break |
||||||
|
} |
||||||
|
} |
||||||
|
default: |
||||||
|
err = fmt.Errorf("proto: can't skip unknown wire type %d for %s", wire, t) |
||||||
|
} |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// Unmarshaler is the interface representing objects that can
|
||||||
|
// unmarshal themselves. The method should reset the receiver before
|
||||||
|
// decoding starts. The argument points to data that may be
|
||||||
|
// overwritten, so implementations should not keep references to the
|
||||||
|
// buffer.
|
||||||
|
type Unmarshaler interface { |
||||||
|
Unmarshal([]byte) error |
||||||
|
} |
||||||
|
|
||||||
|
// Unmarshal parses the protocol buffer representation in buf and places the
|
||||||
|
// decoded result in pb. If the struct underlying pb does not match
|
||||||
|
// the data in buf, the results can be unpredictable.
|
||||||
|
//
|
||||||
|
// Unmarshal resets pb before starting to unmarshal, so any
|
||||||
|
// existing data in pb is always removed. Use UnmarshalMerge
|
||||||
|
// to preserve and append to existing data.
|
||||||
|
func Unmarshal(buf []byte, pb Message) error { |
||||||
|
pb.Reset() |
||||||
|
return UnmarshalMerge(buf, pb) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalMerge parses the protocol buffer representation in buf and
|
||||||
|
// writes the decoded result to pb. If the struct underlying pb does not match
|
||||||
|
// the data in buf, the results can be unpredictable.
|
||||||
|
//
|
||||||
|
// UnmarshalMerge merges into existing data in pb.
|
||||||
|
// Most code should use Unmarshal instead.
|
||||||
|
func UnmarshalMerge(buf []byte, pb Message) error { |
||||||
|
// If the object can unmarshal itself, let it.
|
||||||
|
if u, ok := pb.(Unmarshaler); ok { |
||||||
|
return u.Unmarshal(buf) |
||||||
|
} |
||||||
|
return NewBuffer(buf).Unmarshal(pb) |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeMessage reads a count-delimited message from the Buffer.
|
||||||
|
func (p *Buffer) DecodeMessage(pb Message) error { |
||||||
|
enc, err := p.DecodeRawBytes(false) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
return NewBuffer(enc).Unmarshal(pb) |
||||||
|
} |
||||||
|
|
||||||
|
// DecodeGroup reads a tag-delimited group from the Buffer.
|
||||||
|
func (p *Buffer) DecodeGroup(pb Message) error { |
||||||
|
typ, base, err := getbase(pb) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
return p.unmarshalType(typ.Elem(), GetProperties(typ.Elem()), true, base) |
||||||
|
} |
||||||
|
|
||||||
|
// Unmarshal parses the protocol buffer representation in the
|
||||||
|
// Buffer and places the decoded result in pb. If the struct
|
||||||
|
// underlying pb does not match the data in the buffer, the results can be
|
||||||
|
// unpredictable.
|
||||||
|
//
|
||||||
|
// Unlike proto.Unmarshal, this does not reset pb before starting to unmarshal.
|
||||||
|
func (p *Buffer) Unmarshal(pb Message) error { |
||||||
|
// If the object can unmarshal itself, let it.
|
||||||
|
if u, ok := pb.(Unmarshaler); ok { |
||||||
|
err := u.Unmarshal(p.buf[p.index:]) |
||||||
|
p.index = len(p.buf) |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
typ, base, err := getbase(pb) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
err = p.unmarshalType(typ.Elem(), GetProperties(typ.Elem()), false, base) |
||||||
|
|
||||||
|
if collectStats { |
||||||
|
stats.Decode++ |
||||||
|
} |
||||||
|
|
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// unmarshalType does the work of unmarshaling a structure.
|
||||||
|
func (o *Buffer) unmarshalType(st reflect.Type, prop *StructProperties, is_group bool, base structPointer) error { |
||||||
|
var state errorState |
||||||
|
required, reqFields := prop.reqCount, uint64(0) |
||||||
|
|
||||||
|
var err error |
||||||
|
for err == nil && o.index < len(o.buf) { |
||||||
|
oi := o.index |
||||||
|
var u uint64 |
||||||
|
u, err = o.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
break |
||||||
|
} |
||||||
|
wire := int(u & 0x7) |
||||||
|
if wire == WireEndGroup { |
||||||
|
if is_group { |
||||||
|
if required > 0 { |
||||||
|
// Not enough information to determine the exact field.
|
||||||
|
// (See below.)
|
||||||
|
return &RequiredNotSetError{"{Unknown}"} |
||||||
|
} |
||||||
|
return nil // input is satisfied
|
||||||
|
} |
||||||
|
return fmt.Errorf("proto: %s: wiretype end group for non-group", st) |
||||||
|
} |
||||||
|
tag := int(u >> 3) |
||||||
|
if tag <= 0 { |
||||||
|
return fmt.Errorf("proto: %s: illegal tag %d (wire type %d)", st, tag, wire) |
||||||
|
} |
||||||
|
fieldnum, ok := prop.decoderTags.get(tag) |
||||||
|
if !ok { |
||||||
|
// Maybe it's an extension?
|
||||||
|
if prop.extendable { |
||||||
|
if e, _ := extendable(structPointer_Interface(base, st)); isExtensionField(e, int32(tag)) { |
||||||
|
if err = o.skip(st, tag, wire); err == nil { |
||||||
|
extmap := e.extensionsWrite() |
||||||
|
ext := extmap[int32(tag)] // may be missing
|
||||||
|
ext.enc = append(ext.enc, o.buf[oi:o.index]...) |
||||||
|
extmap[int32(tag)] = ext |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
} |
||||||
|
// Maybe it's a oneof?
|
||||||
|
if prop.oneofUnmarshaler != nil { |
||||||
|
m := structPointer_Interface(base, st).(Message) |
||||||
|
// First return value indicates whether tag is a oneof field.
|
||||||
|
ok, err = prop.oneofUnmarshaler(m, tag, wire, o) |
||||||
|
if err == ErrInternalBadWireType { |
||||||
|
// Map the error to something more descriptive.
|
||||||
|
// Do the formatting here to save generated code space.
|
||||||
|
err = fmt.Errorf("bad wiretype for oneof field in %T", m) |
||||||
|
} |
||||||
|
if ok { |
||||||
|
continue |
||||||
|
} |
||||||
|
} |
||||||
|
err = o.skipAndSave(st, tag, wire, base, prop.unrecField) |
||||||
|
continue |
||||||
|
} |
||||||
|
p := prop.Prop[fieldnum] |
||||||
|
|
||||||
|
if p.dec == nil { |
||||||
|
fmt.Fprintf(os.Stderr, "proto: no protobuf decoder for %s.%s\n", st, st.Field(fieldnum).Name) |
||||||
|
continue |
||||||
|
} |
||||||
|
dec := p.dec |
||||||
|
if wire != WireStartGroup && wire != p.WireType { |
||||||
|
if wire == WireBytes && p.packedDec != nil { |
||||||
|
// a packable field
|
||||||
|
dec = p.packedDec |
||||||
|
} else { |
||||||
|
err = fmt.Errorf("proto: bad wiretype for field %s.%s: got wiretype %d, want %d", st, st.Field(fieldnum).Name, wire, p.WireType) |
||||||
|
continue |
||||||
|
} |
||||||
|
} |
||||||
|
decErr := dec(o, p, base) |
||||||
|
if decErr != nil && !state.shouldContinue(decErr, p) { |
||||||
|
err = decErr |
||||||
|
} |
||||||
|
if err == nil && p.Required { |
||||||
|
// Successfully decoded a required field.
|
||||||
|
if tag <= 64 { |
||||||
|
// use bitmap for fields 1-64 to catch field reuse.
|
||||||
|
var mask uint64 = 1 << uint64(tag-1) |
||||||
|
if reqFields&mask == 0 { |
||||||
|
// new required field
|
||||||
|
reqFields |= mask |
||||||
|
required-- |
||||||
|
} |
||||||
|
} else { |
||||||
|
// This is imprecise. It can be fooled by a required field
|
||||||
|
// with a tag > 64 that is encoded twice; that's very rare.
|
||||||
|
// A fully correct implementation would require allocating
|
||||||
|
// a data structure, which we would like to avoid.
|
||||||
|
required-- |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
if err == nil { |
||||||
|
if is_group { |
||||||
|
return io.ErrUnexpectedEOF |
||||||
|
} |
||||||
|
if state.err != nil { |
||||||
|
return state.err |
||||||
|
} |
||||||
|
if required > 0 { |
||||||
|
// Not enough information to determine the exact field. If we use extra
|
||||||
|
// CPU, we could determine the field only if the missing required field
|
||||||
|
// has a tag <= 64 and we check reqFields.
|
||||||
|
return &RequiredNotSetError{"{Unknown}"} |
||||||
|
} |
||||||
|
} |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// Individual type decoders
|
||||||
|
// For each,
|
||||||
|
// u is the decoded value,
|
||||||
|
// v is a pointer to the field (pointer) in the struct
|
||||||
|
|
||||||
|
// Sizes of the pools to allocate inside the Buffer.
|
||||||
|
// The goal is modest amortization and allocation
|
||||||
|
// on at least 16-byte boundaries.
|
||||||
|
const ( |
||||||
|
boolPoolSize = 16 |
||||||
|
uint32PoolSize = 8 |
||||||
|
uint64PoolSize = 4 |
||||||
|
) |
||||||
|
|
||||||
|
// Decode a bool.
|
||||||
|
func (o *Buffer) dec_bool(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if len(o.bools) == 0 { |
||||||
|
o.bools = make([]bool, boolPoolSize) |
||||||
|
} |
||||||
|
o.bools[0] = u != 0 |
||||||
|
*structPointer_Bool(base, p.field) = &o.bools[0] |
||||||
|
o.bools = o.bools[1:] |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (o *Buffer) dec_proto3_bool(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*structPointer_BoolVal(base, p.field) = u != 0 |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode an int32.
|
||||||
|
func (o *Buffer) dec_int32(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
word32_Set(structPointer_Word32(base, p.field), o, uint32(u)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (o *Buffer) dec_proto3_int32(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
word32Val_Set(structPointer_Word32Val(base, p.field), uint32(u)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode an int64.
|
||||||
|
func (o *Buffer) dec_int64(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
word64_Set(structPointer_Word64(base, p.field), o, u) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (o *Buffer) dec_proto3_int64(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
word64Val_Set(structPointer_Word64Val(base, p.field), o, u) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a string.
|
||||||
|
func (o *Buffer) dec_string(p *Properties, base structPointer) error { |
||||||
|
s, err := o.DecodeStringBytes() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*structPointer_String(base, p.field) = &s |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (o *Buffer) dec_proto3_string(p *Properties, base structPointer) error { |
||||||
|
s, err := o.DecodeStringBytes() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*structPointer_StringVal(base, p.field) = s |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of bytes ([]byte).
|
||||||
|
func (o *Buffer) dec_slice_byte(p *Properties, base structPointer) error { |
||||||
|
b, err := o.DecodeRawBytes(true) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*structPointer_Bytes(base, p.field) = b |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of bools ([]bool).
|
||||||
|
func (o *Buffer) dec_slice_bool(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
v := structPointer_BoolSlice(base, p.field) |
||||||
|
*v = append(*v, u != 0) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of bools ([]bool) in packed format.
|
||||||
|
func (o *Buffer) dec_slice_packed_bool(p *Properties, base structPointer) error { |
||||||
|
v := structPointer_BoolSlice(base, p.field) |
||||||
|
|
||||||
|
nn, err := o.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
nb := int(nn) // number of bytes of encoded bools
|
||||||
|
fin := o.index + nb |
||||||
|
if fin < o.index { |
||||||
|
return errOverflow |
||||||
|
} |
||||||
|
|
||||||
|
y := *v |
||||||
|
for o.index < fin { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
y = append(y, u != 0) |
||||||
|
} |
||||||
|
|
||||||
|
*v = y |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of int32s ([]int32).
|
||||||
|
func (o *Buffer) dec_slice_int32(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
structPointer_Word32Slice(base, p.field).Append(uint32(u)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of int32s ([]int32) in packed format.
|
||||||
|
func (o *Buffer) dec_slice_packed_int32(p *Properties, base structPointer) error { |
||||||
|
v := structPointer_Word32Slice(base, p.field) |
||||||
|
|
||||||
|
nn, err := o.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
nb := int(nn) // number of bytes of encoded int32s
|
||||||
|
|
||||||
|
fin := o.index + nb |
||||||
|
if fin < o.index { |
||||||
|
return errOverflow |
||||||
|
} |
||||||
|
for o.index < fin { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
v.Append(uint32(u)) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of int64s ([]int64).
|
||||||
|
func (o *Buffer) dec_slice_int64(p *Properties, base structPointer) error { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
structPointer_Word64Slice(base, p.field).Append(u) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of int64s ([]int64) in packed format.
|
||||||
|
func (o *Buffer) dec_slice_packed_int64(p *Properties, base structPointer) error { |
||||||
|
v := structPointer_Word64Slice(base, p.field) |
||||||
|
|
||||||
|
nn, err := o.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
nb := int(nn) // number of bytes of encoded int64s
|
||||||
|
|
||||||
|
fin := o.index + nb |
||||||
|
if fin < o.index { |
||||||
|
return errOverflow |
||||||
|
} |
||||||
|
for o.index < fin { |
||||||
|
u, err := p.valDec(o) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
v.Append(u) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of strings ([]string).
|
||||||
|
func (o *Buffer) dec_slice_string(p *Properties, base structPointer) error { |
||||||
|
s, err := o.DecodeStringBytes() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
v := structPointer_StringSlice(base, p.field) |
||||||
|
*v = append(*v, s) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of slice of bytes ([][]byte).
|
||||||
|
func (o *Buffer) dec_slice_slice_byte(p *Properties, base structPointer) error { |
||||||
|
b, err := o.DecodeRawBytes(true) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
v := structPointer_BytesSlice(base, p.field) |
||||||
|
*v = append(*v, b) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a map field.
|
||||||
|
func (o *Buffer) dec_new_map(p *Properties, base structPointer) error { |
||||||
|
raw, err := o.DecodeRawBytes(false) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
oi := o.index // index at the end of this map entry
|
||||||
|
o.index -= len(raw) // move buffer back to start of map entry
|
||||||
|
|
||||||
|
mptr := structPointer_NewAt(base, p.field, p.mtype) // *map[K]V
|
||||||
|
if mptr.Elem().IsNil() { |
||||||
|
mptr.Elem().Set(reflect.MakeMap(mptr.Type().Elem())) |
||||||
|
} |
||||||
|
v := mptr.Elem() // map[K]V
|
||||||
|
|
||||||
|
// Prepare addressable doubly-indirect placeholders for the key and value types.
|
||||||
|
// See enc_new_map for why.
|
||||||
|
keyptr := reflect.New(reflect.PtrTo(p.mtype.Key())).Elem() // addressable *K
|
||||||
|
keybase := toStructPointer(keyptr.Addr()) // **K
|
||||||
|
|
||||||
|
var valbase structPointer |
||||||
|
var valptr reflect.Value |
||||||
|
switch p.mtype.Elem().Kind() { |
||||||
|
case reflect.Slice: |
||||||
|
// []byte
|
||||||
|
var dummy []byte |
||||||
|
valptr = reflect.ValueOf(&dummy) // *[]byte
|
||||||
|
valbase = toStructPointer(valptr) // *[]byte
|
||||||
|
case reflect.Ptr: |
||||||
|
// message; valptr is **Msg; need to allocate the intermediate pointer
|
||||||
|
valptr = reflect.New(reflect.PtrTo(p.mtype.Elem())).Elem() // addressable *V
|
||||||
|
valptr.Set(reflect.New(valptr.Type().Elem())) |
||||||
|
valbase = toStructPointer(valptr) |
||||||
|
default: |
||||||
|
// everything else
|
||||||
|
valptr = reflect.New(reflect.PtrTo(p.mtype.Elem())).Elem() // addressable *V
|
||||||
|
valbase = toStructPointer(valptr.Addr()) // **V
|
||||||
|
} |
||||||
|
|
||||||
|
// Decode.
|
||||||
|
// This parses a restricted wire format, namely the encoding of a message
|
||||||
|
// with two fields. See enc_new_map for the format.
|
||||||
|
for o.index < oi { |
||||||
|
// tagcode for key and value properties are always a single byte
|
||||||
|
// because they have tags 1 and 2.
|
||||||
|
tagcode := o.buf[o.index] |
||||||
|
o.index++ |
||||||
|
switch tagcode { |
||||||
|
case p.mkeyprop.tagcode[0]: |
||||||
|
if err := p.mkeyprop.dec(o, p.mkeyprop, keybase); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
case p.mvalprop.tagcode[0]: |
||||||
|
if err := p.mvalprop.dec(o, p.mvalprop, valbase); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
default: |
||||||
|
// TODO: Should we silently skip this instead?
|
||||||
|
return fmt.Errorf("proto: bad map data tag %d", raw[0]) |
||||||
|
} |
||||||
|
} |
||||||
|
keyelem, valelem := keyptr.Elem(), valptr.Elem() |
||||||
|
if !keyelem.IsValid() { |
||||||
|
keyelem = reflect.Zero(p.mtype.Key()) |
||||||
|
} |
||||||
|
if !valelem.IsValid() { |
||||||
|
valelem = reflect.Zero(p.mtype.Elem()) |
||||||
|
} |
||||||
|
|
||||||
|
v.SetMapIndex(keyelem, valelem) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a group.
|
||||||
|
func (o *Buffer) dec_struct_group(p *Properties, base structPointer) error { |
||||||
|
bas := structPointer_GetStructPointer(base, p.field) |
||||||
|
if structPointer_IsNil(bas) { |
||||||
|
// allocate new nested message
|
||||||
|
bas = toStructPointer(reflect.New(p.stype)) |
||||||
|
structPointer_SetStructPointer(base, p.field, bas) |
||||||
|
} |
||||||
|
return o.unmarshalType(p.stype, p.sprop, true, bas) |
||||||
|
} |
||||||
|
|
||||||
|
// Decode an embedded message.
|
||||||
|
func (o *Buffer) dec_struct_message(p *Properties, base structPointer) (err error) { |
||||||
|
raw, e := o.DecodeRawBytes(false) |
||||||
|
if e != nil { |
||||||
|
return e |
||||||
|
} |
||||||
|
|
||||||
|
bas := structPointer_GetStructPointer(base, p.field) |
||||||
|
if structPointer_IsNil(bas) { |
||||||
|
// allocate new nested message
|
||||||
|
bas = toStructPointer(reflect.New(p.stype)) |
||||||
|
structPointer_SetStructPointer(base, p.field, bas) |
||||||
|
} |
||||||
|
|
||||||
|
// If the object can unmarshal itself, let it.
|
||||||
|
if p.isUnmarshaler { |
||||||
|
iv := structPointer_Interface(bas, p.stype) |
||||||
|
return iv.(Unmarshaler).Unmarshal(raw) |
||||||
|
} |
||||||
|
|
||||||
|
obuf := o.buf |
||||||
|
oi := o.index |
||||||
|
o.buf = raw |
||||||
|
o.index = 0 |
||||||
|
|
||||||
|
err = o.unmarshalType(p.stype, p.sprop, false, bas) |
||||||
|
o.buf = obuf |
||||||
|
o.index = oi |
||||||
|
|
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of embedded messages.
|
||||||
|
func (o *Buffer) dec_slice_struct_message(p *Properties, base structPointer) error { |
||||||
|
return o.dec_slice_struct(p, false, base) |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of embedded groups.
|
||||||
|
func (o *Buffer) dec_slice_struct_group(p *Properties, base structPointer) error { |
||||||
|
return o.dec_slice_struct(p, true, base) |
||||||
|
} |
||||||
|
|
||||||
|
// Decode a slice of structs ([]*struct).
|
||||||
|
func (o *Buffer) dec_slice_struct(p *Properties, is_group bool, base structPointer) error { |
||||||
|
v := reflect.New(p.stype) |
||||||
|
bas := toStructPointer(v) |
||||||
|
structPointer_StructPointerSlice(base, p.field).Append(bas) |
||||||
|
|
||||||
|
if is_group { |
||||||
|
err := o.unmarshalType(p.stype, p.sprop, is_group, bas) |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
raw, err := o.DecodeRawBytes(false) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// If the object can unmarshal itself, let it.
|
||||||
|
if p.isUnmarshaler { |
||||||
|
iv := v.Interface() |
||||||
|
return iv.(Unmarshaler).Unmarshal(raw) |
||||||
|
} |
||||||
|
|
||||||
|
obuf := o.buf |
||||||
|
oi := o.index |
||||||
|
o.buf = raw |
||||||
|
o.index = 0 |
||||||
|
|
||||||
|
err = o.unmarshalType(p.stype, p.sprop, is_group, bas) |
||||||
|
|
||||||
|
o.buf = obuf |
||||||
|
o.index = oi |
||||||
|
|
||||||
|
return err |
||||||
|
} |
||||||
File diff suppressed because it is too large
Load Diff
@ -0,0 +1,300 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2011 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
// Protocol buffer comparison.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"log" |
||||||
|
"reflect" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
/* |
||||||
|
Equal returns true iff protocol buffers a and b are equal. |
||||||
|
The arguments must both be pointers to protocol buffer structs. |
||||||
|
|
||||||
|
Equality is defined in this way: |
||||||
|
- Two messages are equal iff they are the same type, |
||||||
|
corresponding fields are equal, unknown field sets |
||||||
|
are equal, and extensions sets are equal. |
||||||
|
- Two set scalar fields are equal iff their values are equal. |
||||||
|
If the fields are of a floating-point type, remember that |
||||||
|
NaN != x for all x, including NaN. If the message is defined |
||||||
|
in a proto3 .proto file, fields are not "set"; specifically, |
||||||
|
zero length proto3 "bytes" fields are equal (nil == {}). |
||||||
|
- Two repeated fields are equal iff their lengths are the same, |
||||||
|
and their corresponding elements are equal. Note a "bytes" field, |
||||||
|
although represented by []byte, is not a repeated field and the |
||||||
|
rule for the scalar fields described above applies. |
||||||
|
- Two unset fields are equal. |
||||||
|
- Two unknown field sets are equal if their current |
||||||
|
encoded state is equal. |
||||||
|
- Two extension sets are equal iff they have corresponding |
||||||
|
elements that are pairwise equal. |
||||||
|
- Two map fields are equal iff their lengths are the same, |
||||||
|
and they contain the same set of elements. Zero-length map |
||||||
|
fields are equal. |
||||||
|
- Every other combination of things are not equal. |
||||||
|
|
||||||
|
The return value is undefined if a and b are not protocol buffers. |
||||||
|
*/ |
||||||
|
func Equal(a, b Message) bool { |
||||||
|
if a == nil || b == nil { |
||||||
|
return a == b |
||||||
|
} |
||||||
|
v1, v2 := reflect.ValueOf(a), reflect.ValueOf(b) |
||||||
|
if v1.Type() != v2.Type() { |
||||||
|
return false |
||||||
|
} |
||||||
|
if v1.Kind() == reflect.Ptr { |
||||||
|
if v1.IsNil() { |
||||||
|
return v2.IsNil() |
||||||
|
} |
||||||
|
if v2.IsNil() { |
||||||
|
return false |
||||||
|
} |
||||||
|
v1, v2 = v1.Elem(), v2.Elem() |
||||||
|
} |
||||||
|
if v1.Kind() != reflect.Struct { |
||||||
|
return false |
||||||
|
} |
||||||
|
return equalStruct(v1, v2) |
||||||
|
} |
||||||
|
|
||||||
|
// v1 and v2 are known to have the same type.
|
||||||
|
func equalStruct(v1, v2 reflect.Value) bool { |
||||||
|
sprop := GetProperties(v1.Type()) |
||||||
|
for i := 0; i < v1.NumField(); i++ { |
||||||
|
f := v1.Type().Field(i) |
||||||
|
if strings.HasPrefix(f.Name, "XXX_") { |
||||||
|
continue |
||||||
|
} |
||||||
|
f1, f2 := v1.Field(i), v2.Field(i) |
||||||
|
if f.Type.Kind() == reflect.Ptr { |
||||||
|
if n1, n2 := f1.IsNil(), f2.IsNil(); n1 && n2 { |
||||||
|
// both unset
|
||||||
|
continue |
||||||
|
} else if n1 != n2 { |
||||||
|
// set/unset mismatch
|
||||||
|
return false |
||||||
|
} |
||||||
|
b1, ok := f1.Interface().(raw) |
||||||
|
if ok { |
||||||
|
b2 := f2.Interface().(raw) |
||||||
|
// RawMessage
|
||||||
|
if !bytes.Equal(b1.Bytes(), b2.Bytes()) { |
||||||
|
return false |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
f1, f2 = f1.Elem(), f2.Elem() |
||||||
|
} |
||||||
|
if !equalAny(f1, f2, sprop.Prop[i]) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if em1 := v1.FieldByName("XXX_InternalExtensions"); em1.IsValid() { |
||||||
|
em2 := v2.FieldByName("XXX_InternalExtensions") |
||||||
|
if !equalExtensions(v1.Type(), em1.Interface().(XXX_InternalExtensions), em2.Interface().(XXX_InternalExtensions)) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if em1 := v1.FieldByName("XXX_extensions"); em1.IsValid() { |
||||||
|
em2 := v2.FieldByName("XXX_extensions") |
||||||
|
if !equalExtMap(v1.Type(), em1.Interface().(map[int32]Extension), em2.Interface().(map[int32]Extension)) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
uf := v1.FieldByName("XXX_unrecognized") |
||||||
|
if !uf.IsValid() { |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
u1 := uf.Bytes() |
||||||
|
u2 := v2.FieldByName("XXX_unrecognized").Bytes() |
||||||
|
if !bytes.Equal(u1, u2) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// v1 and v2 are known to have the same type.
|
||||||
|
// prop may be nil.
|
||||||
|
func equalAny(v1, v2 reflect.Value, prop *Properties) bool { |
||||||
|
if v1.Type() == protoMessageType { |
||||||
|
m1, _ := v1.Interface().(Message) |
||||||
|
m2, _ := v2.Interface().(Message) |
||||||
|
return Equal(m1, m2) |
||||||
|
} |
||||||
|
switch v1.Kind() { |
||||||
|
case reflect.Bool: |
||||||
|
return v1.Bool() == v2.Bool() |
||||||
|
case reflect.Float32, reflect.Float64: |
||||||
|
return v1.Float() == v2.Float() |
||||||
|
case reflect.Int32, reflect.Int64: |
||||||
|
return v1.Int() == v2.Int() |
||||||
|
case reflect.Interface: |
||||||
|
// Probably a oneof field; compare the inner values.
|
||||||
|
n1, n2 := v1.IsNil(), v2.IsNil() |
||||||
|
if n1 || n2 { |
||||||
|
return n1 == n2 |
||||||
|
} |
||||||
|
e1, e2 := v1.Elem(), v2.Elem() |
||||||
|
if e1.Type() != e2.Type() { |
||||||
|
return false |
||||||
|
} |
||||||
|
return equalAny(e1, e2, nil) |
||||||
|
case reflect.Map: |
||||||
|
if v1.Len() != v2.Len() { |
||||||
|
return false |
||||||
|
} |
||||||
|
for _, key := range v1.MapKeys() { |
||||||
|
val2 := v2.MapIndex(key) |
||||||
|
if !val2.IsValid() { |
||||||
|
// This key was not found in the second map.
|
||||||
|
return false |
||||||
|
} |
||||||
|
if !equalAny(v1.MapIndex(key), val2, nil) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
case reflect.Ptr: |
||||||
|
// Maps may have nil values in them, so check for nil.
|
||||||
|
if v1.IsNil() && v2.IsNil() { |
||||||
|
return true |
||||||
|
} |
||||||
|
if v1.IsNil() != v2.IsNil() { |
||||||
|
return false |
||||||
|
} |
||||||
|
return equalAny(v1.Elem(), v2.Elem(), prop) |
||||||
|
case reflect.Slice: |
||||||
|
if v1.Type().Elem().Kind() == reflect.Uint8 { |
||||||
|
// short circuit: []byte
|
||||||
|
|
||||||
|
// Edge case: if this is in a proto3 message, a zero length
|
||||||
|
// bytes field is considered the zero value.
|
||||||
|
if prop != nil && prop.proto3 && v1.Len() == 0 && v2.Len() == 0 { |
||||||
|
return true |
||||||
|
} |
||||||
|
if v1.IsNil() != v2.IsNil() { |
||||||
|
return false |
||||||
|
} |
||||||
|
return bytes.Equal(v1.Interface().([]byte), v2.Interface().([]byte)) |
||||||
|
} |
||||||
|
|
||||||
|
if v1.Len() != v2.Len() { |
||||||
|
return false |
||||||
|
} |
||||||
|
for i := 0; i < v1.Len(); i++ { |
||||||
|
if !equalAny(v1.Index(i), v2.Index(i), prop) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
case reflect.String: |
||||||
|
return v1.Interface().(string) == v2.Interface().(string) |
||||||
|
case reflect.Struct: |
||||||
|
return equalStruct(v1, v2) |
||||||
|
case reflect.Uint32, reflect.Uint64: |
||||||
|
return v1.Uint() == v2.Uint() |
||||||
|
} |
||||||
|
|
||||||
|
// unknown type, so not a protocol buffer
|
||||||
|
log.Printf("proto: don't know how to compare %v", v1) |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
// base is the struct type that the extensions are based on.
|
||||||
|
// x1 and x2 are InternalExtensions.
|
||||||
|
func equalExtensions(base reflect.Type, x1, x2 XXX_InternalExtensions) bool { |
||||||
|
em1, _ := x1.extensionsRead() |
||||||
|
em2, _ := x2.extensionsRead() |
||||||
|
return equalExtMap(base, em1, em2) |
||||||
|
} |
||||||
|
|
||||||
|
func equalExtMap(base reflect.Type, em1, em2 map[int32]Extension) bool { |
||||||
|
if len(em1) != len(em2) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
for extNum, e1 := range em1 { |
||||||
|
e2, ok := em2[extNum] |
||||||
|
if !ok { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
m1, m2 := e1.value, e2.value |
||||||
|
|
||||||
|
if m1 != nil && m2 != nil { |
||||||
|
// Both are unencoded.
|
||||||
|
if !equalAny(reflect.ValueOf(m1), reflect.ValueOf(m2), nil) { |
||||||
|
return false |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
// At least one is encoded. To do a semantically correct comparison
|
||||||
|
// we need to unmarshal them first.
|
||||||
|
var desc *ExtensionDesc |
||||||
|
if m := extensionMaps[base]; m != nil { |
||||||
|
desc = m[extNum] |
||||||
|
} |
||||||
|
if desc == nil { |
||||||
|
log.Printf("proto: don't know how to compare extension %d of %v", extNum, base) |
||||||
|
continue |
||||||
|
} |
||||||
|
var err error |
||||||
|
if m1 == nil { |
||||||
|
m1, err = decodeExtension(e1.enc, desc) |
||||||
|
} |
||||||
|
if m2 == nil && err == nil { |
||||||
|
m2, err = decodeExtension(e2.enc, desc) |
||||||
|
} |
||||||
|
if err != nil { |
||||||
|
// The encoded form is invalid.
|
||||||
|
log.Printf("proto: badly encoded extension %d of %v: %v", extNum, base, err) |
||||||
|
return false |
||||||
|
} |
||||||
|
if !equalAny(reflect.ValueOf(m1), reflect.ValueOf(m2), nil) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return true |
||||||
|
} |
||||||
@ -0,0 +1,587 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
/* |
||||||
|
* Types and routines for supporting protocol buffer extensions. |
||||||
|
*/ |
||||||
|
|
||||||
|
import ( |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"reflect" |
||||||
|
"strconv" |
||||||
|
"sync" |
||||||
|
) |
||||||
|
|
||||||
|
// ErrMissingExtension is the error returned by GetExtension if the named extension is not in the message.
|
||||||
|
var ErrMissingExtension = errors.New("proto: missing extension") |
||||||
|
|
||||||
|
// ExtensionRange represents a range of message extensions for a protocol buffer.
|
||||||
|
// Used in code generated by the protocol compiler.
|
||||||
|
type ExtensionRange struct { |
||||||
|
Start, End int32 // both inclusive
|
||||||
|
} |
||||||
|
|
||||||
|
// extendableProto is an interface implemented by any protocol buffer generated by the current
|
||||||
|
// proto compiler that may be extended.
|
||||||
|
type extendableProto interface { |
||||||
|
Message |
||||||
|
ExtensionRangeArray() []ExtensionRange |
||||||
|
extensionsWrite() map[int32]Extension |
||||||
|
extensionsRead() (map[int32]Extension, sync.Locker) |
||||||
|
} |
||||||
|
|
||||||
|
// extendableProtoV1 is an interface implemented by a protocol buffer generated by the previous
|
||||||
|
// version of the proto compiler that may be extended.
|
||||||
|
type extendableProtoV1 interface { |
||||||
|
Message |
||||||
|
ExtensionRangeArray() []ExtensionRange |
||||||
|
ExtensionMap() map[int32]Extension |
||||||
|
} |
||||||
|
|
||||||
|
// extensionAdapter is a wrapper around extendableProtoV1 that implements extendableProto.
|
||||||
|
type extensionAdapter struct { |
||||||
|
extendableProtoV1 |
||||||
|
} |
||||||
|
|
||||||
|
func (e extensionAdapter) extensionsWrite() map[int32]Extension { |
||||||
|
return e.ExtensionMap() |
||||||
|
} |
||||||
|
|
||||||
|
func (e extensionAdapter) extensionsRead() (map[int32]Extension, sync.Locker) { |
||||||
|
return e.ExtensionMap(), notLocker{} |
||||||
|
} |
||||||
|
|
||||||
|
// notLocker is a sync.Locker whose Lock and Unlock methods are nops.
|
||||||
|
type notLocker struct{} |
||||||
|
|
||||||
|
func (n notLocker) Lock() {} |
||||||
|
func (n notLocker) Unlock() {} |
||||||
|
|
||||||
|
// extendable returns the extendableProto interface for the given generated proto message.
|
||||||
|
// If the proto message has the old extension format, it returns a wrapper that implements
|
||||||
|
// the extendableProto interface.
|
||||||
|
func extendable(p interface{}) (extendableProto, bool) { |
||||||
|
if ep, ok := p.(extendableProto); ok { |
||||||
|
return ep, ok |
||||||
|
} |
||||||
|
if ep, ok := p.(extendableProtoV1); ok { |
||||||
|
return extensionAdapter{ep}, ok |
||||||
|
} |
||||||
|
return nil, false |
||||||
|
} |
||||||
|
|
||||||
|
// XXX_InternalExtensions is an internal representation of proto extensions.
|
||||||
|
//
|
||||||
|
// Each generated message struct type embeds an anonymous XXX_InternalExtensions field,
|
||||||
|
// thus gaining the unexported 'extensions' method, which can be called only from the proto package.
|
||||||
|
//
|
||||||
|
// The methods of XXX_InternalExtensions are not concurrency safe in general,
|
||||||
|
// but calls to logically read-only methods such as has and get may be executed concurrently.
|
||||||
|
type XXX_InternalExtensions struct { |
||||||
|
// The struct must be indirect so that if a user inadvertently copies a
|
||||||
|
// generated message and its embedded XXX_InternalExtensions, they
|
||||||
|
// avoid the mayhem of a copied mutex.
|
||||||
|
//
|
||||||
|
// The mutex serializes all logically read-only operations to p.extensionMap.
|
||||||
|
// It is up to the client to ensure that write operations to p.extensionMap are
|
||||||
|
// mutually exclusive with other accesses.
|
||||||
|
p *struct { |
||||||
|
mu sync.Mutex |
||||||
|
extensionMap map[int32]Extension |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// extensionsWrite returns the extension map, creating it on first use.
|
||||||
|
func (e *XXX_InternalExtensions) extensionsWrite() map[int32]Extension { |
||||||
|
if e.p == nil { |
||||||
|
e.p = new(struct { |
||||||
|
mu sync.Mutex |
||||||
|
extensionMap map[int32]Extension |
||||||
|
}) |
||||||
|
e.p.extensionMap = make(map[int32]Extension) |
||||||
|
} |
||||||
|
return e.p.extensionMap |
||||||
|
} |
||||||
|
|
||||||
|
// extensionsRead returns the extensions map for read-only use. It may be nil.
|
||||||
|
// The caller must hold the returned mutex's lock when accessing Elements within the map.
|
||||||
|
func (e *XXX_InternalExtensions) extensionsRead() (map[int32]Extension, sync.Locker) { |
||||||
|
if e.p == nil { |
||||||
|
return nil, nil |
||||||
|
} |
||||||
|
return e.p.extensionMap, &e.p.mu |
||||||
|
} |
||||||
|
|
||||||
|
var extendableProtoType = reflect.TypeOf((*extendableProto)(nil)).Elem() |
||||||
|
var extendableProtoV1Type = reflect.TypeOf((*extendableProtoV1)(nil)).Elem() |
||||||
|
|
||||||
|
// ExtensionDesc represents an extension specification.
|
||||||
|
// Used in generated code from the protocol compiler.
|
||||||
|
type ExtensionDesc struct { |
||||||
|
ExtendedType Message // nil pointer to the type that is being extended
|
||||||
|
ExtensionType interface{} // nil pointer to the extension type
|
||||||
|
Field int32 // field number
|
||||||
|
Name string // fully-qualified name of extension, for text formatting
|
||||||
|
Tag string // protobuf tag style
|
||||||
|
Filename string // name of the file in which the extension is defined
|
||||||
|
} |
||||||
|
|
||||||
|
func (ed *ExtensionDesc) repeated() bool { |
||||||
|
t := reflect.TypeOf(ed.ExtensionType) |
||||||
|
return t.Kind() == reflect.Slice && t.Elem().Kind() != reflect.Uint8 |
||||||
|
} |
||||||
|
|
||||||
|
// Extension represents an extension in a message.
|
||||||
|
type Extension struct { |
||||||
|
// When an extension is stored in a message using SetExtension
|
||||||
|
// only desc and value are set. When the message is marshaled
|
||||||
|
// enc will be set to the encoded form of the message.
|
||||||
|
//
|
||||||
|
// When a message is unmarshaled and contains extensions, each
|
||||||
|
// extension will have only enc set. When such an extension is
|
||||||
|
// accessed using GetExtension (or GetExtensions) desc and value
|
||||||
|
// will be set.
|
||||||
|
desc *ExtensionDesc |
||||||
|
value interface{} |
||||||
|
enc []byte |
||||||
|
} |
||||||
|
|
||||||
|
// SetRawExtension is for testing only.
|
||||||
|
func SetRawExtension(base Message, id int32, b []byte) { |
||||||
|
epb, ok := extendable(base) |
||||||
|
if !ok { |
||||||
|
return |
||||||
|
} |
||||||
|
extmap := epb.extensionsWrite() |
||||||
|
extmap[id] = Extension{enc: b} |
||||||
|
} |
||||||
|
|
||||||
|
// isExtensionField returns true iff the given field number is in an extension range.
|
||||||
|
func isExtensionField(pb extendableProto, field int32) bool { |
||||||
|
for _, er := range pb.ExtensionRangeArray() { |
||||||
|
if er.Start <= field && field <= er.End { |
||||||
|
return true |
||||||
|
} |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
// checkExtensionTypes checks that the given extension is valid for pb.
|
||||||
|
func checkExtensionTypes(pb extendableProto, extension *ExtensionDesc) error { |
||||||
|
var pbi interface{} = pb |
||||||
|
// Check the extended type.
|
||||||
|
if ea, ok := pbi.(extensionAdapter); ok { |
||||||
|
pbi = ea.extendableProtoV1 |
||||||
|
} |
||||||
|
if a, b := reflect.TypeOf(pbi), reflect.TypeOf(extension.ExtendedType); a != b { |
||||||
|
return errors.New("proto: bad extended type; " + b.String() + " does not extend " + a.String()) |
||||||
|
} |
||||||
|
// Check the range.
|
||||||
|
if !isExtensionField(pb, extension.Field) { |
||||||
|
return errors.New("proto: bad extension number; not in declared ranges") |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// extPropKey is sufficient to uniquely identify an extension.
|
||||||
|
type extPropKey struct { |
||||||
|
base reflect.Type |
||||||
|
field int32 |
||||||
|
} |
||||||
|
|
||||||
|
var extProp = struct { |
||||||
|
sync.RWMutex |
||||||
|
m map[extPropKey]*Properties |
||||||
|
}{ |
||||||
|
m: make(map[extPropKey]*Properties), |
||||||
|
} |
||||||
|
|
||||||
|
func extensionProperties(ed *ExtensionDesc) *Properties { |
||||||
|
key := extPropKey{base: reflect.TypeOf(ed.ExtendedType), field: ed.Field} |
||||||
|
|
||||||
|
extProp.RLock() |
||||||
|
if prop, ok := extProp.m[key]; ok { |
||||||
|
extProp.RUnlock() |
||||||
|
return prop |
||||||
|
} |
||||||
|
extProp.RUnlock() |
||||||
|
|
||||||
|
extProp.Lock() |
||||||
|
defer extProp.Unlock() |
||||||
|
// Check again.
|
||||||
|
if prop, ok := extProp.m[key]; ok { |
||||||
|
return prop |
||||||
|
} |
||||||
|
|
||||||
|
prop := new(Properties) |
||||||
|
prop.Init(reflect.TypeOf(ed.ExtensionType), "unknown_name", ed.Tag, nil) |
||||||
|
extProp.m[key] = prop |
||||||
|
return prop |
||||||
|
} |
||||||
|
|
||||||
|
// encode encodes any unmarshaled (unencoded) extensions in e.
|
||||||
|
func encodeExtensions(e *XXX_InternalExtensions) error { |
||||||
|
m, mu := e.extensionsRead() |
||||||
|
if m == nil { |
||||||
|
return nil // fast path
|
||||||
|
} |
||||||
|
mu.Lock() |
||||||
|
defer mu.Unlock() |
||||||
|
return encodeExtensionsMap(m) |
||||||
|
} |
||||||
|
|
||||||
|
// encode encodes any unmarshaled (unencoded) extensions in e.
|
||||||
|
func encodeExtensionsMap(m map[int32]Extension) error { |
||||||
|
for k, e := range m { |
||||||
|
if e.value == nil || e.desc == nil { |
||||||
|
// Extension is only in its encoded form.
|
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
// We don't skip extensions that have an encoded form set,
|
||||||
|
// because the extension value may have been mutated after
|
||||||
|
// the last time this function was called.
|
||||||
|
|
||||||
|
et := reflect.TypeOf(e.desc.ExtensionType) |
||||||
|
props := extensionProperties(e.desc) |
||||||
|
|
||||||
|
p := NewBuffer(nil) |
||||||
|
// If e.value has type T, the encoder expects a *struct{ X T }.
|
||||||
|
// Pass a *T with a zero field and hope it all works out.
|
||||||
|
x := reflect.New(et) |
||||||
|
x.Elem().Set(reflect.ValueOf(e.value)) |
||||||
|
if err := props.enc(p, props, toStructPointer(x)); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
e.enc = p.buf |
||||||
|
m[k] = e |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func extensionsSize(e *XXX_InternalExtensions) (n int) { |
||||||
|
m, mu := e.extensionsRead() |
||||||
|
if m == nil { |
||||||
|
return 0 |
||||||
|
} |
||||||
|
mu.Lock() |
||||||
|
defer mu.Unlock() |
||||||
|
return extensionsMapSize(m) |
||||||
|
} |
||||||
|
|
||||||
|
func extensionsMapSize(m map[int32]Extension) (n int) { |
||||||
|
for _, e := range m { |
||||||
|
if e.value == nil || e.desc == nil { |
||||||
|
// Extension is only in its encoded form.
|
||||||
|
n += len(e.enc) |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
// We don't skip extensions that have an encoded form set,
|
||||||
|
// because the extension value may have been mutated after
|
||||||
|
// the last time this function was called.
|
||||||
|
|
||||||
|
et := reflect.TypeOf(e.desc.ExtensionType) |
||||||
|
props := extensionProperties(e.desc) |
||||||
|
|
||||||
|
// If e.value has type T, the encoder expects a *struct{ X T }.
|
||||||
|
// Pass a *T with a zero field and hope it all works out.
|
||||||
|
x := reflect.New(et) |
||||||
|
x.Elem().Set(reflect.ValueOf(e.value)) |
||||||
|
n += props.size(props, toStructPointer(x)) |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// HasExtension returns whether the given extension is present in pb.
|
||||||
|
func HasExtension(pb Message, extension *ExtensionDesc) bool { |
||||||
|
// TODO: Check types, field numbers, etc.?
|
||||||
|
epb, ok := extendable(pb) |
||||||
|
if !ok { |
||||||
|
return false |
||||||
|
} |
||||||
|
extmap, mu := epb.extensionsRead() |
||||||
|
if extmap == nil { |
||||||
|
return false |
||||||
|
} |
||||||
|
mu.Lock() |
||||||
|
_, ok = extmap[extension.Field] |
||||||
|
mu.Unlock() |
||||||
|
return ok |
||||||
|
} |
||||||
|
|
||||||
|
// ClearExtension removes the given extension from pb.
|
||||||
|
func ClearExtension(pb Message, extension *ExtensionDesc) { |
||||||
|
epb, ok := extendable(pb) |
||||||
|
if !ok { |
||||||
|
return |
||||||
|
} |
||||||
|
// TODO: Check types, field numbers, etc.?
|
||||||
|
extmap := epb.extensionsWrite() |
||||||
|
delete(extmap, extension.Field) |
||||||
|
} |
||||||
|
|
||||||
|
// GetExtension parses and returns the given extension of pb.
|
||||||
|
// If the extension is not present and has no default value it returns ErrMissingExtension.
|
||||||
|
func GetExtension(pb Message, extension *ExtensionDesc) (interface{}, error) { |
||||||
|
epb, ok := extendable(pb) |
||||||
|
if !ok { |
||||||
|
return nil, errors.New("proto: not an extendable proto") |
||||||
|
} |
||||||
|
|
||||||
|
if err := checkExtensionTypes(epb, extension); err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
emap, mu := epb.extensionsRead() |
||||||
|
if emap == nil { |
||||||
|
return defaultExtensionValue(extension) |
||||||
|
} |
||||||
|
mu.Lock() |
||||||
|
defer mu.Unlock() |
||||||
|
e, ok := emap[extension.Field] |
||||||
|
if !ok { |
||||||
|
// defaultExtensionValue returns the default value or
|
||||||
|
// ErrMissingExtension if there is no default.
|
||||||
|
return defaultExtensionValue(extension) |
||||||
|
} |
||||||
|
|
||||||
|
if e.value != nil { |
||||||
|
// Already decoded. Check the descriptor, though.
|
||||||
|
if e.desc != extension { |
||||||
|
// This shouldn't happen. If it does, it means that
|
||||||
|
// GetExtension was called twice with two different
|
||||||
|
// descriptors with the same field number.
|
||||||
|
return nil, errors.New("proto: descriptor conflict") |
||||||
|
} |
||||||
|
return e.value, nil |
||||||
|
} |
||||||
|
|
||||||
|
v, err := decodeExtension(e.enc, extension) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// Remember the decoded version and drop the encoded version.
|
||||||
|
// That way it is safe to mutate what we return.
|
||||||
|
e.value = v |
||||||
|
e.desc = extension |
||||||
|
e.enc = nil |
||||||
|
emap[extension.Field] = e |
||||||
|
return e.value, nil |
||||||
|
} |
||||||
|
|
||||||
|
// defaultExtensionValue returns the default value for extension.
|
||||||
|
// If no default for an extension is defined ErrMissingExtension is returned.
|
||||||
|
func defaultExtensionValue(extension *ExtensionDesc) (interface{}, error) { |
||||||
|
t := reflect.TypeOf(extension.ExtensionType) |
||||||
|
props := extensionProperties(extension) |
||||||
|
|
||||||
|
sf, _, err := fieldDefault(t, props) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
if sf == nil || sf.value == nil { |
||||||
|
// There is no default value.
|
||||||
|
return nil, ErrMissingExtension |
||||||
|
} |
||||||
|
|
||||||
|
if t.Kind() != reflect.Ptr { |
||||||
|
// We do not need to return a Ptr, we can directly return sf.value.
|
||||||
|
return sf.value, nil |
||||||
|
} |
||||||
|
|
||||||
|
// We need to return an interface{} that is a pointer to sf.value.
|
||||||
|
value := reflect.New(t).Elem() |
||||||
|
value.Set(reflect.New(value.Type().Elem())) |
||||||
|
if sf.kind == reflect.Int32 { |
||||||
|
// We may have an int32 or an enum, but the underlying data is int32.
|
||||||
|
// Since we can't set an int32 into a non int32 reflect.value directly
|
||||||
|
// set it as a int32.
|
||||||
|
value.Elem().SetInt(int64(sf.value.(int32))) |
||||||
|
} else { |
||||||
|
value.Elem().Set(reflect.ValueOf(sf.value)) |
||||||
|
} |
||||||
|
return value.Interface(), nil |
||||||
|
} |
||||||
|
|
||||||
|
// decodeExtension decodes an extension encoded in b.
|
||||||
|
func decodeExtension(b []byte, extension *ExtensionDesc) (interface{}, error) { |
||||||
|
o := NewBuffer(b) |
||||||
|
|
||||||
|
t := reflect.TypeOf(extension.ExtensionType) |
||||||
|
|
||||||
|
props := extensionProperties(extension) |
||||||
|
|
||||||
|
// t is a pointer to a struct, pointer to basic type or a slice.
|
||||||
|
// Allocate a "field" to store the pointer/slice itself; the
|
||||||
|
// pointer/slice will be stored here. We pass
|
||||||
|
// the address of this field to props.dec.
|
||||||
|
// This passes a zero field and a *t and lets props.dec
|
||||||
|
// interpret it as a *struct{ x t }.
|
||||||
|
value := reflect.New(t).Elem() |
||||||
|
|
||||||
|
for { |
||||||
|
// Discard wire type and field number varint. It isn't needed.
|
||||||
|
if _, err := o.DecodeVarint(); err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
if err := props.dec(o, props, toStructPointer(value.Addr())); err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
if o.index >= len(o.buf) { |
||||||
|
break |
||||||
|
} |
||||||
|
} |
||||||
|
return value.Interface(), nil |
||||||
|
} |
||||||
|
|
||||||
|
// GetExtensions returns a slice of the extensions present in pb that are also listed in es.
|
||||||
|
// The returned slice has the same length as es; missing extensions will appear as nil elements.
|
||||||
|
func GetExtensions(pb Message, es []*ExtensionDesc) (extensions []interface{}, err error) { |
||||||
|
epb, ok := extendable(pb) |
||||||
|
if !ok { |
||||||
|
return nil, errors.New("proto: not an extendable proto") |
||||||
|
} |
||||||
|
extensions = make([]interface{}, len(es)) |
||||||
|
for i, e := range es { |
||||||
|
extensions[i], err = GetExtension(epb, e) |
||||||
|
if err == ErrMissingExtension { |
||||||
|
err = nil |
||||||
|
} |
||||||
|
if err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// ExtensionDescs returns a new slice containing pb's extension descriptors, in undefined order.
|
||||||
|
// For non-registered extensions, ExtensionDescs returns an incomplete descriptor containing
|
||||||
|
// just the Field field, which defines the extension's field number.
|
||||||
|
func ExtensionDescs(pb Message) ([]*ExtensionDesc, error) { |
||||||
|
epb, ok := extendable(pb) |
||||||
|
if !ok { |
||||||
|
return nil, fmt.Errorf("proto: %T is not an extendable proto.Message", pb) |
||||||
|
} |
||||||
|
registeredExtensions := RegisteredExtensions(pb) |
||||||
|
|
||||||
|
emap, mu := epb.extensionsRead() |
||||||
|
if emap == nil { |
||||||
|
return nil, nil |
||||||
|
} |
||||||
|
mu.Lock() |
||||||
|
defer mu.Unlock() |
||||||
|
extensions := make([]*ExtensionDesc, 0, len(emap)) |
||||||
|
for extid, e := range emap { |
||||||
|
desc := e.desc |
||||||
|
if desc == nil { |
||||||
|
desc = registeredExtensions[extid] |
||||||
|
if desc == nil { |
||||||
|
desc = &ExtensionDesc{Field: extid} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
extensions = append(extensions, desc) |
||||||
|
} |
||||||
|
return extensions, nil |
||||||
|
} |
||||||
|
|
||||||
|
// SetExtension sets the specified extension of pb to the specified value.
|
||||||
|
func SetExtension(pb Message, extension *ExtensionDesc, value interface{}) error { |
||||||
|
epb, ok := extendable(pb) |
||||||
|
if !ok { |
||||||
|
return errors.New("proto: not an extendable proto") |
||||||
|
} |
||||||
|
if err := checkExtensionTypes(epb, extension); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
typ := reflect.TypeOf(extension.ExtensionType) |
||||||
|
if typ != reflect.TypeOf(value) { |
||||||
|
return errors.New("proto: bad extension value type") |
||||||
|
} |
||||||
|
// nil extension values need to be caught early, because the
|
||||||
|
// encoder can't distinguish an ErrNil due to a nil extension
|
||||||
|
// from an ErrNil due to a missing field. Extensions are
|
||||||
|
// always optional, so the encoder would just swallow the error
|
||||||
|
// and drop all the extensions from the encoded message.
|
||||||
|
if reflect.ValueOf(value).IsNil() { |
||||||
|
return fmt.Errorf("proto: SetExtension called with nil value of type %T", value) |
||||||
|
} |
||||||
|
|
||||||
|
extmap := epb.extensionsWrite() |
||||||
|
extmap[extension.Field] = Extension{desc: extension, value: value} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// ClearAllExtensions clears all extensions from pb.
|
||||||
|
func ClearAllExtensions(pb Message) { |
||||||
|
epb, ok := extendable(pb) |
||||||
|
if !ok { |
||||||
|
return |
||||||
|
} |
||||||
|
m := epb.extensionsWrite() |
||||||
|
for k := range m { |
||||||
|
delete(m, k) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// A global registry of extensions.
|
||||||
|
// The generated code will register the generated descriptors by calling RegisterExtension.
|
||||||
|
|
||||||
|
var extensionMaps = make(map[reflect.Type]map[int32]*ExtensionDesc) |
||||||
|
|
||||||
|
// RegisterExtension is called from the generated code.
|
||||||
|
func RegisterExtension(desc *ExtensionDesc) { |
||||||
|
st := reflect.TypeOf(desc.ExtendedType).Elem() |
||||||
|
m := extensionMaps[st] |
||||||
|
if m == nil { |
||||||
|
m = make(map[int32]*ExtensionDesc) |
||||||
|
extensionMaps[st] = m |
||||||
|
} |
||||||
|
if _, ok := m[desc.Field]; ok { |
||||||
|
panic("proto: duplicate extension registered: " + st.String() + " " + strconv.Itoa(int(desc.Field))) |
||||||
|
} |
||||||
|
m[desc.Field] = desc |
||||||
|
} |
||||||
|
|
||||||
|
// RegisteredExtensions returns a map of the registered extensions of a
|
||||||
|
// protocol buffer struct, indexed by the extension number.
|
||||||
|
// The argument pb should be a nil pointer to the struct type.
|
||||||
|
func RegisteredExtensions(pb Message) map[int32]*ExtensionDesc { |
||||||
|
return extensionMaps[reflect.TypeOf(pb).Elem()] |
||||||
|
} |
||||||
@ -0,0 +1,898 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
/* |
||||||
|
Package proto converts data structures to and from the wire format of |
||||||
|
protocol buffers. It works in concert with the Go source code generated |
||||||
|
for .proto files by the protocol compiler. |
||||||
|
|
||||||
|
A summary of the properties of the protocol buffer interface |
||||||
|
for a protocol buffer variable v: |
||||||
|
|
||||||
|
- Names are turned from camel_case to CamelCase for export. |
||||||
|
- There are no methods on v to set fields; just treat |
||||||
|
them as structure fields. |
||||||
|
- There are getters that return a field's value if set, |
||||||
|
and return the field's default value if unset. |
||||||
|
The getters work even if the receiver is a nil message. |
||||||
|
- The zero value for a struct is its correct initialization state. |
||||||
|
All desired fields must be set before marshaling. |
||||||
|
- A Reset() method will restore a protobuf struct to its zero state. |
||||||
|
- Non-repeated fields are pointers to the values; nil means unset. |
||||||
|
That is, optional or required field int32 f becomes F *int32. |
||||||
|
- Repeated fields are slices. |
||||||
|
- Helper functions are available to aid the setting of fields. |
||||||
|
msg.Foo = proto.String("hello") // set field
|
||||||
|
- Constants are defined to hold the default values of all fields that |
||||||
|
have them. They have the form Default_StructName_FieldName. |
||||||
|
Because the getter methods handle defaulted values, |
||||||
|
direct use of these constants should be rare. |
||||||
|
- Enums are given type names and maps from names to values. |
||||||
|
Enum values are prefixed by the enclosing message's name, or by the |
||||||
|
enum's type name if it is a top-level enum. Enum types have a String |
||||||
|
method, and a Enum method to assist in message construction. |
||||||
|
- Nested messages, groups and enums have type names prefixed with the name of |
||||||
|
the surrounding message type. |
||||||
|
- Extensions are given descriptor names that start with E_, |
||||||
|
followed by an underscore-delimited list of the nested messages |
||||||
|
that contain it (if any) followed by the CamelCased name of the |
||||||
|
extension field itself. HasExtension, ClearExtension, GetExtension |
||||||
|
and SetExtension are functions for manipulating extensions. |
||||||
|
- Oneof field sets are given a single field in their message, |
||||||
|
with distinguished wrapper types for each possible field value. |
||||||
|
- Marshal and Unmarshal are functions to encode and decode the wire format. |
||||||
|
|
||||||
|
When the .proto file specifies `syntax="proto3"`, there are some differences: |
||||||
|
|
||||||
|
- Non-repeated fields of non-message type are values instead of pointers. |
||||||
|
- Getters are only generated for message and oneof fields. |
||||||
|
- Enum types do not get an Enum method. |
||||||
|
|
||||||
|
The simplest way to describe this is to see an example. |
||||||
|
Given file test.proto, containing |
||||||
|
|
||||||
|
package example; |
||||||
|
|
||||||
|
enum FOO { X = 17; } |
||||||
|
|
||||||
|
message Test { |
||||||
|
required string label = 1; |
||||||
|
optional int32 type = 2 [default=77]; |
||||||
|
repeated int64 reps = 3; |
||||||
|
optional group OptionalGroup = 4 { |
||||||
|
required string RequiredField = 5; |
||||||
|
} |
||||||
|
oneof union { |
||||||
|
int32 number = 6; |
||||||
|
string name = 7; |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
The resulting file, test.pb.go, is: |
||||||
|
|
||||||
|
package example |
||||||
|
|
||||||
|
import proto "github.com/golang/protobuf/proto" |
||||||
|
import math "math" |
||||||
|
|
||||||
|
type FOO int32 |
||||||
|
const ( |
||||||
|
FOO_X FOO = 17 |
||||||
|
) |
||||||
|
var FOO_name = map[int32]string{ |
||||||
|
17: "X", |
||||||
|
} |
||||||
|
var FOO_value = map[string]int32{ |
||||||
|
"X": 17, |
||||||
|
} |
||||||
|
|
||||||
|
func (x FOO) Enum() *FOO { |
||||||
|
p := new(FOO) |
||||||
|
*p = x |
||||||
|
return p |
||||||
|
} |
||||||
|
func (x FOO) String() string { |
||||||
|
return proto.EnumName(FOO_name, int32(x)) |
||||||
|
} |
||||||
|
func (x *FOO) UnmarshalJSON(data []byte) error { |
||||||
|
value, err := proto.UnmarshalJSONEnum(FOO_value, data) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*x = FOO(value) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
type Test struct { |
||||||
|
Label *string `protobuf:"bytes,1,req,name=label" json:"label,omitempty"` |
||||||
|
Type *int32 `protobuf:"varint,2,opt,name=type,def=77" json:"type,omitempty"` |
||||||
|
Reps []int64 `protobuf:"varint,3,rep,name=reps" json:"reps,omitempty"` |
||||||
|
Optionalgroup *Test_OptionalGroup `protobuf:"group,4,opt,name=OptionalGroup" json:"optionalgroup,omitempty"` |
||||||
|
// Types that are valid to be assigned to Union:
|
||||||
|
// *Test_Number
|
||||||
|
// *Test_Name
|
||||||
|
Union isTest_Union `protobuf_oneof:"union"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
func (m *Test) Reset() { *m = Test{} } |
||||||
|
func (m *Test) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Test) ProtoMessage() {} |
||||||
|
|
||||||
|
type isTest_Union interface { |
||||||
|
isTest_Union() |
||||||
|
} |
||||||
|
|
||||||
|
type Test_Number struct { |
||||||
|
Number int32 `protobuf:"varint,6,opt,name=number"` |
||||||
|
} |
||||||
|
type Test_Name struct { |
||||||
|
Name string `protobuf:"bytes,7,opt,name=name"` |
||||||
|
} |
||||||
|
|
||||||
|
func (*Test_Number) isTest_Union() {} |
||||||
|
func (*Test_Name) isTest_Union() {} |
||||||
|
|
||||||
|
func (m *Test) GetUnion() isTest_Union { |
||||||
|
if m != nil { |
||||||
|
return m.Union |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
const Default_Test_Type int32 = 77 |
||||||
|
|
||||||
|
func (m *Test) GetLabel() string { |
||||||
|
if m != nil && m.Label != nil { |
||||||
|
return *m.Label |
||||||
|
} |
||||||
|
return "" |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Test) GetType() int32 { |
||||||
|
if m != nil && m.Type != nil { |
||||||
|
return *m.Type |
||||||
|
} |
||||||
|
return Default_Test_Type |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Test) GetOptionalgroup() *Test_OptionalGroup { |
||||||
|
if m != nil { |
||||||
|
return m.Optionalgroup |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
type Test_OptionalGroup struct { |
||||||
|
RequiredField *string `protobuf:"bytes,5,req" json:"RequiredField,omitempty"` |
||||||
|
} |
||||||
|
func (m *Test_OptionalGroup) Reset() { *m = Test_OptionalGroup{} } |
||||||
|
func (m *Test_OptionalGroup) String() string { return proto.CompactTextString(m) } |
||||||
|
|
||||||
|
func (m *Test_OptionalGroup) GetRequiredField() string { |
||||||
|
if m != nil && m.RequiredField != nil { |
||||||
|
return *m.RequiredField |
||||||
|
} |
||||||
|
return "" |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Test) GetNumber() int32 { |
||||||
|
if x, ok := m.GetUnion().(*Test_Number); ok { |
||||||
|
return x.Number |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Test) GetName() string { |
||||||
|
if x, ok := m.GetUnion().(*Test_Name); ok { |
||||||
|
return x.Name |
||||||
|
} |
||||||
|
return "" |
||||||
|
} |
||||||
|
|
||||||
|
func init() { |
||||||
|
proto.RegisterEnum("example.FOO", FOO_name, FOO_value) |
||||||
|
} |
||||||
|
|
||||||
|
To create and play with a Test object: |
||||||
|
|
||||||
|
package main |
||||||
|
|
||||||
|
import ( |
||||||
|
"log" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
pb "./example.pb" |
||||||
|
) |
||||||
|
|
||||||
|
func main() { |
||||||
|
test := &pb.Test{ |
||||||
|
Label: proto.String("hello"), |
||||||
|
Type: proto.Int32(17), |
||||||
|
Reps: []int64{1, 2, 3}, |
||||||
|
Optionalgroup: &pb.Test_OptionalGroup{ |
||||||
|
RequiredField: proto.String("good bye"), |
||||||
|
}, |
||||||
|
Union: &pb.Test_Name{"fred"}, |
||||||
|
} |
||||||
|
data, err := proto.Marshal(test) |
||||||
|
if err != nil { |
||||||
|
log.Fatal("marshaling error: ", err) |
||||||
|
} |
||||||
|
newTest := &pb.Test{} |
||||||
|
err = proto.Unmarshal(data, newTest) |
||||||
|
if err != nil { |
||||||
|
log.Fatal("unmarshaling error: ", err) |
||||||
|
} |
||||||
|
// Now test and newTest contain the same data.
|
||||||
|
if test.GetLabel() != newTest.GetLabel() { |
||||||
|
log.Fatalf("data mismatch %q != %q", test.GetLabel(), newTest.GetLabel()) |
||||||
|
} |
||||||
|
// Use a type switch to determine which oneof was set.
|
||||||
|
switch u := test.Union.(type) { |
||||||
|
case *pb.Test_Number: // u.Number contains the number.
|
||||||
|
case *pb.Test_Name: // u.Name contains the string.
|
||||||
|
} |
||||||
|
// etc.
|
||||||
|
} |
||||||
|
*/ |
||||||
|
package proto |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"fmt" |
||||||
|
"log" |
||||||
|
"reflect" |
||||||
|
"sort" |
||||||
|
"strconv" |
||||||
|
"sync" |
||||||
|
) |
||||||
|
|
||||||
|
// Message is implemented by generated protocol buffer messages.
|
||||||
|
type Message interface { |
||||||
|
Reset() |
||||||
|
String() string |
||||||
|
ProtoMessage() |
||||||
|
} |
||||||
|
|
||||||
|
// Stats records allocation details about the protocol buffer encoders
|
||||||
|
// and decoders. Useful for tuning the library itself.
|
||||||
|
type Stats struct { |
||||||
|
Emalloc uint64 // mallocs in encode
|
||||||
|
Dmalloc uint64 // mallocs in decode
|
||||||
|
Encode uint64 // number of encodes
|
||||||
|
Decode uint64 // number of decodes
|
||||||
|
Chit uint64 // number of cache hits
|
||||||
|
Cmiss uint64 // number of cache misses
|
||||||
|
Size uint64 // number of sizes
|
||||||
|
} |
||||||
|
|
||||||
|
// Set to true to enable stats collection.
|
||||||
|
const collectStats = false |
||||||
|
|
||||||
|
var stats Stats |
||||||
|
|
||||||
|
// GetStats returns a copy of the global Stats structure.
|
||||||
|
func GetStats() Stats { return stats } |
||||||
|
|
||||||
|
// A Buffer is a buffer manager for marshaling and unmarshaling
|
||||||
|
// protocol buffers. It may be reused between invocations to
|
||||||
|
// reduce memory usage. It is not necessary to use a Buffer;
|
||||||
|
// the global functions Marshal and Unmarshal create a
|
||||||
|
// temporary Buffer and are fine for most applications.
|
||||||
|
type Buffer struct { |
||||||
|
buf []byte // encode/decode byte stream
|
||||||
|
index int // read point
|
||||||
|
|
||||||
|
// pools of basic types to amortize allocation.
|
||||||
|
bools []bool |
||||||
|
uint32s []uint32 |
||||||
|
uint64s []uint64 |
||||||
|
|
||||||
|
// extra pools, only used with pointer_reflect.go
|
||||||
|
int32s []int32 |
||||||
|
int64s []int64 |
||||||
|
float32s []float32 |
||||||
|
float64s []float64 |
||||||
|
} |
||||||
|
|
||||||
|
// NewBuffer allocates a new Buffer and initializes its internal data to
|
||||||
|
// the contents of the argument slice.
|
||||||
|
func NewBuffer(e []byte) *Buffer { |
||||||
|
return &Buffer{buf: e} |
||||||
|
} |
||||||
|
|
||||||
|
// Reset resets the Buffer, ready for marshaling a new protocol buffer.
|
||||||
|
func (p *Buffer) Reset() { |
||||||
|
p.buf = p.buf[0:0] // for reading/writing
|
||||||
|
p.index = 0 // for reading
|
||||||
|
} |
||||||
|
|
||||||
|
// SetBuf replaces the internal buffer with the slice,
|
||||||
|
// ready for unmarshaling the contents of the slice.
|
||||||
|
func (p *Buffer) SetBuf(s []byte) { |
||||||
|
p.buf = s |
||||||
|
p.index = 0 |
||||||
|
} |
||||||
|
|
||||||
|
// Bytes returns the contents of the Buffer.
|
||||||
|
func (p *Buffer) Bytes() []byte { return p.buf } |
||||||
|
|
||||||
|
/* |
||||||
|
* Helper routines for simplifying the creation of optional fields of basic type. |
||||||
|
*/ |
||||||
|
|
||||||
|
// Bool is a helper routine that allocates a new bool value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func Bool(v bool) *bool { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// Int32 is a helper routine that allocates a new int32 value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func Int32(v int32) *int32 { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// Int is a helper routine that allocates a new int32 value
|
||||||
|
// to store v and returns a pointer to it, but unlike Int32
|
||||||
|
// its argument value is an int.
|
||||||
|
func Int(v int) *int32 { |
||||||
|
p := new(int32) |
||||||
|
*p = int32(v) |
||||||
|
return p |
||||||
|
} |
||||||
|
|
||||||
|
// Int64 is a helper routine that allocates a new int64 value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func Int64(v int64) *int64 { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// Float32 is a helper routine that allocates a new float32 value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func Float32(v float32) *float32 { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// Float64 is a helper routine that allocates a new float64 value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func Float64(v float64) *float64 { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// Uint32 is a helper routine that allocates a new uint32 value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func Uint32(v uint32) *uint32 { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// Uint64 is a helper routine that allocates a new uint64 value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func Uint64(v uint64) *uint64 { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// String is a helper routine that allocates a new string value
|
||||||
|
// to store v and returns a pointer to it.
|
||||||
|
func String(v string) *string { |
||||||
|
return &v |
||||||
|
} |
||||||
|
|
||||||
|
// EnumName is a helper function to simplify printing protocol buffer enums
|
||||||
|
// by name. Given an enum map and a value, it returns a useful string.
|
||||||
|
func EnumName(m map[int32]string, v int32) string { |
||||||
|
s, ok := m[v] |
||||||
|
if ok { |
||||||
|
return s |
||||||
|
} |
||||||
|
return strconv.Itoa(int(v)) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSONEnum is a helper function to simplify recovering enum int values
|
||||||
|
// from their JSON-encoded representation. Given a map from the enum's symbolic
|
||||||
|
// names to its int values, and a byte buffer containing the JSON-encoded
|
||||||
|
// value, it returns an int32 that can be cast to the enum type by the caller.
|
||||||
|
//
|
||||||
|
// The function can deal with both JSON representations, numeric and symbolic.
|
||||||
|
func UnmarshalJSONEnum(m map[string]int32, data []byte, enumName string) (int32, error) { |
||||||
|
if data[0] == '"' { |
||||||
|
// New style: enums are strings.
|
||||||
|
var repr string |
||||||
|
if err := json.Unmarshal(data, &repr); err != nil { |
||||||
|
return -1, err |
||||||
|
} |
||||||
|
val, ok := m[repr] |
||||||
|
if !ok { |
||||||
|
return 0, fmt.Errorf("unrecognized enum %s value %q", enumName, repr) |
||||||
|
} |
||||||
|
return val, nil |
||||||
|
} |
||||||
|
// Old style: enums are ints.
|
||||||
|
var val int32 |
||||||
|
if err := json.Unmarshal(data, &val); err != nil { |
||||||
|
return 0, fmt.Errorf("cannot unmarshal %#q into enum %s", data, enumName) |
||||||
|
} |
||||||
|
return val, nil |
||||||
|
} |
||||||
|
|
||||||
|
// DebugPrint dumps the encoded data in b in a debugging format with a header
|
||||||
|
// including the string s. Used in testing but made available for general debugging.
|
||||||
|
func (p *Buffer) DebugPrint(s string, b []byte) { |
||||||
|
var u uint64 |
||||||
|
|
||||||
|
obuf := p.buf |
||||||
|
index := p.index |
||||||
|
p.buf = b |
||||||
|
p.index = 0 |
||||||
|
depth := 0 |
||||||
|
|
||||||
|
fmt.Printf("\n--- %s ---\n", s) |
||||||
|
|
||||||
|
out: |
||||||
|
for { |
||||||
|
for i := 0; i < depth; i++ { |
||||||
|
fmt.Print(" ") |
||||||
|
} |
||||||
|
|
||||||
|
index := p.index |
||||||
|
if index == len(p.buf) { |
||||||
|
break |
||||||
|
} |
||||||
|
|
||||||
|
op, err := p.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
fmt.Printf("%3d: fetching op err %v\n", index, err) |
||||||
|
break out |
||||||
|
} |
||||||
|
tag := op >> 3 |
||||||
|
wire := op & 7 |
||||||
|
|
||||||
|
switch wire { |
||||||
|
default: |
||||||
|
fmt.Printf("%3d: t=%3d unknown wire=%d\n", |
||||||
|
index, tag, wire) |
||||||
|
break out |
||||||
|
|
||||||
|
case WireBytes: |
||||||
|
var r []byte |
||||||
|
|
||||||
|
r, err = p.DecodeRawBytes(false) |
||||||
|
if err != nil { |
||||||
|
break out |
||||||
|
} |
||||||
|
fmt.Printf("%3d: t=%3d bytes [%d]", index, tag, len(r)) |
||||||
|
if len(r) <= 6 { |
||||||
|
for i := 0; i < len(r); i++ { |
||||||
|
fmt.Printf(" %.2x", r[i]) |
||||||
|
} |
||||||
|
} else { |
||||||
|
for i := 0; i < 3; i++ { |
||||||
|
fmt.Printf(" %.2x", r[i]) |
||||||
|
} |
||||||
|
fmt.Printf(" ..") |
||||||
|
for i := len(r) - 3; i < len(r); i++ { |
||||||
|
fmt.Printf(" %.2x", r[i]) |
||||||
|
} |
||||||
|
} |
||||||
|
fmt.Printf("\n") |
||||||
|
|
||||||
|
case WireFixed32: |
||||||
|
u, err = p.DecodeFixed32() |
||||||
|
if err != nil { |
||||||
|
fmt.Printf("%3d: t=%3d fix32 err %v\n", index, tag, err) |
||||||
|
break out |
||||||
|
} |
||||||
|
fmt.Printf("%3d: t=%3d fix32 %d\n", index, tag, u) |
||||||
|
|
||||||
|
case WireFixed64: |
||||||
|
u, err = p.DecodeFixed64() |
||||||
|
if err != nil { |
||||||
|
fmt.Printf("%3d: t=%3d fix64 err %v\n", index, tag, err) |
||||||
|
break out |
||||||
|
} |
||||||
|
fmt.Printf("%3d: t=%3d fix64 %d\n", index, tag, u) |
||||||
|
|
||||||
|
case WireVarint: |
||||||
|
u, err = p.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
fmt.Printf("%3d: t=%3d varint err %v\n", index, tag, err) |
||||||
|
break out |
||||||
|
} |
||||||
|
fmt.Printf("%3d: t=%3d varint %d\n", index, tag, u) |
||||||
|
|
||||||
|
case WireStartGroup: |
||||||
|
fmt.Printf("%3d: t=%3d start\n", index, tag) |
||||||
|
depth++ |
||||||
|
|
||||||
|
case WireEndGroup: |
||||||
|
depth-- |
||||||
|
fmt.Printf("%3d: t=%3d end\n", index, tag) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if depth != 0 { |
||||||
|
fmt.Printf("%3d: start-end not balanced %d\n", p.index, depth) |
||||||
|
} |
||||||
|
fmt.Printf("\n") |
||||||
|
|
||||||
|
p.buf = obuf |
||||||
|
p.index = index |
||||||
|
} |
||||||
|
|
||||||
|
// SetDefaults sets unset protocol buffer fields to their default values.
|
||||||
|
// It only modifies fields that are both unset and have defined defaults.
|
||||||
|
// It recursively sets default values in any non-nil sub-messages.
|
||||||
|
func SetDefaults(pb Message) { |
||||||
|
setDefaults(reflect.ValueOf(pb), true, false) |
||||||
|
} |
||||||
|
|
||||||
|
// v is a pointer to a struct.
|
||||||
|
func setDefaults(v reflect.Value, recur, zeros bool) { |
||||||
|
v = v.Elem() |
||||||
|
|
||||||
|
defaultMu.RLock() |
||||||
|
dm, ok := defaults[v.Type()] |
||||||
|
defaultMu.RUnlock() |
||||||
|
if !ok { |
||||||
|
dm = buildDefaultMessage(v.Type()) |
||||||
|
defaultMu.Lock() |
||||||
|
defaults[v.Type()] = dm |
||||||
|
defaultMu.Unlock() |
||||||
|
} |
||||||
|
|
||||||
|
for _, sf := range dm.scalars { |
||||||
|
f := v.Field(sf.index) |
||||||
|
if !f.IsNil() { |
||||||
|
// field already set
|
||||||
|
continue |
||||||
|
} |
||||||
|
dv := sf.value |
||||||
|
if dv == nil && !zeros { |
||||||
|
// no explicit default, and don't want to set zeros
|
||||||
|
continue |
||||||
|
} |
||||||
|
fptr := f.Addr().Interface() // **T
|
||||||
|
// TODO: Consider batching the allocations we do here.
|
||||||
|
switch sf.kind { |
||||||
|
case reflect.Bool: |
||||||
|
b := new(bool) |
||||||
|
if dv != nil { |
||||||
|
*b = dv.(bool) |
||||||
|
} |
||||||
|
*(fptr.(**bool)) = b |
||||||
|
case reflect.Float32: |
||||||
|
f := new(float32) |
||||||
|
if dv != nil { |
||||||
|
*f = dv.(float32) |
||||||
|
} |
||||||
|
*(fptr.(**float32)) = f |
||||||
|
case reflect.Float64: |
||||||
|
f := new(float64) |
||||||
|
if dv != nil { |
||||||
|
*f = dv.(float64) |
||||||
|
} |
||||||
|
*(fptr.(**float64)) = f |
||||||
|
case reflect.Int32: |
||||||
|
// might be an enum
|
||||||
|
if ft := f.Type(); ft != int32PtrType { |
||||||
|
// enum
|
||||||
|
f.Set(reflect.New(ft.Elem())) |
||||||
|
if dv != nil { |
||||||
|
f.Elem().SetInt(int64(dv.(int32))) |
||||||
|
} |
||||||
|
} else { |
||||||
|
// int32 field
|
||||||
|
i := new(int32) |
||||||
|
if dv != nil { |
||||||
|
*i = dv.(int32) |
||||||
|
} |
||||||
|
*(fptr.(**int32)) = i |
||||||
|
} |
||||||
|
case reflect.Int64: |
||||||
|
i := new(int64) |
||||||
|
if dv != nil { |
||||||
|
*i = dv.(int64) |
||||||
|
} |
||||||
|
*(fptr.(**int64)) = i |
||||||
|
case reflect.String: |
||||||
|
s := new(string) |
||||||
|
if dv != nil { |
||||||
|
*s = dv.(string) |
||||||
|
} |
||||||
|
*(fptr.(**string)) = s |
||||||
|
case reflect.Uint8: |
||||||
|
// exceptional case: []byte
|
||||||
|
var b []byte |
||||||
|
if dv != nil { |
||||||
|
db := dv.([]byte) |
||||||
|
b = make([]byte, len(db)) |
||||||
|
copy(b, db) |
||||||
|
} else { |
||||||
|
b = []byte{} |
||||||
|
} |
||||||
|
*(fptr.(*[]byte)) = b |
||||||
|
case reflect.Uint32: |
||||||
|
u := new(uint32) |
||||||
|
if dv != nil { |
||||||
|
*u = dv.(uint32) |
||||||
|
} |
||||||
|
*(fptr.(**uint32)) = u |
||||||
|
case reflect.Uint64: |
||||||
|
u := new(uint64) |
||||||
|
if dv != nil { |
||||||
|
*u = dv.(uint64) |
||||||
|
} |
||||||
|
*(fptr.(**uint64)) = u |
||||||
|
default: |
||||||
|
log.Printf("proto: can't set default for field %v (sf.kind=%v)", f, sf.kind) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
for _, ni := range dm.nested { |
||||||
|
f := v.Field(ni) |
||||||
|
// f is *T or []*T or map[T]*T
|
||||||
|
switch f.Kind() { |
||||||
|
case reflect.Ptr: |
||||||
|
if f.IsNil() { |
||||||
|
continue |
||||||
|
} |
||||||
|
setDefaults(f, recur, zeros) |
||||||
|
|
||||||
|
case reflect.Slice: |
||||||
|
for i := 0; i < f.Len(); i++ { |
||||||
|
e := f.Index(i) |
||||||
|
if e.IsNil() { |
||||||
|
continue |
||||||
|
} |
||||||
|
setDefaults(e, recur, zeros) |
||||||
|
} |
||||||
|
|
||||||
|
case reflect.Map: |
||||||
|
for _, k := range f.MapKeys() { |
||||||
|
e := f.MapIndex(k) |
||||||
|
if e.IsNil() { |
||||||
|
continue |
||||||
|
} |
||||||
|
setDefaults(e, recur, zeros) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
var ( |
||||||
|
// defaults maps a protocol buffer struct type to a slice of the fields,
|
||||||
|
// with its scalar fields set to their proto-declared non-zero default values.
|
||||||
|
defaultMu sync.RWMutex |
||||||
|
defaults = make(map[reflect.Type]defaultMessage) |
||||||
|
|
||||||
|
int32PtrType = reflect.TypeOf((*int32)(nil)) |
||||||
|
) |
||||||
|
|
||||||
|
// defaultMessage represents information about the default values of a message.
|
||||||
|
type defaultMessage struct { |
||||||
|
scalars []scalarField |
||||||
|
nested []int // struct field index of nested messages
|
||||||
|
} |
||||||
|
|
||||||
|
type scalarField struct { |
||||||
|
index int // struct field index
|
||||||
|
kind reflect.Kind // element type (the T in *T or []T)
|
||||||
|
value interface{} // the proto-declared default value, or nil
|
||||||
|
} |
||||||
|
|
||||||
|
// t is a struct type.
|
||||||
|
func buildDefaultMessage(t reflect.Type) (dm defaultMessage) { |
||||||
|
sprop := GetProperties(t) |
||||||
|
for _, prop := range sprop.Prop { |
||||||
|
fi, ok := sprop.decoderTags.get(prop.Tag) |
||||||
|
if !ok { |
||||||
|
// XXX_unrecognized
|
||||||
|
continue |
||||||
|
} |
||||||
|
ft := t.Field(fi).Type |
||||||
|
|
||||||
|
sf, nested, err := fieldDefault(ft, prop) |
||||||
|
switch { |
||||||
|
case err != nil: |
||||||
|
log.Print(err) |
||||||
|
case nested: |
||||||
|
dm.nested = append(dm.nested, fi) |
||||||
|
case sf != nil: |
||||||
|
sf.index = fi |
||||||
|
dm.scalars = append(dm.scalars, *sf) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return dm |
||||||
|
} |
||||||
|
|
||||||
|
// fieldDefault returns the scalarField for field type ft.
|
||||||
|
// sf will be nil if the field can not have a default.
|
||||||
|
// nestedMessage will be true if this is a nested message.
|
||||||
|
// Note that sf.index is not set on return.
|
||||||
|
func fieldDefault(ft reflect.Type, prop *Properties) (sf *scalarField, nestedMessage bool, err error) { |
||||||
|
var canHaveDefault bool |
||||||
|
switch ft.Kind() { |
||||||
|
case reflect.Ptr: |
||||||
|
if ft.Elem().Kind() == reflect.Struct { |
||||||
|
nestedMessage = true |
||||||
|
} else { |
||||||
|
canHaveDefault = true // proto2 scalar field
|
||||||
|
} |
||||||
|
|
||||||
|
case reflect.Slice: |
||||||
|
switch ft.Elem().Kind() { |
||||||
|
case reflect.Ptr: |
||||||
|
nestedMessage = true // repeated message
|
||||||
|
case reflect.Uint8: |
||||||
|
canHaveDefault = true // bytes field
|
||||||
|
} |
||||||
|
|
||||||
|
case reflect.Map: |
||||||
|
if ft.Elem().Kind() == reflect.Ptr { |
||||||
|
nestedMessage = true // map with message values
|
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if !canHaveDefault { |
||||||
|
if nestedMessage { |
||||||
|
return nil, true, nil |
||||||
|
} |
||||||
|
return nil, false, nil |
||||||
|
} |
||||||
|
|
||||||
|
// We now know that ft is a pointer or slice.
|
||||||
|
sf = &scalarField{kind: ft.Elem().Kind()} |
||||||
|
|
||||||
|
// scalar fields without defaults
|
||||||
|
if !prop.HasDefault { |
||||||
|
return sf, false, nil |
||||||
|
} |
||||||
|
|
||||||
|
// a scalar field: either *T or []byte
|
||||||
|
switch ft.Elem().Kind() { |
||||||
|
case reflect.Bool: |
||||||
|
x, err := strconv.ParseBool(prop.Default) |
||||||
|
if err != nil { |
||||||
|
return nil, false, fmt.Errorf("proto: bad default bool %q: %v", prop.Default, err) |
||||||
|
} |
||||||
|
sf.value = x |
||||||
|
case reflect.Float32: |
||||||
|
x, err := strconv.ParseFloat(prop.Default, 32) |
||||||
|
if err != nil { |
||||||
|
return nil, false, fmt.Errorf("proto: bad default float32 %q: %v", prop.Default, err) |
||||||
|
} |
||||||
|
sf.value = float32(x) |
||||||
|
case reflect.Float64: |
||||||
|
x, err := strconv.ParseFloat(prop.Default, 64) |
||||||
|
if err != nil { |
||||||
|
return nil, false, fmt.Errorf("proto: bad default float64 %q: %v", prop.Default, err) |
||||||
|
} |
||||||
|
sf.value = x |
||||||
|
case reflect.Int32: |
||||||
|
x, err := strconv.ParseInt(prop.Default, 10, 32) |
||||||
|
if err != nil { |
||||||
|
return nil, false, fmt.Errorf("proto: bad default int32 %q: %v", prop.Default, err) |
||||||
|
} |
||||||
|
sf.value = int32(x) |
||||||
|
case reflect.Int64: |
||||||
|
x, err := strconv.ParseInt(prop.Default, 10, 64) |
||||||
|
if err != nil { |
||||||
|
return nil, false, fmt.Errorf("proto: bad default int64 %q: %v", prop.Default, err) |
||||||
|
} |
||||||
|
sf.value = x |
||||||
|
case reflect.String: |
||||||
|
sf.value = prop.Default |
||||||
|
case reflect.Uint8: |
||||||
|
// []byte (not *uint8)
|
||||||
|
sf.value = []byte(prop.Default) |
||||||
|
case reflect.Uint32: |
||||||
|
x, err := strconv.ParseUint(prop.Default, 10, 32) |
||||||
|
if err != nil { |
||||||
|
return nil, false, fmt.Errorf("proto: bad default uint32 %q: %v", prop.Default, err) |
||||||
|
} |
||||||
|
sf.value = uint32(x) |
||||||
|
case reflect.Uint64: |
||||||
|
x, err := strconv.ParseUint(prop.Default, 10, 64) |
||||||
|
if err != nil { |
||||||
|
return nil, false, fmt.Errorf("proto: bad default uint64 %q: %v", prop.Default, err) |
||||||
|
} |
||||||
|
sf.value = x |
||||||
|
default: |
||||||
|
return nil, false, fmt.Errorf("proto: unhandled def kind %v", ft.Elem().Kind()) |
||||||
|
} |
||||||
|
|
||||||
|
return sf, false, nil |
||||||
|
} |
||||||
|
|
||||||
|
// Map fields may have key types of non-float scalars, strings and enums.
|
||||||
|
// The easiest way to sort them in some deterministic order is to use fmt.
|
||||||
|
// If this turns out to be inefficient we can always consider other options,
|
||||||
|
// such as doing a Schwartzian transform.
|
||||||
|
|
||||||
|
func mapKeys(vs []reflect.Value) sort.Interface { |
||||||
|
s := mapKeySorter{ |
||||||
|
vs: vs, |
||||||
|
// default Less function: textual comparison
|
||||||
|
less: func(a, b reflect.Value) bool { |
||||||
|
return fmt.Sprint(a.Interface()) < fmt.Sprint(b.Interface()) |
||||||
|
}, |
||||||
|
} |
||||||
|
|
||||||
|
// Type specialization per https://developers.google.com/protocol-buffers/docs/proto#maps;
|
||||||
|
// numeric keys are sorted numerically.
|
||||||
|
if len(vs) == 0 { |
||||||
|
return s |
||||||
|
} |
||||||
|
switch vs[0].Kind() { |
||||||
|
case reflect.Int32, reflect.Int64: |
||||||
|
s.less = func(a, b reflect.Value) bool { return a.Int() < b.Int() } |
||||||
|
case reflect.Uint32, reflect.Uint64: |
||||||
|
s.less = func(a, b reflect.Value) bool { return a.Uint() < b.Uint() } |
||||||
|
} |
||||||
|
|
||||||
|
return s |
||||||
|
} |
||||||
|
|
||||||
|
type mapKeySorter struct { |
||||||
|
vs []reflect.Value |
||||||
|
less func(a, b reflect.Value) bool |
||||||
|
} |
||||||
|
|
||||||
|
func (s mapKeySorter) Len() int { return len(s.vs) } |
||||||
|
func (s mapKeySorter) Swap(i, j int) { s.vs[i], s.vs[j] = s.vs[j], s.vs[i] } |
||||||
|
func (s mapKeySorter) Less(i, j int) bool { |
||||||
|
return s.less(s.vs[i], s.vs[j]) |
||||||
|
} |
||||||
|
|
||||||
|
// isProto3Zero reports whether v is a zero proto3 value.
|
||||||
|
func isProto3Zero(v reflect.Value) bool { |
||||||
|
switch v.Kind() { |
||||||
|
case reflect.Bool: |
||||||
|
return !v.Bool() |
||||||
|
case reflect.Int32, reflect.Int64: |
||||||
|
return v.Int() == 0 |
||||||
|
case reflect.Uint32, reflect.Uint64: |
||||||
|
return v.Uint() == 0 |
||||||
|
case reflect.Float32, reflect.Float64: |
||||||
|
return v.Float() == 0 |
||||||
|
case reflect.String: |
||||||
|
return v.String() == "" |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
// ProtoPackageIsVersion2 is referenced from generated protocol buffer files
|
||||||
|
// to assert that that code is compatible with this version of the proto package.
|
||||||
|
const ProtoPackageIsVersion2 = true |
||||||
|
|
||||||
|
// ProtoPackageIsVersion1 is referenced from generated protocol buffer files
|
||||||
|
// to assert that that code is compatible with this version of the proto package.
|
||||||
|
const ProtoPackageIsVersion1 = true |
||||||
@ -0,0 +1,311 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
/* |
||||||
|
* Support for message sets. |
||||||
|
*/ |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"encoding/json" |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"reflect" |
||||||
|
"sort" |
||||||
|
) |
||||||
|
|
||||||
|
// errNoMessageTypeID occurs when a protocol buffer does not have a message type ID.
|
||||||
|
// A message type ID is required for storing a protocol buffer in a message set.
|
||||||
|
var errNoMessageTypeID = errors.New("proto does not have a message type ID") |
||||||
|
|
||||||
|
// The first two types (_MessageSet_Item and messageSet)
|
||||||
|
// model what the protocol compiler produces for the following protocol message:
|
||||||
|
// message MessageSet {
|
||||||
|
// repeated group Item = 1 {
|
||||||
|
// required int32 type_id = 2;
|
||||||
|
// required string message = 3;
|
||||||
|
// };
|
||||||
|
// }
|
||||||
|
// That is the MessageSet wire format. We can't use a proto to generate these
|
||||||
|
// because that would introduce a circular dependency between it and this package.
|
||||||
|
|
||||||
|
type _MessageSet_Item struct { |
||||||
|
TypeId *int32 `protobuf:"varint,2,req,name=type_id"` |
||||||
|
Message []byte `protobuf:"bytes,3,req,name=message"` |
||||||
|
} |
||||||
|
|
||||||
|
type messageSet struct { |
||||||
|
Item []*_MessageSet_Item `protobuf:"group,1,rep"` |
||||||
|
XXX_unrecognized []byte |
||||||
|
// TODO: caching?
|
||||||
|
} |
||||||
|
|
||||||
|
// Make sure messageSet is a Message.
|
||||||
|
var _ Message = (*messageSet)(nil) |
||||||
|
|
||||||
|
// messageTypeIder is an interface satisfied by a protocol buffer type
|
||||||
|
// that may be stored in a MessageSet.
|
||||||
|
type messageTypeIder interface { |
||||||
|
MessageTypeId() int32 |
||||||
|
} |
||||||
|
|
||||||
|
func (ms *messageSet) find(pb Message) *_MessageSet_Item { |
||||||
|
mti, ok := pb.(messageTypeIder) |
||||||
|
if !ok { |
||||||
|
return nil |
||||||
|
} |
||||||
|
id := mti.MessageTypeId() |
||||||
|
for _, item := range ms.Item { |
||||||
|
if *item.TypeId == id { |
||||||
|
return item |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (ms *messageSet) Has(pb Message) bool { |
||||||
|
if ms.find(pb) != nil { |
||||||
|
return true |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
func (ms *messageSet) Unmarshal(pb Message) error { |
||||||
|
if item := ms.find(pb); item != nil { |
||||||
|
return Unmarshal(item.Message, pb) |
||||||
|
} |
||||||
|
if _, ok := pb.(messageTypeIder); !ok { |
||||||
|
return errNoMessageTypeID |
||||||
|
} |
||||||
|
return nil // TODO: return error instead?
|
||||||
|
} |
||||||
|
|
||||||
|
func (ms *messageSet) Marshal(pb Message) error { |
||||||
|
msg, err := Marshal(pb) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if item := ms.find(pb); item != nil { |
||||||
|
// reuse existing item
|
||||||
|
item.Message = msg |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
mti, ok := pb.(messageTypeIder) |
||||||
|
if !ok { |
||||||
|
return errNoMessageTypeID |
||||||
|
} |
||||||
|
|
||||||
|
mtid := mti.MessageTypeId() |
||||||
|
ms.Item = append(ms.Item, &_MessageSet_Item{ |
||||||
|
TypeId: &mtid, |
||||||
|
Message: msg, |
||||||
|
}) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (ms *messageSet) Reset() { *ms = messageSet{} } |
||||||
|
func (ms *messageSet) String() string { return CompactTextString(ms) } |
||||||
|
func (*messageSet) ProtoMessage() {} |
||||||
|
|
||||||
|
// Support for the message_set_wire_format message option.
|
||||||
|
|
||||||
|
func skipVarint(buf []byte) []byte { |
||||||
|
i := 0 |
||||||
|
for ; buf[i]&0x80 != 0; i++ { |
||||||
|
} |
||||||
|
return buf[i+1:] |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalMessageSet encodes the extension map represented by m in the message set wire format.
|
||||||
|
// It is called by generated Marshal methods on protocol buffer messages with the message_set_wire_format option.
|
||||||
|
func MarshalMessageSet(exts interface{}) ([]byte, error) { |
||||||
|
var m map[int32]Extension |
||||||
|
switch exts := exts.(type) { |
||||||
|
case *XXX_InternalExtensions: |
||||||
|
if err := encodeExtensions(exts); err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
m, _ = exts.extensionsRead() |
||||||
|
case map[int32]Extension: |
||||||
|
if err := encodeExtensionsMap(exts); err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
m = exts |
||||||
|
default: |
||||||
|
return nil, errors.New("proto: not an extension map") |
||||||
|
} |
||||||
|
|
||||||
|
// Sort extension IDs to provide a deterministic encoding.
|
||||||
|
// See also enc_map in encode.go.
|
||||||
|
ids := make([]int, 0, len(m)) |
||||||
|
for id := range m { |
||||||
|
ids = append(ids, int(id)) |
||||||
|
} |
||||||
|
sort.Ints(ids) |
||||||
|
|
||||||
|
ms := &messageSet{Item: make([]*_MessageSet_Item, 0, len(m))} |
||||||
|
for _, id := range ids { |
||||||
|
e := m[int32(id)] |
||||||
|
// Remove the wire type and field number varint, as well as the length varint.
|
||||||
|
msg := skipVarint(skipVarint(e.enc)) |
||||||
|
|
||||||
|
ms.Item = append(ms.Item, &_MessageSet_Item{ |
||||||
|
TypeId: Int32(int32(id)), |
||||||
|
Message: msg, |
||||||
|
}) |
||||||
|
} |
||||||
|
return Marshal(ms) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalMessageSet decodes the extension map encoded in buf in the message set wire format.
|
||||||
|
// It is called by generated Unmarshal methods on protocol buffer messages with the message_set_wire_format option.
|
||||||
|
func UnmarshalMessageSet(buf []byte, exts interface{}) error { |
||||||
|
var m map[int32]Extension |
||||||
|
switch exts := exts.(type) { |
||||||
|
case *XXX_InternalExtensions: |
||||||
|
m = exts.extensionsWrite() |
||||||
|
case map[int32]Extension: |
||||||
|
m = exts |
||||||
|
default: |
||||||
|
return errors.New("proto: not an extension map") |
||||||
|
} |
||||||
|
|
||||||
|
ms := new(messageSet) |
||||||
|
if err := Unmarshal(buf, ms); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
for _, item := range ms.Item { |
||||||
|
id := *item.TypeId |
||||||
|
msg := item.Message |
||||||
|
|
||||||
|
// Restore wire type and field number varint, plus length varint.
|
||||||
|
// Be careful to preserve duplicate items.
|
||||||
|
b := EncodeVarint(uint64(id)<<3 | WireBytes) |
||||||
|
if ext, ok := m[id]; ok { |
||||||
|
// Existing data; rip off the tag and length varint
|
||||||
|
// so we join the new data correctly.
|
||||||
|
// We can assume that ext.enc is set because we are unmarshaling.
|
||||||
|
o := ext.enc[len(b):] // skip wire type and field number
|
||||||
|
_, n := DecodeVarint(o) // calculate length of length varint
|
||||||
|
o = o[n:] // skip length varint
|
||||||
|
msg = append(o, msg...) // join old data and new data
|
||||||
|
} |
||||||
|
b = append(b, EncodeVarint(uint64(len(msg)))...) |
||||||
|
b = append(b, msg...) |
||||||
|
|
||||||
|
m[id] = Extension{enc: b} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalMessageSetJSON encodes the extension map represented by m in JSON format.
|
||||||
|
// It is called by generated MarshalJSON methods on protocol buffer messages with the message_set_wire_format option.
|
||||||
|
func MarshalMessageSetJSON(exts interface{}) ([]byte, error) { |
||||||
|
var m map[int32]Extension |
||||||
|
switch exts := exts.(type) { |
||||||
|
case *XXX_InternalExtensions: |
||||||
|
m, _ = exts.extensionsRead() |
||||||
|
case map[int32]Extension: |
||||||
|
m = exts |
||||||
|
default: |
||||||
|
return nil, errors.New("proto: not an extension map") |
||||||
|
} |
||||||
|
var b bytes.Buffer |
||||||
|
b.WriteByte('{') |
||||||
|
|
||||||
|
// Process the map in key order for deterministic output.
|
||||||
|
ids := make([]int32, 0, len(m)) |
||||||
|
for id := range m { |
||||||
|
ids = append(ids, id) |
||||||
|
} |
||||||
|
sort.Sort(int32Slice(ids)) // int32Slice defined in text.go
|
||||||
|
|
||||||
|
for i, id := range ids { |
||||||
|
ext := m[id] |
||||||
|
if i > 0 { |
||||||
|
b.WriteByte(',') |
||||||
|
} |
||||||
|
|
||||||
|
msd, ok := messageSetMap[id] |
||||||
|
if !ok { |
||||||
|
// Unknown type; we can't render it, so skip it.
|
||||||
|
continue |
||||||
|
} |
||||||
|
fmt.Fprintf(&b, `"[%s]":`, msd.name) |
||||||
|
|
||||||
|
x := ext.value |
||||||
|
if x == nil { |
||||||
|
x = reflect.New(msd.t.Elem()).Interface() |
||||||
|
if err := Unmarshal(ext.enc, x.(Message)); err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
} |
||||||
|
d, err := json.Marshal(x) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
b.Write(d) |
||||||
|
} |
||||||
|
b.WriteByte('}') |
||||||
|
return b.Bytes(), nil |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalMessageSetJSON decodes the extension map encoded in buf in JSON format.
|
||||||
|
// It is called by generated UnmarshalJSON methods on protocol buffer messages with the message_set_wire_format option.
|
||||||
|
func UnmarshalMessageSetJSON(buf []byte, exts interface{}) error { |
||||||
|
// Common-case fast path.
|
||||||
|
if len(buf) == 0 || bytes.Equal(buf, []byte("{}")) { |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// This is fairly tricky, and it's not clear that it is needed.
|
||||||
|
return errors.New("TODO: UnmarshalMessageSetJSON not yet implemented") |
||||||
|
} |
||||||
|
|
||||||
|
// A global registry of types that can be used in a MessageSet.
|
||||||
|
|
||||||
|
var messageSetMap = make(map[int32]messageSetDesc) |
||||||
|
|
||||||
|
type messageSetDesc struct { |
||||||
|
t reflect.Type // pointer to struct
|
||||||
|
name string |
||||||
|
} |
||||||
|
|
||||||
|
// RegisterMessageSetType is called from the generated code.
|
||||||
|
func RegisterMessageSetType(m Message, fieldNum int32, name string) { |
||||||
|
messageSetMap[fieldNum] = messageSetDesc{ |
||||||
|
t: reflect.TypeOf(m), |
||||||
|
name: name, |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,484 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2012 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
// +build appengine js
|
||||||
|
|
||||||
|
// This file contains an implementation of proto field accesses using package reflect.
|
||||||
|
// It is slower than the code in pointer_unsafe.go but it avoids package unsafe and can
|
||||||
|
// be used on App Engine.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
import ( |
||||||
|
"math" |
||||||
|
"reflect" |
||||||
|
) |
||||||
|
|
||||||
|
// A structPointer is a pointer to a struct.
|
||||||
|
type structPointer struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
// toStructPointer returns a structPointer equivalent to the given reflect value.
|
||||||
|
// The reflect value must itself be a pointer to a struct.
|
||||||
|
func toStructPointer(v reflect.Value) structPointer { |
||||||
|
return structPointer{v} |
||||||
|
} |
||||||
|
|
||||||
|
// IsNil reports whether p is nil.
|
||||||
|
func structPointer_IsNil(p structPointer) bool { |
||||||
|
return p.v.IsNil() |
||||||
|
} |
||||||
|
|
||||||
|
// Interface returns the struct pointer as an interface value.
|
||||||
|
func structPointer_Interface(p structPointer, _ reflect.Type) interface{} { |
||||||
|
return p.v.Interface() |
||||||
|
} |
||||||
|
|
||||||
|
// A field identifies a field in a struct, accessible from a structPointer.
|
||||||
|
// In this implementation, a field is identified by the sequence of field indices
|
||||||
|
// passed to reflect's FieldByIndex.
|
||||||
|
type field []int |
||||||
|
|
||||||
|
// toField returns a field equivalent to the given reflect field.
|
||||||
|
func toField(f *reflect.StructField) field { |
||||||
|
return f.Index |
||||||
|
} |
||||||
|
|
||||||
|
// invalidField is an invalid field identifier.
|
||||||
|
var invalidField = field(nil) |
||||||
|
|
||||||
|
// IsValid reports whether the field identifier is valid.
|
||||||
|
func (f field) IsValid() bool { return f != nil } |
||||||
|
|
||||||
|
// field returns the given field in the struct as a reflect value.
|
||||||
|
func structPointer_field(p structPointer, f field) reflect.Value { |
||||||
|
// Special case: an extension map entry with a value of type T
|
||||||
|
// passes a *T to the struct-handling code with a zero field,
|
||||||
|
// expecting that it will be treated as equivalent to *struct{ X T },
|
||||||
|
// which has the same memory layout. We have to handle that case
|
||||||
|
// specially, because reflect will panic if we call FieldByIndex on a
|
||||||
|
// non-struct.
|
||||||
|
if f == nil { |
||||||
|
return p.v.Elem() |
||||||
|
} |
||||||
|
|
||||||
|
return p.v.Elem().FieldByIndex(f) |
||||||
|
} |
||||||
|
|
||||||
|
// ifield returns the given field in the struct as an interface value.
|
||||||
|
func structPointer_ifield(p structPointer, f field) interface{} { |
||||||
|
return structPointer_field(p, f).Addr().Interface() |
||||||
|
} |
||||||
|
|
||||||
|
// Bytes returns the address of a []byte field in the struct.
|
||||||
|
func structPointer_Bytes(p structPointer, f field) *[]byte { |
||||||
|
return structPointer_ifield(p, f).(*[]byte) |
||||||
|
} |
||||||
|
|
||||||
|
// BytesSlice returns the address of a [][]byte field in the struct.
|
||||||
|
func structPointer_BytesSlice(p structPointer, f field) *[][]byte { |
||||||
|
return structPointer_ifield(p, f).(*[][]byte) |
||||||
|
} |
||||||
|
|
||||||
|
// Bool returns the address of a *bool field in the struct.
|
||||||
|
func structPointer_Bool(p structPointer, f field) **bool { |
||||||
|
return structPointer_ifield(p, f).(**bool) |
||||||
|
} |
||||||
|
|
||||||
|
// BoolVal returns the address of a bool field in the struct.
|
||||||
|
func structPointer_BoolVal(p structPointer, f field) *bool { |
||||||
|
return structPointer_ifield(p, f).(*bool) |
||||||
|
} |
||||||
|
|
||||||
|
// BoolSlice returns the address of a []bool field in the struct.
|
||||||
|
func structPointer_BoolSlice(p structPointer, f field) *[]bool { |
||||||
|
return structPointer_ifield(p, f).(*[]bool) |
||||||
|
} |
||||||
|
|
||||||
|
// String returns the address of a *string field in the struct.
|
||||||
|
func structPointer_String(p structPointer, f field) **string { |
||||||
|
return structPointer_ifield(p, f).(**string) |
||||||
|
} |
||||||
|
|
||||||
|
// StringVal returns the address of a string field in the struct.
|
||||||
|
func structPointer_StringVal(p structPointer, f field) *string { |
||||||
|
return structPointer_ifield(p, f).(*string) |
||||||
|
} |
||||||
|
|
||||||
|
// StringSlice returns the address of a []string field in the struct.
|
||||||
|
func structPointer_StringSlice(p structPointer, f field) *[]string { |
||||||
|
return structPointer_ifield(p, f).(*[]string) |
||||||
|
} |
||||||
|
|
||||||
|
// Extensions returns the address of an extension map field in the struct.
|
||||||
|
func structPointer_Extensions(p structPointer, f field) *XXX_InternalExtensions { |
||||||
|
return structPointer_ifield(p, f).(*XXX_InternalExtensions) |
||||||
|
} |
||||||
|
|
||||||
|
// ExtMap returns the address of an extension map field in the struct.
|
||||||
|
func structPointer_ExtMap(p structPointer, f field) *map[int32]Extension { |
||||||
|
return structPointer_ifield(p, f).(*map[int32]Extension) |
||||||
|
} |
||||||
|
|
||||||
|
// NewAt returns the reflect.Value for a pointer to a field in the struct.
|
||||||
|
func structPointer_NewAt(p structPointer, f field, typ reflect.Type) reflect.Value { |
||||||
|
return structPointer_field(p, f).Addr() |
||||||
|
} |
||||||
|
|
||||||
|
// SetStructPointer writes a *struct field in the struct.
|
||||||
|
func structPointer_SetStructPointer(p structPointer, f field, q structPointer) { |
||||||
|
structPointer_field(p, f).Set(q.v) |
||||||
|
} |
||||||
|
|
||||||
|
// GetStructPointer reads a *struct field in the struct.
|
||||||
|
func structPointer_GetStructPointer(p structPointer, f field) structPointer { |
||||||
|
return structPointer{structPointer_field(p, f)} |
||||||
|
} |
||||||
|
|
||||||
|
// StructPointerSlice the address of a []*struct field in the struct.
|
||||||
|
func structPointer_StructPointerSlice(p structPointer, f field) structPointerSlice { |
||||||
|
return structPointerSlice{structPointer_field(p, f)} |
||||||
|
} |
||||||
|
|
||||||
|
// A structPointerSlice represents the address of a slice of pointers to structs
|
||||||
|
// (themselves messages or groups). That is, v.Type() is *[]*struct{...}.
|
||||||
|
type structPointerSlice struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
func (p structPointerSlice) Len() int { return p.v.Len() } |
||||||
|
func (p structPointerSlice) Index(i int) structPointer { return structPointer{p.v.Index(i)} } |
||||||
|
func (p structPointerSlice) Append(q structPointer) { |
||||||
|
p.v.Set(reflect.Append(p.v, q.v)) |
||||||
|
} |
||||||
|
|
||||||
|
var ( |
||||||
|
int32Type = reflect.TypeOf(int32(0)) |
||||||
|
uint32Type = reflect.TypeOf(uint32(0)) |
||||||
|
float32Type = reflect.TypeOf(float32(0)) |
||||||
|
int64Type = reflect.TypeOf(int64(0)) |
||||||
|
uint64Type = reflect.TypeOf(uint64(0)) |
||||||
|
float64Type = reflect.TypeOf(float64(0)) |
||||||
|
) |
||||||
|
|
||||||
|
// A word32 represents a field of type *int32, *uint32, *float32, or *enum.
|
||||||
|
// That is, v.Type() is *int32, *uint32, *float32, or *enum and v is assignable.
|
||||||
|
type word32 struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
// IsNil reports whether p is nil.
|
||||||
|
func word32_IsNil(p word32) bool { |
||||||
|
return p.v.IsNil() |
||||||
|
} |
||||||
|
|
||||||
|
// Set sets p to point at a newly allocated word with bits set to x.
|
||||||
|
func word32_Set(p word32, o *Buffer, x uint32) { |
||||||
|
t := p.v.Type().Elem() |
||||||
|
switch t { |
||||||
|
case int32Type: |
||||||
|
if len(o.int32s) == 0 { |
||||||
|
o.int32s = make([]int32, uint32PoolSize) |
||||||
|
} |
||||||
|
o.int32s[0] = int32(x) |
||||||
|
p.v.Set(reflect.ValueOf(&o.int32s[0])) |
||||||
|
o.int32s = o.int32s[1:] |
||||||
|
return |
||||||
|
case uint32Type: |
||||||
|
if len(o.uint32s) == 0 { |
||||||
|
o.uint32s = make([]uint32, uint32PoolSize) |
||||||
|
} |
||||||
|
o.uint32s[0] = x |
||||||
|
p.v.Set(reflect.ValueOf(&o.uint32s[0])) |
||||||
|
o.uint32s = o.uint32s[1:] |
||||||
|
return |
||||||
|
case float32Type: |
||||||
|
if len(o.float32s) == 0 { |
||||||
|
o.float32s = make([]float32, uint32PoolSize) |
||||||
|
} |
||||||
|
o.float32s[0] = math.Float32frombits(x) |
||||||
|
p.v.Set(reflect.ValueOf(&o.float32s[0])) |
||||||
|
o.float32s = o.float32s[1:] |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// must be enum
|
||||||
|
p.v.Set(reflect.New(t)) |
||||||
|
p.v.Elem().SetInt(int64(int32(x))) |
||||||
|
} |
||||||
|
|
||||||
|
// Get gets the bits pointed at by p, as a uint32.
|
||||||
|
func word32_Get(p word32) uint32 { |
||||||
|
elem := p.v.Elem() |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int32: |
||||||
|
return uint32(elem.Int()) |
||||||
|
case reflect.Uint32: |
||||||
|
return uint32(elem.Uint()) |
||||||
|
case reflect.Float32: |
||||||
|
return math.Float32bits(float32(elem.Float())) |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
// Word32 returns a reference to a *int32, *uint32, *float32, or *enum field in the struct.
|
||||||
|
func structPointer_Word32(p structPointer, f field) word32 { |
||||||
|
return word32{structPointer_field(p, f)} |
||||||
|
} |
||||||
|
|
||||||
|
// A word32Val represents a field of type int32, uint32, float32, or enum.
|
||||||
|
// That is, v.Type() is int32, uint32, float32, or enum and v is assignable.
|
||||||
|
type word32Val struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
// Set sets *p to x.
|
||||||
|
func word32Val_Set(p word32Val, x uint32) { |
||||||
|
switch p.v.Type() { |
||||||
|
case int32Type: |
||||||
|
p.v.SetInt(int64(x)) |
||||||
|
return |
||||||
|
case uint32Type: |
||||||
|
p.v.SetUint(uint64(x)) |
||||||
|
return |
||||||
|
case float32Type: |
||||||
|
p.v.SetFloat(float64(math.Float32frombits(x))) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// must be enum
|
||||||
|
p.v.SetInt(int64(int32(x))) |
||||||
|
} |
||||||
|
|
||||||
|
// Get gets the bits pointed at by p, as a uint32.
|
||||||
|
func word32Val_Get(p word32Val) uint32 { |
||||||
|
elem := p.v |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int32: |
||||||
|
return uint32(elem.Int()) |
||||||
|
case reflect.Uint32: |
||||||
|
return uint32(elem.Uint()) |
||||||
|
case reflect.Float32: |
||||||
|
return math.Float32bits(float32(elem.Float())) |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
// Word32Val returns a reference to a int32, uint32, float32, or enum field in the struct.
|
||||||
|
func structPointer_Word32Val(p structPointer, f field) word32Val { |
||||||
|
return word32Val{structPointer_field(p, f)} |
||||||
|
} |
||||||
|
|
||||||
|
// A word32Slice is a slice of 32-bit values.
|
||||||
|
// That is, v.Type() is []int32, []uint32, []float32, or []enum.
|
||||||
|
type word32Slice struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
func (p word32Slice) Append(x uint32) { |
||||||
|
n, m := p.v.Len(), p.v.Cap() |
||||||
|
if n < m { |
||||||
|
p.v.SetLen(n + 1) |
||||||
|
} else { |
||||||
|
t := p.v.Type().Elem() |
||||||
|
p.v.Set(reflect.Append(p.v, reflect.Zero(t))) |
||||||
|
} |
||||||
|
elem := p.v.Index(n) |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int32: |
||||||
|
elem.SetInt(int64(int32(x))) |
||||||
|
case reflect.Uint32: |
||||||
|
elem.SetUint(uint64(x)) |
||||||
|
case reflect.Float32: |
||||||
|
elem.SetFloat(float64(math.Float32frombits(x))) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (p word32Slice) Len() int { |
||||||
|
return p.v.Len() |
||||||
|
} |
||||||
|
|
||||||
|
func (p word32Slice) Index(i int) uint32 { |
||||||
|
elem := p.v.Index(i) |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int32: |
||||||
|
return uint32(elem.Int()) |
||||||
|
case reflect.Uint32: |
||||||
|
return uint32(elem.Uint()) |
||||||
|
case reflect.Float32: |
||||||
|
return math.Float32bits(float32(elem.Float())) |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
// Word32Slice returns a reference to a []int32, []uint32, []float32, or []enum field in the struct.
|
||||||
|
func structPointer_Word32Slice(p structPointer, f field) word32Slice { |
||||||
|
return word32Slice{structPointer_field(p, f)} |
||||||
|
} |
||||||
|
|
||||||
|
// word64 is like word32 but for 64-bit values.
|
||||||
|
type word64 struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
func word64_Set(p word64, o *Buffer, x uint64) { |
||||||
|
t := p.v.Type().Elem() |
||||||
|
switch t { |
||||||
|
case int64Type: |
||||||
|
if len(o.int64s) == 0 { |
||||||
|
o.int64s = make([]int64, uint64PoolSize) |
||||||
|
} |
||||||
|
o.int64s[0] = int64(x) |
||||||
|
p.v.Set(reflect.ValueOf(&o.int64s[0])) |
||||||
|
o.int64s = o.int64s[1:] |
||||||
|
return |
||||||
|
case uint64Type: |
||||||
|
if len(o.uint64s) == 0 { |
||||||
|
o.uint64s = make([]uint64, uint64PoolSize) |
||||||
|
} |
||||||
|
o.uint64s[0] = x |
||||||
|
p.v.Set(reflect.ValueOf(&o.uint64s[0])) |
||||||
|
o.uint64s = o.uint64s[1:] |
||||||
|
return |
||||||
|
case float64Type: |
||||||
|
if len(o.float64s) == 0 { |
||||||
|
o.float64s = make([]float64, uint64PoolSize) |
||||||
|
} |
||||||
|
o.float64s[0] = math.Float64frombits(x) |
||||||
|
p.v.Set(reflect.ValueOf(&o.float64s[0])) |
||||||
|
o.float64s = o.float64s[1:] |
||||||
|
return |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
func word64_IsNil(p word64) bool { |
||||||
|
return p.v.IsNil() |
||||||
|
} |
||||||
|
|
||||||
|
func word64_Get(p word64) uint64 { |
||||||
|
elem := p.v.Elem() |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int64: |
||||||
|
return uint64(elem.Int()) |
||||||
|
case reflect.Uint64: |
||||||
|
return elem.Uint() |
||||||
|
case reflect.Float64: |
||||||
|
return math.Float64bits(elem.Float()) |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
func structPointer_Word64(p structPointer, f field) word64 { |
||||||
|
return word64{structPointer_field(p, f)} |
||||||
|
} |
||||||
|
|
||||||
|
// word64Val is like word32Val but for 64-bit values.
|
||||||
|
type word64Val struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
func word64Val_Set(p word64Val, o *Buffer, x uint64) { |
||||||
|
switch p.v.Type() { |
||||||
|
case int64Type: |
||||||
|
p.v.SetInt(int64(x)) |
||||||
|
return |
||||||
|
case uint64Type: |
||||||
|
p.v.SetUint(x) |
||||||
|
return |
||||||
|
case float64Type: |
||||||
|
p.v.SetFloat(math.Float64frombits(x)) |
||||||
|
return |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
func word64Val_Get(p word64Val) uint64 { |
||||||
|
elem := p.v |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int64: |
||||||
|
return uint64(elem.Int()) |
||||||
|
case reflect.Uint64: |
||||||
|
return elem.Uint() |
||||||
|
case reflect.Float64: |
||||||
|
return math.Float64bits(elem.Float()) |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
func structPointer_Word64Val(p structPointer, f field) word64Val { |
||||||
|
return word64Val{structPointer_field(p, f)} |
||||||
|
} |
||||||
|
|
||||||
|
type word64Slice struct { |
||||||
|
v reflect.Value |
||||||
|
} |
||||||
|
|
||||||
|
func (p word64Slice) Append(x uint64) { |
||||||
|
n, m := p.v.Len(), p.v.Cap() |
||||||
|
if n < m { |
||||||
|
p.v.SetLen(n + 1) |
||||||
|
} else { |
||||||
|
t := p.v.Type().Elem() |
||||||
|
p.v.Set(reflect.Append(p.v, reflect.Zero(t))) |
||||||
|
} |
||||||
|
elem := p.v.Index(n) |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int64: |
||||||
|
elem.SetInt(int64(int64(x))) |
||||||
|
case reflect.Uint64: |
||||||
|
elem.SetUint(uint64(x)) |
||||||
|
case reflect.Float64: |
||||||
|
elem.SetFloat(float64(math.Float64frombits(x))) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (p word64Slice) Len() int { |
||||||
|
return p.v.Len() |
||||||
|
} |
||||||
|
|
||||||
|
func (p word64Slice) Index(i int) uint64 { |
||||||
|
elem := p.v.Index(i) |
||||||
|
switch elem.Kind() { |
||||||
|
case reflect.Int64: |
||||||
|
return uint64(elem.Int()) |
||||||
|
case reflect.Uint64: |
||||||
|
return uint64(elem.Uint()) |
||||||
|
case reflect.Float64: |
||||||
|
return math.Float64bits(float64(elem.Float())) |
||||||
|
} |
||||||
|
panic("unreachable") |
||||||
|
} |
||||||
|
|
||||||
|
func structPointer_Word64Slice(p structPointer, f field) word64Slice { |
||||||
|
return word64Slice{structPointer_field(p, f)} |
||||||
|
} |
||||||
@ -0,0 +1,270 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2012 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
// +build !appengine,!js
|
||||||
|
|
||||||
|
// This file contains the implementation of the proto field accesses using package unsafe.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
import ( |
||||||
|
"reflect" |
||||||
|
"unsafe" |
||||||
|
) |
||||||
|
|
||||||
|
// NOTE: These type_Foo functions would more idiomatically be methods,
|
||||||
|
// but Go does not allow methods on pointer types, and we must preserve
|
||||||
|
// some pointer type for the garbage collector. We use these
|
||||||
|
// funcs with clunky names as our poor approximation to methods.
|
||||||
|
//
|
||||||
|
// An alternative would be
|
||||||
|
// type structPointer struct { p unsafe.Pointer }
|
||||||
|
// but that does not registerize as well.
|
||||||
|
|
||||||
|
// A structPointer is a pointer to a struct.
|
||||||
|
type structPointer unsafe.Pointer |
||||||
|
|
||||||
|
// toStructPointer returns a structPointer equivalent to the given reflect value.
|
||||||
|
func toStructPointer(v reflect.Value) structPointer { |
||||||
|
return structPointer(unsafe.Pointer(v.Pointer())) |
||||||
|
} |
||||||
|
|
||||||
|
// IsNil reports whether p is nil.
|
||||||
|
func structPointer_IsNil(p structPointer) bool { |
||||||
|
return p == nil |
||||||
|
} |
||||||
|
|
||||||
|
// Interface returns the struct pointer, assumed to have element type t,
|
||||||
|
// as an interface value.
|
||||||
|
func structPointer_Interface(p structPointer, t reflect.Type) interface{} { |
||||||
|
return reflect.NewAt(t, unsafe.Pointer(p)).Interface() |
||||||
|
} |
||||||
|
|
||||||
|
// A field identifies a field in a struct, accessible from a structPointer.
|
||||||
|
// In this implementation, a field is identified by its byte offset from the start of the struct.
|
||||||
|
type field uintptr |
||||||
|
|
||||||
|
// toField returns a field equivalent to the given reflect field.
|
||||||
|
func toField(f *reflect.StructField) field { |
||||||
|
return field(f.Offset) |
||||||
|
} |
||||||
|
|
||||||
|
// invalidField is an invalid field identifier.
|
||||||
|
const invalidField = ^field(0) |
||||||
|
|
||||||
|
// IsValid reports whether the field identifier is valid.
|
||||||
|
func (f field) IsValid() bool { |
||||||
|
return f != ^field(0) |
||||||
|
} |
||||||
|
|
||||||
|
// Bytes returns the address of a []byte field in the struct.
|
||||||
|
func structPointer_Bytes(p structPointer, f field) *[]byte { |
||||||
|
return (*[]byte)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// BytesSlice returns the address of a [][]byte field in the struct.
|
||||||
|
func structPointer_BytesSlice(p structPointer, f field) *[][]byte { |
||||||
|
return (*[][]byte)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// Bool returns the address of a *bool field in the struct.
|
||||||
|
func structPointer_Bool(p structPointer, f field) **bool { |
||||||
|
return (**bool)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// BoolVal returns the address of a bool field in the struct.
|
||||||
|
func structPointer_BoolVal(p structPointer, f field) *bool { |
||||||
|
return (*bool)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// BoolSlice returns the address of a []bool field in the struct.
|
||||||
|
func structPointer_BoolSlice(p structPointer, f field) *[]bool { |
||||||
|
return (*[]bool)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// String returns the address of a *string field in the struct.
|
||||||
|
func structPointer_String(p structPointer, f field) **string { |
||||||
|
return (**string)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// StringVal returns the address of a string field in the struct.
|
||||||
|
func structPointer_StringVal(p structPointer, f field) *string { |
||||||
|
return (*string)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// StringSlice returns the address of a []string field in the struct.
|
||||||
|
func structPointer_StringSlice(p structPointer, f field) *[]string { |
||||||
|
return (*[]string)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// ExtMap returns the address of an extension map field in the struct.
|
||||||
|
func structPointer_Extensions(p structPointer, f field) *XXX_InternalExtensions { |
||||||
|
return (*XXX_InternalExtensions)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
func structPointer_ExtMap(p structPointer, f field) *map[int32]Extension { |
||||||
|
return (*map[int32]Extension)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// NewAt returns the reflect.Value for a pointer to a field in the struct.
|
||||||
|
func structPointer_NewAt(p structPointer, f field, typ reflect.Type) reflect.Value { |
||||||
|
return reflect.NewAt(typ, unsafe.Pointer(uintptr(p)+uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// SetStructPointer writes a *struct field in the struct.
|
||||||
|
func structPointer_SetStructPointer(p structPointer, f field, q structPointer) { |
||||||
|
*(*structPointer)(unsafe.Pointer(uintptr(p) + uintptr(f))) = q |
||||||
|
} |
||||||
|
|
||||||
|
// GetStructPointer reads a *struct field in the struct.
|
||||||
|
func structPointer_GetStructPointer(p structPointer, f field) structPointer { |
||||||
|
return *(*structPointer)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// StructPointerSlice the address of a []*struct field in the struct.
|
||||||
|
func structPointer_StructPointerSlice(p structPointer, f field) *structPointerSlice { |
||||||
|
return (*structPointerSlice)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// A structPointerSlice represents a slice of pointers to structs (themselves submessages or groups).
|
||||||
|
type structPointerSlice []structPointer |
||||||
|
|
||||||
|
func (v *structPointerSlice) Len() int { return len(*v) } |
||||||
|
func (v *structPointerSlice) Index(i int) structPointer { return (*v)[i] } |
||||||
|
func (v *structPointerSlice) Append(p structPointer) { *v = append(*v, p) } |
||||||
|
|
||||||
|
// A word32 is the address of a "pointer to 32-bit value" field.
|
||||||
|
type word32 **uint32 |
||||||
|
|
||||||
|
// IsNil reports whether *v is nil.
|
||||||
|
func word32_IsNil(p word32) bool { |
||||||
|
return *p == nil |
||||||
|
} |
||||||
|
|
||||||
|
// Set sets *v to point at a newly allocated word set to x.
|
||||||
|
func word32_Set(p word32, o *Buffer, x uint32) { |
||||||
|
if len(o.uint32s) == 0 { |
||||||
|
o.uint32s = make([]uint32, uint32PoolSize) |
||||||
|
} |
||||||
|
o.uint32s[0] = x |
||||||
|
*p = &o.uint32s[0] |
||||||
|
o.uint32s = o.uint32s[1:] |
||||||
|
} |
||||||
|
|
||||||
|
// Get gets the value pointed at by *v.
|
||||||
|
func word32_Get(p word32) uint32 { |
||||||
|
return **p |
||||||
|
} |
||||||
|
|
||||||
|
// Word32 returns the address of a *int32, *uint32, *float32, or *enum field in the struct.
|
||||||
|
func structPointer_Word32(p structPointer, f field) word32 { |
||||||
|
return word32((**uint32)(unsafe.Pointer(uintptr(p) + uintptr(f)))) |
||||||
|
} |
||||||
|
|
||||||
|
// A word32Val is the address of a 32-bit value field.
|
||||||
|
type word32Val *uint32 |
||||||
|
|
||||||
|
// Set sets *p to x.
|
||||||
|
func word32Val_Set(p word32Val, x uint32) { |
||||||
|
*p = x |
||||||
|
} |
||||||
|
|
||||||
|
// Get gets the value pointed at by p.
|
||||||
|
func word32Val_Get(p word32Val) uint32 { |
||||||
|
return *p |
||||||
|
} |
||||||
|
|
||||||
|
// Word32Val returns the address of a *int32, *uint32, *float32, or *enum field in the struct.
|
||||||
|
func structPointer_Word32Val(p structPointer, f field) word32Val { |
||||||
|
return word32Val((*uint32)(unsafe.Pointer(uintptr(p) + uintptr(f)))) |
||||||
|
} |
||||||
|
|
||||||
|
// A word32Slice is a slice of 32-bit values.
|
||||||
|
type word32Slice []uint32 |
||||||
|
|
||||||
|
func (v *word32Slice) Append(x uint32) { *v = append(*v, x) } |
||||||
|
func (v *word32Slice) Len() int { return len(*v) } |
||||||
|
func (v *word32Slice) Index(i int) uint32 { return (*v)[i] } |
||||||
|
|
||||||
|
// Word32Slice returns the address of a []int32, []uint32, []float32, or []enum field in the struct.
|
||||||
|
func structPointer_Word32Slice(p structPointer, f field) *word32Slice { |
||||||
|
return (*word32Slice)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
|
|
||||||
|
// word64 is like word32 but for 64-bit values.
|
||||||
|
type word64 **uint64 |
||||||
|
|
||||||
|
func word64_Set(p word64, o *Buffer, x uint64) { |
||||||
|
if len(o.uint64s) == 0 { |
||||||
|
o.uint64s = make([]uint64, uint64PoolSize) |
||||||
|
} |
||||||
|
o.uint64s[0] = x |
||||||
|
*p = &o.uint64s[0] |
||||||
|
o.uint64s = o.uint64s[1:] |
||||||
|
} |
||||||
|
|
||||||
|
func word64_IsNil(p word64) bool { |
||||||
|
return *p == nil |
||||||
|
} |
||||||
|
|
||||||
|
func word64_Get(p word64) uint64 { |
||||||
|
return **p |
||||||
|
} |
||||||
|
|
||||||
|
func structPointer_Word64(p structPointer, f field) word64 { |
||||||
|
return word64((**uint64)(unsafe.Pointer(uintptr(p) + uintptr(f)))) |
||||||
|
} |
||||||
|
|
||||||
|
// word64Val is like word32Val but for 64-bit values.
|
||||||
|
type word64Val *uint64 |
||||||
|
|
||||||
|
func word64Val_Set(p word64Val, o *Buffer, x uint64) { |
||||||
|
*p = x |
||||||
|
} |
||||||
|
|
||||||
|
func word64Val_Get(p word64Val) uint64 { |
||||||
|
return *p |
||||||
|
} |
||||||
|
|
||||||
|
func structPointer_Word64Val(p structPointer, f field) word64Val { |
||||||
|
return word64Val((*uint64)(unsafe.Pointer(uintptr(p) + uintptr(f)))) |
||||||
|
} |
||||||
|
|
||||||
|
// word64Slice is like word32Slice but for 64-bit values.
|
||||||
|
type word64Slice []uint64 |
||||||
|
|
||||||
|
func (v *word64Slice) Append(x uint64) { *v = append(*v, x) } |
||||||
|
func (v *word64Slice) Len() int { return len(*v) } |
||||||
|
func (v *word64Slice) Index(i int) uint64 { return (*v)[i] } |
||||||
|
|
||||||
|
func structPointer_Word64Slice(p structPointer, f field) *word64Slice { |
||||||
|
return (*word64Slice)(unsafe.Pointer(uintptr(p) + uintptr(f))) |
||||||
|
} |
||||||
@ -0,0 +1,872 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
/* |
||||||
|
* Routines for encoding data into the wire format for protocol buffers. |
||||||
|
*/ |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"log" |
||||||
|
"os" |
||||||
|
"reflect" |
||||||
|
"sort" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
"sync" |
||||||
|
) |
||||||
|
|
||||||
|
const debug bool = false |
||||||
|
|
||||||
|
// Constants that identify the encoding of a value on the wire.
|
||||||
|
const ( |
||||||
|
WireVarint = 0 |
||||||
|
WireFixed64 = 1 |
||||||
|
WireBytes = 2 |
||||||
|
WireStartGroup = 3 |
||||||
|
WireEndGroup = 4 |
||||||
|
WireFixed32 = 5 |
||||||
|
) |
||||||
|
|
||||||
|
const startSize = 10 // initial slice/string sizes
|
||||||
|
|
||||||
|
// Encoders are defined in encode.go
|
||||||
|
// An encoder outputs the full representation of a field, including its
|
||||||
|
// tag and encoder type.
|
||||||
|
type encoder func(p *Buffer, prop *Properties, base structPointer) error |
||||||
|
|
||||||
|
// A valueEncoder encodes a single integer in a particular encoding.
|
||||||
|
type valueEncoder func(o *Buffer, x uint64) error |
||||||
|
|
||||||
|
// Sizers are defined in encode.go
|
||||||
|
// A sizer returns the encoded size of a field, including its tag and encoder
|
||||||
|
// type.
|
||||||
|
type sizer func(prop *Properties, base structPointer) int |
||||||
|
|
||||||
|
// A valueSizer returns the encoded size of a single integer in a particular
|
||||||
|
// encoding.
|
||||||
|
type valueSizer func(x uint64) int |
||||||
|
|
||||||
|
// Decoders are defined in decode.go
|
||||||
|
// A decoder creates a value from its wire representation.
|
||||||
|
// Unrecognized subelements are saved in unrec.
|
||||||
|
type decoder func(p *Buffer, prop *Properties, base structPointer) error |
||||||
|
|
||||||
|
// A valueDecoder decodes a single integer in a particular encoding.
|
||||||
|
type valueDecoder func(o *Buffer) (x uint64, err error) |
||||||
|
|
||||||
|
// A oneofMarshaler does the marshaling for all oneof fields in a message.
|
||||||
|
type oneofMarshaler func(Message, *Buffer) error |
||||||
|
|
||||||
|
// A oneofUnmarshaler does the unmarshaling for a oneof field in a message.
|
||||||
|
type oneofUnmarshaler func(Message, int, int, *Buffer) (bool, error) |
||||||
|
|
||||||
|
// A oneofSizer does the sizing for all oneof fields in a message.
|
||||||
|
type oneofSizer func(Message) int |
||||||
|
|
||||||
|
// tagMap is an optimization over map[int]int for typical protocol buffer
|
||||||
|
// use-cases. Encoded protocol buffers are often in tag order with small tag
|
||||||
|
// numbers.
|
||||||
|
type tagMap struct { |
||||||
|
fastTags []int |
||||||
|
slowTags map[int]int |
||||||
|
} |
||||||
|
|
||||||
|
// tagMapFastLimit is the upper bound on the tag number that will be stored in
|
||||||
|
// the tagMap slice rather than its map.
|
||||||
|
const tagMapFastLimit = 1024 |
||||||
|
|
||||||
|
func (p *tagMap) get(t int) (int, bool) { |
||||||
|
if t > 0 && t < tagMapFastLimit { |
||||||
|
if t >= len(p.fastTags) { |
||||||
|
return 0, false |
||||||
|
} |
||||||
|
fi := p.fastTags[t] |
||||||
|
return fi, fi >= 0 |
||||||
|
} |
||||||
|
fi, ok := p.slowTags[t] |
||||||
|
return fi, ok |
||||||
|
} |
||||||
|
|
||||||
|
func (p *tagMap) put(t int, fi int) { |
||||||
|
if t > 0 && t < tagMapFastLimit { |
||||||
|
for len(p.fastTags) < t+1 { |
||||||
|
p.fastTags = append(p.fastTags, -1) |
||||||
|
} |
||||||
|
p.fastTags[t] = fi |
||||||
|
return |
||||||
|
} |
||||||
|
if p.slowTags == nil { |
||||||
|
p.slowTags = make(map[int]int) |
||||||
|
} |
||||||
|
p.slowTags[t] = fi |
||||||
|
} |
||||||
|
|
||||||
|
// StructProperties represents properties for all the fields of a struct.
|
||||||
|
// decoderTags and decoderOrigNames should only be used by the decoder.
|
||||||
|
type StructProperties struct { |
||||||
|
Prop []*Properties // properties for each field
|
||||||
|
reqCount int // required count
|
||||||
|
decoderTags tagMap // map from proto tag to struct field number
|
||||||
|
decoderOrigNames map[string]int // map from original name to struct field number
|
||||||
|
order []int // list of struct field numbers in tag order
|
||||||
|
unrecField field // field id of the XXX_unrecognized []byte field
|
||||||
|
extendable bool // is this an extendable proto
|
||||||
|
|
||||||
|
oneofMarshaler oneofMarshaler |
||||||
|
oneofUnmarshaler oneofUnmarshaler |
||||||
|
oneofSizer oneofSizer |
||||||
|
stype reflect.Type |
||||||
|
|
||||||
|
// OneofTypes contains information about the oneof fields in this message.
|
||||||
|
// It is keyed by the original name of a field.
|
||||||
|
OneofTypes map[string]*OneofProperties |
||||||
|
} |
||||||
|
|
||||||
|
// OneofProperties represents information about a specific field in a oneof.
|
||||||
|
type OneofProperties struct { |
||||||
|
Type reflect.Type // pointer to generated struct type for this oneof field
|
||||||
|
Field int // struct field number of the containing oneof in the message
|
||||||
|
Prop *Properties |
||||||
|
} |
||||||
|
|
||||||
|
// Implement the sorting interface so we can sort the fields in tag order, as recommended by the spec.
|
||||||
|
// See encode.go, (*Buffer).enc_struct.
|
||||||
|
|
||||||
|
func (sp *StructProperties) Len() int { return len(sp.order) } |
||||||
|
func (sp *StructProperties) Less(i, j int) bool { |
||||||
|
return sp.Prop[sp.order[i]].Tag < sp.Prop[sp.order[j]].Tag |
||||||
|
} |
||||||
|
func (sp *StructProperties) Swap(i, j int) { sp.order[i], sp.order[j] = sp.order[j], sp.order[i] } |
||||||
|
|
||||||
|
// Properties represents the protocol-specific behavior of a single struct field.
|
||||||
|
type Properties struct { |
||||||
|
Name string // name of the field, for error messages
|
||||||
|
OrigName string // original name before protocol compiler (always set)
|
||||||
|
JSONName string // name to use for JSON; determined by protoc
|
||||||
|
Wire string |
||||||
|
WireType int |
||||||
|
Tag int |
||||||
|
Required bool |
||||||
|
Optional bool |
||||||
|
Repeated bool |
||||||
|
Packed bool // relevant for repeated primitives only
|
||||||
|
Enum string // set for enum types only
|
||||||
|
proto3 bool // whether this is known to be a proto3 field; set for []byte only
|
||||||
|
oneof bool // whether this is a oneof field
|
||||||
|
|
||||||
|
Default string // default value
|
||||||
|
HasDefault bool // whether an explicit default was provided
|
||||||
|
def_uint64 uint64 |
||||||
|
|
||||||
|
enc encoder |
||||||
|
valEnc valueEncoder // set for bool and numeric types only
|
||||||
|
field field |
||||||
|
tagcode []byte // encoding of EncodeVarint((Tag<<3)|WireType)
|
||||||
|
tagbuf [8]byte |
||||||
|
stype reflect.Type // set for struct types only
|
||||||
|
sprop *StructProperties // set for struct types only
|
||||||
|
isMarshaler bool |
||||||
|
isUnmarshaler bool |
||||||
|
|
||||||
|
mtype reflect.Type // set for map types only
|
||||||
|
mkeyprop *Properties // set for map types only
|
||||||
|
mvalprop *Properties // set for map types only
|
||||||
|
|
||||||
|
size sizer |
||||||
|
valSize valueSizer // set for bool and numeric types only
|
||||||
|
|
||||||
|
dec decoder |
||||||
|
valDec valueDecoder // set for bool and numeric types only
|
||||||
|
|
||||||
|
// If this is a packable field, this will be the decoder for the packed version of the field.
|
||||||
|
packedDec decoder |
||||||
|
} |
||||||
|
|
||||||
|
// String formats the properties in the protobuf struct field tag style.
|
||||||
|
func (p *Properties) String() string { |
||||||
|
s := p.Wire |
||||||
|
s = "," |
||||||
|
s += strconv.Itoa(p.Tag) |
||||||
|
if p.Required { |
||||||
|
s += ",req" |
||||||
|
} |
||||||
|
if p.Optional { |
||||||
|
s += ",opt" |
||||||
|
} |
||||||
|
if p.Repeated { |
||||||
|
s += ",rep" |
||||||
|
} |
||||||
|
if p.Packed { |
||||||
|
s += ",packed" |
||||||
|
} |
||||||
|
s += ",name=" + p.OrigName |
||||||
|
if p.JSONName != p.OrigName { |
||||||
|
s += ",json=" + p.JSONName |
||||||
|
} |
||||||
|
if p.proto3 { |
||||||
|
s += ",proto3" |
||||||
|
} |
||||||
|
if p.oneof { |
||||||
|
s += ",oneof" |
||||||
|
} |
||||||
|
if len(p.Enum) > 0 { |
||||||
|
s += ",enum=" + p.Enum |
||||||
|
} |
||||||
|
if p.HasDefault { |
||||||
|
s += ",def=" + p.Default |
||||||
|
} |
||||||
|
return s |
||||||
|
} |
||||||
|
|
||||||
|
// Parse populates p by parsing a string in the protobuf struct field tag style.
|
||||||
|
func (p *Properties) Parse(s string) { |
||||||
|
// "bytes,49,opt,name=foo,def=hello!"
|
||||||
|
fields := strings.Split(s, ",") // breaks def=, but handled below.
|
||||||
|
if len(fields) < 2 { |
||||||
|
fmt.Fprintf(os.Stderr, "proto: tag has too few fields: %q\n", s) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
p.Wire = fields[0] |
||||||
|
switch p.Wire { |
||||||
|
case "varint": |
||||||
|
p.WireType = WireVarint |
||||||
|
p.valEnc = (*Buffer).EncodeVarint |
||||||
|
p.valDec = (*Buffer).DecodeVarint |
||||||
|
p.valSize = sizeVarint |
||||||
|
case "fixed32": |
||||||
|
p.WireType = WireFixed32 |
||||||
|
p.valEnc = (*Buffer).EncodeFixed32 |
||||||
|
p.valDec = (*Buffer).DecodeFixed32 |
||||||
|
p.valSize = sizeFixed32 |
||||||
|
case "fixed64": |
||||||
|
p.WireType = WireFixed64 |
||||||
|
p.valEnc = (*Buffer).EncodeFixed64 |
||||||
|
p.valDec = (*Buffer).DecodeFixed64 |
||||||
|
p.valSize = sizeFixed64 |
||||||
|
case "zigzag32": |
||||||
|
p.WireType = WireVarint |
||||||
|
p.valEnc = (*Buffer).EncodeZigzag32 |
||||||
|
p.valDec = (*Buffer).DecodeZigzag32 |
||||||
|
p.valSize = sizeZigzag32 |
||||||
|
case "zigzag64": |
||||||
|
p.WireType = WireVarint |
||||||
|
p.valEnc = (*Buffer).EncodeZigzag64 |
||||||
|
p.valDec = (*Buffer).DecodeZigzag64 |
||||||
|
p.valSize = sizeZigzag64 |
||||||
|
case "bytes", "group": |
||||||
|
p.WireType = WireBytes |
||||||
|
// no numeric converter for non-numeric types
|
||||||
|
default: |
||||||
|
fmt.Fprintf(os.Stderr, "proto: tag has unknown wire type: %q\n", s) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
var err error |
||||||
|
p.Tag, err = strconv.Atoi(fields[1]) |
||||||
|
if err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
for i := 2; i < len(fields); i++ { |
||||||
|
f := fields[i] |
||||||
|
switch { |
||||||
|
case f == "req": |
||||||
|
p.Required = true |
||||||
|
case f == "opt": |
||||||
|
p.Optional = true |
||||||
|
case f == "rep": |
||||||
|
p.Repeated = true |
||||||
|
case f == "packed": |
||||||
|
p.Packed = true |
||||||
|
case strings.HasPrefix(f, "name="): |
||||||
|
p.OrigName = f[5:] |
||||||
|
case strings.HasPrefix(f, "json="): |
||||||
|
p.JSONName = f[5:] |
||||||
|
case strings.HasPrefix(f, "enum="): |
||||||
|
p.Enum = f[5:] |
||||||
|
case f == "proto3": |
||||||
|
p.proto3 = true |
||||||
|
case f == "oneof": |
||||||
|
p.oneof = true |
||||||
|
case strings.HasPrefix(f, "def="): |
||||||
|
p.HasDefault = true |
||||||
|
p.Default = f[4:] // rest of string
|
||||||
|
if i+1 < len(fields) { |
||||||
|
// Commas aren't escaped, and def is always last.
|
||||||
|
p.Default += "," + strings.Join(fields[i+1:], ",") |
||||||
|
break |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func logNoSliceEnc(t1, t2 reflect.Type) { |
||||||
|
fmt.Fprintf(os.Stderr, "proto: no slice oenc for %T = []%T\n", t1, t2) |
||||||
|
} |
||||||
|
|
||||||
|
var protoMessageType = reflect.TypeOf((*Message)(nil)).Elem() |
||||||
|
|
||||||
|
// Initialize the fields for encoding and decoding.
|
||||||
|
func (p *Properties) setEncAndDec(typ reflect.Type, f *reflect.StructField, lockGetProp bool) { |
||||||
|
p.enc = nil |
||||||
|
p.dec = nil |
||||||
|
p.size = nil |
||||||
|
|
||||||
|
switch t1 := typ; t1.Kind() { |
||||||
|
default: |
||||||
|
fmt.Fprintf(os.Stderr, "proto: no coders for %v\n", t1) |
||||||
|
|
||||||
|
// proto3 scalar types
|
||||||
|
|
||||||
|
case reflect.Bool: |
||||||
|
p.enc = (*Buffer).enc_proto3_bool |
||||||
|
p.dec = (*Buffer).dec_proto3_bool |
||||||
|
p.size = size_proto3_bool |
||||||
|
case reflect.Int32: |
||||||
|
p.enc = (*Buffer).enc_proto3_int32 |
||||||
|
p.dec = (*Buffer).dec_proto3_int32 |
||||||
|
p.size = size_proto3_int32 |
||||||
|
case reflect.Uint32: |
||||||
|
p.enc = (*Buffer).enc_proto3_uint32 |
||||||
|
p.dec = (*Buffer).dec_proto3_int32 // can reuse
|
||||||
|
p.size = size_proto3_uint32 |
||||||
|
case reflect.Int64, reflect.Uint64: |
||||||
|
p.enc = (*Buffer).enc_proto3_int64 |
||||||
|
p.dec = (*Buffer).dec_proto3_int64 |
||||||
|
p.size = size_proto3_int64 |
||||||
|
case reflect.Float32: |
||||||
|
p.enc = (*Buffer).enc_proto3_uint32 // can just treat them as bits
|
||||||
|
p.dec = (*Buffer).dec_proto3_int32 |
||||||
|
p.size = size_proto3_uint32 |
||||||
|
case reflect.Float64: |
||||||
|
p.enc = (*Buffer).enc_proto3_int64 // can just treat them as bits
|
||||||
|
p.dec = (*Buffer).dec_proto3_int64 |
||||||
|
p.size = size_proto3_int64 |
||||||
|
case reflect.String: |
||||||
|
p.enc = (*Buffer).enc_proto3_string |
||||||
|
p.dec = (*Buffer).dec_proto3_string |
||||||
|
p.size = size_proto3_string |
||||||
|
|
||||||
|
case reflect.Ptr: |
||||||
|
switch t2 := t1.Elem(); t2.Kind() { |
||||||
|
default: |
||||||
|
fmt.Fprintf(os.Stderr, "proto: no encoder function for %v -> %v\n", t1, t2) |
||||||
|
break |
||||||
|
case reflect.Bool: |
||||||
|
p.enc = (*Buffer).enc_bool |
||||||
|
p.dec = (*Buffer).dec_bool |
||||||
|
p.size = size_bool |
||||||
|
case reflect.Int32: |
||||||
|
p.enc = (*Buffer).enc_int32 |
||||||
|
p.dec = (*Buffer).dec_int32 |
||||||
|
p.size = size_int32 |
||||||
|
case reflect.Uint32: |
||||||
|
p.enc = (*Buffer).enc_uint32 |
||||||
|
p.dec = (*Buffer).dec_int32 // can reuse
|
||||||
|
p.size = size_uint32 |
||||||
|
case reflect.Int64, reflect.Uint64: |
||||||
|
p.enc = (*Buffer).enc_int64 |
||||||
|
p.dec = (*Buffer).dec_int64 |
||||||
|
p.size = size_int64 |
||||||
|
case reflect.Float32: |
||||||
|
p.enc = (*Buffer).enc_uint32 // can just treat them as bits
|
||||||
|
p.dec = (*Buffer).dec_int32 |
||||||
|
p.size = size_uint32 |
||||||
|
case reflect.Float64: |
||||||
|
p.enc = (*Buffer).enc_int64 // can just treat them as bits
|
||||||
|
p.dec = (*Buffer).dec_int64 |
||||||
|
p.size = size_int64 |
||||||
|
case reflect.String: |
||||||
|
p.enc = (*Buffer).enc_string |
||||||
|
p.dec = (*Buffer).dec_string |
||||||
|
p.size = size_string |
||||||
|
case reflect.Struct: |
||||||
|
p.stype = t1.Elem() |
||||||
|
p.isMarshaler = isMarshaler(t1) |
||||||
|
p.isUnmarshaler = isUnmarshaler(t1) |
||||||
|
if p.Wire == "bytes" { |
||||||
|
p.enc = (*Buffer).enc_struct_message |
||||||
|
p.dec = (*Buffer).dec_struct_message |
||||||
|
p.size = size_struct_message |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_struct_group |
||||||
|
p.dec = (*Buffer).dec_struct_group |
||||||
|
p.size = size_struct_group |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
case reflect.Slice: |
||||||
|
switch t2 := t1.Elem(); t2.Kind() { |
||||||
|
default: |
||||||
|
logNoSliceEnc(t1, t2) |
||||||
|
break |
||||||
|
case reflect.Bool: |
||||||
|
if p.Packed { |
||||||
|
p.enc = (*Buffer).enc_slice_packed_bool |
||||||
|
p.size = size_slice_packed_bool |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_bool |
||||||
|
p.size = size_slice_bool |
||||||
|
} |
||||||
|
p.dec = (*Buffer).dec_slice_bool |
||||||
|
p.packedDec = (*Buffer).dec_slice_packed_bool |
||||||
|
case reflect.Int32: |
||||||
|
if p.Packed { |
||||||
|
p.enc = (*Buffer).enc_slice_packed_int32 |
||||||
|
p.size = size_slice_packed_int32 |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_int32 |
||||||
|
p.size = size_slice_int32 |
||||||
|
} |
||||||
|
p.dec = (*Buffer).dec_slice_int32 |
||||||
|
p.packedDec = (*Buffer).dec_slice_packed_int32 |
||||||
|
case reflect.Uint32: |
||||||
|
if p.Packed { |
||||||
|
p.enc = (*Buffer).enc_slice_packed_uint32 |
||||||
|
p.size = size_slice_packed_uint32 |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_uint32 |
||||||
|
p.size = size_slice_uint32 |
||||||
|
} |
||||||
|
p.dec = (*Buffer).dec_slice_int32 |
||||||
|
p.packedDec = (*Buffer).dec_slice_packed_int32 |
||||||
|
case reflect.Int64, reflect.Uint64: |
||||||
|
if p.Packed { |
||||||
|
p.enc = (*Buffer).enc_slice_packed_int64 |
||||||
|
p.size = size_slice_packed_int64 |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_int64 |
||||||
|
p.size = size_slice_int64 |
||||||
|
} |
||||||
|
p.dec = (*Buffer).dec_slice_int64 |
||||||
|
p.packedDec = (*Buffer).dec_slice_packed_int64 |
||||||
|
case reflect.Uint8: |
||||||
|
p.dec = (*Buffer).dec_slice_byte |
||||||
|
if p.proto3 { |
||||||
|
p.enc = (*Buffer).enc_proto3_slice_byte |
||||||
|
p.size = size_proto3_slice_byte |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_byte |
||||||
|
p.size = size_slice_byte |
||||||
|
} |
||||||
|
case reflect.Float32, reflect.Float64: |
||||||
|
switch t2.Bits() { |
||||||
|
case 32: |
||||||
|
// can just treat them as bits
|
||||||
|
if p.Packed { |
||||||
|
p.enc = (*Buffer).enc_slice_packed_uint32 |
||||||
|
p.size = size_slice_packed_uint32 |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_uint32 |
||||||
|
p.size = size_slice_uint32 |
||||||
|
} |
||||||
|
p.dec = (*Buffer).dec_slice_int32 |
||||||
|
p.packedDec = (*Buffer).dec_slice_packed_int32 |
||||||
|
case 64: |
||||||
|
// can just treat them as bits
|
||||||
|
if p.Packed { |
||||||
|
p.enc = (*Buffer).enc_slice_packed_int64 |
||||||
|
p.size = size_slice_packed_int64 |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_int64 |
||||||
|
p.size = size_slice_int64 |
||||||
|
} |
||||||
|
p.dec = (*Buffer).dec_slice_int64 |
||||||
|
p.packedDec = (*Buffer).dec_slice_packed_int64 |
||||||
|
default: |
||||||
|
logNoSliceEnc(t1, t2) |
||||||
|
break |
||||||
|
} |
||||||
|
case reflect.String: |
||||||
|
p.enc = (*Buffer).enc_slice_string |
||||||
|
p.dec = (*Buffer).dec_slice_string |
||||||
|
p.size = size_slice_string |
||||||
|
case reflect.Ptr: |
||||||
|
switch t3 := t2.Elem(); t3.Kind() { |
||||||
|
default: |
||||||
|
fmt.Fprintf(os.Stderr, "proto: no ptr oenc for %T -> %T -> %T\n", t1, t2, t3) |
||||||
|
break |
||||||
|
case reflect.Struct: |
||||||
|
p.stype = t2.Elem() |
||||||
|
p.isMarshaler = isMarshaler(t2) |
||||||
|
p.isUnmarshaler = isUnmarshaler(t2) |
||||||
|
if p.Wire == "bytes" { |
||||||
|
p.enc = (*Buffer).enc_slice_struct_message |
||||||
|
p.dec = (*Buffer).dec_slice_struct_message |
||||||
|
p.size = size_slice_struct_message |
||||||
|
} else { |
||||||
|
p.enc = (*Buffer).enc_slice_struct_group |
||||||
|
p.dec = (*Buffer).dec_slice_struct_group |
||||||
|
p.size = size_slice_struct_group |
||||||
|
} |
||||||
|
} |
||||||
|
case reflect.Slice: |
||||||
|
switch t2.Elem().Kind() { |
||||||
|
default: |
||||||
|
fmt.Fprintf(os.Stderr, "proto: no slice elem oenc for %T -> %T -> %T\n", t1, t2, t2.Elem()) |
||||||
|
break |
||||||
|
case reflect.Uint8: |
||||||
|
p.enc = (*Buffer).enc_slice_slice_byte |
||||||
|
p.dec = (*Buffer).dec_slice_slice_byte |
||||||
|
p.size = size_slice_slice_byte |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
case reflect.Map: |
||||||
|
p.enc = (*Buffer).enc_new_map |
||||||
|
p.dec = (*Buffer).dec_new_map |
||||||
|
p.size = size_new_map |
||||||
|
|
||||||
|
p.mtype = t1 |
||||||
|
p.mkeyprop = &Properties{} |
||||||
|
p.mkeyprop.init(reflect.PtrTo(p.mtype.Key()), "Key", f.Tag.Get("protobuf_key"), nil, lockGetProp) |
||||||
|
p.mvalprop = &Properties{} |
||||||
|
vtype := p.mtype.Elem() |
||||||
|
if vtype.Kind() != reflect.Ptr && vtype.Kind() != reflect.Slice { |
||||||
|
// The value type is not a message (*T) or bytes ([]byte),
|
||||||
|
// so we need encoders for the pointer to this type.
|
||||||
|
vtype = reflect.PtrTo(vtype) |
||||||
|
} |
||||||
|
p.mvalprop.init(vtype, "Value", f.Tag.Get("protobuf_val"), nil, lockGetProp) |
||||||
|
} |
||||||
|
|
||||||
|
// precalculate tag code
|
||||||
|
wire := p.WireType |
||||||
|
if p.Packed { |
||||||
|
wire = WireBytes |
||||||
|
} |
||||||
|
x := uint32(p.Tag)<<3 | uint32(wire) |
||||||
|
i := 0 |
||||||
|
for i = 0; x > 127; i++ { |
||||||
|
p.tagbuf[i] = 0x80 | uint8(x&0x7F) |
||||||
|
x >>= 7 |
||||||
|
} |
||||||
|
p.tagbuf[i] = uint8(x) |
||||||
|
p.tagcode = p.tagbuf[0 : i+1] |
||||||
|
|
||||||
|
if p.stype != nil { |
||||||
|
if lockGetProp { |
||||||
|
p.sprop = GetProperties(p.stype) |
||||||
|
} else { |
||||||
|
p.sprop = getPropertiesLocked(p.stype) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
var ( |
||||||
|
marshalerType = reflect.TypeOf((*Marshaler)(nil)).Elem() |
||||||
|
unmarshalerType = reflect.TypeOf((*Unmarshaler)(nil)).Elem() |
||||||
|
) |
||||||
|
|
||||||
|
// isMarshaler reports whether type t implements Marshaler.
|
||||||
|
func isMarshaler(t reflect.Type) bool { |
||||||
|
// We're checking for (likely) pointer-receiver methods
|
||||||
|
// so if t is not a pointer, something is very wrong.
|
||||||
|
// The calls above only invoke isMarshaler on pointer types.
|
||||||
|
if t.Kind() != reflect.Ptr { |
||||||
|
panic("proto: misuse of isMarshaler") |
||||||
|
} |
||||||
|
return t.Implements(marshalerType) |
||||||
|
} |
||||||
|
|
||||||
|
// isUnmarshaler reports whether type t implements Unmarshaler.
|
||||||
|
func isUnmarshaler(t reflect.Type) bool { |
||||||
|
// We're checking for (likely) pointer-receiver methods
|
||||||
|
// so if t is not a pointer, something is very wrong.
|
||||||
|
// The calls above only invoke isUnmarshaler on pointer types.
|
||||||
|
if t.Kind() != reflect.Ptr { |
||||||
|
panic("proto: misuse of isUnmarshaler") |
||||||
|
} |
||||||
|
return t.Implements(unmarshalerType) |
||||||
|
} |
||||||
|
|
||||||
|
// Init populates the properties from a protocol buffer struct tag.
|
||||||
|
func (p *Properties) Init(typ reflect.Type, name, tag string, f *reflect.StructField) { |
||||||
|
p.init(typ, name, tag, f, true) |
||||||
|
} |
||||||
|
|
||||||
|
func (p *Properties) init(typ reflect.Type, name, tag string, f *reflect.StructField, lockGetProp bool) { |
||||||
|
// "bytes,49,opt,def=hello!"
|
||||||
|
p.Name = name |
||||||
|
p.OrigName = name |
||||||
|
if f != nil { |
||||||
|
p.field = toField(f) |
||||||
|
} |
||||||
|
if tag == "" { |
||||||
|
return |
||||||
|
} |
||||||
|
p.Parse(tag) |
||||||
|
p.setEncAndDec(typ, f, lockGetProp) |
||||||
|
} |
||||||
|
|
||||||
|
var ( |
||||||
|
propertiesMu sync.RWMutex |
||||||
|
propertiesMap = make(map[reflect.Type]*StructProperties) |
||||||
|
) |
||||||
|
|
||||||
|
// GetProperties returns the list of properties for the type represented by t.
|
||||||
|
// t must represent a generated struct type of a protocol message.
|
||||||
|
func GetProperties(t reflect.Type) *StructProperties { |
||||||
|
if t.Kind() != reflect.Struct { |
||||||
|
panic("proto: type must have kind struct") |
||||||
|
} |
||||||
|
|
||||||
|
// Most calls to GetProperties in a long-running program will be
|
||||||
|
// retrieving details for types we have seen before.
|
||||||
|
propertiesMu.RLock() |
||||||
|
sprop, ok := propertiesMap[t] |
||||||
|
propertiesMu.RUnlock() |
||||||
|
if ok { |
||||||
|
if collectStats { |
||||||
|
stats.Chit++ |
||||||
|
} |
||||||
|
return sprop |
||||||
|
} |
||||||
|
|
||||||
|
propertiesMu.Lock() |
||||||
|
sprop = getPropertiesLocked(t) |
||||||
|
propertiesMu.Unlock() |
||||||
|
return sprop |
||||||
|
} |
||||||
|
|
||||||
|
// getPropertiesLocked requires that propertiesMu is held.
|
||||||
|
func getPropertiesLocked(t reflect.Type) *StructProperties { |
||||||
|
if prop, ok := propertiesMap[t]; ok { |
||||||
|
if collectStats { |
||||||
|
stats.Chit++ |
||||||
|
} |
||||||
|
return prop |
||||||
|
} |
||||||
|
if collectStats { |
||||||
|
stats.Cmiss++ |
||||||
|
} |
||||||
|
|
||||||
|
prop := new(StructProperties) |
||||||
|
// in case of recursive protos, fill this in now.
|
||||||
|
propertiesMap[t] = prop |
||||||
|
|
||||||
|
// build properties
|
||||||
|
prop.extendable = reflect.PtrTo(t).Implements(extendableProtoType) || |
||||||
|
reflect.PtrTo(t).Implements(extendableProtoV1Type) |
||||||
|
prop.unrecField = invalidField |
||||||
|
prop.Prop = make([]*Properties, t.NumField()) |
||||||
|
prop.order = make([]int, t.NumField()) |
||||||
|
|
||||||
|
for i := 0; i < t.NumField(); i++ { |
||||||
|
f := t.Field(i) |
||||||
|
p := new(Properties) |
||||||
|
name := f.Name |
||||||
|
p.init(f.Type, name, f.Tag.Get("protobuf"), &f, false) |
||||||
|
|
||||||
|
if f.Name == "XXX_InternalExtensions" { // special case
|
||||||
|
p.enc = (*Buffer).enc_exts |
||||||
|
p.dec = nil // not needed
|
||||||
|
p.size = size_exts |
||||||
|
} else if f.Name == "XXX_extensions" { // special case
|
||||||
|
p.enc = (*Buffer).enc_map |
||||||
|
p.dec = nil // not needed
|
||||||
|
p.size = size_map |
||||||
|
} else if f.Name == "XXX_unrecognized" { // special case
|
||||||
|
prop.unrecField = toField(&f) |
||||||
|
} |
||||||
|
oneof := f.Tag.Get("protobuf_oneof") // special case
|
||||||
|
if oneof != "" { |
||||||
|
// Oneof fields don't use the traditional protobuf tag.
|
||||||
|
p.OrigName = oneof |
||||||
|
} |
||||||
|
prop.Prop[i] = p |
||||||
|
prop.order[i] = i |
||||||
|
if debug { |
||||||
|
print(i, " ", f.Name, " ", t.String(), " ") |
||||||
|
if p.Tag > 0 { |
||||||
|
print(p.String()) |
||||||
|
} |
||||||
|
print("\n") |
||||||
|
} |
||||||
|
if p.enc == nil && !strings.HasPrefix(f.Name, "XXX_") && oneof == "" { |
||||||
|
fmt.Fprintln(os.Stderr, "proto: no encoder for", f.Name, f.Type.String(), "[GetProperties]") |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Re-order prop.order.
|
||||||
|
sort.Sort(prop) |
||||||
|
|
||||||
|
type oneofMessage interface { |
||||||
|
XXX_OneofFuncs() (func(Message, *Buffer) error, func(Message, int, int, *Buffer) (bool, error), func(Message) int, []interface{}) |
||||||
|
} |
||||||
|
if om, ok := reflect.Zero(reflect.PtrTo(t)).Interface().(oneofMessage); ok { |
||||||
|
var oots []interface{} |
||||||
|
prop.oneofMarshaler, prop.oneofUnmarshaler, prop.oneofSizer, oots = om.XXX_OneofFuncs() |
||||||
|
prop.stype = t |
||||||
|
|
||||||
|
// Interpret oneof metadata.
|
||||||
|
prop.OneofTypes = make(map[string]*OneofProperties) |
||||||
|
for _, oot := range oots { |
||||||
|
oop := &OneofProperties{ |
||||||
|
Type: reflect.ValueOf(oot).Type(), // *T
|
||||||
|
Prop: new(Properties), |
||||||
|
} |
||||||
|
sft := oop.Type.Elem().Field(0) |
||||||
|
oop.Prop.Name = sft.Name |
||||||
|
oop.Prop.Parse(sft.Tag.Get("protobuf")) |
||||||
|
// There will be exactly one interface field that
|
||||||
|
// this new value is assignable to.
|
||||||
|
for i := 0; i < t.NumField(); i++ { |
||||||
|
f := t.Field(i) |
||||||
|
if f.Type.Kind() != reflect.Interface { |
||||||
|
continue |
||||||
|
} |
||||||
|
if !oop.Type.AssignableTo(f.Type) { |
||||||
|
continue |
||||||
|
} |
||||||
|
oop.Field = i |
||||||
|
break |
||||||
|
} |
||||||
|
prop.OneofTypes[oop.Prop.OrigName] = oop |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// build required counts
|
||||||
|
// build tags
|
||||||
|
reqCount := 0 |
||||||
|
prop.decoderOrigNames = make(map[string]int) |
||||||
|
for i, p := range prop.Prop { |
||||||
|
if strings.HasPrefix(p.Name, "XXX_") { |
||||||
|
// Internal fields should not appear in tags/origNames maps.
|
||||||
|
// They are handled specially when encoding and decoding.
|
||||||
|
continue |
||||||
|
} |
||||||
|
if p.Required { |
||||||
|
reqCount++ |
||||||
|
} |
||||||
|
prop.decoderTags.put(p.Tag, i) |
||||||
|
prop.decoderOrigNames[p.OrigName] = i |
||||||
|
} |
||||||
|
prop.reqCount = reqCount |
||||||
|
|
||||||
|
return prop |
||||||
|
} |
||||||
|
|
||||||
|
// Return the Properties object for the x[0]'th field of the structure.
|
||||||
|
func propByIndex(t reflect.Type, x []int) *Properties { |
||||||
|
if len(x) != 1 { |
||||||
|
fmt.Fprintf(os.Stderr, "proto: field index dimension %d (not 1) for type %s\n", len(x), t) |
||||||
|
return nil |
||||||
|
} |
||||||
|
prop := GetProperties(t) |
||||||
|
return prop.Prop[x[0]] |
||||||
|
} |
||||||
|
|
||||||
|
// Get the address and type of a pointer to a struct from an interface.
|
||||||
|
func getbase(pb Message) (t reflect.Type, b structPointer, err error) { |
||||||
|
if pb == nil { |
||||||
|
err = ErrNil |
||||||
|
return |
||||||
|
} |
||||||
|
// get the reflect type of the pointer to the struct.
|
||||||
|
t = reflect.TypeOf(pb) |
||||||
|
// get the address of the struct.
|
||||||
|
value := reflect.ValueOf(pb) |
||||||
|
b = toStructPointer(value) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// A global registry of enum types.
|
||||||
|
// The generated code will register the generated maps by calling RegisterEnum.
|
||||||
|
|
||||||
|
var enumValueMaps = make(map[string]map[string]int32) |
||||||
|
|
||||||
|
// RegisterEnum is called from the generated code to install the enum descriptor
|
||||||
|
// maps into the global table to aid parsing text format protocol buffers.
|
||||||
|
func RegisterEnum(typeName string, unusedNameMap map[int32]string, valueMap map[string]int32) { |
||||||
|
if _, ok := enumValueMaps[typeName]; ok { |
||||||
|
panic("proto: duplicate enum registered: " + typeName) |
||||||
|
} |
||||||
|
enumValueMaps[typeName] = valueMap |
||||||
|
} |
||||||
|
|
||||||
|
// EnumValueMap returns the mapping from names to integers of the
|
||||||
|
// enum type enumType, or a nil if not found.
|
||||||
|
func EnumValueMap(enumType string) map[string]int32 { |
||||||
|
return enumValueMaps[enumType] |
||||||
|
} |
||||||
|
|
||||||
|
// A registry of all linked message types.
|
||||||
|
// The string is a fully-qualified proto name ("pkg.Message").
|
||||||
|
var ( |
||||||
|
protoTypes = make(map[string]reflect.Type) |
||||||
|
revProtoTypes = make(map[reflect.Type]string) |
||||||
|
) |
||||||
|
|
||||||
|
// RegisterType is called from generated code and maps from the fully qualified
|
||||||
|
// proto name to the type (pointer to struct) of the protocol buffer.
|
||||||
|
func RegisterType(x Message, name string) { |
||||||
|
if _, ok := protoTypes[name]; ok { |
||||||
|
// TODO: Some day, make this a panic.
|
||||||
|
log.Printf("proto: duplicate proto type registered: %s", name) |
||||||
|
return |
||||||
|
} |
||||||
|
t := reflect.TypeOf(x) |
||||||
|
protoTypes[name] = t |
||||||
|
revProtoTypes[t] = name |
||||||
|
} |
||||||
|
|
||||||
|
// MessageName returns the fully-qualified proto name for the given message type.
|
||||||
|
func MessageName(x Message) string { |
||||||
|
type xname interface { |
||||||
|
XXX_MessageName() string |
||||||
|
} |
||||||
|
if m, ok := x.(xname); ok { |
||||||
|
return m.XXX_MessageName() |
||||||
|
} |
||||||
|
return revProtoTypes[reflect.TypeOf(x)] |
||||||
|
} |
||||||
|
|
||||||
|
// MessageType returns the message type (pointer to struct) for a named message.
|
||||||
|
func MessageType(name string) reflect.Type { return protoTypes[name] } |
||||||
|
|
||||||
|
// A registry of all linked proto files.
|
||||||
|
var ( |
||||||
|
protoFiles = make(map[string][]byte) // file name => fileDescriptor
|
||||||
|
) |
||||||
|
|
||||||
|
// RegisterFile is called from generated code and maps from the
|
||||||
|
// full file name of a .proto file to its compressed FileDescriptorProto.
|
||||||
|
func RegisterFile(filename string, fileDescriptor []byte) { |
||||||
|
protoFiles[filename] = fileDescriptor |
||||||
|
} |
||||||
|
|
||||||
|
// FileDescriptor returns the compressed FileDescriptorProto for a .proto file.
|
||||||
|
func FileDescriptor(filename string) []byte { return protoFiles[filename] } |
||||||
@ -0,0 +1,854 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
// Functions for writing the text protocol buffer format.
|
||||||
|
|
||||||
|
import ( |
||||||
|
"bufio" |
||||||
|
"bytes" |
||||||
|
"encoding" |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"log" |
||||||
|
"math" |
||||||
|
"reflect" |
||||||
|
"sort" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
var ( |
||||||
|
newline = []byte("\n") |
||||||
|
spaces = []byte(" ") |
||||||
|
gtNewline = []byte(">\n") |
||||||
|
endBraceNewline = []byte("}\n") |
||||||
|
backslashN = []byte{'\\', 'n'} |
||||||
|
backslashR = []byte{'\\', 'r'} |
||||||
|
backslashT = []byte{'\\', 't'} |
||||||
|
backslashDQ = []byte{'\\', '"'} |
||||||
|
backslashBS = []byte{'\\', '\\'} |
||||||
|
posInf = []byte("inf") |
||||||
|
negInf = []byte("-inf") |
||||||
|
nan = []byte("nan") |
||||||
|
) |
||||||
|
|
||||||
|
type writer interface { |
||||||
|
io.Writer |
||||||
|
WriteByte(byte) error |
||||||
|
} |
||||||
|
|
||||||
|
// textWriter is an io.Writer that tracks its indentation level.
|
||||||
|
type textWriter struct { |
||||||
|
ind int |
||||||
|
complete bool // if the current position is a complete line
|
||||||
|
compact bool // whether to write out as a one-liner
|
||||||
|
w writer |
||||||
|
} |
||||||
|
|
||||||
|
func (w *textWriter) WriteString(s string) (n int, err error) { |
||||||
|
if !strings.Contains(s, "\n") { |
||||||
|
if !w.compact && w.complete { |
||||||
|
w.writeIndent() |
||||||
|
} |
||||||
|
w.complete = false |
||||||
|
return io.WriteString(w.w, s) |
||||||
|
} |
||||||
|
// WriteString is typically called without newlines, so this
|
||||||
|
// codepath and its copy are rare. We copy to avoid
|
||||||
|
// duplicating all of Write's logic here.
|
||||||
|
return w.Write([]byte(s)) |
||||||
|
} |
||||||
|
|
||||||
|
func (w *textWriter) Write(p []byte) (n int, err error) { |
||||||
|
newlines := bytes.Count(p, newline) |
||||||
|
if newlines == 0 { |
||||||
|
if !w.compact && w.complete { |
||||||
|
w.writeIndent() |
||||||
|
} |
||||||
|
n, err = w.w.Write(p) |
||||||
|
w.complete = false |
||||||
|
return n, err |
||||||
|
} |
||||||
|
|
||||||
|
frags := bytes.SplitN(p, newline, newlines+1) |
||||||
|
if w.compact { |
||||||
|
for i, frag := range frags { |
||||||
|
if i > 0 { |
||||||
|
if err := w.w.WriteByte(' '); err != nil { |
||||||
|
return n, err |
||||||
|
} |
||||||
|
n++ |
||||||
|
} |
||||||
|
nn, err := w.w.Write(frag) |
||||||
|
n += nn |
||||||
|
if err != nil { |
||||||
|
return n, err |
||||||
|
} |
||||||
|
} |
||||||
|
return n, nil |
||||||
|
} |
||||||
|
|
||||||
|
for i, frag := range frags { |
||||||
|
if w.complete { |
||||||
|
w.writeIndent() |
||||||
|
} |
||||||
|
nn, err := w.w.Write(frag) |
||||||
|
n += nn |
||||||
|
if err != nil { |
||||||
|
return n, err |
||||||
|
} |
||||||
|
if i+1 < len(frags) { |
||||||
|
if err := w.w.WriteByte('\n'); err != nil { |
||||||
|
return n, err |
||||||
|
} |
||||||
|
n++ |
||||||
|
} |
||||||
|
} |
||||||
|
w.complete = len(frags[len(frags)-1]) == 0 |
||||||
|
return n, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (w *textWriter) WriteByte(c byte) error { |
||||||
|
if w.compact && c == '\n' { |
||||||
|
c = ' ' |
||||||
|
} |
||||||
|
if !w.compact && w.complete { |
||||||
|
w.writeIndent() |
||||||
|
} |
||||||
|
err := w.w.WriteByte(c) |
||||||
|
w.complete = c == '\n' |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
func (w *textWriter) indent() { w.ind++ } |
||||||
|
|
||||||
|
func (w *textWriter) unindent() { |
||||||
|
if w.ind == 0 { |
||||||
|
log.Print("proto: textWriter unindented too far") |
||||||
|
return |
||||||
|
} |
||||||
|
w.ind-- |
||||||
|
} |
||||||
|
|
||||||
|
func writeName(w *textWriter, props *Properties) error { |
||||||
|
if _, err := w.WriteString(props.OrigName); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if props.Wire != "group" { |
||||||
|
return w.WriteByte(':') |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// raw is the interface satisfied by RawMessage.
|
||||||
|
type raw interface { |
||||||
|
Bytes() []byte |
||||||
|
} |
||||||
|
|
||||||
|
func requiresQuotes(u string) bool { |
||||||
|
// When type URL contains any characters except [0-9A-Za-z./\-]*, it must be quoted.
|
||||||
|
for _, ch := range u { |
||||||
|
switch { |
||||||
|
case ch == '.' || ch == '/' || ch == '_': |
||||||
|
continue |
||||||
|
case '0' <= ch && ch <= '9': |
||||||
|
continue |
||||||
|
case 'A' <= ch && ch <= 'Z': |
||||||
|
continue |
||||||
|
case 'a' <= ch && ch <= 'z': |
||||||
|
continue |
||||||
|
default: |
||||||
|
return true |
||||||
|
} |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
// isAny reports whether sv is a google.protobuf.Any message
|
||||||
|
func isAny(sv reflect.Value) bool { |
||||||
|
type wkt interface { |
||||||
|
XXX_WellKnownType() string |
||||||
|
} |
||||||
|
t, ok := sv.Addr().Interface().(wkt) |
||||||
|
return ok && t.XXX_WellKnownType() == "Any" |
||||||
|
} |
||||||
|
|
||||||
|
// writeProto3Any writes an expanded google.protobuf.Any message.
|
||||||
|
//
|
||||||
|
// It returns (false, nil) if sv value can't be unmarshaled (e.g. because
|
||||||
|
// required messages are not linked in).
|
||||||
|
//
|
||||||
|
// It returns (true, error) when sv was written in expanded format or an error
|
||||||
|
// was encountered.
|
||||||
|
func (tm *TextMarshaler) writeProto3Any(w *textWriter, sv reflect.Value) (bool, error) { |
||||||
|
turl := sv.FieldByName("TypeUrl") |
||||||
|
val := sv.FieldByName("Value") |
||||||
|
if !turl.IsValid() || !val.IsValid() { |
||||||
|
return true, errors.New("proto: invalid google.protobuf.Any message") |
||||||
|
} |
||||||
|
|
||||||
|
b, ok := val.Interface().([]byte) |
||||||
|
if !ok { |
||||||
|
return true, errors.New("proto: invalid google.protobuf.Any message") |
||||||
|
} |
||||||
|
|
||||||
|
parts := strings.Split(turl.String(), "/") |
||||||
|
mt := MessageType(parts[len(parts)-1]) |
||||||
|
if mt == nil { |
||||||
|
return false, nil |
||||||
|
} |
||||||
|
m := reflect.New(mt.Elem()) |
||||||
|
if err := Unmarshal(b, m.Interface().(Message)); err != nil { |
||||||
|
return false, nil |
||||||
|
} |
||||||
|
w.Write([]byte("[")) |
||||||
|
u := turl.String() |
||||||
|
if requiresQuotes(u) { |
||||||
|
writeString(w, u) |
||||||
|
} else { |
||||||
|
w.Write([]byte(u)) |
||||||
|
} |
||||||
|
if w.compact { |
||||||
|
w.Write([]byte("]:<")) |
||||||
|
} else { |
||||||
|
w.Write([]byte("]: <\n")) |
||||||
|
w.ind++ |
||||||
|
} |
||||||
|
if err := tm.writeStruct(w, m.Elem()); err != nil { |
||||||
|
return true, err |
||||||
|
} |
||||||
|
if w.compact { |
||||||
|
w.Write([]byte("> ")) |
||||||
|
} else { |
||||||
|
w.ind-- |
||||||
|
w.Write([]byte(">\n")) |
||||||
|
} |
||||||
|
return true, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (tm *TextMarshaler) writeStruct(w *textWriter, sv reflect.Value) error { |
||||||
|
if tm.ExpandAny && isAny(sv) { |
||||||
|
if canExpand, err := tm.writeProto3Any(w, sv); canExpand { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
st := sv.Type() |
||||||
|
sprops := GetProperties(st) |
||||||
|
for i := 0; i < sv.NumField(); i++ { |
||||||
|
fv := sv.Field(i) |
||||||
|
props := sprops.Prop[i] |
||||||
|
name := st.Field(i).Name |
||||||
|
|
||||||
|
if strings.HasPrefix(name, "XXX_") { |
||||||
|
// There are two XXX_ fields:
|
||||||
|
// XXX_unrecognized []byte
|
||||||
|
// XXX_extensions map[int32]proto.Extension
|
||||||
|
// The first is handled here;
|
||||||
|
// the second is handled at the bottom of this function.
|
||||||
|
if name == "XXX_unrecognized" && !fv.IsNil() { |
||||||
|
if err := writeUnknownStruct(w, fv.Interface().([]byte)); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
if fv.Kind() == reflect.Ptr && fv.IsNil() { |
||||||
|
// Field not filled in. This could be an optional field or
|
||||||
|
// a required field that wasn't filled in. Either way, there
|
||||||
|
// isn't anything we can show for it.
|
||||||
|
continue |
||||||
|
} |
||||||
|
if fv.Kind() == reflect.Slice && fv.IsNil() { |
||||||
|
// Repeated field that is empty, or a bytes field that is unused.
|
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
if props.Repeated && fv.Kind() == reflect.Slice { |
||||||
|
// Repeated field.
|
||||||
|
for j := 0; j < fv.Len(); j++ { |
||||||
|
if err := writeName(w, props); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte(' '); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
v := fv.Index(j) |
||||||
|
if v.Kind() == reflect.Ptr && v.IsNil() { |
||||||
|
// A nil message in a repeated field is not valid,
|
||||||
|
// but we can handle that more gracefully than panicking.
|
||||||
|
if _, err := w.Write([]byte("<nil>\n")); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
if err := tm.writeAny(w, v, props); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
if fv.Kind() == reflect.Map { |
||||||
|
// Map fields are rendered as a repeated struct with key/value fields.
|
||||||
|
keys := fv.MapKeys() |
||||||
|
sort.Sort(mapKeys(keys)) |
||||||
|
for _, key := range keys { |
||||||
|
val := fv.MapIndex(key) |
||||||
|
if err := writeName(w, props); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte(' '); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
// open struct
|
||||||
|
if err := w.WriteByte('<'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
w.indent() |
||||||
|
// key
|
||||||
|
if _, err := w.WriteString("key:"); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte(' '); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
if err := tm.writeAny(w, key, props.mkeyprop); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
// nil values aren't legal, but we can avoid panicking because of them.
|
||||||
|
if val.Kind() != reflect.Ptr || !val.IsNil() { |
||||||
|
// value
|
||||||
|
if _, err := w.WriteString("value:"); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte(' '); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
if err := tm.writeAny(w, val, props.mvalprop); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
// close struct
|
||||||
|
w.unindent() |
||||||
|
if err := w.WriteByte('>'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
if props.proto3 && fv.Kind() == reflect.Slice && fv.Len() == 0 { |
||||||
|
// empty bytes field
|
||||||
|
continue |
||||||
|
} |
||||||
|
if fv.Kind() != reflect.Ptr && fv.Kind() != reflect.Slice { |
||||||
|
// proto3 non-repeated scalar field; skip if zero value
|
||||||
|
if isProto3Zero(fv) { |
||||||
|
continue |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if fv.Kind() == reflect.Interface { |
||||||
|
// Check if it is a oneof.
|
||||||
|
if st.Field(i).Tag.Get("protobuf_oneof") != "" { |
||||||
|
// fv is nil, or holds a pointer to generated struct.
|
||||||
|
// That generated struct has exactly one field,
|
||||||
|
// which has a protobuf struct tag.
|
||||||
|
if fv.IsNil() { |
||||||
|
continue |
||||||
|
} |
||||||
|
inner := fv.Elem().Elem() // interface -> *T -> T
|
||||||
|
tag := inner.Type().Field(0).Tag.Get("protobuf") |
||||||
|
props = new(Properties) // Overwrite the outer props var, but not its pointee.
|
||||||
|
props.Parse(tag) |
||||||
|
// Write the value in the oneof, not the oneof itself.
|
||||||
|
fv = inner.Field(0) |
||||||
|
|
||||||
|
// Special case to cope with malformed messages gracefully:
|
||||||
|
// If the value in the oneof is a nil pointer, don't panic
|
||||||
|
// in writeAny.
|
||||||
|
if fv.Kind() == reflect.Ptr && fv.IsNil() { |
||||||
|
// Use errors.New so writeAny won't render quotes.
|
||||||
|
msg := errors.New("/* nil */") |
||||||
|
fv = reflect.ValueOf(&msg).Elem() |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if err := writeName(w, props); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte(' '); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
if b, ok := fv.Interface().(raw); ok { |
||||||
|
if err := writeRaw(w, b.Bytes()); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
// Enums have a String method, so writeAny will work fine.
|
||||||
|
if err := tm.writeAny(w, fv, props); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Extensions (the XXX_extensions field).
|
||||||
|
pv := sv.Addr() |
||||||
|
if _, ok := extendable(pv.Interface()); ok { |
||||||
|
if err := tm.writeExtensions(w, pv); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// writeRaw writes an uninterpreted raw message.
|
||||||
|
func writeRaw(w *textWriter, b []byte) error { |
||||||
|
if err := w.WriteByte('<'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
w.indent() |
||||||
|
if err := writeUnknownStruct(w, b); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
w.unindent() |
||||||
|
if err := w.WriteByte('>'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// writeAny writes an arbitrary field.
|
||||||
|
func (tm *TextMarshaler) writeAny(w *textWriter, v reflect.Value, props *Properties) error { |
||||||
|
v = reflect.Indirect(v) |
||||||
|
|
||||||
|
// Floats have special cases.
|
||||||
|
if v.Kind() == reflect.Float32 || v.Kind() == reflect.Float64 { |
||||||
|
x := v.Float() |
||||||
|
var b []byte |
||||||
|
switch { |
||||||
|
case math.IsInf(x, 1): |
||||||
|
b = posInf |
||||||
|
case math.IsInf(x, -1): |
||||||
|
b = negInf |
||||||
|
case math.IsNaN(x): |
||||||
|
b = nan |
||||||
|
} |
||||||
|
if b != nil { |
||||||
|
_, err := w.Write(b) |
||||||
|
return err |
||||||
|
} |
||||||
|
// Other values are handled below.
|
||||||
|
} |
||||||
|
|
||||||
|
// We don't attempt to serialise every possible value type; only those
|
||||||
|
// that can occur in protocol buffers.
|
||||||
|
switch v.Kind() { |
||||||
|
case reflect.Slice: |
||||||
|
// Should only be a []byte; repeated fields are handled in writeStruct.
|
||||||
|
if err := writeString(w, string(v.Bytes())); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
case reflect.String: |
||||||
|
if err := writeString(w, v.String()); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
case reflect.Struct: |
||||||
|
// Required/optional group/message.
|
||||||
|
var bra, ket byte = '<', '>' |
||||||
|
if props != nil && props.Wire == "group" { |
||||||
|
bra, ket = '{', '}' |
||||||
|
} |
||||||
|
if err := w.WriteByte(bra); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
w.indent() |
||||||
|
if etm, ok := v.Interface().(encoding.TextMarshaler); ok { |
||||||
|
text, err := etm.MarshalText() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if _, err = w.Write(text); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} else if err := tm.writeStruct(w, v); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
w.unindent() |
||||||
|
if err := w.WriteByte(ket); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
default: |
||||||
|
_, err := fmt.Fprint(w, v.Interface()) |
||||||
|
return err |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// equivalent to C's isprint.
|
||||||
|
func isprint(c byte) bool { |
||||||
|
return c >= 0x20 && c < 0x7f |
||||||
|
} |
||||||
|
|
||||||
|
// writeString writes a string in the protocol buffer text format.
|
||||||
|
// It is similar to strconv.Quote except we don't use Go escape sequences,
|
||||||
|
// we treat the string as a byte sequence, and we use octal escapes.
|
||||||
|
// These differences are to maintain interoperability with the other
|
||||||
|
// languages' implementations of the text format.
|
||||||
|
func writeString(w *textWriter, s string) error { |
||||||
|
// use WriteByte here to get any needed indent
|
||||||
|
if err := w.WriteByte('"'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
// Loop over the bytes, not the runes.
|
||||||
|
for i := 0; i < len(s); i++ { |
||||||
|
var err error |
||||||
|
// Divergence from C++: we don't escape apostrophes.
|
||||||
|
// There's no need to escape them, and the C++ parser
|
||||||
|
// copes with a naked apostrophe.
|
||||||
|
switch c := s[i]; c { |
||||||
|
case '\n': |
||||||
|
_, err = w.w.Write(backslashN) |
||||||
|
case '\r': |
||||||
|
_, err = w.w.Write(backslashR) |
||||||
|
case '\t': |
||||||
|
_, err = w.w.Write(backslashT) |
||||||
|
case '"': |
||||||
|
_, err = w.w.Write(backslashDQ) |
||||||
|
case '\\': |
||||||
|
_, err = w.w.Write(backslashBS) |
||||||
|
default: |
||||||
|
if isprint(c) { |
||||||
|
err = w.w.WriteByte(c) |
||||||
|
} else { |
||||||
|
_, err = fmt.Fprintf(w.w, "\\%03o", c) |
||||||
|
} |
||||||
|
} |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
return w.WriteByte('"') |
||||||
|
} |
||||||
|
|
||||||
|
func writeUnknownStruct(w *textWriter, data []byte) (err error) { |
||||||
|
if !w.compact { |
||||||
|
if _, err := fmt.Fprintf(w, "/* %d unknown bytes */\n", len(data)); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
b := NewBuffer(data) |
||||||
|
for b.index < len(b.buf) { |
||||||
|
x, err := b.DecodeVarint() |
||||||
|
if err != nil { |
||||||
|
_, err := fmt.Fprintf(w, "/* %v */\n", err) |
||||||
|
return err |
||||||
|
} |
||||||
|
wire, tag := x&7, x>>3 |
||||||
|
if wire == WireEndGroup { |
||||||
|
w.unindent() |
||||||
|
if _, err := w.Write(endBraceNewline); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
if _, err := fmt.Fprint(w, tag); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if wire != WireStartGroup { |
||||||
|
if err := w.WriteByte(':'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
if !w.compact || wire == WireStartGroup { |
||||||
|
if err := w.WriteByte(' '); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
switch wire { |
||||||
|
case WireBytes: |
||||||
|
buf, e := b.DecodeRawBytes(false) |
||||||
|
if e == nil { |
||||||
|
_, err = fmt.Fprintf(w, "%q", buf) |
||||||
|
} else { |
||||||
|
_, err = fmt.Fprintf(w, "/* %v */", e) |
||||||
|
} |
||||||
|
case WireFixed32: |
||||||
|
x, err = b.DecodeFixed32() |
||||||
|
err = writeUnknownInt(w, x, err) |
||||||
|
case WireFixed64: |
||||||
|
x, err = b.DecodeFixed64() |
||||||
|
err = writeUnknownInt(w, x, err) |
||||||
|
case WireStartGroup: |
||||||
|
err = w.WriteByte('{') |
||||||
|
w.indent() |
||||||
|
case WireVarint: |
||||||
|
x, err = b.DecodeVarint() |
||||||
|
err = writeUnknownInt(w, x, err) |
||||||
|
default: |
||||||
|
_, err = fmt.Fprintf(w, "/* unknown wire type %d */", wire) |
||||||
|
} |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err = w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func writeUnknownInt(w *textWriter, x uint64, err error) error { |
||||||
|
if err == nil { |
||||||
|
_, err = fmt.Fprint(w, x) |
||||||
|
} else { |
||||||
|
_, err = fmt.Fprintf(w, "/* %v */", err) |
||||||
|
} |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
type int32Slice []int32 |
||||||
|
|
||||||
|
func (s int32Slice) Len() int { return len(s) } |
||||||
|
func (s int32Slice) Less(i, j int) bool { return s[i] < s[j] } |
||||||
|
func (s int32Slice) Swap(i, j int) { s[i], s[j] = s[j], s[i] } |
||||||
|
|
||||||
|
// writeExtensions writes all the extensions in pv.
|
||||||
|
// pv is assumed to be a pointer to a protocol message struct that is extendable.
|
||||||
|
func (tm *TextMarshaler) writeExtensions(w *textWriter, pv reflect.Value) error { |
||||||
|
emap := extensionMaps[pv.Type().Elem()] |
||||||
|
ep, _ := extendable(pv.Interface()) |
||||||
|
|
||||||
|
// Order the extensions by ID.
|
||||||
|
// This isn't strictly necessary, but it will give us
|
||||||
|
// canonical output, which will also make testing easier.
|
||||||
|
m, mu := ep.extensionsRead() |
||||||
|
if m == nil { |
||||||
|
return nil |
||||||
|
} |
||||||
|
mu.Lock() |
||||||
|
ids := make([]int32, 0, len(m)) |
||||||
|
for id := range m { |
||||||
|
ids = append(ids, id) |
||||||
|
} |
||||||
|
sort.Sort(int32Slice(ids)) |
||||||
|
mu.Unlock() |
||||||
|
|
||||||
|
for _, extNum := range ids { |
||||||
|
ext := m[extNum] |
||||||
|
var desc *ExtensionDesc |
||||||
|
if emap != nil { |
||||||
|
desc = emap[extNum] |
||||||
|
} |
||||||
|
if desc == nil { |
||||||
|
// Unknown extension.
|
||||||
|
if err := writeUnknownStruct(w, ext.enc); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
pb, err := GetExtension(ep, desc) |
||||||
|
if err != nil { |
||||||
|
return fmt.Errorf("failed getting extension: %v", err) |
||||||
|
} |
||||||
|
|
||||||
|
// Repeated extensions will appear as a slice.
|
||||||
|
if !desc.repeated() { |
||||||
|
if err := tm.writeExtension(w, desc.Name, pb); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} else { |
||||||
|
v := reflect.ValueOf(pb) |
||||||
|
for i := 0; i < v.Len(); i++ { |
||||||
|
if err := tm.writeExtension(w, desc.Name, v.Index(i).Interface()); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (tm *TextMarshaler) writeExtension(w *textWriter, name string, pb interface{}) error { |
||||||
|
if _, err := fmt.Fprintf(w, "[%s]:", name); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !w.compact { |
||||||
|
if err := w.WriteByte(' '); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
if err := tm.writeAny(w, reflect.ValueOf(pb), nil); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := w.WriteByte('\n'); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (w *textWriter) writeIndent() { |
||||||
|
if !w.complete { |
||||||
|
return |
||||||
|
} |
||||||
|
remain := w.ind * 2 |
||||||
|
for remain > 0 { |
||||||
|
n := remain |
||||||
|
if n > len(spaces) { |
||||||
|
n = len(spaces) |
||||||
|
} |
||||||
|
w.w.Write(spaces[:n]) |
||||||
|
remain -= n |
||||||
|
} |
||||||
|
w.complete = false |
||||||
|
} |
||||||
|
|
||||||
|
// TextMarshaler is a configurable text format marshaler.
|
||||||
|
type TextMarshaler struct { |
||||||
|
Compact bool // use compact text format (one line).
|
||||||
|
ExpandAny bool // expand google.protobuf.Any messages of known types
|
||||||
|
} |
||||||
|
|
||||||
|
// Marshal writes a given protocol buffer in text format.
|
||||||
|
// The only errors returned are from w.
|
||||||
|
func (tm *TextMarshaler) Marshal(w io.Writer, pb Message) error { |
||||||
|
val := reflect.ValueOf(pb) |
||||||
|
if pb == nil || val.IsNil() { |
||||||
|
w.Write([]byte("<nil>")) |
||||||
|
return nil |
||||||
|
} |
||||||
|
var bw *bufio.Writer |
||||||
|
ww, ok := w.(writer) |
||||||
|
if !ok { |
||||||
|
bw = bufio.NewWriter(w) |
||||||
|
ww = bw |
||||||
|
} |
||||||
|
aw := &textWriter{ |
||||||
|
w: ww, |
||||||
|
complete: true, |
||||||
|
compact: tm.Compact, |
||||||
|
} |
||||||
|
|
||||||
|
if etm, ok := pb.(encoding.TextMarshaler); ok { |
||||||
|
text, err := etm.MarshalText() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if _, err = aw.Write(text); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if bw != nil { |
||||||
|
return bw.Flush() |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
// Dereference the received pointer so we don't have outer < and >.
|
||||||
|
v := reflect.Indirect(val) |
||||||
|
if err := tm.writeStruct(aw, v); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if bw != nil { |
||||||
|
return bw.Flush() |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Text is the same as Marshal, but returns the string directly.
|
||||||
|
func (tm *TextMarshaler) Text(pb Message) string { |
||||||
|
var buf bytes.Buffer |
||||||
|
tm.Marshal(&buf, pb) |
||||||
|
return buf.String() |
||||||
|
} |
||||||
|
|
||||||
|
var ( |
||||||
|
defaultTextMarshaler = TextMarshaler{} |
||||||
|
compactTextMarshaler = TextMarshaler{Compact: true} |
||||||
|
) |
||||||
|
|
||||||
|
// TODO: consider removing some of the Marshal functions below.
|
||||||
|
|
||||||
|
// MarshalText writes a given protocol buffer in text format.
|
||||||
|
// The only errors returned are from w.
|
||||||
|
func MarshalText(w io.Writer, pb Message) error { return defaultTextMarshaler.Marshal(w, pb) } |
||||||
|
|
||||||
|
// MarshalTextString is the same as MarshalText, but returns the string directly.
|
||||||
|
func MarshalTextString(pb Message) string { return defaultTextMarshaler.Text(pb) } |
||||||
|
|
||||||
|
// CompactText writes a given protocol buffer in compact text format (one line).
|
||||||
|
func CompactText(w io.Writer, pb Message) error { return compactTextMarshaler.Marshal(w, pb) } |
||||||
|
|
||||||
|
// CompactTextString is the same as CompactText, but returns the string directly.
|
||||||
|
func CompactTextString(pb Message) string { return compactTextMarshaler.Text(pb) } |
||||||
@ -0,0 +1,895 @@ |
|||||||
|
// Go support for Protocol Buffers - Google's data interchange format
|
||||||
|
//
|
||||||
|
// Copyright 2010 The Go Authors. All rights reserved.
|
||||||
|
// https://github.com/golang/protobuf
|
||||||
|
//
|
||||||
|
// Redistribution and use in source and binary forms, with or without
|
||||||
|
// modification, are permitted provided that the following conditions are
|
||||||
|
// met:
|
||||||
|
//
|
||||||
|
// * Redistributions of source code must retain the above copyright
|
||||||
|
// notice, this list of conditions and the following disclaimer.
|
||||||
|
// * Redistributions in binary form must reproduce the above
|
||||||
|
// copyright notice, this list of conditions and the following disclaimer
|
||||||
|
// in the documentation and/or other materials provided with the
|
||||||
|
// distribution.
|
||||||
|
// * Neither the name of Google Inc. nor the names of its
|
||||||
|
// contributors may be used to endorse or promote products derived from
|
||||||
|
// this software without specific prior written permission.
|
||||||
|
//
|
||||||
|
// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
// "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
// A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
// OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
// SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
// LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
// DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
// THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
// (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
|
||||||
|
package proto |
||||||
|
|
||||||
|
// Functions for parsing the Text protocol buffer format.
|
||||||
|
// TODO: message sets.
|
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding" |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"reflect" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
"unicode/utf8" |
||||||
|
) |
||||||
|
|
||||||
|
// Error string emitted when deserializing Any and fields are already set
|
||||||
|
const anyRepeatedlyUnpacked = "Any message unpacked multiple times, or %q already set" |
||||||
|
|
||||||
|
type ParseError struct { |
||||||
|
Message string |
||||||
|
Line int // 1-based line number
|
||||||
|
Offset int // 0-based byte offset from start of input
|
||||||
|
} |
||||||
|
|
||||||
|
func (p *ParseError) Error() string { |
||||||
|
if p.Line == 1 { |
||||||
|
// show offset only for first line
|
||||||
|
return fmt.Sprintf("line 1.%d: %v", p.Offset, p.Message) |
||||||
|
} |
||||||
|
return fmt.Sprintf("line %d: %v", p.Line, p.Message) |
||||||
|
} |
||||||
|
|
||||||
|
type token struct { |
||||||
|
value string |
||||||
|
err *ParseError |
||||||
|
line int // line number
|
||||||
|
offset int // byte number from start of input, not start of line
|
||||||
|
unquoted string // the unquoted version of value, if it was a quoted string
|
||||||
|
} |
||||||
|
|
||||||
|
func (t *token) String() string { |
||||||
|
if t.err == nil { |
||||||
|
return fmt.Sprintf("%q (line=%d, offset=%d)", t.value, t.line, t.offset) |
||||||
|
} |
||||||
|
return fmt.Sprintf("parse error: %v", t.err) |
||||||
|
} |
||||||
|
|
||||||
|
type textParser struct { |
||||||
|
s string // remaining input
|
||||||
|
done bool // whether the parsing is finished (success or error)
|
||||||
|
backed bool // whether back() was called
|
||||||
|
offset, line int |
||||||
|
cur token |
||||||
|
} |
||||||
|
|
||||||
|
func newTextParser(s string) *textParser { |
||||||
|
p := new(textParser) |
||||||
|
p.s = s |
||||||
|
p.line = 1 |
||||||
|
p.cur.line = 1 |
||||||
|
return p |
||||||
|
} |
||||||
|
|
||||||
|
func (p *textParser) errorf(format string, a ...interface{}) *ParseError { |
||||||
|
pe := &ParseError{fmt.Sprintf(format, a...), p.cur.line, p.cur.offset} |
||||||
|
p.cur.err = pe |
||||||
|
p.done = true |
||||||
|
return pe |
||||||
|
} |
||||||
|
|
||||||
|
// Numbers and identifiers are matched by [-+._A-Za-z0-9]
|
||||||
|
func isIdentOrNumberChar(c byte) bool { |
||||||
|
switch { |
||||||
|
case 'A' <= c && c <= 'Z', 'a' <= c && c <= 'z': |
||||||
|
return true |
||||||
|
case '0' <= c && c <= '9': |
||||||
|
return true |
||||||
|
} |
||||||
|
switch c { |
||||||
|
case '-', '+', '.', '_': |
||||||
|
return true |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
func isWhitespace(c byte) bool { |
||||||
|
switch c { |
||||||
|
case ' ', '\t', '\n', '\r': |
||||||
|
return true |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
func isQuote(c byte) bool { |
||||||
|
switch c { |
||||||
|
case '"', '\'': |
||||||
|
return true |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
func (p *textParser) skipWhitespace() { |
||||||
|
i := 0 |
||||||
|
for i < len(p.s) && (isWhitespace(p.s[i]) || p.s[i] == '#') { |
||||||
|
if p.s[i] == '#' { |
||||||
|
// comment; skip to end of line or input
|
||||||
|
for i < len(p.s) && p.s[i] != '\n' { |
||||||
|
i++ |
||||||
|
} |
||||||
|
if i == len(p.s) { |
||||||
|
break |
||||||
|
} |
||||||
|
} |
||||||
|
if p.s[i] == '\n' { |
||||||
|
p.line++ |
||||||
|
} |
||||||
|
i++ |
||||||
|
} |
||||||
|
p.offset += i |
||||||
|
p.s = p.s[i:len(p.s)] |
||||||
|
if len(p.s) == 0 { |
||||||
|
p.done = true |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (p *textParser) advance() { |
||||||
|
// Skip whitespace
|
||||||
|
p.skipWhitespace() |
||||||
|
if p.done { |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// Start of non-whitespace
|
||||||
|
p.cur.err = nil |
||||||
|
p.cur.offset, p.cur.line = p.offset, p.line |
||||||
|
p.cur.unquoted = "" |
||||||
|
switch p.s[0] { |
||||||
|
case '<', '>', '{', '}', ':', '[', ']', ';', ',', '/': |
||||||
|
// Single symbol
|
||||||
|
p.cur.value, p.s = p.s[0:1], p.s[1:len(p.s)] |
||||||
|
case '"', '\'': |
||||||
|
// Quoted string
|
||||||
|
i := 1 |
||||||
|
for i < len(p.s) && p.s[i] != p.s[0] && p.s[i] != '\n' { |
||||||
|
if p.s[i] == '\\' && i+1 < len(p.s) { |
||||||
|
// skip escaped char
|
||||||
|
i++ |
||||||
|
} |
||||||
|
i++ |
||||||
|
} |
||||||
|
if i >= len(p.s) || p.s[i] != p.s[0] { |
||||||
|
p.errorf("unmatched quote") |
||||||
|
return |
||||||
|
} |
||||||
|
unq, err := unquoteC(p.s[1:i], rune(p.s[0])) |
||||||
|
if err != nil { |
||||||
|
p.errorf("invalid quoted string %s: %v", p.s[0:i+1], err) |
||||||
|
return |
||||||
|
} |
||||||
|
p.cur.value, p.s = p.s[0:i+1], p.s[i+1:len(p.s)] |
||||||
|
p.cur.unquoted = unq |
||||||
|
default: |
||||||
|
i := 0 |
||||||
|
for i < len(p.s) && isIdentOrNumberChar(p.s[i]) { |
||||||
|
i++ |
||||||
|
} |
||||||
|
if i == 0 { |
||||||
|
p.errorf("unexpected byte %#x", p.s[0]) |
||||||
|
return |
||||||
|
} |
||||||
|
p.cur.value, p.s = p.s[0:i], p.s[i:len(p.s)] |
||||||
|
} |
||||||
|
p.offset += len(p.cur.value) |
||||||
|
} |
||||||
|
|
||||||
|
var ( |
||||||
|
errBadUTF8 = errors.New("proto: bad UTF-8") |
||||||
|
errBadHex = errors.New("proto: bad hexadecimal") |
||||||
|
) |
||||||
|
|
||||||
|
func unquoteC(s string, quote rune) (string, error) { |
||||||
|
// This is based on C++'s tokenizer.cc.
|
||||||
|
// Despite its name, this is *not* parsing C syntax.
|
||||||
|
// For instance, "\0" is an invalid quoted string.
|
||||||
|
|
||||||
|
// Avoid allocation in trivial cases.
|
||||||
|
simple := true |
||||||
|
for _, r := range s { |
||||||
|
if r == '\\' || r == quote { |
||||||
|
simple = false |
||||||
|
break |
||||||
|
} |
||||||
|
} |
||||||
|
if simple { |
||||||
|
return s, nil |
||||||
|
} |
||||||
|
|
||||||
|
buf := make([]byte, 0, 3*len(s)/2) |
||||||
|
for len(s) > 0 { |
||||||
|
r, n := utf8.DecodeRuneInString(s) |
||||||
|
if r == utf8.RuneError && n == 1 { |
||||||
|
return "", errBadUTF8 |
||||||
|
} |
||||||
|
s = s[n:] |
||||||
|
if r != '\\' { |
||||||
|
if r < utf8.RuneSelf { |
||||||
|
buf = append(buf, byte(r)) |
||||||
|
} else { |
||||||
|
buf = append(buf, string(r)...) |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
ch, tail, err := unescape(s) |
||||||
|
if err != nil { |
||||||
|
return "", err |
||||||
|
} |
||||||
|
buf = append(buf, ch...) |
||||||
|
s = tail |
||||||
|
} |
||||||
|
return string(buf), nil |
||||||
|
} |
||||||
|
|
||||||
|
func unescape(s string) (ch string, tail string, err error) { |
||||||
|
r, n := utf8.DecodeRuneInString(s) |
||||||
|
if r == utf8.RuneError && n == 1 { |
||||||
|
return "", "", errBadUTF8 |
||||||
|
} |
||||||
|
s = s[n:] |
||||||
|
switch r { |
||||||
|
case 'a': |
||||||
|
return "\a", s, nil |
||||||
|
case 'b': |
||||||
|
return "\b", s, nil |
||||||
|
case 'f': |
||||||
|
return "\f", s, nil |
||||||
|
case 'n': |
||||||
|
return "\n", s, nil |
||||||
|
case 'r': |
||||||
|
return "\r", s, nil |
||||||
|
case 't': |
||||||
|
return "\t", s, nil |
||||||
|
case 'v': |
||||||
|
return "\v", s, nil |
||||||
|
case '?': |
||||||
|
return "?", s, nil // trigraph workaround
|
||||||
|
case '\'', '"', '\\': |
||||||
|
return string(r), s, nil |
||||||
|
case '0', '1', '2', '3', '4', '5', '6', '7', 'x', 'X': |
||||||
|
if len(s) < 2 { |
||||||
|
return "", "", fmt.Errorf(`\%c requires 2 following digits`, r) |
||||||
|
} |
||||||
|
base := 8 |
||||||
|
ss := s[:2] |
||||||
|
s = s[2:] |
||||||
|
if r == 'x' || r == 'X' { |
||||||
|
base = 16 |
||||||
|
} else { |
||||||
|
ss = string(r) + ss |
||||||
|
} |
||||||
|
i, err := strconv.ParseUint(ss, base, 8) |
||||||
|
if err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
return string([]byte{byte(i)}), s, nil |
||||||
|
case 'u', 'U': |
||||||
|
n := 4 |
||||||
|
if r == 'U' { |
||||||
|
n = 8 |
||||||
|
} |
||||||
|
if len(s) < n { |
||||||
|
return "", "", fmt.Errorf(`\%c requires %d digits`, r, n) |
||||||
|
} |
||||||
|
|
||||||
|
bs := make([]byte, n/2) |
||||||
|
for i := 0; i < n; i += 2 { |
||||||
|
a, ok1 := unhex(s[i]) |
||||||
|
b, ok2 := unhex(s[i+1]) |
||||||
|
if !ok1 || !ok2 { |
||||||
|
return "", "", errBadHex |
||||||
|
} |
||||||
|
bs[i/2] = a<<4 | b |
||||||
|
} |
||||||
|
s = s[n:] |
||||||
|
return string(bs), s, nil |
||||||
|
} |
||||||
|
return "", "", fmt.Errorf(`unknown escape \%c`, r) |
||||||
|
} |
||||||
|
|
||||||
|
// Adapted from src/pkg/strconv/quote.go.
|
||||||
|
func unhex(b byte) (v byte, ok bool) { |
||||||
|
switch { |
||||||
|
case '0' <= b && b <= '9': |
||||||
|
return b - '0', true |
||||||
|
case 'a' <= b && b <= 'f': |
||||||
|
return b - 'a' + 10, true |
||||||
|
case 'A' <= b && b <= 'F': |
||||||
|
return b - 'A' + 10, true |
||||||
|
} |
||||||
|
return 0, false |
||||||
|
} |
||||||
|
|
||||||
|
// Back off the parser by one token. Can only be done between calls to next().
|
||||||
|
// It makes the next advance() a no-op.
|
||||||
|
func (p *textParser) back() { p.backed = true } |
||||||
|
|
||||||
|
// Advances the parser and returns the new current token.
|
||||||
|
func (p *textParser) next() *token { |
||||||
|
if p.backed || p.done { |
||||||
|
p.backed = false |
||||||
|
return &p.cur |
||||||
|
} |
||||||
|
p.advance() |
||||||
|
if p.done { |
||||||
|
p.cur.value = "" |
||||||
|
} else if len(p.cur.value) > 0 && isQuote(p.cur.value[0]) { |
||||||
|
// Look for multiple quoted strings separated by whitespace,
|
||||||
|
// and concatenate them.
|
||||||
|
cat := p.cur |
||||||
|
for { |
||||||
|
p.skipWhitespace() |
||||||
|
if p.done || !isQuote(p.s[0]) { |
||||||
|
break |
||||||
|
} |
||||||
|
p.advance() |
||||||
|
if p.cur.err != nil { |
||||||
|
return &p.cur |
||||||
|
} |
||||||
|
cat.value += " " + p.cur.value |
||||||
|
cat.unquoted += p.cur.unquoted |
||||||
|
} |
||||||
|
p.done = false // parser may have seen EOF, but we want to return cat
|
||||||
|
p.cur = cat |
||||||
|
} |
||||||
|
return &p.cur |
||||||
|
} |
||||||
|
|
||||||
|
func (p *textParser) consumeToken(s string) error { |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
if tok.value != s { |
||||||
|
p.back() |
||||||
|
return p.errorf("expected %q, found %q", s, tok.value) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Return a RequiredNotSetError indicating which required field was not set.
|
||||||
|
func (p *textParser) missingRequiredFieldError(sv reflect.Value) *RequiredNotSetError { |
||||||
|
st := sv.Type() |
||||||
|
sprops := GetProperties(st) |
||||||
|
for i := 0; i < st.NumField(); i++ { |
||||||
|
if !isNil(sv.Field(i)) { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
props := sprops.Prop[i] |
||||||
|
if props.Required { |
||||||
|
return &RequiredNotSetError{fmt.Sprintf("%v.%v", st, props.OrigName)} |
||||||
|
} |
||||||
|
} |
||||||
|
return &RequiredNotSetError{fmt.Sprintf("%v.<unknown field name>", st)} // should not happen
|
||||||
|
} |
||||||
|
|
||||||
|
// Returns the index in the struct for the named field, as well as the parsed tag properties.
|
||||||
|
func structFieldByName(sprops *StructProperties, name string) (int, *Properties, bool) { |
||||||
|
i, ok := sprops.decoderOrigNames[name] |
||||||
|
if ok { |
||||||
|
return i, sprops.Prop[i], true |
||||||
|
} |
||||||
|
return -1, nil, false |
||||||
|
} |
||||||
|
|
||||||
|
// Consume a ':' from the input stream (if the next token is a colon),
|
||||||
|
// returning an error if a colon is needed but not present.
|
||||||
|
func (p *textParser) checkForColon(props *Properties, typ reflect.Type) *ParseError { |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
if tok.value != ":" { |
||||||
|
// Colon is optional when the field is a group or message.
|
||||||
|
needColon := true |
||||||
|
switch props.Wire { |
||||||
|
case "group": |
||||||
|
needColon = false |
||||||
|
case "bytes": |
||||||
|
// A "bytes" field is either a message, a string, or a repeated field;
|
||||||
|
// those three become *T, *string and []T respectively, so we can check for
|
||||||
|
// this field being a pointer to a non-string.
|
||||||
|
if typ.Kind() == reflect.Ptr { |
||||||
|
// *T or *string
|
||||||
|
if typ.Elem().Kind() == reflect.String { |
||||||
|
break |
||||||
|
} |
||||||
|
} else if typ.Kind() == reflect.Slice { |
||||||
|
// []T or []*T
|
||||||
|
if typ.Elem().Kind() != reflect.Ptr { |
||||||
|
break |
||||||
|
} |
||||||
|
} else if typ.Kind() == reflect.String { |
||||||
|
// The proto3 exception is for a string field,
|
||||||
|
// which requires a colon.
|
||||||
|
break |
||||||
|
} |
||||||
|
needColon = false |
||||||
|
} |
||||||
|
if needColon { |
||||||
|
return p.errorf("expected ':', found %q", tok.value) |
||||||
|
} |
||||||
|
p.back() |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (p *textParser) readStruct(sv reflect.Value, terminator string) error { |
||||||
|
st := sv.Type() |
||||||
|
sprops := GetProperties(st) |
||||||
|
reqCount := sprops.reqCount |
||||||
|
var reqFieldErr error |
||||||
|
fieldSet := make(map[string]bool) |
||||||
|
// A struct is a sequence of "name: value", terminated by one of
|
||||||
|
// '>' or '}', or the end of the input. A name may also be
|
||||||
|
// "[extension]" or "[type/url]".
|
||||||
|
//
|
||||||
|
// The whole struct can also be an expanded Any message, like:
|
||||||
|
// [type/url] < ... struct contents ... >
|
||||||
|
for { |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
if tok.value == terminator { |
||||||
|
break |
||||||
|
} |
||||||
|
if tok.value == "[" { |
||||||
|
// Looks like an extension or an Any.
|
||||||
|
//
|
||||||
|
// TODO: Check whether we need to handle
|
||||||
|
// namespace rooted names (e.g. ".something.Foo").
|
||||||
|
extName, err := p.consumeExtName() |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if s := strings.LastIndex(extName, "/"); s >= 0 { |
||||||
|
// If it contains a slash, it's an Any type URL.
|
||||||
|
messageName := extName[s+1:] |
||||||
|
mt := MessageType(messageName) |
||||||
|
if mt == nil { |
||||||
|
return p.errorf("unrecognized message %q in google.protobuf.Any", messageName) |
||||||
|
} |
||||||
|
tok = p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
// consume an optional colon
|
||||||
|
if tok.value == ":" { |
||||||
|
tok = p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
} |
||||||
|
var terminator string |
||||||
|
switch tok.value { |
||||||
|
case "<": |
||||||
|
terminator = ">" |
||||||
|
case "{": |
||||||
|
terminator = "}" |
||||||
|
default: |
||||||
|
return p.errorf("expected '{' or '<', found %q", tok.value) |
||||||
|
} |
||||||
|
v := reflect.New(mt.Elem()) |
||||||
|
if pe := p.readStruct(v.Elem(), terminator); pe != nil { |
||||||
|
return pe |
||||||
|
} |
||||||
|
b, err := Marshal(v.Interface().(Message)) |
||||||
|
if err != nil { |
||||||
|
return p.errorf("failed to marshal message of type %q: %v", messageName, err) |
||||||
|
} |
||||||
|
if fieldSet["type_url"] { |
||||||
|
return p.errorf(anyRepeatedlyUnpacked, "type_url") |
||||||
|
} |
||||||
|
if fieldSet["value"] { |
||||||
|
return p.errorf(anyRepeatedlyUnpacked, "value") |
||||||
|
} |
||||||
|
sv.FieldByName("TypeUrl").SetString(extName) |
||||||
|
sv.FieldByName("Value").SetBytes(b) |
||||||
|
fieldSet["type_url"] = true |
||||||
|
fieldSet["value"] = true |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
var desc *ExtensionDesc |
||||||
|
// This could be faster, but it's functional.
|
||||||
|
// TODO: Do something smarter than a linear scan.
|
||||||
|
for _, d := range RegisteredExtensions(reflect.New(st).Interface().(Message)) { |
||||||
|
if d.Name == extName { |
||||||
|
desc = d |
||||||
|
break |
||||||
|
} |
||||||
|
} |
||||||
|
if desc == nil { |
||||||
|
return p.errorf("unrecognized extension %q", extName) |
||||||
|
} |
||||||
|
|
||||||
|
props := &Properties{} |
||||||
|
props.Parse(desc.Tag) |
||||||
|
|
||||||
|
typ := reflect.TypeOf(desc.ExtensionType) |
||||||
|
if err := p.checkForColon(props, typ); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
rep := desc.repeated() |
||||||
|
|
||||||
|
// Read the extension structure, and set it in
|
||||||
|
// the value we're constructing.
|
||||||
|
var ext reflect.Value |
||||||
|
if !rep { |
||||||
|
ext = reflect.New(typ).Elem() |
||||||
|
} else { |
||||||
|
ext = reflect.New(typ.Elem()).Elem() |
||||||
|
} |
||||||
|
if err := p.readAny(ext, props); err != nil { |
||||||
|
if _, ok := err.(*RequiredNotSetError); !ok { |
||||||
|
return err |
||||||
|
} |
||||||
|
reqFieldErr = err |
||||||
|
} |
||||||
|
ep := sv.Addr().Interface().(Message) |
||||||
|
if !rep { |
||||||
|
SetExtension(ep, desc, ext.Interface()) |
||||||
|
} else { |
||||||
|
old, err := GetExtension(ep, desc) |
||||||
|
var sl reflect.Value |
||||||
|
if err == nil { |
||||||
|
sl = reflect.ValueOf(old) // existing slice
|
||||||
|
} else { |
||||||
|
sl = reflect.MakeSlice(typ, 0, 1) |
||||||
|
} |
||||||
|
sl = reflect.Append(sl, ext) |
||||||
|
SetExtension(ep, desc, sl.Interface()) |
||||||
|
} |
||||||
|
if err := p.consumeOptionalSeparator(); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
// This is a normal, non-extension field.
|
||||||
|
name := tok.value |
||||||
|
var dst reflect.Value |
||||||
|
fi, props, ok := structFieldByName(sprops, name) |
||||||
|
if ok { |
||||||
|
dst = sv.Field(fi) |
||||||
|
} else if oop, ok := sprops.OneofTypes[name]; ok { |
||||||
|
// It is a oneof.
|
||||||
|
props = oop.Prop |
||||||
|
nv := reflect.New(oop.Type.Elem()) |
||||||
|
dst = nv.Elem().Field(0) |
||||||
|
field := sv.Field(oop.Field) |
||||||
|
if !field.IsNil() { |
||||||
|
return p.errorf("field '%s' would overwrite already parsed oneof '%s'", name, sv.Type().Field(oop.Field).Name) |
||||||
|
} |
||||||
|
field.Set(nv) |
||||||
|
} |
||||||
|
if !dst.IsValid() { |
||||||
|
return p.errorf("unknown field name %q in %v", name, st) |
||||||
|
} |
||||||
|
|
||||||
|
if dst.Kind() == reflect.Map { |
||||||
|
// Consume any colon.
|
||||||
|
if err := p.checkForColon(props, dst.Type()); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// Construct the map if it doesn't already exist.
|
||||||
|
if dst.IsNil() { |
||||||
|
dst.Set(reflect.MakeMap(dst.Type())) |
||||||
|
} |
||||||
|
key := reflect.New(dst.Type().Key()).Elem() |
||||||
|
val := reflect.New(dst.Type().Elem()).Elem() |
||||||
|
|
||||||
|
// The map entry should be this sequence of tokens:
|
||||||
|
// < key : KEY value : VALUE >
|
||||||
|
// However, implementations may omit key or value, and technically
|
||||||
|
// we should support them in any order. See b/28924776 for a time
|
||||||
|
// this went wrong.
|
||||||
|
|
||||||
|
tok := p.next() |
||||||
|
var terminator string |
||||||
|
switch tok.value { |
||||||
|
case "<": |
||||||
|
terminator = ">" |
||||||
|
case "{": |
||||||
|
terminator = "}" |
||||||
|
default: |
||||||
|
return p.errorf("expected '{' or '<', found %q", tok.value) |
||||||
|
} |
||||||
|
for { |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
if tok.value == terminator { |
||||||
|
break |
||||||
|
} |
||||||
|
switch tok.value { |
||||||
|
case "key": |
||||||
|
if err := p.consumeToken(":"); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := p.readAny(key, props.mkeyprop); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := p.consumeOptionalSeparator(); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
case "value": |
||||||
|
if err := p.checkForColon(props.mvalprop, dst.Type().Elem()); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := p.readAny(val, props.mvalprop); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := p.consumeOptionalSeparator(); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
default: |
||||||
|
p.back() |
||||||
|
return p.errorf(`expected "key", "value", or %q, found %q`, terminator, tok.value) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
dst.SetMapIndex(key, val) |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
// Check that it's not already set if it's not a repeated field.
|
||||||
|
if !props.Repeated && fieldSet[name] { |
||||||
|
return p.errorf("non-repeated field %q was repeated", name) |
||||||
|
} |
||||||
|
|
||||||
|
if err := p.checkForColon(props, dst.Type()); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// Parse into the field.
|
||||||
|
fieldSet[name] = true |
||||||
|
if err := p.readAny(dst, props); err != nil { |
||||||
|
if _, ok := err.(*RequiredNotSetError); !ok { |
||||||
|
return err |
||||||
|
} |
||||||
|
reqFieldErr = err |
||||||
|
} |
||||||
|
if props.Required { |
||||||
|
reqCount-- |
||||||
|
} |
||||||
|
|
||||||
|
if err := p.consumeOptionalSeparator(); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
} |
||||||
|
|
||||||
|
if reqCount > 0 { |
||||||
|
return p.missingRequiredFieldError(sv) |
||||||
|
} |
||||||
|
return reqFieldErr |
||||||
|
} |
||||||
|
|
||||||
|
// consumeExtName consumes extension name or expanded Any type URL and the
|
||||||
|
// following ']'. It returns the name or URL consumed.
|
||||||
|
func (p *textParser) consumeExtName() (string, error) { |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return "", tok.err |
||||||
|
} |
||||||
|
|
||||||
|
// If extension name or type url is quoted, it's a single token.
|
||||||
|
if len(tok.value) > 2 && isQuote(tok.value[0]) && tok.value[len(tok.value)-1] == tok.value[0] { |
||||||
|
name, err := unquoteC(tok.value[1:len(tok.value)-1], rune(tok.value[0])) |
||||||
|
if err != nil { |
||||||
|
return "", err |
||||||
|
} |
||||||
|
return name, p.consumeToken("]") |
||||||
|
} |
||||||
|
|
||||||
|
// Consume everything up to "]"
|
||||||
|
var parts []string |
||||||
|
for tok.value != "]" { |
||||||
|
parts = append(parts, tok.value) |
||||||
|
tok = p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return "", p.errorf("unrecognized type_url or extension name: %s", tok.err) |
||||||
|
} |
||||||
|
} |
||||||
|
return strings.Join(parts, ""), nil |
||||||
|
} |
||||||
|
|
||||||
|
// consumeOptionalSeparator consumes an optional semicolon or comma.
|
||||||
|
// It is used in readStruct to provide backward compatibility.
|
||||||
|
func (p *textParser) consumeOptionalSeparator() error { |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
if tok.value != ";" && tok.value != "," { |
||||||
|
p.back() |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (p *textParser) readAny(v reflect.Value, props *Properties) error { |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
if tok.value == "" { |
||||||
|
return p.errorf("unexpected EOF") |
||||||
|
} |
||||||
|
|
||||||
|
switch fv := v; fv.Kind() { |
||||||
|
case reflect.Slice: |
||||||
|
at := v.Type() |
||||||
|
if at.Elem().Kind() == reflect.Uint8 { |
||||||
|
// Special case for []byte
|
||||||
|
if tok.value[0] != '"' && tok.value[0] != '\'' { |
||||||
|
// Deliberately written out here, as the error after
|
||||||
|
// this switch statement would write "invalid []byte: ...",
|
||||||
|
// which is not as user-friendly.
|
||||||
|
return p.errorf("invalid string: %v", tok.value) |
||||||
|
} |
||||||
|
bytes := []byte(tok.unquoted) |
||||||
|
fv.Set(reflect.ValueOf(bytes)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
// Repeated field.
|
||||||
|
if tok.value == "[" { |
||||||
|
// Repeated field with list notation, like [1,2,3].
|
||||||
|
for { |
||||||
|
fv.Set(reflect.Append(fv, reflect.New(at.Elem()).Elem())) |
||||||
|
err := p.readAny(fv.Index(fv.Len()-1), props) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
tok := p.next() |
||||||
|
if tok.err != nil { |
||||||
|
return tok.err |
||||||
|
} |
||||||
|
if tok.value == "]" { |
||||||
|
break |
||||||
|
} |
||||||
|
if tok.value != "," { |
||||||
|
return p.errorf("Expected ']' or ',' found %q", tok.value) |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
// One value of the repeated field.
|
||||||
|
p.back() |
||||||
|
fv.Set(reflect.Append(fv, reflect.New(at.Elem()).Elem())) |
||||||
|
return p.readAny(fv.Index(fv.Len()-1), props) |
||||||
|
case reflect.Bool: |
||||||
|
// true/1/t/True or false/f/0/False.
|
||||||
|
switch tok.value { |
||||||
|
case "true", "1", "t", "True": |
||||||
|
fv.SetBool(true) |
||||||
|
return nil |
||||||
|
case "false", "0", "f", "False": |
||||||
|
fv.SetBool(false) |
||||||
|
return nil |
||||||
|
} |
||||||
|
case reflect.Float32, reflect.Float64: |
||||||
|
v := tok.value |
||||||
|
// Ignore 'f' for compatibility with output generated by C++, but don't
|
||||||
|
// remove 'f' when the value is "-inf" or "inf".
|
||||||
|
if strings.HasSuffix(v, "f") && tok.value != "-inf" && tok.value != "inf" { |
||||||
|
v = v[:len(v)-1] |
||||||
|
} |
||||||
|
if f, err := strconv.ParseFloat(v, fv.Type().Bits()); err == nil { |
||||||
|
fv.SetFloat(f) |
||||||
|
return nil |
||||||
|
} |
||||||
|
case reflect.Int32: |
||||||
|
if x, err := strconv.ParseInt(tok.value, 0, 32); err == nil { |
||||||
|
fv.SetInt(x) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
if len(props.Enum) == 0 { |
||||||
|
break |
||||||
|
} |
||||||
|
m, ok := enumValueMaps[props.Enum] |
||||||
|
if !ok { |
||||||
|
break |
||||||
|
} |
||||||
|
x, ok := m[tok.value] |
||||||
|
if !ok { |
||||||
|
break |
||||||
|
} |
||||||
|
fv.SetInt(int64(x)) |
||||||
|
return nil |
||||||
|
case reflect.Int64: |
||||||
|
if x, err := strconv.ParseInt(tok.value, 0, 64); err == nil { |
||||||
|
fv.SetInt(x) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
case reflect.Ptr: |
||||||
|
// A basic field (indirected through pointer), or a repeated message/group
|
||||||
|
p.back() |
||||||
|
fv.Set(reflect.New(fv.Type().Elem())) |
||||||
|
return p.readAny(fv.Elem(), props) |
||||||
|
case reflect.String: |
||||||
|
if tok.value[0] == '"' || tok.value[0] == '\'' { |
||||||
|
fv.SetString(tok.unquoted) |
||||||
|
return nil |
||||||
|
} |
||||||
|
case reflect.Struct: |
||||||
|
var terminator string |
||||||
|
switch tok.value { |
||||||
|
case "{": |
||||||
|
terminator = "}" |
||||||
|
case "<": |
||||||
|
terminator = ">" |
||||||
|
default: |
||||||
|
return p.errorf("expected '{' or '<', found %q", tok.value) |
||||||
|
} |
||||||
|
// TODO: Handle nested messages which implement encoding.TextUnmarshaler.
|
||||||
|
return p.readStruct(fv, terminator) |
||||||
|
case reflect.Uint32: |
||||||
|
if x, err := strconv.ParseUint(tok.value, 0, 32); err == nil { |
||||||
|
fv.SetUint(uint64(x)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
case reflect.Uint64: |
||||||
|
if x, err := strconv.ParseUint(tok.value, 0, 64); err == nil { |
||||||
|
fv.SetUint(x) |
||||||
|
return nil |
||||||
|
} |
||||||
|
} |
||||||
|
return p.errorf("invalid %v: %v", v.Type(), tok.value) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalText reads a protocol buffer in Text format. UnmarshalText resets pb
|
||||||
|
// before starting to unmarshal, so any existing data in pb is always removed.
|
||||||
|
// If a required field is not set and no other error occurs,
|
||||||
|
// UnmarshalText returns *RequiredNotSetError.
|
||||||
|
func UnmarshalText(s string, pb Message) error { |
||||||
|
if um, ok := pb.(encoding.TextUnmarshaler); ok { |
||||||
|
err := um.UnmarshalText([]byte(s)) |
||||||
|
return err |
||||||
|
} |
||||||
|
pb.Reset() |
||||||
|
v := reflect.ValueOf(pb) |
||||||
|
if pe := newTextParser(s).readStruct(v.Elem(), ""); pe != nil { |
||||||
|
return pe |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
@ -0,0 +1,201 @@ |
|||||||
|
Apache License |
||||||
|
Version 2.0, January 2004 |
||||||
|
http://www.apache.org/licenses/ |
||||||
|
|
||||||
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION |
||||||
|
|
||||||
|
1. Definitions. |
||||||
|
|
||||||
|
"License" shall mean the terms and conditions for use, reproduction, |
||||||
|
and distribution as defined by Sections 1 through 9 of this document. |
||||||
|
|
||||||
|
"Licensor" shall mean the copyright owner or entity authorized by |
||||||
|
the copyright owner that is granting the License. |
||||||
|
|
||||||
|
"Legal Entity" shall mean the union of the acting entity and all |
||||||
|
other entities that control, are controlled by, or are under common |
||||||
|
control with that entity. For the purposes of this definition, |
||||||
|
"control" means (i) the power, direct or indirect, to cause the |
||||||
|
direction or management of such entity, whether by contract or |
||||||
|
otherwise, or (ii) ownership of fifty percent (50%) or more of the |
||||||
|
outstanding shares, or (iii) beneficial ownership of such entity. |
||||||
|
|
||||||
|
"You" (or "Your") shall mean an individual or Legal Entity |
||||||
|
exercising permissions granted by this License. |
||||||
|
|
||||||
|
"Source" form shall mean the preferred form for making modifications, |
||||||
|
including but not limited to software source code, documentation |
||||||
|
source, and configuration files. |
||||||
|
|
||||||
|
"Object" form shall mean any form resulting from mechanical |
||||||
|
transformation or translation of a Source form, including but |
||||||
|
not limited to compiled object code, generated documentation, |
||||||
|
and conversions to other media types. |
||||||
|
|
||||||
|
"Work" shall mean the work of authorship, whether in Source or |
||||||
|
Object form, made available under the License, as indicated by a |
||||||
|
copyright notice that is included in or attached to the work |
||||||
|
(an example is provided in the Appendix below). |
||||||
|
|
||||||
|
"Derivative Works" shall mean any work, whether in Source or Object |
||||||
|
form, that is based on (or derived from) the Work and for which the |
||||||
|
editorial revisions, annotations, elaborations, or other modifications |
||||||
|
represent, as a whole, an original work of authorship. For the purposes |
||||||
|
of this License, Derivative Works shall not include works that remain |
||||||
|
separable from, or merely link (or bind by name) to the interfaces of, |
||||||
|
the Work and Derivative Works thereof. |
||||||
|
|
||||||
|
"Contribution" shall mean any work of authorship, including |
||||||
|
the original version of the Work and any modifications or additions |
||||||
|
to that Work or Derivative Works thereof, that is intentionally |
||||||
|
submitted to Licensor for inclusion in the Work by the copyright owner |
||||||
|
or by an individual or Legal Entity authorized to submit on behalf of |
||||||
|
the copyright owner. For the purposes of this definition, "submitted" |
||||||
|
means any form of electronic, verbal, or written communication sent |
||||||
|
to the Licensor or its representatives, including but not limited to |
||||||
|
communication on electronic mailing lists, source code control systems, |
||||||
|
and issue tracking systems that are managed by, or on behalf of, the |
||||||
|
Licensor for the purpose of discussing and improving the Work, but |
||||||
|
excluding communication that is conspicuously marked or otherwise |
||||||
|
designated in writing by the copyright owner as "Not a Contribution." |
||||||
|
|
||||||
|
"Contributor" shall mean Licensor and any individual or Legal Entity |
||||||
|
on behalf of whom a Contribution has been received by Licensor and |
||||||
|
subsequently incorporated within the Work. |
||||||
|
|
||||||
|
2. Grant of Copyright License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
copyright license to reproduce, prepare Derivative Works of, |
||||||
|
publicly display, publicly perform, sublicense, and distribute the |
||||||
|
Work and such Derivative Works in Source or Object form. |
||||||
|
|
||||||
|
3. Grant of Patent License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
(except as stated in this section) patent license to make, have made, |
||||||
|
use, offer to sell, sell, import, and otherwise transfer the Work, |
||||||
|
where such license applies only to those patent claims licensable |
||||||
|
by such Contributor that are necessarily infringed by their |
||||||
|
Contribution(s) alone or by combination of their Contribution(s) |
||||||
|
with the Work to which such Contribution(s) was submitted. If You |
||||||
|
institute patent litigation against any entity (including a |
||||||
|
cross-claim or counterclaim in a lawsuit) alleging that the Work |
||||||
|
or a Contribution incorporated within the Work constitutes direct |
||||||
|
or contributory patent infringement, then any patent licenses |
||||||
|
granted to You under this License for that Work shall terminate |
||||||
|
as of the date such litigation is filed. |
||||||
|
|
||||||
|
4. Redistribution. You may reproduce and distribute copies of the |
||||||
|
Work or Derivative Works thereof in any medium, with or without |
||||||
|
modifications, and in Source or Object form, provided that You |
||||||
|
meet the following conditions: |
||||||
|
|
||||||
|
(a) You must give any other recipients of the Work or |
||||||
|
Derivative Works a copy of this License; and |
||||||
|
|
||||||
|
(b) You must cause any modified files to carry prominent notices |
||||||
|
stating that You changed the files; and |
||||||
|
|
||||||
|
(c) You must retain, in the Source form of any Derivative Works |
||||||
|
that You distribute, all copyright, patent, trademark, and |
||||||
|
attribution notices from the Source form of the Work, |
||||||
|
excluding those notices that do not pertain to any part of |
||||||
|
the Derivative Works; and |
||||||
|
|
||||||
|
(d) If the Work includes a "NOTICE" text file as part of its |
||||||
|
distribution, then any Derivative Works that You distribute must |
||||||
|
include a readable copy of the attribution notices contained |
||||||
|
within such NOTICE file, excluding those notices that do not |
||||||
|
pertain to any part of the Derivative Works, in at least one |
||||||
|
of the following places: within a NOTICE text file distributed |
||||||
|
as part of the Derivative Works; within the Source form or |
||||||
|
documentation, if provided along with the Derivative Works; or, |
||||||
|
within a display generated by the Derivative Works, if and |
||||||
|
wherever such third-party notices normally appear. The contents |
||||||
|
of the NOTICE file are for informational purposes only and |
||||||
|
do not modify the License. You may add Your own attribution |
||||||
|
notices within Derivative Works that You distribute, alongside |
||||||
|
or as an addendum to the NOTICE text from the Work, provided |
||||||
|
that such additional attribution notices cannot be construed |
||||||
|
as modifying the License. |
||||||
|
|
||||||
|
You may add Your own copyright statement to Your modifications and |
||||||
|
may provide additional or different license terms and conditions |
||||||
|
for use, reproduction, or distribution of Your modifications, or |
||||||
|
for any such Derivative Works as a whole, provided Your use, |
||||||
|
reproduction, and distribution of the Work otherwise complies with |
||||||
|
the conditions stated in this License. |
||||||
|
|
||||||
|
5. Submission of Contributions. Unless You explicitly state otherwise, |
||||||
|
any Contribution intentionally submitted for inclusion in the Work |
||||||
|
by You to the Licensor shall be under the terms and conditions of |
||||||
|
this License, without any additional terms or conditions. |
||||||
|
Notwithstanding the above, nothing herein shall supersede or modify |
||||||
|
the terms of any separate license agreement you may have executed |
||||||
|
with Licensor regarding such Contributions. |
||||||
|
|
||||||
|
6. Trademarks. This License does not grant permission to use the trade |
||||||
|
names, trademarks, service marks, or product names of the Licensor, |
||||||
|
except as required for reasonable and customary use in describing the |
||||||
|
origin of the Work and reproducing the content of the NOTICE file. |
||||||
|
|
||||||
|
7. Disclaimer of Warranty. Unless required by applicable law or |
||||||
|
agreed to in writing, Licensor provides the Work (and each |
||||||
|
Contributor provides its Contributions) on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or |
||||||
|
implied, including, without limitation, any warranties or conditions |
||||||
|
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A |
||||||
|
PARTICULAR PURPOSE. You are solely responsible for determining the |
||||||
|
appropriateness of using or redistributing the Work and assume any |
||||||
|
risks associated with Your exercise of permissions under this License. |
||||||
|
|
||||||
|
8. Limitation of Liability. In no event and under no legal theory, |
||||||
|
whether in tort (including negligence), contract, or otherwise, |
||||||
|
unless required by applicable law (such as deliberate and grossly |
||||||
|
negligent acts) or agreed to in writing, shall any Contributor be |
||||||
|
liable to You for damages, including any direct, indirect, special, |
||||||
|
incidental, or consequential damages of any character arising as a |
||||||
|
result of this License or out of the use or inability to use the |
||||||
|
Work (including but not limited to damages for loss of goodwill, |
||||||
|
work stoppage, computer failure or malfunction, or any and all |
||||||
|
other commercial damages or losses), even if such Contributor |
||||||
|
has been advised of the possibility of such damages. |
||||||
|
|
||||||
|
9. Accepting Warranty or Additional Liability. While redistributing |
||||||
|
the Work or Derivative Works thereof, You may choose to offer, |
||||||
|
and charge a fee for, acceptance of support, warranty, indemnity, |
||||||
|
or other liability obligations and/or rights consistent with this |
||||||
|
License. However, in accepting such obligations, You may act only |
||||||
|
on Your own behalf and on Your sole responsibility, not on behalf |
||||||
|
of any other Contributor, and only if You agree to indemnify, |
||||||
|
defend, and hold each Contributor harmless for any liability |
||||||
|
incurred by, or claims asserted against, such Contributor by reason |
||||||
|
of your accepting any such warranty or additional liability. |
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS |
||||||
|
|
||||||
|
APPENDIX: How to apply the Apache License to your work. |
||||||
|
|
||||||
|
To apply the Apache License to your work, attach the following |
||||||
|
boilerplate notice, with the fields enclosed by brackets "{}" |
||||||
|
replaced with your own identifying information. (Don't include |
||||||
|
the brackets!) The text should be enclosed in the appropriate |
||||||
|
comment syntax for the file format. We also recommend that a |
||||||
|
file or class name and description of purpose be included on the |
||||||
|
same "printed page" as the copyright notice for easier |
||||||
|
identification within third-party archives. |
||||||
|
|
||||||
|
Copyright {yyyy} {name of copyright owner} |
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); |
||||||
|
you may not use this file except in compliance with the License. |
||||||
|
You may obtain a copy of the License at |
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0 |
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software |
||||||
|
distributed under the License is distributed on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||||
|
See the License for the specific language governing permissions and |
||||||
|
limitations under the License. |
||||||
@ -0,0 +1 @@ |
|||||||
|
Copyright 2012 Matt T. Proud (matt.proud@gmail.com) |
||||||
@ -0,0 +1,7 @@ |
|||||||
|
all: |
||||||
|
|
||||||
|
cover: |
||||||
|
go test -cover -v -coverprofile=cover.dat ./...
|
||||||
|
go tool cover -func cover.dat
|
||||||
|
|
||||||
|
.PHONY: cover |
||||||
@ -0,0 +1,75 @@ |
|||||||
|
// Copyright 2013 Matt T. Proud
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package pbutil |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/binary" |
||||||
|
"errors" |
||||||
|
"io" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
) |
||||||
|
|
||||||
|
var errInvalidVarint = errors.New("invalid varint32 encountered") |
||||||
|
|
||||||
|
// ReadDelimited decodes a message from the provided length-delimited stream,
|
||||||
|
// where the length is encoded as 32-bit varint prefix to the message body.
|
||||||
|
// It returns the total number of bytes read and any applicable error. This is
|
||||||
|
// roughly equivalent to the companion Java API's
|
||||||
|
// MessageLite#parseDelimitedFrom. As per the reader contract, this function
|
||||||
|
// calls r.Read repeatedly as required until exactly one message including its
|
||||||
|
// prefix is read and decoded (or an error has occurred). The function never
|
||||||
|
// reads more bytes from the stream than required. The function never returns
|
||||||
|
// an error if a message has been read and decoded correctly, even if the end
|
||||||
|
// of the stream has been reached in doing so. In that case, any subsequent
|
||||||
|
// calls return (0, io.EOF).
|
||||||
|
func ReadDelimited(r io.Reader, m proto.Message) (n int, err error) { |
||||||
|
// Per AbstractParser#parsePartialDelimitedFrom with
|
||||||
|
// CodedInputStream#readRawVarint32.
|
||||||
|
var headerBuf [binary.MaxVarintLen32]byte |
||||||
|
var bytesRead, varIntBytes int |
||||||
|
var messageLength uint64 |
||||||
|
for varIntBytes == 0 { // i.e. no varint has been decoded yet.
|
||||||
|
if bytesRead >= len(headerBuf) { |
||||||
|
return bytesRead, errInvalidVarint |
||||||
|
} |
||||||
|
// We have to read byte by byte here to avoid reading more bytes
|
||||||
|
// than required. Each read byte is appended to what we have
|
||||||
|
// read before.
|
||||||
|
newBytesRead, err := r.Read(headerBuf[bytesRead : bytesRead+1]) |
||||||
|
if newBytesRead == 0 { |
||||||
|
if err != nil { |
||||||
|
return bytesRead, err |
||||||
|
} |
||||||
|
// A Reader should not return (0, nil), but if it does,
|
||||||
|
// it should be treated as no-op (according to the
|
||||||
|
// Reader contract). So let's go on...
|
||||||
|
continue |
||||||
|
} |
||||||
|
bytesRead += newBytesRead |
||||||
|
// Now present everything read so far to the varint decoder and
|
||||||
|
// see if a varint can be decoded already.
|
||||||
|
messageLength, varIntBytes = proto.DecodeVarint(headerBuf[:bytesRead]) |
||||||
|
} |
||||||
|
|
||||||
|
messageBuf := make([]byte, messageLength) |
||||||
|
newBytesRead, err := io.ReadFull(r, messageBuf) |
||||||
|
bytesRead += newBytesRead |
||||||
|
if err != nil { |
||||||
|
return bytesRead, err |
||||||
|
} |
||||||
|
|
||||||
|
return bytesRead, proto.Unmarshal(messageBuf, m) |
||||||
|
} |
||||||
@ -0,0 +1,16 @@ |
|||||||
|
// Copyright 2013 Matt T. Proud
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// Package pbutil provides record length-delimited Protocol Buffer streaming.
|
||||||
|
package pbutil |
||||||
@ -0,0 +1,46 @@ |
|||||||
|
// Copyright 2013 Matt T. Proud
|
||||||
|
//
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package pbutil |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/binary" |
||||||
|
"io" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
) |
||||||
|
|
||||||
|
// WriteDelimited encodes and dumps a message to the provided writer prefixed
|
||||||
|
// with a 32-bit varint indicating the length of the encoded message, producing
|
||||||
|
// a length-delimited record stream, which can be used to chain together
|
||||||
|
// encoded messages of the same type together in a file. It returns the total
|
||||||
|
// number of bytes written and any applicable error. This is roughly
|
||||||
|
// equivalent to the companion Java API's MessageLite#writeDelimitedTo.
|
||||||
|
func WriteDelimited(w io.Writer, m proto.Message) (n int, err error) { |
||||||
|
buffer, err := proto.Marshal(m) |
||||||
|
if err != nil { |
||||||
|
return 0, err |
||||||
|
} |
||||||
|
|
||||||
|
var buf [binary.MaxVarintLen32]byte |
||||||
|
encodedLength := binary.PutUvarint(buf[:], uint64(len(buffer))) |
||||||
|
|
||||||
|
sync, err := w.Write(buf[:encodedLength]) |
||||||
|
if err != nil { |
||||||
|
return sync, err |
||||||
|
} |
||||||
|
|
||||||
|
n, err = w.Write(buffer) |
||||||
|
return n + sync, err |
||||||
|
} |
||||||
@ -0,0 +1,201 @@ |
|||||||
|
Apache License |
||||||
|
Version 2.0, January 2004 |
||||||
|
http://www.apache.org/licenses/ |
||||||
|
|
||||||
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION |
||||||
|
|
||||||
|
1. Definitions. |
||||||
|
|
||||||
|
"License" shall mean the terms and conditions for use, reproduction, |
||||||
|
and distribution as defined by Sections 1 through 9 of this document. |
||||||
|
|
||||||
|
"Licensor" shall mean the copyright owner or entity authorized by |
||||||
|
the copyright owner that is granting the License. |
||||||
|
|
||||||
|
"Legal Entity" shall mean the union of the acting entity and all |
||||||
|
other entities that control, are controlled by, or are under common |
||||||
|
control with that entity. For the purposes of this definition, |
||||||
|
"control" means (i) the power, direct or indirect, to cause the |
||||||
|
direction or management of such entity, whether by contract or |
||||||
|
otherwise, or (ii) ownership of fifty percent (50%) or more of the |
||||||
|
outstanding shares, or (iii) beneficial ownership of such entity. |
||||||
|
|
||||||
|
"You" (or "Your") shall mean an individual or Legal Entity |
||||||
|
exercising permissions granted by this License. |
||||||
|
|
||||||
|
"Source" form shall mean the preferred form for making modifications, |
||||||
|
including but not limited to software source code, documentation |
||||||
|
source, and configuration files. |
||||||
|
|
||||||
|
"Object" form shall mean any form resulting from mechanical |
||||||
|
transformation or translation of a Source form, including but |
||||||
|
not limited to compiled object code, generated documentation, |
||||||
|
and conversions to other media types. |
||||||
|
|
||||||
|
"Work" shall mean the work of authorship, whether in Source or |
||||||
|
Object form, made available under the License, as indicated by a |
||||||
|
copyright notice that is included in or attached to the work |
||||||
|
(an example is provided in the Appendix below). |
||||||
|
|
||||||
|
"Derivative Works" shall mean any work, whether in Source or Object |
||||||
|
form, that is based on (or derived from) the Work and for which the |
||||||
|
editorial revisions, annotations, elaborations, or other modifications |
||||||
|
represent, as a whole, an original work of authorship. For the purposes |
||||||
|
of this License, Derivative Works shall not include works that remain |
||||||
|
separable from, or merely link (or bind by name) to the interfaces of, |
||||||
|
the Work and Derivative Works thereof. |
||||||
|
|
||||||
|
"Contribution" shall mean any work of authorship, including |
||||||
|
the original version of the Work and any modifications or additions |
||||||
|
to that Work or Derivative Works thereof, that is intentionally |
||||||
|
submitted to Licensor for inclusion in the Work by the copyright owner |
||||||
|
or by an individual or Legal Entity authorized to submit on behalf of |
||||||
|
the copyright owner. For the purposes of this definition, "submitted" |
||||||
|
means any form of electronic, verbal, or written communication sent |
||||||
|
to the Licensor or its representatives, including but not limited to |
||||||
|
communication on electronic mailing lists, source code control systems, |
||||||
|
and issue tracking systems that are managed by, or on behalf of, the |
||||||
|
Licensor for the purpose of discussing and improving the Work, but |
||||||
|
excluding communication that is conspicuously marked or otherwise |
||||||
|
designated in writing by the copyright owner as "Not a Contribution." |
||||||
|
|
||||||
|
"Contributor" shall mean Licensor and any individual or Legal Entity |
||||||
|
on behalf of whom a Contribution has been received by Licensor and |
||||||
|
subsequently incorporated within the Work. |
||||||
|
|
||||||
|
2. Grant of Copyright License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
copyright license to reproduce, prepare Derivative Works of, |
||||||
|
publicly display, publicly perform, sublicense, and distribute the |
||||||
|
Work and such Derivative Works in Source or Object form. |
||||||
|
|
||||||
|
3. Grant of Patent License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
(except as stated in this section) patent license to make, have made, |
||||||
|
use, offer to sell, sell, import, and otherwise transfer the Work, |
||||||
|
where such license applies only to those patent claims licensable |
||||||
|
by such Contributor that are necessarily infringed by their |
||||||
|
Contribution(s) alone or by combination of their Contribution(s) |
||||||
|
with the Work to which such Contribution(s) was submitted. If You |
||||||
|
institute patent litigation against any entity (including a |
||||||
|
cross-claim or counterclaim in a lawsuit) alleging that the Work |
||||||
|
or a Contribution incorporated within the Work constitutes direct |
||||||
|
or contributory patent infringement, then any patent licenses |
||||||
|
granted to You under this License for that Work shall terminate |
||||||
|
as of the date such litigation is filed. |
||||||
|
|
||||||
|
4. Redistribution. You may reproduce and distribute copies of the |
||||||
|
Work or Derivative Works thereof in any medium, with or without |
||||||
|
modifications, and in Source or Object form, provided that You |
||||||
|
meet the following conditions: |
||||||
|
|
||||||
|
(a) You must give any other recipients of the Work or |
||||||
|
Derivative Works a copy of this License; and |
||||||
|
|
||||||
|
(b) You must cause any modified files to carry prominent notices |
||||||
|
stating that You changed the files; and |
||||||
|
|
||||||
|
(c) You must retain, in the Source form of any Derivative Works |
||||||
|
that You distribute, all copyright, patent, trademark, and |
||||||
|
attribution notices from the Source form of the Work, |
||||||
|
excluding those notices that do not pertain to any part of |
||||||
|
the Derivative Works; and |
||||||
|
|
||||||
|
(d) If the Work includes a "NOTICE" text file as part of its |
||||||
|
distribution, then any Derivative Works that You distribute must |
||||||
|
include a readable copy of the attribution notices contained |
||||||
|
within such NOTICE file, excluding those notices that do not |
||||||
|
pertain to any part of the Derivative Works, in at least one |
||||||
|
of the following places: within a NOTICE text file distributed |
||||||
|
as part of the Derivative Works; within the Source form or |
||||||
|
documentation, if provided along with the Derivative Works; or, |
||||||
|
within a display generated by the Derivative Works, if and |
||||||
|
wherever such third-party notices normally appear. The contents |
||||||
|
of the NOTICE file are for informational purposes only and |
||||||
|
do not modify the License. You may add Your own attribution |
||||||
|
notices within Derivative Works that You distribute, alongside |
||||||
|
or as an addendum to the NOTICE text from the Work, provided |
||||||
|
that such additional attribution notices cannot be construed |
||||||
|
as modifying the License. |
||||||
|
|
||||||
|
You may add Your own copyright statement to Your modifications and |
||||||
|
may provide additional or different license terms and conditions |
||||||
|
for use, reproduction, or distribution of Your modifications, or |
||||||
|
for any such Derivative Works as a whole, provided Your use, |
||||||
|
reproduction, and distribution of the Work otherwise complies with |
||||||
|
the conditions stated in this License. |
||||||
|
|
||||||
|
5. Submission of Contributions. Unless You explicitly state otherwise, |
||||||
|
any Contribution intentionally submitted for inclusion in the Work |
||||||
|
by You to the Licensor shall be under the terms and conditions of |
||||||
|
this License, without any additional terms or conditions. |
||||||
|
Notwithstanding the above, nothing herein shall supersede or modify |
||||||
|
the terms of any separate license agreement you may have executed |
||||||
|
with Licensor regarding such Contributions. |
||||||
|
|
||||||
|
6. Trademarks. This License does not grant permission to use the trade |
||||||
|
names, trademarks, service marks, or product names of the Licensor, |
||||||
|
except as required for reasonable and customary use in describing the |
||||||
|
origin of the Work and reproducing the content of the NOTICE file. |
||||||
|
|
||||||
|
7. Disclaimer of Warranty. Unless required by applicable law or |
||||||
|
agreed to in writing, Licensor provides the Work (and each |
||||||
|
Contributor provides its Contributions) on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or |
||||||
|
implied, including, without limitation, any warranties or conditions |
||||||
|
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A |
||||||
|
PARTICULAR PURPOSE. You are solely responsible for determining the |
||||||
|
appropriateness of using or redistributing the Work and assume any |
||||||
|
risks associated with Your exercise of permissions under this License. |
||||||
|
|
||||||
|
8. Limitation of Liability. In no event and under no legal theory, |
||||||
|
whether in tort (including negligence), contract, or otherwise, |
||||||
|
unless required by applicable law (such as deliberate and grossly |
||||||
|
negligent acts) or agreed to in writing, shall any Contributor be |
||||||
|
liable to You for damages, including any direct, indirect, special, |
||||||
|
incidental, or consequential damages of any character arising as a |
||||||
|
result of this License or out of the use or inability to use the |
||||||
|
Work (including but not limited to damages for loss of goodwill, |
||||||
|
work stoppage, computer failure or malfunction, or any and all |
||||||
|
other commercial damages or losses), even if such Contributor |
||||||
|
has been advised of the possibility of such damages. |
||||||
|
|
||||||
|
9. Accepting Warranty or Additional Liability. While redistributing |
||||||
|
the Work or Derivative Works thereof, You may choose to offer, |
||||||
|
and charge a fee for, acceptance of support, warranty, indemnity, |
||||||
|
or other liability obligations and/or rights consistent with this |
||||||
|
License. However, in accepting such obligations, You may act only |
||||||
|
on Your own behalf and on Your sole responsibility, not on behalf |
||||||
|
of any other Contributor, and only if You agree to indemnify, |
||||||
|
defend, and hold each Contributor harmless for any liability |
||||||
|
incurred by, or claims asserted against, such Contributor by reason |
||||||
|
of your accepting any such warranty or additional liability. |
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS |
||||||
|
|
||||||
|
APPENDIX: How to apply the Apache License to your work. |
||||||
|
|
||||||
|
To apply the Apache License to your work, attach the following |
||||||
|
boilerplate notice, with the fields enclosed by brackets "[]" |
||||||
|
replaced with your own identifying information. (Don't include |
||||||
|
the brackets!) The text should be enclosed in the appropriate |
||||||
|
comment syntax for the file format. We also recommend that a |
||||||
|
file or class name and description of purpose be included on the |
||||||
|
same "printed page" as the copyright notice for easier |
||||||
|
identification within third-party archives. |
||||||
|
|
||||||
|
Copyright [yyyy] [name of copyright owner] |
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); |
||||||
|
you may not use this file except in compliance with the License. |
||||||
|
You may obtain a copy of the License at |
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0 |
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software |
||||||
|
distributed under the License is distributed on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||||
|
See the License for the specific language governing permissions and |
||||||
|
limitations under the License. |
||||||
@ -0,0 +1,23 @@ |
|||||||
|
Prometheus instrumentation library for Go applications |
||||||
|
Copyright 2012-2015 The Prometheus Authors |
||||||
|
|
||||||
|
This product includes software developed at |
||||||
|
SoundCloud Ltd. (http://soundcloud.com/). |
||||||
|
|
||||||
|
|
||||||
|
The following components are included in this product: |
||||||
|
|
||||||
|
perks - a fork of https://github.com/bmizerany/perks |
||||||
|
https://github.com/beorn7/perks |
||||||
|
Copyright 2013-2015 Blake Mizerany, Björn Rabenstein |
||||||
|
See https://github.com/beorn7/perks/blob/master/README.md for license details. |
||||||
|
|
||||||
|
Go support for Protocol Buffers - Google's data interchange format |
||||||
|
http://github.com/golang/protobuf/ |
||||||
|
Copyright 2010 The Go Authors |
||||||
|
See source code for license details. |
||||||
|
|
||||||
|
Support for streaming Protocol Buffer messages for the Go language (golang). |
||||||
|
https://github.com/matttproud/golang_protobuf_extensions |
||||||
|
Copyright 2013 Matt T. Proud |
||||||
|
Licensed under the Apache License, Version 2.0 |
||||||
@ -0,0 +1 @@ |
|||||||
|
See [](https://godoc.org/github.com/prometheus/client_golang/prometheus). |
||||||
@ -0,0 +1,75 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
// Collector is the interface implemented by anything that can be used by
|
||||||
|
// Prometheus to collect metrics. A Collector has to be registered for
|
||||||
|
// collection. See Registerer.Register.
|
||||||
|
//
|
||||||
|
// The stock metrics provided by this package (Gauge, Counter, Summary,
|
||||||
|
// Histogram, Untyped) are also Collectors (which only ever collect one metric,
|
||||||
|
// namely itself). An implementer of Collector may, however, collect multiple
|
||||||
|
// metrics in a coordinated fashion and/or create metrics on the fly. Examples
|
||||||
|
// for collectors already implemented in this library are the metric vectors
|
||||||
|
// (i.e. collection of multiple instances of the same Metric but with different
|
||||||
|
// label values) like GaugeVec or SummaryVec, and the ExpvarCollector.
|
||||||
|
type Collector interface { |
||||||
|
// Describe sends the super-set of all possible descriptors of metrics
|
||||||
|
// collected by this Collector to the provided channel and returns once
|
||||||
|
// the last descriptor has been sent. The sent descriptors fulfill the
|
||||||
|
// consistency and uniqueness requirements described in the Desc
|
||||||
|
// documentation. (It is valid if one and the same Collector sends
|
||||||
|
// duplicate descriptors. Those duplicates are simply ignored. However,
|
||||||
|
// two different Collectors must not send duplicate descriptors.) This
|
||||||
|
// method idempotently sends the same descriptors throughout the
|
||||||
|
// lifetime of the Collector. If a Collector encounters an error while
|
||||||
|
// executing this method, it must send an invalid descriptor (created
|
||||||
|
// with NewInvalidDesc) to signal the error to the registry.
|
||||||
|
Describe(chan<- *Desc) |
||||||
|
// Collect is called by the Prometheus registry when collecting
|
||||||
|
// metrics. The implementation sends each collected metric via the
|
||||||
|
// provided channel and returns once the last metric has been sent. The
|
||||||
|
// descriptor of each sent metric is one of those returned by
|
||||||
|
// Describe. Returned metrics that share the same descriptor must differ
|
||||||
|
// in their variable label values. This method may be called
|
||||||
|
// concurrently and must therefore be implemented in a concurrency safe
|
||||||
|
// way. Blocking occurs at the expense of total performance of rendering
|
||||||
|
// all registered metrics. Ideally, Collector implementations support
|
||||||
|
// concurrent readers.
|
||||||
|
Collect(chan<- Metric) |
||||||
|
} |
||||||
|
|
||||||
|
// selfCollector implements Collector for a single Metric so that the Metric
|
||||||
|
// collects itself. Add it as an anonymous field to a struct that implements
|
||||||
|
// Metric, and call init with the Metric itself as an argument.
|
||||||
|
type selfCollector struct { |
||||||
|
self Metric |
||||||
|
} |
||||||
|
|
||||||
|
// init provides the selfCollector with a reference to the metric it is supposed
|
||||||
|
// to collect. It is usually called within the factory function to create a
|
||||||
|
// metric. See example.
|
||||||
|
func (c *selfCollector) init(self Metric) { |
||||||
|
c.self = self |
||||||
|
} |
||||||
|
|
||||||
|
// Describe implements Collector.
|
||||||
|
func (c *selfCollector) Describe(ch chan<- *Desc) { |
||||||
|
ch <- c.self.Desc() |
||||||
|
} |
||||||
|
|
||||||
|
// Collect implements Collector.
|
||||||
|
func (c *selfCollector) Collect(ch chan<- Metric) { |
||||||
|
ch <- c.self |
||||||
|
} |
||||||
@ -0,0 +1,164 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"errors" |
||||||
|
) |
||||||
|
|
||||||
|
// Counter is a Metric that represents a single numerical value that only ever
|
||||||
|
// goes up. That implies that it cannot be used to count items whose number can
|
||||||
|
// also go down, e.g. the number of currently running goroutines. Those
|
||||||
|
// "counters" are represented by Gauges.
|
||||||
|
//
|
||||||
|
// A Counter is typically used to count requests served, tasks completed, errors
|
||||||
|
// occurred, etc.
|
||||||
|
//
|
||||||
|
// To create Counter instances, use NewCounter.
|
||||||
|
type Counter interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
|
||||||
|
// Inc increments the counter by 1. Use Add to increment it by arbitrary
|
||||||
|
// non-negative values.
|
||||||
|
Inc() |
||||||
|
// Add adds the given value to the counter. It panics if the value is <
|
||||||
|
// 0.
|
||||||
|
Add(float64) |
||||||
|
} |
||||||
|
|
||||||
|
// CounterOpts is an alias for Opts. See there for doc comments.
|
||||||
|
type CounterOpts Opts |
||||||
|
|
||||||
|
// NewCounter creates a new Counter based on the provided CounterOpts.
|
||||||
|
func NewCounter(opts CounterOpts) Counter { |
||||||
|
desc := NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
) |
||||||
|
result := &counter{value: value{desc: desc, valType: CounterValue, labelPairs: desc.constLabelPairs}} |
||||||
|
result.init(result) // Init self-collection.
|
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
type counter struct { |
||||||
|
value |
||||||
|
} |
||||||
|
|
||||||
|
func (c *counter) Add(v float64) { |
||||||
|
if v < 0 { |
||||||
|
panic(errors.New("counter cannot decrease in value")) |
||||||
|
} |
||||||
|
c.value.Add(v) |
||||||
|
} |
||||||
|
|
||||||
|
// CounterVec is a Collector that bundles a set of Counters that all share the
|
||||||
|
// same Desc, but have different values for their variable labels. This is used
|
||||||
|
// if you want to count the same thing partitioned by various dimensions
|
||||||
|
// (e.g. number of HTTP requests, partitioned by response code and
|
||||||
|
// method). Create instances with NewCounterVec.
|
||||||
|
//
|
||||||
|
// CounterVec embeds MetricVec. See there for a full list of methods with
|
||||||
|
// detailed documentation.
|
||||||
|
type CounterVec struct { |
||||||
|
*MetricVec |
||||||
|
} |
||||||
|
|
||||||
|
// NewCounterVec creates a new CounterVec based on the provided CounterOpts and
|
||||||
|
// partitioned by the given label names. At least one label name must be
|
||||||
|
// provided.
|
||||||
|
func NewCounterVec(opts CounterOpts, labelNames []string) *CounterVec { |
||||||
|
desc := NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
labelNames, |
||||||
|
opts.ConstLabels, |
||||||
|
) |
||||||
|
return &CounterVec{ |
||||||
|
MetricVec: newMetricVec(desc, func(lvs ...string) Metric { |
||||||
|
result := &counter{value: value{ |
||||||
|
desc: desc, |
||||||
|
valType: CounterValue, |
||||||
|
labelPairs: makeLabelPairs(desc, lvs), |
||||||
|
}} |
||||||
|
result.init(result) // Init self-collection.
|
||||||
|
return result |
||||||
|
}), |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWithLabelValues replaces the method of the same name in
|
||||||
|
// MetricVec. The difference is that this method returns a Counter and not a
|
||||||
|
// Metric so that no type conversion is required.
|
||||||
|
func (m *CounterVec) GetMetricWithLabelValues(lvs ...string) (Counter, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWithLabelValues(lvs...) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Counter), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWith replaces the method of the same name in MetricVec. The
|
||||||
|
// difference is that this method returns a Counter and not a Metric so that no
|
||||||
|
// type conversion is required.
|
||||||
|
func (m *CounterVec) GetMetricWith(labels Labels) (Counter, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWith(labels) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Counter), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||||
|
// GetMetricWithLabelValues would have returned an error. By not returning an
|
||||||
|
// error, WithLabelValues allows shortcuts like
|
||||||
|
// myVec.WithLabelValues("404", "GET").Add(42)
|
||||||
|
func (m *CounterVec) WithLabelValues(lvs ...string) Counter { |
||||||
|
return m.MetricVec.WithLabelValues(lvs...).(Counter) |
||||||
|
} |
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||||
|
// returned an error. By not returning an error, With allows shortcuts like
|
||||||
|
// myVec.With(Labels{"code": "404", "method": "GET"}).Add(42)
|
||||||
|
func (m *CounterVec) With(labels Labels) Counter { |
||||||
|
return m.MetricVec.With(labels).(Counter) |
||||||
|
} |
||||||
|
|
||||||
|
// CounterFunc is a Counter whose value is determined at collect time by calling a
|
||||||
|
// provided function.
|
||||||
|
//
|
||||||
|
// To create CounterFunc instances, use NewCounterFunc.
|
||||||
|
type CounterFunc interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
} |
||||||
|
|
||||||
|
// NewCounterFunc creates a new CounterFunc based on the provided
|
||||||
|
// CounterOpts. The value reported is determined by calling the given function
|
||||||
|
// from within the Write method. Take into account that metric collection may
|
||||||
|
// happen concurrently. If that results in concurrent calls to Write, like in
|
||||||
|
// the case where a CounterFunc is directly registered with Prometheus, the
|
||||||
|
// provided function must be concurrency-safe. The function should also honor
|
||||||
|
// the contract for a Counter (values only go up, not down), but compliance will
|
||||||
|
// not be checked.
|
||||||
|
func NewCounterFunc(opts CounterOpts, function func() float64) CounterFunc { |
||||||
|
return newValueFunc(NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
), CounterValue, function) |
||||||
|
} |
||||||
@ -0,0 +1,200 @@ |
|||||||
|
// Copyright 2016 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"sort" |
||||||
|
"strings" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
"github.com/prometheus/common/model" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
) |
||||||
|
|
||||||
|
// reservedLabelPrefix is a prefix which is not legal in user-supplied
|
||||||
|
// label names.
|
||||||
|
const reservedLabelPrefix = "__" |
||||||
|
|
||||||
|
// Labels represents a collection of label name -> value mappings. This type is
|
||||||
|
// commonly used with the With(Labels) and GetMetricWith(Labels) methods of
|
||||||
|
// metric vector Collectors, e.g.:
|
||||||
|
// myVec.With(Labels{"code": "404", "method": "GET"}).Add(42)
|
||||||
|
//
|
||||||
|
// The other use-case is the specification of constant label pairs in Opts or to
|
||||||
|
// create a Desc.
|
||||||
|
type Labels map[string]string |
||||||
|
|
||||||
|
// Desc is the descriptor used by every Prometheus Metric. It is essentially
|
||||||
|
// the immutable meta-data of a Metric. The normal Metric implementations
|
||||||
|
// included in this package manage their Desc under the hood. Users only have to
|
||||||
|
// deal with Desc if they use advanced features like the ExpvarCollector or
|
||||||
|
// custom Collectors and Metrics.
|
||||||
|
//
|
||||||
|
// Descriptors registered with the same registry have to fulfill certain
|
||||||
|
// consistency and uniqueness criteria if they share the same fully-qualified
|
||||||
|
// name: They must have the same help string and the same label names (aka label
|
||||||
|
// dimensions) in each, constLabels and variableLabels, but they must differ in
|
||||||
|
// the values of the constLabels.
|
||||||
|
//
|
||||||
|
// Descriptors that share the same fully-qualified names and the same label
|
||||||
|
// values of their constLabels are considered equal.
|
||||||
|
//
|
||||||
|
// Use NewDesc to create new Desc instances.
|
||||||
|
type Desc struct { |
||||||
|
// fqName has been built from Namespace, Subsystem, and Name.
|
||||||
|
fqName string |
||||||
|
// help provides some helpful information about this metric.
|
||||||
|
help string |
||||||
|
// constLabelPairs contains precalculated DTO label pairs based on
|
||||||
|
// the constant labels.
|
||||||
|
constLabelPairs []*dto.LabelPair |
||||||
|
// VariableLabels contains names of labels for which the metric
|
||||||
|
// maintains variable values.
|
||||||
|
variableLabels []string |
||||||
|
// id is a hash of the values of the ConstLabels and fqName. This
|
||||||
|
// must be unique among all registered descriptors and can therefore be
|
||||||
|
// used as an identifier of the descriptor.
|
||||||
|
id uint64 |
||||||
|
// dimHash is a hash of the label names (preset and variable) and the
|
||||||
|
// Help string. Each Desc with the same fqName must have the same
|
||||||
|
// dimHash.
|
||||||
|
dimHash uint64 |
||||||
|
// err is an error that occurred during construction. It is reported on
|
||||||
|
// registration time.
|
||||||
|
err error |
||||||
|
} |
||||||
|
|
||||||
|
// NewDesc allocates and initializes a new Desc. Errors are recorded in the Desc
|
||||||
|
// and will be reported on registration time. variableLabels and constLabels can
|
||||||
|
// be nil if no such labels should be set. fqName and help must not be empty.
|
||||||
|
//
|
||||||
|
// variableLabels only contain the label names. Their label values are variable
|
||||||
|
// and therefore not part of the Desc. (They are managed within the Metric.)
|
||||||
|
//
|
||||||
|
// For constLabels, the label values are constant. Therefore, they are fully
|
||||||
|
// specified in the Desc. See the Opts documentation for the implications of
|
||||||
|
// constant labels.
|
||||||
|
func NewDesc(fqName, help string, variableLabels []string, constLabels Labels) *Desc { |
||||||
|
d := &Desc{ |
||||||
|
fqName: fqName, |
||||||
|
help: help, |
||||||
|
variableLabels: variableLabels, |
||||||
|
} |
||||||
|
if help == "" { |
||||||
|
d.err = errors.New("empty help string") |
||||||
|
return d |
||||||
|
} |
||||||
|
if !model.IsValidMetricName(model.LabelValue(fqName)) { |
||||||
|
d.err = fmt.Errorf("%q is not a valid metric name", fqName) |
||||||
|
return d |
||||||
|
} |
||||||
|
// labelValues contains the label values of const labels (in order of
|
||||||
|
// their sorted label names) plus the fqName (at position 0).
|
||||||
|
labelValues := make([]string, 1, len(constLabels)+1) |
||||||
|
labelValues[0] = fqName |
||||||
|
labelNames := make([]string, 0, len(constLabels)+len(variableLabels)) |
||||||
|
labelNameSet := map[string]struct{}{} |
||||||
|
// First add only the const label names and sort them...
|
||||||
|
for labelName := range constLabels { |
||||||
|
if !checkLabelName(labelName) { |
||||||
|
d.err = fmt.Errorf("%q is not a valid label name", labelName) |
||||||
|
return d |
||||||
|
} |
||||||
|
labelNames = append(labelNames, labelName) |
||||||
|
labelNameSet[labelName] = struct{}{} |
||||||
|
} |
||||||
|
sort.Strings(labelNames) |
||||||
|
// ... so that we can now add const label values in the order of their names.
|
||||||
|
for _, labelName := range labelNames { |
||||||
|
labelValues = append(labelValues, constLabels[labelName]) |
||||||
|
} |
||||||
|
// Now add the variable label names, but prefix them with something that
|
||||||
|
// cannot be in a regular label name. That prevents matching the label
|
||||||
|
// dimension with a different mix between preset and variable labels.
|
||||||
|
for _, labelName := range variableLabels { |
||||||
|
if !checkLabelName(labelName) { |
||||||
|
d.err = fmt.Errorf("%q is not a valid label name", labelName) |
||||||
|
return d |
||||||
|
} |
||||||
|
labelNames = append(labelNames, "$"+labelName) |
||||||
|
labelNameSet[labelName] = struct{}{} |
||||||
|
} |
||||||
|
if len(labelNames) != len(labelNameSet) { |
||||||
|
d.err = errors.New("duplicate label names") |
||||||
|
return d |
||||||
|
} |
||||||
|
vh := hashNew() |
||||||
|
for _, val := range labelValues { |
||||||
|
vh = hashAdd(vh, val) |
||||||
|
vh = hashAddByte(vh, separatorByte) |
||||||
|
} |
||||||
|
d.id = vh |
||||||
|
// Sort labelNames so that order doesn't matter for the hash.
|
||||||
|
sort.Strings(labelNames) |
||||||
|
// Now hash together (in this order) the help string and the sorted
|
||||||
|
// label names.
|
||||||
|
lh := hashNew() |
||||||
|
lh = hashAdd(lh, help) |
||||||
|
lh = hashAddByte(lh, separatorByte) |
||||||
|
for _, labelName := range labelNames { |
||||||
|
lh = hashAdd(lh, labelName) |
||||||
|
lh = hashAddByte(lh, separatorByte) |
||||||
|
} |
||||||
|
d.dimHash = lh |
||||||
|
|
||||||
|
d.constLabelPairs = make([]*dto.LabelPair, 0, len(constLabels)) |
||||||
|
for n, v := range constLabels { |
||||||
|
d.constLabelPairs = append(d.constLabelPairs, &dto.LabelPair{ |
||||||
|
Name: proto.String(n), |
||||||
|
Value: proto.String(v), |
||||||
|
}) |
||||||
|
} |
||||||
|
sort.Sort(LabelPairSorter(d.constLabelPairs)) |
||||||
|
return d |
||||||
|
} |
||||||
|
|
||||||
|
// NewInvalidDesc returns an invalid descriptor, i.e. a descriptor with the
|
||||||
|
// provided error set. If a collector returning such a descriptor is registered,
|
||||||
|
// registration will fail with the provided error. NewInvalidDesc can be used by
|
||||||
|
// a Collector to signal inability to describe itself.
|
||||||
|
func NewInvalidDesc(err error) *Desc { |
||||||
|
return &Desc{ |
||||||
|
err: err, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (d *Desc) String() string { |
||||||
|
lpStrings := make([]string, 0, len(d.constLabelPairs)) |
||||||
|
for _, lp := range d.constLabelPairs { |
||||||
|
lpStrings = append( |
||||||
|
lpStrings, |
||||||
|
fmt.Sprintf("%s=%q", lp.GetName(), lp.GetValue()), |
||||||
|
) |
||||||
|
} |
||||||
|
return fmt.Sprintf( |
||||||
|
"Desc{fqName: %q, help: %q, constLabels: {%s}, variableLabels: %v}", |
||||||
|
d.fqName, |
||||||
|
d.help, |
||||||
|
strings.Join(lpStrings, ","), |
||||||
|
d.variableLabels, |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
func checkLabelName(l string) bool { |
||||||
|
return model.LabelName(l).IsValid() && |
||||||
|
!strings.HasPrefix(l, reservedLabelPrefix) |
||||||
|
} |
||||||
@ -0,0 +1,181 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// Package prometheus provides metrics primitives to instrument code for
|
||||||
|
// monitoring. It also offers a registry for metrics. Sub-packages allow to
|
||||||
|
// expose the registered metrics via HTTP (package promhttp) or push them to a
|
||||||
|
// Pushgateway (package push).
|
||||||
|
//
|
||||||
|
// All exported functions and methods are safe to be used concurrently unless
|
||||||
|
//specified otherwise.
|
||||||
|
//
|
||||||
|
// A Basic Example
|
||||||
|
//
|
||||||
|
// As a starting point, a very basic usage example:
|
||||||
|
//
|
||||||
|
// package main
|
||||||
|
//
|
||||||
|
// import (
|
||||||
|
// "net/http"
|
||||||
|
//
|
||||||
|
// "github.com/prometheus/client_golang/prometheus"
|
||||||
|
// "github.com/prometheus/client_golang/prometheus/promhttp"
|
||||||
|
// )
|
||||||
|
//
|
||||||
|
// var (
|
||||||
|
// cpuTemp = prometheus.NewGauge(prometheus.GaugeOpts{
|
||||||
|
// Name: "cpu_temperature_celsius",
|
||||||
|
// Help: "Current temperature of the CPU.",
|
||||||
|
// })
|
||||||
|
// hdFailures = prometheus.NewCounterVec(
|
||||||
|
// prometheus.CounterOpts{
|
||||||
|
// Name: "hd_errors_total",
|
||||||
|
// Help: "Number of hard-disk errors.",
|
||||||
|
// },
|
||||||
|
// []string{"device"},
|
||||||
|
// )
|
||||||
|
// )
|
||||||
|
//
|
||||||
|
// func init() {
|
||||||
|
// // Metrics have to be registered to be exposed:
|
||||||
|
// prometheus.MustRegister(cpuTemp)
|
||||||
|
// prometheus.MustRegister(hdFailures)
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// func main() {
|
||||||
|
// cpuTemp.Set(65.3)
|
||||||
|
// hdFailures.With(prometheus.Labels{"device":"/dev/sda"}).Inc()
|
||||||
|
//
|
||||||
|
// // The Handler function provides a default handler to expose metrics
|
||||||
|
// // via an HTTP server. "/metrics" is the usual endpoint for that.
|
||||||
|
// http.Handle("/metrics", promhttp.Handler())
|
||||||
|
// log.Fatal(http.ListenAndServe(":8080", nil))
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
//
|
||||||
|
// This is a complete program that exports two metrics, a Gauge and a Counter,
|
||||||
|
// the latter with a label attached to turn it into a (one-dimensional) vector.
|
||||||
|
//
|
||||||
|
// Metrics
|
||||||
|
//
|
||||||
|
// The number of exported identifiers in this package might appear a bit
|
||||||
|
// overwhelming. Hovever, in addition to the basic plumbing shown in the example
|
||||||
|
// above, you only need to understand the different metric types and their
|
||||||
|
// vector versions for basic usage.
|
||||||
|
//
|
||||||
|
// Above, you have already touched the Counter and the Gauge. There are two more
|
||||||
|
// advanced metric types: the Summary and Histogram. A more thorough description
|
||||||
|
// of those four metric types can be found in the Prometheus docs:
|
||||||
|
// https://prometheus.io/docs/concepts/metric_types/
|
||||||
|
//
|
||||||
|
// A fifth "type" of metric is Untyped. It behaves like a Gauge, but signals the
|
||||||
|
// Prometheus server not to assume anything about its type.
|
||||||
|
//
|
||||||
|
// In addition to the fundamental metric types Gauge, Counter, Summary,
|
||||||
|
// Histogram, and Untyped, a very important part of the Prometheus data model is
|
||||||
|
// the partitioning of samples along dimensions called labels, which results in
|
||||||
|
// metric vectors. The fundamental types are GaugeVec, CounterVec, SummaryVec,
|
||||||
|
// HistogramVec, and UntypedVec.
|
||||||
|
//
|
||||||
|
// While only the fundamental metric types implement the Metric interface, both
|
||||||
|
// the metrics and their vector versions implement the Collector interface. A
|
||||||
|
// Collector manages the collection of a number of Metrics, but for convenience,
|
||||||
|
// a Metric can also “collect itself”. Note that Gauge, Counter, Summary,
|
||||||
|
// Histogram, and Untyped are interfaces themselves while GaugeVec, CounterVec,
|
||||||
|
// SummaryVec, HistogramVec, and UntypedVec are not.
|
||||||
|
//
|
||||||
|
// To create instances of Metrics and their vector versions, you need a suitable
|
||||||
|
// …Opts struct, i.e. GaugeOpts, CounterOpts, SummaryOpts,
|
||||||
|
// HistogramOpts, or UntypedOpts.
|
||||||
|
//
|
||||||
|
// Custom Collectors and constant Metrics
|
||||||
|
//
|
||||||
|
// While you could create your own implementations of Metric, most likely you
|
||||||
|
// will only ever implement the Collector interface on your own. At a first
|
||||||
|
// glance, a custom Collector seems handy to bundle Metrics for common
|
||||||
|
// registration (with the prime example of the different metric vectors above,
|
||||||
|
// which bundle all the metrics of the same name but with different labels).
|
||||||
|
//
|
||||||
|
// There is a more involved use case, too: If you already have metrics
|
||||||
|
// available, created outside of the Prometheus context, you don't need the
|
||||||
|
// interface of the various Metric types. You essentially want to mirror the
|
||||||
|
// existing numbers into Prometheus Metrics during collection. An own
|
||||||
|
// implementation of the Collector interface is perfect for that. You can create
|
||||||
|
// Metric instances “on the fly” using NewConstMetric, NewConstHistogram, and
|
||||||
|
// NewConstSummary (and their respective Must… versions). That will happen in
|
||||||
|
// the Collect method. The Describe method has to return separate Desc
|
||||||
|
// instances, representative of the “throw-away” metrics to be created
|
||||||
|
// later. NewDesc comes in handy to create those Desc instances.
|
||||||
|
//
|
||||||
|
// The Collector example illustrates the use case. You can also look at the
|
||||||
|
// source code of the processCollector (mirroring process metrics), the
|
||||||
|
// goCollector (mirroring Go metrics), or the expvarCollector (mirroring expvar
|
||||||
|
// metrics) as examples that are used in this package itself.
|
||||||
|
//
|
||||||
|
// If you just need to call a function to get a single float value to collect as
|
||||||
|
// a metric, GaugeFunc, CounterFunc, or UntypedFunc might be interesting
|
||||||
|
// shortcuts.
|
||||||
|
//
|
||||||
|
// Advanced Uses of the Registry
|
||||||
|
//
|
||||||
|
// While MustRegister is the by far most common way of registering a Collector,
|
||||||
|
// sometimes you might want to handle the errors the registration might
|
||||||
|
// cause. As suggested by the name, MustRegister panics if an error occurs. With
|
||||||
|
// the Register function, the error is returned and can be handled.
|
||||||
|
//
|
||||||
|
// An error is returned if the registered Collector is incompatible or
|
||||||
|
// inconsistent with already registered metrics. The registry aims for
|
||||||
|
// consistency of the collected metrics according to the Prometheus data
|
||||||
|
// model. Inconsistencies are ideally detected at registration time, not at
|
||||||
|
// collect time. The former will usually be detected at start-up time of a
|
||||||
|
// program, while the latter will only happen at scrape time, possibly not even
|
||||||
|
// on the first scrape if the inconsistency only becomes relevant later. That is
|
||||||
|
// the main reason why a Collector and a Metric have to describe themselves to
|
||||||
|
// the registry.
|
||||||
|
//
|
||||||
|
// So far, everything we did operated on the so-called default registry, as it
|
||||||
|
// can be found in the global DefaultRegistry variable. With NewRegistry, you
|
||||||
|
// can create a custom registry, or you can even implement the Registerer or
|
||||||
|
// Gatherer interfaces yourself. The methods Register and Unregister work in
|
||||||
|
// the same way on a custom registry as the global functions Register and
|
||||||
|
// Unregister on the default registry.
|
||||||
|
//
|
||||||
|
// There are a number of uses for custom registries: You can use registries
|
||||||
|
// with special properties, see NewPedanticRegistry. You can avoid global state,
|
||||||
|
// as it is imposed by the DefaultRegistry. You can use multiple registries at
|
||||||
|
// the same time to expose different metrics in different ways. You can use
|
||||||
|
// separate registries for testing purposes.
|
||||||
|
//
|
||||||
|
// Also note that the DefaultRegistry comes registered with a Collector for Go
|
||||||
|
// runtime metrics (via NewGoCollector) and a Collector for process metrics (via
|
||||||
|
// NewProcessCollector). With a custom registry, you are in control and decide
|
||||||
|
// yourself about the Collectors to register.
|
||||||
|
//
|
||||||
|
// HTTP Exposition
|
||||||
|
//
|
||||||
|
// The Registry implements the Gatherer interface. The caller of the Gather
|
||||||
|
// method can then expose the gathered metrics in some way. Usually, the metrics
|
||||||
|
// are served via HTTP on the /metrics endpoint. That's happening in the example
|
||||||
|
// above. The tools to expose metrics via HTTP are in the promhttp
|
||||||
|
// sub-package. (The top-level functions in the prometheus package are
|
||||||
|
// deprecated.)
|
||||||
|
//
|
||||||
|
// Pushing to the Pushgateway
|
||||||
|
//
|
||||||
|
// Function for pushing to the Pushgateway can be found in the push sub-package.
|
||||||
|
//
|
||||||
|
// Other Means of Exposition
|
||||||
|
//
|
||||||
|
// More ways of exposing metrics can easily be added. Sending metrics to
|
||||||
|
// Graphite would be an example that will soon be implemented.
|
||||||
|
package prometheus |
||||||
@ -0,0 +1,119 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"expvar" |
||||||
|
) |
||||||
|
|
||||||
|
type expvarCollector struct { |
||||||
|
exports map[string]*Desc |
||||||
|
} |
||||||
|
|
||||||
|
// NewExpvarCollector returns a newly allocated expvar Collector that still has
|
||||||
|
// to be registered with a Prometheus registry.
|
||||||
|
//
|
||||||
|
// An expvar Collector collects metrics from the expvar interface. It provides a
|
||||||
|
// quick way to expose numeric values that are already exported via expvar as
|
||||||
|
// Prometheus metrics. Note that the data models of expvar and Prometheus are
|
||||||
|
// fundamentally different, and that the expvar Collector is inherently slower
|
||||||
|
// than native Prometheus metrics. Thus, the expvar Collector is probably great
|
||||||
|
// for experiments and prototying, but you should seriously consider a more
|
||||||
|
// direct implementation of Prometheus metrics for monitoring production
|
||||||
|
// systems.
|
||||||
|
//
|
||||||
|
// The exports map has the following meaning:
|
||||||
|
//
|
||||||
|
// The keys in the map correspond to expvar keys, i.e. for every expvar key you
|
||||||
|
// want to export as Prometheus metric, you need an entry in the exports
|
||||||
|
// map. The descriptor mapped to each key describes how to export the expvar
|
||||||
|
// value. It defines the name and the help string of the Prometheus metric
|
||||||
|
// proxying the expvar value. The type will always be Untyped.
|
||||||
|
//
|
||||||
|
// For descriptors without variable labels, the expvar value must be a number or
|
||||||
|
// a bool. The number is then directly exported as the Prometheus sample
|
||||||
|
// value. (For a bool, 'false' translates to 0 and 'true' to 1). Expvar values
|
||||||
|
// that are not numbers or bools are silently ignored.
|
||||||
|
//
|
||||||
|
// If the descriptor has one variable label, the expvar value must be an expvar
|
||||||
|
// map. The keys in the expvar map become the various values of the one
|
||||||
|
// Prometheus label. The values in the expvar map must be numbers or bools again
|
||||||
|
// as above.
|
||||||
|
//
|
||||||
|
// For descriptors with more than one variable label, the expvar must be a
|
||||||
|
// nested expvar map, i.e. where the values of the topmost map are maps again
|
||||||
|
// etc. until a depth is reached that corresponds to the number of labels. The
|
||||||
|
// leaves of that structure must be numbers or bools as above to serve as the
|
||||||
|
// sample values.
|
||||||
|
//
|
||||||
|
// Anything that does not fit into the scheme above is silently ignored.
|
||||||
|
func NewExpvarCollector(exports map[string]*Desc) Collector { |
||||||
|
return &expvarCollector{ |
||||||
|
exports: exports, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Describe implements Collector.
|
||||||
|
func (e *expvarCollector) Describe(ch chan<- *Desc) { |
||||||
|
for _, desc := range e.exports { |
||||||
|
ch <- desc |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Collect implements Collector.
|
||||||
|
func (e *expvarCollector) Collect(ch chan<- Metric) { |
||||||
|
for name, desc := range e.exports { |
||||||
|
var m Metric |
||||||
|
expVar := expvar.Get(name) |
||||||
|
if expVar == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
var v interface{} |
||||||
|
labels := make([]string, len(desc.variableLabels)) |
||||||
|
if err := json.Unmarshal([]byte(expVar.String()), &v); err != nil { |
||||||
|
ch <- NewInvalidMetric(desc, err) |
||||||
|
continue |
||||||
|
} |
||||||
|
var processValue func(v interface{}, i int) |
||||||
|
processValue = func(v interface{}, i int) { |
||||||
|
if i >= len(labels) { |
||||||
|
copiedLabels := append(make([]string, 0, len(labels)), labels...) |
||||||
|
switch v := v.(type) { |
||||||
|
case float64: |
||||||
|
m = MustNewConstMetric(desc, UntypedValue, v, copiedLabels...) |
||||||
|
case bool: |
||||||
|
if v { |
||||||
|
m = MustNewConstMetric(desc, UntypedValue, 1, copiedLabels...) |
||||||
|
} else { |
||||||
|
m = MustNewConstMetric(desc, UntypedValue, 0, copiedLabels...) |
||||||
|
} |
||||||
|
default: |
||||||
|
return |
||||||
|
} |
||||||
|
ch <- m |
||||||
|
return |
||||||
|
} |
||||||
|
vm, ok := v.(map[string]interface{}) |
||||||
|
if !ok { |
||||||
|
return |
||||||
|
} |
||||||
|
for lv, val := range vm { |
||||||
|
labels[i] = lv |
||||||
|
processValue(val, i+1) |
||||||
|
} |
||||||
|
} |
||||||
|
processValue(v, 0) |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,29 @@ |
|||||||
|
package prometheus |
||||||
|
|
||||||
|
// Inline and byte-free variant of hash/fnv's fnv64a.
|
||||||
|
|
||||||
|
const ( |
||||||
|
offset64 = 14695981039346656037 |
||||||
|
prime64 = 1099511628211 |
||||||
|
) |
||||||
|
|
||||||
|
// hashNew initializies a new fnv64a hash value.
|
||||||
|
func hashNew() uint64 { |
||||||
|
return offset64 |
||||||
|
} |
||||||
|
|
||||||
|
// hashAdd adds a string to a fnv64a hash value, returning the updated hash.
|
||||||
|
func hashAdd(h uint64, s string) uint64 { |
||||||
|
for i := 0; i < len(s); i++ { |
||||||
|
h ^= uint64(s[i]) |
||||||
|
h *= prime64 |
||||||
|
} |
||||||
|
return h |
||||||
|
} |
||||||
|
|
||||||
|
// hashAddByte adds a byte to a fnv64a hash value, returning the updated hash.
|
||||||
|
func hashAddByte(h uint64, b byte) uint64 { |
||||||
|
h ^= uint64(b) |
||||||
|
h *= prime64 |
||||||
|
return h |
||||||
|
} |
||||||
@ -0,0 +1,145 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
// Gauge is a Metric that represents a single numerical value that can
|
||||||
|
// arbitrarily go up and down.
|
||||||
|
//
|
||||||
|
// A Gauge is typically used for measured values like temperatures or current
|
||||||
|
// memory usage, but also "counts" that can go up and down, like the number of
|
||||||
|
// running goroutines.
|
||||||
|
//
|
||||||
|
// To create Gauge instances, use NewGauge.
|
||||||
|
type Gauge interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
|
||||||
|
// Set sets the Gauge to an arbitrary value.
|
||||||
|
Set(float64) |
||||||
|
// Inc increments the Gauge by 1. Use Add to increment it by arbitrary
|
||||||
|
// values.
|
||||||
|
Inc() |
||||||
|
// Dec decrements the Gauge by 1. Use Sub to decrement it by arbitrary
|
||||||
|
// values.
|
||||||
|
Dec() |
||||||
|
// Add adds the given value to the Gauge. (The value can be negative,
|
||||||
|
// resulting in a decrease of the Gauge.)
|
||||||
|
Add(float64) |
||||||
|
// Sub subtracts the given value from the Gauge. (The value can be
|
||||||
|
// negative, resulting in an increase of the Gauge.)
|
||||||
|
Sub(float64) |
||||||
|
|
||||||
|
// SetToCurrentTime sets the Gauge to the current Unix time in seconds.
|
||||||
|
SetToCurrentTime() |
||||||
|
} |
||||||
|
|
||||||
|
// GaugeOpts is an alias for Opts. See there for doc comments.
|
||||||
|
type GaugeOpts Opts |
||||||
|
|
||||||
|
// NewGauge creates a new Gauge based on the provided GaugeOpts.
|
||||||
|
func NewGauge(opts GaugeOpts) Gauge { |
||||||
|
return newValue(NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
), GaugeValue, 0) |
||||||
|
} |
||||||
|
|
||||||
|
// GaugeVec is a Collector that bundles a set of Gauges that all share the same
|
||||||
|
// Desc, but have different values for their variable labels. This is used if
|
||||||
|
// you want to count the same thing partitioned by various dimensions
|
||||||
|
// (e.g. number of operations queued, partitioned by user and operation
|
||||||
|
// type). Create instances with NewGaugeVec.
|
||||||
|
type GaugeVec struct { |
||||||
|
*MetricVec |
||||||
|
} |
||||||
|
|
||||||
|
// NewGaugeVec creates a new GaugeVec based on the provided GaugeOpts and
|
||||||
|
// partitioned by the given label names. At least one label name must be
|
||||||
|
// provided.
|
||||||
|
func NewGaugeVec(opts GaugeOpts, labelNames []string) *GaugeVec { |
||||||
|
desc := NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
labelNames, |
||||||
|
opts.ConstLabels, |
||||||
|
) |
||||||
|
return &GaugeVec{ |
||||||
|
MetricVec: newMetricVec(desc, func(lvs ...string) Metric { |
||||||
|
return newValue(desc, GaugeValue, 0, lvs...) |
||||||
|
}), |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWithLabelValues replaces the method of the same name in
|
||||||
|
// MetricVec. The difference is that this method returns a Gauge and not a
|
||||||
|
// Metric so that no type conversion is required.
|
||||||
|
func (m *GaugeVec) GetMetricWithLabelValues(lvs ...string) (Gauge, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWithLabelValues(lvs...) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Gauge), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWith replaces the method of the same name in MetricVec. The
|
||||||
|
// difference is that this method returns a Gauge and not a Metric so that no
|
||||||
|
// type conversion is required.
|
||||||
|
func (m *GaugeVec) GetMetricWith(labels Labels) (Gauge, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWith(labels) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Gauge), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||||
|
// GetMetricWithLabelValues would have returned an error. By not returning an
|
||||||
|
// error, WithLabelValues allows shortcuts like
|
||||||
|
// myVec.WithLabelValues("404", "GET").Add(42)
|
||||||
|
func (m *GaugeVec) WithLabelValues(lvs ...string) Gauge { |
||||||
|
return m.MetricVec.WithLabelValues(lvs...).(Gauge) |
||||||
|
} |
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||||
|
// returned an error. By not returning an error, With allows shortcuts like
|
||||||
|
// myVec.With(Labels{"code": "404", "method": "GET"}).Add(42)
|
||||||
|
func (m *GaugeVec) With(labels Labels) Gauge { |
||||||
|
return m.MetricVec.With(labels).(Gauge) |
||||||
|
} |
||||||
|
|
||||||
|
// GaugeFunc is a Gauge whose value is determined at collect time by calling a
|
||||||
|
// provided function.
|
||||||
|
//
|
||||||
|
// To create GaugeFunc instances, use NewGaugeFunc.
|
||||||
|
type GaugeFunc interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
} |
||||||
|
|
||||||
|
// NewGaugeFunc creates a new GaugeFunc based on the provided GaugeOpts. The
|
||||||
|
// value reported is determined by calling the given function from within the
|
||||||
|
// Write method. Take into account that metric collection may happen
|
||||||
|
// concurrently. If that results in concurrent calls to Write, like in the case
|
||||||
|
// where a GaugeFunc is directly registered with Prometheus, the provided
|
||||||
|
// function must be concurrency-safe.
|
||||||
|
func NewGaugeFunc(opts GaugeOpts, function func() float64) GaugeFunc { |
||||||
|
return newValueFunc(NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
), GaugeValue, function) |
||||||
|
} |
||||||
@ -0,0 +1,263 @@ |
|||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"runtime" |
||||||
|
"runtime/debug" |
||||||
|
"time" |
||||||
|
) |
||||||
|
|
||||||
|
type goCollector struct { |
||||||
|
goroutines Gauge |
||||||
|
gcDesc *Desc |
||||||
|
|
||||||
|
// metrics to describe and collect
|
||||||
|
metrics memStatsMetrics |
||||||
|
} |
||||||
|
|
||||||
|
// NewGoCollector returns a collector which exports metrics about the current
|
||||||
|
// go process.
|
||||||
|
func NewGoCollector() Collector { |
||||||
|
return &goCollector{ |
||||||
|
goroutines: NewGauge(GaugeOpts{ |
||||||
|
Namespace: "go", |
||||||
|
Name: "goroutines", |
||||||
|
Help: "Number of goroutines that currently exist.", |
||||||
|
}), |
||||||
|
gcDesc: NewDesc( |
||||||
|
"go_gc_duration_seconds", |
||||||
|
"A summary of the GC invocation durations.", |
||||||
|
nil, nil), |
||||||
|
metrics: memStatsMetrics{ |
||||||
|
{ |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("alloc_bytes"), |
||||||
|
"Number of bytes allocated and still in use.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Alloc) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("alloc_bytes_total"), |
||||||
|
"Total number of bytes allocated, even if freed.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.TotalAlloc) }, |
||||||
|
valType: CounterValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("sys_bytes"), |
||||||
|
"Number of bytes obtained from system.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Sys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("lookups_total"), |
||||||
|
"Total number of pointer lookups.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Lookups) }, |
||||||
|
valType: CounterValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("mallocs_total"), |
||||||
|
"Total number of mallocs.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Mallocs) }, |
||||||
|
valType: CounterValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("frees_total"), |
||||||
|
"Total number of frees.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.Frees) }, |
||||||
|
valType: CounterValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("heap_alloc_bytes"), |
||||||
|
"Number of heap bytes allocated and still in use.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapAlloc) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("heap_sys_bytes"), |
||||||
|
"Number of heap bytes obtained from system.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapSys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("heap_idle_bytes"), |
||||||
|
"Number of heap bytes waiting to be used.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapIdle) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("heap_inuse_bytes"), |
||||||
|
"Number of heap bytes that are in use.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapInuse) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("heap_released_bytes"), |
||||||
|
"Number of heap bytes released to OS.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapReleased) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("heap_objects"), |
||||||
|
"Number of allocated objects.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.HeapObjects) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("stack_inuse_bytes"), |
||||||
|
"Number of bytes in use by the stack allocator.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.StackInuse) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("stack_sys_bytes"), |
||||||
|
"Number of bytes obtained from system for stack allocator.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.StackSys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("mspan_inuse_bytes"), |
||||||
|
"Number of bytes in use by mspan structures.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MSpanInuse) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("mspan_sys_bytes"), |
||||||
|
"Number of bytes used for mspan structures obtained from system.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MSpanSys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("mcache_inuse_bytes"), |
||||||
|
"Number of bytes in use by mcache structures.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MCacheInuse) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("mcache_sys_bytes"), |
||||||
|
"Number of bytes used for mcache structures obtained from system.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.MCacheSys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("buck_hash_sys_bytes"), |
||||||
|
"Number of bytes used by the profiling bucket hash table.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.BuckHashSys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("gc_sys_bytes"), |
||||||
|
"Number of bytes used for garbage collection system metadata.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.GCSys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("other_sys_bytes"), |
||||||
|
"Number of bytes used for other system allocations.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.OtherSys) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("next_gc_bytes"), |
||||||
|
"Number of heap bytes when next garbage collection will take place.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.NextGC) }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, { |
||||||
|
desc: NewDesc( |
||||||
|
memstatNamespace("last_gc_time_seconds"), |
||||||
|
"Number of seconds since 1970 of last garbage collection.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
eval: func(ms *runtime.MemStats) float64 { return float64(ms.LastGC) / 1e9 }, |
||||||
|
valType: GaugeValue, |
||||||
|
}, |
||||||
|
}, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func memstatNamespace(s string) string { |
||||||
|
return fmt.Sprintf("go_memstats_%s", s) |
||||||
|
} |
||||||
|
|
||||||
|
// Describe returns all descriptions of the collector.
|
||||||
|
func (c *goCollector) Describe(ch chan<- *Desc) { |
||||||
|
ch <- c.goroutines.Desc() |
||||||
|
ch <- c.gcDesc |
||||||
|
|
||||||
|
for _, i := range c.metrics { |
||||||
|
ch <- i.desc |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Collect returns the current state of all metrics of the collector.
|
||||||
|
func (c *goCollector) Collect(ch chan<- Metric) { |
||||||
|
c.goroutines.Set(float64(runtime.NumGoroutine())) |
||||||
|
ch <- c.goroutines |
||||||
|
|
||||||
|
var stats debug.GCStats |
||||||
|
stats.PauseQuantiles = make([]time.Duration, 5) |
||||||
|
debug.ReadGCStats(&stats) |
||||||
|
|
||||||
|
quantiles := make(map[float64]float64) |
||||||
|
for idx, pq := range stats.PauseQuantiles[1:] { |
||||||
|
quantiles[float64(idx+1)/float64(len(stats.PauseQuantiles)-1)] = pq.Seconds() |
||||||
|
} |
||||||
|
quantiles[0.0] = stats.PauseQuantiles[0].Seconds() |
||||||
|
ch <- MustNewConstSummary(c.gcDesc, uint64(stats.NumGC), float64(stats.PauseTotal.Seconds()), quantiles) |
||||||
|
|
||||||
|
ms := &runtime.MemStats{} |
||||||
|
runtime.ReadMemStats(ms) |
||||||
|
for _, i := range c.metrics { |
||||||
|
ch <- MustNewConstMetric(i.desc, i.valType, i.eval(ms)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// memStatsMetrics provide description, value, and value type for memstat metrics.
|
||||||
|
type memStatsMetrics []struct { |
||||||
|
desc *Desc |
||||||
|
eval func(*runtime.MemStats) float64 |
||||||
|
valType ValueType |
||||||
|
} |
||||||
@ -0,0 +1,444 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"math" |
||||||
|
"sort" |
||||||
|
"sync/atomic" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
) |
||||||
|
|
||||||
|
// A Histogram counts individual observations from an event or sample stream in
|
||||||
|
// configurable buckets. Similar to a summary, it also provides a sum of
|
||||||
|
// observations and an observation count.
|
||||||
|
//
|
||||||
|
// On the Prometheus server, quantiles can be calculated from a Histogram using
|
||||||
|
// the histogram_quantile function in the query language.
|
||||||
|
//
|
||||||
|
// Note that Histograms, in contrast to Summaries, can be aggregated with the
|
||||||
|
// Prometheus query language (see the documentation for detailed
|
||||||
|
// procedures). However, Histograms require the user to pre-define suitable
|
||||||
|
// buckets, and they are in general less accurate. The Observe method of a
|
||||||
|
// Histogram has a very low performance overhead in comparison with the Observe
|
||||||
|
// method of a Summary.
|
||||||
|
//
|
||||||
|
// To create Histogram instances, use NewHistogram.
|
||||||
|
type Histogram interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
|
||||||
|
// Observe adds a single observation to the histogram.
|
||||||
|
Observe(float64) |
||||||
|
} |
||||||
|
|
||||||
|
// bucketLabel is used for the label that defines the upper bound of a
|
||||||
|
// bucket of a histogram ("le" -> "less or equal").
|
||||||
|
const bucketLabel = "le" |
||||||
|
|
||||||
|
// DefBuckets are the default Histogram buckets. The default buckets are
|
||||||
|
// tailored to broadly measure the response time (in seconds) of a network
|
||||||
|
// service. Most likely, however, you will be required to define buckets
|
||||||
|
// customized to your use case.
|
||||||
|
var ( |
||||||
|
DefBuckets = []float64{.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10} |
||||||
|
|
||||||
|
errBucketLabelNotAllowed = fmt.Errorf( |
||||||
|
"%q is not allowed as label name in histograms", bucketLabel, |
||||||
|
) |
||||||
|
) |
||||||
|
|
||||||
|
// LinearBuckets creates 'count' buckets, each 'width' wide, where the lowest
|
||||||
|
// bucket has an upper bound of 'start'. The final +Inf bucket is not counted
|
||||||
|
// and not included in the returned slice. The returned slice is meant to be
|
||||||
|
// used for the Buckets field of HistogramOpts.
|
||||||
|
//
|
||||||
|
// The function panics if 'count' is zero or negative.
|
||||||
|
func LinearBuckets(start, width float64, count int) []float64 { |
||||||
|
if count < 1 { |
||||||
|
panic("LinearBuckets needs a positive count") |
||||||
|
} |
||||||
|
buckets := make([]float64, count) |
||||||
|
for i := range buckets { |
||||||
|
buckets[i] = start |
||||||
|
start += width |
||||||
|
} |
||||||
|
return buckets |
||||||
|
} |
||||||
|
|
||||||
|
// ExponentialBuckets creates 'count' buckets, where the lowest bucket has an
|
||||||
|
// upper bound of 'start' and each following bucket's upper bound is 'factor'
|
||||||
|
// times the previous bucket's upper bound. The final +Inf bucket is not counted
|
||||||
|
// and not included in the returned slice. The returned slice is meant to be
|
||||||
|
// used for the Buckets field of HistogramOpts.
|
||||||
|
//
|
||||||
|
// The function panics if 'count' is 0 or negative, if 'start' is 0 or negative,
|
||||||
|
// or if 'factor' is less than or equal 1.
|
||||||
|
func ExponentialBuckets(start, factor float64, count int) []float64 { |
||||||
|
if count < 1 { |
||||||
|
panic("ExponentialBuckets needs a positive count") |
||||||
|
} |
||||||
|
if start <= 0 { |
||||||
|
panic("ExponentialBuckets needs a positive start value") |
||||||
|
} |
||||||
|
if factor <= 1 { |
||||||
|
panic("ExponentialBuckets needs a factor greater than 1") |
||||||
|
} |
||||||
|
buckets := make([]float64, count) |
||||||
|
for i := range buckets { |
||||||
|
buckets[i] = start |
||||||
|
start *= factor |
||||||
|
} |
||||||
|
return buckets |
||||||
|
} |
||||||
|
|
||||||
|
// HistogramOpts bundles the options for creating a Histogram metric. It is
|
||||||
|
// mandatory to set Name and Help to a non-empty string. All other fields are
|
||||||
|
// optional and can safely be left at their zero value.
|
||||||
|
type HistogramOpts struct { |
||||||
|
// Namespace, Subsystem, and Name are components of the fully-qualified
|
||||||
|
// name of the Histogram (created by joining these components with
|
||||||
|
// "_"). Only Name is mandatory, the others merely help structuring the
|
||||||
|
// name. Note that the fully-qualified name of the Histogram must be a
|
||||||
|
// valid Prometheus metric name.
|
||||||
|
Namespace string |
||||||
|
Subsystem string |
||||||
|
Name string |
||||||
|
|
||||||
|
// Help provides information about this Histogram. Mandatory!
|
||||||
|
//
|
||||||
|
// Metrics with the same fully-qualified name must have the same Help
|
||||||
|
// string.
|
||||||
|
Help string |
||||||
|
|
||||||
|
// ConstLabels are used to attach fixed labels to this
|
||||||
|
// Histogram. Histograms with the same fully-qualified name must have the
|
||||||
|
// same label names in their ConstLabels.
|
||||||
|
//
|
||||||
|
// Note that in most cases, labels have a value that varies during the
|
||||||
|
// lifetime of a process. Those labels are usually managed with a
|
||||||
|
// HistogramVec. ConstLabels serve only special purposes. One is for the
|
||||||
|
// special case where the value of a label does not change during the
|
||||||
|
// lifetime of a process, e.g. if the revision of the running binary is
|
||||||
|
// put into a label. Another, more advanced purpose is if more than one
|
||||||
|
// Collector needs to collect Histograms with the same fully-qualified
|
||||||
|
// name. In that case, those Summaries must differ in the values of
|
||||||
|
// their ConstLabels. See the Collector examples.
|
||||||
|
//
|
||||||
|
// If the value of a label never changes (not even between binaries),
|
||||||
|
// that label most likely should not be a label at all (but part of the
|
||||||
|
// metric name).
|
||||||
|
ConstLabels Labels |
||||||
|
|
||||||
|
// Buckets defines the buckets into which observations are counted. Each
|
||||||
|
// element in the slice is the upper inclusive bound of a bucket. The
|
||||||
|
// values must be sorted in strictly increasing order. There is no need
|
||||||
|
// to add a highest bucket with +Inf bound, it will be added
|
||||||
|
// implicitly. The default value is DefBuckets.
|
||||||
|
Buckets []float64 |
||||||
|
} |
||||||
|
|
||||||
|
// NewHistogram creates a new Histogram based on the provided HistogramOpts. It
|
||||||
|
// panics if the buckets in HistogramOpts are not in strictly increasing order.
|
||||||
|
func NewHistogram(opts HistogramOpts) Histogram { |
||||||
|
return newHistogram( |
||||||
|
NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
), |
||||||
|
opts, |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
func newHistogram(desc *Desc, opts HistogramOpts, labelValues ...string) Histogram { |
||||||
|
if len(desc.variableLabels) != len(labelValues) { |
||||||
|
panic(errInconsistentCardinality) |
||||||
|
} |
||||||
|
|
||||||
|
for _, n := range desc.variableLabels { |
||||||
|
if n == bucketLabel { |
||||||
|
panic(errBucketLabelNotAllowed) |
||||||
|
} |
||||||
|
} |
||||||
|
for _, lp := range desc.constLabelPairs { |
||||||
|
if lp.GetName() == bucketLabel { |
||||||
|
panic(errBucketLabelNotAllowed) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if len(opts.Buckets) == 0 { |
||||||
|
opts.Buckets = DefBuckets |
||||||
|
} |
||||||
|
|
||||||
|
h := &histogram{ |
||||||
|
desc: desc, |
||||||
|
upperBounds: opts.Buckets, |
||||||
|
labelPairs: makeLabelPairs(desc, labelValues), |
||||||
|
} |
||||||
|
for i, upperBound := range h.upperBounds { |
||||||
|
if i < len(h.upperBounds)-1 { |
||||||
|
if upperBound >= h.upperBounds[i+1] { |
||||||
|
panic(fmt.Errorf( |
||||||
|
"histogram buckets must be in increasing order: %f >= %f", |
||||||
|
upperBound, h.upperBounds[i+1], |
||||||
|
)) |
||||||
|
} |
||||||
|
} else { |
||||||
|
if math.IsInf(upperBound, +1) { |
||||||
|
// The +Inf bucket is implicit. Remove it here.
|
||||||
|
h.upperBounds = h.upperBounds[:i] |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
// Finally we know the final length of h.upperBounds and can make counts.
|
||||||
|
h.counts = make([]uint64, len(h.upperBounds)) |
||||||
|
|
||||||
|
h.init(h) // Init self-collection.
|
||||||
|
return h |
||||||
|
} |
||||||
|
|
||||||
|
type histogram struct { |
||||||
|
// sumBits contains the bits of the float64 representing the sum of all
|
||||||
|
// observations. sumBits and count have to go first in the struct to
|
||||||
|
// guarantee alignment for atomic operations.
|
||||||
|
// http://golang.org/pkg/sync/atomic/#pkg-note-BUG
|
||||||
|
sumBits uint64 |
||||||
|
count uint64 |
||||||
|
|
||||||
|
selfCollector |
||||||
|
// Note that there is no mutex required.
|
||||||
|
|
||||||
|
desc *Desc |
||||||
|
|
||||||
|
upperBounds []float64 |
||||||
|
counts []uint64 |
||||||
|
|
||||||
|
labelPairs []*dto.LabelPair |
||||||
|
} |
||||||
|
|
||||||
|
func (h *histogram) Desc() *Desc { |
||||||
|
return h.desc |
||||||
|
} |
||||||
|
|
||||||
|
func (h *histogram) Observe(v float64) { |
||||||
|
// TODO(beorn7): For small numbers of buckets (<30), a linear search is
|
||||||
|
// slightly faster than the binary search. If we really care, we could
|
||||||
|
// switch from one search strategy to the other depending on the number
|
||||||
|
// of buckets.
|
||||||
|
//
|
||||||
|
// Microbenchmarks (BenchmarkHistogramNoLabels):
|
||||||
|
// 11 buckets: 38.3 ns/op linear - binary 48.7 ns/op
|
||||||
|
// 100 buckets: 78.1 ns/op linear - binary 54.9 ns/op
|
||||||
|
// 300 buckets: 154 ns/op linear - binary 61.6 ns/op
|
||||||
|
i := sort.SearchFloat64s(h.upperBounds, v) |
||||||
|
if i < len(h.counts) { |
||||||
|
atomic.AddUint64(&h.counts[i], 1) |
||||||
|
} |
||||||
|
atomic.AddUint64(&h.count, 1) |
||||||
|
for { |
||||||
|
oldBits := atomic.LoadUint64(&h.sumBits) |
||||||
|
newBits := math.Float64bits(math.Float64frombits(oldBits) + v) |
||||||
|
if atomic.CompareAndSwapUint64(&h.sumBits, oldBits, newBits) { |
||||||
|
break |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (h *histogram) Write(out *dto.Metric) error { |
||||||
|
his := &dto.Histogram{} |
||||||
|
buckets := make([]*dto.Bucket, len(h.upperBounds)) |
||||||
|
|
||||||
|
his.SampleSum = proto.Float64(math.Float64frombits(atomic.LoadUint64(&h.sumBits))) |
||||||
|
his.SampleCount = proto.Uint64(atomic.LoadUint64(&h.count)) |
||||||
|
var count uint64 |
||||||
|
for i, upperBound := range h.upperBounds { |
||||||
|
count += atomic.LoadUint64(&h.counts[i]) |
||||||
|
buckets[i] = &dto.Bucket{ |
||||||
|
CumulativeCount: proto.Uint64(count), |
||||||
|
UpperBound: proto.Float64(upperBound), |
||||||
|
} |
||||||
|
} |
||||||
|
his.Bucket = buckets |
||||||
|
out.Histogram = his |
||||||
|
out.Label = h.labelPairs |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// HistogramVec is a Collector that bundles a set of Histograms that all share the
|
||||||
|
// same Desc, but have different values for their variable labels. This is used
|
||||||
|
// if you want to count the same thing partitioned by various dimensions
|
||||||
|
// (e.g. HTTP request latencies, partitioned by status code and method). Create
|
||||||
|
// instances with NewHistogramVec.
|
||||||
|
type HistogramVec struct { |
||||||
|
*MetricVec |
||||||
|
} |
||||||
|
|
||||||
|
// NewHistogramVec creates a new HistogramVec based on the provided HistogramOpts and
|
||||||
|
// partitioned by the given label names. At least one label name must be
|
||||||
|
// provided.
|
||||||
|
func NewHistogramVec(opts HistogramOpts, labelNames []string) *HistogramVec { |
||||||
|
desc := NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
labelNames, |
||||||
|
opts.ConstLabels, |
||||||
|
) |
||||||
|
return &HistogramVec{ |
||||||
|
MetricVec: newMetricVec(desc, func(lvs ...string) Metric { |
||||||
|
return newHistogram(desc, opts, lvs...) |
||||||
|
}), |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWithLabelValues replaces the method of the same name in
|
||||||
|
// MetricVec. The difference is that this method returns a Histogram and not a
|
||||||
|
// Metric so that no type conversion is required.
|
||||||
|
func (m *HistogramVec) GetMetricWithLabelValues(lvs ...string) (Histogram, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWithLabelValues(lvs...) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Histogram), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWith replaces the method of the same name in MetricVec. The
|
||||||
|
// difference is that this method returns a Histogram and not a Metric so that no
|
||||||
|
// type conversion is required.
|
||||||
|
func (m *HistogramVec) GetMetricWith(labels Labels) (Histogram, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWith(labels) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Histogram), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||||
|
// GetMetricWithLabelValues would have returned an error. By not returning an
|
||||||
|
// error, WithLabelValues allows shortcuts like
|
||||||
|
// myVec.WithLabelValues("404", "GET").Observe(42.21)
|
||||||
|
func (m *HistogramVec) WithLabelValues(lvs ...string) Histogram { |
||||||
|
return m.MetricVec.WithLabelValues(lvs...).(Histogram) |
||||||
|
} |
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||||
|
// returned an error. By not returning an error, With allows shortcuts like
|
||||||
|
// myVec.With(Labels{"code": "404", "method": "GET"}).Observe(42.21)
|
||||||
|
func (m *HistogramVec) With(labels Labels) Histogram { |
||||||
|
return m.MetricVec.With(labels).(Histogram) |
||||||
|
} |
||||||
|
|
||||||
|
type constHistogram struct { |
||||||
|
desc *Desc |
||||||
|
count uint64 |
||||||
|
sum float64 |
||||||
|
buckets map[float64]uint64 |
||||||
|
labelPairs []*dto.LabelPair |
||||||
|
} |
||||||
|
|
||||||
|
func (h *constHistogram) Desc() *Desc { |
||||||
|
return h.desc |
||||||
|
} |
||||||
|
|
||||||
|
func (h *constHistogram) Write(out *dto.Metric) error { |
||||||
|
his := &dto.Histogram{} |
||||||
|
buckets := make([]*dto.Bucket, 0, len(h.buckets)) |
||||||
|
|
||||||
|
his.SampleCount = proto.Uint64(h.count) |
||||||
|
his.SampleSum = proto.Float64(h.sum) |
||||||
|
|
||||||
|
for upperBound, count := range h.buckets { |
||||||
|
buckets = append(buckets, &dto.Bucket{ |
||||||
|
CumulativeCount: proto.Uint64(count), |
||||||
|
UpperBound: proto.Float64(upperBound), |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
if len(buckets) > 0 { |
||||||
|
sort.Sort(buckSort(buckets)) |
||||||
|
} |
||||||
|
his.Bucket = buckets |
||||||
|
|
||||||
|
out.Histogram = his |
||||||
|
out.Label = h.labelPairs |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// NewConstHistogram returns a metric representing a Prometheus histogram with
|
||||||
|
// fixed values for the count, sum, and bucket counts. As those parameters
|
||||||
|
// cannot be changed, the returned value does not implement the Histogram
|
||||||
|
// interface (but only the Metric interface). Users of this package will not
|
||||||
|
// have much use for it in regular operations. However, when implementing custom
|
||||||
|
// Collectors, it is useful as a throw-away metric that is generated on the fly
|
||||||
|
// to send it to Prometheus in the Collect method.
|
||||||
|
//
|
||||||
|
// buckets is a map of upper bounds to cumulative counts, excluding the +Inf
|
||||||
|
// bucket.
|
||||||
|
//
|
||||||
|
// NewConstHistogram returns an error if the length of labelValues is not
|
||||||
|
// consistent with the variable labels in Desc.
|
||||||
|
func NewConstHistogram( |
||||||
|
desc *Desc, |
||||||
|
count uint64, |
||||||
|
sum float64, |
||||||
|
buckets map[float64]uint64, |
||||||
|
labelValues ...string, |
||||||
|
) (Metric, error) { |
||||||
|
if len(desc.variableLabels) != len(labelValues) { |
||||||
|
return nil, errInconsistentCardinality |
||||||
|
} |
||||||
|
return &constHistogram{ |
||||||
|
desc: desc, |
||||||
|
count: count, |
||||||
|
sum: sum, |
||||||
|
buckets: buckets, |
||||||
|
labelPairs: makeLabelPairs(desc, labelValues), |
||||||
|
}, nil |
||||||
|
} |
||||||
|
|
||||||
|
// MustNewConstHistogram is a version of NewConstHistogram that panics where
|
||||||
|
// NewConstMetric would have returned an error.
|
||||||
|
func MustNewConstHistogram( |
||||||
|
desc *Desc, |
||||||
|
count uint64, |
||||||
|
sum float64, |
||||||
|
buckets map[float64]uint64, |
||||||
|
labelValues ...string, |
||||||
|
) Metric { |
||||||
|
m, err := NewConstHistogram(desc, count, sum, buckets, labelValues...) |
||||||
|
if err != nil { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
return m |
||||||
|
} |
||||||
|
|
||||||
|
type buckSort []*dto.Bucket |
||||||
|
|
||||||
|
func (s buckSort) Len() int { |
||||||
|
return len(s) |
||||||
|
} |
||||||
|
|
||||||
|
func (s buckSort) Swap(i, j int) { |
||||||
|
s[i], s[j] = s[j], s[i] |
||||||
|
} |
||||||
|
|
||||||
|
func (s buckSort) Less(i, j int) bool { |
||||||
|
return s[i].GetUpperBound() < s[j].GetUpperBound() |
||||||
|
} |
||||||
@ -0,0 +1,523 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"bufio" |
||||||
|
"bytes" |
||||||
|
"compress/gzip" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"net" |
||||||
|
"net/http" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
"sync" |
||||||
|
"time" |
||||||
|
|
||||||
|
"github.com/prometheus/common/expfmt" |
||||||
|
) |
||||||
|
|
||||||
|
// TODO(beorn7): Remove this whole file. It is a partial mirror of
|
||||||
|
// promhttp/http.go (to avoid circular import chains) where everything HTTP
|
||||||
|
// related should live. The functions here are just for avoiding
|
||||||
|
// breakage. Everything is deprecated.
|
||||||
|
|
||||||
|
const ( |
||||||
|
contentTypeHeader = "Content-Type" |
||||||
|
contentLengthHeader = "Content-Length" |
||||||
|
contentEncodingHeader = "Content-Encoding" |
||||||
|
acceptEncodingHeader = "Accept-Encoding" |
||||||
|
) |
||||||
|
|
||||||
|
var bufPool sync.Pool |
||||||
|
|
||||||
|
func getBuf() *bytes.Buffer { |
||||||
|
buf := bufPool.Get() |
||||||
|
if buf == nil { |
||||||
|
return &bytes.Buffer{} |
||||||
|
} |
||||||
|
return buf.(*bytes.Buffer) |
||||||
|
} |
||||||
|
|
||||||
|
func giveBuf(buf *bytes.Buffer) { |
||||||
|
buf.Reset() |
||||||
|
bufPool.Put(buf) |
||||||
|
} |
||||||
|
|
||||||
|
// Handler returns an HTTP handler for the DefaultGatherer. It is
|
||||||
|
// already instrumented with InstrumentHandler (using "prometheus" as handler
|
||||||
|
// name).
|
||||||
|
//
|
||||||
|
// Deprecated: Please note the issues described in the doc comment of
|
||||||
|
// InstrumentHandler. You might want to consider using promhttp.Handler instead
|
||||||
|
// (which is not instrumented).
|
||||||
|
func Handler() http.Handler { |
||||||
|
return InstrumentHandler("prometheus", UninstrumentedHandler()) |
||||||
|
} |
||||||
|
|
||||||
|
// UninstrumentedHandler returns an HTTP handler for the DefaultGatherer.
|
||||||
|
//
|
||||||
|
// Deprecated: Use promhttp.Handler instead. See there for further documentation.
|
||||||
|
func UninstrumentedHandler() http.Handler { |
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { |
||||||
|
mfs, err := DefaultGatherer.Gather() |
||||||
|
if err != nil { |
||||||
|
http.Error(w, "An error has occurred during metrics collection:\n\n"+err.Error(), http.StatusInternalServerError) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
contentType := expfmt.Negotiate(req.Header) |
||||||
|
buf := getBuf() |
||||||
|
defer giveBuf(buf) |
||||||
|
writer, encoding := decorateWriter(req, buf) |
||||||
|
enc := expfmt.NewEncoder(writer, contentType) |
||||||
|
var lastErr error |
||||||
|
for _, mf := range mfs { |
||||||
|
if err := enc.Encode(mf); err != nil { |
||||||
|
lastErr = err |
||||||
|
http.Error(w, "An error has occurred during metrics encoding:\n\n"+err.Error(), http.StatusInternalServerError) |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
if closer, ok := writer.(io.Closer); ok { |
||||||
|
closer.Close() |
||||||
|
} |
||||||
|
if lastErr != nil && buf.Len() == 0 { |
||||||
|
http.Error(w, "No metrics encoded, last error:\n\n"+err.Error(), http.StatusInternalServerError) |
||||||
|
return |
||||||
|
} |
||||||
|
header := w.Header() |
||||||
|
header.Set(contentTypeHeader, string(contentType)) |
||||||
|
header.Set(contentLengthHeader, fmt.Sprint(buf.Len())) |
||||||
|
if encoding != "" { |
||||||
|
header.Set(contentEncodingHeader, encoding) |
||||||
|
} |
||||||
|
w.Write(buf.Bytes()) |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
// decorateWriter wraps a writer to handle gzip compression if requested. It
|
||||||
|
// returns the decorated writer and the appropriate "Content-Encoding" header
|
||||||
|
// (which is empty if no compression is enabled).
|
||||||
|
func decorateWriter(request *http.Request, writer io.Writer) (io.Writer, string) { |
||||||
|
header := request.Header.Get(acceptEncodingHeader) |
||||||
|
parts := strings.Split(header, ",") |
||||||
|
for _, part := range parts { |
||||||
|
part := strings.TrimSpace(part) |
||||||
|
if part == "gzip" || strings.HasPrefix(part, "gzip;") { |
||||||
|
return gzip.NewWriter(writer), "gzip" |
||||||
|
} |
||||||
|
} |
||||||
|
return writer, "" |
||||||
|
} |
||||||
|
|
||||||
|
var instLabels = []string{"method", "code"} |
||||||
|
|
||||||
|
type nower interface { |
||||||
|
Now() time.Time |
||||||
|
} |
||||||
|
|
||||||
|
type nowFunc func() time.Time |
||||||
|
|
||||||
|
func (n nowFunc) Now() time.Time { |
||||||
|
return n() |
||||||
|
} |
||||||
|
|
||||||
|
var now nower = nowFunc(func() time.Time { |
||||||
|
return time.Now() |
||||||
|
}) |
||||||
|
|
||||||
|
func nowSeries(t ...time.Time) nower { |
||||||
|
return nowFunc(func() time.Time { |
||||||
|
defer func() { |
||||||
|
t = t[1:] |
||||||
|
}() |
||||||
|
|
||||||
|
return t[0] |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
// InstrumentHandler wraps the given HTTP handler for instrumentation. It
|
||||||
|
// registers four metric collectors (if not already done) and reports HTTP
|
||||||
|
// metrics to the (newly or already) registered collectors: http_requests_total
|
||||||
|
// (CounterVec), http_request_duration_microseconds (Summary),
|
||||||
|
// http_request_size_bytes (Summary), http_response_size_bytes (Summary). Each
|
||||||
|
// has a constant label named "handler" with the provided handlerName as
|
||||||
|
// value. http_requests_total is a metric vector partitioned by HTTP method
|
||||||
|
// (label name "method") and HTTP status code (label name "code").
|
||||||
|
//
|
||||||
|
// Deprecated: InstrumentHandler has several issues:
|
||||||
|
//
|
||||||
|
// - It uses Summaries rather than Histograms. Summaries are not useful if
|
||||||
|
// aggregation across multiple instances is required.
|
||||||
|
//
|
||||||
|
// - It uses microseconds as unit, which is deprecated and should be replaced by
|
||||||
|
// seconds.
|
||||||
|
//
|
||||||
|
// - The size of the request is calculated in a separate goroutine. Since this
|
||||||
|
// calculator requires access to the request header, it creates a race with
|
||||||
|
// any writes to the header performed during request handling.
|
||||||
|
// httputil.ReverseProxy is a prominent example for a handler
|
||||||
|
// performing such writes.
|
||||||
|
//
|
||||||
|
// Upcoming versions of this package will provide ways of instrumenting HTTP
|
||||||
|
// handlers that are more flexible and have fewer issues. Please prefer direct
|
||||||
|
// instrumentation in the meantime.
|
||||||
|
func InstrumentHandler(handlerName string, handler http.Handler) http.HandlerFunc { |
||||||
|
return InstrumentHandlerFunc(handlerName, handler.ServeHTTP) |
||||||
|
} |
||||||
|
|
||||||
|
// InstrumentHandlerFunc wraps the given function for instrumentation. It
|
||||||
|
// otherwise works in the same way as InstrumentHandler (and shares the same
|
||||||
|
// issues).
|
||||||
|
//
|
||||||
|
// Deprecated: InstrumentHandlerFunc is deprecated for the same reasons as
|
||||||
|
// InstrumentHandler is.
|
||||||
|
func InstrumentHandlerFunc(handlerName string, handlerFunc func(http.ResponseWriter, *http.Request)) http.HandlerFunc { |
||||||
|
return InstrumentHandlerFuncWithOpts( |
||||||
|
SummaryOpts{ |
||||||
|
Subsystem: "http", |
||||||
|
ConstLabels: Labels{"handler": handlerName}, |
||||||
|
Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001}, |
||||||
|
}, |
||||||
|
handlerFunc, |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
// InstrumentHandlerWithOpts works like InstrumentHandler (and shares the same
|
||||||
|
// issues) but provides more flexibility (at the cost of a more complex call
|
||||||
|
// syntax). As InstrumentHandler, this function registers four metric
|
||||||
|
// collectors, but it uses the provided SummaryOpts to create them. However, the
|
||||||
|
// fields "Name" and "Help" in the SummaryOpts are ignored. "Name" is replaced
|
||||||
|
// by "requests_total", "request_duration_microseconds", "request_size_bytes",
|
||||||
|
// and "response_size_bytes", respectively. "Help" is replaced by an appropriate
|
||||||
|
// help string. The names of the variable labels of the http_requests_total
|
||||||
|
// CounterVec are "method" (get, post, etc.), and "code" (HTTP status code).
|
||||||
|
//
|
||||||
|
// If InstrumentHandlerWithOpts is called as follows, it mimics exactly the
|
||||||
|
// behavior of InstrumentHandler:
|
||||||
|
//
|
||||||
|
// prometheus.InstrumentHandlerWithOpts(
|
||||||
|
// prometheus.SummaryOpts{
|
||||||
|
// Subsystem: "http",
|
||||||
|
// ConstLabels: prometheus.Labels{"handler": handlerName},
|
||||||
|
// },
|
||||||
|
// handler,
|
||||||
|
// )
|
||||||
|
//
|
||||||
|
// Technical detail: "requests_total" is a CounterVec, not a SummaryVec, so it
|
||||||
|
// cannot use SummaryOpts. Instead, a CounterOpts struct is created internally,
|
||||||
|
// and all its fields are set to the equally named fields in the provided
|
||||||
|
// SummaryOpts.
|
||||||
|
//
|
||||||
|
// Deprecated: InstrumentHandlerWithOpts is deprecated for the same reasons as
|
||||||
|
// InstrumentHandler is.
|
||||||
|
func InstrumentHandlerWithOpts(opts SummaryOpts, handler http.Handler) http.HandlerFunc { |
||||||
|
return InstrumentHandlerFuncWithOpts(opts, handler.ServeHTTP) |
||||||
|
} |
||||||
|
|
||||||
|
// InstrumentHandlerFuncWithOpts works like InstrumentHandlerFunc (and shares
|
||||||
|
// the same issues) but provides more flexibility (at the cost of a more complex
|
||||||
|
// call syntax). See InstrumentHandlerWithOpts for details how the provided
|
||||||
|
// SummaryOpts are used.
|
||||||
|
//
|
||||||
|
// Deprecated: InstrumentHandlerFuncWithOpts is deprecated for the same reasons
|
||||||
|
// as InstrumentHandler is.
|
||||||
|
func InstrumentHandlerFuncWithOpts(opts SummaryOpts, handlerFunc func(http.ResponseWriter, *http.Request)) http.HandlerFunc { |
||||||
|
reqCnt := NewCounterVec( |
||||||
|
CounterOpts{ |
||||||
|
Namespace: opts.Namespace, |
||||||
|
Subsystem: opts.Subsystem, |
||||||
|
Name: "requests_total", |
||||||
|
Help: "Total number of HTTP requests made.", |
||||||
|
ConstLabels: opts.ConstLabels, |
||||||
|
}, |
||||||
|
instLabels, |
||||||
|
) |
||||||
|
if err := Register(reqCnt); err != nil { |
||||||
|
if are, ok := err.(AlreadyRegisteredError); ok { |
||||||
|
reqCnt = are.ExistingCollector.(*CounterVec) |
||||||
|
} else { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
opts.Name = "request_duration_microseconds" |
||||||
|
opts.Help = "The HTTP request latencies in microseconds." |
||||||
|
reqDur := NewSummary(opts) |
||||||
|
if err := Register(reqDur); err != nil { |
||||||
|
if are, ok := err.(AlreadyRegisteredError); ok { |
||||||
|
reqDur = are.ExistingCollector.(Summary) |
||||||
|
} else { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
opts.Name = "request_size_bytes" |
||||||
|
opts.Help = "The HTTP request sizes in bytes." |
||||||
|
reqSz := NewSummary(opts) |
||||||
|
if err := Register(reqSz); err != nil { |
||||||
|
if are, ok := err.(AlreadyRegisteredError); ok { |
||||||
|
reqSz = are.ExistingCollector.(Summary) |
||||||
|
} else { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
opts.Name = "response_size_bytes" |
||||||
|
opts.Help = "The HTTP response sizes in bytes." |
||||||
|
resSz := NewSummary(opts) |
||||||
|
if err := Register(resSz); err != nil { |
||||||
|
if are, ok := err.(AlreadyRegisteredError); ok { |
||||||
|
resSz = are.ExistingCollector.(Summary) |
||||||
|
} else { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
||||||
|
now := time.Now() |
||||||
|
|
||||||
|
delegate := &responseWriterDelegator{ResponseWriter: w} |
||||||
|
out := computeApproximateRequestSize(r) |
||||||
|
|
||||||
|
_, cn := w.(http.CloseNotifier) |
||||||
|
_, fl := w.(http.Flusher) |
||||||
|
_, hj := w.(http.Hijacker) |
||||||
|
_, rf := w.(io.ReaderFrom) |
||||||
|
var rw http.ResponseWriter |
||||||
|
if cn && fl && hj && rf { |
||||||
|
rw = &fancyResponseWriterDelegator{delegate} |
||||||
|
} else { |
||||||
|
rw = delegate |
||||||
|
} |
||||||
|
handlerFunc(rw, r) |
||||||
|
|
||||||
|
elapsed := float64(time.Since(now)) / float64(time.Microsecond) |
||||||
|
|
||||||
|
method := sanitizeMethod(r.Method) |
||||||
|
code := sanitizeCode(delegate.status) |
||||||
|
reqCnt.WithLabelValues(method, code).Inc() |
||||||
|
reqDur.Observe(elapsed) |
||||||
|
resSz.Observe(float64(delegate.written)) |
||||||
|
reqSz.Observe(float64(<-out)) |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
func computeApproximateRequestSize(r *http.Request) <-chan int { |
||||||
|
// Get URL length in current go routine for avoiding a race condition.
|
||||||
|
// HandlerFunc that runs in parallel may modify the URL.
|
||||||
|
s := 0 |
||||||
|
if r.URL != nil { |
||||||
|
s += len(r.URL.String()) |
||||||
|
} |
||||||
|
|
||||||
|
out := make(chan int, 1) |
||||||
|
|
||||||
|
go func() { |
||||||
|
s += len(r.Method) |
||||||
|
s += len(r.Proto) |
||||||
|
for name, values := range r.Header { |
||||||
|
s += len(name) |
||||||
|
for _, value := range values { |
||||||
|
s += len(value) |
||||||
|
} |
||||||
|
} |
||||||
|
s += len(r.Host) |
||||||
|
|
||||||
|
// N.B. r.Form and r.MultipartForm are assumed to be included in r.URL.
|
||||||
|
|
||||||
|
if r.ContentLength != -1 { |
||||||
|
s += int(r.ContentLength) |
||||||
|
} |
||||||
|
out <- s |
||||||
|
close(out) |
||||||
|
}() |
||||||
|
|
||||||
|
return out |
||||||
|
} |
||||||
|
|
||||||
|
type responseWriterDelegator struct { |
||||||
|
http.ResponseWriter |
||||||
|
|
||||||
|
handler, method string |
||||||
|
status int |
||||||
|
written int64 |
||||||
|
wroteHeader bool |
||||||
|
} |
||||||
|
|
||||||
|
func (r *responseWriterDelegator) WriteHeader(code int) { |
||||||
|
r.status = code |
||||||
|
r.wroteHeader = true |
||||||
|
r.ResponseWriter.WriteHeader(code) |
||||||
|
} |
||||||
|
|
||||||
|
func (r *responseWriterDelegator) Write(b []byte) (int, error) { |
||||||
|
if !r.wroteHeader { |
||||||
|
r.WriteHeader(http.StatusOK) |
||||||
|
} |
||||||
|
n, err := r.ResponseWriter.Write(b) |
||||||
|
r.written += int64(n) |
||||||
|
return n, err |
||||||
|
} |
||||||
|
|
||||||
|
type fancyResponseWriterDelegator struct { |
||||||
|
*responseWriterDelegator |
||||||
|
} |
||||||
|
|
||||||
|
func (f *fancyResponseWriterDelegator) CloseNotify() <-chan bool { |
||||||
|
return f.ResponseWriter.(http.CloseNotifier).CloseNotify() |
||||||
|
} |
||||||
|
|
||||||
|
func (f *fancyResponseWriterDelegator) Flush() { |
||||||
|
f.ResponseWriter.(http.Flusher).Flush() |
||||||
|
} |
||||||
|
|
||||||
|
func (f *fancyResponseWriterDelegator) Hijack() (net.Conn, *bufio.ReadWriter, error) { |
||||||
|
return f.ResponseWriter.(http.Hijacker).Hijack() |
||||||
|
} |
||||||
|
|
||||||
|
func (f *fancyResponseWriterDelegator) ReadFrom(r io.Reader) (int64, error) { |
||||||
|
if !f.wroteHeader { |
||||||
|
f.WriteHeader(http.StatusOK) |
||||||
|
} |
||||||
|
n, err := f.ResponseWriter.(io.ReaderFrom).ReadFrom(r) |
||||||
|
f.written += n |
||||||
|
return n, err |
||||||
|
} |
||||||
|
|
||||||
|
func sanitizeMethod(m string) string { |
||||||
|
switch m { |
||||||
|
case "GET", "get": |
||||||
|
return "get" |
||||||
|
case "PUT", "put": |
||||||
|
return "put" |
||||||
|
case "HEAD", "head": |
||||||
|
return "head" |
||||||
|
case "POST", "post": |
||||||
|
return "post" |
||||||
|
case "DELETE", "delete": |
||||||
|
return "delete" |
||||||
|
case "CONNECT", "connect": |
||||||
|
return "connect" |
||||||
|
case "OPTIONS", "options": |
||||||
|
return "options" |
||||||
|
case "NOTIFY", "notify": |
||||||
|
return "notify" |
||||||
|
default: |
||||||
|
return strings.ToLower(m) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func sanitizeCode(s int) string { |
||||||
|
switch s { |
||||||
|
case 100: |
||||||
|
return "100" |
||||||
|
case 101: |
||||||
|
return "101" |
||||||
|
|
||||||
|
case 200: |
||||||
|
return "200" |
||||||
|
case 201: |
||||||
|
return "201" |
||||||
|
case 202: |
||||||
|
return "202" |
||||||
|
case 203: |
||||||
|
return "203" |
||||||
|
case 204: |
||||||
|
return "204" |
||||||
|
case 205: |
||||||
|
return "205" |
||||||
|
case 206: |
||||||
|
return "206" |
||||||
|
|
||||||
|
case 300: |
||||||
|
return "300" |
||||||
|
case 301: |
||||||
|
return "301" |
||||||
|
case 302: |
||||||
|
return "302" |
||||||
|
case 304: |
||||||
|
return "304" |
||||||
|
case 305: |
||||||
|
return "305" |
||||||
|
case 307: |
||||||
|
return "307" |
||||||
|
|
||||||
|
case 400: |
||||||
|
return "400" |
||||||
|
case 401: |
||||||
|
return "401" |
||||||
|
case 402: |
||||||
|
return "402" |
||||||
|
case 403: |
||||||
|
return "403" |
||||||
|
case 404: |
||||||
|
return "404" |
||||||
|
case 405: |
||||||
|
return "405" |
||||||
|
case 406: |
||||||
|
return "406" |
||||||
|
case 407: |
||||||
|
return "407" |
||||||
|
case 408: |
||||||
|
return "408" |
||||||
|
case 409: |
||||||
|
return "409" |
||||||
|
case 410: |
||||||
|
return "410" |
||||||
|
case 411: |
||||||
|
return "411" |
||||||
|
case 412: |
||||||
|
return "412" |
||||||
|
case 413: |
||||||
|
return "413" |
||||||
|
case 414: |
||||||
|
return "414" |
||||||
|
case 415: |
||||||
|
return "415" |
||||||
|
case 416: |
||||||
|
return "416" |
||||||
|
case 417: |
||||||
|
return "417" |
||||||
|
case 418: |
||||||
|
return "418" |
||||||
|
|
||||||
|
case 500: |
||||||
|
return "500" |
||||||
|
case 501: |
||||||
|
return "501" |
||||||
|
case 502: |
||||||
|
return "502" |
||||||
|
case 503: |
||||||
|
return "503" |
||||||
|
case 504: |
||||||
|
return "504" |
||||||
|
case 505: |
||||||
|
return "505" |
||||||
|
|
||||||
|
case 428: |
||||||
|
return "428" |
||||||
|
case 429: |
||||||
|
return "429" |
||||||
|
case 431: |
||||||
|
return "431" |
||||||
|
case 511: |
||||||
|
return "511" |
||||||
|
|
||||||
|
default: |
||||||
|
return strconv.Itoa(s) |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,166 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"strings" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
) |
||||||
|
|
||||||
|
const separatorByte byte = 255 |
||||||
|
|
||||||
|
// A Metric models a single sample value with its meta data being exported to
|
||||||
|
// Prometheus. Implementations of Metric in this package are Gauge, Counter,
|
||||||
|
// Histogram, Summary, and Untyped.
|
||||||
|
type Metric interface { |
||||||
|
// Desc returns the descriptor for the Metric. This method idempotently
|
||||||
|
// returns the same descriptor throughout the lifetime of the
|
||||||
|
// Metric. The returned descriptor is immutable by contract. A Metric
|
||||||
|
// unable to describe itself must return an invalid descriptor (created
|
||||||
|
// with NewInvalidDesc).
|
||||||
|
Desc() *Desc |
||||||
|
// Write encodes the Metric into a "Metric" Protocol Buffer data
|
||||||
|
// transmission object.
|
||||||
|
//
|
||||||
|
// Metric implementations must observe concurrency safety as reads of
|
||||||
|
// this metric may occur at any time, and any blocking occurs at the
|
||||||
|
// expense of total performance of rendering all registered
|
||||||
|
// metrics. Ideally, Metric implementations should support concurrent
|
||||||
|
// readers.
|
||||||
|
//
|
||||||
|
// While populating dto.Metric, it is the responsibility of the
|
||||||
|
// implementation to ensure validity of the Metric protobuf (like valid
|
||||||
|
// UTF-8 strings or syntactically valid metric and label names). It is
|
||||||
|
// recommended to sort labels lexicographically. (Implementers may find
|
||||||
|
// LabelPairSorter useful for that.) Callers of Write should still make
|
||||||
|
// sure of sorting if they depend on it.
|
||||||
|
Write(*dto.Metric) error |
||||||
|
// TODO(beorn7): The original rationale of passing in a pre-allocated
|
||||||
|
// dto.Metric protobuf to save allocations has disappeared. The
|
||||||
|
// signature of this method should be changed to "Write() (*dto.Metric,
|
||||||
|
// error)".
|
||||||
|
} |
||||||
|
|
||||||
|
// Opts bundles the options for creating most Metric types. Each metric
|
||||||
|
// implementation XXX has its own XXXOpts type, but in most cases, it is just be
|
||||||
|
// an alias of this type (which might change when the requirement arises.)
|
||||||
|
//
|
||||||
|
// It is mandatory to set Name and Help to a non-empty string. All other fields
|
||||||
|
// are optional and can safely be left at their zero value.
|
||||||
|
type Opts struct { |
||||||
|
// Namespace, Subsystem, and Name are components of the fully-qualified
|
||||||
|
// name of the Metric (created by joining these components with
|
||||||
|
// "_"). Only Name is mandatory, the others merely help structuring the
|
||||||
|
// name. Note that the fully-qualified name of the metric must be a
|
||||||
|
// valid Prometheus metric name.
|
||||||
|
Namespace string |
||||||
|
Subsystem string |
||||||
|
Name string |
||||||
|
|
||||||
|
// Help provides information about this metric. Mandatory!
|
||||||
|
//
|
||||||
|
// Metrics with the same fully-qualified name must have the same Help
|
||||||
|
// string.
|
||||||
|
Help string |
||||||
|
|
||||||
|
// ConstLabels are used to attach fixed labels to this metric. Metrics
|
||||||
|
// with the same fully-qualified name must have the same label names in
|
||||||
|
// their ConstLabels.
|
||||||
|
//
|
||||||
|
// Note that in most cases, labels have a value that varies during the
|
||||||
|
// lifetime of a process. Those labels are usually managed with a metric
|
||||||
|
// vector collector (like CounterVec, GaugeVec, UntypedVec). ConstLabels
|
||||||
|
// serve only special purposes. One is for the special case where the
|
||||||
|
// value of a label does not change during the lifetime of a process,
|
||||||
|
// e.g. if the revision of the running binary is put into a
|
||||||
|
// label. Another, more advanced purpose is if more than one Collector
|
||||||
|
// needs to collect Metrics with the same fully-qualified name. In that
|
||||||
|
// case, those Metrics must differ in the values of their
|
||||||
|
// ConstLabels. See the Collector examples.
|
||||||
|
//
|
||||||
|
// If the value of a label never changes (not even between binaries),
|
||||||
|
// that label most likely should not be a label at all (but part of the
|
||||||
|
// metric name).
|
||||||
|
ConstLabels Labels |
||||||
|
} |
||||||
|
|
||||||
|
// BuildFQName joins the given three name components by "_". Empty name
|
||||||
|
// components are ignored. If the name parameter itself is empty, an empty
|
||||||
|
// string is returned, no matter what. Metric implementations included in this
|
||||||
|
// library use this function internally to generate the fully-qualified metric
|
||||||
|
// name from the name component in their Opts. Users of the library will only
|
||||||
|
// need this function if they implement their own Metric or instantiate a Desc
|
||||||
|
// (with NewDesc) directly.
|
||||||
|
func BuildFQName(namespace, subsystem, name string) string { |
||||||
|
if name == "" { |
||||||
|
return "" |
||||||
|
} |
||||||
|
switch { |
||||||
|
case namespace != "" && subsystem != "": |
||||||
|
return strings.Join([]string{namespace, subsystem, name}, "_") |
||||||
|
case namespace != "": |
||||||
|
return strings.Join([]string{namespace, name}, "_") |
||||||
|
case subsystem != "": |
||||||
|
return strings.Join([]string{subsystem, name}, "_") |
||||||
|
} |
||||||
|
return name |
||||||
|
} |
||||||
|
|
||||||
|
// LabelPairSorter implements sort.Interface. It is used to sort a slice of
|
||||||
|
// dto.LabelPair pointers. This is useful for implementing the Write method of
|
||||||
|
// custom metrics.
|
||||||
|
type LabelPairSorter []*dto.LabelPair |
||||||
|
|
||||||
|
func (s LabelPairSorter) Len() int { |
||||||
|
return len(s) |
||||||
|
} |
||||||
|
|
||||||
|
func (s LabelPairSorter) Swap(i, j int) { |
||||||
|
s[i], s[j] = s[j], s[i] |
||||||
|
} |
||||||
|
|
||||||
|
func (s LabelPairSorter) Less(i, j int) bool { |
||||||
|
return s[i].GetName() < s[j].GetName() |
||||||
|
} |
||||||
|
|
||||||
|
type hashSorter []uint64 |
||||||
|
|
||||||
|
func (s hashSorter) Len() int { |
||||||
|
return len(s) |
||||||
|
} |
||||||
|
|
||||||
|
func (s hashSorter) Swap(i, j int) { |
||||||
|
s[i], s[j] = s[j], s[i] |
||||||
|
} |
||||||
|
|
||||||
|
func (s hashSorter) Less(i, j int) bool { |
||||||
|
return s[i] < s[j] |
||||||
|
} |
||||||
|
|
||||||
|
type invalidMetric struct { |
||||||
|
desc *Desc |
||||||
|
err error |
||||||
|
} |
||||||
|
|
||||||
|
// NewInvalidMetric returns a metric whose Write method always returns the
|
||||||
|
// provided error. It is useful if a Collector finds itself unable to collect
|
||||||
|
// a metric and wishes to report an error to the registry.
|
||||||
|
func NewInvalidMetric(desc *Desc, err error) Metric { |
||||||
|
return &invalidMetric{desc, err} |
||||||
|
} |
||||||
|
|
||||||
|
func (m *invalidMetric) Desc() *Desc { return m.desc } |
||||||
|
|
||||||
|
func (m *invalidMetric) Write(*dto.Metric) error { return m.err } |
||||||
@ -0,0 +1,140 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import "github.com/prometheus/procfs" |
||||||
|
|
||||||
|
type processCollector struct { |
||||||
|
pid int |
||||||
|
collectFn func(chan<- Metric) |
||||||
|
pidFn func() (int, error) |
||||||
|
cpuTotal *Desc |
||||||
|
openFDs, maxFDs *Desc |
||||||
|
vsize, rss *Desc |
||||||
|
startTime *Desc |
||||||
|
} |
||||||
|
|
||||||
|
// NewProcessCollector returns a collector which exports the current state of
|
||||||
|
// process metrics including cpu, memory and file descriptor usage as well as
|
||||||
|
// the process start time for the given process id under the given namespace.
|
||||||
|
func NewProcessCollector(pid int, namespace string) Collector { |
||||||
|
return NewProcessCollectorPIDFn( |
||||||
|
func() (int, error) { return pid, nil }, |
||||||
|
namespace, |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
// NewProcessCollectorPIDFn returns a collector which exports the current state
|
||||||
|
// of process metrics including cpu, memory and file descriptor usage as well
|
||||||
|
// as the process start time under the given namespace. The given pidFn is
|
||||||
|
// called on each collect and is used to determine the process to export
|
||||||
|
// metrics for.
|
||||||
|
func NewProcessCollectorPIDFn( |
||||||
|
pidFn func() (int, error), |
||||||
|
namespace string, |
||||||
|
) Collector { |
||||||
|
ns := "" |
||||||
|
if len(namespace) > 0 { |
||||||
|
ns = namespace + "_" |
||||||
|
} |
||||||
|
|
||||||
|
c := processCollector{ |
||||||
|
pidFn: pidFn, |
||||||
|
collectFn: func(chan<- Metric) {}, |
||||||
|
|
||||||
|
cpuTotal: NewDesc( |
||||||
|
ns+"process_cpu_seconds_total", |
||||||
|
"Total user and system CPU time spent in seconds.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
openFDs: NewDesc( |
||||||
|
ns+"process_open_fds", |
||||||
|
"Number of open file descriptors.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
maxFDs: NewDesc( |
||||||
|
ns+"process_max_fds", |
||||||
|
"Maximum number of open file descriptors.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
vsize: NewDesc( |
||||||
|
ns+"process_virtual_memory_bytes", |
||||||
|
"Virtual memory size in bytes.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
rss: NewDesc( |
||||||
|
ns+"process_resident_memory_bytes", |
||||||
|
"Resident memory size in bytes.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
startTime: NewDesc( |
||||||
|
ns+"process_start_time_seconds", |
||||||
|
"Start time of the process since unix epoch in seconds.", |
||||||
|
nil, nil, |
||||||
|
), |
||||||
|
} |
||||||
|
|
||||||
|
// Set up process metric collection if supported by the runtime.
|
||||||
|
if _, err := procfs.NewStat(); err == nil { |
||||||
|
c.collectFn = c.processCollect |
||||||
|
} |
||||||
|
|
||||||
|
return &c |
||||||
|
} |
||||||
|
|
||||||
|
// Describe returns all descriptions of the collector.
|
||||||
|
func (c *processCollector) Describe(ch chan<- *Desc) { |
||||||
|
ch <- c.cpuTotal |
||||||
|
ch <- c.openFDs |
||||||
|
ch <- c.maxFDs |
||||||
|
ch <- c.vsize |
||||||
|
ch <- c.rss |
||||||
|
ch <- c.startTime |
||||||
|
} |
||||||
|
|
||||||
|
// Collect returns the current state of all metrics of the collector.
|
||||||
|
func (c *processCollector) Collect(ch chan<- Metric) { |
||||||
|
c.collectFn(ch) |
||||||
|
} |
||||||
|
|
||||||
|
// TODO(ts): Bring back error reporting by reverting 7faf9e7 as soon as the
|
||||||
|
// client allows users to configure the error behavior.
|
||||||
|
func (c *processCollector) processCollect(ch chan<- Metric) { |
||||||
|
pid, err := c.pidFn() |
||||||
|
if err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
p, err := procfs.NewProc(pid) |
||||||
|
if err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
if stat, err := p.NewStat(); err == nil { |
||||||
|
ch <- MustNewConstMetric(c.cpuTotal, CounterValue, stat.CPUTime()) |
||||||
|
ch <- MustNewConstMetric(c.vsize, GaugeValue, float64(stat.VirtualMemory())) |
||||||
|
ch <- MustNewConstMetric(c.rss, GaugeValue, float64(stat.ResidentMemory())) |
||||||
|
if startTime, err := stat.StartTime(); err == nil { |
||||||
|
ch <- MustNewConstMetric(c.startTime, GaugeValue, startTime) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if fds, err := p.FileDescriptorsLen(); err == nil { |
||||||
|
ch <- MustNewConstMetric(c.openFDs, GaugeValue, float64(fds)) |
||||||
|
} |
||||||
|
|
||||||
|
if limits, err := p.NewLimits(); err == nil { |
||||||
|
ch <- MustNewConstMetric(c.maxFDs, GaugeValue, float64(limits.OpenFiles)) |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,201 @@ |
|||||||
|
// Copyright 2016 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// Copyright (c) 2013, The Prometheus Authors
|
||||||
|
// All rights reserved.
|
||||||
|
//
|
||||||
|
// Use of this source code is governed by a BSD-style license that can be found
|
||||||
|
// in the LICENSE file.
|
||||||
|
|
||||||
|
// Package promhttp contains functions to create http.Handler instances to
|
||||||
|
// expose Prometheus metrics via HTTP. In later versions of this package, it
|
||||||
|
// will also contain tooling to instrument instances of http.Handler and
|
||||||
|
// http.RoundTripper.
|
||||||
|
//
|
||||||
|
// promhttp.Handler acts on the prometheus.DefaultGatherer. With HandlerFor,
|
||||||
|
// you can create a handler for a custom registry or anything that implements
|
||||||
|
// the Gatherer interface. It also allows to create handlers that act
|
||||||
|
// differently on errors or allow to log errors.
|
||||||
|
package promhttp |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"compress/gzip" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"net/http" |
||||||
|
"strings" |
||||||
|
"sync" |
||||||
|
|
||||||
|
"github.com/prometheus/common/expfmt" |
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus" |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
contentTypeHeader = "Content-Type" |
||||||
|
contentLengthHeader = "Content-Length" |
||||||
|
contentEncodingHeader = "Content-Encoding" |
||||||
|
acceptEncodingHeader = "Accept-Encoding" |
||||||
|
) |
||||||
|
|
||||||
|
var bufPool sync.Pool |
||||||
|
|
||||||
|
func getBuf() *bytes.Buffer { |
||||||
|
buf := bufPool.Get() |
||||||
|
if buf == nil { |
||||||
|
return &bytes.Buffer{} |
||||||
|
} |
||||||
|
return buf.(*bytes.Buffer) |
||||||
|
} |
||||||
|
|
||||||
|
func giveBuf(buf *bytes.Buffer) { |
||||||
|
buf.Reset() |
||||||
|
bufPool.Put(buf) |
||||||
|
} |
||||||
|
|
||||||
|
// Handler returns an HTTP handler for the prometheus.DefaultGatherer. The
|
||||||
|
// Handler uses the default HandlerOpts, i.e. report the first error as an HTTP
|
||||||
|
// error, no error logging, and compression if requested by the client.
|
||||||
|
//
|
||||||
|
// If you want to create a Handler for the DefaultGatherer with different
|
||||||
|
// HandlerOpts, create it with HandlerFor with prometheus.DefaultGatherer and
|
||||||
|
// your desired HandlerOpts.
|
||||||
|
func Handler() http.Handler { |
||||||
|
return HandlerFor(prometheus.DefaultGatherer, HandlerOpts{}) |
||||||
|
} |
||||||
|
|
||||||
|
// HandlerFor returns an http.Handler for the provided Gatherer. The behavior
|
||||||
|
// of the Handler is defined by the provided HandlerOpts.
|
||||||
|
func HandlerFor(reg prometheus.Gatherer, opts HandlerOpts) http.Handler { |
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) { |
||||||
|
mfs, err := reg.Gather() |
||||||
|
if err != nil { |
||||||
|
if opts.ErrorLog != nil { |
||||||
|
opts.ErrorLog.Println("error gathering metrics:", err) |
||||||
|
} |
||||||
|
switch opts.ErrorHandling { |
||||||
|
case PanicOnError: |
||||||
|
panic(err) |
||||||
|
case ContinueOnError: |
||||||
|
if len(mfs) == 0 { |
||||||
|
http.Error(w, "No metrics gathered, last error:\n\n"+err.Error(), http.StatusInternalServerError) |
||||||
|
return |
||||||
|
} |
||||||
|
case HTTPErrorOnError: |
||||||
|
http.Error(w, "An error has occurred during metrics gathering:\n\n"+err.Error(), http.StatusInternalServerError) |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
contentType := expfmt.Negotiate(req.Header) |
||||||
|
buf := getBuf() |
||||||
|
defer giveBuf(buf) |
||||||
|
writer, encoding := decorateWriter(req, buf, opts.DisableCompression) |
||||||
|
enc := expfmt.NewEncoder(writer, contentType) |
||||||
|
var lastErr error |
||||||
|
for _, mf := range mfs { |
||||||
|
if err := enc.Encode(mf); err != nil { |
||||||
|
lastErr = err |
||||||
|
if opts.ErrorLog != nil { |
||||||
|
opts.ErrorLog.Println("error encoding metric family:", err) |
||||||
|
} |
||||||
|
switch opts.ErrorHandling { |
||||||
|
case PanicOnError: |
||||||
|
panic(err) |
||||||
|
case ContinueOnError: |
||||||
|
// Handled later.
|
||||||
|
case HTTPErrorOnError: |
||||||
|
http.Error(w, "An error has occurred during metrics encoding:\n\n"+err.Error(), http.StatusInternalServerError) |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
if closer, ok := writer.(io.Closer); ok { |
||||||
|
closer.Close() |
||||||
|
} |
||||||
|
if lastErr != nil && buf.Len() == 0 { |
||||||
|
http.Error(w, "No metrics encoded, last error:\n\n"+err.Error(), http.StatusInternalServerError) |
||||||
|
return |
||||||
|
} |
||||||
|
header := w.Header() |
||||||
|
header.Set(contentTypeHeader, string(contentType)) |
||||||
|
header.Set(contentLengthHeader, fmt.Sprint(buf.Len())) |
||||||
|
if encoding != "" { |
||||||
|
header.Set(contentEncodingHeader, encoding) |
||||||
|
} |
||||||
|
w.Write(buf.Bytes()) |
||||||
|
// TODO(beorn7): Consider streaming serving of metrics.
|
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
// HandlerErrorHandling defines how a Handler serving metrics will handle
|
||||||
|
// errors.
|
||||||
|
type HandlerErrorHandling int |
||||||
|
|
||||||
|
// These constants cause handlers serving metrics to behave as described if
|
||||||
|
// errors are encountered.
|
||||||
|
const ( |
||||||
|
// Serve an HTTP status code 500 upon the first error
|
||||||
|
// encountered. Report the error message in the body.
|
||||||
|
HTTPErrorOnError HandlerErrorHandling = iota |
||||||
|
// Ignore errors and try to serve as many metrics as possible. However,
|
||||||
|
// if no metrics can be served, serve an HTTP status code 500 and the
|
||||||
|
// last error message in the body. Only use this in deliberate "best
|
||||||
|
// effort" metrics collection scenarios. It is recommended to at least
|
||||||
|
// log errors (by providing an ErrorLog in HandlerOpts) to not mask
|
||||||
|
// errors completely.
|
||||||
|
ContinueOnError |
||||||
|
// Panic upon the first error encountered (useful for "crash only" apps).
|
||||||
|
PanicOnError |
||||||
|
) |
||||||
|
|
||||||
|
// Logger is the minimal interface HandlerOpts needs for logging. Note that
|
||||||
|
// log.Logger from the standard library implements this interface, and it is
|
||||||
|
// easy to implement by custom loggers, if they don't do so already anyway.
|
||||||
|
type Logger interface { |
||||||
|
Println(v ...interface{}) |
||||||
|
} |
||||||
|
|
||||||
|
// HandlerOpts specifies options how to serve metrics via an http.Handler. The
|
||||||
|
// zero value of HandlerOpts is a reasonable default.
|
||||||
|
type HandlerOpts struct { |
||||||
|
// ErrorLog specifies an optional logger for errors collecting and
|
||||||
|
// serving metrics. If nil, errors are not logged at all.
|
||||||
|
ErrorLog Logger |
||||||
|
// ErrorHandling defines how errors are handled. Note that errors are
|
||||||
|
// logged regardless of the configured ErrorHandling provided ErrorLog
|
||||||
|
// is not nil.
|
||||||
|
ErrorHandling HandlerErrorHandling |
||||||
|
// If DisableCompression is true, the handler will never compress the
|
||||||
|
// response, even if requested by the client.
|
||||||
|
DisableCompression bool |
||||||
|
} |
||||||
|
|
||||||
|
// decorateWriter wraps a writer to handle gzip compression if requested. It
|
||||||
|
// returns the decorated writer and the appropriate "Content-Encoding" header
|
||||||
|
// (which is empty if no compression is enabled).
|
||||||
|
func decorateWriter(request *http.Request, writer io.Writer, compressionDisabled bool) (io.Writer, string) { |
||||||
|
if compressionDisabled { |
||||||
|
return writer, "" |
||||||
|
} |
||||||
|
header := request.Header.Get(acceptEncodingHeader) |
||||||
|
parts := strings.Split(header, ",") |
||||||
|
for _, part := range parts { |
||||||
|
part := strings.TrimSpace(part) |
||||||
|
if part == "gzip" || strings.HasPrefix(part, "gzip;") { |
||||||
|
return gzip.NewWriter(writer), "gzip" |
||||||
|
} |
||||||
|
} |
||||||
|
return writer, "" |
||||||
|
} |
||||||
@ -0,0 +1,755 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"os" |
||||||
|
"sort" |
||||||
|
"sync" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
// Capacity for the channel to collect metrics and descriptors.
|
||||||
|
capMetricChan = 1000 |
||||||
|
capDescChan = 10 |
||||||
|
) |
||||||
|
|
||||||
|
// DefaultRegisterer and DefaultGatherer are the implementations of the
|
||||||
|
// Registerer and Gatherer interface a number of convenience functions in this
|
||||||
|
// package act on. Initially, both variables point to the same Registry, which
|
||||||
|
// has a process collector (see NewProcessCollector) and a Go collector (see
|
||||||
|
// NewGoCollector) already registered. This approach to keep default instances
|
||||||
|
// as global state mirrors the approach of other packages in the Go standard
|
||||||
|
// library. Note that there are caveats. Change the variables with caution and
|
||||||
|
// only if you understand the consequences. Users who want to avoid global state
|
||||||
|
// altogether should not use the convenience function and act on custom
|
||||||
|
// instances instead.
|
||||||
|
var ( |
||||||
|
defaultRegistry = NewRegistry() |
||||||
|
DefaultRegisterer Registerer = defaultRegistry |
||||||
|
DefaultGatherer Gatherer = defaultRegistry |
||||||
|
) |
||||||
|
|
||||||
|
func init() { |
||||||
|
MustRegister(NewProcessCollector(os.Getpid(), "")) |
||||||
|
MustRegister(NewGoCollector()) |
||||||
|
} |
||||||
|
|
||||||
|
// NewRegistry creates a new vanilla Registry without any Collectors
|
||||||
|
// pre-registered.
|
||||||
|
func NewRegistry() *Registry { |
||||||
|
return &Registry{ |
||||||
|
collectorsByID: map[uint64]Collector{}, |
||||||
|
descIDs: map[uint64]struct{}{}, |
||||||
|
dimHashesByName: map[string]uint64{}, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// NewPedanticRegistry returns a registry that checks during collection if each
|
||||||
|
// collected Metric is consistent with its reported Desc, and if the Desc has
|
||||||
|
// actually been registered with the registry.
|
||||||
|
//
|
||||||
|
// Usually, a Registry will be happy as long as the union of all collected
|
||||||
|
// Metrics is consistent and valid even if some metrics are not consistent with
|
||||||
|
// their own Desc or a Desc provided by their registered Collector. Well-behaved
|
||||||
|
// Collectors and Metrics will only provide consistent Descs. This Registry is
|
||||||
|
// useful to test the implementation of Collectors and Metrics.
|
||||||
|
func NewPedanticRegistry() *Registry { |
||||||
|
r := NewRegistry() |
||||||
|
r.pedanticChecksEnabled = true |
||||||
|
return r |
||||||
|
} |
||||||
|
|
||||||
|
// Registerer is the interface for the part of a registry in charge of
|
||||||
|
// registering and unregistering. Users of custom registries should use
|
||||||
|
// Registerer as type for registration purposes (rather then the Registry type
|
||||||
|
// directly). In that way, they are free to use custom Registerer implementation
|
||||||
|
// (e.g. for testing purposes).
|
||||||
|
type Registerer interface { |
||||||
|
// Register registers a new Collector to be included in metrics
|
||||||
|
// collection. It returns an error if the descriptors provided by the
|
||||||
|
// Collector are invalid or if they — in combination with descriptors of
|
||||||
|
// already registered Collectors — do not fulfill the consistency and
|
||||||
|
// uniqueness criteria described in the documentation of metric.Desc.
|
||||||
|
//
|
||||||
|
// If the provided Collector is equal to a Collector already registered
|
||||||
|
// (which includes the case of re-registering the same Collector), the
|
||||||
|
// returned error is an instance of AlreadyRegisteredError, which
|
||||||
|
// contains the previously registered Collector.
|
||||||
|
//
|
||||||
|
// It is in general not safe to register the same Collector multiple
|
||||||
|
// times concurrently.
|
||||||
|
Register(Collector) error |
||||||
|
// MustRegister works like Register but registers any number of
|
||||||
|
// Collectors and panics upon the first registration that causes an
|
||||||
|
// error.
|
||||||
|
MustRegister(...Collector) |
||||||
|
// Unregister unregisters the Collector that equals the Collector passed
|
||||||
|
// in as an argument. (Two Collectors are considered equal if their
|
||||||
|
// Describe method yields the same set of descriptors.) The function
|
||||||
|
// returns whether a Collector was unregistered.
|
||||||
|
//
|
||||||
|
// Note that even after unregistering, it will not be possible to
|
||||||
|
// register a new Collector that is inconsistent with the unregistered
|
||||||
|
// Collector, e.g. a Collector collecting metrics with the same name but
|
||||||
|
// a different help string. The rationale here is that the same registry
|
||||||
|
// instance must only collect consistent metrics throughout its
|
||||||
|
// lifetime.
|
||||||
|
Unregister(Collector) bool |
||||||
|
} |
||||||
|
|
||||||
|
// Gatherer is the interface for the part of a registry in charge of gathering
|
||||||
|
// the collected metrics into a number of MetricFamilies. The Gatherer interface
|
||||||
|
// comes with the same general implication as described for the Registerer
|
||||||
|
// interface.
|
||||||
|
type Gatherer interface { |
||||||
|
// Gather calls the Collect method of the registered Collectors and then
|
||||||
|
// gathers the collected metrics into a lexicographically sorted slice
|
||||||
|
// of MetricFamily protobufs. Even if an error occurs, Gather attempts
|
||||||
|
// to gather as many metrics as possible. Hence, if a non-nil error is
|
||||||
|
// returned, the returned MetricFamily slice could be nil (in case of a
|
||||||
|
// fatal error that prevented any meaningful metric collection) or
|
||||||
|
// contain a number of MetricFamily protobufs, some of which might be
|
||||||
|
// incomplete, and some might be missing altogether. The returned error
|
||||||
|
// (which might be a MultiError) explains the details. In scenarios
|
||||||
|
// where complete collection is critical, the returned MetricFamily
|
||||||
|
// protobufs should be disregarded if the returned error is non-nil.
|
||||||
|
Gather() ([]*dto.MetricFamily, error) |
||||||
|
} |
||||||
|
|
||||||
|
// Register registers the provided Collector with the DefaultRegisterer.
|
||||||
|
//
|
||||||
|
// Register is a shortcut for DefaultRegisterer.Register(c). See there for more
|
||||||
|
// details.
|
||||||
|
func Register(c Collector) error { |
||||||
|
return DefaultRegisterer.Register(c) |
||||||
|
} |
||||||
|
|
||||||
|
// MustRegister registers the provided Collectors with the DefaultRegisterer and
|
||||||
|
// panics if any error occurs.
|
||||||
|
//
|
||||||
|
// MustRegister is a shortcut for DefaultRegisterer.MustRegister(cs...). See
|
||||||
|
// there for more details.
|
||||||
|
func MustRegister(cs ...Collector) { |
||||||
|
DefaultRegisterer.MustRegister(cs...) |
||||||
|
} |
||||||
|
|
||||||
|
// Unregister removes the registration of the provided Collector from the
|
||||||
|
// DefaultRegisterer.
|
||||||
|
//
|
||||||
|
// Unregister is a shortcut for DefaultRegisterer.Unregister(c). See there for
|
||||||
|
// more details.
|
||||||
|
func Unregister(c Collector) bool { |
||||||
|
return DefaultRegisterer.Unregister(c) |
||||||
|
} |
||||||
|
|
||||||
|
// GathererFunc turns a function into a Gatherer.
|
||||||
|
type GathererFunc func() ([]*dto.MetricFamily, error) |
||||||
|
|
||||||
|
// Gather implements Gatherer.
|
||||||
|
func (gf GathererFunc) Gather() ([]*dto.MetricFamily, error) { |
||||||
|
return gf() |
||||||
|
} |
||||||
|
|
||||||
|
// AlreadyRegisteredError is returned by the Register method if the Collector to
|
||||||
|
// be registered has already been registered before, or a different Collector
|
||||||
|
// that collects the same metrics has been registered before. Registration fails
|
||||||
|
// in that case, but you can detect from the kind of error what has
|
||||||
|
// happened. The error contains fields for the existing Collector and the
|
||||||
|
// (rejected) new Collector that equals the existing one. This can be used to
|
||||||
|
// find out if an equal Collector has been registered before and switch over to
|
||||||
|
// using the old one, as demonstrated in the example.
|
||||||
|
type AlreadyRegisteredError struct { |
||||||
|
ExistingCollector, NewCollector Collector |
||||||
|
} |
||||||
|
|
||||||
|
func (err AlreadyRegisteredError) Error() string { |
||||||
|
return "duplicate metrics collector registration attempted" |
||||||
|
} |
||||||
|
|
||||||
|
// MultiError is a slice of errors implementing the error interface. It is used
|
||||||
|
// by a Gatherer to report multiple errors during MetricFamily gathering.
|
||||||
|
type MultiError []error |
||||||
|
|
||||||
|
func (errs MultiError) Error() string { |
||||||
|
if len(errs) == 0 { |
||||||
|
return "" |
||||||
|
} |
||||||
|
buf := &bytes.Buffer{} |
||||||
|
fmt.Fprintf(buf, "%d error(s) occurred:", len(errs)) |
||||||
|
for _, err := range errs { |
||||||
|
fmt.Fprintf(buf, "\n* %s", err) |
||||||
|
} |
||||||
|
return buf.String() |
||||||
|
} |
||||||
|
|
||||||
|
// MaybeUnwrap returns nil if len(errs) is 0. It returns the first and only
|
||||||
|
// contained error as error if len(errs is 1). In all other cases, it returns
|
||||||
|
// the MultiError directly. This is helpful for returning a MultiError in a way
|
||||||
|
// that only uses the MultiError if needed.
|
||||||
|
func (errs MultiError) MaybeUnwrap() error { |
||||||
|
switch len(errs) { |
||||||
|
case 0: |
||||||
|
return nil |
||||||
|
case 1: |
||||||
|
return errs[0] |
||||||
|
default: |
||||||
|
return errs |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Registry registers Prometheus collectors, collects their metrics, and gathers
|
||||||
|
// them into MetricFamilies for exposition. It implements both Registerer and
|
||||||
|
// Gatherer. The zero value is not usable. Create instances with NewRegistry or
|
||||||
|
// NewPedanticRegistry.
|
||||||
|
type Registry struct { |
||||||
|
mtx sync.RWMutex |
||||||
|
collectorsByID map[uint64]Collector // ID is a hash of the descIDs.
|
||||||
|
descIDs map[uint64]struct{} |
||||||
|
dimHashesByName map[string]uint64 |
||||||
|
pedanticChecksEnabled bool |
||||||
|
} |
||||||
|
|
||||||
|
// Register implements Registerer.
|
||||||
|
func (r *Registry) Register(c Collector) error { |
||||||
|
var ( |
||||||
|
descChan = make(chan *Desc, capDescChan) |
||||||
|
newDescIDs = map[uint64]struct{}{} |
||||||
|
newDimHashesByName = map[string]uint64{} |
||||||
|
collectorID uint64 // Just a sum of all desc IDs.
|
||||||
|
duplicateDescErr error |
||||||
|
) |
||||||
|
go func() { |
||||||
|
c.Describe(descChan) |
||||||
|
close(descChan) |
||||||
|
}() |
||||||
|
r.mtx.Lock() |
||||||
|
defer r.mtx.Unlock() |
||||||
|
// Coduct various tests...
|
||||||
|
for desc := range descChan { |
||||||
|
|
||||||
|
// Is the descriptor valid at all?
|
||||||
|
if desc.err != nil { |
||||||
|
return fmt.Errorf("descriptor %s is invalid: %s", desc, desc.err) |
||||||
|
} |
||||||
|
|
||||||
|
// Is the descID unique?
|
||||||
|
// (In other words: Is the fqName + constLabel combination unique?)
|
||||||
|
if _, exists := r.descIDs[desc.id]; exists { |
||||||
|
duplicateDescErr = fmt.Errorf("descriptor %s already exists with the same fully-qualified name and const label values", desc) |
||||||
|
} |
||||||
|
// If it is not a duplicate desc in this collector, add it to
|
||||||
|
// the collectorID. (We allow duplicate descs within the same
|
||||||
|
// collector, but their existence must be a no-op.)
|
||||||
|
if _, exists := newDescIDs[desc.id]; !exists { |
||||||
|
newDescIDs[desc.id] = struct{}{} |
||||||
|
collectorID += desc.id |
||||||
|
} |
||||||
|
|
||||||
|
// Are all the label names and the help string consistent with
|
||||||
|
// previous descriptors of the same name?
|
||||||
|
// First check existing descriptors...
|
||||||
|
if dimHash, exists := r.dimHashesByName[desc.fqName]; exists { |
||||||
|
if dimHash != desc.dimHash { |
||||||
|
return fmt.Errorf("a previously registered descriptor with the same fully-qualified name as %s has different label names or a different help string", desc) |
||||||
|
} |
||||||
|
} else { |
||||||
|
// ...then check the new descriptors already seen.
|
||||||
|
if dimHash, exists := newDimHashesByName[desc.fqName]; exists { |
||||||
|
if dimHash != desc.dimHash { |
||||||
|
return fmt.Errorf("descriptors reported by collector have inconsistent label names or help strings for the same fully-qualified name, offender is %s", desc) |
||||||
|
} |
||||||
|
} else { |
||||||
|
newDimHashesByName[desc.fqName] = desc.dimHash |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
// Did anything happen at all?
|
||||||
|
if len(newDescIDs) == 0 { |
||||||
|
return errors.New("collector has no descriptors") |
||||||
|
} |
||||||
|
if existing, exists := r.collectorsByID[collectorID]; exists { |
||||||
|
return AlreadyRegisteredError{ |
||||||
|
ExistingCollector: existing, |
||||||
|
NewCollector: c, |
||||||
|
} |
||||||
|
} |
||||||
|
// If the collectorID is new, but at least one of the descs existed
|
||||||
|
// before, we are in trouble.
|
||||||
|
if duplicateDescErr != nil { |
||||||
|
return duplicateDescErr |
||||||
|
} |
||||||
|
|
||||||
|
// Only after all tests have passed, actually register.
|
||||||
|
r.collectorsByID[collectorID] = c |
||||||
|
for hash := range newDescIDs { |
||||||
|
r.descIDs[hash] = struct{}{} |
||||||
|
} |
||||||
|
for name, dimHash := range newDimHashesByName { |
||||||
|
r.dimHashesByName[name] = dimHash |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Unregister implements Registerer.
|
||||||
|
func (r *Registry) Unregister(c Collector) bool { |
||||||
|
var ( |
||||||
|
descChan = make(chan *Desc, capDescChan) |
||||||
|
descIDs = map[uint64]struct{}{} |
||||||
|
collectorID uint64 // Just a sum of the desc IDs.
|
||||||
|
) |
||||||
|
go func() { |
||||||
|
c.Describe(descChan) |
||||||
|
close(descChan) |
||||||
|
}() |
||||||
|
for desc := range descChan { |
||||||
|
if _, exists := descIDs[desc.id]; !exists { |
||||||
|
collectorID += desc.id |
||||||
|
descIDs[desc.id] = struct{}{} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
r.mtx.RLock() |
||||||
|
if _, exists := r.collectorsByID[collectorID]; !exists { |
||||||
|
r.mtx.RUnlock() |
||||||
|
return false |
||||||
|
} |
||||||
|
r.mtx.RUnlock() |
||||||
|
|
||||||
|
r.mtx.Lock() |
||||||
|
defer r.mtx.Unlock() |
||||||
|
|
||||||
|
delete(r.collectorsByID, collectorID) |
||||||
|
for id := range descIDs { |
||||||
|
delete(r.descIDs, id) |
||||||
|
} |
||||||
|
// dimHashesByName is left untouched as those must be consistent
|
||||||
|
// throughout the lifetime of a program.
|
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// MustRegister implements Registerer.
|
||||||
|
func (r *Registry) MustRegister(cs ...Collector) { |
||||||
|
for _, c := range cs { |
||||||
|
if err := r.Register(c); err != nil { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Gather implements Gatherer.
|
||||||
|
func (r *Registry) Gather() ([]*dto.MetricFamily, error) { |
||||||
|
var ( |
||||||
|
metricChan = make(chan Metric, capMetricChan) |
||||||
|
metricHashes = map[uint64]struct{}{} |
||||||
|
dimHashes = map[string]uint64{} |
||||||
|
wg sync.WaitGroup |
||||||
|
errs MultiError // The collected errors to return in the end.
|
||||||
|
registeredDescIDs map[uint64]struct{} // Only used for pedantic checks
|
||||||
|
) |
||||||
|
|
||||||
|
r.mtx.RLock() |
||||||
|
metricFamiliesByName := make(map[string]*dto.MetricFamily, len(r.dimHashesByName)) |
||||||
|
|
||||||
|
// Scatter.
|
||||||
|
// (Collectors could be complex and slow, so we call them all at once.)
|
||||||
|
wg.Add(len(r.collectorsByID)) |
||||||
|
go func() { |
||||||
|
wg.Wait() |
||||||
|
close(metricChan) |
||||||
|
}() |
||||||
|
for _, collector := range r.collectorsByID { |
||||||
|
go func(collector Collector) { |
||||||
|
defer wg.Done() |
||||||
|
collector.Collect(metricChan) |
||||||
|
}(collector) |
||||||
|
} |
||||||
|
|
||||||
|
// In case pedantic checks are enabled, we have to copy the map before
|
||||||
|
// giving up the RLock.
|
||||||
|
if r.pedanticChecksEnabled { |
||||||
|
registeredDescIDs = make(map[uint64]struct{}, len(r.descIDs)) |
||||||
|
for id := range r.descIDs { |
||||||
|
registeredDescIDs[id] = struct{}{} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
r.mtx.RUnlock() |
||||||
|
|
||||||
|
// Drain metricChan in case of premature return.
|
||||||
|
defer func() { |
||||||
|
for range metricChan { |
||||||
|
} |
||||||
|
}() |
||||||
|
|
||||||
|
// Gather.
|
||||||
|
for metric := range metricChan { |
||||||
|
// This could be done concurrently, too, but it required locking
|
||||||
|
// of metricFamiliesByName (and of metricHashes if checks are
|
||||||
|
// enabled). Most likely not worth it.
|
||||||
|
desc := metric.Desc() |
||||||
|
dtoMetric := &dto.Metric{} |
||||||
|
if err := metric.Write(dtoMetric); err != nil { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"error collecting metric %v: %s", desc, err, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
metricFamily, ok := metricFamiliesByName[desc.fqName] |
||||||
|
if ok { |
||||||
|
if metricFamily.GetHelp() != desc.help { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"collected metric %s %s has help %q but should have %q", |
||||||
|
desc.fqName, dtoMetric, desc.help, metricFamily.GetHelp(), |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
// TODO(beorn7): Simplify switch once Desc has type.
|
||||||
|
switch metricFamily.GetType() { |
||||||
|
case dto.MetricType_COUNTER: |
||||||
|
if dtoMetric.Counter == nil { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"collected metric %s %s should be a Counter", |
||||||
|
desc.fqName, dtoMetric, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
case dto.MetricType_GAUGE: |
||||||
|
if dtoMetric.Gauge == nil { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"collected metric %s %s should be a Gauge", |
||||||
|
desc.fqName, dtoMetric, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
case dto.MetricType_SUMMARY: |
||||||
|
if dtoMetric.Summary == nil { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"collected metric %s %s should be a Summary", |
||||||
|
desc.fqName, dtoMetric, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
case dto.MetricType_UNTYPED: |
||||||
|
if dtoMetric.Untyped == nil { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"collected metric %s %s should be Untyped", |
||||||
|
desc.fqName, dtoMetric, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
case dto.MetricType_HISTOGRAM: |
||||||
|
if dtoMetric.Histogram == nil { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"collected metric %s %s should be a Histogram", |
||||||
|
desc.fqName, dtoMetric, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
default: |
||||||
|
panic("encountered MetricFamily with invalid type") |
||||||
|
} |
||||||
|
} else { |
||||||
|
metricFamily = &dto.MetricFamily{} |
||||||
|
metricFamily.Name = proto.String(desc.fqName) |
||||||
|
metricFamily.Help = proto.String(desc.help) |
||||||
|
// TODO(beorn7): Simplify switch once Desc has type.
|
||||||
|
switch { |
||||||
|
case dtoMetric.Gauge != nil: |
||||||
|
metricFamily.Type = dto.MetricType_GAUGE.Enum() |
||||||
|
case dtoMetric.Counter != nil: |
||||||
|
metricFamily.Type = dto.MetricType_COUNTER.Enum() |
||||||
|
case dtoMetric.Summary != nil: |
||||||
|
metricFamily.Type = dto.MetricType_SUMMARY.Enum() |
||||||
|
case dtoMetric.Untyped != nil: |
||||||
|
metricFamily.Type = dto.MetricType_UNTYPED.Enum() |
||||||
|
case dtoMetric.Histogram != nil: |
||||||
|
metricFamily.Type = dto.MetricType_HISTOGRAM.Enum() |
||||||
|
default: |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"empty metric collected: %s", dtoMetric, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
metricFamiliesByName[desc.fqName] = metricFamily |
||||||
|
} |
||||||
|
if err := checkMetricConsistency(metricFamily, dtoMetric, metricHashes, dimHashes); err != nil { |
||||||
|
errs = append(errs, err) |
||||||
|
continue |
||||||
|
} |
||||||
|
if r.pedanticChecksEnabled { |
||||||
|
// Is the desc registered at all?
|
||||||
|
if _, exist := registeredDescIDs[desc.id]; !exist { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"collected metric %s %s with unregistered descriptor %s", |
||||||
|
metricFamily.GetName(), dtoMetric, desc, |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
if err := checkDescConsistency(metricFamily, dtoMetric, desc); err != nil { |
||||||
|
errs = append(errs, err) |
||||||
|
continue |
||||||
|
} |
||||||
|
} |
||||||
|
metricFamily.Metric = append(metricFamily.Metric, dtoMetric) |
||||||
|
} |
||||||
|
return normalizeMetricFamilies(metricFamiliesByName), errs.MaybeUnwrap() |
||||||
|
} |
||||||
|
|
||||||
|
// Gatherers is a slice of Gatherer instances that implements the Gatherer
|
||||||
|
// interface itself. Its Gather method calls Gather on all Gatherers in the
|
||||||
|
// slice in order and returns the merged results. Errors returned from the
|
||||||
|
// Gather calles are all returned in a flattened MultiError. Duplicate and
|
||||||
|
// inconsistent Metrics are skipped (first occurrence in slice order wins) and
|
||||||
|
// reported in the returned error.
|
||||||
|
//
|
||||||
|
// Gatherers can be used to merge the Gather results from multiple
|
||||||
|
// Registries. It also provides a way to directly inject existing MetricFamily
|
||||||
|
// protobufs into the gathering by creating a custom Gatherer with a Gather
|
||||||
|
// method that simply returns the existing MetricFamily protobufs. Note that no
|
||||||
|
// registration is involved (in contrast to Collector registration), so
|
||||||
|
// obviously registration-time checks cannot happen. Any inconsistencies between
|
||||||
|
// the gathered MetricFamilies are reported as errors by the Gather method, and
|
||||||
|
// inconsistent Metrics are dropped. Invalid parts of the MetricFamilies
|
||||||
|
// (e.g. syntactically invalid metric or label names) will go undetected.
|
||||||
|
type Gatherers []Gatherer |
||||||
|
|
||||||
|
// Gather implements Gatherer.
|
||||||
|
func (gs Gatherers) Gather() ([]*dto.MetricFamily, error) { |
||||||
|
var ( |
||||||
|
metricFamiliesByName = map[string]*dto.MetricFamily{} |
||||||
|
metricHashes = map[uint64]struct{}{} |
||||||
|
dimHashes = map[string]uint64{} |
||||||
|
errs MultiError // The collected errors to return in the end.
|
||||||
|
) |
||||||
|
|
||||||
|
for i, g := range gs { |
||||||
|
mfs, err := g.Gather() |
||||||
|
if err != nil { |
||||||
|
if multiErr, ok := err.(MultiError); ok { |
||||||
|
for _, err := range multiErr { |
||||||
|
errs = append(errs, fmt.Errorf("[from Gatherer #%d] %s", i+1, err)) |
||||||
|
} |
||||||
|
} else { |
||||||
|
errs = append(errs, fmt.Errorf("[from Gatherer #%d] %s", i+1, err)) |
||||||
|
} |
||||||
|
} |
||||||
|
for _, mf := range mfs { |
||||||
|
existingMF, exists := metricFamiliesByName[mf.GetName()] |
||||||
|
if exists { |
||||||
|
if existingMF.GetHelp() != mf.GetHelp() { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"gathered metric family %s has help %q but should have %q", |
||||||
|
mf.GetName(), mf.GetHelp(), existingMF.GetHelp(), |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
if existingMF.GetType() != mf.GetType() { |
||||||
|
errs = append(errs, fmt.Errorf( |
||||||
|
"gathered metric family %s has type %s but should have %s", |
||||||
|
mf.GetName(), mf.GetType(), existingMF.GetType(), |
||||||
|
)) |
||||||
|
continue |
||||||
|
} |
||||||
|
} else { |
||||||
|
existingMF = &dto.MetricFamily{} |
||||||
|
existingMF.Name = mf.Name |
||||||
|
existingMF.Help = mf.Help |
||||||
|
existingMF.Type = mf.Type |
||||||
|
metricFamiliesByName[mf.GetName()] = existingMF |
||||||
|
} |
||||||
|
for _, m := range mf.Metric { |
||||||
|
if err := checkMetricConsistency(existingMF, m, metricHashes, dimHashes); err != nil { |
||||||
|
errs = append(errs, err) |
||||||
|
continue |
||||||
|
} |
||||||
|
existingMF.Metric = append(existingMF.Metric, m) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
return normalizeMetricFamilies(metricFamiliesByName), errs.MaybeUnwrap() |
||||||
|
} |
||||||
|
|
||||||
|
// metricSorter is a sortable slice of *dto.Metric.
|
||||||
|
type metricSorter []*dto.Metric |
||||||
|
|
||||||
|
func (s metricSorter) Len() int { |
||||||
|
return len(s) |
||||||
|
} |
||||||
|
|
||||||
|
func (s metricSorter) Swap(i, j int) { |
||||||
|
s[i], s[j] = s[j], s[i] |
||||||
|
} |
||||||
|
|
||||||
|
func (s metricSorter) Less(i, j int) bool { |
||||||
|
if len(s[i].Label) != len(s[j].Label) { |
||||||
|
// This should not happen. The metrics are
|
||||||
|
// inconsistent. However, we have to deal with the fact, as
|
||||||
|
// people might use custom collectors or metric family injection
|
||||||
|
// to create inconsistent metrics. So let's simply compare the
|
||||||
|
// number of labels in this case. That will still yield
|
||||||
|
// reproducible sorting.
|
||||||
|
return len(s[i].Label) < len(s[j].Label) |
||||||
|
} |
||||||
|
for n, lp := range s[i].Label { |
||||||
|
vi := lp.GetValue() |
||||||
|
vj := s[j].Label[n].GetValue() |
||||||
|
if vi != vj { |
||||||
|
return vi < vj |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// We should never arrive here. Multiple metrics with the same
|
||||||
|
// label set in the same scrape will lead to undefined ingestion
|
||||||
|
// behavior. However, as above, we have to provide stable sorting
|
||||||
|
// here, even for inconsistent metrics. So sort equal metrics
|
||||||
|
// by their timestamp, with missing timestamps (implying "now")
|
||||||
|
// coming last.
|
||||||
|
if s[i].TimestampMs == nil { |
||||||
|
return false |
||||||
|
} |
||||||
|
if s[j].TimestampMs == nil { |
||||||
|
return true |
||||||
|
} |
||||||
|
return s[i].GetTimestampMs() < s[j].GetTimestampMs() |
||||||
|
} |
||||||
|
|
||||||
|
// normalizeMetricFamilies returns a MetricFamily slice with empty
|
||||||
|
// MetricFamilies pruned and the remaining MetricFamilies sorted by name within
|
||||||
|
// the slice, with the contained Metrics sorted within each MetricFamily.
|
||||||
|
func normalizeMetricFamilies(metricFamiliesByName map[string]*dto.MetricFamily) []*dto.MetricFamily { |
||||||
|
for _, mf := range metricFamiliesByName { |
||||||
|
sort.Sort(metricSorter(mf.Metric)) |
||||||
|
} |
||||||
|
names := make([]string, 0, len(metricFamiliesByName)) |
||||||
|
for name, mf := range metricFamiliesByName { |
||||||
|
if len(mf.Metric) > 0 { |
||||||
|
names = append(names, name) |
||||||
|
} |
||||||
|
} |
||||||
|
sort.Strings(names) |
||||||
|
result := make([]*dto.MetricFamily, 0, len(names)) |
||||||
|
for _, name := range names { |
||||||
|
result = append(result, metricFamiliesByName[name]) |
||||||
|
} |
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
// checkMetricConsistency checks if the provided Metric is consistent with the
|
||||||
|
// provided MetricFamily. It also hashed the Metric labels and the MetricFamily
|
||||||
|
// name. If the resulting hash is alread in the provided metricHashes, an error
|
||||||
|
// is returned. If not, it is added to metricHashes. The provided dimHashes maps
|
||||||
|
// MetricFamily names to their dimHash (hashed sorted label names). If dimHashes
|
||||||
|
// doesn't yet contain a hash for the provided MetricFamily, it is
|
||||||
|
// added. Otherwise, an error is returned if the existing dimHashes in not equal
|
||||||
|
// the calculated dimHash.
|
||||||
|
func checkMetricConsistency( |
||||||
|
metricFamily *dto.MetricFamily, |
||||||
|
dtoMetric *dto.Metric, |
||||||
|
metricHashes map[uint64]struct{}, |
||||||
|
dimHashes map[string]uint64, |
||||||
|
) error { |
||||||
|
// Type consistency with metric family.
|
||||||
|
if metricFamily.GetType() == dto.MetricType_GAUGE && dtoMetric.Gauge == nil || |
||||||
|
metricFamily.GetType() == dto.MetricType_COUNTER && dtoMetric.Counter == nil || |
||||||
|
metricFamily.GetType() == dto.MetricType_SUMMARY && dtoMetric.Summary == nil || |
||||||
|
metricFamily.GetType() == dto.MetricType_HISTOGRAM && dtoMetric.Histogram == nil || |
||||||
|
metricFamily.GetType() == dto.MetricType_UNTYPED && dtoMetric.Untyped == nil { |
||||||
|
return fmt.Errorf( |
||||||
|
"collected metric %s %s is not a %s", |
||||||
|
metricFamily.GetName(), dtoMetric, metricFamily.GetType(), |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
// Is the metric unique (i.e. no other metric with the same name and the same label values)?
|
||||||
|
h := hashNew() |
||||||
|
h = hashAdd(h, metricFamily.GetName()) |
||||||
|
h = hashAddByte(h, separatorByte) |
||||||
|
dh := hashNew() |
||||||
|
// Make sure label pairs are sorted. We depend on it for the consistency
|
||||||
|
// check.
|
||||||
|
sort.Sort(LabelPairSorter(dtoMetric.Label)) |
||||||
|
for _, lp := range dtoMetric.Label { |
||||||
|
h = hashAdd(h, lp.GetValue()) |
||||||
|
h = hashAddByte(h, separatorByte) |
||||||
|
dh = hashAdd(dh, lp.GetName()) |
||||||
|
dh = hashAddByte(dh, separatorByte) |
||||||
|
} |
||||||
|
if _, exists := metricHashes[h]; exists { |
||||||
|
return fmt.Errorf( |
||||||
|
"collected metric %s %s was collected before with the same name and label values", |
||||||
|
metricFamily.GetName(), dtoMetric, |
||||||
|
) |
||||||
|
} |
||||||
|
if dimHash, ok := dimHashes[metricFamily.GetName()]; ok { |
||||||
|
if dimHash != dh { |
||||||
|
return fmt.Errorf( |
||||||
|
"collected metric %s %s has label dimensions inconsistent with previously collected metrics in the same metric family", |
||||||
|
metricFamily.GetName(), dtoMetric, |
||||||
|
) |
||||||
|
} |
||||||
|
} else { |
||||||
|
dimHashes[metricFamily.GetName()] = dh |
||||||
|
} |
||||||
|
metricHashes[h] = struct{}{} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func checkDescConsistency( |
||||||
|
metricFamily *dto.MetricFamily, |
||||||
|
dtoMetric *dto.Metric, |
||||||
|
desc *Desc, |
||||||
|
) error { |
||||||
|
// Desc help consistency with metric family help.
|
||||||
|
if metricFamily.GetHelp() != desc.help { |
||||||
|
return fmt.Errorf( |
||||||
|
"collected metric %s %s has help %q but should have %q", |
||||||
|
metricFamily.GetName(), dtoMetric, metricFamily.GetHelp(), desc.help, |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
// Is the desc consistent with the content of the metric?
|
||||||
|
lpsFromDesc := make([]*dto.LabelPair, 0, len(dtoMetric.Label)) |
||||||
|
lpsFromDesc = append(lpsFromDesc, desc.constLabelPairs...) |
||||||
|
for _, l := range desc.variableLabels { |
||||||
|
lpsFromDesc = append(lpsFromDesc, &dto.LabelPair{ |
||||||
|
Name: proto.String(l), |
||||||
|
}) |
||||||
|
} |
||||||
|
if len(lpsFromDesc) != len(dtoMetric.Label) { |
||||||
|
return fmt.Errorf( |
||||||
|
"labels in collected metric %s %s are inconsistent with descriptor %s", |
||||||
|
metricFamily.GetName(), dtoMetric, desc, |
||||||
|
) |
||||||
|
} |
||||||
|
sort.Sort(LabelPairSorter(lpsFromDesc)) |
||||||
|
for i, lpFromDesc := range lpsFromDesc { |
||||||
|
lpFromMetric := dtoMetric.Label[i] |
||||||
|
if lpFromDesc.GetName() != lpFromMetric.GetName() || |
||||||
|
lpFromDesc.Value != nil && lpFromDesc.GetValue() != lpFromMetric.GetValue() { |
||||||
|
return fmt.Errorf( |
||||||
|
"labels in collected metric %s %s are inconsistent with descriptor %s", |
||||||
|
metricFamily.GetName(), dtoMetric, desc, |
||||||
|
) |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
@ -0,0 +1,543 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"math" |
||||||
|
"sort" |
||||||
|
"sync" |
||||||
|
"time" |
||||||
|
|
||||||
|
"github.com/beorn7/perks/quantile" |
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
) |
||||||
|
|
||||||
|
// quantileLabel is used for the label that defines the quantile in a
|
||||||
|
// summary.
|
||||||
|
const quantileLabel = "quantile" |
||||||
|
|
||||||
|
// A Summary captures individual observations from an event or sample stream and
|
||||||
|
// summarizes them in a manner similar to traditional summary statistics: 1. sum
|
||||||
|
// of observations, 2. observation count, 3. rank estimations.
|
||||||
|
//
|
||||||
|
// A typical use-case is the observation of request latencies. By default, a
|
||||||
|
// Summary provides the median, the 90th and the 99th percentile of the latency
|
||||||
|
// as rank estimations.
|
||||||
|
//
|
||||||
|
// Note that the rank estimations cannot be aggregated in a meaningful way with
|
||||||
|
// the Prometheus query language (i.e. you cannot average or add them). If you
|
||||||
|
// need aggregatable quantiles (e.g. you want the 99th percentile latency of all
|
||||||
|
// queries served across all instances of a service), consider the Histogram
|
||||||
|
// metric type. See the Prometheus documentation for more details.
|
||||||
|
//
|
||||||
|
// To create Summary instances, use NewSummary.
|
||||||
|
type Summary interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
|
||||||
|
// Observe adds a single observation to the summary.
|
||||||
|
Observe(float64) |
||||||
|
} |
||||||
|
|
||||||
|
// DefObjectives are the default Summary quantile values.
|
||||||
|
//
|
||||||
|
// Deprecated: DefObjectives will not be used as the default objectives in
|
||||||
|
// v0.10 of the library. The default Summary will have no quantiles then.
|
||||||
|
var ( |
||||||
|
DefObjectives = map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001} |
||||||
|
|
||||||
|
errQuantileLabelNotAllowed = fmt.Errorf( |
||||||
|
"%q is not allowed as label name in summaries", quantileLabel, |
||||||
|
) |
||||||
|
) |
||||||
|
|
||||||
|
// Default values for SummaryOpts.
|
||||||
|
const ( |
||||||
|
// DefMaxAge is the default duration for which observations stay
|
||||||
|
// relevant.
|
||||||
|
DefMaxAge time.Duration = 10 * time.Minute |
||||||
|
// DefAgeBuckets is the default number of buckets used to calculate the
|
||||||
|
// age of observations.
|
||||||
|
DefAgeBuckets = 5 |
||||||
|
// DefBufCap is the standard buffer size for collecting Summary observations.
|
||||||
|
DefBufCap = 500 |
||||||
|
) |
||||||
|
|
||||||
|
// SummaryOpts bundles the options for creating a Summary metric. It is
|
||||||
|
// mandatory to set Name and Help to a non-empty string. All other fields are
|
||||||
|
// optional and can safely be left at their zero value.
|
||||||
|
type SummaryOpts struct { |
||||||
|
// Namespace, Subsystem, and Name are components of the fully-qualified
|
||||||
|
// name of the Summary (created by joining these components with
|
||||||
|
// "_"). Only Name is mandatory, the others merely help structuring the
|
||||||
|
// name. Note that the fully-qualified name of the Summary must be a
|
||||||
|
// valid Prometheus metric name.
|
||||||
|
Namespace string |
||||||
|
Subsystem string |
||||||
|
Name string |
||||||
|
|
||||||
|
// Help provides information about this Summary. Mandatory!
|
||||||
|
//
|
||||||
|
// Metrics with the same fully-qualified name must have the same Help
|
||||||
|
// string.
|
||||||
|
Help string |
||||||
|
|
||||||
|
// ConstLabels are used to attach fixed labels to this
|
||||||
|
// Summary. Summaries with the same fully-qualified name must have the
|
||||||
|
// same label names in their ConstLabels.
|
||||||
|
//
|
||||||
|
// Note that in most cases, labels have a value that varies during the
|
||||||
|
// lifetime of a process. Those labels are usually managed with a
|
||||||
|
// SummaryVec. ConstLabels serve only special purposes. One is for the
|
||||||
|
// special case where the value of a label does not change during the
|
||||||
|
// lifetime of a process, e.g. if the revision of the running binary is
|
||||||
|
// put into a label. Another, more advanced purpose is if more than one
|
||||||
|
// Collector needs to collect Summaries with the same fully-qualified
|
||||||
|
// name. In that case, those Summaries must differ in the values of
|
||||||
|
// their ConstLabels. See the Collector examples.
|
||||||
|
//
|
||||||
|
// If the value of a label never changes (not even between binaries),
|
||||||
|
// that label most likely should not be a label at all (but part of the
|
||||||
|
// metric name).
|
||||||
|
ConstLabels Labels |
||||||
|
|
||||||
|
// Objectives defines the quantile rank estimates with their respective
|
||||||
|
// absolute error. If Objectives[q] = e, then the value reported for q
|
||||||
|
// will be the φ-quantile value for some φ between q-e and q+e. The
|
||||||
|
// default value is DefObjectives. It is used if Objectives is left at
|
||||||
|
// its zero value (i.e. nil). To create a Summary without Objectives,
|
||||||
|
// set it to an empty map (i.e. map[float64]float64{}).
|
||||||
|
//
|
||||||
|
// Deprecated: Note that the current value of DefObjectives is
|
||||||
|
// deprecated. It will be replaced by an empty map in v0.10 of the
|
||||||
|
// library. Please explicitly set Objectives to the desired value.
|
||||||
|
Objectives map[float64]float64 |
||||||
|
|
||||||
|
// MaxAge defines the duration for which an observation stays relevant
|
||||||
|
// for the summary. Must be positive. The default value is DefMaxAge.
|
||||||
|
MaxAge time.Duration |
||||||
|
|
||||||
|
// AgeBuckets is the number of buckets used to exclude observations that
|
||||||
|
// are older than MaxAge from the summary. A higher number has a
|
||||||
|
// resource penalty, so only increase it if the higher resolution is
|
||||||
|
// really required. For very high observation rates, you might want to
|
||||||
|
// reduce the number of age buckets. With only one age bucket, you will
|
||||||
|
// effectively see a complete reset of the summary each time MaxAge has
|
||||||
|
// passed. The default value is DefAgeBuckets.
|
||||||
|
AgeBuckets uint32 |
||||||
|
|
||||||
|
// BufCap defines the default sample stream buffer size. The default
|
||||||
|
// value of DefBufCap should suffice for most uses. If there is a need
|
||||||
|
// to increase the value, a multiple of 500 is recommended (because that
|
||||||
|
// is the internal buffer size of the underlying package
|
||||||
|
// "github.com/bmizerany/perks/quantile").
|
||||||
|
BufCap uint32 |
||||||
|
} |
||||||
|
|
||||||
|
// Great fuck-up with the sliding-window decay algorithm... The Merge method of
|
||||||
|
// perk/quantile is actually not working as advertised - and it might be
|
||||||
|
// unfixable, as the underlying algorithm is apparently not capable of merging
|
||||||
|
// summaries in the first place. To avoid using Merge, we are currently adding
|
||||||
|
// observations to _each_ age bucket, i.e. the effort to add a sample is
|
||||||
|
// essentially multiplied by the number of age buckets. When rotating age
|
||||||
|
// buckets, we empty the previous head stream. On scrape time, we simply take
|
||||||
|
// the quantiles from the head stream (no merging required). Result: More effort
|
||||||
|
// on observation time, less effort on scrape time, which is exactly the
|
||||||
|
// opposite of what we try to accomplish, but at least the results are correct.
|
||||||
|
//
|
||||||
|
// The quite elegant previous contraption to merge the age buckets efficiently
|
||||||
|
// on scrape time (see code up commit 6b9530d72ea715f0ba612c0120e6e09fbf1d49d0)
|
||||||
|
// can't be used anymore.
|
||||||
|
|
||||||
|
// NewSummary creates a new Summary based on the provided SummaryOpts.
|
||||||
|
func NewSummary(opts SummaryOpts) Summary { |
||||||
|
return newSummary( |
||||||
|
NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
), |
||||||
|
opts, |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
func newSummary(desc *Desc, opts SummaryOpts, labelValues ...string) Summary { |
||||||
|
if len(desc.variableLabels) != len(labelValues) { |
||||||
|
panic(errInconsistentCardinality) |
||||||
|
} |
||||||
|
|
||||||
|
for _, n := range desc.variableLabels { |
||||||
|
if n == quantileLabel { |
||||||
|
panic(errQuantileLabelNotAllowed) |
||||||
|
} |
||||||
|
} |
||||||
|
for _, lp := range desc.constLabelPairs { |
||||||
|
if lp.GetName() == quantileLabel { |
||||||
|
panic(errQuantileLabelNotAllowed) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
if opts.Objectives == nil { |
||||||
|
opts.Objectives = DefObjectives |
||||||
|
} |
||||||
|
|
||||||
|
if opts.MaxAge < 0 { |
||||||
|
panic(fmt.Errorf("illegal max age MaxAge=%v", opts.MaxAge)) |
||||||
|
} |
||||||
|
if opts.MaxAge == 0 { |
||||||
|
opts.MaxAge = DefMaxAge |
||||||
|
} |
||||||
|
|
||||||
|
if opts.AgeBuckets == 0 { |
||||||
|
opts.AgeBuckets = DefAgeBuckets |
||||||
|
} |
||||||
|
|
||||||
|
if opts.BufCap == 0 { |
||||||
|
opts.BufCap = DefBufCap |
||||||
|
} |
||||||
|
|
||||||
|
s := &summary{ |
||||||
|
desc: desc, |
||||||
|
|
||||||
|
objectives: opts.Objectives, |
||||||
|
sortedObjectives: make([]float64, 0, len(opts.Objectives)), |
||||||
|
|
||||||
|
labelPairs: makeLabelPairs(desc, labelValues), |
||||||
|
|
||||||
|
hotBuf: make([]float64, 0, opts.BufCap), |
||||||
|
coldBuf: make([]float64, 0, opts.BufCap), |
||||||
|
streamDuration: opts.MaxAge / time.Duration(opts.AgeBuckets), |
||||||
|
} |
||||||
|
s.headStreamExpTime = time.Now().Add(s.streamDuration) |
||||||
|
s.hotBufExpTime = s.headStreamExpTime |
||||||
|
|
||||||
|
for i := uint32(0); i < opts.AgeBuckets; i++ { |
||||||
|
s.streams = append(s.streams, s.newStream()) |
||||||
|
} |
||||||
|
s.headStream = s.streams[0] |
||||||
|
|
||||||
|
for qu := range s.objectives { |
||||||
|
s.sortedObjectives = append(s.sortedObjectives, qu) |
||||||
|
} |
||||||
|
sort.Float64s(s.sortedObjectives) |
||||||
|
|
||||||
|
s.init(s) // Init self-collection.
|
||||||
|
return s |
||||||
|
} |
||||||
|
|
||||||
|
type summary struct { |
||||||
|
selfCollector |
||||||
|
|
||||||
|
bufMtx sync.Mutex // Protects hotBuf and hotBufExpTime.
|
||||||
|
mtx sync.Mutex // Protects every other moving part.
|
||||||
|
// Lock bufMtx before mtx if both are needed.
|
||||||
|
|
||||||
|
desc *Desc |
||||||
|
|
||||||
|
objectives map[float64]float64 |
||||||
|
sortedObjectives []float64 |
||||||
|
|
||||||
|
labelPairs []*dto.LabelPair |
||||||
|
|
||||||
|
sum float64 |
||||||
|
cnt uint64 |
||||||
|
|
||||||
|
hotBuf, coldBuf []float64 |
||||||
|
|
||||||
|
streams []*quantile.Stream |
||||||
|
streamDuration time.Duration |
||||||
|
headStream *quantile.Stream |
||||||
|
headStreamIdx int |
||||||
|
headStreamExpTime, hotBufExpTime time.Time |
||||||
|
} |
||||||
|
|
||||||
|
func (s *summary) Desc() *Desc { |
||||||
|
return s.desc |
||||||
|
} |
||||||
|
|
||||||
|
func (s *summary) Observe(v float64) { |
||||||
|
s.bufMtx.Lock() |
||||||
|
defer s.bufMtx.Unlock() |
||||||
|
|
||||||
|
now := time.Now() |
||||||
|
if now.After(s.hotBufExpTime) { |
||||||
|
s.asyncFlush(now) |
||||||
|
} |
||||||
|
s.hotBuf = append(s.hotBuf, v) |
||||||
|
if len(s.hotBuf) == cap(s.hotBuf) { |
||||||
|
s.asyncFlush(now) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (s *summary) Write(out *dto.Metric) error { |
||||||
|
sum := &dto.Summary{} |
||||||
|
qs := make([]*dto.Quantile, 0, len(s.objectives)) |
||||||
|
|
||||||
|
s.bufMtx.Lock() |
||||||
|
s.mtx.Lock() |
||||||
|
// Swap bufs even if hotBuf is empty to set new hotBufExpTime.
|
||||||
|
s.swapBufs(time.Now()) |
||||||
|
s.bufMtx.Unlock() |
||||||
|
|
||||||
|
s.flushColdBuf() |
||||||
|
sum.SampleCount = proto.Uint64(s.cnt) |
||||||
|
sum.SampleSum = proto.Float64(s.sum) |
||||||
|
|
||||||
|
for _, rank := range s.sortedObjectives { |
||||||
|
var q float64 |
||||||
|
if s.headStream.Count() == 0 { |
||||||
|
q = math.NaN() |
||||||
|
} else { |
||||||
|
q = s.headStream.Query(rank) |
||||||
|
} |
||||||
|
qs = append(qs, &dto.Quantile{ |
||||||
|
Quantile: proto.Float64(rank), |
||||||
|
Value: proto.Float64(q), |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
s.mtx.Unlock() |
||||||
|
|
||||||
|
if len(qs) > 0 { |
||||||
|
sort.Sort(quantSort(qs)) |
||||||
|
} |
||||||
|
sum.Quantile = qs |
||||||
|
|
||||||
|
out.Summary = sum |
||||||
|
out.Label = s.labelPairs |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (s *summary) newStream() *quantile.Stream { |
||||||
|
return quantile.NewTargeted(s.objectives) |
||||||
|
} |
||||||
|
|
||||||
|
// asyncFlush needs bufMtx locked.
|
||||||
|
func (s *summary) asyncFlush(now time.Time) { |
||||||
|
s.mtx.Lock() |
||||||
|
s.swapBufs(now) |
||||||
|
|
||||||
|
// Unblock the original goroutine that was responsible for the mutation
|
||||||
|
// that triggered the compaction. But hold onto the global non-buffer
|
||||||
|
// state mutex until the operation finishes.
|
||||||
|
go func() { |
||||||
|
s.flushColdBuf() |
||||||
|
s.mtx.Unlock() |
||||||
|
}() |
||||||
|
} |
||||||
|
|
||||||
|
// rotateStreams needs mtx AND bufMtx locked.
|
||||||
|
func (s *summary) maybeRotateStreams() { |
||||||
|
for !s.hotBufExpTime.Equal(s.headStreamExpTime) { |
||||||
|
s.headStream.Reset() |
||||||
|
s.headStreamIdx++ |
||||||
|
if s.headStreamIdx >= len(s.streams) { |
||||||
|
s.headStreamIdx = 0 |
||||||
|
} |
||||||
|
s.headStream = s.streams[s.headStreamIdx] |
||||||
|
s.headStreamExpTime = s.headStreamExpTime.Add(s.streamDuration) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// flushColdBuf needs mtx locked.
|
||||||
|
func (s *summary) flushColdBuf() { |
||||||
|
for _, v := range s.coldBuf { |
||||||
|
for _, stream := range s.streams { |
||||||
|
stream.Insert(v) |
||||||
|
} |
||||||
|
s.cnt++ |
||||||
|
s.sum += v |
||||||
|
} |
||||||
|
s.coldBuf = s.coldBuf[0:0] |
||||||
|
s.maybeRotateStreams() |
||||||
|
} |
||||||
|
|
||||||
|
// swapBufs needs mtx AND bufMtx locked, coldBuf must be empty.
|
||||||
|
func (s *summary) swapBufs(now time.Time) { |
||||||
|
if len(s.coldBuf) != 0 { |
||||||
|
panic("coldBuf is not empty") |
||||||
|
} |
||||||
|
s.hotBuf, s.coldBuf = s.coldBuf, s.hotBuf |
||||||
|
// hotBuf is now empty and gets new expiration set.
|
||||||
|
for now.After(s.hotBufExpTime) { |
||||||
|
s.hotBufExpTime = s.hotBufExpTime.Add(s.streamDuration) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
type quantSort []*dto.Quantile |
||||||
|
|
||||||
|
func (s quantSort) Len() int { |
||||||
|
return len(s) |
||||||
|
} |
||||||
|
|
||||||
|
func (s quantSort) Swap(i, j int) { |
||||||
|
s[i], s[j] = s[j], s[i] |
||||||
|
} |
||||||
|
|
||||||
|
func (s quantSort) Less(i, j int) bool { |
||||||
|
return s[i].GetQuantile() < s[j].GetQuantile() |
||||||
|
} |
||||||
|
|
||||||
|
// SummaryVec is a Collector that bundles a set of Summaries that all share the
|
||||||
|
// same Desc, but have different values for their variable labels. This is used
|
||||||
|
// if you want to count the same thing partitioned by various dimensions
|
||||||
|
// (e.g. HTTP request latencies, partitioned by status code and method). Create
|
||||||
|
// instances with NewSummaryVec.
|
||||||
|
type SummaryVec struct { |
||||||
|
*MetricVec |
||||||
|
} |
||||||
|
|
||||||
|
// NewSummaryVec creates a new SummaryVec based on the provided SummaryOpts and
|
||||||
|
// partitioned by the given label names. At least one label name must be
|
||||||
|
// provided.
|
||||||
|
func NewSummaryVec(opts SummaryOpts, labelNames []string) *SummaryVec { |
||||||
|
desc := NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
labelNames, |
||||||
|
opts.ConstLabels, |
||||||
|
) |
||||||
|
return &SummaryVec{ |
||||||
|
MetricVec: newMetricVec(desc, func(lvs ...string) Metric { |
||||||
|
return newSummary(desc, opts, lvs...) |
||||||
|
}), |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWithLabelValues replaces the method of the same name in
|
||||||
|
// MetricVec. The difference is that this method returns a Summary and not a
|
||||||
|
// Metric so that no type conversion is required.
|
||||||
|
func (m *SummaryVec) GetMetricWithLabelValues(lvs ...string) (Summary, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWithLabelValues(lvs...) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Summary), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWith replaces the method of the same name in MetricVec. The
|
||||||
|
// difference is that this method returns a Summary and not a Metric so that no
|
||||||
|
// type conversion is required.
|
||||||
|
func (m *SummaryVec) GetMetricWith(labels Labels) (Summary, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWith(labels) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Summary), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||||
|
// GetMetricWithLabelValues would have returned an error. By not returning an
|
||||||
|
// error, WithLabelValues allows shortcuts like
|
||||||
|
// myVec.WithLabelValues("404", "GET").Observe(42.21)
|
||||||
|
func (m *SummaryVec) WithLabelValues(lvs ...string) Summary { |
||||||
|
return m.MetricVec.WithLabelValues(lvs...).(Summary) |
||||||
|
} |
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||||
|
// returned an error. By not returning an error, With allows shortcuts like
|
||||||
|
// myVec.With(Labels{"code": "404", "method": "GET"}).Observe(42.21)
|
||||||
|
func (m *SummaryVec) With(labels Labels) Summary { |
||||||
|
return m.MetricVec.With(labels).(Summary) |
||||||
|
} |
||||||
|
|
||||||
|
type constSummary struct { |
||||||
|
desc *Desc |
||||||
|
count uint64 |
||||||
|
sum float64 |
||||||
|
quantiles map[float64]float64 |
||||||
|
labelPairs []*dto.LabelPair |
||||||
|
} |
||||||
|
|
||||||
|
func (s *constSummary) Desc() *Desc { |
||||||
|
return s.desc |
||||||
|
} |
||||||
|
|
||||||
|
func (s *constSummary) Write(out *dto.Metric) error { |
||||||
|
sum := &dto.Summary{} |
||||||
|
qs := make([]*dto.Quantile, 0, len(s.quantiles)) |
||||||
|
|
||||||
|
sum.SampleCount = proto.Uint64(s.count) |
||||||
|
sum.SampleSum = proto.Float64(s.sum) |
||||||
|
|
||||||
|
for rank, q := range s.quantiles { |
||||||
|
qs = append(qs, &dto.Quantile{ |
||||||
|
Quantile: proto.Float64(rank), |
||||||
|
Value: proto.Float64(q), |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
if len(qs) > 0 { |
||||||
|
sort.Sort(quantSort(qs)) |
||||||
|
} |
||||||
|
sum.Quantile = qs |
||||||
|
|
||||||
|
out.Summary = sum |
||||||
|
out.Label = s.labelPairs |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// NewConstSummary returns a metric representing a Prometheus summary with fixed
|
||||||
|
// values for the count, sum, and quantiles. As those parameters cannot be
|
||||||
|
// changed, the returned value does not implement the Summary interface (but
|
||||||
|
// only the Metric interface). Users of this package will not have much use for
|
||||||
|
// it in regular operations. However, when implementing custom Collectors, it is
|
||||||
|
// useful as a throw-away metric that is generated on the fly to send it to
|
||||||
|
// Prometheus in the Collect method.
|
||||||
|
//
|
||||||
|
// quantiles maps ranks to quantile values. For example, a median latency of
|
||||||
|
// 0.23s and a 99th percentile latency of 0.56s would be expressed as:
|
||||||
|
// map[float64]float64{0.5: 0.23, 0.99: 0.56}
|
||||||
|
//
|
||||||
|
// NewConstSummary returns an error if the length of labelValues is not
|
||||||
|
// consistent with the variable labels in Desc.
|
||||||
|
func NewConstSummary( |
||||||
|
desc *Desc, |
||||||
|
count uint64, |
||||||
|
sum float64, |
||||||
|
quantiles map[float64]float64, |
||||||
|
labelValues ...string, |
||||||
|
) (Metric, error) { |
||||||
|
if len(desc.variableLabels) != len(labelValues) { |
||||||
|
return nil, errInconsistentCardinality |
||||||
|
} |
||||||
|
return &constSummary{ |
||||||
|
desc: desc, |
||||||
|
count: count, |
||||||
|
sum: sum, |
||||||
|
quantiles: quantiles, |
||||||
|
labelPairs: makeLabelPairs(desc, labelValues), |
||||||
|
}, nil |
||||||
|
} |
||||||
|
|
||||||
|
// MustNewConstSummary is a version of NewConstSummary that panics where
|
||||||
|
// NewConstMetric would have returned an error.
|
||||||
|
func MustNewConstSummary( |
||||||
|
desc *Desc, |
||||||
|
count uint64, |
||||||
|
sum float64, |
||||||
|
quantiles map[float64]float64, |
||||||
|
labelValues ...string, |
||||||
|
) Metric { |
||||||
|
m, err := NewConstSummary(desc, count, sum, quantiles, labelValues...) |
||||||
|
if err != nil { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
return m |
||||||
|
} |
||||||
@ -0,0 +1,74 @@ |
|||||||
|
// Copyright 2016 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import "time" |
||||||
|
|
||||||
|
// Observer is the interface that wraps the Observe method, which is used by
|
||||||
|
// Histogram and Summary to add observations.
|
||||||
|
type Observer interface { |
||||||
|
Observe(float64) |
||||||
|
} |
||||||
|
|
||||||
|
// The ObserverFunc type is an adapter to allow the use of ordinary
|
||||||
|
// functions as Observers. If f is a function with the appropriate
|
||||||
|
// signature, ObserverFunc(f) is an Observer that calls f.
|
||||||
|
//
|
||||||
|
// This adapter is usually used in connection with the Timer type, and there are
|
||||||
|
// two general use cases:
|
||||||
|
//
|
||||||
|
// The most common one is to use a Gauge as the Observer for a Timer.
|
||||||
|
// See the "Gauge" Timer example.
|
||||||
|
//
|
||||||
|
// The more advanced use case is to create a function that dynamically decides
|
||||||
|
// which Observer to use for observing the duration. See the "Complex" Timer
|
||||||
|
// example.
|
||||||
|
type ObserverFunc func(float64) |
||||||
|
|
||||||
|
// Observe calls f(value). It implements Observer.
|
||||||
|
func (f ObserverFunc) Observe(value float64) { |
||||||
|
f(value) |
||||||
|
} |
||||||
|
|
||||||
|
// Timer is a helper type to time functions. Use NewTimer to create new
|
||||||
|
// instances.
|
||||||
|
type Timer struct { |
||||||
|
begin time.Time |
||||||
|
observer Observer |
||||||
|
} |
||||||
|
|
||||||
|
// NewTimer creates a new Timer. The provided Observer is used to observe a
|
||||||
|
// duration in seconds. Timer is usually used to time a function call in the
|
||||||
|
// following way:
|
||||||
|
// func TimeMe() {
|
||||||
|
// timer := NewTimer(myHistogram)
|
||||||
|
// defer timer.ObserveDuration()
|
||||||
|
// // Do actual work.
|
||||||
|
// }
|
||||||
|
func NewTimer(o Observer) *Timer { |
||||||
|
return &Timer{ |
||||||
|
begin: time.Now(), |
||||||
|
observer: o, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// ObserveDuration records the duration passed since the Timer was created with
|
||||||
|
// NewTimer. It calls the Observe method of the Observer provided during
|
||||||
|
// construction with the duration in seconds as an argument. ObserveDuration is
|
||||||
|
// usually called with a defer statement.
|
||||||
|
func (t *Timer) ObserveDuration() { |
||||||
|
if t.observer != nil { |
||||||
|
t.observer.Observe(time.Since(t.begin).Seconds()) |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,143 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
// Untyped is a Metric that represents a single numerical value that can
|
||||||
|
// arbitrarily go up and down.
|
||||||
|
//
|
||||||
|
// An Untyped metric works the same as a Gauge. The only difference is that to
|
||||||
|
// no type information is implied.
|
||||||
|
//
|
||||||
|
// To create Untyped instances, use NewUntyped.
|
||||||
|
//
|
||||||
|
// Deprecated: The Untyped type is deprecated because it doesn't make sense in
|
||||||
|
// direct instrumentation. If you need to mirror an external metric of unknown
|
||||||
|
// type (usually while writing exporters), Use MustNewConstMetric to create an
|
||||||
|
// untyped metric instance on the fly.
|
||||||
|
type Untyped interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
|
||||||
|
// Set sets the Untyped metric to an arbitrary value.
|
||||||
|
Set(float64) |
||||||
|
// Inc increments the Untyped metric by 1.
|
||||||
|
Inc() |
||||||
|
// Dec decrements the Untyped metric by 1.
|
||||||
|
Dec() |
||||||
|
// Add adds the given value to the Untyped metric. (The value can be
|
||||||
|
// negative, resulting in a decrease.)
|
||||||
|
Add(float64) |
||||||
|
// Sub subtracts the given value from the Untyped metric. (The value can
|
||||||
|
// be negative, resulting in an increase.)
|
||||||
|
Sub(float64) |
||||||
|
} |
||||||
|
|
||||||
|
// UntypedOpts is an alias for Opts. See there for doc comments.
|
||||||
|
type UntypedOpts Opts |
||||||
|
|
||||||
|
// NewUntyped creates a new Untyped metric from the provided UntypedOpts.
|
||||||
|
func NewUntyped(opts UntypedOpts) Untyped { |
||||||
|
return newValue(NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
), UntypedValue, 0) |
||||||
|
} |
||||||
|
|
||||||
|
// UntypedVec is a Collector that bundles a set of Untyped metrics that all
|
||||||
|
// share the same Desc, but have different values for their variable
|
||||||
|
// labels. This is used if you want to count the same thing partitioned by
|
||||||
|
// various dimensions. Create instances with NewUntypedVec.
|
||||||
|
type UntypedVec struct { |
||||||
|
*MetricVec |
||||||
|
} |
||||||
|
|
||||||
|
// NewUntypedVec creates a new UntypedVec based on the provided UntypedOpts and
|
||||||
|
// partitioned by the given label names. At least one label name must be
|
||||||
|
// provided.
|
||||||
|
func NewUntypedVec(opts UntypedOpts, labelNames []string) *UntypedVec { |
||||||
|
desc := NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
labelNames, |
||||||
|
opts.ConstLabels, |
||||||
|
) |
||||||
|
return &UntypedVec{ |
||||||
|
MetricVec: newMetricVec(desc, func(lvs ...string) Metric { |
||||||
|
return newValue(desc, UntypedValue, 0, lvs...) |
||||||
|
}), |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWithLabelValues replaces the method of the same name in
|
||||||
|
// MetricVec. The difference is that this method returns an Untyped and not a
|
||||||
|
// Metric so that no type conversion is required.
|
||||||
|
func (m *UntypedVec) GetMetricWithLabelValues(lvs ...string) (Untyped, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWithLabelValues(lvs...) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Untyped), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWith replaces the method of the same name in MetricVec. The
|
||||||
|
// difference is that this method returns an Untyped and not a Metric so that no
|
||||||
|
// type conversion is required.
|
||||||
|
func (m *UntypedVec) GetMetricWith(labels Labels) (Untyped, error) { |
||||||
|
metric, err := m.MetricVec.GetMetricWith(labels) |
||||||
|
if metric != nil { |
||||||
|
return metric.(Untyped), err |
||||||
|
} |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics where
|
||||||
|
// GetMetricWithLabelValues would have returned an error. By not returning an
|
||||||
|
// error, WithLabelValues allows shortcuts like
|
||||||
|
// myVec.WithLabelValues("404", "GET").Add(42)
|
||||||
|
func (m *UntypedVec) WithLabelValues(lvs ...string) Untyped { |
||||||
|
return m.MetricVec.WithLabelValues(lvs...).(Untyped) |
||||||
|
} |
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics where GetMetricWithLabels would have
|
||||||
|
// returned an error. By not returning an error, With allows shortcuts like
|
||||||
|
// myVec.With(Labels{"code": "404", "method": "GET"}).Add(42)
|
||||||
|
func (m *UntypedVec) With(labels Labels) Untyped { |
||||||
|
return m.MetricVec.With(labels).(Untyped) |
||||||
|
} |
||||||
|
|
||||||
|
// UntypedFunc is an Untyped whose value is determined at collect time by
|
||||||
|
// calling a provided function.
|
||||||
|
//
|
||||||
|
// To create UntypedFunc instances, use NewUntypedFunc.
|
||||||
|
type UntypedFunc interface { |
||||||
|
Metric |
||||||
|
Collector |
||||||
|
} |
||||||
|
|
||||||
|
// NewUntypedFunc creates a new UntypedFunc based on the provided
|
||||||
|
// UntypedOpts. The value reported is determined by calling the given function
|
||||||
|
// from within the Write method. Take into account that metric collection may
|
||||||
|
// happen concurrently. If that results in concurrent calls to Write, like in
|
||||||
|
// the case where an UntypedFunc is directly registered with Prometheus, the
|
||||||
|
// provided function must be concurrency-safe.
|
||||||
|
func NewUntypedFunc(opts UntypedOpts, function func() float64) UntypedFunc { |
||||||
|
return newValueFunc(NewDesc( |
||||||
|
BuildFQName(opts.Namespace, opts.Subsystem, opts.Name), |
||||||
|
opts.Help, |
||||||
|
nil, |
||||||
|
opts.ConstLabels, |
||||||
|
), UntypedValue, function) |
||||||
|
} |
||||||
@ -0,0 +1,239 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"math" |
||||||
|
"sort" |
||||||
|
"sync/atomic" |
||||||
|
"time" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
) |
||||||
|
|
||||||
|
// ValueType is an enumeration of metric types that represent a simple value.
|
||||||
|
type ValueType int |
||||||
|
|
||||||
|
// Possible values for the ValueType enum.
|
||||||
|
const ( |
||||||
|
_ ValueType = iota |
||||||
|
CounterValue |
||||||
|
GaugeValue |
||||||
|
UntypedValue |
||||||
|
) |
||||||
|
|
||||||
|
var errInconsistentCardinality = errors.New("inconsistent label cardinality") |
||||||
|
|
||||||
|
// value is a generic metric for simple values. It implements Metric, Collector,
|
||||||
|
// Counter, Gauge, and Untyped. Its effective type is determined by
|
||||||
|
// ValueType. This is a low-level building block used by the library to back the
|
||||||
|
// implementations of Counter, Gauge, and Untyped.
|
||||||
|
type value struct { |
||||||
|
// valBits containst the bits of the represented float64 value. It has
|
||||||
|
// to go first in the struct to guarantee alignment for atomic
|
||||||
|
// operations. http://golang.org/pkg/sync/atomic/#pkg-note-BUG
|
||||||
|
valBits uint64 |
||||||
|
|
||||||
|
selfCollector |
||||||
|
|
||||||
|
desc *Desc |
||||||
|
valType ValueType |
||||||
|
labelPairs []*dto.LabelPair |
||||||
|
} |
||||||
|
|
||||||
|
// newValue returns a newly allocated value with the given Desc, ValueType,
|
||||||
|
// sample value and label values. It panics if the number of label
|
||||||
|
// values is different from the number of variable labels in Desc.
|
||||||
|
func newValue(desc *Desc, valueType ValueType, val float64, labelValues ...string) *value { |
||||||
|
if len(labelValues) != len(desc.variableLabels) { |
||||||
|
panic(errInconsistentCardinality) |
||||||
|
} |
||||||
|
result := &value{ |
||||||
|
desc: desc, |
||||||
|
valType: valueType, |
||||||
|
valBits: math.Float64bits(val), |
||||||
|
labelPairs: makeLabelPairs(desc, labelValues), |
||||||
|
} |
||||||
|
result.init(result) |
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) Desc() *Desc { |
||||||
|
return v.desc |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) Set(val float64) { |
||||||
|
atomic.StoreUint64(&v.valBits, math.Float64bits(val)) |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) SetToCurrentTime() { |
||||||
|
v.Set(float64(time.Now().UnixNano()) / 1e9) |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) Inc() { |
||||||
|
v.Add(1) |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) Dec() { |
||||||
|
v.Add(-1) |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) Add(val float64) { |
||||||
|
for { |
||||||
|
oldBits := atomic.LoadUint64(&v.valBits) |
||||||
|
newBits := math.Float64bits(math.Float64frombits(oldBits) + val) |
||||||
|
if atomic.CompareAndSwapUint64(&v.valBits, oldBits, newBits) { |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) Sub(val float64) { |
||||||
|
v.Add(val * -1) |
||||||
|
} |
||||||
|
|
||||||
|
func (v *value) Write(out *dto.Metric) error { |
||||||
|
val := math.Float64frombits(atomic.LoadUint64(&v.valBits)) |
||||||
|
return populateMetric(v.valType, val, v.labelPairs, out) |
||||||
|
} |
||||||
|
|
||||||
|
// valueFunc is a generic metric for simple values retrieved on collect time
|
||||||
|
// from a function. It implements Metric and Collector. Its effective type is
|
||||||
|
// determined by ValueType. This is a low-level building block used by the
|
||||||
|
// library to back the implementations of CounterFunc, GaugeFunc, and
|
||||||
|
// UntypedFunc.
|
||||||
|
type valueFunc struct { |
||||||
|
selfCollector |
||||||
|
|
||||||
|
desc *Desc |
||||||
|
valType ValueType |
||||||
|
function func() float64 |
||||||
|
labelPairs []*dto.LabelPair |
||||||
|
} |
||||||
|
|
||||||
|
// newValueFunc returns a newly allocated valueFunc with the given Desc and
|
||||||
|
// ValueType. The value reported is determined by calling the given function
|
||||||
|
// from within the Write method. Take into account that metric collection may
|
||||||
|
// happen concurrently. If that results in concurrent calls to Write, like in
|
||||||
|
// the case where a valueFunc is directly registered with Prometheus, the
|
||||||
|
// provided function must be concurrency-safe.
|
||||||
|
func newValueFunc(desc *Desc, valueType ValueType, function func() float64) *valueFunc { |
||||||
|
result := &valueFunc{ |
||||||
|
desc: desc, |
||||||
|
valType: valueType, |
||||||
|
function: function, |
||||||
|
labelPairs: makeLabelPairs(desc, nil), |
||||||
|
} |
||||||
|
result.init(result) |
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
func (v *valueFunc) Desc() *Desc { |
||||||
|
return v.desc |
||||||
|
} |
||||||
|
|
||||||
|
func (v *valueFunc) Write(out *dto.Metric) error { |
||||||
|
return populateMetric(v.valType, v.function(), v.labelPairs, out) |
||||||
|
} |
||||||
|
|
||||||
|
// NewConstMetric returns a metric with one fixed value that cannot be
|
||||||
|
// changed. Users of this package will not have much use for it in regular
|
||||||
|
// operations. However, when implementing custom Collectors, it is useful as a
|
||||||
|
// throw-away metric that is generated on the fly to send it to Prometheus in
|
||||||
|
// the Collect method. NewConstMetric returns an error if the length of
|
||||||
|
// labelValues is not consistent with the variable labels in Desc.
|
||||||
|
func NewConstMetric(desc *Desc, valueType ValueType, value float64, labelValues ...string) (Metric, error) { |
||||||
|
if len(desc.variableLabels) != len(labelValues) { |
||||||
|
return nil, errInconsistentCardinality |
||||||
|
} |
||||||
|
return &constMetric{ |
||||||
|
desc: desc, |
||||||
|
valType: valueType, |
||||||
|
val: value, |
||||||
|
labelPairs: makeLabelPairs(desc, labelValues), |
||||||
|
}, nil |
||||||
|
} |
||||||
|
|
||||||
|
// MustNewConstMetric is a version of NewConstMetric that panics where
|
||||||
|
// NewConstMetric would have returned an error.
|
||||||
|
func MustNewConstMetric(desc *Desc, valueType ValueType, value float64, labelValues ...string) Metric { |
||||||
|
m, err := NewConstMetric(desc, valueType, value, labelValues...) |
||||||
|
if err != nil { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
return m |
||||||
|
} |
||||||
|
|
||||||
|
type constMetric struct { |
||||||
|
desc *Desc |
||||||
|
valType ValueType |
||||||
|
val float64 |
||||||
|
labelPairs []*dto.LabelPair |
||||||
|
} |
||||||
|
|
||||||
|
func (m *constMetric) Desc() *Desc { |
||||||
|
return m.desc |
||||||
|
} |
||||||
|
|
||||||
|
func (m *constMetric) Write(out *dto.Metric) error { |
||||||
|
return populateMetric(m.valType, m.val, m.labelPairs, out) |
||||||
|
} |
||||||
|
|
||||||
|
func populateMetric( |
||||||
|
t ValueType, |
||||||
|
v float64, |
||||||
|
labelPairs []*dto.LabelPair, |
||||||
|
m *dto.Metric, |
||||||
|
) error { |
||||||
|
m.Label = labelPairs |
||||||
|
switch t { |
||||||
|
case CounterValue: |
||||||
|
m.Counter = &dto.Counter{Value: proto.Float64(v)} |
||||||
|
case GaugeValue: |
||||||
|
m.Gauge = &dto.Gauge{Value: proto.Float64(v)} |
||||||
|
case UntypedValue: |
||||||
|
m.Untyped = &dto.Untyped{Value: proto.Float64(v)} |
||||||
|
default: |
||||||
|
return fmt.Errorf("encountered unknown type %v", t) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func makeLabelPairs(desc *Desc, labelValues []string) []*dto.LabelPair { |
||||||
|
totalLen := len(desc.variableLabels) + len(desc.constLabelPairs) |
||||||
|
if totalLen == 0 { |
||||||
|
// Super fast path.
|
||||||
|
return nil |
||||||
|
} |
||||||
|
if len(desc.variableLabels) == 0 { |
||||||
|
// Moderately fast path.
|
||||||
|
return desc.constLabelPairs |
||||||
|
} |
||||||
|
labelPairs := make([]*dto.LabelPair, 0, totalLen) |
||||||
|
for i, n := range desc.variableLabels { |
||||||
|
labelPairs = append(labelPairs, &dto.LabelPair{ |
||||||
|
Name: proto.String(n), |
||||||
|
Value: proto.String(labelValues[i]), |
||||||
|
}) |
||||||
|
} |
||||||
|
for _, lp := range desc.constLabelPairs { |
||||||
|
labelPairs = append(labelPairs, lp) |
||||||
|
} |
||||||
|
sort.Sort(LabelPairSorter(labelPairs)) |
||||||
|
return labelPairs |
||||||
|
} |
||||||
@ -0,0 +1,404 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package prometheus |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"sync" |
||||||
|
|
||||||
|
"github.com/prometheus/common/model" |
||||||
|
) |
||||||
|
|
||||||
|
// MetricVec is a Collector to bundle metrics of the same name that
|
||||||
|
// differ in their label values. MetricVec is usually not used directly but as a
|
||||||
|
// building block for implementations of vectors of a given metric
|
||||||
|
// type. GaugeVec, CounterVec, SummaryVec, and UntypedVec are examples already
|
||||||
|
// provided in this package.
|
||||||
|
type MetricVec struct { |
||||||
|
mtx sync.RWMutex // Protects the children.
|
||||||
|
children map[uint64][]metricWithLabelValues |
||||||
|
desc *Desc |
||||||
|
|
||||||
|
newMetric func(labelValues ...string) Metric |
||||||
|
hashAdd func(h uint64, s string) uint64 // replace hash function for testing collision handling
|
||||||
|
hashAddByte func(h uint64, b byte) uint64 |
||||||
|
} |
||||||
|
|
||||||
|
// newMetricVec returns an initialized MetricVec. The concrete value is
|
||||||
|
// returned for embedding into another struct.
|
||||||
|
func newMetricVec(desc *Desc, newMetric func(lvs ...string) Metric) *MetricVec { |
||||||
|
return &MetricVec{ |
||||||
|
children: map[uint64][]metricWithLabelValues{}, |
||||||
|
desc: desc, |
||||||
|
newMetric: newMetric, |
||||||
|
hashAdd: hashAdd, |
||||||
|
hashAddByte: hashAddByte, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// metricWithLabelValues provides the metric and its label values for
|
||||||
|
// disambiguation on hash collision.
|
||||||
|
type metricWithLabelValues struct { |
||||||
|
values []string |
||||||
|
metric Metric |
||||||
|
} |
||||||
|
|
||||||
|
// Describe implements Collector. The length of the returned slice
|
||||||
|
// is always one.
|
||||||
|
func (m *MetricVec) Describe(ch chan<- *Desc) { |
||||||
|
ch <- m.desc |
||||||
|
} |
||||||
|
|
||||||
|
// Collect implements Collector.
|
||||||
|
func (m *MetricVec) Collect(ch chan<- Metric) { |
||||||
|
m.mtx.RLock() |
||||||
|
defer m.mtx.RUnlock() |
||||||
|
|
||||||
|
for _, metrics := range m.children { |
||||||
|
for _, metric := range metrics { |
||||||
|
ch <- metric.metric |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWithLabelValues returns the Metric for the given slice of label
|
||||||
|
// values (same order as the VariableLabels in Desc). If that combination of
|
||||||
|
// label values is accessed for the first time, a new Metric is created.
|
||||||
|
//
|
||||||
|
// It is possible to call this method without using the returned Metric to only
|
||||||
|
// create the new Metric but leave it at its start value (e.g. a Summary or
|
||||||
|
// Histogram without any observations). See also the SummaryVec example.
|
||||||
|
//
|
||||||
|
// Keeping the Metric for later use is possible (and should be considered if
|
||||||
|
// performance is critical), but keep in mind that Reset, DeleteLabelValues and
|
||||||
|
// Delete can be used to delete the Metric from the MetricVec. In that case, the
|
||||||
|
// Metric will still exist, but it will not be exported anymore, even if a
|
||||||
|
// Metric with the same label values is created later. See also the CounterVec
|
||||||
|
// example.
|
||||||
|
//
|
||||||
|
// An error is returned if the number of label values is not the same as the
|
||||||
|
// number of VariableLabels in Desc.
|
||||||
|
//
|
||||||
|
// Note that for more than one label value, this method is prone to mistakes
|
||||||
|
// caused by an incorrect order of arguments. Consider GetMetricWith(Labels) as
|
||||||
|
// an alternative to avoid that type of mistake. For higher label numbers, the
|
||||||
|
// latter has a much more readable (albeit more verbose) syntax, but it comes
|
||||||
|
// with a performance overhead (for creating and processing the Labels map).
|
||||||
|
// See also the GaugeVec example.
|
||||||
|
func (m *MetricVec) GetMetricWithLabelValues(lvs ...string) (Metric, error) { |
||||||
|
h, err := m.hashLabelValues(lvs) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
return m.getOrCreateMetricWithLabelValues(h, lvs), nil |
||||||
|
} |
||||||
|
|
||||||
|
// GetMetricWith returns the Metric for the given Labels map (the label names
|
||||||
|
// must match those of the VariableLabels in Desc). If that label map is
|
||||||
|
// accessed for the first time, a new Metric is created. Implications of
|
||||||
|
// creating a Metric without using it and keeping the Metric for later use are
|
||||||
|
// the same as for GetMetricWithLabelValues.
|
||||||
|
//
|
||||||
|
// An error is returned if the number and names of the Labels are inconsistent
|
||||||
|
// with those of the VariableLabels in Desc.
|
||||||
|
//
|
||||||
|
// This method is used for the same purpose as
|
||||||
|
// GetMetricWithLabelValues(...string). See there for pros and cons of the two
|
||||||
|
// methods.
|
||||||
|
func (m *MetricVec) GetMetricWith(labels Labels) (Metric, error) { |
||||||
|
h, err := m.hashLabels(labels) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
return m.getOrCreateMetricWithLabels(h, labels), nil |
||||||
|
} |
||||||
|
|
||||||
|
// WithLabelValues works as GetMetricWithLabelValues, but panics if an error
|
||||||
|
// occurs. The method allows neat syntax like:
|
||||||
|
// httpReqs.WithLabelValues("404", "POST").Inc()
|
||||||
|
func (m *MetricVec) WithLabelValues(lvs ...string) Metric { |
||||||
|
metric, err := m.GetMetricWithLabelValues(lvs...) |
||||||
|
if err != nil { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
return metric |
||||||
|
} |
||||||
|
|
||||||
|
// With works as GetMetricWith, but panics if an error occurs. The method allows
|
||||||
|
// neat syntax like:
|
||||||
|
// httpReqs.With(Labels{"status":"404", "method":"POST"}).Inc()
|
||||||
|
func (m *MetricVec) With(labels Labels) Metric { |
||||||
|
metric, err := m.GetMetricWith(labels) |
||||||
|
if err != nil { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
return metric |
||||||
|
} |
||||||
|
|
||||||
|
// DeleteLabelValues removes the metric where the variable labels are the same
|
||||||
|
// as those passed in as labels (same order as the VariableLabels in Desc). It
|
||||||
|
// returns true if a metric was deleted.
|
||||||
|
//
|
||||||
|
// It is not an error if the number of label values is not the same as the
|
||||||
|
// number of VariableLabels in Desc. However, such inconsistent label count can
|
||||||
|
// never match an actual Metric, so the method will always return false in that
|
||||||
|
// case.
|
||||||
|
//
|
||||||
|
// Note that for more than one label value, this method is prone to mistakes
|
||||||
|
// caused by an incorrect order of arguments. Consider Delete(Labels) as an
|
||||||
|
// alternative to avoid that type of mistake. For higher label numbers, the
|
||||||
|
// latter has a much more readable (albeit more verbose) syntax, but it comes
|
||||||
|
// with a performance overhead (for creating and processing the Labels map).
|
||||||
|
// See also the CounterVec example.
|
||||||
|
func (m *MetricVec) DeleteLabelValues(lvs ...string) bool { |
||||||
|
m.mtx.Lock() |
||||||
|
defer m.mtx.Unlock() |
||||||
|
|
||||||
|
h, err := m.hashLabelValues(lvs) |
||||||
|
if err != nil { |
||||||
|
return false |
||||||
|
} |
||||||
|
return m.deleteByHashWithLabelValues(h, lvs) |
||||||
|
} |
||||||
|
|
||||||
|
// Delete deletes the metric where the variable labels are the same as those
|
||||||
|
// passed in as labels. It returns true if a metric was deleted.
|
||||||
|
//
|
||||||
|
// It is not an error if the number and names of the Labels are inconsistent
|
||||||
|
// with those of the VariableLabels in the Desc of the MetricVec. However, such
|
||||||
|
// inconsistent Labels can never match an actual Metric, so the method will
|
||||||
|
// always return false in that case.
|
||||||
|
//
|
||||||
|
// This method is used for the same purpose as DeleteLabelValues(...string). See
|
||||||
|
// there for pros and cons of the two methods.
|
||||||
|
func (m *MetricVec) Delete(labels Labels) bool { |
||||||
|
m.mtx.Lock() |
||||||
|
defer m.mtx.Unlock() |
||||||
|
|
||||||
|
h, err := m.hashLabels(labels) |
||||||
|
if err != nil { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
return m.deleteByHashWithLabels(h, labels) |
||||||
|
} |
||||||
|
|
||||||
|
// deleteByHashWithLabelValues removes the metric from the hash bucket h. If
|
||||||
|
// there are multiple matches in the bucket, use lvs to select a metric and
|
||||||
|
// remove only that metric.
|
||||||
|
func (m *MetricVec) deleteByHashWithLabelValues(h uint64, lvs []string) bool { |
||||||
|
metrics, ok := m.children[h] |
||||||
|
if !ok { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
i := m.findMetricWithLabelValues(metrics, lvs) |
||||||
|
if i >= len(metrics) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
if len(metrics) > 1 { |
||||||
|
m.children[h] = append(metrics[:i], metrics[i+1:]...) |
||||||
|
} else { |
||||||
|
delete(m.children, h) |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// deleteByHashWithLabels removes the metric from the hash bucket h. If there
|
||||||
|
// are multiple matches in the bucket, use lvs to select a metric and remove
|
||||||
|
// only that metric.
|
||||||
|
func (m *MetricVec) deleteByHashWithLabels(h uint64, labels Labels) bool { |
||||||
|
metrics, ok := m.children[h] |
||||||
|
if !ok { |
||||||
|
return false |
||||||
|
} |
||||||
|
i := m.findMetricWithLabels(metrics, labels) |
||||||
|
if i >= len(metrics) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
if len(metrics) > 1 { |
||||||
|
m.children[h] = append(metrics[:i], metrics[i+1:]...) |
||||||
|
} else { |
||||||
|
delete(m.children, h) |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// Reset deletes all metrics in this vector.
|
||||||
|
func (m *MetricVec) Reset() { |
||||||
|
m.mtx.Lock() |
||||||
|
defer m.mtx.Unlock() |
||||||
|
|
||||||
|
for h := range m.children { |
||||||
|
delete(m.children, h) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricVec) hashLabelValues(vals []string) (uint64, error) { |
||||||
|
if len(vals) != len(m.desc.variableLabels) { |
||||||
|
return 0, errInconsistentCardinality |
||||||
|
} |
||||||
|
h := hashNew() |
||||||
|
for _, val := range vals { |
||||||
|
h = m.hashAdd(h, val) |
||||||
|
h = m.hashAddByte(h, model.SeparatorByte) |
||||||
|
} |
||||||
|
return h, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricVec) hashLabels(labels Labels) (uint64, error) { |
||||||
|
if len(labels) != len(m.desc.variableLabels) { |
||||||
|
return 0, errInconsistentCardinality |
||||||
|
} |
||||||
|
h := hashNew() |
||||||
|
for _, label := range m.desc.variableLabels { |
||||||
|
val, ok := labels[label] |
||||||
|
if !ok { |
||||||
|
return 0, fmt.Errorf("label name %q missing in label map", label) |
||||||
|
} |
||||||
|
h = m.hashAdd(h, val) |
||||||
|
h = m.hashAddByte(h, model.SeparatorByte) |
||||||
|
} |
||||||
|
return h, nil |
||||||
|
} |
||||||
|
|
||||||
|
// getOrCreateMetricWithLabelValues retrieves the metric by hash and label value
|
||||||
|
// or creates it and returns the new one.
|
||||||
|
//
|
||||||
|
// This function holds the mutex.
|
||||||
|
func (m *MetricVec) getOrCreateMetricWithLabelValues(hash uint64, lvs []string) Metric { |
||||||
|
m.mtx.RLock() |
||||||
|
metric, ok := m.getMetricWithLabelValues(hash, lvs) |
||||||
|
m.mtx.RUnlock() |
||||||
|
if ok { |
||||||
|
return metric |
||||||
|
} |
||||||
|
|
||||||
|
m.mtx.Lock() |
||||||
|
defer m.mtx.Unlock() |
||||||
|
metric, ok = m.getMetricWithLabelValues(hash, lvs) |
||||||
|
if !ok { |
||||||
|
// Copy to avoid allocation in case wo don't go down this code path.
|
||||||
|
copiedLVs := make([]string, len(lvs)) |
||||||
|
copy(copiedLVs, lvs) |
||||||
|
metric = m.newMetric(copiedLVs...) |
||||||
|
m.children[hash] = append(m.children[hash], metricWithLabelValues{values: copiedLVs, metric: metric}) |
||||||
|
} |
||||||
|
return metric |
||||||
|
} |
||||||
|
|
||||||
|
// getOrCreateMetricWithLabelValues retrieves the metric by hash and label value
|
||||||
|
// or creates it and returns the new one.
|
||||||
|
//
|
||||||
|
// This function holds the mutex.
|
||||||
|
func (m *MetricVec) getOrCreateMetricWithLabels(hash uint64, labels Labels) Metric { |
||||||
|
m.mtx.RLock() |
||||||
|
metric, ok := m.getMetricWithLabels(hash, labels) |
||||||
|
m.mtx.RUnlock() |
||||||
|
if ok { |
||||||
|
return metric |
||||||
|
} |
||||||
|
|
||||||
|
m.mtx.Lock() |
||||||
|
defer m.mtx.Unlock() |
||||||
|
metric, ok = m.getMetricWithLabels(hash, labels) |
||||||
|
if !ok { |
||||||
|
lvs := m.extractLabelValues(labels) |
||||||
|
metric = m.newMetric(lvs...) |
||||||
|
m.children[hash] = append(m.children[hash], metricWithLabelValues{values: lvs, metric: metric}) |
||||||
|
} |
||||||
|
return metric |
||||||
|
} |
||||||
|
|
||||||
|
// getMetricWithLabelValues gets a metric while handling possible collisions in
|
||||||
|
// the hash space. Must be called while holding read mutex.
|
||||||
|
func (m *MetricVec) getMetricWithLabelValues(h uint64, lvs []string) (Metric, bool) { |
||||||
|
metrics, ok := m.children[h] |
||||||
|
if ok { |
||||||
|
if i := m.findMetricWithLabelValues(metrics, lvs); i < len(metrics) { |
||||||
|
return metrics[i].metric, true |
||||||
|
} |
||||||
|
} |
||||||
|
return nil, false |
||||||
|
} |
||||||
|
|
||||||
|
// getMetricWithLabels gets a metric while handling possible collisions in
|
||||||
|
// the hash space. Must be called while holding read mutex.
|
||||||
|
func (m *MetricVec) getMetricWithLabels(h uint64, labels Labels) (Metric, bool) { |
||||||
|
metrics, ok := m.children[h] |
||||||
|
if ok { |
||||||
|
if i := m.findMetricWithLabels(metrics, labels); i < len(metrics) { |
||||||
|
return metrics[i].metric, true |
||||||
|
} |
||||||
|
} |
||||||
|
return nil, false |
||||||
|
} |
||||||
|
|
||||||
|
// findMetricWithLabelValues returns the index of the matching metric or
|
||||||
|
// len(metrics) if not found.
|
||||||
|
func (m *MetricVec) findMetricWithLabelValues(metrics []metricWithLabelValues, lvs []string) int { |
||||||
|
for i, metric := range metrics { |
||||||
|
if m.matchLabelValues(metric.values, lvs) { |
||||||
|
return i |
||||||
|
} |
||||||
|
} |
||||||
|
return len(metrics) |
||||||
|
} |
||||||
|
|
||||||
|
// findMetricWithLabels returns the index of the matching metric or len(metrics)
|
||||||
|
// if not found.
|
||||||
|
func (m *MetricVec) findMetricWithLabels(metrics []metricWithLabelValues, labels Labels) int { |
||||||
|
for i, metric := range metrics { |
||||||
|
if m.matchLabels(metric.values, labels) { |
||||||
|
return i |
||||||
|
} |
||||||
|
} |
||||||
|
return len(metrics) |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricVec) matchLabelValues(values []string, lvs []string) bool { |
||||||
|
if len(values) != len(lvs) { |
||||||
|
return false |
||||||
|
} |
||||||
|
for i, v := range values { |
||||||
|
if v != lvs[i] { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricVec) matchLabels(values []string, labels Labels) bool { |
||||||
|
if len(labels) != len(values) { |
||||||
|
return false |
||||||
|
} |
||||||
|
for i, k := range m.desc.variableLabels { |
||||||
|
if values[i] != labels[k] { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricVec) extractLabelValues(labels Labels) []string { |
||||||
|
labelValues := make([]string, len(labels)) |
||||||
|
for i, k := range m.desc.variableLabels { |
||||||
|
labelValues[i] = labels[k] |
||||||
|
} |
||||||
|
return labelValues |
||||||
|
} |
||||||
@ -0,0 +1,201 @@ |
|||||||
|
Apache License |
||||||
|
Version 2.0, January 2004 |
||||||
|
http://www.apache.org/licenses/ |
||||||
|
|
||||||
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION |
||||||
|
|
||||||
|
1. Definitions. |
||||||
|
|
||||||
|
"License" shall mean the terms and conditions for use, reproduction, |
||||||
|
and distribution as defined by Sections 1 through 9 of this document. |
||||||
|
|
||||||
|
"Licensor" shall mean the copyright owner or entity authorized by |
||||||
|
the copyright owner that is granting the License. |
||||||
|
|
||||||
|
"Legal Entity" shall mean the union of the acting entity and all |
||||||
|
other entities that control, are controlled by, or are under common |
||||||
|
control with that entity. For the purposes of this definition, |
||||||
|
"control" means (i) the power, direct or indirect, to cause the |
||||||
|
direction or management of such entity, whether by contract or |
||||||
|
otherwise, or (ii) ownership of fifty percent (50%) or more of the |
||||||
|
outstanding shares, or (iii) beneficial ownership of such entity. |
||||||
|
|
||||||
|
"You" (or "Your") shall mean an individual or Legal Entity |
||||||
|
exercising permissions granted by this License. |
||||||
|
|
||||||
|
"Source" form shall mean the preferred form for making modifications, |
||||||
|
including but not limited to software source code, documentation |
||||||
|
source, and configuration files. |
||||||
|
|
||||||
|
"Object" form shall mean any form resulting from mechanical |
||||||
|
transformation or translation of a Source form, including but |
||||||
|
not limited to compiled object code, generated documentation, |
||||||
|
and conversions to other media types. |
||||||
|
|
||||||
|
"Work" shall mean the work of authorship, whether in Source or |
||||||
|
Object form, made available under the License, as indicated by a |
||||||
|
copyright notice that is included in or attached to the work |
||||||
|
(an example is provided in the Appendix below). |
||||||
|
|
||||||
|
"Derivative Works" shall mean any work, whether in Source or Object |
||||||
|
form, that is based on (or derived from) the Work and for which the |
||||||
|
editorial revisions, annotations, elaborations, or other modifications |
||||||
|
represent, as a whole, an original work of authorship. For the purposes |
||||||
|
of this License, Derivative Works shall not include works that remain |
||||||
|
separable from, or merely link (or bind by name) to the interfaces of, |
||||||
|
the Work and Derivative Works thereof. |
||||||
|
|
||||||
|
"Contribution" shall mean any work of authorship, including |
||||||
|
the original version of the Work and any modifications or additions |
||||||
|
to that Work or Derivative Works thereof, that is intentionally |
||||||
|
submitted to Licensor for inclusion in the Work by the copyright owner |
||||||
|
or by an individual or Legal Entity authorized to submit on behalf of |
||||||
|
the copyright owner. For the purposes of this definition, "submitted" |
||||||
|
means any form of electronic, verbal, or written communication sent |
||||||
|
to the Licensor or its representatives, including but not limited to |
||||||
|
communication on electronic mailing lists, source code control systems, |
||||||
|
and issue tracking systems that are managed by, or on behalf of, the |
||||||
|
Licensor for the purpose of discussing and improving the Work, but |
||||||
|
excluding communication that is conspicuously marked or otherwise |
||||||
|
designated in writing by the copyright owner as "Not a Contribution." |
||||||
|
|
||||||
|
"Contributor" shall mean Licensor and any individual or Legal Entity |
||||||
|
on behalf of whom a Contribution has been received by Licensor and |
||||||
|
subsequently incorporated within the Work. |
||||||
|
|
||||||
|
2. Grant of Copyright License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
copyright license to reproduce, prepare Derivative Works of, |
||||||
|
publicly display, publicly perform, sublicense, and distribute the |
||||||
|
Work and such Derivative Works in Source or Object form. |
||||||
|
|
||||||
|
3. Grant of Patent License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
(except as stated in this section) patent license to make, have made, |
||||||
|
use, offer to sell, sell, import, and otherwise transfer the Work, |
||||||
|
where such license applies only to those patent claims licensable |
||||||
|
by such Contributor that are necessarily infringed by their |
||||||
|
Contribution(s) alone or by combination of their Contribution(s) |
||||||
|
with the Work to which such Contribution(s) was submitted. If You |
||||||
|
institute patent litigation against any entity (including a |
||||||
|
cross-claim or counterclaim in a lawsuit) alleging that the Work |
||||||
|
or a Contribution incorporated within the Work constitutes direct |
||||||
|
or contributory patent infringement, then any patent licenses |
||||||
|
granted to You under this License for that Work shall terminate |
||||||
|
as of the date such litigation is filed. |
||||||
|
|
||||||
|
4. Redistribution. You may reproduce and distribute copies of the |
||||||
|
Work or Derivative Works thereof in any medium, with or without |
||||||
|
modifications, and in Source or Object form, provided that You |
||||||
|
meet the following conditions: |
||||||
|
|
||||||
|
(a) You must give any other recipients of the Work or |
||||||
|
Derivative Works a copy of this License; and |
||||||
|
|
||||||
|
(b) You must cause any modified files to carry prominent notices |
||||||
|
stating that You changed the files; and |
||||||
|
|
||||||
|
(c) You must retain, in the Source form of any Derivative Works |
||||||
|
that You distribute, all copyright, patent, trademark, and |
||||||
|
attribution notices from the Source form of the Work, |
||||||
|
excluding those notices that do not pertain to any part of |
||||||
|
the Derivative Works; and |
||||||
|
|
||||||
|
(d) If the Work includes a "NOTICE" text file as part of its |
||||||
|
distribution, then any Derivative Works that You distribute must |
||||||
|
include a readable copy of the attribution notices contained |
||||||
|
within such NOTICE file, excluding those notices that do not |
||||||
|
pertain to any part of the Derivative Works, in at least one |
||||||
|
of the following places: within a NOTICE text file distributed |
||||||
|
as part of the Derivative Works; within the Source form or |
||||||
|
documentation, if provided along with the Derivative Works; or, |
||||||
|
within a display generated by the Derivative Works, if and |
||||||
|
wherever such third-party notices normally appear. The contents |
||||||
|
of the NOTICE file are for informational purposes only and |
||||||
|
do not modify the License. You may add Your own attribution |
||||||
|
notices within Derivative Works that You distribute, alongside |
||||||
|
or as an addendum to the NOTICE text from the Work, provided |
||||||
|
that such additional attribution notices cannot be construed |
||||||
|
as modifying the License. |
||||||
|
|
||||||
|
You may add Your own copyright statement to Your modifications and |
||||||
|
may provide additional or different license terms and conditions |
||||||
|
for use, reproduction, or distribution of Your modifications, or |
||||||
|
for any such Derivative Works as a whole, provided Your use, |
||||||
|
reproduction, and distribution of the Work otherwise complies with |
||||||
|
the conditions stated in this License. |
||||||
|
|
||||||
|
5. Submission of Contributions. Unless You explicitly state otherwise, |
||||||
|
any Contribution intentionally submitted for inclusion in the Work |
||||||
|
by You to the Licensor shall be under the terms and conditions of |
||||||
|
this License, without any additional terms or conditions. |
||||||
|
Notwithstanding the above, nothing herein shall supersede or modify |
||||||
|
the terms of any separate license agreement you may have executed |
||||||
|
with Licensor regarding such Contributions. |
||||||
|
|
||||||
|
6. Trademarks. This License does not grant permission to use the trade |
||||||
|
names, trademarks, service marks, or product names of the Licensor, |
||||||
|
except as required for reasonable and customary use in describing the |
||||||
|
origin of the Work and reproducing the content of the NOTICE file. |
||||||
|
|
||||||
|
7. Disclaimer of Warranty. Unless required by applicable law or |
||||||
|
agreed to in writing, Licensor provides the Work (and each |
||||||
|
Contributor provides its Contributions) on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or |
||||||
|
implied, including, without limitation, any warranties or conditions |
||||||
|
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A |
||||||
|
PARTICULAR PURPOSE. You are solely responsible for determining the |
||||||
|
appropriateness of using or redistributing the Work and assume any |
||||||
|
risks associated with Your exercise of permissions under this License. |
||||||
|
|
||||||
|
8. Limitation of Liability. In no event and under no legal theory, |
||||||
|
whether in tort (including negligence), contract, or otherwise, |
||||||
|
unless required by applicable law (such as deliberate and grossly |
||||||
|
negligent acts) or agreed to in writing, shall any Contributor be |
||||||
|
liable to You for damages, including any direct, indirect, special, |
||||||
|
incidental, or consequential damages of any character arising as a |
||||||
|
result of this License or out of the use or inability to use the |
||||||
|
Work (including but not limited to damages for loss of goodwill, |
||||||
|
work stoppage, computer failure or malfunction, or any and all |
||||||
|
other commercial damages or losses), even if such Contributor |
||||||
|
has been advised of the possibility of such damages. |
||||||
|
|
||||||
|
9. Accepting Warranty or Additional Liability. While redistributing |
||||||
|
the Work or Derivative Works thereof, You may choose to offer, |
||||||
|
and charge a fee for, acceptance of support, warranty, indemnity, |
||||||
|
or other liability obligations and/or rights consistent with this |
||||||
|
License. However, in accepting such obligations, You may act only |
||||||
|
on Your own behalf and on Your sole responsibility, not on behalf |
||||||
|
of any other Contributor, and only if You agree to indemnify, |
||||||
|
defend, and hold each Contributor harmless for any liability |
||||||
|
incurred by, or claims asserted against, such Contributor by reason |
||||||
|
of your accepting any such warranty or additional liability. |
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS |
||||||
|
|
||||||
|
APPENDIX: How to apply the Apache License to your work. |
||||||
|
|
||||||
|
To apply the Apache License to your work, attach the following |
||||||
|
boilerplate notice, with the fields enclosed by brackets "[]" |
||||||
|
replaced with your own identifying information. (Don't include |
||||||
|
the brackets!) The text should be enclosed in the appropriate |
||||||
|
comment syntax for the file format. We also recommend that a |
||||||
|
file or class name and description of purpose be included on the |
||||||
|
same "printed page" as the copyright notice for easier |
||||||
|
identification within third-party archives. |
||||||
|
|
||||||
|
Copyright [yyyy] [name of copyright owner] |
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); |
||||||
|
you may not use this file except in compliance with the License. |
||||||
|
You may obtain a copy of the License at |
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0 |
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software |
||||||
|
distributed under the License is distributed on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||||
|
See the License for the specific language governing permissions and |
||||||
|
limitations under the License. |
||||||
@ -0,0 +1,5 @@ |
|||||||
|
Data model artifacts for Prometheus. |
||||||
|
Copyright 2012-2015 The Prometheus Authors |
||||||
|
|
||||||
|
This product includes software developed at |
||||||
|
SoundCloud Ltd. (http://soundcloud.com/). |
||||||
@ -0,0 +1,364 @@ |
|||||||
|
// Code generated by protoc-gen-go.
|
||||||
|
// source: metrics.proto
|
||||||
|
// DO NOT EDIT!
|
||||||
|
|
||||||
|
/* |
||||||
|
Package io_prometheus_client is a generated protocol buffer package. |
||||||
|
|
||||||
|
It is generated from these files: |
||||||
|
metrics.proto |
||||||
|
|
||||||
|
It has these top-level messages: |
||||||
|
LabelPair |
||||||
|
Gauge |
||||||
|
Counter |
||||||
|
Quantile |
||||||
|
Summary |
||||||
|
Untyped |
||||||
|
Histogram |
||||||
|
Bucket |
||||||
|
Metric |
||||||
|
MetricFamily |
||||||
|
*/ |
||||||
|
package io_prometheus_client |
||||||
|
|
||||||
|
import proto "github.com/golang/protobuf/proto" |
||||||
|
import math "math" |
||||||
|
|
||||||
|
// Reference imports to suppress errors if they are not otherwise used.
|
||||||
|
var _ = proto.Marshal |
||||||
|
var _ = math.Inf |
||||||
|
|
||||||
|
type MetricType int32 |
||||||
|
|
||||||
|
const ( |
||||||
|
MetricType_COUNTER MetricType = 0 |
||||||
|
MetricType_GAUGE MetricType = 1 |
||||||
|
MetricType_SUMMARY MetricType = 2 |
||||||
|
MetricType_UNTYPED MetricType = 3 |
||||||
|
MetricType_HISTOGRAM MetricType = 4 |
||||||
|
) |
||||||
|
|
||||||
|
var MetricType_name = map[int32]string{ |
||||||
|
0: "COUNTER", |
||||||
|
1: "GAUGE", |
||||||
|
2: "SUMMARY", |
||||||
|
3: "UNTYPED", |
||||||
|
4: "HISTOGRAM", |
||||||
|
} |
||||||
|
var MetricType_value = map[string]int32{ |
||||||
|
"COUNTER": 0, |
||||||
|
"GAUGE": 1, |
||||||
|
"SUMMARY": 2, |
||||||
|
"UNTYPED": 3, |
||||||
|
"HISTOGRAM": 4, |
||||||
|
} |
||||||
|
|
||||||
|
func (x MetricType) Enum() *MetricType { |
||||||
|
p := new(MetricType) |
||||||
|
*p = x |
||||||
|
return p |
||||||
|
} |
||||||
|
func (x MetricType) String() string { |
||||||
|
return proto.EnumName(MetricType_name, int32(x)) |
||||||
|
} |
||||||
|
func (x *MetricType) UnmarshalJSON(data []byte) error { |
||||||
|
value, err := proto.UnmarshalJSONEnum(MetricType_value, data, "MetricType") |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*x = MetricType(value) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
type LabelPair struct { |
||||||
|
Name *string `protobuf:"bytes,1,opt,name=name" json:"name,omitempty"` |
||||||
|
Value *string `protobuf:"bytes,2,opt,name=value" json:"value,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *LabelPair) Reset() { *m = LabelPair{} } |
||||||
|
func (m *LabelPair) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*LabelPair) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *LabelPair) GetName() string { |
||||||
|
if m != nil && m.Name != nil { |
||||||
|
return *m.Name |
||||||
|
} |
||||||
|
return "" |
||||||
|
} |
||||||
|
|
||||||
|
func (m *LabelPair) GetValue() string { |
||||||
|
if m != nil && m.Value != nil { |
||||||
|
return *m.Value |
||||||
|
} |
||||||
|
return "" |
||||||
|
} |
||||||
|
|
||||||
|
type Gauge struct { |
||||||
|
Value *float64 `protobuf:"fixed64,1,opt,name=value" json:"value,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Gauge) Reset() { *m = Gauge{} } |
||||||
|
func (m *Gauge) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Gauge) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Gauge) GetValue() float64 { |
||||||
|
if m != nil && m.Value != nil { |
||||||
|
return *m.Value |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
type Counter struct { |
||||||
|
Value *float64 `protobuf:"fixed64,1,opt,name=value" json:"value,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Counter) Reset() { *m = Counter{} } |
||||||
|
func (m *Counter) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Counter) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Counter) GetValue() float64 { |
||||||
|
if m != nil && m.Value != nil { |
||||||
|
return *m.Value |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
type Quantile struct { |
||||||
|
Quantile *float64 `protobuf:"fixed64,1,opt,name=quantile" json:"quantile,omitempty"` |
||||||
|
Value *float64 `protobuf:"fixed64,2,opt,name=value" json:"value,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Quantile) Reset() { *m = Quantile{} } |
||||||
|
func (m *Quantile) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Quantile) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Quantile) GetQuantile() float64 { |
||||||
|
if m != nil && m.Quantile != nil { |
||||||
|
return *m.Quantile |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Quantile) GetValue() float64 { |
||||||
|
if m != nil && m.Value != nil { |
||||||
|
return *m.Value |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
type Summary struct { |
||||||
|
SampleCount *uint64 `protobuf:"varint,1,opt,name=sample_count" json:"sample_count,omitempty"` |
||||||
|
SampleSum *float64 `protobuf:"fixed64,2,opt,name=sample_sum" json:"sample_sum,omitempty"` |
||||||
|
Quantile []*Quantile `protobuf:"bytes,3,rep,name=quantile" json:"quantile,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Summary) Reset() { *m = Summary{} } |
||||||
|
func (m *Summary) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Summary) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Summary) GetSampleCount() uint64 { |
||||||
|
if m != nil && m.SampleCount != nil { |
||||||
|
return *m.SampleCount |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Summary) GetSampleSum() float64 { |
||||||
|
if m != nil && m.SampleSum != nil { |
||||||
|
return *m.SampleSum |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Summary) GetQuantile() []*Quantile { |
||||||
|
if m != nil { |
||||||
|
return m.Quantile |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
type Untyped struct { |
||||||
|
Value *float64 `protobuf:"fixed64,1,opt,name=value" json:"value,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Untyped) Reset() { *m = Untyped{} } |
||||||
|
func (m *Untyped) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Untyped) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Untyped) GetValue() float64 { |
||||||
|
if m != nil && m.Value != nil { |
||||||
|
return *m.Value |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
type Histogram struct { |
||||||
|
SampleCount *uint64 `protobuf:"varint,1,opt,name=sample_count" json:"sample_count,omitempty"` |
||||||
|
SampleSum *float64 `protobuf:"fixed64,2,opt,name=sample_sum" json:"sample_sum,omitempty"` |
||||||
|
Bucket []*Bucket `protobuf:"bytes,3,rep,name=bucket" json:"bucket,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Histogram) Reset() { *m = Histogram{} } |
||||||
|
func (m *Histogram) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Histogram) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Histogram) GetSampleCount() uint64 { |
||||||
|
if m != nil && m.SampleCount != nil { |
||||||
|
return *m.SampleCount |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Histogram) GetSampleSum() float64 { |
||||||
|
if m != nil && m.SampleSum != nil { |
||||||
|
return *m.SampleSum |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Histogram) GetBucket() []*Bucket { |
||||||
|
if m != nil { |
||||||
|
return m.Bucket |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
type Bucket struct { |
||||||
|
CumulativeCount *uint64 `protobuf:"varint,1,opt,name=cumulative_count" json:"cumulative_count,omitempty"` |
||||||
|
UpperBound *float64 `protobuf:"fixed64,2,opt,name=upper_bound" json:"upper_bound,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Bucket) Reset() { *m = Bucket{} } |
||||||
|
func (m *Bucket) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Bucket) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Bucket) GetCumulativeCount() uint64 { |
||||||
|
if m != nil && m.CumulativeCount != nil { |
||||||
|
return *m.CumulativeCount |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Bucket) GetUpperBound() float64 { |
||||||
|
if m != nil && m.UpperBound != nil { |
||||||
|
return *m.UpperBound |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
type Metric struct { |
||||||
|
Label []*LabelPair `protobuf:"bytes,1,rep,name=label" json:"label,omitempty"` |
||||||
|
Gauge *Gauge `protobuf:"bytes,2,opt,name=gauge" json:"gauge,omitempty"` |
||||||
|
Counter *Counter `protobuf:"bytes,3,opt,name=counter" json:"counter,omitempty"` |
||||||
|
Summary *Summary `protobuf:"bytes,4,opt,name=summary" json:"summary,omitempty"` |
||||||
|
Untyped *Untyped `protobuf:"bytes,5,opt,name=untyped" json:"untyped,omitempty"` |
||||||
|
Histogram *Histogram `protobuf:"bytes,7,opt,name=histogram" json:"histogram,omitempty"` |
||||||
|
TimestampMs *int64 `protobuf:"varint,6,opt,name=timestamp_ms" json:"timestamp_ms,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Metric) Reset() { *m = Metric{} } |
||||||
|
func (m *Metric) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*Metric) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *Metric) GetLabel() []*LabelPair { |
||||||
|
if m != nil { |
||||||
|
return m.Label |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Metric) GetGauge() *Gauge { |
||||||
|
if m != nil { |
||||||
|
return m.Gauge |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Metric) GetCounter() *Counter { |
||||||
|
if m != nil { |
||||||
|
return m.Counter |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Metric) GetSummary() *Summary { |
||||||
|
if m != nil { |
||||||
|
return m.Summary |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Metric) GetUntyped() *Untyped { |
||||||
|
if m != nil { |
||||||
|
return m.Untyped |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Metric) GetHistogram() *Histogram { |
||||||
|
if m != nil { |
||||||
|
return m.Histogram |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Metric) GetTimestampMs() int64 { |
||||||
|
if m != nil && m.TimestampMs != nil { |
||||||
|
return *m.TimestampMs |
||||||
|
} |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
type MetricFamily struct { |
||||||
|
Name *string `protobuf:"bytes,1,opt,name=name" json:"name,omitempty"` |
||||||
|
Help *string `protobuf:"bytes,2,opt,name=help" json:"help,omitempty"` |
||||||
|
Type *MetricType `protobuf:"varint,3,opt,name=type,enum=io.prometheus.client.MetricType" json:"type,omitempty"` |
||||||
|
Metric []*Metric `protobuf:"bytes,4,rep,name=metric" json:"metric,omitempty"` |
||||||
|
XXX_unrecognized []byte `json:"-"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricFamily) Reset() { *m = MetricFamily{} } |
||||||
|
func (m *MetricFamily) String() string { return proto.CompactTextString(m) } |
||||||
|
func (*MetricFamily) ProtoMessage() {} |
||||||
|
|
||||||
|
func (m *MetricFamily) GetName() string { |
||||||
|
if m != nil && m.Name != nil { |
||||||
|
return *m.Name |
||||||
|
} |
||||||
|
return "" |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricFamily) GetHelp() string { |
||||||
|
if m != nil && m.Help != nil { |
||||||
|
return *m.Help |
||||||
|
} |
||||||
|
return "" |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricFamily) GetType() MetricType { |
||||||
|
if m != nil && m.Type != nil { |
||||||
|
return *m.Type |
||||||
|
} |
||||||
|
return MetricType_COUNTER |
||||||
|
} |
||||||
|
|
||||||
|
func (m *MetricFamily) GetMetric() []*Metric { |
||||||
|
if m != nil { |
||||||
|
return m.Metric |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func init() { |
||||||
|
proto.RegisterEnum("io.prometheus.client.MetricType", MetricType_name, MetricType_value) |
||||||
|
} |
||||||
@ -0,0 +1,201 @@ |
|||||||
|
Apache License |
||||||
|
Version 2.0, January 2004 |
||||||
|
http://www.apache.org/licenses/ |
||||||
|
|
||||||
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION |
||||||
|
|
||||||
|
1. Definitions. |
||||||
|
|
||||||
|
"License" shall mean the terms and conditions for use, reproduction, |
||||||
|
and distribution as defined by Sections 1 through 9 of this document. |
||||||
|
|
||||||
|
"Licensor" shall mean the copyright owner or entity authorized by |
||||||
|
the copyright owner that is granting the License. |
||||||
|
|
||||||
|
"Legal Entity" shall mean the union of the acting entity and all |
||||||
|
other entities that control, are controlled by, or are under common |
||||||
|
control with that entity. For the purposes of this definition, |
||||||
|
"control" means (i) the power, direct or indirect, to cause the |
||||||
|
direction or management of such entity, whether by contract or |
||||||
|
otherwise, or (ii) ownership of fifty percent (50%) or more of the |
||||||
|
outstanding shares, or (iii) beneficial ownership of such entity. |
||||||
|
|
||||||
|
"You" (or "Your") shall mean an individual or Legal Entity |
||||||
|
exercising permissions granted by this License. |
||||||
|
|
||||||
|
"Source" form shall mean the preferred form for making modifications, |
||||||
|
including but not limited to software source code, documentation |
||||||
|
source, and configuration files. |
||||||
|
|
||||||
|
"Object" form shall mean any form resulting from mechanical |
||||||
|
transformation or translation of a Source form, including but |
||||||
|
not limited to compiled object code, generated documentation, |
||||||
|
and conversions to other media types. |
||||||
|
|
||||||
|
"Work" shall mean the work of authorship, whether in Source or |
||||||
|
Object form, made available under the License, as indicated by a |
||||||
|
copyright notice that is included in or attached to the work |
||||||
|
(an example is provided in the Appendix below). |
||||||
|
|
||||||
|
"Derivative Works" shall mean any work, whether in Source or Object |
||||||
|
form, that is based on (or derived from) the Work and for which the |
||||||
|
editorial revisions, annotations, elaborations, or other modifications |
||||||
|
represent, as a whole, an original work of authorship. For the purposes |
||||||
|
of this License, Derivative Works shall not include works that remain |
||||||
|
separable from, or merely link (or bind by name) to the interfaces of, |
||||||
|
the Work and Derivative Works thereof. |
||||||
|
|
||||||
|
"Contribution" shall mean any work of authorship, including |
||||||
|
the original version of the Work and any modifications or additions |
||||||
|
to that Work or Derivative Works thereof, that is intentionally |
||||||
|
submitted to Licensor for inclusion in the Work by the copyright owner |
||||||
|
or by an individual or Legal Entity authorized to submit on behalf of |
||||||
|
the copyright owner. For the purposes of this definition, "submitted" |
||||||
|
means any form of electronic, verbal, or written communication sent |
||||||
|
to the Licensor or its representatives, including but not limited to |
||||||
|
communication on electronic mailing lists, source code control systems, |
||||||
|
and issue tracking systems that are managed by, or on behalf of, the |
||||||
|
Licensor for the purpose of discussing and improving the Work, but |
||||||
|
excluding communication that is conspicuously marked or otherwise |
||||||
|
designated in writing by the copyright owner as "Not a Contribution." |
||||||
|
|
||||||
|
"Contributor" shall mean Licensor and any individual or Legal Entity |
||||||
|
on behalf of whom a Contribution has been received by Licensor and |
||||||
|
subsequently incorporated within the Work. |
||||||
|
|
||||||
|
2. Grant of Copyright License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
copyright license to reproduce, prepare Derivative Works of, |
||||||
|
publicly display, publicly perform, sublicense, and distribute the |
||||||
|
Work and such Derivative Works in Source or Object form. |
||||||
|
|
||||||
|
3. Grant of Patent License. Subject to the terms and conditions of |
||||||
|
this License, each Contributor hereby grants to You a perpetual, |
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable |
||||||
|
(except as stated in this section) patent license to make, have made, |
||||||
|
use, offer to sell, sell, import, and otherwise transfer the Work, |
||||||
|
where such license applies only to those patent claims licensable |
||||||
|
by such Contributor that are necessarily infringed by their |
||||||
|
Contribution(s) alone or by combination of their Contribution(s) |
||||||
|
with the Work to which such Contribution(s) was submitted. If You |
||||||
|
institute patent litigation against any entity (including a |
||||||
|
cross-claim or counterclaim in a lawsuit) alleging that the Work |
||||||
|
or a Contribution incorporated within the Work constitutes direct |
||||||
|
or contributory patent infringement, then any patent licenses |
||||||
|
granted to You under this License for that Work shall terminate |
||||||
|
as of the date such litigation is filed. |
||||||
|
|
||||||
|
4. Redistribution. You may reproduce and distribute copies of the |
||||||
|
Work or Derivative Works thereof in any medium, with or without |
||||||
|
modifications, and in Source or Object form, provided that You |
||||||
|
meet the following conditions: |
||||||
|
|
||||||
|
(a) You must give any other recipients of the Work or |
||||||
|
Derivative Works a copy of this License; and |
||||||
|
|
||||||
|
(b) You must cause any modified files to carry prominent notices |
||||||
|
stating that You changed the files; and |
||||||
|
|
||||||
|
(c) You must retain, in the Source form of any Derivative Works |
||||||
|
that You distribute, all copyright, patent, trademark, and |
||||||
|
attribution notices from the Source form of the Work, |
||||||
|
excluding those notices that do not pertain to any part of |
||||||
|
the Derivative Works; and |
||||||
|
|
||||||
|
(d) If the Work includes a "NOTICE" text file as part of its |
||||||
|
distribution, then any Derivative Works that You distribute must |
||||||
|
include a readable copy of the attribution notices contained |
||||||
|
within such NOTICE file, excluding those notices that do not |
||||||
|
pertain to any part of the Derivative Works, in at least one |
||||||
|
of the following places: within a NOTICE text file distributed |
||||||
|
as part of the Derivative Works; within the Source form or |
||||||
|
documentation, if provided along with the Derivative Works; or, |
||||||
|
within a display generated by the Derivative Works, if and |
||||||
|
wherever such third-party notices normally appear. The contents |
||||||
|
of the NOTICE file are for informational purposes only and |
||||||
|
do not modify the License. You may add Your own attribution |
||||||
|
notices within Derivative Works that You distribute, alongside |
||||||
|
or as an addendum to the NOTICE text from the Work, provided |
||||||
|
that such additional attribution notices cannot be construed |
||||||
|
as modifying the License. |
||||||
|
|
||||||
|
You may add Your own copyright statement to Your modifications and |
||||||
|
may provide additional or different license terms and conditions |
||||||
|
for use, reproduction, or distribution of Your modifications, or |
||||||
|
for any such Derivative Works as a whole, provided Your use, |
||||||
|
reproduction, and distribution of the Work otherwise complies with |
||||||
|
the conditions stated in this License. |
||||||
|
|
||||||
|
5. Submission of Contributions. Unless You explicitly state otherwise, |
||||||
|
any Contribution intentionally submitted for inclusion in the Work |
||||||
|
by You to the Licensor shall be under the terms and conditions of |
||||||
|
this License, without any additional terms or conditions. |
||||||
|
Notwithstanding the above, nothing herein shall supersede or modify |
||||||
|
the terms of any separate license agreement you may have executed |
||||||
|
with Licensor regarding such Contributions. |
||||||
|
|
||||||
|
6. Trademarks. This License does not grant permission to use the trade |
||||||
|
names, trademarks, service marks, or product names of the Licensor, |
||||||
|
except as required for reasonable and customary use in describing the |
||||||
|
origin of the Work and reproducing the content of the NOTICE file. |
||||||
|
|
||||||
|
7. Disclaimer of Warranty. Unless required by applicable law or |
||||||
|
agreed to in writing, Licensor provides the Work (and each |
||||||
|
Contributor provides its Contributions) on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or |
||||||
|
implied, including, without limitation, any warranties or conditions |
||||||
|
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A |
||||||
|
PARTICULAR PURPOSE. You are solely responsible for determining the |
||||||
|
appropriateness of using or redistributing the Work and assume any |
||||||
|
risks associated with Your exercise of permissions under this License. |
||||||
|
|
||||||
|
8. Limitation of Liability. In no event and under no legal theory, |
||||||
|
whether in tort (including negligence), contract, or otherwise, |
||||||
|
unless required by applicable law (such as deliberate and grossly |
||||||
|
negligent acts) or agreed to in writing, shall any Contributor be |
||||||
|
liable to You for damages, including any direct, indirect, special, |
||||||
|
incidental, or consequential damages of any character arising as a |
||||||
|
result of this License or out of the use or inability to use the |
||||||
|
Work (including but not limited to damages for loss of goodwill, |
||||||
|
work stoppage, computer failure or malfunction, or any and all |
||||||
|
other commercial damages or losses), even if such Contributor |
||||||
|
has been advised of the possibility of such damages. |
||||||
|
|
||||||
|
9. Accepting Warranty or Additional Liability. While redistributing |
||||||
|
the Work or Derivative Works thereof, You may choose to offer, |
||||||
|
and charge a fee for, acceptance of support, warranty, indemnity, |
||||||
|
or other liability obligations and/or rights consistent with this |
||||||
|
License. However, in accepting such obligations, You may act only |
||||||
|
on Your own behalf and on Your sole responsibility, not on behalf |
||||||
|
of any other Contributor, and only if You agree to indemnify, |
||||||
|
defend, and hold each Contributor harmless for any liability |
||||||
|
incurred by, or claims asserted against, such Contributor by reason |
||||||
|
of your accepting any such warranty or additional liability. |
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS |
||||||
|
|
||||||
|
APPENDIX: How to apply the Apache License to your work. |
||||||
|
|
||||||
|
To apply the Apache License to your work, attach the following |
||||||
|
boilerplate notice, with the fields enclosed by brackets "[]" |
||||||
|
replaced with your own identifying information. (Don't include |
||||||
|
the brackets!) The text should be enclosed in the appropriate |
||||||
|
comment syntax for the file format. We also recommend that a |
||||||
|
file or class name and description of purpose be included on the |
||||||
|
same "printed page" as the copyright notice for easier |
||||||
|
identification within third-party archives. |
||||||
|
|
||||||
|
Copyright [yyyy] [name of copyright owner] |
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License"); |
||||||
|
you may not use this file except in compliance with the License. |
||||||
|
You may obtain a copy of the License at |
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0 |
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software |
||||||
|
distributed under the License is distributed on an "AS IS" BASIS, |
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||||
|
See the License for the specific language governing permissions and |
||||||
|
limitations under the License. |
||||||
@ -0,0 +1,5 @@ |
|||||||
|
Common libraries shared by Prometheus Go components. |
||||||
|
Copyright 2015 The Prometheus Authors |
||||||
|
|
||||||
|
This product includes software developed at |
||||||
|
SoundCloud Ltd. (http://soundcloud.com/). |
||||||
@ -0,0 +1,412 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package expfmt |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"math" |
||||||
|
"mime" |
||||||
|
"net/http" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
|
||||||
|
"github.com/matttproud/golang_protobuf_extensions/pbutil" |
||||||
|
"github.com/prometheus/common/model" |
||||||
|
) |
||||||
|
|
||||||
|
// Decoder types decode an input stream into metric families.
|
||||||
|
type Decoder interface { |
||||||
|
Decode(*dto.MetricFamily) error |
||||||
|
} |
||||||
|
|
||||||
|
type DecodeOptions struct { |
||||||
|
// Timestamp is added to each value from the stream that has no explicit timestamp set.
|
||||||
|
Timestamp model.Time |
||||||
|
} |
||||||
|
|
||||||
|
// ResponseFormat extracts the correct format from a HTTP response header.
|
||||||
|
// If no matching format can be found FormatUnknown is returned.
|
||||||
|
func ResponseFormat(h http.Header) Format { |
||||||
|
ct := h.Get(hdrContentType) |
||||||
|
|
||||||
|
mediatype, params, err := mime.ParseMediaType(ct) |
||||||
|
if err != nil { |
||||||
|
return FmtUnknown |
||||||
|
} |
||||||
|
|
||||||
|
const textType = "text/plain" |
||||||
|
|
||||||
|
switch mediatype { |
||||||
|
case ProtoType: |
||||||
|
if p, ok := params["proto"]; ok && p != ProtoProtocol { |
||||||
|
return FmtUnknown |
||||||
|
} |
||||||
|
if e, ok := params["encoding"]; ok && e != "delimited" { |
||||||
|
return FmtUnknown |
||||||
|
} |
||||||
|
return FmtProtoDelim |
||||||
|
|
||||||
|
case textType: |
||||||
|
if v, ok := params["version"]; ok && v != TextVersion { |
||||||
|
return FmtUnknown |
||||||
|
} |
||||||
|
return FmtText |
||||||
|
} |
||||||
|
|
||||||
|
return FmtUnknown |
||||||
|
} |
||||||
|
|
||||||
|
// NewDecoder returns a new decoder based on the given input format.
|
||||||
|
// If the input format does not imply otherwise, a text format decoder is returned.
|
||||||
|
func NewDecoder(r io.Reader, format Format) Decoder { |
||||||
|
switch format { |
||||||
|
case FmtProtoDelim: |
||||||
|
return &protoDecoder{r: r} |
||||||
|
} |
||||||
|
return &textDecoder{r: r} |
||||||
|
} |
||||||
|
|
||||||
|
// protoDecoder implements the Decoder interface for protocol buffers.
|
||||||
|
type protoDecoder struct { |
||||||
|
r io.Reader |
||||||
|
} |
||||||
|
|
||||||
|
// Decode implements the Decoder interface.
|
||||||
|
func (d *protoDecoder) Decode(v *dto.MetricFamily) error { |
||||||
|
_, err := pbutil.ReadDelimited(d.r, v) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !model.IsValidMetricName(model.LabelValue(v.GetName())) { |
||||||
|
return fmt.Errorf("invalid metric name %q", v.GetName()) |
||||||
|
} |
||||||
|
for _, m := range v.GetMetric() { |
||||||
|
if m == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
for _, l := range m.GetLabel() { |
||||||
|
if l == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
if !model.LabelValue(l.GetValue()).IsValid() { |
||||||
|
return fmt.Errorf("invalid label value %q", l.GetValue()) |
||||||
|
} |
||||||
|
if !model.LabelName(l.GetName()).IsValid() { |
||||||
|
return fmt.Errorf("invalid label name %q", l.GetName()) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// textDecoder implements the Decoder interface for the text protocol.
|
||||||
|
type textDecoder struct { |
||||||
|
r io.Reader |
||||||
|
p TextParser |
||||||
|
fams []*dto.MetricFamily |
||||||
|
} |
||||||
|
|
||||||
|
// Decode implements the Decoder interface.
|
||||||
|
func (d *textDecoder) Decode(v *dto.MetricFamily) error { |
||||||
|
// TODO(fabxc): Wrap this as a line reader to make streaming safer.
|
||||||
|
if len(d.fams) == 0 { |
||||||
|
// No cached metric families, read everything and parse metrics.
|
||||||
|
fams, err := d.p.TextToMetricFamilies(d.r) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if len(fams) == 0 { |
||||||
|
return io.EOF |
||||||
|
} |
||||||
|
d.fams = make([]*dto.MetricFamily, 0, len(fams)) |
||||||
|
for _, f := range fams { |
||||||
|
d.fams = append(d.fams, f) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
*v = *d.fams[0] |
||||||
|
d.fams = d.fams[1:] |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
type SampleDecoder struct { |
||||||
|
Dec Decoder |
||||||
|
Opts *DecodeOptions |
||||||
|
|
||||||
|
f dto.MetricFamily |
||||||
|
} |
||||||
|
|
||||||
|
func (sd *SampleDecoder) Decode(s *model.Vector) error { |
||||||
|
if err := sd.Dec.Decode(&sd.f); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*s = extractSamples(&sd.f, sd.Opts) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Extract samples builds a slice of samples from the provided metric families.
|
||||||
|
func ExtractSamples(o *DecodeOptions, fams ...*dto.MetricFamily) model.Vector { |
||||||
|
var all model.Vector |
||||||
|
for _, f := range fams { |
||||||
|
all = append(all, extractSamples(f, o)...) |
||||||
|
} |
||||||
|
return all |
||||||
|
} |
||||||
|
|
||||||
|
func extractSamples(f *dto.MetricFamily, o *DecodeOptions) model.Vector { |
||||||
|
switch f.GetType() { |
||||||
|
case dto.MetricType_COUNTER: |
||||||
|
return extractCounter(o, f) |
||||||
|
case dto.MetricType_GAUGE: |
||||||
|
return extractGauge(o, f) |
||||||
|
case dto.MetricType_SUMMARY: |
||||||
|
return extractSummary(o, f) |
||||||
|
case dto.MetricType_UNTYPED: |
||||||
|
return extractUntyped(o, f) |
||||||
|
case dto.MetricType_HISTOGRAM: |
||||||
|
return extractHistogram(o, f) |
||||||
|
} |
||||||
|
panic("expfmt.extractSamples: unknown metric family type") |
||||||
|
} |
||||||
|
|
||||||
|
func extractCounter(o *DecodeOptions, f *dto.MetricFamily) model.Vector { |
||||||
|
samples := make(model.Vector, 0, len(f.Metric)) |
||||||
|
|
||||||
|
for _, m := range f.Metric { |
||||||
|
if m.Counter == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
lset := make(model.LabelSet, len(m.Label)+1) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName()) |
||||||
|
|
||||||
|
smpl := &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(m.Counter.GetValue()), |
||||||
|
} |
||||||
|
|
||||||
|
if m.TimestampMs != nil { |
||||||
|
smpl.Timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000) |
||||||
|
} else { |
||||||
|
smpl.Timestamp = o.Timestamp |
||||||
|
} |
||||||
|
|
||||||
|
samples = append(samples, smpl) |
||||||
|
} |
||||||
|
|
||||||
|
return samples |
||||||
|
} |
||||||
|
|
||||||
|
func extractGauge(o *DecodeOptions, f *dto.MetricFamily) model.Vector { |
||||||
|
samples := make(model.Vector, 0, len(f.Metric)) |
||||||
|
|
||||||
|
for _, m := range f.Metric { |
||||||
|
if m.Gauge == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
lset := make(model.LabelSet, len(m.Label)+1) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName()) |
||||||
|
|
||||||
|
smpl := &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(m.Gauge.GetValue()), |
||||||
|
} |
||||||
|
|
||||||
|
if m.TimestampMs != nil { |
||||||
|
smpl.Timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000) |
||||||
|
} else { |
||||||
|
smpl.Timestamp = o.Timestamp |
||||||
|
} |
||||||
|
|
||||||
|
samples = append(samples, smpl) |
||||||
|
} |
||||||
|
|
||||||
|
return samples |
||||||
|
} |
||||||
|
|
||||||
|
func extractUntyped(o *DecodeOptions, f *dto.MetricFamily) model.Vector { |
||||||
|
samples := make(model.Vector, 0, len(f.Metric)) |
||||||
|
|
||||||
|
for _, m := range f.Metric { |
||||||
|
if m.Untyped == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
lset := make(model.LabelSet, len(m.Label)+1) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName()) |
||||||
|
|
||||||
|
smpl := &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(m.Untyped.GetValue()), |
||||||
|
} |
||||||
|
|
||||||
|
if m.TimestampMs != nil { |
||||||
|
smpl.Timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000) |
||||||
|
} else { |
||||||
|
smpl.Timestamp = o.Timestamp |
||||||
|
} |
||||||
|
|
||||||
|
samples = append(samples, smpl) |
||||||
|
} |
||||||
|
|
||||||
|
return samples |
||||||
|
} |
||||||
|
|
||||||
|
func extractSummary(o *DecodeOptions, f *dto.MetricFamily) model.Vector { |
||||||
|
samples := make(model.Vector, 0, len(f.Metric)) |
||||||
|
|
||||||
|
for _, m := range f.Metric { |
||||||
|
if m.Summary == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
timestamp := o.Timestamp |
||||||
|
if m.TimestampMs != nil { |
||||||
|
timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000) |
||||||
|
} |
||||||
|
|
||||||
|
for _, q := range m.Summary.Quantile { |
||||||
|
lset := make(model.LabelSet, len(m.Label)+2) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
// BUG(matt): Update other names to "quantile".
|
||||||
|
lset[model.LabelName(model.QuantileLabel)] = model.LabelValue(fmt.Sprint(q.GetQuantile())) |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName()) |
||||||
|
|
||||||
|
samples = append(samples, &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(q.GetValue()), |
||||||
|
Timestamp: timestamp, |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
lset := make(model.LabelSet, len(m.Label)+1) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_sum") |
||||||
|
|
||||||
|
samples = append(samples, &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(m.Summary.GetSampleSum()), |
||||||
|
Timestamp: timestamp, |
||||||
|
}) |
||||||
|
|
||||||
|
lset = make(model.LabelSet, len(m.Label)+1) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_count") |
||||||
|
|
||||||
|
samples = append(samples, &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(m.Summary.GetSampleCount()), |
||||||
|
Timestamp: timestamp, |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
return samples |
||||||
|
} |
||||||
|
|
||||||
|
func extractHistogram(o *DecodeOptions, f *dto.MetricFamily) model.Vector { |
||||||
|
samples := make(model.Vector, 0, len(f.Metric)) |
||||||
|
|
||||||
|
for _, m := range f.Metric { |
||||||
|
if m.Histogram == nil { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
timestamp := o.Timestamp |
||||||
|
if m.TimestampMs != nil { |
||||||
|
timestamp = model.TimeFromUnixNano(*m.TimestampMs * 1000000) |
||||||
|
} |
||||||
|
|
||||||
|
infSeen := false |
||||||
|
|
||||||
|
for _, q := range m.Histogram.Bucket { |
||||||
|
lset := make(model.LabelSet, len(m.Label)+2) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.LabelName(model.BucketLabel)] = model.LabelValue(fmt.Sprint(q.GetUpperBound())) |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_bucket") |
||||||
|
|
||||||
|
if math.IsInf(q.GetUpperBound(), +1) { |
||||||
|
infSeen = true |
||||||
|
} |
||||||
|
|
||||||
|
samples = append(samples, &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(q.GetCumulativeCount()), |
||||||
|
Timestamp: timestamp, |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
lset := make(model.LabelSet, len(m.Label)+1) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_sum") |
||||||
|
|
||||||
|
samples = append(samples, &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(m.Histogram.GetSampleSum()), |
||||||
|
Timestamp: timestamp, |
||||||
|
}) |
||||||
|
|
||||||
|
lset = make(model.LabelSet, len(m.Label)+1) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_count") |
||||||
|
|
||||||
|
count := &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: model.SampleValue(m.Histogram.GetSampleCount()), |
||||||
|
Timestamp: timestamp, |
||||||
|
} |
||||||
|
samples = append(samples, count) |
||||||
|
|
||||||
|
if !infSeen { |
||||||
|
// Append an infinity bucket sample.
|
||||||
|
lset := make(model.LabelSet, len(m.Label)+2) |
||||||
|
for _, p := range m.Label { |
||||||
|
lset[model.LabelName(p.GetName())] = model.LabelValue(p.GetValue()) |
||||||
|
} |
||||||
|
lset[model.LabelName(model.BucketLabel)] = model.LabelValue("+Inf") |
||||||
|
lset[model.MetricNameLabel] = model.LabelValue(f.GetName() + "_bucket") |
||||||
|
|
||||||
|
samples = append(samples, &model.Sample{ |
||||||
|
Metric: model.Metric(lset), |
||||||
|
Value: count.Value, |
||||||
|
Timestamp: timestamp, |
||||||
|
}) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return samples |
||||||
|
} |
||||||
@ -0,0 +1,88 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package expfmt |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"net/http" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
"github.com/matttproud/golang_protobuf_extensions/pbutil" |
||||||
|
"github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
) |
||||||
|
|
||||||
|
// Encoder types encode metric families into an underlying wire protocol.
|
||||||
|
type Encoder interface { |
||||||
|
Encode(*dto.MetricFamily) error |
||||||
|
} |
||||||
|
|
||||||
|
type encoder func(*dto.MetricFamily) error |
||||||
|
|
||||||
|
func (e encoder) Encode(v *dto.MetricFamily) error { |
||||||
|
return e(v) |
||||||
|
} |
||||||
|
|
||||||
|
// Negotiate returns the Content-Type based on the given Accept header.
|
||||||
|
// If no appropriate accepted type is found, FmtText is returned.
|
||||||
|
func Negotiate(h http.Header) Format { |
||||||
|
for _, ac := range goautoneg.ParseAccept(h.Get(hdrAccept)) { |
||||||
|
// Check for protocol buffer
|
||||||
|
if ac.Type+"/"+ac.SubType == ProtoType && ac.Params["proto"] == ProtoProtocol { |
||||||
|
switch ac.Params["encoding"] { |
||||||
|
case "delimited": |
||||||
|
return FmtProtoDelim |
||||||
|
case "text": |
||||||
|
return FmtProtoText |
||||||
|
case "compact-text": |
||||||
|
return FmtProtoCompact |
||||||
|
} |
||||||
|
} |
||||||
|
// Check for text format.
|
||||||
|
ver := ac.Params["version"] |
||||||
|
if ac.Type == "text" && ac.SubType == "plain" && (ver == TextVersion || ver == "") { |
||||||
|
return FmtText |
||||||
|
} |
||||||
|
} |
||||||
|
return FmtText |
||||||
|
} |
||||||
|
|
||||||
|
// NewEncoder returns a new encoder based on content type negotiation.
|
||||||
|
func NewEncoder(w io.Writer, format Format) Encoder { |
||||||
|
switch format { |
||||||
|
case FmtProtoDelim: |
||||||
|
return encoder(func(v *dto.MetricFamily) error { |
||||||
|
_, err := pbutil.WriteDelimited(w, v) |
||||||
|
return err |
||||||
|
}) |
||||||
|
case FmtProtoCompact: |
||||||
|
return encoder(func(v *dto.MetricFamily) error { |
||||||
|
_, err := fmt.Fprintln(w, v.String()) |
||||||
|
return err |
||||||
|
}) |
||||||
|
case FmtProtoText: |
||||||
|
return encoder(func(v *dto.MetricFamily) error { |
||||||
|
_, err := fmt.Fprintln(w, proto.MarshalTextString(v)) |
||||||
|
return err |
||||||
|
}) |
||||||
|
case FmtText: |
||||||
|
return encoder(func(v *dto.MetricFamily) error { |
||||||
|
_, err := MetricFamilyToText(w, v) |
||||||
|
return err |
||||||
|
}) |
||||||
|
} |
||||||
|
panic("expfmt.NewEncoder: unknown format") |
||||||
|
} |
||||||
@ -0,0 +1,37 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// A package for reading and writing Prometheus metrics.
|
||||||
|
package expfmt |
||||||
|
|
||||||
|
type Format string |
||||||
|
|
||||||
|
const ( |
||||||
|
TextVersion = "0.0.4" |
||||||
|
|
||||||
|
ProtoType = `application/vnd.google.protobuf` |
||||||
|
ProtoProtocol = `io.prometheus.client.MetricFamily` |
||||||
|
ProtoFmt = ProtoType + "; proto=" + ProtoProtocol + ";" |
||||||
|
|
||||||
|
// The Content-Type values for the different wire protocols.
|
||||||
|
FmtUnknown Format = `<unknown>` |
||||||
|
FmtText Format = `text/plain; version=` + TextVersion |
||||||
|
FmtProtoDelim Format = ProtoFmt + ` encoding=delimited` |
||||||
|
FmtProtoText Format = ProtoFmt + ` encoding=text` |
||||||
|
FmtProtoCompact Format = ProtoFmt + ` encoding=compact-text` |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
hdrContentType = "Content-Type" |
||||||
|
hdrAccept = "Accept" |
||||||
|
) |
||||||
@ -0,0 +1,36 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// Build only when actually fuzzing
|
||||||
|
// +build gofuzz
|
||||||
|
|
||||||
|
package expfmt |
||||||
|
|
||||||
|
import "bytes" |
||||||
|
|
||||||
|
// Fuzz text metric parser with with github.com/dvyukov/go-fuzz:
|
||||||
|
//
|
||||||
|
// go-fuzz-build github.com/prometheus/common/expfmt
|
||||||
|
// go-fuzz -bin expfmt-fuzz.zip -workdir fuzz
|
||||||
|
//
|
||||||
|
// Further input samples should go in the folder fuzz/corpus.
|
||||||
|
func Fuzz(in []byte) int { |
||||||
|
parser := TextParser{} |
||||||
|
_, err := parser.TextToMetricFamilies(bytes.NewReader(in)) |
||||||
|
|
||||||
|
if err != nil { |
||||||
|
return 0 |
||||||
|
} |
||||||
|
|
||||||
|
return 1 |
||||||
|
} |
||||||
@ -0,0 +1,303 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package expfmt |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"math" |
||||||
|
"strings" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
"github.com/prometheus/common/model" |
||||||
|
) |
||||||
|
|
||||||
|
// MetricFamilyToText converts a MetricFamily proto message into text format and
|
||||||
|
// writes the resulting lines to 'out'. It returns the number of bytes written
|
||||||
|
// and any error encountered. The output will have the same order as the input,
|
||||||
|
// no further sorting is performed. Furthermore, this function assumes the input
|
||||||
|
// is already sanitized and does not perform any sanity checks. If the input
|
||||||
|
// contains duplicate metrics or invalid metric or label names, the conversion
|
||||||
|
// will result in invalid text format output.
|
||||||
|
//
|
||||||
|
// This method fulfills the type 'prometheus.encoder'.
|
||||||
|
func MetricFamilyToText(out io.Writer, in *dto.MetricFamily) (int, error) { |
||||||
|
var written int |
||||||
|
|
||||||
|
// Fail-fast checks.
|
||||||
|
if len(in.Metric) == 0 { |
||||||
|
return written, fmt.Errorf("MetricFamily has no metrics: %s", in) |
||||||
|
} |
||||||
|
name := in.GetName() |
||||||
|
if name == "" { |
||||||
|
return written, fmt.Errorf("MetricFamily has no name: %s", in) |
||||||
|
} |
||||||
|
|
||||||
|
// Comments, first HELP, then TYPE.
|
||||||
|
if in.Help != nil { |
||||||
|
n, err := fmt.Fprintf( |
||||||
|
out, "# HELP %s %s\n", |
||||||
|
name, escapeString(*in.Help, false), |
||||||
|
) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
} |
||||||
|
metricType := in.GetType() |
||||||
|
n, err := fmt.Fprintf( |
||||||
|
out, "# TYPE %s %s\n", |
||||||
|
name, strings.ToLower(metricType.String()), |
||||||
|
) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
|
||||||
|
// Finally the samples, one line for each.
|
||||||
|
for _, metric := range in.Metric { |
||||||
|
switch metricType { |
||||||
|
case dto.MetricType_COUNTER: |
||||||
|
if metric.Counter == nil { |
||||||
|
return written, fmt.Errorf( |
||||||
|
"expected counter in metric %s %s", name, metric, |
||||||
|
) |
||||||
|
} |
||||||
|
n, err = writeSample( |
||||||
|
name, metric, "", "", |
||||||
|
metric.Counter.GetValue(), |
||||||
|
out, |
||||||
|
) |
||||||
|
case dto.MetricType_GAUGE: |
||||||
|
if metric.Gauge == nil { |
||||||
|
return written, fmt.Errorf( |
||||||
|
"expected gauge in metric %s %s", name, metric, |
||||||
|
) |
||||||
|
} |
||||||
|
n, err = writeSample( |
||||||
|
name, metric, "", "", |
||||||
|
metric.Gauge.GetValue(), |
||||||
|
out, |
||||||
|
) |
||||||
|
case dto.MetricType_UNTYPED: |
||||||
|
if metric.Untyped == nil { |
||||||
|
return written, fmt.Errorf( |
||||||
|
"expected untyped in metric %s %s", name, metric, |
||||||
|
) |
||||||
|
} |
||||||
|
n, err = writeSample( |
||||||
|
name, metric, "", "", |
||||||
|
metric.Untyped.GetValue(), |
||||||
|
out, |
||||||
|
) |
||||||
|
case dto.MetricType_SUMMARY: |
||||||
|
if metric.Summary == nil { |
||||||
|
return written, fmt.Errorf( |
||||||
|
"expected summary in metric %s %s", name, metric, |
||||||
|
) |
||||||
|
} |
||||||
|
for _, q := range metric.Summary.Quantile { |
||||||
|
n, err = writeSample( |
||||||
|
name, metric, |
||||||
|
model.QuantileLabel, fmt.Sprint(q.GetQuantile()), |
||||||
|
q.GetValue(), |
||||||
|
out, |
||||||
|
) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
} |
||||||
|
n, err = writeSample( |
||||||
|
name+"_sum", metric, "", "", |
||||||
|
metric.Summary.GetSampleSum(), |
||||||
|
out, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
written += n |
||||||
|
n, err = writeSample( |
||||||
|
name+"_count", metric, "", "", |
||||||
|
float64(metric.Summary.GetSampleCount()), |
||||||
|
out, |
||||||
|
) |
||||||
|
case dto.MetricType_HISTOGRAM: |
||||||
|
if metric.Histogram == nil { |
||||||
|
return written, fmt.Errorf( |
||||||
|
"expected histogram in metric %s %s", name, metric, |
||||||
|
) |
||||||
|
} |
||||||
|
infSeen := false |
||||||
|
for _, q := range metric.Histogram.Bucket { |
||||||
|
n, err = writeSample( |
||||||
|
name+"_bucket", metric, |
||||||
|
model.BucketLabel, fmt.Sprint(q.GetUpperBound()), |
||||||
|
float64(q.GetCumulativeCount()), |
||||||
|
out, |
||||||
|
) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
if math.IsInf(q.GetUpperBound(), +1) { |
||||||
|
infSeen = true |
||||||
|
} |
||||||
|
} |
||||||
|
if !infSeen { |
||||||
|
n, err = writeSample( |
||||||
|
name+"_bucket", metric, |
||||||
|
model.BucketLabel, "+Inf", |
||||||
|
float64(metric.Histogram.GetSampleCount()), |
||||||
|
out, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
written += n |
||||||
|
} |
||||||
|
n, err = writeSample( |
||||||
|
name+"_sum", metric, "", "", |
||||||
|
metric.Histogram.GetSampleSum(), |
||||||
|
out, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
written += n |
||||||
|
n, err = writeSample( |
||||||
|
name+"_count", metric, "", "", |
||||||
|
float64(metric.Histogram.GetSampleCount()), |
||||||
|
out, |
||||||
|
) |
||||||
|
default: |
||||||
|
return written, fmt.Errorf( |
||||||
|
"unexpected type in metric %s %s", name, metric, |
||||||
|
) |
||||||
|
} |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
} |
||||||
|
return written, nil |
||||||
|
} |
||||||
|
|
||||||
|
// writeSample writes a single sample in text format to out, given the metric
|
||||||
|
// name, the metric proto message itself, optionally an additional label name
|
||||||
|
// and value (use empty strings if not required), and the value. The function
|
||||||
|
// returns the number of bytes written and any error encountered.
|
||||||
|
func writeSample( |
||||||
|
name string, |
||||||
|
metric *dto.Metric, |
||||||
|
additionalLabelName, additionalLabelValue string, |
||||||
|
value float64, |
||||||
|
out io.Writer, |
||||||
|
) (int, error) { |
||||||
|
var written int |
||||||
|
n, err := fmt.Fprint(out, name) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
n, err = labelPairsToText( |
||||||
|
metric.Label, |
||||||
|
additionalLabelName, additionalLabelValue, |
||||||
|
out, |
||||||
|
) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
n, err = fmt.Fprintf(out, " %v", value) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
if metric.TimestampMs != nil { |
||||||
|
n, err = fmt.Fprintf(out, " %v", *metric.TimestampMs) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
} |
||||||
|
n, err = out.Write([]byte{'\n'}) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
return written, nil |
||||||
|
} |
||||||
|
|
||||||
|
// labelPairsToText converts a slice of LabelPair proto messages plus the
|
||||||
|
// explicitly given additional label pair into text formatted as required by the
|
||||||
|
// text format and writes it to 'out'. An empty slice in combination with an
|
||||||
|
// empty string 'additionalLabelName' results in nothing being
|
||||||
|
// written. Otherwise, the label pairs are written, escaped as required by the
|
||||||
|
// text format, and enclosed in '{...}'. The function returns the number of
|
||||||
|
// bytes written and any error encountered.
|
||||||
|
func labelPairsToText( |
||||||
|
in []*dto.LabelPair, |
||||||
|
additionalLabelName, additionalLabelValue string, |
||||||
|
out io.Writer, |
||||||
|
) (int, error) { |
||||||
|
if len(in) == 0 && additionalLabelName == "" { |
||||||
|
return 0, nil |
||||||
|
} |
||||||
|
var written int |
||||||
|
separator := '{' |
||||||
|
for _, lp := range in { |
||||||
|
n, err := fmt.Fprintf( |
||||||
|
out, `%c%s="%s"`, |
||||||
|
separator, lp.GetName(), escapeString(lp.GetValue(), true), |
||||||
|
) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
separator = ',' |
||||||
|
} |
||||||
|
if additionalLabelName != "" { |
||||||
|
n, err := fmt.Fprintf( |
||||||
|
out, `%c%s="%s"`, |
||||||
|
separator, additionalLabelName, |
||||||
|
escapeString(additionalLabelValue, true), |
||||||
|
) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
} |
||||||
|
n, err := out.Write([]byte{'}'}) |
||||||
|
written += n |
||||||
|
if err != nil { |
||||||
|
return written, err |
||||||
|
} |
||||||
|
return written, nil |
||||||
|
} |
||||||
|
|
||||||
|
var ( |
||||||
|
escape = strings.NewReplacer("\\", `\\`, "\n", `\n`) |
||||||
|
escapeWithDoubleQuote = strings.NewReplacer("\\", `\\`, "\n", `\n`, "\"", `\"`) |
||||||
|
) |
||||||
|
|
||||||
|
// escapeString replaces '\' by '\\', new line character by '\n', and - if
|
||||||
|
// includeDoubleQuote is true - '"' by '\"'.
|
||||||
|
func escapeString(v string, includeDoubleQuote bool) string { |
||||||
|
if includeDoubleQuote { |
||||||
|
return escapeWithDoubleQuote.Replace(v) |
||||||
|
} |
||||||
|
|
||||||
|
return escape.Replace(v) |
||||||
|
} |
||||||
@ -0,0 +1,753 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package expfmt |
||||||
|
|
||||||
|
import ( |
||||||
|
"bufio" |
||||||
|
"bytes" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"math" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
|
||||||
|
dto "github.com/prometheus/client_model/go" |
||||||
|
|
||||||
|
"github.com/golang/protobuf/proto" |
||||||
|
"github.com/prometheus/common/model" |
||||||
|
) |
||||||
|
|
||||||
|
// A stateFn is a function that represents a state in a state machine. By
|
||||||
|
// executing it, the state is progressed to the next state. The stateFn returns
|
||||||
|
// another stateFn, which represents the new state. The end state is represented
|
||||||
|
// by nil.
|
||||||
|
type stateFn func() stateFn |
||||||
|
|
||||||
|
// ParseError signals errors while parsing the simple and flat text-based
|
||||||
|
// exchange format.
|
||||||
|
type ParseError struct { |
||||||
|
Line int |
||||||
|
Msg string |
||||||
|
} |
||||||
|
|
||||||
|
// Error implements the error interface.
|
||||||
|
func (e ParseError) Error() string { |
||||||
|
return fmt.Sprintf("text format parsing error in line %d: %s", e.Line, e.Msg) |
||||||
|
} |
||||||
|
|
||||||
|
// TextParser is used to parse the simple and flat text-based exchange format. Its
|
||||||
|
// zero value is ready to use.
|
||||||
|
type TextParser struct { |
||||||
|
metricFamiliesByName map[string]*dto.MetricFamily |
||||||
|
buf *bufio.Reader // Where the parsed input is read through.
|
||||||
|
err error // Most recent error.
|
||||||
|
lineCount int // Tracks the line count for error messages.
|
||||||
|
currentByte byte // The most recent byte read.
|
||||||
|
currentToken bytes.Buffer // Re-used each time a token has to be gathered from multiple bytes.
|
||||||
|
currentMF *dto.MetricFamily |
||||||
|
currentMetric *dto.Metric |
||||||
|
currentLabelPair *dto.LabelPair |
||||||
|
|
||||||
|
// The remaining member variables are only used for summaries/histograms.
|
||||||
|
currentLabels map[string]string // All labels including '__name__' but excluding 'quantile'/'le'
|
||||||
|
// Summary specific.
|
||||||
|
summaries map[uint64]*dto.Metric // Key is created with LabelsToSignature.
|
||||||
|
currentQuantile float64 |
||||||
|
// Histogram specific.
|
||||||
|
histograms map[uint64]*dto.Metric // Key is created with LabelsToSignature.
|
||||||
|
currentBucket float64 |
||||||
|
// These tell us if the currently processed line ends on '_count' or
|
||||||
|
// '_sum' respectively and belong to a summary/histogram, representing the sample
|
||||||
|
// count and sum of that summary/histogram.
|
||||||
|
currentIsSummaryCount, currentIsSummarySum bool |
||||||
|
currentIsHistogramCount, currentIsHistogramSum bool |
||||||
|
} |
||||||
|
|
||||||
|
// TextToMetricFamilies reads 'in' as the simple and flat text-based exchange
|
||||||
|
// format and creates MetricFamily proto messages. It returns the MetricFamily
|
||||||
|
// proto messages in a map where the metric names are the keys, along with any
|
||||||
|
// error encountered.
|
||||||
|
//
|
||||||
|
// If the input contains duplicate metrics (i.e. lines with the same metric name
|
||||||
|
// and exactly the same label set), the resulting MetricFamily will contain
|
||||||
|
// duplicate Metric proto messages. Similar is true for duplicate label
|
||||||
|
// names. Checks for duplicates have to be performed separately, if required.
|
||||||
|
// Also note that neither the metrics within each MetricFamily are sorted nor
|
||||||
|
// the label pairs within each Metric. Sorting is not required for the most
|
||||||
|
// frequent use of this method, which is sample ingestion in the Prometheus
|
||||||
|
// server. However, for presentation purposes, you might want to sort the
|
||||||
|
// metrics, and in some cases, you must sort the labels, e.g. for consumption by
|
||||||
|
// the metric family injection hook of the Prometheus registry.
|
||||||
|
//
|
||||||
|
// Summaries and histograms are rather special beasts. You would probably not
|
||||||
|
// use them in the simple text format anyway. This method can deal with
|
||||||
|
// summaries and histograms if they are presented in exactly the way the
|
||||||
|
// text.Create function creates them.
|
||||||
|
//
|
||||||
|
// This method must not be called concurrently. If you want to parse different
|
||||||
|
// input concurrently, instantiate a separate Parser for each goroutine.
|
||||||
|
func (p *TextParser) TextToMetricFamilies(in io.Reader) (map[string]*dto.MetricFamily, error) { |
||||||
|
p.reset(in) |
||||||
|
for nextState := p.startOfLine; nextState != nil; nextState = nextState() { |
||||||
|
// Magic happens here...
|
||||||
|
} |
||||||
|
// Get rid of empty metric families.
|
||||||
|
for k, mf := range p.metricFamiliesByName { |
||||||
|
if len(mf.GetMetric()) == 0 { |
||||||
|
delete(p.metricFamiliesByName, k) |
||||||
|
} |
||||||
|
} |
||||||
|
// If p.err is io.EOF now, we have run into a premature end of the input
|
||||||
|
// stream. Turn this error into something nicer and more
|
||||||
|
// meaningful. (io.EOF is often used as a signal for the legitimate end
|
||||||
|
// of an input stream.)
|
||||||
|
if p.err == io.EOF { |
||||||
|
p.parseError("unexpected end of input stream") |
||||||
|
} |
||||||
|
return p.metricFamiliesByName, p.err |
||||||
|
} |
||||||
|
|
||||||
|
func (p *TextParser) reset(in io.Reader) { |
||||||
|
p.metricFamiliesByName = map[string]*dto.MetricFamily{} |
||||||
|
if p.buf == nil { |
||||||
|
p.buf = bufio.NewReader(in) |
||||||
|
} else { |
||||||
|
p.buf.Reset(in) |
||||||
|
} |
||||||
|
p.err = nil |
||||||
|
p.lineCount = 0 |
||||||
|
if p.summaries == nil || len(p.summaries) > 0 { |
||||||
|
p.summaries = map[uint64]*dto.Metric{} |
||||||
|
} |
||||||
|
if p.histograms == nil || len(p.histograms) > 0 { |
||||||
|
p.histograms = map[uint64]*dto.Metric{} |
||||||
|
} |
||||||
|
p.currentQuantile = math.NaN() |
||||||
|
p.currentBucket = math.NaN() |
||||||
|
} |
||||||
|
|
||||||
|
// startOfLine represents the state where the next byte read from p.buf is the
|
||||||
|
// start of a line (or whitespace leading up to it).
|
||||||
|
func (p *TextParser) startOfLine() stateFn { |
||||||
|
p.lineCount++ |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
// End of input reached. This is the only case where
|
||||||
|
// that is not an error but a signal that we are done.
|
||||||
|
p.err = nil |
||||||
|
return nil |
||||||
|
} |
||||||
|
switch p.currentByte { |
||||||
|
case '#': |
||||||
|
return p.startComment |
||||||
|
case '\n': |
||||||
|
return p.startOfLine // Empty line, start the next one.
|
||||||
|
} |
||||||
|
return p.readingMetricName |
||||||
|
} |
||||||
|
|
||||||
|
// startComment represents the state where the next byte read from p.buf is the
|
||||||
|
// start of a comment (or whitespace leading up to it).
|
||||||
|
func (p *TextParser) startComment() stateFn { |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentByte == '\n' { |
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
if p.readTokenUntilWhitespace(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
// If we have hit the end of line already, there is nothing left
|
||||||
|
// to do. This is not considered a syntax error.
|
||||||
|
if p.currentByte == '\n' { |
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
keyword := p.currentToken.String() |
||||||
|
if keyword != "HELP" && keyword != "TYPE" { |
||||||
|
// Generic comment, ignore by fast forwarding to end of line.
|
||||||
|
for p.currentByte != '\n' { |
||||||
|
if p.currentByte, p.err = p.buf.ReadByte(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
} |
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
// There is something. Next has to be a metric name.
|
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.readTokenAsMetricName(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentByte == '\n' { |
||||||
|
// At the end of the line already.
|
||||||
|
// Again, this is not considered a syntax error.
|
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
if !isBlankOrTab(p.currentByte) { |
||||||
|
p.parseError("invalid metric name in comment") |
||||||
|
return nil |
||||||
|
} |
||||||
|
p.setOrCreateCurrentMF() |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentByte == '\n' { |
||||||
|
// At the end of the line already.
|
||||||
|
// Again, this is not considered a syntax error.
|
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
switch keyword { |
||||||
|
case "HELP": |
||||||
|
return p.readingHelp |
||||||
|
case "TYPE": |
||||||
|
return p.readingType |
||||||
|
} |
||||||
|
panic(fmt.Sprintf("code error: unexpected keyword %q", keyword)) |
||||||
|
} |
||||||
|
|
||||||
|
// readingMetricName represents the state where the last byte read (now in
|
||||||
|
// p.currentByte) is the first byte of a metric name.
|
||||||
|
func (p *TextParser) readingMetricName() stateFn { |
||||||
|
if p.readTokenAsMetricName(); p.err != nil { |
||||||
|
return nil |
||||||
|
} |
||||||
|
if p.currentToken.Len() == 0 { |
||||||
|
p.parseError("invalid metric name") |
||||||
|
return nil |
||||||
|
} |
||||||
|
p.setOrCreateCurrentMF() |
||||||
|
// Now is the time to fix the type if it hasn't happened yet.
|
||||||
|
if p.currentMF.Type == nil { |
||||||
|
p.currentMF.Type = dto.MetricType_UNTYPED.Enum() |
||||||
|
} |
||||||
|
p.currentMetric = &dto.Metric{} |
||||||
|
// Do not append the newly created currentMetric to
|
||||||
|
// currentMF.Metric right now. First wait if this is a summary,
|
||||||
|
// and the metric exists already, which we can only know after
|
||||||
|
// having read all the labels.
|
||||||
|
if p.skipBlankTabIfCurrentBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
return p.readingLabels |
||||||
|
} |
||||||
|
|
||||||
|
// readingLabels represents the state where the last byte read (now in
|
||||||
|
// p.currentByte) is either the first byte of the label set (i.e. a '{'), or the
|
||||||
|
// first byte of the value (otherwise).
|
||||||
|
func (p *TextParser) readingLabels() stateFn { |
||||||
|
// Summaries/histograms are special. We have to reset the
|
||||||
|
// currentLabels map, currentQuantile and currentBucket before starting to
|
||||||
|
// read labels.
|
||||||
|
if p.currentMF.GetType() == dto.MetricType_SUMMARY || p.currentMF.GetType() == dto.MetricType_HISTOGRAM { |
||||||
|
p.currentLabels = map[string]string{} |
||||||
|
p.currentLabels[string(model.MetricNameLabel)] = p.currentMF.GetName() |
||||||
|
p.currentQuantile = math.NaN() |
||||||
|
p.currentBucket = math.NaN() |
||||||
|
} |
||||||
|
if p.currentByte != '{' { |
||||||
|
return p.readingValue |
||||||
|
} |
||||||
|
return p.startLabelName |
||||||
|
} |
||||||
|
|
||||||
|
// startLabelName represents the state where the next byte read from p.buf is
|
||||||
|
// the start of a label name (or whitespace leading up to it).
|
||||||
|
func (p *TextParser) startLabelName() stateFn { |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentByte == '}' { |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
return p.readingValue |
||||||
|
} |
||||||
|
if p.readTokenAsLabelName(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentToken.Len() == 0 { |
||||||
|
p.parseError(fmt.Sprintf("invalid label name for metric %q", p.currentMF.GetName())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
p.currentLabelPair = &dto.LabelPair{Name: proto.String(p.currentToken.String())} |
||||||
|
if p.currentLabelPair.GetName() == string(model.MetricNameLabel) { |
||||||
|
p.parseError(fmt.Sprintf("label name %q is reserved", model.MetricNameLabel)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
// Special summary/histogram treatment. Don't add 'quantile' and 'le'
|
||||||
|
// labels to 'real' labels.
|
||||||
|
if !(p.currentMF.GetType() == dto.MetricType_SUMMARY && p.currentLabelPair.GetName() == model.QuantileLabel) && |
||||||
|
!(p.currentMF.GetType() == dto.MetricType_HISTOGRAM && p.currentLabelPair.GetName() == model.BucketLabel) { |
||||||
|
p.currentMetric.Label = append(p.currentMetric.Label, p.currentLabelPair) |
||||||
|
} |
||||||
|
if p.skipBlankTabIfCurrentBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentByte != '=' { |
||||||
|
p.parseError(fmt.Sprintf("expected '=' after label name, found %q", p.currentByte)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
return p.startLabelValue |
||||||
|
} |
||||||
|
|
||||||
|
// startLabelValue represents the state where the next byte read from p.buf is
|
||||||
|
// the start of a (quoted) label value (or whitespace leading up to it).
|
||||||
|
func (p *TextParser) startLabelValue() stateFn { |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentByte != '"' { |
||||||
|
p.parseError(fmt.Sprintf("expected '\"' at start of label value, found %q", p.currentByte)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
if p.readTokenAsLabelValue(); p.err != nil { |
||||||
|
return nil |
||||||
|
} |
||||||
|
p.currentLabelPair.Value = proto.String(p.currentToken.String()) |
||||||
|
// Special treatment of summaries:
|
||||||
|
// - Quantile labels are special, will result in dto.Quantile later.
|
||||||
|
// - Other labels have to be added to currentLabels for signature calculation.
|
||||||
|
if p.currentMF.GetType() == dto.MetricType_SUMMARY { |
||||||
|
if p.currentLabelPair.GetName() == model.QuantileLabel { |
||||||
|
if p.currentQuantile, p.err = strconv.ParseFloat(p.currentLabelPair.GetValue(), 64); p.err != nil { |
||||||
|
// Create a more helpful error message.
|
||||||
|
p.parseError(fmt.Sprintf("expected float as value for 'quantile' label, got %q", p.currentLabelPair.GetValue())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
} else { |
||||||
|
p.currentLabels[p.currentLabelPair.GetName()] = p.currentLabelPair.GetValue() |
||||||
|
} |
||||||
|
} |
||||||
|
// Similar special treatment of histograms.
|
||||||
|
if p.currentMF.GetType() == dto.MetricType_HISTOGRAM { |
||||||
|
if p.currentLabelPair.GetName() == model.BucketLabel { |
||||||
|
if p.currentBucket, p.err = strconv.ParseFloat(p.currentLabelPair.GetValue(), 64); p.err != nil { |
||||||
|
// Create a more helpful error message.
|
||||||
|
p.parseError(fmt.Sprintf("expected float as value for 'le' label, got %q", p.currentLabelPair.GetValue())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
} else { |
||||||
|
p.currentLabels[p.currentLabelPair.GetName()] = p.currentLabelPair.GetValue() |
||||||
|
} |
||||||
|
} |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
switch p.currentByte { |
||||||
|
case ',': |
||||||
|
return p.startLabelName |
||||||
|
|
||||||
|
case '}': |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
return p.readingValue |
||||||
|
default: |
||||||
|
p.parseError(fmt.Sprintf("unexpected end of label value %q", p.currentLabelPair.Value)) |
||||||
|
return nil |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// readingValue represents the state where the last byte read (now in
|
||||||
|
// p.currentByte) is the first byte of the sample value (i.e. a float).
|
||||||
|
func (p *TextParser) readingValue() stateFn { |
||||||
|
// When we are here, we have read all the labels, so for the
|
||||||
|
// special case of a summary/histogram, we can finally find out
|
||||||
|
// if the metric already exists.
|
||||||
|
if p.currentMF.GetType() == dto.MetricType_SUMMARY { |
||||||
|
signature := model.LabelsToSignature(p.currentLabels) |
||||||
|
if summary := p.summaries[signature]; summary != nil { |
||||||
|
p.currentMetric = summary |
||||||
|
} else { |
||||||
|
p.summaries[signature] = p.currentMetric |
||||||
|
p.currentMF.Metric = append(p.currentMF.Metric, p.currentMetric) |
||||||
|
} |
||||||
|
} else if p.currentMF.GetType() == dto.MetricType_HISTOGRAM { |
||||||
|
signature := model.LabelsToSignature(p.currentLabels) |
||||||
|
if histogram := p.histograms[signature]; histogram != nil { |
||||||
|
p.currentMetric = histogram |
||||||
|
} else { |
||||||
|
p.histograms[signature] = p.currentMetric |
||||||
|
p.currentMF.Metric = append(p.currentMF.Metric, p.currentMetric) |
||||||
|
} |
||||||
|
} else { |
||||||
|
p.currentMF.Metric = append(p.currentMF.Metric, p.currentMetric) |
||||||
|
} |
||||||
|
if p.readTokenUntilWhitespace(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
value, err := strconv.ParseFloat(p.currentToken.String(), 64) |
||||||
|
if err != nil { |
||||||
|
// Create a more helpful error message.
|
||||||
|
p.parseError(fmt.Sprintf("expected float as value, got %q", p.currentToken.String())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
switch p.currentMF.GetType() { |
||||||
|
case dto.MetricType_COUNTER: |
||||||
|
p.currentMetric.Counter = &dto.Counter{Value: proto.Float64(value)} |
||||||
|
case dto.MetricType_GAUGE: |
||||||
|
p.currentMetric.Gauge = &dto.Gauge{Value: proto.Float64(value)} |
||||||
|
case dto.MetricType_UNTYPED: |
||||||
|
p.currentMetric.Untyped = &dto.Untyped{Value: proto.Float64(value)} |
||||||
|
case dto.MetricType_SUMMARY: |
||||||
|
// *sigh*
|
||||||
|
if p.currentMetric.Summary == nil { |
||||||
|
p.currentMetric.Summary = &dto.Summary{} |
||||||
|
} |
||||||
|
switch { |
||||||
|
case p.currentIsSummaryCount: |
||||||
|
p.currentMetric.Summary.SampleCount = proto.Uint64(uint64(value)) |
||||||
|
case p.currentIsSummarySum: |
||||||
|
p.currentMetric.Summary.SampleSum = proto.Float64(value) |
||||||
|
case !math.IsNaN(p.currentQuantile): |
||||||
|
p.currentMetric.Summary.Quantile = append( |
||||||
|
p.currentMetric.Summary.Quantile, |
||||||
|
&dto.Quantile{ |
||||||
|
Quantile: proto.Float64(p.currentQuantile), |
||||||
|
Value: proto.Float64(value), |
||||||
|
}, |
||||||
|
) |
||||||
|
} |
||||||
|
case dto.MetricType_HISTOGRAM: |
||||||
|
// *sigh*
|
||||||
|
if p.currentMetric.Histogram == nil { |
||||||
|
p.currentMetric.Histogram = &dto.Histogram{} |
||||||
|
} |
||||||
|
switch { |
||||||
|
case p.currentIsHistogramCount: |
||||||
|
p.currentMetric.Histogram.SampleCount = proto.Uint64(uint64(value)) |
||||||
|
case p.currentIsHistogramSum: |
||||||
|
p.currentMetric.Histogram.SampleSum = proto.Float64(value) |
||||||
|
case !math.IsNaN(p.currentBucket): |
||||||
|
p.currentMetric.Histogram.Bucket = append( |
||||||
|
p.currentMetric.Histogram.Bucket, |
||||||
|
&dto.Bucket{ |
||||||
|
UpperBound: proto.Float64(p.currentBucket), |
||||||
|
CumulativeCount: proto.Uint64(uint64(value)), |
||||||
|
}, |
||||||
|
) |
||||||
|
} |
||||||
|
default: |
||||||
|
p.err = fmt.Errorf("unexpected type for metric name %q", p.currentMF.GetName()) |
||||||
|
} |
||||||
|
if p.currentByte == '\n' { |
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
return p.startTimestamp |
||||||
|
} |
||||||
|
|
||||||
|
// startTimestamp represents the state where the next byte read from p.buf is
|
||||||
|
// the start of the timestamp (or whitespace leading up to it).
|
||||||
|
func (p *TextParser) startTimestamp() stateFn { |
||||||
|
if p.skipBlankTab(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.readTokenUntilWhitespace(); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
timestamp, err := strconv.ParseInt(p.currentToken.String(), 10, 64) |
||||||
|
if err != nil { |
||||||
|
// Create a more helpful error message.
|
||||||
|
p.parseError(fmt.Sprintf("expected integer as timestamp, got %q", p.currentToken.String())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
p.currentMetric.TimestampMs = proto.Int64(timestamp) |
||||||
|
if p.readTokenUntilNewline(false); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
if p.currentToken.Len() > 0 { |
||||||
|
p.parseError(fmt.Sprintf("spurious string after timestamp: %q", p.currentToken.String())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
|
||||||
|
// readingHelp represents the state where the last byte read (now in
|
||||||
|
// p.currentByte) is the first byte of the docstring after 'HELP'.
|
||||||
|
func (p *TextParser) readingHelp() stateFn { |
||||||
|
if p.currentMF.Help != nil { |
||||||
|
p.parseError(fmt.Sprintf("second HELP line for metric name %q", p.currentMF.GetName())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
// Rest of line is the docstring.
|
||||||
|
if p.readTokenUntilNewline(true); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
p.currentMF.Help = proto.String(p.currentToken.String()) |
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
|
||||||
|
// readingType represents the state where the last byte read (now in
|
||||||
|
// p.currentByte) is the first byte of the type hint after 'HELP'.
|
||||||
|
func (p *TextParser) readingType() stateFn { |
||||||
|
if p.currentMF.Type != nil { |
||||||
|
p.parseError(fmt.Sprintf("second TYPE line for metric name %q, or TYPE reported after samples", p.currentMF.GetName())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
// Rest of line is the type.
|
||||||
|
if p.readTokenUntilNewline(false); p.err != nil { |
||||||
|
return nil // Unexpected end of input.
|
||||||
|
} |
||||||
|
metricType, ok := dto.MetricType_value[strings.ToUpper(p.currentToken.String())] |
||||||
|
if !ok { |
||||||
|
p.parseError(fmt.Sprintf("unknown metric type %q", p.currentToken.String())) |
||||||
|
return nil |
||||||
|
} |
||||||
|
p.currentMF.Type = dto.MetricType(metricType).Enum() |
||||||
|
return p.startOfLine |
||||||
|
} |
||||||
|
|
||||||
|
// parseError sets p.err to a ParseError at the current line with the given
|
||||||
|
// message.
|
||||||
|
func (p *TextParser) parseError(msg string) { |
||||||
|
p.err = ParseError{ |
||||||
|
Line: p.lineCount, |
||||||
|
Msg: msg, |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// skipBlankTab reads (and discards) bytes from p.buf until it encounters a byte
|
||||||
|
// that is neither ' ' nor '\t'. That byte is left in p.currentByte.
|
||||||
|
func (p *TextParser) skipBlankTab() { |
||||||
|
for { |
||||||
|
if p.currentByte, p.err = p.buf.ReadByte(); p.err != nil || !isBlankOrTab(p.currentByte) { |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// skipBlankTabIfCurrentBlankTab works exactly as skipBlankTab but doesn't do
|
||||||
|
// anything if p.currentByte is neither ' ' nor '\t'.
|
||||||
|
func (p *TextParser) skipBlankTabIfCurrentBlankTab() { |
||||||
|
if isBlankOrTab(p.currentByte) { |
||||||
|
p.skipBlankTab() |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// readTokenUntilWhitespace copies bytes from p.buf into p.currentToken. The
|
||||||
|
// first byte considered is the byte already read (now in p.currentByte). The
|
||||||
|
// first whitespace byte encountered is still copied into p.currentByte, but not
|
||||||
|
// into p.currentToken.
|
||||||
|
func (p *TextParser) readTokenUntilWhitespace() { |
||||||
|
p.currentToken.Reset() |
||||||
|
for p.err == nil && !isBlankOrTab(p.currentByte) && p.currentByte != '\n' { |
||||||
|
p.currentToken.WriteByte(p.currentByte) |
||||||
|
p.currentByte, p.err = p.buf.ReadByte() |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// readTokenUntilNewline copies bytes from p.buf into p.currentToken. The first
|
||||||
|
// byte considered is the byte already read (now in p.currentByte). The first
|
||||||
|
// newline byte encountered is still copied into p.currentByte, but not into
|
||||||
|
// p.currentToken. If recognizeEscapeSequence is true, two escape sequences are
|
||||||
|
// recognized: '\\' tranlates into '\', and '\n' into a line-feed character. All
|
||||||
|
// other escape sequences are invalid and cause an error.
|
||||||
|
func (p *TextParser) readTokenUntilNewline(recognizeEscapeSequence bool) { |
||||||
|
p.currentToken.Reset() |
||||||
|
escaped := false |
||||||
|
for p.err == nil { |
||||||
|
if recognizeEscapeSequence && escaped { |
||||||
|
switch p.currentByte { |
||||||
|
case '\\': |
||||||
|
p.currentToken.WriteByte(p.currentByte) |
||||||
|
case 'n': |
||||||
|
p.currentToken.WriteByte('\n') |
||||||
|
default: |
||||||
|
p.parseError(fmt.Sprintf("invalid escape sequence '\\%c'", p.currentByte)) |
||||||
|
return |
||||||
|
} |
||||||
|
escaped = false |
||||||
|
} else { |
||||||
|
switch p.currentByte { |
||||||
|
case '\n': |
||||||
|
return |
||||||
|
case '\\': |
||||||
|
escaped = true |
||||||
|
default: |
||||||
|
p.currentToken.WriteByte(p.currentByte) |
||||||
|
} |
||||||
|
} |
||||||
|
p.currentByte, p.err = p.buf.ReadByte() |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// readTokenAsMetricName copies a metric name from p.buf into p.currentToken.
|
||||||
|
// The first byte considered is the byte already read (now in p.currentByte).
|
||||||
|
// The first byte not part of a metric name is still copied into p.currentByte,
|
||||||
|
// but not into p.currentToken.
|
||||||
|
func (p *TextParser) readTokenAsMetricName() { |
||||||
|
p.currentToken.Reset() |
||||||
|
if !isValidMetricNameStart(p.currentByte) { |
||||||
|
return |
||||||
|
} |
||||||
|
for { |
||||||
|
p.currentToken.WriteByte(p.currentByte) |
||||||
|
p.currentByte, p.err = p.buf.ReadByte() |
||||||
|
if p.err != nil || !isValidMetricNameContinuation(p.currentByte) { |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// readTokenAsLabelName copies a label name from p.buf into p.currentToken.
|
||||||
|
// The first byte considered is the byte already read (now in p.currentByte).
|
||||||
|
// The first byte not part of a label name is still copied into p.currentByte,
|
||||||
|
// but not into p.currentToken.
|
||||||
|
func (p *TextParser) readTokenAsLabelName() { |
||||||
|
p.currentToken.Reset() |
||||||
|
if !isValidLabelNameStart(p.currentByte) { |
||||||
|
return |
||||||
|
} |
||||||
|
for { |
||||||
|
p.currentToken.WriteByte(p.currentByte) |
||||||
|
p.currentByte, p.err = p.buf.ReadByte() |
||||||
|
if p.err != nil || !isValidLabelNameContinuation(p.currentByte) { |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// readTokenAsLabelValue copies a label value from p.buf into p.currentToken.
|
||||||
|
// In contrast to the other 'readTokenAs...' functions, which start with the
|
||||||
|
// last read byte in p.currentByte, this method ignores p.currentByte and starts
|
||||||
|
// with reading a new byte from p.buf. The first byte not part of a label value
|
||||||
|
// is still copied into p.currentByte, but not into p.currentToken.
|
||||||
|
func (p *TextParser) readTokenAsLabelValue() { |
||||||
|
p.currentToken.Reset() |
||||||
|
escaped := false |
||||||
|
for { |
||||||
|
if p.currentByte, p.err = p.buf.ReadByte(); p.err != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
if escaped { |
||||||
|
switch p.currentByte { |
||||||
|
case '"', '\\': |
||||||
|
p.currentToken.WriteByte(p.currentByte) |
||||||
|
case 'n': |
||||||
|
p.currentToken.WriteByte('\n') |
||||||
|
default: |
||||||
|
p.parseError(fmt.Sprintf("invalid escape sequence '\\%c'", p.currentByte)) |
||||||
|
return |
||||||
|
} |
||||||
|
escaped = false |
||||||
|
continue |
||||||
|
} |
||||||
|
switch p.currentByte { |
||||||
|
case '"': |
||||||
|
return |
||||||
|
case '\n': |
||||||
|
p.parseError(fmt.Sprintf("label value %q contains unescaped new-line", p.currentToken.String())) |
||||||
|
return |
||||||
|
case '\\': |
||||||
|
escaped = true |
||||||
|
default: |
||||||
|
p.currentToken.WriteByte(p.currentByte) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (p *TextParser) setOrCreateCurrentMF() { |
||||||
|
p.currentIsSummaryCount = false |
||||||
|
p.currentIsSummarySum = false |
||||||
|
p.currentIsHistogramCount = false |
||||||
|
p.currentIsHistogramSum = false |
||||||
|
name := p.currentToken.String() |
||||||
|
if p.currentMF = p.metricFamiliesByName[name]; p.currentMF != nil { |
||||||
|
return |
||||||
|
} |
||||||
|
// Try out if this is a _sum or _count for a summary/histogram.
|
||||||
|
summaryName := summaryMetricName(name) |
||||||
|
if p.currentMF = p.metricFamiliesByName[summaryName]; p.currentMF != nil { |
||||||
|
if p.currentMF.GetType() == dto.MetricType_SUMMARY { |
||||||
|
if isCount(name) { |
||||||
|
p.currentIsSummaryCount = true |
||||||
|
} |
||||||
|
if isSum(name) { |
||||||
|
p.currentIsSummarySum = true |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
histogramName := histogramMetricName(name) |
||||||
|
if p.currentMF = p.metricFamiliesByName[histogramName]; p.currentMF != nil { |
||||||
|
if p.currentMF.GetType() == dto.MetricType_HISTOGRAM { |
||||||
|
if isCount(name) { |
||||||
|
p.currentIsHistogramCount = true |
||||||
|
} |
||||||
|
if isSum(name) { |
||||||
|
p.currentIsHistogramSum = true |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
p.currentMF = &dto.MetricFamily{Name: proto.String(name)} |
||||||
|
p.metricFamiliesByName[name] = p.currentMF |
||||||
|
} |
||||||
|
|
||||||
|
func isValidLabelNameStart(b byte) bool { |
||||||
|
return (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || b == '_' |
||||||
|
} |
||||||
|
|
||||||
|
func isValidLabelNameContinuation(b byte) bool { |
||||||
|
return isValidLabelNameStart(b) || (b >= '0' && b <= '9') |
||||||
|
} |
||||||
|
|
||||||
|
func isValidMetricNameStart(b byte) bool { |
||||||
|
return isValidLabelNameStart(b) || b == ':' |
||||||
|
} |
||||||
|
|
||||||
|
func isValidMetricNameContinuation(b byte) bool { |
||||||
|
return isValidLabelNameContinuation(b) || b == ':' |
||||||
|
} |
||||||
|
|
||||||
|
func isBlankOrTab(b byte) bool { |
||||||
|
return b == ' ' || b == '\t' |
||||||
|
} |
||||||
|
|
||||||
|
func isCount(name string) bool { |
||||||
|
return len(name) > 6 && name[len(name)-6:] == "_count" |
||||||
|
} |
||||||
|
|
||||||
|
func isSum(name string) bool { |
||||||
|
return len(name) > 4 && name[len(name)-4:] == "_sum" |
||||||
|
} |
||||||
|
|
||||||
|
func isBucket(name string) bool { |
||||||
|
return len(name) > 7 && name[len(name)-7:] == "_bucket" |
||||||
|
} |
||||||
|
|
||||||
|
func summaryMetricName(name string) string { |
||||||
|
switch { |
||||||
|
case isCount(name): |
||||||
|
return name[:len(name)-6] |
||||||
|
case isSum(name): |
||||||
|
return name[:len(name)-4] |
||||||
|
default: |
||||||
|
return name |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func histogramMetricName(name string) string { |
||||||
|
switch { |
||||||
|
case isCount(name): |
||||||
|
return name[:len(name)-6] |
||||||
|
case isSum(name): |
||||||
|
return name[:len(name)-4] |
||||||
|
case isBucket(name): |
||||||
|
return name[:len(name)-7] |
||||||
|
default: |
||||||
|
return name |
||||||
|
} |
||||||
|
} |
||||||
@ -0,0 +1,67 @@ |
|||||||
|
PACKAGE |
||||||
|
|
||||||
|
package goautoneg |
||||||
|
import "bitbucket.org/ww/goautoneg" |
||||||
|
|
||||||
|
HTTP Content-Type Autonegotiation. |
||||||
|
|
||||||
|
The functions in this package implement the behaviour specified in |
||||||
|
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html |
||||||
|
|
||||||
|
Copyright (c) 2011, Open Knowledge Foundation Ltd. |
||||||
|
All rights reserved. |
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without |
||||||
|
modification, are permitted provided that the following conditions are |
||||||
|
met: |
||||||
|
|
||||||
|
Redistributions of source code must retain the above copyright |
||||||
|
notice, this list of conditions and the following disclaimer. |
||||||
|
|
||||||
|
Redistributions in binary form must reproduce the above copyright |
||||||
|
notice, this list of conditions and the following disclaimer in |
||||||
|
the documentation and/or other materials provided with the |
||||||
|
distribution. |
||||||
|
|
||||||
|
Neither the name of the Open Knowledge Foundation Ltd. nor the |
||||||
|
names of its contributors may be used to endorse or promote |
||||||
|
products derived from this software without specific prior written |
||||||
|
permission. |
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||||
|
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||||
|
|
||||||
|
|
||||||
|
FUNCTIONS |
||||||
|
|
||||||
|
func Negotiate(header string, alternatives []string) (content_type string) |
||||||
|
Negotiate the most appropriate content_type given the accept header |
||||||
|
and a list of alternatives. |
||||||
|
|
||||||
|
func ParseAccept(header string) (accept []Accept) |
||||||
|
Parse an Accept Header string returning a sorted list |
||||||
|
of clauses |
||||||
|
|
||||||
|
|
||||||
|
TYPES |
||||||
|
|
||||||
|
type Accept struct { |
||||||
|
Type, SubType string |
||||||
|
Q float32 |
||||||
|
Params map[string]string |
||||||
|
} |
||||||
|
Structure to represent a clause in an HTTP Accept Header |
||||||
|
|
||||||
|
|
||||||
|
SUBDIRECTORIES |
||||||
|
|
||||||
|
.hg |
||||||
@ -0,0 +1,162 @@ |
|||||||
|
/* |
||||||
|
HTTP Content-Type Autonegotiation. |
||||||
|
|
||||||
|
The functions in this package implement the behaviour specified in |
||||||
|
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
|
||||||
|
|
||||||
|
Copyright (c) 2011, Open Knowledge Foundation Ltd. |
||||||
|
All rights reserved. |
||||||
|
|
||||||
|
Redistribution and use in source and binary forms, with or without |
||||||
|
modification, are permitted provided that the following conditions are |
||||||
|
met: |
||||||
|
|
||||||
|
Redistributions of source code must retain the above copyright |
||||||
|
notice, this list of conditions and the following disclaimer. |
||||||
|
|
||||||
|
Redistributions in binary form must reproduce the above copyright |
||||||
|
notice, this list of conditions and the following disclaimer in |
||||||
|
the documentation and/or other materials provided with the |
||||||
|
distribution. |
||||||
|
|
||||||
|
Neither the name of the Open Knowledge Foundation Ltd. nor the |
||||||
|
names of its contributors may be used to endorse or promote |
||||||
|
products derived from this software without specific prior written |
||||||
|
permission. |
||||||
|
|
||||||
|
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS |
||||||
|
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT |
||||||
|
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR |
||||||
|
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT |
||||||
|
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, |
||||||
|
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT |
||||||
|
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |
||||||
|
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY |
||||||
|
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT |
||||||
|
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE |
||||||
|
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
||||||
|
|
||||||
|
|
||||||
|
*/ |
||||||
|
package goautoneg |
||||||
|
|
||||||
|
import ( |
||||||
|
"sort" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
// Structure to represent a clause in an HTTP Accept Header
|
||||||
|
type Accept struct { |
||||||
|
Type, SubType string |
||||||
|
Q float64 |
||||||
|
Params map[string]string |
||||||
|
} |
||||||
|
|
||||||
|
// For internal use, so that we can use the sort interface
|
||||||
|
type accept_slice []Accept |
||||||
|
|
||||||
|
func (accept accept_slice) Len() int { |
||||||
|
slice := []Accept(accept) |
||||||
|
return len(slice) |
||||||
|
} |
||||||
|
|
||||||
|
func (accept accept_slice) Less(i, j int) bool { |
||||||
|
slice := []Accept(accept) |
||||||
|
ai, aj := slice[i], slice[j] |
||||||
|
if ai.Q > aj.Q { |
||||||
|
return true |
||||||
|
} |
||||||
|
if ai.Type != "*" && aj.Type == "*" { |
||||||
|
return true |
||||||
|
} |
||||||
|
if ai.SubType != "*" && aj.SubType == "*" { |
||||||
|
return true |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
func (accept accept_slice) Swap(i, j int) { |
||||||
|
slice := []Accept(accept) |
||||||
|
slice[i], slice[j] = slice[j], slice[i] |
||||||
|
} |
||||||
|
|
||||||
|
// Parse an Accept Header string returning a sorted list
|
||||||
|
// of clauses
|
||||||
|
func ParseAccept(header string) (accept []Accept) { |
||||||
|
parts := strings.Split(header, ",") |
||||||
|
accept = make([]Accept, 0, len(parts)) |
||||||
|
for _, part := range parts { |
||||||
|
part := strings.Trim(part, " ") |
||||||
|
|
||||||
|
a := Accept{} |
||||||
|
a.Params = make(map[string]string) |
||||||
|
a.Q = 1.0 |
||||||
|
|
||||||
|
mrp := strings.Split(part, ";") |
||||||
|
|
||||||
|
media_range := mrp[0] |
||||||
|
sp := strings.Split(media_range, "/") |
||||||
|
a.Type = strings.Trim(sp[0], " ") |
||||||
|
|
||||||
|
switch { |
||||||
|
case len(sp) == 1 && a.Type == "*": |
||||||
|
a.SubType = "*" |
||||||
|
case len(sp) == 2: |
||||||
|
a.SubType = strings.Trim(sp[1], " ") |
||||||
|
default: |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
if len(mrp) == 1 { |
||||||
|
accept = append(accept, a) |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
for _, param := range mrp[1:] { |
||||||
|
sp := strings.SplitN(param, "=", 2) |
||||||
|
if len(sp) != 2 { |
||||||
|
continue |
||||||
|
} |
||||||
|
token := strings.Trim(sp[0], " ") |
||||||
|
if token == "q" { |
||||||
|
a.Q, _ = strconv.ParseFloat(sp[1], 32) |
||||||
|
} else { |
||||||
|
a.Params[token] = strings.Trim(sp[1], " ") |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
accept = append(accept, a) |
||||||
|
} |
||||||
|
|
||||||
|
slice := accept_slice(accept) |
||||||
|
sort.Sort(slice) |
||||||
|
|
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
// Negotiate the most appropriate content_type given the accept header
|
||||||
|
// and a list of alternatives.
|
||||||
|
func Negotiate(header string, alternatives []string) (content_type string) { |
||||||
|
asp := make([][]string, 0, len(alternatives)) |
||||||
|
for _, ctype := range alternatives { |
||||||
|
asp = append(asp, strings.SplitN(ctype, "/", 2)) |
||||||
|
} |
||||||
|
for _, clause := range ParseAccept(header) { |
||||||
|
for i, ctsp := range asp { |
||||||
|
if clause.Type == ctsp[0] && clause.SubType == ctsp[1] { |
||||||
|
content_type = alternatives[i] |
||||||
|
return |
||||||
|
} |
||||||
|
if clause.Type == ctsp[0] && clause.SubType == "*" { |
||||||
|
content_type = alternatives[i] |
||||||
|
return |
||||||
|
} |
||||||
|
if clause.Type == "*" && clause.SubType == "*" { |
||||||
|
content_type = alternatives[i] |
||||||
|
return |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
@ -0,0 +1,89 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// +build windows
|
||||||
|
|
||||||
|
package log |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"os" |
||||||
|
|
||||||
|
"golang.org/x/sys/windows/svc/eventlog" |
||||||
|
|
||||||
|
"github.com/Sirupsen/logrus" |
||||||
|
) |
||||||
|
|
||||||
|
func init() { |
||||||
|
setEventlogFormatter = func(name string, debugAsInfo bool) error { |
||||||
|
if name == "" { |
||||||
|
return fmt.Errorf("missing name parameter") |
||||||
|
} |
||||||
|
|
||||||
|
fmter, err := newEventlogger(name, debugAsInfo, origLogger.Formatter) |
||||||
|
if err != nil { |
||||||
|
fmt.Fprintf(os.Stderr, "error creating eventlog formatter: %v\n", err) |
||||||
|
origLogger.Errorf("can't connect logger to eventlog: %v", err) |
||||||
|
return err |
||||||
|
} |
||||||
|
origLogger.Formatter = fmter |
||||||
|
return nil |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
type eventlogger struct { |
||||||
|
log *eventlog.Log |
||||||
|
debugAsInfo bool |
||||||
|
wrap logrus.Formatter |
||||||
|
} |
||||||
|
|
||||||
|
func newEventlogger(name string, debugAsInfo bool, fmter logrus.Formatter) (*eventlogger, error) { |
||||||
|
logHandle, err := eventlog.Open(name) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
return &eventlogger{log: logHandle, debugAsInfo: debugAsInfo, wrap: fmter}, nil |
||||||
|
} |
||||||
|
|
||||||
|
func (s *eventlogger) Format(e *logrus.Entry) ([]byte, error) { |
||||||
|
data, err := s.wrap.Format(e) |
||||||
|
if err != nil { |
||||||
|
fmt.Fprintf(os.Stderr, "eventlogger: can't format entry: %v\n", err) |
||||||
|
return data, err |
||||||
|
} |
||||||
|
|
||||||
|
switch e.Level { |
||||||
|
case logrus.PanicLevel: |
||||||
|
fallthrough |
||||||
|
case logrus.FatalLevel: |
||||||
|
fallthrough |
||||||
|
case logrus.ErrorLevel: |
||||||
|
err = s.log.Error(102, e.Message) |
||||||
|
case logrus.WarnLevel: |
||||||
|
err = s.log.Warning(101, e.Message) |
||||||
|
case logrus.InfoLevel: |
||||||
|
err = s.log.Info(100, e.Message) |
||||||
|
case logrus.DebugLevel: |
||||||
|
if s.debugAsInfo { |
||||||
|
err = s.log.Info(100, e.Message) |
||||||
|
} |
||||||
|
default: |
||||||
|
err = s.log.Info(100, e.Message) |
||||||
|
} |
||||||
|
|
||||||
|
if err != nil { |
||||||
|
fmt.Fprintf(os.Stderr, "eventlogger: can't send log to eventlog: %v\n", err) |
||||||
|
} |
||||||
|
|
||||||
|
return data, err |
||||||
|
} |
||||||
@ -0,0 +1,365 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package log |
||||||
|
|
||||||
|
import ( |
||||||
|
"flag" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"io/ioutil" |
||||||
|
"log" |
||||||
|
"net/url" |
||||||
|
"os" |
||||||
|
"runtime" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
|
||||||
|
"github.com/Sirupsen/logrus" |
||||||
|
) |
||||||
|
|
||||||
|
type levelFlag string |
||||||
|
|
||||||
|
// String implements flag.Value.
|
||||||
|
func (f levelFlag) String() string { |
||||||
|
return fmt.Sprintf("%q", string(f)) |
||||||
|
} |
||||||
|
|
||||||
|
// Set implements flag.Value.
|
||||||
|
func (f levelFlag) Set(level string) error { |
||||||
|
l, err := logrus.ParseLevel(level) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
origLogger.Level = l |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// setSyslogFormatter is nil if the target architecture does not support syslog.
|
||||||
|
var setSyslogFormatter func(string, string) error |
||||||
|
|
||||||
|
// setEventlogFormatter is nil if the target OS does not support Eventlog (i.e., is not Windows).
|
||||||
|
var setEventlogFormatter func(string, bool) error |
||||||
|
|
||||||
|
func setJSONFormatter() { |
||||||
|
origLogger.Formatter = &logrus.JSONFormatter{} |
||||||
|
} |
||||||
|
|
||||||
|
type logFormatFlag url.URL |
||||||
|
|
||||||
|
// String implements flag.Value.
|
||||||
|
func (f logFormatFlag) String() string { |
||||||
|
u := url.URL(f) |
||||||
|
return fmt.Sprintf("%q", u.String()) |
||||||
|
} |
||||||
|
|
||||||
|
// Set implements flag.Value.
|
||||||
|
func (f logFormatFlag) Set(format string) error { |
||||||
|
u, err := url.Parse(format) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if u.Scheme != "logger" { |
||||||
|
return fmt.Errorf("invalid scheme %s", u.Scheme) |
||||||
|
} |
||||||
|
jsonq := u.Query().Get("json") |
||||||
|
if jsonq == "true" { |
||||||
|
setJSONFormatter() |
||||||
|
} |
||||||
|
|
||||||
|
switch u.Opaque { |
||||||
|
case "syslog": |
||||||
|
if setSyslogFormatter == nil { |
||||||
|
return fmt.Errorf("system does not support syslog") |
||||||
|
} |
||||||
|
appname := u.Query().Get("appname") |
||||||
|
facility := u.Query().Get("local") |
||||||
|
return setSyslogFormatter(appname, facility) |
||||||
|
case "eventlog": |
||||||
|
if setEventlogFormatter == nil { |
||||||
|
return fmt.Errorf("system does not support eventlog") |
||||||
|
} |
||||||
|
name := u.Query().Get("name") |
||||||
|
debugAsInfo := false |
||||||
|
debugAsInfoRaw := u.Query().Get("debugAsInfo") |
||||||
|
if parsedDebugAsInfo, err := strconv.ParseBool(debugAsInfoRaw); err == nil { |
||||||
|
debugAsInfo = parsedDebugAsInfo |
||||||
|
} |
||||||
|
return setEventlogFormatter(name, debugAsInfo) |
||||||
|
case "stdout": |
||||||
|
origLogger.Out = os.Stdout |
||||||
|
case "stderr": |
||||||
|
origLogger.Out = os.Stderr |
||||||
|
default: |
||||||
|
return fmt.Errorf("unsupported logger %q", u.Opaque) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func init() { |
||||||
|
AddFlags(flag.CommandLine) |
||||||
|
} |
||||||
|
|
||||||
|
// AddFlags adds the flags used by this package to the given FlagSet. That's
|
||||||
|
// useful if working with a custom FlagSet. The init function of this package
|
||||||
|
// adds the flags to flag.CommandLine anyway. Thus, it's usually enough to call
|
||||||
|
// flag.Parse() to make the logging flags take effect.
|
||||||
|
func AddFlags(fs *flag.FlagSet) { |
||||||
|
fs.Var( |
||||||
|
levelFlag(origLogger.Level.String()), |
||||||
|
"log.level", |
||||||
|
"Only log messages with the given severity or above. Valid levels: [debug, info, warn, error, fatal]", |
||||||
|
) |
||||||
|
fs.Var( |
||||||
|
logFormatFlag(url.URL{Scheme: "logger", Opaque: "stderr"}), |
||||||
|
"log.format", |
||||||
|
`Set the log target and format. Example: "logger:syslog?appname=bob&local=7" or "logger:stdout?json=true"`, |
||||||
|
) |
||||||
|
} |
||||||
|
|
||||||
|
// Logger is the interface for loggers used in the Prometheus components.
|
||||||
|
type Logger interface { |
||||||
|
Debug(...interface{}) |
||||||
|
Debugln(...interface{}) |
||||||
|
Debugf(string, ...interface{}) |
||||||
|
|
||||||
|
Info(...interface{}) |
||||||
|
Infoln(...interface{}) |
||||||
|
Infof(string, ...interface{}) |
||||||
|
|
||||||
|
Warn(...interface{}) |
||||||
|
Warnln(...interface{}) |
||||||
|
Warnf(string, ...interface{}) |
||||||
|
|
||||||
|
Error(...interface{}) |
||||||
|
Errorln(...interface{}) |
||||||
|
Errorf(string, ...interface{}) |
||||||
|
|
||||||
|
Fatal(...interface{}) |
||||||
|
Fatalln(...interface{}) |
||||||
|
Fatalf(string, ...interface{}) |
||||||
|
|
||||||
|
With(key string, value interface{}) Logger |
||||||
|
} |
||||||
|
|
||||||
|
type logger struct { |
||||||
|
entry *logrus.Entry |
||||||
|
} |
||||||
|
|
||||||
|
func (l logger) With(key string, value interface{}) Logger { |
||||||
|
return logger{l.entry.WithField(key, value)} |
||||||
|
} |
||||||
|
|
||||||
|
// Debug logs a message at level Debug on the standard logger.
|
||||||
|
func (l logger) Debug(args ...interface{}) { |
||||||
|
l.sourced().Debug(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Debug logs a message at level Debug on the standard logger.
|
||||||
|
func (l logger) Debugln(args ...interface{}) { |
||||||
|
l.sourced().Debugln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Debugf logs a message at level Debug on the standard logger.
|
||||||
|
func (l logger) Debugf(format string, args ...interface{}) { |
||||||
|
l.sourced().Debugf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Info logs a message at level Info on the standard logger.
|
||||||
|
func (l logger) Info(args ...interface{}) { |
||||||
|
l.sourced().Info(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Info logs a message at level Info on the standard logger.
|
||||||
|
func (l logger) Infoln(args ...interface{}) { |
||||||
|
l.sourced().Infoln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Infof logs a message at level Info on the standard logger.
|
||||||
|
func (l logger) Infof(format string, args ...interface{}) { |
||||||
|
l.sourced().Infof(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warn logs a message at level Warn on the standard logger.
|
||||||
|
func (l logger) Warn(args ...interface{}) { |
||||||
|
l.sourced().Warn(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warn logs a message at level Warn on the standard logger.
|
||||||
|
func (l logger) Warnln(args ...interface{}) { |
||||||
|
l.sourced().Warnln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warnf logs a message at level Warn on the standard logger.
|
||||||
|
func (l logger) Warnf(format string, args ...interface{}) { |
||||||
|
l.sourced().Warnf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Error logs a message at level Error on the standard logger.
|
||||||
|
func (l logger) Error(args ...interface{}) { |
||||||
|
l.sourced().Error(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Error logs a message at level Error on the standard logger.
|
||||||
|
func (l logger) Errorln(args ...interface{}) { |
||||||
|
l.sourced().Errorln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Errorf logs a message at level Error on the standard logger.
|
||||||
|
func (l logger) Errorf(format string, args ...interface{}) { |
||||||
|
l.sourced().Errorf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatal logs a message at level Fatal on the standard logger.
|
||||||
|
func (l logger) Fatal(args ...interface{}) { |
||||||
|
l.sourced().Fatal(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatal logs a message at level Fatal on the standard logger.
|
||||||
|
func (l logger) Fatalln(args ...interface{}) { |
||||||
|
l.sourced().Fatalln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatalf logs a message at level Fatal on the standard logger.
|
||||||
|
func (l logger) Fatalf(format string, args ...interface{}) { |
||||||
|
l.sourced().Fatalf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// sourced adds a source field to the logger that contains
|
||||||
|
// the file name and line where the logging happened.
|
||||||
|
func (l logger) sourced() *logrus.Entry { |
||||||
|
_, file, line, ok := runtime.Caller(2) |
||||||
|
if !ok { |
||||||
|
file = "<???>" |
||||||
|
line = 1 |
||||||
|
} else { |
||||||
|
slash := strings.LastIndex(file, "/") |
||||||
|
file = file[slash+1:] |
||||||
|
} |
||||||
|
return l.entry.WithField("source", fmt.Sprintf("%s:%d", file, line)) |
||||||
|
} |
||||||
|
|
||||||
|
var origLogger = logrus.New() |
||||||
|
var baseLogger = logger{entry: logrus.NewEntry(origLogger)} |
||||||
|
|
||||||
|
// Base returns the default Logger logging to
|
||||||
|
func Base() Logger { |
||||||
|
return baseLogger |
||||||
|
} |
||||||
|
|
||||||
|
// NewLogger returns a new Logger logging to out.
|
||||||
|
func NewLogger(w io.Writer) Logger { |
||||||
|
l := logrus.New() |
||||||
|
l.Out = w |
||||||
|
return logger{entry: logrus.NewEntry(l)} |
||||||
|
} |
||||||
|
|
||||||
|
// NewNopLogger returns a logger that discards all log messages.
|
||||||
|
func NewNopLogger() Logger { |
||||||
|
l := logrus.New() |
||||||
|
l.Out = ioutil.Discard |
||||||
|
return logger{entry: logrus.NewEntry(l)} |
||||||
|
} |
||||||
|
|
||||||
|
// With adds a field to the logger.
|
||||||
|
func With(key string, value interface{}) Logger { |
||||||
|
return baseLogger.With(key, value) |
||||||
|
} |
||||||
|
|
||||||
|
// Debug logs a message at level Debug on the standard logger.
|
||||||
|
func Debug(args ...interface{}) { |
||||||
|
baseLogger.sourced().Debug(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Debugln logs a message at level Debug on the standard logger.
|
||||||
|
func Debugln(args ...interface{}) { |
||||||
|
baseLogger.sourced().Debugln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Debugf logs a message at level Debug on the standard logger.
|
||||||
|
func Debugf(format string, args ...interface{}) { |
||||||
|
baseLogger.sourced().Debugf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Info logs a message at level Info on the standard logger.
|
||||||
|
func Info(args ...interface{}) { |
||||||
|
baseLogger.sourced().Info(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Infoln logs a message at level Info on the standard logger.
|
||||||
|
func Infoln(args ...interface{}) { |
||||||
|
baseLogger.sourced().Infoln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Infof logs a message at level Info on the standard logger.
|
||||||
|
func Infof(format string, args ...interface{}) { |
||||||
|
baseLogger.sourced().Infof(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warn logs a message at level Warn on the standard logger.
|
||||||
|
func Warn(args ...interface{}) { |
||||||
|
baseLogger.sourced().Warn(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warnln logs a message at level Warn on the standard logger.
|
||||||
|
func Warnln(args ...interface{}) { |
||||||
|
baseLogger.sourced().Warnln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Warnf logs a message at level Warn on the standard logger.
|
||||||
|
func Warnf(format string, args ...interface{}) { |
||||||
|
baseLogger.sourced().Warnf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Error logs a message at level Error on the standard logger.
|
||||||
|
func Error(args ...interface{}) { |
||||||
|
baseLogger.sourced().Error(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Errorln logs a message at level Error on the standard logger.
|
||||||
|
func Errorln(args ...interface{}) { |
||||||
|
baseLogger.sourced().Errorln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Errorf logs a message at level Error on the standard logger.
|
||||||
|
func Errorf(format string, args ...interface{}) { |
||||||
|
baseLogger.sourced().Errorf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatal logs a message at level Fatal on the standard logger.
|
||||||
|
func Fatal(args ...interface{}) { |
||||||
|
baseLogger.sourced().Fatal(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatalln logs a message at level Fatal on the standard logger.
|
||||||
|
func Fatalln(args ...interface{}) { |
||||||
|
baseLogger.sourced().Fatalln(args...) |
||||||
|
} |
||||||
|
|
||||||
|
// Fatalf logs a message at level Fatal on the standard logger.
|
||||||
|
func Fatalf(format string, args ...interface{}) { |
||||||
|
baseLogger.sourced().Fatalf(format, args...) |
||||||
|
} |
||||||
|
|
||||||
|
type errorLogWriter struct{} |
||||||
|
|
||||||
|
func (errorLogWriter) Write(b []byte) (int, error) { |
||||||
|
baseLogger.sourced().Error(string(b)) |
||||||
|
return len(b), nil |
||||||
|
} |
||||||
|
|
||||||
|
// NewErrorLogger returns a log.Logger that is meant to be used
|
||||||
|
// in the ErrorLog field of an http.Server to log HTTP server errors.
|
||||||
|
func NewErrorLogger() *log.Logger { |
||||||
|
return log.New(&errorLogWriter{}, "", 0) |
||||||
|
} |
||||||
@ -0,0 +1,119 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// +build !windows,!nacl,!plan9
|
||||||
|
|
||||||
|
package log |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"log/syslog" |
||||||
|
"os" |
||||||
|
|
||||||
|
"github.com/Sirupsen/logrus" |
||||||
|
) |
||||||
|
|
||||||
|
func init() { |
||||||
|
setSyslogFormatter = func(appname, local string) error { |
||||||
|
if appname == "" { |
||||||
|
return fmt.Errorf("missing appname parameter") |
||||||
|
} |
||||||
|
if local == "" { |
||||||
|
return fmt.Errorf("missing local parameter") |
||||||
|
} |
||||||
|
|
||||||
|
fmter, err := newSyslogger(appname, local, origLogger.Formatter) |
||||||
|
if err != nil { |
||||||
|
fmt.Fprintf(os.Stderr, "error creating syslog formatter: %v\n", err) |
||||||
|
origLogger.Errorf("can't connect logger to syslog: %v", err) |
||||||
|
return err |
||||||
|
} |
||||||
|
origLogger.Formatter = fmter |
||||||
|
return nil |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
var ceeTag = []byte("@cee:") |
||||||
|
|
||||||
|
type syslogger struct { |
||||||
|
wrap logrus.Formatter |
||||||
|
out *syslog.Writer |
||||||
|
} |
||||||
|
|
||||||
|
func newSyslogger(appname string, facility string, fmter logrus.Formatter) (*syslogger, error) { |
||||||
|
priority, err := getFacility(facility) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
out, err := syslog.New(priority, appname) |
||||||
|
return &syslogger{ |
||||||
|
out: out, |
||||||
|
wrap: fmter, |
||||||
|
}, err |
||||||
|
} |
||||||
|
|
||||||
|
func getFacility(facility string) (syslog.Priority, error) { |
||||||
|
switch facility { |
||||||
|
case "0": |
||||||
|
return syslog.LOG_LOCAL0, nil |
||||||
|
case "1": |
||||||
|
return syslog.LOG_LOCAL1, nil |
||||||
|
case "2": |
||||||
|
return syslog.LOG_LOCAL2, nil |
||||||
|
case "3": |
||||||
|
return syslog.LOG_LOCAL3, nil |
||||||
|
case "4": |
||||||
|
return syslog.LOG_LOCAL4, nil |
||||||
|
case "5": |
||||||
|
return syslog.LOG_LOCAL5, nil |
||||||
|
case "6": |
||||||
|
return syslog.LOG_LOCAL6, nil |
||||||
|
case "7": |
||||||
|
return syslog.LOG_LOCAL7, nil |
||||||
|
} |
||||||
|
return syslog.LOG_LOCAL0, fmt.Errorf("invalid local(%s) for syslog", facility) |
||||||
|
} |
||||||
|
|
||||||
|
func (s *syslogger) Format(e *logrus.Entry) ([]byte, error) { |
||||||
|
data, err := s.wrap.Format(e) |
||||||
|
if err != nil { |
||||||
|
fmt.Fprintf(os.Stderr, "syslogger: can't format entry: %v\n", err) |
||||||
|
return data, err |
||||||
|
} |
||||||
|
// only append tag to data sent to syslog (line), not to what
|
||||||
|
// is returned
|
||||||
|
line := string(append(ceeTag, data...)) |
||||||
|
|
||||||
|
switch e.Level { |
||||||
|
case logrus.PanicLevel: |
||||||
|
err = s.out.Crit(line) |
||||||
|
case logrus.FatalLevel: |
||||||
|
err = s.out.Crit(line) |
||||||
|
case logrus.ErrorLevel: |
||||||
|
err = s.out.Err(line) |
||||||
|
case logrus.WarnLevel: |
||||||
|
err = s.out.Warning(line) |
||||||
|
case logrus.InfoLevel: |
||||||
|
err = s.out.Info(line) |
||||||
|
case logrus.DebugLevel: |
||||||
|
err = s.out.Debug(line) |
||||||
|
default: |
||||||
|
err = s.out.Notice(line) |
||||||
|
} |
||||||
|
|
||||||
|
if err != nil { |
||||||
|
fmt.Fprintf(os.Stderr, "syslogger: can't send log to syslog: %v\n", err) |
||||||
|
} |
||||||
|
|
||||||
|
return data, err |
||||||
|
} |
||||||
@ -0,0 +1,136 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"time" |
||||||
|
) |
||||||
|
|
||||||
|
type AlertStatus string |
||||||
|
|
||||||
|
const ( |
||||||
|
AlertFiring AlertStatus = "firing" |
||||||
|
AlertResolved AlertStatus = "resolved" |
||||||
|
) |
||||||
|
|
||||||
|
// Alert is a generic representation of an alert in the Prometheus eco-system.
|
||||||
|
type Alert struct { |
||||||
|
// Label value pairs for purpose of aggregation, matching, and disposition
|
||||||
|
// dispatching. This must minimally include an "alertname" label.
|
||||||
|
Labels LabelSet `json:"labels"` |
||||||
|
|
||||||
|
// Extra key/value information which does not define alert identity.
|
||||||
|
Annotations LabelSet `json:"annotations"` |
||||||
|
|
||||||
|
// The known time range for this alert. Both ends are optional.
|
||||||
|
StartsAt time.Time `json:"startsAt,omitempty"` |
||||||
|
EndsAt time.Time `json:"endsAt,omitempty"` |
||||||
|
GeneratorURL string `json:"generatorURL"` |
||||||
|
} |
||||||
|
|
||||||
|
// Name returns the name of the alert. It is equivalent to the "alertname" label.
|
||||||
|
func (a *Alert) Name() string { |
||||||
|
return string(a.Labels[AlertNameLabel]) |
||||||
|
} |
||||||
|
|
||||||
|
// Fingerprint returns a unique hash for the alert. It is equivalent to
|
||||||
|
// the fingerprint of the alert's label set.
|
||||||
|
func (a *Alert) Fingerprint() Fingerprint { |
||||||
|
return a.Labels.Fingerprint() |
||||||
|
} |
||||||
|
|
||||||
|
func (a *Alert) String() string { |
||||||
|
s := fmt.Sprintf("%s[%s]", a.Name(), a.Fingerprint().String()[:7]) |
||||||
|
if a.Resolved() { |
||||||
|
return s + "[resolved]" |
||||||
|
} |
||||||
|
return s + "[active]" |
||||||
|
} |
||||||
|
|
||||||
|
// Resolved returns true iff the activity interval ended in the past.
|
||||||
|
func (a *Alert) Resolved() bool { |
||||||
|
return a.ResolvedAt(time.Now()) |
||||||
|
} |
||||||
|
|
||||||
|
// ResolvedAt returns true off the activity interval ended before
|
||||||
|
// the given timestamp.
|
||||||
|
func (a *Alert) ResolvedAt(ts time.Time) bool { |
||||||
|
if a.EndsAt.IsZero() { |
||||||
|
return false |
||||||
|
} |
||||||
|
return !a.EndsAt.After(ts) |
||||||
|
} |
||||||
|
|
||||||
|
// Status returns the status of the alert.
|
||||||
|
func (a *Alert) Status() AlertStatus { |
||||||
|
if a.Resolved() { |
||||||
|
return AlertResolved |
||||||
|
} |
||||||
|
return AlertFiring |
||||||
|
} |
||||||
|
|
||||||
|
// Validate checks whether the alert data is inconsistent.
|
||||||
|
func (a *Alert) Validate() error { |
||||||
|
if a.StartsAt.IsZero() { |
||||||
|
return fmt.Errorf("start time missing") |
||||||
|
} |
||||||
|
if !a.EndsAt.IsZero() && a.EndsAt.Before(a.StartsAt) { |
||||||
|
return fmt.Errorf("start time must be before end time") |
||||||
|
} |
||||||
|
if err := a.Labels.Validate(); err != nil { |
||||||
|
return fmt.Errorf("invalid label set: %s", err) |
||||||
|
} |
||||||
|
if len(a.Labels) == 0 { |
||||||
|
return fmt.Errorf("at least one label pair required") |
||||||
|
} |
||||||
|
if err := a.Annotations.Validate(); err != nil { |
||||||
|
return fmt.Errorf("invalid annotations: %s", err) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Alert is a list of alerts that can be sorted in chronological order.
|
||||||
|
type Alerts []*Alert |
||||||
|
|
||||||
|
func (as Alerts) Len() int { return len(as) } |
||||||
|
func (as Alerts) Swap(i, j int) { as[i], as[j] = as[j], as[i] } |
||||||
|
|
||||||
|
func (as Alerts) Less(i, j int) bool { |
||||||
|
if as[i].StartsAt.Before(as[j].StartsAt) { |
||||||
|
return true |
||||||
|
} |
||||||
|
if as[i].EndsAt.Before(as[j].EndsAt) { |
||||||
|
return true |
||||||
|
} |
||||||
|
return as[i].Fingerprint() < as[j].Fingerprint() |
||||||
|
} |
||||||
|
|
||||||
|
// HasFiring returns true iff one of the alerts is not resolved.
|
||||||
|
func (as Alerts) HasFiring() bool { |
||||||
|
for _, a := range as { |
||||||
|
if !a.Resolved() { |
||||||
|
return true |
||||||
|
} |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
// Status returns StatusFiring iff at least one of the alerts is firing.
|
||||||
|
func (as Alerts) Status() AlertStatus { |
||||||
|
if as.HasFiring() { |
||||||
|
return AlertFiring |
||||||
|
} |
||||||
|
return AlertResolved |
||||||
|
} |
||||||
@ -0,0 +1,105 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"strconv" |
||||||
|
) |
||||||
|
|
||||||
|
// Fingerprint provides a hash-capable representation of a Metric.
|
||||||
|
// For our purposes, FNV-1A 64-bit is used.
|
||||||
|
type Fingerprint uint64 |
||||||
|
|
||||||
|
// FingerprintFromString transforms a string representation into a Fingerprint.
|
||||||
|
func FingerprintFromString(s string) (Fingerprint, error) { |
||||||
|
num, err := strconv.ParseUint(s, 16, 64) |
||||||
|
return Fingerprint(num), err |
||||||
|
} |
||||||
|
|
||||||
|
// ParseFingerprint parses the input string into a fingerprint.
|
||||||
|
func ParseFingerprint(s string) (Fingerprint, error) { |
||||||
|
num, err := strconv.ParseUint(s, 16, 64) |
||||||
|
if err != nil { |
||||||
|
return 0, err |
||||||
|
} |
||||||
|
return Fingerprint(num), nil |
||||||
|
} |
||||||
|
|
||||||
|
func (f Fingerprint) String() string { |
||||||
|
return fmt.Sprintf("%016x", uint64(f)) |
||||||
|
} |
||||||
|
|
||||||
|
// Fingerprints represents a collection of Fingerprint subject to a given
|
||||||
|
// natural sorting scheme. It implements sort.Interface.
|
||||||
|
type Fingerprints []Fingerprint |
||||||
|
|
||||||
|
// Len implements sort.Interface.
|
||||||
|
func (f Fingerprints) Len() int { |
||||||
|
return len(f) |
||||||
|
} |
||||||
|
|
||||||
|
// Less implements sort.Interface.
|
||||||
|
func (f Fingerprints) Less(i, j int) bool { |
||||||
|
return f[i] < f[j] |
||||||
|
} |
||||||
|
|
||||||
|
// Swap implements sort.Interface.
|
||||||
|
func (f Fingerprints) Swap(i, j int) { |
||||||
|
f[i], f[j] = f[j], f[i] |
||||||
|
} |
||||||
|
|
||||||
|
// FingerprintSet is a set of Fingerprints.
|
||||||
|
type FingerprintSet map[Fingerprint]struct{} |
||||||
|
|
||||||
|
// Equal returns true if both sets contain the same elements (and not more).
|
||||||
|
func (s FingerprintSet) Equal(o FingerprintSet) bool { |
||||||
|
if len(s) != len(o) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
for k := range s { |
||||||
|
if _, ok := o[k]; !ok { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// Intersection returns the elements contained in both sets.
|
||||||
|
func (s FingerprintSet) Intersection(o FingerprintSet) FingerprintSet { |
||||||
|
myLength, otherLength := len(s), len(o) |
||||||
|
if myLength == 0 || otherLength == 0 { |
||||||
|
return FingerprintSet{} |
||||||
|
} |
||||||
|
|
||||||
|
subSet := s |
||||||
|
superSet := o |
||||||
|
|
||||||
|
if otherLength < myLength { |
||||||
|
subSet = o |
||||||
|
superSet = s |
||||||
|
} |
||||||
|
|
||||||
|
out := FingerprintSet{} |
||||||
|
|
||||||
|
for k := range subSet { |
||||||
|
if _, ok := superSet[k]; ok { |
||||||
|
out[k] = struct{}{} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return out |
||||||
|
} |
||||||
@ -0,0 +1,42 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
// Inline and byte-free variant of hash/fnv's fnv64a.
|
||||||
|
|
||||||
|
const ( |
||||||
|
offset64 = 14695981039346656037 |
||||||
|
prime64 = 1099511628211 |
||||||
|
) |
||||||
|
|
||||||
|
// hashNew initializies a new fnv64a hash value.
|
||||||
|
func hashNew() uint64 { |
||||||
|
return offset64 |
||||||
|
} |
||||||
|
|
||||||
|
// hashAdd adds a string to a fnv64a hash value, returning the updated hash.
|
||||||
|
func hashAdd(h uint64, s string) uint64 { |
||||||
|
for i := 0; i < len(s); i++ { |
||||||
|
h ^= uint64(s[i]) |
||||||
|
h *= prime64 |
||||||
|
} |
||||||
|
return h |
||||||
|
} |
||||||
|
|
||||||
|
// hashAddByte adds a byte to a fnv64a hash value, returning the updated hash.
|
||||||
|
func hashAddByte(h uint64, b byte) uint64 { |
||||||
|
h ^= uint64(b) |
||||||
|
h *= prime64 |
||||||
|
return h |
||||||
|
} |
||||||
@ -0,0 +1,210 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"fmt" |
||||||
|
"regexp" |
||||||
|
"strings" |
||||||
|
"unicode/utf8" |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
// AlertNameLabel is the name of the label containing the an alert's name.
|
||||||
|
AlertNameLabel = "alertname" |
||||||
|
|
||||||
|
// ExportedLabelPrefix is the prefix to prepend to the label names present in
|
||||||
|
// exported metrics if a label of the same name is added by the server.
|
||||||
|
ExportedLabelPrefix = "exported_" |
||||||
|
|
||||||
|
// MetricNameLabel is the label name indicating the metric name of a
|
||||||
|
// timeseries.
|
||||||
|
MetricNameLabel = "__name__" |
||||||
|
|
||||||
|
// SchemeLabel is the name of the label that holds the scheme on which to
|
||||||
|
// scrape a target.
|
||||||
|
SchemeLabel = "__scheme__" |
||||||
|
|
||||||
|
// AddressLabel is the name of the label that holds the address of
|
||||||
|
// a scrape target.
|
||||||
|
AddressLabel = "__address__" |
||||||
|
|
||||||
|
// MetricsPathLabel is the name of the label that holds the path on which to
|
||||||
|
// scrape a target.
|
||||||
|
MetricsPathLabel = "__metrics_path__" |
||||||
|
|
||||||
|
// ReservedLabelPrefix is a prefix which is not legal in user-supplied
|
||||||
|
// label names.
|
||||||
|
ReservedLabelPrefix = "__" |
||||||
|
|
||||||
|
// MetaLabelPrefix is a prefix for labels that provide meta information.
|
||||||
|
// Labels with this prefix are used for intermediate label processing and
|
||||||
|
// will not be attached to time series.
|
||||||
|
MetaLabelPrefix = "__meta_" |
||||||
|
|
||||||
|
// TmpLabelPrefix is a prefix for temporary labels as part of relabelling.
|
||||||
|
// Labels with this prefix are used for intermediate label processing and
|
||||||
|
// will not be attached to time series. This is reserved for use in
|
||||||
|
// Prometheus configuration files by users.
|
||||||
|
TmpLabelPrefix = "__tmp_" |
||||||
|
|
||||||
|
// ParamLabelPrefix is a prefix for labels that provide URL parameters
|
||||||
|
// used to scrape a target.
|
||||||
|
ParamLabelPrefix = "__param_" |
||||||
|
|
||||||
|
// JobLabel is the label name indicating the job from which a timeseries
|
||||||
|
// was scraped.
|
||||||
|
JobLabel = "job" |
||||||
|
|
||||||
|
// InstanceLabel is the label name used for the instance label.
|
||||||
|
InstanceLabel = "instance" |
||||||
|
|
||||||
|
// BucketLabel is used for the label that defines the upper bound of a
|
||||||
|
// bucket of a histogram ("le" -> "less or equal").
|
||||||
|
BucketLabel = "le" |
||||||
|
|
||||||
|
// QuantileLabel is used for the label that defines the quantile in a
|
||||||
|
// summary.
|
||||||
|
QuantileLabel = "quantile" |
||||||
|
) |
||||||
|
|
||||||
|
// LabelNameRE is a regular expression matching valid label names. Note that the
|
||||||
|
// IsValid method of LabelName performs the same check but faster than a match
|
||||||
|
// with this regular expression.
|
||||||
|
var LabelNameRE = regexp.MustCompile("^[a-zA-Z_][a-zA-Z0-9_]*$") |
||||||
|
|
||||||
|
// A LabelName is a key for a LabelSet or Metric. It has a value associated
|
||||||
|
// therewith.
|
||||||
|
type LabelName string |
||||||
|
|
||||||
|
// IsValid is true iff the label name matches the pattern of LabelNameRE. This
|
||||||
|
// method, however, does not use LabelNameRE for the check but a much faster
|
||||||
|
// hardcoded implementation.
|
||||||
|
func (ln LabelName) IsValid() bool { |
||||||
|
if len(ln) == 0 { |
||||||
|
return false |
||||||
|
} |
||||||
|
for i, b := range ln { |
||||||
|
if !((b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || b == '_' || (b >= '0' && b <= '9' && i > 0)) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalYAML implements the yaml.Unmarshaler interface.
|
||||||
|
func (ln *LabelName) UnmarshalYAML(unmarshal func(interface{}) error) error { |
||||||
|
var s string |
||||||
|
if err := unmarshal(&s); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !LabelName(s).IsValid() { |
||||||
|
return fmt.Errorf("%q is not a valid label name", s) |
||||||
|
} |
||||||
|
*ln = LabelName(s) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements the json.Unmarshaler interface.
|
||||||
|
func (ln *LabelName) UnmarshalJSON(b []byte) error { |
||||||
|
var s string |
||||||
|
if err := json.Unmarshal(b, &s); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !LabelName(s).IsValid() { |
||||||
|
return fmt.Errorf("%q is not a valid label name", s) |
||||||
|
} |
||||||
|
*ln = LabelName(s) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// LabelNames is a sortable LabelName slice. In implements sort.Interface.
|
||||||
|
type LabelNames []LabelName |
||||||
|
|
||||||
|
func (l LabelNames) Len() int { |
||||||
|
return len(l) |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelNames) Less(i, j int) bool { |
||||||
|
return l[i] < l[j] |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelNames) Swap(i, j int) { |
||||||
|
l[i], l[j] = l[j], l[i] |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelNames) String() string { |
||||||
|
labelStrings := make([]string, 0, len(l)) |
||||||
|
for _, label := range l { |
||||||
|
labelStrings = append(labelStrings, string(label)) |
||||||
|
} |
||||||
|
return strings.Join(labelStrings, ", ") |
||||||
|
} |
||||||
|
|
||||||
|
// A LabelValue is an associated value for a LabelName.
|
||||||
|
type LabelValue string |
||||||
|
|
||||||
|
// IsValid returns true iff the string is a valid UTF8.
|
||||||
|
func (lv LabelValue) IsValid() bool { |
||||||
|
return utf8.ValidString(string(lv)) |
||||||
|
} |
||||||
|
|
||||||
|
// LabelValues is a sortable LabelValue slice. It implements sort.Interface.
|
||||||
|
type LabelValues []LabelValue |
||||||
|
|
||||||
|
func (l LabelValues) Len() int { |
||||||
|
return len(l) |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelValues) Less(i, j int) bool { |
||||||
|
return string(l[i]) < string(l[j]) |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelValues) Swap(i, j int) { |
||||||
|
l[i], l[j] = l[j], l[i] |
||||||
|
} |
||||||
|
|
||||||
|
// LabelPair pairs a name with a value.
|
||||||
|
type LabelPair struct { |
||||||
|
Name LabelName |
||||||
|
Value LabelValue |
||||||
|
} |
||||||
|
|
||||||
|
// LabelPairs is a sortable slice of LabelPair pointers. It implements
|
||||||
|
// sort.Interface.
|
||||||
|
type LabelPairs []*LabelPair |
||||||
|
|
||||||
|
func (l LabelPairs) Len() int { |
||||||
|
return len(l) |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelPairs) Less(i, j int) bool { |
||||||
|
switch { |
||||||
|
case l[i].Name > l[j].Name: |
||||||
|
return false |
||||||
|
case l[i].Name < l[j].Name: |
||||||
|
return true |
||||||
|
case l[i].Value > l[j].Value: |
||||||
|
return false |
||||||
|
case l[i].Value < l[j].Value: |
||||||
|
return true |
||||||
|
default: |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelPairs) Swap(i, j int) { |
||||||
|
l[i], l[j] = l[j], l[i] |
||||||
|
} |
||||||
@ -0,0 +1,169 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"fmt" |
||||||
|
"sort" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
// A LabelSet is a collection of LabelName and LabelValue pairs. The LabelSet
|
||||||
|
// may be fully-qualified down to the point where it may resolve to a single
|
||||||
|
// Metric in the data store or not. All operations that occur within the realm
|
||||||
|
// of a LabelSet can emit a vector of Metric entities to which the LabelSet may
|
||||||
|
// match.
|
||||||
|
type LabelSet map[LabelName]LabelValue |
||||||
|
|
||||||
|
// Validate checks whether all names and values in the label set
|
||||||
|
// are valid.
|
||||||
|
func (ls LabelSet) Validate() error { |
||||||
|
for ln, lv := range ls { |
||||||
|
if !ln.IsValid() { |
||||||
|
return fmt.Errorf("invalid name %q", ln) |
||||||
|
} |
||||||
|
if !lv.IsValid() { |
||||||
|
return fmt.Errorf("invalid value %q", lv) |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Equal returns true iff both label sets have exactly the same key/value pairs.
|
||||||
|
func (ls LabelSet) Equal(o LabelSet) bool { |
||||||
|
if len(ls) != len(o) { |
||||||
|
return false |
||||||
|
} |
||||||
|
for ln, lv := range ls { |
||||||
|
olv, ok := o[ln] |
||||||
|
if !ok { |
||||||
|
return false |
||||||
|
} |
||||||
|
if olv != lv { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// Before compares the metrics, using the following criteria:
|
||||||
|
//
|
||||||
|
// If m has fewer labels than o, it is before o. If it has more, it is not.
|
||||||
|
//
|
||||||
|
// If the number of labels is the same, the superset of all label names is
|
||||||
|
// sorted alphanumerically. The first differing label pair found in that order
|
||||||
|
// determines the outcome: If the label does not exist at all in m, then m is
|
||||||
|
// before o, and vice versa. Otherwise the label value is compared
|
||||||
|
// alphanumerically.
|
||||||
|
//
|
||||||
|
// If m and o are equal, the method returns false.
|
||||||
|
func (ls LabelSet) Before(o LabelSet) bool { |
||||||
|
if len(ls) < len(o) { |
||||||
|
return true |
||||||
|
} |
||||||
|
if len(ls) > len(o) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
lns := make(LabelNames, 0, len(ls)+len(o)) |
||||||
|
for ln := range ls { |
||||||
|
lns = append(lns, ln) |
||||||
|
} |
||||||
|
for ln := range o { |
||||||
|
lns = append(lns, ln) |
||||||
|
} |
||||||
|
// It's probably not worth it to de-dup lns.
|
||||||
|
sort.Sort(lns) |
||||||
|
for _, ln := range lns { |
||||||
|
mlv, ok := ls[ln] |
||||||
|
if !ok { |
||||||
|
return true |
||||||
|
} |
||||||
|
olv, ok := o[ln] |
||||||
|
if !ok { |
||||||
|
return false |
||||||
|
} |
||||||
|
if mlv < olv { |
||||||
|
return true |
||||||
|
} |
||||||
|
if mlv > olv { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
// Clone returns a copy of the label set.
|
||||||
|
func (ls LabelSet) Clone() LabelSet { |
||||||
|
lsn := make(LabelSet, len(ls)) |
||||||
|
for ln, lv := range ls { |
||||||
|
lsn[ln] = lv |
||||||
|
} |
||||||
|
return lsn |
||||||
|
} |
||||||
|
|
||||||
|
// Merge is a helper function to non-destructively merge two label sets.
|
||||||
|
func (l LabelSet) Merge(other LabelSet) LabelSet { |
||||||
|
result := make(LabelSet, len(l)) |
||||||
|
|
||||||
|
for k, v := range l { |
||||||
|
result[k] = v |
||||||
|
} |
||||||
|
|
||||||
|
for k, v := range other { |
||||||
|
result[k] = v |
||||||
|
} |
||||||
|
|
||||||
|
return result |
||||||
|
} |
||||||
|
|
||||||
|
func (l LabelSet) String() string { |
||||||
|
lstrs := make([]string, 0, len(l)) |
||||||
|
for l, v := range l { |
||||||
|
lstrs = append(lstrs, fmt.Sprintf("%s=%q", l, v)) |
||||||
|
} |
||||||
|
|
||||||
|
sort.Strings(lstrs) |
||||||
|
return fmt.Sprintf("{%s}", strings.Join(lstrs, ", ")) |
||||||
|
} |
||||||
|
|
||||||
|
// Fingerprint returns the LabelSet's fingerprint.
|
||||||
|
func (ls LabelSet) Fingerprint() Fingerprint { |
||||||
|
return labelSetToFingerprint(ls) |
||||||
|
} |
||||||
|
|
||||||
|
// FastFingerprint returns the LabelSet's Fingerprint calculated by a faster hashing
|
||||||
|
// algorithm, which is, however, more susceptible to hash collisions.
|
||||||
|
func (ls LabelSet) FastFingerprint() Fingerprint { |
||||||
|
return labelSetToFastFingerprint(ls) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements the json.Unmarshaler interface.
|
||||||
|
func (l *LabelSet) UnmarshalJSON(b []byte) error { |
||||||
|
var m map[LabelName]LabelValue |
||||||
|
if err := json.Unmarshal(b, &m); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
// encoding/json only unmarshals maps of the form map[string]T. It treats
|
||||||
|
// LabelName as a string and does not call its UnmarshalJSON method.
|
||||||
|
// Thus, we have to replicate the behavior here.
|
||||||
|
for ln := range m { |
||||||
|
if !ln.IsValid() { |
||||||
|
return fmt.Errorf("%q is not a valid label name", ln) |
||||||
|
} |
||||||
|
} |
||||||
|
*l = LabelSet(m) |
||||||
|
return nil |
||||||
|
} |
||||||
@ -0,0 +1,103 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"regexp" |
||||||
|
"sort" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
var ( |
||||||
|
separator = []byte{0} |
||||||
|
// MetricNameRE is a regular expression matching valid metric
|
||||||
|
// names. Note that the IsValidMetricName function performs the same
|
||||||
|
// check but faster than a match with this regular expression.
|
||||||
|
MetricNameRE = regexp.MustCompile(`^[a-zA-Z_:][a-zA-Z0-9_:]*$`) |
||||||
|
) |
||||||
|
|
||||||
|
// A Metric is similar to a LabelSet, but the key difference is that a Metric is
|
||||||
|
// a singleton and refers to one and only one stream of samples.
|
||||||
|
type Metric LabelSet |
||||||
|
|
||||||
|
// Equal compares the metrics.
|
||||||
|
func (m Metric) Equal(o Metric) bool { |
||||||
|
return LabelSet(m).Equal(LabelSet(o)) |
||||||
|
} |
||||||
|
|
||||||
|
// Before compares the metrics' underlying label sets.
|
||||||
|
func (m Metric) Before(o Metric) bool { |
||||||
|
return LabelSet(m).Before(LabelSet(o)) |
||||||
|
} |
||||||
|
|
||||||
|
// Clone returns a copy of the Metric.
|
||||||
|
func (m Metric) Clone() Metric { |
||||||
|
clone := Metric{} |
||||||
|
for k, v := range m { |
||||||
|
clone[k] = v |
||||||
|
} |
||||||
|
return clone |
||||||
|
} |
||||||
|
|
||||||
|
func (m Metric) String() string { |
||||||
|
metricName, hasName := m[MetricNameLabel] |
||||||
|
numLabels := len(m) - 1 |
||||||
|
if !hasName { |
||||||
|
numLabels = len(m) |
||||||
|
} |
||||||
|
labelStrings := make([]string, 0, numLabels) |
||||||
|
for label, value := range m { |
||||||
|
if label != MetricNameLabel { |
||||||
|
labelStrings = append(labelStrings, fmt.Sprintf("%s=%q", label, value)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
switch numLabels { |
||||||
|
case 0: |
||||||
|
if hasName { |
||||||
|
return string(metricName) |
||||||
|
} |
||||||
|
return "{}" |
||||||
|
default: |
||||||
|
sort.Strings(labelStrings) |
||||||
|
return fmt.Sprintf("%s{%s}", metricName, strings.Join(labelStrings, ", ")) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Fingerprint returns a Metric's Fingerprint.
|
||||||
|
func (m Metric) Fingerprint() Fingerprint { |
||||||
|
return LabelSet(m).Fingerprint() |
||||||
|
} |
||||||
|
|
||||||
|
// FastFingerprint returns a Metric's Fingerprint calculated by a faster hashing
|
||||||
|
// algorithm, which is, however, more susceptible to hash collisions.
|
||||||
|
func (m Metric) FastFingerprint() Fingerprint { |
||||||
|
return LabelSet(m).FastFingerprint() |
||||||
|
} |
||||||
|
|
||||||
|
// IsValidMetricName returns true iff name matches the pattern of MetricNameRE.
|
||||||
|
// This function, however, does not use MetricNameRE for the check but a much
|
||||||
|
// faster hardcoded implementation.
|
||||||
|
func IsValidMetricName(n LabelValue) bool { |
||||||
|
if len(n) == 0 { |
||||||
|
return false |
||||||
|
} |
||||||
|
for i, b := range n { |
||||||
|
if !((b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || b == '_' || b == ':' || (b >= '0' && b <= '9' && i > 0)) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
@ -0,0 +1,16 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
// Package model contains common data structures that are shared across
|
||||||
|
// Prometheus components and libraries.
|
||||||
|
package model |
||||||
@ -0,0 +1,144 @@ |
|||||||
|
// Copyright 2014 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"sort" |
||||||
|
) |
||||||
|
|
||||||
|
// SeparatorByte is a byte that cannot occur in valid UTF-8 sequences and is
|
||||||
|
// used to separate label names, label values, and other strings from each other
|
||||||
|
// when calculating their combined hash value (aka signature aka fingerprint).
|
||||||
|
const SeparatorByte byte = 255 |
||||||
|
|
||||||
|
var ( |
||||||
|
// cache the signature of an empty label set.
|
||||||
|
emptyLabelSignature = hashNew() |
||||||
|
) |
||||||
|
|
||||||
|
// LabelsToSignature returns a quasi-unique signature (i.e., fingerprint) for a
|
||||||
|
// given label set. (Collisions are possible but unlikely if the number of label
|
||||||
|
// sets the function is applied to is small.)
|
||||||
|
func LabelsToSignature(labels map[string]string) uint64 { |
||||||
|
if len(labels) == 0 { |
||||||
|
return emptyLabelSignature |
||||||
|
} |
||||||
|
|
||||||
|
labelNames := make([]string, 0, len(labels)) |
||||||
|
for labelName := range labels { |
||||||
|
labelNames = append(labelNames, labelName) |
||||||
|
} |
||||||
|
sort.Strings(labelNames) |
||||||
|
|
||||||
|
sum := hashNew() |
||||||
|
for _, labelName := range labelNames { |
||||||
|
sum = hashAdd(sum, labelName) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
sum = hashAdd(sum, labels[labelName]) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
} |
||||||
|
return sum |
||||||
|
} |
||||||
|
|
||||||
|
// labelSetToFingerprint works exactly as LabelsToSignature but takes a LabelSet as
|
||||||
|
// parameter (rather than a label map) and returns a Fingerprint.
|
||||||
|
func labelSetToFingerprint(ls LabelSet) Fingerprint { |
||||||
|
if len(ls) == 0 { |
||||||
|
return Fingerprint(emptyLabelSignature) |
||||||
|
} |
||||||
|
|
||||||
|
labelNames := make(LabelNames, 0, len(ls)) |
||||||
|
for labelName := range ls { |
||||||
|
labelNames = append(labelNames, labelName) |
||||||
|
} |
||||||
|
sort.Sort(labelNames) |
||||||
|
|
||||||
|
sum := hashNew() |
||||||
|
for _, labelName := range labelNames { |
||||||
|
sum = hashAdd(sum, string(labelName)) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
sum = hashAdd(sum, string(ls[labelName])) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
} |
||||||
|
return Fingerprint(sum) |
||||||
|
} |
||||||
|
|
||||||
|
// labelSetToFastFingerprint works similar to labelSetToFingerprint but uses a
|
||||||
|
// faster and less allocation-heavy hash function, which is more susceptible to
|
||||||
|
// create hash collisions. Therefore, collision detection should be applied.
|
||||||
|
func labelSetToFastFingerprint(ls LabelSet) Fingerprint { |
||||||
|
if len(ls) == 0 { |
||||||
|
return Fingerprint(emptyLabelSignature) |
||||||
|
} |
||||||
|
|
||||||
|
var result uint64 |
||||||
|
for labelName, labelValue := range ls { |
||||||
|
sum := hashNew() |
||||||
|
sum = hashAdd(sum, string(labelName)) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
sum = hashAdd(sum, string(labelValue)) |
||||||
|
result ^= sum |
||||||
|
} |
||||||
|
return Fingerprint(result) |
||||||
|
} |
||||||
|
|
||||||
|
// SignatureForLabels works like LabelsToSignature but takes a Metric as
|
||||||
|
// parameter (rather than a label map) and only includes the labels with the
|
||||||
|
// specified LabelNames into the signature calculation. The labels passed in
|
||||||
|
// will be sorted by this function.
|
||||||
|
func SignatureForLabels(m Metric, labels ...LabelName) uint64 { |
||||||
|
if len(labels) == 0 { |
||||||
|
return emptyLabelSignature |
||||||
|
} |
||||||
|
|
||||||
|
sort.Sort(LabelNames(labels)) |
||||||
|
|
||||||
|
sum := hashNew() |
||||||
|
for _, label := range labels { |
||||||
|
sum = hashAdd(sum, string(label)) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
sum = hashAdd(sum, string(m[label])) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
} |
||||||
|
return sum |
||||||
|
} |
||||||
|
|
||||||
|
// SignatureWithoutLabels works like LabelsToSignature but takes a Metric as
|
||||||
|
// parameter (rather than a label map) and excludes the labels with any of the
|
||||||
|
// specified LabelNames from the signature calculation.
|
||||||
|
func SignatureWithoutLabels(m Metric, labels map[LabelName]struct{}) uint64 { |
||||||
|
if len(m) == 0 { |
||||||
|
return emptyLabelSignature |
||||||
|
} |
||||||
|
|
||||||
|
labelNames := make(LabelNames, 0, len(m)) |
||||||
|
for labelName := range m { |
||||||
|
if _, exclude := labels[labelName]; !exclude { |
||||||
|
labelNames = append(labelNames, labelName) |
||||||
|
} |
||||||
|
} |
||||||
|
if len(labelNames) == 0 { |
||||||
|
return emptyLabelSignature |
||||||
|
} |
||||||
|
sort.Sort(labelNames) |
||||||
|
|
||||||
|
sum := hashNew() |
||||||
|
for _, labelName := range labelNames { |
||||||
|
sum = hashAdd(sum, string(labelName)) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
sum = hashAdd(sum, string(m[labelName])) |
||||||
|
sum = hashAddByte(sum, SeparatorByte) |
||||||
|
} |
||||||
|
return sum |
||||||
|
} |
||||||
@ -0,0 +1,106 @@ |
|||||||
|
// Copyright 2015 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"fmt" |
||||||
|
"regexp" |
||||||
|
"time" |
||||||
|
) |
||||||
|
|
||||||
|
// Matcher describes a matches the value of a given label.
|
||||||
|
type Matcher struct { |
||||||
|
Name LabelName `json:"name"` |
||||||
|
Value string `json:"value"` |
||||||
|
IsRegex bool `json:"isRegex"` |
||||||
|
} |
||||||
|
|
||||||
|
func (m *Matcher) UnmarshalJSON(b []byte) error { |
||||||
|
type plain Matcher |
||||||
|
if err := json.Unmarshal(b, (*plain)(m)); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if len(m.Name) == 0 { |
||||||
|
return fmt.Errorf("label name in matcher must not be empty") |
||||||
|
} |
||||||
|
if m.IsRegex { |
||||||
|
if _, err := regexp.Compile(m.Value); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Validate returns true iff all fields of the matcher have valid values.
|
||||||
|
func (m *Matcher) Validate() error { |
||||||
|
if !m.Name.IsValid() { |
||||||
|
return fmt.Errorf("invalid name %q", m.Name) |
||||||
|
} |
||||||
|
if m.IsRegex { |
||||||
|
if _, err := regexp.Compile(m.Value); err != nil { |
||||||
|
return fmt.Errorf("invalid regular expression %q", m.Value) |
||||||
|
} |
||||||
|
} else if !LabelValue(m.Value).IsValid() || len(m.Value) == 0 { |
||||||
|
return fmt.Errorf("invalid value %q", m.Value) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Silence defines the representation of a silence definiton
|
||||||
|
// in the Prometheus eco-system.
|
||||||
|
type Silence struct { |
||||||
|
ID uint64 `json:"id,omitempty"` |
||||||
|
|
||||||
|
Matchers []*Matcher `json:"matchers"` |
||||||
|
|
||||||
|
StartsAt time.Time `json:"startsAt"` |
||||||
|
EndsAt time.Time `json:"endsAt"` |
||||||
|
|
||||||
|
CreatedAt time.Time `json:"createdAt,omitempty"` |
||||||
|
CreatedBy string `json:"createdBy"` |
||||||
|
Comment string `json:"comment,omitempty"` |
||||||
|
} |
||||||
|
|
||||||
|
// Validate returns true iff all fields of the silence have valid values.
|
||||||
|
func (s *Silence) Validate() error { |
||||||
|
if len(s.Matchers) == 0 { |
||||||
|
return fmt.Errorf("at least one matcher required") |
||||||
|
} |
||||||
|
for _, m := range s.Matchers { |
||||||
|
if err := m.Validate(); err != nil { |
||||||
|
return fmt.Errorf("invalid matcher: %s", err) |
||||||
|
} |
||||||
|
} |
||||||
|
if s.StartsAt.IsZero() { |
||||||
|
return fmt.Errorf("start time missing") |
||||||
|
} |
||||||
|
if s.EndsAt.IsZero() { |
||||||
|
return fmt.Errorf("end time missing") |
||||||
|
} |
||||||
|
if s.EndsAt.Before(s.StartsAt) { |
||||||
|
return fmt.Errorf("start time must be before end time") |
||||||
|
} |
||||||
|
if s.CreatedBy == "" { |
||||||
|
return fmt.Errorf("creator information missing") |
||||||
|
} |
||||||
|
if s.Comment == "" { |
||||||
|
return fmt.Errorf("comment missing") |
||||||
|
} |
||||||
|
if s.CreatedAt.IsZero() { |
||||||
|
return fmt.Errorf("creation timestamp missing") |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
@ -0,0 +1,249 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"fmt" |
||||||
|
"math" |
||||||
|
"regexp" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
"time" |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
// MinimumTick is the minimum supported time resolution. This has to be
|
||||||
|
// at least time.Second in order for the code below to work.
|
||||||
|
minimumTick = time.Millisecond |
||||||
|
// second is the Time duration equivalent to one second.
|
||||||
|
second = int64(time.Second / minimumTick) |
||||||
|
// The number of nanoseconds per minimum tick.
|
||||||
|
nanosPerTick = int64(minimumTick / time.Nanosecond) |
||||||
|
|
||||||
|
// Earliest is the earliest Time representable. Handy for
|
||||||
|
// initializing a high watermark.
|
||||||
|
Earliest = Time(math.MinInt64) |
||||||
|
// Latest is the latest Time representable. Handy for initializing
|
||||||
|
// a low watermark.
|
||||||
|
Latest = Time(math.MaxInt64) |
||||||
|
) |
||||||
|
|
||||||
|
// Time is the number of milliseconds since the epoch
|
||||||
|
// (1970-01-01 00:00 UTC) excluding leap seconds.
|
||||||
|
type Time int64 |
||||||
|
|
||||||
|
// Interval describes and interval between two timestamps.
|
||||||
|
type Interval struct { |
||||||
|
Start, End Time |
||||||
|
} |
||||||
|
|
||||||
|
// Now returns the current time as a Time.
|
||||||
|
func Now() Time { |
||||||
|
return TimeFromUnixNano(time.Now().UnixNano()) |
||||||
|
} |
||||||
|
|
||||||
|
// TimeFromUnix returns the Time equivalent to the Unix Time t
|
||||||
|
// provided in seconds.
|
||||||
|
func TimeFromUnix(t int64) Time { |
||||||
|
return Time(t * second) |
||||||
|
} |
||||||
|
|
||||||
|
// TimeFromUnixNano returns the Time equivalent to the Unix Time
|
||||||
|
// t provided in nanoseconds.
|
||||||
|
func TimeFromUnixNano(t int64) Time { |
||||||
|
return Time(t / nanosPerTick) |
||||||
|
} |
||||||
|
|
||||||
|
// Equal reports whether two Times represent the same instant.
|
||||||
|
func (t Time) Equal(o Time) bool { |
||||||
|
return t == o |
||||||
|
} |
||||||
|
|
||||||
|
// Before reports whether the Time t is before o.
|
||||||
|
func (t Time) Before(o Time) bool { |
||||||
|
return t < o |
||||||
|
} |
||||||
|
|
||||||
|
// After reports whether the Time t is after o.
|
||||||
|
func (t Time) After(o Time) bool { |
||||||
|
return t > o |
||||||
|
} |
||||||
|
|
||||||
|
// Add returns the Time t + d.
|
||||||
|
func (t Time) Add(d time.Duration) Time { |
||||||
|
return t + Time(d/minimumTick) |
||||||
|
} |
||||||
|
|
||||||
|
// Sub returns the Duration t - o.
|
||||||
|
func (t Time) Sub(o Time) time.Duration { |
||||||
|
return time.Duration(t-o) * minimumTick |
||||||
|
} |
||||||
|
|
||||||
|
// Time returns the time.Time representation of t.
|
||||||
|
func (t Time) Time() time.Time { |
||||||
|
return time.Unix(int64(t)/second, (int64(t)%second)*nanosPerTick) |
||||||
|
} |
||||||
|
|
||||||
|
// Unix returns t as a Unix time, the number of seconds elapsed
|
||||||
|
// since January 1, 1970 UTC.
|
||||||
|
func (t Time) Unix() int64 { |
||||||
|
return int64(t) / second |
||||||
|
} |
||||||
|
|
||||||
|
// UnixNano returns t as a Unix time, the number of nanoseconds elapsed
|
||||||
|
// since January 1, 1970 UTC.
|
||||||
|
func (t Time) UnixNano() int64 { |
||||||
|
return int64(t) * nanosPerTick |
||||||
|
} |
||||||
|
|
||||||
|
// The number of digits after the dot.
|
||||||
|
var dotPrecision = int(math.Log10(float64(second))) |
||||||
|
|
||||||
|
// String returns a string representation of the Time.
|
||||||
|
func (t Time) String() string { |
||||||
|
return strconv.FormatFloat(float64(t)/float64(second), 'f', -1, 64) |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalJSON implements the json.Marshaler interface.
|
||||||
|
func (t Time) MarshalJSON() ([]byte, error) { |
||||||
|
return []byte(t.String()), nil |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements the json.Unmarshaler interface.
|
||||||
|
func (t *Time) UnmarshalJSON(b []byte) error { |
||||||
|
p := strings.Split(string(b), ".") |
||||||
|
switch len(p) { |
||||||
|
case 1: |
||||||
|
v, err := strconv.ParseInt(string(p[0]), 10, 64) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*t = Time(v * second) |
||||||
|
|
||||||
|
case 2: |
||||||
|
v, err := strconv.ParseInt(string(p[0]), 10, 64) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
v *= second |
||||||
|
|
||||||
|
prec := dotPrecision - len(p[1]) |
||||||
|
if prec < 0 { |
||||||
|
p[1] = p[1][:dotPrecision] |
||||||
|
} else if prec > 0 { |
||||||
|
p[1] = p[1] + strings.Repeat("0", prec) |
||||||
|
} |
||||||
|
|
||||||
|
va, err := strconv.ParseInt(p[1], 10, 32) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
*t = Time(v + va) |
||||||
|
|
||||||
|
default: |
||||||
|
return fmt.Errorf("invalid time %q", string(b)) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Duration wraps time.Duration. It is used to parse the custom duration format
|
||||||
|
// from YAML.
|
||||||
|
// This type should not propagate beyond the scope of input/output processing.
|
||||||
|
type Duration time.Duration |
||||||
|
|
||||||
|
var durationRE = regexp.MustCompile("^([0-9]+)(y|w|d|h|m|s|ms)$") |
||||||
|
|
||||||
|
// StringToDuration parses a string into a time.Duration, assuming that a year
|
||||||
|
// always has 365d, a week always has 7d, and a day always has 24h.
|
||||||
|
func ParseDuration(durationStr string) (Duration, error) { |
||||||
|
matches := durationRE.FindStringSubmatch(durationStr) |
||||||
|
if len(matches) != 3 { |
||||||
|
return 0, fmt.Errorf("not a valid duration string: %q", durationStr) |
||||||
|
} |
||||||
|
var ( |
||||||
|
n, _ = strconv.Atoi(matches[1]) |
||||||
|
dur = time.Duration(n) * time.Millisecond |
||||||
|
) |
||||||
|
switch unit := matches[2]; unit { |
||||||
|
case "y": |
||||||
|
dur *= 1000 * 60 * 60 * 24 * 365 |
||||||
|
case "w": |
||||||
|
dur *= 1000 * 60 * 60 * 24 * 7 |
||||||
|
case "d": |
||||||
|
dur *= 1000 * 60 * 60 * 24 |
||||||
|
case "h": |
||||||
|
dur *= 1000 * 60 * 60 |
||||||
|
case "m": |
||||||
|
dur *= 1000 * 60 |
||||||
|
case "s": |
||||||
|
dur *= 1000 |
||||||
|
case "ms": |
||||||
|
// Value already correct
|
||||||
|
default: |
||||||
|
return 0, fmt.Errorf("invalid time unit in duration string: %q", unit) |
||||||
|
} |
||||||
|
return Duration(dur), nil |
||||||
|
} |
||||||
|
|
||||||
|
func (d Duration) String() string { |
||||||
|
var ( |
||||||
|
ms = int64(time.Duration(d) / time.Millisecond) |
||||||
|
unit = "ms" |
||||||
|
) |
||||||
|
factors := map[string]int64{ |
||||||
|
"y": 1000 * 60 * 60 * 24 * 365, |
||||||
|
"w": 1000 * 60 * 60 * 24 * 7, |
||||||
|
"d": 1000 * 60 * 60 * 24, |
||||||
|
"h": 1000 * 60 * 60, |
||||||
|
"m": 1000 * 60, |
||||||
|
"s": 1000, |
||||||
|
"ms": 1, |
||||||
|
} |
||||||
|
|
||||||
|
switch int64(0) { |
||||||
|
case ms % factors["y"]: |
||||||
|
unit = "y" |
||||||
|
case ms % factors["w"]: |
||||||
|
unit = "w" |
||||||
|
case ms % factors["d"]: |
||||||
|
unit = "d" |
||||||
|
case ms % factors["h"]: |
||||||
|
unit = "h" |
||||||
|
case ms % factors["m"]: |
||||||
|
unit = "m" |
||||||
|
case ms % factors["s"]: |
||||||
|
unit = "s" |
||||||
|
} |
||||||
|
return fmt.Sprintf("%v%v", ms/factors[unit], unit) |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalYAML implements the yaml.Marshaler interface.
|
||||||
|
func (d Duration) MarshalYAML() (interface{}, error) { |
||||||
|
return d.String(), nil |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalYAML implements the yaml.Unmarshaler interface.
|
||||||
|
func (d *Duration) UnmarshalYAML(unmarshal func(interface{}) error) error { |
||||||
|
var s string |
||||||
|
if err := unmarshal(&s); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
dur, err := ParseDuration(s) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*d = dur |
||||||
|
return nil |
||||||
|
} |
||||||
@ -0,0 +1,419 @@ |
|||||||
|
// Copyright 2013 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package model |
||||||
|
|
||||||
|
import ( |
||||||
|
"encoding/json" |
||||||
|
"fmt" |
||||||
|
"math" |
||||||
|
"sort" |
||||||
|
"strconv" |
||||||
|
"strings" |
||||||
|
) |
||||||
|
|
||||||
|
var ( |
||||||
|
// ZeroSamplePair is the pseudo zero-value of SamplePair used to signal a
|
||||||
|
// non-existing sample pair. It is a SamplePair with timestamp Earliest and
|
||||||
|
// value 0.0. Note that the natural zero value of SamplePair has a timestamp
|
||||||
|
// of 0, which is possible to appear in a real SamplePair and thus not
|
||||||
|
// suitable to signal a non-existing SamplePair.
|
||||||
|
ZeroSamplePair = SamplePair{Timestamp: Earliest} |
||||||
|
|
||||||
|
// ZeroSample is the pseudo zero-value of Sample used to signal a
|
||||||
|
// non-existing sample. It is a Sample with timestamp Earliest, value 0.0,
|
||||||
|
// and metric nil. Note that the natural zero value of Sample has a timestamp
|
||||||
|
// of 0, which is possible to appear in a real Sample and thus not suitable
|
||||||
|
// to signal a non-existing Sample.
|
||||||
|
ZeroSample = Sample{Timestamp: Earliest} |
||||||
|
) |
||||||
|
|
||||||
|
// A SampleValue is a representation of a value for a given sample at a given
|
||||||
|
// time.
|
||||||
|
type SampleValue float64 |
||||||
|
|
||||||
|
// MarshalJSON implements json.Marshaler.
|
||||||
|
func (v SampleValue) MarshalJSON() ([]byte, error) { |
||||||
|
return json.Marshal(v.String()) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements json.Unmarshaler.
|
||||||
|
func (v *SampleValue) UnmarshalJSON(b []byte) error { |
||||||
|
if len(b) < 2 || b[0] != '"' || b[len(b)-1] != '"' { |
||||||
|
return fmt.Errorf("sample value must be a quoted string") |
||||||
|
} |
||||||
|
f, err := strconv.ParseFloat(string(b[1:len(b)-1]), 64) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
*v = SampleValue(f) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Equal returns true if the value of v and o is equal or if both are NaN. Note
|
||||||
|
// that v==o is false if both are NaN. If you want the conventional float
|
||||||
|
// behavior, use == to compare two SampleValues.
|
||||||
|
func (v SampleValue) Equal(o SampleValue) bool { |
||||||
|
if v == o { |
||||||
|
return true |
||||||
|
} |
||||||
|
return math.IsNaN(float64(v)) && math.IsNaN(float64(o)) |
||||||
|
} |
||||||
|
|
||||||
|
func (v SampleValue) String() string { |
||||||
|
return strconv.FormatFloat(float64(v), 'f', -1, 64) |
||||||
|
} |
||||||
|
|
||||||
|
// SamplePair pairs a SampleValue with a Timestamp.
|
||||||
|
type SamplePair struct { |
||||||
|
Timestamp Time |
||||||
|
Value SampleValue |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalJSON implements json.Marshaler.
|
||||||
|
func (s SamplePair) MarshalJSON() ([]byte, error) { |
||||||
|
t, err := json.Marshal(s.Timestamp) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
v, err := json.Marshal(s.Value) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
return []byte(fmt.Sprintf("[%s,%s]", t, v)), nil |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements json.Unmarshaler.
|
||||||
|
func (s *SamplePair) UnmarshalJSON(b []byte) error { |
||||||
|
v := [...]json.Unmarshaler{&s.Timestamp, &s.Value} |
||||||
|
return json.Unmarshal(b, &v) |
||||||
|
} |
||||||
|
|
||||||
|
// Equal returns true if this SamplePair and o have equal Values and equal
|
||||||
|
// Timestamps. The sematics of Value equality is defined by SampleValue.Equal.
|
||||||
|
func (s *SamplePair) Equal(o *SamplePair) bool { |
||||||
|
return s == o || (s.Value.Equal(o.Value) && s.Timestamp.Equal(o.Timestamp)) |
||||||
|
} |
||||||
|
|
||||||
|
func (s SamplePair) String() string { |
||||||
|
return fmt.Sprintf("%s @[%s]", s.Value, s.Timestamp) |
||||||
|
} |
||||||
|
|
||||||
|
// Sample is a sample pair associated with a metric.
|
||||||
|
type Sample struct { |
||||||
|
Metric Metric `json:"metric"` |
||||||
|
Value SampleValue `json:"value"` |
||||||
|
Timestamp Time `json:"timestamp"` |
||||||
|
} |
||||||
|
|
||||||
|
// Equal compares first the metrics, then the timestamp, then the value. The
|
||||||
|
// sematics of value equality is defined by SampleValue.Equal.
|
||||||
|
func (s *Sample) Equal(o *Sample) bool { |
||||||
|
if s == o { |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
if !s.Metric.Equal(o.Metric) { |
||||||
|
return false |
||||||
|
} |
||||||
|
if !s.Timestamp.Equal(o.Timestamp) { |
||||||
|
return false |
||||||
|
} |
||||||
|
if s.Value.Equal(o.Value) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
func (s Sample) String() string { |
||||||
|
return fmt.Sprintf("%s => %s", s.Metric, SamplePair{ |
||||||
|
Timestamp: s.Timestamp, |
||||||
|
Value: s.Value, |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalJSON implements json.Marshaler.
|
||||||
|
func (s Sample) MarshalJSON() ([]byte, error) { |
||||||
|
v := struct { |
||||||
|
Metric Metric `json:"metric"` |
||||||
|
Value SamplePair `json:"value"` |
||||||
|
}{ |
||||||
|
Metric: s.Metric, |
||||||
|
Value: SamplePair{ |
||||||
|
Timestamp: s.Timestamp, |
||||||
|
Value: s.Value, |
||||||
|
}, |
||||||
|
} |
||||||
|
|
||||||
|
return json.Marshal(&v) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements json.Unmarshaler.
|
||||||
|
func (s *Sample) UnmarshalJSON(b []byte) error { |
||||||
|
v := struct { |
||||||
|
Metric Metric `json:"metric"` |
||||||
|
Value SamplePair `json:"value"` |
||||||
|
}{ |
||||||
|
Metric: s.Metric, |
||||||
|
Value: SamplePair{ |
||||||
|
Timestamp: s.Timestamp, |
||||||
|
Value: s.Value, |
||||||
|
}, |
||||||
|
} |
||||||
|
|
||||||
|
if err := json.Unmarshal(b, &v); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
s.Metric = v.Metric |
||||||
|
s.Timestamp = v.Value.Timestamp |
||||||
|
s.Value = v.Value.Value |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// Samples is a sortable Sample slice. It implements sort.Interface.
|
||||||
|
type Samples []*Sample |
||||||
|
|
||||||
|
func (s Samples) Len() int { |
||||||
|
return len(s) |
||||||
|
} |
||||||
|
|
||||||
|
// Less compares first the metrics, then the timestamp.
|
||||||
|
func (s Samples) Less(i, j int) bool { |
||||||
|
switch { |
||||||
|
case s[i].Metric.Before(s[j].Metric): |
||||||
|
return true |
||||||
|
case s[j].Metric.Before(s[i].Metric): |
||||||
|
return false |
||||||
|
case s[i].Timestamp.Before(s[j].Timestamp): |
||||||
|
return true |
||||||
|
default: |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
func (s Samples) Swap(i, j int) { |
||||||
|
s[i], s[j] = s[j], s[i] |
||||||
|
} |
||||||
|
|
||||||
|
// Equal compares two sets of samples and returns true if they are equal.
|
||||||
|
func (s Samples) Equal(o Samples) bool { |
||||||
|
if len(s) != len(o) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
for i, sample := range s { |
||||||
|
if !sample.Equal(o[i]) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// SampleStream is a stream of Values belonging to an attached COWMetric.
|
||||||
|
type SampleStream struct { |
||||||
|
Metric Metric `json:"metric"` |
||||||
|
Values []SamplePair `json:"values"` |
||||||
|
} |
||||||
|
|
||||||
|
func (ss SampleStream) String() string { |
||||||
|
vals := make([]string, len(ss.Values)) |
||||||
|
for i, v := range ss.Values { |
||||||
|
vals[i] = v.String() |
||||||
|
} |
||||||
|
return fmt.Sprintf("%s =>\n%s", ss.Metric, strings.Join(vals, "\n")) |
||||||
|
} |
||||||
|
|
||||||
|
// Value is a generic interface for values resulting from a query evaluation.
|
||||||
|
type Value interface { |
||||||
|
Type() ValueType |
||||||
|
String() string |
||||||
|
} |
||||||
|
|
||||||
|
func (Matrix) Type() ValueType { return ValMatrix } |
||||||
|
func (Vector) Type() ValueType { return ValVector } |
||||||
|
func (*Scalar) Type() ValueType { return ValScalar } |
||||||
|
func (*String) Type() ValueType { return ValString } |
||||||
|
|
||||||
|
type ValueType int |
||||||
|
|
||||||
|
const ( |
||||||
|
ValNone ValueType = iota |
||||||
|
ValScalar |
||||||
|
ValVector |
||||||
|
ValMatrix |
||||||
|
ValString |
||||||
|
) |
||||||
|
|
||||||
|
// MarshalJSON implements json.Marshaler.
|
||||||
|
func (et ValueType) MarshalJSON() ([]byte, error) { |
||||||
|
return json.Marshal(et.String()) |
||||||
|
} |
||||||
|
|
||||||
|
func (et *ValueType) UnmarshalJSON(b []byte) error { |
||||||
|
var s string |
||||||
|
if err := json.Unmarshal(b, &s); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
switch s { |
||||||
|
case "<ValNone>": |
||||||
|
*et = ValNone |
||||||
|
case "scalar": |
||||||
|
*et = ValScalar |
||||||
|
case "vector": |
||||||
|
*et = ValVector |
||||||
|
case "matrix": |
||||||
|
*et = ValMatrix |
||||||
|
case "string": |
||||||
|
*et = ValString |
||||||
|
default: |
||||||
|
return fmt.Errorf("unknown value type %q", s) |
||||||
|
} |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
func (e ValueType) String() string { |
||||||
|
switch e { |
||||||
|
case ValNone: |
||||||
|
return "<ValNone>" |
||||||
|
case ValScalar: |
||||||
|
return "scalar" |
||||||
|
case ValVector: |
||||||
|
return "vector" |
||||||
|
case ValMatrix: |
||||||
|
return "matrix" |
||||||
|
case ValString: |
||||||
|
return "string" |
||||||
|
} |
||||||
|
panic("ValueType.String: unhandled value type") |
||||||
|
} |
||||||
|
|
||||||
|
// Scalar is a scalar value evaluated at the set timestamp.
|
||||||
|
type Scalar struct { |
||||||
|
Value SampleValue `json:"value"` |
||||||
|
Timestamp Time `json:"timestamp"` |
||||||
|
} |
||||||
|
|
||||||
|
func (s Scalar) String() string { |
||||||
|
return fmt.Sprintf("scalar: %v @[%v]", s.Value, s.Timestamp) |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalJSON implements json.Marshaler.
|
||||||
|
func (s Scalar) MarshalJSON() ([]byte, error) { |
||||||
|
v := strconv.FormatFloat(float64(s.Value), 'f', -1, 64) |
||||||
|
return json.Marshal([...]interface{}{s.Timestamp, string(v)}) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements json.Unmarshaler.
|
||||||
|
func (s *Scalar) UnmarshalJSON(b []byte) error { |
||||||
|
var f string |
||||||
|
v := [...]interface{}{&s.Timestamp, &f} |
||||||
|
|
||||||
|
if err := json.Unmarshal(b, &v); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
value, err := strconv.ParseFloat(f, 64) |
||||||
|
if err != nil { |
||||||
|
return fmt.Errorf("error parsing sample value: %s", err) |
||||||
|
} |
||||||
|
s.Value = SampleValue(value) |
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// String is a string value evaluated at the set timestamp.
|
||||||
|
type String struct { |
||||||
|
Value string `json:"value"` |
||||||
|
Timestamp Time `json:"timestamp"` |
||||||
|
} |
||||||
|
|
||||||
|
func (s *String) String() string { |
||||||
|
return s.Value |
||||||
|
} |
||||||
|
|
||||||
|
// MarshalJSON implements json.Marshaler.
|
||||||
|
func (s String) MarshalJSON() ([]byte, error) { |
||||||
|
return json.Marshal([]interface{}{s.Timestamp, s.Value}) |
||||||
|
} |
||||||
|
|
||||||
|
// UnmarshalJSON implements json.Unmarshaler.
|
||||||
|
func (s *String) UnmarshalJSON(b []byte) error { |
||||||
|
v := [...]interface{}{&s.Timestamp, &s.Value} |
||||||
|
return json.Unmarshal(b, &v) |
||||||
|
} |
||||||
|
|
||||||
|
// Vector is basically only an alias for Samples, but the
|
||||||
|
// contract is that in a Vector, all Samples have the same timestamp.
|
||||||
|
type Vector []*Sample |
||||||
|
|
||||||
|
func (vec Vector) String() string { |
||||||
|
entries := make([]string, len(vec)) |
||||||
|
for i, s := range vec { |
||||||
|
entries[i] = s.String() |
||||||
|
} |
||||||
|
return strings.Join(entries, "\n") |
||||||
|
} |
||||||
|
|
||||||
|
func (vec Vector) Len() int { return len(vec) } |
||||||
|
func (vec Vector) Swap(i, j int) { vec[i], vec[j] = vec[j], vec[i] } |
||||||
|
|
||||||
|
// Less compares first the metrics, then the timestamp.
|
||||||
|
func (vec Vector) Less(i, j int) bool { |
||||||
|
switch { |
||||||
|
case vec[i].Metric.Before(vec[j].Metric): |
||||||
|
return true |
||||||
|
case vec[j].Metric.Before(vec[i].Metric): |
||||||
|
return false |
||||||
|
case vec[i].Timestamp.Before(vec[j].Timestamp): |
||||||
|
return true |
||||||
|
default: |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// Equal compares two sets of samples and returns true if they are equal.
|
||||||
|
func (vec Vector) Equal(o Vector) bool { |
||||||
|
if len(vec) != len(o) { |
||||||
|
return false |
||||||
|
} |
||||||
|
|
||||||
|
for i, sample := range vec { |
||||||
|
if !sample.Equal(o[i]) { |
||||||
|
return false |
||||||
|
} |
||||||
|
} |
||||||
|
return true |
||||||
|
} |
||||||
|
|
||||||
|
// Matrix is a list of time series.
|
||||||
|
type Matrix []*SampleStream |
||||||
|
|
||||||
|
func (m Matrix) Len() int { return len(m) } |
||||||
|
func (m Matrix) Less(i, j int) bool { return m[i].Metric.Before(m[j].Metric) } |
||||||
|
func (m Matrix) Swap(i, j int) { m[i], m[j] = m[j], m[i] } |
||||||
|
|
||||||
|
func (mat Matrix) String() string { |
||||||
|
matCp := make(Matrix, len(mat)) |
||||||
|
copy(matCp, mat) |
||||||
|
sort.Sort(matCp) |
||||||
|
|
||||||
|
strs := make([]string, len(matCp)) |
||||||
|
|
||||||
|
for i, ss := range matCp { |
||||||
|
strs[i] = ss.String() |
||||||
|
} |
||||||
|
|
||||||
|
return strings.Join(strs, "\n") |
||||||
|
} |
||||||
@ -0,0 +1,89 @@ |
|||||||
|
// Copyright 2016 The Prometheus Authors
|
||||||
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
// you may not use this file except in compliance with the License.
|
||||||
|
// You may obtain a copy of the License at
|
||||||
|
//
|
||||||
|
// http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
//
|
||||||
|
// Unless required by applicable law or agreed to in writing, software
|
||||||
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
// See the License for the specific language governing permissions and
|
||||||
|
// limitations under the License.
|
||||||
|
|
||||||
|
package version |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"fmt" |
||||||
|
"runtime" |
||||||
|
"strings" |
||||||
|
"text/template" |
||||||
|
|
||||||
|
"github.com/prometheus/client_golang/prometheus" |
||||||
|
) |
||||||
|
|
||||||
|
// Build information. Populated at build-time.
|
||||||
|
var ( |
||||||
|
Version string |
||||||
|
Revision string |
||||||
|
Branch string |
||||||
|
BuildUser string |
||||||
|
BuildDate string |
||||||
|
GoVersion = runtime.Version() |
||||||
|
) |
||||||
|
|
||||||
|
// NewCollector returns a collector which exports metrics about current version information.
|
||||||
|
func NewCollector(program string) *prometheus.GaugeVec { |
||||||
|
buildInfo := prometheus.NewGaugeVec( |
||||||
|
prometheus.GaugeOpts{ |
||||||
|
Namespace: program, |
||||||
|
Name: "build_info", |
||||||
|
Help: fmt.Sprintf( |
||||||
|
"A metric with a constant '1' value labeled by version, revision, branch, and goversion from which %s was built.", |
||||||
|
program, |
||||||
|
), |
||||||
|
}, |
||||||
|
[]string{"version", "revision", "branch", "goversion"}, |
||||||
|
) |
||||||
|
buildInfo.WithLabelValues(Version, Revision, Branch, GoVersion).Set(1) |
||||||
|
return buildInfo |
||||||
|
} |
||||||
|
|
||||||
|
// versionInfoTmpl contains the template used by Info.
|
||||||
|
var versionInfoTmpl = ` |
||||||
|
{{.program}}, version {{.version}} (branch: {{.branch}}, revision: {{.revision}}) |
||||||
|
build user: {{.buildUser}} |
||||||
|
build date: {{.buildDate}} |
||||||
|
go version: {{.goVersion}} |
||||||
|
` |
||||||
|
|
||||||
|
// Print returns version information.
|
||||||
|
func Print(program string) string { |
||||||
|
m := map[string]string{ |
||||||
|
"program": program, |
||||||
|
"version": Version, |
||||||
|
"revision": Revision, |
||||||
|
"branch": Branch, |
||||||
|
"buildUser": BuildUser, |
||||||
|
"buildDate": BuildDate, |
||||||
|
"goVersion": GoVersion, |
||||||
|
} |
||||||
|
t := template.Must(template.New("version").Parse(versionInfoTmpl)) |
||||||
|
|
||||||
|
var buf bytes.Buffer |
||||||
|
if err := t.ExecuteTemplate(&buf, "version", m); err != nil { |
||||||
|
panic(err) |
||||||
|
} |
||||||
|
return strings.TrimSpace(buf.String()) |
||||||
|
} |
||||||
|
|
||||||
|
// Info returns version, branch and revision information.
|
||||||
|
func Info() string { |
||||||
|
return fmt.Sprintf("(version=%s, branch=%s, revision=%s)", Version, Branch, Revision) |
||||||
|
} |
||||||
|
|
||||||
|
// BuildContext returns goVersion, buildUser and buildDate information.
|
||||||
|
func BuildContext() string { |
||||||
|
return fmt.Sprintf("(go=%s, user=%s, date=%s)", GoVersion, BuildUser, BuildDate) |
||||||
|
} |
||||||
@ -0,0 +1,20 @@ |
|||||||
|
The Prometheus project was started by Matt T. Proud (emeritus) and |
||||||
|
Julius Volz in 2012. |
||||||
|
|
||||||
|
Maintainers of this repository: |
||||||
|
|
||||||
|
* Tobias Schmidt <ts@soundcloud.com> |
||||||
|
|
||||||
|
The following individuals have contributed code to this repository |
||||||
|
(listed in alphabetical order): |
||||||
|
|
||||||
|
* Armen Baghumian <abaghumian@noggin.com.au> |
||||||
|
* Bjoern Rabenstein <beorn@soundcloud.com> |
||||||
|
* David Cournapeau <cournape@gmail.com> |
||||||
|
* Ji-Hoon, Seol <jihoon.seol@gmail.com> |
||||||
|
* Jonas Große Sundrup <cherti@letopolis.de> |
||||||
|
* Julius Volz <julius.volz@gmail.com> |
||||||
|
* Matthias Rampke <mr@soundcloud.com> |
||||||
|
* Nicky Gerritsen <nicky@streamone.nl> |
||||||
|
* Rémi Audebert <contact@halfr.net> |
||||||
|
* Tobias Schmidt <tobidt@gmail.com> |
||||||
@ -0,0 +1,18 @@ |
|||||||
|
# Contributing |
||||||
|
|
||||||
|
Prometheus uses GitHub to manage reviews of pull requests. |
||||||
|
|
||||||
|
* If you have a trivial fix or improvement, go ahead and create a pull |
||||||
|
request, addressing (with `@...`) one or more of the maintainers |
||||||
|
(see [AUTHORS.md](AUTHORS.md)) in the description of the pull request. |
||||||
|
|
||||||
|
* If you plan to do something more involved, first discuss your ideas |
||||||
|
on our [mailing list](https://groups.google.com/forum/?fromgroups#!forum/prometheus-developers). |
||||||
|
This will avoid unnecessary work and surely give you and us a good deal |
||||||
|
of inspiration. |
||||||
|
|
||||||
|
* Relevant coding style guidelines are the [Go Code Review |
||||||
|
Comments](https://code.google.com/p/go-wiki/wiki/CodeReviewComments) |
||||||
|
and the _Formatting and style_ section of Peter Bourgon's [Go: Best |
||||||
|
Practices for Production |
||||||
|
Environments](http://peter.bourgon.org/go-in-production/#formatting-and-style). |
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue