Commit 9bb3555f by Simon Pasquier

cmd/prometheus: support

Signed-off-by: 's avatarSimon Pasquier <>
parent 52c4da16
......@@ -48,7 +48,8 @@ import (
jprom ""
kingpin ""
klog ""
klogv2 ""
promlogflag ""
......@@ -342,6 +343,8 @@ func main() {
// Above level 6, the k8s client would log bearer tokens in clear-text.
klog.SetLogger(log.With(logger, "component", "k8s_client_runtime"))
klogv2.SetLogger(log.With(logger, "component", "k8s_client_runtime"))
level.Info(logger).Log("msg", "Starting Prometheus", "version", version.Info())
level.Info(logger).Log("build_context", version.BuildContext())
......@@ -79,9 +79,13 @@ require ( v0.19.2 v0.19.2 v1.0.0 v2.2.0
replace => v0.1.0
replace ( => v0.3.0 => v2.0.1
exclude (
// Exclude grpc v1.30.0 because of breaking changes. See #7621.
......@@ -217,10 +217,6 @@ v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9 v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= v0.5.0 h1:TrB8swr/68K7m9CcGut2g3UOihhbcbiMAYiuTXdEih4= v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= v0.1.0 h1:M1Tv3VzNlEHg6uyACnRdtrploV2P7wZqH8BoQMtz0cg= v0.1.0/go.mod h1:ixOQHD9gLJUVQQ2ZOR7zLEifBX6tGkNJF4QyIY7sIas= v0.2.0 h1:QvGt2nLcHH0WK9orKa+ppBPAxREcH364nPUedEpK0TY= v0.2.0/go.mod h1:z6/tIYblkpsD+a4lm/fGIIU9mZ+XfAiaFtq7xTgseGU= v0.0.0-20180825180245-b006789cd277/go.mod h1:k70tL6pCuVxPJOHXQ+wIac1FUrvNkHolPie/cLEU6hI= v0.17.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik= v0.18.0/go.mod h1:IowGgpVeD0vNm45So8nr+IcQ3pxVtpRoBWb8PVZO0ik=
......@@ -709,8 +705,10 @@ v0.0.0-20181202132449-6a9ea43bcacd h1:ug7PpSOB5RBPK1K v0.0.0-20181202132449-6a9ea43bcacd/go.mod h1:TrYk7fJVaAttu97ZZKrO9UbRa8izdowaMIZcxYMbVaw= v0.0.0-20200627165143-92b8a710ab6c h1:XLPw6rny9Vrrvrzhw8pNLrC2+x/kH0a/3gOx5xWDa6Y= v0.0.0-20200627165143-92b8a710ab6c/go.mod h1:TrYk7fJVaAttu97ZZKrO9UbRa8izdowaMIZcxYMbVaw= v0.1.0 h1:l3GGzgwlUF4vC1ApCOEsMsV+6nJPM01VoVCUCZgOIUw= v0.1.0/go.mod h1:4lorAA0CyDox4KO34BrvNAJk8J2Ma/M9Q2BDkR38vSI= v0.3.0 h1:TkFK21cbwDRS+CiystjqbAiq5ubJcVTk9hLUck5Ntcs= v0.3.0/go.mod h1:+SUlDQNrhVtGt2FieaqNftzzk8P72zpWlACateWxA9k= v2.0.1 h1:v7vrNd8wve5mHjX6R7kKUfR/ebJJ/LUi06NveGAvdcU= v2.0.1/go.mod h1:VgeTFrwzYYcMH8edEfh3/ai2j/Yg8c/qIm1bkGkhuJg= v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= v1.4.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
......@@ -1153,10 +1151,6 @@ v0.19.2/go.mod h1:DnPGDnARWFvYa3pMHgSxtbZb7gpzzAZ1pTfaUNDVlm v0.19.2 h1:gMJuU3xJZs86L1oQ99R4EViAADUPMHHtS9jFshasHSc= v0.19.2/go.mod h1:S5wPhCqyDNAlzM9CnEdgTGV4OqhsW3jGO1UM1epwfJA= v0.0.0-20200413195148-3a45101e95ac/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0= v2.0.0 h1:Foj74zO6RbjjP4hBEKjnYtjjAhGg4jNynUdYF6fJrok= v2.0.0/go.mod h1:PBfzABfn139FHAV07az/IF9Wp1bkk3vpT2XSJ76fSDE= v2.2.0 h1:XRvcwJozkgZ1UQJmfMGpvRthQHOvihEhYtDfAaxMz/A= v2.2.0/go.mod h1:Od+F08eJP+W3HUb4pSrPpgp9DGU4GzlpG/TmITuYh/Y= v0.0.0-20200805222855-6aeccd4b50c6 h1:+WnxoVtG8TMiudHBSEtrVL1egv36TkkJm+bA8AxicmQ= v0.0.0-20200805222855-6aeccd4b50c6/go.mod h1:UuqjUnNftUyPE5H64/qeyjQoUZhGpeFDVdxjTeEVN2o= v0.0.0-20200729134348-d5654de09c73 h1:uJmqzgNWG7XyClnU/mLPBWwfKKF1K8Hf8whTseBgJcg=
# A more minimal logging API for Go
Before you consider this package, please read [this blog post by the
inimitable Dave Cheney][warning-makes-no-sense]. I really appreciate what
he has to say, and it largely aligns with my own experiences. Too many
choices of levels means inconsistent logs.
This package offers a purely abstract interface, based on these ideas but with
a few twists. Code can depend on just this interface and have the actual
logging implementation be injected from callers. Ideally only `main()` knows
what logging implementation is being used.
# Differences from Dave's ideas
The main differences are:
1) Dave basically proposes doing away with the notion of a logging API in favor
of `fmt.Printf()`. I disagree, especially when you consider things like output
locations, timestamps, file and line decorations, and structured logging. I
restrict the API to just 2 types of logs: info and error.
Info logs are things you want to tell the user which are not errors. Error
logs are, well, errors. If your code receives an `error` from a subordinate
function call and is logging that `error` *and not returning it*, use error
2) Verbosity-levels on info logs. This gives developers a chance to indicate
arbitrary grades of importance for info logs, without assigning names with
semantic meaning such as "warning", "trace", and "debug". Superficially this
may feel very similar, but the primary difference is the lack of semantics.
Because verbosity is a numerical value, it's safe to assume that an app running
with higher verbosity means more (and less important) logs will be generated.
This is a BETA grade API.
There are implementations for the following logging libraries:
- ****: [glogr](
- ****: [klogr](
- ****: [zapr](
- **log** (the Go standard library logger):
- ****: [logrusr](
## Conceptual
## Why structured logging?
- **Structured logs are more easily queriable**: Since you've got
key-value pairs, it's much easier to query your structured logs for
particular values by filtering on the contents of a particular key --
think searching request logs for error codes, Kubernetes reconcilers for
the name and namespace of the reconciled object, etc
- **Structured logging makes it easier to have cross-referencable logs**:
Similarly to searchability, if you maintain conventions around your
keys, it becomes easy to gather all log lines related to a particular
- **Structured logs allow better dimensions of filtering**: if you have
structure to your logs, you've got more precise control over how much
information is logged -- you might choose in a particular configuration
to log certain keys but not others, only log lines where a certain key
matches a certain value, etc, instead of just having v-levels and names
to key off of.
- **Structured logs better represent structured data**: sometimes, the
data that you want to log is inherently structured (think tuple-link
objects). Structured logs allow you to preserve that structure when
## Why V-levels?
**V-levels give operators an easy way to control the chattiness of log
operations**. V-levels provide a way for a given package to distinguish
the relative importance or verbosity of a given log message. Then, if
a particular logger or package is logging too many messages, the user
of the package can simply change the v-levels for that library.
## Why not more named levels, like Warning?
Read [Dave Cheney's post][warning-makes-no-sense]. Then read [Differences
from Dave's ideas](#differences-from-daves-ideas).
## Why not allow format strings, too?
**Format strings negate many of the benefits of structured logs**:
- They're not easily searchable without resorting to fuzzy searching,
regular expressions, etc
- They don't store structured data well, since contents are flattened into
a string
- They're not cross-referencable
- They don't compress easily, since the message is not constant
(unless you turn positional parameters into key-value pairs with numerical
keys, at which point you've gotten key-value logging with meaningless
## Practical
## Why key-value pairs, and not a map?
Key-value pairs are *much* easier to optimize, especially around
allocations. Zap (a structured logger that inspired logr's interface) has
[performance measurements](
that show this quite nicely.
While the interface ends up being a little less obvious, you get
potentially better performance, plus avoid making users type
`map[string]string{}` every time they want to log.
## What if my V-levels differ between libraries?
That's fine. Control your V-levels on a per-logger basis, and use the
`WithName` function to pass different loggers to different libraries.
Generally, you should take care to ensure that you have relatively
consistent V-levels within a given logger, however, as this makes deciding
on what verbosity of logs to request easier.
## But I *really* want to use a format string!
That's not actually a question. Assuming your question is "how do
I convert my mental model of logging with format strings to logging with
constant messages":
1. figure out what the error actually is, as you'd write in a TL;DR style,
and use that as a message
2. For every place you'd write a format specifier, look to the word before
it, and add that as a key value pair
For instance, consider the following examples (all taken from spots in the
Kubernetes codebase):
- `klog.V(4).Infof("Client is returning errors: code %v, error %v",
responseCode, err)` becomes `logger.Error(err, "client returned an
error", "code", responseCode)`
- `klog.V(4).Infof("Got a Retry-After %ds response for attempt %d to %v",
seconds, retries, url)` becomes `logger.V(4).Info("got a retry-after
response when requesting url", "attempt", retries, "after
seconds", seconds, "url", url)`
If you *really* must use a format string, place it as a key value, and
call `fmt.Sprintf` yourself -- for instance, `log.Printf("unable to
reflect over type %T")` becomes `logger.Info("unable to reflect over
type", "type", fmt.Sprintf("%T"))`. In general though, the cases where
this is necessary should be few and far between.
## How do I choose my V-levels?
This is basically the only hard constraint: increase V-levels to denote
more verbose or more debug-y logs.
Otherwise, you can start out with `0` as "you always want to see this",
`1` as "common logging that you might *possibly* want to turn off", and
`10` as "I would like to performance-test your log collection stack".
Then gradually choose levels in between as you need them, working your way
down from 10 (for debug and trace style logs) and up from 1 (for chattier
info-type logs).
## How do I choose my keys
- make your keys human-readable
- constant keys are generally a good idea
- be consistent across your codebase
- keys should naturally match parts of the message string
While key names are mostly unrestricted (and spaces are acceptable),
it's generally a good idea to stick to printable ascii characters, or at
least match the general character set of your log lines.
go 1.14
Copyright 2019 The logr Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
See the License for the specific language governing permissions and
limitations under the License.
// Package logr defines abstract interfaces for logging. Packages can depend on
// these interfaces and callers can implement logging in whatever way is
// appropriate.
// This design derives from Dave Cheney's blog:
// This is a BETA grade API. Until there is a significant 2nd implementation,
// I don't really know how it will change.
// The logging specifically makes it non-trivial to use format strings, to encourage
// attaching structured information instead of unstructured format strings.
// Usage
// Logging is done using a Logger. Loggers can have name prefixes and named
// values attached, so that all log messages logged with that Logger have some
// base context associated.
// The term "key" is used to refer to the name associated with a particular
// value, to disambiguate it from the general Logger name.
// For instance, suppose we're trying to reconcile the state of an object, and
// we want to log that we've made some decision.
// With the traditional log package, we might write:
// log.Printf(
// "decided to set field foo to value %q for object %s/%s",
// targetValue, object.Namespace, object.Name)
// With logr's structured logging, we'd write:
// // elsewhere in the file, set up the logger to log with the prefix of "reconcilers",
// // and the named value target-type=Foo, for extra context.
// log := mainLogger.WithName("reconcilers").WithValues("target-type", "Foo")
// // later on...
// log.Info("setting field foo on object", "value", targetValue, "object", object)
// Depending on our logging implementation, we could then make logging decisions
// based on field values (like only logging such events for objects in a certain
// namespace), or copy the structured information into a structured log store.
// For logging errors, Logger has a method called Error. Suppose we wanted to
// log an error while reconciling. With the traditional log package, we might
// write:
// log.Errorf("unable to reconcile object %s/%s: %v", object.Namespace, object.Name, err)
// With logr, we'd instead write:
// // assuming the above setup for log
// log.Error(err, "unable to reconcile object", "object", object)
// This functions similarly to:
// log.Info("unable to reconcile object", "error", err, "object", object)
// However, it ensures that a standard key for the error value ("error") is used
// across all error logging. Furthermore, certain implementations may choose to
// attach additional information (such as stack traces) on calls to Error, so
// it's preferred to use Error to log errors.
// Parts of a log line
// Each log message from a Logger has four types of context:
// logger name, log verbosity, log message, and the named values.
// The Logger name constists of a series of name "segments" added by successive
// calls to WithName. These name segments will be joined in some way by the
// underlying implementation. It is strongly reccomended that name segements
// contain simple identifiers (letters, digits, and hyphen), and do not contain
// characters that could muddle the log output or confuse the joining operation
// (e.g. whitespace, commas, periods, slashes, brackets, quotes, etc).
// Log verbosity represents how little a log matters. Level zero, the default,
// matters most. Increasing levels matter less and less. Try to avoid lots of
// different verbosity levels, and instead provide useful keys, logger names,
// and log messages for users to filter on. It's illegal to pass a log level
// below zero.
// The log message consists of a constant message attached to the the log line.
// This should generally be a simple description of what's occuring, and should
// never be a format string.
// Variable information can then be attached using named values (key/value
// pairs). Keys are arbitrary strings, while values may be any Go value.
// Key Naming Conventions
// Keys are not strictly required to conform to any specification or regex, but
// it is recommended that they:
// * be human-readable and meaningful (not auto-generated or simple ordinals)
// * be constant (not dependent on input data)
// * contain only printable characters
// * not contain whitespace or punctuation
// These guidelines help ensure that log data is processed properly regardless
// of the log implementation. For example, log implementations will try to
// output JSON data or will store data for later database (e.g. SQL) queries.
// While users are generally free to use key names of their choice, it's
// generally best to avoid using the following keys, as they're frequently used
// by implementations:
// - `"caller"`: the calling information (file/line) of a particular log line.
// - `"error"`: the underlying error value in the `Error` method.
// - `"level"`: the log level.
// - `"logger"`: the name of the associated logger.
// - `"msg"`: the log message.
// - `"stacktrace"`: the stack trace associated with a particular log line or
// error (often from the `Error` message).
// - `"ts"`: the timestamp for a log line.
// Implementations are encouraged to make use of these keys to represent the
// above concepts, when neccessary (for example, in a pure-JSON output form, it
// would be necessary to represent at least message and timestamp as ordinary
// named values).
package logr
// TODO: consider adding back in format strings if they're really needed
// TODO: consider other bits of zap/zapcore functionality like ObjectMarshaller (for arbitrary objects)
// TODO: consider other bits of glog functionality like Flush, InfoDepth, OutputStats
// Logger represents the ability to log messages, both errors and not.
type Logger interface {
// Enabled tests whether this Logger is enabled. For example, commandline
// flags might be used to set the logging verbosity and disable some info
// logs.
Enabled() bool
// Info logs a non-error message with the given key/value pairs as context.
// The msg argument should be used to add some constant description to
// the log line. The key/value pairs can then be used to add additional
// variable information. The key/value pairs should alternate string
// keys and arbitrary values.
Info(msg string, keysAndValues ...interface{})
// Error logs an error, with the given message and key/value pairs as context.
// It functions similarly to calling Info with the "error" named value, but may
// have unique behavior, and should be preferred for logging errors (see the
// package documentations for more information).
// The msg field should be used to add context to any underlying error,
// while the err field should be used to attach the actual error that
// triggered this log line, if present.
Error(err error, msg string, keysAndValues ...interface{})
// V returns an Logger value for a specific verbosity level, relative to
// this Logger. In other words, V values are additive. V higher verbosity
// level means a log message is less important. It's illegal to pass a log
// level less than zero.
V(level int) Logger
// WithValues adds some key-value pairs of context to a logger.
// See Info for documentation on how key/value pairs work.
WithValues(keysAndValues ...interface{}) Logger
// WithName adds a new element to the logger's name.
// Successive calls with WithName continue to append
// suffixes to the logger's name. It's strongly reccomended
// that name segments contain only letters, digits, and hyphens
// (see the package documentation for more information).
WithName(name string) Logger
- path: _test.go
- errcheck
exclude: .errcheck_excludes
# klog-gokit
# klog-gokit [![CircleCI](](
This packages is a replacement for [klog](
This package is a replacement for [](
in projects that use the [go-kit logger](
It also supports [](
It is heavily inspired by the [``]( package.
## Usage
......@@ -24,7 +26,7 @@ In your `Gopkg.toml`:
Add this line to your `go.mod` file:
replace => master
replace => master
In your `main.go`:
......@@ -73,6 +75,11 @@ Setting the logger to the klog package **MUST** happen before using klog in any
This table is rather opinionated and build for use with the Kubernetes' [Go client](
## Disclaimer
This project doesn't aim at covering the complete `klog` API. That being said, it should work ok for
projects that use `` (like [Prometheus]( for instance).
## License
Apache License 2.0, see [LICENSE](
require ( v1.1.1 // indirect v0.8.0 v0.4.0 // indirect v1.8.0 // indirect v1.0.0 // indirect v1.2.2 v0.10.0 v1.4.0
go 1.12
"extends": [
# OSX leaves these everywhere on SMB shares
# OSX trash
# Eclipse files
# Files generated by JetBrains IDEs, e.g. IntelliJ IDEA
# Vscode files
# Contributing Guidelines
Welcome to Kubernetes. We are excited about the prospect of you joining our [community](! The Kubernetes community abides by the CNCF [code of conduct]( Here is an excerpt:
_As contributors and maintainers of this project, and in the interest of fostering an open and welcoming community, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities._
## Getting Started
We have full documentation on how to get started contributing here:
- [Contributor License Agreement]( Kubernetes projects require that you sign a Contributor License Agreement (CLA) before we can accept your pull requests
- [Kubernetes Contributor Guide]( - Main contributor documentation, or you can just jump directly to the [contributing section](
- [Contributor Cheat Sheet]( - Common resources for existing developers
## Mentorship
- [Mentoring Initiatives]( - We have a diverse set of mentorship programs available that are always looking for volunteers!
## Contact Information
- [Slack](
- [Mailing List](!forum/kubernetes-sig-architecture)
# See the OWNERS docs at
- jayunit100
- hoegaarden
- andyxning
- neolit123
- pohly
- yagonobre
- vincepri
- detiber
- dims
- thockin
- justinsb
- tallclair
- piosz
- brancz
- DirectXMan12
- lavalamp
klog is a permanent fork of
## Why was klog created?
The decision to create klog was one that wasn't made lightly, but it was necessary due to some
drawbacks that are present in [glog]( Ultimately, the fork was created due to glog not being under active development; this can be seen in the glog README:
> The code in this repo [...] is not itself under development
This makes us unable to solve many use cases without a fork. The factors that contributed to needing feature development are listed below:
* `glog` [presents a lot "gotchas"]( and introduces challenges in containerized environments, all of which aren't well documented.
* `glog` doesn't provide an easy way to test logs, which detracts from the stability of software using it
* A long term goal is to implement a logging interface that allows us to add context, change output format, etc.
Historical context is available here:
How to use klog
- Replace imports for `` with ``
- Use `klog.InitFlags(nil)` explicitly for initializing global flags as we no longer use `init()` method to register the flags
- You can now use `log_file` instead of `log_dir` for logging to a single file (See `examples/log_file/usage_log_file.go`)
- If you want to redirect everything logged using klog somewhere else (say syslog!), you can use `klog.SetOutput()` method and supply a `io.Writer`. (See `examples/set_output/usage_set_output.go`)
- For more logging conventions (See [Logging Conventions](
**NOTE**: please use the newer go versions that support semantic import versioning in modules, ideally go 1.11.4 or greater.
### Coexisting with glog
This package can be used side by side with glog. [This example](examples/coexist_glog/coexist_glog.go) shows how to initialize and synchronize flags from the global `flag.CommandLine` FlagSet. In addition, the example makes use of stderr as combined output by setting `alsologtostderr` (or `logtostderr`) to `true`.
## Community, discussion, contribution, and support
Learn how to engage with the Kubernetes community on the [community page](
You can reach the maintainers of this project at:
- [Slack](
- [Mailing List](!forum/kubernetes-sig-architecture)
### Code of conduct
Participation in the Kubernetes community is governed by the [Kubernetes Code of Conduct](
Leveled execution logs for Go.
This is an efficient pure Go implementation of leveled logs in the
manner of the open source C++ package
By binding methods to booleans it is possible to use the log package
without paying the expense of evaluating the arguments to the log.
Through the -vmodule flag, the package also provides fine-grained
control over logging at the file level.
The comment from glog.go introduces the ideas:
Package glog implements logging analogous to the Google-internal
C++ INFO/ERROR/V setup. It provides functions Info, Warning,
Error, Fatal, plus formatting variants such as Infof. It
also provides V-style logging controlled by the -v and
-vmodule=file=2 flags.
Basic examples:
glog.Info("Prepare to repel boarders")
glog.Fatalf("Initialization failed: %s", err)
See the documentation for the V function for an explanation
of these examples:
if glog.V(2) {
glog.Info("Starting transaction...")
glog.V(2).Infoln("Processed", nItems, "elements")
The repository contains an open source version of the log package
used inside Google. The master copy of the source lives inside
Google, not here. The code in this repo is for export only and is not itself
under development. Feature requests will be ignored.
Send bug reports to
# Release Process
The `klog` is released on an as-needed basis. The process is as follows:
1. An issue is proposing a new release with a changelog since the last release
1. All [OWNERS](OWNERS) must LGTM this release
1. An OWNER runs `git tag -s $VERSION` and inserts the changelog and pushes the tag with `git push $VERSION`
1. The release issue is closed
1. An announcement email is sent to `` with the subject `[ANNOUNCE] kubernetes-template-project $VERSION is released`
# Defined below are the security contacts for this repo.
# They are the contact point for the Product Security Committee to reach out
# to for triaging and handling of incoming issues.
# The below names agree to abide by the
# [Embargo Policy](
# and will be removed and replaced if they violate that agreement.
# Kubernetes Community Code of Conduct
Please refer to our [Kubernetes Community Code of Conduct](
go 1.13
require v0.10.0
require v0.2.0
go 1.12
// Go support for leveled logs, analogous to
// Copyright 2013 Google Inc. All Rights Reserved.
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// See the License for the specific language governing permissions and
// limitations under the License.
// File I/O for logs.
package klog
import (