CHANGELOG.md 100 KB
Newer Older
Ganesh Vernekar committed
1
## 2.23.0 / 2020-11-26
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29

* [CHANGE] UI: Make the React UI default. #8142
* [CHANGE] Remote write: The following metrics were removed/renamed in remote write. #6815
     - `prometheus_remote_storage_succeeded_samples_total` was removed and `prometheus_remote_storage_samples_total` was introduced for all the samples attempted to send.
     - `prometheus_remote_storage_sent_bytes_total` was removed and replaced with `prometheus_remote_storage_samples_bytes_total` and `prometheus_remote_storage_metadata_bytes_total`.
     - `prometheus_remote_storage_failed_samples_total` -> `prometheus_remote_storage_samples_failed_total` .
     - `prometheus_remote_storage_retried_samples_total` -> `prometheus_remote_storage_samples_retried_total`.
     - `prometheus_remote_storage_dropped_samples_total` -> `prometheus_remote_storage_samples_dropped_total`.
     - `prometheus_remote_storage_pending_samples` -> `prometheus_remote_storage_samples_pending`.
* [CHANGE] Remote: Do not collect non-initialized timestamp metrics. #8060
* [FEATURE] [EXPERIMENTAL] Remote write: Allow metric metadata to be propagated via remote write. The following new metrics were introduced: `prometheus_remote_storage_metadata_total`, `prometheus_remote_storage_metadata_failed_total`, `prometheus_remote_storage_metadata_retried_total`, `prometheus_remote_storage_metadata_bytes_total`. #6815
* [ENHANCEMENT] Remote write: Added a metric `prometheus_remote_storage_max_samples_per_send` for remote write. #8102
* [ENHANCEMENT] TSDB: Make the snapshot directory name always the same length. #8138
* [ENHANCEMENT] TSDB: Create a checkpoint only once at the end of all head compactions. #8067
* [ENHANCEMENT] TSDB: Avoid Series API from hitting the chunks. #8050 
* [ENHANCEMENT] TSDB: Cache label name and last value when adding series during compactions making compactions faster. #8192
* [ENHANCEMENT] PromQL: Improved performance of Hash method making queries a bit faster. #8025
* [ENHANCEMENT] promtool: `tsdb list` now prints block sizes. #7993
* [ENHANCEMENT] promtool: Calculate mint and maxt per test avoiding unnecessary calculations. #8096
* [ENHANCEMENT] SD: Add filtering of services to Docker Swarm SD. #8074
* [BUGFIX] React UI: Fix button display when there are no panels. #8155
* [BUGFIX] PromQL: Fix timestamp() method for vector selector inside parenthesis. #8164
* [BUGFIX] PromQL: Don't include rendered expression on PromQL parse errors. #8177
* [BUGFIX] web: Fix panic with double close() of channel on calling `/-/quit/`. #8166
* [BUGFIX] TSDB: Fixed WAL corruption on partial writes within a page causing `invalid checksum` error on WAL replay. #8125
* [BUGFIX] Update config metrics `prometheus_config_last_reload_successful` and `prometheus_config_last_reload_success_timestamp_seconds` right after initial validation before starting TSDB.
* [BUGFIX] promtool: Correctly detect duplicate label names in exposition.

Frederic Branczyk committed
30 31 32 33
## 2.22.2 / 2020-11-16

* [BUGFIX] Fix race condition in syncing/stopping/reloading scrapers. #8176

Frederic Branczyk committed
34 35 36 37 38 39
## 2.22.1 / 2020-11-03

* [BUGFIX] Fix potential "mmap: invalid argument" errors in loading the head chunks, after an unclean shutdown, by performing read repairs. #8061
* [BUGFIX] Fix serving metrics and API when reloading scrape config. #8104
* [BUGFIX] Fix head chunk size calculation for size based retention. #8139

Frederic Branczyk committed
40
## 2.22.0 / 2020-10-07
Frederic Branczyk committed
41 42 43 44 45 46

As announced in the 2.21.0 release notes, the experimental gRPC API v2 has been
removed.

* [CHANGE] web: Remove APIv2. #7935
* [ENHANCEMENT] React UI: Implement missing TSDB head stats section. #7876
Frederic Branczyk committed
47
* [ENHANCEMENT] UI: Add Collapse all button to targets page. #6957
Frederic Branczyk committed
48 49 50 51 52 53 54 55 56 57
* [ENHANCEMENT] UI: Clarify alert state toggle via checkbox icon. #7936
* [ENHANCEMENT] Add `rule_group_last_evaluation_samples` and `prometheus_tsdb_data_replay_duration_seconds` metrics. #7737 #7977
* [ENHANCEMENT] Gracefully handle unknown WAL record types. #8004
* [ENHANCEMENT] Issue a warning for 64 bit systems running 32 bit binaries. #8012
* [BUGFIX] Adjust scrape timestamps to align them to the intended schedule, effectively reducing block size. Workaround for a regression in go1.14+. #7976
* [BUGFIX] promtool: Ensure alert rules are marked as restored in unit tests. #7661
* [BUGFIX] Eureka: Fix service discovery when compiled in 32-bit. #7964
* [BUGFIX] Don't do literal regex matching optimisation when case insensitive. #8013
* [BUGFIX] Fix classic UI sometimes running queries for instant query when in range query mode. #7984

Julien Pivotto committed
58
## 2.21.0 / 2020-09-11
59

Julien Pivotto committed
60 61
This release is built with Go 1.15, which deprecates [X.509 CommonName](https://golang.org/doc/go1.15#commonname)
in TLS certificates validation.
62

63 64 65 66
In the unlikely case that you use the gRPC API v2 (which is limited to TSDB
admin commands), please note that we will remove this experimental API in the
next minor release 2.22.

Julien Pivotto committed
67
* [CHANGE] Disable HTTP/2 because of concerns with the Go HTTP/2 client. #7588 #7701
68 69 70 71 72 73 74 75 76 77 78 79
* [CHANGE] PromQL: `query_log_file` path is now relative to the config file. #7701
* [CHANGE] Promtool: Replace the tsdb command line tool by a promtool tsdb subcommand. #6088
* [CHANGE] Rules: Label `rule_group_iterations` metric with group name. #7823
* [FEATURE] Eureka SD: New service discovery. #3369
* [FEATURE] Hetzner SD: New service discovery. #7822
* [FEATURE] Kubernetes SD: Support Kubernetes EndpointSlices. #6838
* [FEATURE] Scrape: Add per scrape-config targets limit. #7554
* [ENHANCEMENT] Support composite durations in PromQL, config and UI, e.g. 1h30m. #7713 #7833
* [ENHANCEMENT] DNS SD: Add SRV record target and port meta labels. #7678
* [ENHANCEMENT] Docker Swarm SD: Support tasks and service without published ports. #7686
* [ENHANCEMENT] PromQL: Reduce the amount of data queried by remote read when a subquery has an offset. #7667
* [ENHANCEMENT] Promtool: Add `--time` option to query instant command. #7829
80 81 82 83
* [ENHANCEMENT] UI: Respect the `--web.page-title` parameter in the React UI. #7607
* [ENHANCEMENT] UI: Add duration, labels, annotations to alerts page in the React UI. #7605
* [ENHANCEMENT] UI: Add duration on the React UI rules page, hide annotation and labels if empty. #7606
* [BUGFIX] API: Deduplicate series in /api/v1/series. #7862
84
* [BUGFIX] PromQL: Drop metric name in bool comparison between two instant vectors. #7819
85
* [BUGFIX] PromQL: Exit with an error when time parameters can't be parsed. #7505
Julien Pivotto committed
86
* [BUGFIX] Remote read: Re-add accidentally removed tracing for remote-read requests. #7916
87 88 89 90 91 92 93 94
* [BUGFIX] Rules: Detect extra fields in rule files. #7767
* [BUGFIX] Rules: Disallow overwriting the metric name in the `labels` section of recording rules. #7787
* [BUGFIX] Rules: Keep evaluation timestamp across reloads. #7775
* [BUGFIX] Scrape: Do not stop scrapes in progress during reload. #7752
* [BUGFIX] TSDB: Fix `chunks.HeadReadWriter: maxt of the files are not set` error. #7856
* [BUGFIX] TSDB: Delete blocks atomically to prevent corruption when there is a panic/crash during deletion. #7772
* [BUGFIX] Triton SD: Fix a panic when triton_sd_config is nil. #7671
* [BUGFIX] UI: Fix react UI bug with series going on and off. #7804
Julien Pivotto committed
95
* [BUGFIX] UI: Fix styling bug for target labels with special names in React UI. #7902
96 97
* [BUGFIX] Web: Stop CMUX and GRPC servers even with stale connections, preventing the server to stop on SIGTERM. #7810

beorn7 committed
98 99 100 101
## 2.20.1 / 2020-08-05

* [BUGFIX] SD: Reduce the Consul watch timeout to 2m and adjust the request timeout accordingly. #7724

beorn7 committed
102
## 2.20.0 / 2020-07-22
103

beorn7 committed
104 105
This release changes WAL compression from opt-in to default. WAL compression will prevent a downgrade to v2.10 or earlier without deleting the WAL. Disable WAL compression explicitly by setting the command line flag `--no-storage.tsdb.wal-compression` if you require downgrading to v2.10 or earlier.

beorn7 committed
106 107 108 109 110 111 112 113 114 115 116
* [CHANGE] promtool: Changed rule numbering from 0-based to 1-based when reporting rule errors. #7495
* [CHANGE] Remote read: Added `prometheus_remote_storage_read_queries_total` counter and `prometheus_remote_storage_read_request_duration_seconds` histogram, removed `prometheus_remote_storage_remote_read_queries_total` counter.
* [CHANGE] Remote write: Added buckets for longer durations to `prometheus_remote_storage_sent_batch_duration_seconds` histogram.
* [CHANGE] TSDB: WAL compression is enabled by default. #7410
* [FEATURE] PromQL: Added `group()` aggregator. #7480
* [FEATURE] SD: Added Docker Swarm SD. #7420
* [FEATURE] SD: Added DigitalOcean SD. #7407
* [FEATURE] SD: Added Openstack config option to query alternative endpoints. #7494
* [ENHANCEMENT] Configuration: Exit early on invalid config file and signal it with exit code 2. #7399
* [ENHANCEMENT] PromQL: `without` is now a valid metric identifier. #7533
* [ENHANCEMENT] PromQL: Optimized regex label matching for literals within the pattern or as prefix/suffix. #7453 #7503
117
* [ENHANCEMENT] promtool: Added time range parameters for labels API in promtool. #7463
beorn7 committed
118 119 120 121 122 123 124 125 126 127 128 129 130 131 132
* [ENHANCEMENT] Remote write: Include samples waiting in channel in pending samples metric. Log number of dropped samples on hard shutdown. #7335
* [ENHANCEMENT] Scrape: Ingest synthetic scrape report metrics atomically with the corresponding scraped metrics. #7562
* [ENHANCEMENT] SD: Reduce timeouts for Openstack SD. #7507
* [ENHANCEMENT] SD: Use 10m timeout for Consul watches. #7423
* [ENHANCEMENT] SD: Added AMI meta label for EC2 SD. #7386
* [ENHANCEMENT] TSDB: Increment WAL corruption metric also on WAL corruption during checkpointing. #7491
* [ENHANCEMENT] TSDB: Improved query performance for high-cardinality labels. #7448
* [ENHANCEMENT] UI: Display dates as well as timestamps in status page. #7544
* [ENHANCEMENT] UI: Improved scrolling when following hash-fragment links. #7456
* [ENHANCEMENT] UI: React UI renders numbers in alerts in a more human-readable way. #7426
* [BUGFIX] API: Fixed error status code in the query API. #7435 
* [BUGFIX] PromQL: Fixed `avg` and `avg_over_time` for NaN, Inf, and float64 overflows. #7346
* [BUGFIX] PromQL: Fixed off-by-one error in `histogram_quantile`. #7393
* [BUGFIX] promtool: Support extended durations in rules unit tests. #6297
* [BUGFIX] Scrape: Fix undercounting for `scrape_samples_post_metric_relabeling` in case of errors. #7342
beorn7 committed
133
* [BUGFIX] TSDB: Don't panic on WAL corruptions. #7550
beorn7 committed
134 135 136 137 138
* [BUGFIX] TSDB: Avoid leaving behind empty files in `chunks_head`, causing startup failures. #7573
* [BUGFIX] TSDB: Fixed race between compact (gc, populate) and head append causing unknown symbol error. #7560
* [BUGFIX] TSDB: Fixed unknown symbol error during head compaction. #7526
* [BUGFIX] TSDB: Fixed panic during TSDB metric registration. #7501
* [BUGFIX] TSDB: Fixed `--limit` command line flag in `tsdb` tool. #7430
139

Ganesh Vernekar committed
140 141 142 143
## 2.19.2 / 2020-06-26

* [BUGFIX] Remote Write: Fix panic when reloading config with modified queue parameters. #7452

Ganesh Vernekar committed
144 145 146 147
## 2.19.1 / 2020-06-18

* [BUGFIX] TSDB: Fix m-map file truncation leading to unsequential files. #7414

Ganesh Vernekar committed
148
## 2.19.0 / 2020-06-09
149 150 151 152 153 154 155 156 157 158 159

* [FEATURE] TSDB: Memory-map full chunks of Head (in-memory) block from disk. This reduces memory footprint and makes restarts faster. #6679
* [ENHANCEMENT] Discovery: Added discovery support for Triton global zones. #7250
* [ENHANCEMENT] Increased alert resend delay to be more tolerant towards failures. #7228
* [ENHANCEMENT] Remote Read: Added `prometheus_remote_storage_remote_read_queries_total` counter to count total number of remote read queries. #7328
* [ENHANCEMEMT] Added time range parameters for label names and label values API. #7288
* [ENHANCEMENT] TSDB: Reduced contention in isolation for high load. #7332 
* [BUGFIX] PromQL: Eliminated collision while checking for duplicate labels. #7058 
* [BUGFIX] React UI: Don't null out data when clicking on the current tab. #7243
* [BUGFIX] PromQL: Correctly track number of samples for a query. #7307
* [BUGFIX] PromQL: Return NaN when histogram buckets have 0 observations. #7318
160

161 162
## 2.18.2 / 2020-06-09

Ganesh Vernekar committed
163
* [BUGFIX] TSDB: Fix incorrect query results when using Prometheus with remote reads configured #7361
164

Bartlomiej Plotka committed
165 166 167 168
## 2.18.1 / 2020-05-07

* [BUGFIX] TSDB: Fixed snapshot API. #7217

169
## 2.18.0 / 2020-05-05
170

171
* [CHANGE] Federation: Only use local TSDB for federation (ignore remote read). #7096
172 173 174 175
* [CHANGE] Rules: `rule_evaluations_total` and `rule_evaluation_failures_total` have a `rule_group` label now. #7094
* [FEATURE] Tracing: Added experimental Jaeger support #7148
* [ENHANCEMENT] TSDB: Significantly reduce WAL size kept around after a block cut. #7098
* [ENHANCEMENT] Discovery: Add `architecture` meta label for EC2. #7000
176
* [BUGFIX] UI: Fixed wrong MinTime reported by /status. #7182
177 178 179 180 181
* [BUGFIX] React UI: Fixed multiselect legend on OSX. #6880
* [BUGFIX] Remote Write: Fixed blocked resharding edge case. #7122
* [BUGFIX] Remote Write: Fixed remote write not updating on relabel configs change. #7073


Julien Pivotto committed
182 183 184 185 186 187 188 189
## 2.17.2 / 2020-04-20

* [BUGFIX] Federation: Register federation metrics #7081
* [BUGFIX] PromQL: Fix panic in parser error handling #7132
* [BUGFIX] Rules: Fix reloads hanging when deleting a rule group that is being evaluated #7138
* [BUGFIX] TSDB: Fix a memory leak when prometheus starts with an empty TSDB WAL #7135
* [BUGFIX] TSDB: Make isolation more robust to panics in web handlers #7129 #7136

Julien Pivotto committed
190 191 192 193
## 2.17.1 / 2020-03-26

* [BUGFIX] TSDB: Fix query performance regression that increased memory and CPU usage #7051

Julien Pivotto committed
194
## 2.17.0 / 2020-03-24
195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215

This release implements isolation in TSDB. API queries and recording rules are
guaranteed to only see full scrapes and full recording rules. This comes with a
certain overhead in resource usage. Depending on the situation, there might be
some increase in memory usage, CPU usage, or query latency.

* [FEATURE] TSDB: Support isolation #6841
* [ENHANCEMENT] PromQL: Allow more keywords as metric names #6933
* [ENHANCEMENT] React UI: Add normalization of localhost URLs in targets page #6794
* [ENHANCEMENT] Remote read: Read from remote storage concurrently #6770
* [ENHANCEMENT] Rules: Mark deleted rule series as stale after a reload #6745
* [ENHANCEMENT] Scrape: Log scrape append failures as debug rather than warn #6852
* [ENHANCEMENT] TSDB: Improve query performance for queries that partially hit the head #6676
* [ENHANCEMENT] Consul SD: Expose service health as meta label #5313
* [ENHANCEMENT] EC2 SD: Expose EC2 instance lifecycle as meta label #6914
* [ENHANCEMENT] Kubernetes SD: Expose service type as meta label for K8s service role #6684
* [ENHANCEMENT] Kubernetes SD: Expose label_selector and field_selector #6807
* [ENHANCEMENT] Openstack SD: Expose hypervisor id as meta label #6962
* [BUGFIX] PromQL: Do not escape HTML-like chars in query log #6834 #6795
* [BUGFIX] React UI: Fix data table matrix values #6896
* [BUGFIX] React UI: Fix new targets page not loading when using non-ASCII characters #6892
216 217
* [BUGFIX] Remote read: Fix duplication of metrics read from remote storage with external labels #6967 #7018
* [BUGFIX] Remote write: Register WAL watcher and live reader metrics for all remotes, not just the first one #6998
218
* [BUGFIX] Scrape: Prevent removal of metric names upon relabeling #6891
219
* [BUGFIX] Scrape: Fix 'superfluous response.WriteHeader call' errors when scrape fails under some circonstances #6986
220
* [BUGFIX] Scrape: Fix crash when reloads are separated by two scrape intervals #7011
221

Callum Styan committed
222
## 2.16.0 / 2020-02-13
223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251

* [FEATURE] React UI: Support local timezone on /graph #6692
* [FEATURE] PromQL: add absent_over_time query function #6490
* [FEATURE] Adding optional logging of queries to their own file #6520
* [ENHANCEMENT] React UI: Add support for rules page and "Xs ago" duration displays #6503
* [ENHANCEMENT] React UI: alerts page, replace filtering togglers tabs with checkboxes #6543
* [ENHANCEMENT] TSDB: Export metric for WAL write errors #6647
* [ENHANCEMENT] TSDB: Improve query performance for queries that only touch the most recent 2h of data. #6651
* [ENHANCEMENT] PromQL: Refactoring in parser errors to improve error messages #6634
* [ENHANCEMENT] PromQL: Support trailing commas in grouping opts #6480
* [ENHANCEMENT] Scrape: Reduce memory usage on reloads by reusing scrape cache #6670
* [ENHANCEMENT] Scrape: Add metrics to track bytes and entries in the metadata cache #6675
* [ENHANCEMENT] promtool: Add support for line-column numbers for invalid rules output #6533
* [ENHANCEMENT] Avoid restarting rule groups when it is unnecessary #6450
* [BUGFIX] React UI: Send cookies on fetch() on older browsers #6553
* [BUGFIX] React UI: adopt grafana flot fix for stacked graphs #6603
* [BUFGIX] React UI: broken graph page browser history so that back button works as expected #6659
* [BUGFIX] TSDB: ensure compactionsSkipped metric is registered, and log proper error if one is returned from head.Init #6616
* [BUGFIX] TSDB: return an error on ingesting series with duplicate labels #6664
* [BUGFIX] PromQL: Fix unary operator precedence #6579
* [BUGFIX] PromQL: Respect query.timeout even when we reach query.max-concurrency #6712
* [BUGFIX] PromQL: Fix string and parentheses handling in engine, which affected React UI #6612
* [BUGFIX] PromQL: Remove output labels returned by absent() if they are produced by multiple identical label matchers #6493
* [BUGFIX] Scrape: Validate that OpenMetrics input ends with `# EOF` #6505
* [BUGFIX] Remote read: return the correct error if configs can't be marshal'd to JSON #6622
* [BUGFIX] Remote write: Make remote client `Store` use passed context, which can affect shutdown timing #6673
* [BUGFIX] Remote write: Improve sharding calculation in cases where we would always be consistently behind by tracking pendingSamples #6511
* [BUGFIX] Ensure prometheus_rule_group metrics are deleted when a rule group is removed #6693

Bartlomiej Plotka committed
252 253 254 255 256
## 2.15.2 / 2020-01-06

* [BUGFIX] TSDB: Fixed support for TSDB blocks built with Prometheus before 2.1.0. #6564
* [BUGFIX] TSDB: Fixed block compaction issues on Windows. #6547

Bartlomiej Plotka committed
257 258
## 2.15.1 / 2019-12-25

Bartlomiej Plotka committed
259
* [BUGFIX] TSDB: Fixed race on concurrent queries against same data. #6512
Bartlomiej Plotka committed
260

261 262 263 264 265 266 267 268 269 270 271 272
## 2.15.0 / 2019-12-23

* [CHANGE] Discovery: Removed `prometheus_sd_kubernetes_cache_*` metrics. Additionally `prometheus_sd_kubernetes_workqueue_latency_seconds` and `prometheus_sd_kubernetes_workqueue_work_duration_seconds` metrics now show correct values in seconds. #6393
* [CHANGE] Remote write: Changed `query` label on `prometheus_remote_storage_*` metrics to `remote_name` and `url`. #6043
* [FEATURE] API: Added new endpoint for exposing per metric metadata `/metadata`. #6420 #6442
* [ENHANCEMENT] TSDB: Significantly reduced memory footprint of loaded TSDB blocks. #6418 #6461
* [ENHANCEMENT] TSDB: Significantly optimized what we buffer during compaction which should result in lower memory footprint during compaction. #6422 #6452 #6468 #6475
* [ENHANCEMENT] TSDB: Improve replay latency. #6230
* [ENHANCEMENT] TSDB: WAL size is now used for size based retention calculation. #5886
* [ENHANCEMENT] Remote read: Added query grouping and range hints to the remote read request #6401
* [ENHANCEMENT] Remote write: Added `prometheus_remote_storage_sent_bytes_total` counter per queue. #6344
* [ENHANCEMENT] promql: Improved PromQL parser performance. #6356
273
* [ENHANCEMENT] React UI: Implemented missing pages like `/targets` #6276, TSDB status page #6281 #6267 and many other fixes and performance improvements.
274 275 276 277 278 279
* [ENHANCEMENT] promql: Prometheus now accepts spaces between time range and square bracket. e.g `[ 5m]` #6065
* [BUGFIX] Config: Fixed alertmanager configuration to not miss targets when configurations are similar. #6455
* [BUGFIX] Remote write: Value of `prometheus_remote_storage_shards_desired` gauge shows raw value of desired shards and it's updated correctly. #6378
* [BUGFIX] Rules: Prometheus now fails the evaluation of rules and alerts where metric results collide with labels specified in `labels` field. #6469
* [BUGFIX] API: Targets Metadata API `/targets/metadata` now accepts empty `match_targets` parameter as in the spec. #6303

280
## 2.14.0 / 2019-11-11
281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297

* [SECURITY/BUGFIX] UI: Ensure warnings from the API are escaped. #6279
* [FEATURE] API: `/api/v1/status/runtimeinfo` and `/api/v1/status/buildinfo` endpoints added for use by the React UI. #6243
* [FEATURE] React UI: implement the new experimental React based UI. #5694 and many more
  * Can be found by under `/new`.
  * Not all pages are implemented yet.
* [FEATURE] Status: Cardinality statistics added to the Runtime & Build Information page. #6125
* [ENHANCEMENT/BUGFIX] Remote write: fix delays in remote write after a compaction. #6021
* [ENHANCEMENT] UI: Alerts can be filtered by state. #5758
* [BUGFIX] API: lifecycle endpoints return 403 when not enabled. #6057
* [BUGFIX] Build: Fix Solaris build. #6149
* [BUGFIX] Promtool: Remove false duplicate rule warnings when checking rule files with alerts. #6270
* [BUGFIX] Remote write: restore use of deduplicating logger in remote write. #6113
* [BUGFIX] Remote write: do not reshard when unable to send samples. #6111
* [BUGFIX] Service discovery: errors are no longer logged on context cancellation. #6116, #6133
* [BUGFIX] UI: handle null response from API properly. #6071

298 299 300 301 302 303
## 2.13.1 / 2019-10-16

* [BUGFIX] Fix panic in ARM builds of Prometheus. #6110
* [BUGFIX] promql: fix potential panic in the query logger. #6094
* [BUGFIX] Multiple errors of http: superfluous response.WriteHeader call in the logs. #6145

304
## 2.13.0 / 2019-10-04
305

306
* [SECURITY/BUGFIX] UI: Fix a Stored DOM XSS vulnerability with query history [CVE-2019-10215](http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-10215). #6098
307
* [CHANGE] Metrics: renamed prometheus_sd_configs_failed_total to prometheus_sd_failed_configs and changed to Gauge #5254
308
* [ENHANCEMENT] Include the tsdb tool in builds. #6089
309 310 311 312 313 314 315 316 317 318 319 320 321
* [ENHANCEMENT] Service discovery: add new node address types for kubernetes. #5902
* [ENHANCEMENT] UI: show warnings if query have returned some warnings. #5964
* [ENHANCEMENT] Remote write: reduce memory usage of the series cache. #5849
* [ENHANCEMENT] Remote read: use remote read streaming to reduce memory usage. #5703
* [ENHANCEMENT] Metrics: added metrics for remote write max/min/desired shards to queue manager. #5787
* [ENHANCEMENT] Promtool: show the warnings during label query. #5924
* [ENHANCEMENT] Promtool: improve error messages when parsing bad rules. #5965
* [ENHANCEMENT] Promtool: more promlint rules. #5515
* [BUGFIX] Promtool: fix recording inconsistency due to duplicate labels. #6026
* [BUGFIX] UI: fixes service-discovery view when accessed from unhealthy targets. #5915
* [BUGFIX] Metrics format: OpenMetrics parser crashes on short input. #5939
* [BUGFIX] UI: avoid truncated Y-axis values. #6014

Julius Volz committed
322
## 2.12.0 / 2019-08-17
Julius Volz committed
323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339

* [FEATURE] Track currently active PromQL queries in a log file. #5794
* [FEATURE] Enable and provide binaries for `mips64` / `mips64le` architectures. #5792
* [ENHANCEMENT] Improve responsiveness of targets web UI and API endpoint. #5740
* [ENHANCEMENT] Improve remote write desired shards calculation. #5763
* [ENHANCEMENT] Flush TSDB pages more precisely. tsdb#660
* [ENHANCEMENT] Add `prometheus_tsdb_retention_limit_bytes` metric. tsdb#667
* [ENHANCEMENT] Add logging during TSDB WAL replay on startup. tsdb#662
* [ENHANCEMENT] Improve TSDB memory usage. tsdb#653, tsdb#643, tsdb#654, tsdb#642, tsdb#627
* [BUGFIX] Check for duplicate label names in remote read. #5829
* [BUGFIX] Mark deleted rules' series as stale on next evaluation. #5759
* [BUGFIX] Fix JavaScript error when showing warning about out-of-sync server time. #5833
* [BUGFIX] Fix `promtool test rules` panic when providing empty `exp_labels`. #5774
* [BUGFIX] Only check last directory when discovering checkpoint number. #5756
* [BUGFIX] Fix error propagation in WAL watcher helper functions. #5741
* [BUGFIX] Correctly handle empty labels from alert templates. #5845

Frederic Branczyk committed
340 341 342 343
## 2.11.2 / 2019-08-14

* [BUGFIX/SECURITY] Fix a Stored DOM XSS vulnerability with query history. #5888

Frederic Branczyk committed
344 345 346 347
## 2.11.1 / 2019-07-10

* [BUGFIX] Fix potential panic when prometheus is watching multiple zookeeper paths. #5749

Frederic Branczyk committed
348
## 2.11.0 / 2019-07-09
Frederic Branczyk committed
349 350 351 352 353 354 355 356 357 358 359

* [CHANGE] Remove `max_retries` from queue_config (it has been unused since rewriting remote-write to utilize the write-ahead-log). #5649
* [CHANGE] The meta file `BlockStats` no longer holds size information. This is now dynamically calculated and kept in memory. It also includes the meta file size which was not included before. tsdb#637
* [CHANGE] Renamed metric from `prometheus_tsdb_wal_reader_corruption_errors` to `prometheus_tsdb_wal_reader_corruption_errors_total`. tsdb#622
* [FEATURE] Add option to use Alertmanager API v2. #5482
* [FEATURE] Added `humanizePercentage` function for templates. #5670
* [FEATURE] Include InitContainers in Kubernetes Service Discovery. #5598
* [FEATURE] Provide option to compress WAL records using Snappy. [#609](https://github.com/prometheus/tsdb/pull/609)
* [ENHANCEMENT] Create new clean segment when starting the WAL. tsdb#608
* [ENHANCEMENT] Reduce allocations in PromQL aggregations. #5641
* [ENHANCEMENT] Add storage warnings to LabelValues and LabelNames API results. #5673
Julius Volz committed
360
* [ENHANCEMENT] Add `prometheus_http_requests_total` metric. #5640
Frederic Branczyk committed
361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377
* [ENHANCEMENT] Enable openbsd/arm build. #5696
* [ENHANCEMENT] Remote-write allocation improvements. #5614
* [ENHANCEMENT] Query performance improvement: Efficient iteration and search in HashForLabels and HashWithoutLabels. #5707
* [ENHANCEMENT] Allow injection of arbitrary headers in promtool. #4389
* [ENHANCEMENT] Allow passing `external_labels` in alert unit tests groups. #5608
* [ENHANCEMENT] Allows globs for rules when unit testing. #5595
* [ENHANCEMENT] Improved postings intersection matching. tsdb#616
* [ENHANCEMENT] Reduced disk usage for WAL for small setups. tsdb#605
* [ENHANCEMENT] Optimize queries using regexp for set lookups. tsdb#602
* [BUGFIX] resolve race condition in maxGauge. #5647
* [BUGFIX] Fix ZooKeeper connection leak. #5675
* [BUGFIX] Improved atomicity of .tmp block replacement during compaction for usual case. tsdb#636
* [BUGFIX] Fix "unknown series references" after clean shutdown. tsdb#623
* [BUGFIX] Re-calculate block size when calling `block.Delete`. tsdb#637
* [BUGFIX] Fix unsafe snapshots with head block. tsdb#641
* [BUGFIX] `prometheus_tsdb_compactions_failed_total` is now incremented on any compaction failure. tsdb#613

beorn7 committed
378
## 2.10.0 / 2019-05-25
beorn7 committed
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397

* [CHANGE/BUGFIX] API: Encode alert values as string to correctly represent Inf/NaN. #5582
* [FEATURE] Template expansion: Make external labels available as `$externalLabels` in alert and console template expansion. #5463
* [FEATURE] TSDB: Add `prometheus_tsdb_wal_segment_current` metric for the WAL segment index that TSDB is currently writing to. tsdb#601
* [FEATURE] Scrape: Add `scrape_series_added` per-scrape metric. #5546
* [ENHANCEMENT] Discovery/kubernetes: Add labels `__meta_kubernetes_endpoint_node_name` and `__meta_kubernetes_endpoint_hostname`. #5571
* [ENHANCEMENT] Discovery/azure: Add label `__meta_azure_machine_public_ip`. #5475
* [ENHANCEMENT] TSDB: Simplify mergedPostings.Seek, resulting in better performance if there are many posting lists. tsdb#595
* [ENHANCEMENT] Log filesystem type on startup. #5558
* [ENHANCEMENT] Cmd/promtool: Use POST requests for Query and QueryRange. client_golang#557
* [ENHANCEMENT] Web: Sort alerts by group name. #5448
* [ENHANCEMENT] Console templates: Add convenience variables `$rawParams`, `$params`, `$path`. #5463
* [BUGFIX] TSDB: Don't panic when running out of disk space and recover nicely from the condition. tsdb#582
* [BUGFIX] TSDB: Correctly handle empty labels. tsdb#594
* [BUGFIX] TSDB: Don't crash on an unknown tombstone reference. tsdb#604
* [BUGFIX] Storage/remote: Remove queue-manager specific metrics if queue no longer exists. #5445 #5485 #5555
* [BUGFIX] PromQL: Correctly display `{__name__="a"}`. #5552
* [BUGFIX] Discovery/kubernetes: Use `service` rather than `ingress` as the name for the service workqueue. #5520
* [BUGFIX] Discovery/azure: Don't panic on a VM with a public IP. #5587
beorn7 committed
398
* [BUGFIX] Discovery/triton: Always read HTTP body to completion. #5596
beorn7 committed
399 400
* [BUGFIX] Web: Fixed Content-Type for js and css instead of using `/etc/mime.types`. #5551

Brian Brazil committed
401 402 403 404 405 406 407 408
## 2.9.2 / 2019-04-24

* [BUGFIX] Make sure subquery range is taken into account for selection #5467
* [BUGFIX] Exhaust every request body before closing it #5166
* [BUGFIX] Cmd/promtool: return errors from rule evaluations #5483
* [BUGFIX] Remote Storage: string interner should not panic in release #5487
* [BUGFIX] Fix memory allocation regression in mergedPostings.Seek tsdb#586

Brian Brazil committed
409 410 411 412 413
## 2.9.1 / 2019-04-16

* [BUGFIX] Discovery/kubernetes: fix missing label sanitization #5462
* [BUGFIX] Remote_write: Prevent reshard concurrent with calling stop #5460

Brian Brazil committed
414 415 416 417 418
## 2.9.0 / 2019-04-15

This releases uses Go 1.12, which includes a change in how memory is released
to Linux. This will cause RSS to be reported as higher, however this is harmless
and the memory is available to the kernel when it needs it.
419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437

* [CHANGE/ENHANCEMENT] Update Consul to support catalog.ServiceMultipleTags. #5151
* [FEATURE] Add honor_timestamps scrape option. #5304
* [ENHANCEMENT] Discovery/kubernetes: add present labels for labels/annotations. #5443
* [ENHANCEMENT] OpenStack SD: Add ProjectID and UserID meta labels. #5431
* [ENHANCEMENT] Add GODEBUG and retention to the runtime page. #5324 #5322
* [ENHANCEMENT] Add support for POSTing to /series endpoint. #5422
* [ENHANCEMENT] Support PUT methods for Lifecycle and Admin APIs. #5376
* [ENHANCEMENT] Scrape: Add global jitter for HA server. #5181
* [ENHANCEMENT] Check for cancellation on every step of a range evaluation. #5131
* [ENHANCEMENT] String interning for labels & values in the remote_write path. #5316
* [ENHANCEMENT] Don't lose the scrape cache on a failed scrape. #5414
* [ENHANCEMENT] Reload cert files from disk automatically. common#173
* [ENHANCEMENT] Use fixed length millisecond timestamp format for logs. common#172
* [ENHANCEMENT] Performance improvements for postings. tsdb#509 tsdb#572
* [BUGFIX] Remote Write: fix checkpoint reading. #5429
* [BUGFIX] Check if label value is valid when unmarshaling external labels from YAML. #5316
* [BUGFIX] Promparse: sort all labels when parsing. #5372
* [BUGFIX] Reload rules: copy state on both name and labels. #5368
yuxiaobo committed
438
* [BUGFIX] Exponentiation operator to drop metric name in result of operation. #5329
439 440 441 442 443
* [BUGFIX] Config: resolve more file paths. #5284
* [BUGFIX] Promtool: resolve relative paths in alert test files. #5336
* [BUGFIX] Set TLSHandshakeTimeout in HTTP transport. common#179
* [BUGFIX] Use fsync to be more resilient to machine crashes. tsdb#573 tsdb#578
* [BUGFIX] Keep series that are still in WAL in checkpoints. tsdb#577
444
* [BUGFIX] Fix output sample values for scalar-to-vector comparison operations. #5454
445

Ganesh Vernekar committed
446 447
## 2.8.1 / 2019-03-28

Tariq Ibrahim committed
448
* [BUGFIX] Display the job labels in `/targets` which was removed accidentally. #5406
Ganesh Vernekar committed
449

Ganesh Vernekar committed
450 451 452
## 2.8.0 / 2019-03-12

This release uses Write-Ahead Logging (WAL) for the remote_write API. This currently causes a slight increase in memory usage, which will be addressed in future releases.
453

454
* [CHANGE] Default time retention is used only when no size based retention is specified. These are flags where time retention is specified by the flag `--storage.tsdb.retention` and size retention by `--storage.tsdb.retention.size`. #5216
455 456 457 458 459 460 461 462 463 464
* [CHANGE] `prometheus_tsdb_storage_blocks_bytes_total` is now `prometheus_tsdb_storage_blocks_bytes`. prometheus/tsdb#506
* [FEATURE] [EXPERIMENTAL] Time overlapping blocks are now allowed; vertical compaction and vertical query merge. It is an optional feature which is controlled by the `--storage.tsdb.allow-overlapping-blocks` flag, disabled by default. prometheus/tsdb#370
* [ENHANCEMENT] Use the WAL for remote_write API. #4588
* [ENHANCEMENT] Query performance improvements. prometheus/tsdb#531
* [ENHANCEMENT] UI enhancements with upgrade to Bootstrap 4. #5226
* [ENHANCEMENT] Reduce time that Alertmanagers are in flux when reloaded. #5126
* [ENHANCEMENT] Limit number of metrics displayed on UI to 10000. #5139
* [ENHANCEMENT] (1) Remember All/Unhealthy choice on target-overview when reloading page. (2) Resize text-input area on Graph page on mouseclick. #5201
* [ENHANCEMENT] In `histogram_quantile` merge buckets with equivalent le values. #5158.
* [ENHANCEMENT] Show list of offending labels in the error message in many-to-many scenarios. #5189
Ganesh Vernekar committed
465
* [ENHANCEMENT] Show `Storage Retention` criteria in effect on `/status` page. #5322
466 467 468 469 470 471 472 473 474
* [BUGFIX] Fix sorting of rule groups. #5260
* [BUGFIX] Fix support for password_file and bearer_token_file in Kubernetes SD. #5211
* [BUGFIX] Scrape: catch errors when creating HTTP clients #5182. Adds new metrics:
  * `prometheus_target_scrape_pools_total`
  * `prometheus_target_scrape_pools_failed_total`
  * `prometheus_target_scrape_pool_reloads_total`
  * `prometheus_target_scrape_pool_reloads_failed_total`
* [BUGFIX] Fix panic when aggregator param is not a literal. #5290

Goutham Veeramachaneni committed
475 476 477 478
## 2.7.2 / 2019-03-02

* [BUGFIX] `prometheus_rule_group_last_evaluation_timestamp_seconds` is now a unix timestamp. #5186

479 480 481 482 483 484 485 486
## 2.7.1 / 2019-01-31

This release has a fix for a Stored DOM XSS vulnerability that can be triggered when using the query history functionality. Thanks to Dor Tumarkin from Checkmarx for reporting it.

* [BUGFIX/SECURITY] Fix a Stored DOM XSS vulnerability with query history. #5163
* [BUGFIX] `prometheus_rule_group_last_duration_seconds` now reports seconds instead of nanoseconds. #5153
* [BUGFIX] Make sure the targets are consistently sorted in the targets page. #5161

487
## 2.7.0 / 2019-01-28
488

489
We're rolling back the Dockerfile changes introduced in 2.6.0. If you made changes to your docker deployment in 2.6.0, you will need to roll them back. This release also adds experimental support for disk size based retention. To accommodate that we are deprecating the flag `storage.tsdb.retention` in favour of `storage.tsdb.retention.time`. We print a warning if the flag is in use, but it will function without breaking until Prometheus 3.0.
490 491

* [CHANGE] Rollback Dockerfile to version at 2.5.0. Rollback of the breaking change introduced in 2.6.0. #5122
492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507
* [FEATURE] Add subqueries to PromQL. #4831
* [FEATURE] [EXPERIMENTAL] Add support for disk size based retention. Note that we don't consider the WAL size which could be significant and the time based retention policy also applies. #5109 prometheus/tsdb#343
* [FEATURE] Add CORS origin flag. #5011
* [ENHANCEMENT] Consul SD: Add tagged address to the discovery metadata. #5001
* [ENHANCEMENT] Kubernetes SD: Add service external IP and external name to the discovery metadata. #4940
* [ENHANCEMENT] Azure SD: Add support for Managed Identity authentication. #4590
* [ENHANCEMENT] Azure SD: Add tenant and subscription IDs to the discovery metadata. #4969
* [ENHANCEMENT] OpenStack SD: Add support for application credentials based authentication. #4968
* [ENHANCEMENT] Add metric for number of rule groups loaded. #5090
* [BUGFIX] Avoid duplicate tests for alert unit tests. #4964
* [BUGFIX] Don't depend on given order when comparing samples in alert unit testing. #5049
* [BUGFIX] Make sure the retention period doesn't overflow. #5112
* [BUGFIX] Make sure the blocks don't get very large. #5112
* [BUGFIX] Don't generate blocks with no samples. prometheus/tsdb#374
* [BUGFIX] Reintroduce metric for WAL corruptions. prometheus/tsdb#473

Simon Pasquier committed
508 509 510 511 512 513
## 2.6.1 / 2019-01-15

* [BUGFIX] Azure SD: Fix discovery getting stuck sometimes. #5088
* [BUGFIX] Marathon SD: Use `Tasks.Ports` when `RequirePorts` is `false`. #5026
* [BUGFIX] Promtool: Fix "out-of-order sample" errors when testing rules. #5069

Simon Pasquier committed
514
## 2.6.0 / 2018-12-17
515

Simon Pasquier committed
516
* [CHANGE] Remove default flags from the container's entrypoint, run Prometheus from `/etc/prometheus` and symlink the storage directory to `/etc/prometheus/data`. #4976
517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556
* [CHANGE] Promtool: Remove the `update` command. #3839
* [FEATURE] Add JSON log format via the `--log.format` flag. #4876
* [FEATURE] API: Add /api/v1/labels endpoint to get all label names. #4835
* [FEATURE] Web: Allow setting the page's title via the `--web.ui-title` flag. #4841
* [ENHANCEMENT] Add `prometheus_tsdb_lowest_timestamp_seconds`, `prometheus_tsdb_head_min_time_seconds` and `prometheus_tsdb_head_max_time_seconds` metrics. #4888
* [ENHANCEMENT] Add `rule_group_last_evaluation_timestamp_seconds` metric. #4852
* [ENHANCEMENT] Add `prometheus_template_text_expansion_failures_total` and `prometheus_template_text_expansions_total` metrics. #4747
* [ENHANCEMENT] Set consistent User-Agent header in outgoing requests. #4891
* [ENHANCEMENT] Azure SD: Error out at load time when authentication parameters are missing. #4907
* [ENHANCEMENT] EC2 SD: Add the machine's private DNS name to the discovery metadata. #4693
* [ENHANCEMENT] EC2 SD: Add the operating system's platform to the discovery metadata. #4663
* [ENHANCEMENT] Kubernetes SD: Add the pod's phase to the discovery metadata. #4824
* [ENHANCEMENT] Kubernetes SD: Log Kubernetes messages. #4931
* [ENHANCEMENT] Promtool: Collect CPU and trace profiles. #4897
* [ENHANCEMENT] Promtool: Support writing output as JSON. #4848
* [ENHANCEMENT] Remote Read: Return available data if remote read fails partially. #4832
* [ENHANCEMENT] Remote Write: Improve queue performance. #4772
* [ENHANCEMENT] Remote Write: Add min_shards parameter to set the minimum number of shards. #4924
* [ENHANCEMENT] TSDB: Improve WAL reading. #4953
* [ENHANCEMENT] TSDB: Memory improvements. #4953
* [ENHANCEMENT] Web: Log stack traces on panic. #4221
* [ENHANCEMENT] Web UI: Add copy to clipboard button for configuration. #4410
* [ENHANCEMENT] Web UI: Support console queries at specific times. #4764
* [ENHANCEMENT] Web UI: group targets by job then instance. #4898 #4806
* [BUGFIX] Deduplicate handler labels for HTTP metrics. #4732
* [BUGFIX] Fix leaked queriers causing shutdowns to hang. #4922
* [BUGFIX] Fix configuration loading panics on nil pointer slice elements. #4942
* [BUGFIX] API: Correctly skip mismatching targets on /api/v1/targets/metadata. #4905
* [BUGFIX] API: Better rounding for incoming query timestamps. #4941
* [BUGFIX] Azure SD: Fix panic. #4867
* [BUGFIX] Console templates: Fix hover when the metric has a null value. #4906
* [BUGFIX] Discovery: Remove all targets when the scrape configuration gets empty. #4819
* [BUGFIX] Marathon SD: Fix leaked connections. #4915
* [BUGFIX] Marathon SD: Use 'hostPort' member of portMapping to construct target endpoints. #4887
* [BUGFIX] PromQL: Fix a goroutine leak in the lexer/parser. #4858
* [BUGFIX] Scrape: Pass through content-type for non-compressed output. #4912
* [BUGFIX] Scrape: Fix deadlock in the scrape's manager. #4894
* [BUGFIX] Scrape: Scrape targets at fixed intervals even after Prometheus restarts. #4926
* [BUGFIX] TSDB: Support restored snapshots including the head properly. #4953
* [BUGFIX] TSDB: Repair WAL when the last record in a segment is torn. #4953
Simon Pasquier committed
557
* [BUGFIX] TSDB: Fix unclosed file readers on Windows systems. #4997
558 559
* [BUGFIX] Web: Avoid proxy to connect to the local gRPC server. #4572

Frederic Branczyk committed
560
## 2.5.0 / 2018-11-06
Frederic Branczyk committed
561 562

* [CHANGE] Group targets by scrape config instead of job name. #4806 #4526
563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588
* [CHANGE] Marathon SD: Various changes to adapt to Marathon 1.5+. #4499
* [CHANGE] Discovery: Split `prometheus_sd_discovered_targets` metric by scrape and notify (Alertmanager SD) as well as by section in the respective configuration. #4753
* [FEATURE] Add OpenMetrics support for scraping (EXPERIMENTAL). #4700
* [FEATURE] Add unit testing for rules. #4350
* [FEATURE] Make maximum number of samples per query configurable via `--query.max-samples` flag. #4513
* [FEATURE] Make maximum number of concurrent remote reads configurable via `--storage.remote.read-concurrent-limit` flag. #4656
* [ENHANCEMENT] Support s390x platform for Linux. #4605
* [ENHANCEMENT] API: Add `prometheus_api_remote_read_queries` metric tracking currently executed or waiting remote read API requests. #4699
* [ENHANCEMENT] Remote Read: Add `prometheus_remote_storage_remote_read_queries` metric tracking currently in-flight remote read queries. #4677
* [ENHANCEMENT] Remote Read: Reduced memory usage. #4655
* [ENHANCEMENT] Discovery: Add `prometheus_sd_discovered_targets`, `prometheus_sd_received_updates_total`, `prometheus_sd_updates_delayed_total`, and `prometheus_sd_updates_total` metrics for discovery subsystem. #4667
* [ENHANCEMENT] Discovery: Improve performance of previously slow updates of changes of targets. #4526
* [ENHANCEMENT] Kubernetes SD: Add extended metrics. #4458
* [ENHANCEMENT] OpenStack SD: Support discovering instances from all projects. #4682
* [ENHANCEMENT] OpenStack SD: Discover all interfaces. #4649
* [ENHANCEMENT] OpenStack SD: Support `tls_config` for the used HTTP client. #4654
* [ENHANCEMENT] Triton SD: Add ability to filter triton_sd targets by pre-defined groups. #4701
* [ENHANCEMENT] Web UI: Avoid browser spell-checking in expression field. #4728
* [ENHANCEMENT] Web UI: Add scrape duration and last evaluation time in targets and rules pages. #4722
* [ENHANCEMENT] Web UI: Improve rule view by wrapping lines. #4702
* [ENHANCEMENT] Rules: Error out at load time for invalid templates, rather than at evaluation time. #4537
* [ENHANCEMENT] TSDB: Add metrics for WAL operations. #4692
* [BUGFIX] Change max/min over_time to handle NaNs properly. #4386
* [BUGFIX] Check label name for `count_values` PromQL function. #4585
* [BUGFIX] Ensure that vectors and matrices do not contain identical label-sets. #4589

589 590
## 2.4.3 / 2018-10-04

591 592 593
* [BUGFIX] Fix panic when using custom EC2 API for SD #4672
* [BUGFIX] Fix panic when Zookeeper SD cannot connect to servers #4669
* [BUGFIX] Make the skip_head an optional parameter for snapshot API #4674
594

595 596 597 598 599 600 601
## 2.4.2 / 2018-09-21

 The last release didn't have bugfix included due to a vendoring error.

 * [BUGFIX] Handle WAL corruptions properly prometheus/tsdb#389
 * [BUGFIX] Handle WAL migrations correctly on Windows prometheus/tsdb#392

602 603 604 605 606
## 2.4.1 / 2018-09-19

* [ENHANCEMENT] New TSDB metrics prometheus/tsdb#375 prometheus/tsdb#363
* [BUGFIX] Render UI correctly for Windows #4616

607
## 2.4.0 / 2018-09-11
608

609
This release includes multiple bugfixes and features. Further, the WAL implementation has been re-written so the storage is not forward compatible. Prometheus 2.3 storage will work on 2.4 but not vice-versa.
610 611 612 613 614 615

* [CHANGE] Reduce remote write default retries #4279
* [CHANGE] Remove /heap endpoint #4460
* [FEATURE] Persist alert 'for' state across restarts #4061
* [FEATURE] Add API providing per target metric metadata #4183
* [FEATURE] Add API providing recording and alerting rules #4318 #4501
616 617
* [ENHANCEMENT] Brand new WAL implementation for TSDB. Forwards incompatible with previous WAL.
* [ENHANCEMENT] Show rule evaluation errors in UI #4457
618 619 620 621
* [ENHANCEMENT] Throttle resends of alerts to Alertmanager #4538
* [ENHANCEMENT] Send EndsAt along with the alert to Alertmanager #4550
* [ENHANCEMENT] Limit the samples returned by remote read endpoint #4532
* [ENHANCEMENT] Limit the data read in through remote read #4239
Ben Kochie committed
622
* [ENHANCEMENT] Coalesce identical SD configurations #3912
623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643
* [ENHANCEMENT] `promtool`: Add new commands for debugging and querying #4247 #4308 #4346 #4454
* [ENHANCEMENT] Update console examples for node_exporter v0.16.0 #4208
* [ENHANCEMENT] Optimize PromQL aggregations #4248
* [ENHANCEMENT] Remote read: Add Offset to hints #4226
* [ENHANCEMENT] `consul_sd`: Add support for ServiceMeta field #4280
* [ENHANCEMENT] `ec2_sd`: Maintain order of subnet_id label #4405
* [ENHANCEMENT] `ec2_sd`: Add support for custom endpoint to support EC2 compliant APIs #4333
* [ENHANCEMENT] `ec2_sd`: Add instance_owner label #4514
* [ENHANCEMENT] `azure_sd`: Add support for VMSS discovery and multiple environments #4202 #4569
* [ENHANCEMENT] `gce_sd`: Add instance_id label #4488
* [ENHANCEMENT] Forbid rule-abiding robots from indexing #4266
* [ENHANCEMENT] Log virtual memory limits on startup #4418
* [BUGFIX] Wait for service discovery to stop before exiting #4508
* [BUGFIX] Render SD configs properly #4338
* [BUGFIX] Only add LookbackDelta to vector selectors #4399
* [BUGFIX] `ec2_sd`: Handle panic-ing nil pointer #4469
* [BUGFIX] `consul_sd`: Stop leaking connections #4443
* [BUGFIX] Use templated labels also to identify alerts #4500
* [BUGFIX] Reduce floating point errors in stddev and related functions #4533
* [BUGFIX] Log errors while encoding responses #4359

Brian Brazil committed
644 645 646 647 648 649 650 651
## 2.3.2 / 2018-07-12

* [BUGFIX] Fix various tsdb bugs #4369
* [BUGFIX] Reorder startup and shutdown to prevent panics. #4321
* [BUGFIX] Exit with non-zero code on error #4296
* [BUGFIX] discovery/kubernetes/ingress: fix scheme discovery #4329
* [BUGFIX] Fix race in zookeeper sd #4355
* [BUGFIX] Better timeout handling in promql #4291 #4300
Ben Kochie committed
652
* [BUGFIX] Propagate errors when selecting series from the tsdb #4136
Brian Brazil committed
653

Brian Brazil committed
654 655 656 657
## 2.3.1 / 2018-06-19

* [BUGFIX] Avoid infinite loop on duplicate NaN values. #4275
* [BUGFIX] Fix nil pointer deference when using various API endpoints #4282
Ben Kochie committed
658
* [BUGFIX] config: set target group source index during unmarshaling #4245
Brian Brazil committed
659 660 661 662 663
* [BUGFIX] discovery/file: fix logging #4178
* [BUGFIX] kubernetes_sd: fix namespace filtering #4285
* [BUGFIX] web: restore old path prefix behavior #4273
* [BUGFIX] web: remove security headers added in 2.3.0 #4259

Fabian Reinartz committed
664
## 2.3.0 / 2018-06-05
665

Fabian Reinartz committed
666 667
* [CHANGE] `marathon_sd`: use `auth_token` and `auth_token_file` for token-based authentication instead of `bearer_token` and `bearer_token_file` respectively.
* [CHANGE] Metric names for HTTP server metrics changed
Fabian Reinartz committed
668 669 670
* [FEATURE] Add query commands to promtool
* [FEATURE] Add security headers to HTTP server responses
* [FEATURE] Pass query hints via remote read API
Fabian Reinartz committed
671
* [FEATURE] Basic auth passwords can now be configured via file across all configuration
Ben Kochie committed
672
* [ENHANCEMENT] Optimize PromQL and API serialization for memory usage and allocations
Fabian Reinartz committed
673
* [ENHANCEMENT] Limit number of dropped targets in web UI
Fabian Reinartz committed
674 675
* [ENHANCEMENT] Consul and EC2 service discovery allow using server-side filtering for performance improvement
* [ENHANCEMENT] Add advanced filtering configuration to EC2 service discovery
676
* [ENHANCEMENT] `marathon_sd`: adds support for basic and bearer authentication, plus all other common HTTP client options (TLS config, proxy URL, etc.)
Fabian Reinartz committed
677 678 679 680 681 682 683 684 685 686 687
* [ENHANCEMENT] Provide machine type metadata and labels in GCE service discovery
* [ENHANCEMENT] Add pod controller kind and name to Kubernetes service discovery data
* [ENHANCEMENT] Move TSDB to flock-based log file that works with Docker containers
* [BUGFIX] Properly propagate storage errors in PromQL
* [BUGFIX] Fix path prefix for web pages
* [BUGFIX] Fix goroutine leak in Consul service discovery
* [BUGFIX] Fix races in scrape manager
* [BUGFIX] Fix OOM for very large k in PromQL topk() queries
* [BUGFIX] Make remote write more resilient to unavailable receivers
* [BUGFIX] Make remote write shutdown cleanly
* [BUGFIX] Don't leak files on errors in TSDB's tombstone cleanup
Fabian Reinartz committed
688 689
* [BUGFIX] Unary minus expressions now removes the metric name from results
* [BUGFIX] Fix bug that lead to wrong amount of samples considered for time range expressions
690

Fabian Reinartz committed
691 692 693 694
## 2.2.1 / 2018-03-13

* [BUGFIX] Fix data loss in TSDB on compaction
* [BUGFIX] Correctly stop timer in remote-write path
Ben Kochie committed
695
* [BUGFIX] Fix deadlock triggered by loading targets page
Fabian Reinartz committed
696 697 698
* [BUGFIX] Fix incorrect buffering of samples on range selection queries
* [BUGFIX] Handle large index files on windows properly

Fabian Reinartz committed
699
## 2.2.0 / 2018-03-08
700 701

* [CHANGE] Rename file SD mtime metric.
Fabian Reinartz committed
702 703
* [CHANGE] Send target update on empty pod IP in Kubernetes SD.
* [FEATURE] Add API endpoint for flags.
Fabian Reinartz committed
704 705 706
* [FEATURE] Add API endpoint for dropped targets.
* [FEATURE] Display annotations on alerts page.
* [FEATURE] Add option to skip head data when taking snapshots.
707 708 709 710 711
* [ENHANCEMENT] Federation performance improvement.
* [ENHANCEMENT] Read bearer token file on every scrape.
* [ENHANCEMENT] Improve typeahead on `/graph` page.
* [ENHANCEMENT] Change rule file formatting.
* [ENHANCEMENT] Set consul server default to `localhost:8500`.
Fabian Reinartz committed
712 713
* [ENHANCEMENT] Add dropped Alertmanagers to API info endpoint.
* [ENHANCEMENT] Add OS type meta label to Azure SD.
Fabian Reinartz committed
714
* [ENHANCEMENT] Validate required fields in SD configuration.
Fabian Reinartz committed
715 716 717
* [BUGFIX] Prevent stack overflow on deep recursion in TSDB.
* [BUGFIX] Correctly read offsets in index files that are greater than 4GB.
* [BUGFIX] Fix scraping behavior for empty labels.
718 719 720 721 722 723 724 725 726 727
* [BUGFIX] Drop metric name for bool modifier.
* [BUGFIX] Fix races in discovery.
* [BUGFIX] Fix Kubernetes endpoints SD for empty subsets.
* [BUGFIX] Throttle updates from SD providers, which caused increased CPU usage and allocations.
* [BUGFIX] Fix TSDB block reload issue.
* [BUGFIX] Fix PromQL printing of empty `without()`.
* [BUGFIX] Don't reset FiredAt for inactive alerts.
* [BUGFIX] Fix erroneous file version changes and repair existing data.

## 2.1.0 / 2018-01-19
728 729 730

* [FEATURE] New Service Discovery UI showing labels before and after relabelling.
* [FEATURE] New Admin APIs added to v1 to delete, snapshot and remove tombstones.
Josh Soref committed
731
* [ENHANCEMENT] The graph UI autocomplete now includes your previous queries.
732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747
* [ENHANCEMENT] Federation is now much faster for large numbers of series.
* [ENHANCEMENT] Added new metrics to measure rule timings.
* [ENHANCEMENT] Rule evaluation times added to the rules UI.
* [ENHANCEMENT] Added metrics to measure modified time of file SD files.
* [ENHANCEMENT] Kubernetes SD now includes POD UID in discovery metadata.
* [ENHANCEMENT] The Query APIs now return optional stats on query execution times.
* [ENHANCEMENT] The index now no longer has the 4GiB size limit and is also smaller.
* [BUGFIX] Remote read `read_recent` option is now false by default.
* [BUGFIX] Pass the right configuration to each Alertmanager (AM) when using multiple AM configs.
* [BUGFIX] Fix not-matchers not selecting series with labels unset.
* [BUGFIX] tsdb: Fix occasional panic in head block.
* [BUGFIX] tsdb: Close files before deletion to fix retention issues on Windows and NFS.
* [BUGFIX] tsdb: Cleanup and do not retry failing compactions.
* [BUGFIX] tsdb: Close WAL while shutting down.


Brian Brazil committed
748 749 750 751 752 753 754 755 756 757 758 759 760 761 762
## 2.0.0 / 2017-11-08

This release includes a completely rewritten storage, huge performance
improvements, but also many backwards incompatible changes. For more
information, read the announcement blog post and migration guide.

https://prometheus.io/blog/2017/11/08/announcing-prometheus-2-0/
https://prometheus.io/docs/prometheus/2.0/migration/

* [CHANGE] Completely rewritten storage layer, with WAL. This is not backwards compatible with 1.x storage, and many flags have changed/disappeared.
* [CHANGE] New staleness behavior. Series now marked stale after target scrapes no longer return them, and soon after targets disappear from service discovery.
* [CHANGE] Rules files use YAML syntax now. Conversion tool added to promtool.
* [CHANGE] Removed `count_scalar`, `drop_common_labels` functions and `keep_common` modifier from PromQL.
* [CHANGE] Rewritten exposition format parser with much higher performance. The Protobuf exposition format is no longer supported.
* [CHANGE] Example console templates updated for new storage and metrics names. Examples other than node exporter and Prometheus removed.
763
* [CHANGE] Admin and lifecycle APIs now disabled by default, can be re-enabled via flags
Brian Brazil committed
764 765 766 767 768 769 770 771 772
* [CHANGE] Flags switched to using Kingpin, all flags are now --flagname rather than -flagname.
* [FEATURE/CHANGE] Remote read can be configured to not read data which is available locally. This is enabled by default.
* [FEATURE] Rules can be grouped now. Rules within a rule group are executed sequentially.
* [FEATURE] Added experimental GRPC apis
* [FEATURE] Add timestamp() function to PromQL.
* [ENHANCEMENT] Remove remote read from the query path if no remote storage is configured.
* [ENHANCEMENT] Bump Consul HTTP client timeout to not match the Consul SD watch timeout.
* [ENHANCEMENT] Go-conntrack added to provide HTTP connection metrics.
* [BUGFIX] Fix connection leak in Consul SD.
Fabian Reinartz committed
773

beorn7 committed
774 775 776
## 1.8.2 / 2017-11-04

* [BUGFIX] EC2 service discovery: Do not crash if tags are empty.
Fabian Reinartz committed
777

Brian Brazil committed
778 779
## 1.8.1 / 2017-10-19

780
* [BUGFIX] Correctly handle external labels on remote read endpoint
781

beorn7 committed
782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811
## 1.8.0 / 2017-10-06

* [CHANGE] Rule links link to the _Console_ tab rather than the _Graph_ tab to
  not trigger expensive range queries by default.
* [FEATURE] Ability to act as a remote read endpoint for other Prometheus
  servers.
* [FEATURE] K8s SD: Support discovery of ingresses.
* [FEATURE] Consul SD: Support for node metadata.
* [FEATURE] Openstack SD: Support discovery of hypervisors.
* [FEATURE] Expose current Prometheus config via `/status/config`.
* [FEATURE] Allow to collapse jobs on `/targets` page.
* [FEATURE] Add `/-/healthy` and `/-/ready` endpoints.
* [FEATURE] Add color scheme support to console templates.
* [ENHANCEMENT] Remote storage connections use HTTP keep-alive.
* [ENHANCEMENT] Improved logging about remote storage.
* [ENHANCEMENT] Relaxed URL validation.
* [ENHANCEMENT] Openstack SD: Handle instances without IP.
* [ENHANCEMENT] Make remote storage queue manager configurable.
* [ENHANCEMENT] Validate metrics returned from remote read.
* [ENHANCEMENT] EC2 SD: Set a default region.
* [ENHANCEMENT] Changed help link to `https://prometheus.io/docs`.
* [BUGFIX] Fix floating-point precision issue in `deriv` function.
* [BUGFIX] Fix pprof endpoints when -web.route-prefix or -web.external-url is
  used.
* [BUGFIX] Fix handling of `null` target groups in file-based SD.
* [BUGFIX] Set the sample timestamp in date-related PromQL functions.
* [BUGFIX] Apply path prefix to redirect from deprecated graph URL.
* [BUGFIX] Fixed tests on MS Windows.
* [BUGFIX] Check for invalid UTF-8 in label values after relabeling.

beorn7 committed
812 813 814 815 816 817 818 819 820 821 822 823 824
## 1.7.2 / 2017-09-26

* [BUGFIX] Correctly remove all targets from DNS service discovery if the
  corresponding DNS query succeeds and returns an empty result.
* [BUGFIX] Correctly parse resolution input in expression browser.
* [BUGFIX] Consistently use UTC in the date picker of the expression browser.
* [BUGFIX] Correctly handle multiple ports in Marathon service discovery.
* [BUGFIX] Fix HTML escaping so that HTML templates compile with Go1.9.
* [BUGFIX] Prevent number of remote write shards from going negative.
* [BUGFIX] In the graphs created by the expression browser, render very large
  and small numbers in a readable way.
* [BUGFIX] Fix a rarely occurring iterator issue in varbit encoded chunks.

Frederic Branczyk committed
825 826 827 828
## 1.7.1 / 2017-06-12

* [BUGFIX] Fix double prefix redirect.

Frederic Branczyk committed
829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847
## 1.7.0 / 2017-06-06

* [CHANGE] Compress remote storage requests and responses with unframed/raw snappy.
* [CHANGE] Properly ellide secrets in config.
* [FEATURE] Add OpenStack service discovery.
* [FEATURE] Add ability to limit Kubernetes service discovery to certain namespaces.
* [FEATURE] Add metric for discovered number of Alertmanagers.
* [ENHANCEMENT] Print system information (uname) on start up.
* [ENHANCEMENT] Show gaps in graphs on expression browser.
* [ENHANCEMENT] Promtool linter checks counter naming and more reserved labels.
* [BUGFIX] Fix broken Mesos discovery.
* [BUGFIX] Fix redirect when external URL is set.
* [BUGFIX] Fix mutation of active alert elements by notifier.
* [BUGFIX] Fix HTTP error handling for remote write.
* [BUGFIX] Fix builds for Solaris/Illumos.
* [BUGFIX] Fix overflow checking in global config.
* [BUGFIX] Fix log level reporting issue.
* [BUGFIX] Fix ZooKeeper serverset discovery can become out-of-sync.

848
## 1.6.3 / 2017-05-18
Frederic Branczyk committed
849

Josh Soref committed
850
* [BUGFIX] Fix disappearing Alertmanager targets in Alertmanager discovery.
Frederic Branczyk committed
851 852 853
* [BUGFIX] Fix panic with remote_write on ARMv7.
* [BUGFIX] Fix stacked graphs to adapt min/max values.

854
## 1.6.2 / 2017-05-11
Fabian Reinartz committed
855 856 857

* [BUGFIX] Fix potential memory leak in Kubernetes service discovery

858
## 1.6.1 / 2017-04-19
Brian Brazil committed
859 860 861

* [BUGFIX] Don't panic if storage has no FPs even after initial wait

862
## 1.6.0 / 2017-04-14
beorn7 committed
863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926

* [CHANGE] Replaced the remote write implementations for various backends by a
  generic write interface with example adapter implementation for various
  backends. Note that both the previous and the current remote write
  implementations are **experimental**.
* [FEATURE] New flag `-storage.local.target-heap-size` to tell Prometheus about
  the desired heap size. This deprecates the flags
  `-storage.local.memory-chunks` and `-storage.local.max-chunks-to-persist`,
  which are kept for backward compatibility.
* [FEATURE] Add `check-metrics` to `promtool` to lint metric names.
* [FEATURE] Add Joyent Triton discovery.
* [FEATURE] `X-Prometheus-Scrape-Timeout-Seconds` header in HTTP scrape
  requests.
* [FEATURE] Remote read interface, including example for InfluxDB. **Experimental.**
* [FEATURE] Enable Consul SD to connect via TLS.
* [FEATURE] Marathon SD supports multiple ports.
* [FEATURE] Marathon SD supports bearer token for authentication.
* [FEATURE] Custom timeout for queries.
* [FEATURE] Expose `buildQueryUrl` in `graph.js`.
* [FEATURE] Add `rickshawGraph` property to the graph object in console
  templates.
* [FEATURE] New metrics exported by Prometheus itself:
  * Summary `prometheus_engine_query_duration_seconds`
  * Counter `prometheus_evaluator_iterations_missed_total`
  * Counter `prometheus_evaluator_iterations_total`
  * Gauge `prometheus_local_storage_open_head_chunks`
  * Gauge `prometheus_local_storage_target_heap_size`
* [ENHANCEMENT] Reduce shut-down time by interrupting an ongoing checkpoint
  before starting the final checkpoint.
* [ENHANCEMENT] Auto-tweak times between checkpoints to limit time spent in
  checkpointing to 50%.
* [ENHANCEMENT] Improved crash recovery deals better with certain index
  corruptions.
* [ENHANCEMENT] Graphing deals better with constant time series.
* [ENHANCEMENT] Retry remote writes on recoverable errors.
* [ENHANCEMENT] Evict unused chunk descriptors during crash recovery to limit
  memory usage.
* [ENHANCEMENT] Smoother disk usage during series maintenance.
* [ENHANCEMENT] Targets on targets page sorted by instance within a job.
* [ENHANCEMENT] Sort labels in federation.
* [ENHANCEMENT] Set `GOGC=40` by default, which results in much better memory
  utilization at the price of slightly higher CPU usage. If `GOGC` is set by
  the user, it is still honored as usual.
* [ENHANCEMENT] Close head chunks after being idle for the duration of the
  configured staleness delta. This helps to persist and evict head chunk of
  stale series more quickly.
* [ENHANCEMENT] Stricter checking of relabel config.
* [ENHANCEMENT] Cache busters for static web content.
* [ENHANCEMENT] Send Prometheus-specific user-agent header during scrapes.
* [ENHANCEMENT] Improved performance of series retention cut-off.
* [ENHANCEMENT] Mitigate impact of non-atomic sample ingestion on
  `histogram_quantile` by enforcing buckets to be monotonic.
* [ENHANCEMENT] Released binaries built with Go 1.8.1.
* [BUGFIX] Send `instance=""` with federation if `instance` not set.
* [BUGFIX] Update to new `client_golang` to get rid of unwanted quantile
  metrics in summaries.
* [BUGFIX] Introduce several additional guards against data corruption.
* [BUGFIX] Mark storage dirty and increment
  `prometheus_local_storage_persist_errors_total` on all relevant errors.
* [BUGFIX] Propagate storage errors as 500 in the HTTP API.
* [BUGFIX] Fix int64 overflow in timestamps in the HTTP API.
* [BUGFIX] Fix deadlock in Zookeeper SD.
* [BUGFIX] Fix fuzzy search problems in the web-UI auto-completion.

927
## 1.5.3 / 2017-05-11
Fabian Reinartz committed
928 929 930

* [BUGFIX] Fix potential memory leak in Kubernetes service discovery

931
## 1.5.2 / 2017-02-10
beorn7 committed
932 933 934 935 936 937 938

* [BUGFIX] Fix series corruption in a special case of series maintenance where
  the minimum series-file-shrink-ratio kicks in.
* [BUGFIX] Fix two panic conditions both related to processing a series
  scheduled to be quarantined.
* [ENHANCEMENT] Binaries built with Go1.7.5.

939
## 1.5.1 / 2017-02-07
beorn7 committed
940 941 942 943 944 945

* [BUGFIX] Don't lose fully persisted memory series during checkpointing.
* [BUGFIX] Fix intermittently failing relabeling.
* [BUGFIX] Make `-storage.local.series-file-shrink-ratio` work.
* [BUGFIX] Remove race condition from TestLoop.

946
## 1.5.0 / 2017-01-23
Frederic Branczyk committed
947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962

* [CHANGE] Use lexicographic order to sort alerts by name.
* [FEATURE] Add Joyent Triton discovery.
* [FEATURE] Add scrape targets and alertmanager targets API.
* [FEATURE] Add various persistence related metrics.
* [FEATURE] Add various query engine related metrics.
* [FEATURE] Add ability to limit scrape samples, and related metrics.
* [FEATURE] Add labeldrop and labelkeep relabelling actions.
* [FEATURE] Display current working directory on status-page.
* [ENHANCEMENT] Strictly use ServiceAccount for in cluster configuration on Kubernetes.
* [ENHANCEMENT] Various performance and memory-management improvements.
* [BUGFIX] Fix basic auth for alertmanagers configured via flag.
* [BUGFIX] Don't panic on decoding corrupt data.
* [BUGFIX] Ignore dotfiles in data directory.
* [BUGFIX] Abort on intermediate federation errors.

963
## 1.4.1 / 2016-11-28
Fabian Reinartz committed
964 965 966

* [BUGFIX] Fix Consul service discovery

967
## 1.4.0 / 2016-11-25
Fabian Reinartz committed
968 969 970 971 972 973 974 975 976 977 978 979

* [FEATURE] Allow configuring Alertmanagers via service discovery
* [FEATURE] Display used Alertmanagers on runtime page in the UI
* [FEATURE] Support profiles in AWS EC2 service discovery configuration
* [ENHANCEMENT] Remove duplicated logging of Kubernetes client errors
* [ENHANCEMENT] Add metrics about Kubernetes service discovery
* [BUGFIX] Update alert annotations on re-evaluation
* [BUGFIX] Fix export of group modifier in PromQL queries
* [BUGFIX] Remove potential deadlocks in several service discovery implementations
* [BUGFIX] Use proper float64 modulo in PromQL `%` binary operations
* [BUGFIX] Fix crash bug in Kubernetes service discovery

980
## 1.3.1 / 2016-11-04
beorn7 committed
981

982
This bug-fix release pulls in the fixes from the 1.2.3 release.
beorn7 committed
983 984 985 986 987 988

* [BUGFIX] Correctly handle empty Regex entry in relabel config.
* [BUGFIX] MOD (`%`) operator doesn't panic with small floating point numbers.
* [BUGFIX] Updated miekg/dns vendoring to pick up upstream bug fixes.
* [ENHANCEMENT] Improved DNS error reporting.

989 990 991 992 993 994 995 996 997 998 999 1000
## 1.2.3 / 2016-11-04

Note that this release is chronologically after 1.3.0.

* [BUGFIX] Correctly handle end time before start time in range queries.
* [BUGFIX] Error on negative `-storage.staleness-delta`
* [BUGFIX] Correctly handle empty Regex entry in relabel config.
* [BUGFIX] MOD (`%`) operator doesn't panic with small floating point numbers.
* [BUGFIX] Updated miekg/dns vendoring to pick up upstream bug fixes.
* [ENHANCEMENT] Improved DNS error reporting.

## 1.3.0 / 2016-11-01
Frederic Branczyk committed
1001 1002 1003 1004 1005 1006 1007 1008 1009 1010

This is a breaking change to the Kubernetes service discovery.

* [CHANGE] Rework Kubernetes SD.
* [FEATURE] Add support for interpolating `target_label`.
* [FEATURE] Add GCE metadata as Prometheus meta labels.
* [ENHANCEMENT] Add EC2 SD metrics.
* [ENHANCEMENT] Add Azure SD metrics.
* [ENHANCEMENT] Add fuzzy search to `/graph` textarea.
* [ENHANCEMENT] Always show instance labels on target page.
Fabian Reinartz committed
1011 1012
* [BUGFIX] Validate query end time is not before start time.
* [BUGFIX] Error on negative `-storage.staleness-delta`
beorn7 committed
1013

1014
## 1.2.2 / 2016-10-30
beorn7 committed
1015

Frederic Branczyk committed
1016
* [BUGFIX] Correctly handle on() in alerts.
beorn7 committed
1017 1018 1019 1020 1021
* [BUGFIX] UI: Deal properly with aborted requests.
* [BUGFIX] UI: Decode URL query parameters properly.
* [BUGFIX] Storage: Deal better with data corruption (non-monotonic timestamps).
* [BUGFIX] Remote storage: Re-add accidentally removed timeout flag.
* [BUGFIX] Updated a number of vendored packages to pick up upstream bug fixes.
Frederic Branczyk committed
1022

1023
## 1.2.1 / 2016-10-10
beorn7 committed
1024 1025

* [BUGFIX] Count chunk evictions properly so that the server doesn't
Josh Soref committed
1026
  assume it runs out of memory and subsequently throttles ingestion.
beorn7 committed
1027 1028
* [BUGFIX] Use Go1.7.1 for prebuilt binaries to fix issues on MacOS Sierra.

1029
## 1.2.0 / 2016-10-07
beorn7 committed
1030

1031
* [FEATURE] Cleaner encoding of query parameters in `/graph` URLs.
beorn7 committed
1032 1033
* [FEATURE] PromQL: Add `minute()` function.
* [FEATURE] Add GCE service discovery.
1034
* [FEATURE] Allow any valid UTF-8 string as job name.
beorn7 committed
1035 1036
* [FEATURE] Allow disabling local storage.
* [FEATURE] EC2 service discovery: Expose `ec2_instance_state`.
1037
* [ENHANCEMENT] Various performance improvements in local storage.
beorn7 committed
1038 1039 1040
* [BUGFIX] Zookeeper service discovery: Remove deleted nodes.
* [BUGFIX] Zookeeper service discovery: Resync state after Zookeeper failure.
* [BUGFIX] Remove JSON from HTTP Accept header.
1041 1042
* [BUGFIX] Fix flag validation of Alertmanager URL.
* [BUGFIX] Fix race condition on shutdown.
beorn7 committed
1043 1044 1045
* [BUGFIX] Do not fail Consul discovery on Prometheus startup when Consul
  is down.
* [BUGFIX] Handle NaN in `changes()` correctly.
1046 1047 1048 1049 1050
* [CHANGE] **Experimental** remote write path: Remove use of gRPC.
* [CHANGE] **Experimental** remote write path: Configuration via config file
  rather than command line flags.
* [FEATURE] **Experimental** remote write path: Add HTTP basic auth and TLS.
* [FEATURE] **Experimental** remote write path: Support for relabelling.
beorn7 committed
1051

1052
## 1.1.3 / 2016-09-16
beorn7 committed
1053 1054 1055 1056 1057

* [ENHANCEMENT] Use golang-builder base image for tests in CircleCI.
* [ENHANCEMENT] Added unit tests for federation.
* [BUGFIX] Correctly de-dup metric families in federation output.

1058
## 1.1.2 / 2016-09-08
beorn7 committed
1059 1060 1061

* [BUGFIX] Allow label names that coincide with keywords.

1062
## 1.1.1 / 2016-09-07
Fabian Reinartz committed
1063 1064 1065 1066

* [BUGFIX] Fix IPv6 escaping in service discovery integrations
* [BUGFIX] Fix default scrape port assignment for IPv6

1067
## 1.1.0 / 2016-09-03
beorn7 committed
1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087

* [FEATURE] Add `quantile` and `quantile_over_time`.
* [FEATURE] Add `stddev_over_time` and `stdvar_over_time`.
* [FEATURE] Add various time and date functions.
* [FEATURE] Added `toUpper` and `toLower` formatting to templates.
* [FEATURE] Allow relabeling of alerts.
* [FEATURE] Allow URLs in targets defined via a JSON file.
* [FEATURE] Add idelta function.
* [FEATURE] 'Remove graph' button on the /graph page.
* [FEATURE] Kubernetes SD: Add node name and host IP to pod discovery.
* [FEATURE] New remote storage write path. EXPERIMENTAL!
* [ENHANCEMENT] Improve time-series index lookups.
* [ENHANCEMENT] Forbid invalid relabel configurations.
* [ENHANCEMENT] Improved various tests.
* [ENHANCEMENT] Add crash recovery metric 'started_dirty'.
* [ENHANCEMENT] Fix (and simplify) populating series iterators.
* [ENHANCEMENT] Add job link on target page.
* [ENHANCEMENT] Message on empty Alerts page.
* [ENHANCEMENT] Various internal code refactorings and clean-ups.
* [ENHANCEMENT] Various improvements in the build system.
Ben Kochie committed
1088
* [BUGFIX] Catch errors when unmarshaling delta/doubleDelta encoded chunks.
beorn7 committed
1089 1090 1091 1092 1093 1094 1095 1096
* [BUGFIX] Fix data race in lexer and lexer test.
* [BUGFIX] Trim stray whitespace from bearer token file.
* [BUGFIX] Avoid divide-by-zero panic on query_range?step=0.
* [BUGFIX] Detect invalid rule files at startup.
* [BUGFIX] Fix counter reset treatment in PromQL.
* [BUGFIX] Fix rule HTML escaping issues.
* [BUGFIX] Remove internal labels from alerts sent to AM.

1097
## 1.0.2 / 2016-08-24
beorn7 committed
1098 1099 1100

* [BUGFIX] Clean up old targets after config reload.

1101
## 1.0.1 / 2016-07-21
beorn7 committed
1102 1103 1104 1105 1106

* [BUGFIX] Exit with error on non-flag command-line arguments.
* [BUGFIX] Update example console templates to new HTTP API.
* [BUGFIX] Re-add logging flags.

1107
## 1.0.0 / 2016-07-18
1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125

* [CHANGE] Remove deprecated query language keywords
* [CHANGE] Change Kubernetes SD to require specifying Kubernetes role
* [CHANGE] Use service address in Consul SD if available
* [CHANGE] Standardize all Prometheus internal metrics to second units
* [CHANGE] Remove unversioned legacy HTTP API
* [CHANGE] Remove legacy ingestion of JSON metric format
* [CHANGE] Remove deprecated `target_groups` configuration
* [FEATURE] Add binary power operation to PromQL
* [FEATURE] Add `count_values` aggregator
* [FEATURE] Add `-web.route-prefix` flag
* [FEATURE] Allow `on()`, `by()`, `without()` in PromQL with empty label sets
* [ENHANCEMENT] Make `topk/bottomk` query functions aggregators
* [BUGFIX] Fix annotations in alert rule printing
* [BUGFIX] Expand alert templating at evaluation time
* [BUGFIX] Fix edge case handling in crash recovery
* [BUGFIX] Hide testing package flags from help output

1126
## 0.20.0 / 2016-06-15
Fabian Reinartz committed
1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143

This release contains multiple breaking changes to the configuration schema.

* [FEATURE] Allow configuring multiple Alertmanagers
* [FEATURE] Add server name to TLS configuration
* [FEATURE] Add labels for all node addresses and discover node port if available in Kubernetes SD
* [ENHANCEMENT] More meaningful configuration errors
* [ENHANCEMENT] Round scraping timestamps to milliseconds in web UI
* [ENHANCEMENT] Make number of storage fingerprint locks configurable
* [BUGFIX] Fix date parsing in console template graphs
* [BUGFIX] Fix static console files in Docker images
* [BUGFIX] Fix console JS XHR requests for IE11
* [BUGFIX] Add missing path prefix in new status page
* [CHANGE] Rename `target_groups` to `static_configs` in config files
* [CHANGE] Rename `names` to `files` in file SD configuration
* [CHANGE] Remove kubelet port config option in Kubernetes SD configuration

1144
## 0.19.3 / 2016-06-14
1145 1146 1147 1148

* [BUGFIX] Handle Marathon apps with zero ports
* [BUGFIX] Fix startup panic in retrieval layer

1149
## 0.19.2 / 2016-05-29
beorn7 committed
1150 1151 1152 1153 1154

* [BUGFIX] Correctly handle `GROUP_LEFT` and `GROUP_RIGHT` without labels in
  string representation of expressions and in rules.
* [BUGFIX] Use `-web.external-url` for new status endpoints.

1155
## 0.19.1 / 2016-05-25
1156 1157 1158 1159

* [BUGFIX] Handle service discovery panic affecting Kubernetes SD
* [BUGFIX] Fix web UI display issue in some browsers

1160
## 0.19.0 / 2016-05-24
1161 1162 1163 1164

This version contains a breaking change to the query language. Please read
the documentation on the grouping behavior of vector matching:

Fabian Reinartz committed
1165
https://prometheus.io/docs/querying/operators/#vector-matching
1166 1167 1168 1169 1170 1171

* [FEATURE] Add experimental Microsoft Azure service discovery
* [FEATURE] Add `ignoring` modifier for binary operations
* [FEATURE] Add pod discovery to Kubernetes service discovery
* [CHANGE] Vector matching takes grouping labels from one-side
* [ENHANCEMENT] Support time range on /api/v1/series endpoint
Fabian Reinartz committed
1172
* [ENHANCEMENT] Partition status page into individual pages
1173 1174
* [BUGFIX] Fix issue of hanging target scrapes

1175
## 0.18.0 / 2016-04-18
1176 1177

* [BUGFIX] Fix operator precedence in PromQL
1178 1179
* [BUGFIX] Never drop still open head chunk
* [BUGFIX] Fix missing 'keep_common' when printing AST node
1180 1181
* [CHANGE/BUGFIX] Target identity considers path and parameters additionally to host and port
* [CHANGE] Rename metric `prometheus_local_storage_invalid_preload_requests_total` to `prometheus_local_storage_non_existent_series_matches_total`
1182
* [CHANGE] Support for old alerting rule syntax dropped
1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194
* [FEATURE] Deduplicate targets within the same scrape job
* [FEATURE] Add varbit chunk encoding (higher compression, more CPU usage – disabled by default)
* [FEATURE] Add `holt_winters` query function
* [FEATURE] Add relative complement `unless` operator to PromQL
* [ENHANCEMENT] Quarantine series file if data corruption is encountered (instead of crashing)
* [ENHANCEMENT] Validate Alertmanager URL
* [ENHANCEMENT] Use UTC for build timestamp
* [ENHANCEMENT] Improve index query performance (especially for active time series)
* [ENHANCEMENT] Instrument configuration reload duration
* [ENHANCEMENT] Instrument retrieval layer
* [ENHANCEMENT] Add Go version to `prometheus_build_info` metric

1195
## 0.17.0 / 2016-03-02
1196

1197
This version no longer works with Alertmanager 0.0.4 and earlier!
1198
The alerting rule syntax has changed as well but the old syntax is supported
1199
up until version 0.18.
1200

1201
All regular expressions in PromQL are anchored now, matching the behavior of
1202
regular expressions in config files.
1203

1204
* [CHANGE] Integrate with Alertmanager 0.1.0 and higher
1205 1206 1207 1208
* [CHANGE] Degraded storage mode renamed to rushed mode
* [CHANGE] New alerting rule syntax
* [CHANGE] Add label validation on ingestion
* [CHANGE] Regular expression matchers in PromQL are anchored
1209
* [FEATURE] Add `without` aggregation modifier
1210 1211 1212 1213 1214 1215 1216 1217
* [FEATURE] Send alert resolved notifications to Alertmanager
* [FEATURE] Allow millisecond precision in configuration file
* [FEATURE] Support AirBnB's Smartstack Nerve for service discovery
* [ENHANCEMENT] Storage switches less often between regular and rushed mode.
* [ENHANCEMENT] Storage switches into rushed mode if there are too many memory chunks.
* [ENHANCEMENT] Added more storage instrumentation
* [ENHANCEMENT] Improved instrumentation of notification handler
* [BUGFIX] Do not count head chunks as chunks waiting for persistence
Fabian Reinartz committed
1218
* [BUGFIX] Handle OPTIONS HTTP requests to the API correctly
1219
* [BUGFIX] Parsing of ranges in PromQL fixed
Fabian Reinartz committed
1220
* [BUGFIX] Correctly validate URL flag parameters
1221 1222 1223
* [BUGFIX] Log argument parse errors
* [BUGFIX] Properly handle creation of target with bad TLS config
* [BUGFIX] Fix of checkpoint timing issue
1224

1225
## 0.16.2 / 2016-01-18
1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255

* [FEATURE] Multiple authentication options for EC2 discovery added
* [FEATURE] Several meta labels for EC2 discovery added
* [FEATURE] Allow full URLs in static target groups (used e.g. by the `blackbox_exporter`)
* [FEATURE] Add Graphite remote-storage integration
* [FEATURE] Create separate Kubernetes targets for services and their endpoints
* [FEATURE] Add `clamp_{min,max}` functions to PromQL
* [FEATURE] Omitted time parameter in API query defaults to now
* [ENHANCEMENT] Less frequent time series file truncation
* [ENHANCEMENT] Instrument number of  manually deleted time series
* [ENHANCEMENT] Ignore lost+found directory during storage version detection
* [CHANGE] Kubernetes `masters` renamed to `api_servers`
* [CHANGE] "Healthy" and "unhealthy" targets are now called "up" and "down" in the web UI
* [CHANGE] Remove undocumented 2nd argument of the `delta` function.
  (This is a BREAKING CHANGE for users of the undocumented 2nd argument.)
* [BUGFIX] Return proper HTTP status codes on API errors
* [BUGFIX] Fix Kubernetes authentication configuration
* [BUGFIX] Fix stripped OFFSET from in rule evaluation and display
* [BUGFIX] Do not crash on failing Consul SD initialization
* [BUGFIX] Revert changes to metric auto-completion
* [BUGFIX] Add config overflow validation for TLS configuration
* [BUGFIX] Skip already watched Zookeeper nodes in serverset SD
* [BUGFIX] Don't federate stale samples
* [BUGFIX] Move NaN to end of result for `topk/bottomk/sort/sort_desc/min/max`
* [BUGFIX] Limit extrapolation of `delta/rate/increase`
* [BUGFIX] Fix unhandled error in rule evaluation

Some changes to the Kubernetes service discovery were integration since
it was released as a beta feature.

1256
## 0.16.1 / 2015-10-16
1257 1258 1259 1260 1261 1262

* [FEATURE] Add `irate()` function.
* [ENHANCEMENT] Improved auto-completion in expression browser.
* [CHANGE] Kubernetes SD moves node label to instance label.
* [BUGFIX] Escape regexes in console templates.

1263
## 0.16.0 / 2015-10-09
Julius Volz committed
1264 1265 1266 1267 1268 1269 1270

BREAKING CHANGES:

* Release tarballs now contain the built binaries in a nested directory.
* The `hash_mod` relabeling action now uses MD5 hashes instead of FNV hashes to
  achieve a better distribution.
* The DNS-SD meta label `__meta_dns_srv_name` was renamed to `__meta_dns_name`
Julius Volz committed
1271
  to reflect support for DNS record types other than `SRV`.
Julius Volz committed
1272 1273 1274
* The default full refresh interval for the file-based service discovery has been
  increased from 30 seconds to 5 minutes.
* In relabeling, parts of a source label that weren't matched by
Julius Volz committed
1275 1276
  the specified regular expression are no longer included in the replacement
  output.
Julius Volz committed
1277 1278 1279 1280 1281 1282
* Queries no longer interpolate between two data points. Instead, the resulting
  value will always be the latest value before the evaluation query timestamp.
* Regular expressions supplied via the configuration are now anchored to match
  full strings instead of substrings.
* Global labels are not appended upon storing time series anymore. Instead,
  they are only appended when communicating with external systems
Julius Volz committed
1283 1284
  (Alertmanager, remote storages, federation). They have thus also been renamed
  from `global.labels` to `global.external_labels`.
Julius Volz committed
1285 1286 1287 1288
* The names and units of metrics related to remote storage sample appends have
  been changed.
* The experimental support for writing to InfluxDB has been updated to work
  with InfluxDB 0.9.x. 0.8.x versions of InfluxDB are not supported anymore.
Julius Volz committed
1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301
* Escape sequences in double- and single-quoted string literals in rules or query
  expressions are now interpreted like escape sequences in Go string literals
  (https://golang.org/ref/spec#String_literals).

Future breaking changes / deprecated features:

* The `delta()` function had an undocumented optional second boolean argument
  to make it behave like `increase()`. This second argument will be removed in
  the future. Migrate any occurrences of `delta(x, 1)` to use `increase(x)`
  instead.
* Support for filter operators between two scalar values (like `2 > 1`) will be
  removed in the future. These will require a `bool` modifier on the operator,
  e.g.  `2 > bool 1`.
Julius Volz committed
1302 1303 1304

All changes:

Julius Volz committed
1305
* [CHANGE] Renamed `global.labels` to `global.external_labels`.
Julius Volz committed
1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323
* [CHANGE] Vendoring is now done via govendor instead of godep.
* [CHANGE] Change web UI root page to show the graphing interface instead of
  the server status page.
* [CHANGE] Append global labels only when communicating with external systems
  instead of storing them locally.
* [CHANGE] Change all regexes in the configuration to do full-string matches
  instead of substring matches.
* [CHANGE] Remove interpolation of vector values in queries.
* [CHANGE] For alert `SUMMARY`/`DESCRIPTION` template fields, cast the alert
  value to `float64` to work with common templating functions.
* [CHANGE] In relabeling, don't include unmatched source label parts in the
  replacement.
* [CHANGE] Change default full refresh interval for the file-based service
  discovery from 30 seconds to 5 minutes.
* [CHANGE] Rename the DNS-SD meta label `__meta_dns_srv_name` to
  `__meta_dns_name` to reflect support for other record types than `SRV`.
* [CHANGE] Release tarballs now contain the binaries in a nested directory.
* [CHANGE] Update InfluxDB write support to work with InfluxDB 0.9.x.
Julius Volz committed
1324 1325 1326
* [FEATURE] Support full "Go-style" escape sequences in strings and add raw
  string literals.
* [FEATURE] Add EC2 service discovery support.
Julius Volz committed
1327 1328 1329 1330 1331 1332 1333 1334 1335 1336
* [FEATURE] Allow configuring TLS options in scrape configurations.
* [FEATURE] Add instrumentation around configuration reloads.
* [FEATURE] Add `bool` modifier to comparison operators to enable boolean
  (`0`/`1`) output instead of filtering.
* [FEATURE] In Zookeeper serverset discovery, provide `__meta_serverset_shard`
  label with the serverset shard number.
* [FEATURE] Provide `__meta_consul_service_id` meta label in Consul service
  discovery.
* [FEATURE] Allow scalar expressions in recording rules to enable use cases
  such as building constant metrics.
1337
* [FEATURE] Add `label_replace()` and `vector()` query language functions.
Julius Volz committed
1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363
* [FEATURE] In Consul service discovery, fill in the `__meta_consul_dc`
  datacenter label from the Consul agent when it's not set in the Consul SD
  config.
* [FEATURE] Scrape all services upon empty services list in Consul service
  discovery.
* [FEATURE] Add `labelmap` relabeling action to map a set of input labels to a
  set of output labels using regular expressions.
* [FEATURE] Introduce `__tmp` as a relabeling label prefix that is guaranteed
  to not be used by Prometheus internally.
* [FEATURE] Kubernetes-based service discovery.
* [FEATURE] Marathon-based service discovery.
* [FEATURE] Support multiple series names in console graphs JavaScript library.
* [FEATURE] Allow reloading configuration via web handler at `/-/reload`.
* [FEATURE] Updates to promtool to reflect new Prometheus configuration
  features.
* [FEATURE] Add `proxy_url` parameter to scrape configurations to enable use of
  proxy servers.
* [FEATURE] Add console templates for Prometheus itself.
* [FEATURE] Allow relabeling the protocol scheme of targets.
* [FEATURE] Add `predict_linear()` query language function.
* [FEATURE] Support for authentication using bearer tokens, client certs, and
  CA certs.
* [FEATURE] Implement unary expressions for vector types (`-foo`, `+foo`).
* [FEATURE] Add console templates for the SNMP exporter.
* [FEATURE] Make it possible to relabel target scrape query parameters.
* [FEATURE] Add support for `A` and `AAAA` records in DNS service discovery.
Julius Volz committed
1364 1365 1366
* [ENHANCEMENT] Fix several flaky tests.
* [ENHANCEMENT] Switch to common routing package.
* [ENHANCEMENT] Use more resilient metric decoder.
Julius Volz committed
1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381
* [ENHANCEMENT] Update vendored dependencies.
* [ENHANCEMENT] Add compression to more HTTP handlers.
* [ENHANCEMENT] Make -web.external-url flag help string more verbose.
* [ENHANCEMENT] Improve metrics around remote storage queues.
* [ENHANCEMENT] Use Go 1.5.1 instead of Go 1.4.2 in builds.
* [ENHANCEMENT] Update the architecture diagram in the `README.md`.
* [ENHANCEMENT] Time out sample appends in retrieval layer if the storage is
  backlogging.
* [ENHANCEMENT] Make `hash_mod` relabeling action use MD5 instead of FNV to
  enable better hash distribution.
* [ENHANCEMENT] Better tracking of targets between same service discovery
  mechanisms in one scrape configuration.
* [ENHANCEMENT] Handle parser and query evaluation runtime panics more
  gracefully.
* [ENHANCEMENT] Add IDs to H2 tags on status page to allow anchored linking.
Julius Volz committed
1382 1383 1384 1385 1386 1387 1388
* [BUGFIX] Fix watching multiple paths with Zookeeper serverset discovery.
* [BUGFIX] Fix high CPU usage on configuration reload.
* [BUGFIX] Fix disappearing `__params` on configuration reload.
* [BUGFIX] Make `labelmap` action available through configuration.
* [BUGFIX] Fix direct access of protobuf fields.
* [BUGFIX] Fix panic on Consul request error.
* [BUGFIX] Redirect of graph endpoint for prefixed setups.
Julius Volz committed
1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402
* [BUGFIX] Fix series file deletion behavior when purging archived series.
* [BUGFIX] Fix error checking and logging around checkpointing.
* [BUGFIX] Fix map initialization in target manager.
* [BUGFIX] Fix draining of file watcher events in file-based service discovery.
* [BUGFIX] Add `POST` handler for `/debug` endpoints to fix CPU profiling.
* [BUGFIX] Fix several flaky tests.
* [BUGFIX] Fix busylooping in case a scrape configuration has no target
  providers defined.
* [BUGFIX] Fix exit behavior of static target provider.
* [BUGFIX] Fix configuration reloading loop upon shutdown.
* [BUGFIX] Add missing check for nil expression in expression parser.
* [BUGFIX] Fix error handling bug in test code.
* [BUGFIX] Fix Consul port meta label.
* [BUGFIX] Fix lexer bug that treated non-Latin Unicode digits as digits.
Julius Volz committed
1403 1404 1405
* [CLEANUP] Remove obsolete federation example from console templates.
* [CLEANUP] Remove duplicated Bootstrap JS inclusion on graph page.
* [CLEANUP] Switch to common log package.
Julius Volz committed
1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425
* [CLEANUP] Update build environment scripts and Makefiles to work better with
  native Go build mechanisms and new Go 1.5 experimental vendoring support.
* [CLEANUP] Remove logged notice about 0.14.x configuration file format change.
* [CLEANUP] Move scrape-time metric label modification into SampleAppenders.
* [CLEANUP] Switch from `github.com/client_golang/model` to
  `github.com/common/model` and related type cleanups.
* [CLEANUP] Switch from `github.com/client_golang/extraction` to
  `github.com/common/expfmt` and related type cleanups.
* [CLEANUP] Exit Prometheus when the web server encounters a startup error.
* [CLEANUP] Remove non-functional alert-silencing links on alerting page.
* [CLEANUP] General cleanups to comments and code, derived from `golint`,
  `go vet`, or otherwise.
* [CLEANUP] When entering crash recovery, tell users how to cleanly shut down
  Prometheus.
* [CLEANUP] Remove internal support for multi-statement queries in query engine.
* [CLEANUP] Update AUTHORS.md.
* [CLEANUP] Don't warn/increment metric upon encountering equal timestamps for
  the same series upon append.
* [CLEANUP] Resolve relative paths during configuration loading.

1426
## 0.15.1 / 2015-07-27
1427 1428 1429 1430 1431
* [BUGFIX] Fix vector matching behavior when there is a mix of equality and
  non-equality matchers in a vector selector and one matcher matches no series.
* [ENHANCEMENT] Allow overriding `GOARCH` and `GOOS` in Makefile.INCLUDE.
* [ENHANCEMENT] Update vendored dependencies.

1432
## 0.15.0 / 2015-07-21
beorn7 committed
1433 1434 1435

BREAKING CHANGES:

1436 1437 1438
* Relative paths for rule files are now evaluated relative to the config file.
* External reachability flags (`-web.*`) consolidated.
* The default storage directory has been changed from `/tmp/metrics`
beorn7 committed
1439
  to `data` in the local directory.
1440
* The `rule_checker` tool has been replaced by `promtool` with
beorn7 committed
1441
  different flags and more functionality.
1442
* Empty labels are now removed upon ingestion into the
beorn7 committed
1443 1444 1445
  storage. Matching empty labels is now equivalent to matching unset
  labels (`mymetric{label=""}` now matches series that don't have
  `label` set at all).
1446
* The special `__meta_consul_tags` label in Consul service discovery
beorn7 committed
1447 1448
  now starts and ends with tag separators to enable easier regex
  matching.
1449
* The default scrape interval has been changed back from 1 minute to
beorn7 committed
1450
  10 seconds.
beorn7 committed
1451

Julius Volz committed
1452
All changes:
beorn7 committed
1453

1454
* [CHANGE] Change default storage directory to `data` in the current
beorn7 committed
1455
  working directory.
1456 1457
* [CHANGE] Consolidate external reachability flags (`-web.*`)into one.
* [CHANGE] Deprecate `keeping_extra` modifier keyword, rename it to
beorn7 committed
1458
  `keep_common`.
1459
* [CHANGE] Improve label matching performance and treat unset labels
beorn7 committed
1460
  like empty labels in label matchers.
1461
* [CHANGE] Remove `rule_checker` tool and add generic `promtool` CLI
beorn7 committed
1462
  tool which allows checking rules and configuration files.
1463 1464 1465 1466 1467
* [CHANGE] Resolve rule files relative to config file.
* [CHANGE] Restore default ScrapeInterval of 1 minute instead of 10 seconds.
* [CHANGE] Surround `__meta_consul_tags` value with tag separators.
* [CHANGE] Update node disk console for new filesystem labels.
* [FEATURE] Add Consul's `ServiceAddress`, `Address`, and `ServicePort` as
beorn7 committed
1468
  meta labels to enable setting a custom scrape address if needed.
1469
* [FEATURE] Add `hashmod` relabel action to allow for horizontal
beorn7 committed
1470
  sharding of Prometheus servers.
1471
* [FEATURE] Add `honor_labels` scrape configuration option to not
beorn7 committed
1472
  overwrite any labels exposed by the target.
1473 1474 1475 1476 1477
* [FEATURE] Add basic federation support on `/federate`.
* [FEATURE] Add optional `RUNBOOK` field to alert statements.
* [FEATURE] Add pre-relabel target labels to status page.
* [FEATURE] Add version information endpoint under `/version`.
* [FEATURE] Added initial stable API version 1 under `/api/v1`,
beorn7 committed
1478
  including ability to delete series and query more metadata.
1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501
* [FEATURE] Allow configuring query parameters when scraping metrics endpoints.
* [FEATURE] Allow deleting time series via the new v1 API.
* [FEATURE] Allow individual ingested metrics to be relabeled.
* [FEATURE] Allow loading rule files from an entire directory.
* [FEATURE] Allow scalar expressions in range queries, improve error messages.
* [FEATURE] Support Zookeeper Serversets as a service discovery mechanism.
* [ENHANCEMENT] Add circleci yaml for Dockerfile test build.
* [ENHANCEMENT] Always show selected graph range, regardless of available data.
* [ENHANCEMENT] Change expression input field to multi-line textarea.
* [ENHANCEMENT] Enforce strict monotonicity of time stamps within a series.
* [ENHANCEMENT] Export build information as metric.
* [ENHANCEMENT] Improve UI of `/alerts` page.
* [ENHANCEMENT] Improve display of target labels on status page.
* [ENHANCEMENT] Improve initialization and routing functionality of web service.
* [ENHANCEMENT] Improve target URL handling and display.
* [ENHANCEMENT] New dockerfile using alpine-glibc base image and make.
* [ENHANCEMENT] Other minor fixes.
* [ENHANCEMENT] Preserve alert state across reloads.
* [ENHANCEMENT] Prettify flag help output even more.
* [ENHANCEMENT] README.md updates.
* [ENHANCEMENT] Raise error on unknown config parameters.
* [ENHANCEMENT] Refine v1 HTTP API output.
* [ENHANCEMENT] Show original configuration file contents on status
beorn7 committed
1502
  page instead of serialized YAML.
1503
* [ENHANCEMENT] Start HUP signal handler earlier to not exit upon HUP
beorn7 committed
1504
  during startup.
1505 1506 1507 1508 1509 1510 1511 1512
* [ENHANCEMENT] Updated vendored dependencies.
* [BUGFIX] Do not panic in `StringToDuration()` on wrong duration unit.
* [BUGFIX] Exit on invalid rule files on startup.
* [BUGFIX] Fix a regression in the `.Path` console template variable.
* [BUGFIX] Fix chunk descriptor loading.
* [BUGFIX] Fix consoles "Prometheus" link to point to /
* [BUGFIX] Fix empty configuration file cases
* [BUGFIX] Fix float to int conversions in chunk encoding, which were
beorn7 committed
1513
  broken for some architectures.
1514 1515 1516 1517 1518 1519 1520
* [BUGFIX] Fix overflow detection for serverset config.
* [BUGFIX] Fix race conditions in retrieval layer.
* [BUGFIX] Fix shutdown deadlock in Consul SD code.
* [BUGFIX] Fix the race condition targets in the Makefile.
* [BUGFIX] Fix value display error in web console.
* [BUGFIX] Hide authentication credentials in config `String()` output.
* [BUGFIX] Increment dirty counter metric in storage only if
beorn7 committed
1521
  `setDirty(true)` is called.
1522
* [BUGFIX] Periodically refresh services in Consul to recover from
beorn7 committed
1523
  missing events.
1524
* [BUGFIX] Prevent overwrite of default global config when loading a
beorn7 committed
1525
  configuration.
1526 1527 1528 1529 1530 1531 1532 1533
* [BUGFIX] Properly lex `\r` as whitespace in expression language.
* [BUGFIX] Validate label names in JSON target groups.
* [BUGFIX] Validate presence of regex field in relabeling configurations.
* [CLEANUP] Clean up initialization of remote storage queues.
* [CLEANUP] Fix `go vet` and `golint` violations.
* [CLEANUP] General cleanup of rules and query language code.
* [CLEANUP] Improve and simplify Dockerfile build steps.
* [CLEANUP] Improve and simplify build infrastructure, use go-bindata
beorn7 committed
1534
  for web assets. Allow building without git.
1535 1536 1537 1538 1539 1540 1541
* [CLEANUP] Move all utility packages into common `util` subdirectory.
* [CLEANUP] Refactor main, flag handling, and web package.
* [CLEANUP] Remove unused methods from `Rule` interface.
* [CLEANUP] Simplify default config handling.
* [CLEANUP] Switch human-readable times on web UI to UTC.
* [CLEANUP] Use `templates.TemplateExpander` for all page templates.
* [CLEANUP] Use new v1 HTTP API for querying and graphing.
Julius Volz committed
1542

1543
## 0.14.0 / 2015-06-01
Julius Volz committed
1544
* [CHANGE] Configuration format changed and switched to YAML.
Fabian Reinartz committed
1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567
  (See the provided [migration tool](https://github.com/prometheus/migrate/releases).)
* [ENHANCEMENT] Redesign of state-preserving target discovery.
* [ENHANCEMENT] Allow specifying scrape URL scheme and basic HTTP auth for non-static targets.
* [FEATURE] Allow attaching meaningful labels to targets via relabeling.
* [FEATURE] Configuration/rule reloading at runtime.
* [FEATURE] Target discovery via file watches.
* [FEATURE] Target discovery via Consul.
* [ENHANCEMENT] Simplified binary operation evaluation.
* [ENHANCEMENT] More stable component initialization.
* [ENHANCEMENT] Added internal expression testing language.
* [BUGFIX] Fix graph links with path prefix.
* [ENHANCEMENT] Allow building from source without git.
* [ENHANCEMENT] Improve storage iterator performance.
* [ENHANCEMENT] Change logging output format and flags.
* [BUGFIX] Fix memory alignment bug for 32bit systems.
* [ENHANCEMENT] Improve web redirection behavior.
* [ENHANCEMENT] Allow overriding default hostname for Prometheus URLs.
* [BUGFIX] Fix double slash in URL sent to alertmanager.
* [FEATURE] Add resets() query function to count counter resets.
* [FEATURE] Add changes() query function to count the number of times a gauge changed.
* [FEATURE] Add increase() query function to calculate a counter's increase.
* [ENHANCEMENT] Limit retrievable samples to the storage's retention window.

1568
## 0.13.4 / 2015-05-23
Fabian Reinartz committed
1569 1570
* [BUGFIX] Fix a race while checkpointing fingerprint mappings.

1571
## 0.13.3 / 2015-05-11
beorn7 committed
1572 1573 1574 1575 1576 1577 1578 1579 1580
* [BUGFIX] Handle fingerprint collisions properly.
* [CHANGE] Comments in rules file must start with `#`. (The undocumented `//`
  and `/*...*/` comment styles are no longer supported.)
* [ENHANCEMENT] Switch to custom expression language parser and evaluation
  engine, which generates better error messages, fixes some parsing edge-cases,
  and enables other future enhancements (like the ones below).
* [ENHANCEMENT] Limit maximum number of concurrent queries.
* [ENHANCEMENT] Terminate running queries during shutdown.

1581
## 0.13.2 / 2015-05-05
Rainer Eli committed
1582
* [MAINTENANCE] Updated vendored dependencies to their newest versions.
Johannes 'fish' Ziemke committed
1583 1584 1585 1586 1587 1588 1589 1590
* [MAINTENANCE] Include rule_checker and console templates in release tarball.
* [BUGFIX] Sort NaN as the lowest value.
* [ENHANCEMENT] Add square root, stddev and stdvar functions.
* [BUGFIX] Use scrape_timeout for scrape timeout, not scrape_interval.
* [ENHANCEMENT] Improve chunk and chunkDesc loading, increase performance when
  reading from disk.
* [BUGFIX] Show correct error on wrong DNS response.

1591
## 0.13.1 / 2015-04-09
beorn7 committed
1592 1593 1594
* [BUGFIX] Treat memory series with zero chunks correctly in series maintenance.
* [ENHANCEMENT] Improve readability of usage text even more.

1595
## 0.13.0 / 2015-04-08
beorn7 committed
1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617
* [ENHANCEMENT] Double-delta encoding for chunks, saving typically 40% of
  space, both in RAM and on disk.
* [ENHANCEMENT] Redesign of chunk persistence queuing, increasing performance
  on spinning disks significantly.
* [ENHANCEMENT] Redesign of sample ingestion, increasing ingestion performance.
* [FEATURE] Added ln, log2, log10 and exp functions to the query language.
* [FEATURE] Experimental write support to InfluxDB.
* [FEATURE] Allow custom timestamps in instant query API.
* [FEATURE] Configurable path prefix for URLs to support proxies.
* [ENHANCEMENT] Increase of rule_checker CLI usability.
* [CHANGE] Show special float values as gaps.
* [ENHANCEMENT] Made usage output more readable.
* [ENHANCEMENT] Increased resilience of the storage against data corruption.
* [ENHANCEMENT] Various improvements around chunk encoding.
* [ENHANCEMENT] Nicer formatting of target health table on /status.
* [CHANGE] Rename UNREACHABLE to UNHEALTHY, ALIVE to HEALTHY.
* [BUGFIX] Strip trailing slash in alertmanager URL.
* [BUGFIX] Avoid +InfYs and similar, just display +Inf.
* [BUGFIX] Fixed HTML-escaping at various places.
* [BUGFIX] Fixed special value handling in division and modulo of the query
  language.
* [BUGFIX] Fix embed-static.sh.
1618
* [CLEANUP] Added initial HTTP API tests.
beorn7 committed
1619
* [CLEANUP] Misc. other code cleanups.
1620
* [MAINTENANCE] Updated vendored dependencies to their newest versions.
beorn7 committed
1621

1622
## 0.12.0 / 2015-03-04
beorn7 committed
1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636
* [CHANGE] Use client_golang v0.3.1. THIS CHANGES FINGERPRINTING AND INVALIDATES
  ALL PERSISTED FINGERPRINTS. You have to wipe your storage to use this or
  later versions. There is a version guard in place that will prevent you to
  run Prometheus with the stored data of an older Prometheus.
* [BUGFIX] The change above fixes a weakness in the fingerprinting algorithm.
* [ENHANCEMENT] The change above makes fingerprinting faster and less allocation
  intensive.
* [FEATURE] OR operator and vector matching options. See docs for details.
* [ENHANCEMENT] Scientific notation and special float values (Inf, NaN) now
  supported by the expression language.
* [CHANGE] Dockerfile makes Prometheus use the Docker volume to store data
  (rather than /tmp/metrics).
* [CHANGE] Makefile uses Go 1.4.2.

1637 1638
## 0.11.1 / 2015-02-27
* [BUGFIX] Make series maintenance complete again. (Ever since 0.9.0rc4,
beorn7 committed
1639 1640 1641 1642 1643 1644 1645 1646 1647
  or commit 0851945, series would not be archived, chunk descriptors would
  not be evicted, and stale head chunks would never be closed. This happened
  due to accidental deletion of a line calling a (well tested :) function.
* [BUGFIX] Do not double count head chunks read from checkpoint on startup.
  Also fix a related but less severe bug in counting chunk descriptors.
* [BUGFIX] Check last time in head chunk for head chunk timeout, not first.
* [CHANGE] Update vendoring due to vendoring changes in client_golang.
* [CLEANUP] Code cleanups.
* [ENHANCEMENT] Limit the number of 'dirty' series counted during checkpointing.
beorn7 committed
1648

1649
## 0.11.0 / 2015-02-23
beorn7 committed
1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672
* [FEATURE] Introduce new metric type Histogram with server-side aggregation.
* [FEATURE] Add offset operator.
* [FEATURE] Add floor, ceil and round functions.
* [CHANGE] Change instance identifiers to be host:port.
* [CHANGE] Dependency management and vendoring changed/improved.
* [CHANGE] Flag name changes to create consistency between various Prometheus
  binaries.
* [CHANGE] Show unlimited number of metrics in autocomplete.
* [CHANGE] Add query timeout.
* [CHANGE] Remove labels on persist error counter.
* [ENHANCEMENT] Various performance improvements for sample ingestion.
* [ENHANCEMENT] Various Makefile improvements.
* [ENHANCEMENT] Various console template improvements, including
  proof-of-concept for federation via console templates.
* [ENHANCEMENT] Fix graph JS glitches and simplify graphing code.
* [ENHANCEMENT] Dramatically decrease resources for file embedding.
* [ENHANCEMENT] Crash recovery saves lost series data in 'orphaned' directory.
* [BUGFIX] Fix aggregation grouping key calculation.
* [BUGFIX] Fix Go download path for various architectures.
* [BUGFIX] Fixed the link of the Travis build status image.
* [BUGFIX] Fix Rickshaw/D3 version mismatch.
* [CLEANUP] Various code cleanups.

1673
## 0.10.0 / 2015-01-26
Bjoern Rabenstein committed
1674 1675 1676 1677 1678 1679
* [CHANGE] More efficient JSON result format in query API. This requires
  up-to-date versions of PromDash and prometheus_cli, too.
* [ENHANCEMENT] Excluded non-minified Bootstrap assets and the Bootstrap maps
  from embedding into the binary. Those files are only used for debugging,
  and then you can use -web.use-local-assets. By including fewer files, the
  RAM usage during compilation is much more manageable.
1680
* [ENHANCEMENT] Help link points to https://prometheus.github.io now.
Bjoern Rabenstein committed
1681 1682 1683 1684
* [FEATURE] Consoles for haproxy and cloudwatch.
* [BUGFIX] Several fixes to graphs in consoles.
* [CLEANUP] Removed a file size check that did not check anything.

1685
## 0.9.0 / 2015-01-23