* Adapt UI for Prometheus Agent
UI is not my strongest skill, but I'd like to have something minimal for
the initial release of the agent.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Address review comments
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Add tests
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Add tests, serve only current mode paths
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Update js style, add agent test
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
`BufferedSeriesIterator` and `MemoizedSeriesIterator` use a method
called `Values` for exactly the purpose for which all other iterators
of the same kind use a method called `At`. That alone is confusing,
but on top of that, the `Values` method only returns a single sample,
not multiple values. I assume the naming has historical reasons. This
commit makes it more consistent. It is now easier to read, and now
`BufferedSeriesIterator` and `MemoizedSeriesIterator` implement
`chunkenc.Iterator` like many other iterators, too.
Signed-off-by: beorn7 <beorn@grafana.com>
- Pick At... method via return value of Next/Seek.
- Do not clobber returned buckets.
- Add partial FloatHistogram suppert.
Note that the promql package is now _only_ dealing with
FloatHistograms, following the idea that PromQL only knows float
values.
As a byproduct, I have removed the histogramSeries metric. In my
understanding, series can have both float and histogram samples, so
that metric doesn't make sense anymore.
As another byproduct, I have converged the sampleBuf and the
histogramSampleBuf in memSeries into one. The sample type stored in
the sampleBuf has been extended to also contain histograms even before
this commit.
Signed-off-by: beorn7 <beorn@grafana.com>
* remove vfsgen usages
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
* web: use embed package for static assets
This requires go 1.16.
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
* circleci: drop go generate in web/ui
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
* Makefile: compress web assets before build
This commit add compression before (and decompression after) prometheus
is build. This ensures that gzipped assets are embeded in the prometheus
binary, if the builtinassets build tag is passed. If the build tag is
not passed this step is still executed but has no effect.
All this is executed in a subshell so that we can run the decompress
step even if the build step fails, but retain the exit code of promu.
This cleanup could also cover interrupts, but I left that out for now.
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
* [FIXME]: add new module dependency on common/assets and temp replace
Signed-off-by: Jan Fajerski <jfajersk@redhat.com>
This is to avoid copying the many fields of a histogram.Histogram all
the time.
This also fixes a bunch of formerly broken tests.
Signed-off-by: beorn7 <beorn@grafana.com>
Later on in the test we override fields of handler, so we need a handler
per sub-test, rather than a global one.
Signed-off-by: Mateusz Gozdek <mgozdekof@gmail.com>
This creates a new `model` directory and moves all data-model related
packages over there:
exemplar labels relabel rulefmt textparse timestamp value
All the others are more or less utilities and have been moved to `util`:
gate logging modetimevfs pool runtime
Signed-off-by: beorn7 <beorn@grafana.com>
* add a negative boost for some trigonometric functions that can overlapp other regular promQL functions
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* add comments to explain the purpose of the attribute boost
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
I would like to avoid extra API call's to determine if we are running in
Agent Mode, so I think we could use this approach.
This is a bootstrap of #9612
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Initial draft of prometheus-agent
This commit introduces a new binary, prometheus-agent, based on the
Grafana Agent code. It runs a WAL-only version of prometheus without the
TSDB, alerting, or rule evaluations. It is intended to be used to
remote_write to Prometheus or another remote_write receiver.
By default, prometheus-agent will listen on port 9095 to not collide
with the prometheus default of 9090.
Truncation of the WAL cooperates on a best-effort case with Remote
Write. Every time the WAL is truncated, the minimum timestamp of data to
truncate is determined by the lowest sent timestamp of all samples
across all remote_write endpoints. This gives loose guarantees that data
from the WAL will not try to be removed until the maximum sample
lifetime passes or remote_write starts functionining.
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* add tests for Prometheus agent (#22)
* add tests for Prometheus agent
* add tests for Prometheus agent
* rearranged tests as per the review comments
* update tests for Agent
* changes as per code review comments
Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>
* incremental changes to prometheus agent
Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>
* changes as per code review comments
Signed-off-by: SriKrishna Paparaju <paparaju@gmail.com>
* Commit feedback from code review
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
Co-authored-by: Ganesh Vernekar <ganeshvern@gmail.com>
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* Port over some comments from grafana/agent
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* Rename agent.Storage to agent.DB for tsdb consistency
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* Consolidate agentMode ifs in cmd/prometheus/main.go
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* Document PreAction usage requirements better for agent mode flags
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* remove unnecessary defaultListenAddr
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
* `go fmt ./tsdb/agent` and fix lint errors
Signed-off-by: Robert Fratto <robertfratto@gmail.com>
Co-authored-by: SriKrishna Paparaju <paparaju@gmail.com>
* Fix broken prefixed asset links in webpack build
Fixes https://github.com/prometheus/prometheus/issues/9585
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Use .env file for PUBLIC_URL npm build env var
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* update documentation around react-app and how to upgrade the npm dependencies
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* wording around caution to take when updating the deps
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* fixing the npm version to be used and explain where you should perform the npm install command
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* simplify what is required to build prometheus from the source
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* aligned period and removed redondant word installed
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* set nodeJS version to be used at 16
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* describe manuel steps to update a dependency for the react-app
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* rewording of the manuel step to update the dependencies
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
A lot of this code was hacked together, literally during a
hackathon. This commit intends not to change the code substantially,
but just make the code obey the usual style practices.
A (possibly incomplete) list of areas:
* Generally address linter warnings.
* The `pgk` directory is deprecated as per dev-summit. No new packages should
be added to it. I moved the new `pkg/histogram` package to `model`
anticipating what's proposed in #9478.
* Make the naming of the Sparse Histogram more consistent. Including
abbreviations, there were just too many names for it: SparseHistogram,
Histogram, Histo, hist, his, shs, h. The idea is to call it "Histogram" in
general. Only add "Sparse" if it is needed to avoid confusion with
conventional Histograms (which is rare because the TSDB really has no notion
of conventional Histograms). Use abbreviations only in local scope, and then
really abbreviate (not just removing three out of seven letters like in
"Histo"). This is in the spirit of
https://github.com/golang/go/wiki/CodeReviewComments#variable-names
* Several other minor name changes.
* A lot of formatting of doc comments. For one, following
https://github.com/golang/go/wiki/CodeReviewComments#comment-sentences
, but also layout question, anticipating how things will look like
when rendered by `godoc` (even where `godoc` doesn't render them
right now because they are for unexported types or not a doc comment
at all but just a normal code comment - consistency is queen!).
* Re-enabled `TestQueryLog` and `TestEndopints` (they pass now,
leaving them disabled was presumably an oversight).
* Bucket iterator for histogram.Histogram is now created with a
method.
* HistogramChunk.iterator now allows iterator recycling. (I think
@dieterbe only commented it out because he was confused by the
question in the comment.)
* HistogramAppender.Append panics now because we decided to treat
staleness marker differently.
Signed-off-by: beorn7 <beorn@grafana.com>
This updates React, TypeScript, and some other node packages (but not
everything).
A couple of notes:
- `enzyme-adapter-react-16` does not have a React 17 equivalent yet, so I
switched to the fork `@wojtekmaj/enzyme-adapter-react-17`
- A bunch of tests are still failing because I think in the enzyme testing
environment, a browser API (`ResizeObserver`) is missing, and maybe for other
reasons. This needs to be explored + fixed.
- The TypeScript update introduced more stringent rules, which required fixing
up a bunch of pieces of code a bit.
- The `use-media` package doesn't work with React 17 yet, so I just built our
own minimal `useMedia` hook instead (just a couple of lines).
- I commented out part of the code in `withStartingIndicator.tsx` because it
fails the now-stricter lint checks. It needs to be fixed (and not commented
out).
Signed-off-by: Julius Volz <julius.volz@gmail.com>
This updates React, TypeScript, and some other node packages (but not
everything).
A couple of notes:
- `enzyme-adapter-react-16` does not have a React 17 equivalent yet, so I
switched to the fork `@wojtekmaj/enzyme-adapter-react-17`
- A bunch of tests are still failing because I think in the enzyme testing
environment, a browser API (`ResizeObserver`) is missing, and maybe for other
reasons. This needs to be explored + fixed.
- The TypeScript update introduced more stringent rules, which required fixing
up a bunch of pieces of code a bit.
- The `use-media` package doesn't work with React 17 yet, so I just built our
own minimal `useMedia` hook instead (just a couple of lines).
- I commented out part of the code in `withStartingIndicator.tsx` because it
fails the now-stricter lint checks. It needs to be fixed (and not commented
out).
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Remove query hacks in the API and fix metrics
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Tests for the metrics
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Better way to count series on restart
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* PromQL: Fix start and end keywords masking label and metric names
This commit fixes an issue with the "at modifier" that introduced two
new keywords: `start` and `end`. In grouping options and in metric
names, these keywords took precedence over metric or label names, so
that those metrics and labels could no longer be referenced.
Signed-off-by: Clayton Peters <clayton.peters@man.com>
* Add in additional tests for metrics and/or labels called start/end.
Signed-off-by: Clayton Peters <clayton.peters@man.com>
* *: Cut 2.29.0-rc.0
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
* VERSION: bump to 2.29.0-rc.0
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
* Remove experimental wording on size-based retention
Followup of #9004
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Fix PR reference in changelog
Signed-off-by: George Brighton <george@gebn.co.uk>
* Describe EC2 availability zone IDs at most once per refresh (#9142)
Signed-off-by: George Brighton <george@gebn.co.uk>
* Describe EC2 availability zones at most once per SD load
Closes#9142.
Signed-off-by: George Brighton <george@gebn.co.uk>
* Incorporate feedback
Signed-off-by: George Brighton <george@gebn.co.uk>
* Integrate feedback
Signed-off-by: George Brighton <george@gebn.co.uk>
* Add a compatibility note for macOS users.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* *: Cut v2.29.0-rc.1
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
* Fix `kuma_sd` targetgroup reporting (#9157)
* Bundle all xDS targets into a single group
Signed-off-by: austin ce <austin.cawley@gmail.com>
* *: cut v2.29.0-rc.2
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
* Rename links
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* bump codemirror-promql to 0.17.0
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* *: cut v2.29.0
Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>
* tsdb: align atomically accessed int64 (#9192)
This prevents a panic in 32-bit archs:
https://pkg.go.dev/sync/atomic#pkg-note-BUGFixed#9190
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Release 2.29.1 (#9193)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
Co-authored-by: Clayton Peters <clayton.peters@man.com>
Co-authored-by: Frederic Branczyk <fbranczyk@gmail.com>
Co-authored-by: George Brighton <george@gebn.co.uk>
Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Augustin Husson <husson.augustin@gmail.com>
* Fix `kuma_sd` targetgroup reporting (#9157)
* Bundle all xDS targets into a single group
Signed-off-by: austin ce <austin.cawley@gmail.com>
* Snapshot in-memory chunks on shutdown for faster restarts (#7229)
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Rename links
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Remove Individual Data Type Caps in Per-shard Buffering for Remote Write (#8921)
* Moved everything to nPending buffer
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Simplify exemplar capacity addition
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added pre-allocation
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Don't allocate if not sending exemplars
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Avoid deadlock when processing duplicate series record (#9170)
* Avoid deadlock when processing duplicate series record
`processWALSamples()` needs to be able to send on its output channel
before it can read the input channel, so reads to allow this in case the
output channel is full.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* processWALSamples: update comment
Previous text seems to relate to an earlier implementation.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Optimise WAL loading by removing extra map and caching min-time (#9160)
* BenchmarkLoadWAL: close WAL after use
So that goroutines are stopped and resources released
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* BenchmarkLoadWAL: make series IDs co-prime with #workers
Series are distributed across workers by taking the modulus of the
ID with the number of workers, so multiples of 100 are a poor choice.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* BenchmarkLoadWAL: simulate mmapped chunks
Real Prometheus cuts chunks every 120 samples, then skips those samples
when re-reading the WAL. Simulate this by creating a single mapped chunk
for each series, since the max time is all the reader looks at.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Fix comment
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Remove series map from processWALSamples()
The locks that is commented to reduce contention in are now sharded
32,000 ways, so won't be contended. Removing the map saves memory and
goes just as fast.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* loadWAL: Cache the last mmapped chunk time
So we can skip calling append() for samples it will reject.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Improvements from code review
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Full stops and capitals on comments
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Cache max time in both places mmappedChunks is updated
Including refactor to extract function `setMMappedChunks`, to reduce
code duplication.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Update head min/max time when mmapped chunks added
This ensures we have the correct values if no WAL samples are added for
that series.
Note that `mSeries.maxTime()` was always `math.MinInt64` before, since
that function doesn't consider mmapped chunks.
Signed-off-by: Bryan Boreham <bjboreham@gmail.com>
* Split Go and React Tests (#8897)
* Added go-ci and react-ci
Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Remove search keymap from new expression editor (#9184)
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Co-authored-by: Austin Cawley-Edwards <austin.cawley@gmail.com>
Co-authored-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julien Pivotto <roidelapluie@inuits.eu>
Co-authored-by: Bryan Boreham <bjboreham@gmail.com>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
* Push the matchers for LabelNames all the way into the index.
NB This doesn't actually implement it in the index, just plumbs it through for now...
Signed-off-by: Tom Wilkie <tom@grafana.com>
* Hack it up. Does not work.
Signed-off-by: Tom Wilkie <tom@grafana.com>
* Revert changes I don't understand
Can't see why do we need to hold a mutex on symbols, and the purpose of
the LabelNamesFor method.
Maybe I'll need to re-add this later.
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Implement LabelNamesFor
This method provides the label names that appear in the postings
provided. We do that deeper than the label values because we know
beforehand that most of the label names we'll be the same across
different postings, and we don't want to go down an up looking up the
same symbols for all different series.
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Mutex on symbols should be unlocked
However, I still don't understand why do we need a mutex here.
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Fix head.LabelNamesFor
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Implement mockIndex LabelNames with matchers
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Nitpick on slice initialisation
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Add tests for LabelNamesWithMatchers
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Fix the mutex mess on head.LabelValues/LabelNames
I still don't see why we need to grab that unrelated mutex, but at least
now we're grabbing it consistently
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Check error after iterating postings
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Use the error from posting when there was en error in postings
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Update storage/interface.go comment
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* Update tsdb/index/index.go comment
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* Update tsdb/index/index.go wrapped error msg
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* Update tsdb/index/index.go wrapped error msg
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* Update tsdb/index/index.go warpped error msg
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* Remove unneeded comment
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Add testcases for LabelNames w/matchers in api.go
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
* Use t.Cleanup() instead of defer in tests
Signed-off-by: Oleg Zaytsev <mail@olegzaytsev.com>
Co-authored-by: Tom Wilkie <tom@grafana.com>
Co-authored-by: Ganesh Vernekar <15064823+codesome@users.noreply.github.com>
* Do not panic on histoAppender.Append
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* M-map all chunks on shutdown
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Support negative schema for querying
Signed-off-by: Ganesh Vernekar <ganeshvern@gmail.com>
* Add sorting and filtering to flags page
Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>
* Make filter understand
Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>
* split big state object into smaller ones
Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>
* use fuzzy match and sanitize html for search results
Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>
* use fuzzy.filter
Signed-off-by: Dustin Hooten <dustinhooten@gmail.com>
* replace fuzzy lib by @nexucis/fuzzy + fix flags issues
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* replace fuzzy by @nexucis/fuzzy in ExpressionInput.tsx
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* remove fuzzy lib from package.json
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* fix flags test
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* simplify the input in the fuzzy search
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* cleanup html to be easily compatible with the dark theme
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* fix filtering when there is no result
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* use id to fix the test
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
Co-authored-by: Dustin Hooten <dustinhooten@gmail.com>
* Added selection flot plugin
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added time selection
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Removed irrelevant line in license header
Signed-off-by: Levi Harrison <git@leviharrison.dev>
All this is doing is wrapping the inner alert details display with a
conditional `{open && ...}`.
This already improves https://github.com/prometheus/prometheus/issues/8548 a
lot for cases where there are many individual firing/pending alert elements
under each alerting rule.
E.g. for a list of 200 rules with ~100 alert elements each, this changed the page
render time from 30 seconds to 1s.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Show progress bar when max is not 0
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added semicolon
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fix/removed forwarding
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added global 'wasReady' and 'wasUnexpected'
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Eslint fixes
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added withStartingIndicator wrapper
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed condition
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Removed unused import
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Moved withStartingIndicator calls to pages index
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed withStartingIndicator tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed eslint (maybe?)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Trailing comma
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added prettier ignore
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fix eslint (pt. 2)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Improve graph info tooltip for series and exemplars
It now shows more clearly what the label sets are about (series vs. exemplar +
associated series) and also shows the metric name for both label sets when it
is present.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Fix up inconsistent title casing
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Explicitly indicate empty labelsets in hover tooltip
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Adjust tests to new hover detail
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Added exemplar support
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Modified tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fix eslint suggestions
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Address review comments
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed undefined data property error
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added series label section to tooltip
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed spacing
Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added exemplar info
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed exemplar symbol
Signed-off-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Hide selected exemplar info when 'Show Exemplars' is unchecked
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Include series labels in exemplar info
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* De-densify exemplars
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Moved showExemplars to per-panel control
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Eslint fixes
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Address review comments
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fix state bug
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Removed unused object
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fix eslint
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Encoded 'show_exemplars' in url
Signed-off-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
* Removed min value
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added tests
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Set min value
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added walreplay API endpoint
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added starting page to react-ui
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Documented the new endpoint
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed typos
Signed-off-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
* Removed logo
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed isResponding to isUnexpected
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed width of progress bar
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed width of progress bar
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added DB stats object
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Updated starting page to work with new fields
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Passing nil
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Passing nil (pt. 2)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Passing nil (pt. 3)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Passing nil (and also implementing a method this time) (pt. 4)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Passing nil (and also implementing a method this time) (pt. 5)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed const to let
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Passing nil (pt. 6)
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Remove SetStats method
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added comma
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed api
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed to triple equals
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed data response types
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Don't return pointer
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Changed version
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed interface issue
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed pointer
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed copying lock value error
Signed-off-by: Levi Harrison <git@leviharrison.dev>
Co-authored-by: Julius Volz <julius.volz@gmail.com>
* Do not remove /new because it is not part of the route parameter (CVE-2021-29622)
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Release 2.27.1
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Upgrade bootstrap and reactstrap to the latest version
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add SASS support
node-sass is needed for cra to handle SCSS files instead of pure CSS.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme
This adds a dark theme and UI controls to switch between themes.
Dark theme will require some CSS changes that will follow in future commits.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a margin to Prometheus brand
There is no space between 'Prometheus' brand text and the toggle button when using mobile device.
This adds a margin to the button that's only rendered on mobile
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for CollapsibleAlertPanel
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for RulesContent
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for Config
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Use bootstrap classes for margins
We can override margins via bootstrap css classes instead of loading custom css module.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for QueryStatsView
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for MetricsExplorer
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for 'Clear time' button
This button had some custom css based on light bootstrap theme so it needs to be adjusted for dark theme.
This change re-uses bootstrap styles used for input components instead of copying color values
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add dark theme for Graph panel input
This makes the whole input group look consistent in dark mode as the old styles were made to blend it with the default bootstrap theme.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for CME expression input
This change splits current CME theme into 3:
1 - base theme used for both light and dark mode
2 - light mode specific theme that overrides base
3 - dark mode specific theme that overrides base
To make it all work we also need to move theme to dynamic config, so when theme value
in ThemeContext changes CME input will apply a new theme.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Add a dark theme for /graph page tabs
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Fix metrics explorer modal scroll
bootstrap-dark breaks scrolling on the metrics modal, so we need an extra rule to fix that.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Move App.css rules to themes/_shared.scss
This completes splitting styles into light and dark theme.
It also fixes some small issues with themes as now all styles from App.css are applied correctly.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Move html{} styles to a dedicated file
html block is root document so styles for it cannot be nested under theme classes.
Move it out and add a bit of documentation to explain what which file does.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Fix reboot styles overriding other FontAwesome classes
Both bootstrap themes we use import reboot classes (https://getbootstrap.com/docs/4.6/content/reboot/) which has the side effect of overriding other classes. We need reboot to be applied as defaults for the browser, so it needs to be moved out of theme class selectors. But because reboot requires scss variables we need to feed it something, for that we use the default light theme, so it gets imported there and browser will use style of the default theme to reset default (unthemed) styles.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Move codicon font to app.scss
This needs to be applied globally, not per theme.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* bump cm-promql to v0.14.1
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* enhancement a bit the way to build the codemirror PromQL extension
Signed-off-by: Augustin Husson <husson.augustin@gmail.com>
* Changed error wording
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added another test and moved both into the correct suite
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added 16w and 26w time range steps
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Fixed indentation
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Check response and return alternate page if no statistics
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Added tests for alternate page
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Change error handling in unixToTime
Signed-off-by: Levi Harrison <git@leviharrison.dev>
* Add new codemirror-promql-based expression editor
This adds advanced autocompletion, syntax highlighting, and linting
for PromQL.
Fixes https://github.com/prometheus/prometheus/issues/6160
Fixes https://github.com/prometheus/prometheus/issues/5421
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Group new editor options and float them left
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Improve history autocompletion handling
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Only show info tooltips for unabbreviated completion items
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Rename "new editor" to "experimental editor"
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Add path prefix support
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Revert accidental check-in of go.sum changes
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Remove spurious console.log
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Fix completion item type icon styling
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Checkbox should use onChange, not onClick
This fixes react console errors:
You provided a `checked` prop to a form field without an `onChange` handler. This will render a read-only field. If the field should be mutable use `defaultChecked`. Otherwise, set either `onChange` or `readOnly`.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Correctly pass key in metrics exporer
Instead of passing metric variable we pass 'metric' string, which causes console errors due to duplicated keys.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Update tests
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
If I have a rule with a very long name it will force the table to be wider then the viewport.
This forces the browser to wrap long rule names and uses smaller font, to avoid having overflow.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
Target errors are rendered as badges. If error text is very long it will expand the table since badges are not allowed to wrap.
Replace badge with a span which is allowed to wrap around.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* Fix inconsistent display of word 'ago' in last evaluation column on rules page
Signed-off-by: Alex Petrov <alex.petrov.vt@gmail.com>
* Extract adding word 'ago' to relativeDuration util function
Signed-off-by: Alex Petrov <alex.petrov.vt@gmail.com>
* Move remote read handler to remote package.
This follows the pattern I started with the remote write handler. The api/v1 package is getting pretty cluttered. Moving code to other packages helps reduce this size and also makes it reusable - eg Cortex doesn't do streaming remote writes yet, and will very soon.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Deal with a nil remoteReadHandler for tests.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Remove the global metrics.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Fix test.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Review feedback.
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* Add expression explorer
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Add final new line to all files
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Rename expression to metric
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Pass dedicated metrics array to metrics explorer
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Fix styling of button
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Use append instead of prepend
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Update max width of modal
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Fix code style
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Fix inconsistent variable naming
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Fix modal title
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Fix tests
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Prevent request from being cached
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Remove timestamp from request
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Update button selector in test
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Refactor passing down metric names and query history
Signed-off-by: Lucas Hild <git@lucas-hild.de>
* Fix code style
Signed-off-by: Lucas Hild <git@lucas-hild.de>
Similar to https://github.com/prometheus/prometheus/pull/7902, this could lead
to style bugs for label names that correspond to styled CSS class names.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* [UI] Add a test for duplicated keys in EndpointLink component
I've noticed that I'm getting warnings about multiple children with the same key on /targets page.
This adds a test that fails when that happens.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
* [UI] Fix duplicated keys on /targets page
Since any URI we render on /targets page can have multi-value params we should use both name and value as components keys.
Signed-off-by: Łukasz Mierzwa <l.mierzwa@gmail.com>
- Remove unrelated changes
- Refactor code out of the API module - that is already getting pretty crowded.
- Don't track reference for AddFast in remote write. This has the potential to consume unlimited server-side memory if a malicious client pushes a different label set for every series. For now, its easier and safer to always use the 'slow' path.
- Return 400 on out of order samples.
- Use remote.DecodeWriteRequest in the remote write adapters.
- Put this behing the 'remote-write-server' feature flag
- Add some (very) basic docs.
- Used named return & add test for commit error propagation
Signed-off-by: Tom Wilkie <tom.wilkie@gmail.com>
* API: Fix global URL when external address has no port
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Update test
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Handle IPv6 correctly.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* tsdb: Expose total number of label pairs in head
Signed-off-by: Nguyen Le Vu Long <vulongvn98@gmail.com>
* fix: add comment for NumLabelPairs
Signed-off-by: Nguyen Le Vu Long <vulongvn98@gmail.com>
* fix: remove comment
Signed-off-by: Nguyen Le Vu Long <vulongvn98@gmail.com>
* Small cleanup of API code.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed typo spotted by Julien.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
- First, it is currently not only removing "metric" autocomplete, but
also "query history autocomplete", so the checkbox is confusing.
- Then, in the future, we will want also "functions" autocomplete.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
The React app's assets are now served under /assets, while all old
custom web assets (including the ones for console templates) are now
served from /classic/static.
I tested different combinations of --web.external-url and
--web.route-prefix with proxies in front, and I couldn't find a problem
yet with the routing. Console templates also still work.
While migrating old endpoints to /classic, I noticed that /version was
being treated like a lot of the old UI pages, with readiness check
handler in front of it, etc. I kept it in /version and removed that
readiness wrapper, since it doesn't seem to be needed for that endpoint.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* UI: Add toggle to enable/disable metric autocomplete
This change adds a toggle to enable or disable the metric autocomplete
functionality. By default it is enabled. This is a port of a change I
did in [Thanos][1].
[1]: https://github.com/thanos-io/thanos/pull/3381
Signed-off-by: Jarod Watkins <jarod@42lines.net>
* Adding full variable name
Signed-off-by: Jarod Watkins <jarod@42lines.net>
* Testify: move to require
Moving testify to require to fail tests early in case of errors.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* More moves
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Refactor test assertions
This pull request gets rid of assert.True where possible to use
fine-grained assertions.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* dynamically determine path prefix
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* minor changes per PR review
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* use Context for apiPath and pathPrefix
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* remove unhandled "/version" path
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* only process index once instead of on every req
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* remove unneeded tag fragment
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* switch api path to const
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* revert
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* update tests
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* linter updates
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* simplify
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
* updates per peer review
Signed-off-by: James Ranson <james_ranson@cable.comcast.com>
The current meaning of the alert firing/pending/inactive
toggle seems ambiguouos as #7460 demonstrates.
This commit adds a checkbox icon to clarify the
active state of the button.
Signed-off-by: Christian Hoffmann <mail@hoffmann-christian.info>
* add the "assets" make target and change directory note in readme
Signed-off-by: Luke Chen <showuon@gmail.com>
* address reviewer's comment
Signed-off-by: Luke Chen <showuon@gmail.com>
The collapse all option as suggested in the corresponding issue
has been implemented in this PR.
Signed-off-by: Deepjyoti Mondal <djmdeveloper060796@gmail.com>
Adding the label name as a CSS class can break styling and other
behavior when the label name has a special meaning in CSS. E.g. the
"container" label was displayed at 100% width because it was interpreted
to be a bootstrap container layout element.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
We're using Prometheus behind a reverse proxy which authenticates everything based on cookies.
Without this attribute the cookie is not send with the request for the manifest.json
Signed-off-by: Stefan Bueringer <sbueringer@gmail.com>
* cleanup tempfiles for web_test
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
* cleanup tempfiles for api_test
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
* cleanup tempfiles for file_test
Signed-off-by: Li Zhijian <lizhijian@cn.fujitsu.com>
* Support new duration format in graph range input
This is to make the duration parsing and formatting in the graph range
input field consistent with the new duration formatting introduced for
the configuration and PromQL
(https://github.com/prometheus/prometheus/pull/7713).
Ranges were previously handled in seconds - these are now handled in
milliseconds everywhere, as this makes things nicer / easier.
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Fixups
Signed-off-by: Julius Volz <julius.volz@gmail.com>
I have received a recent report for a user which confirms that sometime
the GRPC server does not stop propery.
It appears that there are 2 issues:
1. The cmux server can refuse to stop if there are stale connections.
For that we set the ReadTimeout.
2. The GRPC server graceful stop can never finish.
What this PR avoids is:
```
goroutine 227 [semacquire, 2 minutes]:
sync.runtime_Semacquire(0xc00059a75c)
/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc00059a75c)
/usr/local/go/src/sync/waitgroup.go:130 +0x64
google.golang.org/grpc.(*Server).GracefulStop(0xc00059a600)
```
This PR stops the GRPC server after 15s. Related to the go routing dumps
in #6747.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Bump jquery and @types/jquery
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Properly close color box span in graph hover detail
jQuery 3.5.0 introcuded a breaking change that was necessary to fix HTML
security issues
(https://blog.jquery.com/2020/04/10/jquery-3-5-0-released/), and as a
result the detail-swatch span wasn't closed properly anymore, because
spans are not valid self-closing tags in
XHTML.
Fixes https://github.com/prometheus/prometheus/issues/7803
Signed-off-by: Julius Volz <julius.volz@gmail.com>
* Fix up tests
Signed-off-by: Julius Volz <julius.volz@gmail.com>
Further precision is truncated by the Prometheus API, so the
steps don't end up quite aligning subsequently.
Fixes#7711
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
* tsdb: Added ChunkQueryable implementations to db; unified compactor, querier and fanout block iterating.
Chained to https://github.com/prometheus/prometheus/pull/7059
* NewMerge(Chunk)Querier now takies multiple primaries allowing tsdb DB code to use it.
* Added single SeriesEntry / ChunkEntry for all series implementations.
* Unified all vertical, and non vertical for compact and querying to single
merge series / chunk sets by reusing VerticalSeriesMergeFunc for overlapping algorithm (same logic as before)
* Added block (Base/Chunk/)Querier for block querying. We then use populateAndTomb(Base/Chunk/) to iterate over chunks or samples.
* Refactored endpoint tests and querier tests to include subtests.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Addressed comments from Brian and Beorn.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed snapshot test and added chunk iterator support for DBReadOnly.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed race when iterating over Ats first.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed tests.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed populate block tests.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed endpoints test.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed test.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Added test & fixed case of head open chunk.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed DBReadOnly tests and bug producing 1 sample chunks.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Added cases for partial block overlap for multiple full chunks.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Added extra tests for chunk meta after compaction.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fixed small vertical merge bug and added more tests for that.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* storage: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* tsdb: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* web: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* notifier: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* cmd: Replace usage of sync/atomic with uber-go/atomic
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* scripts: Verify that we are not using restricted packages
It checks that we are not directly importing 'sync/atomic'.
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* Reorganise imports in blocks
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* notifier/test: Apply PR suggestions
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* storage/remote: avoid storing references on newEntry
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* Revert "scripts: Verify that we are not using restricted packages"
This reverts commit 278d32748e.
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* web: Group imports accordingly
Signed-off-by: Javier Palomo <javier.palomo.almena@gmail.com>
* Backport custom title to React UI
Backports #4841 into the new UI.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Fix typo
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Display dates as well as timestamps in the status page
Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
* Trim trailing whitespaces
Signed-off-by: Sylvain Rabot <sylvain@abstraction.fr>
Previously, hash-fragment links like this:
http://mark-t510:9090/targets#job-alertmanager
Would scroll to have the header at the top, obscured by the nav bar.
Tested in both old and new UIs.
Fixes#7434
Signed-off-by: Mark Hansen <markhansen@google.com>
jQuery prior to 3.4.0 is affected by an Object.prototype pollution
vulnerability (CVE-2019-11358). Even though our code doesn't seem to be
vulnerable to the issue, lets upgrade to the latest jQuery release so we
don't have to bother.
Signed-off-by: Simon Pasquier <spasquie@redhat.com>
* Fixed nits introduced by https://github.com/prometheus/prometheus/pull/7334
* Added ChunkQueryable implementation to fanout and readyStorage.
* Added more comments.
* Changed NewVerticalChunkSeriesMerger to CompactingChunkSeriesMerger, removed tiny interface by reusing VerticalSeriesMergeFunc for overlapping algorithm for
both chunks and series, for both querying and compacting (!) + made sure duplicates are merged.
* Added ErrChunkSeriesSet
* Added Samples interface for seamless []promb.Sample to []tsdbutil.Sample conversion.
* Deprecating non chunks serieset based StreamChunkedReadResponses, added chunk one.
* Improved tests.
* Split remote client into Write (old storage) and read.
* Queryable client is now SampleAndChunkQueryable. Since we cannot use nice QueryableFunc I moved
all config based options to sampleAndChunkQueryableClient to aboid boilerplate.
In next commit: Changes for TSDB.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* value field more human readable
Signed-off-by: kisc <nuno_kisc@hotmail.com>
* fix typo
Signed-off-by: Nuno Cardoso <nuno_kisc@hotmail.com>
* add function convertSCToNumber
Signed-off-by: nunokisc <nuno_kisc@hotmail.com>
* add convertSCToNumber test
Signed-off-by: nunokisc <nuno_kisc@hotmail.com>
* normalize function name
Signed-off-by: kisc <nuno_kisc@hotmail.com>
* convertScientificNotationToNumber to parsePrometheusFloat
Signed-off-by: kisc <nuno_kisc@hotmail.com>
* Fixed returned API status code on error
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Fixed linter
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Simplified code
Signed-off-by: Marco Pracucci <marco@pracucci.com>
* Add errors and Warnings to SeriesSet
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Change Querier interface and refactor accordingly
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Refactor promql/engine to propagate warnings at eval stage
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Address review issues
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Make sure all the series from all Selects are pre-advanced
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Address review issues
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Separate merge series sets
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Clean
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Refactor merge querier failure handling
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Refactored and simplified fanout with improvements from incoming chunk iterator PRs.
* Secondary logic is hidden, instead of weird failed series set logic we had.
* Fanout is well commented
* Fanout closing record all errors
* MergeQuerier improved API (clearer)
* deferredGenericMergeSeriesSet is not needed as we return no samples anyway for failed series sets (next = false).
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Fix formatting
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Fix CI issues
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Added final tests for error handling.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Addressed Brian's comments.
* Moved hints in populate to be allocated only when needed.
* Used sync.Once in secondary Querier to achieve all-or-nothing partial response logic.
* Select after first Next is done will panic.
NOTE: in lazySeriesSet in theory we could just panic, I think however we can
totally just return error, it will panic in expand anyway.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Utilize errWithWarnings
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Fix recently introduced expansion issue
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Add tests for secondary querier error handling
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Implement lazy merge
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Add name to test cases
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Reorganize
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Address review comments
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Address review comments
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Remove redundant warnings
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
* Fix rebase mistake
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>
Co-authored-by: Bartlomiej Plotka <bwplotka@gmail.com>
* add time range params to labelNames api
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* evaluate min/max time range when reading labels from the head
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* add time range params to labelValues api
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* fix test, add docs
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* add a test for head min max range
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* fix test to match comment
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* address CR comments
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* combine vars only used once
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* add time range params to labelNames api
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* evaluate min/max time range when reading labels from the head
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* add time range params to labelValues api
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* fix test, add docs
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* add a test for head min max range
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* fix test to match comment
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* address CR comments
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* combine vars only used once
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* fix test
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* restart ci
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* use range expectedLabelNames instead of range actualLabelNames in test
Signed-off-by: jessicagreben <Jessica.greben1+github@gmail.com>
* Callbacks for lifecycle of series in TSDB
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* Add more comments
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
* API: Allow AlertmanagerRetriever and RulesRetriever to receive a
Context
Signed-off-by: gotjosh <josue@grafana.com>
* Use single line functions where possible
Signed-off-by: gotjosh <josue@grafana.com>
When appending to the head and a chunk is full it is flushed to the disk and m-mapped (memory mapped) to free up memory
Prom startup now happens in these stages
- Iterate the m-maped chunks from disk and keep a map of series reference to its slice of mmapped chunks.
- Iterate the WAL as usual. Whenever we create a new series, look for it's mmapped chunks in the map created before and add it to that series.
If a head chunk is corrupted the currpted one and all chunks after that are deleted and the data after the corruption is recovered from the existing WAL which means that a corruption in m-mapped files results in NO data loss.
[Mmaped chunks format](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/head_chunks.md) - main difference is that the chunk for mmaping now also includes series reference because there is no index for mapping series to chunks.
[The block chunks](https://github.com/prometheus/prometheus/blob/master/tsdb/docs/format/chunks.md) are accessed from the index which includes the offsets for the chunks in the chunks file - example - chunks of series ID have offsets 200, 500 etc in the chunk files.
In case of mmaped chunks, the offsets are stored in memory and accessed from that. During WAL replay, these offsets are restored by iterating all m-mapped chunks as stated above by matching the series id present in the chunk header and offset of that chunk in that file.
**Prombench results**
_WAL Replay_
1h Wal reply time
30% less wal reply time - 4m31 vs 3m36
2h Wal reply time
20% less wal reply time - 8m16 vs 7m
_Memory During WAL Replay_
High Churn:
10-15% less RAM - 32gb vs 28gb
20% less RAM after compaction 34gb vs 27gb
No Churn:
20-30% less RAM - 23gb vs 18gb
40% less RAM after compaction 32.5gb vs 20gb
Screenshots are in [this comment](https://github.com/prometheus/prometheus/pull/6679#issuecomment-621678932)
Signed-off-by: Ganesh Vernekar <cs15btech11018@iith.ac.in>
https://github.com/facebook/create-react-app/issues/8689 is causing our
tests to fail in the CI pipeline. As the comments suggest, downgrading
to react-scripts 3.4.0 fixes the problem.
In addition, fix a test warning due to a missing id field.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
time.Unix attaches the local timezone, which can then
leak out (e.g. in the alert json). While this is harmless,
we should be consistent.
Signed-off-by: Brian Brazil <brian.brazil@robustperception.io>
* storage: Added Chunks{Queryable/Querier/SeriesSet/Series/Iteratable. Added generic Merge{SeriesSet/Querier} implementation.
## Rationales:
In many places (e.g. chunk Remote read, Thanos Receive fetching chunk from TSDB), we operate on encoded chunks not samples.
This means that we unnecessary decode/encode, wasting CPU, time and memory.
This PR adds chunk iterator interfaces and makes the merge code to be reused between both seriesSets
I will make the use of it in following PR inside tsdb itself. For now fanout implements it and mergers.
All merges now also allows passing series mergers. This opens doors for custom deduplications other than TSDB vertical ones (e.g. offline one we have in Thanos).
## Changes
* Added Chunk versions of all iterating methods. It all starts in Querier/ChunkQuerier. The plan is that
Storage will implement both chunked and samples.
* Added Seek to chunks.Iterator interface for iterating over chunks.
* NewMergeChunkQuerier was added; Both this and NewMergeQuerier are now using generigMergeQuerier to share the code. Generic code was added.
* Improved tests.
* Added some TODO for further simplifications in next PRs.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Addressed Brian's comments.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Moved s/Labeled/SeriesLabels as per Krasi suggestion.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Addressed Krasi's comments.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Second iteration of Krasi comments.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Another round of comments.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
This is technically BREAKING CHANGE, but it was like this from the beginning: I just notice that we rely in
Prometheus on remote read being sorted. This is because we use selected data from remote reads in MergeSeriesSet
which rely on sorting.
I found during work on https://github.com/prometheus/prometheus/pull/5882 that
we do so many repetitions because of this, for not good reason. I think
I found a good balance between convenience and readability with just one method.
Smaller the interface = better.
Also I don't know what TestSelectSorted was testing, but now it's testing sorting.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
This is technically BREAKING CHANGE, but it was like this from the beginning: I just notice that we rely in
Prometheus on remote read being sorted. This is because we use selected data from remote reads in MergeSeriesSet
which rely on sorting.
I found during work on https://github.com/prometheus/prometheus/pull/5882 that
we do so many repetitions because of this, for not good reason. I think
I found a good balance between convenience and readability with just one method.
Smaller the interface = better.
Also I don't know what TestSelectSorted was testing, but now it's testing sorting.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
This fixes an issue where the /new/targets page will not load when there
are jobs with invalid CSS characters in them, such as the
namespace/service/0 form used by the Prometheus Operator.
Signed-off-by: Chris Marchbanks <csmarchbanks@gmail.com>
Previously it could return error if RemoteAddr didn't
have correct format, but since this field has no specified
format, that was little too strict.
Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
This is part of https://github.com/prometheus/prometheus/pull/5882 that can be done to simplify things.
All todos I added will be fixed in follow up PRs.
* querier.Querier, querier.Appender, querier.SeriesSet, and querier.Series interfaces merged
with storage interface.go. All imports that.
* querier.SeriesIterator replaced by chunkenc.Iterator
* Added chunkenc.Iterator.Seek method and tests for xor implementation (?)
* Since we properly handle SelectParams for Select methods I adjusted min max
based on that. This should help in terms of performance for queries with functions like offset.
* added Seek to deletedIterator and test.
* storage/tsdb was removed as it was only a unnecessary glue with incompatible structs.
No logic was changed, only different source of abstractions, so no need for benchmarks.
Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
* Make lookbackDelta a option of QueryEngine
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* julius' suggestion
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* remove trivial getter
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Assume lookback delta is always > 0
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* add debug log
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* don't expose loopback delta
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Specify that lookack delta is also used in federation
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Fix federation test
While we have added some logic to the promql engine to keep it backwards
compatible and have a 5 minute loopback by default, the web/ package is
likely to really be internal to Prometheus and we should not add the
same kind of heuritstics here.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* loopback delta: Fix debug log
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
* Add conditional rendering of Navlink for Consoles
Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>
* Replacing if else with only if conditional rendering
Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>
* Add tests and removing global declaration in Navbar
Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>
* Correct Navbar Testcases and add types for ConsolesLink
Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>
* Change names for Console link as per-naming convention
Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>
* Change prop names to AppProps and NavbarProps respectively
Signed-off-by: Drumil Patel <drumilpatel720@gmail.com>
The function HoldDuration and Duration did the exact same thing.
Let's only keep HoldDuration() as Duration() is more confusing.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
Since we use ActiveQueryTracker to check for concurrency in
d992c36b3a it does not make sense to keep
the MaxConcurrent value as an option of the PromQL engine.
This pull request removes it from the PromQL engine options, sets the
max concurrent metric to -1 if there is no active query tracker, and use
the value of the active query tracker otherwise.
It removes dead code and also will inform people who import the promql
package that we made that change, as it breaks the EngineOpts struct.
Signed-off-by: Julien Pivotto <roidelapluie@inuits.eu>
