Commit graph

7132 commits

Author SHA1 Message Date
Fabian Reinartz 3ed2c2a14b Rename Partition to regular DB, DB to PartitionedDB 2017-01-06 11:40:09 +01:00
Fabian Reinartz 937cdb579c Switch to sequential block names
This changes block directory names from the int64 timestamp
to sequential numbering.
2017-01-06 10:45:03 +01:00
Fabian Reinartz 4590b61343 Rename shard to partition 2017-01-06 08:08:02 +01:00
Brian Brazil f9e581907a Make index queue bigger. (#2322)
When a large Prometheus starts up fresh it can take many minutes
to warmup and clear out the index queue. A larger queue means less
blocking, bigger batches and cuts down startup time by ~50%.
2017-01-05 17:57:42 +00:00
Fabian Reinartz 9790aa98ac Add postings wrapper that emits head postings in label set order
This adds a position mapper that takes series from a head block
in the order they were appended and creates a mapping representing
them in order of their label sets.

Write-repair of the postings list would cause very expensive writing.
Hence, we keep them as they are and only apply the postition mapping
at the very end, after a postings list has been sufficienctly reduced
through intersections etc.
2017-01-05 16:05:42 +01:00
Fabian Reinartz 5aa7f7cce8 Compact head block into persisted block 2017-01-04 21:11:15 +01:00
Fabian Reinartz 3f72d5d027 Fix last timestamp initialization
This initializes the chunkDesc's last timestamp to the minimum
value so initial samples with a timestamp of 0 (e.g. in tests)
are not accidentally dropped.
2017-01-04 14:06:40 +01:00
Fabian Reinartz c9f4aea8e2 Merge pull request #2305 from alicebob/favicon
Add a favicon to the web GUI
2017-01-04 10:15:27 +01:00
Fabian Reinartz 40cf215fba Hotfix missing sorting of head block series index 2017-01-04 09:47:20 +01:00
Martin Lehmann 78fae3155f Make relative links in README.md absolute (#2316)
The relative links don't work in other pages that render the README (for example https://hub.docker.com/r/prom/prometheus/). As they are (hopefully) not due to change any time soon, I think using absolute links is better.
2017-01-03 20:07:33 +00:00
Fabian Reinartz ec99f99d3d Fix and test bug in shardSeriesSet 2017-01-03 19:02:42 +01:00
Fabian Reinartz ac49f8c15e Consolidate persistence and compaction 2017-01-03 16:49:37 +01:00
Fabian Reinartz e7f04d14d5 Lock mmapped files 2017-01-03 10:09:20 +01:00
Fabian Reinartz 91b65b55e7 Run persistence in separate goroutine 2017-01-02 22:24:35 +01:00
Fabian Reinartz a648ef5252 Convert persister into function 2017-01-02 16:58:47 +01:00
Fabian Reinartz dbd2b21d2e Make persistence atomic 2017-01-02 14:41:13 +01:00
Fabian Reinartz bc20d93f0a storage: rename iterator value getters to At() 2017-01-02 13:33:37 +01:00
Fabian Reinartz 62b8ded0a5 Rename iterator value getters to At() 2017-01-02 13:33:06 +01:00
Fabian Reinartz 5c45a1cc6f Remove shardSeriesSet caching 2017-01-02 12:05:52 +01:00
Fabian Reinartz beb842a856 Change block interface
This changes the block interface to directly expose index
and series readers rather than the direct querier
2017-01-02 11:12:28 +01:00
Fabian Reinartz 5d75a3dc7b Add basic compaction
This adds a basic compactor that will merge two persisted blocks into
one. It simply fully rewrites the index and concatenates the chunk
lists.

It just writes into the current working dir and doesn't properly handle
which blocks to compact for now.
2017-01-02 10:34:55 +01:00
Fabian Reinartz 5d7ec06e04 Use ChunkMeta in SeriesWriter 2017-01-01 20:01:17 +01:00
Fabian Reinartz c00d17e691 Modify IndexReader API to accomodate compaction
This changes the IndexReader API to expose plain labels
and chunk meta information instead of a Series interface.
Dropping of irrelevant chunks is moved into the querier.

A LabelIndices method is added to query for existing label
value indices.
2016-12-31 16:24:04 +01:00
Julius Volz 90dd216646 Merge pull request #2306 from EdSchouten/sorted-alerts
Use lexicographic order to sort alerts by name.
2016-12-31 13:12:30 +01:00
Fabian Reinartz 1e1a37b15b Remove double-reference in chunk hashmap 2016-12-31 10:19:02 +01:00
Fabian Reinartz 675f0886f0 Append to chunks cannot error 2016-12-31 10:10:27 +01:00
Fabian Reinartz 7280533c42 Add basic shard metrics 2016-12-31 09:48:49 +01:00
Fabian Reinartz 89b6b3be9f vendor: remove unused dependencies 2016-12-31 09:32:55 +01:00
Fabian Reinartz e631a1260d retrieval: use separate appender per target 2016-12-30 21:35:35 +01:00
Fabian Reinartz a009247ab7 Add matching of empty label 2016-12-30 19:36:28 +01:00
Fabian Reinartz 61bd698143 web: implement federation for new storage 2016-12-30 19:34:45 +01:00
Fabian Reinartz 7322c46b8e storage: add mock iterator for test 2016-12-30 10:45:56 +01:00
Fabian Reinartz 28f547bcc7 api/v1: fix tests, restore series queries 2016-12-30 10:43:44 +01:00
Fabian Reinartz e94b0899ee rules: fix tests, remove model types 2016-12-29 17:31:14 +01:00
Fabian Reinartz 68dc358496 cmd/prometheus: remove tests for old flags 2016-12-29 16:55:22 +01:00
Fabian Reinartz 8b4e4a9d2b notifier: fully use labels.Labels 2016-12-29 16:53:11 +01:00
Fabian Reinartz f8fc1f5bb2 *: migrate ingestion to new batch Appender 2016-12-29 11:03:56 +01:00
Fabian Reinartz eb4f366988 Add return error to Appender.Add 2016-12-29 11:03:39 +01:00
Fabian Reinartz 86cb0f30fd pkg/relabel: add relabel pkg using new labels 2016-12-28 19:04:59 +01:00
Fabian Reinartz 3a5468f251 Lock shards during querying and shutdown 2016-12-28 11:41:44 +01:00
Fabian Reinartz 0987a72ec9 pkg/timestamp: create timestamp package 2016-12-28 11:33:00 +01:00
Fabian Reinartz 71fe0c58a8 promql: misc fixes 2016-12-28 11:32:15 +01:00
Fabian Reinartz 286293802b Add merge postings 2016-12-28 11:02:19 +01:00
Fabian Reinartz ab7fbc05ad Zero timestamp as base, use binary search list postings 2016-12-28 08:50:20 +01:00
Mitsuhiro Tanda 7e369b9318 expose max memory chunks metrics (#2303)
* expose max memory chunks metrics
2016-12-27 18:34:07 +00:00
Ed Schouten b3a39ccd8a Use lexicographic order to sort alerts by name.
Right now the /alerts page of Prometheus sorts alerts by severity
(firing, pending, inactive). Once multiple alerts have the same
severity, their order seems to correlate to how they are placed in the
configuration files, but not always. Looking at the code, we make use of
sort.Sort(), which is documented not to provide a stable sort. The
Less() function also only takes the alert state into account.

This change extends the Less() function to provide a lexicographic order
on both the alert state and the name. This means I can finally find the
alerts I'm looking for without using my browser's search feature.
2016-12-27 14:28:44 +01:00
Harmen 135d32ea22 make assets 2016-12-27 13:59:20 +01:00
Harmen dfa4f79bcd add favicon 2016-12-27 13:58:51 +01:00
Fabian Reinartz dd72b52098 Add postings intersection 2016-12-27 11:32:10 +01:00
Fabian Reinartz f8111cef0e Fix chunk series iterator seeking 2016-12-26 16:55:44 +01:00