Commit graph

9 commits

Author SHA1 Message Date
Iván Ovejero f53c482939
perf(core): Improve caching service (#8213)
Story: https://linear.app/n8n/issue/PAY-1188

- Implement Redis hashes on the caching service, based on Micha's work
in #7747, adapted from `node-cache-manager-ioredis-yet`. Optimize
workflow ownership lookups and manual webhook lookups with Redis hashes.
- Simplify the caching service by removing all currently unused methods
and options: `enable`, `disable`, `getCache`, `keys`, `keyValues`,
`refreshFunctionEach`, `refreshFunctionMany`, `refreshTtl`, etc.
- Remove the flag `N8N_CACHE_ENABLED`. Currently some features on
`master` are broken with caching disabled, and test webhooks now rely
entirely on caching, for multi-main setup support. We originally
introduced this flag to protect against excessive memory usage, but
total cache usage is low enough that we decided to drop this setting.
Apparently this flag was also never documented.
- Overall caching service refactor: use generics, reduce branching, add
discriminants for cache kinds for better type safety, type caching
events, improve readability, remove outdated docs, etc. Also refactor
and expand caching service tests.

Follow-up to: https://github.com/n8n-io/n8n/pull/8176

---------

Co-authored-by: Michael Auerswald <michael.auerswald@gmail.com>
2024-01-05 11:52:44 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5edf722209
ci: Stop disabling linting rules that are explicitly set to warn (no-changelog) (#7669) 2023-11-09 14:05:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 05586a900d
refactor(core): Make Logger a service (no-changelog) (#7494) 2023-10-25 16:35:22 +02:00
Michael Auerswald 6d7fe95c58
fix(core): Do not set ttl globally in test (no-changelog) (#7271) 2023-09-27 12:03:15 +02:00
Michael Auerswald adcf5a96e8
feat(core): Add metrics option to cache (#6846)
* add metrics to cache

* use events for metrics

* pr comments / broken test

* lint fix

* update the test

* improve tests

* Update packages/cli/src/config/schema.ts

* disable flaky test

* lint fix

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
2023-08-04 20:51:07 +02:00
Michael Auerswald fdfc6c5a92
fix(core): Handle empty keys in cache service (no-changelog) (#6854)
* fix handle empty keys in cache service

* add test

* add cache mock test

* add simpler mocking, and add tests for all the updated methods

* don't use RedisStore specifically in the mock

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-08-04 19:44:41 +02:00
Michael Auerswald 659ca26fe7
fix(core): Change VariablesService to DI and use caching (#6827)
* support redis cluster

* cleanup, fix config schema

* set default prefix to bull

* initial commit

* improve logging

* improve types and refactor

* list support and refactor

* fix redis service and tests

* add comment

* add redis and cache prefix

* use injection

* lint fix

* clean schema comments

* improve naming, tests, cluster client

* merge master

* cache returns unknown instead of T

* update cache service, tests and doc

* remove console.log

* VariablesService as DI, add caching, fix tests

* do not cache null or undefined values

* import fix

* more DI and remove collections

* fix merge

* lint fix

* rename to ~Cached

* fix test for CI

* fix ActiveWorkflowRunner test
2023-08-02 14:51:09 +02:00
Michael Auerswald 3cad60e918
feat(core): Make Redis available for backend communication (#6719)
* support redis cluster

* cleanup, fix config schema

* set default prefix to bull

* initial commit

* improve logging

* improve types and refactor

* list support and refactor

* fix redis service and tests

* add comment

* add redis and cache prefix

* use injection

* lint fix

* clean schema comments

* improve naming, tests, cluster client

* merge master

* cache returns unknown instead of T

* update cache service, tests and doc

* remove console.log

* do not cache null or undefined values

* fix merge

* lint fix
2023-08-02 12:51:25 +02:00
Michael Auerswald c0d2bac94d
feat(core): Add cache service (#6729)
* add cache service

* PR adjustments

* switch to maxSize for memory cache
2023-07-25 11:56:38 +02:00