Commit graph

22 commits

Author SHA1 Message Date
Omar Ajoue 5cfd87692f
refactor: Add log messages to debug worker init issues (#7238)
Github issue / Community forum post (link here to close automatically):
2023-10-05 13:37:25 +02:00
Michael Auerswald 4b014286cf
fix(core): Make senderId required for all command messages (#7252)
all commands sent between main instance and workers need to contain a
server id to prevent senders from reacting to their own messages,
causing loops

this PR makes sure all sent messages contain a sender id by default as
part of constructing a sending redis client.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-26 13:58:06 +02:00
Iván Ovejero 6d6e2488c6
refactor(core): Generalize binary data manager interface (no-changelog) (#7164)
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)

This PR: 
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.

Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.

Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-22 17:22:12 +02:00
Michael Auerswald 9f797b96d8
feat(core): Add command to trigger license refresh on workers (#7184)
This PR implements the updated license SDK so that worker and webhook
instances do not auto-renew licenses any more.

Instead, they receive a `reloadLicense` command via the Redis client
that will fetch the updated license after it was saved on the main
instance

This also contains some refactoring with moving redis sub and pub
clients into the event bus directly, to prevent cyclic dependency
issues.
2023-09-17 11:05:54 +02:00
Alex Grozav ed927d34b2
feat: External Secrets storage for credentials (#6477)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-08-25 10:33:46 +02:00
Michael Auerswald 6499f42481
feat(core): Add unique id to instances (#6863)
* add serverId and workerId

* adjust id format

* fix logger init / test

* update implementation
2023-08-07 17:03:21 +02:00
Iván Ovejero 72523462ea
refactor: Clear unused ESLint directives from BE packages (no-changelog) (#6798) 2023-07-31 11:00:48 +02:00
Omar Ajoue 42c79cd6f1
fix: Initialize license in queue mode correctly (#6301) 2023-05-23 13:01:50 +02:00
कारतोफ्फेलस्क्रिप्ट™ 9e7b9fb443
feat(core): Improve health check (#6205)
* remove unnecesary Db re-initialization

this is from before we added `Db.init` in `WorkflowRunnerProcess`

* feat(core): Improved health check

* make health check not care about DB connections

* close DB connections, and shutdown the timer
2023-05-10 10:27:04 +02:00
कारतोफ्फेलस्क्रिप्ट™ e67ad29625
fix(core): Initialize queue in the webhook server as well (#5766)
fix(core): Initialize queue in the webhook server as well (no-changelog)
2023-03-24 10:47:21 +01:00
कारतोफ्फेलस्क्रिप्ट™ 52f740b9e8
refactor(core): Use an IoC container to manage singleton classes [Part-1] (no-changelog) (#5509)
* add typedi

* convert ActiveWorkflowRunner into an injectable service

* convert ExternalHooks into an injectable service

* convert InternalHooks into an injectable service

* convert LoadNodesAndCredentials into an injectable service

* convert NodeTypes and CredentialTypes into an injectable service

* convert ActiveExecutions into an injectable service

* convert WaitTracker into an injectable service

* convert Push into an injectable service

* convert ActiveWebhooks and  TestWebhooks into an injectable services

* handle circular references, and log errors when a circular dependency is found
2023-02-21 19:21:56 +01:00
Jan Oberhauser 339655611f
fix(core): Fix issue that worker and webhook service close directly (#5461) 2023-02-11 09:46:10 -06:00
कारतोफ्फेलस्क्रिप्ट™ 9a331ec7b6
fix(core): Stop using unbound calls to stopProcess (no-changelog) (#5456) 2023-02-10 16:34:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5194513850
refactor(core): Delete duplicate code across all commands (#5452) 2023-02-10 14:59:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 72249e0de8
refactor(core): Load and validate all config at startup (no-changelog) (#5283) 2023-01-30 14:42:30 +01:00
कारतोफ्फेलस्क्रिप्ट™ a573db2ef7
fix: Report app startup and DB migration errors to Sentry (#5127) 2023-01-11 18:28:35 +01:00
कारतोफ्फेलस्क्रिप्ट™ 8b19fdd5f0
refactor: Delete a lot of unused and duplicate code in Server and WebhookServer (#5080)
* store n8n version string in a const and use that everywhere

* reduce code duplication between Server and WebhookServer

* unify redis checks

* fix linting
2023-01-04 11:38:48 +01:00
Thomas S 0ec66bfb42
feat(core): Add compatibility to redis > 6 ACLs system using username in queue-mode (#5048)
feat(redis): add compatibility to redis > 6 ACLs system using username

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-03 11:45:03 +01:00
कारतोफ्फेलस्क्रिप्ट™ b828cb31d6
perf: Lazy-load queue-mode and analytics dependencies (#5061)
* refactor: lazy load ioredis and bull

* upgrade bull and hiredis

* refactor: lazy load posthog, rudderstack, and sentry

* upgrade Sentry sdk
2023-01-02 12:14:39 +01:00
Iván Ovejero d9b98fc8be
refactor: Lint for no unneeded backticks (#5057) (no-changelog)
*  Create rule `no-unneeded-backticks`

* 👕 Enable rule

*  Run rule on `cli`

*  Run rule on `core`

*  Run rule on `workflow`

*  Rule rule on `design-system`

*  Run rule on `node-dev`

*  Run rule on `editor-ui`

*  Run rule on `nodes-base`
2022-12-29 12:20:43 +01:00
कारतोफ्फेलस्क्रिप्ट™ b6c57e19fc
feat(core): Lazy-load nodes and credentials to reduce baseline memory usage (#4577) 2022-11-23 16:20:28 +01:00
कारतोफ्फेलस्क्रिप्ट™ 698d96a617
refactor: Setup typescript project references across workflow, core, and cli (#4519)
* refactor: use consistent folder structure across workflow, core, and cli

* setup typescript project references across workflow, core, and cli
2022-11-09 15:25:00 +01:00
Renamed from packages/cli/commands/webhook.ts (Browse further)