Commit graph

38 commits

Author SHA1 Message Date
कारतोफ्फेलस्क्रिप्ट™ 25c8a328a8
refactor(core): Move instanceType to InstanceSettings (no-changelog) (#10640)
Some checks are pending
Test Master / install-and-build (push) Waiting to run
Test Master / Unit tests (18.x) (push) Blocked by required conditions
Test Master / Unit tests (20.x) (push) Blocked by required conditions
Test Master / Unit tests (22.4) (push) Blocked by required conditions
Test Master / Lint (push) Blocked by required conditions
Test Master / Notify Slack on failure (push) Blocked by required conditions
Benchmark Docker Image CI / build (push) Waiting to run
2024-09-16 13:37:14 +02:00
Tomi Turtiainen 5156313074
refactor(core): Enable import/order eslint rule (#10794)
Some checks are pending
Test Master / install-and-build (push) Waiting to run
Test Master / Unit tests (18.x) (push) Blocked by required conditions
Test Master / Unit tests (20.x) (push) Blocked by required conditions
Test Master / Unit tests (22.4) (push) Blocked by required conditions
Test Master / Lint (push) Blocked by required conditions
Test Master / Notify Slack on failure (push) Blocked by required conditions
Benchmark Docker Image CI / build (push) Waiting to run
2024-09-12 19:07:18 +03:00
Iván Ovejero f667b384c9
refactor(core): Standardize filenames in cli (no-changelog) (#10484)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-08-22 11:10:37 +02:00
कारतोफ्फेलस्क्रिप्ट™ 1869c396f3
fix(core): Do not load ScalingService in regular mode (no-changelog) (#10333)
Some checks are pending
Test Master / install-and-build (push) Waiting to run
Test Master / Unit tests (18.x) (push) Blocked by required conditions
Test Master / Unit tests (20.x) (push) Blocked by required conditions
Test Master / Unit tests (22.4) (push) Blocked by required conditions
Test Master / Lint (push) Blocked by required conditions
Test Master / Notify Slack on failure (push) Blocked by required conditions
2024-08-08 17:35:17 +02:00
Iván Ovejero e31d017bdd
refactor(core): Centralize scaling mode (no-changelog) (#9835) 2024-08-07 13:50:46 +02:00
Tomi Turtiainen c8d322a9ba
refactor(core): Reorganize webhook related components under src/webhooks (no-changelog) (#10296)
Some checks are pending
Test Master / install-and-build (push) Waiting to run
Test Master / Unit tests (18.x) (push) Blocked by required conditions
Test Master / Unit tests (20.x) (push) Blocked by required conditions
Test Master / Unit tests (22.4) (push) Blocked by required conditions
Test Master / Lint (push) Blocked by required conditions
Test Master / Notify Slack on failure (push) Blocked by required conditions
2024-08-07 11:23:44 +03:00
कारतोफ्फेलस्क्रिप्ट™ 88086a41ff
feat(core): Support community packages in scaling-mode (#10228) 2024-08-05 11:52:06 +02:00
कारतोफ्फेलस्क्रिप्ट™ c04f92f7fd
refactor(core): Couple of refactors on WorkflowRunner and ActiveExecutions (no-changelog) (#8487) 2024-02-06 18:09:50 +01:00
Iván Ovejero 28aee7c425
refactor(core): Lock webhook process out of multi-main setup (no-changelog) (#8549) 2024-02-05 18:06:54 +01:00
कारतोफ्फेलस्क्रिप्ट™ 913c8c6b0c
feat(core): Upgrade oclif (no-changelog) (#8381) 2024-01-22 18:25:36 +01:00
Tomi Turtiainen 3a881be6c2
feat(core): Unify application components shutdown (#8097)
## Summary

Add `ShutdownService` and `OnShutdown` decorator for more unified way to
shutdown different components. Use this new way in the following
components:

- HTTP(S) server
- Pruning service
- Push connection
- License

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-22 12:39:58 +02:00
Tomi Turtiainen 4cae976a3b
fix(core): Consider timeout in shutdown an error (#8050)
If the process doesn't shutdown within a time limit, exit with error
code.

1. conceptually something timing out is an error.
2. on successful exit we close down the DB connection gracefully. On an
exit timeout we rather not do that, since it will wait for any active
connections to close and would possible block the exit.
2023-12-18 10:53:34 +02:00
Tomi Turtiainen 6ae2f5efea
fix(core): Fix shutdown if terminating before hooks are initialized (#8047)
If the app receives termination signal before hooks have been
initialised, the would be objet is undefined error. This PR fixes that.
2023-12-18 09:23:10 +02:00
Csaba Tuncsik 152883eed1
build: Upgrade lint related packages (#7790) 2023-11-27 09:11:52 +01:00
कारतोफ्फेलस्क्रिप्ट™ 05586a900d
refactor(core): Make Logger a service (no-changelog) (#7494) 2023-10-25 16:35:22 +02:00
Michael Auerswald afa683a06f
refactor(core): Have one orchestration service per instance type (#7303)
webhook instances will not listen to either worker or event log messages
on the Redis pub/sub channel
2023-10-06 13:58:11 +02:00
Omar Ajoue 5cfd87692f
refactor: Add log messages to debug worker init issues (#7238)
Github issue / Community forum post (link here to close automatically):
2023-10-05 13:37:25 +02:00
Michael Auerswald 4b014286cf
fix(core): Make senderId required for all command messages (#7252)
all commands sent between main instance and workers need to contain a
server id to prevent senders from reacting to their own messages,
causing loops

this PR makes sure all sent messages contain a sender id by default as
part of constructing a sending redis client.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-26 13:58:06 +02:00
Iván Ovejero 6d6e2488c6
refactor(core): Generalize binary data manager interface (no-changelog) (#7164)
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)

This PR: 
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.

Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.

Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-22 17:22:12 +02:00
Michael Auerswald 9f797b96d8
feat(core): Add command to trigger license refresh on workers (#7184)
This PR implements the updated license SDK so that worker and webhook
instances do not auto-renew licenses any more.

Instead, they receive a `reloadLicense` command via the Redis client
that will fetch the updated license after it was saved on the main
instance

This also contains some refactoring with moving redis sub and pub
clients into the event bus directly, to prevent cyclic dependency
issues.
2023-09-17 11:05:54 +02:00
Alex Grozav ed927d34b2
feat: External Secrets storage for credentials (#6477)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-08-25 10:33:46 +02:00
Michael Auerswald 6499f42481
feat(core): Add unique id to instances (#6863)
* add serverId and workerId

* adjust id format

* fix logger init / test

* update implementation
2023-08-07 17:03:21 +02:00
Iván Ovejero 72523462ea
refactor: Clear unused ESLint directives from BE packages (no-changelog) (#6798) 2023-07-31 11:00:48 +02:00
Omar Ajoue 42c79cd6f1
fix: Initialize license in queue mode correctly (#6301) 2023-05-23 13:01:50 +02:00
कारतोफ्फेलस्क्रिप्ट™ 9e7b9fb443
feat(core): Improve health check (#6205)
* remove unnecesary Db re-initialization

this is from before we added `Db.init` in `WorkflowRunnerProcess`

* feat(core): Improved health check

* make health check not care about DB connections

* close DB connections, and shutdown the timer
2023-05-10 10:27:04 +02:00
कारतोफ्फेलस्क्रिप्ट™ e67ad29625
fix(core): Initialize queue in the webhook server as well (#5766)
fix(core): Initialize queue in the webhook server as well (no-changelog)
2023-03-24 10:47:21 +01:00
कारतोफ्फेलस्क्रिप्ट™ 52f740b9e8
refactor(core): Use an IoC container to manage singleton classes [Part-1] (no-changelog) (#5509)
* add typedi

* convert ActiveWorkflowRunner into an injectable service

* convert ExternalHooks into an injectable service

* convert InternalHooks into an injectable service

* convert LoadNodesAndCredentials into an injectable service

* convert NodeTypes and CredentialTypes into an injectable service

* convert ActiveExecutions into an injectable service

* convert WaitTracker into an injectable service

* convert Push into an injectable service

* convert ActiveWebhooks and  TestWebhooks into an injectable services

* handle circular references, and log errors when a circular dependency is found
2023-02-21 19:21:56 +01:00
Jan Oberhauser 339655611f
fix(core): Fix issue that worker and webhook service close directly (#5461) 2023-02-11 09:46:10 -06:00
कारतोफ्फेलस्क्रिप्ट™ 9a331ec7b6
fix(core): Stop using unbound calls to stopProcess (no-changelog) (#5456) 2023-02-10 16:34:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5194513850
refactor(core): Delete duplicate code across all commands (#5452) 2023-02-10 14:59:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 72249e0de8
refactor(core): Load and validate all config at startup (no-changelog) (#5283) 2023-01-30 14:42:30 +01:00
कारतोफ्फेलस्क्रिप्ट™ a573db2ef7
fix: Report app startup and DB migration errors to Sentry (#5127) 2023-01-11 18:28:35 +01:00
कारतोफ्फेलस्क्रिप्ट™ 8b19fdd5f0
refactor: Delete a lot of unused and duplicate code in Server and WebhookServer (#5080)
* store n8n version string in a const and use that everywhere

* reduce code duplication between Server and WebhookServer

* unify redis checks

* fix linting
2023-01-04 11:38:48 +01:00
Thomas S 0ec66bfb42
feat(core): Add compatibility to redis > 6 ACLs system using username in queue-mode (#5048)
feat(redis): add compatibility to redis > 6 ACLs system using username

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-03 11:45:03 +01:00
कारतोफ्फेलस्क्रिप्ट™ b828cb31d6
perf: Lazy-load queue-mode and analytics dependencies (#5061)
* refactor: lazy load ioredis and bull

* upgrade bull and hiredis

* refactor: lazy load posthog, rudderstack, and sentry

* upgrade Sentry sdk
2023-01-02 12:14:39 +01:00
Iván Ovejero d9b98fc8be
refactor: Lint for no unneeded backticks (#5057) (no-changelog)
*  Create rule `no-unneeded-backticks`

* 👕 Enable rule

*  Run rule on `cli`

*  Run rule on `core`

*  Run rule on `workflow`

*  Rule rule on `design-system`

*  Run rule on `node-dev`

*  Run rule on `editor-ui`

*  Run rule on `nodes-base`
2022-12-29 12:20:43 +01:00
कारतोफ्फेलस्क्रिप्ट™ b6c57e19fc
feat(core): Lazy-load nodes and credentials to reduce baseline memory usage (#4577) 2022-11-23 16:20:28 +01:00
कारतोफ्फेलस्क्रिप्ट™ 698d96a617
refactor: Setup typescript project references across workflow, core, and cli (#4519)
* refactor: use consistent folder structure across workflow, core, and cli

* setup typescript project references across workflow, core, and cli
2022-11-09 15:25:00 +01:00
Renamed from packages/cli/commands/webhook.ts (Browse further)