mudhorn/n8n
1
0
Fork 0
mirror of https://github.com/n8n-io/n8n.git synced 2025-03-05 20:50:17 -08:00
Code Issues Actions Packages Projects Releases Wiki Activity
12106 commits 1276 branches 1649 tags 365 MiB
Commit graph

270 commits

Author SHA1 Message Date
Iván Ovejero 95b85dd5c1
refactor(core): Port nodes config (no-changelog) (#10140) 
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
 2024-07-23 13:32:50 +02:00
Iván Ovejero aba1c64500
refactor(core): Rename EventRelay to EventService (no-changelog) (#10110) 2024-07-19 12:55:38 +02:00
Iván Ovejero dd54390b0a
refactor(core): Remove unused security audit telemetry event (no-changelog) (#10105) 2024-07-19 11:16:05 +02:00
Iván Ovejero 5eca7c8e28
refactor(core): More decouplings from internal hooks (no-changelog) (#10099) 2024-07-19 10:33:13 +02:00
Iván Ovejero 8b2f76b92e
refactor(core): Decouple projects telemetry (no-changelog) (#10081) 2024-07-17 11:56:27 +02:00
कारतोफ्फेलस्क्रिप्ट™ 3bbeae47f3
refactor(core): Suppress MaxListenersExceededWarning in the logs (#10077) 2024-07-16 18:20:55 +02:00
कारतोफ्फेलस्क्रिप्ट™ c7d4b471c4
refactor(core): Move backend config to a separate package (no-changelog) (#9325) 2024-07-05 11:43:27 +02:00
Iván Ovejero c82579bf76
fix(core): Disconnect Redis after pausing queue during worker shutdown (#9928) 
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
 2024-07-04 18:07:47 +02:00
Iván Ovejero 7c53433191
fix(core): Fix worker logs relay (#9919) 2024-07-03 10:29:25 +02:00
Iván Ovejero 7044d1ca28
fix(core): Filter out certain executions from crash recovery (#9904) 
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
 2024-07-02 17:07:07 +02:00
कारतोफ्फेलस्क्रिप्ट™ 2885091ced
feat(core): Upgrade oclif (no-changelog) (#9881) 2024-06-27 17:26:19 +02:00
Cornelius Suermann 8a09806c5a
feat(core): Log a warning if automatic license renewal is disabled (no-changelog) (#9569) 
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
 2024-06-26 09:00:07 +02:00
Iván Ovejero e33a47311f
refactor: Upgrade to TypeScript 5.5 (no-changelog) (#9828) 2024-06-24 17:49:59 +02:00
Iván Ovejero 44ea4c73eb
refactor: Remove unused lint directives (no-changelog) (#9842) 2024-06-24 12:13:18 +02:00
Iván Ovejero 8e529219df
refactor(core): Lint to restrict @n8n/typeorm to persistence layer (no-changelog) (#9840) 2024-06-24 10:24:05 +02:00
कारतोफ्फेलस्क्रिप्ट™ 65c5609ab5
feat(core): Use WebCrypto to generate all random numbers and strings (#9786) 2024-06-19 13:33:57 +02:00
Iván Ovejero c58621ab79
feat(core): Expand crash recovery to cover queue mode (#9676) 2024-06-18 15:22:02 +02:00
कारतोफ्फेलस्क्रिप्ट™ 2d02c73fbd
refactor(core): Refactor cli command tests (no-changelog) (#9731) 2024-06-18 10:50:39 +02:00
Iván Ovejero f7352b6a8f
refactor(core): Make PruningService.init and WaitTracker.init consistent (no-changelog) (#9761) 2024-06-17 12:49:40 +02:00
Iván Ovejero 797342343f
perf(core): Introduce concurrency control for main mode (#9453) 2024-06-12 15:05:43 +02:00
Iván Ovejero aaa78435b0
refactor(core): Remove event bus channel (no-changelog) (#9663) 2024-06-11 10:02:23 +02:00
Cornelius Suermann 44ecab73d7
feat: Release floating entitlements on license:clear command (no-changelog) (#9603) 
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
 2024-06-07 13:13:31 +02:00
Iván Ovejero 4b345bec03
fix(core): Allow graceful shutdown for main with active executions (#9661) 2024-06-07 10:58:26 +02:00
कारतोफ्फेलस्क्रिप्ट™ 08d9c9a787
fix(core): Start WaitTracker only in the main container (#9600) 2024-06-03 17:43:24 +02:00
कारतोफ्फेलस्क्रिप्ट™ 3298914bc4
refactor(core): Remove dead code that disabled auto-generated ids during import (no-changelog) (#9601) 2024-06-03 16:12:31 +02:00
Iván Ovejero 7fc00d8d10
fix(core): Ensure graceful shutdown for workers (#9547) 2024-05-31 15:41:00 +02:00
कारतोफ्फेलस्क्रिप्ट™ d361b42c70
fix(editor): Replace more variants of BASE_PATH in static assets (#9564) 2024-05-31 14:32:20 +02:00
Danny Martini 8da0d6e9ba
fix(core): Fix type errors (no-changelog) (#9571) 2024-05-31 14:06:13 +02:00
कारतोफ्फेलस्क्रिप्ट™ 5887ed6498
refactor(core): Extract all Auth-related User columns into a separate entity (#9557) 
Co-authored-by: Ricardo Espinoza <ricardo@n8n.io>
 2024-05-31 09:40:19 +02:00
कारतोफ्फेलस्क्रिप्ट™ e55bf0393a
fix(core): Try setting postgres search_path on the database (#9530) 2024-05-30 12:09:49 +02:00
Iván Ovejero 8164ca2398
fix(core): Fix worker encryption key warning docs link (no-changelog) (#9513) 2024-05-27 11:50:57 +02:00
Danny Martini 3b93aae6dc
feat(core): Print the name of the migration that cannot be reverted when using n8n db:revert (#9473) 2024-05-23 18:16:26 +02:00
कारतोफ्फेलस्क्रिप्ट™ 3094f1b886
fix(core): Detect DB connection aquisition deadlocks (no-changelog) (#9485) 
Co-authored-by: Danny Martini <danny@n8n.io>
 2024-05-22 14:53:23 +02:00
Csaba Tuncsik 596c472ecc
feat: RBAC (#8922) 
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Val <68596159+valya@users.noreply.github.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Danny Martini <danny@n8n.io>
Co-authored-by: Danny Martini <despair.blue@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: oleg <me@olegivaniv.com>
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Michael Kret <88898367+michael-radency@users.noreply.github.com>
Co-authored-by: Elias Meire <elias@meire.dev>
Co-authored-by: Giulio Andreini <andreini@netseven.it>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Ayato Hayashi <go12limchangyong@gmail.com>
 2024-05-17 10:53:15 +02:00
Iván Ovejero 7b925ab871
refactor(core): Rename ActiveWorkflowRunner to ActiveWorkflowManager (no-changelog) (#9280) 2024-05-06 17:54:05 +02:00
Iván Ovejero 22b6f90950
fix(core): Prevent occassional 429s on license init in multi-main setup (#9284) 2024-05-06 09:04:16 +02:00
Iván Ovejero 7bda92cc7c
refactor(core): Use logger for packages/cli messages (no-changelog) (#9302) 2024-05-03 15:24:27 +02:00
Danny Martini 3eb5be5f5a
fix(core): Don't create multiple owners when importing credentials or workflows (#9112) 2024-04-12 17:25:59 +02:00
Danny Martini 3bb821f10e
fix(core): Don't revert irreversibble migrations (#9105) 2024-04-11 09:20:48 +02:00
Iván Ovejero a3b59843d5
refactor(core): Lint for cyclomatic complexity in BE packages (no-changelog) (#9111) 2024-04-10 14:02:02 +02:00
Iván Ovejero df56153f8b
refactor(core): Better UX for maxStalledCount queue mode error (#9075) 2024-04-05 20:03:49 +02:00
Iván Ovejero 4052b6c073
refactor(core)!: Remove legacy --file flag for execute CLI command (#9054) 2024-04-05 14:28:04 +02:00
Iván Ovejero b8ab049932
refactor(core, editor): Remove legacy nodesAccess (no-changelog) (#9016) 2024-04-05 13:17:34 +02:00
Iván Ovejero 217b07d735
fix(core): Ensure only leader handles waiting executions (#9014) 2024-04-04 13:28:20 +02:00
Jan Oberhauser a29b41ec55
fix(core): Fix pairedItem issue with partial manual executions (#8575) 
Co-authored-by: Danny Martini <danny@n8n.io>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
 2024-02-23 11:43:08 +01:00
कारतोफ्फेलस्क्रिप्ट™ 059d281fd1
feat(core): Move execution permission checks earlier in the lifecycle (#8677) 2024-02-21 14:47:02 +01:00
कारतोफ्फेलस्क्रिप्ट™ b6c8a0c413
refactor(core): Reduce code duplication in DB config (no-changelog) (#8679) 2024-02-20 14:28:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ a2a3ca160f
fix(core): Improve the startup error when EXECUTIONS_PROCESS is set (#8630) 
Co-authored-by: Omar Ajoue <krynble@gmail.com>
 2024-02-14 15:35:42 +01:00
कारतोफ्फेलस्क्रिप्ट™ 8e392cfc1d
feat(core): Migrate to n8n's typeorm fork (#8590) 2024-02-08 15:13:29 +01:00
कारतोफ्फेलस्क्रिप्ट™ c04f92f7fd
refactor(core): Couple of refactors on WorkflowRunner and ActiveExecutions (no-changelog) (#8487) 2024-02-06 18:09:50 +01:00
Iván Ovejero c4e39451db
refactor(core): Continue breaking dependency cycles (no-changelog) (#8545) 2024-02-06 10:08:46 +01:00
Iván Ovejero 28aee7c425
refactor(core): Lock webhook process out of multi-main setup (no-changelog) (#8549) 2024-02-05 18:06:54 +01:00
Iván Ovejero dc5ec8f946
refactor(core): Streamline flows in multi-main mode (no-changelog) (#8446) 2024-02-05 09:26:55 +01:00
Iván Ovejero cc41fc7c80
fix(core): Fix new graceful shutdown env being always overridden by deprecated env (#8503) 2024-02-01 11:10:40 +01:00
कारतोफ्फेलस्क्रिप्ट™ 121a55b691
feat(core): Remove own execution-process mode (#8490) 2024-01-30 12:51:40 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5cb55270b7
refactor(core): Replace promisify-d node calls with native promises (no-changelog) (#8464) 2024-01-30 09:49:23 +01:00
कारतोफ्फेलस्क्रिप्ट™ c70fa66e76
refactor(core): Use DI for WorkflowRunner (no-changelog) (#8372) 2024-01-26 13:49:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 7c49004018
refactor(core): Use DI for eventBus code - Part 1 (no-changelog) (#8434) 2024-01-26 12:21:15 +01:00
कारतोफ्फेलस्क्रिप्ट™ cc2f0ada76
ci: Enforce noUnusedLocals on all backend and nodes packages (no-changelog) (#8428) 2024-01-25 14:33:35 +01:00
कारतोफ्फेलस्क्रिप्ट™ d6deceacde
refactor(core): Remove roleId indirection (no-changelog) (#8413) 2024-01-24 13:38:57 +01:00
Danny Martini ae06fdeb62
fix(core): Fix update workflow cli command being unable to activate all workflows (#8412) 
Co-authored-by: Daniel Schröder <daniel.schroeder@skriptfabrik.com>
 2024-01-23 10:59:06 +01:00
कारतोफ्फेलस्क्रिप्ट™ 913c8c6b0c
feat(core): Upgrade oclif (no-changelog) (#8381) 2024-01-22 18:25:36 +01:00
Iván Ovejero f35d4fcbd8
refactor(core): Simplify OrchestrationService (no-changelog) (#8364) 2024-01-22 11:16:29 +01:00
Tomi Turtiainen 9a1cc56806
fix: Set '@typescript-eslint/return-await' rule to 'always' for node code (no-changelog) (#8363) 
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
 2024-01-17 17:08:50 +02:00
Iván Ovejero d4c93b1607
fix(core): Prevent issues with missing or mismatching encryption key (#8332) 2024-01-16 18:25:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ 64ceb16af6
refactor(core): Use DI in PermissionChecker (no-changelog) (#8344) 2024-01-16 14:15:29 +01:00
Omar Ajoue e1acb5911a
refactor: Make execution IDs mandatory in BE (#8299) 
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
 2024-01-16 09:53:17 +00:00
Tomi Turtiainen 3b996a7da0
feat(core): Validate shutdown handlers on startup (#8260) 2024-01-08 17:46:45 +02:00
Iván Ovejero 40c1eeeddd
refactor(core): Continue moving typeorm operators to repositories (no-changelog) (#8186) 
Follow-up to: #8163
 2024-01-02 17:53:24 +01:00
Iván Ovejero e418d42450
refactor(core): Move typeorm operators from various sources into repositories (no-changelog) (#8174) 
Follow-up to: #8165
 2023-12-28 13:14:10 +01:00
कारतोफ्फेलस्क्रिप्ट™ f69ddcd796
refactor(core): Use Dependency Injection for all Controller classes (no-changelog) (#8146) 
## Review / Merge checklist
- [x] PR title and summary are descriptive
 2023-12-27 11:50:43 +01:00
कारतोफ्फेलस्क्रिप्ट™ baee47a276
refactor(core): Move all base URLs to UrlService (no-changelog) (#8141) 
This change kept coming up in #6713, #7773, and #8135. 
So this PR moves the existing code without actually changing anything,
to help get rid of some of the circular dependencies.


## Review / Merge checklist
- [x] PR title and summary are descriptive.
 2023-12-22 15:19:50 +01:00
Tomi Turtiainen 3a881be6c2
feat(core): Unify application components shutdown (#8097) 
## Summary

Add `ShutdownService` and `OnShutdown` decorator for more unified way to
shutdown different components. Use this new way in the following
components:

- HTTP(S) server
- Pruning service
- Push connection
- License

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
 2023-12-22 12:39:58 +02:00
कारतोफ्फेलस्क्रिप्ट™ c158ca2471
refactor(core): Upgrade more dependencies to remove axios 0.x (no-changelog) (#8105) 
Had to [fork localtunnel](https://github.com/n8n-io/localtunnel) to get
the axios upgrade, since localtunnel doesn't seem to be actively maintained.
 2023-12-22 11:39:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 464b565283
ci: Remove unnecessary async/await, enable await-thenable linting rule (no-changelog) (#8076) 
## Summary
We accidentally made some functions `async` in
https://github.com/n8n-io/n8n/pull/7846
This PR reverts that change. 

## Review / Merge checklist
- [x] PR title and summary are descriptive.
 2023-12-19 13:52:42 +01:00
Tomi Turtiainen 614f488386
feat(core): Add N8N_GRACEFUL_SHUTDOWN_TIMEOUT env var (#8068) 
Add generic N8N_GRACEFUL_SHUTDOWN_TIMEOUT which controls how long n8n
process will wait for graceful exit before exitting forcefully. This
variables replaces the QUEUE_WORKER_TIMEOUT variable that was used for
worker process.

DEPRECATED: QUEUE_WORKER_TIMEOUT deprected

QUEUE_WORKER_TIMEOUT environment variable has been replaced with
N8N_GRACEFUL_SHUTDOWN_TIMEOUT.
 2023-12-18 14:04:19 +02:00
Tomi Turtiainen 4cae976a3b
fix(core): Consider timeout in shutdown an error (#8050) 
If the process doesn't shutdown within a time limit, exit with error
code.

1. conceptually something timing out is an error.
2. on successful exit we close down the DB connection gracefully. On an
exit timeout we rather not do that, since it will wait for any active
connections to close and would possible block the exit.
 2023-12-18 10:53:34 +02:00
Tomi Turtiainen 6ae2f5efea
fix(core): Fix shutdown if terminating before hooks are initialized (#8047) 
If the app receives termination signal before hooks have been
initialised, the would be objet is undefined error. This PR fixes that.
 2023-12-18 09:23:10 +02:00
Tomi Turtiainen 67bd8ad698
fix(core): Handle multiple termination signals correctly (#8046) 
Prevent possible multiple termination signals initiating the shutdown
process multiple times.
 2023-12-15 17:35:22 +02:00
Tomi Turtiainen e69707efd4
fix(core): Close db connection gracefully when exiting (#8045) 
Close db connection gracefully when exiting
 2023-12-15 17:16:35 +02:00
Iván Ovejero 2da15d0264
refactor(core): Remove ID-less workflow reporting (no-changelog) (#8038) 
We added ID-less workflow reporting at #8031, which has already produced
multiple reports coming from internal, enough info to tackle [this
story](https://linear.app/n8n/issue/PAY-1147). To prevent an
overwhelming number of reports from cloud, this PR removes the reporting
for now.
 2023-12-15 12:50:09 +01:00
Iván Ovejero f18bc5f4b7
refactor(core): Warn on sqlite DB detected during init on queue mode (#8034) 
When setting up queue mode, it is easy to overlook that not exporting
Postgres env vars will default the worker to use sqlite, which will fail
during execution with a non-obvious error. Hence add warnings when
starting a worker with an incompatible DB type.
 2023-12-15 10:56:35 +01:00
Iván Ovejero c5e6ba8cdd
fix(core): Restore workflow ID during execution creation (#8031) 
## Summary
Restore workflow ID during execution creation removed by [this
PR](https://github.com/n8n-io/n8n/pull/8002/files#diff-c8cbb62ca9ab2ae45e5f565cd8c63fff6475809a6241ea0b90acc575615224af).
The missing workflow ID, and more generally the fact that `workflow.id`
is optional when it should not be, causes `PermissionChecker.check` to
misreport a credential as inaccessible when it should be accessible.

More generally, start reporting ID-less workflows so we can root them
out and prevent this at type level.

## Related tickets and issues

https://n8nio.slack.com/archives/C035KBDA917/p1702539465555529
 2023-12-14 18:13:12 +01:00
Iván Ovejero 53c0b49d15
fix(core): Initialize queue once in queue mode (#8025) 
We're initializing the queue twice because of a [bad
merge](2c63474538).
No associated known bugs but no need to init the queue twice. We should
follow up by investigating if any pending bugs can be associated to
this.
 2023-12-14 16:16:12 +01:00
Iván Ovejero 7b5d0a9546
refactor(core): Introduce import service (no-changelog) (#8001) 
Consolidate import logic into import service.

Also fixes:
- https://linear.app/n8n/issue/PAY-1086
- https://github.com/n8n-io/n8n/issues/7881
-
https://community.n8n.io/t/cli-workflow-imports-failing-after-upgrade-to-v1-18-0/33780
- https://linear.app/n8n/issue/PAY-221
- https://github.com/n8n-io/n8n/issues/5477
- https://community.n8n.io/t/export-workflows-with-tags-got-created/6161
 2023-12-13 10:00:21 +01:00
Iván Ovejero f49aebb740
refactor(core): Move instance owner retrieval to ownership service (no-changelog) (#7980) 
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.
Continue breaking down `UserManagementHelper.ts`
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
 2023-12-11 13:35:09 +01:00
Iván Ovejero c48850d74f
fix(core): Fix expression evaluator filename typo (no-changelog) (#7960) 
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
 2023-12-07 17:50:49 +01:00
Iván Ovejero 07f6662aba
refactor(core): Add log about removed TTL keys for binary data (#7892) 
Context:
https://n8nio.slack.com/archives/C035KBDA917/p1701418556761549?thread_ts=1701411854.517989&cid=C035KBDA917
 2023-12-01 12:51:45 +01:00
Iván Ovejero c08c5cc37b
refactor(core): Switch plain errors in cli to ApplicationError (#7857) 
Ensure all errors in `cli` are `ApplicationError` or children of it and
contain no variables in the message, to continue normalizing all the
errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7839
 2023-11-29 12:25:10 +01:00
Iván Ovejero 1c6178759c
refactor(core): Reorganize error hierarchy in cli package (no-changelog) (#7839) 
Ensure all errors in `cli` inherit from `ApplicationError` to continue
normalizing all the errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7820
 2023-11-28 10:19:27 +01:00
Csaba Tuncsik 152883eed1
build: Upgrade lint related packages (#7790) 2023-11-27 09:11:52 +01:00
Iván Ovejero fdb2c18ecc
fix(core): Tighten checks for multi-main setup usage (#7788) 
https://n8nio.slack.com/archives/C05HRPLSGTT/p1700731476321999?thread_ts=1700729359.746899&cid=C05HRPLSGTT
 2023-11-23 12:18:39 +01:00
Iván Ovejero 4c4082503c
feat(core): Coordinate manual workflow activation and deactivation in multi-main scenario (#7643) 
Followup to #7566 | Story: https://linear.app/n8n/issue/PAY-926

### Manual workflow activation and deactivation

In a multi-main scenario, if the user manually activates or deactivates
a workflow, the process (whether leader or follower) that handles the
PATCH request and updates its internal state should send a message into
the command channel, so that all other main processes update their
internal state accordingly:

- Add to `ActiveWorkflows` if activating
- Remove from `ActiveWorkflows` if deactivating
- Remove and re-add to `ActiveWorkflows` if the update did not change
activation status.

After updating their internal state, if activating or deactivating, the
recipient main processes should push a message to all connected
frontends so that these can update their stores and so reflect the value
in the UI.

### Workflow activation errors

On failure to activate a workflow, the main instance should record the
error in Redis - main instances should always pull activation errors
from Redis in a multi-main scenario.

### Leadership change

On leadership change...

- The old leader should stop pruning and the new leader should start
pruning.
- The old leader should remove trigger- and poller-based workflows and
the new leader should add them.
 2023-11-17 15:58:50 +01:00
कारतोफ्फेलस्क्रिप्ट™ 3460eb5eeb
fix(core): Initialize JWT Secret before it's used anywhere (#7707) 
HELP-394
 2023-11-15 12:17:18 +01:00
Iván Ovejero b2ca050031
perf(core): Lazyload security audit reporters (#7696) 
Also converting to service.

Followup to https://github.com/n8n-io/n8n/pull/7663
 2023-11-13 11:50:43 +01:00
कारतोफ्फेलस्क्रिप्ट™ 000e76e3b4
ci(core): Reduce memory usage in tests (part-2) (no-changelog) (#7671) 
This also gets rid of `Db.collection`, which was another source of
circular dependencies.
 2023-11-10 15:04:26 +01:00
कारतोफ्फेलस्क्रिप्ट™ f0fc5b16d3
ci(core): Load config schema after process.env has been overwritten (no-changelog) (#7550) 2023-11-07 15:58:28 +01:00
Iván Ovejero f748de9567
fix(core): Ensure pruning starts only after migrations have completed (#7626) 
https://linear.app/n8n/issue/PAY-986/bug-execution-pruning-timer-is-started-before-the-database-is-ready

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
 2023-11-07 13:56:01 +01:00
Iván Ovejero c857e42677
feat(core): Coordinate workflow activation in multiple main scenario in internal API (#7566) 
Story: https://linear.app/n8n/issue/PAY-926

This PR coordinates workflow activation on instance startup and on
leadership change in multiple main scenario in the internal API. Part 3
on manual workflow activation and deactivation will be a separate PR.

### Part 1: Instance startup

In multi-main scenario, on starting an instance...
- [x] If the instance is the leader, it should add webhooks, triggers
and pollers.
- [x] If the instance is the follower, it should not add webhooks,
triggers or pollers.
- [x] Unit tests.

### Part 2: Leadership change 

In multi-main scenario, if the main instance leader dies…

- [x] The new main instance leader must activate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] The old main instance leader must deactivate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] Unit tests.

To test, start two instances and check behavior on startup and
leadership change:

```
EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug npm run start

EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug N8N_PORT=5679 npm run start
```
 2023-11-07 13:48:48 +01:00
Iván Ovejero a994ba5e8d
fix(core): Ensure init before checking leader or follower in multi-main scenario (#7621) 
This PR ensures `MultiMainInstancePublisher` is initialized before
checking if the instance is leader or follower. Followers skip license
init, license check, and pruning start and stop.
 2023-11-06 12:03:35 +01:00