Commit graph

175 commits

Author SHA1 Message Date
कारतोफ्फेलस्क्रिप्ट™ 059d281fd1
feat(core): Move execution permission checks earlier in the lifecycle (#8677) 2024-02-21 14:47:02 +01:00
कारतोफ्फेलस्क्रिप्ट™ b6c8a0c413
refactor(core): Reduce code duplication in DB config (no-changelog) (#8679) 2024-02-20 14:28:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ a2a3ca160f
fix(core): Improve the startup error when EXECUTIONS_PROCESS is set (#8630)
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2024-02-14 15:35:42 +01:00
कारतोफ्फेलस्क्रिप्ट™ 8e392cfc1d
feat(core): Migrate to n8n's typeorm fork (#8590) 2024-02-08 15:13:29 +01:00
कारतोफ्फेलस्क्रिप्ट™ c04f92f7fd
refactor(core): Couple of refactors on WorkflowRunner and ActiveExecutions (no-changelog) (#8487) 2024-02-06 18:09:50 +01:00
Iván Ovejero c4e39451db
refactor(core): Continue breaking dependency cycles (no-changelog) (#8545) 2024-02-06 10:08:46 +01:00
Iván Ovejero 28aee7c425
refactor(core): Lock webhook process out of multi-main setup (no-changelog) (#8549) 2024-02-05 18:06:54 +01:00
Iván Ovejero dc5ec8f946
refactor(core): Streamline flows in multi-main mode (no-changelog) (#8446) 2024-02-05 09:26:55 +01:00
Iván Ovejero cc41fc7c80
fix(core): Fix new graceful shutdown env being always overridden by deprecated env (#8503) 2024-02-01 11:10:40 +01:00
कारतोफ्फेलस्क्रिप्ट™ 121a55b691
feat(core): Remove own execution-process mode (#8490) 2024-01-30 12:51:40 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5cb55270b7
refactor(core): Replace promisify-d node calls with native promises (no-changelog) (#8464) 2024-01-30 09:49:23 +01:00
कारतोफ्फेलस्क्रिप्ट™ c70fa66e76
refactor(core): Use DI for WorkflowRunner (no-changelog) (#8372) 2024-01-26 13:49:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 7c49004018
refactor(core): Use DI for eventBus code - Part 1 (no-changelog) (#8434) 2024-01-26 12:21:15 +01:00
कारतोफ्फेलस्क्रिप्ट™ cc2f0ada76
ci: Enforce noUnusedLocals on all backend and nodes packages (no-changelog) (#8428) 2024-01-25 14:33:35 +01:00
कारतोफ्फेलस्क्रिप्ट™ d6deceacde
refactor(core): Remove roleId indirection (no-changelog) (#8413) 2024-01-24 13:38:57 +01:00
Danny Martini ae06fdeb62
fix(core): Fix update workflow cli command being unable to activate all workflows (#8412)
Co-authored-by: Daniel Schröder <daniel.schroeder@skriptfabrik.com>
2024-01-23 10:59:06 +01:00
कारतोफ्फेलस्क्रिप्ट™ 913c8c6b0c
feat(core): Upgrade oclif (no-changelog) (#8381) 2024-01-22 18:25:36 +01:00
Iván Ovejero f35d4fcbd8
refactor(core): Simplify OrchestrationService (no-changelog) (#8364) 2024-01-22 11:16:29 +01:00
Tomi Turtiainen 9a1cc56806
fix: Set '@typescript-eslint/return-await' rule to 'always' for node code (no-changelog) (#8363)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-01-17 17:08:50 +02:00
Iván Ovejero d4c93b1607
fix(core): Prevent issues with missing or mismatching encryption key (#8332) 2024-01-16 18:25:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ 64ceb16af6
refactor(core): Use DI in PermissionChecker (no-changelog) (#8344) 2024-01-16 14:15:29 +01:00
Omar Ajoue e1acb5911a
refactor: Make execution IDs mandatory in BE (#8299)
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2024-01-16 09:53:17 +00:00
Tomi Turtiainen 3b996a7da0
feat(core): Validate shutdown handlers on startup (#8260) 2024-01-08 17:46:45 +02:00
Iván Ovejero 40c1eeeddd
refactor(core): Continue moving typeorm operators to repositories (no-changelog) (#8186)
Follow-up to: #8163
2024-01-02 17:53:24 +01:00
Iván Ovejero e418d42450
refactor(core): Move typeorm operators from various sources into repositories (no-changelog) (#8174)
Follow-up to: #8165
2023-12-28 13:14:10 +01:00
कारतोफ्फेलस्क्रिप्ट™ f69ddcd796
refactor(core): Use Dependency Injection for all Controller classes (no-changelog) (#8146)
## Review / Merge checklist
- [x] PR title and summary are descriptive
2023-12-27 11:50:43 +01:00
कारतोफ्फेलस्क्रिप्ट™ baee47a276
refactor(core): Move all base URLs to UrlService (no-changelog) (#8141)
This change kept coming up in #6713, #7773, and #8135. 
So this PR moves the existing code without actually changing anything,
to help get rid of some of the circular dependencies.


## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-22 15:19:50 +01:00
Tomi Turtiainen 3a881be6c2
feat(core): Unify application components shutdown (#8097)
## Summary

Add `ShutdownService` and `OnShutdown` decorator for more unified way to
shutdown different components. Use this new way in the following
components:

- HTTP(S) server
- Pruning service
- Push connection
- License

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-22 12:39:58 +02:00
कारतोफ्फेलस्क्रिप्ट™ c158ca2471
refactor(core): Upgrade more dependencies to remove axios 0.x (no-changelog) (#8105)
Had to [fork localtunnel](https://github.com/n8n-io/localtunnel) to get
the axios upgrade, since localtunnel doesn't seem to be actively maintained.
2023-12-22 11:39:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 464b565283
ci: Remove unnecessary async/await, enable await-thenable linting rule (no-changelog) (#8076)
## Summary
We accidentally made some functions `async` in
https://github.com/n8n-io/n8n/pull/7846
This PR reverts that change. 

## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-19 13:52:42 +01:00
Tomi Turtiainen 614f488386
feat(core): Add N8N_GRACEFUL_SHUTDOWN_TIMEOUT env var (#8068)
Add generic N8N_GRACEFUL_SHUTDOWN_TIMEOUT which controls how long n8n
process will wait for graceful exit before exitting forcefully. This
variables replaces the QUEUE_WORKER_TIMEOUT variable that was used for
worker process.

DEPRECATED: QUEUE_WORKER_TIMEOUT deprected

QUEUE_WORKER_TIMEOUT environment variable has been replaced with
N8N_GRACEFUL_SHUTDOWN_TIMEOUT.
2023-12-18 14:04:19 +02:00
Tomi Turtiainen 4cae976a3b
fix(core): Consider timeout in shutdown an error (#8050)
If the process doesn't shutdown within a time limit, exit with error
code.

1. conceptually something timing out is an error.
2. on successful exit we close down the DB connection gracefully. On an
exit timeout we rather not do that, since it will wait for any active
connections to close and would possible block the exit.
2023-12-18 10:53:34 +02:00
Tomi Turtiainen 6ae2f5efea
fix(core): Fix shutdown if terminating before hooks are initialized (#8047)
If the app receives termination signal before hooks have been
initialised, the would be objet is undefined error. This PR fixes that.
2023-12-18 09:23:10 +02:00
Tomi Turtiainen 67bd8ad698
fix(core): Handle multiple termination signals correctly (#8046)
Prevent possible multiple termination signals initiating the shutdown
process multiple times.
2023-12-15 17:35:22 +02:00
Tomi Turtiainen e69707efd4
fix(core): Close db connection gracefully when exiting (#8045)
Close db connection gracefully when exiting
2023-12-15 17:16:35 +02:00
Iván Ovejero 2da15d0264
refactor(core): Remove ID-less workflow reporting (no-changelog) (#8038)
We added ID-less workflow reporting at #8031, which has already produced
multiple reports coming from internal, enough info to tackle [this
story](https://linear.app/n8n/issue/PAY-1147). To prevent an
overwhelming number of reports from cloud, this PR removes the reporting
for now.
2023-12-15 12:50:09 +01:00
Iván Ovejero f18bc5f4b7
refactor(core): Warn on sqlite DB detected during init on queue mode (#8034)
When setting up queue mode, it is easy to overlook that not exporting
Postgres env vars will default the worker to use sqlite, which will fail
during execution with a non-obvious error. Hence add warnings when
starting a worker with an incompatible DB type.
2023-12-15 10:56:35 +01:00
Iván Ovejero c5e6ba8cdd
fix(core): Restore workflow ID during execution creation (#8031)
## Summary
Restore workflow ID during execution creation removed by [this
PR](https://github.com/n8n-io/n8n/pull/8002/files#diff-c8cbb62ca9ab2ae45e5f565cd8c63fff6475809a6241ea0b90acc575615224af).
The missing workflow ID, and more generally the fact that `workflow.id`
is optional when it should not be, causes `PermissionChecker.check` to
misreport a credential as inaccessible when it should be accessible.

More generally, start reporting ID-less workflows so we can root them
out and prevent this at type level.

## Related tickets and issues

https://n8nio.slack.com/archives/C035KBDA917/p1702539465555529
2023-12-14 18:13:12 +01:00
Iván Ovejero 53c0b49d15
fix(core): Initialize queue once in queue mode (#8025)
We're initializing the queue twice because of a [bad
merge](2c63474538).
No associated known bugs but no need to init the queue twice. We should
follow up by investigating if any pending bugs can be associated to
this.
2023-12-14 16:16:12 +01:00
Iván Ovejero 7b5d0a9546
refactor(core): Introduce import service (no-changelog) (#8001)
Consolidate import logic into import service.

Also fixes:
- https://linear.app/n8n/issue/PAY-1086
- https://github.com/n8n-io/n8n/issues/7881
-
https://community.n8n.io/t/cli-workflow-imports-failing-after-upgrade-to-v1-18-0/33780
- https://linear.app/n8n/issue/PAY-221
- https://github.com/n8n-io/n8n/issues/5477
- https://community.n8n.io/t/export-workflows-with-tags-got-created/6161
2023-12-13 10:00:21 +01:00
Iván Ovejero f49aebb740
refactor(core): Move instance owner retrieval to ownership service (no-changelog) (#7980)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.
Continue breaking down `UserManagementHelper.ts`
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-11 13:35:09 +01:00
Iván Ovejero c48850d74f
fix(core): Fix expression evaluator filename typo (no-changelog) (#7960)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 17:50:49 +01:00
Iván Ovejero 07f6662aba
refactor(core): Add log about removed TTL keys for binary data (#7892)
Context:
https://n8nio.slack.com/archives/C035KBDA917/p1701418556761549?thread_ts=1701411854.517989&cid=C035KBDA917
2023-12-01 12:51:45 +01:00
Iván Ovejero c08c5cc37b
refactor(core): Switch plain errors in cli to ApplicationError (#7857)
Ensure all errors in `cli` are `ApplicationError` or children of it and
contain no variables in the message, to continue normalizing all the
errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7839
2023-11-29 12:25:10 +01:00
Iván Ovejero 1c6178759c
refactor(core): Reorganize error hierarchy in cli package (no-changelog) (#7839)
Ensure all errors in `cli` inherit from `ApplicationError` to continue
normalizing all the errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7820
2023-11-28 10:19:27 +01:00
Csaba Tuncsik 152883eed1
build: Upgrade lint related packages (#7790) 2023-11-27 09:11:52 +01:00
Iván Ovejero fdb2c18ecc
fix(core): Tighten checks for multi-main setup usage (#7788)
https://n8nio.slack.com/archives/C05HRPLSGTT/p1700731476321999?thread_ts=1700729359.746899&cid=C05HRPLSGTT
2023-11-23 12:18:39 +01:00
Iván Ovejero 4c4082503c
feat(core): Coordinate manual workflow activation and deactivation in multi-main scenario (#7643)
Followup to #7566 | Story: https://linear.app/n8n/issue/PAY-926

### Manual workflow activation and deactivation

In a multi-main scenario, if the user manually activates or deactivates
a workflow, the process (whether leader or follower) that handles the
PATCH request and updates its internal state should send a message into
the command channel, so that all other main processes update their
internal state accordingly:

- Add to `ActiveWorkflows` if activating
- Remove from `ActiveWorkflows` if deactivating
- Remove and re-add to `ActiveWorkflows` if the update did not change
activation status.

After updating their internal state, if activating or deactivating, the
recipient main processes should push a message to all connected
frontends so that these can update their stores and so reflect the value
in the UI.

### Workflow activation errors

On failure to activate a workflow, the main instance should record the
error in Redis - main instances should always pull activation errors
from Redis in a multi-main scenario.

### Leadership change

On leadership change...

- The old leader should stop pruning and the new leader should start
pruning.
- The old leader should remove trigger- and poller-based workflows and
the new leader should add them.
2023-11-17 15:58:50 +01:00
कारतोफ्फेलस्क्रिप्ट™ 3460eb5eeb
fix(core): Initialize JWT Secret before it's used anywhere (#7707)
HELP-394
2023-11-15 12:17:18 +01:00
Iván Ovejero b2ca050031
perf(core): Lazyload security audit reporters (#7696)
Also converting to service.

Followup to https://github.com/n8n-io/n8n/pull/7663
2023-11-13 11:50:43 +01:00