Commit graph

32 commits

Author SHA1 Message Date
Ricardo Espinoza 8e0ae3cf8c
refactor: Extract Invitation routes to InvitationController (no-changelog) (#7726)
This PR:

- Creates `InvitationController`
- Moves `POST /users` to `POST /invitations` and move related test to
`invitations.api.tests`
- Moves `POST /users/:id` to `POST /invitations/:id/accept` and move
related test to `invitations.api.tests`
- Adjusts FE to use new endpoints
- Moves all the invitation logic to the `UserService`

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-16 12:39:43 -05:00
कारतोफ्फेलस्क्रिप्ट™ 000e76e3b4
ci(core): Reduce memory usage in tests (part-2) (no-changelog) (#7671)
This also gets rid of `Db.collection`, which was another source of
circular dependencies.
2023-11-10 15:04:26 +01:00
कारतोफ्फेलस्क्रिप्ट™ 0346b211a7
ci(core): Reduce memory usage in tests (part-1) (no-changelog) (#7654) 2023-11-08 16:29:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 60314248f4
fix(core): Make password-reset urls valid only for single-use (#7622) 2023-11-07 15:35:43 +01:00
Iván Ovejero c857e42677
feat(core): Coordinate workflow activation in multiple main scenario in internal API (#7566)
Story: https://linear.app/n8n/issue/PAY-926

This PR coordinates workflow activation on instance startup and on
leadership change in multiple main scenario in the internal API. Part 3
on manual workflow activation and deactivation will be a separate PR.

### Part 1: Instance startup

In multi-main scenario, on starting an instance...
- [x] If the instance is the leader, it should add webhooks, triggers
and pollers.
- [x] If the instance is the follower, it should not add webhooks,
triggers or pollers.
- [x] Unit tests.

### Part 2: Leadership change 

In multi-main scenario, if the main instance leader dies…

- [x] The new main instance leader must activate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] The old main instance leader must deactivate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] Unit tests.

To test, start two instances and check behavior on startup and
leadership change:

```
EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug npm run start

EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug N8N_PORT=5679 npm run start
```
2023-11-07 13:48:48 +01:00
कारतोफ्फेलस्क्रिप्ट™ 05586a900d
refactor(core): Make Logger a service (no-changelog) (#7494) 2023-10-25 16:35:22 +02:00
कारतोफ्फेलस्क्रिप्ट™ b6de910cbe
refactor(core): Abstract away InstanceSettings and encryptionKey into injectable services (no-changelog) (#7471)
This change ensures that things like `encryptionKey` and `instanceId`
are always available directly where they are needed, instead of passing
them around throughout the code.
2023-10-23 13:39:35 +02:00
कारतोफ्फेलस्क्रिप्ट™ 55c6a1b0d3
fix(core): Do not return inviteAcceptUrl in response if email was sent (#7465) 2023-10-19 13:58:06 +02:00
कारतोफ्फेलस्क्रिप्ट™ c5ee06cc61
refactor(core): Refactor nodes loading (no-changelog) (#7283)
fixes PAY-605
2023-10-09 16:09:23 +02:00
Iván Ovejero 34bda535e6
refactor(core): Create controller for binary data (no-changelog) (#7363)
This PR adds a controller for binary data + integration tests.
2023-10-06 16:21:13 +02:00
Iván Ovejero 1a661e6d00
feat(core): Integrate object store as binary data manager (#7253)
Depends on: #7225 | Story:
[PAY-848](https://linear.app/n8n/issue/PAY-848)

This PR integrates the object store service as a new binary data manager
for Enterprise.
2023-10-05 15:25:17 +02:00
Val 0083a9e45d
feat(core): Initial workflow history API (#7234)
Github issue / Community forum post (link here to close automatically):
2023-09-27 15:22:39 +01:00
Iván Ovejero 6d6e2488c6
refactor(core): Generalize binary data manager interface (no-changelog) (#7164)
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)

This PR: 
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.

Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.

Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-22 17:22:12 +02:00
Iván Ovejero cd08c8e4c6
refactor(core): Implement soft-deletions for executions (#7092)
Based on #7065 | Story: https://linear.app/n8n/issue/PAY-771

n8n on filesystem mode marks binary data to delete on manual execution
deletion, on unsaved execution completion, and on every execution
pruning cycle. We later prune binary data in a separate cycle via these
marker files, based on the configured TTL. In the context of introducing
an S3 client to manage binary data, the filesystem mode's mark-and-prune
setup is too tightly coupled to the general binary data management
client interface.

This PR...
- Ensures the deletion of an execution causes the deletion of any binary
data associated to it. This does away with the need for binary data TTL
and simplifies the filesystem mode's mark-and-prune setup.
- Refactors all execution deletions (including pruning) to cause soft
deletions, hard-deletes soft-deleted executions based on the existing
pruning config, and adjusts execution endpoints to filter out
soft-deleted executions. This reduces DB load, and keeps binary data
around long enough for users to access it when building workflows with
unsaved executions.
- Moves all execution pruning work from an execution lifecycle hook to
`execution.repository.ts`. This keeps related logic in a single place.
- Removes all marking logic from the binary data manager. This
simplifies the interface that the S3 client will meet.
- Adds basic sanity-check tests to pruning logic and execution deletion.

Out of scope:

- Improving existing pruning logic.
- Improving existing execution repository logic.
- Adjusting dir structure for filesystem mode.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-20 15:21:42 +02:00
कारतोफ्फेलस्क्रिप्ट™ 2f647974f4
refactor: Use actual nodes/credentials in cli and core tests (no-changelog) (#7115) 2023-09-06 10:23:47 +02:00
कारतोफ्फेलस्क्रिप्ट™ 36f246929e
ci: Fix linting (no-changelog) (#7110) 2023-09-05 14:41:56 +02:00
Michael Auerswald 011ee2e04b
fix(core): Split event bus controller into community and ee (#7107) 2023-09-05 13:32:09 +02:00
कारतोफ्फेलस्क्रिप्ट™ 6aa7b93473
refactor(core): Deprecate prepareOutputData (no-changelog) (#7091) 2023-09-05 12:59:02 +02:00
Iván Ovejero 51093f649d
refactor: Move community package logic to service (no-changelog) (#6973) 2023-09-01 15:13:19 +02:00
Iván Ovejero 87cf1d9c1b
refactor(core): Make controller constructors consistent (no-changelog) (#7015) 2023-08-25 13:23:22 +02:00
Alex Grozav ed927d34b2
feat: External Secrets storage for credentials (#6477)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-08-25 10:33:46 +02:00
Ricardo Espinoza 2b7ba6fdf1
feat(core): Add MFA (#4767)
https://linear.app/n8n/issue/ADO-947/sync-branch-with-master-and-fix-fe-e2e-tets

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-08-23 22:59:16 -04:00
Iván Ovejero 96a9de68a0
refactor(core): Move all user DB access to UserRepository (#6910)
Prep for https://linear.app/n8n/issue/PAY-646
2023-08-22 15:58:05 +02:00
कारतोफ्फेलस्क्रिप्ट™ 10c15874b2
fix(core): Restore body parsing for all content-types for non webhook routes (no-changelog)(#6911)
Changes in https://github.com/n8n-io/n8n/pull/6394 removed xml body parsing for all non-webhook routes. This broken SAML endpoints as they need the XML body parser to function correctly.
2023-08-11 09:18:33 +02:00
Iván Ovejero 11440bfd3c
refactor(core): Move tag collection into repository (no-changelog) (#6860)
* refactor(core): Move tag collection into repository

* Fix tests

* Address feedback

* Fix missing spot
2023-08-08 14:08:56 +02:00
Michael Auerswald adcf5a96e8
feat(core): Add metrics option to cache (#6846)
* add metrics to cache

* use events for metrics

* pr comments / broken test

* lint fix

* update the test

* improve tests

* Update packages/cli/src/config/schema.ts

* disable flaky test

* lint fix

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
2023-08-04 20:51:07 +02:00
Val 371bfa0f48
fix: Auth.api user limit test expecting incorrect status (#6836)
fix: auth.api user limit test expecting incorrect status
2023-08-03 13:03:26 +01:00
Iván Ovejero e4f041815a
perf(core): Cache roles (#6803)
* refactor: Create `RoleService`

* refactor: Refactor to use service

* refactor: Move `getUserRoleForWorkflow`

* refactor: Clear out old `RoleService`

* refactor: Consolidate utils into service

* refactor: Remove unused methods

* test: Add tests

* refactor: Remove redundant return types

* refactor: Missing utility

* chore: Remove commented out bit

* refactor: Make `Db.collections.Repository` inaccessible

* chore: Cleanup

* feat: Prepopulate cache

* chore: Remove logging

* fix: Account for tests where roles are undefined

* fix: Restore `prettier.prettierPath`

* test: Account for cache enabled and disabled

* fix: Restore `Role` in `Db.collections`

* refactor: Simplify by removing `orFail`

* refactor: Rename for clarity

* refactor: Use `cacheKey` for readability

* refactor: Validate role before creation

* refacator: Remove redundant `cache` prefix

* ci: Lint fix

* test: Fix e2e
2023-08-03 08:58:36 +02:00
कारतोफ्फेलस्क्रिप्ट™ 31d8f478ee
refactor(core): Parse Webhook request bodies on-demand (#6394)
Also,
1. Consistent CORS support ~on all three webhook types~ waiting webhooks never supported CORS. I'll fix that in another PR
2. [Fixes binary-data handling when request body is text, json, or xml](https://linear.app/n8n/issue/NODE-505/webhook-binary-data-handling-fails-for-textplain-files).
3. Reduced number of middleware that each request has to go through.
4. Removed the need to maintain webhook endpoints in the auth-exception list.
5. Skip all middlewares (apart from `compression`) on Webhook routes. 
6. move `multipart/form-data` support out of individual nodes
7. upgrade `formidable`
8. fix the filenames on binary-data in webhooks nodes
9. add unit tests and integration tests for webhook request handling, and increase test coverage
2023-08-01 17:32:30 +02:00
Ricardo Espinoza 89f44021b9
fix(core): Use JWT as reset password token (#6714)
* use jwt to reset password

* increase expiration time to 1d

* drop user id query string

* refactor

* use service instead of package in tests

* sqlite migration

* postgres migration

* mysql migration

* remove unused properties

* remove userId from FE

* fix test for users.api

* move migration to the common folder

* move type assertion to the jwt.service

* Add jwt secret as a readonly property

* use signData instead of sign in user.controller

* remove base class

* remove base class

* add tests
2023-07-24 17:40:17 -04:00
Val b1838f7fab
fix: Add missing indices on sqlite (#6673)
* fix: enforce tag name uniqueness on sqlite

* rename migration and add other missing indices

* add tags tests
2023-07-20 10:34:45 +01:00
कारतोफ्फेलस्क्रिप्ट™ b895ba438a
refactor(core): Reduce boilterplate code in between tests 🧹, and fix the tests in node.js 20 (no-changelog) (#6654)
refactor(core): Reduce boilterplate code in between tests

also cleaned up some imports, and fixed the tests in node.js 20
2023-07-13 10:14:48 +02:00