Commit graph

250 commits

Author SHA1 Message Date
Michael Auerswald cbc690907f
feat(editor): Adds a EE view to show worker details and job status (#7600)
This change expands on the command channel communication introduced
lately between the main instance(s) and the workers. The frontend gets a
new menu entry "Workers" which will, when opened, trigger a regular call
to getStatus from the workers. The workers then respond via their
response channel to the backend, which then pushes the status to the
frontend.
This introduces the use of ChartJS for metrics.
This feature is still in MVP state and thus disabled by default for the
moment.
2023-11-10 23:48:31 +01:00
Val 0468ded0db
feat: Add permissions package (no-changelog) (#7650)
Github issue / Community forum post (link here to close automatically):
2023-11-08 15:42:40 +00:00
Ricardo Espinoza 5790e251b8
feat(core): Rate limit forgot password endpoint (#7604)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
2023-11-03 13:44:12 -04:00
कारतोफ्फेलस्क्रिप्ट™ 0bd4e742da
refactor(core): Migrate from crypto-js to native crypto (#7556)
[`crypto-js` has been
discontinued](1da3dabf93)

PS: We'll remove `crypto-js` usage from `n8n-workflow` and
`@n8n_io/license-sdk` in separate PRs.
2023-11-03 13:26:20 +01:00
Alex Grozav 135f9214f5
feat(editor): Update storybook to 7.5.2 (no-changelog) (#7601)
Github issue / Community forum post (link here to close automatically):
2023-11-03 14:14:51 +02:00
कारतोफ्फेलस्क्रिप्ट™ c05bc6728d
fix(editor): Handle localStorage being blocked/unavailable (#7348) 2023-10-27 15:51:20 +02:00
कारतोफ्फेलस्क्रिप्ट™ 65e5593233
fix(core): Upgrade crypto-js to address CVE-2023-46233 (#7519)
[GH Advisory](https://github.com/advisories/GHSA-xwcq-pm8m-c4vf)
2023-10-26 11:21:53 +02:00
Cornelius Suermann 3ef771ea38
feat: Collect usage metrics on license renewal (no-changelog) (#7486) 2023-10-23 21:39:29 +02:00
कारतोफ्फेलस्क्रिप्ट™ b6de910cbe
refactor(core): Abstract away InstanceSettings and encryptionKey into injectable services (no-changelog) (#7471)
This change ensures that things like `encryptionKey` and `instanceId`
are always available directly where they are needed, instead of passing
them around throughout the code.
2023-10-23 13:39:35 +02:00
Elias Meire 7e06b31a5f
fix(editor): Implement canvas zoom UX improvements (#7376)
- Fix pinch-to-zoom
- Support command + scroll to zoom
- Improve accuracy of zooming (scroll more = zoom more)
- Zoom limits
- Zoom relative to mouse position
2023-10-10 11:07:58 +02:00
कारतोफ्फेलस्क्रिप्ट™ 00a4b8b0c6
feat(core): Add support for building LLM applications (#7235)
This extracts all core and editor changes from #7246 and #7137, so that
we can get these changes merged first.

ADO-1120

[DB Tests](https://github.com/n8n-io/n8n/actions/runs/6379749011)
[E2E Tests](https://github.com/n8n-io/n8n/actions/runs/6379751480)
[Workflow Tests](https://github.com/n8n-io/n8n/actions/runs/6379752828)

---------

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-10-02 17:33:43 +02:00
Michael Auerswald 07d072c28f
fix(core): Bump License SDK Version (no-changelog) (#7279) 2023-09-27 16:29:09 +02:00
कारतोफ्फेलस्क्रिप्ट™ 5c57e2ccc3
ci: Update pnpm-lock.yaml (no-changelog) (#7273)
The lockfile got incorrectly formatted in
https://github.com/n8n-io/n8n/pull/7225.
2023-09-27 14:44:13 +02:00
Iván Ovejero fa845453bb
feat(core): Introduce object store service (#7225)
Depends on https://github.com/n8n-io/n8n/pull/7220 | Story:
[PAY-840](https://linear.app/n8n/issue/PAY-840/introduce-object-store-service-and-manager-for-binary-data)

This PR introduces an object store service for Enterprise edition. Note
that the service is tested but currently unused - it will be integrated
soon as a binary data manager, and later for execution data.
`amazonaws.com` in the host is temporarily hardcoded until we integrate
the service and test against AWS, Cloudflare and Backblaze, in the next
PR.

This is ready for review - the PR it depends on is approved and waiting
for CI.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-27 09:42:35 +02:00
Michael Auerswald 4b014286cf
fix(core): Make senderId required for all command messages (#7252)
all commands sent between main instance and workers need to contain a
server id to prevent senders from reacting to their own messages,
causing loops

this PR makes sure all sent messages contain a sender id by default as
part of constructing a sending redis client.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-26 13:58:06 +02:00
Iván Ovejero 6d6e2488c6
refactor(core): Generalize binary data manager interface (no-changelog) (#7164)
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)

This PR: 
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.

Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.

Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-22 17:22:12 +02:00
Val bf74f09d69
feat(core): Add Tournament as the new default expression evaluator (#6964)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-21 13:57:45 +01:00
Elias Meire d41546b899
fix(core): Make parsing of content-type and content-disposition headers more flexible (#7217)
fixes #7149
2023-09-20 14:40:06 +02:00
Michael Auerswald 9f797b96d8
feat(core): Add command to trigger license refresh on workers (#7184)
This PR implements the updated license SDK so that worker and webhook
instances do not auto-renew licenses any more.

Instead, they receive a `reloadLicense` command via the Redis client
that will fetch the updated license after it was saved on the main
instance

This also contains some refactoring with moving redis sub and pub
clients into the event bus directly, to prevent cyclic dependency
issues.
2023-09-17 11:05:54 +02:00
कारतोफ्फेलस्क्रिप्ट™ c18ba370d5
ci: Remove eslint diff plugin. Run full linting on PRs (no-changelog) (#7175) 2023-09-14 16:55:06 +02:00
कारतोफ्फेलस्क्रिप्ट™ 0a35025e5e
fix(Code Node): Upgrade vm2 to address CVE-2023-37466 (#7123)
[GH Advisory](https://github.com/advisories/GHSA-cchq-frgv-rjh5)

Actual fix
[here](26168e6dfe).
2023-09-07 14:18:15 +02:00
कारतोफ्फेलस्क्रिप्ट™ 273d0913fe
feat(HTTP Request Node): Determine binary file name from content-disposition headers (#7032)
Fixes: 

https://community.n8n.io/t/http-request-node-read-filename-from-content-disposition-header-when-downloading-files/13453

https://community.n8n.io/t/read-filename-from-content-disposition-header-when-downloading-files/22192
2023-09-06 12:38:37 +02:00
कारतोफ्फेलस्क्रिप्ट™ 689a77cc87
refactor(Spreadsheet File Node): Reduce memory usage when loading CSV files (#7095)
Co-authored-by: Michael Kret <michael.k@radency.com>
2023-09-06 11:58:00 +02:00
कारतोफ्फेलस्क्रिप्ट™ 2f647974f4
refactor: Use actual nodes/credentials in cli and core tests (no-changelog) (#7115) 2023-09-06 10:23:47 +02:00
Cornelius Suermann e26553f198
feat(core): Add support for floating licenses (#7090)
This PR updates the license-sdk to v2.5.0 which introduces optional
support for floating licenses.
2023-09-04 15:56:20 +02:00
कारतोफ्फेलस्क्रिप्ट™ 97f87ae0fc
refactor(core): Add addColumns and dropColumns to the migrations DSL (no-changelog) (#7073) 2023-09-01 15:19:49 +02:00
कारतोफ्फेलस्क्रिप्ट™ 2432dcc661
fix(core): Prevent bots from scanning the application (no-changelog) (#7066)
N8N-6878
2023-09-01 13:54:35 +02:00
कारतोफ्फेलस्क्रिप्ट™ a693b29134
ci: Fix prettier auto-formatting (no-changelog) (#7063) 2023-09-01 13:29:31 +02:00
कारतोफ्फेलस्क्रिप्ट™ d3f635657c
fix(MongoDB Node): Upgrade mongodb package to address CVE-2021-32050 (#7054)
[GH Advisory](https://github.com/advisories/GHSA-vxvm-qww3-2fh7)
2023-08-31 12:29:43 +02:00
कारतोफ्फेलस्क्रिप्ट™ dfe0fa65f8
fix(Code Node): Switch over to vm2 fork (#7018)
[Fork is here](https://github.com/n8n-io/vm2). Currently there are no
modifications in it.
2023-08-30 11:17:04 +02:00
Iván Ovejero a7785b2c5d
refactor: Upgrade to TypeScript 5.2 (no-changelog) (#7026)
https://devblogs.microsoft.com/typescript/announcing-typescript-5-2/

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-08-28 15:03:07 +02:00
कारतोफ्फेलस्क्रिप्ट™ 8bdb07d33d
fix(core): Setup websocket keep-live messages (#6866)
fixes #6757
2023-08-28 10:26:27 +02:00
Alex Grozav ed927d34b2
feat: External Secrets storage for credentials (#6477)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-08-25 10:33:46 +02:00
Ricardo Espinoza 2b7ba6fdf1
feat(core): Add MFA (#4767)
https://linear.app/n8n/issue/ADO-947/sync-branch-with-master-and-fix-fe-e2e-tets

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-08-23 22:59:16 -04:00
Iván Ovejero f32e993227
fix(editor): Fix parsing for single quoted resolvables (#6982)
Ref: https://github.com/n8n-io/codemirror-lang-n8n-sql/pull/6

<img width="574" alt="Capture 2023-08-21 at 10 08 53@2x"
src="https://github.com/n8n-io/n8n/assets/44588767/e1f88041-3577-41f3-b990-dbbf02741492">
2023-08-22 13:18:35 +02:00
Csaba Tuncsik 6422078a5d
fix(editor): Revert Fix telemetry initialization (no-changelog) (#6968)
Reverts commit b312f2ee54 and
808a92809e
2023-08-18 13:06:13 +02:00
Iván Ovejero 7ce5d8fd90
refactor: Upgrade to Prettier 3 (no-changelog) (#6947)
Supersedes https://github.com/n8n-io/n8n/pull/6937

Excluding fixtures and test workflow JSONs to avoid having to update
tests.
2023-08-16 17:13:57 +02:00
Csaba Tuncsik 808a92809e
fix(editor): Disable telemetry in dev mode and in E2E tests (#6869)
* fix(editor): Disable telemetry in dev mode and in E2E tests

* fix(editor): add external js to rollup config

* fix(editor): using vite plugin to control external script loading

* fix(editor): update lock file version

* fix(editor): block telemetry host
2023-08-11 14:42:25 +02:00
Jon 0e075c9cb5
fix: Fix issue with key formatting introduced in 1.2.0 (#6896) 2023-08-09 16:23:35 +01:00
Iván Ovejero b82cf5603a
ci: Upgrade nodelinter to 1.16.0 (no-changelog) (#6879)
ci: Upgrade nodelinter to 1.16.0
2023-08-09 10:10:08 +02:00
Michael Kret 72814d1f0f
fix(Email Trigger (IMAP) Node): UTF-8 attachments are not correctly named (#6856) 2023-08-07 13:33:06 +03:00
कारतोफ्फेलस्क्रिप्ट™ 75be1a9c0d
feat(core): Create a dsl for writing db agnostic migrations (#6853) 2023-08-04 17:49:02 +02:00
कारतोफ्फेलस्क्रिप्ट™ a5667e6c42
fix(core): Update packages to address CVE-2023-2142 and CVE-2020-28469 (#6844)
GH Advisories:
[ CVE-2023-2142](https://github.com/advisories/GHSA-x77j-w7wf-fjmw)
[CVE-2020-28469](https://github.com/advisories/GHSA-ww39-953v-wcq6)

---------

Co-authored-by: Marcus <marcus@n8n.io>
2023-08-03 16:31:55 +02:00
Michael Auerswald 3cad60e918
feat(core): Make Redis available for backend communication (#6719)
* support redis cluster

* cleanup, fix config schema

* set default prefix to bull

* initial commit

* improve logging

* improve types and refactor

* list support and refactor

* fix redis service and tests

* add comment

* add redis and cache prefix

* use injection

* lint fix

* clean schema comments

* improve naming, tests, cluster client

* merge master

* cache returns unknown instead of T

* update cache service, tests and doc

* remove console.log

* do not cache null or undefined values

* fix merge

* lint fix
2023-08-02 12:51:25 +02:00
कारतोफ्फेलस्क्रिप्ट™ 0ac3d47f26
ci: Fix linting (no-changelog) (#6831) 2023-08-02 11:05:24 +02:00
कारतोफ्फेलस्क्रिप्ट™ 34df8b6238
refactor: Consolidate tsconfig and eslintrc files (no-changelog) (#6816)
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
2023-08-01 17:32:43 +02:00
कारतोफ्फेलस्क्रिप्ट™ 31d8f478ee
refactor(core): Parse Webhook request bodies on-demand (#6394)
Also,
1. Consistent CORS support ~on all three webhook types~ waiting webhooks never supported CORS. I'll fix that in another PR
2. [Fixes binary-data handling when request body is text, json, or xml](https://linear.app/n8n/issue/NODE-505/webhook-binary-data-handling-fails-for-textplain-files).
3. Reduced number of middleware that each request has to go through.
4. Removed the need to maintain webhook endpoints in the auth-exception list.
5. Skip all middlewares (apart from `compression`) on Webhook routes. 
6. move `multipart/form-data` support out of individual nodes
7. upgrade `formidable`
8. fix the filenames on binary-data in webhooks nodes
9. add unit tests and integration tests for webhook request handling, and increase test coverage
2023-08-01 17:32:30 +02:00
Iván Ovejero dec19585bc
refactor: Enforce expanded sensitive inputs rules (no-changelog) (#6815)
* refactor: Enforce expanded sensitive inputs rules (no-changelog)

* refactor: Add extra exemption

* fix: Add setting to `sessionToken` fields

* fix: Restore for `hidden` fields

* fix: More edge case exemptions

* fix: One more
2023-08-01 13:08:25 +02:00
कारतोफ्फेलस्क्रिप्ट™ 37a365fe9f
ci: Downgrade prettier (no-changelog) (#6804)
* ci: downgrade prettier (no-changelog)

* remove indent rule from editor
2023-07-31 14:55:03 +02:00
कारतोफ्फेलस्क्रिप्ट™ 6fb8a9ee39
ci: Fix linting issues (no-changelog) (#6788)
* ci: Fix linting (no-changelog)

* lintfix for nodes-base as well
2023-07-28 18:28:17 +02:00