Commit graph

9303 commits

Author SHA1 Message Date
Iván Ovejero e8e44f6b6e
refactor(core): Log binary data file write errors (no-changelog) (#7237)
This PR adds logging for binary data file write errors, to capture why
executions sometimes point to non-existing binary data files. See
[Sentry
error](https://n8nio.sentry.io/issues/4495134693/?alert_rule_id=14556563&alert_type=issue&notification_uuid=4b50a5da-6ae9-472e-9658-984cca824762&project=4503924908883968&referrer=slack).
2023-09-22 11:48:20 +02:00
Jon c0df5cdfd4
fix(APITemplate.io Node): Fix action description for create pdf (no-changelog) (#7226)
Github issue / Community forum post (link here to close automatically):
https://github.com/n8n-io/n8n/issues/7224
2023-09-22 07:27:00 +01:00
Val bf74f09d69
feat(core): Add Tournament as the new default expression evaluator (#6964)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-21 13:57:45 +01:00
Elias Meire 67b985fe89
fix(core): Handle filename* with quotes in Content-Disposition header (#7229)
Github issue / Community forum post (link here to close automatically):
2023-09-21 14:54:10 +02:00
Michael Kret 2af967cf88
fix(Airtable Node): Attachments field type fix (#7227)
Github issue / Community forum post (link here to close automatically):

https://community.n8n.io/t/possible-airtable-bug-when-creating-attachment-record/29288
2023-09-21 15:00:41 +03:00
Ricardo Espinoza 303bc8e71e
fix: Issue enforcing user limits on start plan (#7188) 2023-09-21 05:56:40 -04:00
Milorad FIlipović b0e98b59a6
feat(editor): Rework banners framework and add email confirmation banner (#7205)
This PR introduces banner framework overhaul:
First version of the banner framework was built to allow multiple
banners to be shown at the same time. Since that proven to be the case
we don't need and it turned out to be pretty messy keeping only one
banner visible in such setup, this PR reworks it so it renders only one
banner at a time, based on [this priority
list](https://www.notion.so/n8n/Banner-stack-60948c4167c743718fde80d6745258d5?pvs=4#6afd052ec8d146a1b0fab8884a19add7)
that is assembled together with our product & design team.

### How to test banner stack:
1. Available banners and their priorities are registered
[here](f9f122d46d/packages/editor-ui/src/components/banners/BannerStack.vue (L14))
2. Banners are pushed to stack using `pushBannerToStack` action, for
example:
```
useUIStore().pushBannerToStack('TRIAL');
```
4. Try pushing different banners to stack and check if only the one with
highest priorities is showing up

### How to test the _Email confirmation_ banner:
1. Comment out [this
line](b80d2e3bec/packages/editor-ui/src/stores/cloudPlan.store.ts (L59)),
so cloud data is always fetched
2. Create an
[override](https://chrome.google.com/webstore/detail/resource-override/pkoacgokdfckfpndoffpifphamojphii)
(URL -> File) that will serve user data that triggers this banner:
- **URL**: `*/rest/cloud/proxy/admin/user/me`
- **File**:
```
{
    "confirmed": false,
    "id": 1,
    "email": "test@test.com",
    "username": "test"
}
```
3. Run n8n
2023-09-21 09:47:21 +02:00
Mutasem Aldmour 2491ccf4d9
fix: Ensure new Set node is on top of search list (#7215)
Github issue / Community forum post (link here to close automatically):
2023-09-20 16:47:51 +02:00
कारतोफ्फेलस्क्रिप्ट™ 8bb22292d5 Merge tag 'n8n@1.8.0' 2023-09-20 16:20:34 +02:00
Iván Ovejero cd08c8e4c6
refactor(core): Implement soft-deletions for executions (#7092)
Based on #7065 | Story: https://linear.app/n8n/issue/PAY-771

n8n on filesystem mode marks binary data to delete on manual execution
deletion, on unsaved execution completion, and on every execution
pruning cycle. We later prune binary data in a separate cycle via these
marker files, based on the configured TTL. In the context of introducing
an S3 client to manage binary data, the filesystem mode's mark-and-prune
setup is too tightly coupled to the general binary data management
client interface.

This PR...
- Ensures the deletion of an execution causes the deletion of any binary
data associated to it. This does away with the need for binary data TTL
and simplifies the filesystem mode's mark-and-prune setup.
- Refactors all execution deletions (including pruning) to cause soft
deletions, hard-deletes soft-deleted executions based on the existing
pruning config, and adjusts execution endpoints to filter out
soft-deleted executions. This reduces DB load, and keeps binary data
around long enough for users to access it when building workflows with
unsaved executions.
- Moves all execution pruning work from an execution lifecycle hook to
`execution.repository.ts`. This keeps related logic in a single place.
- Removes all marking logic from the binary data manager. This
simplifies the interface that the S3 client will meet.
- Adds basic sanity-check tests to pruning logic and execution deletion.

Out of scope:

- Improving existing pruning logic.
- Improving existing execution repository logic.
- Adjusting dir structure for filesystem mode.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-20 15:21:42 +02:00
Giulio Andreini 09a7cf0980
docs(Google Sheets Node): Operations naming update (no-changelog) (#7211)
Github issue / Community forum post (link here to close automatically):
2023-09-20 15:20:56 +02:00
github-actions[bot] ecd5d93c19
🚀 Release 1.8.0 (#7219)
# [1.8.0](https://github.com/n8n-io/n8n/compare/n8n@1.7.0...n8n@1.8.0)
(2023-09-20)


### Bug Fixes

* **core:** Make parsing of content-type and content-disposition headers
more flexible ([#7217](https://github.com/n8n-io/n8n/issues/7217))
([d41546b](d41546b899)),
closes [#7149](https://github.com/n8n-io/n8n/issues/7149)
* **core:** Resolve domains to IPv4 first
([#7206](https://github.com/n8n-io/n8n/issues/7206))
([e9ce531](e9ce531210))
* **editor:** Add ssh key type selection to source control settings when
regenerating key ([#7172](https://github.com/n8n-io/n8n/issues/7172))
([54bf66d](54bf66d335))
* **editor:** No need to add click emitting click events, VUE delegates
the handler to the root element of the component
([#7182](https://github.com/n8n-io/n8n/issues/7182))
([3c055e4](3c055e4d8d))
* **editor:** Prevent duplicate creation of credential for OAuth2
([#7163](https://github.com/n8n-io/n8n/issues/7163))
([07a6417](07a6417f0f))
* **editor:** Testing flaky resource mapper feature in e2e tests
([#7165](https://github.com/n8n-io/n8n/issues/7165))
([aaf87c3](aaf87c3edd))
* **HTML Node:** Add pairedItem support for 'Convert to HTML Table'
operation ([#7196](https://github.com/n8n-io/n8n/issues/7196))
([6bc477b](6bc477b50e))
* **HTTP Request Node:** Decrease default timeout to 5min
([#7177](https://github.com/n8n-io/n8n/issues/7177))
([321780d](321780d4a2))
* **seven Node:** Rename sms77 to seven, fix credentials test
([#7180](https://github.com/n8n-io/n8n/issues/7180))
([cf776b8](cf776b8f17))
* **X (Formerly Twitter) Node:** Rename Twitter to X (keep Twitter
alias) ([#7179](https://github.com/n8n-io/n8n/issues/7179))
([d317e09](d317e09c59))


### Features

* **core:** Add command to trigger license refresh on workers
([#7184](https://github.com/n8n-io/n8n/issues/7184))
([9f797b9](9f797b96d8))
* **core:** Add rsa option to ssh key generation
([#7154](https://github.com/n8n-io/n8n/issues/7154))
([fdac2c8](fdac2c8572))
* **Linear Node:** Add support for OAuth2
([#7201](https://github.com/n8n-io/n8n/issues/7201))
([12a3168](12a3168367))
* **Microsoft Outlook Node:** Node overhaul
([#4449](https://github.com/n8n-io/n8n/issues/4449))
([556a613](556a6132ba))
* **Set Node:** Overhaul
([#6348](https://github.com/n8n-io/n8n/issues/6348))
([3a47455](3a474552b2))

Co-authored-by: krynble <krynble@users.noreply.github.com>
2023-09-20 15:10:20 +02:00
Elias Meire d41546b899
fix(core): Make parsing of content-type and content-disposition headers more flexible (#7217)
fixes #7149
2023-09-20 14:40:06 +02:00
Marcus 6bc477b50e
fix(HTML Node): Add pairedItem support for 'Convert to HTML Table' operation (#7196)
Github issue / Community forum post (link here to close automatically):

Co-authored-by: Michael Kret <michael.k@radency.com>
2023-09-20 13:56:53 +02:00
Elias Meire 321780d4a2
fix(HTTP Request Node): Decrease default timeout to 5min (#7177)
Github issue / Community forum post (link here to close automatically):

fixes https://github.com/n8n-io/n8n/issues/7081
2023-09-20 12:38:34 +02:00
Jon 12a3168367
feat(Linear Node): Add support for OAuth2 (#7201) 2023-09-20 10:19:12 +01:00
Iván Ovejero 6b582d155c
refactor: Ignore large-scale revisions (no-changelog) (#7210)
https://git-scm.com/docs/git-blame#Documentation/git-blame.txt---ignore-revs-fileltfilegt
2023-09-19 16:37:59 +02:00
कारतोफ्फेलस्क्रिप्ट™ e9ce531210
fix(core): Resolve domains to IPv4 first (#7206) 2023-09-19 15:56:38 +02:00
Iván Ovejero 28598ed1a7
ci: Fix tests failing on MySQL (no-changelog) (#7208)
https://github.com/n8n-io/n8n/actions/runs/6229338666/job/16907667212
2023-09-19 15:42:39 +02:00
Michael Kret 3a474552b2
feat(Set Node): Overhaul (#6348)
Github issue / Community forum post (link here to close automatically):
https://github.com/n8n-io/n8n/pull/6348

---------

Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Marcus <marcus@n8n.io>
2023-09-19 13:16:35 +03:00
Cornelius Suermann 050ba706d3
fix: Attempt license renewal when n8n starts (no-changelog) (#7204)
This change is needed so that the newly introduced floating licenses
will be attempted to be claimed when n8n starts up.
2023-09-19 12:10:23 +02:00
Elias Meire cf776b8f17
fix(seven Node): Rename sms77 to seven, fix credentials test (#7180)
Github issue / Community forum post (link here to close automatically):
2023-09-19 10:26:53 +02:00
Jon 8b3aa0d68f
fix(Spotify Node): Fix typos in descriptions (no-changelog) (#7194)
Github issue / Community forum post (link here to close automatically):
https://github.com/n8n-io/n8n/issues/7191
2023-09-18 15:05:15 +01:00
Michael Auerswald 9f797b96d8
feat(core): Add command to trigger license refresh on workers (#7184)
This PR implements the updated license SDK so that worker and webhook
instances do not auto-renew licenses any more.

Instead, they receive a `reloadLicense` command via the Redis client
that will fetch the updated license after it was saved on the main
instance

This also contains some refactoring with moving redis sub and pub
clients into the event bus directly, to prevent cyclic dependency
issues.
2023-09-17 11:05:54 +02:00
Elias Meire d317e09c59
fix(X (Formerly Twitter) Node): Rename Twitter to X (keep Twitter alias) (#7179)
Github issue / Community forum post (link here to close automatically):
2023-09-15 15:04:16 +02:00
Csaba Tuncsik 240b2f075e
feat(editor): Add Workflow history route and base page (no-changelog) (#7161) 2023-09-15 13:17:04 +02:00
Michael Kret 556a6132ba
feat(Microsoft Outlook Node): Node overhaul (#4449)
[N8N-4995](https://linear.app/n8n/issue/N8N-4995)

---------

Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-09-15 12:52:18 +03:00
Iván Ovejero bb215bd12a
ci: Disable intermittently failing MFA e2e tests (no-changelog) (#7106)
MFA e2e tests keep intermittently failing, creating distractions

<img width="917" alt="Capture 2023-09-05 at 09 36 45@2x"
src="https://github.com/n8n-io/n8n/assets/44588767/baed58bf-7871-4c84-a9c1-60109f13bb93">

Followup story to fix them: https://linear.app/n8n/issue/N8N-6896
2023-09-15 11:33:10 +02:00
Csaba Tuncsik 3c055e4d8d
fix(editor): No need to add click emitting click events, VUE delegates the handler to the root element of the component (#7182) 2023-09-15 11:22:30 +02:00
कारतोफ्फेलस्क्रिप्ट™ c18ba370d5
ci: Remove eslint diff plugin. Run full linting on PRs (no-changelog) (#7175) 2023-09-14 16:55:06 +02:00
Csaba Tuncsik 54bf66d335
fix(editor): Add ssh key type selection to source control settings when regenerating key (#7172) 2023-09-14 14:40:34 +02:00
Michael Auerswald fdac2c8572
feat(core): Add rsa option to ssh key generation (#7154)
PR adds a new field to the SourceControlPreferences as well as to the
POST parameters for the `source-control/preferences` and
`source-control/generate-key-pair` endpoints. Both now accept an
optional string parameter `keyGeneratorType` of `'ed25519' | 'rsa'`

Calling the `source-control/generate-key-pair` endpoint with the
parameter set, it will also update the stored preferences accordingly
(so that in the future new keys will use the same method)

By default ed25519 is being used. The default may be changed using a new
environment parameter:

`N8N_SOURCECONTROL_DEFAULT_SSH_KEY_TYPE` which can be `rsa` or `ed25519`

RSA keys are generated with a length of 4096 bytes.
2023-09-14 11:34:51 +02:00
Milorad FIlipović aaf87c3edd
fix(editor): Testing flaky resource mapper feature in e2e tests (#7165) 2023-09-14 10:54:25 +02:00
Michael Auerswald 0c6169ee22
refactor(core): Move event and telemetry handling into workers in queue mode (#7138)
# Motivation

In Queue mode, finished executions would cause the main instance to
always pull all execution data from the database, unflatten it and then
use it to send out event log events and telemetry events, as well as
required returns to Respond to Webhook nodes etc.

This could cause OOM errors when the data was large, since it had to be
fully unpacked and transformed on the main instance’s side, using up a
lot of memory (and time).

This PR attempts to limit this behaviour to only happen in those
required cases where the data has to be forwarded to some waiting
webhook, for example.

# Changes

Execution data is only required in cases, where the active execution has
a `postExecutePromise` attached to it. These usually forward the data to
some other endpoint (e.g. a listening webhook connection).

By adding a helper `getPostExecutePromiseCount()`, we can decide that in
cases where there is nothing listening at all, there is no reason to
pull the data on the main instance.

Previously, there would always be postExecutePromises because the
telemetry events were called. Now, these have been moved into the
workers, which have been given the various InternalHooks calls to their
hook function arrays, so they themselves issue these telemetry and event
calls.

This results in all event log messages to now be logged on the worker’s
event log, as well as the worker’s eventbus being the one to send out
the events to destinations. The main event log does…pretty much nothing.

We are not logging executions on the main event log any more, because
this would require all events to be replicated 1:1 from the workers to
the main instance(s) (this IS possible and implemented, see the worker’s
`replicateToRedisEventLogFunction` - but it is not enabled to reduce the
amount of traffic over redis).

Partial events in the main log could confuse the recovery process and
would result in, ironically, the recovery corrupting the execution data
by considering them crashed.

# Refactor

I have also used the opportunity to reduce duplicate code and move some
of the hook functionality into
`packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts`
in preparation for a future full refactor of the hooks
2023-09-14 07:58:15 +02:00
Csaba Tuncsik 07a6417f0f
fix(editor): Prevent duplicate creation of credential for OAuth2 (#7163)
This PR solves the issue reported in [this
ticket](https://linear.app/n8n/issue/PAY-825/click-to-connect-for-google-sheets-does-not-work-when-cred-is-not).

Basically when trying to connect OAuth2 credentials, n8n would create
the credential twice, possibly causing transaction issues.

This PR prevents n8n from firing a duplicate creation request,
effectively fixing the bug about duplicate creation.
2023-09-13 17:28:43 +02:00
Milorad FIlipović c11dfbbe00
ci: Add reusable unit test workflow (no-changelog) (#7159)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-13 15:24:02 +02:00
github-actions[bot] a6e027b3ff
🚀 Release 1.7.0 (#7162)
# [1.7.0](https://github.com/n8n-io/n8n/compare/n8n@1.6.0...n8n@1.7.0)
(2023-09-13)


### Bug Fixes

* **Code Node:** Disable WASM to address CVE-2023-37903
([#7122](https://github.com/n8n-io/n8n/issues/7122))
([36a8e91](36a8e911e6))
* **Code Node:** Upgrade vm2 to address CVE-2023-37466
([#7123](https://github.com/n8n-io/n8n/issues/7123))
([0a35025](0a35025e5e))
* **core:** Disable Node.js custom inspection to address CVE-2023-37903
([#7125](https://github.com/n8n-io/n8n/issues/7125))
([a223734](a223734a4a))
* **editor** Account for nanoid workflow ids for subworkflow execute
policy ([#7094](https://github.com/n8n-io/n8n/issues/7094))
([67092c0](67092c0a1b))
* **editor:** Tweak hover area of workflow / cred cards
([#7108](https://github.com/n8n-io/n8n/issues/7108))
([217de21](217de21605))
* **editor:** Unbind workflow endpoint events in case of workspace reset
([#7129](https://github.com/n8n-io/n8n/issues/7129))
([c9b7948](c9b79485cf))
* **editor:** Update git repo url validation regex
([#7151](https://github.com/n8n-io/n8n/issues/7151))
([e51f173](e51f173608))
* **Google Cloud Firestore Node:** Fix empty string interpreted as
number ([#7136](https://github.com/n8n-io/n8n/issues/7136))
([915cfa0](915cfa0f6a))
* **HubSpot Node:** Fix issue with contact lists not working
([#5582](https://github.com/n8n-io/n8n/issues/5582))
([6e5a4f6](6e5a4f6a58))
* **Postgres Node:** Fix automatic column mapping
([#7121](https://github.com/n8n-io/n8n/issues/7121))
([92af131](92af1314fe))
* **Zoho CRM Node:** Fix issue with Sales Order not updating
([#6959](https://github.com/n8n-io/n8n/issues/6959))
([fd800b6](fd800b674b))


### Features

* **core:** Add an option to enable WAL mode for SQLite
([#7118](https://github.com/n8n-io/n8n/issues/7118))
([1d1a022](1d1a022def))
* **core:** Add commands to workers to respond with current state
([#7029](https://github.com/n8n-io/n8n/issues/7029))
([7b49cf2](7b49cf2a2c))
* **Salesforce Node:** Add fax field to lead option
([#7030](https://github.com/n8n-io/n8n/issues/7030))
([01f875a](01f875a94d))

Co-authored-by: netroy <netroy@users.noreply.github.com>
2023-09-13 14:55:52 +02:00
कारतोफ्फेलस्क्रिप्ट™ 34ebffea45
refactor(core): Move workerCommandHandler into worker.ts (no-changelog) (#7160) 2023-09-13 14:51:41 +02:00
Csaba Tuncsik 217de21605
fix(editor): Tweak hover area of workflow / cred cards (#7108)
Context

When a user is attempting to interact with a foreground action inside an
entity card (workflow, credential, community node, logging destination),
they might accidentally open that entity instead of interacting with a
foreground action.

For these card components, actions are always placed on right side. 

A/C

Area around right "column" of entity cards (workflow, cred, community
node, logging destination) should not be a hoverable area (that opens
that entity when clicked). This area is roughly highlighted in screen
shot below in orange.


![image](https://github.com/n8n-io/n8n/assets/5410822/0916bcd5-e972-4367-a862-41d2086a2334)
2023-09-13 12:21:26 +02:00
Omar Ajoue 67092c0a1b
fix: Account for nanoid workflow ids for subworkflow execute policy (#7094)
Github issue / Community forum post (link here to close automatically):

Since the change to allow workflow IDs to become strings in Nano ID
formats, this input broke.

This PR allows all characters that comprise workflow IDs.

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-09-13 09:56:58 +02:00
Csaba Tuncsik c9b79485cf
fix(editor): Unbind workflow endpoint events in case of workspace reset (#7129) 2023-09-13 09:00:35 +02:00
कारतोफ्फेलस्क्रिप्ट™ 22edc03cab
fix(core): Ignore missing user-agent on bot check (no-changelog) (#7153) 2023-09-12 19:57:25 +02:00
Jon 6e5a4f6a58
fix(HubSpot Node): Fix issue with contact lists not working (#5582) 2023-09-12 17:04:39 +01:00
Quang-Linh LE 915cfa0f6a
fix(Google Cloud Firestore Node): Fix empty string interpreted as number (#7136) 2023-09-12 10:03:33 +01:00
greyliath b67a6fc432
docs(editor): Update .round() function in NumberExtensions.ts for clarity (#7150) 2023-09-12 09:24:29 +01:00
Csaba Tuncsik e51f173608
fix(editor): Update git repo url validation regex (#7151) 2023-09-11 20:54:03 +02:00
Jon fd800b674b
fix(Zoho CRM Node): Fix issue with Sales Order not updating (#6959) 2023-09-11 17:15:52 +01:00
Jon 5c6cccd4fa
docs: Add concatenate alias to Split in Batches (#7133)
Github issue / Community forum post (link here to close automatically):
2023-09-08 12:49:16 +01:00
कारतोफ्फेलस्क्रिप्ट™ 1d1a022def
feat(core): Add an option to enable WAL mode for SQLite (#7118)
https://www.sqlite.org/wal.html
2023-09-07 15:58:48 +02:00
कारतोफ्फेलस्क्रिप्ट™ 67aaad15eb
refactor(core): Use a Set for deletedProperties in AugmentObject (no-changelog) (#7131) 2023-09-07 15:07:32 +02:00