Commit graph

9467 commits

Author SHA1 Message Date
Iván Ovejero 671c95760b
refactor(core): Make executions pruning more resilient (#7480)
This PR converts the hard-deletion interval to a timeout:
- to prevent the interval from not being restored when hard deletion
throws, and
- to prevent a long-running hard deletion from leading to duplicate
deletions.
2023-10-24 16:16:45 +02:00
Jon 05e6f2a6ac
fix(Spreadsheet File Node): Fix include empty cells not working with v2 (#7505)
Github issue / Community forum post (link here to close automatically):
Ticket#763644
2023-10-24 14:22:45 +01:00
कारतोफ्फेलस्क्रिप्ट™ 8ff9f97493
fix(core): Create instance settings directory recursively (no-changelog) (#7506) 2023-10-24 15:10:22 +02:00
Michael Kret a739245332
fix(Microsoft SQL Node): Prevent SQL injection (#7467)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-10-24 12:36:44 +03:00
Iván Ovejero 78243edd18
refactor(core): Make pruning via lifecycle configuration in S3 mode mandatory (#7482)
Since we do not store which executions produced binary data, for pruning
on S3 we need to query for binary data items for each execution in order
to delete them. To minimize requests to S3, allow the user to skip
pruning requests when setting TTL at bucket level.
2023-10-24 10:37:02 +02:00
कारतोफ्फेलस्क्रिप्ट™ a9fdd018f4
fix(core): Always derive instanceId from the encryption key (no-changlog) (#7501)
This was the expected behavior, until I changed it in
https://github.com/n8n-io/n8n/pull/7471
2023-10-24 09:55:57 +02:00
Val baecb93bef
fix(MQTT Trigger Node): Fix node causing a start up hang when active (#7498)
Github issue / Community forum post (link here to close automatically):
2023-10-24 09:37:49 +02:00
Cornelius Suermann 3ef771ea38
feat: Collect usage metrics on license renewal (no-changelog) (#7486) 2023-10-23 21:39:29 +02:00
Ricardo Espinoza 774fe202bf
fix(core): Fix frontend.settings external hook execution (#7496) 2023-10-23 11:46:13 -04:00
Burak Akgün 4e66023cd4
fix(Redis Node): Fix adding sets data types (#7444)
Fixes #6339
2023-10-23 16:33:44 +01:00
Val 41236b7e08
fix: Save new version of the workflow instead of the previous (no-changelog) (#7428)
Github issue / Community forum post (link here to close automatically):
2023-10-23 16:30:36 +02:00
कारतोफ्फेलस्क्रिप्ट™ b6de910cbe
refactor(core): Abstract away InstanceSettings and encryptionKey into injectable services (no-changelog) (#7471)
This change ensures that things like `encryptionKey` and `instanceId`
are always available directly where they are needed, instead of passing
them around throughout the code.
2023-10-23 13:39:35 +02:00
Léo Martinez 519680c2cf
fix(Customer.io Node): Fix api endpoint when using EU region (#7485)
Fixes #7484
2023-10-23 10:01:48 +01:00
MC Naveen 7b1973c058
feat(Ghost Node): Add support for lexical format (#7488) 2023-10-23 10:00:38 +01:00
OlegIvaniv 450e0cc66a
fix(editor): Fix connections disappearing after reactivating canvas and renaming a node (#7483)
Github issue / Community forum post (link here to close automatically):
-
https://community.n8n.io/t/1-11-1-possible-bug-all-nodes-randomly-losing-their-connectors/31856
- https://community.n8n.io/t/lines-between-nodes-have-disappeared/31846

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
2023-10-23 10:07:33 +02:00
कारतोफ्फेलस्क्रिप्ट™ 76c04815f7
fix(core): Reduce logging overhead for levels that do not output (#7479)
all current logging calls execute `callsites()` to figure out what code
tried to log. This happens even for logging methods that aren't supposed
to create any output. Under moderate load, this can take up quite a lot
of resources. This PR changes the logger to make all ignorable logging
methods a No-Op.

In a small benchmark with a simple webhook, with log-level set to
`warn`, and using `ab -c 50 -n 500
http://localhost:5678/webhook/testing`, these were the response times:

### Before

![Before](https://github.com/n8n-io/n8n/assets/196144/01680fd9-3d2a-4f7f-bb1c-5b03bd7d5bc3)

### After

![After](https://github.com/n8n-io/n8n/assets/196144/ccacb20a-48ca-455a-a8cb-098c9c0e352e)
2023-10-20 18:26:33 +02:00
Iván Ovejero 0b42d1aa71
refactor(core): Limit soft-deletions to pruning only (#7469)
Based on customer feedback, we should limit soft deletions to pruning
only, to prevent executions from piling up in very high volume cases.
2023-10-20 15:02:47 +02:00
कारतोफ्फेलस्क्रिप्ट™ 3c0a166f7f
fix(editor): Allow importing the same workflow multiple times (#7458)
Fixes #7457
2023-10-20 13:58:30 +02:00
Elias Meire ac814a9c61
feat(Facebook Lead Ads Trigger Node): Add Facebook Lead Ads Trigger Node (#7113)
Github issue / Community forum post (link here to close automatically):
https://community.n8n.io/t/facebook-lead-ads-integration/4590/19

---------

Co-authored-by: Marcus <marcus@n8n.io>
2023-10-20 13:43:55 +02:00
Jon 647372be27
feat(core): Add support for oauth based service accounts with UM SMTP (#7311)
This PR adds support for using OAuth based service accounts for the User
Management SMTP connection.

Tested using a Google Service Account.
2023-10-20 11:36:40 +01:00
कारतोफ्फेलस्क्रिप्ट™ 91dfc4d513
feat(core): Update LLM applications building support (no-changelog) (#7418)
extracted out of #7336

---------

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: OlegIvaniv <me@olegivaniv.com>
Co-authored-by: Jan Oberhauser <janober@users.noreply.github.com>
Co-authored-by: Val <68596159+valya@users.noreply.github.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Deborah <deborah@starfallprojects.co.uk>
Co-authored-by: Jesper Bylund <mail@jesperbylund.com>
Co-authored-by: Jon <jonathan.bennetts@gmail.com>
2023-10-20 10:52:56 +02:00
Iván Ovejero 34f3f8001e
fix(n8n Form Trigger Node): Fix typos (no-changelog) (#7472) 2023-10-20 10:49:58 +02:00
Michael Kret 5c46bb09c1
fix(MySQL Node): Resolve expressions in v1 (#7464)
Github issue / Community forum post (link here to close automatically):
2023-10-20 11:09:39 +03:00
Milorad FIlipović bc473655fb
fix(editor): Fix canvas selection breaking after interacting with node actions (#7466)
Sometimes canvas selection stops working after users interact with node
action buttons (for example if node is moved by dragging one of the
buttons)
NOTE: Ticket number in the branch name is wrong, this fixes ADO-1226
2023-10-20 09:26:40 +02:00
Iván Ovejero b50376cf52
refactor(core): Make executions pruning settings configurable (#7468) 2023-10-19 16:57:12 +02:00
Csaba Tuncsik c7c8048430
feat(editor): Workflow history [WIP] - Remove pinned data from workflow history version preview (no-changelog) (#7406) 2023-10-19 14:38:00 +02:00
Csaba Tuncsik 82129694c6
feat(editor): Workflow history [WIP] - Add cloned workflow link to success toast message (no-changelog) (#7405) 2023-10-19 14:02:59 +02:00
कारतोफ्फेलस्क्रिप्ट™ 55c6a1b0d3
fix(core): Do not return inviteAcceptUrl in response if email was sent (#7465) 2023-10-19 13:58:06 +02:00
Michael Kret ab6a9bbac2
fix(Google Sheets Node): Append or update runs forever when without column headers (#7463)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Marcus <marcus@n8n.io>
2023-10-19 14:54:13 +03:00
Jon 689360ee06
feat(RSS Feed Trigger Node): Add RSS feed trigger node (#7386)
Github issue / Community forum post (link here to close automatically):
https://community.n8n.io/t/rss-trigger/14203
2023-10-19 08:49:18 +01:00
कारतोफ्फेलस्क्रिप्ट™ 83762e051d
fix(core): Handle gzip and deflate compressed request payloads (#7461)
NODE-870
2023-10-18 21:52:15 +02:00
github-actions[bot] ef58a23d21
🚀 Release 1.12.0 (#7459)
# [1.12.0](https://github.com/n8n-io/n8n/compare/n8n@1.11.0...n8n@1.12.0)
(2023-10-18)


### Bug Fixes

* **core:** Add check that queue is defined and remove cyclic dependency
([#7404](https://github.com/n8n-io/n8n/issues/7404))
([45f2ef3](45f2ef373e))
* **core:** Do not throw when deleting workflows with executions without
binary-data ([#7411](https://github.com/n8n-io/n8n/issues/7411))
([2b6a15e](2b6a15e478))
* **core:** Fix expression with paired item with multi-input node
([#7424](https://github.com/n8n-io/n8n/issues/7424))
([ec14141](ec141416e2))
* **core:** Fix ignoring crashed executions without event msgs
([#7368](https://github.com/n8n-io/n8n/issues/7368))
([2f4d91b](2f4d91b2cd))
* **core:** Pg-promise de-initialization fix
([#7417](https://github.com/n8n-io/n8n/issues/7417))
([7703904](77039044eb))
* **core:** Prevent false stalled jobs in queue mode from displaying as
errored ([#7435](https://github.com/n8n-io/n8n/issues/7435))
([e01b9e5](e01b9e5ae1))
* **core:** Prevent undefined issues when restoring binary data
([#7419](https://github.com/n8n-io/n8n/issues/7419))
([46977a2](46977a2aff))
* **editor:** Fix remote options fetching on every keystroke
([#7320](https://github.com/n8n-io/n8n/issues/7320))
([367255a](367255ab2c))
* **editor:** Open only one tab with plans page
([#7377](https://github.com/n8n-io/n8n/issues/7377))
([c599006](c599006b91))
* **Google Sheets Node:** Update by row_number, restored 'Handling Extra
Data Option', updated Cell Format default
([#7357](https://github.com/n8n-io/n8n/issues/7357))
([d8531a5](d8531a53b9))
* **Ldap Node:** Fix issue with connections not closing correctly
([#7432](https://github.com/n8n-io/n8n/issues/7432))
([c3f0be8](c3f0be809f))
* **Set Node:** Null should not throw an error
([#7416](https://github.com/n8n-io/n8n/issues/7416))
([e9b6ab0](e9b6ab04cd))
* **TheHive 5 Node:** Observable encoding in alert > create fix
([#7450](https://github.com/n8n-io/n8n/issues/7450))
([a2d2e3d](a2d2e3dda7))


### Features

* **core:** Make executions pruning interval configurable
([#7439](https://github.com/n8n-io/n8n/issues/7439))
([40707fa](40707fa692))
* **Google Calendar Trigger Node:** Add support for cancelled events
([#7436](https://github.com/n8n-io/n8n/issues/7436))
([9d241a0](9d241a0d6d))
* **HubSpot Trigger Node:** Add support for ticket related events
([#7156](https://github.com/n8n-io/n8n/issues/7156))
([57c6093](57c609384a))
* **n8n Form Trigger Node:** New node
([#7130](https://github.com/n8n-io/n8n/issues/7130))
([3ddc176](3ddc176dfa))
* **Spreadsheet File Node:** Improve CSV parsing
([#7448](https://github.com/n8n-io/n8n/issues/7448))
([79f23fb](79f23fb939))

Co-authored-by: netroy <netroy@users.noreply.github.com>
2023-10-18 17:43:29 +02:00
Iván Ovejero 40707fa692
feat(core): Make executions pruning interval configurable (#7439) 2023-10-18 17:01:57 +02:00
कारतोफ्फेलस्क्रिप्ट™ 79f23fb939
feat(Spreadsheet File Node): Improve CSV parsing (#7448)
This adds support for
1. custom delimiters
2. reading offsets to avoid having to read a large CSV all at once
3. excluding byte-order-mark

NODE-861
#7443
2023-10-18 16:57:37 +02:00
Michael Kret d8531a53b9
fix(Google Sheets Node): Tweaks (#7357)
Github issue / Community forum post (link here to close automatically):
2023-10-17 18:41:30 +03:00
Michael Kret a2d2e3dda7
fix(TheHive 5 Node): Observable encoding in alert > create fix (#7450)
Github issue / Community forum post (link here to close automatically):
2023-10-17 16:53:44 +03:00
Michael Kret 3ddc176dfa
feat(n8n Form Trigger Node): New node (#7130)
Github issue / Community forum post (link here to close automatically):

based on https://github.com/joffcom/n8n-nodes-form-trigger

---------

Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-10-17 07:09:30 +03:00
Deborah 869b8f14ca
fix(Convert to/from binary data Node): Change Mime to MIME (no-changelog) (#7446)
Github issue / Community forum post (link here to close automatically):
2023-10-16 16:54:16 +01:00
Jon 9d241a0d6d
feat(Google Calendar Trigger Node): Add support for cancelled events (#7436)
Github issue / Community forum post (link here to close automatically):
https://community.n8n.io/t/google-calendar-trigger-deleted-events/10612

This PR adds support for triggering workflows when Google Calendar
events are cancelled.
2023-10-16 14:09:23 +01:00
Damian Karzon 57c609384a
feat(HubSpot Trigger Node): Add support for ticket related events (#7156) 2023-10-13 16:52:22 +01:00
Michael Auerswald e01b9e5ae1
fix(core): Prevent false stalled jobs in queue mode from displaying as errored (#7435)
This is related to an issue with how Bull handles stalled jobs, see
https://github.com/OptimalBits/bull/issues/1415 for reference.

CPU intensive workflows can in certain cases take a long while to finish
up, thereby blocking the thread and causing Bull queue to think the job
has stalled, even though it finished successfully. In these cases the
error handling could then overwrite the successful execution data with
the error message.
2023-10-13 16:07:08 +02:00
Inga c599006b91
fix(editor): Open only one tab with plans page (#7377)
## Issue

In community edition, clicking on "View plans" button on "Settings" ->
"Usage and plan" page (e.g. http://127.0.0.1:5678/settings/usage) opens
two new tabs with n8n pricing (one of them with UTM tracking, another
without).
This was introduced in #6317 , when click handler of "View plans" link
container [started
calling](https://github.com/n8n-io/n8n/pull/6317/files#diff-0bf26afac8a06e03b3d39d0668f22408859355b585a9ab420800c125e33f0691R109)
`uiStore.goToUpgrade(...)` which opens n8n pricing in a new tab, while
browser opens another tab for the link URL.

The simplest fix, implemented in this PR, is to prevent default event
handling (so that, after `onViewPlans` is called, browser will not
attempt to process the click additionally as clicking on the link),
similarly to how it is prevented on some other pages. It only solves the
immediate problem of browser opening two new tabs on clicking "View
plans".

Note that **I didn't implement any tests for the changed behavior**,
because it was not covered by tests before, and I couldn't quite figure
out how to cover it now within the existing test approach (considering
that testing the fact that only one new tab is open will likely require
to write entirely new tests relying on puppeteer; as far as I can see,
no existing `editor-ui` tests are doing anything like that). I'll gladly
implement tests for the new behavior if you tell me how you would like
them to look.
The existing tests for `editor-ui` still pass; I didn't run tests for
other subpackages (see "additional contribution notes" below).

## Additional notes on the issue.

I'm not sure that the change in this PR is the correct long-term
solution for the issue, because the URLs for these two methods (custom
click handler for link container and default link handling) are slightly
different:
* Custom click handler calls `useTelemetryStore().track('User clicked
upgrade CTA', ...)`; then calls `sendUsageTelemetry('view_plans')` (it
feels weird that two calls to telemetry are made); then opens new tab
for `https://n8n.io/pricing?utm_campaign=open&source=usage_page` (note
that prior to #7316 the second call to telemetry was done after the new
tab is opened, not before);
* Link itself refers to another page, with slightly different tracking
parameters:
`https://subscription.n8n.io/?instanceid=[REDACTED]&version=1.10.0&callback=http%3A%2F%2F127.0.0.1%3A5678%2Fsettings%2Fusage&source=usage_page`;
but this page redirects to `https://n8n.io/pricing/`.
It is not clear which one of the two is the right way of doing things.
Although `goToUpgrade` is called in 20 places throughout `editor-ui`,
while `viewPlansUrl`, as far as I can see, is used for this button only.

Additionally, since Settings pages don't work without JS anyway, I can
only think of two separate scenarios where any tab would be opened:
* Left-clicking the link (or Ctrl-clicking, or pressing Space or Enter
when the link is focused, or tapping): previously, both custom click
handler was executed and link's `href` was opened; in this PR, only
custom click handler is executed (similarly to how it is done in the
other places where `goToUpgrade` is called);
* Right-clicking (or long tapping, or opening context menu in any other
way) and selecting "open link in new tab" (or similar): opens a new tab
for URL from the `href` attribute (and does not send any telemetry at
all).

I'd say that the better permanent solution would probably be to get rid
of one of these methods entirely, and only rely on another in all cases
(for me, as an outside contributor, the preferred way would be for
custom click handler to only send telemetry, while letting my browser
handle the actual navigation). However, that would be a large change,
much more than one line in this PR.
Additionally, other similar places where `goToUpgrade` is currently
called (directly or indirectly) would also need to be adapted for this
change.

## Additional contribution notes

As a first-time contributor, I've encountered several things I didn't
expect; I'm not sure if they should be expected or are issues:

1. Tests for the entire monorepo consume a lot of RAM; 20GB free RAM was
not enough, so I couldn't run tests for the entire monorepo and had to
only run them for `packages/editor-ui`;
2. Linting is very slow; `pnpm lint` in `packages/editor-ui` takes ten
minutes to complete;
3. It seems that types are not checked. Code OSS highlights numerous
errors in code files: for example, `'debug'` is incompatible with
`CloudUpdateLinkSourceType` expected by `goToUpgrade` here:
3e7a4d3b2c/packages/editor-ui/src/composables/useExecutionDebugging.ts (L128)
However, I'm not getting any errors during build. There is a `typecheck`
script defined in `package.json`, but `pnpm typecheck` fails with:
    ```
    n8n-toy-demo:~/projects/n8n/packages/editor-ui$ pnpm typecheck

> n8n-editor-ui@1.10.0 typecheck
/home/inga/projects/n8n/packages/editor-ui
    > vue-tsc --emitDeclarationOnly

error TS5069: Option 'emitDeclarationOnly' cannot be specified without
specifying
    option 'declaration' or option 'composite'.


    Found 1 error.

     ELIFECYCLE  Command failed with exit code 1.
    n8n-toy-demo:~/projects/n8n/packages/editor-ui$ 
    ```
Replacing `--emitDeclarationsOnly` with `--noEmit` in `package.json`
unblocks typechecking and results in seemingly, at first glance, correct
"Found 1924 errors in 306 files" (at least several of the reported
errors that I've checked seem to be correct).
But maybe I'm missing something and there are not in fact two thousands
type errors in `editor-ui`?
2023-10-13 14:14:26 +02:00
Iván Ovejero c6ee1e30c0
refactor(core): Add binary data S3 to telemetry (#7412)
https://n8nio.slack.com/archives/C04B1GZ4T0U/p1697033523039729
2023-10-13 13:16:43 +02:00
pemontto c3f0be809f
fix(Ldap Node): Fix issue with connections not closing correctly (#7432) 2023-10-13 12:15:54 +01:00
Kamil Essekkat bd5ea455e1
docs: Mention UID in the docker examples (#7281) 2023-10-13 11:56:43 +01:00
Milorad FIlipović 1307d488b9
fix: Fix flaky remote options loading test (no-changelog) (#7433)
Background: Looks like remote options loading needs a bit of time to
catch up when dependent parameter changes so when using cypress `type`
method without delay it fails to update the values properly. Since this
does not represent actual component behavior when real user interact
with it we added a delay to automated typing.

This PR increases typing delay, giving the component more time to
breathe when run in CI
- [Latest e2e
run](https://github.com/n8n-io/n8n/actions/runs/6506583716)
- [Video of this specific
test](https://cloud.cypress.io/projects/5hbsdn/runs/aa4f1a7d-b95f-4c8a-acd6-33061d290c7b/test-results/21a1abdf-1e1c-43bf-a640-8cc39b11cc46/video)
2023-10-13 12:30:45 +02:00
Michael Auerswald 45f2ef373e
fix(core): Add check that queue is defined and remove cyclic dependency (#7404)
In a rare edge case an undefined queue could be returned - this should
not happen and now an error is thrown.
Also using the opportunity to remove a cyclic dependency from the Queue.
2023-10-13 11:53:59 +02:00
Iván Ovejero 609f0837cf
test(core): Add test for suppressing error on non-existing filepath (no-changelog) (#7421)
Follow-up to #7411
2023-10-13 11:52:17 +02:00
Milorad FIlipović 1fb5166376
ci: Fix wrong ref in PR workflows which broke external builds (#7431)
Github issue / Community forum post (link here to close automatically):
#7423

This PR updates reference passed to the `checkout` action by the
`cy-pull-request.ym`. This should fix three existing issues:
- Failing unit tests for external pull requests
- Failing e2e tests for external PRs
- Passing empty `ref` to `lint` job which makes linter run on a wrong
branch
2023-10-13 10:44:26 +02:00
Iván Ovejero ec141416e2
fix(core): Fix expression with paired item with multi-input node (#7424)
https://linear.app/n8n/issue/PAY-630

Minimal repro:
https://internal.users.n8n.cloud/workflow/9HSkIy4T1LqXbm1H
2023-10-12 17:32:14 +02:00