कारतोफ्फेलस्क्रिप्ट™
76c04815f7
fix(core): Reduce logging overhead for levels that do not output ( #7479 )
...
all current logging calls execute `callsites()` to figure out what code
tried to log. This happens even for logging methods that aren't supposed
to create any output. Under moderate load, this can take up quite a lot
of resources. This PR changes the logger to make all ignorable logging
methods a No-Op.
In a small benchmark with a simple webhook, with log-level set to
`warn`, and using `ab -c 50 -n 500
http://localhost:5678/webhook/testing `, these were the response times:
### Before
### After
2023-10-20 18:26:33 +02:00
Iván Ovejero
0b42d1aa71
refactor(core): Limit soft-deletions to pruning only ( #7469 )
...
Based on customer feedback, we should limit soft deletions to pruning
only, to prevent executions from piling up in very high volume cases.
2023-10-20 15:02:47 +02:00
Jon
647372be27
feat(core): Add support for oauth based service accounts with UM SMTP ( #7311 )
...
This PR adds support for using OAuth based service accounts for the User
Management SMTP connection.
Tested using a Google Service Account.
2023-10-20 11:36:40 +01:00
Iván Ovejero
b50376cf52
refactor(core): Make executions pruning settings configurable ( #7468 )
2023-10-19 16:57:12 +02:00
कारतोफ्फेलस्क्रिप्ट™
55c6a1b0d3
fix(core): Do not return inviteAcceptUrl in response if email was sent ( #7465 )
2023-10-19 13:58:06 +02:00
कारतोफ्फेलस्क्रिप्ट™
83762e051d
fix(core): Handle gzip and deflate compressed request payloads ( #7461 )
...
NODE-870
2023-10-18 21:52:15 +02:00
github-actions[bot]
ef58a23d21
🚀 Release 1.12.0 ( #7459 )
...
# [1.12.0](https://github.com/n8n-io/n8n/compare/n8n@1.11.0...n8n@1.12.0 )
(2023-10-18)
### Bug Fixes
* **core:** Add check that queue is defined and remove cyclic dependency
([#7404 ](https://github.com/n8n-io/n8n/issues/7404 ))
([45f2ef345f2ef373e#7411 ](https://github.com/n8n-io/n8n/issues/7411 ))
([2b6a15e2b6a15e478#7424 ](https://github.com/n8n-io/n8n/issues/7424 ))
([ec14141ec141416e2#7368 ](https://github.com/n8n-io/n8n/issues/7368 ))
([2f4d91b2f4d91b2cd#7417 ](https://github.com/n8n-io/n8n/issues/7417 ))
([770390477039044eb#7435 ](https://github.com/n8n-io/n8n/issues/7435 ))
([e01b9e5e01b9e5ae1#7419 ](https://github.com/n8n-io/n8n/issues/7419 ))
([46977a246977a2aff#7320 ](https://github.com/n8n-io/n8n/issues/7320 ))
([367255a367255ab2c#7377 ](https://github.com/n8n-io/n8n/issues/7377 ))
([c599006c599006b91#7357 ](https://github.com/n8n-io/n8n/issues/7357 ))
([d8531a5d8531a53b9#7432 ](https://github.com/n8n-io/n8n/issues/7432 ))
([c3f0be8c3f0be809f#7416 ](https://github.com/n8n-io/n8n/issues/7416 ))
([e9b6ab0e9b6ab04cd#7450 ](https://github.com/n8n-io/n8n/issues/7450 ))
([a2d2e3da2d2e3dda7#7439 ](https://github.com/n8n-io/n8n/issues/7439 ))
([40707fa40707fa692#7436 ](https://github.com/n8n-io/n8n/issues/7436 ))
([9d241a09d241a0d6d#7156 ](https://github.com/n8n-io/n8n/issues/7156 ))
([57c609357c609384a#7130 ](https://github.com/n8n-io/n8n/issues/7130 ))
([3ddc1763ddc176dfa#7448 ](https://github.com/n8n-io/n8n/issues/7448 ))
([79f23fb79f23fb939
2023-10-18 17:43:29 +02:00
Iván Ovejero
40707fa692
feat(core): Make executions pruning interval configurable ( #7439 )
2023-10-18 17:01:57 +02:00
Michael Kret
3ddc176dfa
feat(n8n Form Trigger Node): New node ( #7130 )
...
Github issue / Community forum post (link here to close automatically):
based on https://github.com/joffcom/n8n-nodes-form-trigger
---------
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-10-17 07:09:30 +03:00
Michael Auerswald
e01b9e5ae1
fix(core): Prevent false stalled jobs in queue mode from displaying as errored ( #7435 )
...
This is related to an issue with how Bull handles stalled jobs, see
https://github.com/OptimalBits/bull/issues/1415 for reference.
CPU intensive workflows can in certain cases take a long while to finish
up, thereby blocking the thread and causing Bull queue to think the job
has stalled, even though it finished successfully. In these cases the
error handling could then overwrite the successful execution data with
the error message.
2023-10-13 16:07:08 +02:00
Iván Ovejero
c6ee1e30c0
refactor(core): Add binary data S3 to telemetry ( #7412 )
...
https://n8nio.slack.com/archives/C04B1GZ4T0U/p1697033523039729
2023-10-13 13:16:43 +02:00
Michael Auerswald
45f2ef373e
fix(core): Add check that queue is defined and remove cyclic dependency ( #7404 )
...
In a rare edge case an undefined queue could be returned - this should
not happen and now an error is thrown.
Also using the opportunity to remove a cyclic dependency from the Queue.
2023-10-13 11:53:59 +02:00
Omar Ajoue
46977a2aff
fix: Prevent undefined issues when restoring binary data ( #7419 )
...
Github issue / Community forum post (link here to close automatically):
---------
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-10-12 16:41:51 +02:00
Iván Ovejero
54e900955a
refactor(core): Move frontend settings to service (no-changelog) ( #7396 )
...
Quick follow-up to https://github.com/n8n-io/n8n/pull/7283
2023-10-11 17:12:19 +02:00
कारतोफ्फेलस्क्रिप्ट™
1e0753516c
Merge tag 'n8n@1.11.0'
2023-10-11 15:55:52 +02:00
Michael Auerswald
2f4d91b2cd
fix(core): Fix ignoring crashed executions without event msgs ( #7368 )
...
when the event logs do not contain messages for running executions, the
recovery/crash detection on startup would skip these. this PR fixes
that.
2023-10-11 14:22:43 +02:00
github-actions[bot]
e6ba841c5f
🚀 Release 1.11.0 ( #7402 )
...
# [1.11.0](https://github.com/n8n-io/n8n/compare/n8n@1.10.0...n8n@1.11.0 )
(2023-10-11)
### Bug Fixes
* **core:** Add an option to enable postgres ssl with default certs
([#6889 ](https://github.com/n8n-io/n8n/issues/6889 ))
([789e1e7789e1e7ed4#7399 ](https://github.com/n8n-io/n8n/issues/7399 ))
([47e895347e8953ec9#7326 ](https://github.com/n8n-io/n8n/issues/7326 ))
([e2c3c7ae2c3c7aceb#7366 ](https://github.com/n8n-io/n8n/issues/7366 ))
([63e11e463e11e4be9#7382 ](https://github.com/n8n-io/n8n/issues/7382 ))
([86e7ec786e7ec796a#7376 ](https://github.com/n8n-io/n8n/issues/7376 ))
([7e06b317e06b31a5f#7392 ](https://github.com/n8n-io/n8n/issues/7392 ))
([7ed466d7ed466db7f#7372 ](https://github.com/n8n-io/n8n/issues/7372 ))
([3fa27643fa27647d8#7400 ](https://github.com/n8n-io/n8n/issues/7400 ))
([2b075bf2b075bfc2d#7351 ](https://github.com/n8n-io/n8n/issues/7351 ))
([afbf0c3afbf0c3d5e#7354 ](https://github.com/n8n-io/n8n/issues/7354 ))
([97bb70397bb703d0a#7369 ](https://github.com/n8n-io/n8n/issues/7369 ))
([3e7a4d33e7a4d3b2c#7384 ](https://github.com/n8n-io/n8n/issues/7384 ))
([732b15a732b15a1fa#7379 ](https://github.com/n8n-io/n8n/issues/7379 ))
([77643e577643e5ccb#7383 ](https://github.com/n8n-io/n8n/issues/7383 ))
([fbcd1d4fbcd1d40ed#7358 ](https://github.com/n8n-io/n8n/issues/7358 ))
([e5ad1e7e5ad1e7e4d#7385 ](https://github.com/n8n-io/n8n/issues/7385 ))
([6479eb16479eb180f#7360 ](https://github.com/n8n-io/n8n/issues/7360 ))
([b8608ceb8608cee6d#7253 ](https://github.com/n8n-io/n8n/issues/7253 ))
([1a661e61a661e6d00#7307 ](https://github.com/n8n-io/n8n/issues/7307 ))
([0847623](0847623f85#7367 ](https://github.com/n8n-io/n8n/issues/7367 ))
([8187be18187be1b7d#7316 ](https://github.com/n8n-io/n8n/issues/7316 ))
([1dfa0521dfa052301#7289 ](https://github.com/n8n-io/n8n/issues/7289 ))
([c8c14cac8c14ca0af#7297 ](https://github.com/n8n-io/n8n/issues/7297 ))
([965db8f965db8f7f2#7228 ](https://github.com/n8n-io/n8n/issues/7228 ))
([7b773cc7b773cc5cc#7304 ](https://github.com/n8n-io/n8n/issues/7304 ))
([193181a193181a9c6
2023-10-11 13:38:34 +02:00
कारतोफ्फेलस्क्रिप्ट™
6479eb180f
fix(Webhook Node): Backward compatible form-data parsing for non-array files ( #7385 )
...
Fixes
https://community.n8n.io/t/possible-bug-0-added-to-end-of-files-sent-via-webhook/31169
2023-10-10 15:19:05 +02:00
Iván Ovejero
0847623f85
feat(core): Switch binary filesystem mode to nested path structure ( #7307 )
...
Depends on #7253 | Story:
[PAY-863](https://linear.app/n8n/issue/PAY-863/switch-binary-filesystem-mode-to-nested-path-structure )
This PR introduces `filesystem-v2` to store binary data in the
filesystem in the same format as `s3`.
2023-10-10 10:06:06 +02:00
कारतोफ्फेलस्क्रिप्ट™
c5ee06cc61
refactor(core): Refactor nodes loading (no-changelog) ( #7283 )
...
fixes PAY-605
2023-10-09 16:09:23 +02:00
कारतोफ्फेलस्क्रिप्ट™
789e1e7ed4
fix(core): Add an option to enable postgres ssl with default certs ( #6889 )
...
CP-809
2023-10-09 13:55:03 +02:00
Michael Auerswald
b8608cee6d
feat(core): Add Job Summary to Worker response ( #7360 )
2023-10-06 17:52:27 +02:00
Iván Ovejero
34bda535e6
refactor(core): Create controller for binary data (no-changelog) ( #7363 )
...
This PR adds a controller for binary data + integration tests.
2023-10-06 16:21:13 +02:00
Michael Auerswald
afa683a06f
refactor(core): Have one orchestration service per instance type ( #7303 )
...
webhook instances will not listen to either worker or event log messages
on the Redis pub/sub channel
2023-10-06 13:58:11 +02:00
Cornelius Suermann
f4d8c9eed5
feat: Improve error msg when attempting to redeem the same activation code multiple times (no-changelog) ( #7355 )
2023-10-05 17:08:30 +02:00
Iván Ovejero
1a661e6d00
feat(core): Integrate object store as binary data manager ( #7253 )
...
Depends on: #7225 | Story:
[PAY-848](https://linear.app/n8n/issue/PAY-848 )
This PR integrates the object store service as a new binary data manager
for Enterprise.
2023-10-05 15:25:17 +02:00
github-actions[bot]
aa1bf95136
🚀 Release 1.10.0 ( #7350 )
...
# [1.10.0](https://github.com/n8n-io/n8n/compare/n8n@1.9.0...n8n@1.10.0 )
(2023-10-05)
### Bug Fixes
* **Convert to/from binary data Node:** Rename 'Move Binary Data' to
'Convert to/from binary data'
([#7318 ](https://github.com/n8n-io/n8n/issues/7318 ))
([5e6c1d45e6c1d4f4b#7305 ](https://github.com/n8n-io/n8n/issues/7305 ))
([16912231691223789#7333 ](https://github.com/n8n-io/n8n/issues/7333 ))
([1b4848a1b4848afcb#7340 ](https://github.com/n8n-io/n8n/issues/7340 ))
([6d3d1786d3d1789db#7298 ](https://github.com/n8n-io/n8n/issues/7298 ))
([3aac22b3aac22b4c1#7346 ](https://github.com/n8n-io/n8n/issues/7346 ))
([b59b908b59b9086d7#7290 ](https://github.com/n8n-io/n8n/issues/7290 ))
([7ebf8f37ebf8f327a#7295 ](https://github.com/n8n-io/n8n/issues/7295 ))
([ddc26c2ddc26c21bd#7312 ](https://github.com/n8n-io/n8n/issues/7312 ))
([04dfcd704dfcd73be#7322 ](https://github.com/n8n-io/n8n/issues/7322 ))
([37047603704760724#2670 ](https://github.com/n8n-io/n8n/issues/2670 )
* **Notion Node:** Rename Notion API Key to Internal Integration Token
([#7176 ](https://github.com/n8n-io/n8n/issues/7176 ))
([ec2aa38ec2aa3819c#7300 ](https://github.com/n8n-io/n8n/issues/7300 ))
([763d451763d4514fa#7313 ](https://github.com/n8n-io/n8n/issues/7313 ))
([f0a6687f0a66873b9#7315 ](https://github.com/n8n-io/n8n/issues/7315 ))
([b3fc00eb3fc00e045#7183 ](https://github.com/n8n-io/n8n/issues/7183 ))
([8f9fe628f9fe6269b#7235 ](https://github.com/n8n-io/n8n/issues/7235 ))
([00a4b8b00a4b8b0c6#7246 ](https://github.com/n8n-io/n8n/issues/7246 )
[#7137 ](https://github.com/n8n-io/n8n/issues/7137 )
* Workflow History pruning and prune time settings
([#7343 ](https://github.com/n8n-io/n8n/issues/7343 ))
([0adc5330adc533719
2023-10-05 14:12:37 +02:00
Omar Ajoue
5cfd87692f
refactor: Add log messages to debug worker init issues ( #7238 )
...
Github issue / Community forum post (link here to close automatically):
2023-10-05 13:37:25 +02:00
कारतोफ्फेलस्क्रिप्ट™
169175080a
ci: Fix WorkflowHistoryManager tests (no-changelog) ( #7356 )
...
[DB Tests](https://github.com/n8n-io/n8n/actions/runs/6418058186 )
2023-10-05 13:14:57 +02:00
Tomi Turtiainen
1b4848afcb
fix(core): Fix pruning of non-finished executions ( #7333 )
...
This fixes a bug in the pruning (soft-delete). The pruning was a bit too
aggressive, as it also pruned executions that weren't in an end state
yet. This only becomes an issue if there are long-running executions
(e.g. workflow with Wait node) or the prune parameters are set to keep
only a tiny number of executions.
2023-10-04 16:32:05 +03:00
Val
0adc533719
feat: Workflow History pruning and prune time settings ( #7343 )
...
Github issue / Community forum post (link here to close automatically):
2023-10-04 13:57:21 +01:00
कारतोफ्फेलस्क्रिप्ट™
101255d186
ci: Make builds release-channel aware. Add support for scheduled beta builds ( #7323 )
...
ADO-1121
---------
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2023-10-03 20:49:04 +02:00
कारतोफ्फेलस्क्रिप्ट™
00a4b8b0c6
feat(core): Add support for building LLM applications ( #7235 )
...
This extracts all core and editor changes from #7246 and #7137 , so that
we can get these changes merged first.
ADO-1120
[DB Tests](https://github.com/n8n-io/n8n/actions/runs/6379749011 )
[E2E Tests](https://github.com/n8n-io/n8n/actions/runs/6379751480 )
[Workflow Tests](https://github.com/n8n-io/n8n/actions/runs/6379752828 )
---------
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-10-02 17:33:43 +02:00
Iván Ovejero
1691223789
fix(core): Account for itemless case on restoring binary data ID ( #7305 )
...
https://linear.app/n8n/issue/PAY-862
2023-10-02 09:13:55 +02:00
Csaba Tuncsik
d1b6c7fd79
feat(editor): Workflow history [WIP]- create workflow history list component (no-changelog) ( #7186 )
...
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-09-29 17:48:36 +02:00
Omar Ajoue
ec0379378e
feat(core): Add plan name to telemetry (no-changelog) ( #7296 )
...
Github issue / Community forum post (link here to close automatically):
2023-09-29 15:41:32 +02:00
कारतोफ्फेलस्क्रिप्ट™
6a1557bc50
Merge tag 'n8n@1.9.0'
2023-09-29 14:28:06 +02:00
कारतोफ्फेलस्क्रिप्ट™
f8406c04b1
ci: Fix typescript incremental builds (no-changelog) ( #7275 )
...
`tsBuildInfoFile` is supposed to be relative to `tsconfig` like `outDir`
is.
Because of this, we are currently saving the TS incremental build cache
for all packages in the same file. This is likely causing issues where
the built backend code sometimes does not accurately map to the current
source code.
This PR changes the incremental build setup to keep the cache in
individual `dist` folders, like it used to be up until a 2 months ago,
before https://github.com/n8n-io/n8n/pull/6816 .
2023-09-29 13:26:06 +02:00
github-actions[bot]
167124ceb8
🚀 Release 1.9.0 ( #7288 )
...
# [1.9.0](https://github.com/n8n-io/n8n/compare/n8n@1.8.0...n8n@1.9.0 )
(2023-09-28)
### Bug Fixes
* **Airtable Node:** Attachments field type fix
([#7227 ](https://github.com/n8n-io/n8n/issues/7227 ))
([2af967c2af967cf88#7282 ](https://github.com/n8n-io/n8n/issues/7282 ))
([a80abada80abad3af#7251 ](https://github.com/n8n-io/n8n/issues/7251 ))
([484035e484035eb51#7244 ](https://github.com/n8n-io/n8n/issues/7244 ))
([33991e933991e92d0#7229 ](https://github.com/n8n-io/n8n/issues/7229 ))
([67b985f67b985fe89#7272 ](https://github.com/n8n-io/n8n/issues/7272 ))
([5b3121c5b3121c415#7252 ](https://github.com/n8n-io/n8n/issues/7252 ))
([4b014284b014286cf#7261 ](https://github.com/n8n-io/n8n/issues/7261 ))
([861cac5861cac5257#6948 ](https://github.com/n8n-io/n8n/issues/6948 ))
([01325140132514f8b#2178 ](https://github.com/n8n-io/n8n/issues/2178 )
[#2810 ](https://github.com/n8n-io/n8n/issues/2810 )
[#3855 ](https://github.com/n8n-io/n8n/issues/3855 )
[#2813 ](https://github.com/n8n-io/n8n/issues/2813 )
* **editor:** Add debug feature docs link
([#7240 ](https://github.com/n8n-io/n8n/issues/7240 ))
([4614e1e4614e1e1c9#7236 ](https://github.com/n8n-io/n8n/issues/7236 ))
([647fc6c647fc6c555#7215 ](https://github.com/n8n-io/n8n/issues/7215 ))
([2491ccf2491ccf4d9https://github.com/n8n-io/n8n/issues/7188 )
([303bc8e303bc8e71e#7293 ](https://github.com/n8n-io/n8n/issues/7293 ))
([0bc33b10bc33b1cc2#7286 ](https://github.com/n8n-io/n8n/issues/7286 ))
([cde23a1cde23a1bb1#7294 ](https://github.com/n8n-io/n8n/issues/7294 ))
([8bc369d8bc369dd40#7212 ](https://github.com/n8n-io/n8n/issues/7212 ))
([01e934001e9340621#7277 ](https://github.com/n8n-io/n8n/issues/7277 ))
([53a750253a7502d20#6964 ](https://github.com/n8n-io/n8n/issues/6964 ))
([bf74f09bf74f09d69#7234 ](https://github.com/n8n-io/n8n/issues/7234 ))
([0083a9e0083a9e45d#7225 ](https://github.com/n8n-io/n8n/issues/7225 ))
([fa84545fa845453bb#7232 ](https://github.com/n8n-io/n8n/issues/7232 )
([60c152d60c152dc72#7205 ](https://github.com/n8n-io/n8n/issues/7205 ))
([b0e98b5b0e98b59a6#7268 ](https://github.com/n8n-io/n8n/issues/7268 ))
([e4c302ce4c302c683#7260 ](https://github.com/n8n-io/n8n/issues/7260 ))
([db01164db01164ce1
2023-09-28 17:22:28 +02:00
कारतोफ्फेलस्क्रिप्ट™
cf5a0ca456
ci: Refactor DB tests (no-changelog) ( #7292 )
...
[DB tests](https://github.com/n8n-io/n8n/actions/runs/6340094467 )
2023-09-28 16:53:05 +02:00
Michael Auerswald
53a7502d20
feat(core): Add secrets provider reload and refactor ( #7277 )
...
This PR adds a message for queue mode which triggers an external secrets
provider reload inside the workers if the configuration has changed on
the main instance.
It also refactors some of the message handler code to remove cyclic
dependencies, as well as remove unnecessary duplicate redis clients
inside services (thanks to no more cyclic deps)
2023-09-28 12:57:35 +02:00
Val
a80abad3af
fix(core): Change WorkflowHistory nodes/connections columns to be json ( #7282 )
...
Github issue / Community forum post (link here to close automatically):
2023-09-28 10:37:33 +02:00
कारतोफ्फेलस्क्रिप्ट™
0132514f8b
fix(core): Use consistent timezone-aware timestamps in postgres ( #6948 )
...
Fixes:
* ENG-51 / N8N-2490
* PAY-397
* #2178
* #2810
* #3855
Supersedes #2813
[DB
Tests](https://github.com/n8n-io/n8n/actions/runs/6000780146/job/16273596338 )
2023-09-27 18:44:47 +02:00
कारतोफ्फेलस्क्रिप्ट™
ebce6fe1b0
refactor(core): Skip sending webhook activation errors to Sentry (no-changelog) ( #7171 )
2023-09-27 16:57:52 +02:00
Michael Auerswald
07d072c28f
fix(core): Bump License SDK Version (no-changelog) ( #7279 )
2023-09-27 16:29:09 +02:00
Val
0083a9e45d
feat(core): Initial workflow history API ( #7234 )
...
Github issue / Community forum post (link here to close automatically):
2023-09-27 15:22:39 +01:00
कारतोफ्फेलस्क्रिप्ट™
5b3121c415
fix(core): Make DNS resolution order configurable ( #7272 )
2023-09-27 12:19:18 +02:00
Michael Auerswald
6d7fe95c58
fix(core): Do not set ttl globally in test (no-changelog) ( #7271 )
2023-09-27 12:03:15 +02:00
Iván Ovejero
fa845453bb
feat(core): Introduce object store service ( #7225 )
...
Depends on https://github.com/n8n-io/n8n/pull/7220 | Story:
[PAY-840](https://linear.app/n8n/issue/PAY-840/introduce-object-store-service-and-manager-for-binary-data )
This PR introduces an object store service for Enterprise edition. Note
that the service is tested but currently unused - it will be integrated
soon as a binary data manager, and later for execution data.
`amazonaws.com` in the host is temporarily hardcoded until we integrate
the service and test against AWS, Cloudflare and Backblaze, in the next
PR.
This is ready for review - the PR it depends on is approved and waiting
for CI.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-27 09:42:35 +02:00
Iván Ovejero
db01164ce1
perf(core): Skip unneeded calls on every pruning cycle ( #7260 )
2023-09-26 16:53:38 +02:00
Michael Auerswald
4b014286cf
fix(core): Make senderId required for all command messages ( #7252 )
...
all commands sent between main instance and workers need to contain a
server id to prevent senders from reacting to their own messages,
causing loops
this PR makes sure all sent messages contain a sender id by default as
part of constructing a sending redis client.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-26 13:58:06 +02:00
Iván Ovejero
77d6e3fc07
refactor(core): Include workflow ID in binary data writes (no-changelog) ( #7220 )
...
Depends on: https://github.com/n8n-io/n8n/pull/7195 | Story:
[PAY-837](https://linear.app/n8n/issue/PAY-837/implement-object-store-manager-for-binary-data )
This PR includes `workflowId` in binary data writes so that the S3
manager can support this filepath structure
`/workflows/{workflowId}/executions/{executionId}/binaryData/{binaryFilename}`
to easily delete binary data for workflows. Also all binary data service
and manager methods that take `workflowId` and `executionId` are made
consistent in arg order.
Note: `workflowId` is included in filesystem mode for compatibility with
the common interface, but `workflowId` will remain unused by filesystem
mode until we decide to restructure how this mode stores data.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-25 18:04:52 +02:00
Iván Ovejero
75541e91f2
refactor(core)!: Make getBinaryStream async ( #7247 )
...
Story: [PAY-846](https://linear.app/n8n/issue/PAY-846 ) | Related:
https://github.com/n8n-io/n8n/pull/7225
For the S3 backend for external storage of binary data and execution
data, the `getAsStream` method in the binary data manager interface used
by FS and S3 will need to become async. This is a breaking change for
nodes-base.
2023-09-25 16:59:45 +02:00
Ricardo Espinoza
d47986aec3
fix: Bug when trailing user reset password on instance (no-changelog) ( #7250 )
...
Github issue / Community forum post (link here to close automatically):
2023-09-25 09:55:33 -04:00
Mutasem Aldmour
01e9340621
feat: Add onboarding flow ( #7212 )
...
Github issue / Community forum post (link here to close automatically):
2023-09-25 15:49:36 +02:00
Iván Ovejero
33991e92d0
fix(core): Fix missing execution ID in webhook-based workflow producing binary data ( #7244 )
...
Story: https://linear.app/n8n/issue/PAY-839
This is a longstanding bug, fixed now so that the S3 backend for binary
data can use execution IDs as part of the filename.
To reproduce:
1. Set up a workflow with a POST Webhook node that accepts binary data.
2. Activate the workflow and call it sending a binary file, e.g. `curl
-X POST -F "file=@/path/to/binary/file/test.jpg"
http://localhost:5678/webhook/uuid `
3. Check `~/.n8n/binaryData`. The binary data and metadata files will be
missing the execution ID, e.g. `11869055-83c4-4493-876a-9092c4708b9b`
instead of `39011869055-83c4-4493-876a-9092c4708b9b`.
2023-09-25 12:30:28 +02:00
Iván Ovejero
6d6e2488c6
refactor(core): Generalize binary data manager interface (no-changelog) ( #7164 )
...
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768 )
This PR:
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.
Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.
Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-22 17:22:12 +02:00
Val
bf74f09d69
feat(core): Add Tournament as the new default expression evaluator ( #6964 )
...
Github issue / Community forum post (link here to close automatically):
---------
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-21 13:57:45 +01:00
Ricardo Espinoza
303bc8e71e
fix: Issue enforcing user limits on start plan ( #7188 )
2023-09-21 05:56:40 -04:00
कारतोफ्फेलस्क्रिप्ट™
8bb22292d5
Merge tag 'n8n@1.8.0'
2023-09-20 16:20:34 +02:00
Iván Ovejero
cd08c8e4c6
refactor(core): Implement soft-deletions for executions ( #7092 )
...
Based on #7065 | Story: https://linear.app/n8n/issue/PAY-771
n8n on filesystem mode marks binary data to delete on manual execution
deletion, on unsaved execution completion, and on every execution
pruning cycle. We later prune binary data in a separate cycle via these
marker files, based on the configured TTL. In the context of introducing
an S3 client to manage binary data, the filesystem mode's mark-and-prune
setup is too tightly coupled to the general binary data management
client interface.
This PR...
- Ensures the deletion of an execution causes the deletion of any binary
data associated to it. This does away with the need for binary data TTL
and simplifies the filesystem mode's mark-and-prune setup.
- Refactors all execution deletions (including pruning) to cause soft
deletions, hard-deletes soft-deleted executions based on the existing
pruning config, and adjusts execution endpoints to filter out
soft-deleted executions. This reduces DB load, and keeps binary data
around long enough for users to access it when building workflows with
unsaved executions.
- Moves all execution pruning work from an execution lifecycle hook to
`execution.repository.ts`. This keeps related logic in a single place.
- Removes all marking logic from the binary data manager. This
simplifies the interface that the S3 client will meet.
- Adds basic sanity-check tests to pruning logic and execution deletion.
Out of scope:
- Improving existing pruning logic.
- Improving existing execution repository logic.
- Adjusting dir structure for filesystem mode.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-20 15:21:42 +02:00
github-actions[bot]
ecd5d93c19
🚀 Release 1.8.0 ( #7219 )
...
# [1.8.0](https://github.com/n8n-io/n8n/compare/n8n@1.7.0...n8n@1.8.0 )
(2023-09-20)
### Bug Fixes
* **core:** Make parsing of content-type and content-disposition headers
more flexible ([#7217 ](https://github.com/n8n-io/n8n/issues/7217 ))
([d41546bd41546b899#7149 ](https://github.com/n8n-io/n8n/issues/7149 )
* **core:** Resolve domains to IPv4 first
([#7206 ](https://github.com/n8n-io/n8n/issues/7206 ))
([e9ce531e9ce531210#7172 ](https://github.com/n8n-io/n8n/issues/7172 ))
([54bf66d54bf66d335#7182 ](https://github.com/n8n-io/n8n/issues/7182 ))
([3c055e43c055e4d8d#7163 ](https://github.com/n8n-io/n8n/issues/7163 ))
([07a641707a6417f0f#7165 ](https://github.com/n8n-io/n8n/issues/7165 ))
([aaf87c3aaf87c3edd#7196 ](https://github.com/n8n-io/n8n/issues/7196 ))
([6bc477b6bc477b50e#7177 ](https://github.com/n8n-io/n8n/issues/7177 ))
([321780d321780d4a2#7180 ](https://github.com/n8n-io/n8n/issues/7180 ))
([cf776b8cf776b8f17#7179 ](https://github.com/n8n-io/n8n/issues/7179 ))
([d317e09d317e09c59#7184 ](https://github.com/n8n-io/n8n/issues/7184 ))
([9f797b99f797b96d8#7154 ](https://github.com/n8n-io/n8n/issues/7154 ))
([fdac2c8fdac2c8572#7201 ](https://github.com/n8n-io/n8n/issues/7201 ))
([12a316812a3168367#4449 ](https://github.com/n8n-io/n8n/issues/4449 ))
([556a613556a6132ba#6348 ](https://github.com/n8n-io/n8n/issues/6348 ))
([3a474553a474552b2
2023-09-20 15:10:20 +02:00
कारतोफ्फेलस्क्रिप्ट™
e9ce531210
fix(core): Resolve domains to IPv4 first ( #7206 )
2023-09-19 15:56:38 +02:00
Iván Ovejero
28598ed1a7
ci: Fix tests failing on MySQL (no-changelog) ( #7208 )
...
https://github.com/n8n-io/n8n/actions/runs/6229338666/job/16907667212
2023-09-19 15:42:39 +02:00
Cornelius Suermann
050ba706d3
fix: Attempt license renewal when n8n starts (no-changelog) ( #7204 )
...
This change is needed so that the newly introduced floating licenses
will be attempted to be claimed when n8n starts up.
2023-09-19 12:10:23 +02:00
Michael Auerswald
9f797b96d8
feat(core): Add command to trigger license refresh on workers ( #7184 )
...
This PR implements the updated license SDK so that worker and webhook
instances do not auto-renew licenses any more.
Instead, they receive a `reloadLicense` command via the Redis client
that will fetch the updated license after it was saved on the main
instance
This also contains some refactoring with moving redis sub and pub
clients into the event bus directly, to prevent cyclic dependency
issues.
2023-09-17 11:05:54 +02:00
Csaba Tuncsik
240b2f075e
feat(editor): Add Workflow history route and base page (no-changelog) ( #7161 )
2023-09-15 13:17:04 +02:00
Michael Auerswald
fdac2c8572
feat(core): Add rsa option to ssh key generation ( #7154 )
...
PR adds a new field to the SourceControlPreferences as well as to the
POST parameters for the `source-control/preferences` and
`source-control/generate-key-pair` endpoints. Both now accept an
optional string parameter `keyGeneratorType` of `'ed25519' | 'rsa'`
Calling the `source-control/generate-key-pair` endpoint with the
parameter set, it will also update the stored preferences accordingly
(so that in the future new keys will use the same method)
By default ed25519 is being used. The default may be changed using a new
environment parameter:
`N8N_SOURCECONTROL_DEFAULT_SSH_KEY_TYPE` which can be `rsa` or `ed25519`
RSA keys are generated with a length of 4096 bytes.
2023-09-14 11:34:51 +02:00
Michael Auerswald
0c6169ee22
refactor(core): Move event and telemetry handling into workers in queue mode ( #7138 )
...
# Motivation
In Queue mode, finished executions would cause the main instance to
always pull all execution data from the database, unflatten it and then
use it to send out event log events and telemetry events, as well as
required returns to Respond to Webhook nodes etc.
This could cause OOM errors when the data was large, since it had to be
fully unpacked and transformed on the main instance’s side, using up a
lot of memory (and time).
This PR attempts to limit this behaviour to only happen in those
required cases where the data has to be forwarded to some waiting
webhook, for example.
# Changes
Execution data is only required in cases, where the active execution has
a `postExecutePromise` attached to it. These usually forward the data to
some other endpoint (e.g. a listening webhook connection).
By adding a helper `getPostExecutePromiseCount()`, we can decide that in
cases where there is nothing listening at all, there is no reason to
pull the data on the main instance.
Previously, there would always be postExecutePromises because the
telemetry events were called. Now, these have been moved into the
workers, which have been given the various InternalHooks calls to their
hook function arrays, so they themselves issue these telemetry and event
calls.
This results in all event log messages to now be logged on the worker’s
event log, as well as the worker’s eventbus being the one to send out
the events to destinations. The main event log does…pretty much nothing.
We are not logging executions on the main event log any more, because
this would require all events to be replicated 1:1 from the workers to
the main instance(s) (this IS possible and implemented, see the worker’s
`replicateToRedisEventLogFunction` - but it is not enabled to reduce the
amount of traffic over redis).
Partial events in the main log could confuse the recovery process and
would result in, ironically, the recovery corrupting the execution data
by considering them crashed.
# Refactor
I have also used the opportunity to reduce duplicate code and move some
of the hook functionality into
`packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts`
in preparation for a future full refactor of the hooks
2023-09-14 07:58:15 +02:00
github-actions[bot]
a6e027b3ff
🚀 Release 1.7.0 ( #7162 )
...
# [1.7.0](https://github.com/n8n-io/n8n/compare/n8n@1.6.0...n8n@1.7.0 )
(2023-09-13)
### Bug Fixes
* **Code Node:** Disable WASM to address CVE-2023-37903
([#7122 ](https://github.com/n8n-io/n8n/issues/7122 ))
([36a8e9136a8e911e6#7123 ](https://github.com/n8n-io/n8n/issues/7123 ))
([0a350250a35025e5e#7125 ](https://github.com/n8n-io/n8n/issues/7125 ))
([a223734a223734a4a#7094 ](https://github.com/n8n-io/n8n/issues/7094 ))
([67092c067092c0a1b#7108 ](https://github.com/n8n-io/n8n/issues/7108 ))
([217de21217de21605#7129 ](https://github.com/n8n-io/n8n/issues/7129 ))
([c9b7948c9b79485cf#7151 ](https://github.com/n8n-io/n8n/issues/7151 ))
([e51f173e51f173608#7136 ](https://github.com/n8n-io/n8n/issues/7136 ))
([915cfa0915cfa0f6a#5582 ](https://github.com/n8n-io/n8n/issues/5582 ))
([6e5a4f66e5a4f6a58#7121 ](https://github.com/n8n-io/n8n/issues/7121 ))
([92af13192af1314fe#6959 ](https://github.com/n8n-io/n8n/issues/6959 ))
([fd800b6fd800b674b#7118 ](https://github.com/n8n-io/n8n/issues/7118 ))
([1d1a0221d1a022def#7029 ](https://github.com/n8n-io/n8n/issues/7029 ))
([7b49cf27b49cf2a2c#7030 ](https://github.com/n8n-io/n8n/issues/7030 ))
([01f875a01f875a94d
2023-09-13 14:55:52 +02:00
कारतोफ्फेलस्क्रिप्ट™
34ebffea45
refactor(core): Move workerCommandHandler into worker.ts (no-changelog) ( #7160 )
2023-09-13 14:51:41 +02:00
कारतोफ्फेलस्क्रिप्ट™
22edc03cab
fix(core): Ignore missing user-agent on bot check (no-changelog) ( #7153 )
2023-09-12 19:57:25 +02:00
कारतोफ्फेलस्क्रिप्ट™
1d1a022def
feat(core): Add an option to enable WAL mode for SQLite ( #7118 )
...
https://www.sqlite.org/wal.html
2023-09-07 15:58:48 +02:00
Michael Auerswald
7b49cf2a2c
feat(core): Add commands to workers to respond with current state ( #7029 )
...
This PR adds new endpoints to the REST API:
`/orchestration/worker/status` and `/orchestration/worker/id`
Currently these just trigger the return of status / ids from the workers
via the redis back channel, this still needs to be handled and passed
through to the frontend.
It also adds the eventbus to each worker, and triggers a reload of those
eventbus instances when the configuration changes on the main instances.
2023-09-07 14:44:19 +02:00
कारतोफ्फेलस्क्रिप्ट™
a223734a4a
fix(core): Disable Node.js custom inspection to address CVE-2023-37903 ( #7125 )
...
This seems like a better fix than #7122
2023-09-07 10:25:59 +02:00
github-actions[bot]
955bd327ff
🚀 Release 1.6.0 ( #7120 )
...
# [1.6.0](https://github.com/n8n-io/n8n/compare/n8n@1.5.1...n8n@1.6.0 )
(2023-09-06)
### Bug Fixes
* **core:** Add support for in-transit encryption (TLS) on Redis
connections ([#7047 ](https://github.com/n8n-io/n8n/issues/7047 ))
([a910757a910757cc5#7069 ](https://github.com/n8n-io/n8n/issues/7069 ))
([8a28e988a28e98ec8#7107 ](https://github.com/n8n-io/n8n/issues/7107 ))
([011ee2e011ee2e04b#7093 ](https://github.com/n8n-io/n8n/issues/7093 ))
([58b349258b3492b0d#7061 ](https://github.com/n8n-io/n8n/issues/7061 ))
([b8e06d2b8e06d245f#6987 ](https://github.com/n8n-io/n8n/issues/6987 ))
([8a8d4e88a8d4e8bb3#7074 ](https://github.com/n8n-io/n8n/issues/7074 ))
([9dd5f0e9dd5f0e579#7087 ](https://github.com/n8n-io/n8n/issues/7087 ))
([58e55ba58e55ba669#7041 ](https://github.com/n8n-io/n8n/issues/7041 ))
([fd78021fd78021b68#7090 ](https://github.com/n8n-io/n8n/issues/7090 ))
([e26553fe26553f198#7088 ](https://github.com/n8n-io/n8n/issues/7088 ))
([413e0bc413e0bccb4#7032 ](https://github.com/n8n-io/n8n/issues/7032 ))
([273d091273d0913fe#6457 ](https://github.com/n8n-io/n8n/issues/6457 ))
([73e782e73e782e2cf
2023-09-06 13:35:31 +02:00
कारतोफ्फेलस्क्रिप्ट™
273d0913fe
feat(HTTP Request Node): Determine binary file name from content-disposition headers ( #7032 )
...
Fixes:
https://community.n8n.io/t/http-request-node-read-filename-from-content-disposition-header-when-downloading-files/13453
https://community.n8n.io/t/read-filename-from-content-disposition-header-when-downloading-files/22192
2023-09-06 12:38:37 +02:00
Omar Ajoue
25dc4d7825
feat: Add workflow history repository files (no-changelog) ( #7071 )
2023-09-06 12:23:40 +02:00
कारतोफ्फेलस्क्रिप्ट™
2f647974f4
refactor: Use actual nodes/credentials in cli and core tests (no-changelog) ( #7115 )
2023-09-06 10:23:47 +02:00
कारतोफ्फेलस्क्रिप्ट™
a910757cc5
fix(core): Add support for in-transit encryption (TLS) on Redis connections ( #7047 )
...
Fixes https://community.n8n.io/t/upgrade-from-1-1-1-to-1-2-0/30148
2023-09-05 18:18:50 +02:00
कारतोफ्फेलस्क्रिप्ट™
cc37a5046a
fix(core): Lazy-load express-openapi-validator to reduce initial memory spike (no-changelog) ( #7111 )
2023-09-05 14:42:14 +02:00
कारतोफ्फेलस्क्रिप्ट™
36f246929e
ci: Fix linting (no-changelog) ( #7110 )
2023-09-05 14:41:56 +02:00
Iván Ovejero
0036a4726c
refactor(core): Move static data handling to workflow service (no-changelog) ( #7104 )
2023-09-05 13:42:31 +02:00
Michael Auerswald
011ee2e04b
fix(core): Split event bus controller into community and ee ( #7107 )
2023-09-05 13:32:09 +02:00
कारतोफ्फेलस्क्रिप्ट™
6aa7b93473
refactor(core): Deprecate prepareOutputData (no-changelog) ( #7091 )
2023-09-05 12:59:02 +02:00
Iván Ovejero
b7320f5322
refactor(core): Set up ExecutionMetadata service (no-changelog) ( #7103 )
...
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-05 09:13:30 +02:00
कारतोफ्फेलस्क्रिप्ट™
cffda65b33
refactor(core): Add an option to use simple recovery process by default ( #7097 )
2023-09-04 19:58:36 +02:00
कारतोफ्फेलस्क्रिप्ट™
ee36f2d20b
refactor(core): Disable slow-query log by default (no-changelog) ( #7096 )
...
Fixes:
https://community.n8n.io/t/stop-logging-full-query-for-execution/29732
[Slack
Conversation](https://n8nio.slack.com/archives/C035KBDA917/p1693218994681939 )
2023-09-04 17:59:08 +02:00
कारतोफ्फेलस्क्रिप्ट™
f286bd33c1
ci: Fix linting (no-changelog) ( #7099 )
2023-09-04 17:07:11 +02:00
Iván Ovejero
8a28e98ec8
fix(core): Disallow orphan executions ( #7069 )
...
Until https://github.com/n8n-io/n8n/pull/7061 we had an edge case where
a manual unsaved workflow when run creates an orphan execution, i.e. a
saved execution not pointing to any workflow. This execution is only
ever visible to the instance owner (even if triggered by a member), and
is wrongly stored as unfinished and crashed. This PR enforces that the
DB disallows any such executions from making it into the DB.
This is needed also for the S3 client, which will include the
`workflowId` in the path-like filename.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-04 16:57:10 +02:00
Cornelius Suermann
e26553f198
feat(core): Add support for floating licenses ( #7090 )
...
This PR updates the license-sdk to v2.5.0 which introduces optional
support for floating licenses.
2023-09-04 15:56:20 +02:00
Iván Ovejero
fd78021b68
feat(core): Add list query middleware to credentials ( #7041 )
2023-09-04 15:00:25 +02:00
Iván Ovejero
413e0bccb4
feat(core): Migration for soft deletions for executions ( #7088 )
...
Based on https://github.com/n8n-io/n8n/pull/7065
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-04 14:34:03 +02:00
Iván Ovejero
442b910ffb
refactor(core): Consolidate CredentialsService.getMany() (no-changelog) ( #7028 )
...
Consolidate `CredentialsService.getMany()` in preparation for adding
list query middleware to `GET /credentials`.
2023-09-04 10:37:16 +02:00
Iván Ovejero
008cdcce56
ci: Fix lint to fix build ( #7076 )
2023-09-01 16:23:23 +02:00
कारतोफ्फेलस्क्रिप्ट™
97f87ae0fc
refactor(core): Add addColumns and dropColumns to the migrations DSL (no-changelog) ( #7073 )
2023-09-01 15:19:49 +02:00
Iván Ovejero
51093f649d
refactor: Move community package logic to service (no-changelog) ( #6973 )
2023-09-01 15:13:19 +02:00
कारतोफ्फेलस्क्रिप्ट™
2432dcc661
fix(core): Prevent bots from scanning the application (no-changelog) ( #7066 )
...
N8N-6878
2023-09-01 13:54:35 +02:00
कारतोफ्फेलस्क्रिप्ट™
a693b29134
ci: Fix prettier auto-formatting (no-changelog) ( #7063 )
2023-09-01 13:29:31 +02:00
Iván Ovejero
fa3d7070b0
refactor(core): Remove unneeded call from CreateWorkflowHistoryTable migration (no-changelog) ( #7072 )
...
https://github.com/n8n-io/n8n/pull/7069/files#r1312786460
2023-09-01 12:16:24 +02:00
