Commit graph

1825 commits

Author SHA1 Message Date
Iván Ovejero cf435c3311
fix(core): Support MySQL in MoveSshKeysToDatabase migration (#9120) 2024-04-11 11:00:20 +02:00
Danny Martini 3bb821f10e
fix(core): Don't revert irreversibble migrations (#9105) 2024-04-11 09:20:48 +02:00
Iván Ovejero a3b59843d5
refactor(core): Lint for cyclomatic complexity in BE packages (no-changelog) (#9111) 2024-04-10 14:02:02 +02:00
कारतोफ्फेलस्क्रिप्ट™ 9403657e46
refactor(core): Remove unnecessary indirection in SAML code (no-changelog) (#9103) 2024-04-10 10:55:49 +02:00
कारतोफ्फेलस्क्रिप्ट™ a7108d14f9
fix(core): Some more browser-id related fixes (no-changelog) (#9102) 2024-04-10 10:37:23 +02:00
Iván Ovejero c2f4d7d796
fix(core): Fix isLeader check in WaitTracker constructor (#9100) 2024-04-09 15:50:10 +02:00
कारतोफ्फेलस्क्रिप्ट™ 28261047c3
feat(core): Prevent session hijacking (#9057) 2024-04-09 11:20:35 +02:00
oleg 936682eeaa
feat(core): Improve Langsmith traces for AI executions (#9081)
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
2024-04-08 22:51:49 +02:00
Val f4f0a36fe1
feat: Add credential update and delete events to log streaming (#9026) 2024-04-08 08:13:39 +01:00
Iván Ovejero df56153f8b
refactor(core): Better UX for maxStalledCount queue mode error (#9075) 2024-04-05 20:03:49 +02:00
Iván Ovejero 7bf0f900f1
fix(core): Remove binary data when deleting executions by filter (#9056) 2024-04-05 18:00:35 +02:00
Iván Ovejero 0c90c7c8c1
fix(API): Accept settings.executionOrder in workflow creation (#9072) 2024-04-05 17:49:14 +02:00
Iván Ovejero ae3f164c5c
test(core): Fix multiple parallel private key checks (no-changelog) (#9012) 2024-04-05 17:45:21 +02:00
Iván Ovejero 4052b6c073
refactor(core)!: Remove legacy --file flag for execute CLI command (#9054) 2024-04-05 14:28:04 +02:00
Iván Ovejero ff81de3313
fix(core): Ensure TTL safeguard for test webhooks applies only to multi-main setup (#9062) 2024-04-05 14:15:49 +02:00
Iván Ovejero b8ab049932
refactor(core, editor): Remove legacy nodesAccess (no-changelog) (#9016) 2024-04-05 13:17:34 +02:00
Iván Ovejero 217b07d735
fix(core): Ensure only leader handles waiting executions (#9014) 2024-04-04 13:28:20 +02:00
Val 0ac985133b
fix: Workflows executed from other workflows not stopping (#9010) 2024-04-04 10:02:37 +01:00
कारतोफ्फेलस्क्रिप्ट™ a45fd0dbf4
ci(core): Upgrade express-openapi-validator (no-changelog) (#9022) 2024-04-03 16:57:33 +02:00
कारतोफ्फेलस्क्रिप्ट™ 76b73a27a0
refactor(core): Delete all auth exclusion config and checks (no-changelog) (#9044) 2024-04-03 16:56:36 +02:00
कारतोफ्फेलस्क्रिप्ट™ 072c3db97d
refactor(core): Rename push sessionId to pushRef (#8905) 2024-04-03 13:43:14 +02:00
कारतोफ्फेलस्क्रिप्ट™ a6446fe057 feat(core): Rate-limit login endpoint to mitigate brute force password guessing attacks (#9028) 2024-04-03 12:15:19 +02:00
Iván Ovejero 2aab78b058
feat(core): Introduce AWS secrets manager as external secrets store (#8982) 2024-03-28 10:15:58 +01:00
Michael Kret e84c27c0ce
feat(Webhook Node): Overhaul (#8889)
Co-authored-by: Giulio Andreini <andreini@netseven.it>
2024-03-28 10:46:39 +02:00
Michael Kret 335f363ca1
feat: Telemetry include basic llm optional promps, trigger on save workflow event (#8981) 2024-03-27 16:16:02 +01:00
Iván Ovejero ddc0f57116
refactor(core): Store SSH key pair for source control in DB settings (#8965) 2024-03-26 19:09:46 +01:00
Iván Ovejero 260bc07ca9
fix(core): Assign credential ownership correctly in source control import (#8955) 2024-03-26 17:18:41 +01:00
कारतोफ्फेलस्क्रिप्ट™ 372d5c7d01
ci: Upgrade eslint, prettier, typescript, and some other dev tooling (no-changelog) (#8895)
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2024-03-26 14:22:57 +01:00
Iván Ovejero ef45da95f1
test(core): Improve coverage for ActiveWorkflowRunner (no-changelog) (#8946) 2024-03-25 17:59:59 +01:00
Iván Ovejero 7d52419d6f
refactor(core): Remove dead utils (no-changelog) (#8947) 2024-03-25 17:59:42 +01:00
Iván Ovejero 69807a5efb
refactor(core): Unify failed and error execution status (no-changelog) (#8943) 2024-03-25 17:52:07 +01:00
Iván Ovejero 079a1147d4
fix(core): Ensure the generic OAuth2 API credential uses the OAuth2 credential test (#8941) 2024-03-21 13:49:34 +01:00
कारतोफ्फेलस्क्रिप्ट™ 669bd830e9
feat(editor): Block the frontend when trying to access n8n from another host over http (#8906) 2024-03-18 18:34:41 +01:00
कारतोफ्फेलस्क्रिप्ट™ c4c319d7cf
feat(core): Add support for SQLite connection pooling (#8722) 2024-03-14 13:45:16 +01:00
Alex Grozav 948c383999
feat: Add AI Error Debugging using OpenAI (#8805) 2024-03-13 16:48:00 +02:00
Danny Martini 11173a0114
fix: Ignore semver range when upgrading comunity packages (#8863) 2024-03-11 17:12:20 +00:00
Giulio Andreini 5301323906
feat(editor): Improve errors in output panel (#8644)
Co-authored-by: Michael Kret <michael.k@radency.com>
2024-03-07 18:08:01 +02:00
Omar Ajoue c6f6254c0e
fix: Always register webhooks on startup (#8830) 2024-03-07 09:25:12 +00:00
कारतोफ्फेलस्क्रिप्ट™ 0818824a72
feat(core)!: Set the secure flag on issued cookies (#8812) 2024-03-05 18:57:41 +01:00
कारतोफ्फेलस्क्रिप्ट™ 2b0e14e936
fix(core): Refactor push sessionid validation, and add unit tests (no-changelog) (#8815) 2024-03-05 18:57:29 +01:00
कारतोफ्फेलस्क्रिप्ट™ 62282c2bde
fix(core): Disable CollaborationService (no-changelog) (#8813) 2024-03-05 16:34:43 +01:00
कारतोफ्फेलस्क्रिप्ट™ cdec7c9334
feat(core): Update hashing strategy for JWTs (#8810) 2024-03-05 15:06:29 +01:00
कारतोफ्फेलस्क्रिप्ट™ d1b48ddcac
fix(core): Remove sensitive data from User entity during serialization (no-changelog) (#8773) 2024-02-29 14:20:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ db4a419c8d
refactor(core): Enforce authorization by default on all routes (no-changelog) (#8762) 2024-02-28 17:02:18 +01:00
कारतोफ्फेलस्क्रिप्ट™ 2811f77798
refactor(core): Rename RequireGlobalScope to GlobalScope (no-changelog) (#8760) 2024-02-28 14:40:02 +01:00
कारतोफ्फेलस्क्रिप्ट™ 56c8791aff
refactor(core): Remove all legacy auth middleware code (no-changelog) (#8755) 2024-02-28 13:12:28 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5f6da7b84e
fix(core): Do not report generic webhook execution errors (no-changelog) (#8749) 2024-02-27 12:36:49 +01:00
Omar Ajoue 737170893d
feat: Allow instance owners and admins to edit all credentials (#8716)
Co-authored-by: Danny Martini <despair.blue@gmail.com>
2024-02-27 08:26:36 +00:00
Alex Grozav 4b01335aa4
feat: Add env variables to support exposing /workflows/demo route and /nodes.json route (#8506)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-02-23 18:20:12 +02:00
कारतोफ्फेलस्क्रिप्ट™ 35fb7102cb
refactor(core): Update auth related typings (no-changelog) (#8686) 2024-02-23 15:08:18 +01:00
Jan Oberhauser a29b41ec55
fix(core): Fix pairedItem issue with partial manual executions (#8575)
Co-authored-by: Danny Martini <danny@n8n.io>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-02-23 11:43:08 +01:00
कारतोफ्फेलस्क्रिप्ट™ 059d281fd1
feat(core): Move execution permission checks earlier in the lifecycle (#8677) 2024-02-21 14:47:02 +01:00
Iván Ovejero 5304b320c8
refactor(core): Enrich inaccessible credential error (#8574)
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-02-21 13:04:30 +01:00
Tomi Turtiainen 05e13a68ea
fix: Fix typeorm .save usage (no-changelog) (#8678) 2024-02-20 17:34:54 +02:00
कारतोफ्फेलस्क्रिप्ट™ b6c8a0c413
refactor(core): Reduce code duplication in DB config (no-changelog) (#8679) 2024-02-20 14:28:53 +01:00
Omar Ajoue 540a80e9f9
test: Fix broken workflow tag tests (#8641)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-02-15 10:51:59 +00:00
कारतोफ्फेलस्क्रिप्ट™ a2a3ca160f
fix(core): Improve the startup error when EXECUTIONS_PROCESS is set (#8630)
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2024-02-14 15:35:42 +01:00
Danny Martini cd8ca8412d
fix(core): Give better error message if executions.process is still used in the configs (#8618) 2024-02-13 12:40:50 +00:00
Danny Martini ee81015231
docs: Update cursor description in the public api documentation (#8602) 2024-02-09 16:36:57 +01:00
Omar Ajoue a743a40376
feat(API): Add tag support to public API (#8588)
Co-authored-by: Jesús Burgers <jesus.burgers@chakray.co.uk>
Co-authored-by: Jesús Burgers <43568066+jburgers-chakray@users.noreply.github.com>
2024-02-09 15:10:03 +00:00
Cornelius Suermann cd151f1ba9
feat: Include totalUsers in usage metrics during license renewal (no-changelog) (#8598) 2024-02-09 14:15:05 +01:00
oleg 5ab34fe335
fix(API): Fire workflowExecuteBefore hook in runMainProcess (no-changelog) (#8601) 2024-02-09 13:23:00 +01:00
Omar Ajoue 7501ad8f3c
feat: Add support for AI log streaming (#8526)
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
2024-02-09 09:41:01 +00:00
कारतोफ्फेलस्क्रिप्ट™ 8e392cfc1d
feat(core): Migrate to n8n's typeorm fork (#8590) 2024-02-08 15:13:29 +01:00
कारतोफ्फेलस्क्रिप्ट™ 40eee3aa49
ci(core): Avoid slow bcrypt calls in tests (no-changelog) (#8570) 2024-02-07 17:56:02 +01:00
कारतोफ्फेलस्क्रिप्ट™ c04f92f7fd
refactor(core): Couple of refactors on WorkflowRunner and ActiveExecutions (no-changelog) (#8487) 2024-02-06 18:09:50 +01:00
Iván Ovejero dc068ce2e6
build: Fix outdated import to fix build (no-changelog) (#8558) 2024-02-06 10:45:30 +01:00
Iván Ovejero abddbb6227
fix(core): Use trx manager instead of repository for tags overwrite (#8557) 2024-02-06 10:40:32 +01:00
Iván Ovejero c4e39451db
refactor(core): Continue breaking dependency cycles (no-changelog) (#8545) 2024-02-06 10:08:46 +01:00
Iván Ovejero 28aee7c425
refactor(core): Lock webhook process out of multi-main setup (no-changelog) (#8549) 2024-02-05 18:06:54 +01:00
Iván Ovejero dc5ec8f946
refactor(core): Streamline flows in multi-main mode (no-changelog) (#8446) 2024-02-05 09:26:55 +01:00
Iván Ovejero 92f939f827
fix(core): Fix workflow tagging failure due to unique constraint check (#8505) 2024-02-02 12:36:55 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4fed68ee34
fix(core): Fix DropRoleMapping migration (#8521) 2024-02-02 12:22:09 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5832d3ca46
fix(core): Fix PermissionChecker.check, and add additional unit tests (#8528) 2024-02-02 12:21:53 +01:00
Iván Ovejero 528c07134a
fix(core): Fix test runs of triggers that rely on static data (#8524) 2024-02-01 18:05:23 +01:00
Iván Ovejero cc41fc7c80
fix(core): Fix new graceful shutdown env being always overridden by deprecated env (#8503) 2024-02-01 11:10:40 +01:00
कारतोफ्फेलस्क्रिप्ट™ 033fd344b5
fix(core): Handle possibly invalid updatedAt timestamps in source-control (#8485) 2024-01-31 14:25:03 +01:00
कारतोफ्फेलस्क्रिप्ट™ 839dd96c7d
refactor(core): Move all code related to onServerStarted into InternalHooks (no-changelog) (#8500) 2024-01-31 13:29:17 +01:00
Danny Martini 39e8754784
fix(core): Don't report executions that have been paused as failed to rudderstack and log streams (#8501) 2024-01-31 11:21:15 +01:00
Iván Ovejero dac511b710
refactor(core): Modernize credentials controllers and services (no-changelog) (#8488)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-01-31 09:48:48 +01:00
कारतोफ्फेलस्क्रिप्ट™ 121a55b691
feat(core): Remove own execution-process mode (#8490) 2024-01-30 12:51:40 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5cb55270b7
refactor(core): Replace promisify-d node calls with native promises (no-changelog) (#8464) 2024-01-30 09:49:23 +01:00
Iván Ovejero 238b54c77b
fix(core): Fix stopping and retrying failed executions (#8480) 2024-01-29 16:34:58 +01:00
Iván Ovejero 9e93980957
fix(core): Prevent calling internal hook email event if emailing is disabled (#8462)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-01-29 16:15:30 +01:00
Iván Ovejero db48bdd6d1
fix(core): Handle zero execution statistics on metrics collection during license renewal (#8463) 2024-01-26 17:50:19 +01:00
कारतोफ्फेलस्क्रिप्ट™ 2fba0e8d58
feat(core): Upgrade Rudderstack SDK to address CVE-2023-45857 (#8368) 2024-01-26 16:58:44 +01:00
Danny Martini 9f11eba0a4
fix(core): Point users to the official documentation when they use n8n --help (#8440) 2024-01-26 15:16:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ c70fa66e76
refactor(core): Use DI for WorkflowRunner (no-changelog) (#8372) 2024-01-26 13:49:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 7c49004018
refactor(core): Use DI for eventBus code - Part 1 (no-changelog) (#8434) 2024-01-26 12:21:15 +01:00
कारतोफ्फेलस्क्रिप्ट™ cc2f0ada76
ci: Enforce noUnusedLocals on all backend and nodes packages (no-changelog) (#8428) 2024-01-25 14:33:35 +01:00
कारतोफ्फेलस्क्रिप्ट™ f438082442
fix(core): Run the down migration correctly when transaction: false is set (no-changelog) (#8432) 2024-01-24 19:07:47 +01:00
Michael Kret ed7d6b7b3a
feat(core): Display conditions in displayOptions (no-changelog) (#7888) 2024-01-24 18:04:46 +02:00
कारतोफ्फेलस्क्रिप्ट™ d6deceacde
refactor(core): Remove roleId indirection (no-changelog) (#8413) 2024-01-24 13:38:57 +01:00
कारतोफ्फेलस्क्रिप्ट™ 1affebd85e
refactor(core): Remove unused code from Server.ts (no-changelog) (#8426) 2024-01-24 12:55:43 +01:00
Iván Ovejero c0bc94c78f
refactor(core): Finish removing UserManagementHelper (no-changelog) (#8418) 2024-01-23 13:58:31 +01:00
Iván Ovejero a0a1830696
feat(core): Email recipients on resource shared (#8408) 2024-01-23 12:03:59 +01:00
Danny Martini ae06fdeb62
fix(core): Fix update workflow cli command being unable to activate all workflows (#8412)
Co-authored-by: Daniel Schröder <daniel.schroeder@skriptfabrik.com>
2024-01-23 10:59:06 +01:00
Iván Ovejero 49b52c4f1d
refactor(core): Bring active executions into executions controller (no-changelog) (#8371) 2024-01-23 09:48:50 +01:00
कारतोफ्फेलस्क्रिप्ट™ 913c8c6b0c
feat(core): Upgrade oclif (no-changelog) (#8381) 2024-01-22 18:25:36 +01:00
Cornelius Suermann d597c2ab29
feat: Extend collection of usage metrics during license renewal (no-changelog) (#8369)
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2024-01-22 12:29:28 +01:00
Iván Ovejero f35d4fcbd8
refactor(core): Simplify OrchestrationService (no-changelog) (#8364) 2024-01-22 11:16:29 +01:00
Danny Martini 07e6705256
feat(core): Custom session timeout and refresh configuration (#8342) 2024-01-22 09:54:13 +01:00
Iván Ovejero 749ac2b407
fix(core): Adjust starter node priority for manual executions with pinned activators (#8386) 2024-01-19 10:18:04 +01:00
Tomi Turtiainen f31cc0743f
fix: Force posthog recording to be disabled outside cloud (#8374) 2024-01-18 14:44:27 +02:00
Jon fc94377036
fix: Fix issue preventing secrets with a - in the path from being imported (#8378) 2024-01-18 09:24:58 +00:00
Tomi Turtiainen 99457019f7
feat: Nudge users to become template creators if eligible (#8357) 2024-01-17 19:07:34 +02:00
Tomi Turtiainen 9a1cc56806
fix: Set '@typescript-eslint/return-await' rule to 'always' for node code (no-changelog) (#8363)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-01-17 17:08:50 +02:00
Iván Ovejero 2eb829a6b4
refactor(core): Use DI in execution services (no-changelog) (#8358) 2024-01-17 15:42:19 +01:00
Iván Ovejero 771d2fa341
test(core): Fix cleanup in test teardown script (no-changelog) (#8361) 2024-01-17 14:01:56 +01:00
Omar Ajoue d5455d7acc
fix(core): Report when waitTill is invalid and handle it (#8356) 2024-01-17 13:08:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 7cdbb424e3
refactor(core): Move methods from WorkflowHelpers into various workflow services (no-changelog) (#8348) 2024-01-17 10:16:13 +01:00
Iván Ovejero d4c93b1607
fix(core): Prevent issues with missing or mismatching encryption key (#8332) 2024-01-16 18:25:53 +01:00
Iván Ovejero 7bb2d1799e
refactor(core): Consolidate executions controllers (no-changelog) (#8349) 2024-01-16 16:52:21 +01:00
कारतोफ्फेलस्क्रिप्ट™ 64ceb16af6
refactor(core): Use DI in PermissionChecker (no-changelog) (#8344) 2024-01-16 14:15:29 +01:00
Iván Ovejero 420b4271a9
refactor(core): Move typeorm operators from Public API (no-changelog) (#8319) 2024-01-16 13:35:43 +01:00
Omar Ajoue e1acb5911a
refactor: Make execution IDs mandatory in BE (#8299)
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2024-01-16 09:53:17 +00:00
Iván Ovejero 0f4f472a72
refactor(core): Eliminate dead Redis code (no-changelog) (#8292) 2024-01-16 09:31:45 +01:00
Iván Ovejero 5fbd7971e0
fix(core): Account for immediate confirmation request during test webhook creation (#8329) 2024-01-16 09:17:41 +01:00
Ricardo Espinoza 3c2a4000ae
refactor(core): Use DI for LDAP code (no-changelog) (#8248)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-01-15 09:01:48 -05:00
Iván Ovejero 1a0e285553
feat(core): Implement inter-main communication for test webhooks in multi-main setup (#8267) 2024-01-12 11:48:58 +01:00
oleg 884396ea0d
fix(API): Fix manual chat trigger execution (#8300)
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
2024-01-11 14:01:07 +01:00
Omar Ajoue 8c7f39907f
fix: Properly output saml validation errors (#8284) 2024-01-10 16:56:04 +01:00
Omar Ajoue 8a7c629ea1
fix: Store workflow settings when saving an execution (#8288) 2024-01-10 14:20:37 +00:00
Iván Ovejero 5032bf0e34
fix(core): Fix test webhook deregistration (#8247) 2024-01-09 16:02:32 +01:00
Iván Ovejero 0dabe5c74e
fix: Fix user reinvites on FE and BE (#8261) 2024-01-09 13:52:34 +01:00
Tomi Turtiainen 3b996a7da0
feat(core): Validate shutdown handlers on startup (#8260) 2024-01-08 17:46:45 +02:00
Tomi Turtiainen 8affdf680d
fix: Hide cred setup button from canvas (no-changelog) (#8255) 2024-01-08 13:59:04 +02:00
Iván Ovejero 90c065e999
refactor(core): Convert workflows controller to DI (no-changelog) (#8253) 2024-01-08 12:54:23 +01:00
कारतोफ्फेलस्क्रिप्ट™ ac1c642fdd
fix(core): Do not add Authentication header when authentication type is body (#8201) 2024-01-08 12:38:24 +01:00
Jan Oberhauser ccb2b076f8
fix: Resolve expressions in credentials following paired item (#8250)
## Summary
Fixes the issue that pairedItem information was not available in
expressions that got used in credentials


## Related tickets and issues

[PAY-1207](https://linear.app/n8n/issue/PAY-1207/paireditem-expressions-not-working-correctly-in-credentials)


## Review / Merge checklist
- [x] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again.
   > A feature is not complete without tests.

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
2024-01-08 09:48:20 +00:00
Iván Ovejero f2939568cf
perf(core): Optimize workflow activation errors (#8242)
At https://github.com/n8n-io/n8n/pull/8213 we introduced Redis hashes
for workflow ownership and manual webhooks...

- to remove clutter from multiple related keys at the top level,
- to improve performance by preventing serializing-deserializing, and
- to guarantee atomicity during concurrent updates in multi-main setup.

Workflow activation errors can also benefit from this. Added test
coverage as well.

To test manually, create a workflow with a trigger with an invalid
credential, edit the workflow's `active` column to `true`, and restart.
The activation error should show as a red triangle on canvas and in the
workflow list.
2024-01-05 13:06:42 +01:00
Iván Ovejero 23a4ac96c0
refactor(core): Continue moving typeorm operators to repositories (no-changelog) (#8212)
Follow-up to: https://github.com/n8n-io/n8n/pull/8186
2024-01-05 13:06:24 +01:00
Iván Ovejero f53c482939
perf(core): Improve caching service (#8213)
Story: https://linear.app/n8n/issue/PAY-1188

- Implement Redis hashes on the caching service, based on Micha's work
in #7747, adapted from `node-cache-manager-ioredis-yet`. Optimize
workflow ownership lookups and manual webhook lookups with Redis hashes.
- Simplify the caching service by removing all currently unused methods
and options: `enable`, `disable`, `getCache`, `keys`, `keyValues`,
`refreshFunctionEach`, `refreshFunctionMany`, `refreshTtl`, etc.
- Remove the flag `N8N_CACHE_ENABLED`. Currently some features on
`master` are broken with caching disabled, and test webhooks now rely
entirely on caching, for multi-main setup support. We originally
introduced this flag to protect against excessive memory usage, but
total cache usage is low enough that we decided to drop this setting.
Apparently this flag was also never documented.
- Overall caching service refactor: use generics, reduce branching, add
discriminants for cache kinds for better type safety, type caching
events, improve readability, remove outdated docs, etc. Also refactor
and expand caching service tests.

Follow-up to: https://github.com/n8n-io/n8n/pull/8176

---------

Co-authored-by: Michael Auerswald <michael.auerswald@gmail.com>
2024-01-05 11:52:44 +01:00
Iván Ovejero 22a5f5258d
feat(core): Cache test webhook registrations (#8176)
In a multi-main setup, we have the following issue. The user's client
connects to main A and runs a test webhook, so main A starts listening
for a webhook call. A third-party service sends a request to the test
webhook URL. The request is forwarded by the load balancer to main B,
who is not listening for this webhook call. Therefore, the webhook call
is unhandled.

To start addressing this, cache test webhook registrations, using Redis
for queue mode and in-memory for regular mode. When the third-party
service sends a request to the test webhook URL, the request is
forwarded by the load balancer to main B, who fetches test webhooks from
the cache and, if it finds a match, executes the test webhook. This
should be transparent - test webhook behavior should remain the same as
so far.

Notes:
- Test webhook timeouts are not cached. A timeout is only relevant to
the process it was created in, so another process retrieving from Redis
a "foreign" timeout will be unable to act on it. A timeout also has
circular references, so `cache-manager-ioredis-yet` is unable to
serialize it.
- In a single-main scenario, the timeout remains in the single process
and is cleared on test webhook expiration, successful execution, and
manual cancellation - all as usual.
- In a multi-main scenario, we will need to have the process who
received the webhook call send a message to the process who created the
webhook directing this originating process to clear the timeout. This
will likely be implemented via execution lifecycle hooks and Redis
channel messages checking session ID. This implementation is out of
scope for this PR and will come next.
- Additional data in test webhooks is not cached. From what I can tell,
additional data is not needed for test webhooks to be executed.
Additional data also has circular references, so
`cache-manager-ioredis-yet` is unable to serialize it.

Follow-up to: #8155
2024-01-03 16:58:33 +01:00
कारतोफ्फेलस्क्रिप्ट™ cfe9525dd4
fix(core): Better input validation for the changeRole endpoint (#8189)
also refactored the code to
1. stop passing around `scope === 'global'`, since this code can be used
only for changing globalRole.
2. leak less details when input validation fails.

## Review / Merge checklist
- [x] PR title and summary are descriptive
- [x] Tests included
2024-01-03 09:33:35 +01:00
Iván Ovejero 40c1eeeddd
refactor(core): Continue moving typeorm operators to repositories (no-changelog) (#8186)
Follow-up to: #8163
2024-01-02 17:53:24 +01:00
Tomi Turtiainen e126ed74f3
feat(core): Add user.profile.beforeUpdate hook (#8144)
Add `user.profile.beforeUpdate` hook so we can prevent user email change
if it overlaps with other users email.
2024-01-02 10:15:12 +02:00
Iván Ovejero ece48d6a13
refactor(core): Unify workflow controllers (no-changelog) (#8175)
Combine EE workflows controller into main workflows controller,
protecting paid functionality behind feature flag checks.
2023-12-29 14:23:58 +01:00
Iván Ovejero 4bad43dd66
refactor(core): Move typeorm operators from WaitTracker to ExecutionRepository (no-changelog) (#8163)
Follow-up to: #8145

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-28 19:22:09 +01:00
Iván Ovejero e418d42450
refactor(core): Move typeorm operators from various sources into repositories (no-changelog) (#8174)
Follow-up to: #8165
2023-12-28 13:14:10 +01:00
कारतोफ्फेलस्क्रिप्ट™ c84d3c3bbf
refactor(core): Extract ActiveWebhooks out of ActiveWorkflowRunner (no-changelog) (#8171)
## Summary
This PR continues refactoring webhooks code for better modularity.
Continued from #8069 to bring back `ActiveWebhooks`, but this time
actually handling active webhook calls in this class.

## Review / Merge checklist
- [x] PR title and summary are descriptive
2023-12-28 10:04:32 +01:00
Iván Ovejero 639afcd7a5
refactor(core): Simplify state in test webhooks (no-changelog) (#8155)
This PR simplifies state in test webhooks so that it can be cached
easily. Caching this state will allow us to start using Redis for manual
webhooks, to support manual webhooks to work in multi-main setup.

- [x] Convert `workflowWebhooks` to a getter - no need to optimize for
deactivation
- [x] Remove array from value in `TestWebhooks.webhookUrls`
- [x] Consolidate `webhookUrls` and `registeredWebhooks`
2023-12-28 09:28:12 +01:00
Iván Ovejero 0e582594ea
refactor(core): Move more typeorm operators to UserRepository (no-changelog) (#8165)
Follow-up to: #8163
2023-12-28 09:27:47 +01:00
Iván Ovejero 5aee7a1d48
refactor(core): Move typeorm operators from SourceControlExportService to repositories (no-changelog) (#8168)
Follow-up to: #8165
2023-12-28 09:27:38 +01:00
Jan Oberhauser ea7e76fa3b
fix(core): Use pinned data only for manual mode (#8164)
## Summary
Fixes the issue that pinned data gets also used for production executions.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-27 18:22:33 +01:00
Iván Ovejero 2c6ffb0153
fix(core): Minor improvements to multi-main setup (no-changelog) (#8012)
- Move webhook, poller and trigger activation logs closer to activation
event
- Enrich response of `/debug/multi-main-setup`
- Ensure workflow updates broadcast activation state changes only if
state changed
- Fix bug on workflow activation after leadership change
- Ensure debug controller is not available in production

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-12-27 16:55:01 +01:00
कारतोफ्फेलस्क्रिप्ट™ f69ddcd796
refactor(core): Use Dependency Injection for all Controller classes (no-changelog) (#8146)
## Review / Merge checklist
- [x] PR title and summary are descriptive
2023-12-27 11:50:43 +01:00
Jan Oberhauser fa8bd8b9eb
fix(core): Fix issue that pinnedData is not used with Test-Webhooks (#8123)
## Summary
When a workflow gets started via a Test-Webhook the pinned data does get
ignored and the nodes executed anyway.



## Related tickets and issues
> Include links to **Linear ticket** or Github issue or Community forum
post. Important in order to close *automatically* and provide context to
reviewers.



## Review / Merge checklist
- [x] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [X] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again.
   > A feature is not complete without tests.

---------

Co-authored-by: Mutasem Aldmour <mutasem@n8n.io>
2023-12-27 10:51:53 +01:00
Iván Ovejero 7b26a7a621
refactor(core): Move typeorm operators from PruningService to ExecutionRepository (no-changelog) (#8145)
Follow-up to https://github.com/n8n-io/n8n/pull/8143
2023-12-22 17:49:15 +01:00
Iván Ovejero a59d78de18
refactor(core): Move more typeorm operators to repositories (no-changelog) (#8143)
Follow-up to #8139
2023-12-22 16:20:30 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4007163651
refactor(core): Delete unused code, and fix typings in tests (no-changelog) (#8142) 2023-12-22 15:41:29 +01:00
कारतोफ्फेलस्क्रिप्ट™ baee47a276
refactor(core): Move all base URLs to UrlService (no-changelog) (#8141)
This change kept coming up in #6713, #7773, and #8135. 
So this PR moves the existing code without actually changing anything,
to help get rid of some of the circular dependencies.


## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-22 15:19:50 +01:00
Iván Ovejero c6dd935895
refactor(core): Move some typeorm operators to repositories (no-changelog) (#8139)
Moving some persistence logic to repositories to reduce circular
dependencies.
2023-12-22 13:35:23 +01:00
Tomi Turtiainen 3a881be6c2
feat(core): Unify application components shutdown (#8097)
## Summary

Add `ShutdownService` and `OnShutdown` decorator for more unified way to
shutdown different components. Use this new way in the following
components:

- HTTP(S) server
- Pruning service
- Push connection
- License

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-22 12:39:58 +02:00
कारतोफ्फेलस्क्रिप्ट™ c158ca2471
refactor(core): Upgrade more dependencies to remove axios 0.x (no-changelog) (#8105)
Had to [fork localtunnel](https://github.com/n8n-io/localtunnel) to get
the axios upgrade, since localtunnel doesn't seem to be actively maintained.
2023-12-22 11:39:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 021add0f39
refactor(core): Move active workflows endpoints to a decorated controller class (no-changelog) (#8101)
This is a continuation of migrating all rest endpoints to decorated controller classes
2023-12-22 11:28:42 +01:00
कारतोफ्फेलस्क्रिप्ट™ 21788d9153
fix(core): Remove circular dependency in WorkflowService and ActiveWorkflowRunner (#8128)
## Summary
A circular dependency between `WorkflowService` and
`ActiveWorkflowRunner` is sometimes causing `this.activeWorkflowRunner`
to be `undefined` in `WorkflowService`.
Breaking this circular dependency should hopefully fix this issue.

## Related tickets and issues
#8122


## Review / Merge checklist
- [x] PR title and summary are descriptive
- [ ] Tests included
2023-12-21 17:37:08 +01:00
कारतोफ्फेलस्क्रिप्ट™ ffaa30ddc4
fix(core): Handle empty executions table in pruning in migrations (#8121)
In case someone manually prunes their executions table before upgrading
to 1.x, `MigrateIntegerKeysToString` should gracefully handle that,
instead of crashing the application.

## Review / Merge checklist
- [x] PR title and summary are descriptive
2023-12-21 14:52:42 +01:00
Iván Ovejero 9ac8825a67
refactor(core): Move error execution creation to execution service (no-changelog) (#8006)
Continue breaking down legacy helpers.

Note: `getUserById` is unused.
2023-12-21 14:15:46 +01:00
Omar Ajoue 5ffff1bb22
fix: Stop binary data restoration from preventing execution from finishing (#8082)
In the case of a filesystem failure to rename the binary files as part
of the execution's cleanup process, the execution would fail to be saved
and would never finish. This catch prevents it.

## Summary
Whenever an execution is wrapping u to save the data, if it uses binary
data n8n will try to find possibly misallocated files and place them in
the right folder. If this process fails, the execution fails to finish.

Given the execution has already finished at this point, and we cannot
handle the binary data errors more gracefully, all we can do at this
point is log the message as it's a filesystem issue. The rest of the
execution saving process should remain as normal.



## Related tickets and issues
https://linear.app/n8n/issue/HELP-430



## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again.
   > A feature is not complete without tests.

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-12-21 09:40:39 +00:00
Iván Ovejero edfa78414d
feat(editor): Finalize workers view (#8052)
https://linear.app/n8n/issue/PAY-1065
2023-12-20 17:49:14 +01:00
Iván Ovejero 97aa38e783
ci: Fix Postgres and MySQL tests (no-changelog) (#8106)
This role query works for sqlite but [fails for Postgres and
MySQL](https://github.com/n8n-io/n8n/actions/runs/7269009778/job/19805986017),
so generalize by adding alias and accounting for count possibly being
`string` in the resulting rows.

Run in progress: https://github.com/n8n-io/n8n/actions/runs/7275986797
2023-12-20 15:14:31 +01:00
कारतोफ्फेलस्क्रिप्ट™ 81994ce13d
fix(core): Downgrade Rudderstack SDK (no-changelog) (#8107)
This reverts commit a895ee87fc (#8090)

Our telemetry backend is throwing 500s with the updated rudderstack sdk.
Until that is resolved, we need to downgrade.

## Review / Merge checklist
- [x] PR title and summary are descriptive
2023-12-20 11:59:44 +01:00
Iván Ovejero 9dc491c3a5
refactor(core): Improve test-webhooks (no-changelog) (#8069)
Remove duplication, improve readability, and expand tests for
`TestWebhooks.ts` - in anticipation for storing test webhooks in Redis.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-19 17:32:02 +01:00
Iván Ovejero 38d1336fa7
refactor: Add telemetry for RBAC (no-changelog) (#8056)
https://linear.app/n8n/issue/PAY-1142
2023-12-19 17:02:52 +01:00
कारतोफ्फेलस्क्रिप्ट™ a895ee87fc
feat(core): Upgrade Rudderstack SDK (no-changelog) (#8090)
This helps remove some of the older versions of transient dependencies,
like axios 0.x and ioredis 4.x.

## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-19 16:53:12 +01:00
कारतोफ्फेलस्क्रिप्ट™ 64eb9bbc36
fix: Upgrade axios to address CVE-2023-45857 (#7713)
[GH Advisory](https://github.com/advisories/GHSA-wf5p-g6vw-rhxx)
2023-12-19 16:17:01 +01:00
Iván Ovejero 8e6b951a76
fix(core): Do not display error when stopping jobless execution in queue mode (#8007)
No need to surface error to user when stopping a job no longer in queue.

https://linear.app/n8n/issue/PAY-1104
2023-12-19 16:11:21 +01:00
कारतोफ्फेलस्क्रिप्ट™ 464b565283
ci: Remove unnecessary async/await, enable await-thenable linting rule (no-changelog) (#8076)
## Summary
We accidentally made some functions `async` in
https://github.com/n8n-io/n8n/pull/7846
This PR reverts that change. 

## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-19 13:52:42 +01:00
कारतोफ्फेलस्क्रिप्ट™ 785bf9974e
fix(core): Use relative imports for dynamic imports in SecurityAuditService (#8086)
`tsc-alias` doesn't seem to replace imports when using template strings

## Related tickets and issues
#8085

## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-19 12:41:15 +01:00
कारतोफ्फेलस्क्रिप्ट™ a63d94f28c
refactor(core): Move license endpoints to a decorated controller class (no-changelog) (#8074) 2023-12-19 12:13:19 +01:00
Iván Ovejero 73d400a1bf
refactor(core): Inject dependencies into workflow services (no-changelog) (#8066)
Inject dependencies into workflow services (no-changelog)

Up next:

- ~~Make workflow services injectable~~ #8033
- ~~Inject dependencies into workflow services~~ (current)
- Consolidate workflow controllers into one
- Make workflow controller injectable
- Inject dependencies into workflow controller
2023-12-18 16:10:30 +01:00
कारतोफ्फेलस्क्रिप्ट™ a651089a10
refactor(core): Update backend sentry setup to reduce noise (no-changelog) (#8026)
## Summary
This PR updates our backend sentry setup to remove integrations that
don't provide us any value. This also reduces the amount of PII that
gets sent to Sentry.

[Sample event](https://n8nio.sentry.io/issues/4725315362/)

## Related tickets
[ENG-95](https://linear.app/n8n/issue/ENG-95)

## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-18 14:26:07 +01:00
Tomi Turtiainen 614f488386
feat(core): Add N8N_GRACEFUL_SHUTDOWN_TIMEOUT env var (#8068)
Add generic N8N_GRACEFUL_SHUTDOWN_TIMEOUT which controls how long n8n
process will wait for graceful exit before exitting forcefully. This
variables replaces the QUEUE_WORKER_TIMEOUT variable that was used for
worker process.

DEPRECATED: QUEUE_WORKER_TIMEOUT deprected

QUEUE_WORKER_TIMEOUT environment variable has been replaced with
N8N_GRACEFUL_SHUTDOWN_TIMEOUT.
2023-12-18 14:04:19 +02:00
Tomi Turtiainen 4cae976a3b
fix(core): Consider timeout in shutdown an error (#8050)
If the process doesn't shutdown within a time limit, exit with error
code.

1. conceptually something timing out is an error.
2. on successful exit we close down the DB connection gracefully. On an
exit timeout we rather not do that, since it will wait for any active
connections to close and would possible block the exit.
2023-12-18 10:53:34 +02:00
Tomi Turtiainen 6ae2f5efea
fix(core): Fix shutdown if terminating before hooks are initialized (#8047)
If the app receives termination signal before hooks have been
initialised, the would be objet is undefined error. This PR fixes that.
2023-12-18 09:23:10 +02:00
Jon 439a22d68f
feat: Add config option to prefer GET request over LIST when using Hashicorp Vault (#8049)
## Summary
Hashicorp Vault prefers a `LIST` HTTP method to be used when fetching
secrets but not all environments will allow custom http methods through
WAFs. This PR adds `N8N_EXTERNAL_SECRETS_PREFER_GET` which when set to
`true` will use GET instead of LIST to fetch secrets.


## Review / Merge checklist
- [x] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
2023-12-15 16:20:39 +00:00
Tomi Turtiainen 67bd8ad698
fix(core): Handle multiple termination signals correctly (#8046)
Prevent possible multiple termination signals initiating the shutdown
process multiple times.
2023-12-15 17:35:22 +02:00
Tomi Turtiainen e69707efd4
fix(core): Close db connection gracefully when exiting (#8045)
Close db connection gracefully when exiting
2023-12-15 17:16:35 +02:00
Marcus 48d4f4a71b
fix(core): Public API support node's deprecated continueOnFail to upload old workflows (no-changelog) (#8044) 2023-12-15 16:11:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ 989888d9bc
fix(core): Remove circular references before serializing executions in public API (#8043)
## Summary
Handle circular references in the public API for executions created
prior to the fix from #8030

## Related tickets
[PAY-1119](https://linear.app/n8n/issue/PAY-1119)

## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-15 16:01:41 +01:00
Iván Ovejero 1e7a309e63
refactor(core): Make workflow services injectable (no-changelog) (#8033)
Refactor static workflow service classes into DI-compatible classes

Context: https://n8nio.slack.com/archives/C069HS026UF/p1702466571648889

Up next:
- Inject dependencies into workflow services
- Consolidate workflow controllers into one
- Make workflow controller injectable
- Inject dependencies into workflow controller

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-15 12:59:56 +01:00
Iván Ovejero 2da15d0264
refactor(core): Remove ID-less workflow reporting (no-changelog) (#8038)
We added ID-less workflow reporting at #8031, which has already produced
multiple reports coming from internal, enough info to tackle [this
story](https://linear.app/n8n/issue/PAY-1147). To prevent an
overwhelming number of reports from cloud, this PR removes the reporting
for now.
2023-12-15 12:50:09 +01:00
Iván Ovejero f18bc5f4b7
refactor(core): Warn on sqlite DB detected during init on queue mode (#8034)
When setting up queue mode, it is easy to overlook that not exporting
Postgres env vars will default the worker to use sqlite, which will fail
during execution with a non-obvious error. Hence add warnings when
starting a worker with an incompatible DB type.
2023-12-15 10:56:35 +01:00
Iván Ovejero c5e6ba8cdd
fix(core): Restore workflow ID during execution creation (#8031)
## Summary
Restore workflow ID during execution creation removed by [this
PR](https://github.com/n8n-io/n8n/pull/8002/files#diff-c8cbb62ca9ab2ae45e5f565cd8c63fff6475809a6241ea0b90acc575615224af).
The missing workflow ID, and more generally the fact that `workflow.id`
is optional when it should not be, causes `PermissionChecker.check` to
misreport a credential as inaccessible when it should be accessible.

More generally, start reporting ID-less workflows so we can root them
out and prevent this at type level.

## Related tickets and issues

https://n8nio.slack.com/archives/C035KBDA917/p1702539465555529
2023-12-14 18:13:12 +01:00
Iván Ovejero 53c0b49d15
fix(core): Initialize queue once in queue mode (#8025)
We're initializing the queue twice because of a [bad
merge](2c63474538).
No associated known bugs but no need to init the queue twice. We should
follow up by investigating if any pending bugs can be associated to
this.
2023-12-14 16:16:12 +01:00
Michael Kret 953a58f18b
feat(n8n Form Trigger Node): Improvements (#7571)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: Giulio Andreini <andreini@netseven.it>
2023-12-13 17:00:51 +02:00
Omar Ajoue e5581ce802
fix(core): Prevent workflow history saving error from happening (#7812)
When performing actions such as renaming a workflow or updating its
settings, n8n errors with "Failed to save workflow version" in the
console although the saving process was successful. We are now correctly
checking whether `nodes` and `connections` exist and only then save a
snapshot.

Github issue / Community forum post (link here to close automatically):
2023-12-13 11:41:06 +00:00
Iván Ovejero a70a5076ee
refactor(core): Add telemetry for RBAC roles (#7969)
Add telemetry for RBAC roles, see
[requirements](https://linear.app/n8n/issue/PAY-1067/add-telemetry-events-for-adding-and-assigning-admin-users#comment-184619fe).
2023-12-13 12:22:11 +01:00
Iván Ovejero 7b5d0a9546
refactor(core): Introduce import service (no-changelog) (#8001)
Consolidate import logic into import service.

Also fixes:
- https://linear.app/n8n/issue/PAY-1086
- https://github.com/n8n-io/n8n/issues/7881
-
https://community.n8n.io/t/cli-workflow-imports-failing-after-upgrade-to-v1-18-0/33780
- https://linear.app/n8n/issue/PAY-221
- https://github.com/n8n-io/n8n/issues/5477
- https://community.n8n.io/t/export-workflows-with-tags-got-created/6161
2023-12-13 10:00:21 +01:00
Jon 0ac959463f
fix: Fix issue preventing secrets from loading if the path contains - or / (#7988)
## Summary
Fixes an issue preventing n8n from pulling secrets from Hashicorp Vault
KV stores if the secret path contained a `-` or a `/`, An example
provided was `integrations/n8n-workflows` which I have tested in my
local instance of Vault.

This still needs testing with Infisical to make sure nothing breaks
there.
2023-12-12 15:21:18 +00:00
Jon b6c1c04b54
feat: Add config option for external secret update interval (#7995)
## Summary
Adds `N8N_EXTERNAL_SECRETS_UPDATE_INTERVAL` to allow enterprise users to
tweak the update internal for importing new secrets.

If using a config file the value is:

```
"externalSecrets": {
  "updateInterval": 300
}
```

#### How to test the change:
1. Run as normal and check that the secret is updated every 5 minutes
2. Set `N8N_EXTERNAL_SECRETS_UPDATE_INTERVAL` to 10
3. Check the secret is reloaded after 10 seconds


## Review / Merge checklist
- [x] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [x] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
2023-12-12 14:22:14 +00:00
Iván Ovejero d0e44d450f
feat(core): Add multi-main setup debug endpoint (no-changelog) (#7991)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.
Adi's idea here to help diagnose:
https://n8nio.slack.com/archives/C069KJBJ8HE/p1702300349277609?thread_ts=1702299930.728029&cid=C069KJBJ8HE
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-12 15:18:32 +01:00
कारतोफ्फेलस्क्रिप्ट™ 1d870412ca
refactor(core): Don't use DB transactions on ExecutionRepository.createNewExecution (#8002)
Saving execution data is one of the slowest DB operations in the
application, and is likely behind some of the sqlite transaction
concurrency issues we've been seeing.
This not only remove the 2 separate transactions for saving
`ExecutionEntity` and `ExecutionData`, but also remove fields from
`ExecutionData.workflowData` that don't need to be saved (like `tags`,
`shared`, `statistics`, `triggerCount`, etc).
2023-12-12 14:36:56 +01:00
Iván Ovejero c378f60a25
refactor(core): Introduce password utility (no-changelog) (#7979)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.
Continue breaking down `UserManagementHelper.ts`
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-11 18:23:42 +01:00
Iván Ovejero f49aebb740
refactor(core): Move instance owner retrieval to ownership service (no-changelog) (#7980)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.
Continue breaking down `UserManagementHelper.ts`
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-11 13:35:09 +01:00
Iván Ovejero 69c2254742
refactor(core): Introduce naming service (no-changelog) (#7985)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.
Continue breaking down `GenericHelpers.ts`
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-11 12:35:14 +01:00
Iván Ovejero 193fe5ac1f
refactor(core): Log any hard-deletion errors during pruning (no-changelog) (#7965)
## Summary
Pruning is a minor background task so hard-deletion errors during
pruning should be simply logged for later investigation rather than
rethrown.

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-11 11:08:14 +01:00
Iván Ovejero 76ab411bc0
refactor(core): Instrument multi-main to report failures to renew lease (#7970)
## Summary
Instrument multi-main to report failures to renew the lease

https://linear.app/n8n/issue/PAY-1105
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-11 09:45:08 +01:00
Iván Ovejero 1a87f70e84
fix(core): Perform multi-main leader check against key ID (#7964)
## Summary
Current leader check is based on key presence and multi-main instance
type, which can give rise to [this edge
case](https://n8nio.slack.com/archives/C04B1GZ4T0U/p1702025497086379?thread_ts=1701962808.817579&cid=C04B1GZ4T0U)
where leader fails to realize they lost leadership to a former follower.
PR performs the check against the specific key ID instead to prevent
this.

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-08 13:55:55 +01:00
Csaba Tuncsik dbd62a4992
feat: Introduce advanced permissions (#7844)
This PR introduces the possibility of inviting new users with an `admin`
role and changing the role of already invited users.
Also using scoped permission checks where applicable instead of using
user role checks.

---------

Co-authored-by: Val <68596159+valya@users.noreply.github.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-12-08 12:52:25 +01:00
Iván Ovejero 8cb9c6b3ea
ci: Introduce no-plain-errors lint rule for BE packages (no-changelog) (#7961)
## Summary
Require `ApplicationError` or its child classes instead of plain `Error`
in BE packages. This ensures the error will be normalized when reported
to Sentry, if applicable.

Follow-up to:
https://github.com/n8n-io/n8n/pulls?q=is%3Apr+is%3Aclosed+applicationerror

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-08 12:51:49 +01:00
Iván Ovejero 90824b50ed
feat(core): Add package name tag to Sentry errors (no-changelog) (#7958)
## Summary
Add `packageName` tag to errors reported to Sentry, for filtering

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-08 11:30:06 +01:00
Iván Ovejero c48850d74f
fix(core): Fix expression evaluator filename typo (no-changelog) (#7960)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 17:50:49 +01:00
Iván Ovejero 18bb87ac0c
test(core): Isolate restore binary data ID in execution lifecycle hooks (no-changelog) (#7953)
## Summary
Move guard into restore function, move tests into own suite, add tests
for guard.

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 17:30:47 +01:00
Iván Ovejero 1d46983b24
refactor: Unify severity and level for all application errors for Sentry (no-changelog) (#7956)
## Summary
Unify `severity` and `level` for all backend application errors for
Sentry

Follow-up to:
https://github.com/n8n-io/n8n/pull/7914#issuecomment-1840433542

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 16:57:02 +01:00
Val 42e828d5c6
fix: Restrict updating/deleting of shared but not owned credentials (#7950)
## Summary

Fix shared members being able to edit and delete credentials they don't
own

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [x] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [x] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 10:35:40 +00:00
Omar Ajoue 3ba7deb337
fix: Ensure external hooks post workflow execute run in queue mode (#7947)
## Summary
Since 1.8.x a refactor removed the call to `workflow.postExecute`'s
External hook from the execution path. This PR adds it back to the
correct place, where workers are supposed to call this, allowing us to
avoid having to re-read the execution data in the caller just for the
hooks.

It is important to have the hooks running in the worker whenever
possible to prevent having to read the full execution data in the
caller.

#### How to test the change:
1. Use the attached hooks file
[external-hooks.txt](https://github.com/n8n-io/n8n/files/13597270/external-hooks.txt)
setting it up via environment variable using `export
EXTERNAL_HOOK_FILES=/path/to/hooks/external-hooks.js`
2. Set up queue mode loading this file in both main and workers
3. See that the message logs will not be displayed without this fix


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:



## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 10:00:05 +00:00
Iván Ovejero 386bd61967
fix(core): Ensure inviter and invitee are set correctly in invite link (#7943)
## Summary
Ensure inviter and invitee are set correctly in invite link

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

https://linear.app/n8n/issue/ADO-1494


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 10:53:31 +01:00
Michael Kret 565b409a82
fix(Webhook Node): Binary data handling (#7804)
Github issue / Community forum post (link here to close automatically):
2023-12-06 17:46:40 +02:00
Csaba Tuncsik c461025f70
test: Add user type of admin to E2E tests (#7935)
## Summary
Extend existing user types in the E2E database. Currently, we have only
owner and member but we need also admin

---------

Co-authored-by: Val <68596159+valya@users.noreply.github.com>
2023-12-06 14:31:06 +01:00
Iván Ovejero 92bab72cff
fix(core): Fix user comparison in same-user subworkflow caller policy (#7913)
https://linear.app/n8n/issue/PAY-992


https://community.n8n.io/t/executing-workflow-using-owner-role-created-by-another-user-fails/33443

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-12-06 12:27:11 +00:00
Ricardo Espinoza f5502cc628
fix(core): Make sure mfa secret and recovery codes are not returned on login (#7936)
## Summary

What: Fix issue of login endpoint returning secret and recovery codes
when MFA is enabled. Bug was introduced in this
[PR](https://github.com/n8n-io/n8n/pull/6994), specifically in this
[line](https://github.com/n8n-io/n8n/pull/6994/files#diff-95a87cb029a3d26e6722df2e68132453fc254fc1f4540cbdaa95cfdbda1893deL91).

Why: We should not be filtering the secret and recovery codes

Same PR caused the issues on ticket ->
https://linear.app/n8n/issue/ADO-1494/on-user-list-copy-password-reset-link-and-copy-invite-link-are-broken

## Review / Merge checklist
- [x] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [x] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [x] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-06 10:00:13 +01:00
Andrey Starostin 46dd4d3105
fix(core): Fix hard deletes stopping if database query throws (#7848)
I have observed that the next hard deletion timeout is not scheduled if
the `hardDeleteOnPruningCycle` function throws when fetching the data
from the database. That is because the thrown error is not caught and
the `scheduleHardDeletion` method is not called.

This PR moves the call to `scheduleHardDeletion` into the
`scheduleHardDeletion` for better cohesion, and ensures that it is
called even if `hardDeleteOnPruningCycle` throws.
2023-12-05 16:30:32 +01:00
Val 4e55583715
feat: Add admin role to public API (no-changelog) (#7933)
## Summary
Add the admin global role to the public API. This does not include
porting over scopes.

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-05 15:00:14 +00:00
Val 9604b87da9
fix: Return scopes on invitation accept endpoint (no-changelog) (#7917)
## Summary
Return scopes on the invitation accept endpoint. The UI uses information
until the user refreshes the pages so it's causing inconsistency for the
new admin role.

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [x] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-05 11:18:41 +01:00
Iván Ovejero 38b88b946b
fix(core): Consolidate ownership and sharing data on workflows and credentials (#7920)
## Summary

Ensure `ownedBy` and `sharedWith` are present and uniform for
credentials and workflows.

Details in story: https://linear.app/n8n/issue/PAY-987
2023-12-05 10:11:18 +01:00
Iván Ovejero 07f6662aba
refactor(core): Add log about removed TTL keys for binary data (#7892)
Context:
https://n8nio.slack.com/archives/C035KBDA917/p1701418556761549?thread_ts=1701411854.517989&cid=C035KBDA917
2023-12-01 12:51:45 +01:00
Iván Ovejero 61d8aebeaf
fix(core): Allow grace period for binary data deletion after manual execution (#7889)
https://linear.app/n8n/issue/PAY-1079
2023-12-01 10:13:53 +01:00
Val cd474f1562
feat: Allow owner to share workflows/credentials they don't own (no-changelog) (#7869)
Github issue / Community forum post (link here to close automatically):
2023-11-29 16:32:27 +00:00
Omar Ajoue 74b4513298
feat(core): Add Advanced Permissions to FE settings (no-changelog) (#7867)
Github issue / Community forum post (link here to close automatically):
2023-11-29 15:56:35 +01:00
Val 1cb92ffe16
feat: Replace owner checks with scope checks (no-changelog) (#7846)
Github issue / Community forum post (link here to close automatically):
2023-11-29 14:48:36 +00:00
Iván Ovejero 476806ebb0
feat(core): Allow admin creation (#7837)
https://linear.app/n8n/issue/PAY-1038
2023-11-29 13:55:41 +01:00
Val e282ea242d
fix: Return scopes on owner setup endpoint (no-changelog) (#7860)
Github issue / Community forum post (link here to close automatically):
2023-11-29 11:33:32 +00:00
Iván Ovejero c08c5cc37b
refactor(core): Switch plain errors in cli to ApplicationError (#7857)
Ensure all errors in `cli` are `ApplicationError` or children of it and
contain no variables in the message, to continue normalizing all the
errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7839
2023-11-29 12:25:10 +01:00
Jan Oberhauser 87def60979
feat: Add AI tool building capabilities (#7336)
Github issue / Community forum post (link here to close automatically):
https://community.n8n.io/t/langchain-memory-chat/23733

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Val <68596159+valya@users.noreply.github.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: Deborah <deborah@starfallprojects.co.uk>
Co-authored-by: Jesper Bylund <mail@jesperbylund.com>
Co-authored-by: Jon <jonathan.bennetts@gmail.com>
Co-authored-by: Michael Kret <88898367+michael-radency@users.noreply.github.com>
Co-authored-by: Giulio Andreini <andreini@netseven.it>
Co-authored-by: Mason Geloso <Mason.geloso@gmail.com>
Co-authored-by: Mason Geloso <hone@Masons-Mac-mini.local>
Co-authored-by: Mutasem Aldmour <mutasem@n8n.io>
2023-11-29 12:13:55 +01:00
Alex Grozav 72852a60eb
feat(editor): Replace middleware for Role checks with Scope checks (#7847) 2023-11-29 10:35:40 +02:00
कारतोफ्फेलस्क्रिप्ट™ 117962d473
feat(core): Update LLM applications building support (no-changelog) (#7710)
extracted out of #7336

---------

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
2023-11-28 16:47:28 +01:00
Iván Ovejero 2356fb0f0c
feat(core): Set up endpoint for all existing roles with license flag (#7834)
https://linear.app/n8n/issue/PAY-1034/create-endpoint-to-list-all-existing-roles
2023-11-28 14:16:47 +01:00
Val a37f1cb0ba
feat: Add initial scope checks via decorators (#7737) 2023-11-28 11:41:34 +00:00
Csaba Tuncsik 3ab3ec9da8
fix(editor): Allow owners and admins to share workflows and credentials they don't own (#7833) 2023-11-28 11:44:55 +01:00
Iván Ovejero 1c6178759c
refactor(core): Reorganize error hierarchy in cli package (no-changelog) (#7839)
Ensure all errors in `cli` inherit from `ApplicationError` to continue
normalizing all the errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7820
2023-11-28 10:19:27 +01:00
Iván Ovejero 9b87a596ca
fix(core): Ensure member and admin cannot be promoted to owner (#7830)
https://linear.app/n8n/issue/PAY-985/add-user-role-modification-endpoint#comment-62355f6b
2023-11-27 17:35:58 +01:00
Omar Ajoue 847f6ac771
fix(core): Prevent error messages due to statistics about data loading (#7824)
Statistics collection about the first time a workflow loads data simply
attempts an insert to db, and if it fails, we just ignore.

This was causing this query to fire against production workflows
multiple times, and since we want to insert only and detect whether the
insertion failed, performing a select first provides gains both in terms
of performance, as it's usually faster than trying an insertion as well
as preventing unnecessary noise in logs.

Github issue / Community forum post (link here to close automatically):

https://community.n8n.io/t/duplicate-key-value-violates-unique-constraint-workflow-statistics-pkey-still-happening/29283
https://github.com/n8n-io/n8n/issues/7256
https://community.n8n.io/t/error-log-arriving-in-postgres/30191
https://github.com/n8n-io/n8n/issues/7256

https://community.n8n.io/t/cant-launch-webhooks-unable-to-find-data-of-execution/31867

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-27 15:43:48 +01:00
Iván Ovejero dff8456382
refactor(core): Reorganize error hierarchy in core and workflow packages (no-changelog) (#7820)
Ensure all errors in `core` and `workflow` inherit from
`ApplicationError` so that we start normalizing all the errors we report
to Sentry

Follow-up to:
https://github.com/n8n-io/n8n/pull/7757#discussion_r1404338844

### `core` package

`ApplicationError`
- `FileSystemError` (abstract)
	- `FileNotFoundError`
	- `DisallowedFilepathError`
- `BinaryDataError` (abstract)
	- `InvalidModeError`
	- `InvalidManagerError`
- `InvalidExecutionMetadataError`

### `workflow` package

`ApplicationError`
- `ExecutionBaseError` (abstract)
	- `WorkflowActivationError`
		- `WorkflowDeactivationError`
		- `WebhookTakenError`
	- `WorkflowOperationError`
		- `SubworkflowOperationError`
			- `CliWorkflowOperationError`
	- `ExpressionError`
		- `ExpressionExtensionError`
	- `NodeError` (abstract)
		- `NodeOperationError`
		- `NodeApiError`
	- `NodeSSLError`

Up next:
- Reorganize errors in `cli`
- Flatten the hierarchy in `workflow` (do we really need
`ExecutionBaseError`?)
- Remove `ExecutionError` type
- Stop throwing plain `Error`s
- Replace `severity` with `level`
- Add node and credential types as `tags`
- Add workflow IDs and execution IDs as `extras`
2023-11-27 15:33:21 +01:00
Val 27e048c201
feat: Add Licensed decorator (no-changelog) (#7828)
Github issue / Community forum post (link here to close automatically):
2023-11-27 13:46:18 +00:00
Val 5acb7b94c0
refactor: Refactor variables controller into a RestController (no-changelog) (#7822)
Github issue / Community forum post (link here to close automatically):
2023-11-27 12:17:09 +00:00
Iván Ovejero 7b8532d3a3
refactor(core): Move execution progress saving to standalone utility (no-changelog) (#7770)
This PR continues the effort of moving logic inside execution lifecycle
hooks into standalone testable functions, as a stepping stone to
refactoring the hooks themselves.
2023-11-27 13:10:43 +01:00
Csaba Tuncsik 152883eed1
build: Upgrade lint related packages (#7790) 2023-11-27 09:11:52 +01:00
Iván Ovejero eec2ec1ff8
refactor(core): Consolidate path-related errors in Sentry (no-changelog) (#7757)
Keep reporting [path-related
errors](https://n8nio.sentry.io/issues/4649493725) in Sentry but
consolidate them in a single error group.

Also, add `options.extra` as `meta` so they remain visible in debug
logs:

```
2023-11-24T11:50:54.852Z | error    | ReportableError: Something went wrong "{ test: 123, file: 'LoggerProxy.js', function: 'exports.error' }"
```

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-24 14:42:46 +01:00
Iván Ovejero 7a86d36068
feat(core): Allow user role modification (#7797)
https://linear.app/n8n/issue/PAY-985

```
PATCH /users/:id/role
  unauthenticated user
    ✓ should receive 401 (349 ms)
  member
    ✓ should fail to demote owner to member (349 ms)
    ✓ should fail to demote owner to admin (359 ms)
    ✓ should fail to demote admin to member (381 ms)
    ✓ should fail to promote other member to owner (353 ms)
    ✓ should fail to promote other member to admin (377 ms)
    ✓ should fail to promote self to admin (354 ms)
    ✓ should fail to promote self to owner (371 ms)
  admin
    ✓ should receive 400 on invalid payload (351 ms)
    ✓ should receive 404 on unknown target user (351 ms)
    ✓ should fail to demote owner to admin (349 ms)
    ✓ should fail to demote owner to member (347 ms)
    ✓ should fail to promote member to owner (384 ms)
    ✓ should fail to promote admin to owner (350 ms)
    ✓ should be able to demote admin to member (354 ms)
    ✓ should be able to demote self to member (350 ms)
    ✓ should be able to promote member to admin (349 ms)
  owner
    ✓ should be able to promote member to admin (349 ms)
    ✓ should be able to demote admin to member (349 ms)
    ✓ should fail to demote self to admin (348 ms)
    ✓ should fail to demote self to member (354 ms)
```
2023-11-24 11:40:08 +01:00
Val 865192adf0
feat: Add global admin role (no-changelog) (#7781)
Github issue / Community forum post (link here to close automatically):
2023-11-23 13:38:11 +00:00
Michael Auerswald 5835e055d3
fix(editor): Show nice error when environment is not set up (#7778)
Adds a nicer error message with a link for owners who press Push to Git
without having a repository connected yet.
2023-11-23 13:50:03 +01:00
Iván Ovejero fdb2c18ecc
fix(core): Tighten checks for multi-main setup usage (#7788)
https://n8nio.slack.com/archives/C05HRPLSGTT/p1700731476321999?thread_ts=1700729359.746899&cid=C05HRPLSGTT
2023-11-23 12:18:39 +01:00
Csaba Tuncsik e128b23a2b
build: Upgrade to Vite 5 (#7784) 2023-11-23 11:55:02 +01:00
Milorad FIlipović 77bc8ecd4b
feat(editor): Show avatars for users currently working on the same workflow (#7763)
This PR introduces the following changes:
- New Vue stores: `collaborationStore` and `pushConnectionStore`
- Front-end push connection handling overhaul: Keep only a singe
connection open and handle it from the new store
- Add user avatars in the editor header when there are multiple users
working on the same workflow
- Sending a heartbeat event to back-end service periodically to confirm
user is still active

- Back-end overhauls (authored by @tomi):
  - Implementing a cleanup procedure that removes inactive users
  - Refactoring collaboration service current implementation

---------

Co-authored-by: Tomi Turtiainen <10324676+tomi@users.noreply.github.com>
2023-11-23 10:14:34 +01:00
Michael Kret 99a9ea497a
feat(core): Add Support for custom CORS origins for webhooks (#7455)
node-850
https://community.n8n.io/t/add-ability-to-set-cors-allow-list-in-n8n-webhooks/7610
https://community.n8n.io/t/configure-cors-pre-flight-request-option-method-in-the-roadmap/32189

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-22 17:49:56 +01:00
कारतोफ्फेलस्क्रिप्ट™ db77353272
fix(core): Pass correct node reference to NodeExecuteFunctions.getLoadOptionsFunctions (no-changelog) (#7779)
NODE-947
2023-11-22 13:24:23 +01:00
Milorad FIlipović 2064f7f251
fix(editor): Validate user info before submiting (#7608)
Validate first and last names before saving them to database. This
should prevent security issue with un-sanitized data that ends up in
emails.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-22 12:14:44 +01:00
कारतोफ्फेलस्क्रिप्ट™ 9b4856e7de
feat(core): Make postgres pool-size configurable (no-changelog) (#7772)
When we upgrade typeorm in #5151, we switched from no pooling to a
default pool-size of 10. This somehow significantly deteriorates the
performance of queries when the application is under load.
2023-11-21 18:13:08 +01:00
Iván Ovejero 3459eb6c2f
refactor(core): Include execution progress in save settings (no-changelog) (#7769) 2023-11-21 17:33:44 +01:00
Iván Ovejero b7c5c7406f
fix(core): Ensure failed executions are saved in queue mode (#7744)
This PR adds `status` to run data so that
`determineFinalExecutionStatus` resolves correctly on execution failure
and removes the cleanup that is being duplicated in a worker hook.

Followup to https://github.com/n8n-io/n8n/pull/7138

Should fix:
- https://github.com/n8n-io/n8n/issues/7705
-
https://linear.app/n8n/issue/PAY-964/no-execution-found-after-execution-fails
-
https://linear.app/n8n/issue/PAY-1010/execution-deletion-in-queue-mode-not-complying-with-settings
2023-11-20 16:03:02 +01:00
Iván Ovejero 6d19f88080
refactor(core): Stop reporting to Sentry unknown cred on mapping (no-changelog) (#7752)
https://n8nio.sentry.io/issues/4563418905
2023-11-17 16:17:44 +01:00
Iván Ovejero 4c4082503c
feat(core): Coordinate manual workflow activation and deactivation in multi-main scenario (#7643)
Followup to #7566 | Story: https://linear.app/n8n/issue/PAY-926

### Manual workflow activation and deactivation

In a multi-main scenario, if the user manually activates or deactivates
a workflow, the process (whether leader or follower) that handles the
PATCH request and updates its internal state should send a message into
the command channel, so that all other main processes update their
internal state accordingly:

- Add to `ActiveWorkflows` if activating
- Remove from `ActiveWorkflows` if deactivating
- Remove and re-add to `ActiveWorkflows` if the update did not change
activation status.

After updating their internal state, if activating or deactivating, the
recipient main processes should push a message to all connected
frontends so that these can update their stores and so reflect the value
in the UI.

### Workflow activation errors

On failure to activate a workflow, the main instance should record the
error in Redis - main instances should always pull activation errors
from Redis in a multi-main scenario.

### Leadership change

On leadership change...

- The old leader should stop pruning and the new leader should start
pruning.
- The old leader should remove trigger- and poller-based workflows and
the new leader should add them.
2023-11-17 15:58:50 +01:00
कारतोफ्फेलस्क्रिप्ट™ fc60e9a809
refactor(core): Convert dynamic node-parameter routes to a decorated controller (no-changelog) (#7284)
1. Reduce a lot of code duplication
2. Move more endpoints out of `Server.ts`
3. Move all query-param parsing and validation into a middleware to make
the route handlers simpler.
2023-11-17 12:03:05 +01:00
Iván Ovejero 0408299c7d
refactor(core): Stop reporting to Sentry unrecognized node errors (no-changelog) (#7728)
https://n8nio.sentry.io/issues/4636584213
2023-11-17 10:25:10 +01:00
Iván Ovejero b4ebb1a28d
fix(core): Account for non-ASCII chars in filename on binary data download (#7742)
https://n8nio.sentry.io/issues/4641538638
2023-11-17 10:07:44 +01:00
Ricardo Espinoza 8e0ae3cf8c
refactor: Extract Invitation routes to InvitationController (no-changelog) (#7726)
This PR:

- Creates `InvitationController`
- Moves `POST /users` to `POST /invitations` and move related test to
`invitations.api.tests`
- Moves `POST /users/:id` to `POST /invitations/:id/accept` and move
related test to `invitations.api.tests`
- Adjusts FE to use new endpoints
- Moves all the invitation logic to the `UserService`

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-16 12:39:43 -05:00
Csaba Tuncsik e2ffd397fc
fix(editor): Handle permission edge cases (empty scopes) (#7723) 2023-11-16 18:08:23 +01:00
Val d39bb2540f
feat: Add scopes to /login endpoint (no-changelog) (#7718)
Github issue / Community forum post (link here to close automatically):
2023-11-16 11:11:55 +00:00
Michael Auerswald f96c1d2044
fix(core): Correct permissions for getstatus (#7724) 2023-11-15 16:54:33 +01:00
कारतोफ्फेलस्क्रिप्ट™ 2844ce2bd2
ci: Fix env issues in e2e tests (no-changelog) (#7719) 2023-11-15 13:58:38 +01:00
Ricardo Espinoza 4020c14d59
refactor: Use POST /users to re-invite users (no-changelog) (#7714) 2023-11-15 06:40:57 -05:00
कारतोफ्फेलस्क्रिप्ट™ 3460eb5eeb
fix(core): Initialize JWT Secret before it's used anywhere (#7707)
HELP-394
2023-11-15 12:17:18 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4441ed5116
fix(core): Fix named parameter resolution in migrations (#7688)
Fixes #7628
2023-11-15 10:31:08 +01:00
Michael Auerswald 0e00dab9f5
fix(core): Consider subworkflows successfully run when in waiting state (#7699)
Github issue / Community forum post (link here to close automatically):
https://github.com/n8n-io/n8n/issues/7189
2023-11-14 11:04:24 +01:00
Elias Meire 14035e1244
feat(editor): Add HTTP request nodes for credentials without a node (#7157)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-13 12:11:16 +01:00
Iván Ovejero b2ca050031
perf(core): Lazyload security audit reporters (#7696)
Also converting to service.

Followup to https://github.com/n8n-io/n8n/pull/7663
2023-11-13 11:50:43 +01:00
Michael Auerswald cbc690907f
feat(editor): Adds a EE view to show worker details and job status (#7600)
This change expands on the command channel communication introduced
lately between the main instance(s) and the workers. The frontend gets a
new menu entry "Workers" which will, when opened, trigger a regular call
to getStatus from the workers. The workers then respond via their
response channel to the backend, which then pushes the status to the
frontend.
This introduces the use of ChartJS for metrics.
This feature is still in MVP state and thus disabled by default for the
moment.
2023-11-10 23:48:31 +01:00
कारतोफ्फेलस्क्रिप्ट™ 000e76e3b4
ci(core): Reduce memory usage in tests (part-2) (no-changelog) (#7671)
This also gets rid of `Db.collection`, which was another source of
circular dependencies.
2023-11-10 15:04:26 +01:00
Iván Ovejero f73a0597ba
refactor: Add rule no-constant-binary-expression (no-changelog) (#7670)
https://eslint.org/docs/latest/rules/no-constant-binary-expression
2023-11-09 17:50:59 +01:00
कारतोफ्फेलस्क्रिप्ट™ 9fd6319583
fix(core): Reduce memory usage in credentials risk auditing (#7663) 2023-11-09 14:39:16 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5edf722209
ci: Stop disabling linting rules that are explicitly set to warn (no-changelog) (#7669) 2023-11-09 14:05:39 +01:00
Iván Ovejero 16a1a92f18
refactor(core): Stop reporting to Sentry credential-not-found error (no-changelog) (#7665)
https://n8nio.sentry.io/issues/4528134408
2023-11-09 10:55:13 +01:00
कारतोफ्फेलस्क्रिप्ट™ 0346b211a7
ci(core): Reduce memory usage in tests (part-1) (no-changelog) (#7654) 2023-11-08 16:29:39 +01:00
Iván Ovejero 3a776e0132
refactor(core): Stop reporting to Sentry missing-node-on-retry error (no-changelog) (#7648)
https://n8nio.sentry.io/issues/4612370603
2023-11-08 10:09:33 +01:00
Iván Ovejero 48c068f97b
fix(core): Comply with custom default for workflow saving settings (#7634)
https://linear.app/n8n/issue/PAY-982
2023-11-07 16:26:55 +01:00
Tomi Turtiainen ac877014ed
feat(core): Initial support for two-way communication over websockets (#7570)
- Enable two-way communication with web sockets
- Enable sending push messages to specific users
- Add collaboration service for managing active users for workflow

Missing things:
- State is currently kept only in memory, making this not work in
multi-master setups
- Removing a user from active users in situations where they go inactive
or we miss the "workflow closed" message
- I think a timer based solution for this would cover most edge cases.
I.e. have FE ping every X minutes, BE removes the user unless they have
received a ping in Y minutes, where Y > X
- FE changes to be added later by @MiloradFilipovic 

Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-07 17:26:45 +02:00
कारतोफ्फेलस्क्रिप्ट™ f0fc5b16d3
ci(core): Load config schema after process.env has been overwritten (no-changelog) (#7550) 2023-11-07 15:58:28 +01:00
कारतोफ्फेलस्क्रिप्ट™ 60314248f4
fix(core): Make password-reset urls valid only for single-use (#7622) 2023-11-07 15:35:43 +01:00
Ricardo Espinoza b3470fd64d
fix: Error handling on forgot password page (no-changelog) (#7633)
fixes:
https://linear.app/n8n/issue/ADO-1339/fix-error-handling-on-forgot-password-page
2023-11-07 08:45:58 -05:00
Iván Ovejero f748de9567
fix(core): Ensure pruning starts only after migrations have completed (#7626)
https://linear.app/n8n/issue/PAY-986/bug-execution-pruning-timer-is-started-before-the-database-is-ready

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-11-07 13:56:01 +01:00
Iván Ovejero c857e42677
feat(core): Coordinate workflow activation in multiple main scenario in internal API (#7566)
Story: https://linear.app/n8n/issue/PAY-926

This PR coordinates workflow activation on instance startup and on
leadership change in multiple main scenario in the internal API. Part 3
on manual workflow activation and deactivation will be a separate PR.

### Part 1: Instance startup

In multi-main scenario, on starting an instance...
- [x] If the instance is the leader, it should add webhooks, triggers
and pollers.
- [x] If the instance is the follower, it should not add webhooks,
triggers or pollers.
- [x] Unit tests.

### Part 2: Leadership change 

In multi-main scenario, if the main instance leader dies…

- [x] The new main instance leader must activate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] The old main instance leader must deactivate all trigger- and
poller-based workflows, excluding webhook-based workflows.
- [x] Unit tests.

To test, start two instances and check behavior on startup and
leadership change:

```
EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug npm run start

EXECUTIONS_MODE=queue N8N_LEADER_SELECTION_ENABLED=true N8N_LICENSE_TENANT_ID=... N8N_LICENSE_ACTIVATION_KEY=... N8N_LOG_LEVEL=debug N8N_PORT=5679 npm run start
```
2023-11-07 13:48:48 +01:00
Iván Ovejero a994ba5e8d
fix(core): Ensure init before checking leader or follower in multi-main scenario (#7621)
This PR ensures `MultiMainInstancePublisher` is initialized before
checking if the instance is leader or follower. Followers skip license
init, license check, and pruning start and stop.
2023-11-06 12:03:35 +01:00
Ricardo Espinoza 5790e251b8
feat(core): Rate limit forgot password endpoint (#7604)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
2023-11-03 13:44:12 -04:00
कारतोफ्फेलस्क्रिप्ट™ acec9bad71
refactor(core): Convert OAuth1/OAuth2 routes to decorated controller classes (no-changelog) (#5973) 2023-11-03 17:20:54 +01:00
Tomi Turtiainen 2aa7f6375a
fix(core): Decrease reset password token expire time (#7598)
Decrease the expiration time from 1 day to 20 minutes

Github issue / Community forum post (link here to close automatically):
2023-11-03 13:32:08 +02:00
Iván Ovejero 1c77d6597f
refactor(core): Ensure only leader handles licensing in multi-main scenario (#7558)
https://linear.app/n8n/issue/PAY-953/ensure-only-main-instance-leader-handles-licensing
2023-11-02 14:16:22 +01:00
Tomi Turtiainen 7dac9ab82c
feat(core): Log executed migrations with info level (#7586)
To help debugging possible issues in startup and migrations, log the
executed migrations with log level 'info', instead of 'debug'.

Github issue / Community forum post (link here to close automatically):
2023-11-02 14:24:04 +02:00
Iván Ovejero 921d213ae5
refactor(core): Create pruning service (no-changelog) (#7564)
https://linear.app/n8n/issue/PAY-954/ensure-only-main-instance-leader-handles-pruning
2023-11-02 12:24:25 +01:00
Jon 479f90231d
fix(core): Fix issue that prevents owner logging in when using ldap (#7408)
This PR prioritises the internal email account over LDAP for the Owner.

---------

Co-authored-by: ricardo <ricardoespinoza105@gmail.com>
2023-11-01 23:02:49 -04:00
Omar Ajoue 437c95e84e
fix(core): Permission check for subworkflow properly checking for workflow settings (#7576)
The `sharing` related code is legacy that was not removed. Subworkflow
execution should check workflow settings alone, and this is now
reflected in the code.

Github issue / Community forum post (link here to close automatically):

https://community.n8n.io/t/bug-when-using-the-execute-workflow-node-when-workflow-is-shared/32207

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-11-01 18:31:34 +01:00
कारतोफ्फेलस्क्रिप्ट™ 9bdb85c4ce
fix(core): Prevent executions from becoming forever running (#7569)
Fixes CP-867
Possibly also fixes PAY-323 and PAY-412
2023-11-01 13:51:13 +01:00
Michael Auerswald b350568505
fix(core): Fix data decryption on credentials import (#7560)
Due to a change, during the credentials import command, the core's
Credential object is being called through its prototype. This caused the
Credential's cipher variable to not be set, thus no cipher service being
available during import. This fix catches this edge case and provides a
fix.
2023-10-31 13:15:09 +01:00
Jan Oberhauser 655efeaf66
feat(core): Add optional Error-Output (#7460)
Add an additional optional error output to which all items get sent that
could not be processed.
![Screenshot from 2023-10-18
17-29-15](https://github.com/n8n-io/n8n/assets/6249596/e9732807-ab2b-4662-a5f6-bdff24f7ad55)

Github issue / Community forum post (link here to close automatically):
https://community.n8n.io/t/error-connector-for-nodes/3094

https://community.n8n.io/t/error-handling-at-node-level-detect-node-execution-status/26791

---------

Co-authored-by: OlegIvaniv <me@olegivaniv.com>
2023-10-30 18:42:47 +01:00
Iván Ovejero 442c73e63b
feat(core): Set up leader selection for multiple main instances (#7527)
https://linear.app/n8n/issue/PAY-933/set-up-leader-selection-for-multiple-main-instances

- [x] Set up new envs
- [x] Add config and license checks
- [x] Implement `MultiMainInstancePublisher`
- [x] Expand `RedisServicePubSubPublisher` to support
`MultiMainInstancePublisher`
- [x] Init `MultiMainInstancePublisher` on startup and destroy on
shutdown
- [x] Add to sandbox plans
- [x] Test manually

Note: This is only for setup - coordinating in reaction to leadership
changes will come in later PRs.
2023-10-30 16:22:32 +01:00
OlegIvaniv 3ab04e4f9e
ci(core): Extract local e2e run script (no-changelog) (#7551)
Github issue / Community forum post (link here to close automatically):

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-10-30 11:18:51 +01:00
Iván Ovejero 233683ce1a
refactor(core): Remove webhook deregistration on shutdown flag (#7543)
Removal missing from [original
PR](https://github.com/n8n-io/n8n/pull/7515) after initial revert.
2023-10-27 18:48:21 +02:00
Iván Ovejero 6796ba75a1
refactor(core): Stop reporting to Sentry stopping execution with wrong status (no-changelog) (#7541)
https://n8nio.sentry.io/issues/4257040218/
2023-10-27 16:14:01 +02:00
कारतोफ्फेलस्क्रिप्ट™ 35bb42c1b9
refactor(core): Avoid passing around static state like default timezone (no-changelog) (#7221) 2023-10-27 14:17:52 +02:00
Iván Ovejero 62c096710f
refactor: Run lintfix (no-changelog) (#7537)
- Fix autofixable violations
- Remove unused directives
- Allow for PascalCased variables - needed for dynamically imported or
assigned classes, decorators, routers, etc.
2023-10-27 14:15:02 +02:00
Omar Ajoue 87996a1fcf
refactor(core): Stop reporting non-error to sentry (issue 4229454473) (no-changelog) (#7525)
This PR aims to stop reporting issues such as [this
one](https://n8nio.sentry.io/issues/4229454473/events/42b96bfd6a334c15a84499e981cf90eb/?project=4503924908883968).

Github issue / Community forum post (link here to close automatically):
2023-10-27 09:25:07 +02:00
Omar Ajoue 3d95b243e9
feat(core): Make queue mode settings configurable (#7526)
This PR allows users to configure the settings to Bull, possibly
reducing the errors with `maxStalledCount` and other issues, that
usually happen either when a worker crashes or when the event loop is
super busy. Increasing the lease time and the `maxStalledCount` settings
might improve UX.

Github issue / Community forum post (link here to close automatically):
2023-10-26 17:30:16 +02:00
Iván Ovejero ae8c7a635e
refactor(core)!: Remove webhook deregistration at startup and shutdown (#7515)
https://linear.app/n8n/issue/PAY-932/deprecate-flag-to-skip-webhook-deregistration-on-shutdown
2023-10-26 14:37:54 +02:00
Iván Ovejero 5477e3fb45
refactor(core): Move execution save settings into lifecycle function (no-changelog) (#7370)
Move the handling of execution save settings into a tested lifecycle
function as discussed with Omar
2023-10-26 14:35:38 +02:00
कारतोफ्फेलस्क्रिप्ट™ 1055bd3762
fix(core): Handle missing resultData in runData (#7523) 2023-10-26 13:15:19 +02:00
Iván Ovejero 742c8a8534
fix(core): Ensure execution deletion in worker lifecycle hook (#7481)
Reported by customer
[here](https://n8nio.slack.com/archives/C05PUALKZHD/p1697446945481249?thread_ts=1697196557.638169&cid=C05PUALKZHD),
apparently a very old long-standing bug for queue mode. Please review
closely as I am not familiar with queue mode.
2023-10-25 19:13:06 +02:00
कारतोफ्फेलस्क्रिप्ट™ 05586a900d
refactor(core): Make Logger a service (no-changelog) (#7494) 2023-10-25 16:35:22 +02:00
कारतोफ्फेलस्क्रिप्ट™ 0643487403 ci: Fix oclif manifest generation 2023-10-25 14:51:10 +02:00
कारतोफ्फेलस्क्रिप्ट™ 6f45298d3d
fix(core): Ensure nodes post-processors run in the correct order (#7500)
Fixes #7497
2023-10-25 13:59:38 +02:00
Val 93cfabbeac
fix(core): Fix workflow activation with history and workflow history for EE (no-changelog) (#7508)
Github issue / Community forum post (link here to close automatically):
2023-10-25 11:07:11 +01:00
Iván Ovejero 671c95760b
refactor(core): Make executions pruning more resilient (#7480)
This PR converts the hard-deletion interval to a timeout:
- to prevent the interval from not being restored when hard deletion
throws, and
- to prevent a long-running hard deletion from leading to duplicate
deletions.
2023-10-24 16:16:45 +02:00
Iván Ovejero 78243edd18
refactor(core): Make pruning via lifecycle configuration in S3 mode mandatory (#7482)
Since we do not store which executions produced binary data, for pruning
on S3 we need to query for binary data items for each execution in order
to delete them. To minimize requests to S3, allow the user to skip
pruning requests when setting TTL at bucket level.
2023-10-24 10:37:02 +02:00
Cornelius Suermann 3ef771ea38
feat: Collect usage metrics on license renewal (no-changelog) (#7486) 2023-10-23 21:39:29 +02:00
Ricardo Espinoza 774fe202bf
fix(core): Fix frontend.settings external hook execution (#7496) 2023-10-23 11:46:13 -04:00
Val 41236b7e08
fix: Save new version of the workflow instead of the previous (no-changelog) (#7428)
Github issue / Community forum post (link here to close automatically):
2023-10-23 16:30:36 +02:00
कारतोफ्फेलस्क्रिप्ट™ b6de910cbe
refactor(core): Abstract away InstanceSettings and encryptionKey into injectable services (no-changelog) (#7471)
This change ensures that things like `encryptionKey` and `instanceId`
are always available directly where they are needed, instead of passing
them around throughout the code.
2023-10-23 13:39:35 +02:00
कारतोफ्फेलस्क्रिप्ट™ 76c04815f7
fix(core): Reduce logging overhead for levels that do not output (#7479)
all current logging calls execute `callsites()` to figure out what code
tried to log. This happens even for logging methods that aren't supposed
to create any output. Under moderate load, this can take up quite a lot
of resources. This PR changes the logger to make all ignorable logging
methods a No-Op.

In a small benchmark with a simple webhook, with log-level set to
`warn`, and using `ab -c 50 -n 500
http://localhost:5678/webhook/testing`, these were the response times:

### Before

![Before](https://github.com/n8n-io/n8n/assets/196144/01680fd9-3d2a-4f7f-bb1c-5b03bd7d5bc3)

### After

![After](https://github.com/n8n-io/n8n/assets/196144/ccacb20a-48ca-455a-a8cb-098c9c0e352e)
2023-10-20 18:26:33 +02:00
Iván Ovejero 0b42d1aa71
refactor(core): Limit soft-deletions to pruning only (#7469)
Based on customer feedback, we should limit soft deletions to pruning
only, to prevent executions from piling up in very high volume cases.
2023-10-20 15:02:47 +02:00
Jon 647372be27
feat(core): Add support for oauth based service accounts with UM SMTP (#7311)
This PR adds support for using OAuth based service accounts for the User
Management SMTP connection.

Tested using a Google Service Account.
2023-10-20 11:36:40 +01:00
Iván Ovejero b50376cf52
refactor(core): Make executions pruning settings configurable (#7468) 2023-10-19 16:57:12 +02:00
कारतोफ्फेलस्क्रिप्ट™ 55c6a1b0d3
fix(core): Do not return inviteAcceptUrl in response if email was sent (#7465) 2023-10-19 13:58:06 +02:00
कारतोफ्फेलस्क्रिप्ट™ 83762e051d
fix(core): Handle gzip and deflate compressed request payloads (#7461)
NODE-870
2023-10-18 21:52:15 +02:00
Iván Ovejero 40707fa692
feat(core): Make executions pruning interval configurable (#7439) 2023-10-18 17:01:57 +02:00
Michael Kret 3ddc176dfa
feat(n8n Form Trigger Node): New node (#7130)
Github issue / Community forum post (link here to close automatically):

based on https://github.com/joffcom/n8n-nodes-form-trigger

---------

Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-10-17 07:09:30 +03:00
Michael Auerswald e01b9e5ae1
fix(core): Prevent false stalled jobs in queue mode from displaying as errored (#7435)
This is related to an issue with how Bull handles stalled jobs, see
https://github.com/OptimalBits/bull/issues/1415 for reference.

CPU intensive workflows can in certain cases take a long while to finish
up, thereby blocking the thread and causing Bull queue to think the job
has stalled, even though it finished successfully. In these cases the
error handling could then overwrite the successful execution data with
the error message.
2023-10-13 16:07:08 +02:00
Iván Ovejero c6ee1e30c0
refactor(core): Add binary data S3 to telemetry (#7412)
https://n8nio.slack.com/archives/C04B1GZ4T0U/p1697033523039729
2023-10-13 13:16:43 +02:00
Michael Auerswald 45f2ef373e
fix(core): Add check that queue is defined and remove cyclic dependency (#7404)
In a rare edge case an undefined queue could be returned - this should
not happen and now an error is thrown.
Also using the opportunity to remove a cyclic dependency from the Queue.
2023-10-13 11:53:59 +02:00
Omar Ajoue 46977a2aff
fix: Prevent undefined issues when restoring binary data (#7419)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-10-12 16:41:51 +02:00
Iván Ovejero 54e900955a
refactor(core): Move frontend settings to service (no-changelog) (#7396)
Quick follow-up to https://github.com/n8n-io/n8n/pull/7283
2023-10-11 17:12:19 +02:00
Michael Auerswald 2f4d91b2cd
fix(core): Fix ignoring crashed executions without event msgs (#7368)
when the event logs do not contain messages for running executions, the
recovery/crash detection on startup would skip these. this PR fixes
that.
2023-10-11 14:22:43 +02:00
कारतोफ्फेलस्क्रिप्ट™ 6479eb180f
fix(Webhook Node): Backward compatible form-data parsing for non-array files (#7385)
Fixes
https://community.n8n.io/t/possible-bug-0-added-to-end-of-files-sent-via-webhook/31169
2023-10-10 15:19:05 +02:00
Iván Ovejero 0847623f85
feat(core): Switch binary filesystem mode to nested path structure (#7307)
Depends on #7253 | Story:
[PAY-863](https://linear.app/n8n/issue/PAY-863/switch-binary-filesystem-mode-to-nested-path-structure)

This PR introduces `filesystem-v2` to store binary data in the
filesystem in the same format as `s3`.
2023-10-10 10:06:06 +02:00
कारतोफ्फेलस्क्रिप्ट™ c5ee06cc61
refactor(core): Refactor nodes loading (no-changelog) (#7283)
fixes PAY-605
2023-10-09 16:09:23 +02:00
कारतोफ्फेलस्क्रिप्ट™ 789e1e7ed4
fix(core): Add an option to enable postgres ssl with default certs (#6889)
CP-809
2023-10-09 13:55:03 +02:00
Michael Auerswald b8608cee6d
feat(core): Add Job Summary to Worker response (#7360) 2023-10-06 17:52:27 +02:00
Iván Ovejero 34bda535e6
refactor(core): Create controller for binary data (no-changelog) (#7363)
This PR adds a controller for binary data + integration tests.
2023-10-06 16:21:13 +02:00
Michael Auerswald afa683a06f
refactor(core): Have one orchestration service per instance type (#7303)
webhook instances will not listen to either worker or event log messages
on the Redis pub/sub channel
2023-10-06 13:58:11 +02:00
Cornelius Suermann f4d8c9eed5
feat: Improve error msg when attempting to redeem the same activation code multiple times (no-changelog) (#7355) 2023-10-05 17:08:30 +02:00
Iván Ovejero 1a661e6d00
feat(core): Integrate object store as binary data manager (#7253)
Depends on: #7225 | Story:
[PAY-848](https://linear.app/n8n/issue/PAY-848)

This PR integrates the object store service as a new binary data manager
for Enterprise.
2023-10-05 15:25:17 +02:00
Omar Ajoue 5cfd87692f
refactor: Add log messages to debug worker init issues (#7238)
Github issue / Community forum post (link here to close automatically):
2023-10-05 13:37:25 +02:00
कारतोफ्फेलस्क्रिप्ट™ 169175080a
ci: Fix WorkflowHistoryManager tests (no-changelog) (#7356)
[DB Tests](https://github.com/n8n-io/n8n/actions/runs/6418058186)
2023-10-05 13:14:57 +02:00
Tomi Turtiainen 1b4848afcb
fix(core): Fix pruning of non-finished executions (#7333)
This fixes a bug in the pruning (soft-delete). The pruning was a bit too
aggressive, as it also pruned executions that weren't in an end state
yet. This only becomes an issue if there are long-running executions
(e.g. workflow with Wait node) or the prune parameters are set to keep
only a tiny number of executions.
2023-10-04 16:32:05 +03:00
Val 0adc533719
feat: Workflow History pruning and prune time settings (#7343)
Github issue / Community forum post (link here to close automatically):
2023-10-04 13:57:21 +01:00
कारतोफ्फेलस्क्रिप्ट™ 101255d186
ci: Make builds release-channel aware. Add support for scheduled beta builds (#7323)
ADO-1121

---------

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2023-10-03 20:49:04 +02:00
कारतोफ्फेलस्क्रिप्ट™ 00a4b8b0c6
feat(core): Add support for building LLM applications (#7235)
This extracts all core and editor changes from #7246 and #7137, so that
we can get these changes merged first.

ADO-1120

[DB Tests](https://github.com/n8n-io/n8n/actions/runs/6379749011)
[E2E Tests](https://github.com/n8n-io/n8n/actions/runs/6379751480)
[Workflow Tests](https://github.com/n8n-io/n8n/actions/runs/6379752828)

---------

Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-10-02 17:33:43 +02:00
Iván Ovejero 1691223789
fix(core): Account for itemless case on restoring binary data ID (#7305)
https://linear.app/n8n/issue/PAY-862
2023-10-02 09:13:55 +02:00
Csaba Tuncsik d1b6c7fd79
feat(editor): Workflow history [WIP]- create workflow history list component (no-changelog) (#7186)
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-09-29 17:48:36 +02:00
Omar Ajoue ec0379378e
feat(core): Add plan name to telemetry (no-changelog) (#7296)
Github issue / Community forum post (link here to close automatically):
2023-09-29 15:41:32 +02:00
Michael Auerswald 53a7502d20
feat(core): Add secrets provider reload and refactor (#7277)
This PR adds a message for queue mode which triggers an external secrets
provider reload inside the workers if the configuration has changed on
the main instance.

It also refactors some of the message handler code to remove cyclic
dependencies, as well as remove unnecessary duplicate redis clients
inside services (thanks to no more cyclic deps)
2023-09-28 12:57:35 +02:00
Val a80abad3af
fix(core): Change WorkflowHistory nodes/connections columns to be json (#7282)
Github issue / Community forum post (link here to close automatically):
2023-09-28 10:37:33 +02:00
कारतोफ्फेलस्क्रिप्ट™ 0132514f8b
fix(core): Use consistent timezone-aware timestamps in postgres (#6948)
Fixes:
* ENG-51 / N8N-2490
* PAY-397
* #2178
* #2810
* #3855

Supersedes #2813

[DB
Tests](https://github.com/n8n-io/n8n/actions/runs/6000780146/job/16273596338)
2023-09-27 18:44:47 +02:00
कारतोफ्फेलस्क्रिप्ट™ ebce6fe1b0
refactor(core): Skip sending webhook activation errors to Sentry (no-changelog) (#7171) 2023-09-27 16:57:52 +02:00
Val 0083a9e45d
feat(core): Initial workflow history API (#7234)
Github issue / Community forum post (link here to close automatically):
2023-09-27 15:22:39 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5b3121c415
fix(core): Make DNS resolution order configurable (#7272) 2023-09-27 12:19:18 +02:00
Iván Ovejero fa845453bb
feat(core): Introduce object store service (#7225)
Depends on https://github.com/n8n-io/n8n/pull/7220 | Story:
[PAY-840](https://linear.app/n8n/issue/PAY-840/introduce-object-store-service-and-manager-for-binary-data)

This PR introduces an object store service for Enterprise edition. Note
that the service is tested but currently unused - it will be integrated
soon as a binary data manager, and later for execution data.
`amazonaws.com` in the host is temporarily hardcoded until we integrate
the service and test against AWS, Cloudflare and Backblaze, in the next
PR.

This is ready for review - the PR it depends on is approved and waiting
for CI.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-27 09:42:35 +02:00
Iván Ovejero db01164ce1
perf(core): Skip unneeded calls on every pruning cycle (#7260) 2023-09-26 16:53:38 +02:00
Michael Auerswald 4b014286cf
fix(core): Make senderId required for all command messages (#7252)
all commands sent between main instance and workers need to contain a
server id to prevent senders from reacting to their own messages,
causing loops

this PR makes sure all sent messages contain a sender id by default as
part of constructing a sending redis client.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-26 13:58:06 +02:00
Iván Ovejero 77d6e3fc07
refactor(core): Include workflow ID in binary data writes (no-changelog) (#7220)
Depends on: https://github.com/n8n-io/n8n/pull/7195 | Story:
[PAY-837](https://linear.app/n8n/issue/PAY-837/implement-object-store-manager-for-binary-data)

This PR includes `workflowId` in binary data writes so that the S3
manager can support this filepath structure
`/workflows/{workflowId}/executions/{executionId}/binaryData/{binaryFilename}`
to easily delete binary data for workflows. Also all binary data service
and manager methods that take `workflowId` and `executionId` are made
consistent in arg order.

Note: `workflowId` is included in filesystem mode for compatibility with
the common interface, but `workflowId` will remain unused by filesystem
mode until we decide to restructure how this mode stores data.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-25 18:04:52 +02:00
Iván Ovejero 75541e91f2
refactor(core)!: Make getBinaryStream async (#7247)
Story: [PAY-846](https://linear.app/n8n/issue/PAY-846) | Related:
https://github.com/n8n-io/n8n/pull/7225

For the S3 backend for external storage of binary data and execution
data, the `getAsStream` method in the binary data manager interface used
by FS and S3 will need to become async. This is a breaking change for
nodes-base.
2023-09-25 16:59:45 +02:00
Mutasem Aldmour 01e9340621
feat: Add onboarding flow (#7212)
Github issue / Community forum post (link here to close automatically):
2023-09-25 15:49:36 +02:00
Iván Ovejero 33991e92d0
fix(core): Fix missing execution ID in webhook-based workflow producing binary data (#7244)
Story: https://linear.app/n8n/issue/PAY-839

This is a longstanding bug, fixed now so that the S3 backend for binary
data can use execution IDs as part of the filename.

To reproduce:

1. Set up a workflow with a POST Webhook node that accepts binary data.
2. Activate the workflow and call it sending a binary file, e.g. `curl
-X POST -F "file=@/path/to/binary/file/test.jpg"
http://localhost:5678/webhook/uuid`
3. Check `~/.n8n/binaryData`. The binary data and metadata files will be
missing the execution ID, e.g. `11869055-83c4-4493-876a-9092c4708b9b`
instead of `39011869055-83c4-4493-876a-9092c4708b9b`.
2023-09-25 12:30:28 +02:00
Iván Ovejero 6d6e2488c6
refactor(core): Generalize binary data manager interface (no-changelog) (#7164)
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)

This PR: 
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.

Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.

Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-22 17:22:12 +02:00
Val bf74f09d69
feat(core): Add Tournament as the new default expression evaluator (#6964)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-21 13:57:45 +01:00
Ricardo Espinoza 303bc8e71e
fix: Issue enforcing user limits on start plan (#7188) 2023-09-21 05:56:40 -04:00
Iván Ovejero cd08c8e4c6
refactor(core): Implement soft-deletions for executions (#7092)
Based on #7065 | Story: https://linear.app/n8n/issue/PAY-771

n8n on filesystem mode marks binary data to delete on manual execution
deletion, on unsaved execution completion, and on every execution
pruning cycle. We later prune binary data in a separate cycle via these
marker files, based on the configured TTL. In the context of introducing
an S3 client to manage binary data, the filesystem mode's mark-and-prune
setup is too tightly coupled to the general binary data management
client interface.

This PR...
- Ensures the deletion of an execution causes the deletion of any binary
data associated to it. This does away with the need for binary data TTL
and simplifies the filesystem mode's mark-and-prune setup.
- Refactors all execution deletions (including pruning) to cause soft
deletions, hard-deletes soft-deleted executions based on the existing
pruning config, and adjusts execution endpoints to filter out
soft-deleted executions. This reduces DB load, and keeps binary data
around long enough for users to access it when building workflows with
unsaved executions.
- Moves all execution pruning work from an execution lifecycle hook to
`execution.repository.ts`. This keeps related logic in a single place.
- Removes all marking logic from the binary data manager. This
simplifies the interface that the S3 client will meet.
- Adds basic sanity-check tests to pruning logic and execution deletion.

Out of scope:

- Improving existing pruning logic.
- Improving existing execution repository logic.
- Adjusting dir structure for filesystem mode.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-20 15:21:42 +02:00
कारतोफ्फेलस्क्रिप्ट™ e9ce531210
fix(core): Resolve domains to IPv4 first (#7206) 2023-09-19 15:56:38 +02:00
Cornelius Suermann 050ba706d3
fix: Attempt license renewal when n8n starts (no-changelog) (#7204)
This change is needed so that the newly introduced floating licenses
will be attempted to be claimed when n8n starts up.
2023-09-19 12:10:23 +02:00
Michael Auerswald 9f797b96d8
feat(core): Add command to trigger license refresh on workers (#7184)
This PR implements the updated license SDK so that worker and webhook
instances do not auto-renew licenses any more.

Instead, they receive a `reloadLicense` command via the Redis client
that will fetch the updated license after it was saved on the main
instance

This also contains some refactoring with moving redis sub and pub
clients into the event bus directly, to prevent cyclic dependency
issues.
2023-09-17 11:05:54 +02:00
Csaba Tuncsik 240b2f075e
feat(editor): Add Workflow history route and base page (no-changelog) (#7161) 2023-09-15 13:17:04 +02:00
Michael Auerswald fdac2c8572
feat(core): Add rsa option to ssh key generation (#7154)
PR adds a new field to the SourceControlPreferences as well as to the
POST parameters for the `source-control/preferences` and
`source-control/generate-key-pair` endpoints. Both now accept an
optional string parameter `keyGeneratorType` of `'ed25519' | 'rsa'`

Calling the `source-control/generate-key-pair` endpoint with the
parameter set, it will also update the stored preferences accordingly
(so that in the future new keys will use the same method)

By default ed25519 is being used. The default may be changed using a new
environment parameter:

`N8N_SOURCECONTROL_DEFAULT_SSH_KEY_TYPE` which can be `rsa` or `ed25519`

RSA keys are generated with a length of 4096 bytes.
2023-09-14 11:34:51 +02:00
Michael Auerswald 0c6169ee22
refactor(core): Move event and telemetry handling into workers in queue mode (#7138)
# Motivation

In Queue mode, finished executions would cause the main instance to
always pull all execution data from the database, unflatten it and then
use it to send out event log events and telemetry events, as well as
required returns to Respond to Webhook nodes etc.

This could cause OOM errors when the data was large, since it had to be
fully unpacked and transformed on the main instance’s side, using up a
lot of memory (and time).

This PR attempts to limit this behaviour to only happen in those
required cases where the data has to be forwarded to some waiting
webhook, for example.

# Changes

Execution data is only required in cases, where the active execution has
a `postExecutePromise` attached to it. These usually forward the data to
some other endpoint (e.g. a listening webhook connection).

By adding a helper `getPostExecutePromiseCount()`, we can decide that in
cases where there is nothing listening at all, there is no reason to
pull the data on the main instance.

Previously, there would always be postExecutePromises because the
telemetry events were called. Now, these have been moved into the
workers, which have been given the various InternalHooks calls to their
hook function arrays, so they themselves issue these telemetry and event
calls.

This results in all event log messages to now be logged on the worker’s
event log, as well as the worker’s eventbus being the one to send out
the events to destinations. The main event log does…pretty much nothing.

We are not logging executions on the main event log any more, because
this would require all events to be replicated 1:1 from the workers to
the main instance(s) (this IS possible and implemented, see the worker’s
`replicateToRedisEventLogFunction` - but it is not enabled to reduce the
amount of traffic over redis).

Partial events in the main log could confuse the recovery process and
would result in, ironically, the recovery corrupting the execution data
by considering them crashed.

# Refactor

I have also used the opportunity to reduce duplicate code and move some
of the hook functionality into
`packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts`
in preparation for a future full refactor of the hooks
2023-09-14 07:58:15 +02:00
कारतोफ्फेलस्क्रिप्ट™ 34ebffea45
refactor(core): Move workerCommandHandler into worker.ts (no-changelog) (#7160) 2023-09-13 14:51:41 +02:00
कारतोफ्फेलस्क्रिप्ट™ 22edc03cab
fix(core): Ignore missing user-agent on bot check (no-changelog) (#7153) 2023-09-12 19:57:25 +02:00
कारतोफ्फेलस्क्रिप्ट™ 1d1a022def
feat(core): Add an option to enable WAL mode for SQLite (#7118)
https://www.sqlite.org/wal.html
2023-09-07 15:58:48 +02:00
Michael Auerswald 7b49cf2a2c
feat(core): Add commands to workers to respond with current state (#7029)
This PR adds new endpoints to the REST API:
`/orchestration/worker/status` and `/orchestration/worker/id`

Currently these just trigger the return of status / ids from the workers
via the redis back channel, this still needs to be handled and passed
through to the frontend.

It also adds the eventbus to each worker, and triggers a reload of those
eventbus instances when the configuration changes on the main instances.
2023-09-07 14:44:19 +02:00
कारतोफ्फेलस्क्रिप्ट™ 273d0913fe
feat(HTTP Request Node): Determine binary file name from content-disposition headers (#7032)
Fixes: 

https://community.n8n.io/t/http-request-node-read-filename-from-content-disposition-header-when-downloading-files/13453

https://community.n8n.io/t/read-filename-from-content-disposition-header-when-downloading-files/22192
2023-09-06 12:38:37 +02:00
Omar Ajoue 25dc4d7825
feat: Add workflow history repository files (no-changelog) (#7071) 2023-09-06 12:23:40 +02:00
कारतोफ्फेलस्क्रिप्ट™ a910757cc5
fix(core): Add support for in-transit encryption (TLS) on Redis connections (#7047)
Fixes https://community.n8n.io/t/upgrade-from-1-1-1-to-1-2-0/30148
2023-09-05 18:18:50 +02:00
कारतोफ्फेलस्क्रिप्ट™ cc37a5046a
fix(core): Lazy-load express-openapi-validator to reduce initial memory spike (no-changelog) (#7111) 2023-09-05 14:42:14 +02:00
Iván Ovejero 0036a4726c
refactor(core): Move static data handling to workflow service (no-changelog) (#7104) 2023-09-05 13:42:31 +02:00
Michael Auerswald 011ee2e04b
fix(core): Split event bus controller into community and ee (#7107) 2023-09-05 13:32:09 +02:00
Iván Ovejero b7320f5322
refactor(core): Set up ExecutionMetadata service (no-changelog) (#7103)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-05 09:13:30 +02:00
कारतोफ्फेलस्क्रिप्ट™ cffda65b33
refactor(core): Add an option to use simple recovery process by default (#7097) 2023-09-04 19:58:36 +02:00
कारतोफ्फेलस्क्रिप्ट™ ee36f2d20b
refactor(core): Disable slow-query log by default (no-changelog) (#7096)
Fixes:
https://community.n8n.io/t/stop-logging-full-query-for-execution/29732

[Slack
Conversation](https://n8nio.slack.com/archives/C035KBDA917/p1693218994681939)
2023-09-04 17:59:08 +02:00
कारतोफ्फेलस्क्रिप्ट™ f286bd33c1
ci: Fix linting (no-changelog) (#7099) 2023-09-04 17:07:11 +02:00
Iván Ovejero 8a28e98ec8
fix(core): Disallow orphan executions (#7069)
Until https://github.com/n8n-io/n8n/pull/7061 we had an edge case where
a manual unsaved workflow when run creates an orphan execution, i.e. a
saved execution not pointing to any workflow. This execution is only
ever visible to the instance owner (even if triggered by a member), and
is wrongly stored as unfinished and crashed. This PR enforces that the
DB disallows any such executions from making it into the DB.

This is needed also for the S3 client, which will include the
`workflowId` in the path-like filename.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-04 16:57:10 +02:00
Cornelius Suermann e26553f198
feat(core): Add support for floating licenses (#7090)
This PR updates the license-sdk to v2.5.0 which introduces optional
support for floating licenses.
2023-09-04 15:56:20 +02:00
Iván Ovejero fd78021b68
feat(core): Add list query middleware to credentials (#7041) 2023-09-04 15:00:25 +02:00
Iván Ovejero 413e0bccb4
feat(core): Migration for soft deletions for executions (#7088)
Based on https://github.com/n8n-io/n8n/pull/7065

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-09-04 14:34:03 +02:00
Iván Ovejero 442b910ffb
refactor(core): Consolidate CredentialsService.getMany() (no-changelog) (#7028)
Consolidate `CredentialsService.getMany()` in preparation for adding
list query middleware to `GET /credentials`.
2023-09-04 10:37:16 +02:00
Iván Ovejero 008cdcce56
ci: Fix lint to fix build (#7076) 2023-09-01 16:23:23 +02:00
कारतोफ्फेलस्क्रिप्ट™ 97f87ae0fc
refactor(core): Add addColumns and dropColumns to the migrations DSL (no-changelog) (#7073) 2023-09-01 15:19:49 +02:00
Iván Ovejero 51093f649d
refactor: Move community package logic to service (no-changelog) (#6973) 2023-09-01 15:13:19 +02:00
कारतोफ्फेलस्क्रिप्ट™ 2432dcc661
fix(core): Prevent bots from scanning the application (no-changelog) (#7066)
N8N-6878
2023-09-01 13:54:35 +02:00
कारतोफ्फेलस्क्रिप्ट™ a693b29134
ci: Fix prettier auto-formatting (no-changelog) (#7063) 2023-09-01 13:29:31 +02:00
Iván Ovejero fa3d7070b0
refactor(core): Remove unneeded call from CreateWorkflowHistoryTable migration (no-changelog) (#7072)
https://github.com/n8n-io/n8n/pull/7069/files#r1312786460
2023-09-01 12:16:24 +02:00
Iván Ovejero 95cb127dfa
refactor(core): Remove slow LDAP query and unused telemetry event (#7068)
https://linear.app/n8n/issue/PAY-660
2023-09-01 12:05:14 +02:00
Mutasem Aldmour 8a8d4e8bb3
fix: Load remote resources even if expressions in non requried parameters resolve (#6987)
Github issue / Community forum post (link here to close automatically):
2023-08-31 16:40:20 +02:00
Iván Ovejero 8cd4db0ab7
refactor(core): Simplify marking logic in binary data manager (no-changelog) (#7046)
- For a saved execution, we write to disk binary data and metadata.
These two are only ever deleted via `POST /executions/delete`. No marker
file, so untouched by pruning.
- For an unsaved execution, we write to disk binary data, binary data
metadata, and a marker file at `/meta`. We later delete all three during
pruning.
- The third flow is legacy. Currently, if the execution is unsaved, we
actually store it in the DB while running the workflow and immediately
after the workflow is finished during the `onWorkflowPostExecute()` hook
we delete that execution, so the second flow applies. But formerly, we
did not store unsaved executions in the DB ("ephemeral executions") and
so we needed to write a marker file at `/persistMeta` so that, if the
ephemeral execution crashed after the step where binary data was stored,
we had a way to later delete its associated dangling binary data via a
second pruning cycle, and if the ephemeral execution succeeded, then we
immediately cleaned up the marker file at `/persistMeta` during the
`onWorkflowPostExecute()` hook.

This creation and cleanup at `/persistMeta` is still happening, but this
third flow no longer has a purpose, as we now store unsaved executions
in the DB and delete them immediately after. Hence the third flow can be
removed.
2023-08-31 16:02:20 +02:00
कारतोफ्फेलस्क्रिप्ट™ 9d70ce3a15
ci: Fix occasionally failing MFA e2e test (no-changelog) (#7040) 2023-08-29 15:39:10 +02:00
Omar Ajoue a4578d68a5
feat: Create workflow history database migration (no-changelog) (#7031)
Github issue / Community forum post (link here to close automatically):

For the upcoming workflow history feature, we're creating the necessary
database tables.

Also changes the schema for Postgres so the versionId column is now
properly a UUID. The `using` statement prevents losing data, basically
converting the strings to UUIDs.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
2023-08-29 12:01:11 +02:00
Iván Ovejero 53361d1d62
refactor: Move node i18n logic to NodeTypes (no-changelog) (#7035) 2023-08-29 10:46:24 +02:00
Iván Ovejero b716241b42
feat(core): Add filtering, selection and pagination to users (#6994)
https://linear.app/n8n/issue/PAY-646
2023-08-28 16:13:17 +02:00
कारतोफ्फेलस्क्रिप्ट™ 8bdb07d33d
fix(core): Setup websocket keep-live messages (#6866)
fixes #6757
2023-08-28 10:26:27 +02:00
कारतोफ्फेलस्क्रिप्ट™ 596b5695cd
fix(Webhook Node): Fix URL params for webhooks (#6986)
Fixes:

https://community.n8n.io/t/empty-params-variables-in-webhook-node-after-upgrading-to-n8n-1-3-1/29624
2023-08-25 13:28:32 +02:00
Iván Ovejero 87cf1d9c1b
refactor(core): Make controller constructors consistent (no-changelog) (#7015) 2023-08-25 13:23:22 +02:00
Alex Grozav ed927d34b2
feat: External Secrets storage for credentials (#6477)
Github issue / Community forum post (link here to close automatically):

---------

Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-08-25 10:33:46 +02:00
कारतोफ्फेलस्क्रिप्ट™ 72f65dcdd6
refactor(core): Do not import BaseCommand outside src/commands (no-changelog) (#7008) 2023-08-25 00:17:00 +02:00
कारतोफ्फेलस्क्रिप्ट™ 92d4befea6
fix(core): Fix AddMfaColumns migration for sqlite (no-changelog) (#7006)
When ever we have migrations that use `.addColumn` or `.dropColumn`,
typeorm recreates tables for sqlite. so, we need to disable foreign key
enforcement for sqlite, or else data in some tables can get deleted
because of `ON DELETE CASCADE`

[This has happened in the
past](https://github.com/n8n-io/n8n/pull/6739), and we should really
come up with a way to prevent this from happening again.

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
2023-08-24 09:31:37 -04:00
Ricardo Espinoza 2b7ba6fdf1
feat(core): Add MFA (#4767)
https://linear.app/n8n/issue/ADO-947/sync-branch-with-master-and-fix-fe-e2e-tets

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-08-23 22:59:16 -04:00
Iván Ovejero 8c38d85e76
fix(core): Fix import:workflow command (#6996)
Ref: https://github.com/n8n-io/n8n/actions/runs/5946170960

To test:

```sh
./packages/cli/bin/n8n export:workflow --all --output=./all-workflows.json
./packages/cli/bin/n8n import:workflow --input=./all-workflows.json
```
2023-08-23 15:15:14 +02:00
Iván Ovejero 96a9de68a0
refactor(core): Move all user DB access to UserRepository (#6910)
Prep for https://linear.app/n8n/issue/PAY-646
2023-08-22 15:58:05 +02:00
Omar Ajoue 67b88f75f4
feat(core): Add a warning to error workflows that cannot be started due to permission or settings (#6961)
Github issue / Community forum post (link here to close automatically):

This PR aims to address an issue where an Error workflow cannot be
started, either due to insufficient permissions or because its settings
prevent it from being called.

The way of addressing this is by creating a failed execution for the
appointed error workflow stating the error, as can be seen below.

This means the execution itself won't start, as it's prevented before
the execution beings, but we save a "stub" execution to show the error.

![Screenshot 2023-08-17 at 16 17
02](https://github.com/n8n-io/n8n/assets/219272/d8ec0144-13c5-4b11-b91c-a6b440816ccf)
2023-08-22 15:26:33 +02:00
Iván Ovejero 2cfa6d344e
refactor: Consolidate WorkflowService.getMany() (no-changelog) (#6892)
In scope:

- Consolidate `WorkflowService.getMany()`.
- Support non-entity field `ownedBy` for `select`.
- Support `tags` for `filter`.
- Move `addOwnerId` to `OwnershipService`.
- Remove unneeded check for `filter.id`.
- Simplify DTO validation for `filter` and `select`.
- Expand tests for `GET /workflows`.

Workflow list query DTOs:

```
filter → name, active, tags
select → id, name, active, tags, createdAt, updatedAt, versionId, ownedBy
```

Out of scope:

- Migrate `shared_workflow.roleId` and `shared_credential.roleId` to
string IDs.
- Refactor `WorkflowHelpers.getSharedWorkflowIds()`.
2023-08-22 13:19:37 +02:00
Iván Ovejero 9b9b891e68
refactor(core): Move remaining tags logic to service (no-changelog) (#6920)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-08-22 12:24:43 +02:00
Michael Auerswald 7b96820218
fix(core): Add recoveryInProgress flag file (#6962)
Issue: during startup, unfinished executions trigger a recovery process
that, under certain circumstances, can in itself crash the instance
(e.g. by running our of memory), resulting in an infinite recovery loop

This PR aims to change this behaviour by writing a flag file when the
recovery process starts, and removing it when it finishes. In the case
of a crash, this flag will persist and upon the next attempt, the
recovery will instead do the absolute minimal (marking executions as
'crashed'), without attempting any 'crashable' actions.
2023-08-18 17:12:24 +02:00
Iván Ovejero 4fc69b776c
fix(core): Set up OAuth2 cred test (#6960)
https://linear.app/n8n/issue/PAY-728
2023-08-18 16:18:44 +02:00
Michael Auerswald b6a00febbd
fix(core): Replace throw with warning when deactivating a non-active workflow (#6969)
Replaces a throw with a warning message instead, since the failure in
question is not serious enough to warrant stopping the application.
2023-08-18 14:04:49 +02:00
कारतोफ्फेलस्क्रिप्ट™ 9455bcfef5
fix(Webhook Node): Backward compatible form-data parsing for non-array fields (#6967) 2023-08-18 12:34:42 +02:00
Ricardo Espinoza 270946a93b
feat: Remove PostHog event calls (#6915) 2023-08-17 11:39:32 -04:00
कारतोफ्फेलस्क्रिप्ट™ 41c3cc89ca
refactor(core): Delete some duplicate code between ActiveWebhooks and ActiveWorkflowRunner (no-changelog) (#6951) 2023-08-17 17:18:14 +02:00
कारतोफ्फेलस्क्रिप्ट™ 8f8a1de3dd
feat(core): Add support for ready hooks, and credentials overwrite endpoint in workers (#6954) 2023-08-17 15:42:57 +02:00
Iván Ovejero 413570c49d
feat(editor): Show banner for non-production licenses (#6943)
https://linear.app/n8n/issue/PAY-692
2023-08-17 14:00:17 +02:00
Iván Ovejero 7ce5d8fd90
refactor: Upgrade to Prettier 3 (no-changelog) (#6947)
Supersedes https://github.com/n8n-io/n8n/pull/6937

Excluding fixtures and test workflow JSONs to avoid having to update
tests.
2023-08-16 17:13:57 +02:00
कारतोफ्फेलस्क्रिप्ट™ bb76c18073
ci: Fix executeBatch command to fix workflow tests (no-changelog) (#6946)
This broke because [we started returning an explicit `undefined` on
error
descriptions](https://github.com/n8n-io/n8n/pull/6841/files#diff-ea9bff15fe880ac829cc642c7e72a25d2348e331e77fc060cbe11a6effb6aeaeR430)

[Test
run](https://github.com/n8n-io/n8n/actions/runs/5879776481/job/15945235163)
2023-08-16 16:53:17 +02:00
OlegIvaniv fde6ad1e7f
feat(editor): Ask AI in Code node (#6672)
* feat(editor): Ask AI tab and CLi connection

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Remove old getSchema util method

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Increase CSS specificity of the CodeNodeEditor global overrides

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* feat(editor): Magic Connect

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Improve AI controller, load conditionally, UX modal imporvements

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Extract-out AI curl

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Move loading phrases to locale, add support for ask ai experiment

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix build

* adjust communication

* fix: Remove duplicate source control preferences fetching (no-changelog) (#6675)

fix: remove duplicate source control preferences fetching (no-changelog)

* fix(Slack Node): Add UTM params to n8n reference in Slack message (no-changelog) (#6668)

fix(Slack Node): Add UTM params to n8n reference in Slack message

* fix(FileMaker Node): Improve returned error responses (#6585)

* fix(Microsoft Outlook Node): Fix issue with category not correctly applying (#6583)

* feat(Airtable Node): Overhaul (#6200)

* fix(core): Deleting manual executions should defer deleting binary data (#6680)

deleting manual executions should defer deleting binary data

* fix(editor): Add paywall state to non owner users for Variables (#6679)

* fix(editor): Add paywall state to non owner users for Variables

* fix(editor): Add variables view tests

* fix(editor): remove link from paywall state for non owner

* fix(editor): fix displaying logic

* refactor(core): Refactor WorkflowStatistics code (no-changelog) (#6617)

refactor(core): Refactor WorkflowStatistics code

* fix(editor): Hide Execute Node button for unknown nodes (#6684)

* feat: Allow hiding credential params on cloud (#6687)

* fix: Stop n8n from complaining about credentials when saving a new workflow form a template (#6671)

* fix(core): Upgrade semver to address CVE-2022-25883 (#6689)

* fix(core): Upgrade semver to address CVE-2022-25883

[GH Advisory](https://github.com/advisories/GHSA-c2qf-rxjj-qqgw)

* enforce the patched version of semver everywhere in the dev setup

* ci: Fix test checker glob (no changelog) (#6682)

ci: Fix test checker glob

* fix(API): Do not add starting node on workflow creation (#6686)

* fix(API): Do not add starting node on workflow creation

* chore: Remove comment

* fix(core): Filter out workflows that failed to activate on startup (#6676)

* fix(core): Deactivate on init workflow that should not be retried

* fix(core): Filter out workflows with activation errors

* fix(core): Load SAML libraries dynamically (#6690)

load SAML dynamically

* fix(crowd.dev Node): Fix documentation urls for crowd.dev credentials and nodes (#6696)

* feat(Read PDF Node): Replace pdf-parse with pdfjs, and add support for streaming and encrypted PDFs (#6640)

* feat: Allow `eslint-config` to be externally consumable (#6694)

* feat: Allow `eslint-config` to be externally consumable

* refactor: Adjust import styles

* fix(Contentful Node): Fix typo in credential name (no-changelog) (#6692)

* fix(editor): Ensure default credential values are not detected as dirty state (#6677)

* fix(editor): Ensure default credential values are not detected as dirty state

* chore: Remove logging

* refactor: Improve comment

* feat(Google Cloud Storage Node): Use streaming for file uploads (#6462)

fix(Google Cloud Storage Node): Use streaming for file uploads

* fix(editor): Prevent RMC from loading schema if it's already cached (#6695)

* fix(editor): Prevent RMC from loading schema if it's already cached
*  Adding new tests for RMC
* 👕 Fixing lint errors
* 👌 Updating inline loader styling

* fix(API): Fix issue with workflow setting not supporting newer nanoids (#6699)

* ci: Fix test workflows (no-changelog) (#6698)

* ci: Fix test workflows (no-changelog)

We removed `pdf-parse` in #6640, so we need to get these test PDF files from the `test-workflows` repo instead ([which has been updated to include these files](0f6ef1c804))

* remove `\n` from ids and skipList text files

* fix(core): Banner dismissal should also work for users migrating to v1 (no-changelog) (#6700)

* fix(Postgres Node): For select queries, empty result should be be replaced with `{"success":true}` (#6703)

* fix(Postgres Node): For select queries, empty result should be be replaced with `{"success":true}`

*  less checks

---------

Co-authored-by: Michael Kret <michael.k@radency.com>

* feat(editor): Removing `ph-no-capture` class from some elements (#6674)

* feat(editor): Remove `.ph-no-capture` class from some of the fields
* ✔️ Updating test snapshots
*  Redacting expressions preview in credentials form
* 🔧 Disable posthog input masking
* 🚨 Testing PostHog iFrame settings
* Reverting iframe test
*  Hiding API key in PostHog recordings
*  Added tests for redacted values
* ✔️ Updating checkbox snapshots after label component update
* ✔️ Updating test snapshots in editor-ui
* 👕 Fix lint errors

* fix(editor): Remove global link styling in v1 banner (#6705)

* fix: Add missing indices on sqlite (#6673)

* fix: enforce tag name uniqueness on sqlite

* rename migration and add other missing indices

* add tags tests

* test: Move test timeout to `/cli` (no-changelog) (#6712)

* fix(core): Redirect user to previous url after SSO signin (#6710)

redirect user to previous url after SSO signin

* fix(FTP Node): List recursive ignore . and .. to prevent infinite loops (#6707)

ignore . and .. to prevent infinite loop

Co-authored-by: Michael Kret <michael.k@radency.com>

* ci: Fix running e2e tests in dev mode (no-changelog) (#6717)

* fix(Google BigQuery Node): Error description improvement (#6715)

* fix(GitLab Trigger Node): Fix trigger activation 404 error  (#6711)

* fix webhook checkExists not deleting static data

* improve webhook checkExists not deleting static data

* fix(core): Support redis cluster in queue mode (#6708)

* support redis cluster

* cleanup, fix config schema

* set default prefix to bull

* fix(editor): Skip error line highlighting if out of range (#6721)

* fix(AwsS3 Node): Fix issue if bucket name contains a '.' (#6542)

* test(editor): Add canvas actions E2E tests (#6723)

* test(editor): Add canvas actions E2E tests

* test(editor): Open category items in node creator when category dropped on canvas

* test(editor): Have new position counted only once in drag

* test(editor): rename test

* feat(Rundeck Node): Add support for node filters  (#5633)

* fix(Gmail Trigger Node): Early returns in case of no data (#6727)

* fix(core): Use JWT as reset password token (#6714)

* use jwt to reset password

* increase expiration time to 1d

* drop user id query string

* refactor

* use service instead of package in tests

* sqlite migration

* postgres migration

* mysql migration

* remove unused properties

* remove userId from FE

* fix test for users.api

* move migration to the common folder

* move type assertion to the jwt.service

* Add jwt secret as a readonly property

* use signData instead of sign in user.controller

* remove base class

* remove base class

* add tests

* ci: Fix tests on postgres (no-changelog)

* refactor(core): Prevent community packages queries if feature is disabled (#6728)

* feat(core): Add cache service (#6729)

* add cache service

* PR adjustments

* switch to maxSize for memory cache

* Revert "test(editor): Add canvas actions E2E tests" (#6736)

Revert "test(editor): Add canvas actions E2E tests (#6723)"

This reverts commit 052d82b220.

* fix(Postgres Node): Arrays in query replacement fix (#6718)

* fix(Telegram Trigger Node): Add guard to 'include' call on null or undefined (#6730)

* fix(core): Use `exec` in docker images to forward signals correctly (#6732)

* refactor(core): Move webhook DB access to repository (no-changelog) (#6706)

* refactor(core): Move webhook DB access to repository (no-changelog)

* make sure `DataSource` is initialized before it's dependencies

at some point I hope to replace `DataSource` with a custom `DatabaseConnection` service class that can then disconnect and reconnect from DB without having to update all repositories.

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>

* feat: Environments release using source control (#6653)

* initial telemetry setup and adjusted pull return

* quicksave before merge

* feat: add conflicting workflow list to pull modal

* feat: update source control pull modal

* fix: fix linting issue

* feat: add Enter keydown event for submitting source control push modal (no-changelog)

feat: add Enter keydown event for submitting source control push modal

* quicksave

* user workflow table for export

* improve telemetry data

* pull api telemetry

* fix lint

* Copy tweaks.

* remove authorName and authorEmail and pick from user

* rename owners.json to workflow_owners.json

* ignore credential conflicts on pull

* feat: several push/pull flow changes and design update

* pull and push return same data format

* fix: add One last step toast for successful pull

* feat: add up to date pull toast

* fix: add proper Learn more link for push and pull modals

* do not await tracking being sent

* fix import

* fix await

* add more sourcecontrolfile status

* Minor copy tweak for "More info".

* Minor copy tweak for "More info".

* ignore variable_stub conflicts on pull

* ignore whitespace differences

* do not show remote workflows that are not yet created

* fix telemetry

* fix toast when pulling deleted wf

* lint fix

* refactor and make some imports dynamic

* fix variable edit validation

* fix telemetry response

* improve telemetry

* fix unintenional delete commit

* fix status unknown issue

* fix up to date toast

* do not export active state and reapply versionid

* use update instead of upsert

* fix: show all workflows when clicking push to git

* feat: update Up to date pull translation

* fix: update read only env checks

* do not update versionid of only active flag changes

* feat: prevent access to new workflow and templates import when read only env

* feat: send only active state and version if workflow state is not dirty

* fix: Detect when only active state has changed and prevent generation a new version ID

* feat: improve readonly env messages

* make getPreferences public

* fix telemetry issue

* fix: add partial workflow update based on dirty state when changing active state

* update unit tests

* fix: remove unsaved changes check in readOnlyEnv

* fix: disable push to git button when read onyl env

* fix: update readonly toast duration

* fix: fix pinning and title input in protected mode

* initial commit (NOT working)

* working push

* cleanup and implement pull

* fix getstatus

* update import to new method

* var and tag diffs are no conflicts

* only show pull conflict for workflows

* refactor and ignore faulty credentials

* add sanitycheck for missing git folder

* prefer fetch over pull and limit depth to 1

* back to pull...

* fix setting branch on initial connect

* fix test

* remove clean workfolder

* refactor: Remove some unnecessary code

* Fixed links to docs.

* fix getstatus query params

* lint fix

* dialog to show local and remote name on conflict

* only show remote name on conflict

* fix credential expression export

* fix: Broken test

* dont show toast on pull with empty var/tags and refactor

* apply frontend changes from old branch

* fix tag with same name import

* fix buttons shown for non instance owners

* prepare local storage key for removal

* refactor: Change wording on pushing and pulling

* refactor: Change menu item

* test: Fix broken test

* Update packages/cli/src/environments/sourceControl/types/sourceControlPushWorkFolder.ts

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>

---------

Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>

* fix(core): Fix RemoveResetPasswordColumns migration for sqlite (no-changelog) (#6739)

* ci: Update changelog generation to work with node 18

* refactor: Remove webhook from `IDatabaseCollections` (no-changelog) (#6745)

* refactor: Remove webhook from `IDatabaseCollections`

* refactor: Remove also from `collections`

* 🚀 Release 1.1.0 (#6746)

Co-authored-by: netroy <netroy@users.noreply.github.com>

* fix(Lemlist Node): Fix pagination issues with campaigns and activities (#6734)

* ci: Fix linting issues (no-changelog) (#6747)

* fix(core): Allow ignoring SSL issues on generic oauth2 credentials (#6702)

* refactor: Remove all references to the resetPasswordToken field (no-changelog) (#6751)

refactor: remove all references to the resetPasswordToken field (no-changelog)

* refactor(core): Use mixins to delete redundant code between Entity classes (no-changelog) (#6616)

* db entities don't need an ID before they are inserted

* don't define constructors on entity classes, use repository.create instead

* use mixins to reduce duplicate code in db entity classes

* fix: Display source control buttons properly (#6756)

* feat(editor): Migrate Design System and Editor UI to Vue 3 (#6476)

* feat: remove vue-fragment (no-changelog)

* feat: partial design-system migration

* feat: migrate info-accordion and info-tip components

* feat: migrate several components to vue 3

* feat: migrated several components

* feat: migrate several components

* feat: migrate several components

* feat: migrate several components

* feat: re-exported all design system components

* fix: fix design for popper components

* fix: editor kind of working, lots of issues to fix

* fix: fix several vue 3 migration issues

* fix: replace @change with @update:modelValue in several places

* fix: fix translation linking

* fix: fix inline-edit input

* fix: fix ndv and dialog design

* fix: update parameter input event bindings

* fix: rename deprecated lifecycle methods

* fix: fix json view mapping

* build: update lock file

* fix(editor): revisit last conflict with master and fix issues

* fix(editor): revisit last conflict with master and fix issues

* fix: fix expression editor bug causing code mirror to no longer be reactive

* fix: fix resource locator bug

* fix: fix vue-agile integration

* fix: remove global import for vue-agile

* fix: replace element-plus buttons with n8n-buttons everywhere

* fix(editor): Fix various element-plus styles (#6571)

* fix(editor): Fix various element-plus styles

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Remove debugging code

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Address PR comments

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): Fix loading in production mode [Vue 3] (#6578)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): First round of e2e tests fixes with Vue 3 (#6579)

* fix(editor): Fix broken smoke and workflow list e2e tests
* ✔️ Fix failing canvas action tests. Updating some selectors used in credentials and workflow tests

* feat: add vue 3 eslint rules and fix issues

* fix: fix tags-dropdown

* fix: fix white-space issues caused by i18n-t

* fix: rename non-generic click events

* fix: fix search in resources list layout

* fix: fix datatable paginator

* fix: fix popper select caret and dropdown size

* fix: add width to action-dropdown

* fix: fix workflow settings icon not being hidden

* fix: refactor newly added code

* fix: fix merge issue

* fix: fix ndv credentials watcher

* fix: fix workflow saving and grabber notch

* fix: fix nodes list panel transition

* fix: fix node title visibility

* fix: fix data unpinning

* fix: fix value access

* fix: show  input panel only if trigger panel enabled or not trigger node

* fix: fix tags dropdown and executions status spcing

* fix(editor): Prevent execution list to load back when leaving the route (#6697)

fix(editor): prevent execution list to load back when leaving the route

* fix: fix drawer visibility

* fix: fix expression toggle padding

* fix: fix expressions editor styling

* chore: prepare for testing

* fix: fix styling for el-button without patching

* test: fix unit tests in design-system

* test: fix most unit tests

* fix: remove import cycle.

* fix: fix personalization modal tests

* fix further resource mapper test adjustments

* fix: fix multiple tests and n8n-route attr duplication

* fix: fix source control tets

* fix: fixed remaining unit tests

* fix: fix workflows and credentials e2e tests

* fix: fix localizeNodeNames

* fix: update ndv e2e tests

* fix: fix popper left placement arrow

* fix: fix 5-ndv e2e tests

* fix: fix 6-code-node e2e tests

* fix(editor): Drop click outside directive from NodeCreator (#6716)

* fix(editor): Drop click outside directive from NodeCreator

* fix(editor): make sure mouseup outside is unbound at least before the component is unmounted

* fix: fix 10-settings-log-streaming e2e tests

* fix: fix node redrawing

* fix: fix tooltip buttons styling

* fix: fix varous e2e suites

* fix: fix 15-scheduler-node e2e suite

* fix: fix route watcher

* fix: fixed param name update and credential edit

* feat: update event names

* refactor: Remove deprecated `$data` (#6576)

Co-authored-by: Alex Grozav <alex@grozav.com>

* fix: fix 17-sharing e2e suite

* fix: fix tags dropdown

* fix: fix tags manager

* fix(editor): move :deep selectors to a separate scoped style block

* fix: fix sticky component and inline text edit

* fix: update e2e tests

* fix: remove button override references

* fix(editor): Adjust spacing in templates for Vue 3 (#6744)

* fix(editor): Adjust spacing in templates

* fix: Undo unneeded change

* fix: Undo unneeded change

* fix(editor): Adjust NDV height for Vue 3 (#6742)

fix(editor): Adjust NDV height

* fix(editor): Restore collapsed sidebar items for Vue 3 (#6743)

fix(editor): Restore collapsed sidebar items

* fix: fix linting issues

* fix: fix design-system deps

* fix: post-merge fixes

* fix: update tests

* fix: increase timeout for executionslist tets

* chore: fix linting issue

* fix: fix 14-mapping e2e tests in ci

* fix: re-enable tests

* fix: fix workflow duplication e2e tests after tags update

* fix(editor): Change component prop to be typed

* fix: fix tags dropdown in duplicate wf modal

* fix: fix focus behaviour in tags selector

* fix: fix tag creation

* fix: fix log streaming e2e race condition

* fix(editor): Fix Vue 3 linting issues (#6748)

* fix(editor): Fix Vue 3 linting issues

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix MainSidebar linter issues

* revert pnpm lock

* update pnpm lock file

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>

* fix(editor): Some css fixes for vue3 branch (#6749)

*  Fixing filter button height

*  Update input modal button position

*  Updating tags styling

*  Fix event logging settings spacing

* 👕 Fixing lint errors

* fix: fix linting issues

* Revert to `// eslint-disable-next-line @typescript-eslint/no-misused-promises` disabling of mixins init

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix: fix css issue

* fix(editor): Lint fix

* fix(editor): Fix settings initialisation (#6750)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix: fix initial settings loading

* fix: replace realClick with click force

* fix: fix randomly failing mapping e2e tests

* fix(editor): Fix menu item event handling

* fix: fix resource filters dropdown events (#6752)

* fix: fix resource filters dropdown events

* fix: remove teleported:false

* fix: fix event selection event naming (#6753)

* fix: removed console.log (#6754)

* fix: rever await nextTick changes

* fix: redo linting changes

* fix(editor): Redraw node connections if adding more than one node to canvas (#6755)

* fix(editor): Redraw node connections if adding more than one node to canvas

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Update position before connection two nodes

* Lint fix

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>

* fix(editor): Fix `ResourceMapper` unit tests (#6758)

* ✔️ Fix matching columns test

* ✔️ Fix multiple matching columns test

* ✔️ Removing `skip` from the last test

* fix: Allow pasting a big workflow (#6760)

* fix: pasting a big workflow

* chore: update comment

* refactor: move try/catch to function

* refactor: move try/catch to function

* fix(editor): Fix modal layer width

* fix: fix position changes

* fix: undo it.only

* fix: make undo/redo multiple steps more verbose

* fix: Fix value survey styles (#6764)

* fix: fix value survey styles

* fix: lint

* Revert "fix: lint"

72869c431f

* fix: lint

* fix(editor): Fix collapsed sub menu

* fix: Fix drawer animation (#6767)

fix: drawer animation

* fix(editor): Fix source control buttons (#6769)

* fix(editor): Fix App loading & auth  (#6768)

* fix(editor): Fix App loading & auth

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Await promises

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Fix eslint error

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: OlegIvaniv <me@olegivaniv.com>
Co-authored-by: Milorad FIlipović <milorad@n8n.io>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Mutasem Aldmour <4711238+mutdmour@users.noreply.github.com>

* perf(editor): Memoize locale translate calls during actions generation (#6773)

performance(editor): Memoize locale translate calls during actions generation

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): Close tags dropdown when modal is opened (#6766)

* feat: remove vue-fragment (no-changelog)

* feat: partial design-system migration

* feat: migrate info-accordion and info-tip components

* feat: migrate several components to vue 3

* feat: migrated several components

* feat: migrate several components

* feat: migrate several components

* feat: migrate several components

* feat: re-exported all design system components

* fix: fix design for popper components

* fix: editor kind of working, lots of issues to fix

* fix: fix several vue 3 migration issues

* fix: replace @change with @update:modelValue in several places

* fix: fix translation linking

* fix: fix inline-edit input

* fix: fix ndv and dialog design

* fix: update parameter input event bindings

* fix: rename deprecated lifecycle methods

* fix: fix json view mapping

* build: update lock file

* fix(editor): revisit last conflict with master and fix issues

* fix(editor): revisit last conflict with master and fix issues

* fix: fix expression editor bug causing code mirror to no longer be reactive

* fix: fix resource locator bug

* fix: fix vue-agile integration

* fix: remove global import for vue-agile

* fix: replace element-plus buttons with n8n-buttons everywhere

* fix(editor): Fix various element-plus styles (#6571)

* fix(editor): Fix various element-plus styles

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Remove debugging code

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Address PR comments

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): Fix loading in production mode [Vue 3] (#6578)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): First round of e2e tests fixes with Vue 3 (#6579)

* fix(editor): Fix broken smoke and workflow list e2e tests
* ✔️ Fix failing canvas action tests. Updating some selectors used in credentials and workflow tests

* feat: add vue 3 eslint rules and fix issues

* fix: fix tags-dropdown

* fix: fix white-space issues caused by i18n-t

* fix: rename non-generic click events

* fix: fix search in resources list layout

* fix: fix datatable paginator

* fix: fix popper select caret and dropdown size

* fix: add width to action-dropdown

* fix: fix workflow settings icon not being hidden

* fix: refactor newly added code

* fix: fix merge issue

* fix: fix ndv credentials watcher

* fix: fix workflow saving and grabber notch

* fix: fix nodes list panel transition

* fix: fix node title visibility

* fix: fix data unpinning

* fix: fix value access

* fix: show  input panel only if trigger panel enabled or not trigger node

* fix: fix tags dropdown and executions status spcing

* fix(editor): Prevent execution list to load back when leaving the route (#6697)

fix(editor): prevent execution list to load back when leaving the route

* fix: fix drawer visibility

* fix: fix expression toggle padding

* fix: fix expressions editor styling

* chore: prepare for testing

* fix: fix styling for el-button without patching

* test: fix unit tests in design-system

* test: fix most unit tests

* fix: remove import cycle.

* fix: fix personalization modal tests

* fix further resource mapper test adjustments

* fix: fix multiple tests and n8n-route attr duplication

* fix: fix source control tets

* fix: fixed remaining unit tests

* fix: fix workflows and credentials e2e tests

* fix: fix localizeNodeNames

* fix: update ndv e2e tests

* fix: fix popper left placement arrow

* fix: fix 5-ndv e2e tests

* fix: fix 6-code-node e2e tests

* fix(editor): Drop click outside directive from NodeCreator (#6716)

* fix(editor): Drop click outside directive from NodeCreator

* fix(editor): make sure mouseup outside is unbound at least before the component is unmounted

* fix: fix 10-settings-log-streaming e2e tests

* fix: fix node redrawing

* fix: fix tooltip buttons styling

* fix: fix varous e2e suites

* fix: fix 15-scheduler-node e2e suite

* fix: fix route watcher

* fix: fixed param name update and credential edit

* feat: update event names

* refactor: Remove deprecated `$data` (#6576)

Co-authored-by: Alex Grozav <alex@grozav.com>

* fix: fix 17-sharing e2e suite

* fix: fix tags dropdown

* fix: fix tags manager

* fix(editor): move :deep selectors to a separate scoped style block

* fix: fix sticky component and inline text edit

* fix: update e2e tests

* fix: remove button override references

* fix(editor): Adjust spacing in templates for Vue 3 (#6744)

* fix(editor): Adjust spacing in templates

* fix: Undo unneeded change

* fix: Undo unneeded change

* fix(editor): Adjust NDV height for Vue 3 (#6742)

fix(editor): Adjust NDV height

* fix(editor): Restore collapsed sidebar items for Vue 3 (#6743)

fix(editor): Restore collapsed sidebar items

* fix: fix linting issues

* fix: fix design-system deps

* fix: post-merge fixes

* fix: update tests

* fix: increase timeout for executionslist tets

* chore: fix linting issue

* fix: fix 14-mapping e2e tests in ci

* fix: re-enable tests

* fix: fix workflow duplication e2e tests after tags update

* fix(editor): Change component prop to be typed

* fix: fix tags dropdown in duplicate wf modal

* fix: fix focus behaviour in tags selector

* fix: fix tag creation

* fix: fix log streaming e2e race condition

* fix(editor): Fix Vue 3 linting issues (#6748)

* fix(editor): Fix Vue 3 linting issues

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix MainSidebar linter issues

* revert pnpm lock

* update pnpm lock file

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>

* fix(editor): Some css fixes for vue3 branch (#6749)

*  Fixing filter button height

*  Update input modal button position

*  Updating tags styling

*  Fix event logging settings spacing

* 👕 Fixing lint errors

* fix: fix linting issues

* Revert to `// eslint-disable-next-line @typescript-eslint/no-misused-promises` disabling of mixins init

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix: fix css issue

* fix(editor): Lint fix

* fix(editor): Fix settings initialisation (#6750)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix: fix initial settings loading

* fix: replace realClick with click force

* fix: fix randomly failing mapping e2e tests

* fix(editor): Fix menu item event handling

* fix: fix resource filters dropdown events (#6752)

* fix: fix resource filters dropdown events

* fix: remove teleported:false

* fix: fix event selection event naming (#6753)

* fix: removed console.log (#6754)

* fix: rever await nextTick changes

* fix: redo linting changes

* fix(editor): Redraw node connections if adding more than one node to canvas (#6755)

* fix(editor): Redraw node connections if adding more than one node to canvas

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Update position before connection two nodes

* Lint fix

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>

* fix(editor): Fix `ResourceMapper` unit tests (#6758)

* ✔️ Fix matching columns test

* ✔️ Fix multiple matching columns test

* ✔️ Removing `skip` from the last test

* fix: Allow pasting a big workflow (#6760)

* fix: pasting a big workflow

* chore: update comment

* refactor: move try/catch to function

* refactor: move try/catch to function

* fix(editor): Fix modal layer width

* fix: fix position changes

* fix: undo it.only

* fix: make undo/redo multiple steps more verbose

* fix: Fix value survey styles (#6764)

* fix: fix value survey styles

* fix: lint

* Revert "fix: lint"

72869c431f

* fix: lint

* fix(editor): Close tags dropdown when modal is opened

* ✔️ Updating tag selectors in e2e tests

* ✔️ Using tab to blur dropdown after adding tags

* ✔️ Clicking on the New Tab button instead of the tags dropdown to open it

* Reverting merge changes added by mistake

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: OlegIvaniv <me@olegivaniv.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Mutasem Aldmour <4711238+mutdmour@users.noreply.github.com>

* fix: Show NodeIcon tooltips by removing pointer-events: none (#6777)

fix: show NodeIcon tooltips by removing pointer-events: none

* fix: Respect set modal widths (#6771)

* feat: remove vue-fragment (no-changelog)

* feat: partial design-system migration

* feat: migrate info-accordion and info-tip components

* feat: migrate several components to vue 3

* feat: migrated several components

* feat: migrate several components

* feat: migrate several components

* feat: migrate several components

* feat: re-exported all design system components

* fix: fix design for popper components

* fix: editor kind of working, lots of issues to fix

* fix: fix several vue 3 migration issues

* fix: replace @change with @update:modelValue in several places

* fix: fix translation linking

* fix: fix inline-edit input

* fix: fix ndv and dialog design

* fix: update parameter input event bindings

* fix: rename deprecated lifecycle methods

* fix: fix json view mapping

* build: update lock file

* fix(editor): revisit last conflict with master and fix issues

* fix(editor): revisit last conflict with master and fix issues

* fix: fix expression editor bug causing code mirror to no longer be reactive

* fix: fix resource locator bug

* fix: fix vue-agile integration

* fix: remove global import for vue-agile

* fix: replace element-plus buttons with n8n-buttons everywhere

* fix(editor): Fix various element-plus styles (#6571)

* fix(editor): Fix various element-plus styles

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Remove debugging code

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Address PR comments

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): Fix loading in production mode [Vue 3] (#6578)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): First round of e2e tests fixes with Vue 3 (#6579)

* fix(editor): Fix broken smoke and workflow list e2e tests
* ✔️ Fix failing canvas action tests. Updating some selectors used in credentials and workflow tests

* feat: add vue 3 eslint rules and fix issues

* fix: fix tags-dropdown

* fix: fix white-space issues caused by i18n-t

* fix: rename non-generic click events

* fix: fix search in resources list layout

* fix: fix datatable paginator

* fix: fix popper select caret and dropdown size

* fix: add width to action-dropdown

* fix: fix workflow settings icon not being hidden

* fix: refactor newly added code

* fix: fix merge issue

* fix: fix ndv credentials watcher

* fix: fix workflow saving and grabber notch

* fix: fix nodes list panel transition

* fix: fix node title visibility

* fix: fix data unpinning

* fix: fix value access

* fix: show  input panel only if trigger panel enabled or not trigger node

* fix: fix tags dropdown and executions status spcing

* fix(editor): Prevent execution list to load back when leaving the route (#6697)

fix(editor): prevent execution list to load back when leaving the route

* fix: fix drawer visibility

* fix: fix expression toggle padding

* fix: fix expressions editor styling

* chore: prepare for testing

* fix: fix styling for el-button without patching

* test: fix unit tests in design-system

* test: fix most unit tests

* fix: remove import cycle.

* fix: fix personalization modal tests

* fix further resource mapper test adjustments

* fix: fix multiple tests and n8n-route attr duplication

* fix: fix source control tets

* fix: fixed remaining unit tests

* fix: fix workflows and credentials e2e tests

* fix: fix localizeNodeNames

* fix: update ndv e2e tests

* fix: fix popper left placement arrow

* fix: fix 5-ndv e2e tests

* fix: fix 6-code-node e2e tests

* fix(editor): Drop click outside directive from NodeCreator (#6716)

* fix(editor): Drop click outside directive from NodeCreator

* fix(editor): make sure mouseup outside is unbound at least before the component is unmounted

* fix: fix 10-settings-log-streaming e2e tests

* fix: fix node redrawing

* fix: fix tooltip buttons styling

* fix: fix varous e2e suites

* fix: fix 15-scheduler-node e2e suite

* fix: fix route watcher

* fix: fixed param name update and credential edit

* feat: update event names

* refactor: Remove deprecated `$data` (#6576)

Co-authored-by: Alex Grozav <alex@grozav.com>

* fix: fix 17-sharing e2e suite

* fix: fix tags dropdown

* fix: fix tags manager

* fix(editor): move :deep selectors to a separate scoped style block

* fix: fix sticky component and inline text edit

* fix: update e2e tests

* fix: remove button override references

* fix(editor): Adjust spacing in templates for Vue 3 (#6744)

* fix(editor): Adjust spacing in templates

* fix: Undo unneeded change

* fix: Undo unneeded change

* fix(editor): Adjust NDV height for Vue 3 (#6742)

fix(editor): Adjust NDV height

* fix(editor): Restore collapsed sidebar items for Vue 3 (#6743)

fix(editor): Restore collapsed sidebar items

* fix: fix linting issues

* fix: fix design-system deps

* fix: post-merge fixes

* fix: update tests

* fix: increase timeout for executionslist tets

* chore: fix linting issue

* fix: fix 14-mapping e2e tests in ci

* fix: re-enable tests

* fix: fix workflow duplication e2e tests after tags update

* fix(editor): Change component prop to be typed

* fix: fix tags dropdown in duplicate wf modal

* fix: fix focus behaviour in tags selector

* fix: fix tag creation

* fix: fix log streaming e2e race condition

* fix(editor): Fix Vue 3 linting issues (#6748)

* fix(editor): Fix Vue 3 linting issues

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix MainSidebar linter issues

* revert pnpm lock

* update pnpm lock file

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>

* fix(editor): Some css fixes for vue3 branch (#6749)

*  Fixing filter button height

*  Update input modal button position

*  Updating tags styling

*  Fix event logging settings spacing

* 👕 Fixing lint errors

* fix: fix linting issues

* Revert to `// eslint-disable-next-line @typescript-eslint/no-misused-promises` disabling of mixins init

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix: fix css issue

* fix(editor): Lint fix

* fix(editor): Fix settings initialisation (#6750)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix: fix initial settings loading

* fix: replace realClick with click force

* fix: fix randomly failing mapping e2e tests

* fix(editor): Fix menu item event handling

* fix: fix resource filters dropdown events (#6752)

* fix: fix resource filters dropdown events

* fix: remove teleported:false

* fix: fix event selection event naming (#6753)

* fix: removed console.log (#6754)

* fix: rever await nextTick changes

* fix: redo linting changes

* fix(editor): Redraw node connections if adding more than one node to canvas (#6755)

* fix(editor): Redraw node connections if adding more than one node to canvas

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Update position before connection two nodes

* Lint fix

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>

* fix(editor): Fix `ResourceMapper` unit tests (#6758)

* ✔️ Fix matching columns test

* ✔️ Fix multiple matching columns test

* ✔️ Removing `skip` from the last test

* fix: Allow pasting a big workflow (#6760)

* fix: pasting a big workflow

* chore: update comment

* refactor: move try/catch to function

* refactor: move try/catch to function

* fix(editor): Fix modal layer width

* fix: fix position changes

* fix: undo it.only

* fix: make undo/redo multiple steps more verbose

* fix: Fix value survey styles (#6764)

* fix: fix value survey styles

* fix: lint

* Revert "fix: lint"

72869c431f

* fix: lint

* fix(editor): Fix collapsed sub menu

* fix: Fix drawer animation (#6767)

fix: drawer animation

* fix(editor): Fix source control buttons (#6769)

* fix: Respect modal width

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: OlegIvaniv <me@olegivaniv.com>
Co-authored-by: Milorad FIlipović <milorad@n8n.io>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>

* fix(editor): Fix tooltip opening delay prop name (#6776)

fix(editor): fix tooltip opening delay prop name

* fix(editor): Fix collapsed sub menu elements (#6778)

* fix: Remove number input arrows (no-changelog) (#6782)

fix: remove number input arrows

* ci: Update most of the dev tooling (no-changelog) (#6780)

* fix(TheHive Node): Treat  `ApiKey` as a secret (#6786)

* test(editor): Prevent node view unload by default in e2e run (#6787)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): Resolve vue 3 related console-warnings (#6779)

* fix(editor): Resolve vue 3 related console-warnings

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Use span as component wrapper instead of div

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Wrap popover component in span

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(editor): Vue3 - Fix modal positioning and multi-select tag sizing (#6783)

*  Updating modals positioning within the overlay
* 💄 Implemented multi-select variant with small tabs
* ✔️ Removing password link clicks while modal is open in e2e tests
* Set generous timeout for $paramter resolve
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
---------
Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>

* ci: Fix linting issues (no-changelog) (#6788)

* ci: Fix linting (no-changelog)

* lintfix for nodes-base as well

* fix(editor): Fix code node highlight error (#6791)

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* feat(core): Credentials for popular SecOps services, Part 1 (#6775)

* refactor: Clear unused ESLint directives from BE packages (no-changelog) (#6798)

* refactor(core): Cache workflow ownership (#6738)

* refactor: Set up ownership service

* refactor: Specify cache keys and values

* refactor: Replace util with service calls

* test: Mock service in tests

* refactor: Use dependency injection

* test: Write tests

* refactor: Apply feedback from Omar and Micha

* test: Fix tests

* test: Fix missing spot

* refactor: Return user entity from cache

* refactor: More dependency injection!

* fix(editor): Prevent text edit dialog from re-opening in same tick (#6781)

* fix: prevent reopenning textedit dialog in same tick

* fix: add same logic for code edit dialog

* fix: remove stop modifier

* fix: blur input field when closing modal, removing default element-plus behaviour

* test(editor): Do not chain invoke calls after assertions in 24-ndv-paired-item e2e spec (no-changelog) (#6800)

* test(editor): Do not chaing invoke calls after assertions in 24-ndv-paired-item e2e spec

* Do not chaing realHover after assertion

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Remove .only

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix(Todoist Node): Fix issue with section id being ignored (#6799)

* test(editor): Add canvas actions E2E tests (#6723) (#6790)

* test(editor): Add canvas actions E2E tests (#6723)

* test(editor): Add canvas actions E2E tests

* test(editor): Open category items in node creator when category dropped on canvas

* test(editor): Have new position counted only once in drag

* test(editor): rename test

(cherry picked from commit 052d82b220)

* test: fix drag positioning

* fix(core): Add missing primary key on the `execution_data` table on postgres (#6797)

* fix: Review fixes

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* fix: Fin locales

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Fix merging errors

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Map erros based on statusCode

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Fix code replacing

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Fix code formatting

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Address review points

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Optionally access total_tokens

* Clean-up Ask AI modal

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Store prompt in sessionStorage

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Improve schema generation, only get parent nodes

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Send error messages to telemetry, aske before switching tabs

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Add locale

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Post-merge cleanup

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Move Ask AI into separate folder

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Lint fix

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Constants lint fix

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Add Ask AI e2e tests and fix linting issues

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Move CircleLoader to design-lib

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Replace circle-lodaer and move el-tabs styles to n8n theme

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Fix placeholder & e2e tests

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

* Remove old CircleLoader

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>

---------

Signed-off-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: ricardo <ricardoespinoza105@gmail.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Romain Dunand <romain@1-more-thing.com>
Co-authored-by: Jon <jonathan.bennetts@gmail.com>
Co-authored-by: Michael Kret <88898367+michael-radency@users.noreply.github.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Michael Auerswald <michael.auerswald@gmail.com>
Co-authored-by: Milorad FIlipović <milorad@n8n.io>
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Val <68596159+valya@users.noreply.github.com>
Co-authored-by: Marcus <56945030+maspio@users.noreply.github.com>
Co-authored-by: Jordan Hall <Jordan@libertyware.co.uk>
Co-authored-by: qg-horie <36725144+qg-horie@users.noreply.github.com>
Co-authored-by: Ricardo Espinoza <ricardo@n8n.io>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: Ali Afsharzadeh <afsharzadeh8@gmail.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Mutasem Aldmour <4711238+mutdmour@users.noreply.github.com>
2023-08-16 13:08:10 +02:00
Cornelius Suermann 198a977f57
feat(core): Show a banner when running a non-prod license (no-changelog) (#6849)
* add feat:showNonProdBanner

* fix linting

* fix linting
2023-08-16 10:05:03 +02:00
Xavier Calland eead6d49f2
feat: Add support for not requiring SMTP auth with user management (#3742) 2023-08-15 09:55:58 +01:00
कारतोफ्फेलस्क्रिप्ट™ 96ff1f847d
fix(HTTP Request Node): Improve error handling for TCP socket errors when Continue On Fail is enabled (#6925) 2023-08-14 17:26:40 +02:00
कारतोफ्फेलस्क्रिप्ट™ a82107fb05
fix(Respond to Webhook Node): Return headers in response (#6921) 2023-08-14 12:38:17 +02:00
Omar Ajoue 139e08a549
fix: Add only necessary information to executions (no-changelog) (#6916)
* fix: Add only necessary information to executions

* Fix lint

* fix: lint issue

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2023-08-14 09:44:04 +02:00
कारतोफ्फेलस्क्रिप्ट™ 10c15874b2
fix(core): Restore body parsing for all content-types for non webhook routes (no-changelog)(#6911)
Changes in https://github.com/n8n-io/n8n/pull/6394 removed xml body parsing for all non-webhook routes. This broken SAML endpoints as they need the XML body parser to function correctly.
2023-08-11 09:18:33 +02:00
Michael Kret 7f0db60f15
refactor(core): Abstract away duplication when finding first pinned trigger (no-changelog) (#5927) 2023-08-10 15:06:16 +03:00
कारतोफ्फेलस्क्रिप्ट™ f3fce48155
fix(core): Fix fetching of EE executions (#6901)
fix(core): fix fetching of EE executions
2023-08-10 10:38:22 +02:00
कारतोफ्फेलस्क्रिप्ट™ 718e61354d
fix(core): Update frontend urls when using the --tunnel option (#6898)
fix(core): update frontend urls when using the `--tunnel` option
2023-08-09 19:07:45 +02:00
कारतोफ्फेलस्क्रिप्ट™ cfeb322b3b
fix(core): Don't let bull override the default redis config (#6897) 2023-08-09 18:10:58 +02:00
Omar Ajoue 3d2f4406d7
feat(core): Debug in Editor flag (no-changelog) (#6888)
* feat(cli): Debug in Editor flag (no-changelog)

* fix: Lint issue

* chore: Use correct lock version

* fix: Remove dependency cycle
2023-08-09 16:38:17 +02:00
कारतोफ्फेलस्क्रिप्ट™ 6553d92c7c
fix(core): Log crash causes to console when sentry is disabled (#6890) 2023-08-09 13:29:57 +02:00
Iván Ovejero dceff675ec
perf(core): Add filtering and pagination to GET /workflows (#6845)
* Initial setup

* Specify max paginated items

* Simplify

* Add tests

* Add more tests

* Add migrations

* Add top-level property

* Add field selection

* Cleanup

* Rename `total` to `count`

* More cleanup

* Move query logic into `WorkflowRepository`

* Create `AbstractRepository`

* Cleanup

* Fix name

* Remove leftover comments

* Replace reference

* Add default for `rawSkip`

* Remove unneeded typing

* Switch to `class-validator`

* Simplify

* Simplify

* Type as optional

* Make typing more accurate

* Fix lint

* Use `getOwnPropertyNames`

* Use DSL

* Set schema at repo level

* Cleanup

* Remove comment

* Refactor repository methods to middleware

* Add middleware tests

* Remove old test files

* Remove generic experiment

* Reuse `reportError`

* Remove unused type

* Cleanup

* Improve wording

* Reduce diff

* Add missing mw

* Use `Container.get`

* Adjust lint rule

* Reorganize into subdir

* Remove unused directive

* Remove nodes

* Silly mistake

* Validate take

* refactor(core): Adjust index handling in new migrations DSL (no-changelog) (#6876)

* refactor(core): Adjust index handling in new migrations DSL (no-changelog)

* Account for custom index name

* Also for dropping

* Fix `select` issue with `relations`

* Tighten validation

* Ensure `ownerId` is not added when specifying `select`
2023-08-09 12:30:02 +02:00
Iván Ovejero 6796d9e5d6
fix(core): Add sharing data to workflows in EE executions (#6872)
* fix(core): Add sharing data to workflows in EE executions

* Address feedback
2023-08-08 14:28:05 +02:00
Iván Ovejero 11440bfd3c
refactor(core): Move tag collection into repository (no-changelog) (#6860)
* refactor(core): Move tag collection into repository

* Fix tests

* Address feedback

* Fix missing spot
2023-08-08 14:08:56 +02:00
Michael Auerswald 6499f42481
feat(core): Add unique id to instances (#6863)
* add serverId and workerId

* adjust id format

* fix logger init / test

* update implementation
2023-08-07 17:03:21 +02:00
Michael Auerswald adcf5a96e8
feat(core): Add metrics option to cache (#6846)
* add metrics to cache

* use events for metrics

* pr comments / broken test

* lint fix

* update the test

* improve tests

* Update packages/cli/src/config/schema.ts

* disable flaky test

* lint fix

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
2023-08-04 20:51:07 +02:00
Michael Auerswald fdfc6c5a92
fix(core): Handle empty keys in cache service (no-changelog) (#6854)
* fix handle empty keys in cache service

* add test

* add cache mock test

* add simpler mocking, and add tests for all the updated methods

* don't use RedisStore specifically in the mock

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-08-04 19:44:41 +02:00
Michael Kret 5ab30fdd95
fix(core): OAuth1 authentication fix for Clever Cloud API (#6847) 2023-08-04 19:49:00 +03:00
कारतोफ्फेलस्क्रिप्ट™ 75be1a9c0d
feat(core): Create a dsl for writing db agnostic migrations (#6853) 2023-08-04 17:49:02 +02:00
Jan Oberhauser a8bfb46183 fix(core): Fix WebSocket close codes 2023-08-04 16:40:03 +02:00
Omar Ajoue 63ffe67dad
feat(core): Add workflow history initial setup (no-changelog) (#6852)
* feat: Add workflow history initial setup

* fix: Add feature to missing controller
2023-08-04 12:27:06 +02:00
Iván Ovejero 0511458d41
perf(core): Cache webhooks (#6825)
* refactor: Initial setup

* Refactor for clarity

* Comments to clarify

* More replacements

* Simplify with `fullPath`

* Fix tests

* Implement remaining methods

* chore: Fix misresolved conflicts

* Simplify syntax

* Reduce diff

* Minor cleanup

* Fix lint

* Inject dependency

* Improve typings

* Remove unused method

* Restore method

* Add comment

* Rename in test

* Restore comments

* Clean up dynamic webhook handling

* Clean up tests

* Remove redundant `cache` prefix

* fix: Correct `uniquePath` for dynamic webhooks
2023-08-04 11:52:45 +02:00