Commit graph

275 commits

Author SHA1 Message Date
Iván Ovejero 9b651cf4f8
refactor: Accumulate loadOptions from all node versions to validate (no-changelog) (#6014)
 Accumulate loadOptions from node versions to validate
2023-04-20 16:21:07 +02:00
Jan Oberhauser 62f993c84f
feat(core): Add support for digestAuth to httpRequest and declarative style (#5676)
feat(core): Add support to digestAuth to httpRequest and declarative style
2023-04-19 13:44:41 +02:00
Val 1bb987140a
feat: Add variables feature (#5602)
* feat: add variables db models and migrations

* feat: variables api endpoints

* feat: add $variables to expressions

* test: fix ActiveWorkflowRunner tests failing

* test: a different fix for the tests broken by $variables

* feat: variables licensing

* fix: could create one extra variable than licensed for

* feat: Add Variables UI page and $vars global property (#5750)

* feat: add support for row slot to datatable

* feat: add variables create, read, update, delete

* feat: add vars autocomplete

* chore: remove alert

* feat: add variables autocomplete for code and expressions

* feat: add tests for variable components

* feat: add variables search and sort

* test: update tests for variables view

* chore: fix test and linting issue

* refactor: review changes

* feat: add variable creation telemetry

* fix: Improve variables listing and disabled case, fix resource sorting (no-changelog) (#5903)

* fix: Improve variables disabled experience and fix sorting

* fix: update action box margin

* test: update tests for variables row and datatable

* fix: Add ee controller to base controller

* fix: variables.ee routes not being added

* feat: add variables validation

* fix: fix vue-fragment bug that breaks everything

* chore: Update lock

* feat: Add variables input validation and permissions (no-changelog) (#5910)

* feat: add input validation

* feat: handle variables view for non-instance-owner users

* test: update variables tests

* fix: fix data-testid pattern

* feat: improve overflow styles

* test: fix variables row snapshot

* feat: update sorting to take newly created variables into account

* fix: fix list layout overflow

* fix: fix adding variables on page other than 1. fix validation

* feat: add docs link

* fix: fix default displayName function for resource-list-layout

* feat: improve vars expressions ux, cm-tooltip

* test: fix datatable test

* feat: add MATCH_REGEX validation rule

* fix: overhaul how datatable pagination selector works

* feat: update  completer description

* fix: conditionally update usage syntax based on key validation

* test: update datatable snapshot

* fix: fix variables-row button margins

* fix: fix pagination overflow

* test: Fix broken test

* test: Update snapshot

* fix: Remove duplicate declaration

* feat: add custom variables icon

---------

Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-04-18 12:41:55 +02:00
कारतोफ्फेलस्क्रिप्ट™ c42820e82e
fix(core): Make getExecutionId available on all nodes types (#5990)
fixes https://community.n8n.io/t/this-getexecutionid-in-custom-node-development-not-available-anymore/25259/1
2023-04-17 10:11:26 +02:00
Iván Ovejero 5227ccd75a
ci: Validate load options methods in nodes-base (no-changelog) (#5862) 2023-04-12 15:46:11 +02:00
कारतोफ्फेलस्क्रिप्ट™ e79679c023
fix(HTTP Request Node): Show detailed error message in the UI again (#5959) 2023-04-12 14:58:05 +02:00
Val 323e26acfd
fix(core): Validate customData keys and values (#5920) (no-changelog)
* fix(core): Validate customData keys and values

Throws errors in manual mode and ignores and logs values in production

* fix: validate customData key characters

* refactor: review changes

* fix: logger not initialised for metadata tests

* fix: allow numbers for values
2023-04-12 09:18:26 +01:00
कारतोफ्फेलस्क्रिप्ट™ 6689451e8c
fix(core): Do not execute workflowExecuteBefore hook when resuming executions from a waiting state (#5727) 2023-04-06 10:18:19 +02:00
agobrech 33c67f45ba
fix(HTTP Request Node): Refresh token properly on never fail option (#5861)
* Add handle for simple request options and refresh token

* Remove console.logs

* Add safe check for full response
2023-04-05 15:27:04 +02:00
कारतोफ्फेलस्क्रिप्ट™ a260c05fa8
fix(core): Improve axios error handling in nodes (#5891) 2023-04-03 18:37:22 +02:00
कारतोफ्फेलस्क्रिप्ट™ 69efde7a09
fix(HTTP Request Node): Detect mime-type from streaming responses (#5896) 2023-04-03 18:37:07 +02:00
Jan Oberhauser aed5674530 Revert "fix(core): Improve axios error handling in nodes (#5699)"
This reverts commit 33d9784319.
2023-04-01 19:03:12 +02:00
Csaba Tuncsik d78a41db54
feat: Execution custom data saving and filtering (#5496)
* wip: workflow execution filtering

* fix: import type failing to build

* fix: remove console.logs

* feat: execution metadata migrations

* fix(editor): Move global executions filter to its own component

* fix(editor): Using the same filter component in workflow level

* fix(editor): a small housekeeping

* checking workflowId in filter applied

* fix(editor): update filter after resolving merge conflicts

* fix(editor): unify empy filter status

* feat(editor): add datetime picker to filter

* feat(editor): add meta fields

* fix: fix button override in datepicker panel

* feat(editor): add filter metadata

* feat(core): add 'startedBefore' execution filter prop

* feat(core): add 'tags' execution query filter

* Revert "feat(core): add 'tags' execution query filter"

This reverts commit a7b968081c.

* feat(editor): add translations and tooltip and counting selected filter props

* fix(editor): fix label layouts

* fix(editor): update custom data docs link

* fix(editor): update custom data tooltip position

* fix(editor): update tooltip text

* refactor: Ignore metadata if not enabled by license

* fix(editor): Add paywall states to advanced execution filter

* refactor: Save custom data also for worker mode

* fix: Remove duplicate migration name from list

* fix(editor): Reducing filter complexity and add debounce to text inputs

* fix(editor): Remove unused import, add comment

* fix(editor): simplify event listener

* fix: Prevent error when there are running executions

* test(editor): Add advanced execution filter basic unit test

* test(editor): Add advanced execution filter state change unit test

* fix: Small lint issue

* feat: Add indices to speed up queries

* feat: add customData limits

* refactor: put metadata save in transaction

* chore: remove unneed comment

* test: add tests for execution metadata

* fix(editor): Fixes after merge conflict

* fix(editor): Remove unused import

* wordings and ui fixes

* fix(editor): type fixes

* feat: add code node autocompletions for customData

* fix: Prevent transaction issues and ambiguous ID in sql clauses

* fix(editor): Suppress requesting current executions if metadata is used in filter (#5739)

* fix(editor): Suppress requesting current executions if metadata is used in filter

* fix(editor): Fix arrows for select in popover

* refactor: Improve performance by correcting database indices

* fix: Lint issue

* test: Fix broken test

* fix: Broken test

* test: add call data check for saveExecutionMetadata test

---------

Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
2023-03-23 18:07:46 +01:00
कारतोफ्फेलस्क्रिप्ट™ b0cfd69f2b
fix(core): Setup nodeHelpers that aren't exposed in the code sandbox (no-changelog) (#5753) 2023-03-23 15:11:18 +01:00
कारतोफ्फेलस्क्रिप्ट™ 38e91ab730
refactor(core): Stop importing LoggerProxy and createDeferredPromise in nodes-base (no-changelog) (#5742)
* refactor(core): Stop importing LoggerProxy in nodes-base

* refactor(core): Stop importing createDeferredPromise in nodes-base
2023-03-22 14:04:15 +01:00
agobrech 33d9784319
fix(core): Improve axios error handling in nodes (#5699)
* fix(core): Improve axios error handling in nodes

* handle axios errors without a response (like when connection fails)

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-03-20 17:34:14 +01:00
कारतोफ्फेलस्क्रिप्ट™ 7a4e9ef5fa
refactor: Remove n8n-core dependency in nodes-base (no-changelog) (#5649) 2023-03-09 18:13:15 +01:00
agobrech 5790e5e719
fix(core): Fix trying to pipe a non stream on errors (no-changelog) (#5660) 2023-03-09 17:48:33 +01:00
agobrech ce0d9d2bed
feat(HTTP Request Node): Move from Binary Buffer to Binary streaming (#5610)
*  Change from binary buffer to binary streaming

* Remove console.logs

* Import Readable from the correct lib

* stream response

* parametersToKeyValue doesn't need to be async anymore

* Fix bodyParameter reduce method

* parametersToKeyValue doesn't need to be async anymore

* handle streaming responses

* send `Content-Length` and `Content-Type` on binary requests

* Add new helper function for binary data

* Add binary function to helpers interface

* Fix bug in error handler

* Fix issue with wrongfully assigned headers to body

* Fix test workflow

* Remove console.logs

* Remove unnecsessary type

* Remove concat dependency already imported in workflow package

* Update pnpm-lock file

* Small fixes, asyncronous error message

* reset the lockfile

* Remove buffer check and simplify error handling

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-03-09 15:38:54 +01:00
agobrech 0d49ad8b93
fix(core): Add a helper function to convert binary streams to buffers (no-changelog) (#5641) 2023-03-08 14:32:27 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5eb0d52459
refactor: Unify binary-data assertion across all nodes (no-changelog) (#5624) 2023-03-06 17:33:32 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4e244937c9
refactor: catch doesn't need to have a param (no-changelog) (#5614) 2023-03-03 18:18:49 +01:00
कारतोफ्फेलस्क्रिप्ट™ 52f740b9e8
refactor(core): Use an IoC container to manage singleton classes [Part-1] (no-changelog) (#5509)
* add typedi

* convert ActiveWorkflowRunner into an injectable service

* convert ExternalHooks into an injectable service

* convert InternalHooks into an injectable service

* convert LoadNodesAndCredentials into an injectable service

* convert NodeTypes and CredentialTypes into an injectable service

* convert ActiveExecutions into an injectable service

* convert WaitTracker into an injectable service

* convert Push into an injectable service

* convert ActiveWebhooks and  TestWebhooks into an injectable services

* handle circular references, and log errors when a circular dependency is found
2023-02-21 19:21:56 +01:00
कारतोफ्फेलस्क्रिप्ट™ eef2574067
fix(core): Do not allow arbitrary path traversal in BinaryDataManager (#5523) 2023-02-21 11:21:17 +01:00
Michael Auerswald d143f3f2ec
feat(core): Add execution runData recovery and status field (#5112)
* adds ExecutionEvents view modal to ExecutionList

* fix time rendering and remove wf column

* checks for unfinished executions and fails them

* prevent re-setting stoppedAt for execution

* some cleanup / manually create rundata after crash

* quicksave

* remove Threads lib, log worker rewrite

* cleanup comment

* fix sentry destination return value

* test for tests...

* run tests with single worker

* fix tests

* remove console log

* add endpoint for execution data recovery

* lint cleanup and some refactoring

* fix accidental recursion

* remove cyclic imports

* add rundata recovery to Workflowrunner

* remove comments

* cleanup and refactor

* adds a status field to executions

* setExecutionStatus on queued worker

* fix onWorkflowPostExecute

* set waiting from worker

* get crashed status into frontend

* remove comment

* merge fix

* cleanup

* catch empty rundata in recovery

* refactor IExecutionsSummary and inject nodeExecution Errors

* reduce default event log size to 10mb from 100mb

* add per node execution status

* lint fix

* merge and lint fix

* phrasing change

* improve preview rendering and messaging

* remove debug

* Improve partial rundata recovery

* fix labels

* fix line through

* send manual rundata to ui at crash

* some type and msg push fixes

* improve recovered item rendering in preview

* update workflowStatistics on recover

* merge fix

* review fixes

* merge fix

* notify eventbus when ui is back up

* add a small timeout to make sure the UI is back up

* increase reconnect timeout to 30s

* adjust recover timeout and ui connection lost msg

* do not stop execution in editor after x reconnects

* add executionRecovered push event

* fix recovered connection not green

* remove reconnect toast and  merge existing rundata

* merge editor and recovered data for own mode
2023-02-17 10:54:07 +01:00
कारतोफ्फेलस्क्रिप्ट™ a9f08fc5ba
fix(core): Fix issues with community node installation (no-changelog) (#5481)
This fixes the following issues:
* After a community node is installed, we were not calling `postProcessLoaders`, which was causing a bunch of unexpected behaviors, and sometimes even crashes. This was only happening in the session where the package was installed. After a crash, the restarted service was working without these issues.
* After a community node is installed, the icon for the nodes and credentials were missing in the UI, as we were creating one icons route per installed package at startup, and this did not handle newly installed packages. restarting the service fixes this issue as well.

Fixes https://community.n8n.io/t/showing-weird-count-on-community-nodes/23035
2023-02-15 16:09:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ 59f5c4221e
fix(core): Handle versioned custom nodes correctly (#5313) 2023-02-10 18:33:04 +01:00
Valya 9c1f827dad
feat(core): Live reload node/credential descriptions in development (no-changelog) (#4939) 2023-02-08 19:26:07 +01:00
कारतोफ्फेलस्क्रिप्ट™ f23fb92696
fix(core): Stop copying icons to cache (#5419)
Fixes 

- https://github.com/n8n-io/n8n/issues/4973
- https://github.com/n8n-io/n8n/issues/5274
- https://community.n8n.io/t/starting-n8n-fails-with-ebusy-error/21243
- https://community.n8n.io/t/problem-executing-workflow-ebusy-resource-busy-or-locked-copyfile/21280

Replaces

- https://github.com/n8n-io/n8n/pull/5052
- https://github.com/n8n-io/n8n/pull/5401
2023-02-08 18:57:43 +01:00
OlegIvaniv 6985500a7d
fix(core): Fix populating of node custom api call options (#5347)
* feat(core): Fix populating of node custom api call options

* lint fixes

* Adress PR comments

* Add e2e test and only inject custom API options for latest version

* Make sure to injectCustomApiCallOption for the latest version of node

* feat(cli): Move apiCallOption injection to LoadNodesAndCredentials and add e2e tests to check for custom nodes credentials

* Load nodes and credentials fixtures from a single place

* Console warning if credential is invalid during customApiOptions injection
2023-02-03 13:14:59 +01:00
OlegIvaniv 616074158c
fix(core): Revert custom API option injecting (#5345)
Revert "feat(core): Fix populating of node custom api call options (#5303)"

This reverts commit e58bc41d24.
2023-02-02 20:03:45 +01:00
OlegIvaniv e58bc41d24
feat(core): Fix populating of node custom api call options (#5303)
* feat(core): Fix populating of node custom api call options

* lint fixes

* Adress PR comments

* Add e2e test and only inject custom API options for latest version

* Make sure to injectCustomApiCallOption for the latest version of node
2023-02-02 14:53:08 +01:00
Jonathan Bennetts ec7575b032
fix(core): Fix oauth2 client credentials not always working (#5327)
fix oauth client credentials not working as expected
2023-02-01 19:05:21 +01:00
agobrech a8637a0bc6
fix(Set Node): Fix behaviour when selecting continueOnFail & PairedItem (#5257)
* 🐛 Fix bug on continueOnFail with pairedItem

* 🐛  Fix pairedItem continueOnFail for the Set Node

* 🎨  Add the ExpressionExtensionError back

*  import fix

---------

Co-authored-by: Michael Kret <michael.k@radency.com>
2023-01-31 09:25:08 +01:00
Iván Ovejero 2ca959b383
refactor: Integrate consistent-type-imports in BE packages (no-changelog) (#5270) 2023-01-27 14:56:56 +01:00
कारतोफ्फेलस्क्रिप्ट™ c97f3cad59
fix(Read Binary File Node): Do not crash the execution when the source file does not exist (#5100)
This issue was introduced in https://github.com/n8n-io/n8n/pull/5069
2023-01-06 14:15:46 +01:00
कारतोफ्फेलस्क्रिप्ट™ 54126b2c87
refactor(Google Drive Node): Use node streams for uploading and downloading files (#5017)
* use streams to upload files to google drive

* use streams to download files from google drive

* use resumable uploads api for google drive

* avoid dangling promises, and reduce memory usage in error logging
2023-01-04 12:29:56 +01:00
कारतोफ्फेलस्क्रिप्ट™ 8bee04cd2a
refactor(Read Binary File Node): Use node streams for to reduce memory usage (#5069) 2023-01-02 17:07:10 +01:00
Iván Ovejero d9b98fc8be
refactor: Lint for no unneeded backticks (#5057) (no-changelog)
*  Create rule `no-unneeded-backticks`

* 👕 Enable rule

*  Run rule on `cli`

*  Run rule on `core`

*  Run rule on `workflow`

*  Rule rule on `design-system`

*  Run rule on `node-dev`

*  Run rule on `editor-ui`

*  Run rule on `nodes-base`
2022-12-29 12:20:43 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5cf70e1007
refactor: Consolidate redundant code in workflow execution-functions (no-changelog) (#4884)
* refactor: Consolidate redundant code in workflow interfaces (no-changelog)

* consolidate all execution-functions interfaces code

* move rawBody definition back

* address PR comments
2022-12-23 18:27:07 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5db9c46043
ci: Break most of the circular dependencies in code (no-changelog) (#4990) 2022-12-21 10:46:26 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5cbb5f4bc8
fix: BinaryDataManager should store metadata when saving from buffer as well (#4964) 2022-12-19 14:11:34 +01:00
Daemonxiao 6fac502f9e
fix(core): Fix HTTP Digest Auth for responses without an opaque parameter (#4806) 2022-12-16 06:10:26 +01:00
कारतोफ्फेलस्क्रिप्ट™ 3b969d2cd1
fix(Move Binary Data Node): Stringify objects before encoding them in MoveBinaryData (#4882)
* stringify objects before encoding them objects in MoveBinaryData

* add fileSize and fileType on MoveBinaryData converted data

* show `view` option for text files as well

* improve how JSON binary data is shown in the UI
2022-12-11 14:10:54 +01:00
Jan Oberhauser 38d7300d2a
fix(Split In Batches Node): Fix issue with pairedItem (#4873)
fix(SplitInBatches Node): Fix issue with pairedItem
2022-12-09 13:39:06 +01:00
कारतोफ्फेलस्क्रिप्ट™ 85241fd230
fix: Make nodes.exclude and nodes.include work with lazy-loaded nodes (#4833) 2022-12-06 19:15:15 +01:00
freya 1722c6b0c5
feat(core): Workflow Execution Statistics (#4200)
Add recording and reporting of workflow execution statistics
2022-12-06 14:55:40 +00:00
कारतोफ्फेलस्क्रिप्ट™ 46c75b9fd2
fix: Stop using prefixed core modules from nodejs (no-changelog) (#4813)
fix: stop using prefixed core modules from nodejs (no-changelog)

Starting node 18, there are some core modules that only work with the `node:` prefix, like `node:test`. It's very likely that nodejs will switch to prefix-only core modules in the next versions as un-prefixed core modules are susceptible to supply-chain attacks.
2022-12-05 11:41:30 +01:00
कारतोफ्फेलस्क्रिप्ट™ 8698e02079
fix: Register community nodes as known nodes (no-changelog) (#4775) 2022-11-30 18:17:57 +01:00
कारतोफ्फेलस्क्रिप्ट™ ada73ed41d
ci: Fix linting issues on master (no-changelog) (#4768)
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2022-11-30 12:16:19 +01:00