Commit graph

762 commits

Author SHA1 Message Date
कारतोफ्फेलस्क्रिप्ट™ b4e60c3b47
refactor: Rewrite workflow public-api tests to avoid timeouts (no-changelog) (#5696) 2023-03-15 14:24:09 +01:00
Michael Auerswald ef07528cc2
feat(core): Improve SAML connection test (#5680)
* improve saml test

* cleanup

* remove unused SamlConfiguration types
2023-03-13 19:47:57 +01:00
Michael Auerswald c6ba0bd8de
feat(core): Convert saml controller to decorator style (#5653) (no-changelog) 2023-03-10 19:19:52 +01:00
कारतोफ्फेलस्क्रिप्ट™ 90afa5e55f
fix(core): Initialize License and LDAP in the correct order (#5673) 2023-03-10 19:05:51 +01:00
कारतोफ्फेलस्क्रिप्ट™ 7a4e9ef5fa
refactor: Remove n8n-core dependency in nodes-base (no-changelog) (#5649) 2023-03-09 18:13:15 +01:00
कारतोफ्फेलस्क्रिप्ट™ 1979eb8a9c
fix(core): Fix issues with LDAP reset and LDAP init (no-changelog) (#5657)
* fix(core): Ldap reset should recreate the default config

* fix(core): init ldap before registering the routes for it
2023-03-09 16:24:44 +01:00
कारतोफ्फेलस्क्रिप्ट™ d872866add
feat(core): Allow using middlewares with decorators on a per-route basis (no-changelog) (#5656) 2023-03-09 15:08:48 +01:00
कारतोफ्फेलस्क्रिप्ट™ 356e916194
refactor(core): Convert more routes to use the decorator pattern (no-changelog) (#5611)
* move nodeTypes api to a controller class
* move tags api to a controller class
* move LDAP routes to a controller class
* move nodes routes to a controller class
2023-03-09 14:42:13 +01:00
Michael Auerswald 6f27b445ca
feat(core): Refactor and add SAML preferences for service provider instance (#5637)
* create SP through parameters instead of metadata

* refactor SAML prefs and add SP configurations
2023-03-09 09:08:23 +01:00
Michael Auerswald 419969c0d7
fix(core): Rename advancedFilters to advancedExecutionFilters (#5643)
rename advancedFilters to advancedExecutionFilters
2023-03-07 18:35:52 +01:00
Michael Auerswald 0b5ef09e7c
feat(core): Add advancedFilters feature flag (#5638)
adds advancedFilters feature flag
2023-03-07 14:18:10 +01:00
कारतोफ्फेलस्क्रिप्ट™ 45ccdd3bb5
fix(core): Allow serving icons for custom nodes with npm scoped names (#5626) 2023-03-06 13:36:47 +01:00
Michael Auerswald ca66ec8f4d
feat(core): Add SAML XML validation (#5600)
* consolidate SSO settings

* update saml settings

* fix type error

* limit user changes when saml is enabled

* add test

* add toggle endpoint and fetch metadata

* rename enabled param

* add handling of POST saml login request

* add config test endpoint

* adds saml XML validation

* add comment

* protect test endpoint

* improve ignoreSSL and some cleanup

* fix wrong schema used

* remove console.log
2023-03-06 09:44:25 +01:00
कारतोफ्फेलस्क्रिप्ट™ 3c57062571
refactor: Fix some typos (no-changelog) (#5616) 2023-03-03 18:49:19 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4e244937c9
refactor: catch doesn't need to have a param (no-changelog) (#5614) 2023-03-03 18:18:49 +01:00
Michael Auerswald 523fa71705
feat(core): Add SAML post and test endpoints (#5595)
* consolidate SSO settings

* update saml settings

* fix type error

* limit user changes when saml is enabled

* add test

* add toggle endpoint and fetch metadata

* rename enabled param

* add handling of POST saml login request

* add config test endpoint
2023-03-03 10:19:43 +01:00
Michael Auerswald b5179597f3
feat(core): Limit user changes when saml is enabled (#5577)
* consolidate SSO settings

* update saml settings

* fix type error

* limit user changes when saml is enabled

* add test
2023-03-03 10:05:30 +01:00
OlegIvaniv 0004dc7ee8
ci(editor): Run e2e tests in parallel and improve build caching (#5445)
* WIP: Cypress parallel CI run test

* Trigger action on branch push

* Change build artifacts path

* Make sure to checkout the repo for testing job

* Use Cypress action for installing

* Lock cypress action userd version

* Skip node install step since we're using cypress node16 container

* Let Cypress handle pnpm install

* Use setup-node action for caching pnpm

* Set CYPRESS_CACHE_FOLDER

* Set CYPRESS_CACHE_FOLDER

* Manually cache pnpm store

* Dont fix pnpm version

* Use caching action also in testing job

* Zip packages dist before uploading the artifacts and change caching key

* Use absolute build paths for zipping job

* Use zip command in action

* Use tar for zipping packages

* Debuggin directory ls

* Debugging caching of modules

* Attempt to fix permissions issue

* Porivde Cypress executable via `CYPRESS_RUN_BINARY`

* Cache /github/home

* Adjust caching keys

* Debug: search for cypress exec

* Debugging: List dirs

* Use pnpm install action to install node_modules

* Do not log /home/runner

* Use node_modules/.bin Cypress binary

* Use absolute path to nodue modules

* Run Cypress via custom command

* Try with patched cypress action

* Revert logging

* Manually specify cypress config file

* Use absolute paths

* Fix cypress config name

* Debug print cypress config

* Remove debugging, increase to 4 containers

* Increase amount of containers

* Add env-version matrix

* Replace node14 with node18 in testing matrix

* Remove debugging and add node 14

* Use just node14

* Use cypress:base and remove browser req

* Give more general timeouts

* Try with node16

* Change cache directive position

* Replace zip artifact upload with cache

* Cache full packages not just dist

* Test with variable inputs

* Add commit info message

* Remove wrongly commited code

* Allow WF API dispatch

* Try Chrome browser again for comparison

* Include Monaco in the build

* Make e2e workflow re-usable

* Comment out invalid reusable workflow args

* Use electron and add node 14 run

* Fix env arg

* Provide custom ci-build-id

* Refactor remaining e2e workflow to use reusable action

* Remove single matrix directive

* Refactor ci-pull-req

* Make lint job dependant on test jobs

* Disable debugging job

* Make containers dynamic

* Cleanup & install git for linting action

* Use regular buntu image for PR linting

* Debugging failing tests

* Remove fixed spec name

* Debug e2e env var

* Do not use realkeypress which crashes electron runner

* Debugging

* chore: remove console

* chore: remove console

* test: remove node 14 tests

* test: replace test branch with master

* test: use tests in current branch

* test: use relative path

* chore: clean up

* test: only trigger on approval

* ci: update test PR

* ci: use curr branch

* ci: only run 14 on schedule, not for slack command

* ci: only run test on approval

* ci: clean up branch, rename step

* ci: rename steps

* ci: clean up cancel

* ci: clean up env var

* ci: set var

* ci: use chromef

* ci: use electron

* chore: add console log

* chore: add console log

* ci: update to string

* ci: set all env options

* test: build

* ci: fix step issue

* Fix failing tests & upgrade to Cypress 12

* Allow WF dispatch of e2e reusable

* Fix wrong naming in e2e-tests workflow

* Redeploy

* Fix tests

* Fix NDV tests and remove skipping of webhooks execution tests

* Fix clipboard read command

* Fix execution failing tests

* Reset before each 15 and 3

* Fix flaky tests

* Cleanup and log envs

* Test fixes

* Default owner spec fixes

* Get rid of CYPRESS_RUN_ENV

* Increase amount of containers, cleanup and add mock for credentials test call

* Cleanup & fix PR tests unit tests

* Wait for WF to loade in sharing spec

* Do linting and unit tests first

* Use frozen lockfile

* Revert back ci pull request jobs order

* Refine credential input selector and move cy.waitForLoad to correct position in 15-scheduler spec

* test: build

* Wait for WF execution instead of arbitraty timeout in WF execution spec, change order of jobs for ci pull request

* Fix flaky 3-default owner spec and wait for execution list to load in 20-workflow-executions

* Use setup node action

* Remove caching for lint/unit tests

* Experiment with parallel test & lint on ci

* Provide cache key dynamically

* Run e2e in parallel on pr

* Only run node14 e2e on daily schedule

* Make sure to generate generate new ci-build-id on re-runs

* Remove debugging prints

* Address PR comments

* Rename custom onBeforeUnload handler

* Make sure 19-execution spec waits for wf to load properly before import fixtures

---------

Co-authored-by: Mutasem <mutdmour@gmail.com>
2023-03-02 16:50:21 +01:00
Michael Auerswald 31cc8de829
feat(core): Add SAML settings and consolidate LDAP under SSO (#5574)
* consolidate SSO settings

* update saml settings

* fix type error
2023-03-02 09:00:51 +01:00
कारतोफ्फेलस्क्रिप्ट™ a19ec6ac94
fix(core): Revert isPending check on the user entity (#5571) 2023-02-27 12:35:59 +01:00
Csaba Tuncsik ea2035b510
fix(core): Fix Filtering of Workflow by Tags (#5570) 2023-02-27 12:25:45 +01:00
Michael Auerswald 40a934bbb4
feat(core): Add SAML login setup (#5515)
* initial commit with sample data

* basic saml setup

* cleanup console logs

* limit saml endpoints through middleware

* basic login and token issue

* saml service and cleanup

* refactor and create user

* get/set saml prefs

* fix authentication issue

* redirect to user details

* merge fix

* add generated password to saml user

* update user from attributes where possible

* refactor and fix creating new user

* rename saml prefs key

* minor cleanup

* Update packages/cli/src/config/schema.ts

Co-authored-by: Omar Ajoue <krynble@gmail.com>

* Update packages/cli/src/config/schema.ts

Co-authored-by: Omar Ajoue <krynble@gmail.com>

* Update packages/cli/src/controllers/auth.controller.ts

Co-authored-by: Omar Ajoue <krynble@gmail.com>

* code review changes

* fix default saml enabled

* remove console.log

* fix isSamlLicensed

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-02-24 20:37:19 +01:00
कारतोफ्फेलस्क्रिप्ट™ 88de6613bd
fix(core): Fix execution pruning queries (#5562)
* fix(core): Execution pruning should delete query should use the `OR` operator

* fix(core): Prune executions in a chunk to avoid sqlite error "Expression tree is too large"

* reduce the memory usage during execution pruning
2023-02-24 18:02:34 +01:00
Omar Ajoue 46d9ac6c6f
fix: Prevent executions from displaying as running forever (#5563)
* fix: Prevent executions from displaying as running forever

* fix: Correct migration query
2023-02-24 14:53:49 +01:00
OlegIvaniv 0fbb3f0f02
fix(core): Do not remove empty output connections arrays in PurgeInvalidWorkflowConnections migration (#5546)
fix(cli): Do not remove empty output connections arrays in PurgeInvalidWorkflowConnections migration
2023-02-22 14:20:06 +01:00
Michael Auerswald 17eff4d7d6
fix(core): Fix execution status filters (#5533)
* fix status filters

* fix countfilter

* add migrations to backfill status

* fix migrations
2023-02-21 21:44:46 +01:00
कारतोफ्फेलस्क्रिप्ट™ 52f740b9e8
refactor(core): Use an IoC container to manage singleton classes [Part-1] (no-changelog) (#5509)
* add typedi

* convert ActiveWorkflowRunner into an injectable service

* convert ExternalHooks into an injectable service

* convert InternalHooks into an injectable service

* convert LoadNodesAndCredentials into an injectable service

* convert NodeTypes and CredentialTypes into an injectable service

* convert ActiveExecutions into an injectable service

* convert WaitTracker into an injectable service

* convert Push into an injectable service

* convert ActiveWebhooks and  TestWebhooks into an injectable services

* handle circular references, and log errors when a circular dependency is found
2023-02-21 19:21:56 +01:00
कारतोफ्फेलस्क्रिप्ट™ f58573dba3
fix(core): Do not explicitly bypass auth on urls containing .svg (#5525) 2023-02-21 11:52:22 +01:00
कारतोफ्फेलस्क्रिप्ट™ 510855d958
fix(core): User update endpoint should only allow updating email, firstName, and lastName (#5526) 2023-02-21 11:22:54 +01:00
कारतोफ्फेलस्क्रिप्ट™ eef2574067
fix(core): Do not allow arbitrary path traversal in BinaryDataManager (#5523) 2023-02-21 11:21:17 +01:00
कारतोफ्फेलस्क्रिप्ट™ f0f8d59fee
fix(core): Do not allow arbitrary path traversal in the credential-translation endpoint (#5522) 2023-02-21 11:21:04 +01:00
Mutasem Aldmour 26a20ed47e
feat: Support feature flag evaluation server side (#5511)
* feat(editor): roll out schema view

* feat(editor): add posthog tracking

* refactor: use composables

* refactor: clean up console log

* refactor: clean up impl

* chore: clean up impl

* fix: fix demo var

* chore: add comment

* refactor: clean up

* chore: wrap error func

* refactor: clean up import

* refactor: make store

* feat: enable rudderstack usebeacon, move event to unload

* chore: clean up alert

* refactor: move tracking from hooks

* fix: reload flags on login

* fix: add func to setup

* fix: clear duplicate import

* chore: add console to tesT

* chore: add console to tesT

* fix: try reload

* chore: randomize instnace id for testing

* chore: randomize instnace id for testing

* chore: add console logs for testing

* chore: move random id to fe

* chore: use query param for testing

* feat: update PostHog api endpoint

* feat: update rs host

* feat: update rs host

* feat: update rs endpoints

* refactor: use api host for BE events as well

* refactor: refactor out posthog client

* feat: add feature flags to login

* feat: add feature flags to login

* feat: get feature flags to work

* feat: add created at to be events

* chore: add todos

* chore: clean up store

* chore: add created at to identify

* feat: add posthog config to settings

* feat: add bootstrapping

* chore: clean up

* chore: fix build

* fix: get dates to work

* fix: get posthog to recognize dates

* chore: refactor

* fix: update back to number

* fix: update key

* fix: get experiment evals to work

* feat: add posthog to signup router

* feat: add feature flags on sign up

* chore: clean up

* fix: fix import

* chore: clean up loading script

* feat: add timeout, fix: script loader

* fix: test timeout and get working on 8080

* refactor: move out posthog

* feat: add experiment tracking

* fix: clear tracked on reset

* fix: fix signup bug

* fix: handle errors when telmetry is disabled

* refactor: remove redundant await

* fix: add back posthog to telemetry

* test: fix test

* test: fix test

* test: add tests for posthog client

* lint: fix

* fix: fix issue with slow decide endpoint

* lint: fix

* lint: fix

* lint: fix

* lint: fix

* chore: address PR feedback

* chore: address PR feedback

* feat: add onboarding experiment
2023-02-21 11:35:35 +03:00
Omar Ajoue 75a094a8c0
fix: Fixes an issue when saving an active workflow without triggers would cause n8n to be stuck (#5513)
fix: Allow saving and editing when an active workflow is saved without triggers
2023-02-20 12:22:27 +01:00
Ahsan Virani db702932f3
feat(core): Add configurable execution history limit (#5505)
* Prune execution data when more than cofnfigured limit

* use stricter typings

* use `pruneDataMaxCount`

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-02-20 10:28:38 +01:00
कारतोफ्फेलस्क्रिप्ट™ 561882f599
refactor(core): Improve instance owner setup and add unit tests (no-changelog) (#5499)
* refactor(core): Avoid fetching all workflows and credentials for the owner setup screen

* refactor(core): Add unit tests for the owner controller
2023-02-17 10:59:09 +01:00
कारतोफ्फेलस्क्रिप्ट™ 12104bc4a3
refactor(core): Delete unused GET /me endpoint (no-changelog) (#5498) 2023-02-17 10:58:48 +01:00
Michael Auerswald d143f3f2ec
feat(core): Add execution runData recovery and status field (#5112)
* adds ExecutionEvents view modal to ExecutionList

* fix time rendering and remove wf column

* checks for unfinished executions and fails them

* prevent re-setting stoppedAt for execution

* some cleanup / manually create rundata after crash

* quicksave

* remove Threads lib, log worker rewrite

* cleanup comment

* fix sentry destination return value

* test for tests...

* run tests with single worker

* fix tests

* remove console log

* add endpoint for execution data recovery

* lint cleanup and some refactoring

* fix accidental recursion

* remove cyclic imports

* add rundata recovery to Workflowrunner

* remove comments

* cleanup and refactor

* adds a status field to executions

* setExecutionStatus on queued worker

* fix onWorkflowPostExecute

* set waiting from worker

* get crashed status into frontend

* remove comment

* merge fix

* cleanup

* catch empty rundata in recovery

* refactor IExecutionsSummary and inject nodeExecution Errors

* reduce default event log size to 10mb from 100mb

* add per node execution status

* lint fix

* merge and lint fix

* phrasing change

* improve preview rendering and messaging

* remove debug

* Improve partial rundata recovery

* fix labels

* fix line through

* send manual rundata to ui at crash

* some type and msg push fixes

* improve recovered item rendering in preview

* update workflowStatistics on recover

* merge fix

* review fixes

* merge fix

* notify eventbus when ui is back up

* add a small timeout to make sure the UI is back up

* increase reconnect timeout to 30s

* adjust recover timeout and ui connection lost msg

* do not stop execution in editor after x reconnects

* add executionRecovered push event

* fix recovered connection not green

* remove reconnect toast and  merge existing rundata

* merge editor and recovered data for own mode
2023-02-17 10:54:07 +01:00
Michael Auerswald 3a9c257f55
feat(core): Add saml feature flag (#5494)
adds saml feature flag
2023-02-16 15:05:39 +01:00
Mutasem Aldmour 3de49e8f78
feat: Update telemetry api endpoints (#5482)
* feat: update PostHog api endpoint

* feat: update rs host

* feat: update rs host

* feat: update rs endpoints

* refactor: use api host for BE events as well
2023-02-16 15:07:08 +03:00
Csaba Tuncsik a2c6ea9e11
fix(core): Remove unnecessary info from GET /workflows response (#5311)
* fix(core): Remove unnecessary info from `GET /workflows` response

* fix(core): Remove unnecessary info from `GET /workflows` response

* fix(core): Remove credentials from `GET /workflows` response

* fix(core): Update unit tests for `GET /workflows` response

* fix(core): Remove `usedCredentials` from `GET /workflows` response

* fix(core): Update unit tests for `GET /workflows` response

* fix(core): remove nodes from getMany

* fix(core): remove unnecessary owner props from workflow list items

* fix(core): fix lint error

* fix(core): remove unused function

* fix(core): simplifying ownerId usage

* fix(core): trim down the query for workflow listing
2023-02-16 10:36:24 +01:00
कारतोफ्फेलस्क्रिप्ट™ a9f08fc5ba
fix(core): Fix issues with community node installation (no-changelog) (#5481)
This fixes the following issues:
* After a community node is installed, we were not calling `postProcessLoaders`, which was causing a bunch of unexpected behaviors, and sometimes even crashes. This was only happening in the session where the package was installed. After a crash, the restarted service was working without these issues.
* After a community node is installed, the icon for the nodes and credentials were missing in the UI, as we were creating one icons route per installed package at startup, and this did not handle newly installed packages. restarting the service fixes this issue as well.

Fixes https://community.n8n.io/t/showing-weird-count-on-community-nodes/23035
2023-02-15 16:09:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ a6c59fcbc2
fix(core): Use stricter typing on queries in cli commands (no-changelog) (#5476)
This has been broken since TypeORM upgrade

Fixes:
https://linear.app/n8n/issue/ENG-50
https://community.n8n.io/t/execute-command-stdout/22994
2023-02-14 17:05:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4dc458eca5
fix(core): Fix the issue with test webhooks getting removed incorrectly (no-changelog) (#5466)
This broke because of the change [here](https://github.com/n8n-io/n8n/pull/5443/files#diff-b386248ff00977749c873ed85821c241b773e9740d7e7adf94e05b73b350ed74L152).
`finally` block is called even if there is a `return` in the `try` block, and this is causing the test webhook to be removed, even when it shouldn't be removed.
2023-02-13 16:16:53 +01:00
Jan Oberhauser 339655611f
fix(core): Fix issue that worker and webhook service close directly (#5461) 2023-02-11 09:46:10 -06:00
agobrech 309c995364
test: Change workflows to test from safe list to skip list (#5457)
* Remove check by safe list to skip list

* add an option to skip using statistics events

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-02-10 17:11:06 +01:00
कारतोफ्फेलस्क्रिप्ट™ 9a331ec7b6
fix(core): Stop using unbound calls to stopProcess (no-changelog) (#5456) 2023-02-10 16:34:39 +01:00
Omar Ajoue a2e2ec5442
test: Add tests for ActiveWorkflowRunner class (#5278) 2023-02-10 15:24:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 538984dc2f
feat(core): Add support for WebSockets as an alternative to Server-Sent Events (#5443)
Co-authored-by: Matthijs Knigge <matthijs@volcano.nl>
2023-02-10 15:02:47 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5194513850
refactor(core): Delete duplicate code across all commands (#5452) 2023-02-10 14:59:20 +01:00
Omar Ajoue 2eb72a6c9f
fix: MySQL migration parses database contents if necessary (fix for MariaDB) (#5441) 2023-02-09 18:15:20 +01:00