Commit graph

741 commits

Author SHA1 Message Date
Michael Auerswald 40a934bbb4
feat(core): Add SAML login setup (#5515)
* initial commit with sample data

* basic saml setup

* cleanup console logs

* limit saml endpoints through middleware

* basic login and token issue

* saml service and cleanup

* refactor and create user

* get/set saml prefs

* fix authentication issue

* redirect to user details

* merge fix

* add generated password to saml user

* update user from attributes where possible

* refactor and fix creating new user

* rename saml prefs key

* minor cleanup

* Update packages/cli/src/config/schema.ts

Co-authored-by: Omar Ajoue <krynble@gmail.com>

* Update packages/cli/src/config/schema.ts

Co-authored-by: Omar Ajoue <krynble@gmail.com>

* Update packages/cli/src/controllers/auth.controller.ts

Co-authored-by: Omar Ajoue <krynble@gmail.com>

* code review changes

* fix default saml enabled

* remove console.log

* fix isSamlLicensed

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-02-24 20:37:19 +01:00
कारतोफ्फेलस्क्रिप्ट™ 88de6613bd
fix(core): Fix execution pruning queries (#5562)
* fix(core): Execution pruning should delete query should use the `OR` operator

* fix(core): Prune executions in a chunk to avoid sqlite error "Expression tree is too large"

* reduce the memory usage during execution pruning
2023-02-24 18:02:34 +01:00
Omar Ajoue 46d9ac6c6f
fix: Prevent executions from displaying as running forever (#5563)
* fix: Prevent executions from displaying as running forever

* fix: Correct migration query
2023-02-24 14:53:49 +01:00
OlegIvaniv 0fbb3f0f02
fix(core): Do not remove empty output connections arrays in PurgeInvalidWorkflowConnections migration (#5546)
fix(cli): Do not remove empty output connections arrays in PurgeInvalidWorkflowConnections migration
2023-02-22 14:20:06 +01:00
Michael Auerswald 17eff4d7d6
fix(core): Fix execution status filters (#5533)
* fix status filters

* fix countfilter

* add migrations to backfill status

* fix migrations
2023-02-21 21:44:46 +01:00
कारतोफ्फेलस्क्रिप्ट™ 52f740b9e8
refactor(core): Use an IoC container to manage singleton classes [Part-1] (no-changelog) (#5509)
* add typedi

* convert ActiveWorkflowRunner into an injectable service

* convert ExternalHooks into an injectable service

* convert InternalHooks into an injectable service

* convert LoadNodesAndCredentials into an injectable service

* convert NodeTypes and CredentialTypes into an injectable service

* convert ActiveExecutions into an injectable service

* convert WaitTracker into an injectable service

* convert Push into an injectable service

* convert ActiveWebhooks and  TestWebhooks into an injectable services

* handle circular references, and log errors when a circular dependency is found
2023-02-21 19:21:56 +01:00
कारतोफ्फेलस्क्रिप्ट™ f58573dba3
fix(core): Do not explicitly bypass auth on urls containing .svg (#5525) 2023-02-21 11:52:22 +01:00
कारतोफ्फेलस्क्रिप्ट™ 510855d958
fix(core): User update endpoint should only allow updating email, firstName, and lastName (#5526) 2023-02-21 11:22:54 +01:00
कारतोफ्फेलस्क्रिप्ट™ eef2574067
fix(core): Do not allow arbitrary path traversal in BinaryDataManager (#5523) 2023-02-21 11:21:17 +01:00
कारतोफ्फेलस्क्रिप्ट™ f0f8d59fee
fix(core): Do not allow arbitrary path traversal in the credential-translation endpoint (#5522) 2023-02-21 11:21:04 +01:00
Mutasem Aldmour 26a20ed47e
feat: Support feature flag evaluation server side (#5511)
* feat(editor): roll out schema view

* feat(editor): add posthog tracking

* refactor: use composables

* refactor: clean up console log

* refactor: clean up impl

* chore: clean up impl

* fix: fix demo var

* chore: add comment

* refactor: clean up

* chore: wrap error func

* refactor: clean up import

* refactor: make store

* feat: enable rudderstack usebeacon, move event to unload

* chore: clean up alert

* refactor: move tracking from hooks

* fix: reload flags on login

* fix: add func to setup

* fix: clear duplicate import

* chore: add console to tesT

* chore: add console to tesT

* fix: try reload

* chore: randomize instnace id for testing

* chore: randomize instnace id for testing

* chore: add console logs for testing

* chore: move random id to fe

* chore: use query param for testing

* feat: update PostHog api endpoint

* feat: update rs host

* feat: update rs host

* feat: update rs endpoints

* refactor: use api host for BE events as well

* refactor: refactor out posthog client

* feat: add feature flags to login

* feat: add feature flags to login

* feat: get feature flags to work

* feat: add created at to be events

* chore: add todos

* chore: clean up store

* chore: add created at to identify

* feat: add posthog config to settings

* feat: add bootstrapping

* chore: clean up

* chore: fix build

* fix: get dates to work

* fix: get posthog to recognize dates

* chore: refactor

* fix: update back to number

* fix: update key

* fix: get experiment evals to work

* feat: add posthog to signup router

* feat: add feature flags on sign up

* chore: clean up

* fix: fix import

* chore: clean up loading script

* feat: add timeout, fix: script loader

* fix: test timeout and get working on 8080

* refactor: move out posthog

* feat: add experiment tracking

* fix: clear tracked on reset

* fix: fix signup bug

* fix: handle errors when telmetry is disabled

* refactor: remove redundant await

* fix: add back posthog to telemetry

* test: fix test

* test: fix test

* test: add tests for posthog client

* lint: fix

* fix: fix issue with slow decide endpoint

* lint: fix

* lint: fix

* lint: fix

* lint: fix

* chore: address PR feedback

* chore: address PR feedback

* feat: add onboarding experiment
2023-02-21 11:35:35 +03:00
Omar Ajoue 75a094a8c0
fix: Fixes an issue when saving an active workflow without triggers would cause n8n to be stuck (#5513)
fix: Allow saving and editing when an active workflow is saved without triggers
2023-02-20 12:22:27 +01:00
Ahsan Virani db702932f3
feat(core): Add configurable execution history limit (#5505)
* Prune execution data when more than cofnfigured limit

* use stricter typings

* use `pruneDataMaxCount`

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-02-20 10:28:38 +01:00
कारतोफ्फेलस्क्रिप्ट™ 561882f599
refactor(core): Improve instance owner setup and add unit tests (no-changelog) (#5499)
* refactor(core): Avoid fetching all workflows and credentials for the owner setup screen

* refactor(core): Add unit tests for the owner controller
2023-02-17 10:59:09 +01:00
कारतोफ्फेलस्क्रिप्ट™ 12104bc4a3
refactor(core): Delete unused GET /me endpoint (no-changelog) (#5498) 2023-02-17 10:58:48 +01:00
Michael Auerswald d143f3f2ec
feat(core): Add execution runData recovery and status field (#5112)
* adds ExecutionEvents view modal to ExecutionList

* fix time rendering and remove wf column

* checks for unfinished executions and fails them

* prevent re-setting stoppedAt for execution

* some cleanup / manually create rundata after crash

* quicksave

* remove Threads lib, log worker rewrite

* cleanup comment

* fix sentry destination return value

* test for tests...

* run tests with single worker

* fix tests

* remove console log

* add endpoint for execution data recovery

* lint cleanup and some refactoring

* fix accidental recursion

* remove cyclic imports

* add rundata recovery to Workflowrunner

* remove comments

* cleanup and refactor

* adds a status field to executions

* setExecutionStatus on queued worker

* fix onWorkflowPostExecute

* set waiting from worker

* get crashed status into frontend

* remove comment

* merge fix

* cleanup

* catch empty rundata in recovery

* refactor IExecutionsSummary and inject nodeExecution Errors

* reduce default event log size to 10mb from 100mb

* add per node execution status

* lint fix

* merge and lint fix

* phrasing change

* improve preview rendering and messaging

* remove debug

* Improve partial rundata recovery

* fix labels

* fix line through

* send manual rundata to ui at crash

* some type and msg push fixes

* improve recovered item rendering in preview

* update workflowStatistics on recover

* merge fix

* review fixes

* merge fix

* notify eventbus when ui is back up

* add a small timeout to make sure the UI is back up

* increase reconnect timeout to 30s

* adjust recover timeout and ui connection lost msg

* do not stop execution in editor after x reconnects

* add executionRecovered push event

* fix recovered connection not green

* remove reconnect toast and  merge existing rundata

* merge editor and recovered data for own mode
2023-02-17 10:54:07 +01:00
Michael Auerswald 3a9c257f55
feat(core): Add saml feature flag (#5494)
adds saml feature flag
2023-02-16 15:05:39 +01:00
Mutasem Aldmour 3de49e8f78
feat: Update telemetry api endpoints (#5482)
* feat: update PostHog api endpoint

* feat: update rs host

* feat: update rs host

* feat: update rs endpoints

* refactor: use api host for BE events as well
2023-02-16 15:07:08 +03:00
Csaba Tuncsik a2c6ea9e11
fix(core): Remove unnecessary info from GET /workflows response (#5311)
* fix(core): Remove unnecessary info from `GET /workflows` response

* fix(core): Remove unnecessary info from `GET /workflows` response

* fix(core): Remove credentials from `GET /workflows` response

* fix(core): Update unit tests for `GET /workflows` response

* fix(core): Remove `usedCredentials` from `GET /workflows` response

* fix(core): Update unit tests for `GET /workflows` response

* fix(core): remove nodes from getMany

* fix(core): remove unnecessary owner props from workflow list items

* fix(core): fix lint error

* fix(core): remove unused function

* fix(core): simplifying ownerId usage

* fix(core): trim down the query for workflow listing
2023-02-16 10:36:24 +01:00
कारतोफ्फेलस्क्रिप्ट™ a9f08fc5ba
fix(core): Fix issues with community node installation (no-changelog) (#5481)
This fixes the following issues:
* After a community node is installed, we were not calling `postProcessLoaders`, which was causing a bunch of unexpected behaviors, and sometimes even crashes. This was only happening in the session where the package was installed. After a crash, the restarted service was working without these issues.
* After a community node is installed, the icon for the nodes and credentials were missing in the UI, as we were creating one icons route per installed package at startup, and this did not handle newly installed packages. restarting the service fixes this issue as well.

Fixes https://community.n8n.io/t/showing-weird-count-on-community-nodes/23035
2023-02-15 16:09:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ a6c59fcbc2
fix(core): Use stricter typing on queries in cli commands (no-changelog) (#5476)
This has been broken since TypeORM upgrade

Fixes:
https://linear.app/n8n/issue/ENG-50
https://community.n8n.io/t/execute-command-stdout/22994
2023-02-14 17:05:53 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4dc458eca5
fix(core): Fix the issue with test webhooks getting removed incorrectly (no-changelog) (#5466)
This broke because of the change [here](https://github.com/n8n-io/n8n/pull/5443/files#diff-b386248ff00977749c873ed85821c241b773e9740d7e7adf94e05b73b350ed74L152).
`finally` block is called even if there is a `return` in the `try` block, and this is causing the test webhook to be removed, even when it shouldn't be removed.
2023-02-13 16:16:53 +01:00
Jan Oberhauser 339655611f
fix(core): Fix issue that worker and webhook service close directly (#5461) 2023-02-11 09:46:10 -06:00
agobrech 309c995364
test: Change workflows to test from safe list to skip list (#5457)
* Remove check by safe list to skip list

* add an option to skip using statistics events

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-02-10 17:11:06 +01:00
कारतोफ्फेलस्क्रिप्ट™ 9a331ec7b6
fix(core): Stop using unbound calls to stopProcess (no-changelog) (#5456) 2023-02-10 16:34:39 +01:00
Omar Ajoue a2e2ec5442
test: Add tests for ActiveWorkflowRunner class (#5278) 2023-02-10 15:24:20 +01:00
कारतोफ्फेलस्क्रिप्ट™ 538984dc2f
feat(core): Add support for WebSockets as an alternative to Server-Sent Events (#5443)
Co-authored-by: Matthijs Knigge <matthijs@volcano.nl>
2023-02-10 15:02:47 +01:00
कारतोफ्फेलस्क्रिप्ट™ 5194513850
refactor(core): Delete duplicate code across all commands (#5452) 2023-02-10 14:59:20 +01:00
Omar Ajoue 2eb72a6c9f
fix: MySQL migration parses database contents if necessary (fix for MariaDB) (#5441) 2023-02-09 18:15:20 +01:00
OlegIvaniv 018f8a3510
fix(editor): Prevent creation of input connections for nodes without input slot (#5425)
* fix(editor): Prevent creation of input connections for nodes without input

* WIP: Workflow checks service and controller

* fix: Created SQLite migration to remove broken connections

* Cleanup & add mysql/posgres migrations

* Linter fixes

* Unify the migration scripts

* Escape migration workflow_entity

* Wrap the migration in try/catch and do not parse nodes and connection if mysql/postgres

* Do migration changes also fro mysql

* refactor: Wrap only the necessary call in try catch block

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
2023-02-09 16:04:26 +01:00
agobrech fdf47a96de
fix(core): Fix import command for workflows with old format(pre UM) (#5403)
* Replace invalid credentials when importing workflows

* Remove useless console.logs
2023-02-09 15:24:09 +01:00
कारतोफ्फेलस्क्रिप्ट™ 0fb4d7fc19
fix(core): Stop creating icons directories (no-changelog) (#5431)
forgot to remove this in https://github.com/n8n-io/n8n/pull/5419
2023-02-09 11:45:32 +01:00
कारतोफ्फेलस्क्रिप्ट™ 52cb185637
refactor(core): Upgrade typeorm (no-changelog) (#5423)
[Now we can use aggregate functions directly over the repository API](https://github.com/typeorm/typeorm/pull/9737)
2023-02-09 10:44:07 +01:00
Valya 9c1f827dad
feat(core): Live reload node/credential descriptions in development (no-changelog) (#4939) 2023-02-08 19:26:07 +01:00
कारतोफ्फेलस्क्रिप्ट™ f23fb92696
fix(core): Stop copying icons to cache (#5419)
Fixes 

- https://github.com/n8n-io/n8n/issues/4973
- https://github.com/n8n-io/n8n/issues/5274
- https://community.n8n.io/t/starting-n8n-fails-with-ebusy-error/21243
- https://community.n8n.io/t/problem-executing-workflow-ebusy-resource-busy-or-locked-copyfile/21280

Replaces

- https://github.com/n8n-io/n8n/pull/5052
- https://github.com/n8n-io/n8n/pull/5401
2023-02-08 18:57:43 +01:00
Omar Ajoue 5e3e70b83b
feat: Change desktop UM experience (#5312)
* refactor: Hide prompt for desktop

* feat: add email field to personalization modal

* fix: update survey interfaces

* chore: enable personalization survey email key display condition

* feat: add users page upsell for desktop client

* feat: disable UM on desktop where possible

* refactor: Have a single function to decide whether UM is enabled

* feat: update community nodes upsell link

---------

Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: krynble <omar@n8n.io>
Co-authored-by: freyamade <freya@n8n.io>
2023-02-08 11:42:22 +02:00
Omar Ajoue c8245b9f87
fix: Error workflow now correctly checks for subworkflow permissions (#5390) 2023-02-07 17:40:36 +01:00
कारतोफ्फेलस्क्रिप्ट™ 7e2f2f7453
refactor(core): Add unit tests for all external auth middlewares (no-changelog) (#5386) 2023-02-07 15:49:35 +01:00
कारतोफ्फेलस्क्रिप्ट™ 3a435f7057
fix(core): Disable transactions on sqlite migrations that use PRAGMA foreign_keys (#5392)
`PRAGMA foreign_keys` is [a no-op within a sqlite transaction](https://www.sqlite.org/pragma.html#pragma_foreign_keys).
This causes cascade deletes to happen when we create a new user table, and drop the old user table.
2023-02-07 15:14:59 +01:00
Michael Auerswald dd652c0b0d
ci: Add E2E test for scheduled trigger (#5391)
* scheduler e2e test and switch to main mode

* adjust e2e options

* rolling back config changes
2023-02-07 12:41:44 +01:00
कारतोफ्फेलस्क्रिप्ट™ 58b0cf66b5
fix(core): Do not block app startup by telemetry (no-changelog) (#5387) 2023-02-07 12:10:16 +01:00
कारतोफ्फेलस्क्रिप्ट™ a115baa1ac
fix(core): CORS middleware should not process the entire handler chain on OPTIONS requests (no-changelog) (#5368)
fix(core): CORS middleware should not process the entire handler chain on OPTIONS requests
2023-02-06 12:43:50 +01:00
OlegIvaniv 6985500a7d
fix(core): Fix populating of node custom api call options (#5347)
* feat(core): Fix populating of node custom api call options

* lint fixes

* Adress PR comments

* Add e2e test and only inject custom API options for latest version

* Make sure to injectCustomApiCallOption for the latest version of node

* feat(cli): Move apiCallOption injection to LoadNodesAndCredentials and add e2e tests to check for custom nodes credentials

* Load nodes and credentials fixtures from a single place

* Console warning if credential is invalid during customApiOptions injection
2023-02-03 13:14:59 +01:00
कारतोफ्फेलस्क्रिप्ट™ 4dab2fec49
fix(core): Add missing schema keys for LDAP (no-changelog) (#5348) 2023-02-03 12:50:08 +01:00
कारतोफ्फेलस्क्रिप्ट™ 56326cb317
fix(core): Use custom cache control only on editor-ui assets (no-changelog) (#5346) 2023-02-03 11:27:24 +01:00
OlegIvaniv 616074158c
fix(core): Revert custom API option injecting (#5345)
Revert "feat(core): Fix populating of node custom api call options (#5303)"

This reverts commit e58bc41d24.
2023-02-02 20:03:45 +01:00
Omar Ajoue b5154d9be5
fix: Prevent unnecessarily touching updatedAt when n8n starts (#5340)
* Include MariaDB in exception list

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <netroy@users.noreply.github.com>
2023-02-02 17:01:45 +01:00
freya 6ca49f9d54
fix(core): Prevent shared user details being saved alongside execution data (#5334)
* 🔨 - Remove `shared` key from execution save data

* 👕 - Using import type where needed

* remove console.log

* 🔨 - Create new clean workflowData instead of removing shared

If IWorkflowBase changes in future, TS will error out here ensuring it's kept up to date

* 🔨 - use lodash.pick for less verbosity

* 🔨 - fix lodash imports
2023-02-02 15:27:00 +00:00
कारतोफ्फेलस्क्रिप्ट™ 93a2dac063
fix(core): Fix env file config loading (no-changelog) (#5339)
* fix(core): Do not trim config values read from files (no-changelog)

* do not use config.load. it does not work
2023-02-02 15:34:29 +01:00
OlegIvaniv e58bc41d24
feat(core): Fix populating of node custom api call options (#5303)
* feat(core): Fix populating of node custom api call options

* lint fixes

* Adress PR comments

* Add e2e test and only inject custom API options for latest version

* Make sure to injectCustomApiCallOption for the latest version of node
2023-02-02 14:53:08 +01:00