extracted out of #7336
---------
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: Oleg Ivaniv <me@olegivaniv.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
This PR introduces the following changes:
- New Vue stores: `collaborationStore` and `pushConnectionStore`
- Front-end push connection handling overhaul: Keep only a singe
connection open and handle it from the new store
- Add user avatars in the editor header when there are multiple users
working on the same workflow
- Sending a heartbeat event to back-end service periodically to confirm
user is still active
- Back-end overhauls (authored by @tomi):
- Implementing a cleanup procedure that removes inactive users
- Refactoring collaboration service current implementation
---------
Co-authored-by: Tomi Turtiainen <10324676+tomi@users.noreply.github.com>
Followup to #7566 | Story: https://linear.app/n8n/issue/PAY-926
### Manual workflow activation and deactivation
In a multi-main scenario, if the user manually activates or deactivates
a workflow, the process (whether leader or follower) that handles the
PATCH request and updates its internal state should send a message into
the command channel, so that all other main processes update their
internal state accordingly:
- Add to `ActiveWorkflows` if activating
- Remove from `ActiveWorkflows` if deactivating
- Remove and re-add to `ActiveWorkflows` if the update did not change
activation status.
After updating their internal state, if activating or deactivating, the
recipient main processes should push a message to all connected
frontends so that these can update their stores and so reflect the value
in the UI.
### Workflow activation errors
On failure to activate a workflow, the main instance should record the
error in Redis - main instances should always pull activation errors
from Redis in a multi-main scenario.
### Leadership change
On leadership change...
- The old leader should stop pruning and the new leader should start
pruning.
- The old leader should remove trigger- and poller-based workflows and
the new leader should add them.
Github issue / Community forum post (link here to close automatically):
---------
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
This change expands on the command channel communication introduced
lately between the main instance(s) and the workers. The frontend gets a
new menu entry "Workers" which will, when opened, trigger a regular call
to getStatus from the workers. The workers then respond via their
response channel to the backend, which then pushes the status to the
frontend.
This introduces the use of ChartJS for metrics.
This feature is still in MVP state and thus disabled by default for the
moment.
useStorage takes the default value `undefined` and sets it in local
storage.. also returns "undefined" as string which breaks onboarding
flows
Github issue / Community forum post (link here to close automatically):
extracted out of #7336
---------
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: OlegIvaniv <me@olegivaniv.com>
Co-authored-by: Jan Oberhauser <janober@users.noreply.github.com>
Co-authored-by: Val <68596159+valya@users.noreply.github.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Deborah <deborah@starfallprojects.co.uk>
Co-authored-by: Jesper Bylund <mail@jesperbylund.com>
Co-authored-by: Jon <jonathan.bennetts@gmail.com>
Github issue / Community forum post (link here to close automatically):
This PR aims to address an issue where an Error workflow cannot be
started, either due to insufficient permissions or because its settings
prevent it from being called.
The way of addressing this is by creating a failed execution for the
appointed error workflow stating the error, as can be seen below.
This means the execution itself won't start, as it's prevented before
the execution beings, but we save a "stub" execution to show the error.
* initial telemetry setup and adjusted pull return
* quicksave before merge
* feat: add conflicting workflow list to pull modal
* feat: update source control pull modal
* fix: fix linting issue
* feat: add Enter keydown event for submitting source control push modal (no-changelog)
feat: add Enter keydown event for submitting source control push modal
* quicksave
* user workflow table for export
* improve telemetry data
* pull api telemetry
* fix lint
* Copy tweaks.
* remove authorName and authorEmail and pick from user
* rename owners.json to workflow_owners.json
* ignore credential conflicts on pull
* feat: several push/pull flow changes and design update
* pull and push return same data format
* fix: add One last step toast for successful pull
* feat: add up to date pull toast
* fix: add proper Learn more link for push and pull modals
* do not await tracking being sent
* fix import
* fix await
* add more sourcecontrolfile status
* Minor copy tweak for "More info".
* Minor copy tweak for "More info".
* ignore variable_stub conflicts on pull
* ignore whitespace differences
* do not show remote workflows that are not yet created
* fix telemetry
* fix toast when pulling deleted wf
* lint fix
* refactor and make some imports dynamic
* fix variable edit validation
* fix telemetry response
* improve telemetry
* fix unintenional delete commit
* fix status unknown issue
* fix up to date toast
* do not export active state and reapply versionid
* use update instead of upsert
* fix: show all workflows when clicking push to git
* feat: update Up to date pull translation
* fix: update read only env checks
* do not update versionid of only active flag changes
* feat: prevent access to new workflow and templates import when read only env
* feat: send only active state and version if workflow state is not dirty
* fix: Detect when only active state has changed and prevent generation a new version ID
* feat: improve readonly env messages
* make getPreferences public
* fix telemetry issue
* fix: add partial workflow update based on dirty state when changing active state
* update unit tests
* fix: remove unsaved changes check in readOnlyEnv
* fix: disable push to git button when read onyl env
* fix: update readonly toast duration
* fix: fix pinning and title input in protected mode
* initial commit (NOT working)
* working push
* cleanup and implement pull
* fix getstatus
* update import to new method
* var and tag diffs are no conflicts
* only show pull conflict for workflows
* refactor and ignore faulty credentials
* add sanitycheck for missing git folder
* prefer fetch over pull and limit depth to 1
* back to pull...
* fix setting branch on initial connect
* fix test
* remove clean workfolder
* refactor: Remove some unnecessary code
* Fixed links to docs.
* fix getstatus query params
* lint fix
* dialog to show local and remote name on conflict
* only show remote name on conflict
* fix credential expression export
* fix: Broken test
* dont show toast on pull with empty var/tags and refactor
* apply frontend changes from old branch
* fix tag with same name import
* fix buttons shown for non instance owners
* prepare local storage key for removal
* refactor: Change wording on pushing and pulling
* refactor: Change menu item
* test: Fix broken test
* Update packages/cli/src/environments/sourceControl/types/sourceControlPushWorkFolder.ts
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
---------
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
* Draft setup
* ⚡ Implemented expression evaluation in Postgres node, minor SQL editor UI improvements, minor refacring
* ⚡ Added initial version of expression preview for SQL editor
* ⚡ Linking npm package for codemirror sql grammar instead of a local file
* ⚡ Moving expression editor wrapper elements to the component
* ⚡ Using expression preview in SQL editor
* Use SQL parser skipping whitespace
* ✨ Added support for custom skipped segments specification
* ✨ Fixing highlight problems with dots and expressions that resolve to zero
* 👕 Fixing linting error
* ✨ Added current item support
* ⚡ Added expression support to more nodes with sql editor
* ✨ Added expression support for other nodes
* ✨ Implemented different SQL dialect support
* 🐛 Fixing hard-coded parameter names for editors
* ✨ Fixing preview for nested queries, updating query when input data changes, adding keyboard shortcut to toggle comments
* ✨ Adding a custom automcomplete notice for different editors
* ⚡ Updating SQL autocomplete notice
* ✅ Added unit tests for SQL editor
* ⚡ Using latest grammar
* 🐛 Fixing code node editor rendering
* 💄 SQL preview dropdown matches editor width. Removing unnecessary css
* ⚡ Addressing PR review feedback
* 👌 Addressing PR review feedback pt2
* 👌 Added path alias for utils in nodes-base package
* 👌 Addressing more PR review feedback
* ✅ Adding tests for `getResolvables` utility function
* ⚡Fixing lodash imports
* 👌 Better focus handling, adding more plugins to the editor, other minor imrovements
* ⚡ Not showing SQL autocomplete suggestions inside expressions
* ⚡ Using npm package for sql grammar
* ⚡ Removing autocomplete notice, adding line highlight on syntax error
* 👌 Addressing code review feedback
---------
Co-authored-by: Milorad Filipovic <milorad@n8n.io>
* ✨ Removing the `makeApiRequest` method from `restAPI` mixin, removing the mixing from the App component
* ✨ Removing `restApi` mixin
* 👕 Fixing lint errors
* ✔️ Fixing execution list unit tests and merge bug in workflowRun mixin
* 🐛 Added missing useStore
* refactor(editor): Refactor history and debounce mixins to composables and add unit tests (no-changelog)
* Lint fix and use userEvent to fire keydown events
* Fix debounce spec
* refactor: replace new Vue() with custom event bus (no-changelog)
* fix: export types from design system main
* fix: update component types
* fix: update form inputs event bus
* refactor(editor): Migrate mouseSelect & deviceSupportHelpers mixins to composables (no-changelog)
* Fix node drop position and correct event listeners
* adds ExecutionEvents view modal to ExecutionList
* fix time rendering and remove wf column
* checks for unfinished executions and fails them
* prevent re-setting stoppedAt for execution
* some cleanup / manually create rundata after crash
* quicksave
* remove Threads lib, log worker rewrite
* cleanup comment
* fix sentry destination return value
* test for tests...
* run tests with single worker
* fix tests
* remove console log
* add endpoint for execution data recovery
* lint cleanup and some refactoring
* fix accidental recursion
* remove cyclic imports
* add rundata recovery to Workflowrunner
* remove comments
* cleanup and refactor
* adds a status field to executions
* setExecutionStatus on queued worker
* fix onWorkflowPostExecute
* set waiting from worker
* get crashed status into frontend
* remove comment
* merge fix
* cleanup
* catch empty rundata in recovery
* refactor IExecutionsSummary and inject nodeExecution Errors
* reduce default event log size to 10mb from 100mb
* add per node execution status
* lint fix
* merge and lint fix
* phrasing change
* improve preview rendering and messaging
* remove debug
* Improve partial rundata recovery
* fix labels
* fix line through
* send manual rundata to ui at crash
* some type and msg push fixes
* improve recovered item rendering in preview
* update workflowStatistics on recover
* merge fix
* review fixes
* merge fix
* notify eventbus when ui is back up
* add a small timeout to make sure the UI is back up
* increase reconnect timeout to 30s
* adjust recover timeout and ui connection lost msg
* do not stop execution in editor after x reconnects
* add executionRecovered push event
* fix recovered connection not green
* remove reconnect toast and merge existing rundata
* merge editor and recovered data for own mode
* refactor(editor): Fix duplicate NodeView keys when navigating between routes (no-changelog)
* Prettier fixes
* Use computed to export jsPlumb instance from canvas
* Force jsPlumb computed instance type
* ⚡ Removing authentication parameter from NDV
* ⚡ Added auth type selector to credentials modal
* 🔨 Extracting reusable logic to util functions
* ⚡ Updating credentials position, adding label for radio buttons
* ⚡ Using first node credentials for nodes with single auth options and hiding auth selector UI in that case
* ⚡ Fixing credentials modal when opened from credentials page
* ⚡ Showing all available credentials in NDV credentials dropdown
* ⚡ Updating node credentials dropdown component to show credentials description. Disabling `Credentials of type not found` error in node
* ⚡ Moving auth related fields from NDV to credentials modal. Added support for multiple auth fileds
* ⚡ Moving NDV fields that authentication depends on to credentials modal
* ⚡ Keeping old auth/credentials UI in NDV for HTTP Request and Webhook nodes. Pre-populating credential type for HTTP request node when selected from 'app action' menu
* 💄 Use old label and field position for nodes that use old credentials UI in NDV
* ⚡ Implementing more generic way to find node's auth fileds
* 📚 Adding comments on parameter hiding logic
* ⚡ Fixing node auth options logic for multiple auth fields
* 👕 Fixing lint errors
* 💄 Addressing design review comments
* ⚡ Not selecting first auth option when opening new credential dialog
* ⚡ Using default credentials name and icon if authentication type is not selected
* ⚡ Updating credential data when auth type is changed
* ⚡ Setting new credentials type for HTTP Request and Webhook nodes
* ⚡ Setting nodes with access when changing auth type
* 👕 Fixing lint error
* ⚡ Updating active node auth type from credentials modal
* ⚡ Syncronizing credentials modal and dropdown
* 👕 Fixing linter error
* ⚡ Handling credential dropdown UI for multiple credentials
* 👕 Removing unused imports
* ⚡ Handling auth selection when default auth type is the first option
* ⚡ Updating credentials change listening logic
* ⚡ Resetting credential data when deleting a credential, disabling 'Details' and 'Sharing' tabs if auth type is not selected
* 🐛 Skipping credentials type check when showing mixed credentials in the dropdown and switching credentials type
* ⚡ Showing credential modal tabs for saved credentials
* ⚡ Preventing renaming credentials when no auth type is selected
* 🐛 Fixing credentials modal when opened from credentials page
* ⚡ Keeping auth radio buttons selected when switching tabs
* ✅ Adding initial batch of credentials NDV tests
* ⚡ Updating node auth filed value when new credential type is selected
* ⚡ Using all available credential types for current node to sync credential dropdown with modal
* ⚡ Sorting mixed credentials by date, simplifying credential dropdown option logic
* 🔨 Extracting some reusable logic to utils
* ⚡ Improving required vs optional credentials detection and using it to show auth radio buttons
* 👕 Fixing lint errors
* ✅ Adding more credentials tests
* ⚡ Filtering credential options based on authentication type
* 🔨 Refactoring credentials and auth utils
* ⚡ Updated handling of auth options in credentials modal to work with new logic
* 🔨 Getting the terminology in line
* 📚 Removing leftover comment
* ⚡ Updating node auth filed detection logic to account for different edge-cases
* ⚡ Adding Wait node as an exception for new UI
* ⚡ Updating NDV display when auth type changes
* ⚡ Updating default credentials name when auth type changes
* ⚡ Hiding auth settings after credentials are saved
* ⚡ Always showing credentials modal menu tabs
* ⚡ Improving main auth field detection logic so it doesn't account for authentication fields which can have `none` value
* ⚡ Restoring accidentally deleted not existing credential issue logic
* ⚡ Updating other nodes when deleted credentials have been updated
* ⚡ Using filtered auth type list to show or hide radio buttons section in credentials modal
* 👕 Addressing lint error
* 👌 Addressing PR review feedback
* 👕 Fixing lint issues
* ⚡ Updating main auth filed detection logic so it checks full dependency path to determine if the field is required or optional
* 👌 Addressing the rest of PR feedback
* ✅ Updating credential tests
* ⚡ Resetting credential data on authentication type change
* ⚡ Created AuthTypeSelector component
* 👌 Addressing PR comments
* ⚡ Not resetting overwritten credential properties when changing auth type
* ⚡ Hiding auth selector section if there are no options to show
