Depends on https://github.com/n8n-io/n8n/pull/7220 | Story:
[PAY-840](https://linear.app/n8n/issue/PAY-840/introduce-object-store-service-and-manager-for-binary-data)
This PR introduces an object store service for Enterprise edition. Note
that the service is tested but currently unused - it will be integrated
soon as a binary data manager, and later for execution data.
`amazonaws.com` in the host is temporarily hardcoded until we integrate
the service and test against AWS, Cloudflare and Backblaze, in the next
PR.
This is ready for review - the PR it depends on is approved and waiting
for CI.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)
This PR:
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.
Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.
Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Based on #7065 | Story: https://linear.app/n8n/issue/PAY-771
n8n on filesystem mode marks binary data to delete on manual execution
deletion, on unsaved execution completion, and on every execution
pruning cycle. We later prune binary data in a separate cycle via these
marker files, based on the configured TTL. In the context of introducing
an S3 client to manage binary data, the filesystem mode's mark-and-prune
setup is too tightly coupled to the general binary data management
client interface.
This PR...
- Ensures the deletion of an execution causes the deletion of any binary
data associated to it. This does away with the need for binary data TTL
and simplifies the filesystem mode's mark-and-prune setup.
- Refactors all execution deletions (including pruning) to cause soft
deletions, hard-deletes soft-deleted executions based on the existing
pruning config, and adjusts execution endpoints to filter out
soft-deleted executions. This reduces DB load, and keeps binary data
around long enough for users to access it when building workflows with
unsaved executions.
- Moves all execution pruning work from an execution lifecycle hook to
`execution.repository.ts`. This keeps related logic in a single place.
- Removes all marking logic from the binary data manager. This
simplifies the interface that the S3 client will meet.
- Adds basic sanity-check tests to pruning logic and execution deletion.
Out of scope:
- Improving existing pruning logic.
- Improving existing execution repository logic.
- Adjusting dir structure for filesystem mode.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
Github issue / Community forum post (link here to close automatically):
https://github.com/n8n-io/n8n/pull/6348
---------
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
Co-authored-by: Marcus <marcus@n8n.io>
- For a saved execution, we write to disk binary data and metadata.
These two are only ever deleted via `POST /executions/delete`. No marker
file, so untouched by pruning.
- For an unsaved execution, we write to disk binary data, binary data
metadata, and a marker file at `/meta`. We later delete all three during
pruning.
- The third flow is legacy. Currently, if the execution is unsaved, we
actually store it in the DB while running the workflow and immediately
after the workflow is finished during the `onWorkflowPostExecute()` hook
we delete that execution, so the second flow applies. But formerly, we
did not store unsaved executions in the DB ("ephemeral executions") and
so we needed to write a marker file at `/persistMeta` so that, if the
ephemeral execution crashed after the step where binary data was stored,
we had a way to later delete its associated dangling binary data via a
second pruning cycle, and if the ephemeral execution succeeded, then we
immediately cleaned up the marker file at `/persistMeta` during the
`onWorkflowPostExecute()` hook.
This creation and cleanup at `/persistMeta` is still happening, but this
third flow no longer has a purpose, as we now store unsaved executions
in the DB and delete them immediately after. Hence the third flow can be
removed.
* fix(core): Make node execution order configurable, and backward-compatible
* ⚡ Also add new Merge-Node behaviour
* ⚡ Fix typo
* Fix lint issue
* update labels
* rename legacy to v0
* remove the unnecessary log
* default all new workflows to use v1 execution-order
* remove the controller changes
* clone default settings to avoid it getting modified
---------
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* feat(core): Change node execution order (most top-left one first)
* ⚡ Fix issue with multi-output-nodes
* ⚡ Remove not needed meta-entry in test
* fix the e2e test
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
* 🔧 Adjust base ESLint config
* 🔧 Adjust `lint` and `lintfix` in `nodes-base`
* 🔧 Include `test` and `utils` in `nodes-base`
* 📘 Convert JS tests to TS
* 👕 Apply lintfixes
* fix(core): Validate customData keys and values
Throws errors in manual mode and ignores and logs values in production
* fix: validate customData key characters
* refactor: review changes
* fix: logger not initialised for metadata tests
* fix: allow numbers for values
* wip: workflow execution filtering
* fix: import type failing to build
* fix: remove console.logs
* feat: execution metadata migrations
* fix(editor): Move global executions filter to its own component
* fix(editor): Using the same filter component in workflow level
* fix(editor): a small housekeeping
* checking workflowId in filter applied
* fix(editor): update filter after resolving merge conflicts
* fix(editor): unify empy filter status
* feat(editor): add datetime picker to filter
* feat(editor): add meta fields
* fix: fix button override in datepicker panel
* feat(editor): add filter metadata
* feat(core): add 'startedBefore' execution filter prop
* feat(core): add 'tags' execution query filter
* Revert "feat(core): add 'tags' execution query filter"
This reverts commit a7b968081c.
* feat(editor): add translations and tooltip and counting selected filter props
* fix(editor): fix label layouts
* fix(editor): update custom data docs link
* fix(editor): update custom data tooltip position
* fix(editor): update tooltip text
* refactor: Ignore metadata if not enabled by license
* fix(editor): Add paywall states to advanced execution filter
* refactor: Save custom data also for worker mode
* fix: Remove duplicate migration name from list
* fix(editor): Reducing filter complexity and add debounce to text inputs
* fix(editor): Remove unused import, add comment
* fix(editor): simplify event listener
* fix: Prevent error when there are running executions
* test(editor): Add advanced execution filter basic unit test
* test(editor): Add advanced execution filter state change unit test
* fix: Small lint issue
* feat: Add indices to speed up queries
* feat: add customData limits
* refactor: put metadata save in transaction
* chore: remove unneed comment
* test: add tests for execution metadata
* fix(editor): Fixes after merge conflict
* fix(editor): Remove unused import
* wordings and ui fixes
* fix(editor): type fixes
* feat: add code node autocompletions for customData
* fix: Prevent transaction issues and ambiguous ID in sql clauses
* fix(editor): Suppress requesting current executions if metadata is used in filter (#5739)
* fix(editor): Suppress requesting current executions if metadata is used in filter
* fix(editor): Fix arrows for select in popover
* refactor: Improve performance by correcting database indices
* fix: Lint issue
* test: Fix broken test
* fix: Broken test
* test: add call data check for saveExecutionMetadata test
---------
Co-authored-by: Valya Bullions <valya@n8n.io>
Co-authored-by: Alex Grozav <alex@grozav.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
* 📘 Clear all `@ts-ignore` comments from workflow package
* 👕 Default to error with package-level overrides
* refactor(core): clear all `@ts-ignore` comments from core package (#4473)
👕 Clear all `@ts-ignore` comments from core package
* ✏️ Update comment
* Improve code health
Fix TS typos in local variables
Fix CSS typos in local styles
Fix typos in comments
Fix typos in strings
* Fix order of n8n setup sections in CONTRIBUTING.md
* ✨ Introduce simple node versioning
* ⚡ Add example how to read version in node-code for custom logic
* 🐛 Fix setting of parameters
* 🐛 Fix another instance where it sets the wrong parameter
* ⚡ Remove unnecessary TOODs
* ⚡ Revert Set Node example changes
* ;rotating_light: Add test
* ✨ change FE to handle new object type
* 🚸 improve UX of handling invalid credentials
* 🚧 WIP
* 🎨 fix typescript issues
* 🐘 add migrations for all supported dbs
* ✏️ add description to migrations
* ⚡ add credential update on import
* ⚡ resolve after merge issues
* 👕 fix lint issues
* ⚡ check credentials on workflow create/update
* update interface
* 👕 fix ts issues
* ⚡ adaption to new credentials UI
* 🐛 intialize cache on BE for credentials check
* 🐛 fix undefined oldCredentials
* 🐛 fix deleting credential
* 🐛 fix check for undefined keys
* 🐛 fix disabling edit in execution
* 🎨 just show credential name on execution view
* ✏️ remove TODO
* ⚡ implement review suggestions
* ⚡ add cache to getCredentialsByType
* ⏪ use getter instead of cache
* ✏️ fix variable name typo
* 🐘 include waiting nodes to migrations
* 🐛 fix reverting migrations command
* ⚡ update typeorm command
* ✨ create db:revert command
* 👕 fix lint error
* ✨ Add optional authenticate method to credentials
* ⚡ Simplify code and add authentication support to MattermostApi
* 👕 Fix lint issue
* ⚡ Add support to own-mode
* 👕 Fix lint issue
* ✨ Add support for predefined auth types bearer and headerAuth
* ⚡ Make sure that DateTime Node always returns strings
* ⚡ Add support for moment types to If Node
* ⚡ Make it possible for HTTP Request Node to use all credential types
* ✨ Add basicAuth support
* Add a new dropcontact node
* ✨ First basic implementation of mainly JSON based nodes
* ✨ Add fixedCollection support, added value parameter and
expression support for value and property
* Improvements to #2389
* ⚡ Add credentials verification
* ⚡ Small improvement
* ⚡ set default time to 45 seconds
* ✨ Add support for preSend and postReceive methods
* ➕ Add lodash merge and set depedency to workflow
* 👕 Fix lint issue
* ⚡ Improvements
* ⚡ Improvements
* ⚡ Improvements
* ⚡ Improvements
* ⚡ Improvements
* 🐛 Set siren and language correctly
* ⚡ Add support for requestDefaults
* ⚡ Add support for baseURL to httpRequest
* ⚡ Move baseURL to correct location
* ✨ Add support for options loading
* 🐛 Fix error with fullAccess nodes
* ✨ Add credential test functionality
* 🐛 Fix issue with OAuth autentication and lint issue
* ⚡ Fix build issue
* 🐛 Fix issue that url got always overwritten to empty
* ✨ Add pagination support
* ⚡ Code fix required after merge
* ⚡ Remove not needed imports
* ⚡ Fix credential test
* ✨ Add expression support for request properties and $self
support on properties
* ⚡ Rename $self to $value
* 👕 Fix lint issue
* ⚡ Add example how to send data in path
* ✨ Make it possible to not sent in dot notation
* ✨ Add support for postReceive:rootProperty
* ⚡ Fix typo
* ✨ Add support for postReceive:set
* ⚡ Some fixes
* ⚡ Small improvement
* ;zap: Separate RoutingNode code
* ⚡ Simplify code and fix bug
* ⚡ Remove unused code
* ✨ Make it possible to define "request" and "requestProperty" on
options
* 👕 Fix lint issue
* ⚡ Change $credentials variables name
* ✨ Enable expressions and access to credentials in requestDefaults
* ⚡ Make parameter option loading use RoutingNode.makeRoutingRequest
* ✨ Allow requestOperations overwrite on LoadOptions
* ✨ Make it possible to access current node parameters in loadOptions
* ⚡ Rename parameters variable to make future proof
* ⚡ Make it possible to use offset-pagination with body
* ✨ Add support for queryAuth
* ⚡ Never return more items than requested
* ✨ Make it possible to overwrite requestOperations on parameter
and option level
* 👕 Fix lint issue
* ✨ Allow simplified auth also with regular nodes
* ✨ Add support for receiving binary data
* 🐛 Fix example node
* ⚡ Rename property "name" to "displayName" in loadOptions
* ⚡ Send data by default as "query" if nothing is set
* ⚡ Rename $self to $parent
* ⚡ Change to work with INodeExecutionData instead of IDataObject
* ⚡ Improve binaryData handling
* ⚡ Property design improvements
* ⚡ Fix property name
* 🚨 Add some tests
* ⚡ Add also test for request
* ⚡ Improve test and fix issues
* ⚡ Improvements to loadOptions
* ⚡ Normalize loadOptions with rest of code
* ⚡ Add info text
* ✨ Add support for $value in postReceive
* 🚨 Add tests for RoutingNode.runNode
* ⚡ Remove TODOs and make url property optional
* ⚡ Fix bug and lint issue
* 🐛 Fix bug that not the correct property got used
* 🚨 Add tests for CredentialsHelper.authenticate
* ⚡ Improve code and resolve expressions also everywhere for
loadOptions and credential test requests
* ✨ Make it possible to define multiple preSend and postReceive
actions
* ✨ Allow to define tests on credentials
* ⚡ Remove test data
* ⬆️ Update package-lock.json file
* ⚡ Remove old not longer used code
Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
Co-authored-by: Mutasem <mutdmour@gmail.com>
Co-authored-by: PaulineDropcontact <pauline@dropcontact.io>
Co-authored-by: ricardo <ricardoespinoza105@gmail.com>
* ✨ Add Webhook-Response-Node
* ⚡ Replace callback function with promise
* ✨ Add support for Bull and binary-data
* ✨ Add string response option
* ⚡ Remove some comments
* ✨ Make more generically possible & fix issue multi call in
queue mode
* ⚡ Fix startup and eslint issues
* ⚡ Improvements to webhook response node and functionality
* ⚡ Replace data with more generic type
* ⚡ Make statusMessage optional
* ⚡ Change parameter order
* ⚡ Move Response Code underneath options
* ⚡ Hide Response Code on Webhook node if mode responseNode got selected
* ⚡ Minor improvements
* ⚡ Add missing file and fix lint issue
* ⚡ Fix some node linting issues
* ⚡ Apply feedback
* ⚡ Minor improvements
* ⚡ introduce versioned nodes
* Export versioned nodes for separate process run
* Add bse node for versioned nodes
* fix node name for versioned nodes
* extend node from nodeVersionedType
* improve nodes base and flow to FE
* revert lib es2019 to es2017
* include version in key to prevent duplicate key
* handle type versions on FE
* clean up
* cleanup nodes base
* add type versions in getNodeParameterOptions
* cleanup
* code review
* code review + add default version to node type description
* remove node default types from store
* 💄 cleanups
* Draft for migrated Mattermost node
* First version of Mattermost node versioned according to node standards
* Correcting deactivate operations name to match currently used one
* ✨ Create utility types
* ⚡ Simplify Mattermost types
* ⚡ Rename exports for consistency
* ⚡ Type channel properties
* ⚡ Type message properties
* ⚡ Type reaction properties
* ⚡ Type user properties
* ⚡ Add type import to router
* 🐛 Add missing key
* 🔨 Adjust typo in operation name
* 🔨 Inline exports for channel properties
* 🔨 Inline exports for message properties
* 🔨 Inline exports for reaction properties
* 🔨 Inline exports for user properties
* 🔨 Inline exports for load options
* 👕 Fix lint issue
* 🔨 Inline export for description
* 🔨 Rename descriptions for clarity
* 🔨 Refactor imports/exports for methods
* 🔨 Refactor latest version retrieval
* 🔥 Remove unneeded else clause
When the string literal union is exhausted, the resource key becomes never, so TS disallows wrong key usage.
* ✨ Add overloads to getNodeParameter
* ⚡ Improve overload
* 🔥 Remove superfluous INodeVersions type
* 🔨 Relocate pre-existing interface
* 🔥 Remove JSDoc arg descriptions
* ⚡ Minor reformatting in transport file
* ⚡ Fix API call function type
* Created first draft for Axios requests
* Working version of mattermost node with Axios
* Work in progress for replacing request library
* Improvements to request translations
* Fixed sending files via multipart / form-data
* Fixing translation from request to axios and loading node parameter options
* Improved typing for new http helper
* Added ignore any for specific lines for linting
* Fixed follow redirects changes on http request node and manual execution of previously existing workflow with older node versions
* Adding default headers according to body on httpRequest helper
* Spec error handling and fixed workflows with older node versions
* Showcase how to export errors in a standard format
* Merging master
* Refactored mattermost node to keep files in a uniform structure. Also fix bugs with merges
* Reverting changes to http request node
* Changed nullish comparison and removed repeated code from nodes
* Renamed queryString back to qs and simplified node output
* Simplified some comparisons
* Changed header names to be uc first
* Added default user agent to requests and patch http method support
* Fixed indentation, remove unnecessary file and console log
* Fixed mattermost node name
* Fixed lint issues
* Further fix linting issues
* Further fix lint issues
* Fixed http request helper's return type
Co-authored-by: ahsan-virani <ahsan.virani@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
* refactor saving
* refactor api layer to be stateless
* refactor header details
* set variable for menu height
* clean up scss
* clean up indentation
* clean up dropdown impl
* refactor no tags view
* split away header
* Fix tslint issues
* Refactor tag manager
* add tags to patch request
* clean up scss
* ⚡ Refactor types to entities
* fix issues
* update no workflow error
* clean up tagscontainer
* use getters instead of state
* remove imports
* use custom colors
* clean up tags container
* clean up dropdown
* clean up focusoncreate
* ⚡ Ignore mistaken ID in POST /workflows
* ⚡ Fix undefined tag ID in PATCH /workflows
* ⚡ Shorten response for POST /tags
* remove scss mixins
* clean up imports
* ⚡ Implement validation with class-validator
* address ivan's comments
* implement modals
* Fix lint issues
* fix disabling shortcuts
* fix focus issues
* fix focus issues
* fix focus issues with modal
* fix linting issues
* use dispatch
* use constants for modal keys
* fix focus
* fix lint issues
* remove unused prop
* add modal root
* fix lint issues
* remove unused methods
* fix shortcut
* remove max width
* ⚡ Fix duplicate entry error for pg and MySQL
* update rename messaging
* update order of buttons
* fix firefox overflow on windows
* fix dropdown height
* 🔨 refactor tag crud controllers
* 🧹 remove unused imports
* use variable for number of items
* fix dropdown spacing
* ⚡ Restore type to fix build
* ⚡ Fix post-refactor PATCH /workflows/:id
* ⚡ Fix PATCH /workflows/:id for zero tags
* ⚡ Fix usage count becoming stringified
* address max's comments
* fix filter spacing
* fix blur bug
* address most of ivan's comments
* address tags type concern
* remove defaults
* ⚡ return tag id as string
* 🔨 add hooks to tag CUD operations
* 🏎 simplify timestamp pruning
* remove blur event
* fix onblur bug
* ⚡ Fix fs import to fix build
* address max's comments
* implement responsive tag container
* fix lint issues
* update tag limits
* address ivan's comments
* remove rename, refactor header, implement new designs for save, remove responsive tag container
* update styling
* update styling
* implement responsive tag container
* implement header tags edit
* implement header tags edit
* fix lint issues
* implement expandable input
* minor fixes
* minor fixes
* use variable
* rename save as
* duplicate fixes
* minor edit fixes
* lint fixes
* style fixes
* hook up saving name
* hook up tags
* clean up impl
* fix dirty state bug
* update limit
* update notification messages
* on click outside
* fix minor bug with count
* lint fixes
* handle minor edge cases
* handle minor edge cases
* handle minor bugs; fix firefox dropdown issue
* Fix min width
* apply tags only after api success
* remove count fix
* clean up workflow tags impl, fix tags delete bug
* fix minor issue
* fix minor spacing issue
* disable wrap for ops
* fix viewport root; save on click in dropdown
* save button loading when saving name/tags
* implement max width on tags container
* implement cleaner create experience
* disable edit while updating
* codacy hex color
* refactor tags container
* fix clickability
* fix workflow open and count
* clean up structure
* fix up lint issues
* fix button size
* increase workflow name limit for larger screen
* tslint fixes
* disable responsiveness for workflow modal
* rename event
* change min width for tags
* clean up pr
* address max's comments on styles
* remove success toasts
* add hover mode to name
* minor fixes
* refactor name preview
* fix name input not to jiggle
* finish up name input
* Fix up add tags
* clean up param
* clean up scss
* fix resizing name
* fix resizing name
* fix resize bug
* clean up edit spacing
* ignore on esc
* fix input bug
* focus input on clear
* build
* fix up add tags clickablity
* remove scrollbars
* move into folders
* clean up multiple patch req
* remove padding top from edit
* update tags on enter
* build
* rollout blur on enter behavior
* rollout esc behavior
* fix tags bug when duplicating tags
* move key to reload tags
* update header spacing
* build
* update hex case
* refactor workflow title
* remove unusued prop
* keep focus on error, fix bug on error
* Fix bug with name / tags toggle on error
* fix connection push bug
* :spakles: Implement wait functionality
* 🐛 Do not delete waiting executions with prune
* ⚡ Improve SQLite migration to not lose execution data anymore
* ⚡ Make it possible to restart waiting execution via webhook
* ⚡ Add missing file
* 🐛 Some more merge fixes
* ⚡ Do not show error for Wait-Nodes if in time-mode
* ⚡ Make $executionId available in expressions
* 👕 Fix lint issue
* 👕 Fix lint issue
* 👕 Fix lint issue
* ⚡ Set the unlimited sleep time as a variable
* ⚡ Add also sleeping webhook path to config
* ⚡ Make it possible to retrieve restartUrl in workflow
* ⚡ Add authentication to Wait-Node in Webhook-Mode
* ⚡ Return 404 when trying to restart execution via webhook which does
not support it
* ✨ Make it possible to set absolute time on Wait-Node
* ⚡ Remove not needed imports
* ⚡ Fix description format
* ✨ Implement missing webhook features on Wait-Node
* ⚡ Display webhook variable in NodeWebhooks
* ⚡ Include also date in displayed sleep time
* ⚡ Make it possible to see sleep time on node
* ⚡ Make sure that no executions does get executed twice
* ⚡ Add comment
* ⚡ Further improvements
* ⚡ Make Wait-Node easier to use
* ✨ Add support for "notice" parameter type
* Fixing wait node to work with queue, improved logging and execution view
* Added support for mysql and pg
* ✨ Add support for webhook postfix path
* ✨ Make it possible to stop sleeping executions
* ⚡ Fix issue with webhook paths in not webhook mode
* ⚡ Remove not needed console.log
* ⚡ Update TODOs
* ⚡ Increase min time of workflow staying active to descrease possible issue
with overlap
* 👕 Fix lint issue
* 🐛 Fix issues with webhooks
* ⚡ Make error message clearer
* ⚡ Fix issue with missing execution ID in scaling mode
* Fixed execution list to correctly display waiting executins
* Feature: enable webhook wait workflows to continue after specified time
* Fixed linting
* ⚡ Improve waiting description text
* ⚡ Fix parameter display issue and rename
* ⚡ Remove comment
* ⚡ Do not display webhooks on Wait-Node
* Changed wording from restart to resume on wait node
* Fixed wording and inconsistent screen when changing resume modes
* Removed dots from the descriptions
* Changed docs url and renaming postfix to suffix
* Changed names from sleep to wait
* ⚡ Apply suggestions from ben
Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>
* Some fixes by Ben
* ⚡ Remove console.logs
* ⚡ Fixes and improvements
Co-authored-by: Mutasem <mutdmour@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>
Co-authored-by: Omar Ajoue <krynble@gmail.com>
* Changes to types so that credentials can be always loaded from DB
This first commit changes all return types from the execute functions
and calls to get credentials to be async so we can use await.
This is a first step as previously credentials were loaded in memory and
always available. We will now be loading them from the DB which requires
turning the whole call chain async.
* Fix updated files
* Removed unnecessary credential loading to improve performance
* Fix typo
* ⚡ Fix issue
* Updated new nodes to load credentials async
* ⚡ Remove not needed comment
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* introduce binary data management
* cr
* add binary data changes to awsS3 node
* add binary data changes to Box node
* add binary data changes to CiscoWebex node
* add binary data changes to HumaniticAi node
* add binary data changes to Jira node
* add binary data changes to Line node
* add binary data changes to MicrosoftOneDrive node
* add binary data changes to MicrosoftOutlook node
* add binary data changes to Mindee node
* add binary data changes to NocoDB node
* add binary data changes to Pushbullet node
* add binary data changes to Pushover node
* add binary data changes to Raindrop node
* add binary data changes to S3 node
* add binary data changes to Salesforce node
* add binary data changes to Ssh node
* add binary data changes to TheHive node
* add binary data changes to Twist node
* add binary data changes to Twitter node
* remove changes not needed right now
* 🐛 Fix issue with multiple runs
* 🐛 Revert fix and add support for multiple inputs
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* ✨ Send console.log messages to editor-UI
* ⚡ Send message only to session which started workflow
* ⚡ Made it also work in own process
* ⚡ Add support for console.log UI forward also to FunctionItem Node
* 👕 Fix lint issue
* 👕 Fix linting issue
* ⚡ Improve code
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
* Added logging to n8n
This commit adds logging to n8n using the Winston library.
For now, this commit only allows logging to console (default behavior)
or file (need to pass in config via environment variables).
Other logging methods can be further implemented using hooks. These were
skipped for now as it would require adding more dependencies.
Logging level is notice by default, meaning no additional messages would
be displayed at the moment. Logging level can be set to info or debug as
well to enrich the generated logs.
The ILogger interface was added to the workflow project as it would make
it available for all other projects but the implementation was done on
the cli project.
* Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously
* Improvements to logging framework
Using appropriate single quotes
Improving the way the logger is declared
* Improved naming for Log Types
* Removed logger global variable, replacing it by a proxy
* Add logging to CLI commands
* Remove unused GenericHelpers
* Changed back some messages to console instead of logger and added npm
shortcuts for worker and webhook
* Fix typos
* Adding basic file rotation to logs as suggested by @mutdmour
* Fixed linting issues
* Correcting comment to correctly reflect space usage
* Added settings for log files rotation
* Correcting config type from String to Number
* Changed default file settings to number
To reflect previous changes to the type
* Changed the way log messages are added to be called statically. Also minor naming improvements
* Applying latest corrections sent by @ivov
* ⚡ Some logging improvements
* Saving logs to a folder inside n8n home instead of root
* Fixed broken tests and linting
* Changed some log messages to improve formatting
* Adding quotes to names on log messages
* Added execution and session IDs to logs. Also removed unnecessary line breaks
* ⚡ Added file caller to log messages (#1657)
This is done using callsites library which already existed
in the project as another library's dependency. So in fact
it does not add any new dependency.
* Adding logs to help debug Salesforce node
* ⚡ Add function name to logs and add more logs
* ⚡ Improve some error messages
* ⚡ Improve some more log messages
* ⚡ Rename logging env variables to match others
Co-authored-by: dali <servfrdali@yahoo.fr>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
