n8n/packages/cli/commands/webhook.ts

240 lines
8 KiB
TypeScript
Raw Normal View History

:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
/* eslint-disable no-console */
/* eslint-disable @typescript-eslint/no-unsafe-call */
/* eslint-disable @typescript-eslint/no-unsafe-member-access */
/* eslint-disable @typescript-eslint/no-unsafe-assignment */
/* eslint-disable @typescript-eslint/unbound-method */
import { BinaryDataManager, UserSettings } from 'n8n-core';
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
import { Command, flags } from '@oclif/command';
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line import/no-extraneous-dependencies
import Redis from 'ioredis';
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
import { IDataObject, LoggerProxy } from 'n8n-workflow';
import config from '../config';
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
import {
ActiveExecutions,
ActiveWorkflowRunner,
CredentialsOverwrites,
CredentialTypes,
Db,
ExternalHooks,
GenericHelpers,
2021-10-21 16:25:31 -07:00
InternalHooksManager,
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
LoadNodesAndCredentials,
NodeTypes,
WebhookServer,
2021-03-25 03:23:54 -07:00
} from '../src';
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
import { getLogger } from '../src/Logger';
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
let activeWorkflowRunner: ActiveWorkflowRunner.ActiveWorkflowRunner | undefined;
let processExistCode = 0;
export class Webhook extends Command {
static description = 'Starts n8n webhook process. Intercepts only production URLs.';
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
static examples = [`$ n8n webhook`];
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
static flags = {
help: flags.help({ char: 'h' }),
};
/**
* Stops the n8n in a graceful way.
* Make for example sure that all the webhooks from third party services
* get removed.
*/
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
static async stopProcess() {
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
LoggerProxy.info(`\nStopping n8n...`);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
try {
const externalHooks = ExternalHooks();
await externalHooks.run('n8n.stop', []);
setTimeout(() => {
// In case that something goes wrong with shutdown we
// kill after max. 30 seconds no matter what
process.exit(processExistCode);
}, 30000);
// Wait for active workflow executions to finish
const activeExecutionsInstance = ActiveExecutions.getInstance();
let executingWorkflows = activeExecutionsInstance.getActiveExecutions();
let count = 0;
while (executingWorkflows.length !== 0) {
if (count++ % 4 === 0) {
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
LoggerProxy.info(
`Waiting for ${executingWorkflows.length} active executions to finish...`,
);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
}
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line no-await-in-loop
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
await new Promise((resolve) => {
setTimeout(resolve, 500);
});
executingWorkflows = activeExecutionsInstance.getActiveExecutions();
}
} catch (error) {
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
LoggerProxy.error('There was an error shutting down n8n.', error);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
}
process.exit(processExistCode);
}
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
async run() {
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
const logger = getLogger();
LoggerProxy.init(logger);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
// Make sure that n8n shuts down gracefully if possible
process.on('SIGTERM', Webhook.stopProcess);
process.on('SIGINT', Webhook.stopProcess);
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line @typescript-eslint/no-unused-vars, @typescript-eslint/no-shadow
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
const { flags } = this.parse(Webhook);
// Wrap that the process does not close but we can still use async
await (async () => {
if (config.getEnv('executions.mode') !== 'queue') {
2021-03-25 03:23:54 -07:00
/**
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
* It is technically possible to run without queues but
* there are 2 known bugs when running in this mode:
* - Executions list will be problematic as the main process
* is not aware of current executions in the webhook processes
* and therefore will display all current executions as error
* as it is unable to determine if it is still running or crashed
* - You cannot stop currently executing jobs from webhook processes
* when running without queues as the main process cannot talk to
* the wehbook processes to communicate workflow execution interruption.
*/
this.error('Webhook processes can only run with execution mode as queue.');
}
try {
// Start directly with the init of the database to improve startup time
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
const startDbInitPromise = Db.init().catch((error) => {
// eslint-disable-next-line @typescript-eslint/restrict-template-expressions, @typescript-eslint/no-unsafe-member-access
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
logger.error(`There was an error initializing DB: "${error.message}"`);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
processExistCode = 1;
// @ts-ignore
process.emit('SIGINT');
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
process.exit(1);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
});
// Make sure the settings exist
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line @typescript-eslint/no-unused-vars
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
const userSettings = await UserSettings.prepareUserSettings();
// Load all node and credential types
const loadNodesAndCredentials = LoadNodesAndCredentials();
await loadNodesAndCredentials.init();
// Load the credentials overwrites if any exist
const credentialsOverwrites = CredentialsOverwrites();
await credentialsOverwrites.init();
// Load all external hooks
const externalHooks = ExternalHooks();
await externalHooks.init();
// Add the found types to an instance other parts of the application can use
const nodeTypes = NodeTypes();
await nodeTypes.init(loadNodesAndCredentials.nodeTypes);
const credentialTypes = CredentialTypes();
await credentialTypes.init(loadNodesAndCredentials.credentialTypes);
// Wait till the database is ready
await startDbInitPromise;
2021-10-21 16:25:31 -07:00
const instanceId = await UserSettings.getInstanceId();
const { cli } = await GenericHelpers.getVersions();
InternalHooksManager.init(instanceId, cli, nodeTypes);
2021-10-21 16:25:31 -07:00
const binaryDataConfig = config.getEnv('binaryDataManager');
await BinaryDataManager.init(binaryDataConfig);
if (config.getEnv('executions.mode') === 'queue') {
const redisHost = config.getEnv('queue.bull.redis.host');
const redisPassword = config.getEnv('queue.bull.redis.password');
const redisPort = config.getEnv('queue.bull.redis.port');
const redisDB = config.getEnv('queue.bull.redis.db');
const redisConnectionTimeoutLimit = config.getEnv('queue.bull.redis.timeoutThreshold');
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
let lastTimer = 0;
let cumulativeTimeout = 0;
2021-03-25 03:23:54 -07:00
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
const settings = {
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line @typescript-eslint/no-unused-vars
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
retryStrategy: (times: number): number | null => {
const now = Date.now();
if (now - lastTimer > 30000) {
// Means we had no timeout at all or last timeout was temporary and we recovered
lastTimer = now;
cumulativeTimeout = 0;
} else {
cumulativeTimeout += now - lastTimer;
lastTimer = now;
if (cumulativeTimeout > redisConnectionTimeoutLimit) {
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
logger.error(
// eslint-disable-next-line @typescript-eslint/restrict-template-expressions
`Unable to connect to Redis after ${redisConnectionTimeoutLimit}. Exiting process.`,
);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
process.exit(1);
}
}
return 500;
},
} as IDataObject;
if (redisHost) {
settings.host = redisHost;
}
if (redisPassword) {
settings.password = redisPassword;
}
if (redisPort) {
settings.port = redisPort;
}
if (redisDB) {
settings.db = redisDB;
}
2021-03-25 03:23:54 -07:00
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
// This connection is going to be our heartbeat
// IORedis automatically pings redis and tries to reconnect
// We will be using the retryStrategy above
// to control how and when to exit.
const redis = new Redis(settings);
redis.on('error', (error) => {
if (error.toString().includes('ECONNREFUSED') === true) {
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
logger.warn('Redis unavailable - trying to reconnect...');
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
} else {
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
logger.warn('Error with Redis: ', error);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
}
});
}
2021-03-25 03:23:54 -07:00
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
await WebhookServer.start();
// Start to get active workflows and run their triggers
activeWorkflowRunner = ActiveWorkflowRunner.getInstance();
await activeWorkflowRunner.initWebhooks();
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line @typescript-eslint/no-unused-vars
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
const editorUrl = GenericHelpers.getBaseUrl();
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
console.info('Webhook listener waiting for requests.');
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
} catch (error) {
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
console.error('Exiting due to error. See log message for details.');
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
// eslint-disable-next-line @typescript-eslint/restrict-template-expressions
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
logger.error(`Webhook process cannot continue. "${error.message}"`);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
processExistCode = 1;
// @ts-ignore
process.emit('SIGINT');
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
process.exit(1);
:sparkles: Separate webhooks from core (#1408) * Unify execution ID across executions * Fix indentation and improved comments * WIP: saving data after each node execution * Added on/off to save data after each step, saving initial data and retries working * Fixing lint issues * Fixing more lint issues * :sparkles: Add bull to execute workflows * :shirt: Fix lint issue * :zap: Add graceful shutdown to worker * :zap: Add loading staticData to worker * :shirt: Fix lint issue * :zap: Fix import * Changed tables metadata to add nullable to stoppedAt * Reload database on migration run * Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration * Added checks to Redis and exiting process if connection is unavailable * Fixing error with new installations * Fix issue with data not being sent back to browser on manual executions with defined destination * Merging bull and unify execution id branch fixes * Main process will now get execution success from database instead of redis * Omit execution duration if execution did not stop * Fix issue with execution list displaying inconsistant information information while a workflow is running * Remove unused hooks to clarify for developers that these wont run in queue mode * Added active pooling to help recover from Redis crashes * Lint issues * Changing default polling interval to 60 seconds * Removed unnecessary attributes from bull job * Added webhooks service and setting to disable webhooks from main process * Fixed executions list when running with queues. Now we get the list of actively running workflows from bull. * Add option to disable deregistration of webhooks on shutdown * Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat. * Added auto refresh to executions list * Improvements to workflow stop process when running with queues * Refactor queue system to use a singleton and avoid code duplication * Improve comments and remove unnecessary commits * Remove console.log from vue file * Blocking webhook process to run without queues * Handling execution stop graciously when possible * Removing initialization of all workflows from webhook process * Refactoring code to remove code duplication for job stop * Improved execution list to be more fluid and less intrusive * Fixing workflow name for current executions when auto updating * :zap: Right align autorefresh checkbox Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-02-09 14:32:40 -08:00
}
})();
}
}