n8n/packages/cli/src/WebhookHelpers.ts

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

849 lines
24 KiB
TypeScript
Raw Normal View History

/* eslint-disable @typescript-eslint/no-unsafe-argument */
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
/* eslint-disable @typescript-eslint/prefer-optional-chain */
/* eslint-disable @typescript-eslint/no-shadow */
/* eslint-disable @typescript-eslint/no-unsafe-assignment */
2019-06-23 03:35:23 -07:00
/* eslint-disable id-denylist */
/* eslint-disable prefer-spread */
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
/* eslint-disable @typescript-eslint/no-unsafe-member-access */
/* eslint-disable @typescript-eslint/restrict-template-expressions */
import type express from 'express';
import { Container } from 'typedi';
import get from 'lodash/get';
import { finished } from 'stream/promises';
import formidable from 'formidable';
2019-06-23 03:35:23 -07:00
import { BinaryDataService, NodeExecuteFunctions } from 'n8n-core';
2019-06-23 03:35:23 -07:00
import type {
IBinaryData,
2019-06-23 03:35:23 -07:00
IBinaryKeyData,
IDataObject,
IDeferredPromise,
2019-06-23 03:35:23 -07:00
IExecuteData,
IExecuteResponsePromiseData,
IHttpRequestMethods,
IN8nHttpFullResponse,
2019-06-23 03:35:23 -07:00
INode,
IPinData,
2019-06-23 03:35:23 -07:00
IRunExecutionData,
IWebhookData,
IWebhookResponseData,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
IWorkflowDataProxyAdditionalKeys,
2019-06-23 03:35:23 -07:00
IWorkflowExecuteAdditionalData,
WebhookResponseMode,
Workflow,
WorkflowExecuteMode,
} from 'n8n-workflow';
import {
ApplicationError,
BINARY_ENCODING,
createDeferredPromise,
ErrorReporterProxy as ErrorReporter,
2019-06-23 03:35:23 -07:00
NodeHelpers,
} from 'n8n-workflow';
import type {
2019-06-23 03:35:23 -07:00
IExecutionDb,
IResponseCallbackData,
IWebhookManager,
2019-06-23 03:35:23 -07:00
IWorkflowDb,
IWorkflowExecutionDataProcess,
WebhookCORSRequest,
WebhookRequest,
} from '@/Interfaces';
import * as ResponseHelper from '@/ResponseHelper';
import * as WorkflowHelpers from '@/WorkflowHelpers';
import { WorkflowRunner } from '@/WorkflowRunner';
import * as WorkflowExecuteAdditionalData from '@/WorkflowExecuteAdditionalData';
import { ActiveExecutions } from '@/ActiveExecutions';
import { WorkflowStatisticsService } from '@/services/workflow-statistics.service';
import { OwnershipService } from './services/ownership.service';
import { parseBody } from './middlewares';
import { Logger } from './Logger';
import { NotFoundError } from './errors/response-errors/not-found.error';
import { InternalServerError } from './errors/response-errors/internal-server.error';
import { UnprocessableRequestError } from './errors/response-errors/unprocessable.error';
import type { Project } from './databases/entities/Project';
2019-06-23 03:35:23 -07:00
export const WEBHOOK_METHODS: IHttpRequestMethods[] = [
'DELETE',
'GET',
'HEAD',
'PATCH',
'POST',
'PUT',
];
export const webhookRequestHandler =
(webhookManager: IWebhookManager) =>
async (req: WebhookRequest | WebhookCORSRequest, res: express.Response) => {
const { path } = req.params;
const method = req.method;
if (method !== 'OPTIONS' && !WEBHOOK_METHODS.includes(method)) {
return ResponseHelper.sendErrorResponse(
res,
new Error(`The method ${method} is not supported.`),
);
}
// Setup CORS headers only if the incoming request has an `origin` header
if ('origin' in req.headers) {
if (webhookManager.getWebhookMethods) {
try {
const allowedMethods = await webhookManager.getWebhookMethods(path);
res.header('Access-Control-Allow-Methods', ['OPTIONS', ...allowedMethods].join(', '));
} catch (error) {
return ResponseHelper.sendErrorResponse(res, error as Error);
}
}
const requestedMethod =
method === 'OPTIONS'
? (req.headers['access-control-request-method'] as IHttpRequestMethods)
: method;
if (webhookManager.findAccessControlOptions && requestedMethod) {
const options = await webhookManager.findAccessControlOptions(path, requestedMethod);
const { allowedOrigins } = options ?? {};
if (allowedOrigins && allowedOrigins !== '*' && allowedOrigins !== req.headers.origin) {
const originsList = allowedOrigins.split(',');
const defaultOrigin = originsList[0];
if (originsList.length === 1) {
res.header('Access-Control-Allow-Origin', defaultOrigin);
}
if (originsList.includes(req.headers.origin as string)) {
res.header('Access-Control-Allow-Origin', req.headers.origin);
} else {
res.header('Access-Control-Allow-Origin', defaultOrigin);
}
} else {
res.header('Access-Control-Allow-Origin', req.headers.origin);
}
if (method === 'OPTIONS') {
res.header('Access-Control-Max-Age', '300');
const requestedHeaders = req.headers['access-control-request-headers'];
if (requestedHeaders?.length) {
res.header('Access-Control-Allow-Headers', requestedHeaders);
}
}
}
}
if (method === 'OPTIONS') {
return ResponseHelper.sendSuccessResponse(res, {}, true, 204);
}
let response;
try {
response = await webhookManager.executeWebhook(req, res);
} catch (error) {
return ResponseHelper.sendErrorResponse(res, error as Error);
}
// Don't respond, if already responded
if (response.noWebhookResponse !== true) {
ResponseHelper.sendSuccessResponse(
res,
response.data,
true,
response.responseCode,
response.headers,
);
}
};
2022-02-20 07:44:30 -08:00
2019-06-23 03:35:23 -07:00
/**
feat(editor): Add usage and plan pages (#4819) * feat(editor): Usage and plan page (#4793) feat(editor): usage and plan page * feat(editor): Update Usage and plan page (#4842) * feat(editor): usage and plan store * feat(editor): usage and plan page updates * feat(editor): usage and plan add buttons and alert * tes(editor): usage and plan store * tes(editor): usage remove refresh button and add link to view plans * tes(editor): usage use info tip * tes(editor): usage info style * feat(editor): Get quotas data (#4866) feat(editor): get quotas data * feat(editor): In-app experience (#4875) * feat: Add license quotas endpoint * feat: Add trigger count to workflow activation process * refactor: Get quotas from db * feat: Add license information * :sparkles: - finalised GET /license endpoint * :hammer: - getActiveTriggerCount return 0 instead of null * :bug: - ignore manualTrigger when counting active triggers * :sparkles: - add activation endpoint * :sparkles: - added renew endpoint * :hammer: - added return type interfaces * :hammer: - handle license errors where methods are called * :hammer: - rename function to match name from lib * feat(editor): usage add plans buttons logic * :rotating_light: - testing new License methods * feat(editor): usage add more business logic * chore(editor): code formatting * :rotating_light: - added license api tests * fix(editor): usage store * fix(editor): usage update translations * feat(editor): usage add license activation modal * feat(editor): usage change subscription app url * feat(editor): usage add contact us link * feat(editor): usage fix modal width * :sparkles: - Add renewal tracking metric * :sparkles: - add license data to pulse event * :hammer: - set default triggercount on entity model * :sparkles: - add db migrations for mysql and postgres * fix(editor): Usage api call data processing and error handling * fix(editor): Usage fix activation query key * :rotating_light: - add initDb to telemetry tests * :hammer: - move getlicensedata to licenseservice * :hammer: - return 403 instead of 404 to non owners * :hammer: - move owner checking to middleware * :bug: - fixed incorrectly returned error from middleware * :bug: - using mock instead of test db for pulse tests * fix(editor): Usage fix activation and add success messages * fix(editor): Usage should not renew activation right after activation * :rotating_light: - skipping failing pulse tests for now * fix(editor): Usage add telemetry calls and apply design review outcomes * feat(editor): Hide usage page according to BE flag * feat(editor): Usage modify key activation flow * feat(editor): Usage change subscription app url * feat(editor): Usage add telemetry for manage plan * feat(editor): Usage extend link url query params * feat(editor): Usage add line chart if there is a workflow limit * feat(editor): Usage remove query after key activation redirection * fix(editor): Usage handle limit exceeded workflow chart, add focus to input when modal opened * fix(editor): Usage activation can return router promise when removing query * fix(editor): Usage and plan design review * :bug: - fix renew endpoint hanging issue * :bug: - fix license activation bug * fix(editor): Usage proper translation for plans and/or editions * fix(editor): Usage apply David's review results * fix(editor): Usage page set as default and first under Settings * fix(editor): Usage open subscription app in new tab * fix(editor): Usage page having key query param a plan links * test: Fix broken test * fix(editor): Usage page address review * :test_tube: Flush promises on telemetry tests * :zap: Extract helper with `setImmediate` * :fire: Remove leftovers * :zap: Use Adi's helper * refactor: Comment broken tests * refactor: add Tenant id to settings * feat: add environment to license endpoints * refactor: Move license environment to general settings * fix: fix routing bug * fix(editor): Usage page some code review changes and formatting * fix(editor): Usage page remove direct usage of reusable translation keys * fix(editor): Usage page async await instead of then * fix(editor): Usage page show some content only if network requests in component mounted were successful * chore(editor): code formatting * fix(editor): Usage checking license environment * feat(editor): Improve license activation error messages (no-changelog) (#4958) * fix(editor): Usage changing activation error title * remove unnecessary import * fix(editor): Usage refactor notification showing * fix(editor): Usage using notification directly in store actions Co-authored-by: Omar Ajoue <krynble@gmail.com> Co-authored-by: freyamade <freya@n8n.io> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Cornelius Suermann <cornelius@n8n.io> * fix(editor): Usage change mounted lifecycle logic * fix(editor): Usage return after successful activation in mounted * fix: remove console log * test: fix tests related to settings (#4979) Co-authored-by: Omar Ajoue <krynble@gmail.com> Co-authored-by: freyamade <freya@n8n.io> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Cornelius Suermann <cornelius@n8n.io> Co-authored-by: Mutasem Aldmour <4711238+mutdmour@users.noreply.github.com>
2022-12-20 01:52:01 -08:00
* Returns all the webhooks which should be created for the given workflow
2019-06-23 03:35:23 -07:00
*/
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
export function getWorkflowWebhooks(
workflow: Workflow,
additionalData: IWorkflowExecuteAdditionalData,
destinationNode?: string,
ignoreRestartWebhooks = false,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
): IWebhookData[] {
2019-06-23 03:35:23 -07:00
// Check all the nodes in the workflow if they have webhooks
const returnData: IWebhookData[] = [];
let parentNodes: string[] | undefined;
if (destinationNode !== undefined) {
parentNodes = workflow.getParentNodes(destinationNode);
// Also add the destination node in case it itself is a webhook node
parentNodes.push(destinationNode);
2019-06-23 03:35:23 -07:00
}
for (const node of Object.values(workflow.nodes)) {
if (parentNodes !== undefined && !parentNodes.includes(node.name)) {
// If parentNodes are given check only them if they have webhooks
// and no other ones
2019-06-23 03:35:23 -07:00
continue;
}
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
returnData.push.apply(
returnData,
NodeHelpers.getNodeWebhooks(workflow, node, additionalData, ignoreRestartWebhooks),
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
);
2019-06-23 03:35:23 -07:00
}
return returnData;
}
export function encodeWebhookResponse(
response: IExecuteResponsePromiseData,
): IExecuteResponsePromiseData {
if (typeof response === 'object' && Buffer.isBuffer(response.body)) {
response.body = {
'__@N8nEncodedBuffer@__': response.body.toString(BINARY_ENCODING),
};
}
return response;
}
const normalizeFormData = <T>(values: Record<string, T | T[]>) => {
for (const key in values) {
const value = values[key];
if (Array.isArray(value) && value.length === 1) {
values[key] = value[0];
}
}
};
2019-06-23 03:35:23 -07:00
/**
* Executes a webhook
*/
// eslint-disable-next-line complexity
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
export async function executeWebhook(
workflow: Workflow,
webhookData: IWebhookData,
workflowData: IWorkflowDb,
workflowStartNode: INode,
executionMode: WorkflowExecuteMode,
pushRef: string | undefined,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
runExecutionData: IRunExecutionData | undefined,
executionId: string | undefined,
req: WebhookRequest,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
res: express.Response,
responseCallback: (error: Error | null, data: IResponseCallbackData) => void,
destinationNode?: string,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
): Promise<string | undefined> {
2019-06-23 03:35:23 -07:00
// Get the nodeType to know which responseMode is set
:sparkles: Updated node design and node versioning (#1961) * :zap: introduce versioned nodes * Export versioned nodes for separate process run * Add bse node for versioned nodes * fix node name for versioned nodes * extend node from nodeVersionedType * improve nodes base and flow to FE * revert lib es2019 to es2017 * include version in key to prevent duplicate key * handle type versions on FE * clean up * cleanup nodes base * add type versions in getNodeParameterOptions * cleanup * code review * code review + add default version to node type description * remove node default types from store * :lipstick: cleanups * Draft for migrated Mattermost node * First version of Mattermost node versioned according to node standards * Correcting deactivate operations name to match currently used one * :sparkles: Create utility types * :zap: Simplify Mattermost types * :zap: Rename exports for consistency * :zap: Type channel properties * :zap: Type message properties * :zap: Type reaction properties * :zap: Type user properties * :zap: Add type import to router * :bug: Add missing key * :hammer: Adjust typo in operation name * :hammer: Inline exports for channel properties * :hammer: Inline exports for message properties * :hammer: Inline exports for reaction properties * :hammer: Inline exports for user properties * :hammer: Inline exports for load options * :shirt: Fix lint issue * :hammer: Inline export for description * :hammer: Rename descriptions for clarity * :hammer: Refactor imports/exports for methods * :hammer: Refactor latest version retrieval * :fire: Remove unneeded else clause When the string literal union is exhausted, the resource key becomes never, so TS disallows wrong key usage. * :sparkles: Add overloads to getNodeParameter * :zap: Improve overload * :fire: Remove superfluous INodeVersions type * :hammer: Relocate pre-existing interface * :fire: Remove JSDoc arg descriptions * :zap: Minor reformatting in transport file * :zap: Fix API call function type * Created first draft for Axios requests * Working version of mattermost node with Axios * Work in progress for replacing request library * Improvements to request translations * Fixed sending files via multipart / form-data * Fixing translation from request to axios and loading node parameter options * Improved typing for new http helper * Added ignore any for specific lines for linting * Fixed follow redirects changes on http request node and manual execution of previously existing workflow with older node versions * Adding default headers according to body on httpRequest helper * Spec error handling and fixed workflows with older node versions * Showcase how to export errors in a standard format * Merging master * Refactored mattermost node to keep files in a uniform structure. Also fix bugs with merges * Reverting changes to http request node * Changed nullish comparison and removed repeated code from nodes * Renamed queryString back to qs and simplified node output * Simplified some comparisons * Changed header names to be uc first * Added default user agent to requests and patch http method support * Fixed indentation, remove unnecessary file and console log * Fixed mattermost node name * Fixed lint issues * Further fix linting issues * Further fix lint issues * Fixed http request helper's return type Co-authored-by: ahsan-virani <ahsan.virani@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2021-09-21 10:38:24 -07:00
const nodeType = workflow.nodeTypes.getByNameAndVersion(
workflowStartNode.type,
workflowStartNode.typeVersion,
);
2019-06-23 03:35:23 -07:00
if (nodeType === undefined) {
const errorMessage = `The type of the webhook node "${workflowStartNode.name}" is not known`;
2019-06-23 03:35:23 -07:00
responseCallback(new Error(errorMessage), {});
throw new InternalServerError(errorMessage);
2019-06-23 03:35:23 -07:00
}
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
const additionalKeys: IWorkflowDataProxyAdditionalKeys = {
$executionId: executionId,
};
let project: Project | undefined = undefined;
try {
project = await Container.get(OwnershipService).getWorkflowProjectCached(workflowData.id);
} catch (error) {
throw new NotFoundError('Cannot find workflow');
}
// Prepare everything that is needed to run the workflow
const additionalData = await WorkflowExecuteAdditionalData.getBase();
2019-06-23 03:35:23 -07:00
// Get the responseMode
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
const responseMode = workflow.expression.getSimpleParameterValue(
workflowStartNode,
webhookData.webhookDescription.responseMode,
executionMode,
additionalKeys,
undefined,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
'onReceived',
) as WebhookResponseMode;
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
const responseCode = workflow.expression.getSimpleParameterValue(
workflowStartNode,
webhookData.webhookDescription.responseCode as string,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
executionMode,
additionalKeys,
undefined,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
200,
) as number;
2019-06-23 03:35:23 -07:00
const responseData = workflow.expression.getComplexParameterValue(
workflowStartNode,
webhookData.webhookDescription.responseData,
executionMode,
additionalKeys,
undefined,
'firstEntryJson',
);
if (!['onReceived', 'lastNode', 'responseNode'].includes(responseMode)) {
2019-06-23 03:35:23 -07:00
// If the mode is not known we error. Is probably best like that instead of using
// the default that people know as early as possible (probably already testing phase)
// that something does not resolve properly.
const errorMessage = `The response mode '${responseMode}' is not valid!`;
2019-06-23 03:35:23 -07:00
responseCallback(new Error(errorMessage), {});
throw new InternalServerError(errorMessage);
2019-06-23 03:35:23 -07:00
}
// Add the Response and Request so that this data can be accessed in the node
additionalData.httpRequest = req;
additionalData.httpResponse = res;
let binaryData;
const nodeVersion = workflowStartNode.typeVersion;
if (nodeVersion === 1) {
// binaryData option is removed in versions higher than 1
binaryData = workflow.expression.getSimpleParameterValue(
workflowStartNode,
'={{$parameter["options"]["binaryData"]}}',
executionMode,
additionalKeys,
undefined,
false,
);
}
2019-06-23 03:35:23 -07:00
let didSendResponse = false;
let runExecutionDataMerge = {};
2019-06-23 03:35:23 -07:00
try {
// Run the webhook function to see what should be returned and if
// the workflow should be executed or not
let webhookResultData: IWebhookResponseData;
2019-06-23 03:35:23 -07:00
// if `Webhook` or `Wait` node, and binaryData is enabled, skip pre-parse the request-body
// always falsy for versions higher than 1
if (!binaryData) {
const { contentType, encoding } = req;
if (contentType === 'multipart/form-data') {
const form = formidable({
multiples: true,
encoding: encoding as formidable.BufferEncoding,
// TODO: pass a custom `fileWriteStreamHandler` to create binary data files directly
});
req.body = await new Promise((resolve) => {
form.parse(req, async (_err, data, files) => {
normalizeFormData(data);
normalizeFormData(files);
resolve({ data, files });
});
});
} else {
if (nodeVersion > 1) {
if (
contentType?.startsWith('application/json') ||
contentType?.startsWith('text/plain') ||
contentType?.startsWith('application/x-www-form-urlencoded') ||
contentType?.endsWith('/xml') ||
contentType?.endsWith('+xml')
) {
await parseBody(req);
}
} else {
await parseBody(req);
}
}
}
try {
webhookResultData = await workflow.runWebhook(
webhookData,
workflowStartNode,
additionalData,
NodeExecuteFunctions,
executionMode,
);
Container.get(WorkflowStatisticsService).emit(
'nodeFetchedData',
workflow.id,
workflowStartNode,
);
:sparkles: Improve node error handling (#1309) * Add path mapping and response error interfaces * Add error handling and throwing functionality * Refactor error handling into a single function * Re-implement error handling in Hacker News node * Fix linting details * Re-implement error handling in Spotify node * Re-implement error handling in G Suite Admin node * :construction: create basic setup NodeError * :construction: add httpCodes * :construction: add path priolist * :construction: handle statusCode in error, adjust interfaces * :construction: fixing type issues w/Ivan * :construction: add error exploration * 👔 fix linter issues * :wrench: improve object check * :construction: remove path passing from NodeApiError * :construction: add multi error + refactor findProperty method * 👔 allow any * :wrench: handle multi error message callback * :zap: change return type of callback * :zap: add customCallback to MultiError * :construction: refactor to use INode * :hammer: handle arrays, continue search after first null property found * 🚫 refactor method access * :construction: setup NodeErrorView * :zap: change timestamp to Date.now * :books: Add documentation for methods and constants * :construction: change message setting * 🚚 move NodeErrors to workflow * :sparkles: add new ErrorView for Nodes * :art: improve error notification * :art: refactor interfaces * :zap: add WorkflowOperationError, refactor error throwing * 👕 fix linter issues * :art: rename param * :bug: fix handling normal errors * :zap: add usage of NodeApiError * :art: fix throw new error instead of constructor * :art: remove unnecessary code/comments * :art: adjusted spacing + updated status messages * :art: fix tab indentation * ✨ Replace current errors with custom errors (#1576) * :zap: Introduce NodeApiError in catch blocks * :zap: Introduce NodeOperationError in nodes * :zap: Add missing errors and remove incompatible * :zap: Fix NodeOperationError in incompatible nodes * :wrench: Adjust error handling in missed nodes PayPal, FileMaker, Reddit, Taiga and Facebook Graph API nodes * :hammer: Adjust Strava Trigger node error handling * :hammer: Adjust AWS nodes error handling * :hammer: Remove duplicate instantiation of NodeApiError * :bug: fix strava trigger node error handling * Add XML parsing to NodeApiError constructor (#1633) * :bug: Remove type annotation from catch variable * :sparkles: Add XML parsing to NodeApiError * :zap: Simplify error handling in Rekognition node * :zap: Pass in XML flag in generic functions * :fire: Remove try/catch wrappers at call sites * :hammer: Refactor setting description from XML * :hammer: Refactor let to const in resource loaders * :zap: Find property in parsed XML * :zap: Change let to const * :fire: Remove unneeded try/catch block * :shirt: Fix linting issues * :bug: Fix errors from merge conflict resolution * :zap: Add custom errors to latest contributions * :shirt: Fix linting issues * :zap: Refactor MongoDB helpers for custom errors * :bug: Correct custom error type * :zap: Apply feedback to A nodes * :zap: Apply feedback to missed A node * :zap: Apply feedback to B-D nodes * :zap: Apply feedback to E-F nodes * :zap: Apply feedback to G nodes * :zap: Apply feedback to H-L nodes * :zap: Apply feedback to M nodes * :zap: Apply feedback to P nodes * :zap: Apply feedback to R nodes * :zap: Apply feedback to S nodes * :zap: Apply feedback to T nodes * :zap: Apply feedback to V-Z nodes * :zap: Add HTTP code to iterable node error * :hammer: Standardize e as error * :hammer: Standardize err as error * :zap: Fix error handling for non-standard nodes Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>
2021-04-16 09:33:36 -07:00
} catch (err) {
// Send error response to webhook caller
const errorMessage = 'Workflow Webhook Error: Workflow could not be started!';
responseCallback(new Error(errorMessage), {});
didSendResponse = true;
// Add error to execution data that it can be logged and send to Editor-UI
runExecutionDataMerge = {
resultData: {
runData: {},
lastNodeExecuted: workflowStartNode.name,
error: {
:sparkles: Improve node error handling (#1309) * Add path mapping and response error interfaces * Add error handling and throwing functionality * Refactor error handling into a single function * Re-implement error handling in Hacker News node * Fix linting details * Re-implement error handling in Spotify node * Re-implement error handling in G Suite Admin node * :construction: create basic setup NodeError * :construction: add httpCodes * :construction: add path priolist * :construction: handle statusCode in error, adjust interfaces * :construction: fixing type issues w/Ivan * :construction: add error exploration * 👔 fix linter issues * :wrench: improve object check * :construction: remove path passing from NodeApiError * :construction: add multi error + refactor findProperty method * 👔 allow any * :wrench: handle multi error message callback * :zap: change return type of callback * :zap: add customCallback to MultiError * :construction: refactor to use INode * :hammer: handle arrays, continue search after first null property found * 🚫 refactor method access * :construction: setup NodeErrorView * :zap: change timestamp to Date.now * :books: Add documentation for methods and constants * :construction: change message setting * 🚚 move NodeErrors to workflow * :sparkles: add new ErrorView for Nodes * :art: improve error notification * :art: refactor interfaces * :zap: add WorkflowOperationError, refactor error throwing * 👕 fix linter issues * :art: rename param * :bug: fix handling normal errors * :zap: add usage of NodeApiError * :art: fix throw new error instead of constructor * :art: remove unnecessary code/comments * :art: adjusted spacing + updated status messages * :art: fix tab indentation * ✨ Replace current errors with custom errors (#1576) * :zap: Introduce NodeApiError in catch blocks * :zap: Introduce NodeOperationError in nodes * :zap: Add missing errors and remove incompatible * :zap: Fix NodeOperationError in incompatible nodes * :wrench: Adjust error handling in missed nodes PayPal, FileMaker, Reddit, Taiga and Facebook Graph API nodes * :hammer: Adjust Strava Trigger node error handling * :hammer: Adjust AWS nodes error handling * :hammer: Remove duplicate instantiation of NodeApiError * :bug: fix strava trigger node error handling * Add XML parsing to NodeApiError constructor (#1633) * :bug: Remove type annotation from catch variable * :sparkles: Add XML parsing to NodeApiError * :zap: Simplify error handling in Rekognition node * :zap: Pass in XML flag in generic functions * :fire: Remove try/catch wrappers at call sites * :hammer: Refactor setting description from XML * :hammer: Refactor let to const in resource loaders * :zap: Find property in parsed XML * :zap: Change let to const * :fire: Remove unneeded try/catch block * :shirt: Fix linting issues * :bug: Fix errors from merge conflict resolution * :zap: Add custom errors to latest contributions * :shirt: Fix linting issues * :zap: Refactor MongoDB helpers for custom errors * :bug: Correct custom error type * :zap: Apply feedback to A nodes * :zap: Apply feedback to missed A node * :zap: Apply feedback to B-D nodes * :zap: Apply feedback to E-F nodes * :zap: Apply feedback to G nodes * :zap: Apply feedback to H-L nodes * :zap: Apply feedback to M nodes * :zap: Apply feedback to P nodes * :zap: Apply feedback to R nodes * :zap: Apply feedback to S nodes * :zap: Apply feedback to T nodes * :zap: Apply feedback to V-Z nodes * :zap: Add HTTP code to iterable node error * :hammer: Standardize e as error * :hammer: Standardize err as error * :zap: Fix error handling for non-standard nodes Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>
2021-04-16 09:33:36 -07:00
...err,
message: err.message,
stack: err.stack,
},
},
};
webhookResultData = {
noWebhookResponse: true,
// Add empty data that it at least tries to "execute" the webhook
// which then so gets the chance to throw the error.
workflowData: [[{ json: {} }]],
};
}
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
const additionalKeys: IWorkflowDataProxyAdditionalKeys = {
$executionId: executionId,
};
if (webhookData.webhookDescription.responseHeaders !== undefined) {
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
const responseHeaders = workflow.expression.getComplexParameterValue(
workflowStartNode,
webhookData.webhookDescription.responseHeaders,
executionMode,
additionalKeys,
undefined,
undefined,
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
) as {
entries?:
| Array<{
name: string;
value: string;
}>
| undefined;
};
if (responseHeaders !== undefined && responseHeaders.entries !== undefined) {
for (const item of responseHeaders.entries) {
res.setHeader(item.name, item.value);
}
}
}
if (webhookResultData.noWebhookResponse === true && !didSendResponse) {
2019-06-23 03:35:23 -07:00
// The response got already send
responseCallback(null, {
noWebhookResponse: true,
});
didSendResponse = true;
}
if (webhookResultData.workflowData === undefined) {
// Workflow should not run
if (webhookResultData.webhookResponse !== undefined) {
// Data to respond with is given
if (!didSendResponse) {
responseCallback(null, {
data: webhookResultData.webhookResponse,
responseCode,
});
didSendResponse = true;
}
2019-06-23 03:35:23 -07:00
} else {
// Send default response
if (!didSendResponse) {
responseCallback(null, {
data: {
message: 'Webhook call received',
},
responseCode,
});
didSendResponse = true;
}
2019-06-23 03:35:23 -07:00
}
return;
}
// Now that we know that the workflow should run we can return the default response
// directly if responseMode it set to "onReceived" and a response should be sent
2019-08-28 08:16:09 -07:00
if (responseMode === 'onReceived' && !didSendResponse) {
2019-06-23 03:35:23 -07:00
// Return response directly and do not wait for the workflow to finish
if (responseData === 'noData') {
// Return without data
responseCallback(null, {
responseCode,
});
} else if (responseData) {
// Return the data specified in the response data option
responseCallback(null, {
data: responseData as IDataObject,
responseCode,
});
} else if (webhookResultData.webhookResponse !== undefined) {
2019-06-23 03:35:23 -07:00
// Data to respond with is given
responseCallback(null, {
data: webhookResultData.webhookResponse,
responseCode,
2019-06-23 03:35:23 -07:00
});
} else {
responseCallback(null, {
data: {
message: 'Workflow was started',
},
responseCode,
2019-06-23 03:35:23 -07:00
});
}
didSendResponse = true;
}
// Initialize the data of the webhook node
const nodeExecutionStack: IExecuteData[] = [];
nodeExecutionStack.push({
node: workflowStartNode,
data: {
main: webhookResultData.workflowData,
},
source: null,
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
});
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
runExecutionData =
runExecutionData ||
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
({
2019-06-23 03:35:23 -07:00
startData: {},
resultData: {
2019-06-23 03:35:23 -07:00
runData: {},
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
},
2019-06-23 03:35:23 -07:00
executionData: {
contextData: {},
nodeExecutionStack,
waitingExecution: {},
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
},
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
} as IRunExecutionData);
if (destinationNode && runExecutionData.startData) {
runExecutionData.startData.destinationNode = destinationNode;
}
:sparkles: Implement Wait functionality (#1817) * refactor saving * refactor api layer to be stateless * refactor header details * set variable for menu height * clean up scss * clean up indentation * clean up dropdown impl * refactor no tags view * split away header * Fix tslint issues * Refactor tag manager * add tags to patch request * clean up scss * :zap: Refactor types to entities * fix issues * update no workflow error * clean up tagscontainer * use getters instead of state * remove imports * use custom colors * clean up tags container * clean up dropdown * clean up focusoncreate * :zap: Ignore mistaken ID in POST /workflows * :zap: Fix undefined tag ID in PATCH /workflows * :zap: Shorten response for POST /tags * remove scss mixins * clean up imports * :zap: Implement validation with class-validator * address ivan's comments * implement modals * Fix lint issues * fix disabling shortcuts * fix focus issues * fix focus issues * fix focus issues with modal * fix linting issues * use dispatch * use constants for modal keys * fix focus * fix lint issues * remove unused prop * add modal root * fix lint issues * remove unused methods * fix shortcut * remove max width * :zap: Fix duplicate entry error for pg and MySQL * update rename messaging * update order of buttons * fix firefox overflow on windows * fix dropdown height * :hammer: refactor tag crud controllers * 🧹 remove unused imports * use variable for number of items * fix dropdown spacing * :zap: Restore type to fix build * :zap: Fix post-refactor PATCH /workflows/:id * :zap: Fix PATCH /workflows/:id for zero tags * :zap: Fix usage count becoming stringified * address max's comments * fix filter spacing * fix blur bug * address most of ivan's comments * address tags type concern * remove defaults * :zap: return tag id as string * :hammer: add hooks to tag CUD operations * 🏎 simplify timestamp pruning * remove blur event * fix onblur bug * :zap: Fix fs import to fix build * address max's comments * implement responsive tag container * fix lint issues * update tag limits * address ivan's comments * remove rename, refactor header, implement new designs for save, remove responsive tag container * update styling * update styling * implement responsive tag container * implement header tags edit * implement header tags edit * fix lint issues * implement expandable input * minor fixes * minor fixes * use variable * rename save as * duplicate fixes * minor edit fixes * lint fixes * style fixes * hook up saving name * hook up tags * clean up impl * fix dirty state bug * update limit * update notification messages * on click outside * fix minor bug with count * lint fixes * handle minor edge cases * handle minor edge cases * handle minor bugs; fix firefox dropdown issue * Fix min width * apply tags only after api success * remove count fix * clean up workflow tags impl, fix tags delete bug * fix minor issue * fix minor spacing issue * disable wrap for ops * fix viewport root; save on click in dropdown * save button loading when saving name/tags * implement max width on tags container * implement cleaner create experience * disable edit while updating * codacy hex color * refactor tags container * fix clickability * fix workflow open and count * clean up structure * fix up lint issues * fix button size * increase workflow name limit for larger screen * tslint fixes * disable responsiveness for workflow modal * rename event * change min width for tags * clean up pr * address max's comments on styles * remove success toasts * add hover mode to name * minor fixes * refactor name preview * fix name input not to jiggle * finish up name input * Fix up add tags * clean up param * clean up scss * fix resizing name * fix resizing name * fix resize bug * clean up edit spacing * ignore on esc * fix input bug * focus input on clear * build * fix up add tags clickablity * remove scrollbars * move into folders * clean up multiple patch req * remove padding top from edit * update tags on enter * build * rollout blur on enter behavior * rollout esc behavior * fix tags bug when duplicating tags * move key to reload tags * update header spacing * build * update hex case * refactor workflow title * remove unusued prop * keep focus on error, fix bug on error * Fix bug with name / tags toggle on error * fix connection push bug * :spakles: Implement wait functionality * :bug: Do not delete waiting executions with prune * :zap: Improve SQLite migration to not lose execution data anymore * :zap: Make it possible to restart waiting execution via webhook * :zap: Add missing file * :bug: Some more merge fixes * :zap: Do not show error for Wait-Nodes if in time-mode * :zap: Make $executionId available in expressions * :shirt: Fix lint issue * :shirt: Fix lint issue * :shirt: Fix lint issue * :zap: Set the unlimited sleep time as a variable * :zap: Add also sleeping webhook path to config * :zap: Make it possible to retrieve restartUrl in workflow * :zap: Add authentication to Wait-Node in Webhook-Mode * :zap: Return 404 when trying to restart execution via webhook which does not support it * :sparkles: Make it possible to set absolute time on Wait-Node * :zap: Remove not needed imports * :zap: Fix description format * :sparkles: Implement missing webhook features on Wait-Node * :zap: Display webhook variable in NodeWebhooks * :zap: Include also date in displayed sleep time * :zap: Make it possible to see sleep time on node * :zap: Make sure that no executions does get executed twice * :zap: Add comment * :zap: Further improvements * :zap: Make Wait-Node easier to use * :sparkles: Add support for "notice" parameter type * Fixing wait node to work with queue, improved logging and execution view * Added support for mysql and pg * :sparkles: Add support for webhook postfix path * :sparkles: Make it possible to stop sleeping executions * :zap: Fix issue with webhook paths in not webhook mode * :zap: Remove not needed console.log * :zap: Update TODOs * :zap: Increase min time of workflow staying active to descrease possible issue with overlap * :shirt: Fix lint issue * :bug: Fix issues with webhooks * :zap: Make error message clearer * :zap: Fix issue with missing execution ID in scaling mode * Fixed execution list to correctly display waiting executins * Feature: enable webhook wait workflows to continue after specified time * Fixed linting * :zap: Improve waiting description text * :zap: Fix parameter display issue and rename * :zap: Remove comment * :zap: Do not display webhooks on Wait-Node * Changed wording from restart to resume on wait node * Fixed wording and inconsistent screen when changing resume modes * Removed dots from the descriptions * Changed docs url and renaming postfix to suffix * Changed names from sleep to wait * :zap: Apply suggestions from ben Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> * Some fixes by Ben * :zap: Remove console.logs * :zap: Fixes and improvements Co-authored-by: Mutasem <mutdmour@gmail.com> Co-authored-by: Iván Ovejero <ivov.src@gmail.com> Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com> Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com> Co-authored-by: Omar Ajoue <krynble@gmail.com>
2021-08-21 05:11:32 -07:00
if (executionId !== undefined) {
// Set the data the webhook node did return on the waiting node if executionId
// already exists as it means that we are restarting an existing execution.
runExecutionData.executionData!.nodeExecutionStack[0].data.main =
webhookResultData.workflowData;
}
2019-06-23 03:35:23 -07:00
if (Object.keys(runExecutionDataMerge).length !== 0) {
// If data to merge got defined add it to the execution data
Object.assign(runExecutionData, runExecutionDataMerge);
}
let pinData: IPinData | undefined;
const usePinData = executionMode === 'manual';
if (usePinData) {
pinData = workflowData.pinData;
runExecutionData.resultData.pinData = pinData;
}
const runData: IWorkflowExecutionDataProcess = {
executionMode,
executionData: runExecutionData,
pushRef,
workflowData,
pinData,
projectId: project?.id,
};
let responsePromise: IDeferredPromise<IN8nHttpFullResponse> | undefined;
if (responseMode === 'responseNode') {
responsePromise = await createDeferredPromise<IN8nHttpFullResponse>();
responsePromise
.promise()
.then(async (response: IN8nHttpFullResponse) => {
if (didSendResponse) {
return;
}
const binaryData = (response.body as IDataObject)?.binaryData as IBinaryData;
if (binaryData?.id) {
res.header(response.headers);
const stream = await Container.get(BinaryDataService).getAsStream(binaryData.id);
stream.pipe(res, { end: false });
await finished(stream);
responseCallback(null, { noWebhookResponse: true });
} else if (Buffer.isBuffer(response.body)) {
res.header(response.headers);
res.end(response.body);
responseCallback(null, { noWebhookResponse: true });
} else {
// TODO: This probably needs some more changes depending on the options on the
// Webhook Response node
const headers = response.headers;
let responseCode = response.statusCode;
let data = response.body as IDataObject;
// for formTrigger node redirection has to be handled by sending redirectURL in response body
if (
nodeType.description.name === 'formTrigger' &&
headers.location &&
String(responseCode).startsWith('3')
) {
responseCode = 200;
data = {
redirectURL: headers.location,
};
headers.location = undefined;
}
responseCallback(null, {
data,
headers,
responseCode,
});
}
process.nextTick(() => res.end());
didSendResponse = true;
})
.catch(async (error) => {
ErrorReporter.error(error);
Container.get(Logger).error(
`Error with Webhook-Response for execution "${executionId}": "${error.message}"`,
{ executionId, workflowId: workflow.id },
);
});
}
2019-06-23 03:35:23 -07:00
// Start now to run the workflow
executionId = await Container.get(WorkflowRunner).run(
runData,
true,
!didSendResponse,
executionId,
responsePromise,
);
2019-06-23 03:35:23 -07:00
Container.get(Logger).verbose(
:sparkles: Added logging to n8n (#1381) * Added logging to n8n This commit adds logging to n8n using the Winston library. For now, this commit only allows logging to console (default behavior) or file (need to pass in config via environment variables). Other logging methods can be further implemented using hooks. These were skipped for now as it would require adding more dependencies. Logging level is notice by default, meaning no additional messages would be displayed at the moment. Logging level can be set to info or debug as well to enrich the generated logs. The ILogger interface was added to the workflow project as it would make it available for all other projects but the implementation was done on the cli project. * Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously * Improvements to logging framework Using appropriate single quotes Improving the way the logger is declared * Improved naming for Log Types * Removed logger global variable, replacing it by a proxy * Add logging to CLI commands * Remove unused GenericHelpers * Changed back some messages to console instead of logger and added npm shortcuts for worker and webhook * Fix typos * Adding basic file rotation to logs as suggested by @mutdmour * Fixed linting issues * Correcting comment to correctly reflect space usage * Added settings for log files rotation * Correcting config type from String to Number * Changed default file settings to number To reflect previous changes to the type * Changed the way log messages are added to be called statically. Also minor naming improvements * Applying latest corrections sent by @ivov * :zap: Some logging improvements * Saving logs to a folder inside n8n home instead of root * Fixed broken tests and linting * Changed some log messages to improve formatting * Adding quotes to names on log messages * Added execution and session IDs to logs. Also removed unnecessary line breaks * :zap: Added file caller to log messages (#1657) This is done using callsites library which already existed in the project as another library's dependency. So in fact it does not add any new dependency. * Adding logs to help debug Salesforce node * :zap: Add function name to logs and add more logs * :zap: Improve some error messages * :zap: Improve some more log messages * :zap: Rename logging env variables to match others Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-05-01 20:43:01 -07:00
`Started execution of workflow "${workflow.name}" from webhook with execution ID ${executionId}`,
{ executionId },
);
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (!didSendResponse) {
// Get a promise which resolves when the workflow did execute and send then response
const executePromise = Container.get(ActiveExecutions).getPostExecutePromise(
executionId,
) as Promise<IExecutionDb | undefined>;
executePromise
// eslint-disable-next-line complexity
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
.then(async (data) => {
if (data === undefined) {
if (!didSendResponse) {
responseCallback(null, {
data: {
message: 'Workflow executed successfully but no data was returned',
},
responseCode,
});
didSendResponse = true;
}
return undefined;
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
}
feat(editor): Add data pinning functionality (#3511) * feat: Design system color improvements and button component redesign. * feat: Added button focus state and unit tests. * refactor: Aligned n8n-button usage inside of editor-ui. * test: Updated snapshots. * refactor: Extracted focus outline width into scss variable. * fix: Fixed select input border-radius. * refactor: Removed element-ui references in button. * fix: Fixed scss variable imports. * feat: Added color-neutral variable story. * fix: Fixed color-secondary variable definition. * feat: Added color-white story. * test: Updated button snapshot. * feat: Replaced zoom buttons with new n8n-icon-button. * feat: Added stories for float utilities. * chore: Updated color shades generation code for later use. * chore: Removed color-white code. * chore: Updated story properties for button components. * fix: Added el-button fallback for places where el-button is not replaceable (messagebox). * feat: Reverted to css modules. Replaced el-button with n8n-button at application level. * test: Updated button snapshot. * fix: Fixed element-ui locally referenced buttons (via components: {}). * fix: Updated colors. Removed irrelevant validation. Added ElButton override component. * test: Updated button override snapshot. * fix: Various button adjustments and fixes. * fix: Updated button disabled state. * test: Updated snapshots. * fix: Consolidated css variables changes. * Data pinning (#3512) * refactor: Aligned n8n-button usage inside of editor-ui. * feat: Added edit data button on json hover. * feat: Extracted code editor into separate form component. * feat: Added edit data button on json hover. * feat: Added pinData and edit mode methods. * :fire: Remove conflict markers * :pencil2: Update i18n keys * :zap: Add JSON validation * :card_file_box: Add `pinData` column to `workflow_entity` * :blue_book: Tighten type * :zap: Make `pinData` column nullable * :zap: Adjust workflow endpoints for pin data * :blue_book: Improve types * :pencil2: Improve wording * Inject pindata into items flow (#3420) * :zap: Inject pin data - Second approach * :fire: Remove unneeded lint exception * feat: Added edit data button on json hover. * feat: Extracted code editor into separate form component. * feat: Added edit data button on json hover. * fix: Fixed rebase conflicts. * :rewind: Undo button change * :bug: Fix runNode call Adjust per update in bdb84130d687811d65337ff6b025e7cb0eae8256 * :test_tube: Fix workflow tests * :bug: More merge conflict fixes * feat: Added pin/unpin button and store mutations. * feat: Size check. Various design and ux improvements. * :zap: Add transformer * :zap: Hoist pin data * :zap: Adjust endpoints for hoisted pin data * :blue_book: Expand interface * :bug: Fix stray array * :shirt: Fix build * :shirt: Add lint exception * :shirt: Fix header * :art: Add color secondary tints * :sparkles: Create `HeaderMessage` component * :zap: Adjust `InfoTip` component * :sparkles: Add `HeaderMessage` to `RunData` * :bug: Fix console error * :shirt: Fix lint * :zap: Consolidate `HeaderMessage` and `Callout` * :rewind: Undo `InfoTip` changes * :fire: Remove duplicate icons * :zap: Simplify template * :art: Change cursor for action text * :shirt: Fix lint * :zap: Add URL * :bug: Fix handler name * :zap: Use constant * :recycle: Refactor per feedback * fix: Various fixes after data pinning relocation. * fix: Added store mutation for setting pinned data. * feat: Added pinned state for workflow canvas node. * fix: Fixed workflow saving. * fix: Removed pinData hoisting (no longer necessary). * feat: Added canPinData flag to hide for input pane and binary data. Fixed unpin and execute flow. * :zap: Fixes for canvas pin data (#3587) * :zap: Fixes for canvas pin data * :blue_book: Rename type * :test_tube: Fix unrelated Public API test * :fire: Remove logging * feat: Updated pinData mixin to no longer include extra fields. * :zap: Output same pindata for every run * :art: Fix cropping * :fire: Remove unrelated logging * feat: Moved edit button next to pin button. * feat: Changed data to be inserted for empty state. * chore: Changed invalid editor output translation. * feat: Added error line reporting on JSON Validation. * feat: Migrated pinData edit mode to store. * chore: Merged duplicate node border color condition. * feat: Moved pin data validation to mixin. Added check before closing ndv modal. * fix: Changed pinned data size calculation to discard active node pin data. * feat: Added support for rename and delete node with pin data. * feat: Simplified editing state. Fixed edit mode in input panel after store migration. * feat: Various data pinning improvements. * fix: Fixed callout link underline. * refactor: Added support for both string and objects for data size check. * feat: Added disabled node check for input panel. Fixed monaco editor resizing. * fix: Fixed edit mode footer size. * :zap: Fix pindata items per run * :shirt: Remove unneeded exception * refactor: Added isValidPinData() helper method. * refactor: Changed how string size in bytes in calculated.g * refactor: Updated pinData mixin interface. * refactor: Merged filter and reduce in pinDataSize calculation. * fix: Changed code-editor to correct type. * fix: Added insert test data message to trigger nodes. * feat: Disabled data pinning for multiple output nodes. * refactor: Updated ndv.input.disabled translation to include node name. * refactor: Aligned n8n-button usage inside of editor-ui. * feat: Added edit data button on json hover. * feat: Extracted code editor into separate form component. * feat: Added edit data button on json hover. * feat: Added pinData and edit mode methods. * :fire: Remove conflict markers * :pencil2: Update i18n keys * :zap: Add JSON validation * :card_file_box: Add `pinData` column to `workflow_entity` * :blue_book: Tighten type * :zap: Make `pinData` column nullable * :zap: Adjust workflow endpoints for pin data * :blue_book: Improve types * :pencil2: Improve wording * Inject pindata into items flow (#3420) * :zap: Inject pin data - Second approach * :fire: Remove unneeded lint exception * feat: Added edit data button on json hover. * feat: Extracted code editor into separate form component. * feat: Added edit data button on json hover. * fix: Fixed rebase conflicts. * :rewind: Undo button change * :bug: Fix runNode call Adjust per update in bdb84130d687811d65337ff6b025e7cb0eae8256 * :test_tube: Fix workflow tests * :bug: More merge conflict fixes * feat: Added pin/unpin button and store mutations. * feat: Size check. Various design and ux improvements. * :zap: Add transformer * :zap: Hoist pin data * :zap: Adjust endpoints for hoisted pin data * :blue_book: Expand interface * :bug: Fix stray array * :shirt: Fix build * :art: Add color secondary tints * :sparkles: Create `HeaderMessage` component * :zap: Adjust `InfoTip` component * :sparkles: Add `HeaderMessage` to `RunData` * :bug: Fix console error * :shirt: Fix lint * :zap: Consolidate `HeaderMessage` and `Callout` * :rewind: Undo `InfoTip` changes * :fire: Remove duplicate icons * :zap: Simplify template * :art: Change cursor for action text * :shirt: Fix lint * :zap: Add URL * :bug: Fix handler name * :zap: Use constant * :recycle: Refactor per feedback * fix: Various fixes after data pinning relocation. * fix: Added store mutation for setting pinned data. * feat: Added pinned state for workflow canvas node. * :zap: Fixes for canvas pin data (#3587) * :zap: Fixes for canvas pin data * :blue_book: Rename type * :test_tube: Fix unrelated Public API test * :fire: Remove logging * feat: Updated pinData mixin to no longer include extra fields. * fix: Removed pinData hoisting (no longer necessary). * chore: Merged duplicate node border color condition. * :zap: Output same pindata for every run * :art: Fix cropping * :bug: Fix excess closing template tag * fix: Removed rogue template tag after merge. * fix: Fixed code-editor resizing when moving ndv panel. * feat: Added node duplication pin data. * :zap: Implement telemetry * :recycle: Add clarifications from call * fix: Fixed run data header height. * feat: Removed border from pin data callout. * feat: Added line-break before 'or insert pin data'. * feat: Changed enterEditMode to always insert test data if there's no execution data. * feat: Removed copy output tooltip. * feat: Removed unpin tooltip. * fix: Removed thumbtack icon rotation. * fix: Removed run info from Edit Output title. * feat: Hid edit and pin buttons when editing. * feat: Updated monaco code-editor padding and borders. * feat: Progress on pinData error message format * feat: Updated copy feature to work without any selected value. * feat: Moved save and cancel buttons. Cleared notifications on save. * feat: Changed pin data beforeClosing confirm text. * feat: Closing ndv when discarding or saving pindata on close. * feat: Added split in batches node to pin data denylist. * fix: Added missing margin-bottom to webhook node. * feat: Moved thumbtack icon to the right, replacing the checkmark. * fix: Hid pagination while editing. * feat: Added pin data discovery flow. * feat: Changed pin data discovery flow to avoid tooltip glitching. * fix: Changed copy selection to copy all input data. * feat: Updated pin data validation error message for unexpected single quotes. * fix: Replaced :manual='true' prop with manual shorthand. * fix: Removed unused variable. * chore: Renamed translation key to node.discovery.pinData. * refactor: Extracted isPinDataNodeType to pinData mixin. * fix: Updated watch condition to improve performance. * refactor: Renamed some pin data variables and methods as per review. * fix: Added partial translation for JSON.parse pin data error messages. * chore: Temporarily disabled failing unit test. * :test_tube: Fix data pinning workflow retrieval test * :fire: Remove unused imports * :fire: Remove leftover line * :zap: Skip pindata node issues on BE * :zap: Skip pindata node issues on FE * :zap: Hide `RunInfo` for pindata node * :zap: Hide purple banner in edit output mode * feat: Updated data pinning discoverability flow. * fix: Fixed paginated data pinning. * fix: Disabled pin data in read only mode. * :bug: Fix runtime error with non-array * fix: Loading pin data when opening execution. * :zap: Adjust stale data warning for pinned data * :zap: Skip auth in endpoint * :zap: Mark start node for pinned trigger * :pencil2: Comment on passthrough * :fire: Remove comment * Final pindata metrics changes (#3673) * :bug: Fix `pinData` tracked as `0` * :zap: Add `is_pinned` to `nodesGraph` * :blue_book: Extend `IWorkflowBase` * :zap: Handle `pinData` being `undefined` * :zap: Add `data_pinning_tooltip_presented` * :recycle: Refactor to remove circular dependency * fix: Added pin data handling when importing workflow. (#3698) * :fire: Remove helper from WorkflowExecute * :zap: Add logic for single pinned trigger * :shirt: Remove lint exception * fix: Added pin data handling in importWorkflowExact. * N8N-4077 data pinning discoverability part 2 (#3701) * fix: Fixed pin data discovery tooltip position when moving canvas. * feat: Updated data pinning discovery tooltip copy. * Fix data pinning build (#3702) * :zap: Disable edit button for disabled node * :zap: Ensure disabled pinned nodes are passthrough * :bug: Fix JSON key unfurling in edit mode * :zap: Improve implementation * :bug: Fix console error * fix: Fixed copying pinned output data. (#3715) * Fix pinning for webhook responding with output from last node (#3719) * fix: Fixed entering edit mode after refresh. * fix: Fixed type error during build. * fix: RunData import formatting. * chore: Updated pin data types. * fix: Added missing type to stringSizeInBytes. Co-authored-by: Iván Ovejero <ivov.src@gmail.com> * fix: Showing pin data without executing the node only in output pane. * fix: Updated no data message when previous node not executed. * feat: Added expression input and evaluation for pin data nodes without execution. * chore: Fixed linting issues and removed remnant console.log(). * chore: Undone package-lock changes. * fix: Removed pin data store changes. * fix: Created a new object using vuex runExecutionData. * fix: Fixed bug appearing when adding a new node after executing. * fix: Fix editor-ui build * feat: Added green node connectors when having pin data output. * chore: Fixed linting errors. * fix: Added pin data eventBus unsubscribe. * fix: Added pin data color check after adding a connection. * :art: Add pindata styles Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2022-07-20 08:50:39 -07:00
if (usePinData) {
data.data.resultData.pinData = pinData;
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
const returnData = WorkflowHelpers.getDataLastExecutedNodeData(data);
if (data.data.resultData.error || returnData?.error !== undefined) {
if (!didSendResponse) {
responseCallback(null, {
data: {
message: 'Error in workflow',
},
responseCode: 500,
});
}
didSendResponse = true;
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
return data;
}
// in `responseNode` mode `responseCallback` is called by `responsePromise`
if (responseMode === 'responseNode' && responsePromise) {
await Promise.allSettled([responsePromise.promise()]);
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
return undefined;
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
}
2019-06-23 03:35:23 -07:00
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (returnData === undefined) {
if (!didSendResponse) {
responseCallback(null, {
data: {
message:
'Workflow executed successfully but the last node did not return any data',
},
responseCode,
});
}
didSendResponse = true;
return data;
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
const additionalKeys: IWorkflowDataProxyAdditionalKeys = {
$executionId: executionId,
};
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (!didSendResponse) {
let data: IDataObject | IDataObject[] | undefined;
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (responseData === 'firstEntryJson') {
// Return the JSON data of the first entry
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (returnData.data!.main[0]![0] === undefined) {
responseCallback(new Error('No item to return got found'), {});
didSendResponse = true;
return undefined;
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
data = returnData.data!.main[0]![0].json;
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
const responsePropertyName = workflow.expression.getSimpleParameterValue(
workflowStartNode,
webhookData.webhookDescription.responsePropertyName,
executionMode,
additionalKeys,
undefined,
undefined,
);
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (responsePropertyName !== undefined) {
data = get(data, responsePropertyName as string) as IDataObject;
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
const responseContentType = workflow.expression.getSimpleParameterValue(
workflowStartNode,
webhookData.webhookDescription.responseContentType,
executionMode,
additionalKeys,
undefined,
undefined,
);
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (responseContentType !== undefined) {
// Send the webhook response manually to be able to set the content-type
res.setHeader('Content-Type', responseContentType as string);
// Returning an object, boolean, number, ... causes problems so make sure to stringify if needed
if (
data !== null &&
data !== undefined &&
['Buffer', 'String'].includes(data.constructor.name)
) {
res.end(data);
} else {
res.end(JSON.stringify(data));
}
responseCallback(null, {
noWebhookResponse: true,
});
didSendResponse = true;
}
} else if (responseData === 'firstEntryBinary') {
// Return the binary data of the first entry
data = returnData.data!.main[0]![0];
if (data === undefined) {
responseCallback(new Error('No item was found to return'), {});
didSendResponse = true;
return undefined;
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (data.binary === undefined) {
responseCallback(new Error('No binary data was found to return'), {});
didSendResponse = true;
return undefined;
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
const responseBinaryPropertyName = workflow.expression.getSimpleParameterValue(
workflowStartNode,
webhookData.webhookDescription.responseBinaryPropertyName,
executionMode,
additionalKeys,
undefined,
'data',
);
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (responseBinaryPropertyName === undefined && !didSendResponse) {
responseCallback(new Error("No 'responseBinaryPropertyName' is set"), {});
didSendResponse = true;
}
2019-06-23 03:35:23 -07:00
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
const binaryData = (data.binary as IBinaryKeyData)[
responseBinaryPropertyName as string
];
if (binaryData === undefined && !didSendResponse) {
responseCallback(
new Error(
`The binary property '${responseBinaryPropertyName}' which should be returned does not exist`,
),
{},
);
didSendResponse = true;
}
2019-06-23 03:35:23 -07:00
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (!didSendResponse) {
// Send the webhook response manually
res.setHeader('Content-Type', binaryData.mimeType);
if (binaryData.id) {
const stream = await Container.get(BinaryDataService).getAsStream(binaryData.id);
stream.pipe(res, { end: false });
await finished(stream);
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
} else {
res.write(Buffer.from(binaryData.data, BINARY_ENCODING));
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
}
responseCallback(null, {
noWebhookResponse: true,
});
process.nextTick(() => res.end());
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
}
} else if (responseData === 'noData') {
// Return without data
data = undefined;
} else {
// Return the JSON data of all the entries
data = [];
for (const entry of returnData.data!.main[0]!) {
data.push(entry.json);
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
}
2019-06-23 03:35:23 -07:00
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
if (!didSendResponse) {
2019-06-23 03:35:23 -07:00
responseCallback(null, {
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
data,
responseCode,
2019-06-23 03:35:23 -07:00
});
:art: Set up linting and formatting (#2120) * :arrow_up: Upgrade TS to 4.3.5 * :shirt: Add ESLint configs * :art: Add Prettier config * :package: Add deps and commands * :zap: Adjust global .editorconfig to new ruleset * :fire: Remove unneeded local .editorconfig * :package: Update deps in editor-ui * :hammer: Limit Prettier to only TS files * :zap: Add recommended VSCode extensions * :shirt: Fix build * :fire: Remove Vue setting from global config * :zap: Disable prefer-default-export per feedback * :pencil2: Add forgotten divider * :shirt: Disable no-plusplus * :shirt: Disable class-methods-use-this * :pencil2: Alphabetize overrides * :shirt: Add one-var consecutive override * :rewind: Revert one-var consecutive override This reverts commit b9252cf935659ba6d76727ad484a1d3c00008fcc. * 🎨 👕 Lint and format workflow package (#2121) * :art: Format /workflow package * :shirt: Lint /workflow package * :art: Re-format /workflow package * :shirt: Re-lint /workflow package * :pencil2: Fix typo * :zap: Consolidate if-checks * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format node-dev package (#2122) * :art: Format /node-dev package * :zap: Exclude templates from ESLint config This keeps the templates consistent with the codebase while preventing lint exceptions from being made part of the templates. * :shirt: Lint /node-dev package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * 🎨 👕 Lint and format core package (#2123) * :art: Format /core package * :shirt: Lint /core package * :art: Re-format /core package * :shirt: Re-lint /core package * :fire: Remove prefer-default-export exceptions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 🎨 👕 Lint and format cli package (#2124) * :art: Format /cli package * :shirt: Exclude migrations from linting * :shirt: Lint /cli package * :art: Re-format /cli package * :shirt: Re-lint /cli package * :shirt: Fix build * :fire: Remove prefer-default-export exceptions * :zap: Update exceptions in ActiveExecutions * :fire: Remove no-plusplus exceptions * :fire: Remove class-methods-use-this exceptions * 👕 fix lint issues * :wrench: use package specific linter, remove tslint command * :hammer: resolve build issue, sync dependencies * :wrench: change lint command Co-authored-by: Ben Hesseldieck <b.hesseldieck@gmail.com>
2021-08-29 11:58:11 -07:00
}
2019-06-23 03:35:23 -07:00
}
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
didSendResponse = true;
2019-06-23 03:35:23 -07:00
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
return data;
})
.catch((e) => {
if (!didSendResponse) {
responseCallback(
new ApplicationError('There was a problem executing the workflow', {
level: 'warning',
cause: e,
}),
{},
);
}
2019-06-23 03:35:23 -07:00
throw new InternalServerError(e.message);
refactor(core): Move event and telemetry handling into workers in queue mode (#7138) # Motivation In Queue mode, finished executions would cause the main instance to always pull all execution data from the database, unflatten it and then use it to send out event log events and telemetry events, as well as required returns to Respond to Webhook nodes etc. This could cause OOM errors when the data was large, since it had to be fully unpacked and transformed on the main instance’s side, using up a lot of memory (and time). This PR attempts to limit this behaviour to only happen in those required cases where the data has to be forwarded to some waiting webhook, for example. # Changes Execution data is only required in cases, where the active execution has a `postExecutePromise` attached to it. These usually forward the data to some other endpoint (e.g. a listening webhook connection). By adding a helper `getPostExecutePromiseCount()`, we can decide that in cases where there is nothing listening at all, there is no reason to pull the data on the main instance. Previously, there would always be postExecutePromises because the telemetry events were called. Now, these have been moved into the workers, which have been given the various InternalHooks calls to their hook function arrays, so they themselves issue these telemetry and event calls. This results in all event log messages to now be logged on the worker’s event log, as well as the worker’s eventbus being the one to send out the events to destinations. The main event log does…pretty much nothing. We are not logging executions on the main event log any more, because this would require all events to be replicated 1:1 from the workers to the main instance(s) (this IS possible and implemented, see the worker’s `replicateToRedisEventLogFunction` - but it is not enabled to reduce the amount of traffic over redis). Partial events in the main log could confuse the recovery process and would result in, ironically, the recovery corrupting the execution data by considering them crashed. # Refactor I have also used the opportunity to reduce duplicate code and move some of the hook functionality into `packages/cli/src/executionLifecycleHooks/shared/sharedHookFunctions.ts` in preparation for a future full refactor of the hooks
2023-09-13 22:58:15 -07:00
});
}
2019-06-23 03:35:23 -07:00
return executionId;
} catch (e) {
const error =
e instanceof UnprocessableRequestError
? e
: new ApplicationError('There was a problem executing the workflow', {
level: 'warning',
cause: e,
});
if (didSendResponse) throw error;
responseCallback(error, {});
return;
2019-06-23 03:35:23 -07:00
}
}