n8n/packages/cli/src/eventbus/MessageEventBusDestination/MessageEventBusDestination.ee.ts

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

132 lines
3.8 KiB
TypeScript
Raw Normal View History

feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
import { v4 as uuid } from 'uuid';
import { Container } from 'typedi';
import type { DeleteResult, InsertResult } from 'typeorm';
import type { INodeCredentials, MessageEventBusDestinationOptions } from 'n8n-workflow';
import { MessageEventBusDestinationTypeNames } from 'n8n-workflow';
import { Logger } from '@/Logger';
import type { AbstractEventMessage } from '../EventMessageClasses/AbstractEventMessage';
import type { EventMessageTypes } from '../EventMessageClasses';
feat(core): Add execution runData recovery and status field (#5112) * adds ExecutionEvents view modal to ExecutionList * fix time rendering and remove wf column * checks for unfinished executions and fails them * prevent re-setting stoppedAt for execution * some cleanup / manually create rundata after crash * quicksave * remove Threads lib, log worker rewrite * cleanup comment * fix sentry destination return value * test for tests... * run tests with single worker * fix tests * remove console log * add endpoint for execution data recovery * lint cleanup and some refactoring * fix accidental recursion * remove cyclic imports * add rundata recovery to Workflowrunner * remove comments * cleanup and refactor * adds a status field to executions * setExecutionStatus on queued worker * fix onWorkflowPostExecute * set waiting from worker * get crashed status into frontend * remove comment * merge fix * cleanup * catch empty rundata in recovery * refactor IExecutionsSummary and inject nodeExecution Errors * reduce default event log size to 10mb from 100mb * add per node execution status * lint fix * merge and lint fix * phrasing change * improve preview rendering and messaging * remove debug * Improve partial rundata recovery * fix labels * fix line through * send manual rundata to ui at crash * some type and msg push fixes * improve recovered item rendering in preview * update workflowStatistics on recover * merge fix * review fixes * merge fix * notify eventbus when ui is back up * add a small timeout to make sure the UI is back up * increase reconnect timeout to 30s * adjust recover timeout and ui connection lost msg * do not stop execution in editor after x reconnects * add executionRecovered push event * fix recovered connection not green * remove reconnect toast and merge existing rundata * merge editor and recovered data for own mode
2023-02-17 01:54:07 -08:00
import type { EventMessageConfirmSource } from '../EventMessageClasses/EventMessageConfirm';
import type { MessageEventBus, MessageWithCallback } from '../MessageEventBus/MessageEventBus';
import { EventDestinationsRepository } from '@db/repositories/eventDestinations.repository';
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
export abstract class MessageEventBusDestination implements MessageEventBusDestinationOptions {
// Since you can't have static abstract functions - this just serves as a reminder that you need to implement these. Please.
// static abstract deserialize(): MessageEventBusDestination | null;
readonly id: string;
feat(core): Add execution runData recovery and status field (#5112) * adds ExecutionEvents view modal to ExecutionList * fix time rendering and remove wf column * checks for unfinished executions and fails them * prevent re-setting stoppedAt for execution * some cleanup / manually create rundata after crash * quicksave * remove Threads lib, log worker rewrite * cleanup comment * fix sentry destination return value * test for tests... * run tests with single worker * fix tests * remove console log * add endpoint for execution data recovery * lint cleanup and some refactoring * fix accidental recursion * remove cyclic imports * add rundata recovery to Workflowrunner * remove comments * cleanup and refactor * adds a status field to executions * setExecutionStatus on queued worker * fix onWorkflowPostExecute * set waiting from worker * get crashed status into frontend * remove comment * merge fix * cleanup * catch empty rundata in recovery * refactor IExecutionsSummary and inject nodeExecution Errors * reduce default event log size to 10mb from 100mb * add per node execution status * lint fix * merge and lint fix * phrasing change * improve preview rendering and messaging * remove debug * Improve partial rundata recovery * fix labels * fix line through * send manual rundata to ui at crash * some type and msg push fixes * improve recovered item rendering in preview * update workflowStatistics on recover * merge fix * review fixes * merge fix * notify eventbus when ui is back up * add a small timeout to make sure the UI is back up * increase reconnect timeout to 30s * adjust recover timeout and ui connection lost msg * do not stop execution in editor after x reconnects * add executionRecovered push event * fix recovered connection not green * remove reconnect toast and merge existing rundata * merge editor and recovered data for own mode
2023-02-17 01:54:07 -08:00
readonly eventBusInstance: MessageEventBus;
protected readonly logger: Logger;
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
__type: MessageEventBusDestinationTypeNames;
label: string;
enabled: boolean;
subscribedEvents: string[];
credentials: INodeCredentials = {};
anonymizeAuditMessages: boolean;
feat(core): Add execution runData recovery and status field (#5112) * adds ExecutionEvents view modal to ExecutionList * fix time rendering and remove wf column * checks for unfinished executions and fails them * prevent re-setting stoppedAt for execution * some cleanup / manually create rundata after crash * quicksave * remove Threads lib, log worker rewrite * cleanup comment * fix sentry destination return value * test for tests... * run tests with single worker * fix tests * remove console log * add endpoint for execution data recovery * lint cleanup and some refactoring * fix accidental recursion * remove cyclic imports * add rundata recovery to Workflowrunner * remove comments * cleanup and refactor * adds a status field to executions * setExecutionStatus on queued worker * fix onWorkflowPostExecute * set waiting from worker * get crashed status into frontend * remove comment * merge fix * cleanup * catch empty rundata in recovery * refactor IExecutionsSummary and inject nodeExecution Errors * reduce default event log size to 10mb from 100mb * add per node execution status * lint fix * merge and lint fix * phrasing change * improve preview rendering and messaging * remove debug * Improve partial rundata recovery * fix labels * fix line through * send manual rundata to ui at crash * some type and msg push fixes * improve recovered item rendering in preview * update workflowStatistics on recover * merge fix * review fixes * merge fix * notify eventbus when ui is back up * add a small timeout to make sure the UI is back up * increase reconnect timeout to 30s * adjust recover timeout and ui connection lost msg * do not stop execution in editor after x reconnects * add executionRecovered push event * fix recovered connection not green * remove reconnect toast and merge existing rundata * merge editor and recovered data for own mode
2023-02-17 01:54:07 -08:00
constructor(eventBusInstance: MessageEventBus, options: MessageEventBusDestinationOptions) {
this.logger = Container.get(Logger);
feat(core): Add execution runData recovery and status field (#5112) * adds ExecutionEvents view modal to ExecutionList * fix time rendering and remove wf column * checks for unfinished executions and fails them * prevent re-setting stoppedAt for execution * some cleanup / manually create rundata after crash * quicksave * remove Threads lib, log worker rewrite * cleanup comment * fix sentry destination return value * test for tests... * run tests with single worker * fix tests * remove console log * add endpoint for execution data recovery * lint cleanup and some refactoring * fix accidental recursion * remove cyclic imports * add rundata recovery to Workflowrunner * remove comments * cleanup and refactor * adds a status field to executions * setExecutionStatus on queued worker * fix onWorkflowPostExecute * set waiting from worker * get crashed status into frontend * remove comment * merge fix * cleanup * catch empty rundata in recovery * refactor IExecutionsSummary and inject nodeExecution Errors * reduce default event log size to 10mb from 100mb * add per node execution status * lint fix * merge and lint fix * phrasing change * improve preview rendering and messaging * remove debug * Improve partial rundata recovery * fix labels * fix line through * send manual rundata to ui at crash * some type and msg push fixes * improve recovered item rendering in preview * update workflowStatistics on recover * merge fix * review fixes * merge fix * notify eventbus when ui is back up * add a small timeout to make sure the UI is back up * increase reconnect timeout to 30s * adjust recover timeout and ui connection lost msg * do not stop execution in editor after x reconnects * add executionRecovered push event * fix recovered connection not green * remove reconnect toast and merge existing rundata * merge editor and recovered data for own mode
2023-02-17 01:54:07 -08:00
this.eventBusInstance = eventBusInstance;
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
this.id = !options.id || options.id.length !== 36 ? uuid() : options.id;
this.__type = options.__type ?? MessageEventBusDestinationTypeNames.abstract;
this.label = options.label ?? 'Log Destination';
this.enabled = options.enabled ?? false;
this.subscribedEvents = options.subscribedEvents ?? [];
this.anonymizeAuditMessages = options.anonymizeAuditMessages ?? false;
if (options.credentials) this.credentials = options.credentials;
this.logger.debug(`${this.__type}(${this.id}) event destination constructed`);
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
}
startListening() {
if (this.enabled) {
feat(core): Add execution runData recovery and status field (#5112) * adds ExecutionEvents view modal to ExecutionList * fix time rendering and remove wf column * checks for unfinished executions and fails them * prevent re-setting stoppedAt for execution * some cleanup / manually create rundata after crash * quicksave * remove Threads lib, log worker rewrite * cleanup comment * fix sentry destination return value * test for tests... * run tests with single worker * fix tests * remove console log * add endpoint for execution data recovery * lint cleanup and some refactoring * fix accidental recursion * remove cyclic imports * add rundata recovery to Workflowrunner * remove comments * cleanup and refactor * adds a status field to executions * setExecutionStatus on queued worker * fix onWorkflowPostExecute * set waiting from worker * get crashed status into frontend * remove comment * merge fix * cleanup * catch empty rundata in recovery * refactor IExecutionsSummary and inject nodeExecution Errors * reduce default event log size to 10mb from 100mb * add per node execution status * lint fix * merge and lint fix * phrasing change * improve preview rendering and messaging * remove debug * Improve partial rundata recovery * fix labels * fix line through * send manual rundata to ui at crash * some type and msg push fixes * improve recovered item rendering in preview * update workflowStatistics on recover * merge fix * review fixes * merge fix * notify eventbus when ui is back up * add a small timeout to make sure the UI is back up * increase reconnect timeout to 30s * adjust recover timeout and ui connection lost msg * do not stop execution in editor after x reconnects * add executionRecovered push event * fix recovered connection not green * remove reconnect toast and merge existing rundata * merge editor and recovered data for own mode
2023-02-17 01:54:07 -08:00
this.eventBusInstance.on(
this.getId(),
async (
msg: EventMessageTypes,
confirmCallback: (message: EventMessageTypes, src: EventMessageConfirmSource) => void,
) => {
await this.receiveFromEventBus({ msg, confirmCallback });
},
);
this.logger.debug(`${this.id} listener started`);
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
}
}
stopListening() {
feat(core): Add execution runData recovery and status field (#5112) * adds ExecutionEvents view modal to ExecutionList * fix time rendering and remove wf column * checks for unfinished executions and fails them * prevent re-setting stoppedAt for execution * some cleanup / manually create rundata after crash * quicksave * remove Threads lib, log worker rewrite * cleanup comment * fix sentry destination return value * test for tests... * run tests with single worker * fix tests * remove console log * add endpoint for execution data recovery * lint cleanup and some refactoring * fix accidental recursion * remove cyclic imports * add rundata recovery to Workflowrunner * remove comments * cleanup and refactor * adds a status field to executions * setExecutionStatus on queued worker * fix onWorkflowPostExecute * set waiting from worker * get crashed status into frontend * remove comment * merge fix * cleanup * catch empty rundata in recovery * refactor IExecutionsSummary and inject nodeExecution Errors * reduce default event log size to 10mb from 100mb * add per node execution status * lint fix * merge and lint fix * phrasing change * improve preview rendering and messaging * remove debug * Improve partial rundata recovery * fix labels * fix line through * send manual rundata to ui at crash * some type and msg push fixes * improve recovered item rendering in preview * update workflowStatistics on recover * merge fix * review fixes * merge fix * notify eventbus when ui is back up * add a small timeout to make sure the UI is back up * increase reconnect timeout to 30s * adjust recover timeout and ui connection lost msg * do not stop execution in editor after x reconnects * add executionRecovered push event * fix recovered connection not green * remove reconnect toast and merge existing rundata * merge editor and recovered data for own mode
2023-02-17 01:54:07 -08:00
this.eventBusInstance.removeAllListeners(this.getId());
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
}
enable() {
this.enabled = true;
this.startListening();
}
disable() {
this.enabled = false;
this.stopListening();
}
getId() {
return this.id;
}
hasSubscribedToEvent(msg: AbstractEventMessage) {
if (!this.enabled) return false;
for (const eventName of this.subscribedEvents) {
if (eventName === '*' || msg.eventName.startsWith(eventName)) {
return true;
}
}
return false;
}
async saveToDb() {
const data = {
id: this.getId(),
destination: this.serialize(),
};
const dbResult: InsertResult = await Container.get(EventDestinationsRepository).upsert(data, {
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
skipUpdateIfNoValuesChanged: true,
conflictPaths: ['id'],
});
return dbResult;
}
async deleteFromDb() {
return MessageEventBusDestination.deleteFromDb(this.getId());
}
static async deleteFromDb(id: string): Promise<DeleteResult> {
const dbResult = await Container.get(EventDestinationsRepository).delete({ id });
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
return dbResult;
}
serialize(): MessageEventBusDestinationOptions {
return {
__type: this.__type,
id: this.getId(),
label: this.label,
enabled: this.enabled,
subscribedEvents: this.subscribedEvents,
anonymizeAuditMessages: this.anonymizeAuditMessages,
};
}
feat(core): Add execution runData recovery and status field (#5112) * adds ExecutionEvents view modal to ExecutionList * fix time rendering and remove wf column * checks for unfinished executions and fails them * prevent re-setting stoppedAt for execution * some cleanup / manually create rundata after crash * quicksave * remove Threads lib, log worker rewrite * cleanup comment * fix sentry destination return value * test for tests... * run tests with single worker * fix tests * remove console log * add endpoint for execution data recovery * lint cleanup and some refactoring * fix accidental recursion * remove cyclic imports * add rundata recovery to Workflowrunner * remove comments * cleanup and refactor * adds a status field to executions * setExecutionStatus on queued worker * fix onWorkflowPostExecute * set waiting from worker * get crashed status into frontend * remove comment * merge fix * cleanup * catch empty rundata in recovery * refactor IExecutionsSummary and inject nodeExecution Errors * reduce default event log size to 10mb from 100mb * add per node execution status * lint fix * merge and lint fix * phrasing change * improve preview rendering and messaging * remove debug * Improve partial rundata recovery * fix labels * fix line through * send manual rundata to ui at crash * some type and msg push fixes * improve recovered item rendering in preview * update workflowStatistics on recover * merge fix * review fixes * merge fix * notify eventbus when ui is back up * add a small timeout to make sure the UI is back up * increase reconnect timeout to 30s * adjust recover timeout and ui connection lost msg * do not stop execution in editor after x reconnects * add executionRecovered push event * fix recovered connection not green * remove reconnect toast and merge existing rundata * merge editor and recovered data for own mode
2023-02-17 01:54:07 -08:00
abstract receiveFromEventBus(emitterPayload: MessageWithCallback): Promise<boolean>;
feat: Add global event bus (#4860) * fix branch * fix deserialize, add filewriter * add catchAll eventGroup/Name * adding simple Redis sender and receiver to eventbus * remove native node threads * improve eventbus * refactor and simplify * more refactoring and syslog client * more refactor, improved endpoints and eventbus * remove local broker and receivers from mvp * destination de/serialization * create MessageEventBusDestinationEntity * db migrations, load destinations at startup * add delete destination endpoint * pnpm merge and circular import fix * delete destination fix * trigger log file shuffle after size reached * add environment variables for eventbus * reworking event messages * serialize to thread fix * some refactor and lint fixing * add emit to eventbus * cleanup and fix sending unsent * quicksave frontend trial * initial EventTree vue component * basic log streaming settings in vue * http request code merge * create destination settings modals * fix eventmessage options types * credentials are loaded * fix and clean up frontend code * move request code to axios * update lock file * merge fix * fix redis build * move destination interfaces into workflow pkg * revive sentry as destination * migration fixes and frontend cleanup * N8N-5777 / N8N-5789 N8N-5788 * N8N-5784 * N8N-5782 removed event levels * N8N-5790 sentry destination cleanup * N8N-5786 and refactoring * N8N-5809 and refactor/cleanup * UI fixes and anonymize renaming * N8N-5837 * N8N-5834 * fix no-items UI issues * remove card / settings label in modal * N8N-5842 fix * disable webhook auth for now and update ui * change sidebar to tabs * remove payload option * extend audit events with more user data * N8N-5853 and UI revert to sidebar * remove redis destination * N8N-5864 / N8N-5868 / N8N-5867 / N8N-5865 * ui and licensing fixes * add node events and info bubbles to frontend * ui wording changes * frontend tests * N8N-5896 and ee rename * improves backend tests * merge fix * fix backend test * make linter happy * remove unnecessary cfg / limit actions to owners * fix multiple sentry DSN and anon bug * eslint fix * more tests and fixes * merge fix * fix workflow audit events * remove 'n8n.workflow.execution.error' event * merge fix * lint fix * lint fix * review fixes * fix merge * prettier fixes * merge * review changes * use loggerproxy * remove catch from internal hook promises * fix tests * lint fix * include review PR changes * review changes * delete duplicate lines from a bad merge * decouple log-streaming UI options from public API * logstreaming -> log-streaming for consistency * do not make unnecessary api calls when log streaming is disabled * prevent sentryClient.close() from being called if init failed * fix the e2e test for log-streaming * review changes * cleanup * use `private` for one last private property * do not use node prefix package names.. just yet * remove unused import * fix the tests because there is a folder called `events`, tsc-alias is messing up all imports for native events module. https://github.com/justkey007/tsc-alias/issues/152 Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-01-04 00:47:48 -08:00
toString() {
return JSON.stringify(this.serialize());
}
close(): void | Promise<void> {
this.stopListening();
}
}