n8n/packages/nodes-base/nodes/Code/Sandbox.ts

272 lines
8.8 KiB
TypeScript
Raw Normal View History

feat(Code Node): create Code node (#3965) * Introduce node deprecation (#3930) :sparkles: Introduce node deprecation * :construction: Scaffold out Code node * :shirt: Fix lint * :blue_book: Create types file * :truck: Rename theme * :fire: Remove unneeded prop * :zap: Override keybindings * :zap: Expand lintings * :zap: Create editor content getter * :truck: Ensure all helpers use `$` * :sparkles: Add autocompletion * :zap: Filter out welcome note node * :zap: Convey error line number * :zap: Highlight error line * :zap: Restore logging from node * :sparkles: More autocompletions * :zap: Streamline completions * :pencil2: Update placeholders * :zap: Update linter to new methods * :fire: Remove `$nodeItem` completions * :zap: Re-update placeholders * :art: Fix formatting * :package: Update `package-lock.json` * :zap: Refresh with multi-line empty string * :zap: Account for syntax errors * :fire: Remove unneeded variant * :zap: Minor improvements * :zap: Add more autocompletions * :truck: Rename extension * :fire: Remove outdated comments * :truck: Rename field * :sparkles: More autocompletions * :zap: Fix up error display when empty text * :fire: Remove logging * :sparkles: More error validation * :bug: Fix `pairedItem` to `pairedItem()` * :zap: Add item to validation info * :package: Update `package-lock.json` * :zap: Leftover fixes * :zap: Set `insertNewlineAndIndent` * :package: Update `package-lock.json` * :package: Re-update `package-lock.json` * :shirt: Add lint exception * :blue_book: Add type to mixin type * Clean up comment * :zap: Refactor completion per new requirements * :zap: Adjust placeholders * :zap: Add `json` autocompletions for `$input` * :art: Set border * :zap: Restore local completion source * :zap: Implement autocompletion for imports * :zap: Add `.*` to follow user typing on autocompletion * :blue_book: Fix typings in autocompletions * :shirt: Add linting for use of `item()` * :package: Update `package-lock.json` * :bug: Fix for `$items(nodeName)[0]` * :zap: Filter down built-in modules list * :zap: Refactor error handling * :zap: Linter and validation improvements * :zap: Apply review feedback * :recycle: More general refactorings * :zap: Add dot notation utility * Customize input handler * :zap: Support `.json.` completions * :zap: Adjust placeholder * :zap: Sort imports * :fire: Remove blank rows addition * :zap: Add more error validation * :package: Update `package-lock.json` * :zap: Make date logging consistent * :wrench: Adjust linting highlight range * :zap: Add line numbers to each item mode errors * :zap: Allow for links in error descriptions * :zap: More input validation * :zap: Expand linting to loops * :zap: Deprecate Function and Function Item nodes * :bug: Fix placeholder syntax * :blue_book: Narrow down type * :truck: Rename using kebab-case * :fire: Remove `mapGetters` * :pencil2: Fix casing * :zap: Adjust import for type * :pencil2: Fix quotes * :bug: Fix `activeNode` reference * :zap: Use constant * :fire: Remove logging * :pencil2: Fix typo * :zap: Add missing `notice` * :pencil2: Add tags * :pencil2: Fix alias * :pencil2: Update copy * :fire: Remove wrong linting * :pencil2: Update copy * :zap: Add validation for `null` * :zap: Add validation for non-object and non-array * :zap: Add validation for non-array with json * :pencil2: Intentionally use wrong spelling * :zap: More validation * :pencil2: More copy updates * :pencil2: Placeholder updates * :rewind: Restore spelling * :zap: Fix var name * :pencil2: More copy updates * :zap: Add luxon autocompletions * :zap: Make scrollable * :zap: Fix comma from merge conflict resolution * :package: Update `package-lock.json` * :shirt: Fix lint detail * :art: Set font family * :zap: Bring in expressions fix * :recycle: Address feedback * :zap: Exclude codemirror packages from render chunks * :bug: Fix placeholder not showing on first load * feat(editor-ui): Replace `lezer` with `esprima` in client linter (#4192) * :fire: Remove addition from misresolved conflict * :zap: Replace `lezer` with `esprima` in client linter * :zap: Add missing key * :package: Update `package-lock.json` * :zap: Match dependencies * :package: Update `package-lock.json` * :package: Re-update `package-lock.json` * :zap: Match whitespace * :bug: Fix selection * :zap: Expand validation * :fire: Remove validation * :pencil2: Update copy * :truck: Move to constants * :zap: More `null` validation * :zap: Support `all()` with index to access item * :zap: Gloss over n8n syntax error * :art: Re-style diagnostic button * :fire: Remove `item` as `itemAlias` * :zap: Add linting for `item.json` in single item mode * :zap: Refactor to add label info descriptions * :zap: More autocompletions * :shirt: Fix lint * :zap: Simplify typings * feat(nodes-base): Multiline autocompletion for `code-node-editor` (#4220) * :zap: Simplify typings * :zap: Consolidate helpers in utils * :zap: Multiline autocompletion for standalone vars * :fire: Remove unneeded mixins * :pencil2: Update copy * :pencil2: Prep TODOs * :zap: Multiline completion for `$input.method` + `$input.item` * :fire: Remove unused method * :fire: Remove another unused method * :truck: Move luxon strings to helpers * :zap: Multiline autocompletion for methods output * :zap: Refactor to use optional chaining * :shirt: Fix lint * :pencil2: Update TODOs * :zap: Multiline autocompletion for `json` fields * :blue_book: Add typings * :zap: De-duplicate callback to forEach * :bug: Fix autocompletions not working with leading whitespace * :globe_with_meridians: Apply i18n * :shirt: Fix lint * :constructor: Second-period var usage completions * :shirt: Fix lint * :shirt: Add exception * :zap: Add completion telemetry * :blue_book: Add typing * :zap: Major refactoring to organize * :bug: Fix multiline `.all()[index]` * :bug: Do not autoclose square brackets prior to `.json` * :bug: Fix accessor for multiline `jsonField` completions * :zap: Add completions for half-assignments * :bug: Fix `jsonField` completions for `x.json` * :pencil2: Improve comments * :bug: Fix `.json[field]` for multiline matches * :zap: Cleanup * :package: Update `package-lock.json` * :shirt: Fix lint * :bug: Rely on original value for custom matcher * :zap: Create `customMatcherJsonFieldCompletions` to simplify setup * :bug: Include selector in `customMatcherJsonField` completions * :pencil2: Make naming consistent * :pencil2: Add docline * :zap: Finish self-review cleanup * :fire: Remove outdated comment * :pushpin: Pin luxon to major-minor * :pencil2: Fix typo * :package: Update `package-lock.json` * :package: Update `package-lock.json` * :package: Re-update `package-lock.json` * :heavy_plus_sign: Add `luxon` for Gmail node * :package: Update `package-lock.json` * :zap: Replace Function with Code in suggested nodes * :bug: Fix `$prevNode` completions * :pencil2: Update `$execution.mode` copy * :zap: Separate luxon getters from methods * :zap: Adjusting linter to tolerate `.binary` * :zap: Adjust top-level item keys check * :zap: Anticipate user expecting `item` to pre-exist * :zap: Add linting for legacy item access * :zap: Add hint for attempted `items` access * :zap: Add keybinding for toggling comments * :pencil2: Update copy of `all`, `first`, `last` and `itemMatching` * :bug: Make `input.all()` etc act on copies * :package: Update `package-lock.json` * :bug: Fix guard in `$input.last()` * :recycle: Address Jan's feedback * :arrow_up: Upgrade `eslint-plugin-n8n-nodes-base` * :package: Update `package-lock.json` * :fire: Remove unneeded exceptions * :zap: Restore placeholder logic * :zap: Add placeholders to client * :zap: Account for shadow item * :pencil2: More completion info labels * :shirt: Fix lint * :pencil2: Update copy * :pencil2: Update copy * :pencil2: More copy updates * :package: Update `package-lock.json` * :zap: Add more validation * :zap: Add placheolder on first load * Replace `Cmd` with `Mod` * :package: Update `package-lock.json`
2022-10-13 05:28:02 -07:00
import { normalizeItems } from 'n8n-core';
import { NodeVM, NodeVMOptions } from 'vm2';
import { ValidationError } from './ValidationError';
import { ExecutionError } from './ExecutionError';
import { CodeNodeMode, isObject, SUPPORTED_ITEM_KEYS } from './utils';
import type { IExecuteFunctions, IWorkflowDataProxyData, WorkflowExecuteMode } from 'n8n-workflow';
export class Sandbox extends NodeVM {
private jsCode = '';
private itemIndex: number | undefined = undefined;
constructor(
context: ReturnType<typeof getSandboxContext>,
workflowMode: WorkflowExecuteMode,
private nodeMode: CodeNodeMode,
) {
super(Sandbox.getSandboxOptions(context, workflowMode));
}
static getSandboxOptions(
context: ReturnType<typeof getSandboxContext>,
workflowMode: WorkflowExecuteMode,
): NodeVMOptions {
const { NODE_FUNCTION_ALLOW_BUILTIN: builtIn, NODE_FUNCTION_ALLOW_EXTERNAL: external } =
process.env;
return {
console: workflowMode === 'manual' ? 'redirect' : 'inherit',
sandbox: context,
require: {
builtin: builtIn ? builtIn.split(',') : [],
external: external ? { modules: external.split(','), transitive: false } : false,
},
};
}
async runCode(jsCode: string, itemIndex?: number) {
this.jsCode = jsCode;
this.itemIndex = itemIndex;
return this.nodeMode === 'runOnceForAllItems' ? this.runCodeAllItems() : this.runCodeEachItem();
}
private async runCodeAllItems() {
const script = `module.exports = async function() {${this.jsCode}\n}()`;
const match = script.match(
/(?<disallowedSyntax>\)\.item(?!Matching)|\$input\.item(?!Matching)|\$json|\$binary|\$itemIndex)/,
); // disallow .item but tolerate .itemMatching
if (match?.groups?.disallowedSyntax) {
const { disallowedSyntax } = match.groups;
const lineNumber =
this.jsCode.split('\n').findIndex((line) => {
return line.includes(disallowedSyntax) && !line.startsWith('//') && !line.startsWith('*');
}) + 1;
const disallowedSyntaxFound = lineNumber !== 0;
if (disallowedSyntaxFound) {
throw new ValidationError({
message: `Can't use ${disallowedSyntax} here`,
description: "This is only available in 'Run Once for Each Item' mode",
itemIndex: this.itemIndex,
lineNumber,
});
}
}
let executionResult;
try {
executionResult = await this.run(script, __dirname);
} catch (error) {
// anticipate user expecting `items` to pre-exist as in Function Item node
if (error.message === 'items is not defined' && !/(let|const|var) items =/.test(script)) {
const quoted = error.message.replace('items', '`items`');
error.message = quoted + '. Did you mean `$input.all()`?';
}
throw new ExecutionError(error);
}
if (executionResult === null) return [];
if (executionResult === undefined || typeof executionResult !== 'object') {
throw new ValidationError({
message: "Code doesn't return items properly",
description:
'Please return an array of objects, one for each item you would like to output',
itemIndex: this.itemIndex,
});
}
if (Array.isArray(executionResult)) {
for (const item of executionResult) {
if (item.json !== undefined && !isObject(item.json)) {
throw new ValidationError({
message: "A 'json' property isn't an object",
description: "In the returned data, every key named 'json' must point to an object",
itemIndex: this.itemIndex,
});
}
// If at least one top-level key is a supported item key (`json`, `binary`, etc.),
// then validate all keys to be a supported item key, else allow user keys
// to be wrapped in `json` when normalizing items below.
if (
executionResult.some((item) =>
Object.keys(item).find((key) => SUPPORTED_ITEM_KEYS.has(key)),
)
) {
Object.keys(item).forEach((key) => {
if (SUPPORTED_ITEM_KEYS.has(key)) return;
throw new ValidationError({
message: `Unknown top-level item key: ${key}`,
description: 'Access the properties of an item under `.json`, e.g. `item.json`',
itemIndex: this.itemIndex,
});
});
}
if (item.binary !== undefined && !isObject(item.binary)) {
throw new ValidationError({
message: "A 'binary' property isn't an object",
description: "In the returned data, every key named 'binary must point to an object.",
itemIndex: this.itemIndex,
});
}
}
} else {
if (executionResult.json !== undefined && !isObject(executionResult.json)) {
throw new ValidationError({
message: "A 'json' property isn't an object",
description: "In the returned data, every key named 'json' must point to an object",
itemIndex: this.itemIndex,
});
}
if (executionResult.binary !== undefined && !isObject(executionResult.binary)) {
throw new ValidationError({
message: "A 'binary' property isn't an object",
description: "In the returned data, every key named 'binary must point to an object.",
itemIndex: this.itemIndex,
});
}
}
return normalizeItems(executionResult);
}
private async runCodeEachItem() {
const script = `module.exports = async function() {${this.jsCode}\n}()`;
const match = this.jsCode.match(/\$input\.(?<disallowedMethod>first|last|all|itemMatching)/);
if (match?.groups?.disallowedMethod) {
const { disallowedMethod } = match.groups;
const lineNumber =
this.jsCode.split('\n').findIndex((line) => {
return line.includes(disallowedMethod) && !line.startsWith('//') && !line.startsWith('*');
}) + 1;
const disallowedMethodFound = lineNumber !== 0;
if (disallowedMethodFound) {
throw new ValidationError({
message: `Can't use .${disallowedMethod}() here`,
description: "This is only available in 'Run Once for All Items' mode",
itemIndex: this.itemIndex,
lineNumber,
});
}
}
let executionResult;
try {
executionResult = await this.run(script, __dirname);
} catch (error) {
// anticipate user expecting `item` to pre-exist as in Function Item node
if (error.message === 'item is not defined' && !/(let|const|var) item =/.test(script)) {
const quoted = error.message.replace('item', '`item`');
error.message = quoted + '. Did you mean `$input.item.json`?';
}
throw new ExecutionError(error, this.itemIndex);
}
if (executionResult === null) return;
if (executionResult === undefined || typeof executionResult !== 'object') {
throw new ValidationError({
message: "Code doesn't return an object",
description: `Please return an object representing the output item. ('${executionResult}' was returned instead.)`,
itemIndex: this.itemIndex,
});
}
if (executionResult.json !== undefined && !isObject(executionResult.json)) {
throw new ValidationError({
message: "A 'json' property isn't an object",
description: "In the returned data, every key named 'json' must point to an object",
itemIndex: this.itemIndex,
});
}
if (executionResult.binary !== undefined && !isObject(executionResult.binary)) {
throw new ValidationError({
message: "A 'binary' property isn't an object",
description: "In the returned data, every key named 'binary must point to an object.",
itemIndex: this.itemIndex,
});
}
// If at least one top-level key is a supported item key (`json`, `binary`, etc.),
// and another top-level key is unrecognized, then the user mis-added a property
// directly on the item, when they intended to add it on the `json` property
Object.keys(executionResult).forEach((key) => {
if (SUPPORTED_ITEM_KEYS.has(key)) return;
throw new ValidationError({
message: `Unknown top-level item key: ${key}`,
description: 'Access the properties of an item under `.json`, e.g. `item.json`',
itemIndex: this.itemIndex,
});
});
if (Array.isArray(executionResult)) {
const firstSentence =
executionResult.length > 0
? `An array of ${typeof executionResult[0]}s was returned.`
: 'An empty array was returned.';
throw new ValidationError({
message: "Code doesn't return a single object",
description: `${firstSentence} If you need to output multiple items, please use the 'Run Once for All Items' mode instead`,
itemIndex: this.itemIndex,
});
}
return executionResult.json ? executionResult : { json: executionResult };
}
}
export function getSandboxContext(this: IExecuteFunctions, index?: number) {
const sandboxContext: Record<string, unknown> & {
$item: (i: number) => IWorkflowDataProxyData;
$input: any; // tslint:disable-line: no-any
} = {
// from NodeExecuteFunctions
$getNodeParameter: this.getNodeParameter,
$getWorkflowStaticData: this.getWorkflowStaticData,
helpers: this.helpers,
// to bring in all $-prefixed vars and methods from WorkflowDataProxy
$item: this.getWorkflowDataProxy,
$input: null,
};
feat(Code Node): create Code node (#3965) * Introduce node deprecation (#3930) :sparkles: Introduce node deprecation * :construction: Scaffold out Code node * :shirt: Fix lint * :blue_book: Create types file * :truck: Rename theme * :fire: Remove unneeded prop * :zap: Override keybindings * :zap: Expand lintings * :zap: Create editor content getter * :truck: Ensure all helpers use `$` * :sparkles: Add autocompletion * :zap: Filter out welcome note node * :zap: Convey error line number * :zap: Highlight error line * :zap: Restore logging from node * :sparkles: More autocompletions * :zap: Streamline completions * :pencil2: Update placeholders * :zap: Update linter to new methods * :fire: Remove `$nodeItem` completions * :zap: Re-update placeholders * :art: Fix formatting * :package: Update `package-lock.json` * :zap: Refresh with multi-line empty string * :zap: Account for syntax errors * :fire: Remove unneeded variant * :zap: Minor improvements * :zap: Add more autocompletions * :truck: Rename extension * :fire: Remove outdated comments * :truck: Rename field * :sparkles: More autocompletions * :zap: Fix up error display when empty text * :fire: Remove logging * :sparkles: More error validation * :bug: Fix `pairedItem` to `pairedItem()` * :zap: Add item to validation info * :package: Update `package-lock.json` * :zap: Leftover fixes * :zap: Set `insertNewlineAndIndent` * :package: Update `package-lock.json` * :package: Re-update `package-lock.json` * :shirt: Add lint exception * :blue_book: Add type to mixin type * Clean up comment * :zap: Refactor completion per new requirements * :zap: Adjust placeholders * :zap: Add `json` autocompletions for `$input` * :art: Set border * :zap: Restore local completion source * :zap: Implement autocompletion for imports * :zap: Add `.*` to follow user typing on autocompletion * :blue_book: Fix typings in autocompletions * :shirt: Add linting for use of `item()` * :package: Update `package-lock.json` * :bug: Fix for `$items(nodeName)[0]` * :zap: Filter down built-in modules list * :zap: Refactor error handling * :zap: Linter and validation improvements * :zap: Apply review feedback * :recycle: More general refactorings * :zap: Add dot notation utility * Customize input handler * :zap: Support `.json.` completions * :zap: Adjust placeholder * :zap: Sort imports * :fire: Remove blank rows addition * :zap: Add more error validation * :package: Update `package-lock.json` * :zap: Make date logging consistent * :wrench: Adjust linting highlight range * :zap: Add line numbers to each item mode errors * :zap: Allow for links in error descriptions * :zap: More input validation * :zap: Expand linting to loops * :zap: Deprecate Function and Function Item nodes * :bug: Fix placeholder syntax * :blue_book: Narrow down type * :truck: Rename using kebab-case * :fire: Remove `mapGetters` * :pencil2: Fix casing * :zap: Adjust import for type * :pencil2: Fix quotes * :bug: Fix `activeNode` reference * :zap: Use constant * :fire: Remove logging * :pencil2: Fix typo * :zap: Add missing `notice` * :pencil2: Add tags * :pencil2: Fix alias * :pencil2: Update copy * :fire: Remove wrong linting * :pencil2: Update copy * :zap: Add validation for `null` * :zap: Add validation for non-object and non-array * :zap: Add validation for non-array with json * :pencil2: Intentionally use wrong spelling * :zap: More validation * :pencil2: More copy updates * :pencil2: Placeholder updates * :rewind: Restore spelling * :zap: Fix var name * :pencil2: More copy updates * :zap: Add luxon autocompletions * :zap: Make scrollable * :zap: Fix comma from merge conflict resolution * :package: Update `package-lock.json` * :shirt: Fix lint detail * :art: Set font family * :zap: Bring in expressions fix * :recycle: Address feedback * :zap: Exclude codemirror packages from render chunks * :bug: Fix placeholder not showing on first load * feat(editor-ui): Replace `lezer` with `esprima` in client linter (#4192) * :fire: Remove addition from misresolved conflict * :zap: Replace `lezer` with `esprima` in client linter * :zap: Add missing key * :package: Update `package-lock.json` * :zap: Match dependencies * :package: Update `package-lock.json` * :package: Re-update `package-lock.json` * :zap: Match whitespace * :bug: Fix selection * :zap: Expand validation * :fire: Remove validation * :pencil2: Update copy * :truck: Move to constants * :zap: More `null` validation * :zap: Support `all()` with index to access item * :zap: Gloss over n8n syntax error * :art: Re-style diagnostic button * :fire: Remove `item` as `itemAlias` * :zap: Add linting for `item.json` in single item mode * :zap: Refactor to add label info descriptions * :zap: More autocompletions * :shirt: Fix lint * :zap: Simplify typings * feat(nodes-base): Multiline autocompletion for `code-node-editor` (#4220) * :zap: Simplify typings * :zap: Consolidate helpers in utils * :zap: Multiline autocompletion for standalone vars * :fire: Remove unneeded mixins * :pencil2: Update copy * :pencil2: Prep TODOs * :zap: Multiline completion for `$input.method` + `$input.item` * :fire: Remove unused method * :fire: Remove another unused method * :truck: Move luxon strings to helpers * :zap: Multiline autocompletion for methods output * :zap: Refactor to use optional chaining * :shirt: Fix lint * :pencil2: Update TODOs * :zap: Multiline autocompletion for `json` fields * :blue_book: Add typings * :zap: De-duplicate callback to forEach * :bug: Fix autocompletions not working with leading whitespace * :globe_with_meridians: Apply i18n * :shirt: Fix lint * :constructor: Second-period var usage completions * :shirt: Fix lint * :shirt: Add exception * :zap: Add completion telemetry * :blue_book: Add typing * :zap: Major refactoring to organize * :bug: Fix multiline `.all()[index]` * :bug: Do not autoclose square brackets prior to `.json` * :bug: Fix accessor for multiline `jsonField` completions * :zap: Add completions for half-assignments * :bug: Fix `jsonField` completions for `x.json` * :pencil2: Improve comments * :bug: Fix `.json[field]` for multiline matches * :zap: Cleanup * :package: Update `package-lock.json` * :shirt: Fix lint * :bug: Rely on original value for custom matcher * :zap: Create `customMatcherJsonFieldCompletions` to simplify setup * :bug: Include selector in `customMatcherJsonField` completions * :pencil2: Make naming consistent * :pencil2: Add docline * :zap: Finish self-review cleanup * :fire: Remove outdated comment * :pushpin: Pin luxon to major-minor * :pencil2: Fix typo * :package: Update `package-lock.json` * :package: Update `package-lock.json` * :package: Re-update `package-lock.json` * :heavy_plus_sign: Add `luxon` for Gmail node * :package: Update `package-lock.json` * :zap: Replace Function with Code in suggested nodes * :bug: Fix `$prevNode` completions * :pencil2: Update `$execution.mode` copy * :zap: Separate luxon getters from methods * :zap: Adjusting linter to tolerate `.binary` * :zap: Adjust top-level item keys check * :zap: Anticipate user expecting `item` to pre-exist * :zap: Add linting for legacy item access * :zap: Add hint for attempted `items` access * :zap: Add keybinding for toggling comments * :pencil2: Update copy of `all`, `first`, `last` and `itemMatching` * :bug: Make `input.all()` etc act on copies * :package: Update `package-lock.json` * :bug: Fix guard in `$input.last()` * :recycle: Address Jan's feedback * :arrow_up: Upgrade `eslint-plugin-n8n-nodes-base` * :package: Update `package-lock.json` * :fire: Remove unneeded exceptions * :zap: Restore placeholder logic * :zap: Add placeholders to client * :zap: Account for shadow item * :pencil2: More completion info labels * :shirt: Fix lint * :pencil2: Update copy * :pencil2: Update copy * :pencil2: More copy updates * :package: Update `package-lock.json` * :zap: Add more validation * :zap: Add placheolder on first load * Replace `Cmd` with `Mod` * :package: Update `package-lock.json`
2022-10-13 05:28:02 -07:00
// $node, $items(), $parameter, $json, $env, etc.
Object.assign(sandboxContext, sandboxContext.$item(index ?? 0));
return sandboxContext;
}