feat(editor): Implement Resource Mapper component (#6207)

*  scaffolding
*  finished scaffolding
*  renamed types
*  updated subtitle
*  renamed functions file, UI updates
*  query parameters fixes, ui updates, refactoring
*  fixes for credentials test, setup for error parsing
*  rlc for schema and table, error handling tweaks
*  delete operation, new options
*  columns loader
*  linter fixes
*  where clauses setup
*  logic for processing where clauses
*  select operation
*  refactoring
*  data mode for insert and update, wip
*  data mapping, insert update, skip on conflict option
*  select columns with spaces fix
*  update operation update, wip
*  finished update operation
*  upsert operation
*  ui fixes
* Copy updates.
* Copy updates.
*  option to convert empty strings to nulls, schema checks
*  UI requested updates
*  ssh setup WIP
*  fixes, ssh WIP
*  ssh fixes, credentials
*  credentials testing update
*  uncaught error fix
*  clean up
*  address in use fix
*  improved error message
*  tests setup
*  unit tests wip
*  config files clean up
*  utils unit tests
*  refactoring
*  setup for testing operations, tests for deleteTable operation
*  executeQuery and insert operations tests
*  select, update, upsert operations tests
*  runQueries tests setup
*  hint to query
* Copy updates.
*  ui fixes
*  clean up
*  error message update
*  ui update
* Minor tweaks to query params decription.
* feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752)
*  Added initial resource mapping support in google sheets node
*  Wired mapping API endpoint with node-specific logic for fetching mapping fields
*  Implementing mapping fields logic for google sheets
*  Updating Google Sheets execute methods to support resource mapper fields
* 🚧 Added initial version of `ResourceLocator` component
* 👌 Added `update` mode to resource mapper modes
* 👌 Addressing PR feedback
* 👌 Removing leftover const reference
* 👕 Fixing lint errors
*  singlton for conections
*  credentials test fix, clean up
* feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814)
*  scaffolding
*  finished scaffolding
*  renamed types
*  updated subtitle
*  renamed functions file, UI updates
*  query parameters fixes, ui updates, refactoring
*  fixes for credentials test, setup for error parsing
*  rlc for schema and table, error handling tweaks
*  delete operation, new options
*  columns loader
*  linter fixes
*  where clauses setup
*  logic for processing where clauses
*  select operation
*  refactoring
*  data mode for insert and update, wip
*  data mapping, insert update, skip on conflict option
*  select columns with spaces fix
*  update operation update, wip
*  finished update operation
*  upsert operation
*  ui fixes
* Copy updates.
* Copy updates.
*  option to convert empty strings to nulls, schema checks
*  UI requested updates
*  ssh setup WIP
*  fixes, ssh WIP
*  ssh fixes, credentials
*  credentials testing update
*  uncaught error fix
*  clean up
*  address in use fix
*  improved error message
*  tests setup
*  unit tests wip
*  config files clean up
*  utils unit tests
*  refactoring
*  setup for testing operations, tests for deleteTable operation
*  executeQuery and insert operations tests
*  select, update, upsert operations tests
*  runQueries tests setup
*  hint to query
* Copy updates.
*  ui fixes
*  clean up
*  error message update
*  ui update
* Minor tweaks to query params decription.
*  Updated Postgres node to use resource mapper component
*  Implemented postgres <-> resource mapper type mapping
*  Updated Postgres node execution to use resource mapper fields in v3
* 🔥 Removing unused import
---------
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>

* feat(core): Resource editor componend P0 (#5970)
*  Added inital value of mapping mode dropdown
*  Finished mapping mode selector
*  Finished implementing mapping mode selector
*  Implemented 'Columns to match on' dropdown
*  Implemented `loadOptionsDependOn` support in resource mapper
*  Implemented initial version of mapping fields
*  Implementing dependant fields watcher in new component setup
*  Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one
*  Handling matching columns correctly in UI
*  Saving and loading resourceMapper values in component
*  Implemented proper data saving and loading
*  ResourceMapper component refactor, fixing value save/load
*  Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key
*  Updated Google Sheets node to work with the new UI
*  Updating Postgres Node to work with new UI
*  Additional loading indicator that shown if there is no mapping mode selector
*  Removing hard-coded values, fixing matching columns ordering, refactoring
*  Updating field names in nodes
*  Fixing minor UI issues
*  Implemented matching fields filter logic
*  Moving loading label outside of fields list
*  Added initial unit tests for resource mapper
*  Finished default rendering test
*  Test refactoring
*  Finished unit tests
* 🔨 Updating the way i18n is used in resource mapper components
* ✔️ Fixing value to match on logic for postgres node
*  Hiding mapping fields when auto-map mode is selected
*  Syncing selected mapping mode between components
*  Fixing dateTime input rendering and adding update check to Postgres node
*  Properly handling database connections. Sending null for empty string values.
* 💄 Updated wording in the error message for non-existing rows
*  Fixing issues with selected matching values
* ✔️ Updating unit tests after matching logic update
*  Updating matching columns when new fields are loaded
*  Defaulting to null for empty parameter values
*  Allowing zero as valid value for number imputs
*  Updated list of types that use datepicker as widger
*  Using text inputs for time types
*  Initial mapping field rework
*  Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors
*  Added tooltip for columns that cannot be deleted
*  Saving deleted values in parameter value
*  Implemented control to add/remove mapping fields
*  Syncing field list with add field dropdown when changing dependent values
*  Not showing removed fields in matching columns selector. Updating wording in matching columns selector description
*  Implementing disabled states for add/remove all fields options
*  Saving removed columns separately, updating copy
*  Implemented resource mapper values validation
*  Updated validation logic and error input styling
*  Validating resource mapper fields when new nodes are added
*  Using node field words in validation, refactoring resource mapper component
*  Implemented schema syncing and add/remove all fields
*  Implemented custom parameter actions
*  Implemented loading indicator in parameter options
* 🔨 Removing unnecessary constants and vue props
*  Handling default values properly
*  Fixing validation logic
* 👕 Fixing lint errors
*  Fixing type issues
*  Not showing fields by default if `addAllFields` is set to `false`
*  Implemented field type validation in resource mapper
*  Updated casing in copy, removed all/remove all option from bottom menu
*  Added auto mapping mode notice
*  Added support for more types in validation
*  Added support for enumerated values
*  Fixing imports after merging
*  Not showing removed fields in matching columns selector. Refactoring validation logic.
* 👕 Fixing imports
* ✔️ Updating unit tests
*  Added resource mapper schema tests
*  Removing `match` from resource mapper field definition, fixing matching columns loading
*  Fixed schema merging
*  update operation return data fix
*  review
* 🐛 Added missing import
* 💄 Updating parameter actions icon based on the ui review
* 💄 Updating word capitalisation in tooltips
* 💄 Added empty state to mapping fields list
* 💄 Removing asterisk from fields, updating tooltips for matching fields
*  Preventing matching fields from being removed by 'Remove All option'
*  Not showing hidden fields in the `Add field` dropdown
*  Added support for custom matching columns labels
*  query optimization
*  fix
*  Optimizing Postgres node enumeration logic
*  Added empty state for matching columns
*  Only fully loading fields if there is no schema fetched
*  Hiding mapping fields if there is no matching columns available in the schema
* ✔️ Fixing minor issues
*  Implemented runtime type validation
* 🔨 Refactoring validation logic
*  Implemented required check, added more custom messages
*  Skipping boolean type in required check
* Type check improvements
*  Only reloading fields if dependent values actually change
*  Adding item index to validation error title
*  Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted
*  Resetting field values when adding them via the addAll option
*  Using minor version (2.2) for new Postgres node
*  Implemented proper date validation and type casting
* 👕 Consolidating typing
*  Added unit tests for type validations
* 👌 Addressing front-end review comments
*  More refactoring to address review changes
*  Updating leftover props
*  Added fallback for ISO dates with invalid timezones
* Added timestamp to datetime test cases
*  Reseting matching columns if operation changes
*  Not forcing auto-increment fields to be filled in in Postgres node. Handling null values
* 💄 Added a custom message for invalid dates
*  Better handling of JSON values
*  Updating codemirror readonly stauts based on component property, handling objects in json validation
* Deleting leftover console.log
*  Better time validation
*  Fixing build error after merging
* 👕 Fixing lint error
*  Updating node configuration values
*  Handling postgres arrays better
*  Handling SQL array syntax
*  Updating time validation rules to include timezone
*  Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets
*  Allowing removed fields to be selected for match
*  Updated the query for fetching unique columns and primary keys
*  Optimizing the unique query
*  Setting timezone to all parsed dates
*  Addressing PR review feedback
*  Configuring Sheets node for production, minor vue component update
* New cases added to the TypeValidation test.
*  Tweaking validation rules for arrays/objects and updating test cases
---------
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
This commit is contained in:
Milorad FIlipović 2023-05-31 11:56:09 +02:00 committed by GitHub
parent 5ae1124106
commit 04cfa548af
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
57 changed files with 3436 additions and 183 deletions

View file

@ -31,6 +31,7 @@ import clientOAuth1 from 'oauth-1.0a';
import { import {
BinaryDataManager, BinaryDataManager,
Credentials, Credentials,
LoadMappingOptions,
LoadNodeParameterOptions, LoadNodeParameterOptions,
LoadNodeListSearch, LoadNodeListSearch,
UserSettings, UserSettings,
@ -49,6 +50,7 @@ import type {
ICredentialTypes, ICredentialTypes,
ExecutionStatus, ExecutionStatus,
IExecutionsSummary, IExecutionsSummary,
ResourceMapperFields,
IN8nUISettings, IN8nUISettings,
} from 'n8n-workflow'; } from 'n8n-workflow';
import { LoggerProxy, jsonParse } from 'n8n-workflow'; import { LoggerProxy, jsonParse } from 'n8n-workflow';
@ -79,6 +81,7 @@ import type {
NodeListSearchRequest, NodeListSearchRequest,
NodeParameterOptionsRequest, NodeParameterOptionsRequest,
OAuthRequest, OAuthRequest,
ResourceMapperRequest,
WorkflowRequest, WorkflowRequest,
} from '@/requests'; } from '@/requests';
import { registerController } from '@/decorators'; import { registerController } from '@/decorators';
@ -756,6 +759,58 @@ export class Server extends AbstractServer {
), ),
); );
this.app.get(
`/${this.restEndpoint}/get-mapping-fields`,
ResponseHelper.send(
async (
req: ResourceMapperRequest,
res: express.Response,
): Promise<ResourceMapperFields | undefined> => {
const nodeTypeAndVersion = jsonParse(
req.query.nodeTypeAndVersion,
) as INodeTypeNameVersion;
const { path, methodName } = req.query;
if (!req.query.currentNodeParameters) {
throw new ResponseHelper.BadRequestError(
'Parameter currentNodeParameters is required.',
);
}
const currentNodeParameters = jsonParse(
req.query.currentNodeParameters,
) as INodeParameters;
let credentials: INodeCredentials | undefined;
if (req.query.credentials) {
credentials = jsonParse(req.query.credentials);
}
const loadMappingOptionsInstance = new LoadMappingOptions(
nodeTypeAndVersion,
this.nodeTypes,
path,
currentNodeParameters,
credentials,
);
const additionalData = await WorkflowExecuteAdditionalData.getBase(
req.user.id,
currentNodeParameters,
);
const fields = await loadMappingOptionsInstance.getOptionsViaMethodName(
methodName,
additionalData,
);
return fields;
},
),
);
// ---------------------------------------- // ----------------------------------------
// Active Workflows // Active Workflows
// ---------------------------------------- // ----------------------------------------

View file

@ -348,6 +348,23 @@ export type NodeListSearchRequest = AuthenticatedRequest<
} }
>; >;
// ----------------------------------
// /get-mapping-fields
// ----------------------------------
export type ResourceMapperRequest = AuthenticatedRequest<
{},
{},
{},
{
nodeTypeAndVersion: string;
methodName: string;
path: string;
currentNodeParameters: string;
credentials: string;
}
>;
// ---------------------------------- // ----------------------------------
// /tags // /tags
// ---------------------------------- // ----------------------------------

View file

@ -11,6 +11,7 @@ import type {
ITriggerFunctions as ITriggerFunctionsBase, ITriggerFunctions as ITriggerFunctionsBase,
IWebhookFunctions as IWebhookFunctionsBase, IWebhookFunctions as IWebhookFunctionsBase,
BinaryMetadata, BinaryMetadata,
ValidationResult,
} from 'n8n-workflow'; } from 'n8n-workflow';
// TODO: remove these after removing `n8n-core` dependency from `nodes-bases` // TODO: remove these after removing `n8n-core` dependency from `nodes-bases`
@ -89,3 +90,5 @@ export namespace n8n {
}; };
} }
} }
export type ExtendedValidationResult = Partial<ValidationResult> & { fieldName?: string };

View file

@ -0,0 +1,34 @@
import type { IWorkflowExecuteAdditionalData, ResourceMapperFields } from 'n8n-workflow';
import * as NodeExecuteFunctions from './NodeExecuteFunctions';
import { LoadNodeDetails } from './LoadNodeDetails';
export class LoadMappingOptions extends LoadNodeDetails {
/**
* Returns the available mapping fields for the ResourceMapper component
*/
async getOptionsViaMethodName(
methodName: string,
additionalData: IWorkflowExecuteAdditionalData,
): Promise<ResourceMapperFields> {
const node = this.getTempNode();
const nodeType = this.workflow.nodeTypes.getByNameAndVersion(node.type, node.typeVersion);
const method = nodeType?.methods?.resourceMapping?.[methodName];
if (typeof method !== 'function') {
throw new Error(
`The node-type "${node.type}" does not have the method "${methodName}" defined!`,
);
}
const thisArgs = NodeExecuteFunctions.getLoadOptionsFunctions(
this.workflow,
node,
this.path,
additionalData,
);
return method.call(thisArgs);
}
}

View file

@ -61,10 +61,12 @@ import type {
IWebhookFunctions, IWebhookFunctions,
BinaryMetadata, BinaryMetadata,
FileSystemHelperFunctions, FileSystemHelperFunctions,
INodeType,
} from 'n8n-workflow'; } from 'n8n-workflow';
import { import {
createDeferredPromise, createDeferredPromise,
isObjectEmpty, isObjectEmpty,
isResourceMapperValue,
NodeApiError, NodeApiError,
NodeHelpers, NodeHelpers,
NodeOperationError, NodeOperationError,
@ -74,6 +76,7 @@ import {
deepCopy, deepCopy,
fileTypeFromMimeType, fileTypeFromMimeType,
ExpressionError, ExpressionError,
validateFieldType,
} from 'n8n-workflow'; } from 'n8n-workflow';
import pick from 'lodash.pick'; import pick from 'lodash.pick';
@ -114,7 +117,7 @@ import { access as fsAccess } from 'fs/promises';
import { createReadStream } from 'fs'; import { createReadStream } from 'fs';
import { BinaryDataManager } from './BinaryDataManager'; import { BinaryDataManager } from './BinaryDataManager';
import type { IResponseError, IWorkflowSettings } from './Interfaces'; import type { ExtendedValidationResult, IResponseError, IWorkflowSettings } from './Interfaces';
import { extractValue } from './ExtractValue'; import { extractValue } from './ExtractValue';
import { getClientCredentialsToken } from './OAuth2Helper'; import { getClientCredentialsToken } from './OAuth2Helper';
import { PLACEHOLDER_EMPTY_EXECUTION_ID } from './Constants'; import { PLACEHOLDER_EMPTY_EXECUTION_ID } from './Constants';
@ -1867,7 +1870,7 @@ function cleanupParameterData(inputData: NodeParameterValueType): void {
} }
if (Array.isArray(inputData)) { if (Array.isArray(inputData)) {
inputData.forEach((value) => cleanupParameterData(value)); inputData.forEach((value) => cleanupParameterData(value as NodeParameterValueType));
return; return;
} }
@ -1886,6 +1889,103 @@ function cleanupParameterData(inputData: NodeParameterValueType): void {
} }
} }
const validateResourceMapperValue = (
parameterName: string,
paramValues: { [key: string]: unknown },
node: INode,
skipRequiredCheck = false,
): ExtendedValidationResult => {
const result: ExtendedValidationResult = { valid: true, newValue: paramValues };
const paramNameParts = parameterName.split('.');
if (paramNameParts.length !== 2) {
return result;
}
const resourceMapperParamName = paramNameParts[0];
const resourceMapperField = node.parameters[resourceMapperParamName];
if (!resourceMapperField || !isResourceMapperValue(resourceMapperField)) {
return result;
}
const schema = resourceMapperField.schema;
const paramValueNames = Object.keys(paramValues);
for (let i = 0; i < paramValueNames.length; i++) {
const key = paramValueNames[i];
const resolvedValue = paramValues[key];
const schemaEntry = schema.find((s) => s.id === key);
if (
!skipRequiredCheck &&
schemaEntry?.required === true &&
schemaEntry.type !== 'boolean' &&
!resolvedValue
) {
return {
valid: false,
errorMessage: `The value "${String(key)}" is required but not set`,
fieldName: key,
};
}
if (schemaEntry?.type) {
const validationResult = validateFieldType(
key,
resolvedValue,
schemaEntry.type,
schemaEntry.options,
);
if (!validationResult.valid) {
return { ...validationResult, fieldName: key };
} else {
// If it's valid, set the casted value
paramValues[key] = validationResult.newValue;
}
}
}
return result;
};
const validateValueAgainstSchema = (
node: INode,
nodeType: INodeType,
inputValues: string | number | boolean | object | null | undefined,
parameterName: string,
runIndex: number,
itemIndex: number,
) => {
let validationResult: ExtendedValidationResult = { valid: true, newValue: inputValues };
// Currently only validate resource mapper values
const resourceMapperField = nodeType.description.properties.find(
(prop) =>
NodeHelpers.displayParameter(node.parameters, prop, node) &&
prop.type === 'resourceMapper' &&
parameterName === `${prop.name}.value`,
);
if (resourceMapperField && typeof inputValues === 'object') {
validationResult = validateResourceMapperValue(
parameterName,
inputValues as { [key: string]: unknown },
node,
resourceMapperField.typeOptions?.resourceMapper?.mode !== 'add',
);
}
if (!validationResult.valid) {
throw new ExpressionError(
`Invalid input for '${
String(validationResult.fieldName) || parameterName
}' [item ${itemIndex}]`,
{
description: validationResult.errorMessage,
failExecution: true,
runIndex,
itemIndex,
nodeCause: node.name,
},
);
}
return validationResult.newValue;
};
/** /**
* Returns the requested resolved (all expressions replaced) node parameters. * Returns the requested resolved (all expressions replaced) node parameters.
* *
@ -1947,6 +2047,16 @@ export function getNodeParameter(
returnData = extractValue(returnData, parameterName, node, nodeType); returnData = extractValue(returnData, parameterName, node, nodeType);
} }
// Validate parameter value if it has a schema defined
returnData = validateValueAgainstSchema(
node,
nodeType,
returnData,
parameterName,
runIndex,
itemIndex,
);
return returnData; return returnData;
} }

View file

@ -9,6 +9,7 @@ export * from './Constants';
export * from './Credentials'; export * from './Credentials';
export * from './DirectoryLoader'; export * from './DirectoryLoader';
export * from './Interfaces'; export * from './Interfaces';
export * from './LoadMappingOptions';
export * from './LoadNodeParameterOptions'; export * from './LoadNodeParameterOptions';
export * from './LoadNodeListSearch'; export * from './LoadNodeListSearch';
export * from './NodeExecuteFunctions'; export * from './NodeExecuteFunctions';

View file

@ -8,7 +8,7 @@ export default {
argTypes: { argTypes: {
placement: { placement: {
type: 'select', type: 'select',
options: ['top', 'top-start', 'top-end', 'bottom', 'bottom-end'], options: ['top', 'top-end', 'top-start', 'bottom', 'bottom-end', 'bottom-start'],
}, },
size: { size: {
type: 'select', type: 'select',

View file

@ -9,7 +9,10 @@
@visible-change="onVisibleChange" @visible-change="onVisibleChange"
> >
<span :class="{ [$style.button]: true, [$style[theme]]: !!theme }"> <span :class="{ [$style.button]: true, [$style[theme]]: !!theme }">
<n8n-icon icon="ellipsis-v" :size="iconSize" /> <n8n-icon
:icon="iconOrientation === 'horizontal' ? 'ellipsis-h' : 'ellipsis-v'"
:size="iconSize"
/>
</span> </span>
<template #dropdown> <template #dropdown>
@ -79,6 +82,11 @@ export default defineComponent({
default: 'default', default: 'default',
validator: (value: string): boolean => ['default', 'dark'].includes(value), validator: (value: string): boolean => ['default', 'dark'].includes(value),
}, },
iconOrientation: {
type: String,
default: 'vertical',
validator: (value: string): boolean => ['horizontal', 'vertical'].includes(value),
},
}, },
methods: { methods: {
onCommand(value: string) { onCommand(value: string) {

View file

@ -33,7 +33,11 @@
v-if="$slots.options && label" v-if="$slots.options && label"
:class="{ [$style.overlay]: true, [$style.visible]: showOptions }" :class="{ [$style.overlay]: true, [$style.visible]: showOptions }"
/> />
<div v-if="$slots.options" :class="{ [$style.options]: true, [$style.visible]: showOptions }"> <div
v-if="$slots.options"
:class="{ [$style.options]: true, [$style.visible]: showOptions }"
data-test-id="parameter-input-options-container"
>
<slot name="options" /> <slot name="options" />
</div> </div>
</label> </label>

View file

@ -1361,6 +1361,13 @@ export type NodeAuthenticationOption = {
displayOptions?: IDisplayOptions; displayOptions?: IDisplayOptions;
}; };
export interface ResourceMapperReqParams {
nodeTypeAndVersion: INodeTypeNameVersion;
path: string;
methodName?: string;
currentNodeParameters: INodeParameters;
credentials?: INodeCredentials;
}
export interface EnvironmentVariable { export interface EnvironmentVariable {
id: number; id: number;
key: string; key: string;

View file

@ -3,6 +3,7 @@ import type {
INodeTranslationHeaders, INodeTranslationHeaders,
IResourceLocatorReqParams, IResourceLocatorReqParams,
IRestApiContext, IRestApiContext,
ResourceMapperReqParams,
} from '@/Interface'; } from '@/Interface';
import type { import type {
IDataObject, IDataObject,
@ -15,6 +16,7 @@ import type {
INodeTypeNameVersion, INodeTypeNameVersion,
} from 'n8n-workflow'; } from 'n8n-workflow';
import axios from 'axios'; import axios from 'axios';
import type { ResourceMapperFields } from 'n8n-workflow/src/Interfaces';
export async function getNodeTypes(baseUrl: string) { export async function getNodeTypes(baseUrl: string) {
const { data } = await axios.get(baseUrl + 'types/nodes.json', { withCredentials: true }); const { data } = await axios.get(baseUrl + 'types/nodes.json', { withCredentials: true });
@ -59,3 +61,15 @@ export async function getResourceLocatorResults(
sendData as unknown as IDataObject, sendData as unknown as IDataObject,
); );
} }
export async function getResourceMapperFields(
context: IRestApiContext,
sendData: ResourceMapperReqParams,
): Promise<ResourceMapperFields> {
return makeRestApiRequest(
context,
'GET',
'/get-mapping-fields',
sendData as unknown as IDataObject,
);
}

View file

@ -6,7 +6,7 @@
import { defineComponent } from 'vue'; import { defineComponent } from 'vue';
import { mapStores } from 'pinia'; import { mapStores } from 'pinia';
import { EditorView, keymap } from '@codemirror/view'; import { EditorView, keymap } from '@codemirror/view';
import { EditorState, Prec } from '@codemirror/state'; import { Compartment, EditorState, Prec } from '@codemirror/state';
import { history, redo } from '@codemirror/commands'; import { history, redo } from '@codemirror/commands';
import { acceptCompletion, autocompletion, completionStatus } from '@codemirror/autocomplete'; import { acceptCompletion, autocompletion, completionStatus } from '@codemirror/autocomplete';
@ -19,6 +19,8 @@ import { inputTheme } from './theme';
import { n8nLang } from '@/plugins/codemirror/n8nLang'; import { n8nLang } from '@/plugins/codemirror/n8nLang';
import { completionManager } from '@/mixins/completionManager'; import { completionManager } from '@/mixins/completionManager';
const editableConf = new Compartment();
export default defineComponent({ export default defineComponent({
name: 'InlineExpressionEditorInput', name: 'InlineExpressionEditorInput',
mixins: [completionManager, expressionManager, workflowHelpers], mixins: [completionManager, expressionManager, workflowHelpers],
@ -39,6 +41,11 @@ export default defineComponent({
}, },
}, },
watch: { watch: {
isReadOnly(newValue: boolean) {
this.editor?.dispatch({
effects: editableConf.reconfigure(EditorView.editable.of(!newValue)),
});
},
value(newValue) { value(newValue) {
const isInternalChange = newValue === this.editor?.state.doc.toString(); const isInternalChange = newValue === this.editor?.state.doc.toString();
@ -97,7 +104,7 @@ export default defineComponent({
history(), history(),
expressionInputHandler(), expressionInputHandler(),
EditorView.lineWrapping, EditorView.lineWrapping,
EditorView.editable.of(!this.isReadOnly), editableConf.of(EditorView.editable.of(!this.isReadOnly)),
EditorView.contentAttributes.of({ 'data-gramm': 'false' }), // disable grammarly EditorView.contentAttributes.of({ 'data-gramm': 'false' }), // disable grammarly
EditorView.domEventHandlers({ EditorView.domEventHandlers({
focus: () => { focus: () => {

View file

@ -362,6 +362,7 @@ import type {
INodeProperties, INodeProperties,
INodePropertyCollection, INodePropertyCollection,
NodeParameterValueType, NodeParameterValueType,
IParameterLabel,
EditorType, EditorType,
CodeNodeEditorLanguage, CodeNodeEditorLanguage,
} from 'n8n-workflow'; } from 'n8n-workflow';
@ -380,7 +381,6 @@ import { externalHooks } from '@/mixins/externalHooks';
import { nodeHelpers } from '@/mixins/nodeHelpers'; import { nodeHelpers } from '@/mixins/nodeHelpers';
import { workflowHelpers } from '@/mixins/workflowHelpers'; import { workflowHelpers } from '@/mixins/workflowHelpers';
import { hasExpressionMapping, isValueExpression, isResourceLocatorValue } from '@/utils'; import { hasExpressionMapping, isValueExpression, isResourceLocatorValue } from '@/utils';
import { CODE_NODE_TYPE, CUSTOM_API_CALL_KEY, HTML_NODE_TYPE } from '@/constants'; import { CODE_NODE_TYPE, CUSTOM_API_CALL_KEY, HTML_NODE_TYPE } from '@/constants';
import type { PropType } from 'vue'; import type { PropType } from 'vue';
import { debounceHelper } from '@/mixins/debounce'; import { debounceHelper } from '@/mixins/debounce';
@ -457,6 +457,12 @@ export default defineComponent({
expressionEvaluated: { expressionEvaluated: {
type: String as PropType<string | undefined>, type: String as PropType<string | undefined>,
}, },
label: {
type: Object as PropType<IParameterLabel>,
default: () => ({
size: 'small',
}),
},
}, },
data() { data() {
return { return {

View file

@ -10,6 +10,7 @@
> >
<template #options> <template #options>
<parameter-options <parameter-options
v-if="displayOptions"
:parameter="parameter" :parameter="parameter"
:value="value" :value="value"
:isReadOnly="isReadOnly" :isReadOnly="isReadOnly"
@ -54,6 +55,7 @@
:forceShowExpression="forceShowExpression" :forceShowExpression="forceShowExpression"
:hint="hint" :hint="hint"
:hide-issues="hideIssues" :hide-issues="hideIssues"
:label="label"
@valueChanged="valueChanged" @valueChanged="valueChanged"
@textInput="onTextInput" @textInput="onTextInput"
@focus="onFocus" @focus="onFocus"

View file

@ -81,7 +81,16 @@
@valueChanged="valueChanged" @valueChanged="valueChanged"
/> />
</div> </div>
<resource-mapper
v-else-if="parameter.type === 'resourceMapper'"
:parameter="parameter"
:node="node"
:path="getPath(parameter.name)"
:dependentParametersValues="getDependentParametersValues(parameter)"
inputSize="small"
labelSize="small"
@valueChanged="valueChanged"
/>
<div v-else-if="displayNodeParameter(parameter)" class="parameter-item"> <div v-else-if="displayNodeParameter(parameter)" class="parameter-item">
<div <div
class="delete-option clickable" class="delete-option clickable"
@ -100,9 +109,11 @@
:parameter="parameter" :parameter="parameter"
:hide-issues="hiddenIssuesInputs.includes(parameter.name)" :hide-issues="hiddenIssuesInputs.includes(parameter.name)"
:value="getParameterValue(nodeValues, parameter.name, path)" :value="getParameterValue(nodeValues, parameter.name, path)"
:displayOptions="true" :displayOptions="shouldShowOptions(parameter)"
:path="getPath(parameter.name)" :path="getPath(parameter.name)"
:isReadOnly="isReadOnly" :isReadOnly="isReadOnly"
:hideLabel="false"
:nodeValues="nodeValues"
@valueChanged="valueChanged" @valueChanged="valueChanged"
@blur="onParameterBlur(parameter.name)" @blur="onParameterBlur(parameter.name)"
/> />
@ -127,7 +138,7 @@ import MultipleParameter from '@/components/MultipleParameter.vue';
import { workflowHelpers } from '@/mixins/workflowHelpers'; import { workflowHelpers } from '@/mixins/workflowHelpers';
import ParameterInputFull from '@/components/ParameterInputFull.vue'; import ParameterInputFull from '@/components/ParameterInputFull.vue';
import ImportParameter from '@/components/ImportParameter.vue'; import ImportParameter from '@/components/ImportParameter.vue';
import ResourceMapper from '@/components/ResourceMapper/ResourceMapper.vue';
import { get, set } from 'lodash-es'; import { get, set } from 'lodash-es';
import { useNDVStore } from '@/stores/ndv.store'; import { useNDVStore } from '@/stores/ndv.store';
import { useNodeTypesStore } from '@/stores/nodeTypes.store'; import { useNodeTypesStore } from '@/stores/nodeTypes.store';
@ -143,6 +154,7 @@ export default defineComponent({
FixedCollectionParameter: async () => import('./FixedCollectionParameter.vue'), FixedCollectionParameter: async () => import('./FixedCollectionParameter.vue'),
CollectionParameter: async () => import('./CollectionParameter.vue'), CollectionParameter: async () => import('./CollectionParameter.vue'),
ImportParameter, ImportParameter,
ResourceMapper,
}, },
props: { props: {
nodeValues: { nodeValues: {
@ -398,6 +410,33 @@ export default defineComponent({
// since there is no such case, omitting it to avoid additional computation // since there is no such case, omitting it to avoid additional computation
return isAuthRelatedParameter(this.nodeAuthFields, parameter); return isAuthRelatedParameter(this.nodeAuthFields, parameter);
}, },
shouldShowOptions(parameter: INodeProperties): boolean {
return parameter.type !== 'resourceMapper';
},
getDependentParametersValues(parameter: INodeProperties): string | null {
const loadOptionsDependsOn = this.getArgument('loadOptionsDependsOn', parameter) as
| string[]
| undefined;
if (loadOptionsDependsOn === undefined) {
return null;
}
// Get the resolved parameter values of the current node
const currentNodeParameters = this.ndvStore.activeNode?.parameters;
try {
const resolvedNodeParameters = this.resolveParameter(currentNodeParameters);
const returnValues: string[] = [];
for (const parameterPath of loadOptionsDependsOn) {
returnValues.push(get(resolvedNodeParameters, parameterPath) as string);
}
return returnValues.join('|');
} catch (error) {
return null;
}
},
}, },
watch: { watch: {
filteredParameterNames(newValue, oldValue) { filteredParameterNames(newValue, oldValue) {

View file

@ -16,6 +16,7 @@
:isForCredential="isForCredential" :isForCredential="isForCredential"
:eventSource="eventSource" :eventSource="eventSource"
:expressionEvaluated="expressionValueComputed" :expressionEvaluated="expressionValueComputed"
:label="label"
:data-test-id="`parameter-input-${parameter.name}`" :data-test-id="`parameter-input-${parameter.name}`"
@focus="onFocus" @focus="onFocus"
@blur="onBlur" @blur="onBlur"
@ -51,6 +52,7 @@ import InputHint from '@/components/ParameterInputHint.vue';
import type { import type {
INodeProperties, INodeProperties,
INodePropertyMode, INodePropertyMode,
IParameterLabel,
NodeParameterValue, NodeParameterValue,
NodeParameterValueType, NodeParameterValueType,
} from 'n8n-workflow'; } from 'n8n-workflow';
@ -116,6 +118,12 @@ export default defineComponent({
eventSource: { eventSource: {
type: String, type: String,
}, },
label: {
type: Object as PropType<IParameterLabel>,
default: () => ({
size: 'small',
}),
},
}, },
computed: { computed: {
...mapStores(useNDVStore), ...mapStores(useNDVStore),

View file

@ -1,26 +1,41 @@
<template> <template>
<div :class="$style.container"> <div :class="$style.container">
<n8n-action-toggle <div v-if="loading" :class="$style.loader">
v-if="shouldShowOptions" <n8n-text v-if="loading" size="small">
placement="bottom-end" <n8n-icon icon="sync-alt" size="xsmall" :spin="true" />
size="small" {{ loadingMessage }}
color="foreground-xdark" </n8n-text>
iconSize="small" </div>
:actions="actions" <div v-else :class="$style.controlsContainer">
@action="(action) => $emit('optionSelected', action)" <div
@visible-change="onMenuToggle" :class="{
/> [$style.noExpressionSelector]: !shouldShowExpressionSelector,
<n8n-radio-buttons }"
v-if="parameter.noDataExpression !== true && showExpressionSelector" >
size="small" <n8n-action-toggle
:value="selectedView" v-if="shouldShowOptions"
:disabled="isReadOnly" placement="bottom-end"
@input="onViewSelected" size="small"
:options="[ color="foreground-xdark"
{ label: $locale.baseText('parameterInput.fixed'), value: 'fixed' }, iconSize="small"
{ label: $locale.baseText('parameterInput.expression'), value: 'expression' }, :actions="actions"
]" :iconOrientation="iconOrientation"
/> @action="(action) => $emit('optionSelected', action)"
@visible-change="onMenuToggle"
/>
</div>
<n8n-radio-buttons
v-if="shouldShowExpressionSelector"
size="small"
:value="selectedView"
:disabled="isReadOnly"
@input="onViewSelected"
:options="[
{ label: $locale.baseText('parameterInput.fixed'), value: 'fixed' },
{ label: $locale.baseText('parameterInput.expression'), value: 'expression' },
]"
/>
</div>
</div> </div>
</template> </template>
@ -50,6 +65,25 @@ export default defineComponent({
type: Boolean, type: Boolean,
default: true, default: true,
}, },
customActions: {
type: Array as PropType<Array<{ label: string; value: string; disabled?: boolean }>>,
default: () => [],
},
iconOrientation: {
type: String,
default: 'vertical',
validator: (value: string): boolean => ['horizontal', 'vertical'].includes(value),
},
loading: {
type: Boolean,
default: false,
},
loadingMessage: {
type: String,
default() {
return this.$locale.baseText('genericHelpers.loading');
},
},
}, },
computed: { computed: {
isDefault(): boolean { isDefault(): boolean {
@ -61,6 +95,9 @@ export default defineComponent({
isHtmlEditor(): boolean { isHtmlEditor(): boolean {
return this.getArgument('editor') === 'htmlEditor'; return this.getArgument('editor') === 'htmlEditor';
}, },
shouldShowExpressionSelector(): boolean {
return this.parameter.noDataExpression !== true && this.showExpressionSelector;
},
shouldShowOptions(): boolean { shouldShowOptions(): boolean {
if (this.isReadOnly === true) { if (this.isReadOnly === true) {
return false; return false;
@ -91,6 +128,10 @@ export default defineComponent({
return !!this.getArgument('loadOptionsMethod') || !!this.getArgument('loadOptions'); return !!this.getArgument('loadOptionsMethod') || !!this.getArgument('loadOptions');
}, },
actions(): Array<{ label: string; value: string; disabled?: boolean }> { actions(): Array<{ label: string; value: string; disabled?: boolean }> {
if (Array.isArray(this.customActions) && this.customActions.length > 0) {
return this.customActions;
}
if (this.isHtmlEditor && !this.isValueExpression) { if (this.isHtmlEditor && !this.isValueExpression) {
return [ return [
{ {
@ -158,4 +199,19 @@ export default defineComponent({
.container { .container {
display: flex; display: flex;
} }
.loader > span {
line-height: 1em;
}
.controlsContainer {
display: flex;
}
.noExpressionSelector {
margin-bottom: var(--spacing-4xs);
span {
padding-right: 0 !important;
}
}
</style> </style>

View file

@ -0,0 +1,392 @@
<script setup lang="ts">
import type { IUpdateInformation } from '@/Interface';
import type {
FieldType,
INodeIssues,
INodeParameters,
INodeProperties,
NodePropertyTypes,
ResourceMapperField,
ResourceMapperValue,
} from 'n8n-workflow';
import ParameterInputFull from '@/components/ParameterInputFull.vue';
import ParameterIssues from '../ParameterIssues.vue';
import ParameterOptions from '../ParameterOptions.vue';
import { computed } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
import { useNDVStore } from '@/stores';
import { fieldCannotBeDeleted, isMatchingField, parseResourceMapperFieldName } from '@/utils';
import { useNodeSpecificationValues } from '@/composables';
interface Props {
parameter: INodeProperties;
path: string;
nodeValues: INodeParameters | undefined;
fieldsToMap: ResourceMapperField[];
paramValue: ResourceMapperValue;
labelSize: string;
showMatchingColumnsSelector: boolean;
showMappingModeSelect: boolean;
loading: boolean;
refreshInProgress: boolean;
}
const props = defineProps<Props>();
const FORCE_TEXT_INPUT_FOR_TYPES: FieldType[] = ['time', 'object', 'array'];
const {
resourceMapperTypeOptions,
singularFieldWord,
singularFieldWordCapitalized,
pluralFieldWord,
pluralFieldWordCapitalized,
} = useNodeSpecificationValues(props.parameter.typeOptions);
const emit = defineEmits<{
(event: 'fieldValueChanged', value: IUpdateInformation): void;
(event: 'removeField', field: string): void;
(event: 'addField', field: string): void;
(event: 'refreshFieldList'): void;
}>();
const ndvStore = useNDVStore();
const fieldsUi = computed<INodeProperties[]>(() => {
return props.fieldsToMap
.filter((field) => field.display !== false && field.removed !== true)
.map((field) => {
return {
displayName: getFieldLabel(field),
// Set part of the path to each param name so value can be fetched properly by input parameter list component
name: `value["${field.id}"]`,
type: getParamType(field),
default: field.type === 'boolean' ? false : '',
required: field.required,
description: getFieldDescription(field),
options: field.options,
};
});
});
const orderedFields = computed<INodeProperties[]>(() => {
// Sort so that matching columns are first
if (props.paramValue.matchingColumns) {
fieldsUi.value.forEach((field, i) => {
const fieldName = parseResourceMapperFieldName(field.name);
if (fieldName) {
if (props.paramValue.matchingColumns.includes(fieldName)) {
fieldsUi.value.splice(i, 1);
fieldsUi.value.unshift(field);
}
}
});
}
return fieldsUi.value;
});
const removedFields = computed<ResourceMapperField[]>(() => {
return props.fieldsToMap.filter((field) => field.removed === true && field.display !== false);
});
const addFieldOptions = computed<Array<{ name: string; value: string; disabled?: boolean }>>(() => {
return removedFields.value.map((field) => {
return {
name: field.displayName,
value: field.id,
disabled: false,
};
});
});
const parameterActions = computed<Array<{ label: string; value: string; disabled?: boolean }>>(
() => {
return [
{
label: locale.baseText('resourceMapper.refreshFieldList', {
interpolate: { fieldWord: singularFieldWordCapitalized.value },
}),
value: 'refreshFieldList',
},
{
label: locale.baseText('resourceMapper.addAllFields', {
interpolate: { fieldWord: pluralFieldWordCapitalized.value },
}),
value: 'addAllFields',
disabled: removedFields.value.length === 0,
},
{
label: locale.baseText('resourceMapper.removeAllFields', {
interpolate: { fieldWord: pluralFieldWordCapitalized.value },
}),
value: 'removeAllFields',
disabled: isRemoveAllAvailable.value === false,
},
];
},
);
const isRemoveAllAvailable = computed<boolean>(() => {
return (
removedFields.value.length !== props.fieldsToMap.length &&
props.fieldsToMap.some((field) => {
return (
field.removed !== true &&
!fieldCannotBeDeleted(
field,
props.showMatchingColumnsSelector,
resourceMapperMode.value,
props.paramValue.matchingColumns,
)
);
})
);
});
const resourceMapperMode = computed<string | undefined>(() => {
return resourceMapperTypeOptions.value?.mode;
});
const valuesLabel = computed<string>(() => {
if (resourceMapperMode.value && resourceMapperMode.value === 'update') {
return locale.baseText('resourceMapper.valuesToUpdate.label');
}
return locale.baseText('resourceMapper.valuesToSend.label');
});
const fetchingFieldsLabel = computed<string>(() => {
return locale.baseText('resourceMapper.fetchingFields.message', {
interpolate: {
fieldWord: pluralFieldWord.value,
},
});
});
function getFieldLabel(field: ResourceMapperField): string {
if (
isMatchingField(field.id, props.paramValue.matchingColumns, props.showMatchingColumnsSelector)
) {
const suffix = locale.baseText('resourceMapper.usingToMatch') || '';
return `${field.displayName} ${suffix}`;
}
return field.displayName;
}
function getFieldDescription(field: ResourceMapperField): string {
if (
isMatchingField(field.id, props.paramValue.matchingColumns, props.showMatchingColumnsSelector)
) {
return (
locale.baseText('resourceMapper.usingToMatch.description', {
interpolate: {
fieldWord: singularFieldWord.value,
},
}) || ''
);
}
return '';
}
function getParameterValue(parameterName: string) {
const fieldName = parseResourceMapperFieldName(parameterName);
if (fieldName && props.paramValue.value) {
return props.paramValue.value[fieldName];
}
return null;
}
function getFieldIssues(field: INodeProperties): string[] {
if (!ndvStore.activeNode) return [];
const nodeIssues = ndvStore.activeNode.issues || ({} as INodeIssues);
const fieldName = parseResourceMapperFieldName(field.name);
if (!fieldName) return [];
let fieldIssues: string[] = [];
const key = `${props.parameter.name}.${fieldName}`;
if (nodeIssues['parameters'] && key in nodeIssues['parameters']) {
fieldIssues = fieldIssues.concat(nodeIssues['parameters'][key]);
}
return fieldIssues;
}
function getParamType(field: ResourceMapperField): NodePropertyTypes {
if (field.type && !FORCE_TEXT_INPUT_FOR_TYPES.includes(field.type)) {
return field.type as NodePropertyTypes;
}
return 'string';
}
function onValueChanged(value: IUpdateInformation): void {
emit('fieldValueChanged', value);
}
function removeField(fieldName: string) {
emit('removeField', fieldName);
}
function addField(fieldName: string) {
emit('addField', fieldName);
}
function onParameterActionSelected(action: string): void {
switch (action) {
case 'addAllFields':
emit('addField', action);
break;
case 'removeAllFields':
emit('removeField', action);
break;
case 'refreshFieldList':
emit('refreshFieldList');
break;
default:
break;
}
}
defineExpose({
orderedFields,
});
</script>
<template>
<div class="mt-xs" data-test-id="mapping-fields-container">
<n8n-input-label
:label="valuesLabel"
:underline="true"
:size="labelSize"
:showOptions="true"
:showExpressionSelector="false"
color="text-dark"
>
<template #options>
<parameter-options
:parameter="parameter"
:customActions="parameterActions"
:loading="props.refreshInProgress"
:loadingMessage="fetchingFieldsLabel"
@optionSelected="onParameterActionSelected"
/>
</template>
</n8n-input-label>
<div v-if="orderedFields.length === 0" class="mt-3xs mb-xs">
<n8n-text size="small">{{
$locale.baseText('fixedCollectionParameter.currentlyNoItemsExist')
}}</n8n-text>
</div>
<div
v-for="field in orderedFields"
:key="field.name"
:class="{
['parameter-item']: true,
[$style.parameterItem]: true,
[$style.hasIssues]: getFieldIssues(field).length > 0,
}"
>
<div
v-if="resourceMapperMode === 'add' && field.required"
:class="['delete-option', 'mt-5xs', $style.parameterTooltipIcon]"
>
<n8n-tooltip placement="top">
<template #content>
<span>{{
locale.baseText('resourceMapper.mandatoryField.title', {
interpolate: { fieldWord: singularFieldWord },
})
}}</span>
</template>
<font-awesome-icon icon="question-circle" />
</n8n-tooltip>
</div>
<div
v-else-if="
!isMatchingField(
field.name,
props.paramValue.matchingColumns,
props.showMatchingColumnsSelector,
)
"
:class="['delete-option', 'clickable', 'mt-5xs']"
>
<font-awesome-icon
icon="trash"
:title="
locale.baseText('resourceMapper.removeField', {
interpolate: {
fieldWord: singularFieldWordCapitalized,
},
})
"
data-test-id="remove-field-button"
@click="removeField(field.name)"
/>
</div>
<div :class="$style.parameterInput">
<parameter-input-full
:parameter="field"
:value="getParameterValue(field.name)"
:displayOptions="true"
:path="`${props.path}.${field.name}`"
:isReadOnly="refreshInProgress"
:hideIssues="true"
:nodeValues="nodeValues"
:class="$style.parameterInputFull"
@valueChanged="onValueChanged"
/>
</div>
<parameter-issues
v-if="getFieldIssues(field).length > 0"
:issues="getFieldIssues(field)"
:class="[$style.parameterIssues, 'ml-5xs']"
/>
</div>
<div class="add-option" data-test-id="add-fields-select">
<n8n-select
:placeholder="
locale.baseText('resourceMapper.addFieldToSend', {
interpolate: { fieldWord: singularFieldWordCapitalized },
})
"
size="small"
:disabled="addFieldOptions.length == 0"
@change="addField"
>
<n8n-option
v-for="item in addFieldOptions"
:key="item.value"
:label="item.name"
:value="item.value"
:disabled="item.disabled"
>
</n8n-option>
</n8n-select>
</div>
</div>
</template>
<style module lang="scss">
.parameterItem {
display: flex;
padding: 0 0 0 1em;
.parameterInput {
width: 100%;
}
&.hasIssues {
.parameterIssues {
float: none;
padding-top: var(--spacing-xl);
}
input,
input:focus {
--input-border-color: var(--color-danger);
border-color: var(--color-danger);
}
}
}
.parameterTooltipIcon {
color: var(--color-text-light) !important;
}
</style>

View file

@ -0,0 +1,153 @@
<script setup lang="ts">
import type { INodePropertyTypeOptions, ResourceMapperFields } from 'n8n-workflow';
import { computed, ref, watch } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
import { useNodeSpecificationValues } from '@/composables';
interface Props {
initialValue: string;
fieldsToMap: ResourceMapperFields['fields'];
inputSize: string;
labelSize: string;
typeOptions: INodePropertyTypeOptions | undefined;
serviceName: string;
loading: boolean;
loadingError: boolean;
}
const props = defineProps<Props>();
const { resourceMapperTypeOptions, pluralFieldWord, singularFieldWord } =
useNodeSpecificationValues(props.typeOptions);
// Mapping mode options: Labels here use field words defined in parameter type options
const mappingModeOptions = [
{
name: locale.baseText('resourceMapper.mappingMode.defineBelow.name'),
value: 'defineBelow',
description: locale.baseText('resourceMapper.mappingMode.defineBelow.description', {
interpolate: {
fieldWord: singularFieldWord.value,
},
}),
},
{
name: locale.baseText('resourceMapper.mappingMode.autoMapInputData.name'),
value: 'autoMapInputData',
description: locale.baseText('resourceMapper.mappingMode.autoMapInputData.description', {
interpolate: {
fieldWord: pluralFieldWord.value,
serviceName: props.serviceName,
},
}),
},
];
const emit = defineEmits<{
(event: 'modeChanged', value: string): void;
(event: 'retryFetch'): void;
}>();
const selected = ref(props.initialValue);
watch(
() => props.initialValue,
() => {
selected.value = props.initialValue;
},
);
const errorMessage = computed<string>(() => {
if (selected.value === 'defineBelow') {
// Loading error message
if (props.loadingError) {
return locale.baseText('resourceMapper.fetchingFields.errorMessage', {
interpolate: {
fieldWord: pluralFieldWord.value,
},
});
}
// No data error message
if (props.fieldsToMap.length === 0) {
return (
// Use custom error message if defined
resourceMapperTypeOptions.value?.noFieldsError ||
locale.baseText('resourceMapper.fetchingFields.noFieldsFound', {
interpolate: {
fieldWord: pluralFieldWord.value,
serviceName: props.serviceName,
},
})
);
}
return '';
}
return '';
});
function onModeChanged(value: string): void {
selected.value = value;
emit('modeChanged', value);
}
function onRetryClick(): void {
emit('retryFetch');
}
defineExpose({
selected,
onModeChanged,
mappingModeOptions,
});
</script>
<template>
<div data-test-id="mapping-mode-select">
<n8n-input-label
:label="locale.baseText('resourceMapper.mappingMode.label')"
:bold="false"
:required="false"
:size="labelSize"
color="text-dark"
>
<template>
<div class="mt-5xs">
<n8n-select :value="selected" :size="props.inputSize" @change="onModeChanged">
<n8n-option
v-for="option in mappingModeOptions"
:key="option.value"
:value="option.value"
:label="option.name"
description="sadasd"
>
<div class="list-option">
<div class="option-headline">
{{ option.name }}
</div>
<div class="option-description" v-html="option.description" />
</div>
</n8n-option>
</n8n-select>
</div>
<div class="mt-5xs">
<n8n-text v-if="loading" size="small">
<n8n-icon icon="sync-alt" size="xsmall" :spin="true" />
{{
locale.baseText('resourceMapper.fetchingFields.message', {
interpolate: {
fieldWord: pluralFieldWord,
},
})
}}
</n8n-text>
<n8n-text v-else-if="errorMessage !== ''" size="small" color="danger">
<n8n-icon icon="exclamation-triangle" size="xsmall" />
{{ errorMessage }}
<n8n-link size="small" theme="danger" :underline="true" @click="onRetryClick">
{{ locale.baseText('generic.retry') }}
</n8n-link>
</n8n-text>
</div>
</template>
</n8n-input-label>
</div>
</template>

View file

@ -0,0 +1,155 @@
<script setup lang="ts">
import type {
INodePropertyTypeOptions,
ResourceMapperField,
ResourceMapperFields,
} from 'n8n-workflow';
import { computed, reactive, watch } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
import { useNodeSpecificationValues } from '@/composables';
interface Props {
initialValue: string[];
fieldsToMap: ResourceMapperFields['fields'];
typeOptions: INodePropertyTypeOptions | undefined;
labelSize: string;
inputSize: string;
loading: boolean;
serviceName: string;
}
const props = defineProps<Props>();
const {
resourceMapperTypeOptions,
singularFieldWord,
singularFieldWordCapitalized,
pluralFieldWord,
pluralFieldWordCapitalized,
} = useNodeSpecificationValues(props.typeOptions);
// Depending on the mode (multiple/singe key column), the selected value can be a string or an array of strings
const state = reactive({
selected: props.initialValue as string[] | string,
});
watch(
() => props.initialValue,
() => {
state.selected =
resourceMapperTypeOptions.value?.multiKeyMatch === true
? props.initialValue
: props.initialValue[0];
},
);
const emit = defineEmits<{
(event: 'matchingColumnsChanged', value: string[]): void;
}>();
const availableMatchingFields = computed<ResourceMapperField[]>(() => {
return props.fieldsToMap.filter((field) => {
return field.canBeUsedToMatch !== false && field.display !== false;
});
});
// Field label, description and tooltip: Labels here use content and field words defined in parameter type options
const fieldLabel = computed<string>(() => {
if (resourceMapperTypeOptions.value?.matchingFieldsLabels?.title) {
return resourceMapperTypeOptions.value.matchingFieldsLabels.title;
}
const fieldWord =
resourceMapperTypeOptions.value?.multiKeyMatch === true
? pluralFieldWordCapitalized.value
: singularFieldWordCapitalized.value;
return locale.baseText('resourceMapper.columnsToMatchOn.label', {
interpolate: {
fieldWord,
},
});
});
const fieldDescription = computed<string>(() => {
if (resourceMapperTypeOptions.value?.matchingFieldsLabels?.hint) {
return resourceMapperTypeOptions.value.matchingFieldsLabels.hint;
}
const labeli18nKey =
resourceMapperTypeOptions.value?.multiKeyMatch === true
? 'resourceMapper.columnsToMatchOn.multi.description'
: 'resourceMapper.columnsToMatchOn.single.description';
return locale.baseText(labeli18nKey, {
interpolate: {
fieldWord:
resourceMapperTypeOptions.value?.multiKeyMatch === true
? `${pluralFieldWord.value}`
: `${singularFieldWord.value}`,
},
});
});
const fieldTooltip = computed<string>(() => {
if (resourceMapperTypeOptions.value?.matchingFieldsLabels?.description) {
return resourceMapperTypeOptions.value.matchingFieldsLabels.description;
}
return locale.baseText('resourceMapper.columnsToMatchOn.tooltip', {
interpolate: {
fieldWord:
resourceMapperTypeOptions.value?.multiKeyMatch === true
? `${pluralFieldWord.value}`
: `${singularFieldWord.value}`,
},
});
});
function onSelectionChange(value: string | string[]) {
if (resourceMapperTypeOptions.value?.multiKeyMatch === true) {
state.selected = value as string[];
} else {
state.selected = value as string;
}
emitValueChanged();
}
function emitValueChanged() {
emit('matchingColumnsChanged', Array.isArray(state.selected) ? state.selected : [state.selected]);
}
defineExpose({
state,
});
</script>
<template>
<div class="mt-2xs" data-test-id="matching-column-select">
<n8n-input-label
v-if="availableMatchingFields.length > 0"
:label="fieldLabel"
:tooltipText="fieldTooltip"
:bold="false"
:required="false"
:size="labelSize"
color="text-dark"
>
<n8n-select
:multiple="resourceMapperTypeOptions?.multiKeyMatch === true"
:value="state.selected"
:size="props.inputSize"
:disabled="loading"
@change="onSelectionChange"
>
<n8n-option v-for="field in availableMatchingFields" :key="field.id" :value="field.id">
{{ field.displayName }}
</n8n-option>
</n8n-select>
<n8n-text size="small">
{{ fieldDescription }}
</n8n-text>
</n8n-input-label>
<n8n-notice v-else>
{{
locale.baseText('resourceMapper.columnsToMatchOn.noFieldsFound', {
interpolate: { fieldWord: singularFieldWord, serviceName: props.serviceName },
})
}}
</n8n-notice>
</div>
</template>

View file

@ -0,0 +1,500 @@
<script setup lang="ts">
import type { IUpdateInformation, ResourceMapperReqParams } from '@/Interface';
import { resolveParameter } from '@/mixins/workflowHelpers';
import { useNodeTypesStore } from '@/stores/nodeTypes.store';
import type {
INode,
INodeParameters,
INodeProperties,
INodeTypeDescription,
ResourceMapperField,
ResourceMapperValue,
} from 'n8n-workflow';
import { NodeHelpers } from 'n8n-workflow';
import { computed, onMounted, reactive, watch } from 'vue';
import MappingModeSelect from './MappingModeSelect.vue';
import MatchingColumnsSelect from './MatchingColumnsSelect.vue';
import MappingFields from './MappingFields.vue';
import { fieldCannotBeDeleted, isResourceMapperValue, parseResourceMapperFieldName } from '@/utils';
import { i18n as locale } from '@/plugins/i18n';
import { useNDVStore } from '@/stores/ndv.store';
interface Props {
parameter: INodeProperties;
node: INode | null;
path: string;
inputSize: string;
labelSize: string;
dependentParametersValues: string | null;
}
const nodeTypesStore = useNodeTypesStore();
const ndvStore = useNDVStore();
const props = defineProps<Props>();
const emit = defineEmits<{
(event: 'valueChanged', value: IUpdateInformation): void;
}>();
const state = reactive({
paramValue: {
mappingMode: 'defineBelow',
value: {},
matchingColumns: [] as string[],
schema: [] as ResourceMapperField[],
} as ResourceMapperValue,
parameterValues: {} as INodeParameters,
loading: false,
refreshInProgress: false, // Shows inline loader when refreshing fields
loadingError: false,
});
// Reload fields to map when dependent parameters change
watch(
() => props.dependentParametersValues,
async (currentValue, oldValue) => {
if (oldValue !== null && currentValue !== null && oldValue !== currentValue) {
state.paramValue = {
...state.paramValue,
value: null,
schema: [],
};
emitValueChanged();
await initFetching();
setDefaultFieldValues(true);
}
},
);
onMounted(async () => {
if (props.node) {
state.parameterValues = {
...state.parameterValues,
parameters: props.node.parameters,
};
}
const params = state.parameterValues.parameters as INodeParameters;
const parameterName = props.parameter.name;
if (!(parameterName in params)) {
return;
}
let hasSchema = false;
const nodeValues = params[parameterName] as unknown as ResourceMapperValue;
state.paramValue = {
...state.paramValue,
...nodeValues,
};
if (!state.paramValue.schema) {
state.paramValue = {
...state.paramValue,
schema: [],
};
} else {
hasSchema = state.paramValue.schema.length > 0;
}
Object.keys(state.paramValue.value || {}).forEach((key) => {
if (state.paramValue.value && state.paramValue.value[key] === '') {
state.paramValue = {
...state.paramValue,
value: {
...state.paramValue.value,
[key]: null,
},
};
}
});
if (nodeValues.matchingColumns) {
state.paramValue = {
...state.paramValue,
matchingColumns: nodeValues.matchingColumns,
};
}
await initFetching(hasSchema);
// Set default values if this is the first time the parameter is being set
if (!state.paramValue.value) {
setDefaultFieldValues();
}
updateNodeIssues();
});
const resourceMapperMode = computed<string | undefined>(() => {
return props.parameter.typeOptions?.resourceMapper?.mode;
});
const nodeType = computed<INodeTypeDescription | null>(() => {
if (props.node) {
return nodeTypesStore.getNodeType(props.node.type, props.node.typeVersion);
}
return null;
});
const showMappingModeSelect = computed<boolean>(() => {
return props.parameter.typeOptions?.resourceMapper?.supportAutoMap !== false;
});
const showMatchingColumnsSelector = computed<boolean>(() => {
return (
!state.loading &&
props.parameter.typeOptions?.resourceMapper?.mode !== 'add' &&
state.paramValue.schema.length > 0
);
});
const showMappingFields = computed<boolean>(() => {
return (
state.paramValue.mappingMode === 'defineBelow' &&
!state.loading &&
!state.loadingError &&
state.paramValue.schema.length > 0 &&
hasAvailableMatchingColumns.value
);
});
const matchingColumns = computed<string[]>(() => {
if (!showMatchingColumnsSelector.value) {
return [];
}
if (state.paramValue.matchingColumns) {
return state.paramValue.matchingColumns;
}
return defaultSelectedMatchingColumns.value;
});
const hasAvailableMatchingColumns = computed<boolean>(() => {
if (resourceMapperMode.value !== 'add') {
return (
state.paramValue.schema.filter(
(field) =>
field.canBeUsedToMatch !== false && field.display !== false && field.removed !== true,
).length > 0
);
}
return true;
});
const defaultSelectedMatchingColumns = computed<string[]>(() => {
return state.paramValue.schema.length === 1
? [state.paramValue.schema[0].id]
: state.paramValue.schema.reduce((acc, field) => {
if (field.defaultMatch && field.canBeUsedToMatch === true) {
acc.push(field.id);
}
return acc;
}, [] as string[]);
});
const pluralFieldWord = computed<string>(() => {
return (
props.parameter.typeOptions?.resourceMapper?.fieldWords?.plural ||
locale.baseText('generic.fields')
);
});
async function initFetching(inlineLading = false): Promise<void> {
state.loadingError = false;
if (inlineLading) {
state.refreshInProgress = true;
} else {
state.loading = true;
}
try {
await loadFieldsToMap();
if (!state.paramValue.matchingColumns || state.paramValue.matchingColumns.length === 0) {
onMatchingColumnsChanged(defaultSelectedMatchingColumns.value);
}
} catch (error) {
state.loadingError = true;
} finally {
state.loading = false;
state.refreshInProgress = false;
}
}
async function loadFieldsToMap(): Promise<void> {
if (!props.node) {
return;
}
const requestParams: ResourceMapperReqParams = {
nodeTypeAndVersion: {
name: props.node?.type,
version: props.node.typeVersion,
},
currentNodeParameters: resolveParameter(props.node.parameters) as INodeParameters,
path: props.path,
methodName: props.parameter.typeOptions?.resourceMapper?.resourceMapperMethod,
credentials: props.node.credentials,
};
const fetchedFields = await nodeTypesStore.getResourceMapperFields(requestParams);
if (fetchedFields !== null) {
const newSchema = fetchedFields.fields.map((field) => {
const existingField = state.paramValue.schema.find((f) => f.id === field.id);
if (existingField) {
field.removed = existingField.removed;
} else if (state.paramValue.value !== null && !(field.id in state.paramValue.value)) {
// New fields are shown by default
field.removed = false;
}
return field;
});
state.paramValue = {
...state.paramValue,
schema: newSchema,
};
emitValueChanged();
}
}
async function onModeChanged(mode: string): Promise<void> {
state.paramValue.mappingMode = mode;
if (mode === 'defineBelow') {
await initFetching();
} else {
state.loadingError = false;
}
emitValueChanged();
}
function setDefaultFieldValues(forceMatchingFieldsUpdate = false): void {
state.paramValue.value = {};
const hideAllFields = props.parameter.typeOptions?.resourceMapper?.addAllFields === false;
state.paramValue.schema.forEach((field) => {
if (state.paramValue.value) {
// Hide all non-required fields if it's configured in node definition
if (hideAllFields) {
field.removed = !field.required;
}
if (field.type === 'boolean') {
state.paramValue.value = {
...state.paramValue.value,
[field.id]: false,
};
} else {
state.paramValue.value = {
...state.paramValue.value,
[field.id]: null,
};
}
}
});
emitValueChanged();
if (!state.paramValue.matchingColumns || forceMatchingFieldsUpdate) {
state.paramValue.matchingColumns = defaultSelectedMatchingColumns.value;
emitValueChanged();
}
}
function updateNodeIssues(): void {
if (props.node) {
const parameterIssues = NodeHelpers.getNodeParametersIssues(
nodeType.value?.properties || [],
props.node,
);
if (parameterIssues) {
ndvStore.updateNodeParameterIssues(parameterIssues);
}
}
}
function onMatchingColumnsChanged(matchingColumns: string[]): void {
state.paramValue = {
...state.paramValue,
matchingColumns,
};
// Set all matching fields to be visible
state.paramValue.schema.forEach((field) => {
if (state.paramValue.matchingColumns?.includes(field.id)) {
field.removed = false;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
});
if (!state.loading) {
emitValueChanged();
}
}
function fieldValueChanged(updateInfo: IUpdateInformation): void {
let newValue = null;
if (
updateInfo.value !== undefined &&
updateInfo.value !== '' &&
updateInfo.value !== null &&
isResourceMapperValue(updateInfo.value)
) {
newValue = updateInfo.value;
}
const fieldName = parseResourceMapperFieldName(updateInfo.name);
if (fieldName && state.paramValue.value) {
state.paramValue.value = {
...state.paramValue.value,
[fieldName]: newValue,
};
emitValueChanged();
}
}
function removeField(name: string): void {
if (name === 'removeAllFields') {
return removeAllFields();
}
const fieldName = parseResourceMapperFieldName(name);
if (fieldName) {
if (state.paramValue.value) {
delete state.paramValue.value[fieldName];
const field = state.paramValue.schema.find((f) => f.id === fieldName);
if (field) {
field.removed = true;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
emitValueChanged();
}
}
}
function addField(name: string): void {
if (name === 'addAllFields') {
return addAllFields();
}
if (name === 'removeAllFields') {
return removeAllFields();
}
state.paramValue.value = {
...state.paramValue.value,
[name]: null,
};
const field = state.paramValue.schema.find((f) => f.id === name);
if (field) {
field.removed = false;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
emitValueChanged();
}
function addAllFields(): void {
const newValues: { [name: string]: null } = {};
state.paramValue.schema.forEach((field) => {
if (field.display && field.removed) {
newValues[field.id] = null;
field.removed = false;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
});
state.paramValue.value = {
...state.paramValue.value,
...newValues,
};
emitValueChanged();
}
function removeAllFields(): void {
state.paramValue.schema.forEach((field) => {
if (
!fieldCannotBeDeleted(
field,
showMatchingColumnsSelector.value,
resourceMapperMode.value,
matchingColumns.value,
)
) {
field.removed = true;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
});
emitValueChanged();
}
function emitValueChanged(): void {
pruneParamValues();
emit('valueChanged', {
name: `${props.path}`,
value: state.paramValue,
node: props.node?.name,
});
updateNodeIssues();
}
function pruneParamValues(): void {
if (!state.paramValue.value) {
return;
}
const valueKeys = Object.keys(state.paramValue.value);
valueKeys.forEach((key) => {
if (state.paramValue.value && state.paramValue.value[key] === null) {
delete state.paramValue.value[key];
}
});
}
defineExpose({
state,
});
</script>
<template>
<div class="mt-4xs" data-test-id="resource-mapper-container">
<mapping-mode-select
v-if="showMappingModeSelect"
:inputSize="inputSize"
:labelSize="labelSize"
:initialValue="state.paramValue.mappingMode || 'defineBelow'"
:typeOptions="props.parameter.typeOptions"
:serviceName="nodeType?.displayName || locale.baseText('generic.service')"
:loading="state.loading"
:loadingError="state.loadingError"
:fieldsToMap="state.paramValue.schema"
@modeChanged="onModeChanged"
@retryFetch="initFetching"
/>
<matching-columns-select
v-if="showMatchingColumnsSelector"
:label-size="labelSize"
:fieldsToMap="state.paramValue.schema"
:typeOptions="props.parameter.typeOptions"
:inputSize="inputSize"
:loading="state.loading"
:initialValue="matchingColumns"
:serviceName="nodeType?.displayName || locale.baseText('generic.service')"
@matchingColumnsChanged="onMatchingColumnsChanged"
/>
<n8n-text v-if="!showMappingModeSelect && state.loading" size="small">
<n8n-icon icon="sync-alt" size="xsmall" :spin="true" />
{{
locale.baseText('resourceMapper.fetchingFields.message', {
interpolate: {
fieldWord: pluralFieldWord,
},
})
}}
</n8n-text>
<mapping-fields
v-if="showMappingFields"
:parameter="props.parameter"
:path="props.path"
:nodeValues="state.parameterValues"
:fieldsToMap="state.paramValue.schema"
:paramValue="state.paramValue"
:labelSize="labelSize"
:showMatchingColumnsSelector="showMatchingColumnsSelector"
:showMappingModeSelect="showMappingModeSelect"
:loading="state.loading"
:refreshInProgress="state.refreshInProgress"
@fieldValueChanged="fieldValueChanged"
@removeField="removeField"
@addField="addField"
@refreshFieldList="initFetching(true)"
/>
<n8n-notice
v-if="state.paramValue.mappingMode === 'autoMapInputData' && hasAvailableMatchingColumns"
>
{{
locale.baseText('resourceMapper.autoMappingNotice', {
interpolate: {
fieldWord: pluralFieldWord,
serviceName: nodeType?.displayName || locale.baseText('generic.service'),
},
})
}}
</n8n-notice>
</div>
</template>

View file

@ -0,0 +1,232 @@
import { PiniaVuePlugin } from 'pinia';
import { render, within } from '@testing-library/vue';
import { merge } from 'lodash-es';
import {
DEFAULT_SETUP,
MAPPING_COLUMNS_RESPONSE,
NODE_PARAMETER_VALUES,
UPDATED_SCHEMA,
} from './utils/ResourceMapper.utils';
import { useNodeTypesStore } from '@/stores/nodeTypes.store';
import { waitAllPromises } from '@/__tests__/utils';
import * as workflowHelpers from '@/mixins/workflowHelpers';
import ResourceMapper from '@/components/ResourceMapper/ResourceMapper.vue';
import userEvent from '@testing-library/user-event';
let nodeTypeStore: ReturnType<typeof useNodeTypesStore>;
const renderComponent = (renderOptions: Parameters<typeof render>[1] = {}) =>
render(ResourceMapper, merge(DEFAULT_SETUP, renderOptions), (vue) => {
vue.use(PiniaVuePlugin);
});
describe('ResourceMapper.vue', () => {
beforeEach(() => {
nodeTypeStore = useNodeTypesStore();
vi.spyOn(workflowHelpers, 'resolveParameter').mockReturnValue(NODE_PARAMETER_VALUES);
vi.spyOn(nodeTypeStore, 'getResourceMapperFields').mockResolvedValue(MAPPING_COLUMNS_RESPONSE);
});
afterEach(() => {
vi.clearAllMocks();
});
it('renders default configuration properly', async () => {
const { getByTestId } = renderComponent();
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
expect(getByTestId('mapping-mode-select')).toBeInTheDocument();
expect(getByTestId('matching-column-select')).toBeInTheDocument();
expect(getByTestId('mapping-fields-container')).toBeInTheDocument();
// Should render one parameter input for each fetched column
expect(
getByTestId('mapping-fields-container').querySelectorAll('.parameter-input').length,
).toBe(MAPPING_COLUMNS_RESPONSE.fields.length);
});
it('renders add mode properly', async () => {
const { getByTestId, queryByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
mode: 'add',
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
// This mode doesn't render matching column selector
expect(queryByTestId('matching-column-select')).not.toBeInTheDocument();
});
it('renders multi-key match selector properly', async () => {
const { container, getByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
mode: 'upsert',
multiKeyMatch: true,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
expect(container.querySelector('.el-select__tags')).toBeInTheDocument();
});
it('does not render mapping mode selector if it is disabled', async () => {
const { getByTestId, queryByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
supportAutoMap: false,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
expect(queryByTestId('mapping-mode-select')).not.toBeInTheDocument();
});
it('renders field on top of the list when they are selected for matching', async () => {
const { container, getByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
supportAutoMap: true,
mode: 'upsert',
multiKeyMatch: false,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
// Id should be the first field in the list
expect(container.querySelector('.parameter-item')).toContainHTML('id (using to match)');
// // Select Last Name as matching column
await userEvent.click(getByTestId('matching-column-select'));
const matchingColumnDropdown = getByTestId('matching-column-select');
await userEvent.click(within(matchingColumnDropdown).getByText('Last Name'));
// // Now, last name should be the first field in the list
expect(container.querySelector('.parameter-item')).toContainHTML('Last Name (using to match)');
});
it('renders selected matching columns properly when multiple key matching is enabled', async () => {
const { getByTestId, getByText, queryByText } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
supportAutoMap: true,
mode: 'upsert',
multiKeyMatch: true,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
const matchingColumnDropdown = getByTestId('matching-column-select');
await userEvent.click(matchingColumnDropdown);
await userEvent.click(within(matchingColumnDropdown).getByText('Username'));
// Both matching columns should be rendered in the dropdown
expect(getByTestId('matching-column-select')).toContainHTML(
'<span class="el-select__tags-text">id</span>',
);
expect(getByTestId('matching-column-select')).toContainHTML(
'<span class="el-select__tags-text">Username</span>',
);
// All selected columns should have correct labels
expect(getByText('id (using to match)')).toBeInTheDocument();
expect(getByText('Username (using to match)')).toBeInTheDocument();
expect(queryByText('First Name (using to match)')).not.toBeInTheDocument();
});
it('uses field words defined in node definition', async () => {
const { getByText } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
fieldWords: {
singular: 'foo',
plural: 'foos',
},
},
},
},
},
});
await waitAllPromises();
expect(getByText('Set the value for each foo')).toBeInTheDocument();
expect(
getByText('Look for incoming data that matches the foos in the service'),
).toBeInTheDocument();
expect(getByText('Foos to Match On')).toBeInTheDocument();
expect(getByText('The foos that identify the row(s) to modify')).toBeInTheDocument();
});
it('should render correct fields based on saved schema', async () => {
const { getByTestId, queryAllByTestId } = renderComponent({
props: {
node: {
parameters: {
columns: {
schema: UPDATED_SCHEMA,
},
},
},
parameter: {
typeOptions: {
resourceMapper: {
mode: 'add',
},
},
},
},
});
await waitAllPromises();
// There should be 5 fields rendered and only 2 of them should have remove button
expect(
getByTestId('mapping-fields-container').querySelectorAll('.parameter-input').length,
).toBe(5);
expect(queryAllByTestId('remove-field-button').length).toBe(2);
});
it('should render correct options based on saved schema', async () => {
const { getByTestId } = renderComponent({
props: {
node: {
parameters: {
columns: {
schema: UPDATED_SCHEMA,
},
},
},
parameter: {
typeOptions: {
resourceMapper: {
mode: 'add',
},
},
},
},
});
await waitAllPromises();
// Should have one option in the bottom dropdown for one removed field
expect(getByTestId('add-fields-select').querySelectorAll('li').length).toBe(1);
});
});

View file

@ -0,0 +1,214 @@
import { SETTINGS_STORE_DEFAULT_STATE } from '@/__tests__/utils';
import { STORES } from '@/constants';
import { createTestingPinia } from '@pinia/testing';
import { merge } from 'lodash-es';
import type { ResourceMapperFields } from 'n8n-workflow';
export const NODE_PARAMETER_VALUES = {
authentication: 'oAuth2',
resource: 'sheet',
operation: 'appendOrUpdate',
documentId: {
__rl: true,
value:
'https://docs.google.com/spreadsheets/d/1BAjxEhlUu5tXDCMQcjqjguIZDFuct3FYkdo7flxl3yc/edit#gid=0',
mode: 'url',
__regex: 'https:\\/\\/(?:drive|docs)\\.google\\.com\\/\\w+\\/d\\/([0-9a-zA-Z\\-_]+)(?:\\/.*|)',
},
sheetName: {
__rl: true,
value: 'gid=0',
mode: 'list',
cachedResultName: 'Users',
cachedResultUrl:
'https://docs.google.com/spreadsheets/d/1BAjxEhlUu5tXDCMQcjqjguIZDFuct3FYkdo7flxl3yc/edit#gid=0',
},
columns: {
mappingMode: 'defineBelow',
value: null,
},
options: {},
};
export const UPDATED_SCHEMA = [
{
id: 'First name',
displayName: 'First name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
{
id: 'Last name',
displayName: 'Last name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
{
id: 'Username',
displayName: 'Username',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
{
id: 'Address',
displayName: 'Address',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: true,
},
{
id: 'id',
displayName: 'id',
match: false,
required: true,
defaultMatch: true,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
];
export const DEFAULT_SETUP = {
pinia: createTestingPinia({
initialState: {
[STORES.SETTINGS]: {
settings: merge({}, SETTINGS_STORE_DEFAULT_STATE.settings),
},
},
}),
props: {
path: 'parameters.columns',
dependentParametersValues: 'gid=0',
inputSize: 'small',
labelSize: 'small',
node: {
parameters: NODE_PARAMETER_VALUES,
id: 'f63efb2d-3cc5-4500-89f9-b39aab19baf5',
name: 'Google Sheets',
type: 'n8n-nodes-base.googleSheets',
typeVersion: 4,
position: [1120, 380],
credentials: {},
disabled: false,
},
parameter: {
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
default: {
mappingMode: 'defineBelow',
value: {},
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'upsert',
addAllFields: true,
noFieldsError: 'No columns found in sheet.',
multiKeyMatch: false,
fieldWords: {
singular: 'column',
plural: 'columns',
},
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['appendOrUpdate'],
'@version': [4],
},
hide: {
sheetName: [''],
},
},
},
},
};
export const MAPPING_COLUMNS_RESPONSE: ResourceMapperFields = {
fields: [
{
id: 'First name',
displayName: 'First name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Last name',
displayName: 'Last name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Username',
displayName: 'Username',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Address',
displayName: 'Address',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'id',
displayName: 'id',
match: true,
required: true,
defaultMatch: true,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Last Name',
displayName: 'Last Name',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
],
};

View file

@ -11,3 +11,4 @@ export * from './useTelemetry';
export * from './useTitleChange'; export * from './useTitleChange';
export * from './useToast'; export * from './useToast';
export * from './useUpgradeLink'; export * from './useUpgradeLink';
export * from './useNodeSpecificationValues';

View file

@ -0,0 +1,35 @@
import type { INodePropertyTypeOptions, ResourceMapperTypeOptions } from 'n8n-workflow';
import { computed } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
export function useNodeSpecificationValues(typeOptions: INodePropertyTypeOptions | undefined) {
const resourceMapperTypeOptions = computed<ResourceMapperTypeOptions | undefined>(() => {
return typeOptions?.resourceMapper;
});
const singularFieldWord = computed<string>(() => {
const singularFieldWord =
resourceMapperTypeOptions.value?.fieldWords?.singular || locale.baseText('generic.field');
return singularFieldWord;
});
const singularFieldWordCapitalized = computed<string>(() => {
return singularFieldWord.value.charAt(0).toUpperCase() + singularFieldWord.value.slice(1);
});
const pluralFieldWord = computed<string>(() => {
return resourceMapperTypeOptions.value?.fieldWords?.plural || locale.baseText('generic.fields');
});
const pluralFieldWordCapitalized = computed<string>(() => {
return pluralFieldWord.value.charAt(0).toUpperCase() + pluralFieldWord.value.slice(1);
});
return {
resourceMapperTypeOptions,
singularFieldWord,
singularFieldWordCapitalized,
pluralFieldWord,
pluralFieldWordCapitalized,
};
}

View file

@ -21,6 +21,8 @@
"generic.confirm": "Confirm", "generic.confirm": "Confirm",
"generic.deleteWorkflowError": "Problem deleting workflow", "generic.deleteWorkflowError": "Problem deleting workflow",
"generic.filtersApplied": "Filters are currently applied.", "generic.filtersApplied": "Filters are currently applied.",
"generic.field": "field",
"generic.fields": "fields",
"generic.learnMore": "Learn more", "generic.learnMore": "Learn more",
"generic.reset": "Reset", "generic.reset": "Reset",
"generic.resetAllFilters": "Reset all filters", "generic.resetAllFilters": "Reset all filters",
@ -35,7 +37,9 @@
"generic.beta": "beta", "generic.beta": "beta",
"generic.yes": "Yes", "generic.yes": "Yes",
"generic.no": "No", "generic.no": "No",
"generic.retry": "Retry",
"generic.settings": "Settings", "generic.settings": "Settings",
"generic.service": "the service",
"generic.unsavedWork.confirmMessage.headline": "Save changes before leaving?", "generic.unsavedWork.confirmMessage.headline": "Save changes before leaving?",
"generic.unsavedWork.confirmMessage.message": "If you don't save, you will lose your changes.", "generic.unsavedWork.confirmMessage.message": "If you don't save, you will lose your changes.",
"generic.unsavedWork.confirmMessage.confirmButtonText": "Save", "generic.unsavedWork.confirmMessage.confirmButtonText": "Save",
@ -1054,6 +1058,31 @@
"resourceLocator.openResource": "Open in {appName}", "resourceLocator.openResource": "Open in {appName}",
"resourceLocator.search.placeholder": "Search...", "resourceLocator.search.placeholder": "Search...",
"resourceLocator.url.placeholder": "Enter URL...", "resourceLocator.url.placeholder": "Enter URL...",
"resourceMapper.autoMappingNotice": "In this mode, make sure the incoming data fields are named the same as the {fieldWord} in {serviceName}. (Use a 'set' node before this node to change them if required.)",
"resourceMapper.mappingMode.label": "Mapping Column Mode",
"resourceMapper.mappingMode.defineBelow.name": "Map Each Column Manually",
"resourceMapper.mappingMode.defineBelow.description": "Set the value for each {fieldWord}",
"resourceMapper.mappingMode.autoMapInputData.name": "Map Automatically",
"resourceMapper.mappingMode.autoMapInputData.description": "Look for incoming data that matches the {fieldWord} in {serviceName}",
"resourceMapper.fetchingFields.message": "Fetching {fieldWord}",
"resourceMapper.fetchingFields.errorMessage": "Can't get {fieldWord}.",
"resourceMapper.fetchingFields.noFieldsFound": "No {fieldWord} found in {serviceName}.",
"resourceMapper.columnsToMatchOn.label": "{fieldWord} to Match On",
"resourceMapper.columnsToMatchOn.multi.description": "The {fieldWord} that identify the row(s) to modify",
"resourceMapper.columnsToMatchOn.single.description": "The {fieldWord} that identifies the row(s) to modify",
"resourceMapper.columnsToMatchOn.tooltip": "The {fieldWord} to compare when finding the rows to update",
"resourceMapper.columnsToMatchOn.noFieldsFound": "No {fieldWord} that can be used for matching found in {serviceName}.",
"resourceMapper.valuesToSend.label": "Values to Send",
"resourceMapper.valuesToUpdate.label": "Values to Update",
"resourceMapper.usingToMatch": "(using to match)",
"resourceMapper.usingToMatch.description": "This {fieldWord} won't be updated and can't be removed, as it's used for matching",
"resourceMapper.removeField": "Remove {fieldWord}",
"resourceMapper.mandatoryField.title": "This {fieldWord} is mandatory and cant be removed",
"resourceMapper.addFieldToSend": "Add {fieldWord} to Send",
"resourceMapper.matching.title": "This {fieldWord} is used for matching and cant be removed",
"resourceMapper.addAllFields": "Add All {fieldWord}",
"resourceMapper.removeAllFields": "Remove All {fieldWord}",
"resourceMapper.refreshFieldList": "Refresh {fieldWord} List",
"runData.emptyItemHint": "This is an item, but it's empty.", "runData.emptyItemHint": "This is an item, but it's empty.",
"runData.emptyArray": "[empty array]", "runData.emptyArray": "[empty array]",
"runData.emptyString": "[empty]", "runData.emptyString": "[empty]",

View file

@ -39,6 +39,7 @@ import {
faCut, faCut,
faDotCircle, faDotCircle,
faEdit, faEdit,
faEllipsisH,
faEllipsisV, faEllipsisV,
faEnvelope, faEnvelope,
faEye, faEye,
@ -176,6 +177,7 @@ addIcon(faCut);
addIcon(faDotCircle); addIcon(faDotCircle);
addIcon(faGripVertical); addIcon(faGripVertical);
addIcon(faEdit); addIcon(faEdit);
addIcon(faEllipsisH);
addIcon(faEllipsisV); addIcon(faEllipsisV);
addIcon(faEnvelope); addIcon(faEnvelope);
addIcon(faEye); addIcon(faEye);

View file

@ -6,7 +6,7 @@ import type {
NodePanelType, NodePanelType,
XYPosition, XYPosition,
} from '@/Interface'; } from '@/Interface';
import type { IRunData } from 'n8n-workflow'; import type { INodeIssues, IRunData } from 'n8n-workflow';
import { defineStore } from 'pinia'; import { defineStore } from 'pinia';
import Vue from 'vue'; import Vue from 'vue';
import { useWorkflowsStore } from './workflows.store'; import { useWorkflowsStore } from './workflows.store';
@ -206,5 +206,14 @@ export const useNDVStore = defineStore(STORES.NDV, {
window.localStorage.setItem(LOCAL_STORAGE_MAPPING_IS_ONBOARDED, 'true'); window.localStorage.setItem(LOCAL_STORAGE_MAPPING_IS_ONBOARDED, 'true');
} }
}, },
updateNodeParameterIssues(issues: INodeIssues): void {
const activeNode = this.activeNode;
if (activeNode) {
Vue.set(activeNode, 'issues', {
...activeNode.issues,
...issues,
});
}
},
}, },
}); });

View file

@ -4,9 +4,14 @@ import {
getNodeTranslationHeaders, getNodeTranslationHeaders,
getNodeTypes, getNodeTypes,
getResourceLocatorResults, getResourceLocatorResults,
getResourceMapperFields,
} from '@/api/nodeTypes'; } from '@/api/nodeTypes';
import { DEFAULT_NODETYPE_VERSION, STORES } from '@/constants'; import { DEFAULT_NODETYPE_VERSION, STORES } from '@/constants';
import type { INodeTypesState, IResourceLocatorReqParams } from '@/Interface'; import type {
INodeTypesState,
IResourceLocatorReqParams,
ResourceMapperReqParams,
} from '@/Interface';
import { addHeaders, addNodeTranslation } from '@/plugins/i18n'; import { addHeaders, addNodeTranslation } from '@/plugins/i18n';
import { omit } from '@/utils'; import { omit } from '@/utils';
import type { import type {
@ -17,6 +22,7 @@ import type {
INodePropertyOptions, INodePropertyOptions,
INodeTypeDescription, INodeTypeDescription,
INodeTypeNameVersion, INodeTypeNameVersion,
ResourceMapperFields,
} from 'n8n-workflow'; } from 'n8n-workflow';
import { defineStore } from 'pinia'; import { defineStore } from 'pinia';
import Vue from 'vue'; import Vue from 'vue';
@ -177,5 +183,15 @@ export const useNodeTypesStore = defineStore(STORES.NODE_TYPES, {
const rootStore = useRootStore(); const rootStore = useRootStore();
return getResourceLocatorResults(rootStore.getRestApiContext, sendData); return getResourceLocatorResults(rootStore.getRestApiContext, sendData);
}, },
async getResourceMapperFields(
sendData: ResourceMapperReqParams,
): Promise<ResourceMapperFields | null> {
const rootStore = useRootStore();
try {
return await getResourceMapperFields(rootStore.getRestApiContext, sendData);
} catch (error) {
return null;
}
},
}, },
}); });

View file

@ -22,6 +22,7 @@ import type {
NodeParameterValueType, NodeParameterValueType,
INodePropertyOptions, INodePropertyOptions,
INodePropertyCollection, INodePropertyCollection,
ResourceMapperField,
} from 'n8n-workflow'; } from 'n8n-workflow';
import { isResourceLocatorValue, isJsonKeyObject } from '@/utils'; import { isResourceLocatorValue, isJsonKeyObject } from '@/utils';
import { useCredentialsStore } from '@/stores/credentials.store'; import { useCredentialsStore } from '@/stores/credentials.store';
@ -35,6 +36,7 @@ import { i18n as locale } from '@/plugins/i18n';
const CRED_KEYWORDS_TO_FILTER = ['API', 'OAuth1', 'OAuth2']; const CRED_KEYWORDS_TO_FILTER = ['API', 'OAuth1', 'OAuth2'];
const NODE_KEYWORDS_TO_FILTER = ['Trigger']; const NODE_KEYWORDS_TO_FILTER = ['Trigger'];
const COMMUNITY_PACKAGE_NAME_REGEX = /(@\w+\/)?n8n-nodes-(?!base\b)\b\w+/g; const COMMUNITY_PACKAGE_NAME_REGEX = /(@\w+\/)?n8n-nodes-(?!base\b)\b\w+/g;
const RESOURCE_MAPPER_FIELD_NAME_REGEX = /value\[\"(.+)\"\]/;
export function getAppNameFromCredType(name: string) { export function getAppNameFromCredType(name: string) {
return name return name
@ -400,3 +402,35 @@ export const isNodeParameterRequired = (
}); });
return true; return true;
}; };
export const parseResourceMapperFieldName = (fullName: string) => {
const match = fullName.match(RESOURCE_MAPPER_FIELD_NAME_REGEX);
const fieldName = match ? match.pop() : fullName;
return fieldName;
};
export const fieldCannotBeDeleted = (
field: INodeProperties | ResourceMapperField,
showMatchingColumnsSelector: boolean,
resourceMapperMode = '',
matchingFields: string[] = [],
): boolean => {
const fieldIdentifier = 'id' in field ? field.id : field.name;
return (
(resourceMapperMode === 'add' && field.required === true) ||
isMatchingField(fieldIdentifier, matchingFields, showMatchingColumnsSelector)
);
};
export const isMatchingField = (
field: string,
matchingFields: string[],
showMatchingColumnsSelector: boolean,
): boolean => {
const fieldName = parseResourceMapperFieldName(field);
if (fieldName) {
return showMatchingColumnsSelector && (matchingFields || []).includes(fieldName);
}
return false;
};

View file

@ -35,3 +35,7 @@ export function isNumber(value: unknown): value is number {
export const isCredentialModalState = (value: unknown): value is NewCredentialsModal => { export const isCredentialModalState = (value: unknown): value is NewCredentialsModal => {
return typeof value === 'object' && value !== null && 'showAuthSelector' in value; return typeof value === 'object' && value !== null && 'showAuthSelector' in value;
}; };
export const isResourceMapperValue = (value: unknown): value is string | number | boolean => {
return ['string', 'number', 'boolean'].includes(typeof value);
};

View file

@ -11,7 +11,7 @@ export class GoogleSheets extends VersionedNodeType {
name: 'googleSheets', name: 'googleSheets',
icon: 'file:googleSheets.svg', icon: 'file:googleSheets.svg',
group: ['input', 'output'], group: ['input', 'output'],
defaultVersion: 3, defaultVersion: 4,
subtitle: '={{$parameter["operation"] + ": " + $parameter["resource"]}}', subtitle: '={{$parameter["operation"] + ": " + $parameter["resource"]}}',
description: 'Read, update and write data to Google Sheets', description: 'Read, update and write data to Google Sheets',
}; };
@ -20,6 +20,7 @@ export class GoogleSheets extends VersionedNodeType {
1: new GoogleSheetsV1(baseDescription), 1: new GoogleSheetsV1(baseDescription),
2: new GoogleSheetsV1(baseDescription), 2: new GoogleSheetsV1(baseDescription),
3: new GoogleSheetsV2(baseDescription), 3: new GoogleSheetsV2(baseDescription),
4: new GoogleSheetsV2(baseDescription),
}; };
super(nodeVersions, baseDescription); super(nodeVersions, baseDescription);

View file

@ -5,7 +5,7 @@ import type {
INodeTypeDescription, INodeTypeDescription,
} from 'n8n-workflow'; } from 'n8n-workflow';
import { versionDescription } from './actions/versionDescription'; import { versionDescription } from './actions/versionDescription';
import { credentialTest, listSearch, loadOptions } from './methods'; import { credentialTest, listSearch, loadOptions, resourceMapping } from './methods';
import { router } from './actions/router'; import { router } from './actions/router';
export class GoogleSheetsV2 implements INodeType { export class GoogleSheetsV2 implements INodeType {
@ -22,6 +22,7 @@ export class GoogleSheetsV2 implements INodeType {
loadOptions, loadOptions,
credentialTest, credentialTest,
listSearch, listSearch,
resourceMapping,
}; };
async execute(this: IExecuteFunctions) { async execute(this: IExecuteFunctions) {

View file

@ -30,6 +30,7 @@ export const description: SheetProperties = [
show: { show: {
resource: ['sheet'], resource: ['sheet'],
operation: ['append'], operation: ['append'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -48,6 +49,7 @@ export const description: SheetProperties = [
show: { show: {
operation: ['append'], operation: ['append'],
dataMode: ['autoMapInputData'], dataMode: ['autoMapInputData'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -68,6 +70,7 @@ export const description: SheetProperties = [
resource: ['sheet'], resource: ['sheet'],
operation: ['append'], operation: ['append'],
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -101,6 +104,40 @@ export const description: SheetProperties = [
}, },
], ],
}, },
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'add',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: false,
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['append'],
'@version': [4],
},
hide: {
...untilSheetSelected,
},
},
},
{ {
displayName: 'Options', displayName: 'Options',
name: 'options', name: 'options',
@ -156,7 +193,11 @@ export async function execute(
sheetId: string, sheetId: string,
): Promise<INodeExecutionData[]> { ): Promise<INodeExecutionData[]> {
const items = this.getInputData(); const items = this.getInputData();
const dataMode = this.getNodeParameter('dataMode', 0) as string; const nodeVersion = this.getNode().typeVersion;
const dataMode =
nodeVersion < 4
? (this.getNodeParameter('dataMode', 0) as string)
: (this.getNodeParameter('columns.mappingMode', 0) as string);
if (!items.length || dataMode === 'nothing') return []; if (!items.length || dataMode === 'nothing') return [];
@ -190,5 +231,9 @@ export async function execute(
false, false,
); );
return items; if (nodeVersion < 4 || dataMode === 'autoMapInputData') {
return items;
} else {
return this.helpers.returnJsonArray(setData);
}
} }

View file

@ -36,6 +36,7 @@ export const description: SheetProperties = [
show: { show: {
resource: ['sheet'], resource: ['sheet'],
operation: ['appendOrUpdate'], operation: ['appendOrUpdate'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -61,6 +62,7 @@ export const description: SheetProperties = [
show: { show: {
resource: ['sheet'], resource: ['sheet'],
operation: ['appendOrUpdate'], operation: ['appendOrUpdate'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -77,6 +79,7 @@ export const description: SheetProperties = [
resource: ['sheet'], resource: ['sheet'],
operation: ['appendOrUpdate'], operation: ['appendOrUpdate'],
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -96,6 +99,7 @@ export const description: SheetProperties = [
resource: ['sheet'], resource: ['sheet'],
operation: ['appendOrUpdate'], operation: ['appendOrUpdate'],
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -141,6 +145,40 @@ export const description: SheetProperties = [
}, },
], ],
}, },
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'upsert',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: false,
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['appendOrUpdate'],
'@version': [4],
},
hide: {
...untilSheetSelected,
},
},
},
{ {
displayName: 'Options', displayName: 'Options',
name: 'options', name: 'options',
@ -200,10 +238,21 @@ export async function execute(
} }
columnNames = sheetData[headerRow]; columnNames = sheetData[headerRow];
const nodeVersion = this.getNode().typeVersion;
const newColumns = new Set<string>(); const newColumns = new Set<string>();
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', 0) as string; const columnsToMatchOn: string[] =
const keyIndex = columnNames.indexOf(columnToMatchOn); nodeVersion < 4
? [this.getNodeParameter('columnToMatchOn', 0) as string]
: (this.getNodeParameter('columns.matchingColumns', 0) as string[]);
const dataMode =
nodeVersion < 4
? (this.getNodeParameter('dataMode', 0) as string)
: (this.getNodeParameter('columns.mappingMode', 0) as string);
// TODO: Add support for multiple columns to match on in the next overhaul
const keyIndex = columnNames.indexOf(columnsToMatchOn[0]);
const columnValues = await sheet.getColumnValues( const columnValues = await sheet.getColumnValues(
range, range,
@ -216,12 +265,8 @@ export async function execute(
const updateData: ISheetUpdateData[] = []; const updateData: ISheetUpdateData[] = [];
const appendData: IDataObject[] = []; const appendData: IDataObject[] = [];
const mappedValues: IDataObject[] = [];
for (let i = 0; i < items.length; i++) { for (let i = 0; i < items.length; i++) {
const dataMode = this.getNodeParameter('dataMode', i) as
| 'defineBelow'
| 'autoMapInputData'
| 'nothing';
if (dataMode === 'nothing') continue; if (dataMode === 'nothing') continue;
const data: IDataObject[] = []; const data: IDataObject[] = [];
@ -251,33 +296,50 @@ export async function execute(
data.push(items[i].json); data.push(items[i].json);
} }
} else { } else {
const valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string; const valueToMatchOn =
nodeVersion < 4
? (this.getNodeParameter('valueToMatchOn', i) as string)
: (this.getNodeParameter(`columns.value[${columnsToMatchOn[0]}]`, i) as string);
const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[]; if (nodeVersion < 4) {
if (!valuesToSend?.length) { const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[];
throw new NodeOperationError( if (!valuesToSend?.length) {
this.getNode(), throw new NodeOperationError(
"At least one value has to be added under 'Values to Send'", this.getNode(),
); "At least one value has to be added under 'Values to Send'",
} );
const fields = valuesToSend.reduce((acc, entry) => {
if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
}
acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
} }
return acc; const fields = valuesToSend.reduce((acc, entry) => {
}, {} as IDataObject); if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
fields[columnToMatchOn] = valueToMatchOn; if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
data.push(fields); }
acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
}
return acc;
}, {} as IDataObject);
fields[columnsToMatchOn[0]] = valueToMatchOn;
data.push(fields);
} else {
const mappingValues = this.getNodeParameter('columns.value', i) as IDataObject;
if (Object.keys(mappingValues).length === 0) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
// Setting empty values to empty string so that they are not ignored by the API
Object.keys(mappingValues).forEach((key) => {
if (mappingValues[key] === undefined || mappingValues[key] === null) {
mappingValues[key] = '';
}
});
data.push(mappingValues);
mappedValues.push(mappingValues);
}
} }
if (newColumns.size) { if (newColumns.size) {
@ -291,7 +353,7 @@ export async function execute(
const preparedData = await sheet.prepareDataForUpdateOrUpsert( const preparedData = await sheet.prepareDataForUpdateOrUpsert(
data, data,
columnToMatchOn, columnsToMatchOn[0],
range, range,
headerRow, headerRow,
firstDataRow, firstDataRow,
@ -321,5 +383,10 @@ export async function execute(
lastRow, lastRow,
); );
} }
return items;
if (nodeVersion < 4 || dataMode === 'autoMapInputData') {
return items;
} else {
return this.helpers.returnJsonArray(mappedValues);
}
} }

View file

@ -36,6 +36,7 @@ export const description: SheetProperties = [
show: { show: {
resource: ['sheet'], resource: ['sheet'],
operation: ['update'], operation: ['update'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -61,6 +62,7 @@ export const description: SheetProperties = [
show: { show: {
resource: ['sheet'], resource: ['sheet'],
operation: ['update'], operation: ['update'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -77,6 +79,7 @@ export const description: SheetProperties = [
resource: ['sheet'], resource: ['sheet'],
operation: ['update'], operation: ['update'],
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -96,6 +99,7 @@ export const description: SheetProperties = [
resource: ['sheet'], resource: ['sheet'],
operation: ['update'], operation: ['update'],
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [3],
}, },
hide: { hide: {
...untilSheetSelected, ...untilSheetSelected,
@ -141,6 +145,40 @@ export const description: SheetProperties = [
}, },
], ],
}, },
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'update',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: false,
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['update'],
'@version': [4],
},
hide: {
...untilSheetSelected,
},
},
},
{ {
displayName: 'Options', displayName: 'Options',
name: 'options', name: 'options',
@ -175,6 +213,8 @@ export async function execute(
const locationDefineOptions = (options.locationDefine as IDataObject)?.values as IDataObject; const locationDefineOptions = (options.locationDefine as IDataObject)?.values as IDataObject;
const nodeVersion = this.getNode().typeVersion;
let headerRow = 0; let headerRow = 0;
let firstDataRow = 1; let firstDataRow = 1;
@ -201,8 +241,18 @@ export async function execute(
columnNames = sheetData[headerRow]; columnNames = sheetData[headerRow];
const newColumns = new Set<string>(); const newColumns = new Set<string>();
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', 0) as string; const columnsToMatchOn: string[] =
const keyIndex = columnNames.indexOf(columnToMatchOn); nodeVersion < 4
? [this.getNodeParameter('columnToMatchOn', 0) as string]
: (this.getNodeParameter('columns.matchingColumns', 0) as string[]);
const dataMode =
nodeVersion < 4
? (this.getNodeParameter('dataMode', 0) as string)
: (this.getNodeParameter('columns.mappingMode', 0) as string);
// TODO: Add support for multiple columns to match on in the next overhaul
const keyIndex = columnNames.indexOf(columnsToMatchOn[0]);
const columnValues = await sheet.getColumnValues( const columnValues = await sheet.getColumnValues(
range, range,
@ -214,12 +264,9 @@ export async function execute(
const updateData: ISheetUpdateData[] = []; const updateData: ISheetUpdateData[] = [];
for (let i = 0; i < items.length; i++) { const mappedValues: IDataObject[] = [];
const dataMode = this.getNodeParameter('dataMode', i) as
| 'defineBelow'
| 'autoMapInputData'
| 'nothing';
for (let i = 0; i < items.length; i++) {
if (dataMode === 'nothing') continue; if (dataMode === 'nothing') continue;
const data: IDataObject[] = []; const data: IDataObject[] = [];
@ -249,33 +296,54 @@ export async function execute(
data.push(items[i].json); data.push(items[i].json);
} }
} else { } else {
const valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string; const valueToMatchOn =
nodeVersion < 4
? (this.getNodeParameter('valueToMatchOn', i) as string)
: (this.getNodeParameter(`columns.value[${columnsToMatchOn[0]}]`, i) as string);
const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[]; if (nodeVersion < 4) {
if (!valuesToSend?.length) { const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[];
throw new NodeOperationError( if (!valuesToSend?.length) {
this.getNode(), throw new NodeOperationError(
"At least one value has to be added under 'Values to Send'", this.getNode(),
); "At least one value has to be added under 'Values to Send'",
} );
const fields = valuesToSend.reduce((acc, entry) => {
if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
}
acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
} }
return acc; const fields = valuesToSend.reduce((acc, entry) => {
}, {} as IDataObject); if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
fields[columnToMatchOn] = valueToMatchOn; if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
}
data.push(fields); acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
}
return acc;
}, {} as IDataObject);
fields[columnsToMatchOn[0]] = valueToMatchOn;
data.push(fields);
} else {
const mappingValues = this.getNodeParameter('columns.value', i) as IDataObject;
if (Object.keys(mappingValues).length === 0) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
// Setting empty values to empty string so that they are not ignored by the API
Object.keys(mappingValues).forEach((key) => {
if (mappingValues[key] === undefined || mappingValues[key] === null) {
mappingValues[key] = '';
}
});
data.push(mappingValues);
mappedValues.push(mappingValues);
}
} }
if (newColumns.size) { if (newColumns.size) {
@ -289,7 +357,7 @@ export async function execute(
const preparedData = await sheet.prepareDataForUpdateOrUpsert( const preparedData = await sheet.prepareDataForUpdateOrUpsert(
data, data,
columnToMatchOn, columnsToMatchOn[0],
range, range,
headerRow, headerRow,
firstDataRow, firstDataRow,
@ -306,5 +374,9 @@ export async function execute(
await sheet.batchUpdate(updateData, valueInputMode); await sheet.batchUpdate(updateData, valueInputMode);
} }
return items; if (nodeVersion < 4 || dataMode === 'autoMapInputData') {
return items;
} else {
return this.helpers.returnJsonArray(mappedValues);
}
} }

View file

@ -9,7 +9,7 @@ export const versionDescription: INodeTypeDescription = {
name: 'googleSheets', name: 'googleSheets',
icon: 'file:googleSheets.svg', icon: 'file:googleSheets.svg',
group: ['input', 'output'], group: ['input', 'output'],
version: 3, version: [3, 4],
subtitle: '={{$parameter["operation"] + ": " + $parameter["resource"]}}', subtitle: '={{$parameter["operation"] + ": " + $parameter["resource"]}}',
description: 'Read, update and write data to Google Sheets', description: 'Read, update and write data to Google Sheets',
defaults: { defaults: {

View file

@ -195,12 +195,24 @@ export function mapFields(this: IExecuteFunctions, inputSize: number) {
const returnData: IDataObject[] = []; const returnData: IDataObject[] = [];
for (let i = 0; i < inputSize; i++) { for (let i = 0; i < inputSize; i++) {
const fields = this.getNodeParameter('fieldsUi.fieldValues', i, []) as IDataObject[]; const nodeVersion = this.getNode().typeVersion;
let dataToSend: IDataObject = {}; if (nodeVersion < 4) {
for (const field of fields) { const fields = this.getNodeParameter('fieldsUi.fieldValues', i, []) as IDataObject[];
dataToSend = { ...dataToSend, [field.fieldId as string]: field.fieldValue }; let dataToSend: IDataObject = {};
for (const field of fields) {
dataToSend = { ...dataToSend, [field.fieldId as string]: field.fieldValue };
}
returnData.push(dataToSend);
} else {
const mappingValues = this.getNodeParameter('columns.value', i) as IDataObject;
if (Object.keys(mappingValues).length === 0) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
returnData.push(mappingValues);
} }
returnData.push(dataToSend);
} }
return returnData; return returnData;

View file

@ -1,3 +1,4 @@
export * as loadOptions from './loadOptions'; export * as loadOptions from './loadOptions';
export * as listSearch from './listSearch'; export * as listSearch from './listSearch';
export * as credentialTest from './credentialTest'; export * as credentialTest from './credentialTest';
export * as resourceMapping from './resourceMapping';

View file

@ -0,0 +1,39 @@
import type { ILoadOptionsFunctions } from 'n8n-core';
import type { IDataObject, ResourceMapperFields } from 'n8n-workflow';
import { GoogleSheet } from '../helpers/GoogleSheet';
import type { ResourceLocator } from '../helpers/GoogleSheets.types';
import { getSpreadsheetId } from '../helpers/GoogleSheets.utils';
export async function getMappingColumns(
this: ILoadOptionsFunctions,
): Promise<ResourceMapperFields> {
const { mode, value } = this.getNodeParameter('documentId', 0) as IDataObject;
const spreadsheetId = getSpreadsheetId(mode as ResourceLocator, value as string);
const sheet = new GoogleSheet(spreadsheetId, this);
let sheetWithinDocument = this.getNodeParameter('sheetName', undefined, {
extractValue: true,
}) as string;
if (sheetWithinDocument === 'gid=0') {
sheetWithinDocument = '0';
}
const sheetName = await sheet.spreadsheetGetSheetNameById(sheetWithinDocument);
const sheetData = await sheet.getData(`${sheetName}!1:1`, 'FORMATTED_VALUE');
const columns = sheet.testFilter(sheetData || [], 0, 0).filter((col) => col !== '');
const columnData: ResourceMapperFields = {
fields: columns.map((col) => ({
id: col,
displayName: col,
required: false,
defaultMatch: col === 'id',
display: true,
type: 'string',
canBeUsedToMatch: true,
})),
};
return columnData;
}

View file

@ -11,7 +11,7 @@ export class Postgres extends VersionedNodeType {
name: 'postgres', name: 'postgres',
icon: 'file:postgres.svg', icon: 'file:postgres.svg',
group: ['input'], group: ['input'],
defaultVersion: 2.1, defaultVersion: 2.2,
description: 'Get, add and update data in Postgres', description: 'Get, add and update data in Postgres',
}; };
@ -19,6 +19,7 @@ export class Postgres extends VersionedNodeType {
1: new PostgresV1(baseDescription), 1: new PostgresV1(baseDescription),
2: new PostgresV2(baseDescription), 2: new PostgresV2(baseDescription),
2.1: new PostgresV2(baseDescription), 2.1: new PostgresV2(baseDescription),
2.2: new PostgresV2(baseDescription),
}; };
super(nodeVersions, baseDescription); super(nodeVersions, baseDescription);

View file

@ -9,7 +9,7 @@ import type {
import { router } from './actions/router'; import { router } from './actions/router';
import { versionDescription } from './actions/versionDescription'; import { versionDescription } from './actions/versionDescription';
import { credentialTest, listSearch, loadOptions } from './methods'; import { credentialTest, listSearch, loadOptions, resourceMapping } from './methods';
export class PostgresV2 implements INodeType { export class PostgresV2 implements INodeType {
description: INodeTypeDescription; description: INodeTypeDescription;
@ -21,7 +21,7 @@ export class PostgresV2 implements INodeType {
}; };
} }
methods = { credentialTest, listSearch, loadOptions }; methods = { credentialTest, listSearch, loadOptions, resourceMapping };
async execute(this: IExecuteFunctions): Promise<INodeExecutionData[][]> { async execute(this: IExecuteFunctions): Promise<INodeExecutionData[][]> {
return router.call(this); return router.call(this);

View file

@ -40,6 +40,11 @@ const properties: INodeProperties[] = [
default: 'autoMapInputData', default: 'autoMapInputData',
description: description:
'Whether to map node input properties and the table data automatically or manually', 'Whether to map node input properties and the table data automatically or manually',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
}, },
{ {
displayName: ` displayName: `
@ -51,6 +56,7 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['autoMapInputData'], dataMode: ['autoMapInputData'],
'@version': [2, 2.1],
}, },
}, },
}, },
@ -66,6 +72,7 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [2, 2.1],
}, },
}, },
default: {}, default: {},
@ -97,6 +104,35 @@ const properties: INodeProperties[] = [
}, },
], ],
}, },
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
default: {
mappingMode: 'defineBelow',
value: null,
},
noDataExpression: true,
required: true,
typeOptions: {
loadOptionsDependsOn: ['table.value', 'operation'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'add',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: true,
},
},
displayOptions: {
show: {
'@version': [2.2],
},
},
},
optionsCollection, optionsCollection,
]; ];
@ -142,7 +178,11 @@ export async function execute(
let query = `INSERT INTO $1:name.$2:name($3:name) VALUES($3:csv)${onConflict}`; let query = `INSERT INTO $1:name.$2:name($3:name) VALUES($3:csv)${onConflict}`;
let values: QueryValues = [schema, table]; let values: QueryValues = [schema, table];
const dataMode = this.getNodeParameter('dataMode', i) as string; const nodeVersion = this.getNode().typeVersion;
const dataMode =
nodeVersion < 2.2
? (this.getNodeParameter('dataMode', i) as string)
: (this.getNodeParameter('columns.mappingMode', i) as string);
let item: IDataObject = {}; let item: IDataObject = {};
@ -151,10 +191,17 @@ export async function execute(
} }
if (dataMode === 'defineBelow') { if (dataMode === 'defineBelow') {
const valuesToSend = (this.getNodeParameter('valuesToSend', i, []) as IDataObject) const valuesToSend =
.values as IDataObject[]; nodeVersion < 2.2
? ((this.getNodeParameter('valuesToSend', i, []) as IDataObject).values as IDataObject[])
: ((this.getNodeParameter('columns.values', i, []) as IDataObject)
.values as IDataObject[]);
item = prepareItem(valuesToSend); if (nodeVersion < 2.2) {
item = prepareItem(valuesToSend);
} else {
item = this.getNodeParameter('columns.value', i) as IDataObject;
}
} }
const tableSchema = await getTableSchema(db, schema, table); const tableSchema = await getTableSchema(db, schema, table);

View file

@ -14,6 +14,7 @@ import type {
import { import {
addReturning, addReturning,
checkItemAgainstSchema, checkItemAgainstSchema,
doesRowExist,
getTableSchema, getTableSchema,
prepareItem, prepareItem,
replaceEmptyStringsByNulls, replaceEmptyStringsByNulls,
@ -41,6 +42,11 @@ const properties: INodeProperties[] = [
default: 'autoMapInputData', default: 'autoMapInputData',
description: description:
'Whether to map node input properties and the table data automatically or manually', 'Whether to map node input properties and the table data automatically or manually',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
}, },
{ {
displayName: ` displayName: `
@ -52,6 +58,7 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['autoMapInputData'], dataMode: ['autoMapInputData'],
'@version': [2],
}, },
}, },
}, },
@ -69,6 +76,11 @@ const properties: INodeProperties[] = [
}, },
default: '', default: '',
hint: 'The column that identifies the row(s) to modify', hint: 'The column that identifies the row(s) to modify',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
}, },
{ {
displayName: 'Value of Column to Match On', displayName: 'Value of Column to Match On',
@ -80,6 +92,7 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [2, 2.1],
}, },
}, },
}, },
@ -95,6 +108,7 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [2, 2.1],
}, },
}, },
default: {}, default: {},
@ -126,6 +140,35 @@ const properties: INodeProperties[] = [
}, },
], ],
}, },
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['table.value', 'operation'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'update',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: true,
},
},
displayOptions: {
show: {
'@version': [2.2],
},
},
},
optionsCollection, optionsCollection,
]; ];
@ -161,32 +204,80 @@ export async function execute(
extractValue: true, extractValue: true,
}) as string; }) as string;
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', i) as string; const nodeVersion = this.getNode().typeVersion;
const columnsToMatchOn: string[] =
nodeVersion < 2.2
? [this.getNodeParameter('columnToMatchOn', i) as string]
: (this.getNodeParameter('columns.matchingColumns', i) as string[]);
const dataMode = this.getNodeParameter('dataMode', i) as string; const dataMode =
nodeVersion < 2.2
? (this.getNodeParameter('dataMode', i) as string)
: (this.getNodeParameter('columns.mappingMode', i) as string);
let item: IDataObject = {}; let item: IDataObject = {};
let valueToMatchOn: string | IDataObject = ''; let valueToMatchOn: string | IDataObject = '';
if (nodeVersion < 2.2) {
if (dataMode === 'autoMapInputData') {
item = items[i].json;
valueToMatchOn = item[columnToMatchOn] as string;
}
if (dataMode === 'defineBelow') {
const valuesToSend = (this.getNodeParameter('valuesToSend', i, []) as IDataObject)
.values as IDataObject[];
item = prepareItem(valuesToSend);
valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string; valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string;
} }
if (!item[columnToMatchOn] && dataMode === 'autoMapInputData') { if (dataMode === 'autoMapInputData') {
throw new NodeOperationError( item = items[i].json;
this.getNode(), if (nodeVersion < 2.2) {
"Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.", valueToMatchOn = item[columnsToMatchOn[0]] as string;
); }
}
if (dataMode === 'defineBelow') {
const valuesToSend =
nodeVersion < 2.2
? ((this.getNodeParameter('valuesToSend', i, []) as IDataObject).values as IDataObject[])
: ((this.getNodeParameter('columns.values', i, []) as IDataObject)
.values as IDataObject[]);
if (nodeVersion < 2.2) {
item = prepareItem(valuesToSend);
item[columnsToMatchOn[0]] = this.getNodeParameter('valueToMatchOn', i) as string;
} else {
item = this.getNodeParameter('columns.value', i) as IDataObject;
}
}
const matchValues: string[] = [];
if (nodeVersion < 2.2) {
if (!item[columnsToMatchOn[0]] && dataMode === 'autoMapInputData') {
throw new NodeOperationError(
this.getNode(),
"Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.",
);
}
matchValues.push(valueToMatchOn);
matchValues.push(columnsToMatchOn[0]);
} else {
columnsToMatchOn.forEach((column) => {
matchValues.push(column);
matchValues.push(item[column] as string);
});
const rowExists = await doesRowExist(db, schema, table, matchValues);
if (!rowExists) {
const descriptionValues: string[] = [];
matchValues.forEach((val, index) => {
if (index % 2 === 0) {
descriptionValues.push(`${matchValues[index]}=${matchValues[index + 1]}`);
}
});
throw new NodeOperationError(
this.getNode(),
"The row you are trying to update doesn't exist",
{
description: `No rows matching the provided values (${descriptionValues.join(
', ',
)}) were found in the table "${table}".`,
itemIndex: i,
},
);
}
} }
const tableSchema = await getTableSchema(db, schema, table); const tableSchema = await getTableSchema(db, schema, table);
@ -197,11 +288,22 @@ export async function execute(
let valuesLength = values.length + 1; let valuesLength = values.length + 1;
const condition = `$${valuesLength}:name = $${valuesLength + 1}`; let condition = '';
valuesLength = valuesLength + 2; if (nodeVersion < 2.2) {
values.push(columnToMatchOn, valueToMatchOn); condition = `$${valuesLength}:name = $${valuesLength + 1}`;
valuesLength = valuesLength + 2;
values.push(columnsToMatchOn[0], valueToMatchOn);
} else {
const conditions: string[] = [];
for (const column of columnsToMatchOn) {
conditions.push(`$${valuesLength}:name = $${valuesLength + 1}`);
valuesLength = valuesLength + 2;
values.push(column, item[column] as string);
}
condition = conditions.join(' AND ');
}
const updateColumns = Object.keys(item).filter((column) => column !== columnToMatchOn); const updateColumns = Object.keys(item).filter((column) => !columnsToMatchOn.includes(column));
if (!Object.keys(updateColumns).length) { if (!Object.keys(updateColumns).length) {
throw new NodeOperationError( throw new NodeOperationError(
@ -227,5 +329,6 @@ export async function execute(
queries.push({ query, values }); queries.push({ query, values });
} }
return runQueries(queries, items, nodeOptions); const results = await runQueries(queries, items, nodeOptions);
return results;
} }

View file

@ -41,6 +41,11 @@ const properties: INodeProperties[] = [
default: 'autoMapInputData', default: 'autoMapInputData',
description: description:
'Whether to map node input properties and the table data automatically or manually', 'Whether to map node input properties and the table data automatically or manually',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
}, },
{ {
displayName: ` displayName: `
@ -52,12 +57,13 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['autoMapInputData'], dataMode: ['autoMapInputData'],
'@version': [2, 2.1],
}, },
}, },
}, },
{ {
// eslint-disable-next-line n8n-nodes-base/node-param-display-name-miscased, n8n-nodes-base/node-param-display-name-wrong-for-dynamic-options // eslint-disable-next-line n8n-nodes-base/node-param-display-name-miscased, n8n-nodes-base/node-param-display-name-wrong-for-dynamic-options
displayName: 'Column to Match On', displayName: 'Unique Column',
name: 'columnToMatchOn', name: 'columnToMatchOn',
type: 'options', type: 'options',
required: true, required: true,
@ -69,9 +75,14 @@ const properties: INodeProperties[] = [
}, },
default: '', default: '',
hint: "Used to find the correct row(s) to update. Doesn't get changed. Has to be unique.", hint: "Used to find the correct row(s) to update. Doesn't get changed. Has to be unique.",
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
}, },
{ {
displayName: 'Value of Column to Match On', displayName: 'Value of Unique Column',
name: 'valueToMatchOn', name: 'valueToMatchOn',
type: 'string', type: 'string',
default: '', default: '',
@ -80,6 +91,7 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [2, 2.1],
}, },
}, },
}, },
@ -95,6 +107,7 @@ const properties: INodeProperties[] = [
displayOptions: { displayOptions: {
show: { show: {
dataMode: ['defineBelow'], dataMode: ['defineBelow'],
'@version': [2, 2.1],
}, },
}, },
default: {}, default: {},
@ -126,6 +139,35 @@ const properties: INodeProperties[] = [
}, },
], ],
}, },
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['table.value', 'operation'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'upsert',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: true,
},
},
displayOptions: {
show: {
'@version': [2.2],
},
},
},
optionsCollection, optionsCollection,
]; ];
@ -161,9 +203,16 @@ export async function execute(
extractValue: true, extractValue: true,
}) as string; }) as string;
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', i) as string; const nodeVersion = this.getNode().typeVersion;
const columnsToMatchOn: string[] =
nodeVersion < 2.2
? [this.getNodeParameter('columnToMatchOn', i) as string]
: (this.getNodeParameter('columns.matchingColumns', i) as string[]);
const dataMode = this.getNodeParameter('dataMode', i) as string; const dataMode =
nodeVersion < 2.2
? (this.getNodeParameter('dataMode', i) as string)
: (this.getNodeParameter('columns.mappingMode', i) as string);
let item: IDataObject = {}; let item: IDataObject = {};
@ -172,22 +221,28 @@ export async function execute(
} }
if (dataMode === 'defineBelow') { if (dataMode === 'defineBelow') {
const valuesToSend = (this.getNodeParameter('valuesToSend', i, []) as IDataObject) const valuesToSend =
.values as IDataObject[]; nodeVersion < 2.2
? ((this.getNodeParameter('valuesToSend', i, []) as IDataObject).values as IDataObject[])
: ((this.getNodeParameter('columns.values', i, []) as IDataObject)
.values as IDataObject[]);
item = prepareItem(valuesToSend); if (nodeVersion < 2.2) {
item = prepareItem(valuesToSend);
item[columnToMatchOn] = this.getNodeParameter('valueToMatchOn', i) as string; item[columnsToMatchOn[0]] = this.getNodeParameter('valueToMatchOn', i) as string;
} else {
item = this.getNodeParameter('columns.value', i) as IDataObject;
}
} }
if (!item[columnToMatchOn]) { if (!item[columnsToMatchOn[0]]) {
throw new NodeOperationError( throw new NodeOperationError(
this.getNode(), this.getNode(),
"Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.", "Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.",
); );
} }
if (item[columnToMatchOn] && Object.keys(item).length === 1) { if (item[columnsToMatchOn[0]] && Object.keys(item).length === 1) {
throw new NodeOperationError( throw new NodeOperationError(
this.getNode(), this.getNode(),
"Add values to update or insert to the input item or set the 'Data Mode' to 'Define Below' to define the values to insert or update.", "Add values to update or insert to the input item or set the 'Data Mode' to 'Define Below' to define the values to insert or update.",
@ -201,16 +256,19 @@ export async function execute(
let values: QueryValues = [schema, table]; let values: QueryValues = [schema, table];
let valuesLength = values.length + 1; let valuesLength = values.length + 1;
const onConflict = ` ON CONFLICT ($${valuesLength}:name) DO UPDATE `; const conflictColumns: string[] = [];
valuesLength = valuesLength + 1; columnsToMatchOn.forEach((column) => {
values.push(columnToMatchOn); conflictColumns.push(`$${valuesLength}:name`);
valuesLength = valuesLength + 1;
values.push(column);
});
const onConflict = ` ON CONFLICT (${conflictColumns.join(',')}) DO UPDATE `;
const insertQuery = `INSERT INTO $1:name.$2:name($${valuesLength}:name) VALUES($${valuesLength}:csv)${onConflict}`; const insertQuery = `INSERT INTO $1:name.$2:name($${valuesLength}:name) VALUES($${valuesLength}:csv)${onConflict}`;
valuesLength = valuesLength + 1; valuesLength = valuesLength + 1;
values.push(item); values.push(item);
const updateColumns = Object.keys(item).filter((column) => column !== columnToMatchOn); const updateColumns = Object.keys(item).filter((column) => !columnsToMatchOn.includes(column));
const updates: string[] = []; const updates: string[] = [];
for (const column of updateColumns) { for (const column of updateColumns) {

View file

@ -8,7 +8,7 @@ export const versionDescription: INodeTypeDescription = {
name: 'postgres', name: 'postgres',
icon: 'file:postgres.svg', icon: 'file:postgres.svg',
group: ['input'], group: ['input'],
version: [2, 2.1], version: [2, 2.1, 2.2],
subtitle: '={{ $parameter["operation"] }}', subtitle: '={{ $parameter["operation"] }}',
description: 'Get, add and update data in Postgres', description: 'Get, add and update data in Postgres',
defaults: { defaults: {

View file

@ -16,7 +16,17 @@ export type QueryWithValues = { query: string; values?: QueryValues };
export type WhereClause = { column: string; condition: string; value: string | number }; export type WhereClause = { column: string; condition: string; value: string | number };
export type SortRule = { column: string; direction: string }; export type SortRule = { column: string; direction: string };
export type ColumnInfo = { column_name: string; data_type: string; is_nullable: string }; export type ColumnInfo = {
column_name: string;
data_type: string;
is_nullable: string;
udt_name: string;
column_default?: string;
};
export type EnumInfo = {
typname: string;
enumlabel: string;
};
export type PgpClient = pgPromise.IMain<{}, pg.IClient>; export type PgpClient = pgPromise.IMain<{}, pg.IClient>;
export type PgpDatabase = pgPromise.IDatabase<{}, pg.IClient>; export type PgpDatabase = pgPromise.IDatabase<{}, pg.IClient>;

View file

@ -1,9 +1,10 @@
import type { IDataObject, INode, INodeExecutionData } from 'n8n-workflow'; import type { IDataObject, INode, INodeExecutionData, INodePropertyOptions } from 'n8n-workflow';
import { NodeOperationError } from 'n8n-workflow'; import { NodeOperationError } from 'n8n-workflow';
import type { import type {
ColumnInfo, ColumnInfo,
ConstructExecutionMetaData, ConstructExecutionMetaData,
EnumInfo,
PgpClient, PgpClient,
PgpDatabase, PgpDatabase,
QueryMode, QueryMode,
@ -13,6 +14,8 @@ import type {
WhereClause, WhereClause,
} from './interfaces'; } from './interfaces';
const ENUM_VALUES_REGEX = /\{(.+?)\}/gm;
export function wrapData(data: IDataObject | IDataObject[]): INodeExecutionData[] { export function wrapData(data: IDataObject | IDataObject[]): INodeExecutionData[] {
if (!Array.isArray(data)) { if (!Array.isArray(data)) {
return [{ json: data }]; return [{ json: data }];
@ -324,13 +327,60 @@ export async function getTableSchema(
table: string, table: string,
): Promise<ColumnInfo[]> { ): Promise<ColumnInfo[]> {
const columns = await db.any( const columns = await db.any(
'SELECT column_name, data_type, is_nullable FROM information_schema.columns WHERE table_schema = $1 AND table_name = $2', 'SELECT column_name, data_type, is_nullable, udt_name, column_default FROM information_schema.columns WHERE table_schema = $1 AND table_name = $2',
[schema, table], [schema, table],
); );
return columns; return columns;
} }
export async function uniqueColumns(db: PgpDatabase, table: string) {
// Using the modified query from https://wiki.postgresql.org/wiki/Retrieve_primary_key_columns
const unique = await db.any(
`
SELECT DISTINCT a.attname
FROM pg_index i JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = ANY(i.indkey)
WHERE i.indrelid = quote_ident($1)::regclass
AND (i.indisprimary OR i.indisunique);
`,
[table],
);
return unique as IDataObject[];
}
export async function getEnums(db: PgpDatabase): Promise<EnumInfo[]> {
const enumsData = await db.any(
'SELECT pg_type.typname, pg_enum.enumlabel FROM pg_type JOIN pg_enum ON pg_enum.enumtypid = pg_type.oid;',
);
return enumsData as EnumInfo[];
}
export function getEnumValues(enumInfo: EnumInfo[], enumName: string): INodePropertyOptions[] {
return enumInfo.reduce((acc, current) => {
if (current.typname === enumName) {
acc.push({ name: current.enumlabel, value: current.enumlabel });
}
return acc;
}, [] as INodePropertyOptions[]);
}
export async function doesRowExist(
db: PgpDatabase,
schema: string,
table: string,
values: string[],
): Promise<boolean> {
const where = [];
for (let i = 3; i < 3 + values.length; i += 2) {
where.push(`$${i}:name=$${i + 1}`);
}
const exists = await db.any(
`SELECT EXISTS(SELECT 1 FROM $1:name.$2:name WHERE ${where.join(' AND ')})`,
[schema, table, ...values],
);
return exists[0].exists;
}
export function checkItemAgainstSchema( export function checkItemAgainstSchema(
node: INode, node: INode,
item: IDataObject, item: IDataObject,

View file

@ -1,3 +1,4 @@
export * as credentialTest from './credentialTest'; export * as credentialTest from './credentialTest';
export * as listSearch from './listSearch'; export * as listSearch from './listSearch';
export * as loadOptions from './loadOptions'; export * as loadOptions from './loadOptions';
export * as resourceMapping from './resourceMapping';

View file

@ -0,0 +1,91 @@
import type { ILoadOptionsFunctions, ResourceMapperFields, FieldType } from 'n8n-workflow';
import { getEnumValues, getEnums, getTableSchema, uniqueColumns } from '../helpers/utils';
import { configurePostgres } from '../transport';
const fieldTypeMapping: Partial<Record<FieldType, string[]>> = {
string: ['text', 'varchar', 'character varying', 'character', 'char'],
number: [
'integer',
'smallint',
'bigint',
'decimal',
'numeric',
'real',
'double precision',
'smallserial',
'serial',
'bigserial',
],
boolean: ['boolean'],
dateTime: [
'timestamp',
'date',
'timestampz',
'timestamp without time zone',
'timestamp with time zone',
],
time: ['time', 'time without time zone', 'time with time zone'],
object: ['json', 'jsonb'],
options: ['enum', 'USER-DEFINED'],
array: ['ARRAY'],
};
function mapPostgresType(postgresType: string): FieldType {
let mappedType: FieldType = 'string';
for (const t of Object.keys(fieldTypeMapping)) {
const postgresTypes = fieldTypeMapping[t as FieldType];
if (postgresTypes?.includes(postgresType)) {
mappedType = t as FieldType;
}
}
return mappedType;
}
export async function getMappingColumns(
this: ILoadOptionsFunctions,
): Promise<ResourceMapperFields> {
const credentials = await this.getCredentials('postgres');
const { db } = await configurePostgres(credentials);
const schema = this.getNodeParameter('schema', 0, {
extractValue: true,
}) as string;
const table = this.getNodeParameter('table', 0, {
extractValue: true,
}) as string;
const operation = this.getNodeParameter('operation', 0, {
extractValue: true,
}) as string;
try {
const columns = await getTableSchema(db, schema, table);
const unique = operation === 'upsert' ? await uniqueColumns(db, table) : [];
const enumInfo = await getEnums(db);
const fields = await Promise.all(
columns.map(async (col) => {
const canBeUsedToMatch =
operation === 'upsert' ? unique.some((u) => u.attname === col.column_name) : true;
const type = mapPostgresType(col.data_type);
const options = type === 'options' ? getEnumValues(enumInfo, col.udt_name) : undefined;
const isAutoIncrement = col.column_default?.startsWith('nextval');
return {
id: col.column_name,
displayName: col.column_name,
required: col.is_nullable !== 'YES' && !isAutoIncrement,
defaultMatch: col.column_name === 'id',
display: true,
type,
canBeUsedToMatch,
options,
};
}),
);
return { fields };
} catch (error) {
throw error;
}
}

View file

@ -552,7 +552,9 @@ export class Expression {
// The parameter value is complex so resolve depending on type // The parameter value is complex so resolve depending on type
if (Array.isArray(parameterValue)) { if (Array.isArray(parameterValue)) {
// Data is an array // Data is an array
const returnData = parameterValue.map((item) => resolveParameterValue(item, {})); const returnData = parameterValue.map((item) =>
resolveParameterValue(item as NodeParameterValueType, {}),
);
return returnData as NodeParameterValue[] | INodeParameters[]; return returnData as NodeParameterValue[] | INodeParameters[];
} }

View file

@ -995,7 +995,8 @@ export type NodeParameterValueType =
| INodeParameterResourceLocator | INodeParameterResourceLocator
| NodeParameterValue[] | NodeParameterValue[]
| INodeParameters[] | INodeParameters[]
| INodeParameterResourceLocator[]; | INodeParameterResourceLocator[]
| ResourceMapperValue[];
export interface INodeParameters { export interface INodeParameters {
[key: string]: NodeParameterValueType; [key: string]: NodeParameterValueType;
@ -1016,7 +1017,8 @@ export type NodePropertyTypes =
| 'string' | 'string'
| 'credentialsSelect' | 'credentialsSelect'
| 'resourceLocator' | 'resourceLocator'
| 'curlImport'; | 'curlImport'
| 'resourceMapper';
export type CodeAutocompleteTypes = 'function' | 'functionItem'; export type CodeAutocompleteTypes = 'function' | 'functionItem';
@ -1052,9 +1054,25 @@ export interface INodePropertyTypeOptions {
showAlpha?: boolean; // Supported by: color showAlpha?: boolean; // Supported by: color
sortable?: boolean; // Supported when "multipleValues" set to true sortable?: boolean; // Supported when "multipleValues" set to true
expirable?: boolean; // Supported by: hidden (only in the credentials) expirable?: boolean; // Supported by: hidden (only in the credentials)
resourceMapper?: ResourceMapperTypeOptions;
[key: string]: any; [key: string]: any;
} }
export interface ResourceMapperTypeOptions {
resourceMapperMethod: string;
mode: 'add' | 'update' | 'upsert';
fieldWords?: { singular: string; plural: string };
addAllFields?: boolean;
noFieldsError?: string;
multiKeyMatch?: boolean;
supportAutoMap?: boolean;
matchingFieldsLabels?: {
title?: string;
description?: string;
hint?: string;
};
}
export interface IDisplayOptions { export interface IDisplayOptions {
hide?: { hide?: {
[key: string]: NodeParameterValue[] | undefined; [key: string]: NodeParameterValue[] | undefined;
@ -1221,6 +1239,9 @@ export interface INodeType {
// Contains a group of functions that test credentials. // Contains a group of functions that test credentials.
[functionName: string]: ICredentialTestFunction; [functionName: string]: ICredentialTestFunction;
}; };
resourceMapping?: {
[functionName: string]: (this: ILoadOptionsFunctions) => Promise<ResourceMapperFields>;
};
}; };
webhookMethods?: { webhookMethods?: {
[name in IWebhookDescription['name']]?: { [name in IWebhookDescription['name']]?: {
@ -1934,6 +1955,44 @@ export interface IExceutionSummaryNodeExecutionResult {
}>; }>;
} }
export interface ResourceMapperFields {
fields: ResourceMapperField[];
}
export interface ResourceMapperField {
id: string;
displayName: string;
defaultMatch: boolean;
canBeUsedToMatch?: boolean;
required: boolean;
display: boolean;
type?: FieldType;
removed?: boolean;
options?: INodePropertyOptions[];
}
export type FieldType =
| 'string'
| 'number'
| 'dateTime'
| 'boolean'
| 'time'
| 'array'
| 'object'
| 'options';
export type ValidationResult = {
valid: boolean;
errorMessage?: string;
newValue?: string | number | boolean | object | null | undefined;
};
export type ResourceMapperValue = {
mappingMode: string;
value: { [key: string]: string | number | boolean | null } | null;
matchingColumns: string[];
schema: ResourceMapperField[];
};
export interface ExecutionOptions { export interface ExecutionOptions {
limit?: number; limit?: number;
} }

View file

@ -37,11 +37,16 @@ import type {
IWorkflowExecuteAdditionalData, IWorkflowExecuteAdditionalData,
NodeParameterValue, NodeParameterValue,
WebhookHttpMethod, WebhookHttpMethod,
FieldType,
INodePropertyOptions,
ResourceMapperValue,
ValidationResult,
} from './Interfaces'; } from './Interfaces';
import { isValidResourceLocatorParameterValue } from './type-guards'; import { isResourceMapperValue, isValidResourceLocatorParameterValue } from './type-guards';
import { deepCopy } from './utils'; import { deepCopy } from './utils';
import type { Workflow } from './Workflow'; import type { Workflow } from './Workflow';
import { DateTime } from 'luxon';
export const cronNodeOptions: INodePropertyCollection[] = [ export const cronNodeOptions: INodePropertyCollection[] = [
{ {
@ -1097,6 +1102,173 @@ export function nodeIssuesToString(issues: INodeIssues, node?: INode): string[]
return nodeIssues; return nodeIssues;
} }
// Validates field against the schema and tries to parse it to the correct type
export const validateFieldType = (
fieldName: string,
value: unknown,
type: FieldType,
options?: INodePropertyOptions[],
): ValidationResult => {
if (value === null || value === undefined) return { valid: true };
const defaultErrorMessage = `'${fieldName}' expects a ${type} but we got '${String(value)}'.`;
switch (type.toLowerCase()) {
case 'number': {
try {
return { valid: true, newValue: tryToParseNumber(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'boolean': {
try {
return { valid: true, newValue: tryToParseBoolean(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'datetime': {
try {
return { valid: true, newValue: tryToParseDateTime(value) };
} catch (e) {
const luxonDocsURL =
'https://moment.github.io/luxon/api-docs/index.html#datetimefromformat';
const errorMessage = `${defaultErrorMessage} <br/><br/> Consider using <a href="${luxonDocsURL}" target="_blank"><code>DateTime.fromFormat</code></a> to work with custom date formats.`;
return { valid: false, errorMessage };
}
}
case 'time': {
try {
return { valid: true, newValue: tryToParseTime(value) };
} catch (e) {
return {
valid: false,
errorMessage: `'${fieldName}' expects time (hh:mm:(:ss)) but we got '${String(value)}'.`,
};
}
}
case 'object': {
try {
return { valid: true, newValue: tryToParseObject(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'array': {
try {
return { valid: true, newValue: tryToParseArray(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'options': {
const validOptions = options?.map((option) => option.value).join(', ') || '';
const isValidOption = options?.some((option) => option.value === value) || false;
if (!isValidOption) {
return {
valid: false,
errorMessage: `'${fieldName}' expects one of the following values: [${validOptions}] but we got '${String(
value,
)}'`,
};
}
return { valid: true, newValue: value };
}
default: {
return { valid: true, newValue: value };
}
}
};
export const tryToParseNumber = (value: unknown): number => {
const isValidNumber = !isNaN(Number(value));
if (!isValidNumber) {
throw new Error(`Could not parse '${String(value)}' to number.`);
}
return Number(value);
};
export const tryToParseBoolean = (value: unknown): value is boolean => {
if (typeof value === 'boolean') {
return value;
}
if (typeof value === 'string' && ['true', 'false'].includes(value.toLowerCase())) {
return value.toLowerCase() === 'true';
}
const num = Number(value);
if (num === 0) {
return false;
} else if (num === 1) {
return true;
}
throw new Error(`Could not parse '${String(value)}' to boolean.`);
};
export const tryToParseDateTime = (value: unknown): DateTime => {
const dateString = String(value).trim();
// Rely on luxon to parse different date formats
const isoDate = DateTime.fromISO(dateString, { setZone: true });
if (isoDate.isValid) {
return isoDate;
}
const httpDate = DateTime.fromHTTP(dateString, { setZone: true });
if (httpDate.isValid) {
return httpDate;
}
const rfc2822Date = DateTime.fromRFC2822(dateString, { setZone: true });
if (rfc2822Date.isValid) {
return rfc2822Date;
}
const sqlDate = DateTime.fromSQL(dateString, { setZone: true });
if (sqlDate.isValid) {
return sqlDate;
}
throw new Error(`The value "${dateString}" is not a valid date.`);
};
export const tryToParseTime = (value: unknown): string => {
const isTimeInput = /^\d{2}:\d{2}(:\d{2})?((\-|\+)\d{4})?((\-|\+)\d{1,2}(:\d{2})?)?$/s.test(
String(value),
);
if (!isTimeInput) {
throw new Error(`The value "${String(value)}" is not a valid time.`);
}
return String(value);
};
export const tryToParseArray = (value: unknown): unknown[] => {
try {
const parsed = JSON.parse(String(value));
if (!Array.isArray(parsed)) {
throw new Error(`The value "${String(value)}" is not a valid array.`);
}
return parsed;
} catch (e) {
throw new Error(`The value "${String(value)}" is not a valid array.`);
}
};
export const tryToParseObject = (value: unknown): object => {
if (value && typeof value === 'object' && !Array.isArray(value)) {
return value;
}
try {
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
const o = JSON.parse(String(value));
if (typeof o !== 'object' || Array.isArray(o)) {
throw new Error(`The value "${String(value)}" is not a valid object.`);
}
return o;
} catch (e) {
throw new Error(`The value "${String(value)}" is not a valid object.`);
}
};
/* /*
* Validates resource locator node parameters based on validation ruled defined in each parameter mode * Validates resource locator node parameters based on validation ruled defined in each parameter mode
* *
@ -1128,6 +1300,42 @@ export const validateResourceLocatorParameter = (
return validationErrors; return validationErrors;
}; };
/*
* Validates resource mapper values based on service schema
*
*/
export const validateResourceMapperParameter = (
nodeProperties: INodeProperties,
value: ResourceMapperValue,
skipRequiredCheck = false,
): Record<string, string[]> => {
const issues: Record<string, string[]> = {};
let fieldWordSingular =
nodeProperties.typeOptions?.resourceMapper?.fieldWords?.singular || 'Field';
fieldWordSingular = fieldWordSingular.charAt(0).toUpperCase() + fieldWordSingular.slice(1);
value.schema.forEach((field) => {
const fieldValue = value.value ? value.value[field.id] : null;
const key = `${nodeProperties.name}.${field.id}`;
const fieldErrors: string[] = [];
if (field.required && !skipRequiredCheck) {
if (value.value === null || fieldValue === null || fieldValue === undefined) {
const error = `${fieldWordSingular} "${field.id}" is required`;
fieldErrors.push(error);
}
}
if (!fieldValue?.toString().startsWith('=') && field.type) {
const validationResult = validateFieldType(field.id, fieldValue, field.type, field.options);
if (!validationResult.valid && validationResult.errorMessage) {
fieldErrors.push(validationResult.errorMessage);
}
}
if (fieldErrors.length > 0) {
issues[key] = fieldErrors;
}
});
return issues;
};
/** /**
* Adds an issue if the parameter is not defined * Adds an issue if the parameter is not defined
* *
@ -1196,8 +1404,9 @@ export function getParameterIssues(
node: INode, node: INode,
): INodeIssues { ): INodeIssues {
const foundIssues: INodeIssues = {}; const foundIssues: INodeIssues = {};
const isDisplayed = displayParameterPath(nodeValues, nodeProperties, path, node);
if (nodeProperties.required === true) { if (nodeProperties.required === true) {
if (displayParameterPath(nodeValues, nodeProperties, path, node)) { if (isDisplayed) {
const value = getParameterValueByPath(nodeValues, nodeProperties.name, path); const value = getParameterValueByPath(nodeValues, nodeProperties.name, path);
if ( if (
@ -1218,24 +1427,37 @@ export function getParameterIssues(
} }
} }
if (nodeProperties.type === 'resourceLocator') { if (nodeProperties.type === 'resourceLocator' && isDisplayed) {
if (displayParameterPath(nodeValues, nodeProperties, path, node)) { const value = getParameterValueByPath(nodeValues, nodeProperties.name, path);
const value = getParameterValueByPath(nodeValues, nodeProperties.name, path); if (isINodeParameterResourceLocator(value)) {
if (isINodeParameterResourceLocator(value)) { const mode = nodeProperties.modes?.find((option) => option.name === value.mode);
const mode = nodeProperties.modes?.find((option) => option.name === value.mode); if (mode) {
if (mode) { const errors = validateResourceLocatorParameter(value, mode);
const errors = validateResourceLocatorParameter(value, mode); errors.forEach((error) => {
errors.forEach((error) => { if (foundIssues.parameters === undefined) {
if (foundIssues.parameters === undefined) { foundIssues.parameters = {};
foundIssues.parameters = {}; }
} if (foundIssues.parameters[nodeProperties.name] === undefined) {
if (foundIssues.parameters[nodeProperties.name] === undefined) { foundIssues.parameters[nodeProperties.name] = [];
foundIssues.parameters[nodeProperties.name] = []; }
}
foundIssues.parameters[nodeProperties.name].push(error); foundIssues.parameters[nodeProperties.name].push(error);
}); });
}
}
} else if (nodeProperties.type === 'resourceMapper' && isDisplayed) {
const skipRequiredCheck = nodeProperties.typeOptions?.resourceMapper?.mode !== 'add';
const value = getParameterValueByPath(nodeValues, nodeProperties.name, path);
if (isResourceMapperValue(value)) {
const issues = validateResourceMapperParameter(nodeProperties, value, skipRequiredCheck);
if (Object.keys(issues).length > 0) {
if (foundIssues.parameters === undefined) {
foundIssues.parameters = {};
} }
if (foundIssues.parameters[nodeProperties.name] === undefined) {
foundIssues.parameters[nodeProperties.name] = [];
}
foundIssues.parameters = { ...foundIssues.parameters, ...issues };
} }
} }
} }

View file

@ -465,7 +465,13 @@ export class Workflow {
const returnArray: any[] = []; const returnArray: any[] = [];
for (const currentValue of parameterValue) { for (const currentValue of parameterValue) {
returnArray.push(this.renameNodeInParameterValue(currentValue, currentName, newName)); returnArray.push(
this.renameNodeInParameterValue(
currentValue as NodeParameterValueType,
currentName,
newName,
),
);
} }
return returnArray; return returnArray;

View file

@ -39,6 +39,7 @@ export {
isINodePropertiesList, isINodePropertiesList,
isINodePropertyCollectionList, isINodePropertyCollectionList,
isINodePropertyOptionsList, isINodePropertyOptionsList,
isResourceMapperValue,
} from './type-guards'; } from './type-guards';
export { ExpressionExtensions } from './Extensions'; export { ExpressionExtensions } from './Extensions';

View file

@ -3,6 +3,7 @@ import type {
INodePropertyOptions, INodePropertyOptions,
INodePropertyCollection, INodePropertyCollection,
INodeParameterResourceLocator, INodeParameterResourceLocator,
ResourceMapperValue,
} from './Interfaces'; } from './Interfaces';
export const isINodeProperties = ( export const isINodeProperties = (
@ -43,3 +44,13 @@ export const isValidResourceLocatorParameterValue = (
return !!value; return !!value;
} }
}; };
export const isResourceMapperValue = (value: unknown): value is ResourceMapperValue => {
return (
typeof value === 'object' &&
value !== null &&
'mappingMode' in value &&
'schema' in value &&
'value' in value
);
};

View file

@ -0,0 +1,206 @@
import { validateFieldType } from '@/NodeHelpers';
import type { DateTime } from 'luxon';
const VALID_ISO_DATES = [
'1994-11-05T08:15:30-05:00',
'1994-11-05T13:15:30Z',
'1997-07-16T19:20+01:00',
'1997-07-16T19:20:30+01:00',
'1997-07-16T19:20:30.45+01:00',
'2018-05-16',
'1972-06-30T23:59:40Z',
'2019-03-26T14:00:00.9Z',
'2019-03-26T14:00:00.4999Z',
'2023-05-17T10:52:32+0000',
'2023-05-17T10:52:32+0000',
];
const VALID_HTTP_DATES = [
'Wed, 21 Oct 2015 07:28:00 GMT',
'Wed, 01 Jun 2022 08:00:00 GMT',
'Tue, 15 Nov 1994 12:45:26 GMT',
'Wed, 1 Jun 2022 08:00:00 GMT',
];
const VALID_RFC_DATES = [
'Tue, 04 Jun 2013 07:40:03 -0400',
'Tue, 4 Jun 2013 02:24:39 +0530',
'Wed, 17 May 2023 10:52:32 +0000',
];
const VALID_SQL_DATES = ['2008-11-11', '2008-11-11 13:23:44'];
const INVALID_DATES = [
'1994-11-05M08:15:30-05:00',
'18-05-2020',
'',
'Wed, 17 May 2023 10:52:32',
'SMT, 17 May 2023 10:52:32',
'1685084980', // We are not supporting timestamps
'1685085012135',
1685084980,
1685085012135,
true,
[],
];
describe('Type Validation', () => {
it('should validate and cast ISO dates', () => {
VALID_ISO_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should validate and cast RFC 2822 dates', () => {
VALID_RFC_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should validate and cast HTTP dates', () => {
VALID_HTTP_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should validate and cast SQL dates', () => {
VALID_SQL_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should not validate invalid dates', () => {
INVALID_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(false);
});
});
it('should validate boolean values properly', () => {
expect(validateFieldType('boolean', 'true', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', 'TRUE', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', 1, 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', '1', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', '01', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', 'false', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', 'FALSE', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', '0', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', '000', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', '0000', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', 0, 'boolean').newValue).toBe(false);
});
it('should not validate invalid boolean values', () => {
expect(validateFieldType('boolean', 'tru', 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 'fals', 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 1111, 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 2, 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', -1, 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 'yes', 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 'no', 'boolean').valid).toBe(false);
});
it('should validate and cast numbers', () => {
expect(validateFieldType('number', '1', 'number').newValue).toBe(1);
expect(validateFieldType('number', '-1', 'number').newValue).toBe(-1);
expect(validateFieldType('number', '1.1', 'number').newValue).toBe(1.1);
expect(validateFieldType('number', '-1.1', 'number').newValue).toBe(-1.1);
expect(validateFieldType('number', 1, 'number').newValue).toBe(1);
expect(validateFieldType('number', 'A', 'number').valid).toBe(false);
expect(validateFieldType('number', '1,1', 'number').valid).toBe(false);
expect(validateFieldType('number', true, 'number').valid).toBe(true);
expect(validateFieldType('number', '1972-06-30T23:59:40Z', 'number').valid).toBe(false);
expect(validateFieldType('number', [1, 2], 'number').valid).toBe(false);
});
it('should validate and cast JSON properly', () => {
expect(validateFieldType('json', '{"a": 1}', 'object').newValue).toEqual({ a: 1 });
expect(
validateFieldType('json', '{"a": 1, "b": { "c": 10, "d": "test"}}', 'object').valid,
).toEqual(true);
expect(validateFieldType('json', { name: 'John' }, 'object').valid).toEqual(true);
expect(
validateFieldType(
'json',
{ name: 'John', address: { street: 'Via Roma', city: 'Milano' } },
'object',
).valid,
).toEqual(true);
// Invalid value:
expect(validateFieldType('json', ['one', 'two'], 'object').valid).toEqual(false);
// eslint-disable-next-line prettier/prettier
expect(validateFieldType('json', ["one", "two"], 'object').valid).toEqual(false);
expect(validateFieldType('json', '1', 'object').valid).toEqual(false);
expect(validateFieldType('json', '[1]', 'object').valid).toEqual(false);
expect(validateFieldType('json', '1.1', 'object').valid).toEqual(false);
expect(validateFieldType('json', 1.1, 'object').valid).toEqual(false);
expect(validateFieldType('json', '"a"', 'object').valid).toEqual(false);
expect(validateFieldType('json', '{a: 1}', 'object').valid).toEqual(false);
expect(validateFieldType('json', '["apples", "oranges"]', 'object').valid).toEqual(false);
expect(validateFieldType('json', [{ name: 'john' }, { name: 'bob' }], 'object').valid).toEqual(
false,
);
expect(
validateFieldType('json', '[ { name: "john" }, { name: "bob" } ]', 'object').valid,
).toEqual(false);
});
it('should validate and cast arrays properly', () => {
expect(validateFieldType('array', '["apples", "oranges"]', 'array').newValue).toEqual([
'apples',
'oranges',
]);
expect(validateFieldType('array', '[1]', 'array').newValue).toEqual([1]);
expect(validateFieldType('array', '[1, 2]', 'array').newValue).toEqual([1, 2]);
// Invalid values:
expect(validateFieldType('array', '"apples", "oranges"', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1.1', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1, 2', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1. 2. 3', 'array').valid).toEqual(false);
expect(validateFieldType('array', '[1, 2, 3', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1, 2, 3]', 'array').valid).toEqual(false);
expect(validateFieldType('array', '{1, 2, {3, 4}, 5}', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1, 2, {3, 4}, 5', 'array').valid).toEqual(false);
expect(validateFieldType('array', { name: 'John' }, 'array').valid).toEqual(false);
});
it('should validate options properly', () => {
expect(
validateFieldType('options', 'oranges', 'options', [
{ name: 'apples', value: 'apples' },
{ name: 'oranges', value: 'oranges' },
]).valid,
).toEqual(true);
expect(
validateFieldType('options', 'something else', 'options', [
{ name: 'apples', value: 'apples' },
{ name: 'oranges', value: 'oranges' },
]).valid,
).toEqual(false);
});
it('should validate and cast time properly', () => {
expect(validateFieldType('time', '23:23', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23+1000', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23-1000', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00+01:00', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00-01:00', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00+01', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00-01', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23:23', 'time').valid).toEqual(false);
expect(validateFieldType('time', '23', 'time').valid).toEqual(false);
expect(validateFieldType('time', 'foo', 'time').valid).toEqual(false);
expect(validateFieldType('time', '23:23:', 'time').valid).toEqual(false);
expect(validateFieldType('time', '23::23::23', 'time').valid).toEqual(false);
});
});