feat(editor): Implement Resource Mapper component (#6207)

*  scaffolding
*  finished scaffolding
*  renamed types
*  updated subtitle
*  renamed functions file, UI updates
*  query parameters fixes, ui updates, refactoring
*  fixes for credentials test, setup for error parsing
*  rlc for schema and table, error handling tweaks
*  delete operation, new options
*  columns loader
*  linter fixes
*  where clauses setup
*  logic for processing where clauses
*  select operation
*  refactoring
*  data mode for insert and update, wip
*  data mapping, insert update, skip on conflict option
*  select columns with spaces fix
*  update operation update, wip
*  finished update operation
*  upsert operation
*  ui fixes
* Copy updates.
* Copy updates.
*  option to convert empty strings to nulls, schema checks
*  UI requested updates
*  ssh setup WIP
*  fixes, ssh WIP
*  ssh fixes, credentials
*  credentials testing update
*  uncaught error fix
*  clean up
*  address in use fix
*  improved error message
*  tests setup
*  unit tests wip
*  config files clean up
*  utils unit tests
*  refactoring
*  setup for testing operations, tests for deleteTable operation
*  executeQuery and insert operations tests
*  select, update, upsert operations tests
*  runQueries tests setup
*  hint to query
* Copy updates.
*  ui fixes
*  clean up
*  error message update
*  ui update
* Minor tweaks to query params decription.
* feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752)
*  Added initial resource mapping support in google sheets node
*  Wired mapping API endpoint with node-specific logic for fetching mapping fields
*  Implementing mapping fields logic for google sheets
*  Updating Google Sheets execute methods to support resource mapper fields
* 🚧 Added initial version of `ResourceLocator` component
* 👌 Added `update` mode to resource mapper modes
* 👌 Addressing PR feedback
* 👌 Removing leftover const reference
* 👕 Fixing lint errors
*  singlton for conections
*  credentials test fix, clean up
* feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814)
*  scaffolding
*  finished scaffolding
*  renamed types
*  updated subtitle
*  renamed functions file, UI updates
*  query parameters fixes, ui updates, refactoring
*  fixes for credentials test, setup for error parsing
*  rlc for schema and table, error handling tweaks
*  delete operation, new options
*  columns loader
*  linter fixes
*  where clauses setup
*  logic for processing where clauses
*  select operation
*  refactoring
*  data mode for insert and update, wip
*  data mapping, insert update, skip on conflict option
*  select columns with spaces fix
*  update operation update, wip
*  finished update operation
*  upsert operation
*  ui fixes
* Copy updates.
* Copy updates.
*  option to convert empty strings to nulls, schema checks
*  UI requested updates
*  ssh setup WIP
*  fixes, ssh WIP
*  ssh fixes, credentials
*  credentials testing update
*  uncaught error fix
*  clean up
*  address in use fix
*  improved error message
*  tests setup
*  unit tests wip
*  config files clean up
*  utils unit tests
*  refactoring
*  setup for testing operations, tests for deleteTable operation
*  executeQuery and insert operations tests
*  select, update, upsert operations tests
*  runQueries tests setup
*  hint to query
* Copy updates.
*  ui fixes
*  clean up
*  error message update
*  ui update
* Minor tweaks to query params decription.
*  Updated Postgres node to use resource mapper component
*  Implemented postgres <-> resource mapper type mapping
*  Updated Postgres node execution to use resource mapper fields in v3
* 🔥 Removing unused import
---------
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>

* feat(core): Resource editor componend P0 (#5970)
*  Added inital value of mapping mode dropdown
*  Finished mapping mode selector
*  Finished implementing mapping mode selector
*  Implemented 'Columns to match on' dropdown
*  Implemented `loadOptionsDependOn` support in resource mapper
*  Implemented initial version of mapping fields
*  Implementing dependant fields watcher in new component setup
*  Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one
*  Handling matching columns correctly in UI
*  Saving and loading resourceMapper values in component
*  Implemented proper data saving and loading
*  ResourceMapper component refactor, fixing value save/load
*  Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key
*  Updated Google Sheets node to work with the new UI
*  Updating Postgres Node to work with new UI
*  Additional loading indicator that shown if there is no mapping mode selector
*  Removing hard-coded values, fixing matching columns ordering, refactoring
*  Updating field names in nodes
*  Fixing minor UI issues
*  Implemented matching fields filter logic
*  Moving loading label outside of fields list
*  Added initial unit tests for resource mapper
*  Finished default rendering test
*  Test refactoring
*  Finished unit tests
* 🔨 Updating the way i18n is used in resource mapper components
* ✔️ Fixing value to match on logic for postgres node
*  Hiding mapping fields when auto-map mode is selected
*  Syncing selected mapping mode between components
*  Fixing dateTime input rendering and adding update check to Postgres node
*  Properly handling database connections. Sending null for empty string values.
* 💄 Updated wording in the error message for non-existing rows
*  Fixing issues with selected matching values
* ✔️ Updating unit tests after matching logic update
*  Updating matching columns when new fields are loaded
*  Defaulting to null for empty parameter values
*  Allowing zero as valid value for number imputs
*  Updated list of types that use datepicker as widger
*  Using text inputs for time types
*  Initial mapping field rework
*  Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors
*  Added tooltip for columns that cannot be deleted
*  Saving deleted values in parameter value
*  Implemented control to add/remove mapping fields
*  Syncing field list with add field dropdown when changing dependent values
*  Not showing removed fields in matching columns selector. Updating wording in matching columns selector description
*  Implementing disabled states for add/remove all fields options
*  Saving removed columns separately, updating copy
*  Implemented resource mapper values validation
*  Updated validation logic and error input styling
*  Validating resource mapper fields when new nodes are added
*  Using node field words in validation, refactoring resource mapper component
*  Implemented schema syncing and add/remove all fields
*  Implemented custom parameter actions
*  Implemented loading indicator in parameter options
* 🔨 Removing unnecessary constants and vue props
*  Handling default values properly
*  Fixing validation logic
* 👕 Fixing lint errors
*  Fixing type issues
*  Not showing fields by default if `addAllFields` is set to `false`
*  Implemented field type validation in resource mapper
*  Updated casing in copy, removed all/remove all option from bottom menu
*  Added auto mapping mode notice
*  Added support for more types in validation
*  Added support for enumerated values
*  Fixing imports after merging
*  Not showing removed fields in matching columns selector. Refactoring validation logic.
* 👕 Fixing imports
* ✔️ Updating unit tests
*  Added resource mapper schema tests
*  Removing `match` from resource mapper field definition, fixing matching columns loading
*  Fixed schema merging
*  update operation return data fix
*  review
* 🐛 Added missing import
* 💄 Updating parameter actions icon based on the ui review
* 💄 Updating word capitalisation in tooltips
* 💄 Added empty state to mapping fields list
* 💄 Removing asterisk from fields, updating tooltips for matching fields
*  Preventing matching fields from being removed by 'Remove All option'
*  Not showing hidden fields in the `Add field` dropdown
*  Added support for custom matching columns labels
*  query optimization
*  fix
*  Optimizing Postgres node enumeration logic
*  Added empty state for matching columns
*  Only fully loading fields if there is no schema fetched
*  Hiding mapping fields if there is no matching columns available in the schema
* ✔️ Fixing minor issues
*  Implemented runtime type validation
* 🔨 Refactoring validation logic
*  Implemented required check, added more custom messages
*  Skipping boolean type in required check
* Type check improvements
*  Only reloading fields if dependent values actually change
*  Adding item index to validation error title
*  Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted
*  Resetting field values when adding them via the addAll option
*  Using minor version (2.2) for new Postgres node
*  Implemented proper date validation and type casting
* 👕 Consolidating typing
*  Added unit tests for type validations
* 👌 Addressing front-end review comments
*  More refactoring to address review changes
*  Updating leftover props
*  Added fallback for ISO dates with invalid timezones
* Added timestamp to datetime test cases
*  Reseting matching columns if operation changes
*  Not forcing auto-increment fields to be filled in in Postgres node. Handling null values
* 💄 Added a custom message for invalid dates
*  Better handling of JSON values
*  Updating codemirror readonly stauts based on component property, handling objects in json validation
* Deleting leftover console.log
*  Better time validation
*  Fixing build error after merging
* 👕 Fixing lint error
*  Updating node configuration values
*  Handling postgres arrays better
*  Handling SQL array syntax
*  Updating time validation rules to include timezone
*  Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets
*  Allowing removed fields to be selected for match
*  Updated the query for fetching unique columns and primary keys
*  Optimizing the unique query
*  Setting timezone to all parsed dates
*  Addressing PR review feedback
*  Configuring Sheets node for production, minor vue component update
* New cases added to the TypeValidation test.
*  Tweaking validation rules for arrays/objects and updating test cases
---------
Co-authored-by: Michael Kret <michael.k@radency.com>
Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
This commit is contained in:
Milorad FIlipović 2023-05-31 11:56:09 +02:00 committed by GitHub
parent 5ae1124106
commit 04cfa548af
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
57 changed files with 3436 additions and 183 deletions

View file

@ -31,6 +31,7 @@ import clientOAuth1 from 'oauth-1.0a';
import {
BinaryDataManager,
Credentials,
LoadMappingOptions,
LoadNodeParameterOptions,
LoadNodeListSearch,
UserSettings,
@ -49,6 +50,7 @@ import type {
ICredentialTypes,
ExecutionStatus,
IExecutionsSummary,
ResourceMapperFields,
IN8nUISettings,
} from 'n8n-workflow';
import { LoggerProxy, jsonParse } from 'n8n-workflow';
@ -79,6 +81,7 @@ import type {
NodeListSearchRequest,
NodeParameterOptionsRequest,
OAuthRequest,
ResourceMapperRequest,
WorkflowRequest,
} from '@/requests';
import { registerController } from '@/decorators';
@ -756,6 +759,58 @@ export class Server extends AbstractServer {
),
);
this.app.get(
`/${this.restEndpoint}/get-mapping-fields`,
ResponseHelper.send(
async (
req: ResourceMapperRequest,
res: express.Response,
): Promise<ResourceMapperFields | undefined> => {
const nodeTypeAndVersion = jsonParse(
req.query.nodeTypeAndVersion,
) as INodeTypeNameVersion;
const { path, methodName } = req.query;
if (!req.query.currentNodeParameters) {
throw new ResponseHelper.BadRequestError(
'Parameter currentNodeParameters is required.',
);
}
const currentNodeParameters = jsonParse(
req.query.currentNodeParameters,
) as INodeParameters;
let credentials: INodeCredentials | undefined;
if (req.query.credentials) {
credentials = jsonParse(req.query.credentials);
}
const loadMappingOptionsInstance = new LoadMappingOptions(
nodeTypeAndVersion,
this.nodeTypes,
path,
currentNodeParameters,
credentials,
);
const additionalData = await WorkflowExecuteAdditionalData.getBase(
req.user.id,
currentNodeParameters,
);
const fields = await loadMappingOptionsInstance.getOptionsViaMethodName(
methodName,
additionalData,
);
return fields;
},
),
);
// ----------------------------------------
// Active Workflows
// ----------------------------------------

View file

@ -348,6 +348,23 @@ export type NodeListSearchRequest = AuthenticatedRequest<
}
>;
// ----------------------------------
// /get-mapping-fields
// ----------------------------------
export type ResourceMapperRequest = AuthenticatedRequest<
{},
{},
{},
{
nodeTypeAndVersion: string;
methodName: string;
path: string;
currentNodeParameters: string;
credentials: string;
}
>;
// ----------------------------------
// /tags
// ----------------------------------

View file

@ -11,6 +11,7 @@ import type {
ITriggerFunctions as ITriggerFunctionsBase,
IWebhookFunctions as IWebhookFunctionsBase,
BinaryMetadata,
ValidationResult,
} from 'n8n-workflow';
// TODO: remove these after removing `n8n-core` dependency from `nodes-bases`
@ -89,3 +90,5 @@ export namespace n8n {
};
}
}
export type ExtendedValidationResult = Partial<ValidationResult> & { fieldName?: string };

View file

@ -0,0 +1,34 @@
import type { IWorkflowExecuteAdditionalData, ResourceMapperFields } from 'n8n-workflow';
import * as NodeExecuteFunctions from './NodeExecuteFunctions';
import { LoadNodeDetails } from './LoadNodeDetails';
export class LoadMappingOptions extends LoadNodeDetails {
/**
* Returns the available mapping fields for the ResourceMapper component
*/
async getOptionsViaMethodName(
methodName: string,
additionalData: IWorkflowExecuteAdditionalData,
): Promise<ResourceMapperFields> {
const node = this.getTempNode();
const nodeType = this.workflow.nodeTypes.getByNameAndVersion(node.type, node.typeVersion);
const method = nodeType?.methods?.resourceMapping?.[methodName];
if (typeof method !== 'function') {
throw new Error(
`The node-type "${node.type}" does not have the method "${methodName}" defined!`,
);
}
const thisArgs = NodeExecuteFunctions.getLoadOptionsFunctions(
this.workflow,
node,
this.path,
additionalData,
);
return method.call(thisArgs);
}
}

View file

@ -61,10 +61,12 @@ import type {
IWebhookFunctions,
BinaryMetadata,
FileSystemHelperFunctions,
INodeType,
} from 'n8n-workflow';
import {
createDeferredPromise,
isObjectEmpty,
isResourceMapperValue,
NodeApiError,
NodeHelpers,
NodeOperationError,
@ -74,6 +76,7 @@ import {
deepCopy,
fileTypeFromMimeType,
ExpressionError,
validateFieldType,
} from 'n8n-workflow';
import pick from 'lodash.pick';
@ -114,7 +117,7 @@ import { access as fsAccess } from 'fs/promises';
import { createReadStream } from 'fs';
import { BinaryDataManager } from './BinaryDataManager';
import type { IResponseError, IWorkflowSettings } from './Interfaces';
import type { ExtendedValidationResult, IResponseError, IWorkflowSettings } from './Interfaces';
import { extractValue } from './ExtractValue';
import { getClientCredentialsToken } from './OAuth2Helper';
import { PLACEHOLDER_EMPTY_EXECUTION_ID } from './Constants';
@ -1867,7 +1870,7 @@ function cleanupParameterData(inputData: NodeParameterValueType): void {
}
if (Array.isArray(inputData)) {
inputData.forEach((value) => cleanupParameterData(value));
inputData.forEach((value) => cleanupParameterData(value as NodeParameterValueType));
return;
}
@ -1886,6 +1889,103 @@ function cleanupParameterData(inputData: NodeParameterValueType): void {
}
}
const validateResourceMapperValue = (
parameterName: string,
paramValues: { [key: string]: unknown },
node: INode,
skipRequiredCheck = false,
): ExtendedValidationResult => {
const result: ExtendedValidationResult = { valid: true, newValue: paramValues };
const paramNameParts = parameterName.split('.');
if (paramNameParts.length !== 2) {
return result;
}
const resourceMapperParamName = paramNameParts[0];
const resourceMapperField = node.parameters[resourceMapperParamName];
if (!resourceMapperField || !isResourceMapperValue(resourceMapperField)) {
return result;
}
const schema = resourceMapperField.schema;
const paramValueNames = Object.keys(paramValues);
for (let i = 0; i < paramValueNames.length; i++) {
const key = paramValueNames[i];
const resolvedValue = paramValues[key];
const schemaEntry = schema.find((s) => s.id === key);
if (
!skipRequiredCheck &&
schemaEntry?.required === true &&
schemaEntry.type !== 'boolean' &&
!resolvedValue
) {
return {
valid: false,
errorMessage: `The value "${String(key)}" is required but not set`,
fieldName: key,
};
}
if (schemaEntry?.type) {
const validationResult = validateFieldType(
key,
resolvedValue,
schemaEntry.type,
schemaEntry.options,
);
if (!validationResult.valid) {
return { ...validationResult, fieldName: key };
} else {
// If it's valid, set the casted value
paramValues[key] = validationResult.newValue;
}
}
}
return result;
};
const validateValueAgainstSchema = (
node: INode,
nodeType: INodeType,
inputValues: string | number | boolean | object | null | undefined,
parameterName: string,
runIndex: number,
itemIndex: number,
) => {
let validationResult: ExtendedValidationResult = { valid: true, newValue: inputValues };
// Currently only validate resource mapper values
const resourceMapperField = nodeType.description.properties.find(
(prop) =>
NodeHelpers.displayParameter(node.parameters, prop, node) &&
prop.type === 'resourceMapper' &&
parameterName === `${prop.name}.value`,
);
if (resourceMapperField && typeof inputValues === 'object') {
validationResult = validateResourceMapperValue(
parameterName,
inputValues as { [key: string]: unknown },
node,
resourceMapperField.typeOptions?.resourceMapper?.mode !== 'add',
);
}
if (!validationResult.valid) {
throw new ExpressionError(
`Invalid input for '${
String(validationResult.fieldName) || parameterName
}' [item ${itemIndex}]`,
{
description: validationResult.errorMessage,
failExecution: true,
runIndex,
itemIndex,
nodeCause: node.name,
},
);
}
return validationResult.newValue;
};
/**
* Returns the requested resolved (all expressions replaced) node parameters.
*
@ -1947,6 +2047,16 @@ export function getNodeParameter(
returnData = extractValue(returnData, parameterName, node, nodeType);
}
// Validate parameter value if it has a schema defined
returnData = validateValueAgainstSchema(
node,
nodeType,
returnData,
parameterName,
runIndex,
itemIndex,
);
return returnData;
}

View file

@ -9,6 +9,7 @@ export * from './Constants';
export * from './Credentials';
export * from './DirectoryLoader';
export * from './Interfaces';
export * from './LoadMappingOptions';
export * from './LoadNodeParameterOptions';
export * from './LoadNodeListSearch';
export * from './NodeExecuteFunctions';

View file

@ -8,7 +8,7 @@ export default {
argTypes: {
placement: {
type: 'select',
options: ['top', 'top-start', 'top-end', 'bottom', 'bottom-end'],
options: ['top', 'top-end', 'top-start', 'bottom', 'bottom-end', 'bottom-start'],
},
size: {
type: 'select',

View file

@ -9,7 +9,10 @@
@visible-change="onVisibleChange"
>
<span :class="{ [$style.button]: true, [$style[theme]]: !!theme }">
<n8n-icon icon="ellipsis-v" :size="iconSize" />
<n8n-icon
:icon="iconOrientation === 'horizontal' ? 'ellipsis-h' : 'ellipsis-v'"
:size="iconSize"
/>
</span>
<template #dropdown>
@ -79,6 +82,11 @@ export default defineComponent({
default: 'default',
validator: (value: string): boolean => ['default', 'dark'].includes(value),
},
iconOrientation: {
type: String,
default: 'vertical',
validator: (value: string): boolean => ['horizontal', 'vertical'].includes(value),
},
},
methods: {
onCommand(value: string) {

View file

@ -33,7 +33,11 @@
v-if="$slots.options && label"
:class="{ [$style.overlay]: true, [$style.visible]: showOptions }"
/>
<div v-if="$slots.options" :class="{ [$style.options]: true, [$style.visible]: showOptions }">
<div
v-if="$slots.options"
:class="{ [$style.options]: true, [$style.visible]: showOptions }"
data-test-id="parameter-input-options-container"
>
<slot name="options" />
</div>
</label>

View file

@ -1361,6 +1361,13 @@ export type NodeAuthenticationOption = {
displayOptions?: IDisplayOptions;
};
export interface ResourceMapperReqParams {
nodeTypeAndVersion: INodeTypeNameVersion;
path: string;
methodName?: string;
currentNodeParameters: INodeParameters;
credentials?: INodeCredentials;
}
export interface EnvironmentVariable {
id: number;
key: string;

View file

@ -3,6 +3,7 @@ import type {
INodeTranslationHeaders,
IResourceLocatorReqParams,
IRestApiContext,
ResourceMapperReqParams,
} from '@/Interface';
import type {
IDataObject,
@ -15,6 +16,7 @@ import type {
INodeTypeNameVersion,
} from 'n8n-workflow';
import axios from 'axios';
import type { ResourceMapperFields } from 'n8n-workflow/src/Interfaces';
export async function getNodeTypes(baseUrl: string) {
const { data } = await axios.get(baseUrl + 'types/nodes.json', { withCredentials: true });
@ -59,3 +61,15 @@ export async function getResourceLocatorResults(
sendData as unknown as IDataObject,
);
}
export async function getResourceMapperFields(
context: IRestApiContext,
sendData: ResourceMapperReqParams,
): Promise<ResourceMapperFields> {
return makeRestApiRequest(
context,
'GET',
'/get-mapping-fields',
sendData as unknown as IDataObject,
);
}

View file

@ -6,7 +6,7 @@
import { defineComponent } from 'vue';
import { mapStores } from 'pinia';
import { EditorView, keymap } from '@codemirror/view';
import { EditorState, Prec } from '@codemirror/state';
import { Compartment, EditorState, Prec } from '@codemirror/state';
import { history, redo } from '@codemirror/commands';
import { acceptCompletion, autocompletion, completionStatus } from '@codemirror/autocomplete';
@ -19,6 +19,8 @@ import { inputTheme } from './theme';
import { n8nLang } from '@/plugins/codemirror/n8nLang';
import { completionManager } from '@/mixins/completionManager';
const editableConf = new Compartment();
export default defineComponent({
name: 'InlineExpressionEditorInput',
mixins: [completionManager, expressionManager, workflowHelpers],
@ -39,6 +41,11 @@ export default defineComponent({
},
},
watch: {
isReadOnly(newValue: boolean) {
this.editor?.dispatch({
effects: editableConf.reconfigure(EditorView.editable.of(!newValue)),
});
},
value(newValue) {
const isInternalChange = newValue === this.editor?.state.doc.toString();
@ -97,7 +104,7 @@ export default defineComponent({
history(),
expressionInputHandler(),
EditorView.lineWrapping,
EditorView.editable.of(!this.isReadOnly),
editableConf.of(EditorView.editable.of(!this.isReadOnly)),
EditorView.contentAttributes.of({ 'data-gramm': 'false' }), // disable grammarly
EditorView.domEventHandlers({
focus: () => {

View file

@ -362,6 +362,7 @@ import type {
INodeProperties,
INodePropertyCollection,
NodeParameterValueType,
IParameterLabel,
EditorType,
CodeNodeEditorLanguage,
} from 'n8n-workflow';
@ -380,7 +381,6 @@ import { externalHooks } from '@/mixins/externalHooks';
import { nodeHelpers } from '@/mixins/nodeHelpers';
import { workflowHelpers } from '@/mixins/workflowHelpers';
import { hasExpressionMapping, isValueExpression, isResourceLocatorValue } from '@/utils';
import { CODE_NODE_TYPE, CUSTOM_API_CALL_KEY, HTML_NODE_TYPE } from '@/constants';
import type { PropType } from 'vue';
import { debounceHelper } from '@/mixins/debounce';
@ -457,6 +457,12 @@ export default defineComponent({
expressionEvaluated: {
type: String as PropType<string | undefined>,
},
label: {
type: Object as PropType<IParameterLabel>,
default: () => ({
size: 'small',
}),
},
},
data() {
return {

View file

@ -10,6 +10,7 @@
>
<template #options>
<parameter-options
v-if="displayOptions"
:parameter="parameter"
:value="value"
:isReadOnly="isReadOnly"
@ -54,6 +55,7 @@
:forceShowExpression="forceShowExpression"
:hint="hint"
:hide-issues="hideIssues"
:label="label"
@valueChanged="valueChanged"
@textInput="onTextInput"
@focus="onFocus"

View file

@ -81,7 +81,16 @@
@valueChanged="valueChanged"
/>
</div>
<resource-mapper
v-else-if="parameter.type === 'resourceMapper'"
:parameter="parameter"
:node="node"
:path="getPath(parameter.name)"
:dependentParametersValues="getDependentParametersValues(parameter)"
inputSize="small"
labelSize="small"
@valueChanged="valueChanged"
/>
<div v-else-if="displayNodeParameter(parameter)" class="parameter-item">
<div
class="delete-option clickable"
@ -100,9 +109,11 @@
:parameter="parameter"
:hide-issues="hiddenIssuesInputs.includes(parameter.name)"
:value="getParameterValue(nodeValues, parameter.name, path)"
:displayOptions="true"
:displayOptions="shouldShowOptions(parameter)"
:path="getPath(parameter.name)"
:isReadOnly="isReadOnly"
:hideLabel="false"
:nodeValues="nodeValues"
@valueChanged="valueChanged"
@blur="onParameterBlur(parameter.name)"
/>
@ -127,7 +138,7 @@ import MultipleParameter from '@/components/MultipleParameter.vue';
import { workflowHelpers } from '@/mixins/workflowHelpers';
import ParameterInputFull from '@/components/ParameterInputFull.vue';
import ImportParameter from '@/components/ImportParameter.vue';
import ResourceMapper from '@/components/ResourceMapper/ResourceMapper.vue';
import { get, set } from 'lodash-es';
import { useNDVStore } from '@/stores/ndv.store';
import { useNodeTypesStore } from '@/stores/nodeTypes.store';
@ -143,6 +154,7 @@ export default defineComponent({
FixedCollectionParameter: async () => import('./FixedCollectionParameter.vue'),
CollectionParameter: async () => import('./CollectionParameter.vue'),
ImportParameter,
ResourceMapper,
},
props: {
nodeValues: {
@ -398,6 +410,33 @@ export default defineComponent({
// since there is no such case, omitting it to avoid additional computation
return isAuthRelatedParameter(this.nodeAuthFields, parameter);
},
shouldShowOptions(parameter: INodeProperties): boolean {
return parameter.type !== 'resourceMapper';
},
getDependentParametersValues(parameter: INodeProperties): string | null {
const loadOptionsDependsOn = this.getArgument('loadOptionsDependsOn', parameter) as
| string[]
| undefined;
if (loadOptionsDependsOn === undefined) {
return null;
}
// Get the resolved parameter values of the current node
const currentNodeParameters = this.ndvStore.activeNode?.parameters;
try {
const resolvedNodeParameters = this.resolveParameter(currentNodeParameters);
const returnValues: string[] = [];
for (const parameterPath of loadOptionsDependsOn) {
returnValues.push(get(resolvedNodeParameters, parameterPath) as string);
}
return returnValues.join('|');
} catch (error) {
return null;
}
},
},
watch: {
filteredParameterNames(newValue, oldValue) {

View file

@ -16,6 +16,7 @@
:isForCredential="isForCredential"
:eventSource="eventSource"
:expressionEvaluated="expressionValueComputed"
:label="label"
:data-test-id="`parameter-input-${parameter.name}`"
@focus="onFocus"
@blur="onBlur"
@ -51,6 +52,7 @@ import InputHint from '@/components/ParameterInputHint.vue';
import type {
INodeProperties,
INodePropertyMode,
IParameterLabel,
NodeParameterValue,
NodeParameterValueType,
} from 'n8n-workflow';
@ -116,6 +118,12 @@ export default defineComponent({
eventSource: {
type: String,
},
label: {
type: Object as PropType<IParameterLabel>,
default: () => ({
size: 'small',
}),
},
},
computed: {
...mapStores(useNDVStore),

View file

@ -1,26 +1,41 @@
<template>
<div :class="$style.container">
<n8n-action-toggle
v-if="shouldShowOptions"
placement="bottom-end"
size="small"
color="foreground-xdark"
iconSize="small"
:actions="actions"
@action="(action) => $emit('optionSelected', action)"
@visible-change="onMenuToggle"
/>
<n8n-radio-buttons
v-if="parameter.noDataExpression !== true && showExpressionSelector"
size="small"
:value="selectedView"
:disabled="isReadOnly"
@input="onViewSelected"
:options="[
{ label: $locale.baseText('parameterInput.fixed'), value: 'fixed' },
{ label: $locale.baseText('parameterInput.expression'), value: 'expression' },
]"
/>
<div v-if="loading" :class="$style.loader">
<n8n-text v-if="loading" size="small">
<n8n-icon icon="sync-alt" size="xsmall" :spin="true" />
{{ loadingMessage }}
</n8n-text>
</div>
<div v-else :class="$style.controlsContainer">
<div
:class="{
[$style.noExpressionSelector]: !shouldShowExpressionSelector,
}"
>
<n8n-action-toggle
v-if="shouldShowOptions"
placement="bottom-end"
size="small"
color="foreground-xdark"
iconSize="small"
:actions="actions"
:iconOrientation="iconOrientation"
@action="(action) => $emit('optionSelected', action)"
@visible-change="onMenuToggle"
/>
</div>
<n8n-radio-buttons
v-if="shouldShowExpressionSelector"
size="small"
:value="selectedView"
:disabled="isReadOnly"
@input="onViewSelected"
:options="[
{ label: $locale.baseText('parameterInput.fixed'), value: 'fixed' },
{ label: $locale.baseText('parameterInput.expression'), value: 'expression' },
]"
/>
</div>
</div>
</template>
@ -50,6 +65,25 @@ export default defineComponent({
type: Boolean,
default: true,
},
customActions: {
type: Array as PropType<Array<{ label: string; value: string; disabled?: boolean }>>,
default: () => [],
},
iconOrientation: {
type: String,
default: 'vertical',
validator: (value: string): boolean => ['horizontal', 'vertical'].includes(value),
},
loading: {
type: Boolean,
default: false,
},
loadingMessage: {
type: String,
default() {
return this.$locale.baseText('genericHelpers.loading');
},
},
},
computed: {
isDefault(): boolean {
@ -61,6 +95,9 @@ export default defineComponent({
isHtmlEditor(): boolean {
return this.getArgument('editor') === 'htmlEditor';
},
shouldShowExpressionSelector(): boolean {
return this.parameter.noDataExpression !== true && this.showExpressionSelector;
},
shouldShowOptions(): boolean {
if (this.isReadOnly === true) {
return false;
@ -91,6 +128,10 @@ export default defineComponent({
return !!this.getArgument('loadOptionsMethod') || !!this.getArgument('loadOptions');
},
actions(): Array<{ label: string; value: string; disabled?: boolean }> {
if (Array.isArray(this.customActions) && this.customActions.length > 0) {
return this.customActions;
}
if (this.isHtmlEditor && !this.isValueExpression) {
return [
{
@ -158,4 +199,19 @@ export default defineComponent({
.container {
display: flex;
}
.loader > span {
line-height: 1em;
}
.controlsContainer {
display: flex;
}
.noExpressionSelector {
margin-bottom: var(--spacing-4xs);
span {
padding-right: 0 !important;
}
}
</style>

View file

@ -0,0 +1,392 @@
<script setup lang="ts">
import type { IUpdateInformation } from '@/Interface';
import type {
FieldType,
INodeIssues,
INodeParameters,
INodeProperties,
NodePropertyTypes,
ResourceMapperField,
ResourceMapperValue,
} from 'n8n-workflow';
import ParameterInputFull from '@/components/ParameterInputFull.vue';
import ParameterIssues from '../ParameterIssues.vue';
import ParameterOptions from '../ParameterOptions.vue';
import { computed } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
import { useNDVStore } from '@/stores';
import { fieldCannotBeDeleted, isMatchingField, parseResourceMapperFieldName } from '@/utils';
import { useNodeSpecificationValues } from '@/composables';
interface Props {
parameter: INodeProperties;
path: string;
nodeValues: INodeParameters | undefined;
fieldsToMap: ResourceMapperField[];
paramValue: ResourceMapperValue;
labelSize: string;
showMatchingColumnsSelector: boolean;
showMappingModeSelect: boolean;
loading: boolean;
refreshInProgress: boolean;
}
const props = defineProps<Props>();
const FORCE_TEXT_INPUT_FOR_TYPES: FieldType[] = ['time', 'object', 'array'];
const {
resourceMapperTypeOptions,
singularFieldWord,
singularFieldWordCapitalized,
pluralFieldWord,
pluralFieldWordCapitalized,
} = useNodeSpecificationValues(props.parameter.typeOptions);
const emit = defineEmits<{
(event: 'fieldValueChanged', value: IUpdateInformation): void;
(event: 'removeField', field: string): void;
(event: 'addField', field: string): void;
(event: 'refreshFieldList'): void;
}>();
const ndvStore = useNDVStore();
const fieldsUi = computed<INodeProperties[]>(() => {
return props.fieldsToMap
.filter((field) => field.display !== false && field.removed !== true)
.map((field) => {
return {
displayName: getFieldLabel(field),
// Set part of the path to each param name so value can be fetched properly by input parameter list component
name: `value["${field.id}"]`,
type: getParamType(field),
default: field.type === 'boolean' ? false : '',
required: field.required,
description: getFieldDescription(field),
options: field.options,
};
});
});
const orderedFields = computed<INodeProperties[]>(() => {
// Sort so that matching columns are first
if (props.paramValue.matchingColumns) {
fieldsUi.value.forEach((field, i) => {
const fieldName = parseResourceMapperFieldName(field.name);
if (fieldName) {
if (props.paramValue.matchingColumns.includes(fieldName)) {
fieldsUi.value.splice(i, 1);
fieldsUi.value.unshift(field);
}
}
});
}
return fieldsUi.value;
});
const removedFields = computed<ResourceMapperField[]>(() => {
return props.fieldsToMap.filter((field) => field.removed === true && field.display !== false);
});
const addFieldOptions = computed<Array<{ name: string; value: string; disabled?: boolean }>>(() => {
return removedFields.value.map((field) => {
return {
name: field.displayName,
value: field.id,
disabled: false,
};
});
});
const parameterActions = computed<Array<{ label: string; value: string; disabled?: boolean }>>(
() => {
return [
{
label: locale.baseText('resourceMapper.refreshFieldList', {
interpolate: { fieldWord: singularFieldWordCapitalized.value },
}),
value: 'refreshFieldList',
},
{
label: locale.baseText('resourceMapper.addAllFields', {
interpolate: { fieldWord: pluralFieldWordCapitalized.value },
}),
value: 'addAllFields',
disabled: removedFields.value.length === 0,
},
{
label: locale.baseText('resourceMapper.removeAllFields', {
interpolate: { fieldWord: pluralFieldWordCapitalized.value },
}),
value: 'removeAllFields',
disabled: isRemoveAllAvailable.value === false,
},
];
},
);
const isRemoveAllAvailable = computed<boolean>(() => {
return (
removedFields.value.length !== props.fieldsToMap.length &&
props.fieldsToMap.some((field) => {
return (
field.removed !== true &&
!fieldCannotBeDeleted(
field,
props.showMatchingColumnsSelector,
resourceMapperMode.value,
props.paramValue.matchingColumns,
)
);
})
);
});
const resourceMapperMode = computed<string | undefined>(() => {
return resourceMapperTypeOptions.value?.mode;
});
const valuesLabel = computed<string>(() => {
if (resourceMapperMode.value && resourceMapperMode.value === 'update') {
return locale.baseText('resourceMapper.valuesToUpdate.label');
}
return locale.baseText('resourceMapper.valuesToSend.label');
});
const fetchingFieldsLabel = computed<string>(() => {
return locale.baseText('resourceMapper.fetchingFields.message', {
interpolate: {
fieldWord: pluralFieldWord.value,
},
});
});
function getFieldLabel(field: ResourceMapperField): string {
if (
isMatchingField(field.id, props.paramValue.matchingColumns, props.showMatchingColumnsSelector)
) {
const suffix = locale.baseText('resourceMapper.usingToMatch') || '';
return `${field.displayName} ${suffix}`;
}
return field.displayName;
}
function getFieldDescription(field: ResourceMapperField): string {
if (
isMatchingField(field.id, props.paramValue.matchingColumns, props.showMatchingColumnsSelector)
) {
return (
locale.baseText('resourceMapper.usingToMatch.description', {
interpolate: {
fieldWord: singularFieldWord.value,
},
}) || ''
);
}
return '';
}
function getParameterValue(parameterName: string) {
const fieldName = parseResourceMapperFieldName(parameterName);
if (fieldName && props.paramValue.value) {
return props.paramValue.value[fieldName];
}
return null;
}
function getFieldIssues(field: INodeProperties): string[] {
if (!ndvStore.activeNode) return [];
const nodeIssues = ndvStore.activeNode.issues || ({} as INodeIssues);
const fieldName = parseResourceMapperFieldName(field.name);
if (!fieldName) return [];
let fieldIssues: string[] = [];
const key = `${props.parameter.name}.${fieldName}`;
if (nodeIssues['parameters'] && key in nodeIssues['parameters']) {
fieldIssues = fieldIssues.concat(nodeIssues['parameters'][key]);
}
return fieldIssues;
}
function getParamType(field: ResourceMapperField): NodePropertyTypes {
if (field.type && !FORCE_TEXT_INPUT_FOR_TYPES.includes(field.type)) {
return field.type as NodePropertyTypes;
}
return 'string';
}
function onValueChanged(value: IUpdateInformation): void {
emit('fieldValueChanged', value);
}
function removeField(fieldName: string) {
emit('removeField', fieldName);
}
function addField(fieldName: string) {
emit('addField', fieldName);
}
function onParameterActionSelected(action: string): void {
switch (action) {
case 'addAllFields':
emit('addField', action);
break;
case 'removeAllFields':
emit('removeField', action);
break;
case 'refreshFieldList':
emit('refreshFieldList');
break;
default:
break;
}
}
defineExpose({
orderedFields,
});
</script>
<template>
<div class="mt-xs" data-test-id="mapping-fields-container">
<n8n-input-label
:label="valuesLabel"
:underline="true"
:size="labelSize"
:showOptions="true"
:showExpressionSelector="false"
color="text-dark"
>
<template #options>
<parameter-options
:parameter="parameter"
:customActions="parameterActions"
:loading="props.refreshInProgress"
:loadingMessage="fetchingFieldsLabel"
@optionSelected="onParameterActionSelected"
/>
</template>
</n8n-input-label>
<div v-if="orderedFields.length === 0" class="mt-3xs mb-xs">
<n8n-text size="small">{{
$locale.baseText('fixedCollectionParameter.currentlyNoItemsExist')
}}</n8n-text>
</div>
<div
v-for="field in orderedFields"
:key="field.name"
:class="{
['parameter-item']: true,
[$style.parameterItem]: true,
[$style.hasIssues]: getFieldIssues(field).length > 0,
}"
>
<div
v-if="resourceMapperMode === 'add' && field.required"
:class="['delete-option', 'mt-5xs', $style.parameterTooltipIcon]"
>
<n8n-tooltip placement="top">
<template #content>
<span>{{
locale.baseText('resourceMapper.mandatoryField.title', {
interpolate: { fieldWord: singularFieldWord },
})
}}</span>
</template>
<font-awesome-icon icon="question-circle" />
</n8n-tooltip>
</div>
<div
v-else-if="
!isMatchingField(
field.name,
props.paramValue.matchingColumns,
props.showMatchingColumnsSelector,
)
"
:class="['delete-option', 'clickable', 'mt-5xs']"
>
<font-awesome-icon
icon="trash"
:title="
locale.baseText('resourceMapper.removeField', {
interpolate: {
fieldWord: singularFieldWordCapitalized,
},
})
"
data-test-id="remove-field-button"
@click="removeField(field.name)"
/>
</div>
<div :class="$style.parameterInput">
<parameter-input-full
:parameter="field"
:value="getParameterValue(field.name)"
:displayOptions="true"
:path="`${props.path}.${field.name}`"
:isReadOnly="refreshInProgress"
:hideIssues="true"
:nodeValues="nodeValues"
:class="$style.parameterInputFull"
@valueChanged="onValueChanged"
/>
</div>
<parameter-issues
v-if="getFieldIssues(field).length > 0"
:issues="getFieldIssues(field)"
:class="[$style.parameterIssues, 'ml-5xs']"
/>
</div>
<div class="add-option" data-test-id="add-fields-select">
<n8n-select
:placeholder="
locale.baseText('resourceMapper.addFieldToSend', {
interpolate: { fieldWord: singularFieldWordCapitalized },
})
"
size="small"
:disabled="addFieldOptions.length == 0"
@change="addField"
>
<n8n-option
v-for="item in addFieldOptions"
:key="item.value"
:label="item.name"
:value="item.value"
:disabled="item.disabled"
>
</n8n-option>
</n8n-select>
</div>
</div>
</template>
<style module lang="scss">
.parameterItem {
display: flex;
padding: 0 0 0 1em;
.parameterInput {
width: 100%;
}
&.hasIssues {
.parameterIssues {
float: none;
padding-top: var(--spacing-xl);
}
input,
input:focus {
--input-border-color: var(--color-danger);
border-color: var(--color-danger);
}
}
}
.parameterTooltipIcon {
color: var(--color-text-light) !important;
}
</style>

View file

@ -0,0 +1,153 @@
<script setup lang="ts">
import type { INodePropertyTypeOptions, ResourceMapperFields } from 'n8n-workflow';
import { computed, ref, watch } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
import { useNodeSpecificationValues } from '@/composables';
interface Props {
initialValue: string;
fieldsToMap: ResourceMapperFields['fields'];
inputSize: string;
labelSize: string;
typeOptions: INodePropertyTypeOptions | undefined;
serviceName: string;
loading: boolean;
loadingError: boolean;
}
const props = defineProps<Props>();
const { resourceMapperTypeOptions, pluralFieldWord, singularFieldWord } =
useNodeSpecificationValues(props.typeOptions);
// Mapping mode options: Labels here use field words defined in parameter type options
const mappingModeOptions = [
{
name: locale.baseText('resourceMapper.mappingMode.defineBelow.name'),
value: 'defineBelow',
description: locale.baseText('resourceMapper.mappingMode.defineBelow.description', {
interpolate: {
fieldWord: singularFieldWord.value,
},
}),
},
{
name: locale.baseText('resourceMapper.mappingMode.autoMapInputData.name'),
value: 'autoMapInputData',
description: locale.baseText('resourceMapper.mappingMode.autoMapInputData.description', {
interpolate: {
fieldWord: pluralFieldWord.value,
serviceName: props.serviceName,
},
}),
},
];
const emit = defineEmits<{
(event: 'modeChanged', value: string): void;
(event: 'retryFetch'): void;
}>();
const selected = ref(props.initialValue);
watch(
() => props.initialValue,
() => {
selected.value = props.initialValue;
},
);
const errorMessage = computed<string>(() => {
if (selected.value === 'defineBelow') {
// Loading error message
if (props.loadingError) {
return locale.baseText('resourceMapper.fetchingFields.errorMessage', {
interpolate: {
fieldWord: pluralFieldWord.value,
},
});
}
// No data error message
if (props.fieldsToMap.length === 0) {
return (
// Use custom error message if defined
resourceMapperTypeOptions.value?.noFieldsError ||
locale.baseText('resourceMapper.fetchingFields.noFieldsFound', {
interpolate: {
fieldWord: pluralFieldWord.value,
serviceName: props.serviceName,
},
})
);
}
return '';
}
return '';
});
function onModeChanged(value: string): void {
selected.value = value;
emit('modeChanged', value);
}
function onRetryClick(): void {
emit('retryFetch');
}
defineExpose({
selected,
onModeChanged,
mappingModeOptions,
});
</script>
<template>
<div data-test-id="mapping-mode-select">
<n8n-input-label
:label="locale.baseText('resourceMapper.mappingMode.label')"
:bold="false"
:required="false"
:size="labelSize"
color="text-dark"
>
<template>
<div class="mt-5xs">
<n8n-select :value="selected" :size="props.inputSize" @change="onModeChanged">
<n8n-option
v-for="option in mappingModeOptions"
:key="option.value"
:value="option.value"
:label="option.name"
description="sadasd"
>
<div class="list-option">
<div class="option-headline">
{{ option.name }}
</div>
<div class="option-description" v-html="option.description" />
</div>
</n8n-option>
</n8n-select>
</div>
<div class="mt-5xs">
<n8n-text v-if="loading" size="small">
<n8n-icon icon="sync-alt" size="xsmall" :spin="true" />
{{
locale.baseText('resourceMapper.fetchingFields.message', {
interpolate: {
fieldWord: pluralFieldWord,
},
})
}}
</n8n-text>
<n8n-text v-else-if="errorMessage !== ''" size="small" color="danger">
<n8n-icon icon="exclamation-triangle" size="xsmall" />
{{ errorMessage }}
<n8n-link size="small" theme="danger" :underline="true" @click="onRetryClick">
{{ locale.baseText('generic.retry') }}
</n8n-link>
</n8n-text>
</div>
</template>
</n8n-input-label>
</div>
</template>

View file

@ -0,0 +1,155 @@
<script setup lang="ts">
import type {
INodePropertyTypeOptions,
ResourceMapperField,
ResourceMapperFields,
} from 'n8n-workflow';
import { computed, reactive, watch } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
import { useNodeSpecificationValues } from '@/composables';
interface Props {
initialValue: string[];
fieldsToMap: ResourceMapperFields['fields'];
typeOptions: INodePropertyTypeOptions | undefined;
labelSize: string;
inputSize: string;
loading: boolean;
serviceName: string;
}
const props = defineProps<Props>();
const {
resourceMapperTypeOptions,
singularFieldWord,
singularFieldWordCapitalized,
pluralFieldWord,
pluralFieldWordCapitalized,
} = useNodeSpecificationValues(props.typeOptions);
// Depending on the mode (multiple/singe key column), the selected value can be a string or an array of strings
const state = reactive({
selected: props.initialValue as string[] | string,
});
watch(
() => props.initialValue,
() => {
state.selected =
resourceMapperTypeOptions.value?.multiKeyMatch === true
? props.initialValue
: props.initialValue[0];
},
);
const emit = defineEmits<{
(event: 'matchingColumnsChanged', value: string[]): void;
}>();
const availableMatchingFields = computed<ResourceMapperField[]>(() => {
return props.fieldsToMap.filter((field) => {
return field.canBeUsedToMatch !== false && field.display !== false;
});
});
// Field label, description and tooltip: Labels here use content and field words defined in parameter type options
const fieldLabel = computed<string>(() => {
if (resourceMapperTypeOptions.value?.matchingFieldsLabels?.title) {
return resourceMapperTypeOptions.value.matchingFieldsLabels.title;
}
const fieldWord =
resourceMapperTypeOptions.value?.multiKeyMatch === true
? pluralFieldWordCapitalized.value
: singularFieldWordCapitalized.value;
return locale.baseText('resourceMapper.columnsToMatchOn.label', {
interpolate: {
fieldWord,
},
});
});
const fieldDescription = computed<string>(() => {
if (resourceMapperTypeOptions.value?.matchingFieldsLabels?.hint) {
return resourceMapperTypeOptions.value.matchingFieldsLabels.hint;
}
const labeli18nKey =
resourceMapperTypeOptions.value?.multiKeyMatch === true
? 'resourceMapper.columnsToMatchOn.multi.description'
: 'resourceMapper.columnsToMatchOn.single.description';
return locale.baseText(labeli18nKey, {
interpolate: {
fieldWord:
resourceMapperTypeOptions.value?.multiKeyMatch === true
? `${pluralFieldWord.value}`
: `${singularFieldWord.value}`,
},
});
});
const fieldTooltip = computed<string>(() => {
if (resourceMapperTypeOptions.value?.matchingFieldsLabels?.description) {
return resourceMapperTypeOptions.value.matchingFieldsLabels.description;
}
return locale.baseText('resourceMapper.columnsToMatchOn.tooltip', {
interpolate: {
fieldWord:
resourceMapperTypeOptions.value?.multiKeyMatch === true
? `${pluralFieldWord.value}`
: `${singularFieldWord.value}`,
},
});
});
function onSelectionChange(value: string | string[]) {
if (resourceMapperTypeOptions.value?.multiKeyMatch === true) {
state.selected = value as string[];
} else {
state.selected = value as string;
}
emitValueChanged();
}
function emitValueChanged() {
emit('matchingColumnsChanged', Array.isArray(state.selected) ? state.selected : [state.selected]);
}
defineExpose({
state,
});
</script>
<template>
<div class="mt-2xs" data-test-id="matching-column-select">
<n8n-input-label
v-if="availableMatchingFields.length > 0"
:label="fieldLabel"
:tooltipText="fieldTooltip"
:bold="false"
:required="false"
:size="labelSize"
color="text-dark"
>
<n8n-select
:multiple="resourceMapperTypeOptions?.multiKeyMatch === true"
:value="state.selected"
:size="props.inputSize"
:disabled="loading"
@change="onSelectionChange"
>
<n8n-option v-for="field in availableMatchingFields" :key="field.id" :value="field.id">
{{ field.displayName }}
</n8n-option>
</n8n-select>
<n8n-text size="small">
{{ fieldDescription }}
</n8n-text>
</n8n-input-label>
<n8n-notice v-else>
{{
locale.baseText('resourceMapper.columnsToMatchOn.noFieldsFound', {
interpolate: { fieldWord: singularFieldWord, serviceName: props.serviceName },
})
}}
</n8n-notice>
</div>
</template>

View file

@ -0,0 +1,500 @@
<script setup lang="ts">
import type { IUpdateInformation, ResourceMapperReqParams } from '@/Interface';
import { resolveParameter } from '@/mixins/workflowHelpers';
import { useNodeTypesStore } from '@/stores/nodeTypes.store';
import type {
INode,
INodeParameters,
INodeProperties,
INodeTypeDescription,
ResourceMapperField,
ResourceMapperValue,
} from 'n8n-workflow';
import { NodeHelpers } from 'n8n-workflow';
import { computed, onMounted, reactive, watch } from 'vue';
import MappingModeSelect from './MappingModeSelect.vue';
import MatchingColumnsSelect from './MatchingColumnsSelect.vue';
import MappingFields from './MappingFields.vue';
import { fieldCannotBeDeleted, isResourceMapperValue, parseResourceMapperFieldName } from '@/utils';
import { i18n as locale } from '@/plugins/i18n';
import { useNDVStore } from '@/stores/ndv.store';
interface Props {
parameter: INodeProperties;
node: INode | null;
path: string;
inputSize: string;
labelSize: string;
dependentParametersValues: string | null;
}
const nodeTypesStore = useNodeTypesStore();
const ndvStore = useNDVStore();
const props = defineProps<Props>();
const emit = defineEmits<{
(event: 'valueChanged', value: IUpdateInformation): void;
}>();
const state = reactive({
paramValue: {
mappingMode: 'defineBelow',
value: {},
matchingColumns: [] as string[],
schema: [] as ResourceMapperField[],
} as ResourceMapperValue,
parameterValues: {} as INodeParameters,
loading: false,
refreshInProgress: false, // Shows inline loader when refreshing fields
loadingError: false,
});
// Reload fields to map when dependent parameters change
watch(
() => props.dependentParametersValues,
async (currentValue, oldValue) => {
if (oldValue !== null && currentValue !== null && oldValue !== currentValue) {
state.paramValue = {
...state.paramValue,
value: null,
schema: [],
};
emitValueChanged();
await initFetching();
setDefaultFieldValues(true);
}
},
);
onMounted(async () => {
if (props.node) {
state.parameterValues = {
...state.parameterValues,
parameters: props.node.parameters,
};
}
const params = state.parameterValues.parameters as INodeParameters;
const parameterName = props.parameter.name;
if (!(parameterName in params)) {
return;
}
let hasSchema = false;
const nodeValues = params[parameterName] as unknown as ResourceMapperValue;
state.paramValue = {
...state.paramValue,
...nodeValues,
};
if (!state.paramValue.schema) {
state.paramValue = {
...state.paramValue,
schema: [],
};
} else {
hasSchema = state.paramValue.schema.length > 0;
}
Object.keys(state.paramValue.value || {}).forEach((key) => {
if (state.paramValue.value && state.paramValue.value[key] === '') {
state.paramValue = {
...state.paramValue,
value: {
...state.paramValue.value,
[key]: null,
},
};
}
});
if (nodeValues.matchingColumns) {
state.paramValue = {
...state.paramValue,
matchingColumns: nodeValues.matchingColumns,
};
}
await initFetching(hasSchema);
// Set default values if this is the first time the parameter is being set
if (!state.paramValue.value) {
setDefaultFieldValues();
}
updateNodeIssues();
});
const resourceMapperMode = computed<string | undefined>(() => {
return props.parameter.typeOptions?.resourceMapper?.mode;
});
const nodeType = computed<INodeTypeDescription | null>(() => {
if (props.node) {
return nodeTypesStore.getNodeType(props.node.type, props.node.typeVersion);
}
return null;
});
const showMappingModeSelect = computed<boolean>(() => {
return props.parameter.typeOptions?.resourceMapper?.supportAutoMap !== false;
});
const showMatchingColumnsSelector = computed<boolean>(() => {
return (
!state.loading &&
props.parameter.typeOptions?.resourceMapper?.mode !== 'add' &&
state.paramValue.schema.length > 0
);
});
const showMappingFields = computed<boolean>(() => {
return (
state.paramValue.mappingMode === 'defineBelow' &&
!state.loading &&
!state.loadingError &&
state.paramValue.schema.length > 0 &&
hasAvailableMatchingColumns.value
);
});
const matchingColumns = computed<string[]>(() => {
if (!showMatchingColumnsSelector.value) {
return [];
}
if (state.paramValue.matchingColumns) {
return state.paramValue.matchingColumns;
}
return defaultSelectedMatchingColumns.value;
});
const hasAvailableMatchingColumns = computed<boolean>(() => {
if (resourceMapperMode.value !== 'add') {
return (
state.paramValue.schema.filter(
(field) =>
field.canBeUsedToMatch !== false && field.display !== false && field.removed !== true,
).length > 0
);
}
return true;
});
const defaultSelectedMatchingColumns = computed<string[]>(() => {
return state.paramValue.schema.length === 1
? [state.paramValue.schema[0].id]
: state.paramValue.schema.reduce((acc, field) => {
if (field.defaultMatch && field.canBeUsedToMatch === true) {
acc.push(field.id);
}
return acc;
}, [] as string[]);
});
const pluralFieldWord = computed<string>(() => {
return (
props.parameter.typeOptions?.resourceMapper?.fieldWords?.plural ||
locale.baseText('generic.fields')
);
});
async function initFetching(inlineLading = false): Promise<void> {
state.loadingError = false;
if (inlineLading) {
state.refreshInProgress = true;
} else {
state.loading = true;
}
try {
await loadFieldsToMap();
if (!state.paramValue.matchingColumns || state.paramValue.matchingColumns.length === 0) {
onMatchingColumnsChanged(defaultSelectedMatchingColumns.value);
}
} catch (error) {
state.loadingError = true;
} finally {
state.loading = false;
state.refreshInProgress = false;
}
}
async function loadFieldsToMap(): Promise<void> {
if (!props.node) {
return;
}
const requestParams: ResourceMapperReqParams = {
nodeTypeAndVersion: {
name: props.node?.type,
version: props.node.typeVersion,
},
currentNodeParameters: resolveParameter(props.node.parameters) as INodeParameters,
path: props.path,
methodName: props.parameter.typeOptions?.resourceMapper?.resourceMapperMethod,
credentials: props.node.credentials,
};
const fetchedFields = await nodeTypesStore.getResourceMapperFields(requestParams);
if (fetchedFields !== null) {
const newSchema = fetchedFields.fields.map((field) => {
const existingField = state.paramValue.schema.find((f) => f.id === field.id);
if (existingField) {
field.removed = existingField.removed;
} else if (state.paramValue.value !== null && !(field.id in state.paramValue.value)) {
// New fields are shown by default
field.removed = false;
}
return field;
});
state.paramValue = {
...state.paramValue,
schema: newSchema,
};
emitValueChanged();
}
}
async function onModeChanged(mode: string): Promise<void> {
state.paramValue.mappingMode = mode;
if (mode === 'defineBelow') {
await initFetching();
} else {
state.loadingError = false;
}
emitValueChanged();
}
function setDefaultFieldValues(forceMatchingFieldsUpdate = false): void {
state.paramValue.value = {};
const hideAllFields = props.parameter.typeOptions?.resourceMapper?.addAllFields === false;
state.paramValue.schema.forEach((field) => {
if (state.paramValue.value) {
// Hide all non-required fields if it's configured in node definition
if (hideAllFields) {
field.removed = !field.required;
}
if (field.type === 'boolean') {
state.paramValue.value = {
...state.paramValue.value,
[field.id]: false,
};
} else {
state.paramValue.value = {
...state.paramValue.value,
[field.id]: null,
};
}
}
});
emitValueChanged();
if (!state.paramValue.matchingColumns || forceMatchingFieldsUpdate) {
state.paramValue.matchingColumns = defaultSelectedMatchingColumns.value;
emitValueChanged();
}
}
function updateNodeIssues(): void {
if (props.node) {
const parameterIssues = NodeHelpers.getNodeParametersIssues(
nodeType.value?.properties || [],
props.node,
);
if (parameterIssues) {
ndvStore.updateNodeParameterIssues(parameterIssues);
}
}
}
function onMatchingColumnsChanged(matchingColumns: string[]): void {
state.paramValue = {
...state.paramValue,
matchingColumns,
};
// Set all matching fields to be visible
state.paramValue.schema.forEach((field) => {
if (state.paramValue.matchingColumns?.includes(field.id)) {
field.removed = false;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
});
if (!state.loading) {
emitValueChanged();
}
}
function fieldValueChanged(updateInfo: IUpdateInformation): void {
let newValue = null;
if (
updateInfo.value !== undefined &&
updateInfo.value !== '' &&
updateInfo.value !== null &&
isResourceMapperValue(updateInfo.value)
) {
newValue = updateInfo.value;
}
const fieldName = parseResourceMapperFieldName(updateInfo.name);
if (fieldName && state.paramValue.value) {
state.paramValue.value = {
...state.paramValue.value,
[fieldName]: newValue,
};
emitValueChanged();
}
}
function removeField(name: string): void {
if (name === 'removeAllFields') {
return removeAllFields();
}
const fieldName = parseResourceMapperFieldName(name);
if (fieldName) {
if (state.paramValue.value) {
delete state.paramValue.value[fieldName];
const field = state.paramValue.schema.find((f) => f.id === fieldName);
if (field) {
field.removed = true;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
emitValueChanged();
}
}
}
function addField(name: string): void {
if (name === 'addAllFields') {
return addAllFields();
}
if (name === 'removeAllFields') {
return removeAllFields();
}
state.paramValue.value = {
...state.paramValue.value,
[name]: null,
};
const field = state.paramValue.schema.find((f) => f.id === name);
if (field) {
field.removed = false;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
emitValueChanged();
}
function addAllFields(): void {
const newValues: { [name: string]: null } = {};
state.paramValue.schema.forEach((field) => {
if (field.display && field.removed) {
newValues[field.id] = null;
field.removed = false;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
});
state.paramValue.value = {
...state.paramValue.value,
...newValues,
};
emitValueChanged();
}
function removeAllFields(): void {
state.paramValue.schema.forEach((field) => {
if (
!fieldCannotBeDeleted(
field,
showMatchingColumnsSelector.value,
resourceMapperMode.value,
matchingColumns.value,
)
) {
field.removed = true;
state.paramValue.schema.splice(state.paramValue.schema.indexOf(field), 1, field);
}
});
emitValueChanged();
}
function emitValueChanged(): void {
pruneParamValues();
emit('valueChanged', {
name: `${props.path}`,
value: state.paramValue,
node: props.node?.name,
});
updateNodeIssues();
}
function pruneParamValues(): void {
if (!state.paramValue.value) {
return;
}
const valueKeys = Object.keys(state.paramValue.value);
valueKeys.forEach((key) => {
if (state.paramValue.value && state.paramValue.value[key] === null) {
delete state.paramValue.value[key];
}
});
}
defineExpose({
state,
});
</script>
<template>
<div class="mt-4xs" data-test-id="resource-mapper-container">
<mapping-mode-select
v-if="showMappingModeSelect"
:inputSize="inputSize"
:labelSize="labelSize"
:initialValue="state.paramValue.mappingMode || 'defineBelow'"
:typeOptions="props.parameter.typeOptions"
:serviceName="nodeType?.displayName || locale.baseText('generic.service')"
:loading="state.loading"
:loadingError="state.loadingError"
:fieldsToMap="state.paramValue.schema"
@modeChanged="onModeChanged"
@retryFetch="initFetching"
/>
<matching-columns-select
v-if="showMatchingColumnsSelector"
:label-size="labelSize"
:fieldsToMap="state.paramValue.schema"
:typeOptions="props.parameter.typeOptions"
:inputSize="inputSize"
:loading="state.loading"
:initialValue="matchingColumns"
:serviceName="nodeType?.displayName || locale.baseText('generic.service')"
@matchingColumnsChanged="onMatchingColumnsChanged"
/>
<n8n-text v-if="!showMappingModeSelect && state.loading" size="small">
<n8n-icon icon="sync-alt" size="xsmall" :spin="true" />
{{
locale.baseText('resourceMapper.fetchingFields.message', {
interpolate: {
fieldWord: pluralFieldWord,
},
})
}}
</n8n-text>
<mapping-fields
v-if="showMappingFields"
:parameter="props.parameter"
:path="props.path"
:nodeValues="state.parameterValues"
:fieldsToMap="state.paramValue.schema"
:paramValue="state.paramValue"
:labelSize="labelSize"
:showMatchingColumnsSelector="showMatchingColumnsSelector"
:showMappingModeSelect="showMappingModeSelect"
:loading="state.loading"
:refreshInProgress="state.refreshInProgress"
@fieldValueChanged="fieldValueChanged"
@removeField="removeField"
@addField="addField"
@refreshFieldList="initFetching(true)"
/>
<n8n-notice
v-if="state.paramValue.mappingMode === 'autoMapInputData' && hasAvailableMatchingColumns"
>
{{
locale.baseText('resourceMapper.autoMappingNotice', {
interpolate: {
fieldWord: pluralFieldWord,
serviceName: nodeType?.displayName || locale.baseText('generic.service'),
},
})
}}
</n8n-notice>
</div>
</template>

View file

@ -0,0 +1,232 @@
import { PiniaVuePlugin } from 'pinia';
import { render, within } from '@testing-library/vue';
import { merge } from 'lodash-es';
import {
DEFAULT_SETUP,
MAPPING_COLUMNS_RESPONSE,
NODE_PARAMETER_VALUES,
UPDATED_SCHEMA,
} from './utils/ResourceMapper.utils';
import { useNodeTypesStore } from '@/stores/nodeTypes.store';
import { waitAllPromises } from '@/__tests__/utils';
import * as workflowHelpers from '@/mixins/workflowHelpers';
import ResourceMapper from '@/components/ResourceMapper/ResourceMapper.vue';
import userEvent from '@testing-library/user-event';
let nodeTypeStore: ReturnType<typeof useNodeTypesStore>;
const renderComponent = (renderOptions: Parameters<typeof render>[1] = {}) =>
render(ResourceMapper, merge(DEFAULT_SETUP, renderOptions), (vue) => {
vue.use(PiniaVuePlugin);
});
describe('ResourceMapper.vue', () => {
beforeEach(() => {
nodeTypeStore = useNodeTypesStore();
vi.spyOn(workflowHelpers, 'resolveParameter').mockReturnValue(NODE_PARAMETER_VALUES);
vi.spyOn(nodeTypeStore, 'getResourceMapperFields').mockResolvedValue(MAPPING_COLUMNS_RESPONSE);
});
afterEach(() => {
vi.clearAllMocks();
});
it('renders default configuration properly', async () => {
const { getByTestId } = renderComponent();
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
expect(getByTestId('mapping-mode-select')).toBeInTheDocument();
expect(getByTestId('matching-column-select')).toBeInTheDocument();
expect(getByTestId('mapping-fields-container')).toBeInTheDocument();
// Should render one parameter input for each fetched column
expect(
getByTestId('mapping-fields-container').querySelectorAll('.parameter-input').length,
).toBe(MAPPING_COLUMNS_RESPONSE.fields.length);
});
it('renders add mode properly', async () => {
const { getByTestId, queryByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
mode: 'add',
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
// This mode doesn't render matching column selector
expect(queryByTestId('matching-column-select')).not.toBeInTheDocument();
});
it('renders multi-key match selector properly', async () => {
const { container, getByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
mode: 'upsert',
multiKeyMatch: true,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
expect(container.querySelector('.el-select__tags')).toBeInTheDocument();
});
it('does not render mapping mode selector if it is disabled', async () => {
const { getByTestId, queryByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
supportAutoMap: false,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
expect(queryByTestId('mapping-mode-select')).not.toBeInTheDocument();
});
it('renders field on top of the list when they are selected for matching', async () => {
const { container, getByTestId } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
supportAutoMap: true,
mode: 'upsert',
multiKeyMatch: false,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
// Id should be the first field in the list
expect(container.querySelector('.parameter-item')).toContainHTML('id (using to match)');
// // Select Last Name as matching column
await userEvent.click(getByTestId('matching-column-select'));
const matchingColumnDropdown = getByTestId('matching-column-select');
await userEvent.click(within(matchingColumnDropdown).getByText('Last Name'));
// // Now, last name should be the first field in the list
expect(container.querySelector('.parameter-item')).toContainHTML('Last Name (using to match)');
});
it('renders selected matching columns properly when multiple key matching is enabled', async () => {
const { getByTestId, getByText, queryByText } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
supportAutoMap: true,
mode: 'upsert',
multiKeyMatch: true,
},
},
},
},
});
await waitAllPromises();
expect(getByTestId('resource-mapper-container')).toBeInTheDocument();
const matchingColumnDropdown = getByTestId('matching-column-select');
await userEvent.click(matchingColumnDropdown);
await userEvent.click(within(matchingColumnDropdown).getByText('Username'));
// Both matching columns should be rendered in the dropdown
expect(getByTestId('matching-column-select')).toContainHTML(
'<span class="el-select__tags-text">id</span>',
);
expect(getByTestId('matching-column-select')).toContainHTML(
'<span class="el-select__tags-text">Username</span>',
);
// All selected columns should have correct labels
expect(getByText('id (using to match)')).toBeInTheDocument();
expect(getByText('Username (using to match)')).toBeInTheDocument();
expect(queryByText('First Name (using to match)')).not.toBeInTheDocument();
});
it('uses field words defined in node definition', async () => {
const { getByText } = renderComponent({
props: {
parameter: {
typeOptions: {
resourceMapper: {
fieldWords: {
singular: 'foo',
plural: 'foos',
},
},
},
},
},
});
await waitAllPromises();
expect(getByText('Set the value for each foo')).toBeInTheDocument();
expect(
getByText('Look for incoming data that matches the foos in the service'),
).toBeInTheDocument();
expect(getByText('Foos to Match On')).toBeInTheDocument();
expect(getByText('The foos that identify the row(s) to modify')).toBeInTheDocument();
});
it('should render correct fields based on saved schema', async () => {
const { getByTestId, queryAllByTestId } = renderComponent({
props: {
node: {
parameters: {
columns: {
schema: UPDATED_SCHEMA,
},
},
},
parameter: {
typeOptions: {
resourceMapper: {
mode: 'add',
},
},
},
},
});
await waitAllPromises();
// There should be 5 fields rendered and only 2 of them should have remove button
expect(
getByTestId('mapping-fields-container').querySelectorAll('.parameter-input').length,
).toBe(5);
expect(queryAllByTestId('remove-field-button').length).toBe(2);
});
it('should render correct options based on saved schema', async () => {
const { getByTestId } = renderComponent({
props: {
node: {
parameters: {
columns: {
schema: UPDATED_SCHEMA,
},
},
},
parameter: {
typeOptions: {
resourceMapper: {
mode: 'add',
},
},
},
},
});
await waitAllPromises();
// Should have one option in the bottom dropdown for one removed field
expect(getByTestId('add-fields-select').querySelectorAll('li').length).toBe(1);
});
});

View file

@ -0,0 +1,214 @@
import { SETTINGS_STORE_DEFAULT_STATE } from '@/__tests__/utils';
import { STORES } from '@/constants';
import { createTestingPinia } from '@pinia/testing';
import { merge } from 'lodash-es';
import type { ResourceMapperFields } from 'n8n-workflow';
export const NODE_PARAMETER_VALUES = {
authentication: 'oAuth2',
resource: 'sheet',
operation: 'appendOrUpdate',
documentId: {
__rl: true,
value:
'https://docs.google.com/spreadsheets/d/1BAjxEhlUu5tXDCMQcjqjguIZDFuct3FYkdo7flxl3yc/edit#gid=0',
mode: 'url',
__regex: 'https:\\/\\/(?:drive|docs)\\.google\\.com\\/\\w+\\/d\\/([0-9a-zA-Z\\-_]+)(?:\\/.*|)',
},
sheetName: {
__rl: true,
value: 'gid=0',
mode: 'list',
cachedResultName: 'Users',
cachedResultUrl:
'https://docs.google.com/spreadsheets/d/1BAjxEhlUu5tXDCMQcjqjguIZDFuct3FYkdo7flxl3yc/edit#gid=0',
},
columns: {
mappingMode: 'defineBelow',
value: null,
},
options: {},
};
export const UPDATED_SCHEMA = [
{
id: 'First name',
displayName: 'First name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
{
id: 'Last name',
displayName: 'Last name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
{
id: 'Username',
displayName: 'Username',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
{
id: 'Address',
displayName: 'Address',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: true,
},
{
id: 'id',
displayName: 'id',
match: false,
required: true,
defaultMatch: true,
display: true,
type: 'string',
canBeUsedToMatch: true,
removed: false,
},
];
export const DEFAULT_SETUP = {
pinia: createTestingPinia({
initialState: {
[STORES.SETTINGS]: {
settings: merge({}, SETTINGS_STORE_DEFAULT_STATE.settings),
},
},
}),
props: {
path: 'parameters.columns',
dependentParametersValues: 'gid=0',
inputSize: 'small',
labelSize: 'small',
node: {
parameters: NODE_PARAMETER_VALUES,
id: 'f63efb2d-3cc5-4500-89f9-b39aab19baf5',
name: 'Google Sheets',
type: 'n8n-nodes-base.googleSheets',
typeVersion: 4,
position: [1120, 380],
credentials: {},
disabled: false,
},
parameter: {
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
default: {
mappingMode: 'defineBelow',
value: {},
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'upsert',
addAllFields: true,
noFieldsError: 'No columns found in sheet.',
multiKeyMatch: false,
fieldWords: {
singular: 'column',
plural: 'columns',
},
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['appendOrUpdate'],
'@version': [4],
},
hide: {
sheetName: [''],
},
},
},
},
};
export const MAPPING_COLUMNS_RESPONSE: ResourceMapperFields = {
fields: [
{
id: 'First name',
displayName: 'First name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Last name',
displayName: 'Last name',
match: false,
required: true,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Username',
displayName: 'Username',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Address',
displayName: 'Address',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'id',
displayName: 'id',
match: true,
required: true,
defaultMatch: true,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
{
id: 'Last Name',
displayName: 'Last Name',
match: false,
required: false,
defaultMatch: false,
display: true,
type: 'string',
canBeUsedToMatch: true,
},
],
};

View file

@ -11,3 +11,4 @@ export * from './useTelemetry';
export * from './useTitleChange';
export * from './useToast';
export * from './useUpgradeLink';
export * from './useNodeSpecificationValues';

View file

@ -0,0 +1,35 @@
import type { INodePropertyTypeOptions, ResourceMapperTypeOptions } from 'n8n-workflow';
import { computed } from 'vue';
import { i18n as locale } from '@/plugins/i18n';
export function useNodeSpecificationValues(typeOptions: INodePropertyTypeOptions | undefined) {
const resourceMapperTypeOptions = computed<ResourceMapperTypeOptions | undefined>(() => {
return typeOptions?.resourceMapper;
});
const singularFieldWord = computed<string>(() => {
const singularFieldWord =
resourceMapperTypeOptions.value?.fieldWords?.singular || locale.baseText('generic.field');
return singularFieldWord;
});
const singularFieldWordCapitalized = computed<string>(() => {
return singularFieldWord.value.charAt(0).toUpperCase() + singularFieldWord.value.slice(1);
});
const pluralFieldWord = computed<string>(() => {
return resourceMapperTypeOptions.value?.fieldWords?.plural || locale.baseText('generic.fields');
});
const pluralFieldWordCapitalized = computed<string>(() => {
return pluralFieldWord.value.charAt(0).toUpperCase() + pluralFieldWord.value.slice(1);
});
return {
resourceMapperTypeOptions,
singularFieldWord,
singularFieldWordCapitalized,
pluralFieldWord,
pluralFieldWordCapitalized,
};
}

View file

@ -21,6 +21,8 @@
"generic.confirm": "Confirm",
"generic.deleteWorkflowError": "Problem deleting workflow",
"generic.filtersApplied": "Filters are currently applied.",
"generic.field": "field",
"generic.fields": "fields",
"generic.learnMore": "Learn more",
"generic.reset": "Reset",
"generic.resetAllFilters": "Reset all filters",
@ -35,7 +37,9 @@
"generic.beta": "beta",
"generic.yes": "Yes",
"generic.no": "No",
"generic.retry": "Retry",
"generic.settings": "Settings",
"generic.service": "the service",
"generic.unsavedWork.confirmMessage.headline": "Save changes before leaving?",
"generic.unsavedWork.confirmMessage.message": "If you don't save, you will lose your changes.",
"generic.unsavedWork.confirmMessage.confirmButtonText": "Save",
@ -1054,6 +1058,31 @@
"resourceLocator.openResource": "Open in {appName}",
"resourceLocator.search.placeholder": "Search...",
"resourceLocator.url.placeholder": "Enter URL...",
"resourceMapper.autoMappingNotice": "In this mode, make sure the incoming data fields are named the same as the {fieldWord} in {serviceName}. (Use a 'set' node before this node to change them if required.)",
"resourceMapper.mappingMode.label": "Mapping Column Mode",
"resourceMapper.mappingMode.defineBelow.name": "Map Each Column Manually",
"resourceMapper.mappingMode.defineBelow.description": "Set the value for each {fieldWord}",
"resourceMapper.mappingMode.autoMapInputData.name": "Map Automatically",
"resourceMapper.mappingMode.autoMapInputData.description": "Look for incoming data that matches the {fieldWord} in {serviceName}",
"resourceMapper.fetchingFields.message": "Fetching {fieldWord}",
"resourceMapper.fetchingFields.errorMessage": "Can't get {fieldWord}.",
"resourceMapper.fetchingFields.noFieldsFound": "No {fieldWord} found in {serviceName}.",
"resourceMapper.columnsToMatchOn.label": "{fieldWord} to Match On",
"resourceMapper.columnsToMatchOn.multi.description": "The {fieldWord} that identify the row(s) to modify",
"resourceMapper.columnsToMatchOn.single.description": "The {fieldWord} that identifies the row(s) to modify",
"resourceMapper.columnsToMatchOn.tooltip": "The {fieldWord} to compare when finding the rows to update",
"resourceMapper.columnsToMatchOn.noFieldsFound": "No {fieldWord} that can be used for matching found in {serviceName}.",
"resourceMapper.valuesToSend.label": "Values to Send",
"resourceMapper.valuesToUpdate.label": "Values to Update",
"resourceMapper.usingToMatch": "(using to match)",
"resourceMapper.usingToMatch.description": "This {fieldWord} won't be updated and can't be removed, as it's used for matching",
"resourceMapper.removeField": "Remove {fieldWord}",
"resourceMapper.mandatoryField.title": "This {fieldWord} is mandatory and cant be removed",
"resourceMapper.addFieldToSend": "Add {fieldWord} to Send",
"resourceMapper.matching.title": "This {fieldWord} is used for matching and cant be removed",
"resourceMapper.addAllFields": "Add All {fieldWord}",
"resourceMapper.removeAllFields": "Remove All {fieldWord}",
"resourceMapper.refreshFieldList": "Refresh {fieldWord} List",
"runData.emptyItemHint": "This is an item, but it's empty.",
"runData.emptyArray": "[empty array]",
"runData.emptyString": "[empty]",

View file

@ -39,6 +39,7 @@ import {
faCut,
faDotCircle,
faEdit,
faEllipsisH,
faEllipsisV,
faEnvelope,
faEye,
@ -176,6 +177,7 @@ addIcon(faCut);
addIcon(faDotCircle);
addIcon(faGripVertical);
addIcon(faEdit);
addIcon(faEllipsisH);
addIcon(faEllipsisV);
addIcon(faEnvelope);
addIcon(faEye);

View file

@ -6,7 +6,7 @@ import type {
NodePanelType,
XYPosition,
} from '@/Interface';
import type { IRunData } from 'n8n-workflow';
import type { INodeIssues, IRunData } from 'n8n-workflow';
import { defineStore } from 'pinia';
import Vue from 'vue';
import { useWorkflowsStore } from './workflows.store';
@ -206,5 +206,14 @@ export const useNDVStore = defineStore(STORES.NDV, {
window.localStorage.setItem(LOCAL_STORAGE_MAPPING_IS_ONBOARDED, 'true');
}
},
updateNodeParameterIssues(issues: INodeIssues): void {
const activeNode = this.activeNode;
if (activeNode) {
Vue.set(activeNode, 'issues', {
...activeNode.issues,
...issues,
});
}
},
},
});

View file

@ -4,9 +4,14 @@ import {
getNodeTranslationHeaders,
getNodeTypes,
getResourceLocatorResults,
getResourceMapperFields,
} from '@/api/nodeTypes';
import { DEFAULT_NODETYPE_VERSION, STORES } from '@/constants';
import type { INodeTypesState, IResourceLocatorReqParams } from '@/Interface';
import type {
INodeTypesState,
IResourceLocatorReqParams,
ResourceMapperReqParams,
} from '@/Interface';
import { addHeaders, addNodeTranslation } from '@/plugins/i18n';
import { omit } from '@/utils';
import type {
@ -17,6 +22,7 @@ import type {
INodePropertyOptions,
INodeTypeDescription,
INodeTypeNameVersion,
ResourceMapperFields,
} from 'n8n-workflow';
import { defineStore } from 'pinia';
import Vue from 'vue';
@ -177,5 +183,15 @@ export const useNodeTypesStore = defineStore(STORES.NODE_TYPES, {
const rootStore = useRootStore();
return getResourceLocatorResults(rootStore.getRestApiContext, sendData);
},
async getResourceMapperFields(
sendData: ResourceMapperReqParams,
): Promise<ResourceMapperFields | null> {
const rootStore = useRootStore();
try {
return await getResourceMapperFields(rootStore.getRestApiContext, sendData);
} catch (error) {
return null;
}
},
},
});

View file

@ -22,6 +22,7 @@ import type {
NodeParameterValueType,
INodePropertyOptions,
INodePropertyCollection,
ResourceMapperField,
} from 'n8n-workflow';
import { isResourceLocatorValue, isJsonKeyObject } from '@/utils';
import { useCredentialsStore } from '@/stores/credentials.store';
@ -35,6 +36,7 @@ import { i18n as locale } from '@/plugins/i18n';
const CRED_KEYWORDS_TO_FILTER = ['API', 'OAuth1', 'OAuth2'];
const NODE_KEYWORDS_TO_FILTER = ['Trigger'];
const COMMUNITY_PACKAGE_NAME_REGEX = /(@\w+\/)?n8n-nodes-(?!base\b)\b\w+/g;
const RESOURCE_MAPPER_FIELD_NAME_REGEX = /value\[\"(.+)\"\]/;
export function getAppNameFromCredType(name: string) {
return name
@ -400,3 +402,35 @@ export const isNodeParameterRequired = (
});
return true;
};
export const parseResourceMapperFieldName = (fullName: string) => {
const match = fullName.match(RESOURCE_MAPPER_FIELD_NAME_REGEX);
const fieldName = match ? match.pop() : fullName;
return fieldName;
};
export const fieldCannotBeDeleted = (
field: INodeProperties | ResourceMapperField,
showMatchingColumnsSelector: boolean,
resourceMapperMode = '',
matchingFields: string[] = [],
): boolean => {
const fieldIdentifier = 'id' in field ? field.id : field.name;
return (
(resourceMapperMode === 'add' && field.required === true) ||
isMatchingField(fieldIdentifier, matchingFields, showMatchingColumnsSelector)
);
};
export const isMatchingField = (
field: string,
matchingFields: string[],
showMatchingColumnsSelector: boolean,
): boolean => {
const fieldName = parseResourceMapperFieldName(field);
if (fieldName) {
return showMatchingColumnsSelector && (matchingFields || []).includes(fieldName);
}
return false;
};

View file

@ -35,3 +35,7 @@ export function isNumber(value: unknown): value is number {
export const isCredentialModalState = (value: unknown): value is NewCredentialsModal => {
return typeof value === 'object' && value !== null && 'showAuthSelector' in value;
};
export const isResourceMapperValue = (value: unknown): value is string | number | boolean => {
return ['string', 'number', 'boolean'].includes(typeof value);
};

View file

@ -11,7 +11,7 @@ export class GoogleSheets extends VersionedNodeType {
name: 'googleSheets',
icon: 'file:googleSheets.svg',
group: ['input', 'output'],
defaultVersion: 3,
defaultVersion: 4,
subtitle: '={{$parameter["operation"] + ": " + $parameter["resource"]}}',
description: 'Read, update and write data to Google Sheets',
};
@ -20,6 +20,7 @@ export class GoogleSheets extends VersionedNodeType {
1: new GoogleSheetsV1(baseDescription),
2: new GoogleSheetsV1(baseDescription),
3: new GoogleSheetsV2(baseDescription),
4: new GoogleSheetsV2(baseDescription),
};
super(nodeVersions, baseDescription);

View file

@ -5,7 +5,7 @@ import type {
INodeTypeDescription,
} from 'n8n-workflow';
import { versionDescription } from './actions/versionDescription';
import { credentialTest, listSearch, loadOptions } from './methods';
import { credentialTest, listSearch, loadOptions, resourceMapping } from './methods';
import { router } from './actions/router';
export class GoogleSheetsV2 implements INodeType {
@ -22,6 +22,7 @@ export class GoogleSheetsV2 implements INodeType {
loadOptions,
credentialTest,
listSearch,
resourceMapping,
};
async execute(this: IExecuteFunctions) {

View file

@ -30,6 +30,7 @@ export const description: SheetProperties = [
show: {
resource: ['sheet'],
operation: ['append'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -48,6 +49,7 @@ export const description: SheetProperties = [
show: {
operation: ['append'],
dataMode: ['autoMapInputData'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -68,6 +70,7 @@ export const description: SheetProperties = [
resource: ['sheet'],
operation: ['append'],
dataMode: ['defineBelow'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -101,6 +104,40 @@ export const description: SheetProperties = [
},
],
},
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'add',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: false,
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['append'],
'@version': [4],
},
hide: {
...untilSheetSelected,
},
},
},
{
displayName: 'Options',
name: 'options',
@ -156,7 +193,11 @@ export async function execute(
sheetId: string,
): Promise<INodeExecutionData[]> {
const items = this.getInputData();
const dataMode = this.getNodeParameter('dataMode', 0) as string;
const nodeVersion = this.getNode().typeVersion;
const dataMode =
nodeVersion < 4
? (this.getNodeParameter('dataMode', 0) as string)
: (this.getNodeParameter('columns.mappingMode', 0) as string);
if (!items.length || dataMode === 'nothing') return [];
@ -190,5 +231,9 @@ export async function execute(
false,
);
return items;
if (nodeVersion < 4 || dataMode === 'autoMapInputData') {
return items;
} else {
return this.helpers.returnJsonArray(setData);
}
}

View file

@ -36,6 +36,7 @@ export const description: SheetProperties = [
show: {
resource: ['sheet'],
operation: ['appendOrUpdate'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -61,6 +62,7 @@ export const description: SheetProperties = [
show: {
resource: ['sheet'],
operation: ['appendOrUpdate'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -77,6 +79,7 @@ export const description: SheetProperties = [
resource: ['sheet'],
operation: ['appendOrUpdate'],
dataMode: ['defineBelow'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -96,6 +99,7 @@ export const description: SheetProperties = [
resource: ['sheet'],
operation: ['appendOrUpdate'],
dataMode: ['defineBelow'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -141,6 +145,40 @@ export const description: SheetProperties = [
},
],
},
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'upsert',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: false,
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['appendOrUpdate'],
'@version': [4],
},
hide: {
...untilSheetSelected,
},
},
},
{
displayName: 'Options',
name: 'options',
@ -200,10 +238,21 @@ export async function execute(
}
columnNames = sheetData[headerRow];
const nodeVersion = this.getNode().typeVersion;
const newColumns = new Set<string>();
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', 0) as string;
const keyIndex = columnNames.indexOf(columnToMatchOn);
const columnsToMatchOn: string[] =
nodeVersion < 4
? [this.getNodeParameter('columnToMatchOn', 0) as string]
: (this.getNodeParameter('columns.matchingColumns', 0) as string[]);
const dataMode =
nodeVersion < 4
? (this.getNodeParameter('dataMode', 0) as string)
: (this.getNodeParameter('columns.mappingMode', 0) as string);
// TODO: Add support for multiple columns to match on in the next overhaul
const keyIndex = columnNames.indexOf(columnsToMatchOn[0]);
const columnValues = await sheet.getColumnValues(
range,
@ -216,12 +265,8 @@ export async function execute(
const updateData: ISheetUpdateData[] = [];
const appendData: IDataObject[] = [];
const mappedValues: IDataObject[] = [];
for (let i = 0; i < items.length; i++) {
const dataMode = this.getNodeParameter('dataMode', i) as
| 'defineBelow'
| 'autoMapInputData'
| 'nothing';
if (dataMode === 'nothing') continue;
const data: IDataObject[] = [];
@ -251,33 +296,50 @@ export async function execute(
data.push(items[i].json);
}
} else {
const valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string;
const valueToMatchOn =
nodeVersion < 4
? (this.getNodeParameter('valueToMatchOn', i) as string)
: (this.getNodeParameter(`columns.value[${columnsToMatchOn[0]}]`, i) as string);
const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[];
if (!valuesToSend?.length) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
const fields = valuesToSend.reduce((acc, entry) => {
if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
}
acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
if (nodeVersion < 4) {
const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[];
if (!valuesToSend?.length) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
return acc;
}, {} as IDataObject);
fields[columnToMatchOn] = valueToMatchOn;
data.push(fields);
const fields = valuesToSend.reduce((acc, entry) => {
if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
}
acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
}
return acc;
}, {} as IDataObject);
fields[columnsToMatchOn[0]] = valueToMatchOn;
data.push(fields);
} else {
const mappingValues = this.getNodeParameter('columns.value', i) as IDataObject;
if (Object.keys(mappingValues).length === 0) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
// Setting empty values to empty string so that they are not ignored by the API
Object.keys(mappingValues).forEach((key) => {
if (mappingValues[key] === undefined || mappingValues[key] === null) {
mappingValues[key] = '';
}
});
data.push(mappingValues);
mappedValues.push(mappingValues);
}
}
if (newColumns.size) {
@ -291,7 +353,7 @@ export async function execute(
const preparedData = await sheet.prepareDataForUpdateOrUpsert(
data,
columnToMatchOn,
columnsToMatchOn[0],
range,
headerRow,
firstDataRow,
@ -321,5 +383,10 @@ export async function execute(
lastRow,
);
}
return items;
if (nodeVersion < 4 || dataMode === 'autoMapInputData') {
return items;
} else {
return this.helpers.returnJsonArray(mappedValues);
}
}

View file

@ -36,6 +36,7 @@ export const description: SheetProperties = [
show: {
resource: ['sheet'],
operation: ['update'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -61,6 +62,7 @@ export const description: SheetProperties = [
show: {
resource: ['sheet'],
operation: ['update'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -77,6 +79,7 @@ export const description: SheetProperties = [
resource: ['sheet'],
operation: ['update'],
dataMode: ['defineBelow'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -96,6 +99,7 @@ export const description: SheetProperties = [
resource: ['sheet'],
operation: ['update'],
dataMode: ['defineBelow'],
'@version': [3],
},
hide: {
...untilSheetSelected,
@ -141,6 +145,40 @@ export const description: SheetProperties = [
},
],
},
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['sheetName.value'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'update',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: false,
},
},
displayOptions: {
show: {
resource: ['sheet'],
operation: ['update'],
'@version': [4],
},
hide: {
...untilSheetSelected,
},
},
},
{
displayName: 'Options',
name: 'options',
@ -175,6 +213,8 @@ export async function execute(
const locationDefineOptions = (options.locationDefine as IDataObject)?.values as IDataObject;
const nodeVersion = this.getNode().typeVersion;
let headerRow = 0;
let firstDataRow = 1;
@ -201,8 +241,18 @@ export async function execute(
columnNames = sheetData[headerRow];
const newColumns = new Set<string>();
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', 0) as string;
const keyIndex = columnNames.indexOf(columnToMatchOn);
const columnsToMatchOn: string[] =
nodeVersion < 4
? [this.getNodeParameter('columnToMatchOn', 0) as string]
: (this.getNodeParameter('columns.matchingColumns', 0) as string[]);
const dataMode =
nodeVersion < 4
? (this.getNodeParameter('dataMode', 0) as string)
: (this.getNodeParameter('columns.mappingMode', 0) as string);
// TODO: Add support for multiple columns to match on in the next overhaul
const keyIndex = columnNames.indexOf(columnsToMatchOn[0]);
const columnValues = await sheet.getColumnValues(
range,
@ -214,12 +264,9 @@ export async function execute(
const updateData: ISheetUpdateData[] = [];
for (let i = 0; i < items.length; i++) {
const dataMode = this.getNodeParameter('dataMode', i) as
| 'defineBelow'
| 'autoMapInputData'
| 'nothing';
const mappedValues: IDataObject[] = [];
for (let i = 0; i < items.length; i++) {
if (dataMode === 'nothing') continue;
const data: IDataObject[] = [];
@ -249,33 +296,54 @@ export async function execute(
data.push(items[i].json);
}
} else {
const valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string;
const valueToMatchOn =
nodeVersion < 4
? (this.getNodeParameter('valueToMatchOn', i) as string)
: (this.getNodeParameter(`columns.value[${columnsToMatchOn[0]}]`, i) as string);
const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[];
if (!valuesToSend?.length) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
const fields = valuesToSend.reduce((acc, entry) => {
if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
}
acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
if (nodeVersion < 4) {
const valuesToSend = this.getNodeParameter('fieldsUi.values', i, []) as IDataObject[];
if (!valuesToSend?.length) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
return acc;
}, {} as IDataObject);
const fields = valuesToSend.reduce((acc, entry) => {
if (entry.column === 'newColumn') {
const columnName = entry.columnName as string;
fields[columnToMatchOn] = valueToMatchOn;
if (!columnNames.includes(columnName)) {
newColumns.add(columnName);
}
data.push(fields);
acc[columnName] = entry.fieldValue as string;
} else {
acc[entry.column as string] = entry.fieldValue as string;
}
return acc;
}, {} as IDataObject);
fields[columnsToMatchOn[0]] = valueToMatchOn;
data.push(fields);
} else {
const mappingValues = this.getNodeParameter('columns.value', i) as IDataObject;
if (Object.keys(mappingValues).length === 0) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
// Setting empty values to empty string so that they are not ignored by the API
Object.keys(mappingValues).forEach((key) => {
if (mappingValues[key] === undefined || mappingValues[key] === null) {
mappingValues[key] = '';
}
});
data.push(mappingValues);
mappedValues.push(mappingValues);
}
}
if (newColumns.size) {
@ -289,7 +357,7 @@ export async function execute(
const preparedData = await sheet.prepareDataForUpdateOrUpsert(
data,
columnToMatchOn,
columnsToMatchOn[0],
range,
headerRow,
firstDataRow,
@ -306,5 +374,9 @@ export async function execute(
await sheet.batchUpdate(updateData, valueInputMode);
}
return items;
if (nodeVersion < 4 || dataMode === 'autoMapInputData') {
return items;
} else {
return this.helpers.returnJsonArray(mappedValues);
}
}

View file

@ -9,7 +9,7 @@ export const versionDescription: INodeTypeDescription = {
name: 'googleSheets',
icon: 'file:googleSheets.svg',
group: ['input', 'output'],
version: 3,
version: [3, 4],
subtitle: '={{$parameter["operation"] + ": " + $parameter["resource"]}}',
description: 'Read, update and write data to Google Sheets',
defaults: {

View file

@ -195,12 +195,24 @@ export function mapFields(this: IExecuteFunctions, inputSize: number) {
const returnData: IDataObject[] = [];
for (let i = 0; i < inputSize; i++) {
const fields = this.getNodeParameter('fieldsUi.fieldValues', i, []) as IDataObject[];
let dataToSend: IDataObject = {};
for (const field of fields) {
dataToSend = { ...dataToSend, [field.fieldId as string]: field.fieldValue };
const nodeVersion = this.getNode().typeVersion;
if (nodeVersion < 4) {
const fields = this.getNodeParameter('fieldsUi.fieldValues', i, []) as IDataObject[];
let dataToSend: IDataObject = {};
for (const field of fields) {
dataToSend = { ...dataToSend, [field.fieldId as string]: field.fieldValue };
}
returnData.push(dataToSend);
} else {
const mappingValues = this.getNodeParameter('columns.value', i) as IDataObject;
if (Object.keys(mappingValues).length === 0) {
throw new NodeOperationError(
this.getNode(),
"At least one value has to be added under 'Values to Send'",
);
}
returnData.push(mappingValues);
}
returnData.push(dataToSend);
}
return returnData;

View file

@ -1,3 +1,4 @@
export * as loadOptions from './loadOptions';
export * as listSearch from './listSearch';
export * as credentialTest from './credentialTest';
export * as resourceMapping from './resourceMapping';

View file

@ -0,0 +1,39 @@
import type { ILoadOptionsFunctions } from 'n8n-core';
import type { IDataObject, ResourceMapperFields } from 'n8n-workflow';
import { GoogleSheet } from '../helpers/GoogleSheet';
import type { ResourceLocator } from '../helpers/GoogleSheets.types';
import { getSpreadsheetId } from '../helpers/GoogleSheets.utils';
export async function getMappingColumns(
this: ILoadOptionsFunctions,
): Promise<ResourceMapperFields> {
const { mode, value } = this.getNodeParameter('documentId', 0) as IDataObject;
const spreadsheetId = getSpreadsheetId(mode as ResourceLocator, value as string);
const sheet = new GoogleSheet(spreadsheetId, this);
let sheetWithinDocument = this.getNodeParameter('sheetName', undefined, {
extractValue: true,
}) as string;
if (sheetWithinDocument === 'gid=0') {
sheetWithinDocument = '0';
}
const sheetName = await sheet.spreadsheetGetSheetNameById(sheetWithinDocument);
const sheetData = await sheet.getData(`${sheetName}!1:1`, 'FORMATTED_VALUE');
const columns = sheet.testFilter(sheetData || [], 0, 0).filter((col) => col !== '');
const columnData: ResourceMapperFields = {
fields: columns.map((col) => ({
id: col,
displayName: col,
required: false,
defaultMatch: col === 'id',
display: true,
type: 'string',
canBeUsedToMatch: true,
})),
};
return columnData;
}

View file

@ -11,7 +11,7 @@ export class Postgres extends VersionedNodeType {
name: 'postgres',
icon: 'file:postgres.svg',
group: ['input'],
defaultVersion: 2.1,
defaultVersion: 2.2,
description: 'Get, add and update data in Postgres',
};
@ -19,6 +19,7 @@ export class Postgres extends VersionedNodeType {
1: new PostgresV1(baseDescription),
2: new PostgresV2(baseDescription),
2.1: new PostgresV2(baseDescription),
2.2: new PostgresV2(baseDescription),
};
super(nodeVersions, baseDescription);

View file

@ -9,7 +9,7 @@ import type {
import { router } from './actions/router';
import { versionDescription } from './actions/versionDescription';
import { credentialTest, listSearch, loadOptions } from './methods';
import { credentialTest, listSearch, loadOptions, resourceMapping } from './methods';
export class PostgresV2 implements INodeType {
description: INodeTypeDescription;
@ -21,7 +21,7 @@ export class PostgresV2 implements INodeType {
};
}
methods = { credentialTest, listSearch, loadOptions };
methods = { credentialTest, listSearch, loadOptions, resourceMapping };
async execute(this: IExecuteFunctions): Promise<INodeExecutionData[][]> {
return router.call(this);

View file

@ -40,6 +40,11 @@ const properties: INodeProperties[] = [
default: 'autoMapInputData',
description:
'Whether to map node input properties and the table data automatically or manually',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
},
{
displayName: `
@ -51,6 +56,7 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['autoMapInputData'],
'@version': [2, 2.1],
},
},
},
@ -66,6 +72,7 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['defineBelow'],
'@version': [2, 2.1],
},
},
default: {},
@ -97,6 +104,35 @@ const properties: INodeProperties[] = [
},
],
},
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
default: {
mappingMode: 'defineBelow',
value: null,
},
noDataExpression: true,
required: true,
typeOptions: {
loadOptionsDependsOn: ['table.value', 'operation'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'add',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: true,
},
},
displayOptions: {
show: {
'@version': [2.2],
},
},
},
optionsCollection,
];
@ -142,7 +178,11 @@ export async function execute(
let query = `INSERT INTO $1:name.$2:name($3:name) VALUES($3:csv)${onConflict}`;
let values: QueryValues = [schema, table];
const dataMode = this.getNodeParameter('dataMode', i) as string;
const nodeVersion = this.getNode().typeVersion;
const dataMode =
nodeVersion < 2.2
? (this.getNodeParameter('dataMode', i) as string)
: (this.getNodeParameter('columns.mappingMode', i) as string);
let item: IDataObject = {};
@ -151,10 +191,17 @@ export async function execute(
}
if (dataMode === 'defineBelow') {
const valuesToSend = (this.getNodeParameter('valuesToSend', i, []) as IDataObject)
.values as IDataObject[];
const valuesToSend =
nodeVersion < 2.2
? ((this.getNodeParameter('valuesToSend', i, []) as IDataObject).values as IDataObject[])
: ((this.getNodeParameter('columns.values', i, []) as IDataObject)
.values as IDataObject[]);
item = prepareItem(valuesToSend);
if (nodeVersion < 2.2) {
item = prepareItem(valuesToSend);
} else {
item = this.getNodeParameter('columns.value', i) as IDataObject;
}
}
const tableSchema = await getTableSchema(db, schema, table);

View file

@ -14,6 +14,7 @@ import type {
import {
addReturning,
checkItemAgainstSchema,
doesRowExist,
getTableSchema,
prepareItem,
replaceEmptyStringsByNulls,
@ -41,6 +42,11 @@ const properties: INodeProperties[] = [
default: 'autoMapInputData',
description:
'Whether to map node input properties and the table data automatically or manually',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
},
{
displayName: `
@ -52,6 +58,7 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['autoMapInputData'],
'@version': [2],
},
},
},
@ -69,6 +76,11 @@ const properties: INodeProperties[] = [
},
default: '',
hint: 'The column that identifies the row(s) to modify',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
},
{
displayName: 'Value of Column to Match On',
@ -80,6 +92,7 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['defineBelow'],
'@version': [2, 2.1],
},
},
},
@ -95,6 +108,7 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['defineBelow'],
'@version': [2, 2.1],
},
},
default: {},
@ -126,6 +140,35 @@ const properties: INodeProperties[] = [
},
],
},
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['table.value', 'operation'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'update',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: true,
},
},
displayOptions: {
show: {
'@version': [2.2],
},
},
},
optionsCollection,
];
@ -161,32 +204,80 @@ export async function execute(
extractValue: true,
}) as string;
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', i) as string;
const nodeVersion = this.getNode().typeVersion;
const columnsToMatchOn: string[] =
nodeVersion < 2.2
? [this.getNodeParameter('columnToMatchOn', i) as string]
: (this.getNodeParameter('columns.matchingColumns', i) as string[]);
const dataMode = this.getNodeParameter('dataMode', i) as string;
const dataMode =
nodeVersion < 2.2
? (this.getNodeParameter('dataMode', i) as string)
: (this.getNodeParameter('columns.mappingMode', i) as string);
let item: IDataObject = {};
let valueToMatchOn: string | IDataObject = '';
if (dataMode === 'autoMapInputData') {
item = items[i].json;
valueToMatchOn = item[columnToMatchOn] as string;
}
if (dataMode === 'defineBelow') {
const valuesToSend = (this.getNodeParameter('valuesToSend', i, []) as IDataObject)
.values as IDataObject[];
item = prepareItem(valuesToSend);
if (nodeVersion < 2.2) {
valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string;
}
if (!item[columnToMatchOn] && dataMode === 'autoMapInputData') {
throw new NodeOperationError(
this.getNode(),
"Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.",
);
if (dataMode === 'autoMapInputData') {
item = items[i].json;
if (nodeVersion < 2.2) {
valueToMatchOn = item[columnsToMatchOn[0]] as string;
}
}
if (dataMode === 'defineBelow') {
const valuesToSend =
nodeVersion < 2.2
? ((this.getNodeParameter('valuesToSend', i, []) as IDataObject).values as IDataObject[])
: ((this.getNodeParameter('columns.values', i, []) as IDataObject)
.values as IDataObject[]);
if (nodeVersion < 2.2) {
item = prepareItem(valuesToSend);
item[columnsToMatchOn[0]] = this.getNodeParameter('valueToMatchOn', i) as string;
} else {
item = this.getNodeParameter('columns.value', i) as IDataObject;
}
}
const matchValues: string[] = [];
if (nodeVersion < 2.2) {
if (!item[columnsToMatchOn[0]] && dataMode === 'autoMapInputData') {
throw new NodeOperationError(
this.getNode(),
"Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.",
);
}
matchValues.push(valueToMatchOn);
matchValues.push(columnsToMatchOn[0]);
} else {
columnsToMatchOn.forEach((column) => {
matchValues.push(column);
matchValues.push(item[column] as string);
});
const rowExists = await doesRowExist(db, schema, table, matchValues);
if (!rowExists) {
const descriptionValues: string[] = [];
matchValues.forEach((val, index) => {
if (index % 2 === 0) {
descriptionValues.push(`${matchValues[index]}=${matchValues[index + 1]}`);
}
});
throw new NodeOperationError(
this.getNode(),
"The row you are trying to update doesn't exist",
{
description: `No rows matching the provided values (${descriptionValues.join(
', ',
)}) were found in the table "${table}".`,
itemIndex: i,
},
);
}
}
const tableSchema = await getTableSchema(db, schema, table);
@ -197,11 +288,22 @@ export async function execute(
let valuesLength = values.length + 1;
const condition = `$${valuesLength}:name = $${valuesLength + 1}`;
valuesLength = valuesLength + 2;
values.push(columnToMatchOn, valueToMatchOn);
let condition = '';
if (nodeVersion < 2.2) {
condition = `$${valuesLength}:name = $${valuesLength + 1}`;
valuesLength = valuesLength + 2;
values.push(columnsToMatchOn[0], valueToMatchOn);
} else {
const conditions: string[] = [];
for (const column of columnsToMatchOn) {
conditions.push(`$${valuesLength}:name = $${valuesLength + 1}`);
valuesLength = valuesLength + 2;
values.push(column, item[column] as string);
}
condition = conditions.join(' AND ');
}
const updateColumns = Object.keys(item).filter((column) => column !== columnToMatchOn);
const updateColumns = Object.keys(item).filter((column) => !columnsToMatchOn.includes(column));
if (!Object.keys(updateColumns).length) {
throw new NodeOperationError(
@ -227,5 +329,6 @@ export async function execute(
queries.push({ query, values });
}
return runQueries(queries, items, nodeOptions);
const results = await runQueries(queries, items, nodeOptions);
return results;
}

View file

@ -41,6 +41,11 @@ const properties: INodeProperties[] = [
default: 'autoMapInputData',
description:
'Whether to map node input properties and the table data automatically or manually',
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
},
{
displayName: `
@ -52,12 +57,13 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['autoMapInputData'],
'@version': [2, 2.1],
},
},
},
{
// eslint-disable-next-line n8n-nodes-base/node-param-display-name-miscased, n8n-nodes-base/node-param-display-name-wrong-for-dynamic-options
displayName: 'Column to Match On',
displayName: 'Unique Column',
name: 'columnToMatchOn',
type: 'options',
required: true,
@ -69,9 +75,14 @@ const properties: INodeProperties[] = [
},
default: '',
hint: "Used to find the correct row(s) to update. Doesn't get changed. Has to be unique.",
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
},
{
displayName: 'Value of Column to Match On',
displayName: 'Value of Unique Column',
name: 'valueToMatchOn',
type: 'string',
default: '',
@ -80,6 +91,7 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['defineBelow'],
'@version': [2, 2.1],
},
},
},
@ -95,6 +107,7 @@ const properties: INodeProperties[] = [
displayOptions: {
show: {
dataMode: ['defineBelow'],
'@version': [2, 2.1],
},
},
default: {},
@ -126,6 +139,35 @@ const properties: INodeProperties[] = [
},
],
},
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['table.value', 'operation'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'upsert',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: true,
},
},
displayOptions: {
show: {
'@version': [2.2],
},
},
},
optionsCollection,
];
@ -161,9 +203,16 @@ export async function execute(
extractValue: true,
}) as string;
const columnToMatchOn = this.getNodeParameter('columnToMatchOn', i) as string;
const nodeVersion = this.getNode().typeVersion;
const columnsToMatchOn: string[] =
nodeVersion < 2.2
? [this.getNodeParameter('columnToMatchOn', i) as string]
: (this.getNodeParameter('columns.matchingColumns', i) as string[]);
const dataMode = this.getNodeParameter('dataMode', i) as string;
const dataMode =
nodeVersion < 2.2
? (this.getNodeParameter('dataMode', i) as string)
: (this.getNodeParameter('columns.mappingMode', i) as string);
let item: IDataObject = {};
@ -172,22 +221,28 @@ export async function execute(
}
if (dataMode === 'defineBelow') {
const valuesToSend = (this.getNodeParameter('valuesToSend', i, []) as IDataObject)
.values as IDataObject[];
const valuesToSend =
nodeVersion < 2.2
? ((this.getNodeParameter('valuesToSend', i, []) as IDataObject).values as IDataObject[])
: ((this.getNodeParameter('columns.values', i, []) as IDataObject)
.values as IDataObject[]);
item = prepareItem(valuesToSend);
item[columnToMatchOn] = this.getNodeParameter('valueToMatchOn', i) as string;
if (nodeVersion < 2.2) {
item = prepareItem(valuesToSend);
item[columnsToMatchOn[0]] = this.getNodeParameter('valueToMatchOn', i) as string;
} else {
item = this.getNodeParameter('columns.value', i) as IDataObject;
}
}
if (!item[columnToMatchOn]) {
if (!item[columnsToMatchOn[0]]) {
throw new NodeOperationError(
this.getNode(),
"Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.",
);
}
if (item[columnToMatchOn] && Object.keys(item).length === 1) {
if (item[columnsToMatchOn[0]] && Object.keys(item).length === 1) {
throw new NodeOperationError(
this.getNode(),
"Add values to update or insert to the input item or set the 'Data Mode' to 'Define Below' to define the values to insert or update.",
@ -201,16 +256,19 @@ export async function execute(
let values: QueryValues = [schema, table];
let valuesLength = values.length + 1;
const onConflict = ` ON CONFLICT ($${valuesLength}:name) DO UPDATE `;
valuesLength = valuesLength + 1;
values.push(columnToMatchOn);
const conflictColumns: string[] = [];
columnsToMatchOn.forEach((column) => {
conflictColumns.push(`$${valuesLength}:name`);
valuesLength = valuesLength + 1;
values.push(column);
});
const onConflict = ` ON CONFLICT (${conflictColumns.join(',')}) DO UPDATE `;
const insertQuery = `INSERT INTO $1:name.$2:name($${valuesLength}:name) VALUES($${valuesLength}:csv)${onConflict}`;
valuesLength = valuesLength + 1;
values.push(item);
const updateColumns = Object.keys(item).filter((column) => column !== columnToMatchOn);
const updateColumns = Object.keys(item).filter((column) => !columnsToMatchOn.includes(column));
const updates: string[] = [];
for (const column of updateColumns) {

View file

@ -8,7 +8,7 @@ export const versionDescription: INodeTypeDescription = {
name: 'postgres',
icon: 'file:postgres.svg',
group: ['input'],
version: [2, 2.1],
version: [2, 2.1, 2.2],
subtitle: '={{ $parameter["operation"] }}',
description: 'Get, add and update data in Postgres',
defaults: {

View file

@ -16,7 +16,17 @@ export type QueryWithValues = { query: string; values?: QueryValues };
export type WhereClause = { column: string; condition: string; value: string | number };
export type SortRule = { column: string; direction: string };
export type ColumnInfo = { column_name: string; data_type: string; is_nullable: string };
export type ColumnInfo = {
column_name: string;
data_type: string;
is_nullable: string;
udt_name: string;
column_default?: string;
};
export type EnumInfo = {
typname: string;
enumlabel: string;
};
export type PgpClient = pgPromise.IMain<{}, pg.IClient>;
export type PgpDatabase = pgPromise.IDatabase<{}, pg.IClient>;

View file

@ -1,9 +1,10 @@
import type { IDataObject, INode, INodeExecutionData } from 'n8n-workflow';
import type { IDataObject, INode, INodeExecutionData, INodePropertyOptions } from 'n8n-workflow';
import { NodeOperationError } from 'n8n-workflow';
import type {
ColumnInfo,
ConstructExecutionMetaData,
EnumInfo,
PgpClient,
PgpDatabase,
QueryMode,
@ -13,6 +14,8 @@ import type {
WhereClause,
} from './interfaces';
const ENUM_VALUES_REGEX = /\{(.+?)\}/gm;
export function wrapData(data: IDataObject | IDataObject[]): INodeExecutionData[] {
if (!Array.isArray(data)) {
return [{ json: data }];
@ -324,13 +327,60 @@ export async function getTableSchema(
table: string,
): Promise<ColumnInfo[]> {
const columns = await db.any(
'SELECT column_name, data_type, is_nullable FROM information_schema.columns WHERE table_schema = $1 AND table_name = $2',
'SELECT column_name, data_type, is_nullable, udt_name, column_default FROM information_schema.columns WHERE table_schema = $1 AND table_name = $2',
[schema, table],
);
return columns;
}
export async function uniqueColumns(db: PgpDatabase, table: string) {
// Using the modified query from https://wiki.postgresql.org/wiki/Retrieve_primary_key_columns
const unique = await db.any(
`
SELECT DISTINCT a.attname
FROM pg_index i JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = ANY(i.indkey)
WHERE i.indrelid = quote_ident($1)::regclass
AND (i.indisprimary OR i.indisunique);
`,
[table],
);
return unique as IDataObject[];
}
export async function getEnums(db: PgpDatabase): Promise<EnumInfo[]> {
const enumsData = await db.any(
'SELECT pg_type.typname, pg_enum.enumlabel FROM pg_type JOIN pg_enum ON pg_enum.enumtypid = pg_type.oid;',
);
return enumsData as EnumInfo[];
}
export function getEnumValues(enumInfo: EnumInfo[], enumName: string): INodePropertyOptions[] {
return enumInfo.reduce((acc, current) => {
if (current.typname === enumName) {
acc.push({ name: current.enumlabel, value: current.enumlabel });
}
return acc;
}, [] as INodePropertyOptions[]);
}
export async function doesRowExist(
db: PgpDatabase,
schema: string,
table: string,
values: string[],
): Promise<boolean> {
const where = [];
for (let i = 3; i < 3 + values.length; i += 2) {
where.push(`$${i}:name=$${i + 1}`);
}
const exists = await db.any(
`SELECT EXISTS(SELECT 1 FROM $1:name.$2:name WHERE ${where.join(' AND ')})`,
[schema, table, ...values],
);
return exists[0].exists;
}
export function checkItemAgainstSchema(
node: INode,
item: IDataObject,

View file

@ -1,3 +1,4 @@
export * as credentialTest from './credentialTest';
export * as listSearch from './listSearch';
export * as loadOptions from './loadOptions';
export * as resourceMapping from './resourceMapping';

View file

@ -0,0 +1,91 @@
import type { ILoadOptionsFunctions, ResourceMapperFields, FieldType } from 'n8n-workflow';
import { getEnumValues, getEnums, getTableSchema, uniqueColumns } from '../helpers/utils';
import { configurePostgres } from '../transport';
const fieldTypeMapping: Partial<Record<FieldType, string[]>> = {
string: ['text', 'varchar', 'character varying', 'character', 'char'],
number: [
'integer',
'smallint',
'bigint',
'decimal',
'numeric',
'real',
'double precision',
'smallserial',
'serial',
'bigserial',
],
boolean: ['boolean'],
dateTime: [
'timestamp',
'date',
'timestampz',
'timestamp without time zone',
'timestamp with time zone',
],
time: ['time', 'time without time zone', 'time with time zone'],
object: ['json', 'jsonb'],
options: ['enum', 'USER-DEFINED'],
array: ['ARRAY'],
};
function mapPostgresType(postgresType: string): FieldType {
let mappedType: FieldType = 'string';
for (const t of Object.keys(fieldTypeMapping)) {
const postgresTypes = fieldTypeMapping[t as FieldType];
if (postgresTypes?.includes(postgresType)) {
mappedType = t as FieldType;
}
}
return mappedType;
}
export async function getMappingColumns(
this: ILoadOptionsFunctions,
): Promise<ResourceMapperFields> {
const credentials = await this.getCredentials('postgres');
const { db } = await configurePostgres(credentials);
const schema = this.getNodeParameter('schema', 0, {
extractValue: true,
}) as string;
const table = this.getNodeParameter('table', 0, {
extractValue: true,
}) as string;
const operation = this.getNodeParameter('operation', 0, {
extractValue: true,
}) as string;
try {
const columns = await getTableSchema(db, schema, table);
const unique = operation === 'upsert' ? await uniqueColumns(db, table) : [];
const enumInfo = await getEnums(db);
const fields = await Promise.all(
columns.map(async (col) => {
const canBeUsedToMatch =
operation === 'upsert' ? unique.some((u) => u.attname === col.column_name) : true;
const type = mapPostgresType(col.data_type);
const options = type === 'options' ? getEnumValues(enumInfo, col.udt_name) : undefined;
const isAutoIncrement = col.column_default?.startsWith('nextval');
return {
id: col.column_name,
displayName: col.column_name,
required: col.is_nullable !== 'YES' && !isAutoIncrement,
defaultMatch: col.column_name === 'id',
display: true,
type,
canBeUsedToMatch,
options,
};
}),
);
return { fields };
} catch (error) {
throw error;
}
}

View file

@ -552,7 +552,9 @@ export class Expression {
// The parameter value is complex so resolve depending on type
if (Array.isArray(parameterValue)) {
// Data is an array
const returnData = parameterValue.map((item) => resolveParameterValue(item, {}));
const returnData = parameterValue.map((item) =>
resolveParameterValue(item as NodeParameterValueType, {}),
);
return returnData as NodeParameterValue[] | INodeParameters[];
}

View file

@ -995,7 +995,8 @@ export type NodeParameterValueType =
| INodeParameterResourceLocator
| NodeParameterValue[]
| INodeParameters[]
| INodeParameterResourceLocator[];
| INodeParameterResourceLocator[]
| ResourceMapperValue[];
export interface INodeParameters {
[key: string]: NodeParameterValueType;
@ -1016,7 +1017,8 @@ export type NodePropertyTypes =
| 'string'
| 'credentialsSelect'
| 'resourceLocator'
| 'curlImport';
| 'curlImport'
| 'resourceMapper';
export type CodeAutocompleteTypes = 'function' | 'functionItem';
@ -1052,9 +1054,25 @@ export interface INodePropertyTypeOptions {
showAlpha?: boolean; // Supported by: color
sortable?: boolean; // Supported when "multipleValues" set to true
expirable?: boolean; // Supported by: hidden (only in the credentials)
resourceMapper?: ResourceMapperTypeOptions;
[key: string]: any;
}
export interface ResourceMapperTypeOptions {
resourceMapperMethod: string;
mode: 'add' | 'update' | 'upsert';
fieldWords?: { singular: string; plural: string };
addAllFields?: boolean;
noFieldsError?: string;
multiKeyMatch?: boolean;
supportAutoMap?: boolean;
matchingFieldsLabels?: {
title?: string;
description?: string;
hint?: string;
};
}
export interface IDisplayOptions {
hide?: {
[key: string]: NodeParameterValue[] | undefined;
@ -1221,6 +1239,9 @@ export interface INodeType {
// Contains a group of functions that test credentials.
[functionName: string]: ICredentialTestFunction;
};
resourceMapping?: {
[functionName: string]: (this: ILoadOptionsFunctions) => Promise<ResourceMapperFields>;
};
};
webhookMethods?: {
[name in IWebhookDescription['name']]?: {
@ -1934,6 +1955,44 @@ export interface IExceutionSummaryNodeExecutionResult {
}>;
}
export interface ResourceMapperFields {
fields: ResourceMapperField[];
}
export interface ResourceMapperField {
id: string;
displayName: string;
defaultMatch: boolean;
canBeUsedToMatch?: boolean;
required: boolean;
display: boolean;
type?: FieldType;
removed?: boolean;
options?: INodePropertyOptions[];
}
export type FieldType =
| 'string'
| 'number'
| 'dateTime'
| 'boolean'
| 'time'
| 'array'
| 'object'
| 'options';
export type ValidationResult = {
valid: boolean;
errorMessage?: string;
newValue?: string | number | boolean | object | null | undefined;
};
export type ResourceMapperValue = {
mappingMode: string;
value: { [key: string]: string | number | boolean | null } | null;
matchingColumns: string[];
schema: ResourceMapperField[];
};
export interface ExecutionOptions {
limit?: number;
}

View file

@ -37,11 +37,16 @@ import type {
IWorkflowExecuteAdditionalData,
NodeParameterValue,
WebhookHttpMethod,
FieldType,
INodePropertyOptions,
ResourceMapperValue,
ValidationResult,
} from './Interfaces';
import { isValidResourceLocatorParameterValue } from './type-guards';
import { isResourceMapperValue, isValidResourceLocatorParameterValue } from './type-guards';
import { deepCopy } from './utils';
import type { Workflow } from './Workflow';
import { DateTime } from 'luxon';
export const cronNodeOptions: INodePropertyCollection[] = [
{
@ -1097,6 +1102,173 @@ export function nodeIssuesToString(issues: INodeIssues, node?: INode): string[]
return nodeIssues;
}
// Validates field against the schema and tries to parse it to the correct type
export const validateFieldType = (
fieldName: string,
value: unknown,
type: FieldType,
options?: INodePropertyOptions[],
): ValidationResult => {
if (value === null || value === undefined) return { valid: true };
const defaultErrorMessage = `'${fieldName}' expects a ${type} but we got '${String(value)}'.`;
switch (type.toLowerCase()) {
case 'number': {
try {
return { valid: true, newValue: tryToParseNumber(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'boolean': {
try {
return { valid: true, newValue: tryToParseBoolean(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'datetime': {
try {
return { valid: true, newValue: tryToParseDateTime(value) };
} catch (e) {
const luxonDocsURL =
'https://moment.github.io/luxon/api-docs/index.html#datetimefromformat';
const errorMessage = `${defaultErrorMessage} <br/><br/> Consider using <a href="${luxonDocsURL}" target="_blank"><code>DateTime.fromFormat</code></a> to work with custom date formats.`;
return { valid: false, errorMessage };
}
}
case 'time': {
try {
return { valid: true, newValue: tryToParseTime(value) };
} catch (e) {
return {
valid: false,
errorMessage: `'${fieldName}' expects time (hh:mm:(:ss)) but we got '${String(value)}'.`,
};
}
}
case 'object': {
try {
return { valid: true, newValue: tryToParseObject(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'array': {
try {
return { valid: true, newValue: tryToParseArray(value) };
} catch (e) {
return { valid: false, errorMessage: defaultErrorMessage };
}
}
case 'options': {
const validOptions = options?.map((option) => option.value).join(', ') || '';
const isValidOption = options?.some((option) => option.value === value) || false;
if (!isValidOption) {
return {
valid: false,
errorMessage: `'${fieldName}' expects one of the following values: [${validOptions}] but we got '${String(
value,
)}'`,
};
}
return { valid: true, newValue: value };
}
default: {
return { valid: true, newValue: value };
}
}
};
export const tryToParseNumber = (value: unknown): number => {
const isValidNumber = !isNaN(Number(value));
if (!isValidNumber) {
throw new Error(`Could not parse '${String(value)}' to number.`);
}
return Number(value);
};
export const tryToParseBoolean = (value: unknown): value is boolean => {
if (typeof value === 'boolean') {
return value;
}
if (typeof value === 'string' && ['true', 'false'].includes(value.toLowerCase())) {
return value.toLowerCase() === 'true';
}
const num = Number(value);
if (num === 0) {
return false;
} else if (num === 1) {
return true;
}
throw new Error(`Could not parse '${String(value)}' to boolean.`);
};
export const tryToParseDateTime = (value: unknown): DateTime => {
const dateString = String(value).trim();
// Rely on luxon to parse different date formats
const isoDate = DateTime.fromISO(dateString, { setZone: true });
if (isoDate.isValid) {
return isoDate;
}
const httpDate = DateTime.fromHTTP(dateString, { setZone: true });
if (httpDate.isValid) {
return httpDate;
}
const rfc2822Date = DateTime.fromRFC2822(dateString, { setZone: true });
if (rfc2822Date.isValid) {
return rfc2822Date;
}
const sqlDate = DateTime.fromSQL(dateString, { setZone: true });
if (sqlDate.isValid) {
return sqlDate;
}
throw new Error(`The value "${dateString}" is not a valid date.`);
};
export const tryToParseTime = (value: unknown): string => {
const isTimeInput = /^\d{2}:\d{2}(:\d{2})?((\-|\+)\d{4})?((\-|\+)\d{1,2}(:\d{2})?)?$/s.test(
String(value),
);
if (!isTimeInput) {
throw new Error(`The value "${String(value)}" is not a valid time.`);
}
return String(value);
};
export const tryToParseArray = (value: unknown): unknown[] => {
try {
const parsed = JSON.parse(String(value));
if (!Array.isArray(parsed)) {
throw new Error(`The value "${String(value)}" is not a valid array.`);
}
return parsed;
} catch (e) {
throw new Error(`The value "${String(value)}" is not a valid array.`);
}
};
export const tryToParseObject = (value: unknown): object => {
if (value && typeof value === 'object' && !Array.isArray(value)) {
return value;
}
try {
// eslint-disable-next-line @typescript-eslint/no-unsafe-assignment
const o = JSON.parse(String(value));
if (typeof o !== 'object' || Array.isArray(o)) {
throw new Error(`The value "${String(value)}" is not a valid object.`);
}
return o;
} catch (e) {
throw new Error(`The value "${String(value)}" is not a valid object.`);
}
};
/*
* Validates resource locator node parameters based on validation ruled defined in each parameter mode
*
@ -1128,6 +1300,42 @@ export const validateResourceLocatorParameter = (
return validationErrors;
};
/*
* Validates resource mapper values based on service schema
*
*/
export const validateResourceMapperParameter = (
nodeProperties: INodeProperties,
value: ResourceMapperValue,
skipRequiredCheck = false,
): Record<string, string[]> => {
const issues: Record<string, string[]> = {};
let fieldWordSingular =
nodeProperties.typeOptions?.resourceMapper?.fieldWords?.singular || 'Field';
fieldWordSingular = fieldWordSingular.charAt(0).toUpperCase() + fieldWordSingular.slice(1);
value.schema.forEach((field) => {
const fieldValue = value.value ? value.value[field.id] : null;
const key = `${nodeProperties.name}.${field.id}`;
const fieldErrors: string[] = [];
if (field.required && !skipRequiredCheck) {
if (value.value === null || fieldValue === null || fieldValue === undefined) {
const error = `${fieldWordSingular} "${field.id}" is required`;
fieldErrors.push(error);
}
}
if (!fieldValue?.toString().startsWith('=') && field.type) {
const validationResult = validateFieldType(field.id, fieldValue, field.type, field.options);
if (!validationResult.valid && validationResult.errorMessage) {
fieldErrors.push(validationResult.errorMessage);
}
}
if (fieldErrors.length > 0) {
issues[key] = fieldErrors;
}
});
return issues;
};
/**
* Adds an issue if the parameter is not defined
*
@ -1196,8 +1404,9 @@ export function getParameterIssues(
node: INode,
): INodeIssues {
const foundIssues: INodeIssues = {};
const isDisplayed = displayParameterPath(nodeValues, nodeProperties, path, node);
if (nodeProperties.required === true) {
if (displayParameterPath(nodeValues, nodeProperties, path, node)) {
if (isDisplayed) {
const value = getParameterValueByPath(nodeValues, nodeProperties.name, path);
if (
@ -1218,24 +1427,37 @@ export function getParameterIssues(
}
}
if (nodeProperties.type === 'resourceLocator') {
if (displayParameterPath(nodeValues, nodeProperties, path, node)) {
const value = getParameterValueByPath(nodeValues, nodeProperties.name, path);
if (isINodeParameterResourceLocator(value)) {
const mode = nodeProperties.modes?.find((option) => option.name === value.mode);
if (mode) {
const errors = validateResourceLocatorParameter(value, mode);
errors.forEach((error) => {
if (foundIssues.parameters === undefined) {
foundIssues.parameters = {};
}
if (foundIssues.parameters[nodeProperties.name] === undefined) {
foundIssues.parameters[nodeProperties.name] = [];
}
if (nodeProperties.type === 'resourceLocator' && isDisplayed) {
const value = getParameterValueByPath(nodeValues, nodeProperties.name, path);
if (isINodeParameterResourceLocator(value)) {
const mode = nodeProperties.modes?.find((option) => option.name === value.mode);
if (mode) {
const errors = validateResourceLocatorParameter(value, mode);
errors.forEach((error) => {
if (foundIssues.parameters === undefined) {
foundIssues.parameters = {};
}
if (foundIssues.parameters[nodeProperties.name] === undefined) {
foundIssues.parameters[nodeProperties.name] = [];
}
foundIssues.parameters[nodeProperties.name].push(error);
});
foundIssues.parameters[nodeProperties.name].push(error);
});
}
}
} else if (nodeProperties.type === 'resourceMapper' && isDisplayed) {
const skipRequiredCheck = nodeProperties.typeOptions?.resourceMapper?.mode !== 'add';
const value = getParameterValueByPath(nodeValues, nodeProperties.name, path);
if (isResourceMapperValue(value)) {
const issues = validateResourceMapperParameter(nodeProperties, value, skipRequiredCheck);
if (Object.keys(issues).length > 0) {
if (foundIssues.parameters === undefined) {
foundIssues.parameters = {};
}
if (foundIssues.parameters[nodeProperties.name] === undefined) {
foundIssues.parameters[nodeProperties.name] = [];
}
foundIssues.parameters = { ...foundIssues.parameters, ...issues };
}
}
}

View file

@ -465,7 +465,13 @@ export class Workflow {
const returnArray: any[] = [];
for (const currentValue of parameterValue) {
returnArray.push(this.renameNodeInParameterValue(currentValue, currentName, newName));
returnArray.push(
this.renameNodeInParameterValue(
currentValue as NodeParameterValueType,
currentName,
newName,
),
);
}
return returnArray;

View file

@ -39,6 +39,7 @@ export {
isINodePropertiesList,
isINodePropertyCollectionList,
isINodePropertyOptionsList,
isResourceMapperValue,
} from './type-guards';
export { ExpressionExtensions } from './Extensions';

View file

@ -3,6 +3,7 @@ import type {
INodePropertyOptions,
INodePropertyCollection,
INodeParameterResourceLocator,
ResourceMapperValue,
} from './Interfaces';
export const isINodeProperties = (
@ -43,3 +44,13 @@ export const isValidResourceLocatorParameterValue = (
return !!value;
}
};
export const isResourceMapperValue = (value: unknown): value is ResourceMapperValue => {
return (
typeof value === 'object' &&
value !== null &&
'mappingMode' in value &&
'schema' in value &&
'value' in value
);
};

View file

@ -0,0 +1,206 @@
import { validateFieldType } from '@/NodeHelpers';
import type { DateTime } from 'luxon';
const VALID_ISO_DATES = [
'1994-11-05T08:15:30-05:00',
'1994-11-05T13:15:30Z',
'1997-07-16T19:20+01:00',
'1997-07-16T19:20:30+01:00',
'1997-07-16T19:20:30.45+01:00',
'2018-05-16',
'1972-06-30T23:59:40Z',
'2019-03-26T14:00:00.9Z',
'2019-03-26T14:00:00.4999Z',
'2023-05-17T10:52:32+0000',
'2023-05-17T10:52:32+0000',
];
const VALID_HTTP_DATES = [
'Wed, 21 Oct 2015 07:28:00 GMT',
'Wed, 01 Jun 2022 08:00:00 GMT',
'Tue, 15 Nov 1994 12:45:26 GMT',
'Wed, 1 Jun 2022 08:00:00 GMT',
];
const VALID_RFC_DATES = [
'Tue, 04 Jun 2013 07:40:03 -0400',
'Tue, 4 Jun 2013 02:24:39 +0530',
'Wed, 17 May 2023 10:52:32 +0000',
];
const VALID_SQL_DATES = ['2008-11-11', '2008-11-11 13:23:44'];
const INVALID_DATES = [
'1994-11-05M08:15:30-05:00',
'18-05-2020',
'',
'Wed, 17 May 2023 10:52:32',
'SMT, 17 May 2023 10:52:32',
'1685084980', // We are not supporting timestamps
'1685085012135',
1685084980,
1685085012135,
true,
[],
];
describe('Type Validation', () => {
it('should validate and cast ISO dates', () => {
VALID_ISO_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should validate and cast RFC 2822 dates', () => {
VALID_RFC_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should validate and cast HTTP dates', () => {
VALID_HTTP_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should validate and cast SQL dates', () => {
VALID_SQL_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(true);
expect((validationResult.newValue as DateTime).isValid).toBe(true);
});
});
it('should not validate invalid dates', () => {
INVALID_DATES.forEach((date) => {
const validationResult = validateFieldType('date', date, 'dateTime');
expect(validationResult.valid).toBe(false);
});
});
it('should validate boolean values properly', () => {
expect(validateFieldType('boolean', 'true', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', 'TRUE', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', 1, 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', '1', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', '01', 'boolean').newValue).toBe(true);
expect(validateFieldType('boolean', 'false', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', 'FALSE', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', '0', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', '000', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', '0000', 'boolean').newValue).toBe(false);
expect(validateFieldType('boolean', 0, 'boolean').newValue).toBe(false);
});
it('should not validate invalid boolean values', () => {
expect(validateFieldType('boolean', 'tru', 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 'fals', 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 1111, 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 2, 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', -1, 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 'yes', 'boolean').valid).toBe(false);
expect(validateFieldType('boolean', 'no', 'boolean').valid).toBe(false);
});
it('should validate and cast numbers', () => {
expect(validateFieldType('number', '1', 'number').newValue).toBe(1);
expect(validateFieldType('number', '-1', 'number').newValue).toBe(-1);
expect(validateFieldType('number', '1.1', 'number').newValue).toBe(1.1);
expect(validateFieldType('number', '-1.1', 'number').newValue).toBe(-1.1);
expect(validateFieldType('number', 1, 'number').newValue).toBe(1);
expect(validateFieldType('number', 'A', 'number').valid).toBe(false);
expect(validateFieldType('number', '1,1', 'number').valid).toBe(false);
expect(validateFieldType('number', true, 'number').valid).toBe(true);
expect(validateFieldType('number', '1972-06-30T23:59:40Z', 'number').valid).toBe(false);
expect(validateFieldType('number', [1, 2], 'number').valid).toBe(false);
});
it('should validate and cast JSON properly', () => {
expect(validateFieldType('json', '{"a": 1}', 'object').newValue).toEqual({ a: 1 });
expect(
validateFieldType('json', '{"a": 1, "b": { "c": 10, "d": "test"}}', 'object').valid,
).toEqual(true);
expect(validateFieldType('json', { name: 'John' }, 'object').valid).toEqual(true);
expect(
validateFieldType(
'json',
{ name: 'John', address: { street: 'Via Roma', city: 'Milano' } },
'object',
).valid,
).toEqual(true);
// Invalid value:
expect(validateFieldType('json', ['one', 'two'], 'object').valid).toEqual(false);
// eslint-disable-next-line prettier/prettier
expect(validateFieldType('json', ["one", "two"], 'object').valid).toEqual(false);
expect(validateFieldType('json', '1', 'object').valid).toEqual(false);
expect(validateFieldType('json', '[1]', 'object').valid).toEqual(false);
expect(validateFieldType('json', '1.1', 'object').valid).toEqual(false);
expect(validateFieldType('json', 1.1, 'object').valid).toEqual(false);
expect(validateFieldType('json', '"a"', 'object').valid).toEqual(false);
expect(validateFieldType('json', '{a: 1}', 'object').valid).toEqual(false);
expect(validateFieldType('json', '["apples", "oranges"]', 'object').valid).toEqual(false);
expect(validateFieldType('json', [{ name: 'john' }, { name: 'bob' }], 'object').valid).toEqual(
false,
);
expect(
validateFieldType('json', '[ { name: "john" }, { name: "bob" } ]', 'object').valid,
).toEqual(false);
});
it('should validate and cast arrays properly', () => {
expect(validateFieldType('array', '["apples", "oranges"]', 'array').newValue).toEqual([
'apples',
'oranges',
]);
expect(validateFieldType('array', '[1]', 'array').newValue).toEqual([1]);
expect(validateFieldType('array', '[1, 2]', 'array').newValue).toEqual([1, 2]);
// Invalid values:
expect(validateFieldType('array', '"apples", "oranges"', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1.1', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1, 2', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1. 2. 3', 'array').valid).toEqual(false);
expect(validateFieldType('array', '[1, 2, 3', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1, 2, 3]', 'array').valid).toEqual(false);
expect(validateFieldType('array', '{1, 2, {3, 4}, 5}', 'array').valid).toEqual(false);
expect(validateFieldType('array', '1, 2, {3, 4}, 5', 'array').valid).toEqual(false);
expect(validateFieldType('array', { name: 'John' }, 'array').valid).toEqual(false);
});
it('should validate options properly', () => {
expect(
validateFieldType('options', 'oranges', 'options', [
{ name: 'apples', value: 'apples' },
{ name: 'oranges', value: 'oranges' },
]).valid,
).toEqual(true);
expect(
validateFieldType('options', 'something else', 'options', [
{ name: 'apples', value: 'apples' },
{ name: 'oranges', value: 'oranges' },
]).valid,
).toEqual(false);
});
it('should validate and cast time properly', () => {
expect(validateFieldType('time', '23:23', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23+1000', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23-1000', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00+01:00', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00-01:00', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00+01', 'time').valid).toEqual(true);
expect(validateFieldType('time', '22:00:00-01', 'time').valid).toEqual(true);
expect(validateFieldType('time', '23:23:23:23', 'time').valid).toEqual(false);
expect(validateFieldType('time', '23', 'time').valid).toEqual(false);
expect(validateFieldType('time', 'foo', 'time').valid).toEqual(false);
expect(validateFieldType('time', '23:23:', 'time').valid).toEqual(false);
expect(validateFieldType('time', '23::23::23', 'time').valid).toEqual(false);
});
});