n8n/packages/nodes-base/nodes/Postgres/v2/actions/database/update.operation.ts

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

353 lines
9.4 KiB
TypeScript
Raw Normal View History

import type {
IDataObject,
IExecuteFunctions,
INodeExecutionData,
INodeProperties,
} from 'n8n-workflow';
import { NodeOperationError } from 'n8n-workflow';
2023-04-03 08:18:01 -07:00
import type {
PgpDatabase,
QueriesRunner,
QueryValues,
QueryWithValues,
} from '../../helpers/interfaces';
import {
addReturning,
checkItemAgainstSchema,
configureTableSchemaUpdater,
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
doesRowExist,
2023-04-03 08:18:01 -07:00
getTableSchema,
prepareItem,
replaceEmptyStringsByNulls,
} from '../../helpers/utils';
import { optionsCollection } from '../common.descriptions';
feat(editor): SQL editor overhaul (#6282) * Draft setup * ⚡ Implemented expression evaluation in Postgres node, minor SQL editor UI improvements, minor refacring * ⚡ Added initial version of expression preview for SQL editor * ⚡ Linking npm package for codemirror sql grammar instead of a local file * ⚡ Moving expression editor wrapper elements to the component * ⚡ Using expression preview in SQL editor * Use SQL parser skipping whitespace * ✨ Added support for custom skipped segments specification * ✨ Fixing highlight problems with dots and expressions that resolve to zero * 👕 Fixing linting error * ✨ Added current item support * ⚡ Added expression support to more nodes with sql editor * ✨ Added expression support for other nodes * ✨ Implemented different SQL dialect support * 🐛 Fixing hard-coded parameter names for editors * ✨ Fixing preview for nested queries, updating query when input data changes, adding keyboard shortcut to toggle comments * ✨ Adding a custom automcomplete notice for different editors * ⚡ Updating SQL autocomplete notice * ✅ Added unit tests for SQL editor * ⚡ Using latest grammar * 🐛 Fixing code node editor rendering * 💄 SQL preview dropdown matches editor width. Removing unnecessary css * ⚡ Addressing PR review feedback * 👌 Addressing PR review feedback pt2 * 👌 Added path alias for utils in nodes-base package * 👌 Addressing more PR review feedback * ✅ Adding tests for `getResolvables` utility function * ⚡Fixing lodash imports * 👌 Better focus handling, adding more plugins to the editor, other minor imrovements * ⚡ Not showing SQL autocomplete suggestions inside expressions * ⚡ Using npm package for sql grammar * ⚡ Removing autocomplete notice, adding line highlight on syntax error * 👌 Addressing code review feedback --------- Co-authored-by: Milorad Filipovic <milorad@n8n.io>
2023-06-22 07:47:28 -07:00
import { updateDisplayOptions } from '@utils/utilities';
2023-04-03 08:18:01 -07:00
const properties: INodeProperties[] = [
{
displayName: 'Data Mode',
name: 'dataMode',
type: 'options',
options: [
{
name: 'Auto-Map Input Data to Columns',
value: 'autoMapInputData',
description: 'Use when node input properties names exactly match the table column names',
},
{
name: 'Map Each Column Manually',
value: 'defineBelow',
description: 'Set the value for each destination column manually',
},
],
default: 'autoMapInputData',
description:
'Whether to map node input properties and the table data automatically or manually',
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
2023-04-03 08:18:01 -07:00
},
{
displayName: `
In this mode, make sure incoming data fields are named the same as the columns in your table. If needed, use an 'Edit Fields' node before this node to change the field names.
2023-04-03 08:18:01 -07:00
`,
name: 'notice',
type: 'notice',
default: '',
displayOptions: {
show: {
dataMode: ['autoMapInputData'],
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
'@version': [2],
2023-04-03 08:18:01 -07:00
},
},
},
{
// eslint-disable-next-line n8n-nodes-base/node-param-display-name-wrong-for-dynamic-options
2023-04-03 08:18:01 -07:00
displayName: 'Column to Match On',
name: 'columnToMatchOn',
type: 'options',
required: true,
// eslint-disable-next-line n8n-nodes-base/node-param-description-wrong-for-dynamic-options
2023-04-03 08:18:01 -07:00
description:
'The column to compare when finding the rows to update. Choose from the list, or specify an ID using an <a href="https://docs.n8n.io/code-examples/expressions/" target="_blank">expression</a>.',
2023-04-03 08:18:01 -07:00
typeOptions: {
loadOptionsMethod: 'getColumns',
loadOptionsDependsOn: ['schema.value', 'table.value'],
},
default: '',
hint: 'The column that identifies the row(s) to modify',
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
displayOptions: {
show: {
'@version': [2, 2.1],
},
},
2023-04-03 08:18:01 -07:00
},
{
displayName: 'Value of Column to Match On',
name: 'valueToMatchOn',
type: 'string',
default: '',
description:
'Rows with a value in the specified "Column to Match On" that corresponds to the value in this field will be updated',
displayOptions: {
show: {
dataMode: ['defineBelow'],
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
'@version': [2, 2.1],
2023-04-03 08:18:01 -07:00
},
},
},
{
displayName: 'Values to Send',
name: 'valuesToSend',
placeholder: 'Add Value',
type: 'fixedCollection',
typeOptions: {
multipleValueButtonText: 'Add Value',
multipleValues: true,
},
displayOptions: {
show: {
dataMode: ['defineBelow'],
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
'@version': [2, 2.1],
2023-04-03 08:18:01 -07:00
},
},
default: {},
options: [
{
displayName: 'Values',
name: 'values',
values: [
{
// eslint-disable-next-line n8n-nodes-base/node-param-display-name-wrong-for-dynamic-options
displayName: 'Column',
name: 'column',
type: 'options',
// eslint-disable-next-line n8n-nodes-base/node-param-description-wrong-for-dynamic-options
2023-04-03 08:18:01 -07:00
description:
'Choose from the list, or specify an ID using an <a href="https://docs.n8n.io/code-examples/expressions/" target="_blank">expression</a>',
2023-04-03 08:18:01 -07:00
typeOptions: {
loadOptionsMethod: 'getColumnsWithoutColumnToMatchOn',
loadOptionsDependsOn: ['schema.value', 'table.value'],
},
default: [],
},
{
displayName: 'Value',
name: 'value',
type: 'string',
default: '',
},
],
},
],
},
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
{
displayName: 'Columns',
name: 'columns',
type: 'resourceMapper',
noDataExpression: true,
default: {
mappingMode: 'defineBelow',
value: null,
},
required: true,
typeOptions: {
loadOptionsDependsOn: ['table.value', 'operation'],
resourceMapper: {
resourceMapperMethod: 'getMappingColumns',
mode: 'update',
fieldWords: {
singular: 'column',
plural: 'columns',
},
addAllFields: true,
multiKeyMatch: true,
},
},
displayOptions: {
show: {
'@version': [2.2, 2.3],
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
},
},
},
2023-04-03 08:18:01 -07:00
optionsCollection,
];
const displayOptions = {
show: {
resource: ['database'],
operation: ['update'],
},
hide: {
table: [''],
},
};
export const description = updateDisplayOptions(displayOptions, properties);
export async function execute(
this: IExecuteFunctions,
runQueries: QueriesRunner,
items: INodeExecutionData[],
nodeOptions: IDataObject,
db: PgpDatabase,
): Promise<INodeExecutionData[]> {
items = replaceEmptyStringsByNulls(items, nodeOptions.replaceEmptyStrings as boolean);
const nodeVersion = nodeOptions.nodeVersion as number;
let schema = this.getNodeParameter('schema', 0, undefined, {
extractValue: true,
}) as string;
let table = this.getNodeParameter('table', 0, undefined, {
extractValue: true,
}) as string;
const updateTableSchema = configureTableSchemaUpdater(schema, table);
let tableSchema = await getTableSchema(db, schema, table);
2023-04-03 08:18:01 -07:00
const queries: QueryWithValues[] = [];
for (let i = 0; i < items.length; i++) {
schema = this.getNodeParameter('schema', i, undefined, {
2023-04-03 08:18:01 -07:00
extractValue: true,
}) as string;
table = this.getNodeParameter('table', i, undefined, {
2023-04-03 08:18:01 -07:00
extractValue: true,
}) as string;
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
const columnsToMatchOn: string[] =
nodeVersion < 2.2
? [this.getNodeParameter('columnToMatchOn', i) as string]
: (this.getNodeParameter('columns.matchingColumns', i) as string[]);
2023-04-03 08:18:01 -07:00
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
const dataMode =
nodeVersion < 2.2
? (this.getNodeParameter('dataMode', i) as string)
: (this.getNodeParameter('columns.mappingMode', i) as string);
2023-04-03 08:18:01 -07:00
let item: IDataObject = {};
let valueToMatchOn: string | IDataObject = '';
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
if (nodeVersion < 2.2) {
valueToMatchOn = this.getNodeParameter('valueToMatchOn', i) as string;
}
2023-04-03 08:18:01 -07:00
if (dataMode === 'autoMapInputData') {
item = items[i].json;
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
if (nodeVersion < 2.2) {
valueToMatchOn = item[columnsToMatchOn[0]] as string;
}
2023-04-03 08:18:01 -07:00
}
if (dataMode === 'defineBelow') {
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
const valuesToSend =
nodeVersion < 2.2
? ((this.getNodeParameter('valuesToSend', i, []) as IDataObject).values as IDataObject[])
: ((this.getNodeParameter('columns.values', i, []) as IDataObject)
.values as IDataObject[]);
if (nodeVersion < 2.2) {
item = prepareItem(valuesToSend);
item[columnsToMatchOn[0]] = this.getNodeParameter('valueToMatchOn', i) as string;
} else {
item = this.getNodeParameter('columns.value', i) as IDataObject;
}
2023-04-03 08:18:01 -07:00
}
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
const matchValues: string[] = [];
if (nodeVersion < 2.2) {
if (!item[columnsToMatchOn[0]] && dataMode === 'autoMapInputData') {
throw new NodeOperationError(
this.getNode(),
"Column to match on not found in input item. Add a column to match on or set the 'Data Mode' to 'Define Below' to define the value to match on.",
);
}
matchValues.push(valueToMatchOn);
matchValues.push(columnsToMatchOn[0]);
} else {
columnsToMatchOn.forEach((column) => {
matchValues.push(column);
matchValues.push(item[column] as string);
});
const rowExists = await doesRowExist(db, schema, table, matchValues);
if (!rowExists) {
const descriptionValues: string[] = [];
matchValues.forEach((val, index) => {
if (index % 2 === 0) {
descriptionValues.push(`${matchValues[index]}=${matchValues[index + 1]}`);
}
});
throw new NodeOperationError(
this.getNode(),
"The row you are trying to update doesn't exist",
{
description: `No rows matching the provided values (${descriptionValues.join(
', ',
)}) were found in the table "${table}".`,
itemIndex: i,
},
);
}
}
tableSchema = await updateTableSchema(db, tableSchema, schema, table);
2023-04-03 08:18:01 -07:00
item = checkItemAgainstSchema(this.getNode(), item, tableSchema, i);
let values: QueryValues = [schema, table];
let valuesLength = values.length + 1;
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
let condition = '';
if (nodeVersion < 2.2) {
condition = `$${valuesLength}:name = $${valuesLength + 1}`;
valuesLength = valuesLength + 2;
values.push(columnsToMatchOn[0], valueToMatchOn);
} else {
const conditions: string[] = [];
for (const column of columnsToMatchOn) {
conditions.push(`$${valuesLength}:name = $${valuesLength + 1}`);
valuesLength = valuesLength + 2;
values.push(column, item[column] as string);
}
condition = conditions.join(' AND ');
}
2023-04-03 08:18:01 -07:00
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
const updateColumns = Object.keys(item).filter((column) => !columnsToMatchOn.includes(column));
2023-04-03 08:18:01 -07:00
if (!Object.keys(updateColumns).length) {
throw new NodeOperationError(
this.getNode(),
"Add values to update to the input item or set the 'Data Mode' to 'Define Below' to define the values to update.",
);
}
2023-04-03 08:18:01 -07:00
const updates: string[] = [];
for (const column of updateColumns) {
updates.push(`$${valuesLength}:name = $${valuesLength + 1}`);
valuesLength = valuesLength + 2;
values.push(column, item[column] as string);
}
let query = `UPDATE $1:name.$2:name SET ${updates.join(', ')} WHERE ${condition}`;
const outputColumns = this.getNodeParameter('options.outputColumns', i, ['*']) as string[];
[query, values] = addReturning(query, outputColumns, values);
queries.push({ query, values });
}
feat(editor): Implement Resource Mapper component (#6207) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * feat(Google Sheets Node): Implement Resource mapper in Google Sheets node (#5752) * ✨ Added initial resource mapping support in google sheets node * ✨ Wired mapping API endpoint with node-specific logic for fetching mapping fields * ✨ Implementing mapping fields logic for google sheets * ✨ Updating Google Sheets execute methods to support resource mapper fields * 🚧 Added initial version of `ResourceLocator` component * 👌 Added `update` mode to resource mapper modes * 👌 Addressing PR feedback * 👌 Removing leftover const reference * 👕 Fixing lint errors * :zap: singlton for conections * :zap: credentials test fix, clean up * feat(Postgres Node): Add resource mapper to new version of Postgres node (#5814) * :zap: scaffolding * :zap: finished scaffolding * :zap: renamed types * :zap: updated subtitle * :zap: renamed functions file, UI updates * :zap: query parameters fixes, ui updates, refactoring * :zap: fixes for credentials test, setup for error parsing * :zap: rlc for schema and table, error handling tweaks * :zap: delete operation, new options * :zap: columns loader * :zap: linter fixes * :zap: where clauses setup * :zap: logic for processing where clauses * :zap: select operation * :zap: refactoring * :zap: data mode for insert and update, wip * :zap: data mapping, insert update, skip on conflict option * :zap: select columns with spaces fix * :zap: update operation update, wip * :zap: finished update operation * :zap: upsert operation * :zap: ui fixes * Copy updates. * Copy updates. * :zap: option to convert empty strings to nulls, schema checks * :zap: UI requested updates * :zap: ssh setup WIP * :zap: fixes, ssh WIP * :zap: ssh fixes, credentials * :zap: credentials testing update * :zap: uncaught error fix * :zap: clean up * :zap: address in use fix * :zap: improved error message * :zap: tests setup * :zap: unit tests wip * :zap: config files clean up * :zap: utils unit tests * :zap: refactoring * :zap: setup for testing operations, tests for deleteTable operation * :zap: executeQuery and insert operations tests * :zap: select, update, upsert operations tests * :zap: runQueries tests setup * :zap: hint to query * Copy updates. * :zap: ui fixes * :zap: clean up * :zap: error message update * :zap: ui update * Minor tweaks to query params decription. * ✨ Updated Postgres node to use resource mapper component * ✨ Implemented postgres <-> resource mapper type mapping * ✨ Updated Postgres node execution to use resource mapper fields in v3 * 🔥 Removing unused import --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com> * feat(core): Resource editor componend P0 (#5970) * ✨ Added inital value of mapping mode dropdown * ✨ Finished mapping mode selector * ✨ Finished implementing mapping mode selector * ✨ Implemented 'Columns to match on' dropdown * ✨ Implemented `loadOptionsDependOn` support in resource mapper * ✨ Implemented initial version of mapping fields * ✨ Implementing dependant fields watcher in new component setup * ✨ Generating correct resource mapper field types. Added `supportAutoMap` to node specification and UI. Not showing fields with `display=false`. Pre-selecting matching columns if it's the only one * ✨ Handling matching columns correctly in UI * ✨ Saving and loading resourceMapper values in component * ✨ Implemented proper data saving and loading * ✨ ResourceMapper component refactor, fixing value save/load * ✨ Refactoring MatchingColumnSelect component. Updating Sheets node to use single key match and Postgres to use multi key * ✨ Updated Google Sheets node to work with the new UI * ✨ Updating Postgres Node to work with new UI * ✨ Additional loading indicator that shown if there is no mapping mode selector * ✨ Removing hard-coded values, fixing matching columns ordering, refactoring * ✨ Updating field names in nodes * ✨ Fixing minor UI issues * ✨ Implemented matching fields filter logic * ✨ Moving loading label outside of fields list * ✅ Added initial unit tests for resource mapper * ✅ Finished default rendering test * ✅ Test refactoring * ✅ Finished unit tests * 🔨 Updating the way i18n is used in resource mapper components * ✔️ Fixing value to match on logic for postgres node * ✨ Hiding mapping fields when auto-map mode is selected * ✨ Syncing selected mapping mode between components * ✨ Fixing dateTime input rendering and adding update check to Postgres node * ✨ Properly handling database connections. Sending null for empty string values. * 💄 Updated wording in the error message for non-existing rows * ✨ Fixing issues with selected matching values * ✔️ Updating unit tests after matching logic update * ✨ Updating matching columns when new fields are loaded * ✨ Defaulting to null for empty parameter values * ✨ Allowing zero as valid value for number imputs * ✨ Updated list of types that use datepicker as widger * ✨ Using text inputs for time types * ✨ Initial mapping field rework * ✨ Added new component for mapping fields, moved bit of logic from root component to matching selector, fixing some lint errors * ✨ Added tooltip for columns that cannot be deleted * ✨ Saving deleted values in parameter value * ✨ Implemented control to add/remove mapping fields * ✨ Syncing field list with add field dropdown when changing dependent values * ✨ Not showing removed fields in matching columns selector. Updating wording in matching columns selector description * ✨ Implementing disabled states for add/remove all fields options * ✨ Saving removed columns separately, updating copy * ✨ Implemented resource mapper values validation * ✨ Updated validation logic and error input styling * ✨ Validating resource mapper fields when new nodes are added * ✨ Using node field words in validation, refactoring resource mapper component * ✨ Implemented schema syncing and add/remove all fields * ✨ Implemented custom parameter actions * ✨ Implemented loading indicator in parameter options * 🔨 Removing unnecessary constants and vue props * ✨ Handling default values properly * ✨ Fixing validation logic * 👕 Fixing lint errors * ⚡ Fixing type issues * ⚡ Not showing fields by default if `addAllFields` is set to `false` * ✨ Implemented field type validation in resource mapper * ✨ Updated casing in copy, removed all/remove all option from bottom menu * ✨ Added auto mapping mode notice * ✨ Added support for more types in validation * ✨ Added support for enumerated values * ✨ Fixing imports after merging * ✨ Not showing removed fields in matching columns selector. Refactoring validation logic. * 👕 Fixing imports * ✔️ Updating unit tests * ✅ Added resource mapper schema tests * ⚡ Removing `match` from resource mapper field definition, fixing matching columns loading * ⚡ Fixed schema merging * :zap: update operation return data fix * :zap: review * 🐛 Added missing import * 💄 Updating parameter actions icon based on the ui review * 💄 Updating word capitalisation in tooltips * 💄 Added empty state to mapping fields list * 💄 Removing asterisk from fields, updating tooltips for matching fields * ⚡ Preventing matching fields from being removed by 'Remove All option' * ⚡ Not showing hidden fields in the `Add field` dropdown * ⚡ Added support for custom matching columns labels * :zap: query optimization * :zap: fix * ⚡ Optimizing Postgres node enumeration logic * ⚡ Added empty state for matching columns * ⚡ Only fully loading fields if there is no schema fetched * ⚡ Hiding mapping fields if there is no matching columns available in the schema * ✔️ Fixing minor issues * ✨ Implemented runtime type validation * 🔨 Refactoring validation logic * ✨ Implemented required check, added more custom messages * ✨ Skipping boolean type in required check * Type check improvements * ✨ Only reloading fields if dependent values actually change * ✨ Adding item index to validation error title * ✨ Updating Postgres fetching logic, using resource mapper mode to determine if a field can be deleted * ✨ Resetting field values when adding them via the addAll option * ⚡ Using minor version (2.2) for new Postgres node * ⚡ Implemented proper date validation and type casting * 👕 Consolidating typing * ✅ Added unit tests for type validations * 👌 Addressing front-end review comments * ⚡ More refactoring to address review changes * ⚡ Updating leftover props * ⚡ Added fallback for ISO dates with invalid timezones * Added timestamp to datetime test cases * ⚡ Reseting matching columns if operation changes * ⚡ Not forcing auto-increment fields to be filled in in Postgres node. Handling null values * 💄 Added a custom message for invalid dates * ⚡ Better handling of JSON values * ⚡ Updating codemirror readonly stauts based on component property, handling objects in json validation * Deleting leftover console.log * ⚡ Better time validation * ⚡ Fixing build error after merging * 👕 Fixing lint error * ⚡ Updating node configuration values * ⚡ Handling postgres arrays better * ⚡ Handling SQL array syntax * ⚡ Updating time validation rules to include timezone * ⚡ Sending expressions that resolve to `null` or `undefined` by the resource mapper to delete cell content in Google Sheets * ⚡ Allowing removed fields to be selected for match * ⚡ Updated the query for fetching unique columns and primary keys * ⚡ Optimizing the unique query * ⚡ Setting timezone to all parsed dates * ⚡ Addressing PR review feedback * ⚡ Configuring Sheets node for production, minor vue component update * New cases added to the TypeValidation test. * ✅ Tweaking validation rules for arrays/objects and updating test cases --------- Co-authored-by: Michael Kret <michael.k@radency.com> Co-authored-by: Giulio Andreini <g.andreini@gmail.com>
2023-05-31 02:56:09 -07:00
const results = await runQueries(queries, items, nodeOptions);
return results;
2023-04-03 08:18:01 -07:00
}