n8n/packages/core/src/UserSettings.ts
Jan Oberhauser c85faff4f1
feat: Make it possible to dynamically load community nodes (#2849)
*  Make it possible to dynamically load node packages

*  Fix comment

*  Make possible to dynamically install nodes from npm

* Created migration for sqlite regarding community nodes

* Saving to db whenever a package with nodes is installed

* Created endpoint to fetch installed packages

* WIP - uninstall package with nodes

* Fix lint issues

* Updating nodes via API

* Lint and improvement fixes

* Created community node helpers and removed packages taht do not contain nodes

* Check for package updates when fetching installed packages

* Blocked access to non-owner and preventing incorrect install of packages

* Added auto healing process

* Unit tests for helpers

* Finishing tests for helpers

* Improved unit tests, refactored more helpers and created integration tests for GET

* Implemented detection of missing packages on init and added warning to frontend settings

* Add check for banned packages and fix broken tests

* Create migrations for other db systems

* Updated with latest changes from master

* Fixed conflict errors

* Improved unit tests, refactored more helpers and created integration tests for GET

* Implemented detection of missing packages on init and added warning to frontend settings

* 🔥 Removing access check for the Settings sidebar item

*  Added inital community nodes settings screen

* Added executionMode flag to settings

*  Implemented N8N-callout component

* 💄Updating Callout component template propery names

* 💄 Updating Callout component styling.

* 💄Updating Callout component sizing and colors.

* ✔️ Updating Callout component test snapshots after styling changes

*  Updating the `ActionBox` component so it supports callouts and conditional button rendering

* 💄 Removing duplicate callout theme validation in the `ActionBox` component. Adding a selection control for it in the storybook.

*  Added warning message if instance is in the queue mode. Updated colors based on the new design.

*  Added a custom permission support to router

* 🔨 Implemented UM detection as a custom permission.

* 👌Updating route permission logic.

*  Implemented installed community packages list in the settings view

* 👌 Updating settings routes rules and community nodes setting view.

* Allow installation of packages that failed to load

* 👌 Updating `ActionBox`, `CommuntyPackageCard` components and settings loading logic.

* 👌 Fixing community nodes loading state and sidebar icon spacing.

*  Implemented loading skeletons for community package cards

* 👌 Handling errrors while loading installed package list. Updating spacing.

* 👌 Updating community nodes error messages.

* Added disable flag

* 🐛 Fixing a community nodes update detection bug when there are missing packages. (#3497)

*  Added front-end support for community nodes feature flag

*  Implemented community package installation modal dialog

* 💄 Community nodes installation modal updates: Moved links to constants and used them in translations, disabling inputs in loading state.

*  Implemented community packages install flow

* Standardize error codes (#3501)

* Standardize error: 400 for request issues such as invalid package name and 500 for installation problems

* Fix http status code for when package is not found

*  Implemented community package installation modal dialog

* 💄 Community nodes installation modal updates: Moved links to constants and used them in translations, disabling inputs in loading state.

*  Implemented community packages install flow

*  Updated error handling based on the response codes

*  Implemented community package installation modal dialog

*  Implemented community package uninstall flow.

*  Finished update confirm modal UI

* 💄 Replaced community nodes tooltip image with the one exported from figma.

*  Implemented community package update process

*  Updating community nodes list after successful package update

* 🔒 Updating public API setting route to use new access rules. Updating express app definition in community nodes tests

*  Implemented community package installation modal dialog

* 💄 Community nodes installation modal updates: Moved links to constants and used them in translations, disabling inputs in loading state.

*  Implemented community packages install flow

*  Updated error handling based on the response codes

* Change output for installation request

* Improve payload for update requests

* 👌 Updating community nodes install modal UI

* 👌 Updating community nodes confirm modal logic

* 👌 Refactoring community nodes confirm modal dialog

* 👌 Separating community nodes components loading states

* 💄 Updating community nodes install modal spacing.

* Fix behavior for installing already installed packages

* 💡 Commenting community nodes install process

* 🔥 Removing leftover commits of deleted Vue mutations

*  Updated node list to identify community nodes and handle node name clash

*  Implemented missing community node dialog.

* 💄 Updating n8n-tabs component to support tooltips

*  Updating node details with community node details.

* 🔨 Using back-end response when updating community packages

* 👌 Updating tabs component and refactoring community nodes store mutations

* 👌 Adding community node flag to node type descriptions and using it to identify community nodes

* 👌 Hiding unnecessary elements from missing node details panel.

* 👌 Updating missing node type descriptions for custom and community nodes

* 👌 Updating community node package name detection logic

* 👌 Removing communityNode flag from node description

*  Adding `force` flag to credentials fetching (#3527)

*  Adding `force` flag to credentials fetching which can be used to skip check when loading credentials

*  Forcing credentials loading when opening nodeView

* 👌 Minor updates to community nodes details panel

* tests for post endpoint

* duplicate comments

* Add Patch and Delete enpoints tests

* 🔒 Using `pageCategory`prop to assemble the list of settings routes instead of hard-coded array (#3562)

* 📈 Added front-end telemetry events for community nodes

* 📈 Updating community nodes telemetry events

* 💄 Updating community nodes settings UI elements based on product/design review

* 💄 Updating node view & node details view for community nodes based on product/design feedback

* 💄 Fixing community node text capitalisation

*  Adding community node install error message under the package name input field

* Fixed and improved tests

* Fix lint issue

* feat: Migrated to npm release of riot-tmpl fork.

* 📈 Updating community nodes telemetry events based on the product review

* 💄 Updating community nodes UI based on the design feedback

* 🔀 Merging recent node draggable panels changes

* Implement self healing process

* Improve error messages for package name requirement and disk space

* 💄 Removing front-end error message override since appropriate response is available from the back-end

* Fix lint issues

* Fix installed node name

* 💄 Removed additional node name parsing

* 📈 Updating community nodes telemetry events

* Fix postgres migration for cascading nodes when package is removed

* Remove postman mock for banned packages

* 📈 Adding missing telemetry event for community node documentation click

* 🐛 Fixing community nodes UI bugs reported during the bug bash

* Fix issue with uninstalling packages not reflecting UI

* 🐛 Fixing a missing node type bug when trying to run a workflow.

* Improve error detection for installing packages

* 💄 Updating community nodes components styling and wording based on the product feedback

* Implement telemetry be events

* Add author name and email to packages

* Fix telemetry be events for community packages

* 📈 Updating front-end telemetry events with community nodes author data

* 💄 Updating credentials documentation link logic to handle community nodes credentials

* 🐛 Fixing draggable panels logic

* Fix duplicate wrong import

* 💄 Hiding community nodes credentials documentation links when they don't contain an absolute URL

* Fix issue with detection of missing packages

* 💄 Adding the `Docs` tab to community nodes

* 💄 Adding a failed loading indicator to community nodes list

* Prevent n8n from crashing on startup

* Refactor and improve code quality

*  Remove not needed depenedency

Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Milorad Filipović <milorad@n8n.io>
Co-authored-by: Milorad FIlipović <miloradfilipovic19@gmail.com>
Co-authored-by: agobrech <ael.gobrecht@gmail.com>
Co-authored-by: Alex Grozav <alex@grozav.com>
2022-07-20 16:24:03 +02:00

302 lines
7.2 KiB
TypeScript

/* eslint-disable no-param-reassign */
/* eslint-disable @typescript-eslint/no-use-before-define */
/* eslint-disable @typescript-eslint/no-unsafe-call */
/* eslint-disable @typescript-eslint/no-unsafe-assignment */
import fs from 'fs';
import path from 'path';
import { createHash, randomBytes } from 'crypto';
// eslint-disable-next-line import/no-cycle
import {
ENCRYPTION_KEY_ENV_OVERWRITE,
EXTENSIONS_SUBDIRECTORY,
DOWNLOADED_NODES_SUBDIRECTORY,
IUserSettings,
RESPONSE_ERROR_MESSAGES,
USER_FOLDER_ENV_OVERWRITE,
USER_SETTINGS_FILE_NAME,
USER_SETTINGS_SUBFOLDER,
} from '.';
// eslint-disable-next-line @typescript-eslint/no-var-requires
const { promisify } = require('util');
const fsAccess = promisify(fs.access);
const fsReadFile = promisify(fs.readFile);
const fsMkdir = promisify(fs.mkdir);
const fsWriteFile = promisify(fs.writeFile);
let settingsCache: IUserSettings | undefined;
/**
* Creates the user settings if they do not exist yet
*
* @export
*/
export async function prepareUserSettings(): Promise<IUserSettings> {
const settingsPath = getUserSettingsPath();
let userSettings = await getUserSettings(settingsPath);
if (userSettings !== undefined) {
// Settings already exist, check if they contain the encryptionKey
if (userSettings.encryptionKey !== undefined) {
// Key already exists
if (userSettings.instanceId === undefined) {
userSettings.instanceId = await generateInstanceId(userSettings.encryptionKey);
settingsCache = userSettings;
}
return userSettings;
}
} else {
userSettings = {};
}
if (process.env[ENCRYPTION_KEY_ENV_OVERWRITE] !== undefined) {
// Use the encryption key which got set via environment
userSettings.encryptionKey = process.env[ENCRYPTION_KEY_ENV_OVERWRITE];
} else {
// Generate a new encryption key
userSettings.encryptionKey = randomBytes(24).toString('base64');
}
userSettings.instanceId = await generateInstanceId(userSettings.encryptionKey);
// eslint-disable-next-line no-console
console.log(`UserSettings were generated and saved to: ${settingsPath}`);
return writeUserSettings(userSettings, settingsPath);
}
/**
* Returns the encryption key which is used to encrypt
* the credentials.
*
* @export
* @returns
*/
export async function getEncryptionKey(): Promise<string> {
if (process.env[ENCRYPTION_KEY_ENV_OVERWRITE] !== undefined) {
return process.env[ENCRYPTION_KEY_ENV_OVERWRITE] as string;
}
const userSettings = await getUserSettings();
if (userSettings === undefined || userSettings.encryptionKey === undefined) {
throw new Error(RESPONSE_ERROR_MESSAGES.NO_ENCRYPTION_KEY);
}
return userSettings.encryptionKey;
}
/**
* Returns the instance ID
*
* @export
* @returns
*/
export async function getInstanceId(): Promise<string> {
const userSettings = await getUserSettings();
if (userSettings === undefined) {
return '';
}
if (userSettings.instanceId === undefined) {
return '';
}
return userSettings.instanceId;
}
async function generateInstanceId(key?: string) {
const hash = key
? createHash('sha256')
.update(key.slice(Math.round(key.length / 2)))
.digest('hex')
: undefined;
return hash;
}
/**
* Adds/Overwrite the given settings in the currently
* saved user settings
*
* @export
* @param {IUserSettings} addSettings The settings to add/overwrite
* @param {string} [settingsPath] Optional settings file path
* @returns {Promise<IUserSettings>}
*/
export async function addToUserSettings(
addSettings: IUserSettings,
settingsPath?: string,
): Promise<IUserSettings> {
if (settingsPath === undefined) {
settingsPath = getUserSettingsPath();
}
let userSettings = await getUserSettings(settingsPath);
if (userSettings === undefined) {
userSettings = {};
}
// Add the settings
Object.assign(userSettings, addSettings);
return writeUserSettings(userSettings, settingsPath);
}
/**
* Writes a user settings file
*
* @export
* @param {IUserSettings} userSettings The settings to write
* @param {string} [settingsPath] Optional settings file path
* @returns {Promise<IUserSettings>}
*/
export async function writeUserSettings(
userSettings: IUserSettings,
settingsPath?: string,
): Promise<IUserSettings> {
if (settingsPath === undefined) {
settingsPath = getUserSettingsPath();
}
if (userSettings === undefined) {
userSettings = {};
}
// Check if parent folder exists if not create it.
try {
await fsAccess(path.dirname(settingsPath));
} catch (error) {
// Parent folder does not exist so create
await fsMkdir(path.dirname(settingsPath));
}
const settingsToWrite = { ...userSettings };
if (settingsToWrite.instanceId !== undefined) {
delete settingsToWrite.instanceId;
}
await fsWriteFile(settingsPath, JSON.stringify(settingsToWrite, null, '\t'));
settingsCache = JSON.parse(JSON.stringify(userSettings));
return userSettings;
}
/**
* Returns the content of the user settings
*
* @export
* @returns {UserSettings}
*/
export async function getUserSettings(
settingsPath?: string,
ignoreCache?: boolean,
): Promise<IUserSettings | undefined> {
if (settingsCache !== undefined && ignoreCache !== true) {
return settingsCache;
}
if (settingsPath === undefined) {
settingsPath = getUserSettingsPath();
}
try {
await fsAccess(settingsPath);
} catch (error) {
// The file does not exist
return undefined;
}
const settingsFile = await fsReadFile(settingsPath, 'utf8');
try {
settingsCache = JSON.parse(settingsFile);
} catch (error) {
throw new Error(
`Error parsing n8n-config file "${settingsPath}". It does not seem to be valid JSON.`,
);
}
return settingsCache as IUserSettings;
}
/**
* Returns the path to the user settings
*
* @export
* @returns {string}
*/
export function getUserSettingsPath(): string {
const n8nFolder = getUserN8nFolderPath();
return path.join(n8nFolder, USER_SETTINGS_FILE_NAME);
}
/**
* Retruns the path to the n8n folder in which all n8n
* related data gets saved
*
* @export
* @returns {string}
*/
export function getUserN8nFolderPath(): string {
let userFolder;
if (process.env[USER_FOLDER_ENV_OVERWRITE] !== undefined) {
userFolder = process.env[USER_FOLDER_ENV_OVERWRITE] as string;
} else {
userFolder = getUserHome();
}
return path.join(userFolder, USER_SETTINGS_SUBFOLDER);
}
/**
* Returns the path to the n8n user folder with the custom
* extensions like nodes and credentials
*
* @export
* @returns {string}
*/
export function getUserN8nFolderCustomExtensionPath(): string {
return path.join(getUserN8nFolderPath(), EXTENSIONS_SUBDIRECTORY);
}
/**
* Returns the path to the n8n user folder with the nodes that
* have been downloaded
*
* @export
* @returns {string}
*/
export function getUserN8nFolderDowloadedNodesPath(): string {
return path.join(getUserN8nFolderPath(), DOWNLOADED_NODES_SUBDIRECTORY);
}
/**
* Returns the home folder path of the user if
* none can be found it falls back to the current
* working directory
*
* @export
* @returns {string}
*/
export function getUserHome(): string {
let variableName = 'HOME';
if (process.platform === 'win32') {
variableName = 'USERPROFILE';
}
if (process.env[variableName] === undefined) {
// If for some reason the variable does not exist
// fall back to current folder
return process.cwd();
}
return process.env[variableName] as string;
}