Story: [PAY-846](https://linear.app/n8n/issue/PAY-846) | Related:
https://github.com/n8n-io/n8n/pull/7225
For the S3 backend for external storage of binary data and execution
data, the `getAsStream` method in the binary data manager interface used
by FS and S3 will need to become async. This is a breaking change for
nodes-base.
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)
This PR:
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.
Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.
Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
* Create new version for S3
* Update S3 to new aws s3 methods
* Switch from SAOP to Rest api
* Add multipart request
* Seperate stream into chunks and send the multipart
* Fix chunk into buffer
* Fix wrong sha256 mismatch
* Add abort multipart on error
* Complete multipart and list parts
* Change format to xml and add a minmum size of 5MB for each part
* Fix returned data for uploading a file
* Remove console.logs
* Seperate needed headers and multipart headers
* Throw error on aborting, remove console.logs
* Remove soap request from generic function
* Keep buffer
* Add unit test for V2
* fix upload file content body
* removed unused import
* Fix bug where the object was too smal and used only one part
* Fix naming for bucket name
* Fix issue with file name not returning data
* Add parent name
* Remove console.logs
* Add content type
* fix headears for other upload mode
---------
Co-authored-by: Marcus <marcus@n8n.io>
* Fix broken logic with type action
* Unit test base
* Add unit test using only nock
* Fix failing test
* remove console log
---------
Co-authored-by: Marcus <marcus@n8n.io>
* refactor(core): Add more overloads for string-type node parameter (no-changelog)
continued from https://github.com/n8n-io/n8n/pull/4797
* include binaryPropertyOutput
* ✨ Create rule `no-unneeded-backticks`
* 👕 Enable rule
* ⚡ Run rule on `cli`
* ⚡ Run rule on `core`
* ⚡ Run rule on `workflow`
* ⚡ Rule rule on `design-system`
* ⚡ Run rule on `node-dev`
* ⚡ Run rule on `editor-ui`
* ⚡ Run rule on `nodes-base`
* add or update documentation URLs for all credentials
* add UTM params to documentation urls even when they are absolute urls
* Setup a CI job to validate documentation urls after every release
* Fix FacebookGraphApi documentation URL
* also validate node documentation urls
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Added 'M' type to decode attribute for dynamodb
* Fixed bug with return all. Headers from the previous iteration were being passed causing an invalid signature error
* WIP: Node Actions List UI
* WIP: Recommended Actions and preseting of fields
* WIP: Resource category
* 🎨 Moved actions categorisation to the server
* 🏷️ Add missing INodeAction type
* ✨ Improve SSR categorisation, fix adding of mixed actions
* ♻️ Refactor CategorizedItems to composition api, style fixes
* WIP: Adding multiple nodes
* ♻️ Refactor rest of the NodeCreator component to composition API, conver globalLinkActions to composable
* ✨ Allow actions dragging, fix search and refactor passing of actions to categorized items
* 💄 Fix node actions title
* Migrate to the pinia store, add posthog feature and various fixes
* 🐛 Fix filtering of trigger actions when not merged
* fix: N8N-5439 — Do not use simple node item when at NodeHelperPanel root
* 🐛 Design review fixes
* 🐛 Fix disabling of merged actions
* Fix trigger root filtering
* ✨ Allow for custom node actions parser, introduce hubspot parser
* 🐛 Fix initial node params validation, fix position of second added node
* 🐛 Introduce operations category, removed canvas node names overrride, fix API actions display and prevent dragging of action nodes
* ✨ Prevent NDV auto-open feature flag
* 🐛 Inject recommened action for trigger nodes without actions
* Refactored NodeCreatorNode to Storybook, change filtering of merged nodes for the trigger helper panel, minor fixes
* Improve rendering of app nodes and animation
* Cleanup, any only enable accordion transition on triggerhelperpanel
* Hide node creator scrollbars in Firefox
* Minor styles fixes
* Do not copy the array in rendering method
* Removed unused props
* Fix memory leak
* Fix categorisation of regular nodes with a single resource
* Implement telemetry calls for node actions
* Move categorization to FE
* Fix client side actions categorisation
* Skip custom action show
* Only load tooltip for NodeIcon if necessary
* Fix lodash startCase import
* Remove lodash.startcase
* Cleanup
* Fix node creator autofocus on "tab"
* Prevent posthog getFeatureFlag from crashing
* Debugging preview env search issues
* Remove logs
* Make sure the pre-filled params are update not overwritten
* Get rid of transition in itemiterator
* WIP: Rough version of NodeActions keyboard navigation, replace nodeCreator composable with Pinia store module
* Rewrite to add support for ActionItem to ItemIterator and make CategorizedItems accept items props
* Fix category item counter & cleanup
* Add APIHint to actions search no-result, clean up NodeCreatorNode
* Improve node actions no results message
* Remove logging, fix filtering of recommended placeholder category
* Remove unused NodeActions component and node merging feature falg
* Do not show regular nodes without actions
* Make sure to add manual trigger when adding http node via actions hint
* Fixed api hint footer line height
* Prevent pointer-events od NodeIcon img and remove "this" from template
* Address PR points
* Fix e2e specs
* Make sure canvas ia loaded
* Make sure canvas ia loaded before opening nodeCreator in e2e spec
* Fix flaky workflows tags e2e getter
* Imrpove node creator click outside UX, add manual node to regular nodes added from trigger panel
* Add manual trigger node if dragging regular from trigger panel
* ⚡ enabled array-type
* ⚡ await-thenable on
* ⚡ ban-types on
* ⚡ default-param-last on
* ⚡ dot-notation on
* ⚡ member-delimiter-style on
* ⚡ no-duplicate-imports on
* ⚡ no-empty-interface on
* ⚡ no-floating-promises on
* ⚡ no-for-in-array on
* ⚡ no-invalid-void-type on
* ⚡ no-loop-func on
* ⚡ no-shadow on
* ⚡ ban-ts-comment re enabled
* ⚡ @typescript-eslint/lines-between-class-members on
* address my own comment
* @typescript-eslint/return-await on
* @typescript-eslint/promise-function-async on
* @typescript-eslint/no-unnecessary-boolean-literal-compare on
* @typescript-eslint/no-unnecessary-type-assertion on
* prefer-const on
* @typescript-eslint/prefer-optional-chain on
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
