Github issue / Community forum post (link here to close automatically):
---------
Co-authored-by: Giulio Andreini <andreini@netseven.it>
Co-authored-by: Marcus <marcus@n8n.io>
Story: [PAY-846](https://linear.app/n8n/issue/PAY-846) | Related:
https://github.com/n8n-io/n8n/pull/7225
For the S3 backend for external storage of binary data and execution
data, the `getAsStream` method in the binary data manager interface used
by FS and S3 will need to become async. This is a breaking change for
nodes-base.
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)
This PR:
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.
Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.
Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
* Create new version for S3
* Update S3 to new aws s3 methods
* Switch from SAOP to Rest api
* Add multipart request
* Seperate stream into chunks and send the multipart
* Fix chunk into buffer
* Fix wrong sha256 mismatch
* Add abort multipart on error
* Complete multipart and list parts
* Change format to xml and add a minmum size of 5MB for each part
* Fix returned data for uploading a file
* Remove console.logs
* Seperate needed headers and multipart headers
* Throw error on aborting, remove console.logs
* Remove soap request from generic function
* Keep buffer
* Add unit test for V2
* fix upload file content body
* removed unused import
* Fix bug where the object was too smal and used only one part
* Fix naming for bucket name
* Fix issue with file name not returning data
* Add parent name
* Remove console.logs
* Add content type
* fix headears for other upload mode
---------
Co-authored-by: Marcus <marcus@n8n.io>
* refactor(core): Add more overloads for string-type node parameter (no-changelog)
continued from https://github.com/n8n-io/n8n/pull/4797
* include binaryPropertyOutput
* ⚡ enabled array-type
* ⚡ await-thenable on
* ⚡ ban-types on
* ⚡ default-param-last on
* ⚡ dot-notation on
* ⚡ member-delimiter-style on
* ⚡ no-duplicate-imports on
* ⚡ no-empty-interface on
* ⚡ no-floating-promises on
* ⚡ no-for-in-array on
* ⚡ no-invalid-void-type on
* ⚡ no-loop-func on
* ⚡ no-shadow on
* ⚡ ban-ts-comment re enabled
* ⚡ @typescript-eslint/lines-between-class-members on
* address my own comment
* @typescript-eslint/return-await on
* @typescript-eslint/promise-function-async on
* @typescript-eslint/no-unnecessary-boolean-literal-compare on
* @typescript-eslint/no-unnecessary-type-assertion on
* prefer-const on
* @typescript-eslint/prefer-optional-chain on
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
* fix(nodes-base): fix and harmonize all primaryDocumentation links
* feat(workflow, cli): expose documentation links to UI via node codex
* fix(editor-ui): link to correct node and credential documentation URLs
* config(nodes-base): update 'format' script to also format node descriptor json
* chore: fix outdated links to node reference documentation
