This adds support for
1. custom delimiters
2. reading offsets to avoid having to read a large CSV all at once
3. excluding byte-order-mark
NODE-861
#7443
Depends on: #7092 | Story:
[PAY-768](https://linear.app/n8n/issue/PAY-768)
This PR:
- Generalizes the `IBinaryDataManager` interface.
- Adjusts `Filesystem.ts` to satisfy the interface.
- Sets up an S3 client stub to be filled in in the next PR.
- Turns `BinaryDataManager` into an injectable service.
- Adjusts the config schema and adds new validators.
Note that the PR looks large but all the main changes are in
`packages/core/src/binaryData`.
Out of scope:
- `BinaryDataManager` (now `BinaryDataService`) and `Filesystem.ts` (now
`fs.client.ts`) were slightly refactored for maintainability, but fully
overhauling them is **not** the focus of this PR, which is meant to
clear the way for the S3 implementation. Future improvements for these
two should include setting up a backwards-compatible dir structure that
makes it easier to locate binary data files to delete, removing
duplication, simplifying cloning methods, using integers for binary data
size instead of `prettyBytes()`, writing tests for existing binary data
logic, etc.
---------
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
* ⚡️test setup
* ⚡️fix 'testData' implicitly has an 'any' type.
* ✅ test github action file binary data reading
* ✅ checking for output binary equality
* ✅ writing files to different formats
* ✅ reading spreadsheet with different options
* ⚡️improve workflow file path replacement
* 🐛 fixing string.at() not supported in node 14.
* 🐛 trying to fix github action test error
* ⚡ fix for empty binary
* ⚡ switch for binary test
* ⚡️test helpers now return/compare json and binary (if not empty))
* ⚡ removed commented console log
---------
Co-authored-by: Michael Kret <michael.k@radency.com>
