n8n/.github/workflows/test-workflows.yml

86 lines
2.6 KiB
YAML
Raw Normal View History

:rotating_light: Add Workflow testing framework (#1814) * Added flag to simplify output to execute command and created executeAll Also created a command that lists workflows so it can be used by other applications that wish to interact with n8n via CLI. * Added compare funcionality that helps us identify possible breaking changes * :zap: Add JSON flag to output execution result in json format * Add execution time to execution result * Add --output flag to save JSON formated results * Fix typos * Adding usage examples and warning message * Changing command to always output JSON information * :zap: Add concurrency to executeAll command * :sparkles: Add IDs filter to executeAll command * :pencil2: Fix typos * Fix lint issues * Improvements to execute all * Added colors and beautified execute all command output * Improving premature termination of execute all command * Refactored output * Added detection for warnings instead of errors for a few cases * Fixed linting * Improved process finishing * Added encryption key init, removed messages from non-debug run and created sample github actions file * Updated test command to use correct encryption key * Correcting paths * Changed command name and changed concurrency to be slot based * Added the retry functionality * Improved typing and fixed executions filtering * Updated test description * Added concurrency * Added skip list and concurrency * Fixed termination behavior, removed unnecessary code and added main mode to executions * Added special edge cases handling * Added safe debug flag for non tty terminals * Removed comparison and retries for testing * Changed the way we run commands to add more debug info * Test adding concurrency * Testing with regular testing procedure without comparison * Adding compare flag * Fixing short output command * Upgraded concurrency to 16, fixed short output and covered nodes count * Fixed short output condition * Correcting short output information * Correcting the output for short output * Added a string to possible warnings list * Added example usages with more arguments * Improvements to code made by Ivan * Removed colorize output setting (always use colors), moved execution timeout to become static and removed node list from executions * Moving types to a separate file on execute batch command * Changed interfaces file extension to prevent oclif warnings * Updated workflow pdf files * :zap: Change rule when tests run + minor formatting fixes Co-authored-by: dali <servfrdali@yahoo.fr> Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
2021-07-01 00:04:24 -07:00
name: Run test workflows
on:
schedule:
- cron: "0 2 * * *"
workflow_dispatch:
jobs:
run-test-workflows:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x]
steps:
-
name: Checkout
uses: actions/checkout@v2
with:
path: n8n
-
name: Checkout workflows repo
uses: actions/checkout@v2
with:
repository: n8n-io/test-workflows
path: test-workflows
-
name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}
-
name: npm install and build
run: |
cd n8n
npm install
npm run bootstrap
npm run build --if-present
env:
CI: true
shell: bash
-
name: Import credentials
run: n8n/packages/cli/bin/n8n import:credentials --input=test-workflows/credentials.json
shell: bash
env:
N8N_ENCRYPTION_KEY: ${{secrets.ENCRYPTION_KEY}}
-
name: Import workflows
run: n8n/packages/cli/bin/n8n import:workflow --separate --input=test-workflows/workflows
shell: bash
env:
N8N_ENCRYPTION_KEY: ${{secrets.ENCRYPTION_KEY}}
-
name: Copy static assets
run: |
cp n8n/assets/n8n-logo.png /tmp/n8n-logo.png
cp n8n/assets/n8n-screenshot.png /tmp/n8n-screenshot.png
cp n8n/node_modules/pdf-parse/test/data/05-versions-space.pdf /tmp/05-versions-space.pdf
cp n8n/node_modules/pdf-parse/test/data/04-valid.pdf /tmp/04-valid.pdf
shell: bash
-
name: Run tests
run: n8n/packages/cli/bin/n8n executeBatch --shallow --skipList=test-workflows/skipList.txt --shortOutput --concurrency=16 --compare=test-workflows/snapshots
shell: bash
env:
N8N_ENCRYPTION_KEY: ${{secrets.ENCRYPTION_KEY}}
-
name: Export credentials
if: always()
run: n8n/packages/cli/bin/n8n export:credentials --output=test-workflows/credentials.json --all --pretty
shell: bash
env:
N8N_ENCRYPTION_KEY: ${{secrets.ENCRYPTION_KEY}}
-
name: Commit and push credential changes
if: always()
run: |
cd test-workflows
git config --global user.name 'n8n test bot'
git config --global user.email 'n8n-test-bot@users.noreply.github.com'
git commit -am "Automated credential update"
git push --force --quiet "https://janober:${{ secrets.TOKEN }}@github.com/n8n-io/test-workflows.git" main:main