* Log incoming jobs.
Log the full contents of the job protobuf to make debugging jobs easier
* Ensure that the parallel executor always uses at least one thread.
The caller may mis-calculate the number of CPUs as zero, in which case
ensure that at least one thread is spawned.
* Use runtime.NumCPU for CPU counts.
For hosts without docker, GetHostInfo() returns a blank struct which
has zero CPUs and causes downstream trouble.
---------
Co-authored-by: Paul Armstrong <psa@users.noreply.gitea.com>
Co-authored-by: Jason Song <i@wolfogre.com>
* feat: use logger from context wherever possible
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add step/job id and results to json logs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: value to be masked should not be hard-coded in the action
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* fix: replace values following ::add-mask:: in evaluated strings
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: [DEBUG] identifier for debug logs to distinguish them
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: replace logger with step logger
The container gets injected a job logger, but during the time that steps
are run, we want to use the step logger.
This commit wraps pre/main/post steps in an executor that replaces the
job logger with a step logger.
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add pre/post stage identifier fields to json log output
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add job/step result status to skipped steps/jobs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* feat: run jobs in parallel
This changes fixes and restructures the parallel execution of jobs.
The previous changes limiting the parallel execution did break this
and allowed only one job in parallel.
While we run #CPU jobs in parallel now, the jobs added per job-matrix
add to this. So we might over-commit to the capacity, but at least
it is limited.
* fix: correctly build job pipeline
The job pipeline should just append all required pipeline steps.
The parallelism will be handled by the ParallelExecutor and we
shouldn't handle it during building the pipelines.
Also this adds a test, that the ParallelExecutor does run
a limited amount of parallel goroutines.
* test: correct test implementation
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* refactor: remove debug error output
Errors should always be logged with an error level and not debug level.
Since the error is returned here, it will be logged later as an error.
Presumably this was a leftover from debugging the executor chain in:
PR: #971
* refactor: debug log wich expression is going to be evaluated
* fix: handle nil in EvalBool
We've seen this issue when the env map is not set-up properly,
i.e. when the env map is nil, EvalBool might return nil, which should
be handled as a falsy value.
* fix: fail on error in if expression and return the evaluation error
Stop running the workflow in case an expression cannot be evaluated.
Fixes: #1008
* fix: remove quotes from inside expression syntax in test
It looks like having an expression inside double quotes inside the
expression syntax is not valid: https://github.com/ZauberNerd/act-test/actions/runs/1881986429
The workflow is not valid. .github/workflows/test.yml (Line: 10, Col: 13): Unexpected symbol: '"endsWith'. Located at position 1 within expression: "endsWith('Hello world', 'ld')"
* refactor: export IsTruthy function from exprparser package
* refactor: use IsTruthy function in EvalBool
* refactor: move debug log for expression rewrite to rewrite function
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix: continue jobs + steps after failure
To allow proper if expression handling on jobs and steps (like always,
success, failure, ...) we need to continue running all executors in
the prepared chain.
To keep the error handling intact we add an occurred error to the
go context and handle it later in the pipeline/chain.
Also we add the job result to the needs context to give expressions
access to it.
The needs object, failure and success functions are split between
run context (on jobs) and step context.
Closes#442
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* style: correct linter warnings
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* fix: job if value defaults to success()
As described in the documentation, a default value of "success()" is
applied when no "if" value is present on the job.
https://docs.github.com/en/actions/learn-github-actions/expressions#job-status-check-functions
Co-authored-by: Markus Wolf <mail@markus-wolf.de>
* fix: check job needs recursively
Ensure job result includes results of previous jobs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: add runner test for job status check functions
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: add unit tests for run context if evaluation
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* refactor: move if expression evaluation
Move if expression evaluation into own function (step context) to
better support unit testing.
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* test: add unit tests for step context if evaluation
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* fix: handle job error more resilient
The job error is not stored in a context map instead of a context
added value.
Since context values are immutable an added value requires to keep
the new context in all cases. This is fragile since it might slip
unnoticed to other parts of the code.
Storing the error of a job in the context map will make it more stable,
since the map is always there and the context of the pipeline is stable
for the whole run.
* feat: steps should use a default if expression of success()
* test: add integration test for if-expressions
* chore: disable editorconfig-checker for yaml multiline string
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
