For log processing of the JSON logs, we want to be able to know which
keys/values of the matrices were used.
This commit adds the current matrix map to the job logger.
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* Made env interpolated instead of evaluated.
* [skip ci] Add Test Workflow file
* Activate Test
* fix Test
Co-authored-by: ChristopherHX <christopher.homberger@web.de>
* feat(#1161): add --through-action to assigned actions from GitHub
* docs(flags): add --through-action and --through-action-token flags description
* test(action, remote): add test case for ThroughAction
* refactor(command): rename command from --through-action to --actions-from-github
* refactor(command): rename command from --actions-from-github to --replace-ghe-action-with-github-com
* feat: use logger from context wherever possible
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add step/job id and results to json logs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: value to be masked should not be hard-coded in the action
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* fix: replace values following ::add-mask:: in evaluated strings
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: [DEBUG] identifier for debug logs to distinguish them
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: replace logger with step logger
The container gets injected a job logger, but during the time that steps
are run, we want to use the step logger.
This commit wraps pre/main/post steps in an executor that replaces the
job logger with a step logger.
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add pre/post stage identifier fields to json log output
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: add job/step result status to skipped steps/jobs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
There might be use cases where users want to use GitHub's variables in
the environment variables, which is a valid use case.
This commits adds support for replacement of GitHub's env with GitHub's
values.
Signed-off-by: André Martins <aanm90@gmail.com>
* feat: handle context cancelation during docker exec
To allow interrupting docker exec (which could be long running)
we process the log output in a go routine and handle
context cancelation as well as command result.
In case of context cancelation a CTRL+C is written into the docker
container. This should be enough to terminate the running
command.
To make sure we do not get stuck during cleanup, we do
set the cleanup contexts with a timeout of 5 minutes
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se>
* feat: handle SIGTERM signal and abort run
* test: on context cancel, abort running command
This test makes sure that whenever the act Context was canceled, the
currently running docker exec is sent a 0x03 (ctrl+c).
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* test: make sure the exec funcction handles command exit code
This test makes sure that the exec function does handle
docker command error results
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Philipp Hinrichsen <philipp.hinrichsen@new-work.se>
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* refactor: remove composite action runcontext workaround
The RunContext is cloned to execute a composite action with all its
steps in a similar context. This required some workaround, since
the command handler has kept a reference to the original RunContext.
This is solved now, by replacing the docker LogWriter with a proper
scoped LogWriter.
This prepares for a simpler setup of composite actions to be able
to create and re-create the composite RunContext for pre/main/post
action steps.
* test: check env-vars for local js and docker actions
* test: test remote docker and js actions
* fix: merge github context into env when read and setup
* refacotr: simplify composite context setup
* test: use a map matcher to test input setup
* fix: restore composite log output
Since we create a new line writer, we need to log the raw_output as well.
Otherwise no output will be available from the log-writer
* fix: add RunContext JobName to fill GITHUB_JOBNAME
* test: use nektos/act-test-actions
* fix: allow masking values in composite actions
To allow masking of values from composite actions, we need
to use a custom job logger with a reference to the masked
values for the composite run context.
* refactor: keep existing logger for composite actions
To not introduce another new logger while still be able to use
the masking from the composite action, we add the masks to
the go context. To leverage that context, we also add the context
to the log entries where the valueMasker then could get the actual
mask values.
With this way to 'inject' the masked values into the logger, we do
- keep the logger
- keep the coloring
- stay away from inconsistencies due to parallel jobs
* fix: re-add removed color increase
This one should have never removed :-)
* fix: add missing ExtraPath attribute
* fix: merge run context env into composite run context env
This adds a test and fix for the parent environment. It should be
inherited by the composite environment.
* test: add missing test case
* fix: store github token next to secrets
We must not expose the secrets to composite actions, but the
`github.token` is available inside composite actions.
To provide this we store the token in the config and create it in
the GithubContext from there.
The token can be used with `github.token` but is not available as
`secrets.GITHUB_TOKEN`.
This implements the same behavior as on GitHub.
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Marcus Noll <markus.noll@new-work.se>
* fixup! fix: allow masking values in composite actions
* style: use tabs instead of spaces to fix linter errors
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Marcus Noll <markus.noll@new-work.se>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
act has a feature that skips the checkout action to do a remote
checkout when a local checkout exists. in some cases, e.g. when
running act in a CI, you always want to clone the repository.
* feat: add json logger output
This will allow to format log output as json.
This is helpful in cases where act is not executed on a 'local' machine.
* refactor: use runner config
Using the runner config to configure logging is cleaner.
Co-authored-by: Casey Lee <cplee@nektos.com>
this commit adds support for the `::add-mask::` command, which was
implemented as a stub before.
it does not cover debug output that appears when you run act in
verbose mode
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* feat: run jobs in parallel
This changes fixes and restructures the parallel execution of jobs.
The previous changes limiting the parallel execution did break this
and allowed only one job in parallel.
While we run #CPU jobs in parallel now, the jobs added per job-matrix
add to this. So we might over-commit to the capacity, but at least
it is limited.
* fix: correctly build job pipeline
The job pipeline should just append all required pipeline steps.
The parallelism will be handled by the ParallelExecutor and we
shouldn't handle it during building the pipelines.
Also this adds a test, that the ParallelExecutor does run
a limited amount of parallel goroutines.
* test: correct test implementation
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This commit moves the githubContext, jobContext and stepResult structs
from the runner package to the model package in preparation for #908
because the expression.go file lives in the runner package and would
introduce cyclic dependencies with the exprparser package.
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Feature: uses in composite
* Negate logic
* Reduce complexity
* Update step_context.go
* Update step_context.go
* Update step_context.go
* Fix syntax error in test
* Bump
* Disable usage of actions/setup-node@v2
* Bump
* Fix step id collision
* Fix output command workaround
* Make secrets context inaccessible in composite
* Fix order after adding a workaround (needs tests)
Fixes https://github.com/nektos/act/pull/793#issuecomment-922329838
* Evaluate env before passing one step deeper
If env would contain any inputs, steps ctx or secrets there was undefined behaviour
* [no ci] prepare secret test
* Initial test pass inputs as env
* Fix syntax error
* extend test also for direct invoke
* Fix passing provided env as composite output
* Fix syntax error
* toUpper 'no such secret', act has a bug
* fix indent
* Fix env outputs in composite
* Test env outputs of composite
* Fix inputs not defined in docker actions
* Fix interpolate args input of docker actions
* Fix lint
* AllowCompositeIf now defaults to true
see https://github.com/actions/runner/releases/tag/v2.284.0
* Fix lint
* Fix env of docker action.yml
* Test calling a local docker action from composite
With input context hirachy
* local-action-dockerfile Test pass on action/runner
It seems action/runner ignores overrides of args,
if the target docker action has the args property set.
* Fix exec permissions of docker-local-noargs
* Revert getStepsContext change
* fix: handle composite action on error and continue
This change is a follow up of https://github.com/nektos/act/pull/840
and integrates with https://github.com/nektos/act/pull/793
There are two things included here:
- The default value for a step.if in an action need to be 'success()'
- We need to hand the error from a composite action back to the
calling executor
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
* Patch inputs can be bool, float64 and string
for workflow_call
Also inputs is now always defined, but may be null
* Simplify cherry-picked commit
* Minor style adjustments
* Remove chmod +x from tests
now fails on windows like before
* Fix GITHUB_ACTION_PATH some action env vars
Fixes GITHUB_ACTION_REPOSITORY, GITHUB_ACTION_REF.
* Add comment to CompositeRestrictions
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Co-authored-by: Ryan <me@hackerc.at>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* fix: continue jobs + steps after failure
To allow proper if expression handling on jobs and steps (like always,
success, failure, ...) we need to continue running all executors in
the prepared chain.
To keep the error handling intact we add an occurred error to the
go context and handle it later in the pipeline/chain.
Also we add the job result to the needs context to give expressions
access to it.
The needs object, failure and success functions are split between
run context (on jobs) and step context.
Closes#442
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* style: correct linter warnings
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* fix: job if value defaults to success()
As described in the documentation, a default value of "success()" is
applied when no "if" value is present on the job.
https://docs.github.com/en/actions/learn-github-actions/expressions#job-status-check-functions
Co-authored-by: Markus Wolf <mail@markus-wolf.de>
* fix: check job needs recursively
Ensure job result includes results of previous jobs
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: add runner test for job status check functions
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* test: add unit tests for run context if evaluation
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* refactor: move if expression evaluation
Move if expression evaluation into own function (step context) to
better support unit testing.
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* test: add unit tests for step context if evaluation
Co-authored-by: Markus Wolf <markus.wolf@new-work.se>
* fix: handle job error more resilient
The job error is not stored in a context map instead of a context
added value.
Since context values are immutable an added value requires to keep
the new context in all cases. This is fragile since it might slip
unnoticed to other parts of the code.
Storing the error of a job in the context map will make it more stable,
since the map is always there and the context of the pipeline is stable
for the whole run.
* feat: steps should use a default if expression of success()
* test: add integration test for if-expressions
* chore: disable editorconfig-checker for yaml multiline string
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
Adds option to rebuild local action docker images
Fixed up README due to missing flags after
PR #714 and #716
Signed-off-by: hackercat <me@hackerc.at>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Add asset server and upload handling of binary files
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* Add asset download parts to the asset server
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* Add artifact-server-path flag
If the flag is not defined, the artifact server isn't started.
This includes the configuration of ACTIONS_RUNTIME_URL and
ACTIONS_RUNTIME_TOKEN which are set if the server is started.
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* Move ACTIONS_RUNTIME_* vars into the withGithubEnv setup
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* feat: add artifact server port as flag
This commits adds a flag to define the artifact server port.
If not given, the port defaults to 34567.
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
* test: add artifact server tests
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
* refactor: use fs.FS
This allows to add tests with in-memory file system
* feat: add support for gzip encoded uploads
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
* test: add artifact integration test
* chore: run act tests with asset server path
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
* docs: add new cli flags
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
* test: add test workflow to testdata
* feat: add log output
* refactor: log shutdown error instead of panic
* feat: use outbound ip for the asset server
This change should allow to use the host ip in macos and windows.
Since docker is running in an intermediate vm, localhost is not
sufficient to have the artifacts in the host system.
* fix: do not use canceled context
To shutdown artifact server, we should not use the already canceled
context but the parent context instead.
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* feat: shutdown artifact server at end of pipeline
When the pipeline is done the asset server should be shut down
gracefully.
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* fix: close server if graceful shutdown failed
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
* fix: ignore server closed error from listen call
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: Björn Brauer <zaubernerd@zaubernerd.de>
Co-authored-by: Björn Brauer <bjoern.brauer@new-work.se>
This patch adds a new command-line flag (`--rm`) to automatically
stop workflow container(s), just prior to exit. The default
behavior is kept, e.g.: the containers continue running at exit.
Fixes: #694
Signed-off-by: Joseph Benden <joe@benden.us>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
This patch adds two new command-line flags to specify one or
more kernel capabilities to add or remove from the workflow
containers.
The command-line flag `--container-cap-add` allows for adding
specific capabilities on the workflow containers; where as,
The command-line flag `--container-cap-drop` allows for removing
specific capabilities on the workflow containers.
This was developed to specifically be able to add `SYS_PTRACE`
to a workflow I maintain. It involves using this capability to
monitor a make build, to then build a compilation database.
Signed-off-by: Joseph Benden <joe@benden.us>
* define GITHUB_ACTION_PATH #603
* Add more environment variables
* Add job name
Note: the job name gets a suffix for matrix builds, but this is not part of the env var
* fix: remove unnecessary variables
* feat: add `RepositoryOwner`
credit: @KnisterPeter
* feat: add test for `getGithubContext()`
Co-authored-by: Ryan (hackercat) <me@hackerc.at>
* Add option to specify custom GitHub instance
* Use correct GHE API endpoint URLs
Co-authored-by: Markus Wolf <knister.peter@shadowrun-clan.de>
* Extract slug from GitHub Enterprise URLs
Co-authored-by: Markus Wolf <knister.peter@shadowrun-clan.de>
* Use GITHUB_TOKEN for clone authenticate if provided
This change will allow use authentication for cloning actions
from private repositories or github enterprise instances.
Co-Authored-By: Markus Wolf <knister.peter@shadowrun-clan.de>
* Add section about using act on GitHub Enterprise to README
Co-authored-by: Markus Wolf <knister.peter@shadowrun-clan.de>
* Set GitHubInstance in runnerConfig in runner_test
Co-authored-by: Markus Wolf <knister.peter@shadowrun-clan.de>
Co-authored-by: hackercat <me@hackerc.at>
Co-authored-by: Markus Wolf <knister.peter@shadowrun-clan.de>
* feat: bump `golangci-lint`, add `super-linter`, replace outdated linter
Bump `golangci-lint` version.
Add `super-linter` to lint other languages.
Go linter is disabled because it's currently broken:
https://github.com/github/super-linter/pull/370
Replacing `scopelint` with `exportloopref`: "[runner] The linter
'scopelint' is deprecated (since v1.39.0) due to: The repository of the
linter has been deprecated by the owner. Replaced by exportloopref."
Fixed formatting in `.golangci.yml`
Add addtional linters:
`misspell`: purely style, detects typos in comments
`whitespace`: detects leading and trailing whitespace
`goimports`: it's gofmt + checks unused imports
* fix: lint/fix `go` files
* fix: lint with `standardjs`
* fix: lint/fix with `markdownlint`, make template more verbose
* feat: add lint stuff to makefile
* fix: `UseGitIgnore` formatting
* fix: lint/fix `README.md`
Co-authored-by: Casey Lee <cplee@nektos.com>
* disable gitignore for actions
* feat: Add option to allow/disallow paths specified in .gitignore
Co-authored-by: Alan Birtles <alan.birtles@eu.sony.com>
* Comment for public function
* Add git describe fallback
* spelling: github
* Set initial branch to satisfy tests for modern git
* Clarify -even- if
* Go 1.16
* Support running commands in repositories without action.yaml
Support runnings commands with only a Docker file
Co-authored-by: Casey Lee <cplee@nektos.com>
* Add QEMU to run different architectures
* Update dependencies in `go.mod`
* Add `--container-architecture` flag to specify custom image architecture
Co-authored-by: Casey Lee <cplee@nektos.com>
I got an error like this after hitting `act` command.
> Error: Error response from daemon: cannot share the host's network namespace when user namespaces are enabled
According to the document, when user namespaces are enabled on the Docker daemon,
neither host network mode and --privileged work without --userns=host. Since `act`
uses host network mode to match GitHub Actions runners, it cannot run jobs when
user namespaces are enabled. So I added the flag.
https://docs.docker.com/engine/security/userns-remap/#user-namespace-known-limitations
Co-authored-by: Casey Lee <cplee@nektos.com>
Previously secrets would be shown in log output as provided. This
commit updates the stepLogFormatter to replace any instance of the secret
string with "***", as GitHub Actions would
Known issues: If the secret is a generic string (such as "docker"), all
occurances of that string will be replaced in the output
Co-authored-by: Casey Lee <cplee@nektos.com>
This adds the `-a` flag when running `act` to change the username of the GITHUB_ACTOR environment variable
Co-authored-by: Casey Lee <cplee@nektos.com>
* feat: load every environment from --env-file to workflow
* fix: pass dotenv's environments through by context
* updates to support --secret-file
Co-authored-by: Casey Lee <cplee@nektos.com>
