Update images, fix extrapath (#723)

* fix: typo

* fix: remove debug for git references

it generates a massive amount of logs

* feat: read values from env

allows to test act on GHA when it's not a main repo

* fix: merge extrapath with PATH

* fix(tests): add additional shells for testing

* fix(image): update images

pin node to major version only, current node version: 12.22.1
replace most images with `node:12-buster-slim` to prevent
errors on macOS runner due to DockerHub pull limit
replace ocaml image

Co-authored-by: Casey Lee <cplee@nektos.com>

IMAGES.md

@@ -8,12 +8,12 @@
 
 **Note 2: `node` `-slim` images don't have `python` installed, if you want to use actions or software that is depending on `python`, you need to specify image manually**
 
-| Image                                     | Size                                                          |
-| ----------------------------------------- | ------------------------------------------------------------- |
-| [`node:12.20.1-buster`][hub/_/node]       | ![`buster-size`][hub/_/node/12.20.1-buster/size]              |
-| [`node:12.20.1-buster-slim`][hub/_/node]  | ![`micro-buster-size`][hub/_/node/12.20.1-buster-slim/size]   |
-| [`node:12.20.1-stretch`][hub/_/node]      | ![`stretch-size`][hub/_/node/12.20.1-stretch/size]            |
-| [`node:12.20.1-stretch-slim`][hub/_/node] | ![`micro-stretch-size`][hub/_/node/12.20.1-stretch-slim/size] |
+| Image                                | Size                                                     |
+| ------------------------------------ | -------------------------------------------------------- |
+| [`node:12-buster`][hub/_/node]       | ![`buster-size`][hub/_/node/12-buster/size]              |
+| [`node:12-buster-slim`][hub/_/node]  | ![`micro-buster-size`][hub/_/node/12-buster-slim/size]   |
+| [`node:12-stretch`][hub/_/node]      | ![`stretch-size`][hub/_/node/12-stretch/size]            |
+| [`node:12-stretch-slim`][hub/_/node] | ![`micro-stretch-size`][hub/_/node/12-stretch-slim/size] |
 
 **Note: `catthehacker/ubuntu` images are based on Ubuntu root filesystem**
 
@@ -48,10 +48,10 @@ Feel free to make a pull request with your image added here
 
 [hub/_/buildpack-deps]: https://hub.docker.com/_/buildpack-deps
 [hub/_/node]: https://hub.docker.com/r/_/node
-[hub/_/node/12.20.1-buster/size]: https://img.shields.io/docker/image-size/_/node/12.20.1-buster
-[hub/_/node/12.20.1-buster-slim/size]: https://img.shields.io/docker/image-size/_/node/12.20.1-buster-slim
-[hub/_/node/12.20.1-stretch/size]: https://img.shields.io/docker/image-size/_/node/12.20.1-stretch
-[hub/_/node/12.20.1-stretch-slim/size]: https://img.shields.io/docker/image-size/_/node/12.20.1-stretch-slim
+[hub/_/node/12-buster/size]: https://img.shields.io/docker/image-size/_/node/12-buster
+[hub/_/node/12-buster-slim/size]: https://img.shields.io/docker/image-size/_/node/12-buster-slim
+[hub/_/node/12-stretch/size]: https://img.shields.io/docker/image-size/_/node/12-stretch
+[hub/_/node/12-stretch-slim/size]: https://img.shields.io/docker/image-size/_/node/12-stretch-slim
 [hub/catthehacker/ubuntu]: https://hub.docker.com/r/catthehacker/ubuntu
 [hub/catthehacker/ubuntu/act-latest/size]: https://img.shields.io/docker/image-size/catthehacker/ubuntu/act-latest
 [hub/catthehacker/ubuntu/act-20.04/size]: https://img.shields.io/docker/image-size/catthehacker/ubuntu/act-20.04

README.md

@@ -139,7 +139,7 @@ It will save that information to `~/.actrc`, please refer to [Configuration](#co
       --detect-event                     Use first event type from workflow as event that triggered the workflow
   -C, --directory string                 working directory (default ".")
   -n, --dryrun                           dryrun mode
-      --env stringArray                  env to make available to actions with optional value (e.g. --e myenv=foo or -s myenv)
+      --env stringArray                  env to make available to actions with optional value (e.g. --env myenv=foo or --env myenv)
       --env-file string                  environment file to read and use as env in the containers (default ".env")
   -e, --eventpath string                 path to event JSON file
       --github-instance string           GitHub instance to use. Don't use this if you are not using GitHub Enterprise Server. (default "github.com")
@@ -192,12 +192,12 @@ If the `path:` value doesn't match the name of the repository, a `MODULE_NOT_FOU
 
 GitHub Actions offers managed [virtual environments](https://help.github.com/en/actions/reference/virtual-environments-for-github-hosted-runners) for running workflows. In order for `act` to run your workflows locally, it must run a container for the runner defined in your workflow file. Here are the images that `act` uses for each runner type and size:
 
-| GitHub Runner   | Micro Docker Image                   | Medium Docker Image                        | Large Docker Image                                  |
-| --------------- | ------------------------------------ | ------------------------------------------ | --------------------------------------------------- |
-| `ubuntu-latest` | [`node:12.20.1-buster-slim`][micro]  | [`catthehacker/ubuntu:act-latest`][medium] | [`catthehacker/ubuntu:full-20.04`][large-cat]       |
-| `ubuntu-20.04`  | [`node:12.20.1-buster-slim`][micro]  | [`catthehacker/ubuntu:act-20.04`][medium]  | [`catthehacker/ubuntu:full-20.04`][large-cat]       |
-| `ubuntu-18.04`  | [`node:12.20.1-buster-slim`][micro]  | [`catthehacker/ubuntu:act-18.04`][medium]  | [`nektos/act-environments-ubuntu:18.04`][large-act] |
-| `ubuntu-16.04`  | [`node:12.20.1-stretch-slim`][micro] | [`catthehacker/ubuntu:act-16.04`][medium]  | `unavailable`                                       |
+| GitHub Runner   | Micro Docker Image              | Medium Docker Image                        | Large Docker Image                                  |
+| --------------- | ------------------------------- | ------------------------------------------ | --------------------------------------------------- |
+| `ubuntu-latest` | [`node:12-buster-slim`][micro]  | [`catthehacker/ubuntu:act-latest`][medium] | [`catthehacker/ubuntu:full-20.04`][large-cat]       |
+| `ubuntu-20.04`  | [`node:12-buster-slim`][micro]  | [`catthehacker/ubuntu:act-20.04`][medium]  | [`catthehacker/ubuntu:full-20.04`][large-cat]       |
+| `ubuntu-18.04`  | [`node:12-buster-slim`][micro]  | [`catthehacker/ubuntu:act-18.04`][medium]  | [`nektos/act-environments-ubuntu:18.04`][large-act] |
+| `ubuntu-16.04`  | [`node:12-stretch-slim`][micro] | [`catthehacker/ubuntu:act-16.04`][medium]  | `unavailable`                                       |
 
 [micro]: https://hub.docker.com/_/buildpack-deps
 [medium]: https://github.com/catthehacker/docker_images
@@ -249,7 +249,7 @@ If you use multiple platforms in your workflow, you have to specify them to chan
 For example, if your workflow uses `ubuntu-18.04`, `ubuntu-16.04` and `ubuntu-latest`, specify all platforms like below
 
 ```sh
-act -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04 -P ubuntu-latest=ubuntu:latest -P ubuntu-16.04=node:12.20.1-buster-slim
+act -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04 -P ubuntu-latest=ubuntu:latest -P ubuntu-16.04=node:12-buster-slim
 ```
 
 # Secrets

cmd/platforms.go

@@ -6,10 +6,10 @@ import (
 
 func (i *Input) newPlatforms() map[string]string {
 	platforms := map[string]string{
-		"ubuntu-latest":  "node:12.20.1-buster-slim",
-		"ubuntu-20.04":   "node:12.20.1-buster-slim",
-		"ubuntu-18.04":   "node:12.20.1-buster-slim",
-		"ubuntu-16.04":   "node:12.20.1-stretch-slim",
+		"ubuntu-latest":  "node:12-buster-slim",
+		"ubuntu-20.04":   "node:12-buster-slim",
+		"ubuntu-18.04":   "node:12-buster-slim",
+		"ubuntu-16.04":   "node:12-stretch-slim",
 		"windows-latest": "",
 		"windows-2019":   "",
 		"macos-latest":   "",

cmd/root.go

@@ -39,7 +39,7 @@ func Execute(ctx context.Context, version string) {
 	rootCmd.Flags().BoolP("graph", "g", false, "draw workflows")
 	rootCmd.Flags().StringP("job", "j", "", "run job")
 	rootCmd.Flags().StringArrayVarP(&input.secrets, "secret", "s", []string{}, "secret to make available to actions with optional value (e.g. -s mysecret=foo or -s mysecret)")
-	rootCmd.Flags().StringArrayVarP(&input.envs, "env", "", []string{}, "env to make available to actions with optional value (e.g. --e myenv=foo or -s myenv)")
+	rootCmd.Flags().StringArrayVarP(&input.envs, "env", "", []string{}, "env to make available to actions with optional value (e.g. --env myenv=foo or --env myenv)")
 	rootCmd.Flags().StringArrayVarP(&input.platforms, "platform", "P", []string{}, "custom image to use per platform (e.g. -P ubuntu-18.04=nektos/act-environments-ubuntu:18.04)")
 	rootCmd.Flags().BoolVarP(&input.reuseContainers, "reuse", "r", false, "reuse action containers to maintain state")
 	rootCmd.Flags().BoolVarP(&input.bindWorkdir, "bind", "b", false, "bind working directory to container, rather than copy")
@@ -303,7 +303,7 @@ func defaultImageSurvey(actrc string) error {
 	case "Medium":
 		option = "-P ubuntu-latest=catthehacker/ubuntu:act-latest\n-P ubuntu-20.04=catthehacker/ubuntu:act-20.04\n-P ubuntu-18.04=catthehacker/ubuntu:act-18.04\nubuntu-16.04=catthehacker/ubuntu:act-16.04"
 	case "Micro":
-		option = "-P ubuntu-latest=node:12.20.1-buster-slim\n-P ubuntu-20.04=node:12.20.1-buster-slim\n-P ubuntu-18.04=node:12.20.1-buster-slim\n-P ubuntu-16.04=node:12.20.1-stretch-slim"
+		option = "-P ubuntu-latest=node:12-buster-slim\n-P ubuntu-20.04=node:12-buster-slim\n-P ubuntu-18.04=node:12-buster-slim\n-P ubuntu-16.04=node:12-stretch-slim"
 	}
 
 	f, err := os.Create(actrc)

pkg/common/git.go

@@ -86,7 +86,7 @@ func FindGitRef(file string) (string, error) {
 				if r == nil || err != nil {
 					break
 				}
-				log.Debugf("Reference: name=%s sha=%s", r.Name().String(), r.Hash().String())
+				// log.Debugf("Reference: name=%s sha=%s", r.Name().String(), r.Hash().String())
 				if r.Hash().String() == ref {
 					if r.Name().IsTag() {
 						refTag = r.Name().String()

pkg/container/docker_images_test.go

@@ -40,7 +40,7 @@ func TestImageExistsLocally(t *testing.T) {
 
 	// Chose alpine latest because it's so small
 	// maybe we should build an image instead so that tests aren't reliable on dockerhub
-	readerDefault, err := cli.ImagePull(ctx, "alpine:latest", types.ImagePullOptions{
+	readerDefault, err := cli.ImagePull(ctx, "node:12-buster-slim", types.ImagePullOptions{
 		Platform: "linux/amd64",
 	})
 	assert.Nil(t, err)
@@ -48,12 +48,12 @@ func TestImageExistsLocally(t *testing.T) {
 	_, err = ioutil.ReadAll(readerDefault)
 	assert.Nil(t, err)
 
-	imageDefaultArchExists, err := ImageExistsLocally(ctx, "alpine:latest", "linux/amd64")
+	imageDefaultArchExists, err := ImageExistsLocally(ctx, "node:12-buster-slim", "linux/amd64")
 	assert.Nil(t, err)
 	assert.Equal(t, true, imageDefaultArchExists)
 
 	// Validate if another architecture platform can be pulled
-	readerArm64, err := cli.ImagePull(ctx, "alpine:latest", types.ImagePullOptions{
+	readerArm64, err := cli.ImagePull(ctx, "node:12-buster-slim", types.ImagePullOptions{
 		Platform: "linux/arm64",
 	})
 	assert.Nil(t, err)
@@ -61,7 +61,7 @@ func TestImageExistsLocally(t *testing.T) {
 	_, err = ioutil.ReadAll(readerArm64)
 	assert.Nil(t, err)
 
-	imageArm64Exists, err := ImageExistsLocally(ctx, "alpine:latest", "linux/arm64")
+	imageArm64Exists, err := ImageExistsLocally(ctx, "node:12-buster-slim", "linux/arm64")
 	assert.Nil(t, err)
 	assert.Equal(t, true, imageArm64Exists)
 }

pkg/runner/run_context_test.go

@@ -308,13 +308,28 @@ func TestGetGitHubContext(t *testing.T) {
 
 	log.Debugf("%v", ghc)
 
+	actor := "nektos/act"
+	if a := os.Getenv("ACT_ACTOR"); a != "" {
+		actor = a
+	}
+
+	repo := "nektos/act"
+	if r := os.Getenv("ACT_REPOSITORY"); r != "" {
+		repo = r
+	}
+
+	owner := "nektos"
+	if o := os.Getenv("ACT_OWNER"); o != "" {
+		owner = o
+	}
+
 	assert.Equal(t, ghc.RunID, "1")
 	assert.Equal(t, ghc.Workspace, rc.Config.containerPath(cwd))
 	assert.Equal(t, ghc.RunNumber, "1")
 	assert.Equal(t, ghc.RetentionDays, "0")
-	assert.Equal(t, ghc.Actor, "nektos/act")
-	assert.Equal(t, ghc.Repository, "nektos/act")
-	assert.Equal(t, ghc.RepositoryOwner, "nektos")
+	assert.Equal(t, ghc.Actor, actor)
+	assert.Equal(t, ghc.Repository, repo)
+	assert.Equal(t, ghc.RepositoryOwner, owner)
 	assert.Equal(t, ghc.RunnerPerflog, "/dev/null")
 	assert.Equal(t, ghc.EventPath, ActPath+"/workflow/event.json")
 	assert.Equal(t, ghc.Token, rc.Config.Secrets["GITHUB_TOKEN"])

pkg/runner/runner_test.go

@@ -16,6 +16,14 @@ import (
 	"github.com/nektos/act/pkg/model"
 )
 
+var baseImage string = "node:12-buster-slim"
+
+func init() {
+	if p := os.Getenv("ACT_TEST_IMAGE"); p != "" {
+		baseImage = p
+	}
+}
+
 func TestGraphEvent(t *testing.T) {
 	planner, err := model.NewWorkflowPlanner("testdata/basic", true)
 	assert.Nil(t, err)
@@ -81,7 +89,7 @@ func TestRunEvent(t *testing.T) {
 	}
 
 	platforms := map[string]string{
-		"ubuntu-latest": "node:12.20.1-buster-slim",
+		"ubuntu-latest": baseImage,
 	}
 
 	tables := []TestJobFileInfo{
@@ -89,8 +97,10 @@ func TestRunEvent(t *testing.T) {
 		{"testdata", "fail", "push", "exit with `FAILURE`: 1", platforms, ""},
 		{"testdata", "runs-on", "push", "", platforms, ""},
 		{"testdata", "checkout", "push", "", platforms, ""},
-		// Pwsh is not available in default worker (yet) so we use a separate image for testing
-		{"testdata", "powershell", "push", "", map[string]string{"ubuntu-latest": "ghcr.io/justingrote/act-pwsh:latest"}, ""},
+		{"testdata", "shells/pwsh", "push", "", map[string]string{"ubuntu-latest": "ghcr.io/justingrote/act-pwsh:latest"}, ""}, // custom image with pwsh
+		{"testdata", "shells/bash", "push", "", platforms, ""},
+		{"testdata", "shells/python", "push", "", map[string]string{"ubuntu-latest": "node:12-buster"}, ""}, // slim doesn't have python
+		{"testdata", "shells/sh", "push", "", platforms, ""},
 		{"testdata", "job-container", "push", "", platforms, ""},
 		{"testdata", "job-container-non-root", "push", "", platforms, ""},
 		{"testdata", "uses-docker-url", "push", "", platforms, ""},
@@ -131,7 +141,7 @@ func TestRunEventSecrets(t *testing.T) {
 	ctx := context.Background()
 
 	platforms := map[string]string{
-		"ubuntu-latest": "node:12.20.1-buster-slim",
+		"ubuntu-latest": baseImage,
 	}
 
 	workflowPath := "secrets"
@@ -172,7 +182,7 @@ func TestRunEventPullRequest(t *testing.T) {
 	ctx := context.Background()
 
 	platforms := map[string]string{
-		"ubuntu-latest": "node:12.20.1-buster-slim",
+		"ubuntu-latest": baseImage,
 	}
 
 	workflowPath := "pull-request"

pkg/runner/step_context.go

@@ -125,9 +125,13 @@ func (sc *StepContext) mergeEnv() map[string]string {
 		env = mergeMaps(rc.GetEnv(), step.GetEnv())
 	}
 
-	env["PATH"] = `/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin`
-	if (rc.ExtraPath != nil) && (len(rc.ExtraPath) > 0) {
+	if env["PATH"] == "" {
+		env["PATH"] = `/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin`
+	}
+	if rc.ExtraPath != nil && len(rc.ExtraPath) > 0 {
+		p := env["PATH"]
 		env["PATH"] = strings.Join(rc.ExtraPath, `:`)
+		env["PATH"] += `:` + p
 	}
 
 	sc.Env = rc.withGithubEnv(env)

pkg/runner/step_context_test.go

@@ -9,7 +9,7 @@ import (
 
 func TestStepContextExecutor(t *testing.T) {
 	platforms := map[string]string{
-		"ubuntu-latest": "node:12.20.1-buster-slim",
+		"ubuntu-latest": baseImage,
 	}
 	tables := []TestJobFileInfo{
 		{"testdata", "uses-and-run-in-one-step", "push", "Invalid run/uses syntax for job:test step:Test", platforms, ""},

pkg/runner/testdata/actions/docker-local/Dockerfile

@@ -1,5 +1,5 @@
 # Container image that runs your code
-FROM alpine:3.13
+FROM node:12-buster-slim
 
 # Copies your code file from your action repository to the filesystem path `/` of the container
 COPY entrypoint.sh /entrypoint.sh

pkg/runner/testdata/actions/docker-url/action.yml

@@ -8,9 +8,8 @@ inputs:
     default: World
 runs:
   using: docker
-  #image: docker://alpine:3.13
-  image: docker://node:12-alpine
+  image: docker://node:12-buster-slim
   env:
     TEST: enabled
-  args: 
-  - env
+  args:
+    - env

pkg/runner/testdata/basic/push.yml

@@ -10,12 +10,12 @@ jobs:
     steps:
       - run: echo ${{ env.TEST }} | grep value
       - run: env
-      - uses: docker://alpine:3.13
+      - uses: docker://node:12-buster-slim
         with:
           somekey: ${{ env.TEST }}
           args: echo ${INPUT_SOMEKEY} | grep somevalue
       - run: ls
-      - run: echo 'hello world' 
+      - run: echo 'hello world'
       - run: echo ${GITHUB_SHA} >> $(dirname "${GITHUB_WORKSPACE}")/sha.txt
       - run: cat $(dirname "${GITHUB_WORKSPACE}")/sha.txt | grep ${GITHUB_SHA}
   build:
@@ -29,10 +29,10 @@ jobs:
     runs-on: ubuntu-latest
     needs: [build]
     steps:
-      - uses: docker://ubuntu:18.04
+      - uses: docker://node:12-buster-slim
         with:
           args: env
-      - uses: docker://ubuntu:18.04
+      - uses: docker://node:12-buster-slim
         with:
           entrypoint: /bin/echo
           args: ${{github.event_name}}

pkg/runner/testdata/commands/push.yml

@@ -14,8 +14,8 @@ jobs:
         id: set_output
         run: echo "::set-output name=zoo::zar"
 
-      #- run: echo "::add-path::/zip"
-      #- run: echo $PATH | grep /zip
+      - run: echo "::add-path::/zip"
+      - run: echo $PATH | grep /zip
 
       - name: TEST conditional
         if: steps.set_output.outputs.zoo

pkg/runner/testdata/fail/push.yml

@@ -4,8 +4,8 @@ on: push
 jobs:
   build:
     runs-on: ubuntu-latest
-    container: 
-      image: node:12.20.1-buster
+    container:
+      image: node:12-buster-slim
       env:
         TEST_ENV: test-value
     steps:

pkg/runner/testdata/job-container-non-root/push.yml

@@ -4,7 +4,7 @@ on: push
 jobs:
   test:
     runs-on: ubuntu-latest
-    container: 
-      image: ocaml/opam2:debian-9-opam
+    container:
+      image: ghcr.io/catthehacker/ubuntu:runner-latest # image with user 'runner:runner' built on tag 'act-latest'
     steps:
       - run: echo PASS

pkg/runner/testdata/job-container/push.yml

@@ -4,8 +4,8 @@ on: push
 jobs:
   test:
     runs-on: ubuntu-latest
-    container: 
-      image: node:12.20.1-buster
+    container:
+      image: node:12-buster-slim
       env:
         TEST_ENV: test-value
     steps:
@@ -13,7 +13,7 @@ jobs:
 
   test2:
     runs-on: ubuntu-latest
-    container: node:12.20.1-buster
+    container: node:12-buster-slim
     steps:
       - run: echo ${TEST_ENV} | grep test-value
         env:

pkg/runner/testdata/parallel/push.yml

@@ -15,7 +15,7 @@ jobs:
     runs-on: ubuntu-latest
     needs: [build]
     steps:
-      - uses: docker://ubuntu:18.04
+      - uses: docker://node:12-buster-slim
         with:
           args: echo ${GITHUB_REF} | grep nektos/act
       - uses: ./actions/docker-url

pkg/runner/testdata/shells/bash/push.yml

@@ -0,0 +1,10 @@
+on: push
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - shell: bash
+        run: |
+          if [[ -n "$PATH" ]] ; then
+            echo "I'm Bash!"
+          fi

pkg/runner/testdata/shells/pwsh/push.yml

@@ -1,12 +1,8 @@
-name: powershell
 on: push
-
-env:
-  TEST: value
 jobs:
   check:
     runs-on: ubuntu-latest
     steps:
       - shell: pwsh
         run: |
-          echo "hello test"
+          $PSVersionTable

pkg/runner/testdata/shells/python/push.yml

@@ -0,0 +1,9 @@
+on: push
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - shell: python
+        run: |
+          import platform
+          print(platform.python_version())

pkg/runner/testdata/shells/sh/push.yml

@@ -0,0 +1,10 @@
+on: push
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - shell: sh
+        run: |
+          if [ -n "$PATH" ] ; then
+            echo "I'm sh!"
+          fi

pkg/runner/testdata/uses-docker-url/push.yml

@@ -5,7 +5,7 @@ jobs:
   test:
     runs-on: ubuntu-latest
     steps:
-    - uses: docker://alpine:3.13
-      with:
-        somekey: somevalue
-        args: echo ${INPUT_SOMEKEY} | grep somevalue
+      - uses: docker://node:12-buster-slim
+        with:
+          somekey: somevalue
+          args: echo ${INPUT_SOMEKEY} | grep somevalue