1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
|
//go:build !workflows
/*
The workflows program manages the .gitea directory to avoid having to write
yaml. It is never invoked directly or built for the distribution tarball.
This program is an internal detail of the hakurei project and is not usable
on its own. It is not covered by the compatibility promise.
# Configuring act_runner
It is unreasonable and impractical to bootstrap Rosa OS each time a job runs.
This section documents an ideal configuration for exposing the cmd/mbf CI
service to all job containers, while avoiding some issues with the container
setup documented upstream.
Microsoft Github workflows expect a docker socket in the job container. This is
problematic because docker has a hard time running inside a docker container.
The Gitea act_runner simply bind mounts whatever socket it sees into the
container. With a regular docker daemon, this allows not only a simple container
escape, but also privilege escalation as unconstrained root in the init
namespace. To mitigate this, set up an unprivileged podman daemon and expose its
socket to the container instead. Since mountinfo always use credentials from the
init user namespace, subordinate user and group ID must always be 100000.
On Alpine Linux, this is achieved by:
# Configuration for /etc/init.d/podman
# See podman-system-service(1) for service description
# and available options.
#podman_opts="--time 0"
# API endpoint in URI form. Leave empty to use defaults.
podman_uri="unix:///srv/act_runner/podman"
# Setting root user will start rootful service.
# Use any other user for rootless mode.
podman_user="act_runner"
A symlink must then be created at /var/run/docker.sock pointing to the socket
owned by the unprivileged daemon. This should be done in a /etc/local.d script.
The container can be specified via docker-compose:
version: "3.8"
services:
runner:
restart: always
image: docker.io/gitea/act_runner:nightly
network_mode: host
environment:
CONFIG_FILE: /config.yaml
GITEA_INSTANCE_URL: "https://git.gensokyo.uk"
GITEA_RUNNER_LABELS: "rosa:docker://docker.gitea.com/runner-images:ubuntu-latest"
GITEA_RUNNER_REGISTRATION_TOKEN: "${REGISTRATION_TOKEN}"
GITEA_RUNNER_NAME: "${RUNNER_NAME}"
volumes:
- ./config.yaml:/config.yaml
- ./data:/data
- ./podman:/var/run/docker.sock
Before starting the container, configure act_runner via config.yaml:
container:
options: -e PAGER=cat -e MBF_CACHE_DIR=/rosa -e MBF_POISON_OPEN=1
-v /var/lib/rosa:/rosa
--security-opt='unmask=/proc/*'
--cap-add=SYS_ADMIN
--cap-add=SYS_PTRACE
--device=/dev/kvm
--device=/dev/fuse
valid_volumes:
- /var/lib/rosa
where /var/lib/rosa is the absolute pathname of the cache directory in the init
namespace. Setting MBF_POISON_OPEN enables cmd/mbf to run as root. It is also
a good idea here to set runner.capacity to reflect the capacity of the guest, so
jobs can be consumed quicker. Removing mount points covering /proc enables
testing of cmd/hakurei. Exposing the fuse device and adding capability
CAP_SYS_ADMIN enables testing of cmd/sharefs. Adding capability CAP_SYS_PTRACE
enables dumping seccomp filters via ptrace on the patched kernel.
Build a statically-linked cmd/mbf:
go build -trimpath \
-ldflags='-s -w
-buildid=
-linkmode external
-extldflags=-static' \
./cmd/mbf
Create a directory named "bin" in the cache directory and install the resulting
binary.
The cmd/mbf CI service should generally run as a separate user. To achieve this,
change the socket permission to 0777 after the service starts. On Alpine Linux,
this can be achieved by the init script:
#!/sbin/openrc-run
supervisor=supervise-daemon
name="Rosa OS cmd/mbf CI backend"
description="Listening service that services CI workloads from act_runner"
command=/var/lib/rosa/bin/mbf
command_args="-d /var/lib/rosa ci daemon"
command_user="${mbf_user:=mbf}"
output_logger="logger"
error_logger="logger"
depend() {
need sysfs
}
start_post() {
while [ ! -e /var/lib/rosa/ci ]; do sleep 0.1; done
chmod 0777 /var/lib/rosa/ci
}
It is often a good idea to populate the cache from a mirror service before the
first workflow job is started and re-populate it after every cmd/mbf update.
# Configuring the kernel
In order to attach to the container process, the sysctl kernel.yama.ptrace_scope
must be set to 0. After which, apply the patch test/sandbox/seccomp.patch to
your kernel sources, compile and install the new kernel. Refer to
https://wiki.alpinelinux.org/wiki/Custom_Kernel if the guest runs Alpine Linux.
If running podman or docker as root, the patch is not required. Do not apply
this patch on a system meant to be secure.
# Security
The design of Microsoft Github workflows is inherently insecure: it requires
internet access just to fetch actions, and these actions all expect internet
access as well. Gitea, being a drop-in replacement, retains the same problems.
The setup documented above already avoids the obvious container escape via the
bind-mounted docker socket, but it is still easily possible to escape as the
context of the unconstrained runner user. For this reason, the runner must
reside in a hardened guest. Since internet access cannot be disabled, extra care
must be taken when setting up the host firewall to prevent access to internal
systems from the untrusted guest.
*/
package workflows
//go:generate go run -tags=workflows .
|
