path: root/internal/rosa/doc.go

/*
Package rosa implements the Rosa OS build system via package [hakurei.app/pkg].

Software packages required by the Rosa OS base system are included in the
package directory. By default, these built-in packages are embedded at compile
time. The penv build tag may be used to specify the pathname of the source code
of this package at runtime via the environment variable ROSA_SOURCE. Note that
changes to Go code still requires recompilation.

Software packages not part of the Rosa OS base system must not be added to
the built-in package directory. Additional Azalea source code may be registered
using the [S.RegisterAzalea] function in a new instance of [S]. The
[Rosa OS packages repository] is a good place to put additional packages.

# Native artifacts

Package rosa implements custom [hakurei.app/pkg] artifacts. This package must
not be used alongside other packages also implementing custom artifacts. As a
protection against simple cases of this, importing this package sets the
extension variant identification string to [Extension].

# Toolchain stages

Rosa OS has three kinds of toolchain stages, with two sets of each and an
additional [Stage3] final stage. This adds up to a total of seven stages,
however not all of them are normally used.

Every Rosa OS package is rooted in a binary seed containing at least a
shell and a LLVM toolchain. The early stage, a stage where the unexported
isStage0 method returns true, provides an environment from this binary seed
using only native artifacts. This stage does not support many fields of the
[PackageAttr] structure or [Helper] primitives. Special care must be taken for
packages compiled against this toolchain.

The intermediate stage (where the unexported isIntermediate method returns true)
and the std stage (where the unexported isStd method returns true) are
functionally identical. They support all [PackageAttr] fields and [Helper]
primitives. The purpose of the intermediate stage is to complete a 3-stage
bootstrap for the mksh shell and LLVM toolchain.

The final [Stage3] toolchain should generally only be used to validate 3-stage
determinism of the LLVM toolchain itself, by building a fourth LLVM stage. This
works around deficiencies in the CMake-based build system of the LLVM project
when compiling a dynamically-linked LLVM toolchain against a dynamically-linked
host LLVM toolchain.

The set of Gentoo-derived toolchains are rooted in a Gentoo Linux stage3
tarball. These are for completing the 3-stage bootstrap of the initial Rosa OS
stage0 toolchain distribution only and is never used again. This bootstrap
path might become broken from time to time. Historically, this is also required
when bootstrapping a completely new architecture, however that is now done via
cross-compilation, so the Gentoo bootstrap path is no longer invoked. The
unexported isRosa method returns whether a toolchain is not rooted in Gentoo.

# Declaring packages

Rosa OS packages are described using the Azalea language. Packages must be
declared at the top level:

	package hakurei-source {
		// short user-facing description, included in package metadata
		description = "hakurei source tree";
		// whether to exclude from listings in exported functions
		exclude = true;

		// package version, # adds it to the current scope; if source is set and
		// refers to a package declared at the top level, this defaults to the
		// version of that package
		version# = "0.4.10";

		// a non-nil output artifact bypasses the build system
		output = remoteGit {
			url = "https://src.rosa.moe/hakurei";
			// using the version variable previously added to the scope
			tag = "v"+version;
			checksum = "SZdto_lQCgwebCqUHJREQcHwwO1pHn3euOwFvXYEri0G-46zUhx0hX4buFt2XvkS";
		};
	}

	package hakurei-dist {
		// short user-facing description, included in package metadata
		description = "low-level userspace tooling for Rosa OS "+
			"(distribution tarball)";

		// project home page, included in package metadata
		website = "https://hakurei.app";

		// project identifier on release-monitoring.org, to check for updates
		anitya = 388834;

		// if non-empty, provides a maintainer-facing reason of blocked update
		// and prevent checking for updates via anitya
		block = "";

		// if non-nil, overrides the version filtering function
		latest = anityaFallback;

		// artifact containing the source tree, placed in /usr/src/ under a
		// directory named after the package, see section "Source fetchers";
		// this can also refer to a package declared at the top level like done
		// here
		source = hakurei-source;

		// pathnames of patches to apply to source, relative to the Azalea file;
		// must be within the base directory
		patches = [];

		// unconditionally enter source directory early
		enterSource = true;

		// allow writes to the source tree, with all writes going to an
		// anonymous instance of tmpfs
		writable = true;

		// make all files writable in the source tree
		chmod = true;

		// additional files to make available, relative to the Azalea file,
		// within the base directory
		files = {
			// specifying a relative pathname places the file in /usr/src/ under
			// the specified name
			".config": arch+".config";

			// specifying an absolute pathname places the file at the specified
			// pathname as is
			"/sbin/installkernel": "installkernel";
			// if the contents of the file begins with certain sequences of
			// bytes, it is automatically made executable
			"/sbin/depmod": "depmod";

			// prefix is the Rosa OS installation prefix, with the leading path
			// separator
			prefix+"/oldconfig": "oldconfig";
		};

		// whether the resulting artifact dodges all other artifacts with this
		// field set when being scheduled to cure
		exclusive = false;

		// whether the build process does not require the LLVM toolchain
		minimal = false;

		// whether this package is allowed in bootstrap stages
		bootstrap = true;

		// whether this package requires libc; defaults to true
		libc = true;

		// additional environment variables
		env = [
			"CGO_ENABLED=1",
			"GOCACHE=/tmp/gocache",
			"CC=clang -O3 -Werror",
		];

		// runs before script emitted by exec, implies enterSource if non-empty
		early = "";

		// programs to make available in /bin/
		bin = [ "hostname" ];

		// whether to replace /usr/bin/ with a symlink to /bin/
		populateUsrBin = false;

		// build system helper, see section "Build system helpers"
		exec = generic {
			chdir = false;
			cross = `
	export HAKUREI_ARCH='`+arch+`'
	export HAKUREI_SYSROOT='`+prefix+`/`+arch+`'

	export PKG_CONFIG_SYSROOT_DIR='`+prefix+`/`+arch+`'
	export PKG_CONFIG_LIBDIR='`+prefix+`/`+arch+prefix+`/lib/pkgconfig:`+
		prefix+`/`+arch+``+prefix+`/share/pkgconfig'
	`;
			build = `
	NAME=make`;
			check = `
	NAME=all`;
			install = `
	DESTDIR=/work "./${NAME}.sh"
	`;
		};

		// if non-nil, measures the outcome of the package and fails if it does
		// not match the specified checksum value
		checksum = nil;

		// native utilities used during the build
		tools = [
			go,
			pkgconf,
			net-tools,
			acl,
			wayland,
			wayland-protocols,
		];

		// packages made available for the target architecture, in the cross
		// sysroot if cross-compiling
		inputs = [
			libseccomp,
			acl,
			fuse,
			xcb,
			wayland,
			wayland-protocols,
			util-linux,
			kernel-headers,
		];

		// additional packages to place under directories named after each
		// package in /usr/src/
		extra = [];

		// if non-empty, packages specified in extra are overlay-mounted under
		// this pathname instead
		overlay = "";

		// inputs made available only during cross-compilation; setting this to
		// nil indicates that this package does not require cross-compilation
		// because its outcome is identical regardless of the target
		cross = [];

		// native utilities made available only during cross-compilation
		native = [];

		// whether explicitly specified runtime dependencies are combined with
		// resolved runtimes instead of overriding it
		join = false;

		// runtime dependencies, resolved from inputs if unset
		runtime = nil;

		// whether this package is unlikely to become a runtime dependency
		unlikely = true;
	}

# Built-in constants

	prefix         Rosa OS installation prefix
	prefixR        Rosa OS installation prefix relative to sysroot
	unversioned    special version string denoting an unversioned package

	hostTriple     Rosa OS host triple of the current system
	hostArch       architecture target name of the current system
	hostLinuxArch  architecture name used by Linux for the current system

	triple         Rosa OS host triple of the target system
	arch           architecture target name of the target system
	linuxArch      architecture name used by Linux for the target system
	llvmArch       architecture name used by LLVM for the target system

	jobsE          expression for preferred job count
	jobsFlagE      expression for flag with preferred job count (such as -j8)
	jobsLE         expression for twice of preferred job count
	jobsLFlagE     expression for flag with twice of preferred job count (such
	               as -j16)

	loadE          expression for preferred loadavg target
	loadFlagE      expression for flag with preferred loadavg target (such as
	               -l9)

	anityaFallback version filter function pointer, returns the first stable
	               version, or latest if that is unavailable

# Library functions

replace returns a copy of the string s with the first n non-overlapping
instances of old replaced by new. If old is empty, it matches at the beginning
of the string and after each UTF-8 sequence, yielding up to k+1 replacements for
a k-rune string. If n == 0, there is no limit on the number of replacements.

	replace {
		s = "2.9.0";
		old = ".";
		new = "_";
		n = 2;
	}
	// returns "2_9_0"

slice returns a subslice of elems.

	slice {
		elems = split {
			s = "2.42.4";
			sep = ".";
			n = 3;
		};
		start = 0;
		end = 2;
	}
	// returns [ "2", "42" ]

join concatenates elems to create a single string. The separator string sep is
placed between elements in the resulting string.

	join {
		elems = slice {
			elems = split {
				s = "2.42.4";
				sep = ".";
				n = 3;
			};
			end = 2;
		};
		sep = ".";
	}
	// returns "2.42"

arch returns the value specified for the identifier referring to the target
architecture. As a special case, the cross identifier is matched if currently
cross-compiling, and the target architecture did not match any value specified
before the cross identifier.

	arch {
		cross = unset;
		default = arch;

		amd64 = "x86";
		ppc64le = "powerpc";
		riscv64 = "riscv";
	}

# Source fetchers

remoteTar reads an (optionally compressed) tar stream from url and unpacks its
contents. It is a cure error if the checksum of the full stream read until EOF
from url does not match the specified checksum.

	remoteTar {
		url = "https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-"+version+".tar.gz";
		checksum = "e-OfydG1EoUUuOCLHZn2XXt1h3ONKo9fg67IAcYSfU-3c8HrBXJ259AtdC8CZdk0";

		// available decompressors: gzip, bzip2, zstd
		//
		// defaults to uncompressed
		compress = gzip;
	};

remoteFile reads the stream from url until EOF. It is a cure error if the
checksum of the full stream read from url does not match the specified checksum.

	remoteFile {
		url = "https://web.archive.org/web/"+version+"id_/"+
			"https://www.iana.org/assignments/enterprise-numbers.txt";
		checksum = "IplVyf7P4PDhhDovWCzL6jPGmF0T75InOCv2IOTMoR7khHUcRfki1pY93kobuXHF";
	};

remoteGit downloads a remote git repository using the git package. It is a cure
error if the checksum of the [hakurei.app/pkg] archive of the resulting
directory does not match the specified checksum.

	output = remoteGit {
		url = "https://src.rosa.moe/hakurei";
		tag = "v"+version;
		checksum = "SZdto_lQCgwebCqUHJREQcHwwO1pHn3euOwFvXYEri0G-46zUhx0hX4buFt2XvkS";
	};

remoteGitLab downloads the specified ref from a GitLab instance. It is a cure
error if the checksum of the compressed tar stream read from the GitLab instance
does not match the specified checksum.

	remoteGitLab {
		domain = "gitlab.freedesktop.org";
		suffix = "wayland/wayland";
		ref = version;
		checksum = "79UJi1LDcQBoYl_yR7Jnt3OXptvyklTdJB7GZIeF_Zb0q0ETQRBp5RqsltdZOTAp";
	};

remoteGitHub downloads the specified tag from Microsoft Github. It is a cure
error if the checksum of the compressed tar stream read from Microsoft Github
does not match the specified checksum.

	remoteGitHub {
		suffix = "nss-dev/nss";
		tag = "NSS_"+join {
			elems = split {
				s = version;
				sep = ".";
				n = 3;
			};
			sep = "_";
		}+"_RTM";
		checksum = "GBkrmgdeEjvmLuWq6_EX5nkfebL4ojBE0yHsBgeijX7cnoLA5aOY2Z-GIolJwFy_";
	};

# Build system helpers

generic directly passes build script segments for invocations of one-off build
scripts too specific to fit into any other [Helper] implementation.

	generic {
			// whether to enter a newly created work directory; defaults to true
			chdir = false;

			// runs only if cross-compiling
			cross = `
	export HAKUREI_ARCH='`+arch+`'
	export HAKUREI_SYSROOT='`+prefix+`/`+arch+`'

	export PKG_CONFIG_SYSROOT_DIR='`+prefix+`/`+arch+`'
	export PKG_CONFIG_LIBDIR='`+prefix+`/`+arch+prefix+`/lib/pkgconfig:`+
		prefix+`/`+arch+``+prefix+`/share/pkgconfig'
	`;

			// runs first, or after cross if cross-compiling
			build = `
	NAME=make`;

			// runs after build, if testing is enabled
			check = `
	NAME=all`;

			// runs after build and check
			install = `
	DESTDIR=/work "./${NAME}.sh"
	`;
		}

make wraps the make program and optionally an autotools-generated build system.

	make {
		// whether to enter a newly created work directory; defaults to true
		chdir = true;

		// command to generate the build system if non-empty
		generate = "";

		// runs before configure
		preConfigure = "";

		// cross targets to pass to the configure script, the zero value passes
		// the Rosa OS target triples, the string `""` causes the corresponding
		// flag(s) to be omitted
		host, build = "";

		// whether to let autotools infer the target triple
		autoTarget = false;

		// whether to pass sysroot pathname to the configure script, defaults
		// to true
		sysroot = true;

		// do not clear program prefix/suffix
		defaultName = false;

		// alternative name for the configure script if non-empty
		configureName = "";

		// generate autoconf arguments regardless of configureName
		forceAutoconf = false;

		// flags passed to the configure script
		configure = {
			"disable-multilib";
			"enable-default-pie";
			"disable-nls";
			"with-gnu-as";
			"with-gnu-ld";
			"with-system-zlib";
			"enable-languages": "c,c++,go";
			"with-native-system-header-dir": prefix+"/include";

			"with-multilib-list": arch {
				amd64, arm64 = "''";
			};
		};

		// runs before make
		preMake = "";

		// additional make arguments
		make = [
			"BOOT_CFLAGS='-O2 -g'",
			"bootstrap",
		];

		// runs before check
		preCheck = "";

		// replaces the check arguments if non-empty
		check = nil;

		// replaces the default install command if non-empty
		install = "";
	};

cmake builds and tests a CMake project with specified CACHE entries.

	cmake {
		// path elements joined with source
		append = [ "compiler-rt/lib/builtins" ];

		// CMake CACHE entries
		cache = {
			"LLVM_HOST_TRIPLE": triple;
			"LLVM_ENABLE_RTTI": "ON";
			"LLVM_BUILD_LLVM_DYLIB": "ON";
			"LLVM_LINK_LLVM_DYLIB": "ON";
			"LLVM_ENABLE_PER_TARGET_RUNTIME_DIR": "ON";

			"COMPILER_RT_DEFAULT_TARGET_ONLY": "ON";
		};

		// runs after install
		postInstall = "";

		// replaces the default test command if non-empty; explicitly setting
		// this to nil disables testing entirely
		test = nil;
	};

meson builds and tests a meson project.

	meson {
		// whether to use muon instead of meson
		muon = false;

		// -D flags passed to the setup command
		setup = {
			"default_library": "both";
			"documentation": "false";
			"tests": "true";
		};

		// runs after setup
		preCompile = "";

		// runs after compile
		postCompile = "";

		// whether to run the test suite; defaults to true
		test = true;

		// names of test suites to skip
		skipTests = [];

		// whether to run tests with interactive input/output
		interactiveTest = false;

		// runs after install
		postInstall = "";
	};

[Rosa OS packages repository]: https://src.rosa.moe/packages
*/
package rosa