| Age | Commit message (Collapse) | Author |
|---|
|
This should hopefully provide good separation between the artifact curing backend implementation and the (still work in progress) language. Making the IR parseable also guarantees uniqueness of the representation.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This was taking way too long for early failures.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This also rearranges the ident ir to be more predictable, and avoids an obvious and somewhat easy to get into inconsistent state.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This alleviates scheduler overhead when curing many artifacts.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This uses the new measured reader provided by Cache. This should make httpArtifact zero-copy.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This will be fully implemented in httpArtifact in a future commit.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This avoids blocking while not in Cure method of the implementation.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This is an interesting value to know when profiling.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This enables deduplication by value as implemented in execArtifact.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This significantly simplifies error reporting for caller.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
Any fine-grained file-based locking here significantly hurts performance and is not part of the use case of the package. This change guarantees exclusive access to prevent inconsistent state on the filesystem.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This significantly improves scrubbing performance. Since the cache directory structure is friendly to simultaneous access, this is possible without synchronisation.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This eliminates duplicate identifier computations. The new implementation also significantly reduces allocations while computing identifier for a large dependency tree.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This provides a quick check against cyclic dependencies without hurting cure performance.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This is significantly more practical than keeping track of them in directory flattening format and setting this in every non-artifact implementation. Only tarArtifact can have meaningful deterministic checksums that are not zero and zeroing them still keeps autotools happy.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This reports cure completions to the user.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This makes verbose messages much more useful.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This result was mistakenly unused resulting in incorrect identifiers for artifacts with duplicate dependencies.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This removes the left over embedded contexts.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This imposes a hard upper limit to concurrency during dependency satisfaction and moves all dependency-related code out of individual implementations of Artifact. This change also includes ctx and msg as part of Cache.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This is useful for custom implementations of Artifact.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This improves efficiency for cache hits.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This is an Artifact implementing File, backed by a constant, caller-supplied byte slice.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This optionally attaches an output checksum to an execArtifact and enables host networking for the resulting container.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This runs a program in a container environment. Artifacts can be made available to the container, they are cured concurrently and mounted in order.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This avoids missing the cleanup when cure returns an error.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This allows a curing Artifact to cure Artifact it depends on.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This cleans up the function signature and makes backwards compatible API changes possible.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This change adds a method to check on-disk cache consistency and destroy inconsistent entries as they are encountered. This primarily helps verify artifact implementation correctness, but can also repair a cache that got into an inconsistent state from curing a misbehaving artifact, without having to destroy the entire cache.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This makes the checksum consistent with the final resting state of artifact directories without incurring the cost of an extra pair of chown syscalls.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
There is no reason to keep these around.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This allows for more flexibility during implementation. The use case that required this was for expanding single directory tarballs.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
The previous implementation exposes arbitrary user input to the cache as an identifier, which is highly error-prone and can cause the cache to enter an inconsistent state if the user is not careful. This change replaces the implementation to compute identifier late, using url string as params.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This moves all cache I/O code to Cache. Artifact now only contains methods for constructing their actual contents.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
These are much easier to handle than hard links and should be just as transparent for this use case.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This must be writable to enable renaming, and the final result is conventionally read-only alongside the entire directory contents. This change overrides the permission bits as part of Store.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This is still not ideal as it makes entry into Store sequential. This will be improved after more usage code is written.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This works on any directories and should be robust against any bad state the artifact curing process might have failed at.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This wraps base64.URLEncoding.EncodeToString for cleaner call site.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This provides infrastructure for computing a deterministic identifier based on current artifact kind, opaque parameters data, and optional dependency kind and identifiers.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This is used for artifacts that cure into directories.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
|
This change contains primitives for validating and caching single-file artifacts.
Signed-off-by: Ophestra <cat@gensokyo.uk>
|
