Skip to content

dev: version checks, README, merge gate #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ozankiratli merged 3 commits into from
Aug 17, 2026
Merged

dev: version checks, README, merge gate #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
922c3b3
Manual is mostly moved to github pages
ozankiratli Aug 16, 2026
f9b3437
Version check is added.
ozankiratli Aug 16, 2026
85fbe74
Merge checks added
ozankiratli Aug 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 96 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
name: Checks

# The gate on merging into main, and an early warning on dev.
#
# Everything here is static: nothing runs the pipeline or needs sequencing data.
# It catches the class of breakage that would otherwise only surface at tag time
# (release.yml) or on someone else's machine - a syntax error in a helper, a
# version that drifted between the three places it lives, a template that no
# longer parses, an archive missing a file a run needs.
#
# It deliberately does NOT run on push to main: by then it is too late to gate
# anything, and main is only written by merges that already passed here.

on:
pull_request:
branches: [main]
push:
branches: [dev]
workflow_dispatch:

permissions:
contents: read

env:
# Pinned to the version install/environment.yml ships, so CI checks the same
# Nextflow a user will actually run.
NXF_VER: 26.04.6

concurrency:
group: checks-${{ github.ref }}
cancel-in-progress: true

jobs:
checks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
# verify-archive.sh builds from a git ref, so it needs real history.
fetch-depth: 0

- name: The version agrees with itself
run: |
set -euo pipefail
declared=$(sed -n 's/^VERSION="\(.*\)"$/\1/p' PoolSeqFlow)
header=$(sed -n 's/^# Version: \(.*\)$/\1/p' PoolSeqFlow)
manifest=$(sed -n "s/^[[:space:]]*version[[:space:]]*=[[:space:]]*'\(.*\)'.*/\1/p" nextflow.config)
echo " ./PoolSeqFlow VERSION= : $declared"
echo " ./PoolSeqFlow header : $header"
echo " nextflow.config : $manifest"
[ -n "$declared" ] || { echo "no VERSION= in ./PoolSeqFlow" >&2; exit 1; }
[ "$header" = "$declared" ] && [ "$manifest" = "$declared" ] || {
echo "the three version strings disagree" >&2; exit 1; }

- name: Shell scripts parse
run: |
set -euo pipefail
for s in PoolSeqFlow bin/*.sh install/*.sh dev/scripts/*.sh; do
bash -n "$s" && echo " ok $s"
done

- name: Python and awk helpers parse
run: |
set -euo pipefail
python3 -m py_compile bin/MajorAlleleToRef.py && echo " ok bin/MajorAlleleToRef.py"
# Running it on empty input executes BEGIN and nothing else, which is
# enough to reject a syntax error without needing a VCF.
awk -f bin/depth2freq.awk /dev/null && echo " ok bin/depth2freq.awk"

- uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "17"

- name: Install Nextflow
run: |
curl -s https://get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/
nextflow -version

- name: Pipeline lints without errors
run: nextflow lint poolseqflow.nf scripts/*.nf

# The failure this catches is the one users hit as `null: command not
# found` partway through a run - a parameter the template no longer
# resolves. Cheap to check, invisible otherwise until it bites.
- name: The config template parses
run: |
set -euo pipefail
cp parameters.config.template parameters.config
nextflow config -flat > /dev/null
echo " ok parameters.config.template resolves"
rm -f parameters.config

- name: The release archive is a working copy
run: dev/scripts/verify-archive.sh
73 changes: 14 additions & 59 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,16 @@ jobs:
exit 1
fi

# The version lives in nextflow.config's manifest too - that is what Nextflow
# reports and what step 0 records alongside a project's outputs, so a stale
# value there would misattribute results to the wrong release.
nfversion=$(sed -n "s/^[[:space:]]*version[[:space:]]*=[[:space:]]*'\(.*\)'.*/\1/p" nextflow.config)
if [ "$nfversion" != "$declared" ]; then
echo "./PoolSeqFlow says VERSION=\"$declared\" but nextflow.config's" >&2
echo " manifest says version = '$nfversion'" >&2
exit 1
fi

# On a tag, the tag has to match. On workflow_dispatch there is no tag,
# and the run builds and verifies the archive without publishing.
if [ "${GITHUB_REF_TYPE:-}" = "tag" ]; then
Expand All @@ -79,13 +89,14 @@ jobs:
echo "name=PoolSeqFlow-$declared" >> "$GITHUB_OUTPUT"
echo "Building PoolSeqFlow-$declared"

- name: Build the archive
# Same script ci.yml runs on every pull request, so what is published here
# has already been checked in review rather than only at tag time.
- name: Build and verify the archive
env:
NAME: ${{ steps.version.outputs.name }}
run: |
set -euo pipefail
mkdir -p dist
git archive --format=tar.gz --prefix="${NAME}/" -o "dist/${NAME}.tar.gz" HEAD
dev/scripts/verify-archive.sh HEAD dist

# The same archive under a stable name. GitHub's
# /releases/latest/download/<asset> only resolves a filename it can
Expand All @@ -97,62 +108,6 @@ jobs:
( cd dist && sha256sum ./*.tar.gz > SHA256SUMS )
ls -lh dist/

- name: Verify the archive is a working copy of the pipeline
env:
NAME: ${{ steps.version.outputs.name }}
VERSION: ${{ steps.version.outputs.version }}
run: |
set -euo pipefail
tmp=$(mktemp -d)
tar -xzf "dist/${NAME}.tar.gz" -C "$tmp"
root="$tmp/${NAME}"

fail() { echo "ARCHIVE CHECK FAILED: $*" >&2; exit 1; }

# Everything a run needs.
for f in PoolSeqFlow poolseqflow.nf nextflow.config \
parameters.config.template RGTags.csv.template \
install/environment.yml install/check_install.sh \
LICENSE README.md CHANGELOG.md; do
[ -f "$root/$f" ] || fail "missing from archive: $f"
done

# All nine step modules, not however many happened to be committed.
steps_found=$(find "$root/scripts" -name '*.nf' | wc -l)
[ "$steps_found" -eq 9 ] || fail "expected 9 scripts/*.nf, found $steps_found"

# The helpers the process scripts call by bare name via nextflow.config's PATH.
for f in atomic_mv.sh config_migrate.sh createDepthFile.sh \
depth2freq.awk filterFalsePositives.sh MajorAlleleToRef.py; do
[ -f "$root/bin/$f" ] || fail "missing from archive: bin/$f"
done

# Repository furniture must NOT ship. This is the other half of
# .gitattributes: if an export-ignore is dropped, this catches it.
for f in docs .github mkdocs.yml .gitignore .gitattributes dev Project; do
[ ! -e "$root/$f" ] || fail "should have been export-ignored: $f"
done

# The executable bit is the whole reason for git archive over tar.
[ -x "$root/PoolSeqFlow" ] || fail "./PoolSeqFlow is not executable"
for s in "$root"/bin/*; do
[ -x "$s" ] || fail "$(basename "$s") is not executable"
done

# Syntax-check every shell script that ships.
bash -n "$root/PoolSeqFlow" || fail "./PoolSeqFlow has a syntax error"
for s in "$root"/bin/*.sh "$root"/install/*.sh; do
bash -n "$s" || fail "$(basename "$s") has a syntax error"
done

# The wrapper cannot simply be run here: it calls `conda shell.bash hook`
# at the top under `set -e`, and the runner has no conda. Check the
# version it would report instead.
grep -q "^VERSION=\"${VERSION}\"$" "$root/PoolSeqFlow" \
|| fail "extracted wrapper does not report ${VERSION}"

echo "Archive verified: $(du -h "dist/${NAME}.tar.gz" | cut -f1)"

- name: Publish the release
if: github.ref_type == 'tag'
uses: softprops/action-gh-release@v3
Expand Down
9 changes: 6 additions & 3 deletions PoolSeqFlow
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ case $COMMAND in
echo " $STORAGE/Reference - indices and the SnpEff database"
echo " $STORAGE/.poolseqflow_params"
echo " $STORAGE/.poolseqflow_rgtags"
echo " $STORAGE/.poolseqflow_versions"
[ -n "$WORKDIR" ] && echo " $WORKDIR"
echo ""
echo "Your raw sequencing data in $DATADIR is not touched."
Expand All @@ -166,9 +167,11 @@ case $COMMAND in
if [ -n "$WORKDIR" ]; then rm -rf "$WORKDIR"; fi
rm -rf "$STORAGE"/Reports "$STORAGE"/Logs "$STORAGE"/Output \
"$STORAGE"/Reference "$STORAGE"/PoolSeqFlow_*
# These record the parameters and RG tags behind outputs that no longer exist;
# leaving them would make the next run fail its checks for no reason.
rm -f "$STORAGE"/.poolseqflow_params "$STORAGE"/.poolseqflow_rgtags
# These record the parameters, RG tags and pipeline versions behind outputs that
# no longer exist; leaving them would make the next run fail its checks for no
# reason, and would attribute fresh results to an earlier release.
rm -f "$STORAGE"/.poolseqflow_params "$STORAGE"/.poolseqflow_rgtags \
"$STORAGE"/.poolseqflow_versions
echo "Reset complete."
;;
version)
Expand Down
Loading
Loading