On computer cluster

Preparations

# Prepare variables in terminal
TAG_OR_BRANCH="vX.Y.Z" # eg TAG_OR_BRANCH="v0.18.3"
PIPELINE_NAME="Your_pipeline_name" # eg PIPELINE_NAME="Twist_Solid"
PIPELINE_SHORT_NAME="wpX_name" # wg PIPELINE_SHORT_NAME="wp1_gms560"
USER="username_on_remote_cluster"

Uncompress files

Extract environment and repositories, and run conda-unpack to remove prefixes from build environment.

# Miarka
cd /proj/ngi2024001/nobackup/bin/${PIPELINE_SHORT_NAME}/${TAG_OR_BRANCH}/
tar --strip-components=1 -zxvf ${PIPELINE_NAME}_${TAG_OR_BRANCH}.tar.gz
mkdir venv && tar zxvf env.tar.gz -C venv/
source venv/bin/activate
conda-unpack 

Decompress reference files

# Miarka
cd /proj/ngi2024001/nobackup/bin/${PIPELINE_SHORT_NAME}/
tar -xvf design_and_ref_files.tar.gz design_and_ref_files

Validate reference files

Validate that all design and reference files exists and haven't changed

OBS! Warnings for possible file PATH/hydra-genetics and missing tbi files in config can be ignored

# Twist solid example for Miarka
hydra-genetics --debug references validate -c config/config.yaml -c config/config.data.hg19.yaml -v config/references/design_files.hg19.yaml -v config/references/nextseq.hg19.pon.yaml -v config/references/references.hg19.yaml -p /proj/ngi2024001/nobackup/bin/${PIPELINE_SHORT_NAME}/design_and_ref_files

Final adjustments of profile and start-scripts

  • Set correct paths in the profile for singularity-prefix and wrapper-prefix if not already done.
  • Update start-script to use the correct profile

Run Pipeline

Load any required modules (e.g., slurm-drmaa for the clusters job submission system) Run the pipeline as usual but with updated profile or updated start script.