rule mask_self fails on Docker container

[lucasnell]

I'm trying to use dentist on a docker container, but the example always fails during the mask_self step. From my mac, I'm starting the container using docker run -it --rm=true --platform linux/x86_64 centos:7 /bin/bash. I'm then running the following:

yum update -y -q
yum install -y wget

wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh
sh Mambaforge-Linux-x86_64.sh -b -p /opt/mamba3
export PATH=/opt/mamba3/bin:$PATH
rm Mambaforge-Linux-x86_64.sh

mamba install -c conda-forge -c bioconda -y snakemake

wget https://github.com/a-ludi/dentist/releases/download/v3.0.0/dentist-example.tar.gz
tar -xzf dentist-example.tar.gz
cd dentist-example

snakemake --configfile=snakemake.yml --use-conda --cores=1

This causes the following error:

Error in rule mask_self:
    jobid: 11
    output: workdir/.reference.dentist-self.anno, workdir/.reference.dentist-self.data
    log: logs/mask-self.reference.log (check log file(s) for error message)
    conda-env: /dentist-example/.snakemake/conda/850bc5c09e81d3d6b875839f8fe0ed70
    shell:
        dentist mask --config=dentist.yml  workdir/reference.dam workdir/reference.reference.las dentist-self 2> logs/mask-self.reference.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

The contents of logs/mask-self.reference.log:

{"executableVersion":"v3.0.0","refDb":"workdir/reference.dam","readsDb":"","dbAlignmentFile":"workdir/reference.reference.las","repeatMask":"dentist-self","configFile":"dentist.yml","debugRepeatMasks":false,"help":false,"maxCoverageReads":4294967295,"coverageBoundsReads":[0,0],"maxCoverageSelf":3,"coverageBoundsSelf":[0,3],"maxImproperCoverageReads":4294967295,"improperCoverageBoundsReads":[0,0],"properAlignmentAllowance":100,"quiet":false,"readCoverage":20,"revertOptionNames":[],"tracePointDistance":100,"verbosity":2}
{"thread":274939902976,"logLevel":"diagnostic","timestamp":637783965479794038,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.run","state":"enter"}
{"thread":274939902976,"logLevel":"diagnostic","timestamp":637783965479801084,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.readInputs","state":"enter"}
{"thread":274939902976,"logLevel":"diagnostic","state":"pre","command":["DBdump","workdir/reference.dam"],"timestamp":637783965479844193,"action":"execute","type":"pipe"}
{"thread":274939902976,"logLevel":"diagnostic","timestamp":637783965480006401,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.readInputs","state":"exit","timeElapsed":203652}
{"thread":274939902976,"logLevel":"diagnostic","timestamp":637783965480009609,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.run","state":"exit","timeElapsed":208842}
core.exception.AssertError@source/dentist/util/process.d(215): Attempting to fetch the front of an empty LinesPipe
----------------
??:? _d_assert_msg [0x40010b3e06]
??:? uint dentist.dazzler.numDbRecords(in immutable(char)[]) [0x4000dbef18]
??:? dentist.util.process.LinesPipe!(dentist.util.process.ProcessInfo, 0).LinesPipe dentist.dazzler.dbdump!(ulong[]).dbdump(in immutable(char)[], ulong[], in immutable(char)[][]) [0x4000dcbdad]
??:? uint[] dentist.dazzler.LocalAlignmentReader.contigLengths(immutable(char)[]) [0x4000dbfff3]
??:? dentist.dazzler.LocalAlignmentReader dentist.dazzler.LocalAlignmentReader.__ctor(const(immutable(char)[]), immutable(char)[], immutable(char)[], dentist.dazzler.BufferMode, dentist.common.alignments.base.TracePoint[]) [0x4000dbfc21]
??:? void dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.readInputs() [0x4000c73d52]
??:? void dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.run() [0x4000c73b09]
??:? dentist.commandline.ReturnCode dentist.commandline.runCommand!(2).runCommand(in immutable(char)[][]) [0x4000bbd656]
??:? dentist.commandline.ReturnCode dentist.commandline.run(in immutable(char)[][]) [0x4000b2850d]
??:? _Dmain [0x40009bee67]

Strangely, I can run the same series of commands on a CentOS 7 cluster with no issue. The contents of logs/mask-self.reference.log for the successful run are

{"executableVersion":"v3.0.0","refDb":"workdir/reference.dam","readsDb":"","dbAlignmentFile":"workdir/reference.reference.las","repeatMask":"dentist-self","configFile":"dentist.yml","debugRepeatMasks":false,"help":false,"maxCoverageReads":4294967295,"coverageBoundsReads":[0,0],"maxCoverageSelf":3,"coverageBoundsSelf":[0,3],"maxImproperCoverageReads":4294967295,"improperCoverageBoundsReads":[0,0],"properAlignmentAllowance":100,"quiet":false,"readCoverage":20,"revertOptionNames":[],"tracePointDistance":100,"verbosity":2}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913924743264,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.run","state":"enter"}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913924743707,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.readInputs","state":"enter"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","workdir/reference.dam"],"timestamp":637783913924766164,"action":"execute","type":"pipe"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","-h","workdir/reference.dam"],"timestamp":637783913924790405,"action":"execute","type":"pipe"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","workdir/reference.dam"],"timestamp":637783913924830211,"action":"execute","type":"pipe"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","-h","workdir/reference.dam"],"timestamp":637783913924851842,"action":"execute","type":"pipe"}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913924889160,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.readInputs","state":"exit","timeElapsed":145216}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913924889516,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.assessRepeatStructure","state":"enter"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","workdir/reference.dam"],"timestamp":637783913924890069,"action":"execute","type":"pipe"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","-r","-h","workdir/reference.dam"],"timestamp":637783913924911577,"action":"execute","type":"pipe"}
{"alignmentType":"self","thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913925023014,"repetitiveRegions":null,"numRepetitiveRegions":185}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913925023427,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.assessRepeatStructure","state":"exit","timeElapsed":133711}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913925023646,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.writeRepeatMask","state":"enter"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","workdir/reference.dam"],"timestamp":637783913925025791,"action":"execute","type":"pipe"}
{"thread":22377401060544,"logLevel":"diagnostic","state":"pre","command":["DBdump","workdir/reference.dam"],"timestamp":637783913925047965,"action":"execute","type":"pipe"}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913925069691,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.writeRepeatMask","state":"exit","timeElapsed":45797}
{"thread":22377401060544,"logLevel":"diagnostic","timestamp":637783913925069958,"function":"dentist.commands.maskRepetitiveRegions.RepeatMaskAssessor.run","state":"exit","timeElapsed":326260}

I've also tried this on the condaforge/mambaforge:4.11.0-0 container, but I get the same error.

Any insights are greatly appreciated!

[a-ludi]

Hi, that seems strange. I have no clue what is going on. I suspect it has nothing directly to do with DENTIST. It might be something file system-related.

What's the output of DBdump workdir/reference.dam? Is it empty?

[lucasnell]

I just re-ran the exact same commands and am getting a new error:

SourceFileError in line 78 of /dentist-example/Snakefile:
Error in source file definition: must be given as Python string or one of the predefined source file marker types (see docs)
  File "/dentist-example/Snakefile", line 775, in <module>
  File "/dentist-example/Snakefile", line 78, in prefetch_conda_env

I'm completely confused now, especially since it works fine when it's not on a docker container. I'd be okay closing this issue since it doesn't appear to be a problem with DENTIST.

[a-ludi]

I tried running your code on my machine. First, I could reproduce the latest error which occurred because of an update to Snakemake. I have created
a patch (will be included in next version). Please try applying it to see if the error vanishes:

wget https://github.com/a-ludi/dentist/files/8031135/snake.patch.txt
yum install -y patch
patch Snakefile snake.patch.txt 

After this first fix, everything works as expected, though. It seems your original error is related to the machine you are running Docker on or its setup/configuration. You could try it on a different machine.