What’s New (History)¶
Post Version 1.0.1¶
User-visible improvements¶
- ZSTD compression is available for output and intermediate files use it for reduced temporary space usage (and possibly faster processing).
- The countfile now reorders its input if it is not ordered. This is necessary for correct usage.
- Added
include_fragments
option to orf_find(). - More flexible loading of
functional_map
arguments in count to accept multiple comment lines at the top of the file as produced by eggnog-mapper. - Faster check for column headers in
functional_map
argument to count() function: now it is performed as soon as possible (including at the top of the script if the arguments are literal strings), thus NGLess can fail faster. - Added
sense
argument to the count function, generalizing the previousstrand
argument (which is deprecated). Whereas before it was only possible to consider features either to be present on both strands or only on the strand to which they are annotated, now it is also possible to consider them present only on the opposite strand (which is necessary for some strand-specific protocols as they produce the opposite strand). - Added
interleaved
argument to fastq
Version 1.0.1¶
This is a bugfix release and results should not change.
Bugfixes¶
- Fix bug with external modules and multiple fastQ inputs.
- Fix bug with resaving input files where the original file was sometimes moved (thus removing it).
Version 1.0¶
User-visible improvements¶
- The handling of multiple annotations in count (i.e., when the user
requests multiple
features
and/orsubfeatures
) has changed. The previous model caused a few issues (#63, but also mixing with collect(). Unfortunately, this means that scripts asking for the old behaviour in their version declaration are no longer supported if they use multiple features.
Version 0.11¶
Released March 15 2019 (0.11.0) and March 21 2019 (0.11.1).
Version 0.11.0 used ZStdandard compression, which was not reliable (the official haskell zstd wrapper has issues). Thus, it was removed in v0.11.1. Using v0.11.0 is not recommended.
User-visible improvements¶
- Module samtools (version 0.1) now includes samtools_view
- Add –verbose flag to check-install mode (ngless –check-install –verbose)
- Add early checks for input files in more situations (#33)
- Support compression in collect() output (#42)
- Add smoothtrim() function
Bugfixes¶
- Fix bug with orf_find & prots_out argument
- Fix bug in garbage collection where intermediate files were often left on disk for far longer than necessary.
- Fix CIGAR (#92) for select() blocks
Internal improvements¶
- Switched to diagrams package for plotting. This should make building easier as cairo was often a complicated dependency.
- Update to LTS-13 (GHC 8.6)
- Update minimap2 version to 2.14
- Call bwa/minimap2 with interleaved fastq files. This avoids calling it twice (which would mean that the indices were read twice).
- Avoid leaving open file descriptors after FastQ encoding detection
- Tar extraction uses much less memory now (#77)
Version 0.10.0¶
Released Nov 12 2018
Bugfixes¶
- Fixed bug where header was printed even when STDOUT was used
- Fix to lock1’s return value when used with paths (#68 - reopen)
- Fixed bug where writing interleaved FastQ to STDOUT did not work as expected
- Fix saving fastq sets with –subsample (issue #85)
- Fix (hypothetical) case where the two mate files have different FastQ encodings
User-visible improvements¶
- samtools_sort() now accepts by={name} to sort by read name
- Add __extra_megahit_args to assemble() (issue #86)
- arg1 in external modules is no longer always treated as a path
- Added expand_searchdir to external modules API (issue #56)
- Support _F/_R suffixes for forward/reverse in load_mocat_sample
- Better error messages when version is mis-specified
- Support NO_COLOR standard: when
NO_COLOR
is present in the environment, print no colours. - Always check output file writability (issue #91)
paired()
now acceptsencoding
argument (it was documented to, but mis-implemented)
Version 0.9¶
Released July 12th 2018
User-visible improvements¶
- Added
allbest()
method to MappedRead. - NGLess will issue a warning before overwriting an existing file.
- Output directory contains PNG files with basic QC stats
- Added modules for gut gene catalogs of mouse, pig, and dog
- Updated the integrated gene catalog
Internal improvements¶
- All lock files now are continuously “touched” (i.e., their modification time is updated every 10 minutes). This makes it easier to discover stale lock files.
- The automated downloading of builtin references now uses versioned URLs, so that, in the future, we can change them without breaking backwards compatibility.
Version 0.8¶
Released May 6th 2018
Incompatible changes¶
- Added an extra field to the FastQ statistics, with the fraction of basepairs that are not ATCG. This means that uses of qcstats must use an up-to-date version declaration.
- In certain cases (see below), the output of count when using a GFF will change.
User-visible improvements¶
Better handling of multiple features in a GFF. For example, using a GFF containing “gene_name=nameA,nameB” would result in:
nameA,nameB 1 Now the same results in:: nameA 1 nameB 1
This follows after https://git.io/vpagq and the case of Parent=AF2312,AB2812,abc-3
Support for minimap2 as alternative mapper. Import the
minimap2
module and specify themapper
when calling map. For example:ngless '0.8' import "minimap2" version "1.0" input = paired('sample.1.fq', 'sample.2.fq', singles='sample.singles.fq') mapped = map(input, fafile='ref.fna', mapper='minimap2') write(mapped, ofile='output.sam')
Added the
</>
operator. This can be used to concatenate filepaths.p0 </> p1
is short forp0 + "/" + p1
(except that it avoids double forward slashes).Fixed a bug in select where in some edge cases, the sequence would be incorrectly omitted from the result. Given that this is a rare case, if a version prior to 0.8 is specified in the version header, the old behaviour is emulated.
Added bzip2 support to write.
Added reference argument to count.
Bug fixes¶
- Fix writing multiple compressed Fastq outputs.
- Fix corner case in select <Functions.html#select>__. Previously, it was possible that some sequences were wrongly removed from the output.
Version 0.7.1¶
Released Mar 17 2018
Improves memory usage in count()
and the use the when-true
flag in
external modules.
Version 0.7¶
Released Mar 7 2018
New functionality in NGLess language¶
- Added max_trim argument to
filter
method ofMappedReadSet
. - Support saving compressed SAM files
- Support for saving interleaved FastQ files
- Compute number Basepairs in FastQ stats
- Add
headers
argument to samfile function
Bug fixes¶
- Fix
count
’s mode{intersection_strict}
to no longer behave as{union}
- Fix
as_reads()
for single-end reads - Fix
select()
corner case
In addition, this release also improves both speed and memory usage.
Version 0.6¶
Released Nov 29 2017
New functionality in NGLess language¶
New functionality in NGLess tools¶
- Add –index-path functionality to define where to write indices.
- Allow citations as key in external modules (generally better citations information)
- Use multiple threads in SAM->BAM conversion
- Better error checking/script validation
Bug fixes¶
- Output preprocessed FQ statistics (had been erroneously removed)
- Fix –strict-threads command-line option spelling
- Version embedded megahit binary
- Fixed inconsistency between reference identifiers and underlying files