W36: Advanced UNIX: Scripting and Queueing for Large Data

Biological data (e.g., from sequencing) continues to grow rapidly, and day-to-day work of a bioinformatician increasingly requires efficient facility with command line management, preparation, computation over, and summarization of large files. As the most ubiquitous and stable interface to command line computing, advanced use of the universal “glue” that is command line shells (e.g., GNU BASH) is an important skill. Large computations also often require sophisticated use of the Hoffman2 job queueing system. This Workshop will go over (with examples) advanced shell features (complex pipelines, functions, control flow, arrays, string manipulation, arithmetic, robustness) and parallelization (both on any single machine, as well as with the Hoffman2 job array system).