How to write a bash script for your reproducible data analysis pipeline (CC247)

Bash and Zsh are tremendously powerful tools for doing file operations from the command line. You can combine multiple bash programs in a script to automate your data analysis pipelines. In this Code Club, Pat uses bash scripts and a driver script to download files from a website and will share some tricks for using wget and tar within his bash scripts. The goal of this project is to highlight reproducible research practices using a number of tools. The specific output from this project will be a map-based visual that shows the level of drought across the globe. You can find my blog post for this episode at https://www.riffomonas.org/code_club/2022-09-12-bash-scripts. #bash #wget #tar #R #Rstats Support Riffomonas by becoming a Patreon member! https://www.patreon.com/riffomonas Want more practice on the concepts covered in Code Club? You can sign up for my weekly newsletter at https://shop.riffomonas.org/youtube to get practice problems, tips, and insights. If you're interested in taking an upcoming 3 day R workshop be sure to check out our schedule at https://riffomonas.org/workshops/ You can also find complete tutorials for learning R with the tidyverse using... Microbial ecology data: https://www.riffomonas.org/minimalR/ General data: https://www.riffomonas.org/generalR/ 0:00 Introduction 1:09 Navigating NOAA's GHCN data portal 3:47 Creating an executable bash script 6:27 Creating a bash script to download data with wget 14:15 Creating command line arguments for bash scripts 17:43 Creating a driver script for pipeline 19:57 Add another step to pipeline with tar