Instead of doing a simple
#!/bin/bash chmod +x tesscurl_sector_1_ffic.sh ./tesscurl_sector_1_ffic.sh
call for these download scripts, implement the following:
#!/bin/bash cat tesscurl_sector_1_ffic.sh | \ awk '{ if ( 6<=NF ) print $6; }' | \ sort | \ awk '{ n++; a=a " " $1; if ( n==16 ) { print a; n=0; a=""; } } END { print a; }' | \ while read list; do cmd="" for fits in $list; do test -f $fits && continue cmd="$cmd -o $fits https://mast.stsci.edu/api/v0.1/Download/file/?uri=mast:TESS/product/$fits" done if test -n "$cmd"; then t0=`date +%s.%N` curl -C - -L $cmd t1=`date +%s.%N` echo $t0 $t1 | awk '{ printf("Time: %.3f seconds\n",$2-$1); }' fi done
With this optimization, one can achieve a speed-up of a factor of 2.5 ... 3.