To list IME in text file: cat /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmmm_20170817/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmmm_20170817/imevals_ime_gtif_1000ppmmm_20170817.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmm_500fetch/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmm_500fetch/imevals_ime_gtif_1000ppmm_500fetch.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_500ppmm_250fetch_5merge/ang2016*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_500ppmm_250fetch_5merge/imevals_500ppmm_250fetch_5merge.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_250fetch_5merge/ang2016*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_250fetch_5merge/imevals_1000ppmm_250fetch_5merge.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_125fetch_5merge/ang2016*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_125fetch_5merge/imevals_1000ppmm_125fetch_5merge.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_250fetch_5merge/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_250fetch_5merge/imevals_1000ppmm_250fetch_5merge.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge/imevals_1000ppmm_250fetch_5merge.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_50fetch_5merge/ang2016*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_50fetch_5merge/imevals_1000ppmm_50fetch_5merge.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_50fetch_5merge/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_50fetch_5merge/imevals_1000ppmm_50fetch_5merge.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_50fetch_5merge/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_50fetch_5merge/imevals_1000ppmm_50fetch_5merge.txt To run GeoTIFF: /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_20170823.xlsx "SP17 plumes 2" -o /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmmm_20170817 -p 1000 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_20170823.xlsx "SP17 plumes 2" -o /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmm_500fetch -p 1000 -f 500 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_750ppmm_250fetch_5merge_nodefilt -p 750 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_500ppmm_250fetch_5merge -p 500 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_250fetch_5merge -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA17_Master_Source_List_IME_GasStorage.xlsx "FA17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA17_Gas_Storage_1000ppmm_250fetch_5merge -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_250fetch_5merge -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_250fetch_5merge_v2 -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py -p 1000 -f 250 -m 5 /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_250fetch_5merge_v2 --calids v2m2 -p 500 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge --calids v1n2 -p 1000 -f 250 -m 5 Woirks: /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_250fetch_5merge_v2 --calids v1n2 -p 500 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_125fetch_5merge_v2 --calids v1n2 -p 1000 -f 125 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_50fetch_5merge_v2 --calids v1n2 -p 1000 -f 50 -m 5 FA17: /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA17_Master_Source_List_IME_GasStorage.xlsx "FA17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA17_Gas_Storage_1000ppmm_250fetch_5merge --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA17a_Master_Source_List_IME_GasStorage.xlsx "FA17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_250fetch_5merge --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA17a_Master_Source_List_IME_GasStorage.xlsx "FA17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_125fetch_5merge --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -f 125 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA17a_Master_Source_List_IME_GasStorage.xlsx "FA17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_50fetch_5merge --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -f 50 -m 5 SP17: /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -f 250 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_50fetch_5merge --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -f 50 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_GasStorage.xlsx "SP17 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_500ppmm_500fetch_5merge_New_Color --calids v2p4 v2p7 v2p9 v2p10 -p 500 -f 500 -m 5 Rerun 2016 CARB report lines with new color scheme /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_new/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Source_Master_List_Sort_20170818a.xlsx "Fall 2016 plumes for report" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_500ppmm_500fetch_5merge_New_Color --calids v1n2 v2m4 -p 500 -f 500 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Master_Source_List_20170806.xlsx "Fall 2016 plumes for report v5" -o /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_500ppmm_500fetch_5merge_New_Color -p 500 -f 500 -m 5 Rerun 2016 Spring gas storage with new color scheme /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_new/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Source_Master_List_Sort_20170818a.xlsx "Fall 2016 plumes for report" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_500ppmm_500fetch_5merge_New_Color -p 500 -f 500 -m 5 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_new/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/FA16_Master_Source_List_IME_GasStorage.xlsx "FA16 plumes IME" -o /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_500ppmm_500fetch_5merge_New_Color --calids v1n2 v2m4 -p 500 -f 500 -m 5 2016 v1n2 v2m4 2017 v2p4 v2p7 v2p9 v2p10 Copy only TIF from merge50: cp /lustre/athorpe/ch4/Test_GeoTIFF/v3/ang2016*/mergedist50/*.tif /lustre/athorpe/ch4/Test_GeoTIFF/v3/c cp /lustre/bbue/ch4/ime_gtif_v3/ang2016*/mergedist50/*.tif /lustre/athorpe/ch4/ime_gtif_v3_mergedist50 cp /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_250fetch_5merge/ang2017*/mergedist5/*ime.png /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_250fetch_5merge/select_images cp /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_250fetch_5merge/ang2017*/mergedist5/*rgb.png /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_250fetch_5merge/select_images cp /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge/ang2017*/mergedist5/*rgb.png /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge/select_images cp /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge/ang2017*/mergedist5/*ime.png /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_250fetch_5merge/select_images cp /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_50fetch_5merge/ang2017*/mergedist5/*rgb.png /lustre/athorpe/ch4/gen_ime_geotif/output/FA17a_Gas_Storage_1000ppmm_50fetch_5merge/select_images cp /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_50fetch_5merge/ang2017*/mergedist5/*rgb.png /lustre/athorpe/ch4/gen_ime_geotif/output/SP17_Gas_Storage_1000ppmm_50fetch_5merge/select_images cp /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_50fetch_5merge/ang2016*/mergedist5/*rgb.png /lustre/athorpe/ch4/gen_ime_geotif/output/FA16_Gas_Storage_1000ppmm_50fetch_5merge/select_images Count files: find /lustre/bbue/ch4/ime_gtif_v3/ang2016*/mergedist50/*.tif | wc find /lustre/athorpe/ch4/ime_gtif_v3_mergedist50/*.tif | wc Count folders: find /lustre/bbue/ch4/ime_gtif_v3 -maxdepth 1 -type d -print| wc -1 Instructions for IME parameters: Hi Andrew, Done, I actually had this in the script already but the option wasn't exposed. To set the ppmm threshold, use the -p/--ppmmthr command line argument (default=500.0 ppmm). Also, you can specify a list of mergedist values via the -m/--mergedists argument and set the maximum fetch value via -f/--fetchmax if you'd like to vary those parameters as well. See usage string below for other options which may or may not be useful. Let me know if you run into any issues. _B bbue@gibson:srcfinder$ python compute_ime/compute_ime.py --help usage: compute_ime.py [-h] [-m MERGEDISTS [MERGEDISTS ...]] [-c] [-o OUTDIR] [-f FETCHMAX] [-p PPMMTHR] [--nodetfilt] [-i inputfile] [-l listfile] [--randomize] [--filter filter] xlsfile sheetname CMF IME Computation + Product Generation positional arguments: xlsfile Source master list spreadsheet sheetname Source master list spreadsheet sheet name optional arguments: -h, --help show this help message and exit -m MERGEDISTS [MERGEDISTS ...], --mergedists MERGEDISTS [MERGEDISTS ...] List of one or more distances (in meters) to merge neighboring blobs (default=[10, 20, 50]) -c, --clobber Clobber (overwrite) existing outputs -o OUTDIR, --outdir OUTDIR Save filtered detection image to OUTDIR -f FETCHMAX, --fetchmax FETCHMAX Maximum fetch value (default=unlimited) -p PPMMTHR, --ppmmthr PPMMTHR Minimum ppmm threshold (default=500.00) --nodetfilt Do not filter out small or weakly connected detections -i inputfile, --inputfile inputfile CMF image (assumed listed in spreadsheet) -l listfile, --listfile listfile List of CMF images to process (subset of images in provided spreadsheet) --randomize Randomize spreadsheet processing order (by source id) --filter filter Compute IME for spreadsheet rows matching filters of the form "spreadsheet_column:filter_string (e.g., "flightline:ang201605")" Hi Andrew, Sorry for the confusion; the [] in the usage string means the argument is optional, and assumes a default value if unspecified. So, to set the ppmm threshold to 350 you'd use "-p 350" and to use mergedists=10,25 you'd use "-m 10 25". So, for the params you specified below it'd be: /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_20170822.xlsx "SP17 plumes 2" -o /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmmm_20170817 -p 1000 -m 10 25 -f 200 Note that mergedists is the only param that can take multiple values (hence the multivalued spec in the usage string: [-m MERGEDISTS [MERGEDISTS ...]]), the other params -p/-f are floating point scalars. _B