Test version with new meta data: /shared/anaconda2/bin/python2.7 /lustre/athorpe/ch4/compute_ime_040318_akt/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_2016_20180629.xlsx "Plume_List" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518 --calids v1n2 v2m4 -p 1000 -m 10 -f 150 /shared/anaconda2/bin/python2.7 /lustre/athorpe/ch4/compute_ime_040318_akt/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_2017_20180629.xlsx "Plume_List" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518 --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -m 10 -f 150 Generate IME: cat /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2016*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/imevals_FA16_Landfills_1000ppmm_150fetch_20180725.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/imevals_SP17_FA17_Landfills_1000ppmm_150fetch_20180725.txt FA16: cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2016*/mergedist10/*.tif /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2016*/mergedist10/*.pdf /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2016*/mergedist10/*.geojson /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/TIF_PDF_JSON/mergedist10 SP17, FA17: cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2017*/mergedist10/*.tif /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2017*/mergedist10/*.pdf /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/ang2017*/mergedist10/*.geojson /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_072518/TIF_PDF_JSON/mergedist10 Test version with Brian's latest code (revised distance calculations) /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_061518/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_2016_20180629.xlsx "Plume_List" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518 --calids v1n2 v2m4 -p 1000 -m 5 10 20 -f 150 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_061518/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_2017_20180629.xlsx "Plume_List" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518 --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -m 5 10 20 -f 150 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_061518/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_Aggregated_2016_20180629.xlsx "Supersources_ALL_columns" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_150fetch_5_10_20merge_ime_061518 --calids v1n2 v2m4 -p 1000 -m 5 10 20 -f 150 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_061518/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_Aggregated_2017_20180629.xlsx "Supersources_ALL_columns" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_150fetch_5_10_20merge_ime_061518 --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -m 5 10 20 -f 150 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_061518/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_Aggregated_2016_20180629.xlsx "Supersources_ALL_columns" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518 --calids v1n2 v2m4 -p 1000 -m 5 10 20 -f 500 /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime_061518/compute_ime.py /lustre/athorpe/ch4/gen_ime_geotif/input/Landfills_Aggregated_2017_20180629.xlsx "Supersources_ALL_columns" -o /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518 --calids v2p4 v2p7 v2p9 v2p10 -p 1000 -m 5 10 20 -f 500 Generate IME: Unaggregated cat /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2016*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/imevals_FA16_Landfills_1000ppmm_150fetch_20180709.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/imevals_SP17_FA17_Landfills_1000ppmm_150fetch_20180709.txt Aggregated cat /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2016*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/imevals_FA16_Landfills_Aggregate_1000ppmm_500fetch_20180709.txt cat /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2017*/*_ime.txt|sort|uniq >> /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/imevals_SP17_FA17_Landfills_Aggregate_1000ppmm_500fetch_20180709.txt Aggregate .JSON, .PDF, .TIF: FA16: cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2016*/mergedist10/*.tif /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2016*/mergedist10/*.pdf /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2016*/mergedist10/*.geojson /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 SP17, FA17: cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2017*/mergedist10/*.tif /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2017*/mergedist10/*.pdf /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/ang2017*/mergedist10/*.geojson /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_1000ppmm_150fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 FA16 (aggregate): cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2016*/mergedist10/*.tif /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2016*/mergedist10/*.pdf /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2016*/mergedist10/*.geojson /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/FA16_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 SP17, FA17 (aggregate): cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2017*/mergedist10/*.tif /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2017*/mergedist10/*.pdf /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 cp /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/ang2017*/mergedist10/*.geojson /lustre/athorpe/ch4/gen_ime_geotif/output/Landfills/SP17_FA17_Landfill_Aggregate_1000ppmm_500fetch_5_10_20merge_ime_061518/TIF_PDF_JSON/mergedist10 Instructions for IME parameters: Hi Andrew, Done, I actually had this in the script already but the option wasn't exposed. To set the ppmm threshold, use the -p/--ppmmthr command line argument (default=500.0 ppmm). Also, you can specify a list of mergedist values via the -m/--mergedists argument and set the maximum fetch value via -f/--fetchmax if you'd like to vary those parameters as well. See usage string below for other options which may or may not be useful. Let me know if you run into any issues. _B bbue@gibson:srcfinder$ python compute_ime/compute_ime.py --help usage: compute_ime.py [-h] [-m MERGEDISTS [MERGEDISTS ...]] [-c] [-o OUTDIR] [-f FETCHMAX] [-p PPMMTHR] [--nodetfilt] [-i inputfile] [-l listfile] [--randomize] [--filter filter] xlsfile sheetname CMF IME Computation + Product Generation positional arguments: xlsfile Source master list spreadsheet sheetname Source master list spreadsheet sheet name optional arguments: -h, --help show this help message and exit -m MERGEDISTS [MERGEDISTS ...], --mergedists MERGEDISTS [MERGEDISTS ...] List of one or more distances (in meters) to merge neighboring blobs (default=[10, 20, 50]) -c, --clobber Clobber (overwrite) existing outputs -o OUTDIR, --outdir OUTDIR Save filtered detection image to OUTDIR -f FETCHMAX, --fetchmax FETCHMAX Maximum fetch value (default=unlimited) -p PPMMTHR, --ppmmthr PPMMTHR Minimum ppmm threshold (default=500.00) --nodetfilt Do not filter out small or weakly connected detections -i inputfile, --inputfile inputfile CMF image (assumed listed in spreadsheet) -l listfile, --listfile listfile List of CMF images to process (subset of images in provided spreadsheet) --randomize Randomize spreadsheet processing order (by source id) --filter filter Compute IME for spreadsheet rows matching filters of the form "spreadsheet_column:filter_string (e.g., "flightline:ang201605")" Hi Andrew, Sorry for the confusion; the [] in the usage string means the argument is optional, and assumes a default value if unspecified. So, to set the ppmm threshold to 350 you'd use "-p 350" and to use mergedists=10,25 you'd use "-m 10 25". So, for the params you specified below it'd be: /shared/anaconda2/bin/python2.7 /lustre/bbue/ch4/compute_ime/compute_ime.py lustre/athorpe/ch4/gen_ime_geotif/input/SP17_Master_Source_List_IME_20170822.xlsx "SP17 plumes 2" -o /lustre/athorpe/ch4/gen_ime_geotif/output/ime_gtif_1000ppmmm_20170817 -p 1000 -m 10 25 -f 200 Note that mergedists is the only param that can take multiple values (hence the multivalued spec in the usage string: [-m MERGEDISTS [MERGEDISTS ...]]), the other params -p/-f are floating point scalars. _B