This readme file will guide you through the set-up of the baseline algorithm publicly available as part of the multiview stereo benchmark package. http://www.jhuapl.edu/satellite-benchmark.html % The Johns Hopkins University, Applied Physics Laboratory % Reference: % [1] M. Bosch, Z. Kurtz, S. Hagstrom, M. Brown % A Multiple View Stereo Benchmark for Satellite Imagery. % Proceedings of IEEE Applied Imagery Pattern Recognition (AIPR). % Washington DC, October 2016 -------------DIRECTORY STRUCTURE------------- 'algo': contains the example algorithm to create point clouds from multiple satellite images. It can also run the benchmark to generate the metrics 'crop_imgs': pre-loaded tif images and txt metadata that the algorithm uses for all sites 'execs': preloaded 'windows' executables to run the metrics software directly from Matlab as well. -crop-image.exe: it reads a NITF image and writes into an 8-bit TIFF only the region specified by the KML file. Usage:$crop-image.exe -k KMLFILE.kml -o OUTPUTTIFNAME.tif INPUTNITFIMAGE.NTF -crop-image_master.exe: it reads a NITF image and writes into an 8-bit TIFF the region spanning the centroid of the KML coordinates with a 1000 pixel radius. Usage:$crop-image_master.exe -k KMLFILE.kml -o OUTPUTTIFNAME.tif INPUTNITFIMAGE.NTF -crop-image_650rad.exe: it reads a NITF image and writes into an 8-bit TIFF the region spanning the centroid of the KML coordinates with a 650 pixel radius. Usage:$crop-image_650rad.exe -k KMLFILE.kml -o OUTPUTTIFNAME.tif INPUTNITFIMAGE.NTF -run-metrics_master,exe: it produces the metrics scores for a given point cloud and grountruth input Usage:$run-metrics_master.exe -i POINTCLOUDINPUT.las -t truth_forSITE.tif 'gt': Contains the geotiff groundtruth files for all 8 sites of the benchmark 'imgs': it is a temporal directory where the algorithm stores the 8-bit TIF files of the sites being run 'rpc': it is a temporal directory where the algorithm stores the metadata files of the sites being run 'kml': It contains all the kml files for the benchmark. Drag and drop them into Google Earth to see the areas 'las': It contains the resulting point cloud from the algorithm (in LAS format) and metric summary 'utilities': other dependencies and sample code to read NITF images, run metrics code,... -------------RUN THE ALGORITHM--------------- MVS_JHUAPL_example. (Inside 'algo' directory) It is an algorithm implemented in Matlab for the most part. main_mvs_baseline_array.m is the main program to run. Usage: Run in matlab: main_mvs_baseline_array Configuration as delivered: FASTCONFIG=1;siteID=0;cropread=1;evaluation=0; Configuration to reproduce results published in [1]: FASTCONFIG=0;siteID=1;cropread=1;evaluation=1; This will need the BENCHMARK dependencies installed (see below) ALGO DEPENDENCIES: It has few dependencies in C++/matlab that have been slightly adapted to our problem (included with the release). They are in '3rdparty' folder in the 'algo' directory. - matchSIFT: to find pixel correspondences between pairs of images (pre-compiled for mac and windows -64bit, tested only on windows) - vgg: for planar registration using robust estimator (RANSAC) - SIFTflow: to compute the optical flow of two images (pre-compiled for mac and windows -64bit, tested only on windows) - deg2utm.m: script to convert from degrees to UTM standard coordinates - utm2deg.m:script to convert from UTM standard coordinates to degress - lasdata.m: script that takes a x,y,z buffer and creates a LAS file to display point clouds. BENCHMARK DEPENDENCIES There are two main dependencies for the becnhmark in order to read pixel data from NITFs and run the metrics: - masterchallenge_cropNITFs: This code contains sample code that shows how to read/access pixel data and metadata in a NITF image. The matlab algorithm uses it to generate TIF images out of the larger NITF for the selected site. SETUP: 1. Have Visual Studio (tested on version 2012) and CMake installed 2. Extract the built dependencies from here: utilities\dependencies.zip 3. Open the CMake GUI * select the root code directory using "Browse Source..." * Select an empty build directory using "Browse Build..." * Press Configure, select "Visual Studio 11 2012 Win64" from the drop-down, press "Finish" (It will fail, but that's OK) * Select the EXTERNAL_LIBS_ROOT entry and press the "..." on the far right. * In the dialog select the folder of the built dependencies (containing include, lib, bin). * Press Configure (Should succeed) * Press Generate. 4. Navigate to the build directory you created and open the eopcc-samples.sln 5. Standard Visual Studio usage from here out. Change to release mode to generate the executable (it replicates crop-image_master.exe) 6. Note: You'll get missing dll complaints when you try to run unless dependencies/bin is in the PATH (standard Windows stuff). - masterchallenge_metrics: This code executes the metrics and it computes relative accuracy and completeness values. It performs X,Y,Z translational registration to correct for global errors. It reports the following metrics: "completeness" : percentage of valid truth points that your solution covers. For a point to be counted towards completeness needs to be within 1m. of the ground truth corresponding 3D point "error_rms (a.k.a. accuracy)":3d geometric error for all valid points of the solution: sqrt((Ztruth-Zinput)^2/Nvalidpoints) "error_median":The error in meters at 50%-th point "error_percentile":error in meters at 10%,25%,50%,75%,90%,99% of points "cumulative histogram": ratio of points with smaller or equal error than 0.5,1,2,5,25 meters "reg_shift": overall error before regustration SETUP: 1. Have Visual Studio 2012 and CMake installed 2. Extract the built dependencies from here: dependencies.zip 3. Open the CMake GUI * select the root code directory using "Browse Source..." * Select an empty build directory using "Browse Build..." * Press Configure, select "Visual Studio 11 2012 Win64" from the drop-down, press "Finish" (It will fail, but that's OK) * Select the EXTERNAL_LIBS_ROOT entry and press the "..." on the far right. * In the dialog select the folder of the built dependencies (containing include, lib, bin). * Press Configure (Should succeed) * Press Generate. 4. Navigate to the build directory you created and open the eopcc-samples.sln 5. Standard Visual Studio usage from here out. Change to release mode to generate the executable (it replicates run-metrics_master.exe) 6. Note: You'll get missing dll complaints when you try to run unless dependencies/bin is in the PATH (standard Windows stuff). HELP SOFTWARE: - Google Earth: to visualize the areas of the benchmark using the kml files provided - Applied Imagery’s QT Reader (http://appliedimagery.com/download/) is a free tool for viewing 3D point clouds and other geospatial data. QT Reader supports LAS, LAZ, KML, and NITF files used for this challenge. - LizardTech’s GeoViewer (https://www.lizardtech.com/products/geoviewer/) is a free tool for viewing images. GeoViewer supports NITF files used for this challenge.