View source for UAS Phenotyping and Aerial Image Acquisition Procedures

'''Welcome to the Poland Lab UAS Phenotyping Wiki page''' !

== '''UAS Data Acquisition & Processing Pipeline Overview''' ==

[[File:UAS_Pipeline_overview_Kevin.png|right|thumb|500px|'''Figure 1a.''': Unmanned Aerial Systems (UAS) data acquisition and processing pipeline.]]

❏ The better understanding of the biological processes requires quality phenotypic and genotypic information. 

❏ Superior spatial and temporal resolution and throughput of the Small Unmanned Aerial Systems (sUAS) can be leveraged for precise plant phenotypic measurements.  

❏ Along with the nucleotide sequence data, the precise phenotypic measurements would allow connecting the phenotype-to-genotype link, leading to better understanding of the biological mechanisms. 

❏ To ensure proper repeatability and integrity of the collected observations, sUAS based standard operational protocols are required. 

❏ The following sections in this document should guide the readers through some of the key steps involved in the UAS based data acquisition and processing.

== '''Field-setup Guide''' ==

=== Ground Control Points ===

::: '''❏''' Early in the crop growing season, distribute at least 12-15 ground control points (GCP) uniformly throughout the field. 

::: '''❏''' Six to eight GCPs should be installed in the outside periphery and the rest can be distributed uniformly inside the field, preferably, in the alleyways, at the row-column intersections for better visibility ('''Figure 1a''') [[File:GCP_layout.png|250px|thumb|'''Figure 2a''': Example layout of the uniformly distributed Ground Control Points in a phenotyping field.]].  In case the field layout permits leaving a few blank plots, the GCPs can also be installed in the middle of those plots. 

::: '''❏''' Survey the GCPs with either RTK or a DGPS GNSS systems (whichever available) multiple times during the season to reduce the day/time bias. For best results, we strongly recommend recording multiple readings per GCP to get the precise positioning values. 

::: '''❏''' Some GPS units allow recording of raw NMEA GNSS strings at high frequency (1-10Hz). Therefore, these units should be preferred, depending upon the availability. The averaged Latitude, Longitude, and Altitude readings can then be used in the photogrammetry softwares to generate highly accurate orthomosaics and Digital Elevation Models or DEMs. 

::: '''❏''' It is highly important '''<u>not to move</u>''' the surveyed GCPs from their fixed positions throughout the crop season.  

::: '''❏''' Generally, the white color GCPs are easy to spot, hence, should be preferred.  Some photogrammetry softwares support automatic GCP detection. When available, these software customized GCPs can be used.  Here are a few examples of GCPs that have worked well for us:

:::: '''❏''' Metal sheets screwed on top of the iron rods ('''Figure 2b''')[[File:Metal_GCP_india.jpeg|250px|thumb|'''Figure 2b''': Ground Control Point made by screwing metal sheet on top of a thin metal rod.]].

:::: '''❏''' White color tile (12" x 12") placed on top of the brick/paver base:  White tiles and bricks can be purchased from any hardware store. A base with 2-2.5" thickness should be preferred so as to avoid any dust deposition on the white surface. ('''Figure 2c''') [[File:GCP_brick_1.jpg|250px|thumb|'''Figure 2c''': White tile sitting on top of a concrete paver used as a Ground Control Point.]]

=== Reference Panels === 

::: '''❏''' Place the reference calibration panel on a flat surface close to the first mission waypoint, at least 2-meter away from the plot. 
[[File:Folding_color_panel_Daljit_Singh.jpg|250px|thumb|'''Figure 2d''': A foldable color reference panel (Design '''1''') with marked corners.]]
::: '''❏''' It is very important to put the reference panels in the exact same orientation and position for each flight. This is to facilitate automatic processing of the panels based on coordinate positions. Small-size stakes/flags can be used to mark the corners of panels ('''Figure 2d''').

::: '''❏''' In order to minimize the variations due to changing sun illumination and overcast, maximum aerial shots of calibration panel at regular intervals are recommended. This can be achieved either by using multiple calibration targets or by capturing images of a target at every landing and takeoff event during the mission (for bigger fields where multiple flights are necessary). 

::: After conducting several tests with different panel designs, we decided to use these two most portable panels for field based imaging data. Both the Designs 1 and 2 can be carried in the field easily due to their foldable design ('''Figure 2d-e'''). [[File:New_color_panel_box_04292016.jpeg|250px|thumb|'''Figure 1e''': A foldable carry-on color reference panel (Design '''2''').]]

::: '''❏''' A 15-20% gradient grayscale reference panel can be used as a reference panel. The details to build a reference panel are described below.

::: '''<u>Materials required</u>''': 
:::: '''1.''' 7mm thick lightweight plywood. 
:::: '''2.''' Flat black and white water based color (1 liter each).
:::: '''3.''' Wood primer (1/2 liter)
:::: '''4.''' Paint mixing trays (8 units).
:::: '''5.''' Brush rollers (8 units)
:::: '''6.''' 100 ml liquid syringes with 10ml markings (2 units)
:::: '''7.''' 5-7mm metal hinges with 2-3 holes (4 units)
:::: '''8.''' Painters tape

==  <u>'''Flight Checklists'''</u> ==

=== <u>Pre-Flight Guide – Before Leaving For Field</u> ===
==== UAS Accessory List ====
:: ❏ Transmitter ('''Figure 3a''')[[File:UAV_accessory_IRIS%2B_DSingh.jpg|250px|thumb|'''Figure 3a''': UAV accesory for IRIS+ quadcoptor.]]
:: ❏ Radio with USB cable
:: ❏ UAV batteries and battery chargers
:: ❏ Extra set of propellers/wings
:: ❏ Camera(s) with fully charged batteries
:: ❏ Multimeter (optional)
:: ❏ Toolkit (wrench set, propeller hex-key, camera mount screws and screwdriver etc.) ('''Figure 3b''') [[File:Hex-key_screw_keys.jpg|250px|thumb|'''Figure 3b''': Tool keys, screws and screwdriver.]]
:: ❏ Camera gimbal
:: ❏ SD cards with CHDK installed
:: ❏ Color calibration panel
:: ❏ Ground Control Points installed in field

==== Things to check before leaving lab ====

:: ❏ An updated version of Mission Planner Software is installed on the field laptop. 
:: ❏ Has the field map been cached in the Mission Planner program? Details on map caching procedure are discussed below in section 3.3 (Note: This step is required only once per phenotyping field.)
:: ❏ Does RC Transmitter have enough voltage?  FrSky transmitter should have ≥ 9.4V charge.  
:: ❏ Are UAV batteries charged? 3-cell batteries should charge up to 12.6V. 
:: ❏ Is the laptop charged? Do you need/have a laptop charger? 
:: ❏ Ensure that you are familiar with the laws and regulations related to flying UAVs in your country/area. General laws in the United States require 400 feet maximum height and within line-of-sight flying. 
:: ❏ Make sure you have watched all the UAV manufacturer recommended tutorials before your first flight.  
:: ❏ When first flying your IRIS+, make sure to fly it away from populated areas and only in open space.  
:: ❏ Before the first flight, make sure to perform a compass calibration otherwise this could lead to a bad behavior in GPS enabled modes. You can perform the compass calibration through APM or Mission Planner programs following this '''youtube video'''[https://www.youtube.com/watch?v=oD9Z9IR2TNU].

=== <u> At-field- Pre-Flight Checklist </u> ===

==== Camera settings ====

Note: These settings are based on Canon PowerShot S100/S95/S110 Point & Shoot cameras.

: → '''General recommendations for aerial imagery''':

:: ❏ Keep the camera focal length fixed throughout the mission.
:: ❏ Shoot in RAW since the RAW format retains maximum scene details and provides greater flexibility during post-processing. 
:: ❏ Use faster shutter speeds to avoid blur due to vehicle speed and vibration.  
:: ❏ Use gimbal to get maximum nadir shots of the scene and to avoid image blurring.
:: ❏ Set the Manual Focus (MF) setting to infinity for >5 meter altitude flights. 

: → '''Choose one of the following camera shooting modes''':

:: '''1. Shutter Priority (Tv) mode''': 

::: ❏  In Tv mode, only shutter speed needs to be adjusted, ISO and F-number are automatically adjusted by the camera based on scene conditions.
::: ❏  Shutter speed: fastest or close to fastest (for S100 canon, 1/1250, 1/1600, or 1/2000 would work, depending on the requirements).
::: ❏  Optional: Ev (exposure): 0, -1/3 or -2/3 (again adjust based on ambient light conditions)

:: '''2. Manual (M) mode''':

::: ❏  Shutter speed: fastest or close to fastest (for S100 canon, 1/1600 or 1/2000 would work, depending upon the UAV speed and weather conditions)
::: ❏  ISO: ~80-250 (adjust based on ambient light). Hint: Higher ISO, brighter image but higher noise.
::: ❏  Aperture (f-number): 4.0-5.6. Note: Small f-number means wider apertures and shallow depth of field. In our experience, F-number of 4-5.6 is a sweet spot. It is recommended that you play around with the manual settings to setup your camera to suit your specific requirements. 
::: ❏  Ev (exposure): -1/3 or -2/3. Again, adjust based on ambient light conditions.
::: ❏  Additional note: Use the vertical bar with moving pointer to adjust the camera settings to get proper exposure (not too bright and dark). The pointer should point at or below the mid-mark. 

: →  '''Under MENU option, change the default settings to''':

:: ❏  GPS '''ON''' 
:: ❏  MF-point ZOOM '''OFF'''
:: ❏  Safety '''MF OFF'''
:: ❏  Wind Filter '''OFF'''
:: ❏ Image format: Camera raw ('''CR2''')

=== <u>UAS In-field Pre-flight Checklist</u> ===
          
: ❏  Place the UAV on a flat surface, plug-in the battery and close the lid. Note: Avoid any disturbance to the UAV during initial few seconds as the UAV performs automatic sensor-calibrations.

: ❏ Lock the SD card and insert it in camera.

: ❏ Switch-on the camera by pressing the preview button.Half-press the shutter to let the lens out. 

: ❏ Set the camera parameters by following the instructions given in '''section 3.2.1'''. 
: ❏ Attach the camera to the gimbal mount and plug-in the wire. 
: ❏ Load and run CHDK script by half-pressing the camera shutter button. 
: ❏ Open mission planner on field laptop and connect mission planner to UAV Autopilot using telemetry radio link. Use a correct COM PORT Number and 57600 Baud rate from the drop down menu on   top right corner of mission planner software. Then hit connect. 
: ❏ Check flight parameters (see parameter section below for detailed parameter list)
: ❏ Load proper mission waypoint file and write the waypoints into the Autopilot through radio link. Note: You can read back the WP from UAV to double check the correct WayPoints are written to the UAV.
: ❏ Make sure the GPS signal is good (3D fix: number of satellites > 7 and hdop < 2) and there is no error message flashing on the mission planner main window.
: ❏ Check if the camera trigger is working properly by walking the UAV. Note: Make sure the UAV safety light is flashing to avoid accidental arming.
: ❏ Check battery voltage.
: ❏ Turn the radio controller (RC) transmitter on.
: ❏ Check to make sure the RC switch positions are set to: 
: ❏ Stabilized Mode [ON], RTL [OFF], Throttle [All the way Down].
: ❏ Depending upon how the other RC switches are mapped, make sure the switches are set to a mode that is safe for take-off.
: ❏ Check the trims are zero (offsets to stick positions that can be set with small slide switches).
: ❏ Make sure no one is close to UAV and survey area.

=== In Field After Arming Checklist ===

# Arm the UAV in Stabilized mode ('''STB''') and take off in '''AUTO''' mode.
# After take off, adjust the yaw position with controller stick if necessary.
# Keep a close eye on UAV during first 3-4 WPs to make sure mission plan is being followed correctly.
# At the end of the mission, during landing (when UAV is ~5-6 inches above ground), swiftly flip the flight mode switch to stabilized mode while making sure throttle is all the way down.  Note: The landing will gets better with regular practice.
# Try this trick for bigger experimental fields that require changing batteries on regular intervals:
## As soon as the battery hits ~10.80-10.85V mark and UAV is at the most closest waypoint to the take-off position, hit the RTL (return to launch) button on  the controller.
## Make a note of the last WP the UAV was about to head to. Let the UAV land safely on the launch pad and un-arm it.
## Disconnect the mission planner program. 
## Press safety button on UAV.  
## Stop the CHDK camera script by half-pressing the shutter button. 
## Replace the battery. Connect the Mission Planner program. 
## UAV takes off again in '''AUTO''' but this time as soon as the UAV hits '''Do_Change_Speed''' Waypoint, on Mission Planner, go to ‘'''ACTIONS'''’ menu and choose the last UAV WP from step 2, and hit '''SET WP'''. Now the UAV will jump to the selected WP. (Note: Always let the UAV hit '''Do_Change_Speed''' and '''Do_Set_Cam_Trigger_Distance''' WP before pushing the '''SET''' button.)

=== Post-Flight Checklist ===

==  '''How-to Guide''' ==

=== CHDK Installation on SD Card ===

# Low-level format the SD card in the camera.
# Check to find out the correct camera firmware version using Acid App on Windows 8 and onwards.
# Download and run EOScard utility program to make the SD card bootable (the link is in the references section).
# Download and unzip an appropriate CHDK version for your camera firmware.
# Copy the unzipped CHDK files into the root (main) folder of your SD card. The Other relevant scripts such as 'NEG_APM.bas' should be copied to the scripts folder inside the CHDK. NEG_APM.bas script can be accessed here. 
# Lock your SD card and put it back into the camera. 
# Fire-up the camera using playback button and go to '''CHDK Settings''' → '''Remote Parameters''' and check '''Enable Remote''' option. Remote parameters are enabled to facilitate the camera trigger using the UAS autopilot.
# Select an appropriate script for your UAS operation.

Here is the official instruction page for [http://chdk.wikia.com/wiki/Prepare_your_SD_card CHDK] installation.

=== Ground Station Program and Driver Installation ===

=== Ground Map Caching on Mission Planner ===
=== Mission Planning ===
=== Downloading Data Flash Logs ===
=== Preparing Data For Archiving ===

==  '''Ground Truth and Agronomic Notes (Optional)''' ==

=== Plant Height ===
=== NDVI with a spectroradiometer or GreenSeeker ===
=== Ohter agronomics or physiological notes ===

== '''References and other relevant links''' ==

# Mission Planner download [http://ardupilot.com/downloads/?did=82] 
# Canon CHDK installation instructions [http://chdk.wikia.com/wiki/Prepare_your_SD_card] 
# Download link for Windows <u>''7zip''</u> [http://www.7-zip.org/download.html] and ''fciv''[http://www.microsoft.com/en-us/download/details.aspx?id=11533] tools.
# QGIS software [https://www.qgis.org/en/site/forusers/download.html]
# Canon Digital Photo Professional (DPP) download v3.14.xx [https://www.usa.canon.com/internet/portal/us/home/support/details/cameras/support-point-and-shoot/powershot-s-series/powershot-s100?tab=drivers#Z7_MQH8HIC0L88RB0AMD0F1Q42K25]
# Canon S100 firmware [https://www.usa.canon.com/internet/portal/us/home/support/details/cameras/support-point-and-shoot/powershot-s-series/powershot-s100?tab=drivers#Z7_MQH8HIC0L88RB0AMD0F1Q42K25]