Difference between revisions of "UAS Phenotyping and Aerial Image Acquisition Procedures"

Latest revision as of 02:10, 15 April 2017

Welcome to the Poland Lab sUAS Phenotyping Wiki page !

sUAS Data Acquisition & Processing Pipeline Overview

Figure 1a.: Small Unmanned Aerial Systems (sUAS) based data acquisition and processing pipeline. (Figure credit: Xu "Kevin" Wang, KSU)

❏ The better understanding of the biological processes requires quality phenotypic and genotypic information.

❏ Superior spatial and temporal resolution and throughput of the Small Unmanned Aerial Systems (sUAS) can be leveraged for precise plant phenotypic measurements.

❏ Along with the nucleotide sequence data, the precise phenotypic measurements would allow connecting the phenotype-to-genotype link, leading to better understanding of the biological mechanisms.

❏ To ensure proper repeatability and integrity of the collected observations, sUAS based standard operational protocols are developed.

❏ The following sections in this document should guide the readers through some of the key steps involved in the sUAS based data acquisition and processing.

Field-setup Guide

Ground Control Points

❏ Early in the crop growing season, distribute at least 12-15 ground control points (GCP) uniformly throughout the field. The idea is to get full coverage of the field terrain.

❏ Six to eight GCPs should be installed in the outside periphery and the rest can be distributed uniformly inside the field (Figure 2a)

Figure 2a: Example layout of the uniformly distributed Ground Control Points in a phenotyping field.

, preferably, in the alleyways, at the row-column intersections for better visibility . In case the field layout permits leaving a few blank plots, the GCPs can also be installed in the middle of those plots.

❏ Survey the GCPs with either RTK or a DGPS GNSS systems (whichever available) multiple times during the season to reduce the day/time bias. For best results, we strongly recommend recording multiple readings per GCP to get the precise positioning readings.

❏ Some GPS units allow recording of raw NMEA GNSS strings at high frequency (1-10Hz). Therefore, these units should be preferred, depending upon the availability. The averaged Latitude, Longitude, and Altitude (LLA) readings can then be used in the photogrammetry softwares to generate highly accurate orthomosaics and Digital Elevation Models or DEMs.

❏ It is highly important not to move the surveyed GCPs from their fixed positions throughout the crop season.

❏ Generally, the white color GCPs are easy to spot, hence, should be preferred. Some photogrammetry softwares support automatic GCP detection. When available, these software customized GCPs can be used. Here are a few examples of GCPs that have worked well for us:

❏ Metal sheets screwed on top of the iron rods (Figure 2b)

Figure 2b: Ground Control Point made by screwing metal sheet on top of a thin metal rod.

.

❏ White color tile (12" x 12") placed on top of the brick/paver base: White tiles and bricks can be purchased from any hardware store. A base with 2-2.5" thickness should be preferred so as to avoid any dust deposition on the white surface. (Figure 2c)

Figure 2c: White tile sitting on top of a concrete paver used as a Ground Control Point.

Reference Panels

❏ Place the reference calibration panel on a flat surface close to the first mission waypoint, at least 2-meter away from the plot.

Figure 2d: A foldable color reference panel (Design 1) with marked corners.

❏ It is very important to put the reference panels in the exact same orientation and position for each flight. This is to facilitate automatic processing of the panels based on coordinate positions. Small-size stakes/flags can be used to mark the corners of panels (Figure 2d). Survey the reference panel positions with a RTK or DGPS GNSS system.

❏ In order to minimize the variations due to changing sun illumination and overcast, maximum aerial shots of calibration panel at regular intervals are recommended. This can be achieved either by using multiple calibration targets or by capturing images of a target at every landing and takeoff event during the mission (for bigger fields where multiple flights are necessary).

After conducting several tests with different panel designs, we decided to use these two most portable panels for field based imaging data. Both the Designs 1 and 2 can be carried in the field easily due to their foldable design (Figure 2d-e).

Figure 2e: A foldable carry-on color reference panel (Design 2).

❏ A 15-20% gradient grayscale reference panel can be used as a reference panel. The details to build a reference panel are described below.

Materials required:

1. 7mm thick lightweight plywood.

2. Flat black and white water based color (1 liter each).

3. Wood primer (1/2 liter)

4. Paint mixing trays (8 units).

5. Brush rollers (8 units)

6. 100 ml liquid syringes with 10ml markings (2 units)

7. 5-7mm metal hinges with 2-3 holes (4 units)

8. Painters tape

Flight Checklists

Pre-Flight Guide – Before Leaving For Field

sUAS Accessory List

❏ Transmitter (Figure 3a)

Figure 3a: sUAV accesory for IRIS+ quadcoptor.

❏ Radio with USB cable

❏ UAV batteries and battery chargers

❏ Extra set of propellers/wings

❏ Camera(s) with fully charged batteries

❏ Multimeter (optional)

❏ Toolkit (wrench set, propeller hex-key, camera mount screws and screwdriver etc.) (Figure 3b)

Figure 3b: Tool keys, screws and screwdriver.

❏ Camera gimbal

❏ SD cards with CHDK installed (follow instructions in section 3.1)

❏ Color calibration panel

❏ Ground Control Points installed in field

Things to check before leaving lab

❏ An updated version of Mission Planner Software is installed on the field laptop.

❏ Has the field map been cached in the Mission Planner program? Details on map caching procedure are discussed below in section 3.3 (Note: This step is required only once per phenotyping field.)

❏ Does RC Transmitter have enough voltage? FrSky transmitter should have ≥ 9.4V charge.

❏ Are UAV batteries charged? 3-cell batteries should charge up to 12.6V.

❏ Is the laptop charged? Do you need/have a laptop charger?

❏ Ensure that you are familiar with the laws and regulations related to flying UAVs in your country/area. General laws in the United States require 400 feet maximum height and within line-of-sight flying.

❏ Make sure you have watched all the UAV manufacturer recommended tutorials before your first flight.

❏ When first flying your IRIS+, make sure to fly it away from populated areas and only in open space.

❏ Before the first flight, make sure to perform a compass calibration otherwise this could lead to a bad behavior in GPS enabled modes. You can perform the compass calibration through APM or Mission Planner programs following this youtube video[1].

At-field- Pre-Flight Checklist

Camera settings

Note: These settings are based on Canon PowerShot S100/S95/S110 Point & Shoot cameras.

→ General recommendations for aerial imagery:

❏ Keep the camera focal length fixed throughout the mission.

❏ Shoot in RAW since the RAW format retains maximum scene details and provides greater flexibility during post-processing.

❏ Use faster shutter speeds to avoid blur due to vehicle speed and vibration.

❏ Use gimbal to get maximum nadir shots of the scene and to avoid image blurring.

❏ Set the Manual Focus (MF) setting to infinity for >5 meter altitude flights.

→ Choose one of the following camera shooting modes:

1. Shutter Priority (Tv) mode:

❏ In Tv mode, only shutter speed needs to be adjusted, ISO and F-number are automatically adjusted by the camera based on scene conditions.

❏ Shutter speed: fastest or close to fastest (for S100 canon, 1/1250, 1/1600, or 1/2000 would work, depending on the requirements).

❏ Optional: Ev (exposure): 0, -1/3 or -2/3 (again adjust based on ambient light conditions)

2. Manual (M) mode:

❏ Shutter speed: fastest or close to fastest (for S100 canon, 1/1600 or 1/2000 would work, depending upon the UAV speed and weather conditions)

❏ ISO: ~80-250 (adjust based on ambient light). Hint: Higher ISO, brighter image but higher noise.

❏ Aperture (f-number): 4.0-5.6. Note: Small f-number means wider apertures and shallow depth of field. In our experience, F-number of 4-5.6 is a sweet spot. However, we recommend that you play around with the manual settings to setup your camera to suit your specific requirements.

❏ Ev (exposure): -1/3 or -2/3. Again, adjust based on ambient light conditions.

❏ Additional note: Use the vertical bar with moving pointer to adjust the camera settings to get proper exposure (not too bright and dark). The pointer should point at or below the mid-mark.

→ Under MENU option, change the default settings to:

❏ GPS ON

❏ MF-point ZOOM OFF

❏ Safety MF OFF

❏ Wind Filter OFF

❏ Image format: Camera raw (CR2)

sUAV In-field Pre-flight Checklist

❏ Place the UAV on a flat surface, carefully plug-in the battery and wait 10-seconds for it to finish performing initial sensor calibrations. DO NOT DISTURB/MOVE the coptor during that time (after plugging in the battery). Close the lid after the purple/blue light goes off.

❏ Leave the UAV on a flat surface for good 4-5 minutes while you set up the field plan and calibration panels.

❏ Perform necessary accelerometer and compass calibrations through Mission Planner (Initial Setup >> Mandatory hardware >> Accel/Compass). It is recommended to perform accelerometer calibration step every time: a) a field location/site is changed, b) involves a travel of more than 10-20 miles. It is important to understand that properly calibrated sensors ensure the best flight performance and help avoid unusual n-flight coptor behaviour.

❏ Lock the SD card and insert it in camera.

❏ Switch-on the camera by pressing the preview button.Half-press the shutter to let the lens out.

❏ Set the camera parameters by following the instructions given in section 3.2.1.

❏ Attach the camera to the gimbal mount and plug-in the wire.

❏ Load and run CHDK script by half-pressing the camera shutter button.

❏ Open Mission Planner program on field laptop and connect the UAV to the program. Select an appropriate COM PORT and 57600 Baud rate from the drop down menu on top right corner of the program. Hit connect.

❏ Check flight parameters (see parameter section below for detailed list)

❏ Load proper mission waypoint (WP) file and write onto the Autopilot through the radio link. Note: You can read back the WP from UAV to double check the mission plan is written properly.

❏ Make sure the GPS signal is good (3D fix: number of satellites > 7 and hdop < 2) and there is no error message flashing on the mission planner main window.

❏ Check if the camera trigger is working properly by walking the UAV (if necessary). Note: Make sure the UAV safety light is flashing to avoid accidental arming.

❏ Check battery voltage.

❏ Turn the radio controller (RC) transmitter on.

❏ Check to make sure the RC switch positions are set to:

❏ Stabilized Mode [ON], RTL [OFF], Throttle [All the way Down towards the pilot].

❏ Depending upon how the other RC switches are mapped, make sure the switches are set to a mode that is safe for take-off.

❏ Check the trims are zero (offsets to stick positions that can be set with small slide switches). Pushing trim buttons does a tik..tik..tik sound !

❏ Make sure no one is close to UAV and survey area.

In Field After Arming Checklist

❏ Arm the UAV in Stabilized mode (STB) and take off in AUTO mode.

❏ After take off, adjust the yaw position with controller stick if necessary.

❏ Keep a close eye on UAV during first 3-4 WPs to make sure mission plan is being followed correctly.

❏ At the end of the mission, during landing (when UAV is ~5-6 inches above ground), swiftly flip the flight mode switch to stabilized mode while making sure throttle is all the way down. Note: The landing will get better with regular practice.

❏ Try this trick for large experimental fields that require changing batteries on regular intervals:

❏ As soon as the battery hits ~10.80-10.85V mark and UAV is at the most closest waypoint to the take-off position, hit the RTL (return to launch) button on the controller.

❏ Make a note of the last WP the UAV was about to head to. Let the UAV land safely on the launch pad and un-arm it.

❏ Disconnect the mission planner program.

❏ Press safety button on UAV.

❏ Stop the CHDK camera script by half-pressing the shutter button.

❏ Replace the battery. Connect the Mission Planner program.

❏ UAV takes off again in AUTO but this time as soon as the UAV hits Do_Change_Speed Waypoint, on Mission Planner, go to ‘ACTIONS’ menu and choose the last UAV WP from step 2, and hit SET WP. Now the UAV will jump to the selected WP. (Note: Always let the UAV hit Do_Change_Speed and Do_Set_Cam_Trigger_Distance WP before pushing the SET button.)

Post-Flight Checklist

❏ Unarm the UAV right after landing.

❏ Disconnect the battery to end the current log.

❏ Turn off RC transmitter if it’s the last flight.

❏ Place used battery upside down in UAV case to distinguish the charged and discharged batteries. Use the voltmeter in IRIS box to check the battery status.

❏ Fill out flight log. Kevin’s JAVA program. [optional]

❏ Download log(s) from UAV (see instructions below)

❏ Download CR2 files from camera to your computer or hardware.

How-to Guide

CHDK Installation on SD Card

❏ Low-level format the SD card in the camera.

❏ Check to find out the correct camera firmware version using Acid App on Windows 8 and onwards.

❏ Download and run EOScard utility program to make the SD card bootable (the link is in the references section).

❏ Download and unzip an appropriate CHDK version for your camera firmware.

❏ Copy the unzipped CHDK files into the root (main) folder of your SD card. The Other relevant scripts such as 'NEG_APM.bas' should be copied to the scripts folder inside the CHDK. NEG_APM.bas script can be accessed here.

❏ Lock your SD card and put it back into the camera.

❏ Fire-up the camera using playback button and go to CHDK Settings → Remote Parameters and check Enable Remote option. Remote parameters are enabled to facilitate the camera trigger using the UAS autopilot.

❏ Select an appropriate script for your UAS operation.

Here is the official instruction page for CHDK installation.

Ground Station Program and Driver Installation

Ground Map Caching on Mission Planner

Mission Planning

Downloading Data Flash Logs

Preparing Data For Archiving

Ground Truth and Agronomic Notes (Optional)

Plant Height

NDVI with a spectroradiometer or GreenSeeker

Other agronomic or physiological notes

References and other relevant links

1. In case of questions or queries, contact me at singhdj2[at]ksu[dot]edu
2. Mission Planner download [2]
3. Canon CHDK installation instructions [3]
4. Download link for Windows 7zip [4] and fciv[5] tools.
5. QGIS software [6]
6. Canon Digital Photo Professional (DPP) download v3.14.xx [7]
7. Canon S100 firmware [8]