Difference between revisions of "Poland Lab Wiki"

From Poland Lab Wiki
Jump to: navigation, search
(Standard Operating Procedures)
(Database)
 
(115 intermediate revisions by 9 users not shown)
Line 1: Line 1:
 
<strong>Welcome to the Poland Lab Wiki!</strong>
 
<strong>Welcome to the Poland Lab Wiki!</strong>
  
== [Protocols] ==
+
== General Lab Information ==
  
 +
[[Lab Procedures]]
  
 +
[[Reading List]]
  
== Standard Operating Procedures ==
+
[[Travel Instructions]]
===UAS Data Management Operating Procedure===
+
  
====Field Computer Operating Procedure====
+
== Projects==
  
1. Create 'flight' folder for the data collection run
+
[[NSF BREAD]]
a. Execute the python program create_flight_directory_v03.py from a command window.
+
  
usage: create_flight_directory.py [-h] [-l LOG] [-f FLIGHT]
+
== Genotyping Procedures ==
  
optional arguments:
+
Lab protocols
-h, --help            show this help message and exit
+
-l LOG, --log LOG    The full path to apm log file.
+
-f FLIGHT, --flight  The full path to the directory where     the flight folder is to be created.
+
  
Example Command Line Invocation MAC OS X: ./create_flight_directory_v03.py -l /Users/mlucas/Desktop/"2015-01-19 09-08-54.log" -f /Users/mlucas/Desktop/
+
[[Media:GBS_NGS_Service_Order_Quebec.pdf|NGS Ordering]]
  
Example Command Line Invocation Windows: create_flight_directory_v03.py -l C:\Users\mlucas\Desktop\"2015-01-19 09-08-54.log" -f C:\Users\mlucas\Desktop\
+
[[GBS Data Management]]
  
 +
== High-Throughput Phenotyping Procedures ==
  
Note that since the log file has a space in it, it needs to be enclosed in quotes. Alternatively, you could rename the log files to replace the space with a hyphen '-' to avoid having to use quotes.
+
[[UAS Phenotyping and Aerial Image Acquisition Procedures]]
  
Note also that all folder paths have to have '/' on the end to make sure that they reference the correct folder.
+
[[UAS Phenotyping and Aerial Image Acquisition Procedure for DJI]]
  
Example Folder Name: uas_20150119_091005_020150119
+
[[Media:UAV_Data_Upload_SOP_v1.3.pdf|UAS Data Management Procedures]]
  
 +
[[UAS HTP Data Analysis]]
  
2. Store all data for a flight in the 'flight' folder:
+
[[Media:CAMERAS_README.txt|Cameras README]]
a. 1 operator log file e.g. DroneSettingsLog_2015-01-19_09-05-18.txt
+
b. 1 autopilot log file e.g. 2015-01-19-09-08-54.log
+
c. 1 telemetry log file e.g. 2015-01-19-09-03-03.tlog
+
d. All image files e.g. *.CR2. N.B. There is no need to put these in a separate sub-folder. Just copy all images to the same level as the other log files.
+
  
3. tar the 'flight' folder. If on Window 7zip will do this.
+
== Research Methods ==
a. filename e.g. uas_20150119_091005_020150119.tar
+
  
4. Compute MD5 checksum for tar file. If on Windows the Microsoft utility fciv.exe will do this.
+
[[Genomic Selection]]
  
5. Store checksum for the tar file in text file that contains the same 'flight ID' as the tarfile with '_md5.txt' appended.
+
[[Meta-GWAS]]
  
      Example using fciv utility:
+
== Database ==
  
fciv  uas_20150119_091005_020150119.tar > uas_20150119_091005_020150119_md5.txt
+
[[Connecting to the Database]]
  
This will output the md5 checksum to a text file. Edit the text file to remove all text except for the line with the checksum:
+
[[Media:Wheatgenetics_Quick_Start_Guide_ML_20171016.pdf|Wheatgenetics Database Quick Start Guide]]
  
427b87f05c72a0d7aea0cb2d8582aa2b uas_20150119_091005_020150119.tar
+
[[Media:DatabaseQueryFromR.pdf|Connecting to the Database with R]]
  
 +
[[Genotyping Database]]
  
6. Transmit data to server 'uas_staging' folder: /homes/jpoland/images/staging/uas_staging
+
[[Land-based High-Throughput Phenotyping Database]]
a. Copy the tar file and the md5.txt file to the server together.
+
b. Ideally via sftp if not too large. Otherwise, store a copy on external hard drive and return hard drive when all runs for the experiment have been completed.
+
c. N.B. Keep a backup copy of data for each run on the field laptop!
+
+
====Server Operating Procedure====
+
  
7. Navigate to the 'incoming' folder where new run data is staged.
+
[[UAV High-Throughput Phenotyping Database]]
  
8. Compute checksum for the 'flight ' data tar file and verify that it matches the value in the checksum file.
+
[[Manual Phenotyping Database]]
  
9. Untar the 'flight' data folder.
+
[[Seed and Plant Database]]
  
10. Post-process the flight data:
+
[[CIMMYT Wheat Breeding Database]]
a. Rename each image file to a unique file name per image
+
b. Generate a geo-referenced metadata file for the full set of images in the run (one row per image)
+
i. Include a checksum for each image in the metadata
+
ii. N.B. The program to create the metadata could load the database table directly without having to create an intermediate metadata file for import.
+
  
11. Perform a sanity check on geo-referenced image file to make sure that it does not contain any spurious data
+
[[Media:CIMMYT_Automation_File_Interfaces.pdf|CIMMYT Database Import File Interface Specifications]]
a. N.B. The software for doing this is TBD. It needs to be able to read the position of every image and generate a visual plot of the flight path.
+
  
12. Import geo-referenced image metadata file into database 'uas_images' (new) table.
+
== Scripts ==
a. N.B. If database is loaded by geo-referencing program directly this step is not required.
+
  
13. Cleanup the run folder after verifying successful completion of post-processing steps
+
[[Tassel 5 GBS v2 Pipeline sample script for calling SNPs using reference genome]]
a. Delete the geo-referenced metadata file
+
  
14. Tar the 'flight' data file folder containing the renamed image files and original log files.
+
== Equipment ==
  
15. Compute checksum for image folder tar file.
+
The links below provide access to the user manuals for the Toshiba Stake Printer and the Zebra Label Printer.  
  
16. Store a record for the run into database 'uas_run' (new) table that contains at least the flight ID, checksum and notes columns.  
+
If you encounter problems while using the printers please refer to the Troubleshooting section in the relevant user manual.
  
17. Move the  'flight' folder tar file containing the images and log files to production directory.
+
[[Media:B-SX5T-R_Owners-Manual_EN.pdf|Toshiba Stake Printer User Manual]]
  
18. Backup the flight data to NAS.
+
[[Media:zt200series_userguide.pdf|Zebra Printer User Manual]]
 +
 
 +
== Guides ==
 +
 
 +
[[ How to Submit GBS Data to the NCBI Sequence Read Archive (SRA) ]]

Latest revision as of 20:04, 13 May 2020

Welcome to the Poland Lab Wiki!

General Lab Information

Lab Procedures

Reading List

Travel Instructions

Projects

NSF BREAD

Genotyping Procedures

Lab protocols

NGS Ordering

GBS Data Management

High-Throughput Phenotyping Procedures

UAS Phenotyping and Aerial Image Acquisition Procedures

UAS Phenotyping and Aerial Image Acquisition Procedure for DJI

UAS Data Management Procedures

UAS HTP Data Analysis

Cameras README

Research Methods

Genomic Selection

Meta-GWAS

Database

Connecting to the Database

Wheatgenetics Database Quick Start Guide

Connecting to the Database with R

Genotyping Database

Land-based High-Throughput Phenotyping Database

UAV High-Throughput Phenotyping Database

Manual Phenotyping Database

Seed and Plant Database

CIMMYT Wheat Breeding Database

CIMMYT Database Import File Interface Specifications

Scripts

Tassel 5 GBS v2 Pipeline sample script for calling SNPs using reference genome

Equipment

The links below provide access to the user manuals for the Toshiba Stake Printer and the Zebra Label Printer.

If you encounter problems while using the printers please refer to the Troubleshooting section in the relevant user manual.

Toshiba Stake Printer User Manual

Zebra Printer User Manual

Guides

How to Submit GBS Data to the NCBI Sequence Read Archive (SRA)