Data_Preparation.Rmd

```{r,message=FALSE,echo=FALSE}
  html <- TRUE
  library("knitcitations")
  library("knitr")
  cite_options(citation_format = "pandoc", max.names = 3, style = "html", hyperlink = "to.doc")
  bib <- read.bibtex("bibtexlib.bib")
  opts_chunk$set(tidy = FALSE, message = FALSE, warning = FALSE,
                 fig.width = 10, fig.height = 6, cache = TRUE)
  if (html) opts_chunk$set(out.width = "700px", dpi = 300)
  # use this to set knitr options:
  # http://yihui.name/knitr/options #chunk_options
```

---
title: 'Data Preparation'
subtitle: '*ZN Kamvar, SE Everhart and NJ Gr&uuml;nwald*'
---


This section will briefly go over the basics of data import into *poppr*. For
this section, we will focus on the GenAlEx format. Other formats are supported
and details are given in the R help page for the *adegenet* function
`import2genind`. We will show examples of haploid, diploid, and polyploid data
sets and show you how you can format your data if it's grouped into multiple
stratifications.

> [Appendix 1](Data_sets.html) provides an overview and description of the data
> sets used in this primer.

Importing data
------

### GenAlEx formatted data

To start simple, let's examine the input file for the `monpop` data set
containing 694 isolates of the plant pathogen *Monilinia fructicola* genotyped
over 13 haploid microsatellite loci `r citep(bib["everhart2014finescale"])`. The
data set is called **monpop.csv**.

><a class="btn btn-info btn-sm" target = "blank" href="https://github.com/grunwaldlab/Population_Genetics_in_R/blob/master/monpop.csv" role="button">view</a>
or
<a class="btn btn-primary btn-sm" target = "blank" href="http://grunwaldlab.github.io/Population_Genetics_in_R/monpop.csv" role="button"> download</a> monpop.csv

> Note that `*.csv` stands for comma-separated-value. This can be opened in a
> text editor or excel. `*.csv` is a good format for portability into R.

To import GenAlEx formatted data into *poppr*, you should use the function
`read.genalex`. Below is an example using **monpop.csv**.

```{r, eval = FALSE}
library("poppr")
monpop <- read.genalex("~/Downloads/monpop.csv")
monpop
```

```{r, echo = FALSE}
library("poppr")
data(monpop)
monpop
```

> There are other options for genalex formatted data, such as xy coordinates. We
> won't cover the other options, but type `help("read.genalex")` in your R
> console for details.

### Other data formats

Given dependence of *poppr* on *adegenet*, users can import data from the following formats

- **FSTAT** (file.dat)
- **GENETIX** (file.gtx)
- **GENEPOP** (file.gen)
- **STRUCTURE** (file.str)

The *adegenet* function `import2genind` will import all of these formats.
If you have sequence data, you can use the `read.FASTA` function from the *ape*
package. If your data is in any other format, type `help("df2genind")` for guidance.

GenAlEx data format
----

[GenAlEx](http://biology-assets.anu.edu.au/GenAlEx/Welcome.html) is a very
popular add-on for Microsoft Excel. It is relatively easy to use because of its
familiar, menu-driven interface. It also gives the user the option to include
information on population groupings, regional groupings, and xy coordinates. The
flexibility of this format made it a clear choice for import into *poppr*.

The data format is standard in that *individuals are defined in the rows* and
*loci are defined in the columns*. The first two rows are reserved for metadata
and the first two columns are reserved for the individual names and population
names, respectively. The examples we will be using include haploid, diploid and
polyploid data.

### Basic Format

Below is what the `monpop` (haploid) data looks like. Highlighted in red is
how missing data should be coded for SSR markers. Highlighted in blue are
the parts of the metadata rows used by *poppr*. These three numbers represent:

<table align="left" cellpadding=20 border=2>
<thead>
<tr>
<th class="col-xs-3">A</th>
<th class="col-xs-3">B</th>
<th class="col-xs-3">C</th>
</tr>
</thead>
<tbody>
<tr>
<td class="text-center">loci</td>
<td class="text-center">samples</td>
<td class="text-center">populations</td>
</tr>
</tbody>
</table>

![monpop](images/monpop.png)

The columns of the metadata beyond those three rows define the number of
individuals contained within each population. Since this data is redundant with
the second column, it is not necessary. Notice, also, that the second column,
reserved for the population assignments, has a pattern of underscores in the
populations. This will be important at the end of this section. Below is a
modified version of the input format that should make it easier to format.

![monpop_ideal](images/monpop_ideal.png)

Highlighted in blue is the cell that defines the number of columns highlighted
in red. If we set this number to `1`, then we do not have to enter in any
information in those columns. Try it for yourself.

### Diploids

Diploid data is only different in the fact that you will have two alleles
per locus. This is coded such that each allele is in a separate
column. Below is an example of the `nancycats` data set (from the *adegenet*
package), exported like above. Highlighted in blue and red are the first and
second loci, respectively.

![diploid](images/diploid.png)

### Polyploids

GenAlEx does not handle polyploids, but since poppr can do it, we have set up a
scheme to allow import of polyploids via this format. The limitation is that all
of your loci have to have the same observed ploidy. Below is the example of
*Phytophthora infestans* in the data set `Pinf` where some genotypes had
observed tetraploid loci `r citep(bib["goss2014irish"])`.

![tetraploid](images/tetraploid.png)

Highlighted in blue is the first locus and highlighted in red are two samples at
that locus, an observed diploid and observed triploid. Note the extra zeroes
needed to make the genotype tetraploid.


Population strata
----

A hierarchical sampling approach is necessary to infer structure of populations
in space or time. *Poppr* facilitates definition of stratified data by
concatenating the different stratifications into a single column by a common
separator ("\_" by default). Here's an example of the three stratifications
of the `monpop` data set introduced above:


<table border=2 cellpadding=20 align=center>
<tr> <th class="col-xs3">
Tree
</th> <th class="col-xs3">
Year
</th> <th class="col-xs3">
Symptom
</th>  </tr>
  <tr> <td align="center">
7
</td> <td align="center">
9
</td> <td align="center">
BB
</td> </tr>
  <tr> <td align="center">
7
</td> <td align="center">
9
</td> <td align="center">
BB
</td> </tr>
  <tr> <td align="center">
7
</td> <td align="center">
9
</td> <td align="center">
BB
</td> </tr>
  <tr> <td align="center">
7
</td> <td align="center">
9
</td> <td align="center">
BB
</td> </tr>
  <tr> <td align="center">
7
</td> <td align="center">
9
</td> <td align="center">
BB
</td> </tr>
  <tr> <td align="center">
7
</td> <td align="center">
9
</td> <td align="center">
BB
</td> </tr>
</table>

resulting in definition for use in *poppr* as:

<!-- html table generated in R 3.1.2 by xtable 1.7-4 package -->
<!-- Fri Jan  9 20:12:57 2015 -->
<table border=2 cellpadding=20 align=center>
<tr> <th class="col-xs3">
Tree_Year_Symptom
</th>  </tr>
  <tr> <td align="center">
7_9_BB
</td> </tr>
  <tr> <td align="center">
7_9_BB
</td> </tr>
  <tr> <td align="center">
7_9_BB
</td> </tr>
  <tr> <td align="center">
7_9_BB
</td> </tr>
  <tr> <td align="center">
7_9_BB
</td> </tr>
  <tr> <td align="center">
7_9_BB
</td> </tr>
</table>



> If you do not want to concatenate your population factors into one row,
save them as a separate csv file and use the functions `read.table` and `strata`
after you've imported your genetic data to achieve the same result.

The data imported this way can be accessed with the function `splitStrata`:

```{r}
# using monpop from above
monpop # Before (One level, Pop)
splitStrata(monpop) <- ~Tree/Year/Symptom
monpop # After (Three distinct levels)
```

This is detailed in [our chapter on population strata](Population_Strata.html).

References
----------

<!------>