use DataFrames 0.11 and Missings, drop Gadfly and RCall

related:
- plots and sexp migrated to PhyloPlots: related files removed
- major overhaul of DataFrames: use Missing, drop NullableArrays
- use StatsModels and CSV: which were previously part of DataFrames
  readtable -> CSV.read

not updated yet: documentation, barely started

broken tests: in test/test_lm_tree.jl, search @test_skip

Author: cecileane

.travis.yml

@@ -11,24 +11,15 @@ notifications:
   email: false
 
 before_install:
-  # linux
-  - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9; fi
-  - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo add-apt-repository -y "deb http://cran.rstudio.com/bin/linux/ubuntu $(lsb_release -s -c)/"; fi
-  - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo apt-get update -qq -y; fi
-  - if [ "$TRAVIS_OS_NAME" == "linux" ]; then sudo apt-get install git r-base r-base-dev r-recommended -y; fi
-
-  # osx
-  # faster than using homebrew/science tap
-  # but no permalink to release download
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then wget "https://cran.rstudio.com/bin/macosx/$(wget -qO- https://cran.rstudio.com/bin/macosx/ | sed -n 's/.*href="\(R-[^"]*.pkg\)".*/\1/p' | head -n 1)"; fi
-  - if [ "$TRAVIS_OS_NAME" == "osx" ]; then sudo installer -pkg R-*.pkg -target /; fi
 
   # Track master of NLopt. This line should be removed when a a new release of NLopt is out.
   - if [ "$TRAVIS_OS_NAME" == "osx" ]; then julia -e 'Pkg.clone("https://github.com/JuliaOpt/NLopt.jl"); Pkg.build("NLopt")'; fi
 
 after_success:
-  # Build doc
+  # push coverage results to Coveralls
+  - julia -e 'cd(Pkg.dir("PhyloNetworks")); Pkg.add("Coverage"); using Coverage; Coveralls.submit(Coveralls.process_folder())'
+  # push coverage results to Codecov
+  - julia -e 'cd(Pkg.dir("PhyloNetworks")); Pkg.add("Coverage"); using Coverage; Codecov.submit(Codecov.process_folder())'
+  # build doc
   - chmod +x docs/make.sh
   - ./docs/make.sh
-  # Code coverage
-  - julia -e 'cd(Pkg.dir("PhyloNetworks")); Pkg.add("Coverage"); using Coverage; Codecov.submit(process_folder())'

LICENSE.md

@@ -3,10 +3,11 @@
 [![Build Status](https://travis-ci.org/crsl4/PhyloNetworks.jl.svg)](https://travis-ci.org/crsl4/PhyloNetworks.jl)
 [![](https://img.shields.io/badge/docs-stable-blue.svg)](https://crsl4.github.io/PhyloNetworks.jl/stable)
 [![](https://img.shields.io/badge/docs-latest-blue.svg)](https://crsl4.github.io/PhyloNetworks.jl/latest)
-[![codecov](https://codecov.io/gh/crsl4/PhyloNetworks.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/crsl4/PhyloNetworks.jl)
+[![codecov.io](http://codecov.io/github/crsl4/PhyloNetworks.jl/coverage.svg?branch=master)](http://codecov.io/github/crsl4/PhyloNetworks.jl?branch=master)
 <!--
-[![Coverage Status](https://coveralls.io/repos/crsl4/PhyloNetworks/badge.svg?branch=master&service=github)](https://coveralls.io/github/crsl4/PhyloNetworks?branch=master)
+[![codecov](https://codecov.io/gh/crsl4/PhyloNetworks.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/crsl4/PhyloNetworks.jl)
 -->
+[![Coverage Status](https://coveralls.io/repos/crsl4/PhyloNetworks.jl/badge.svg?branch=master&service=github)](https://coveralls.io/github/crsl4/PhyloNetworks?branch=master)
 
 ## Overview
 
@@ -15,7 +16,6 @@ PhyloNetworks is a [Julia](http://julialang.org) package with utilities to:
   in (extended) Newick format.
   Networks are considered explicit: nodes represent ancestral species.
   They can be rooted or unrooted.
-- plot networks (and trees)
 - manipulate networks: re-root, prune taxa, remove hybrid edges,
   extract the major tree from a network, extract displayed networks / trees
 - compare networks / trees with dissimilarity measures
@@ -25,6 +25,8 @@ PhyloNetworks is a [Julia](http://julialang.org) package with utilities to:
 - estimate species networks from multilocus data (see below)
 - phylogenetic comparative methods for continuous trait evolution
   on species networks / trees
+- plot networks (and trees), via the companion package
+  [PhyloPlots](https://github.com/cecileane/PhyloPlots.jl)
 
 To get help, check
 

README.md

@@ -1,15 +1,13 @@
 julia 0.6
 StatsBase 0.18
-NullableArrays 0.1
-DataFrames 0.9
+DataFrames 0.11 # requires and re-exports Missings
 DataStructures 0.5.2
 NLopt 0.3
-ColorTypes 0.2
-Gadfly 0.4
-GLM 0.8.1
+GLM 0.10 # requires StatsModels, Distributions
+CSV 0.2.2
 Combinatorics 0.2.1
-RCall 0.6.1
-StatsFuns 0.4.0 #already required by GLM
-IterTools 0.2.1 # possibly 0.1.0
+StatsFuns 0.4.0 # already required by GLM
+IterTools 0.2.1
 BioSequences 0.8.1
 BioSymbols 1.2.0
+StaticArrays 0.7.0

REQUIRE

@@ -3,9 +3,8 @@ repo_url:         https://github.com/crsl4/PhyloNetworks.jl
 site_description: PhyloNetworks is a Julia package for the manipulation, visualization and inference of phylogenetic networks.
 site_author:      Claudia Solís-Lemus
 
-theme: material
-
-extra:
+theme:
+  name: 'material'
   palette:
     primary: 'teal'
     accent:  'teal'

docs/mkdocs.yml

@@ -30,7 +30,7 @@ Pages = [
     "man/multiplealleles.md",
     "man/trait_tree.md"
 ]
-Depth = 2
+Depth = 3
 ```
 
 ## Library Outline

docs/src/index.md

@@ -10,8 +10,10 @@ For a quick & basic tutorial on Julia, see
 
 Editors:
 
+- [Visual Studio Code](https://code.visualstudio.com) provides an editor
+  and an integrated development environment (IDE) for Julia: highly recommended!
 - [Juno](http://junolab.org) provides an IDE for Julia,
-  based on the (fantastic) [Atom](http://atom.io/) editor.
+  based on the [Atom](http://atom.io/) editor.
 - you can also run Julia within a [Jupyter](http://jupyter.org) notebook
   (formerly IPython notebook).
 
@@ -20,9 +22,6 @@ first time you run a function, it will be compiled at that moment. So,
 please be patient! Future calls to the function will be much much
 faster. Trying out toy examples for the first calls is a good idea.
 
-Users should have Julia 0.4 or above.
-
-
 ## Installation of the package PhyloNetworks
 
 To install the package, type inside Julia:
@@ -59,9 +58,22 @@ users can do the following through the terminal:
 where HOME is replaced by your home directory.
 -->
 
-The PhyloNetworks package has dependencies like NLopt and Gadfly
+The PhyloNetworks package has dependencies like NLopt and
+[DataFrames](http://juliadata.github.io/DataFrames.jl/stable/)
 (see the REQUIRE file for the full list), but everything is installed automatically.
 
+The companion package [PhyloPlots](https://github.com/cecileane/PhyloPlots.jl)
+has utilities to visualize networks, and for interoperability,
+such as to export networks to R (which can then be plotted via R).
+To install:
+
+```julia
+Pkg.add("PhyloPlots")
+```
+
+PhyloPlots depends on PhyloNetworks, and has further dependencies
+like [Gadfly](http://gadflyjl.org/stable/) and
+[RCall](https://github.com/JuliaInterop/RCall.jl)
 
 ## Test example
 

docs/src/man/src/installation.jmd

@@ -1,4 +1,6 @@
-using Weave, Fontconfig, Cairo
+using Weave
+# using Fontconfig, Cairo
+# using PhyloPlots
 
 set_chunk_defaults(Dict{Symbol, Any}(:results => "hidden", :eval => false))
 

docs/src/man/src/make_weave.jl

@@ -1,4 +1,4 @@
-"tipNames","AVG.SVL","AVG.ltoe.IV","AVG.lfing.IV","region"
+"tipNames","AVG_SVL","AVG_ltoe_IV","AVG_lfing_IV","region"
 "ahli",4.039125443,2.553343811,1.943287666,0
 "alayoni",3.815704818,1.875257738,1.40302856,0
 "alfaroi",3.526654599,1.73254011,1.040276712,0

examples/lizard_trait.txt

@@ -4,31 +4,25 @@ module PhyloNetworks
 
     using DataStructures # for updateInCycle with priority queue
     using DataFrames # for functions to read/write tables, and names()
+    using StatsBase # sample, coef etc.
+    using StatsModels
     using GLM # for the lm function
     using NLopt # for branch lengths optimization
-    using Gadfly # for plots
-    using ColorTypes # used by Gadfly already. To resolve data type names (Colorant)
-    using StatsBase: sample
     using Combinatorics.combinations
-    using RCall
-    using RCall: protect, unprotect, RClass
-    using NullableArrays
     using StaticArrays
     using IterTools
     using BioSequences
     using BioSymbols
 
     import Base.show
-    import Gadfly.plot
     import GLM.ftest
-    import RCall.sexp
-    export plot, ftest, sexp
 
     global DEBUG = false #for debugging only
     const DEBUGC = false #more detailed prints
     global CHECKNET = false #for debugging only
     global REDIRECT = false # changed for debugging to a file
 
+    export ftest
     export
         ## Network Definition
         HybridNetwork,
@@ -126,11 +120,7 @@ module PhyloNetworks
         nStates,
         Q, P,
         randomTrait,
-        randomTrait!,
-        #
-        apeRExport
-
-    # export part
+        randomTrait!
 
     include("types.jl")
     include("auxiliary.jl")
@@ -147,13 +137,11 @@ module PhyloNetworks
     include("manipulateNet.jl")
     include("bootstrap.jl")
     include("multipleAlleles.jl")
-    include("plotsGadfly.jl")
-    include("plotsRCall.jl")
     include("compareNetworks.jl")
     include("traits.jl")
     include("parsimony.jl")
     include("pairwiseDistanceLS.jl")
-    include("apeRExport.jl")
+    include("interop.jl")
     include("substitutionModels.jl")
     include("biconnectedComponents.jl")
 

src/PhyloNetworks.jl

@@ -352,36 +352,36 @@ addHybridizationUpdateSmart!(net::HybridNetwork, N::Integer) = addHybridizationU
 ## can we put in a different color in the plot?
 ## Cecile: you could add a new option to our RCall-based plot function to take a vector of colors.
 """
-`addAlternativeHybridizations!(net::HybridNetwork,BSe::DataFrame; cutoff=10::Number,top=3::Int)`
+    addAlternativeHybridizations!(net::HybridNetwork, BSe::DataFrame;
+                                  cutoff=10::Number, top=3::Int)
 
 Modify the network `net` (the best network estimated with snaq) by adding other hybridizations
 that are present in the bootstrap networks. By default, it will only consider hybrid edges with
 more than 10% bootstrap support (`cutoff`) and it will only include the three top hybridizations
 (`top`) sorted by bootstrap support.
 The function also modifies the dataframe `BSe` obtained with `hybridBootstrapSupport`. In the original
-`BSe` dataframe, hybrid edges that do not appear in the best network have NA as number.
+`BSe` dataframe, hybrid edges that do not appear in the best network have a missing number.
 After the hybrid edges are added with `addAlternativeHybridizations`, `BSe` is modified to include the
 edge numbers of the newly added hybrid edges. Note that the function only adds the hybrid edges as minor to
 keep the underlying tree topology.
 
-Example:
+# example
 
+```julia
 bootnet = readMultiTopology("bootstrap-networks.txt")
-
 bestnet = readTopology("best.tre")
-
 BSn, BSe, BSc, BSgam, BSedgenum = hybridBootstrapSupport(bootnet, bestnet);
-
 addAlternativeHybridizations!(bestnet,BSe)
-
+using PhyloPlots
 plot(bestnet, edgeLabel=BSe[[:edge,:BS_hybrid_edge]])
+```
 """
 function addAlternativeHybridizations!(net::HybridNetwork,BSe::DataFrame; cutoff=10::Number,top=3::Int)
     top > 0 || error("top must be greater than 0")
     BSe[:alternative] = falses(length(BSe[:hybrid]))
     newBSe = BSe[BSe[:BS_hybrid_edge] .> cutoff,:]
-    newBSe = newBSe[.!isna.(newBSe[:hybrid]) & .!isna.(newBSe[:sister]),:]
-    newBSe = newBSe[isna.(newBSe[:edge]),:]
+    newBSe = newBSe[.!ismissing.(newBSe[:hybrid]) & .!ismissing.(newBSe[:sister]),:]
+    newBSe = newBSe[ismissing.(newBSe[:edge]),:]
     newHyb = newBSe[1:top,:]
 
     if(size(newHyb,1) == 0)
@@ -393,8 +393,8 @@ function addAlternativeHybridizations!(net::HybridNetwork,BSe::DataFrame; cutoff
         hybnum = newHyb[:hybrid][i]
         sisnum = newHyb[:sister][i]
         edgenum = addHybridBetweenClades!(hybnum,sisnum,net)
-        ind1 = find(x->!isna(x) && x==hybnum,BSe[:hybrid])
-        ind2 = find(x->!isna(x) && x==sisnum,BSe[:sister])
+        ind1 = find(x->!ismissing(x) && x==hybnum,BSe[:hybrid])
+        ind2 = find(x->!ismissing(x) && x==sisnum,BSe[:sister])
         ind = intersect(ind1,ind2)
         BSe[ind,:edge] = edgenum
         BSe[ind,:alternative] = true