Skip to content
This repository was archived by the owner on Jun 12, 2023. It is now read-only.

Building TileDB

Jump to bottom
Stavros Papadopoulos edited this page Jan 13, 2017 · 95 revisions
# Requirements

TileDB has been tested on Ubuntu Linux (v.15.10), CentOS Linux (v.7.3.1611) and Mac OS X El Capitan (v.10.11.2), but TileDB should work with any reasonably recent version of Ubuntu, CentOS or Mac OS X. Moreover, it has been tested with g++ v.5.3.0 on Ubuntu, g++ v.4.8.5 on CentOS, and Clang Apple LLVM version 7.3.0 on Mac OS X.

Library Dependencies

TileDB has minimal dependencies, relying only on the following standard libraries:

Mandatory dependencies:

Optional dependencies:

  • MPI (for multi-processing operations)
  • OpenMP (for multi-threaded operations)

Other Software

In case you wish to run the TileDB code tests, or generate the TileDB code documentation, you will also need:

# Build on Ubuntu Linux

Step 1 - Install cmake and wget

You first need to install two useful programs, cmake and GNU wget. Simply run:

sudo apt-get install cmake
sudo apt-get install wget

Step 2 - Install zlib, OpenSSL, MPI and Google Test

Simply run:

sudo apt-get install zlib1g-dev
sudo apt-get install libssl-dev
sudo apt-get install mpich
sudo apt-get install libgtest-dev

Note that we have tested with the mpich version of MPI, but one may alternatively install OpenMPI.

Step 3 - Install Doxygen

If you wish to compile the TileDB code documentation, you need Doxygen. Simply run:

sudo apt-get install doxygen

Setp 4 - Clone the TileDB Repo

Install git if you have not already done so, by running:

sudo apt-get install git

Then, run:

git clone https://github.com/Intel-HLS/TileDB.git

This will create a directory TileDB in you current working directory, containing the TileDB source files.

Step 5 - Build TileDB

Now you are ready to build TileDB. First, enter the TileDB source file directory:

cd <TileDB source file directory path>

Then, create the TileDB C API library by running

make -j X

where X is the number of jobs to run simultaneously (typically equal to the number of available cores - this will speed up the make process significantly).

To create the examples, run:

make examples -j X

To run the code tests, run:

make test -j X

To run in debug mode, add flag BUILD=debug. Also, to enable verbose error messages, add flag VERBOSE=1. Finally, to enable OpenMP for multi-threading, add flag OPENMP=1.

Note that the compiler may complain about missing the OpenSSL header files, or the linker may complain about missing the corresponding library. In this case you may have to run:

make INCLUDE_PATHS=-I<your-openssl-include-path> LIBRARY_PATHS=-L<your-openssl-lib-path> -j X

You can create the Doxygen documentation by running:

make doc

Finally, you can clean up the built files by running:

make clean

Step 6 - Proper Care for MPI

Assuming that you have installed MPI (mpich in our examples), you need to add some extra flags upon making. Specifically, you need to run:

make MPI=1 CXX=mpic++ INCLUDE_PATHS="-I<your-openssl-include-path> -I<your-MPI-include-path>" LIBRARY_PATHS="-L<your-openssl-lib-path> -L<your-mpi-lib-path>" MPILIB=-lmpich -j X
# Build on CentOS Linux

Step 1 - cmake and wget

You first need to install two useful programs, cmake and GNU wget. Simply run:

sudo yum install cmake
sudo yum install wget

Step 2 - Install zlib, OpenSSL, MPI and Google Test

Simply run:

sudo yum install zlib-devel
sudo yum install openssl-devel
sudo yum install mpich-devel
sudo yum install gtest-devel

Note that we have tested with the mpich version of MPI, but one may alternatively install OpenMPI.

Step 3 - Install Doxygen

If you wish to compile the TileDB code documentation, you need Doxygen. Simply run:

sudo yum install doxygen

Setp 4 - Clone the TileDB Repo

Install git if you have not already done so, by running:

sudo yum install git

Then, run:

git clone https://github.com/Intel-HLS/TileDB.git

This will create a directory TileDB in you current working directory, containing the TileDB source files.

Step 5 - Build TileDB

Now you are ready to build TileDB. First, enter the TileDB source file directory:

cd <TileDB source file directory path>

Then, create the TileDB C API library by running

make -j X

where X is the number of jobs to run simultaneously (typically equal to the number of available cores - this will speed up the make process significantly).

To create the examples, run:

make examples -j X

To run the code tests, run:

make test -j X

To run in debug mode, add flag BUILD=debug. Also, to enable verbose error messages, add flag VERBOSE=1. Finally, to enable OpenMP for multi-threading, add flag OPENMP=1.

Note that the compiler may complain about missing the OpenSSL header files, or the linker may complain about missing the corresponding library. In this case you may have to run:

make INCLUDE_PATHS=-I<your-openssl-include-path> LIBRARY_PATHS=-L<your-openssl-lib-path> -j X

You can create the Doxygen documentation by running:

make doc

Finally, you can clean up the built files by running:

make clean

Step 6 - Proper Care for MPI

Assuming that you have installed MPI (mpich in our examples), you need to add some extra flags upon making. Specifically, you need to run:

make MPI=1 CXX=mpic++ INCLUDE_PATHS="-I<your-openssl-include-path> -I<your-MPI-include-path>" LIBRARY_PATHS="-L<your-openssl-lib-path> -L<your-mpi-lib-path>" MPILIB=-lmpich -j X
# Build on Mac OS X

Mac OS X uses Clang as its default C++ compiler, which unfortunately does not support OpenMP. Therefore, the TileDB builder deactivates OpenMP under Mac OS X, which comes at some small performance cost (e.g., internal sorting is not parallel any more when loading unsorted sparse cells into an array).

Step 1 - Install Homebrew

First, you need to install Homebrew, which is a package manager for Mac OS X. After installing Homebrew, you will be able to install packages to your Mac by running:

brew install <package-name>

You can find the installation guidelines on its website, but an easy way to install Homebrew is by running the following command:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Step 2 - Install cmake and wget

You also need to install two useful programs, cmake and GNU wget. Simply run

brew install cmake
brew install wget

Step 3 - Install zlib, OpenSSL and MPI

Simply run

brew install lzlib
brew install openssl
brew install mpich

Note that we have tested with the mpich version of MPI, but one may alternatively install OpenMPI.

Step 4 - Install Google Test

Get the Google Test development release:

wget https://github.com/google/googletest/archive/release-1.7.0.tar.gz
tar xf release-1.7.0.tar.gz

Compile the sources:

cd googletest-release-1.7.0
cmake .
make

Move the header and library files:

sudo mv include/gtest /usr/include/gtest
sudo mv libgtest_main.a libgtest.a /usr/lib/

Note: The last commands may not work on your Max OS due to Apple's System Integrity Protection. If that is the case, you need to reboot your system and press cmd+r while it is booting to enter the recovery mode. Then go to Utilities > Terminal from the menu at the top, and run the following commands:

csrutil disable
reboot

This will reboot your system, and then you will be able to move the header and library files as described above.

Step 5 - Install Doxygen

If you wish to compile the TileDB code documentation, you need Doxygen. Simply run:

brew install doxygen

Step 6 - Clone the TileDB Repo

Install git if have not already done so, by running:

brew install git

Then, run:

git clone https://github.com/Intel-HLS/TileDB.git

This will create a directory TileDB in you current working directory, containing the TileDB source files.

Step 7 - Build TileDB

Now you are ready to build TileDB. First, enter the TileDB source file directory:

cd <TileDB source file directory path>

Then, create the TileDB C API library by running

make -j X

where X is the number of jobs to run simultaneously (typically equal to the number of available cores - this will speed up the make process significantly).

To create the examples, run:

make examples -j X

To run the code tests, run:

make test -j X

To run in debug mode, add flag BUILD=debug. Also, to enable verbose error messages, add flag VERBOSE=1.

Note that the compiler may complain about missing OpenSSL header files, or the linker may complain about missing the corresponding library. In this case you may have to run:

make INCLUDE_PATHS=-I<your-openssl-include-path> LIBRARY_PATHS=-L<your-openssl-lib-path> -j X

You can create the Doxygen documentation by running:

make doc

Finally, you can clean up the built files by running:

make clean

Step 8 - Proper Care for MPI

Assuming that you have installed MPI (mpich in our examples), you need to add some extra flags upon making. Specifically, you need to run:

make MPI=1 CXX=mpic++ INCLUDE_PATHS="-I<your-openssl-include-path> -I<your-MPI-include-path>" LIBRARY_PATHS="-L<your-openssl-lib-path> -L<your-mpi-lib-path>" MPILIB=-lmpich -j X

If you attempt to run an MPI-based program with mpiexec, depending on the Mac OS X version, the program may fail with a weird message about the hostname. You need to do the following. First, go to the System Preferences > Sharing settings and make sure you enable Remote Login. Then, set a proper host name by running:

sudo scutil HostName your-host-name

where your-host-name is a name of your choice. Next, add 127.0.0.1 your-host-name to the /etc/hosts/ file. Finally, add your public key to the authorized keys file by running:

cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

The above should do the trick.

# Test If It Works

Try running the tutorial examples. For example,

./examples/bin/release/tiledb_workspace_group_create

should create a directory called my_workspace in your current working directory.

Moreover, you can try running the current Google tests, e.g.,

make test INCLUDE_PATHS=-I/usr/local/opt/openssl/include -j 4

which should show that all tests passed.

Table of Contents

Clone this wiki locally