Skip to content

Code for our paper "Evaluating SIMD Compiler-Intrinsics for Database Systems"

Notifications You must be signed in to change notification settings

hpides/autovec-db

Folders and files

NameName
Last commit message
Last commit date

Latest commit

8bf194a · Jul 5, 2023
Apr 14, 2023
Jun 12, 2023
Jun 12, 2023
Jun 9, 2023
Aug 26, 2022
Feb 7, 2023
Mar 8, 2023
May 3, 2023
Jun 6, 2023
Jul 5, 2023

Repository files navigation

Microbenchmarks for our paper "Evaluating SIMD Compiler-Intrinsics for Database Systems"

This file focuses on the microbenchmarks. For reproducing the Velox measurements, we have a separate README file.

Our measurement results can be found in eval/results/

Running the benchmarks

The measurements were done using a docker image based on ubuntu 22.10 with clang 15 and gcc 12.

To repeat the measurements, run the following commands:

git clone https://github.com/hpides/autovec-db.git

# using enroot:
enroot import docker://hpides/autovec-db
enroot create hpides+autovec-db.sqsh
enroot start -m ./autovec-db/:/autovec-db --rw hpides+autovec-db

# alternatively, using docker:
docker pull hpides/autovec-db
# note: we're using --privileged as we observed up to 5x slower measurements without it (likely a seccomp problem)
docker run -it --privileged -v "$(pwd):/autovec-db" hpides/autovec-db

The commands that run inside the container can be found at scripts/docker_entrypoint.sh.

The benchmarks log their results to .csv files that can be compared using our diff script at eval/scripts/diff.py.

The docker image was built using:

# x86
DOCKER_BUILDKIT=1 docker build -t hpides/autovec-db:x86 --target=x86 .
# AArch64
DOCKER_BUILDKIT=1 docker build -t hpides/autovec-db:aarch64 --target=aarch64 .

Note: To build the AArch64 image, you need QEMU on an x86 system.

To upload the images and automatically detect the architecture on download, run:

# Create
docker manifest create hpides/autovec-db:latest \
          --amend hpides/autovec-db:x86 \
          --amend hpides/autovec-db:aarch64

# Upload
docker manifest push hpides/autovec-db:latest

Plotting the results

This is a short guide on how to work with the results from the paper and generate the plots.

Setup

We use a basic Python + Pandas + Matplot setup. To create it, run the following commands in the eval directory:

$ python3 -m virtualenv venv
$ source venv/bin/activate
$ pip3 install -r requirements.txt

Plots

Our scripts always generate a PNG and an SVG for each plot. By default, it will generate a version optimized for PNG. If you want to create the SVG plots used in the paper, set AUTOVEC_DB_PLOTS=ON in your environment variables.

To generate a plot for a benchmark, run the following command:

$ python3 scripts/<script_name>.py results/ /path/to/plot/dir

The script will output a command that you can use to view the file. Generally, the plot is stored at /path/to/plot/dir/<script_name>.[png|svg].

You can also create all plots with:

$ ./all_plots.sh results /path/to/plot/dir