Preregistration.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SJj2vLdPMndg"
   },
   "source": [
    "# Preregistration"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "k9MpoJBjMqeQ"
   },
   "source": [
    "## Study Information\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o8GIVRdTMwDP"
   },
   "source": [
    "### Title\n",
    "\n",
    "Benchmarking the Visual Utility of Private Scatterplots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5PAR72bOM3HH"
   },
   "source": [
    "### Authors\n",
    "Anonymous for peer review. Online form on [osf.io](https://osf.io/) will list authors upon publication or embargo expiration."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1_N3sngWMxv-"
   },
   "source": [
    "### Hypotheses\n",
    "\n",
    "\n",
    "1.   Different algorithms' visual utility will vary depending on the shape of the data. This will be determined by the median utility rating being different for at least two of the algorithms as determined by the Friedman test. \n",
    "2.   Different algorithms' visual utility will vary depending on the task associated with the data.  This will be determined by the median utility rating being different for at least two of the algorithms as determined by the Friedman test. \n",
    "3.   Different algorithms' visual utility will vary depending on the privacy level chosen.  This will be determined by the median utility rating being different for at least two of the algorithms as determined by the Friedman test. \n",
    "4.   Different algorithms' visual utility will vary depending on the bin size chosen. This will be determined by the median utility rating being different for at least two of the algorithms as determined by the Friedman test. \n",
    "5.   At least two common metrics of utility will have different strengths of association with the visual utility scores generated. \n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AWVH0m_NjVVp"
   },
   "source": [
    "### Contributions\n",
    "\n",
    "\n",
    "\n",
    "1.   A comparison of differential privacy algorithms' visual utility across a range of tasks, bin sizes, privacy levels, and scatterplot shapes.\n",
    "2.   Guidance on how to display 2D histrograms privately using differential privacy. Discussion on how to improve the visual quality of the output of the data.\n",
    "3.   An examination of how utility metrics correspond to human-percieved utilty.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "634G2JoSNB8Y"
   },
   "source": [
    "## Design Plan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MzsOWfgbNEP3"
   },
   "source": [
    "### Study type \n",
    "\n",
    "Observational Study/Data study"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "okIvQsdKNIoI"
   },
   "source": [
    "### Blinding \n",
    "\n",
    "Experts will not be able to see either the privacy parameter nor the algorithm used. They will only see the two plots side by side."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V4vRPISdNL3e"
   },
   "source": [
    "### Study Design\n",
    "The questions are blocked by task. The experts will therefore first go through all questions related to their ability to identify the same correlation in the private plot as the original plot. Then they will proceed through clusters and distribution questions. Inside of these blocks the questions are randomly ordered. Because this is a data study we are not concerned with ordering effects.\n",
    "\n",
    "3 raters will see all the stimuli and give a rating for each one.\n",
    "\n",
    "Five factors:\n",
    "\n",
    "* Algorithm (DAWA, AHP, AGrid, Laplace, Geometric Truncated)\n",
    "* Epsilon, $\\epsilon$ (.5, .1, .05, .01)\n",
    "* Task (Clusters, Distributions, Correlation)\n",
    "* Scatterplot shape (20 different shapes)\n",
    "* Bin Size (32x32, 64x64)\n",
    "\n",
    "Utility Metrics tested:\n",
    "\n",
    "* Average per Query Error - [Paper](https://doi.org/10.48550/arXiv.1512.04817), [Implementation](https://github.com/dpcomp-org/dpcomp_core)\n",
    "* MS-SSIM - [Paper](https://doi.org/10.1109/ACSSC.2003.1292216), [Implementation](https://github.com/VainF/pytorch-msssim)\n",
    "* Scagnostics - [Paper](https://research.tableau.com/sites/default/files/Wilkinson_Infovis-05.pdf), [Implementation](https://pypi.org/project/pyscagnostics/)\n",
    "* Earth Movers Distance - [Paper](https://doi.org/10.1109/TVCG.2017.2745139), [Implementation](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wasserstein_distance.html)\n",
    "* 1\\Epsilon - [Paper](https://doi.org/10.48550/arXiv.2201.05964)\n",
    "* SDMetrics - [Paper](https://doi.org/10.48550/arXiv.2112.09238), [Implementation](https://github.com/sdv-dev/SDMetrics)\n",
    "\n",
    "    * KSTest\n",
    "    * BNLikelihood\n",
    "    \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BmB02J38NSjO"
   },
   "source": [
    "## Sampling Plan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "l03A5uC9NXIX"
   },
   "source": [
    "### Existing data\n",
    "\n",
    "Registration prior to creation of data: As of the date of submission of this research plan for preregistration, the data have not yet been collected, created, or realized. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KyG9N4O4NX-R"
   },
   "source": [
    "### Data collection procedures\n",
    "\n",
    "Population: Visualization Experts\n",
    "\n",
    "Recruitment efforts: Project collaborators\n",
    "\n",
    "Payment: None\n",
    "\n",
    "Eligibility: Data Visualization Professor\n",
    "\n",
    "Timeline: Data collected in July\n",
    "\n",
    "Data is collected through a study website. There each expert will be able to rate the utility of each plot they see. The data for each expert is exported in a json format indicating the underlying parameters that generated the private plot and the expert rating. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AgkoJvWGNcXe"
   },
   "source": [
    "### Sample size \n",
    "\n",
    "\n",
    "1,200 judgements from three expert reviewers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p7J0VnxSNda-"
   },
   "source": [
    "### Sample size rationale\n",
    "\n",
    "Our study is a qualitative data study. We are interested in the way the manipulated variables affected the utility of the data and the not differences of people's perception of utility. Our goal is to test as many parameters as possible and this can be done with a smaller amount of dedicated coders. Previous work done by [Seldmair et al.](https://doi.org/10.1109/TVCG.2013.153) uses two coders when they propose the concept of a qualitative data study. We use three coders since the definition of utility can be hard to pinpoint and having another perspective ensures that our consensus on the private plots utility stems from a broad background. The three coders make 1,200 judgements (5 algorithms x 2 bins sizes x 4 privacy levels x 30 scaterplot/tasks). Not every shape is conducive to all three tasks so each plot was labeled with the tasks an analyst would likely use it for. \n",
    "\n",
    "We did not do the typical user study with many participants and a power analysis for several reasons. Asking participants to answer over a thousand questions requires an unreasonable amount of time and resources. Additionally, participants recruited from a general pool have likely not done extensive data analysis. All three of our experts are experienced in analyzing scatterplot data for the specified tasks. Therefore, they were able to discuss what utility means for a data analyst and come to an agreed upon ranking of utility related to their experience that would have been impossible if a random population sample had been chosen as the study participants. Finally, the majority variation in the ratings stems from the data and not from the reviewers perception of the data.\n",
    "\n",
    "Since we are not comparing across individuals but instead comparing across data variables, we first ran a pilot study to ensure the trained coders agreed upon the evaluation of the data. We use an inter-rater reliability (IRR) metric to ensure the coders consistently agree upon their evaluation of utility. A high IRR ensures that the variation in the utility of the private plots arises from the differences in the plot parameters and not from the differences in coder perception. IRR scores range from a score of 0 to 1. As an example, a score of .8 would mean that 80\\% of the variance is due to the true variance of the underlying data and 20\\% of the variance is due to the differences in coder ratings. A common practice is to set an a-priori value of inter-rater reliability to ensure there is an appropriate amount of agreement amongst the coders before they are presented with the real data. Our a-priori was set at .6 which is classified as substantial aggreement by [Kevin Hallgren](https://doi.org/10.20982%2Ftqmp.08.1.p023). This allows us to attribute the differences in the scores to the data and not to the users. \n",
    "\n",
    "For our pilot study we found an IRR of .7311 with a 95% confidence interval of [0.68, 0.78]. This was an acceptable score based of Kevin Hallgren's recommendations. The statistical analysis can be found in the inter-coder reliability [section](#IRR).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gfc8J8B4NgPl"
   },
   "source": [
    "## Variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "rCTsgrw9Ni4W"
   },
   "source": [
    "### Manipulated variables\n",
    "\n",
    "We manipulated 5 variables: task type, scatterplot type, privacy level, bin number, algorithm used. \n",
    "\n",
    "There are 3 tasks that we wanted to ensure the user of the private scatterplot could still complete. They are listed below with the wording of the question from the study. \n",
    "\n",
    "\n",
    "*   <b>Distribution: </b>\n",
    "            The distribution of points in space for the graph on the right\n",
    "            is comparable to the graph on the left, including the visibility of \n",
    "            manifolds and the relative density of each region.\n",
    "*   <b>Correlation: </b>\n",
    "            The graph on the right preserves the level of\n",
    "            dependence between the two attributes—including non-linear\n",
    "            dependence.\n",
    "*   <b>Clusters: </b>\n",
    "            The clusters visible in the graph on the left—and\n",
    "            no other clusters—are visible on the graph on the right and occur in\n",
    "            the same places.\n",
    "\n",
    "\n",
    "For our scatterplot types we chose scatterplots from each of the 20 categories generated by [Pandey et al.](https://doi.org/10.1145/2858036.2858155). The scatterplots had points added until all scatterplots had 5000 points. The 20 scatterplots can all be seen in [scatterplotStimuli.pdf](scatterplotStimuli.pdf) and the code to generate them can be found at [scatterplotDensification.ipynb](scatterplotDensification.ipynb). The website that the study was conducted with can be found under [pilotPrivatePlotsStudyWebsite.zip](pilotPrivatePlotsStudyWebsite.zip) with the pilot data added.\n",
    "\n",
    "The privacy level was adjusted by epsilon. The four different levels of epsilon chosen were .5, .1, .05, .01. The two different bin numbers used to generate the binned scatterplots were 32 and 64 evenly-sized bins. The five algorithms selected were [DAWA](https://doi.org/10.48550/arXiv.1410.0265), [AHP](https://doi.org/10.1137/1.9781611973440.68), [AGrid](https://doi.org/10.1109/ICDE.2013.6544872), [Laplace](https://doi.org/10.1007/978-3-540-32732-5_32), and [Geometric Truncated](https://doi.org/10.48550/arXiv.0811.2841). The implementation for DAWA, AHP, and AGrid can be found [here](https://github.com/dpcomp-org/dpcomp_core) and the implementation for Laplace and Geometric Truncated can be found [here](https://diffprivlib.readthedocs.io/en/latest/index.html). The full code to generate the plots can be found at [pilotPrivatePlotGenerationAndStatsErrors.ipynb](pilotPrivatePlotGenerationAndStatsErrors.ipynb).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FQWUuDc9NlUv"
   },
   "source": [
    "### Measured variables\n",
    "\n",
    "The ability of the private plot to retain the task utility is evaluated on a 4 point Likert scale. The categories were limited to four for two reasons. The nuances between different ratings of a more fine grained scale would between difficult to discern and our goal was to keep inter rater reliability high to validate our approach of many datasets tested with a small group of people. Our abbreviated scale can be described as: the user [doesn't, suggests, somewhat, does] retain the ability to complete the task using the private scatterplot when compared to the original binned plot.\n",
    "\n",
    "*   0: Doesn’t preserve the feature\n",
    "*   1: Suggests the feature could exist\n",
    "*   2: Somewhat preserves the feature\n",
    "*   3: Preserves the feature very well"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9JKrJ23wNorN"
   },
   "source": [
    "### Indices \n",
    "\n",
    "* <b>Median Utility Rating</b>: To create one rating for each stimulus to use for our analysis from the three ratings that are provided by the coders we use the median of the ratings. Since the data is ordinal and does not have an equal defined spacing between the four ratings we do not use the mean. The median provides us with one homogenous rating from the three raters. \n",
    "\n",
    "* <b>Kendall's $\\tau_b$</b>: For each automated utility metric we will calculate Kendall's $\\tau_b$ for the metric against the median utility rating. This indice will be used to test hypothesis 5."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5siVoXudNrFH"
   },
   "source": [
    "## Analysis Plan\n",
    "\n",
    "The current analysis code is run with pilot data. Once the study data is collected the exact same code will be run on the full data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7HGm_NE7Nt32"
   },
   "source": [
    "### Inter-Rater Reliability\n",
    "\n",
    "<a id='#IRR'></a>\n",
    "\n",
    "To ensure that our task questions and structure were clear across all raters, we ran pilot tests to ensure inter-rater reliability was acceptable. To test the interrater reliability we follow the guidelines for choosing intraclass correlation coefficient (ICC) selection as set out by [Koo and Li](https://doi.org/10.1016%2Fj.jcm.2016.02.012). ICC's are a common way of testing interrater reliatiblity. \n",
    "\n",
    "There are 10 different forms of ICC that rely on different assumptions. To start we are testing for interrater reliability which should not be mistaken for test-retest reliability or intrarater reliability. Interrater reliability measures the variance of across raters on a set of stimuli (stimuli in our case are the private data plots). The selection of the correct ICC comes by defining the model, type, and definition of relationship. Our model is a two-way random effects model. We choose this model because we have the same set of raters evaluate all the data and we generalize our results to any raters who have the same characteristics as our raters. Type asks how the rating will be used in the application. Multiple raters means the average of the ratings will be used while single rater means we will treat the rating as if it came from a single rater though multiple raters made the judgement. Our study involves ordinal data and therefore the average of the three raters does not work. Therefore, we use the single rater type. Finally, for the relationship of the evaluations we can choose between absolute agreement and consistency. For our ratings we are looking for absolute agreement since we want a consensus on utility. \n",
    "\n",
    "Therefore our final ICC selection corresponds to a Two-way mixed effects, absolute agreement, single rater/measurement - ICC(3,1).\n",
    "\n",
    "$\\frac{MS_R-MS_E}{MS_R+(k-1)MS_E}$\n",
    "\n",
    "where $MS_R$ = mean square for rows; $MS_E$ = mean square for error; and k = number of raters/measurements.\n",
    "\n",
    "We used .6 as our minimum cutoff for the kappa to proceed with the actual study. This corresponds with good and substantial agreement from the papers presented by [Cicchetti](https://psycnet.apa.org/doi/10.1037/1040-3590.6.4.284) and [Landis and Koch](https://doi.org/10.2307/2529310)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "Q2h0D2alF0Hw"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "\n",
    "!pip install pingouin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 424
    },
    "id": "edWwYHGBF45_",
    "outputId": "9696be6f-7f97-4e10-bb7c-7290b57b4d9d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3_AHP_clusters_0.01_64</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3_Geometric_distribution_0.5_32</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3_AGrid_clusters_0.05_64</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3_Laplace_clusters_0.5_32</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2_AHP_distribution_0.5_64</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3_AGrid_clusters_0.5_64</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2_DAWA_distribution_0.05_32</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3_Geometric_clusters_0.01_32</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9_Geometric_distribution_0.1_64</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2_Laplace_distribution_0.05_32</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>240 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 0  1  2\n",
       "3_AHP_clusters_0.01_64           0  0  0\n",
       "3_Geometric_distribution_0.5_32  2  3  2\n",
       "3_AGrid_clusters_0.05_64         0  0  0\n",
       "3_Laplace_clusters_0.5_32        3  1  3\n",
       "2_AHP_distribution_0.5_64        1  3  2\n",
       "...                             .. .. ..\n",
       "3_AGrid_clusters_0.5_64          3  1  3\n",
       "2_DAWA_distribution_0.05_32      1  1  0\n",
       "3_Geometric_clusters_0.01_32     0  0  0\n",
       "9_Geometric_distribution_0.1_64  1  0  1\n",
       "2_Laplace_distribution_0.05_32   1  0  0\n",
       "\n",
       "[240 rows x 3 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import os\n",
    "import json\n",
    "# assign directory\n",
    "directory = './pilotResults'\n",
    " \n",
    "# iterate over files in\n",
    "# that directory\n",
    "answerDict = {}\n",
    "for filename in os.listdir(directory):\n",
    "    f = os.path.join(directory, filename)\n",
    "    # checking if it is a file\n",
    "    if os.path.isfile(f):\n",
    "      # print(f)\n",
    "        file=open(f)\n",
    "        data=json.load(file)\n",
    "        for d in data['answers']:\n",
    "          if d[0] in answerDict:\n",
    "            answerDict[d[0]].append(d[1])\n",
    "          else:\n",
    "            answerDict[d[0]] = [d[1]]\n",
    "\n",
    "answerDataFrame = pd.DataFrame.from_dict(answerDict)\n",
    "answerDataFrame = answerDataFrame.T\n",
    "answerDataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "id": "Mg24VRivF6ev"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "question = answerDataFrame.index.to_numpy()\n",
    "\n",
    "score0 = answerDataFrame[0].to_numpy()\n",
    "score1 = answerDataFrame[1].to_numpy()\n",
    "score2 = answerDataFrame[2].to_numpy()\n",
    "name0 = np.full(len(score0), 0)\n",
    "name1 = np.full(len(score1),1)\n",
    "name2 = np.full(len(score2),2)\n",
    "\n",
    "questions = np.concatenate((question, question, question))\n",
    "raters = np.concatenate((name0, name1, name2))\n",
    "ratings = np.concatenate((score0, score1, score2))\n",
    "\n",
    "df = pd.DataFrame({'question': questions, 'rater': raters, 'ratings': ratings})\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 238
    },
    "id": "3bqXFWRnF95i",
    "outputId": "5d0930ce-c673-44e0-8b0d-9e303b72d098"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Type</th>\n",
       "      <th>Description</th>\n",
       "      <th>ICC</th>\n",
       "      <th>F</th>\n",
       "      <th>df1</th>\n",
       "      <th>df2</th>\n",
       "      <th>pval</th>\n",
       "      <th>CI95%</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ICC1</td>\n",
       "      <td>Single raters absolute</td>\n",
       "      <td>0.699226</td>\n",
       "      <td>7.974255</td>\n",
       "      <td>239</td>\n",
       "      <td>480</td>\n",
       "      <td>6.451216e-82</td>\n",
       "      <td>[0.64, 0.75]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ICC2</td>\n",
       "      <td>Single random raters</td>\n",
       "      <td>0.703071</td>\n",
       "      <td>9.157029</td>\n",
       "      <td>239</td>\n",
       "      <td>478</td>\n",
       "      <td>5.489759e-92</td>\n",
       "      <td>[0.62, 0.77]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>ICC3</td>\n",
       "      <td>Single fixed raters</td>\n",
       "      <td>0.731111</td>\n",
       "      <td>9.157029</td>\n",
       "      <td>239</td>\n",
       "      <td>478</td>\n",
       "      <td>5.489759e-92</td>\n",
       "      <td>[0.68, 0.78]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ICC1k</td>\n",
       "      <td>Average raters absolute</td>\n",
       "      <td>0.874596</td>\n",
       "      <td>7.974255</td>\n",
       "      <td>239</td>\n",
       "      <td>480</td>\n",
       "      <td>6.451216e-82</td>\n",
       "      <td>[0.84, 0.9]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ICC2k</td>\n",
       "      <td>Average random raters</td>\n",
       "      <td>0.876595</td>\n",
       "      <td>9.157029</td>\n",
       "      <td>239</td>\n",
       "      <td>478</td>\n",
       "      <td>5.489759e-92</td>\n",
       "      <td>[0.83, 0.91]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>ICC3k</td>\n",
       "      <td>Average fixed raters</td>\n",
       "      <td>0.890794</td>\n",
       "      <td>9.157029</td>\n",
       "      <td>239</td>\n",
       "      <td>478</td>\n",
       "      <td>5.489759e-92</td>\n",
       "      <td>[0.86, 0.91]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Type              Description       ICC         F  df1  df2          pval  \\\n",
       "0   ICC1   Single raters absolute  0.699226  7.974255  239  480  6.451216e-82   \n",
       "1   ICC2     Single random raters  0.703071  9.157029  239  478  5.489759e-92   \n",
       "2   ICC3      Single fixed raters  0.731111  9.157029  239  478  5.489759e-92   \n",
       "3  ICC1k  Average raters absolute  0.874596  7.974255  239  480  6.451216e-82   \n",
       "4  ICC2k    Average random raters  0.876595  9.157029  239  478  5.489759e-92   \n",
       "5  ICC3k     Average fixed raters  0.890794  9.157029  239  478  5.489759e-92   \n",
       "\n",
       "          CI95%  \n",
       "0  [0.64, 0.75]  \n",
       "1  [0.62, 0.77]  \n",
       "2  [0.68, 0.78]  \n",
       "3   [0.84, 0.9]  \n",
       "4  [0.83, 0.91]  \n",
       "5  [0.86, 0.91]  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pingouin as pg\n",
    "icc = pg.intraclass_corr(data=df, targets='question', raters='rater',\n",
    "                         ratings='ratings')\n",
    "icc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mGOEt-x7Gjq-"
   },
   "source": [
    "### Algorithm Comparison\n",
    "\n",
    "The selection of best algorithm for all the subsequent analysis follows a three step process:\n",
    "\n",
    "\n",
    "\n",
    "1.   Using the Friedman test, we ensure that there is a difference between the 5 different algorithms. If the Friedman test returns a p value < .05 we continue on step 2 - the post-hoc analysis.\n",
    "2.   We conduct a post-hoc conover test to see which algorithms significantly differ from each other. This gives us more specific insight into which algorithms specifically are different from one another. The post hoc test is corrected for multiple hypothesis testing using the Benjamini/Hochberg method.\n",
    "3.   We visualize the data to examine the results from the post-hoc conover test to see which of the algorithms that have statistically significant differences in their results yield higher visual utility.\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Automated Utility Benchmarking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fhtaK3snrCLr"
   },
   "source": [
    "We want to measure the strength of association between the visual utility ratings generated by our coders and metrics of utility that can be generated computationaly. Our coder rankings are ordinal and the metrics are continuous. When obtaining the association of ordinal-continuous variables [Harry Khamis](https://doi.org/10.1177/8756479308317006) recommends using Kendall's coeficient of rank correlation $τ_b$. $τ_b$ can range from -1 (perfect negative association) to 1 (perfect positive assocation). \n",
    "\n",
    "\n",
    "We calculate every metric (Average per query error, MS-SSIM, Scagnostics, 1/epsilon, Earth Movers Distance, KSTest, BNLikelihood) on every stimulus comparing the binned non private with the binned private plot. We match these plots based on all the parameters (algorithm, task, bin size, shape, epsilon). For each metric we find Kendall's $\\tau_b$. We use the [implementation](https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.kendalltau.html) provided by scipy.stats.\n",
    "\n",
    "\n",
    "We are submitting the preregistration without the implementation for all the metrics in our code. Links to the respective libraries we plan to use are provided.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inference Criteria\n",
    "\n",
    "To test our hypotheses we will be using a holistic approach. We will look at both the p-value the Friedman test returns as well as examining the distributions found in the data to see if there is a meaningful difference between the algorithms. We will also this approach for the post-hoc analysis.\n",
    "\n",
    "While we have no specific hypothesis for the statistical utility correlations with visual utility, we will determine which metric is best used by finding the one with the highest positive correlation that has a p-value of less than .05. To test hypothesis 5, we will use the fisher z-transformation to test the different metrics corrrelation coefficients."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### P-Value Interpretation\n",
    "\n",
    "This p-value is the conditional probability of the data (or any result even more extreme) assuming:\n",
    "\n",
    "1. the null hypothesis H0 is exactly true,\n",
    "2. the study is repeated an infinite number times by drawing random samples from the same populations(s),\n",
    "3. all distributional requirements are met, and\n",
    "4. there is no source of error besides sampling or measurement error.\n",
    "\n",
    "p-value interpretation based on ([Kline, 2013](https://doi.org/10.1037/14136-000)])."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vY8Fv_N1lSvS"
   },
   "source": [
    "#### Code for analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "id": "cDJbS032j-ZJ"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "import statistics\n",
    "from scipy.stats import friedmanchisquare\n",
    "\n",
    "\n",
    "!pip install scikit-posthocs\n",
    "import scikit_posthocs as sp\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "id": "aIdQ2hhMIqJu"
   },
   "outputs": [],
   "source": [
    "\n",
    "medianArray = []\n",
    "for index, row in answerDataFrame.iterrows():\n",
    "    medianArray.append( statistics.median([row[0], row[1], row[2]]))\n",
    "\n",
    "answerDataFrame['totalRating'] = medianArray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "id": "heLs_KKdRNNW"
   },
   "outputs": [],
   "source": [
    "tasks = ['distribution', 'clusters', 'correlation']\n",
    "epsilons = ['0.5', '0.1', '0.05', '0.01']\n",
    "algorithms = ['DAWA', 'AHP', \"AGrid\", 'Geometric', 'Laplace']\n",
    "bins = ['32','64']\n",
    "chart = ['0','2','3','9']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "id": "4Uq4GegheUnH"
   },
   "outputs": [],
   "source": [
    "def getFilteredArray(df, name):\n",
    "  dataFiltered = df.filter(like=name, axis=0)\n",
    "  DAWA = []\n",
    "  AHP = []\n",
    "  AGrid = []\n",
    "  Geometric = []\n",
    "  Laplace = []\n",
    "  exDF = dataFiltered\n",
    "  while len(exDF.index)>0:\n",
    "    count = 0\n",
    "    for e in epsilons:\n",
    "      for b in bins:\n",
    "        for t in tasks:\n",
    "          for c in chart:            \n",
    "            for index, row in exDF.iterrows():\n",
    "              if count !=5:\n",
    "                \n",
    "                if c+'_' in index and b in index and t in index and e in index:\n",
    "\n",
    "\n",
    "                  if 'Laplace' in index:\n",
    "                      \n",
    "                      Laplace.append(row['totalRating'])\n",
    "                      exDF = exDF.drop(index)\n",
    "                      count = count+1\n",
    "                  if 'AHP' in index:\n",
    "                    \n",
    "\n",
    "                    AHP.append(row['totalRating'])\n",
    "                    exDF = exDF.drop(index)\n",
    "                    count = count+1\n",
    "                  if 'AGrid' in index:\n",
    "                    \n",
    "\n",
    "                    AGrid.append(row['totalRating'])\n",
    "                    exDF = exDF.drop(index) \n",
    "                    count = count+1\n",
    "                  if 'Geometric' in index:\n",
    "                    \n",
    "\n",
    "                    Geometric.append(row['totalRating'])\n",
    "                    exDF = exDF.drop(index) \n",
    "                    count = count+1\n",
    "                  if 'DAWA' in index:\n",
    "                  \n",
    "\n",
    "                    DAWA.append(row['totalRating'])\n",
    "                    exDF = exDF.drop(index) \n",
    "                    count = count+1\n",
    "    \n",
    "  dfAlgorithms =  pd.DataFrame({'DAWA': DAWA, 'AHP': AHP, 'AGrid': AGrid, 'Geometric': Geometric, 'Laplace': Laplace})\n",
    "  return dfAlgorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CY2lQvHzlDyQ"
   },
   "source": [
    "##### Step 1: Friedman Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "id": "yNv3RBq0erYp"
   },
   "outputs": [],
   "source": [
    "def getFriedmanResult(df):\n",
    "  stat, p = friedmanchisquare(df['DAWA'], df['AHP'],df['AGrid'], df['Geometric'], df['Laplace'])\n",
    "  print('Statistics=%.3f, p=%.3f' % (stat, p))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FWM7a5uWlHFY"
   },
   "source": [
    "##### Step 2: Post Hoc Conover\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "id": "ZDocnOxXe0en"
   },
   "outputs": [],
   "source": [
    "def getConoverPostHoc(df):\n",
    "  newLook = sp.__convert_to_block_df(df)\n",
    "  pc = sp.posthoc_conover(newLook[0], val_col='y', group_col='groups', p_adjust = 'fdr_bh')\n",
    "  pd.set_option('display.float_format', lambda x: '%.5f' % x)\n",
    "  return pc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5ykz2jcDlKlZ"
   },
   "source": [
    "##### Step 3: Visualization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "id": "x1a00Mmwe-mv"
   },
   "outputs": [],
   "source": [
    "def getVisual(df):\n",
    "  frequencies = {}\n",
    "  for i in df.columns:\n",
    "      frequencies[i] = df[i].value_counts()\n",
    "  plotdata = pd.DataFrame(frequencies)\n",
    "  plotdata = plotdata.transpose()\n",
    "  plotdata2 = plotdata.div(plotdata.sum(axis=1), axis=0)*100\n",
    "  plotdata2.plot(kind=\"barh\", stacked=True)\n",
    "  plt.legend(bbox_to_anchor=(1.05, 1))\n",
    "  plt.ylabel('Algorithm')\n",
    "  plt.xlabel('Cumulative Percent')\n",
    "  # return plotdata2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mFCzFvTgY9nj"
   },
   "source": [
    "#### Best Overall Algorithm\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "id": "VrPup6PtkRSJ"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, '0')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "uk57a3Bskb6y",
    "outputId": "1665353f-2c51-49e1-a64a-4efa4fe175dc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=31.911, p=0.000\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "tcLoe003keqO",
    "outputId": "9917974e-9f2a-4376-c29c-a1a6c5d1d37d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.92348</td>\n",
       "      <td>0.32095</td>\n",
       "      <td>0.50400</td>\n",
       "      <td>0.42810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.92348</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.32095</td>\n",
       "      <td>0.50580</td>\n",
       "      <td>0.42810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.32095</td>\n",
       "      <td>0.32095</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.54470</td>\n",
       "      <td>0.06261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.50400</td>\n",
       "      <td>0.50580</td>\n",
       "      <td>0.54470</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.19885</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.42810</td>\n",
       "      <td>0.42810</td>\n",
       "      <td>0.06261</td>\n",
       "      <td>0.19885</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.92348 0.32095    0.50400  0.42810\n",
       "AHP       0.92348 1.00000 0.32095    0.50580  0.42810\n",
       "DAWA      0.32095 0.32095 1.00000    0.54470  0.06261\n",
       "Geometric 0.50400 0.50580 0.54470    1.00000  0.19885\n",
       "Laplace   0.42810 0.42810 0.06261    0.19885  1.00000"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "lSfG4tpTkgyZ",
    "outputId": "ae8fc9a1-8e90-4f91-d1d9-688bf7afda46"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdyElEQVR4nO3de5zWdZ338dd7GITUwAMjmgZDBAuISjKe3cBDpXlMNATbG7ZaqruDYtZSe4h2Hxu0ZZaVGre1WnfpKmli7gqtQqZpMSA6IKAmCGIk3NgoHsCZ+dx//H6TF+Mcrhmua67fXNf7+XjMg9/5+nwHmPd8f6evIgIzMzPLhqpSF2BmZmZvcjCbmZlliIPZzMwsQxzMZmZmGeJgNjMzy5DqUhfQ1w0ZMiRqa2tLXYaZWZ+yYsWK7RFRU+o6ssjBvJdqa2upr68vdRlmZn2KpGdLXUNW+VS2mZlZhjiYzczMMsTBbGZmliG+xmxmZmVjxYoVh1RXV98IjCebnc8WYHVTU9PHJ06c+EJ7GziYzcysbFRXV9946KGHjq2pqXmxqqoqc4NBtLS0aNu2beO2bt16I3B+e9tk8bcJMzOznhpfU1PzUhZDGaCqqipqamoaSXr07W/Ti/WYmZkVW1VWQ7lVWl+H+etgNjMzyxBfYzYzs7JVO+eeiYU83sb556zoapuFCxcOuuqqq4a1tLTwkY98ZPvXvva1rd35DAfzXmrY0kjtnHtKXYZZ3jYOnF7qEvq0o0YMK3UJPXLbvKaCHm/surUFPV65aGpqYvbs2cMWL1785Lve9a43jjnmmLFTpkz588SJE1/P9xg+lW1mZlYgy5Yt22/48OG7xo0bt3vgwIFx0UUX7Vi4cOEB3TmGg9nMzKxANm/evM/hhx++u3X+iCOO2L1ly5Z9unMMB7OZmVmBRLz1hnBJ3bpL3MFsZmZWIMOGDdujh/zcc8/t8453vOON7hzDwWxmZlYgkyZNemXjxo0D161bt8/rr7+uO+6446ApU6b8uTvH8F3ZZmZWtvJ5vKmQ+vfvz9VXX73prLPOGt3c3Mz06dO319XV5X1HNjiYzczMCmrq1KmNU6dObezp/pk7lS1pZwGOUStpdSHqMTMz602ZC2YzM7NK1ieCWdJ5kn4n6VFJ/yNpaLp8rqSfSLpf0lOS/q6dfWsl/UbSyvTr5Jx1X5TUIOkxSfPTZSMl3StpRbrfmN5rqZmZVbq+co35QeDEiAhJHwe+CHw+XXc0cCKwH/CopLbvx3wBeF9EvC5pFHALUCfpbOBC4ISIeFXSQen2C4BPRsRTkk4ArgNOzz2gpFnALIB+g2oK3FQzM6tkfSWYjwD+U9JhwD7Ahpx1d0XEa8BrkpYCxwOrctb3B74naQLQDIxOl58J/EdEvAoQETsk7Q+cDNwuqXX/AW2LiYgFJAHOgMNGZXp4MTMz61v6SjB/F/hWRCySNBmYm7OubTC2nZ8N/Ak4huTUfett62pn2yrgzxExYa8rNjMz64G+EsyDgS3p9Iw26y6QNI/kVPZkYA5Jrzp33+ciokXSDKBfunwJ8M+SftZ6KjvtNW+QdElE3K6k23x0RDxWrIaZmVkRzR1c0GEfmdvY5XPRl1xySe199903+OCDD2566qmn1nT3I7J489e+kp7L+bqSpId8u6TfANvbbP974B7gEeBfI+L5NuuvA2ZIeoTkNPYrABFxL7AIqJe0Crgq3f4y4GOSHgPWABcUuoFmZla+PvrRj25ftGjRUz3dP3M95ojo6JeFuzpY/mREzGpzjI3A+HT6KZIbxFp9KWe7+cD8NvtuAM7qXtVmZmaJs88+e+f69eu7NaJUriz2mM3MzCpW5nrM3RERc0tdg5mZWSG5x2xmZpYhDmYzM7MM6dOnss3MzDqVx+NNhXbeeeeNeOSRR97+4osvVg8dOvToOXPmPD979uy2TxR1yMG8l446fDD1888pdRlm3dDj0egMaCh1AT3V9g0QVjR33333hq636phPZZuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsSPS5mZWdk66uajCjrsY8OMhk6fi3766af7X3bZZSO2bdvWv6qqihkzZmz7p3/6pxe68xnuMZuZmRVI//79ufrqq5975pln1ixfvnztD3/4w0NWrFgxsDvHcDCbmZkVyPDhw9849dRTXwU48MADW0aOHPnapk2bujUEpIPZzMysCNavX7/PE088se+kSZN2dmc/B7OZmVmBNTY2Vl100UUj58+fv/mggw5q6c6+DmYzM7MC2rVrl84555yRl1xyyY4ZM2b8ubv7O5jNzMwKpKWlhUsvvXT46NGjX587d+6fenIMPy5lZmZlq6vHmwrtV7/61f6/+MUvDh41atRrY8aMGQfw1a9+dcvUqVPzHtbNwWxmZlYgH/jAB3ZGxF79MuBT2WZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDPHjUnupYUsjtXPuKXUZViQbB04vdQmZd9SIYaUuoU+7bV5TqUvokbHr1pa6hLysHTO2oMM+jl23ttNHoV599VWdcMIJY3bv3q3m5madd955L15zzTXPd+czHMxmZmYFMnDgwHjwwQfXDx48uGXXrl067rjj/uq+++5rPOOMM17J9xg+lW1mZlYgVVVVDB48uAVg9+7dampqkqTuHaMolZmZmVWopqYmxowZM27o0KHHTJo06aXTTz89794yOJjNzMwKqrq6mnXr1j2xadOmx1euXLnf8uXLB3ZnfwezmZlZEQwZMqT51FNPffnuu+8e3J39HMxmZmYF8vzzz1dv3769H8DOnTu1bNmyQWPHjn29O8co6l3ZkoYC1wAnAi8Cu4F/j4g7i/m57dQxGdgdEb/tYP35wLiImN+bdZmZWXF19XhToW3evLn/zJkzRzQ3NxMRuuCCC3ZMmzYt7yEfoYjBnN6G9gvg5oiYni4bDpxfrM/sxGRgJ/CWYJZUHRGLgEW9XZSZmZWXE0444bW1a9c+sTfHKOap7NNJeqk3tC6IiGcj4ruS+kn6hqTlkh6X9AlIwjxdvlpSg6Sp6fLJkn4t6TZJT0qaL+kySb9PtxuZblcj6efpcZdLOkVSLfBJYLakVZL+WtJNkr4laSnwdUkzJX0vPcZQSXdKeiz9OrmI3yMzM7M9FPNU9pHAyg7WfQxojIjjJA0AHpK0BDgWmAAcAwwBlkt6IN3nGGAssAN4BrgxIo6XdDnwWeAK4DvANRHxoKRhwOKIGCvpBmBnRHwTQNLHgNHAmRHRLGlmTm3XAr+OiA9J6gfsX4hvhpmZWT567c1fkr4PnEpynflZ4GhJF6erBwOj0vW3REQz8CdJvwaOA14ClkfEH9Nj/QFYku7bAJyWTp8JjMt5mHuQpLd3UNLt6ee0dTrwvwDS9W+5NiBpFjALoN+gmq4bb2ZmvaWlpaVFVVVVUepCOtLS0iKgpaP1xQzmNcCU1pmI+LSkIUA9sAn4bEQszt1B0gc7Od6unOmWnPkW3mxHFXBSRLzW5rjtHa9bD3zniogFwAKAAYeNyuxfvplZBVq9bdu2cTU1NY1ZDOeWlhZt27ZtMLC6o22KGcz3A1+T9KmIuD5dtm/652LgU5Luj4g3JI0GtgAPAJ+QdDNwEPBe4AvAmDw/cwnwGeAbAJImRMQq4GVgUJ7HuA/4FPDt9FT2fhHxUp77mplZCTU1NX1869atN27dunU82XwkuAVY3dTU9PGONihaMEdESLoQuEbSF4FtJL3UvwduB2qBlend29uAC4E7gZOAx4AAvhgRWyXlG8yfA74v6XGStj1AcuPX3cBCSReQXI/uzOXAgvQ6dDNJSD+c5+ebmVkJTZw48QVK8/RPwSgicz39PmXAYaPisBnfLnUZViQe9rFrHvZx71TqsI+SVkREXYHKKStZ7OabmZlVLAezmZlZhjiYzczMMsTBbGZmliEOZjMzswzxXdl7qa6uLurr60tdhplZn+K7sjvmHrOZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwypDqfjSQdCLwzd/uIWFmsoszMzCpVl8Es6V+BmcAfgNYXawdwevHKMjMzq0z59Jg/DIyMiN3FLsbMzKzS5XONeTVwQJHrMDMzM/LrMc8DHpW0GtjVujAizi9aVWZmZhUqn2C+Gfg60AC0FLccMzOzypZPMG+PiGuLXomZmZnlFcwrJM0DFrHnqWw/LmVmZlZg+QTze9I/T8xZ5selzMzMiqDLYI6I03qjEDMzM8vvBSMDgClALXu++etfileWmZlZZcrnVPZdQCOwgpxrzJZo2NJI7Zx7Sl2GZcjGgdNLXULFOWrEsFKXkHm3zWsq6PHGrltb0OPZm/IJ5iMi4qyiV2JmZmZ5vfnrt5KOKnolZmZm1nGPWVIDyd3X1cDfSnqG5FS2gIiIo3unRDMzs8rR2ansc3utCjMzMwM6CeaIeBZA0k8i4m9y10n6CfA37e5oZmZmPZbPNeYjc2ck9QMmFqccMzOzytZhMEv6kqSXgaMlvZR+vQy8QPIIlZmZmRVYh8EcEfMi4u3ANyJiUPr19og4OCK+1Is1dkjShySFpDE5y46XtEzSU5JWSrqno7vKJZ0vaU4H63YWq24zM7OOdHZX9piIWAfcLunYtuszMojFNOBB4FJgrqShwG3A9Ij4LYCkU4GRJMNW/oWk6ohYRDI4h5mZWSZ0dlf2lcAs4Op21pV8EAtJ+wOnAKeRhOtc4DPAza2hDBARD+bscxOwg2RgjpXpI2F1EfEZSSOAn5F8T+7tpWaYmZntobO7smdJqgL+MSIe6sWa8nUhcG9EPClpR9qrPxK4uYv9RgNnRkSzpJk5y78DXB8RP5b06c4OIGkWyS8t9BtU09P6zczM3qLTu7IjogX4Zi/V0l3TgFvT6VvT+T1I+p2ktZK+k7P49ohobud4pwC3pNM/6eyDI2JBRNRFRF2/fQf3oHQzM7P25fOu7CWSpgB3REQUu6B8SDqY5FT6eEkB9CM5vX4zcCzpXeMRcYKki9nzZSmvdHLoTLTPzMwqVz7PMV8J3A7sbn1kStJLRa6rKxcDP46I4RFRGxHvBDYAS4CZkk7O2XbfPI/5EMlNZACXFa5UMzOz/HXZY04fmcqaacD8Nst+DkwHpgJfl3Q4yTPX24F8xo6+HPiZpMvTY5mZmfW6fE5lI+l84L3p7LKI+GXxSupaRExuZ9m1ObOTOthvZpv5m4Cb0ukNwEk5q9sGv5mZWdF1eSpb0nyS3uQT6dfl6TIzMzMrsHx6zB8EJqR3aCPpZuBRoN03ZpmZmVnP5XPzF8ABOdN+PsjMzKxI8ukxzwMelbQUEMm15ky8K9vMzKzc5HNX9i2SlgHHkQTz30fE1mIXZmZmVonU1TtD2hvAAmgEno2IpqJU1YfU1dVFfX19qcswM+tTJK2IiLpS15FF+ZzKvo7kbVqPk/SYx6fTB0v6ZEQsKWJ9ZmZmFSWfm782Au9J3w09kWRkptXAmcC/F7E2MzOzipNPMI+JiDWtMxHxBElQP1O8sszMzCpTPqey10u6njdHcpoKPClpAPBG0SozMzOrQPn0mGcCTwNXALOBZ9JlbwCnFakuMzOzipTP41KvAVenX23tLHhFZmZmFazDYJbUQMfjE0dEHFOckszMzCpXZz3mc9tZJuAI4MvFKcfMzKyydRjMEfFs67SkCSRjHX8Y2IDHKzYzMyuKzk5ljwYuBaYB/w/4T5I3hfmGLzMzsyLp7FT2OuA3wHkR8TSApNm9UpWZmVmF6uxxqSnAVmCppP8j6QySa8xmZmZWJB0Gc0TcGRFTgTHAMpJnmIdKul7S+3upPjMzs4rS5QtGIuKViPhpRJxLckf2KmBOsQszMzOrRPm8+esvImJHRPwgIk4vVkFmZmaVrFvBbGZmZsXlYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDOlyPGbrXMOWRmrn3FPqMqyMbRw4vdQlZMJRI4aVugQAbpvXVOoSMmHsurWlLqFsucdsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8uQsg5mSR+SFJLGpPO1kla32WaupKvS6ZskbZC0StJKSSeVom4zM6tcZR3MwDTgQeDSbuzzhYiYQDLm9A+KUZSZmVlHyjaYJe0PnAJ8jO4Fc6sHgHcXtCgzM7MulPObvy4E7o2IJyXtkHQssAMYKWlVznaHAt9sZ//zgIaiV2lmZpajbHvMJKexb02nb03nAf4QERNav4Ab2uz3jTS4Z5H0tt9C0ixJ9ZLqm19tLHzlZmZWscqyxyzpYOB0YLykAPoBAVyXx+5fiIiFnW0QEQuABQADDhsVe1mumZnZX5Rrj/li4McRMTwiaiPincAG4IgS12VmZtapcg3macCdbZb9HPhyCWoxMzPLW1meyo6Iye0suxa4tp3lc3OmZxazLjMzs66Ua4/ZzMysT3Iwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQRfj9GHujrq4u6uvrS12GmVmfImlFRNSVuo4sco/ZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDKkutQF9HUNWxqpnXNPqcvIhI0Dp5e6BCtjR40YVuoSiuK2eU2lLqFHxq5bW+oSypZ7zGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZUifC2ZJzZJWSVoj6TFJV0qqarPNXZIezpm/RtIVOfOLJd2YM3+1pCvT6WpJ2yXN64XmmJmZ7aHPBTPwWkRMiIgjgfcBHwS+0rpS0gHAscABkkaki38LnJyurwKGAEfmHPNk4KF0+v3AeuDDklTEdpiZmb1FXwzmv4iIF4BZwGdyQnQKcDdwK3Bpuuwh0mAmCeTVwMuSDpQ0ABgLPJqunwZ8B9gEnFj0RpiZmeXo08EMEBHPkLTjkHTRNOCW9Gtaus3zQJOkYSQB/TDwO+AkoA54PCJ2S3obcAbwy9z925I0S1K9pPrmVxuL1jYzM6s8fT6YUwKQNBR4N/BgRDxJEsbj021ae82twfxwzvxv023OBZZGxKvAz4EPSerX9sMiYkFE1EVEXb99BxexWWZmVmn6fDBLehfQDLwATAUOBDZI2gjU8ubp7NbrzEeRnMp+hKTHnHt9eRpwZrrvCuBg4LReaIaZmRnQx4NZUg1wA/C9iAiSYD0rImojohaYyJ7Xmc8FdkREc0TsAA4gCeeHJQ0CTgWG5ez/aTo4nW1mZlYMfTGY39b6uBTwP8AS4KuSaoFhJD1hACJiA/CSpBOABpK7sR/JOVYD0BgR24GLgPsjYlfO+ruA89MbxMzMzIquz43HHBFvueab2ggc3s72x+bMDmqzbmbO9E3ATW3W7wBqelSomZlZD/TFHrOZmVnZcjCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAlj/9aT9XV1UV9fX2pyzAz61MkrYiIulLXkUXuMZuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxD/ErOvSTpZWB9qesooSHA9lIXUUJuf+W2v5LbDnvf/uERUVOoYspJdakLKAPrK/l9r5Lq3X63v9R1lEIltx3c/mLyqWwzM7MMcTCbmZlliIN57y0odQEl5vZXtkpufyW3Hdz+ovHNX2ZmZhniHrOZmVmGOJjNzMwyxMG8FySdJWm9pKclzSl1PcUk6Z2SlkpaK2mNpMvT5QdJ+pWkp9I/Dyx1rcUkqZ+kRyX9Mp2vmPZLOkDSQknr0n8HJ1VY+2en//ZXS7pF0sBybr+kH0l6QdLqnGUdtlfSl9KfheslfaA0VZcHB3MPSeoHfB84GxgHTJM0rrRVFVUT8PmIGAucCHw6be8c4L6IGAXcl86Xs8uBtTnzldT+7wD3RsQY4BiS70NFtF/S4cDngLqIGA/0Ay6lvNt/E3BWm2Xttjf9WXApcGS6z3Xpz0jrAQdzzx0PPB0Rz0TEbuBW4IIS11Q0EfHHiFiZTr9M8kP5cJI235xudjNwYUkK7AWSjgDOAW7MWVwR7Zc0CHgv8EOAiNgdEX+mQtqfqgbeJqka2Bd4njJuf0Q8AOxos7ij9l4A3BoRuyJiA/A0yc9I6wEHc88dDmzOmX8uXVb2JNUC7wF+BwyNiD9CEt7AISUsrdi+DXwRaMlZVintfxewDfiP9FT+jZL2o0LaHxFbgG8Cm4A/Ao0RsYQKaX+OjtpbsT8Pi8HB3HNqZ1nZP3smaX/g58AVEfFSqevpLZLOBV6IiBWlrqVEqoFjgesj4j3AK5TXadtOpddSLwBGAO8A9pP0kdJWlSkV+fOwWBzMPfcc8M6c+SNITm2VLUn9SUL5pxFxR7r4T5IOS9cfBrxQqvqK7BTgfEkbSS5bnC7p/1I57X8OeC4ifpfOLyQJ6kpp/5nAhojYFhFvAHcAJ1M57W/VUXsr7udhMTmYe245MErSCEn7kNz4sKjENRWNJJFcX1wbEd/KWbUImJFOzwDu6u3aekNEfCkijoiIWpK/6/sj4iNUTvu3Apsl/VW66AzgCSqk/SSnsE+UtG/6f+EMkvssKqX9rTpq7yLgUkkDJI0ARgG/L0F9ZcFv/toLkj5Ict2xH/CjiPi30lZUPJJOBX4DNPDmNdYvk1xnvg0YRvLD65KIaHvDSFmRNBm4KiLOlXQwFdJ+SRNIbnzbB3gG+FuSX+4rpf1fBaaSPKHwKPBxYH/KtP2SbgEmkwzv+CfgK8Av6KC9kv4B+CjJ9+eKiPjv3q+6PDiYzczMMsSnss3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZp2QdKikWyX9QdITkv5L0ugif+YySXVdbHOFpH1z5v9L0gEF+OyNkhokPSZpiaRD9/aYPahhQvooollFcjCbdSB9kcSdwLKIGBkR40ie3R5a2soAuIJkIAUAIuKD6aAShXBaRBwD1JO0t0sFHkloAuBgtorlYDbr2GnAGxFxQ+uCiFgVEb+RNLl1TGYASd+TNDOd3ijpa5IellQv6VhJi9Ne9yfTbTrcP5ek69NjrElfcIGkz5G8r3mppKU5nzlE0tcl/e+c/edK+nw6/QVJyyU93nqsLjwAvFvJGNTfyNn3EzltWCrpZ0BDut030x7345I+m243UdKvJa1Ivw+tr3Rcltb7e0lPSvrr9C16/wJMlbRK0tQ86jQrK9WlLsAsw8YDPR20YnNEnCTpGpJxbU8BBgJrgBs627GNf4iIHWmP9D5JR0fEtZKuJOnZbm+z/a0kb6O7Lp3/MHCWpPeTvCbxeJIBBxZJem86tF9HziV509vHSEZTOk7SAOAhSUvSbY4HxkfEBkmfIhnk4T0R0STpoPT96t8FLoiIbWnQ/hvJG6IAqiPi+PTU9Vci4kxJ/0wy7vFnuvF9MisbDmaz4mh9b3oDsH86hvXLkl7v5rXgD0uaRfJ/9TBgHPB4RxtHxKOSDpH0DqAGeDEiNqW97PeTvEoSkldJjiLpFbe1VFJz+jn/SPIazqMlXZyuH5zuuxv4fTr+LiQDPdwQEU1pLTskjSf5BedXyZUB+pEMm9iqdTCUFUBtHt8Ps7LnYDbr2Brg4g7WNbHnpaCBbdbvSv9syZluna/OY3/SwQCuAo6LiBcl3dTedu1YmNZ9KEkPGpJe8ryI+EEe++/RE0+vtX82Iha3qW8yyfCPf1nEW4f6E7AmIk7q4LNavzfN+OeRGeBrzGaduR8YIOnvWhdIOk7SJOBZYFw6ms5gktGGuiOf/QeRBF+jpKHA2TnrXgbe3sGxbyUZAetikpAGWAx8VMl42kg6XNIhHezf1mLgU+lpaSSNlrRfO9stAT4pqTrd7iBgPVAj6aR0WX9JR3bxeZ21zazsOZjNOhDJCC8fAt6X3ri1BpgLPB8Rm0lG2Xkc+ClvniLO99hd7h8Rj6XL1wA/Ah7KWb0A+O/Wm7/a7LeGJNi2RMQf02VLgJ8BD0tqIAnsfMPvRpIhHldKWg38gPZ7tzeSjDj0uKTHgOkRsZvkF4Svp8tWkYxj3JmlJL+0+OYvq0geXcrMzCxD3GM2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxD/j/B5ryLBFNDnwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hsRc5rJJGmcW"
   },
   "source": [
    "#### Different Privacy Levels\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1COY3knYbJq5"
   },
   "source": [
    "##### Epsilon = .5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "id": "UTDGrD7bfOex"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, '_0.5_')\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9squYfUff9o-",
    "outputId": "589b3c39-fd25-4268-bf65-0290d5e7458a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=16.867, p=0.002\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "TqaQISS3gT-G",
    "outputId": "7dafd5c0-1cf0-423c-e347-30572b2f603b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.01535</td>\n",
       "      <td>0.13403</td>\n",
       "      <td>0.37330</td>\n",
       "      <td>0.81896</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.01535</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.32047</td>\n",
       "      <td>0.11331</td>\n",
       "      <td>0.01535</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.13403</td>\n",
       "      <td>0.32047</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.46240</td>\n",
       "      <td>0.11331</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.37330</td>\n",
       "      <td>0.11331</td>\n",
       "      <td>0.46240</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.32047</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.81896</td>\n",
       "      <td>0.01535</td>\n",
       "      <td>0.11331</td>\n",
       "      <td>0.32047</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.01535 0.13403    0.37330  0.81896\n",
       "AHP       0.01535 1.00000 0.32047    0.11331  0.01535\n",
       "DAWA      0.13403 0.32047 1.00000    0.46240  0.11331\n",
       "Geometric 0.37330 0.11331 0.46240    1.00000  0.32047\n",
       "Laplace   0.81896 0.01535 0.11331    0.32047  1.00000"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "Zvo7GUzTgYc9",
    "outputId": "4a11c949-5981-431a-c9fa-19bf98d4b7ad"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcWElEQVR4nO3de5hddX3v8fcnCUIRiQIxigqhFAoIyCWIqJWL1CsXURDQ9gQvpXrUItZa7OU0tk8PeNR6tNYLB1uop0KrVoXaChZBRUVIuE0AEQQERSSc2AiCxCTf88dao5sxk0yS2bPXzH6/nmeeWff9/Q1hPvNbt1+qCkmS1A2zBl2AJEn6JYNZkqQOMZglSeoQg1mSpA4xmCVJ6pA5gy5gutthhx1qwYIFgy5DkqaVpUuX3l9V8wZdRxcZzJtpwYIFLFmyZNBlSNK0kuR7g66hqzyVLUlShxjMkiR1iMEsSVKHeI1ZkjRjLF269Ilz5sw5B9ibbnY+1wLLVq9e/foDDzzwvnVtYDBLkmaMOXPmnPOkJz1pz3nz5v141qxZnRsMYu3atVm+fPle99577znAMevapot/TUiStKn2njdv3k+6GMoAs2bNqnnz5q2k6dGve5sprEeSpH6b1dVQHtXWN27+GsySJHWI15glSTPWgjO+cOBkHu/Os166dEPbnHDCCQsuvfTSudtvv/3qW2+99caN/QyDeTON/GAlC874wqDLmFR3bvWqQZcg9d0+u+w06BKmtZFFI4MuobNe+9rX3n/aaafd95rXvGaXTdnfU9mSJE2iF7/4xQ/Omzdv9abubzBLktQhBrMkSR1iMEuS1CEGsyRJHeJd2ZKkGWsijzdNtqOPPnqXK6+88nE//vGP58yfP3/fM844457TTz/9/onubzBLkjSJLrroojs2Z//OncpO8uAkHGNBkmWTUY8kSVOpc8EsSdIwmxbBnOToJN9Kcm2S/0wyv12+OMknknw5ya1Jfm8d+y5I8rUk17Rfz+5Z944kI0muT3JWu2zXJF9MsrTdb4+pa6kkadhNl2vMVwDPqqpK8nrgHcAftuv2BZ4FPBa4NsnY92PeB/x2Vf0syW7A+cDCJC8GXgYcXFUPJdmu3f5s4A1VdWuSg4EPA0f0HjDJqcCpALO3nTfJTZUkDbPpEsxPBf45yZOBxwC9F9Y/X1UPAw8nuQx4JnBdz/otgA8l2Q9YA+zeLj8S+IeqegigqlYk2QZ4NvCpJKP7bzm2mKo6mybA2fLJu3V6eDFJ0vQyXYL5b4G/qaoLkxwGLO5ZNzYYx86fDvwIeAbNqfuftcuzjm1nAf9VVfttdsWSJG2C6RLMc4EftNOLxqw7NsmZNKeyDwPOoOlV9+77/apam2QRMLtdfgnwP5J8cvRUdttrviPJCVX1qTTd5n2r6vp+NUyS1EeL507qsI8sXrne56Jvu+22LV796lfvsnz58i1mzZrFokWLlv/5n//5fRvzEV28+WvrJN/v+XobTQ/5U0m+Box9SPsq4AvAlcBfVdU9Y9Z/GFiU5Eqa09g/BaiqLwIXAkuSXAe8vd3+1cDrklwP3AgcO9kNlCTNTFtssQXve9/7vn/77bffePXVV9/88Y9//IlLly7damOO0bkec1WN98fC58dZ/p2qOnXMMe4E9m6nb6W5QWzUO3u2Ows4a8y+dwAv2riqJUmCnXfe+ec777zzzwGe8IQnrN11110fvuuuux5z4IEH/mxD+47qYo9ZkqRp75ZbbnnMTTfdtPWhhx66US/O6lyPeWNU1eJB1yBJ0lgrV66c9fKXv3zXs8466+7ttttu7cbsa49ZkqRJ9Mgjj+SlL33prieccMKKRYsW/dfG7m8wS5I0SdauXctJJ5208+677/6zxYsX/2hTjjGtT2VLkrReG3i8abJ96Utf2uZzn/vc9rvtttvDe+yxx14A73rXu35w4oknrpzoMQzmzbTPU+ay5KyXDrqMSTbhfz/StDUy6AI0I73whS98sKo2648BT2VLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkd4uNSkqQZa5/z9pnUYR9HFo2s91Gohx56KAcffPAeq1atypo1a3L00Uf/+P3vf//YUQ/Xy2CWJGmSbLXVVnXFFVfcMnfu3LWPPPJIDjrooN+89NJLVz7/+c//6USP4alsSZImyaxZs5g7d+5agFWrVmX16tVJsnHH6EtlkiQNqdWrV7PHHnvsNX/+/GcceuihPzniiCMm3FsGg1mSpEk1Z84cvv3tb99011133XDNNdc89uqrr95qY/Y3mCVJ6oMddthhzXOf+9wHLrroorkbs5/BLEnSJLnnnnvm3H///bMBHnzwwVx++eXb7rnnnj/bmGN4V7Ykacba0ONNk+3uu+/e4pRTTtllzZo1VFWOPfbYFSeffPJGDdlnMEuSNEkOPvjgh2+++eabNucYnsqWJKlDDGZJkjrEYJYkzSRr165du3Fv9JhibX1rx1tvMEuSZpJly5cvn9vVcF67dm2WL18+F1g23jbe/CVJmjFWr179+nvvvfece++9d2+62flcCyxbvXr168fbwGCWJM0YBx544H3AMYOuY3MYzJvrnmth8Ua91EWadvbZZadBl6COGVk0MugSZqwudvMlSRpaBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkd0tdgTjI/ySeT3J5kaZJvJjmun585Th2HJXn2etYfk+SMqaxJkqR16dtzzEkCfA44r6pe1S7bmcE8+H0Y8CDwjbErksypqguBC6e6KEmSxupnj/kIYFVVfXR0QVV9r6r+NsnsJO9JcnWSG5L8PjRh3i5flmQkyYnt8sOSfCXJvyT5TpKzkrw6yVXtdru2281L8pn2uFcneU6SBcAbgNOTXJfkt5Kcm+RvklwGvDvJKUk+1B5jfpLPJrm+/Rq3py1J0mTr55u/ng5cM8661wErq+qgJFsCX09yCXAAsB/wDGAH4OokX233eQawJ7ACuB04p6qemeQ04C3AW4EPAO+vqiuS7ARcXFV7Jvko8GBVvRcgyeuA3YEjq2pNklN6avsg8JWqOi7JbGCbyfhhSJI0EVP2Ss4kfwc8F1gFfA/YN8nx7eq5wG7t+vOrag3woyRfAQ4CfgJcXVU/bI/1XeCSdt8R4PB2+khgr+YsOgDbJnncOCV9qv2csY4A/htAu37lOtpyKnAqwE5zOzmAiSRpmupnMN8IvGJ0pqrelGQHYAlwF/CWqrq4d4ckL1nP8R7pmV7bM7+WX7ZjFnBIVT085rjrOt5PJ9CGdaqqs4GzARbuOLs29TiSJI3Vz2vMXwa2SvLGnmVbt98vBt6YZAuAJLsneSzwVeDE9hr0POB5wFUb8ZmXAG8enUmyXzv5ADBez3msS4E3tvvPTrLtRny+JEmbpW/BXFUFvAw4NMkdSa4CzgP+GDgHuAm4Jsky4GM0vd7PAjcA19ME+zuq6t6N+Ng/ABa2N5TdRHPTF8BFwHGjN39t4BinAYcnGQGW0lwrlyRpSqTJT22qhTvOriWnen+YZjaHfdRYmzvsY5KlVbVwksqZUXzzlyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR0yZW/+mrF23B8WLxl0FVJfbd79t5I2hj1mSZI6xGCWJKlDDGZJkjrEYJYkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUMm9ErOJE8Anta7fVVd06+iJEkaVhsM5iR/BZwCfBeodnEBR/SvLEmShtNEesyvBHatqlX9LkaSpGE3kWvMy4DH97kOSZLExHrMZwLXJlkGPDK6sKqO6VtVkiQNqYkE83nAu2mGZF3b33IkSRpuEwnm+6vqg32vRJIkTSiYlyY5E7iQR5/K9nEpSZIm2USCef/2+7N6lvm4lCRJfbDBYK6qw6eiEEmSNLEXjGwJvAJYwKPf/PWX/StLkqThNJFT2Z8HVgJL6bnGrMbID1ay4IwvTMln3bnVq6bkc6RhsM8uOw26hGltZNHIoEuYsSYSzE+tqhf1vRJJkjShN399I8k+fa9EkiSN32NOMkJz9/Uc4DVJbqc5lR2gqmrfqSlRkqThsb5T2UdNWRWSJAlYTzBX1fcAknyiqn63d12STwC/u84dJUnSJpvINean984kmQ0c2J9yJEkabuMGc5J3JnkA2DfJT9qvB4D7aB6hkiRJk2zcYK6qM6vqccB7qmrb9utxVbV9Vb1zCmscV5LjklSSPXqWPTPJ5UluTXJNki+Md1d5kmOSnDHOugf7VbckSeNZ313Ze1TVt4FPJTlg7PqODGJxMnAFcBKwOMl84F+AV1XVNwCSPBfYlWbYyl9IMqeqLqQZnEOSpE5Y313ZbwNOBd63jnUDH8QiyTbAc4DDacJ1MfBm4LzRUAaoqit69jkXWEEzMMc17SNhC6vqzUl2AT5J8zP54hQ1Q5KkR1nfXdmnJpkF/FlVfX0Ka5qolwFfrKrvJFnR9uqfDpy3gf12B46sqjVJTulZ/gHgI1X1j0netL4DJDmV5o8WZm87b1PrlyTpV6z3ruyqWgu8d4pq2VgnAxe00xe084+S5FtJbk7ygZ7Fn6qqNes43nOA89vpT6zvg6vq7KpaWFULZ289dxNKlyRp3SbyruxLkrwC+Neqqn4XNBFJtqc5lb53kgJm05xePw84gPau8ao6OMnxPPplKT9dz6E70T5J0vCayHPMbwM+BawafWQqyU/6XNeGHA/8Y1XtXFULquppwB3AJcApSZ7ds+3WEzzm12luIgN49eSVKknSxG2wx9w+MtU1JwNnjVn2GeBVwInAu5M8heaZ6/uBiYwdfRrwySSntceSJGnKTeRUNkmOAZ7Xzl5eVf/Wv5I2rKoOW8eyD/bMHjrOfqeMmT8XOLedvgM4pGf12OCXJKnvNngqO8lZNL3Jm9qv09plkiRpkk2kx/wSYL/2Dm2SnAdcC6zzjVmSJGnTTeTmL4DH90z7fJAkSX0ykR7zmcC1SS4DQnOtuRPvypYkaaaZyF3Z5ye5HDiIJpj/uKru7XdhkiQNo2zonSHrGsACWAl8r6pW96WqaWThwoW1ZMmSQZchSdNKkqVVtXDQdXTRRE5lf5jmbVo30PSY926nt0/yhqq6pI/1SZI0VCZy89edwP7tu6EPpBmZaRlwJPC/+libJElDZyLBvEdV3Tg6U1U30QT17f0rS5Kk4TSRU9m3JPkIvxzJ6UTgO0m2BH7et8okSRpCE+kxnwLcBrwVOB24vV32c+DwPtUlSdJQmsjjUg8D72u/xnpw0iuSJGmIjRvMSUYYf3ziqqpn9KckSZKG1/p6zEetY1mApwJ/0p9yJEkabuMGc1V9b3Q6yX40Yx2/ErgDxyuWJKkv1ncqe3fgJOBk4P8B/0zzpjBv+JIkqU/Wdyr728DXgKOr6jaAJKdPSVWSJA2p9T0u9QrgXuCyJP8nyfNprjFLkqQ+GTeYq+qzVXUisAdwOc0zzPOTfCTJC6aoPkmShsoGXzBSVT+tqn+qqqNo7si+Djij34VJkjSMJvLmr1+oqhVV9bGqOqJfBUmSNMw2KpglSVJ/GcySJHWIwSxJUocYzJIkdYjBLElShxjMkiR1yAbHY9YG3HMtLJ476Com1T677DToEiR13MiikUGXMGPZY5YkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjpkRgdzkuOSVJI92vkFSZaN2WZxkre30+cmuSPJdUmuSXLIIOqWJA2vGR3MwMnAFcBJG7HPH1XVfjRjTn+sH0VJkjSeGRvMSbYBngO8jo0L5lFfBX5jUouSJGkDZvKbv14GfLGqvpNkRZIDgBXArkmu69nuScB717H/0YCvtpEkTakZ22OmOY19QTt9QTsP8N2q2m/0C/jomP3e0wb3qTS97V+R5NQkS5IsWf5QTX7lkqShNSN7zEm2B44A9k5SwGyggA9PYPc/qqpPr2+DqjobOBtg4Y6zTWZJ0qSZqT3m44F/rKqdq2pBVT0NuAN46oDrkiRpvWZqMJ8MfHbMss8AfzKAWiRJmrAZeSq7qg5bx7IPAh9cx/LFPdOn9LMuSZI2ZKb2mCVJmpYMZkmSOsRgliSpQwxmSZI6xGCWJKlDZuRd2VNqx/1h8ZJBVzGpfA+pJA2OPWZJkjrEYJYkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUMMZkmSOmTOoAuY9u65FhbPHXQV6pB9dtlp0CVIfTeyaGTQJcxY9pglSeoQg1mSpA4xmCVJ6hCDWZKkDjGYJUnqEINZkqQOMZglSeqQaRfMSdYkuS7JjUmuT/K2JLPGbPP5JN/smX9/krf2zF+c5Jye+fcleVs7PSfJ/UnOnILmSJL0KNMumIGHq2q/qno68NvAS4C/GF2Z5PHAAcDjk+zSLv4G8Ox2/SxgB+DpPcd8NvD1dvoFwC3AK5Okj+2QJOlXTMdg/oWqug84FXhzT4i+ArgIuAA4qV32ddpgpgnkZcADSZ6QZEtgT+Dadv3JwAeAu4Bn9b0RkiT1mNbBDFBVt9O044ntopOB89uvk9tt7gFWJ9mJJqC/CXwLOARYCNxQVauS/BrwfODfevcfK8mpSZYkWbL8oepb2yRJw2faB3MrAEnmA78BXFFV36EJ473bbUZ7zaPB/M2e+W+02xwFXFZVDwGfAY5LMnvsh1XV2VW1sKoWztvas92SpMkz7YM5ya8Da4D7gBOBJwB3JLkTWMAvT2ePXmfeh+ZU9pU0Pebe68snA0e2+y4FtgcOn4JmSJIETPNgTjIP+CjwoaoqmmB9UVUtqKoFwIE8+jrzUcCKqlpTVSuAx9OE8zeTbAs8F9ipZ/83Mc7pbEmS+mE6BvOvjT4uBfwncAnwriQLgJ1oesIAVNUdwE+SHAyM0NyNfWXPsUaAlVV1P/By4MtV9UjP+s8Dx7Q3iEmS1HfTbjzmqvqVa76tO4GnrGP7A3pmtx2z7pSe6XOBc8esXwHM26RCJUnaBNOxxyxJ0oxlMEuS1CEGsyRJHWIwS5LUIQazJEkdMu3uyu6cHfeHxUsGXYU6ZGTQBUia1uwxS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR2Sqhp0DdNakgeAWwZdxwDtANw/6CIGyPYPb/uHue2w+e3fuarmTVYxM4nvyt58t1TVwkEXMShJlth+2z/oOgZhmNsOtr+fPJUtSVKHGMySJHWIwbz5zh50AQNm+4fbMLd/mNsOtr9vvPlLkqQOsccsSVKHGMySJHWIwbwZkrwoyS1JbktyxqDr6ackT0tyWZKbk9yY5LR2+XZJvpTk1vb7EwZdaz8lmZ3k2iT/1s4PTfuTPD7Jp5N8u/13cMiQtf/09t/+siTnJ9lqJrc/yd8nuS/Jsp5l47Y3yTvb34W3JHnhYKqeGQzmTZRkNvB3wIuBvYCTk+w12Kr6ajXwh1W1J/As4E1te88ALq2q3YBL2/mZ7DTg5p75YWr/B4AvVtUewDNofg5D0f4kTwH+AFhYVXsDs4GTmNntPxd40Zhl62xv+7vgJODp7T4fbn9HahMYzJvumcBtVXV7Va0CLgCOHXBNfVNVP6yqa9rpB2h+KT+Fps3ntZudB7xsIAVOgSRPBV4KnNOzeCjan2Rb4HnAxwGqalVV/RdD0v7WHODXkswBtgbuYQa3v6q+CqwYs3i89h4LXFBVj1TVHcBtNL8jtQkM5k33FODunvnvt8tmvCQLgP2BbwHzq+qH0IQ38MQBltZv/xt4B7C2Z9mwtP/XgeXAP7Sn8s9J8liGpP1V9QPgvcBdwA+BlVV1CUPS/h7jtXdofx/2g8G86bKOZTP+2bMk2wCfAd5aVT8ZdD1TJclRwH1VtXTQtQzIHOAA4CNVtT/wU2bWadv1aq+lHgvsAuwIPDbJ7wy2qk4Zyt+H/WIwb7rvA0/rmX8qzamtGSvJFjSh/E9V9a/t4h8leXK7/snAfYOqr8+eAxyT5E6ayxZHJPm/DE/7vw98v6q+1c5/miaoh6X9RwJ3VNXyqvo58K/Asxme9o8ar71D9/uwnwzmTXc1sFuSXZI8hubGhwsHXFPfJAnN9cWbq+pvelZdCCxqpxcBn5/q2qZCVb2zqp5aVQto/lt/uap+h+Fp/73A3Ul+s130fOAmhqT9NKewn5Vk6/b/hefT3GcxLO0fNV57LwROSrJlkl2A3YCrBlDfjOCbvzZDkpfQXHecDfx9Vf31YCvqnyTPBb4GjPDLa6x/QnOd+V+AnWh+eZ1QVWNvGJlRkhwGvL2qjkqyPUPS/iT70dz49hjgduA1NH/cD0v73wWcSPOEwrXA64FtmKHtT3I+cBjN8I4/Av4C+BzjtDfJnwKvpfn5vLWq/mPqq54ZDGZJkjrEU9mSJHWIwSxJUocYzJIkdYjBLElShxjMkiR1iMEsrUeSJyW5IMl3k9yU5N+T7N7nz7w8ycINbPPWJFv3zP97ksdPwmffmWQkyfVJLknypM095ibUsF/7KKI0lAxmaRztiyQ+C1xeVbtW1V40z27PH2xlALyVZiAFAKrqJe2gEpPh8Kp6BrCEpr0bNMkjCe0HGMwaWgazNL7DgZ9X1UdHF1TVdVX1tSSHjY7JDJDkQ0lOaafvTPI/k3wzyZIkByS5uO11v6HdZtz9eyX5SHuMG9sXXJDkD2je13xZkst6PnOHJO9O8t979l+c5A/b6T9KcnWSG0aPtQFfBX4jzRjU7+nZ9/d72nBZkk8CI+1272173DckeUu73YFJvpJkaftzGH2l4+VtvVcl+U6S32rfoveXwIlJrkty4gTqlGaUOYMuQOqwvYFNHbTi7qo6JMn7aca1fQ6wFXAj8NH17TjGn1bVirZHemmSfavqg0neRtOzvX/M9hfQvI3uw+38K4EXJXkBzWsSn0kz4MCFSZ7XDu03nqNo3vT2OprRlA5KsiXw9SSXtNs8E9i7qu5I8kaaQR72r6rVSbZr36/+t8CxVbW8Ddq/pnlDFMCcqnpme+r6L6rqyCT/g2bc4zdvxM9JmjEMZqk/Rt+bPgJs045h/UCSn23kteBXJjmV5v/VJwN7ATeMt3FVXZvkiUl2BOYBP66qu9pe9gtoXiUJzaskd6PpFY91WZI17ef8Gc1rOPdNcny7fm677yrgqnb8XWgGevhoVa1ua1mRZG+aP3C+1FwZYDbNsImjRgdDWQosmMDPQ5rxDGZpfDcCx4+zbjWPvhS01Zj1j7Tf1/ZMj87PmcD+tIMBvB04qKp+nOTcdW23Dp9u634STQ8aml7ymVX1sQns/6ieeHut/S1VdfGY+g6jGf7xF4v41aH+AtxYVYeM81mjP5s1+PtIArzGLK3Pl4Etk/ze6IIkByU5FPgesFc7ms5cmtGGNsZE9t+WJvhWJpkPvLhn3QPA48Y59gU0I2AdTxPSABcDr00znjZJnpLkiePsP9bFwBvb09Ik2T3JY9ex3SXAG5LMabfbDrgFmJfkkHbZFkmevoHPW1/bpBnPYJbGUc0IL8cBv93euHUjsBi4p6ruphll5wbgn/jlKeKJHnuD+1fV9e3yG4G/B77es/ps4D9Gb/4as9+NNMH2g6r6YbvsEuCTwDeTjNAE9kTD7xyaIR6vSbIM+Bjr7t2eQzPi0A1JrgdeVVWraP5AeHe77DqacYzX5zKaP1q8+UtDydGlJEnqEHvMkiR1iMEsSVKHGMySJHWIwSxJUocYzJIkdYjBLElShxjMkiR1yP8HZkoVX1QludAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RYAHfIcugy7X"
   },
   "source": [
    "##### Epsilon .1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "id": "75poyMBPg2as"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, '_0.1_')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "gcO6lbMHg6Gs",
    "outputId": "f52505ff-160e-4598-890d-6833167ad470"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=18.503, p=0.001\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "CCGONrf-g8BM",
    "outputId": "f3c8a157-5dd1-47aa-b6ff-068b5a6bb80c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.37908</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.37908</td>\n",
       "      <td>0.03017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.37908</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.37908</td>\n",
       "      <td>0.14311</td>\n",
       "      <td>0.18067</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.37908</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.37908</td>\n",
       "      <td>0.03017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.37908</td>\n",
       "      <td>0.14311</td>\n",
       "      <td>0.37908</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.00556</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.03017</td>\n",
       "      <td>0.18067</td>\n",
       "      <td>0.03017</td>\n",
       "      <td>0.00556</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.37908 1.00000    0.37908  0.03017\n",
       "AHP       0.37908 1.00000 0.37908    0.14311  0.18067\n",
       "DAWA      1.00000 0.37908 1.00000    0.37908  0.03017\n",
       "Geometric 0.37908 0.14311 0.37908    1.00000  0.00556\n",
       "Laplace   0.03017 0.18067 0.03017    0.00556  1.00000"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "whp4CYtvg9GV",
    "outputId": "16cf02e8-3f5b-446a-8a11-83068810f606"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAddUlEQVR4nO3deZiedX3v8fcnCZACEgRiZBGCFBqQTQgiSmWRKiqLihhAe0LVUj1qFasWu8b2qmCVUq114WAr9VSp4Aa1FSyCiisJW9hBQDaRcIIBZAnJfM8fzz06DJlkJpknzz3zvF/XNdfc+/P9DWE+87u3X6oKSZLUDlN6XYAkSfoNg1mSpBYxmCVJahGDWZKkFjGYJUlqkWm9LmCi22qrrWr27Nm9LkOSJpRFixY9UFUze11HGxnM62j27NksXLiw12VI0oSS5Ge9rqGtPJUtSVKLGMySJLWIwSxJUot4jVmSNGksWrToWdOmTTsL2J12dj4HgGtXrFjxln333ff+VW1gMEuSJo1p06ad9exnP3vXmTNnPjhlypTWDQYxMDCQJUuW7HbfffedBRy1qm3a+NeEJElra/eZM2c+1MZQBpgyZUrNnDlzGZ0e/aq3WY/1SJLUbVPaGsqDmvpGzF+DWZKkFvEasyRp0pp9yjf2Hc/j3XHaqxataZvzzjtvs/e+973bDwwM8MY3vvGBD33oQ/eN5TMM5nW0+J5lzD7lG70uQy1yx/QTel2C1H0LlvW6glZasWIFJ5988vYXXnjhzc997nOf3GuvvXY95phjfrnvvvs+PtpjeCpbkqRxcumll26yww47PLHbbrstnz59er32ta9det55520+lmMYzJIkjZO77rprw2233Xb54Px22223/J577tlwLMcwmCVJGidVT78hPMmY7hI3mCVJGifbb7/9U3rId99994bbbLPNk2M5hsEsSdI4Oeigg351xx13TL/xxhs3fPzxx/OVr3xli2OOOeaXYzmGd2VLkiat0TzeNJ422GADTj/99DsPP/zwXVauXMkJJ5zwwNy5c0d9RzYYzJIkjat58+Ytmzdv3lo/T9a6U9lJHhmHY8xOcu141CNJ0vrUumCWJKmfTYhgTnJkkh8nuTLJ/ySZ1SxfkOTzSb6d5JYkf7iKfWcn+V6SK5qvFw1Z9/4ki5NcneS0ZtlOSb6ZZFGz35z111JJUr+bKNeYLwNeWFWV5C3A+4E/adbtCbwQ2AS4Msnw92PeD/xeVT2eZGfgi8DcJK8AXg3sX1WPJtmi2f5M4K1VdUuS/YFPAocOPWCSk4CTAKZuNnOcmypJ6mcTJZi3A/4jydbAhsDtQ9Z9vaoeAx5LcgnwAuCqIes3AD6RZG9gJbBLs/ww4F+r6lGAqlqaZFPgRcC5SQb332h4MVV1Jp0AZ6Otd2718GKSpIllogTzPwH/UFXnJzkYWDBk3fBgHD5/MvALYC86p+4Hb1vPKradAvyyqvZe54olSVoLEyWYZwD3NNPzh607OsmpdE5lHwycQqdXPXTfu6tqIMl8YGqz/CLgr5J8YfBUdtNrvj3JsVV1bjrd5j2r6upuNUyS1EULZozrsI8sWLbG56KPPfbY2RdffPGMLbfccsUtt9xy3Vg/oo03f22c5O4hX++h00M+N8n3gAeGbf8T4BvAj4C/rap7h63/JDA/yY/onMb+FUBVfRM4H1iY5Crgvc32bwDenORq4Drg6PFuoCRp8nrTm970wPnnn3/L2u7fuh5zVY30x8LXR1h+c1WdNOwYdwC7N9O30LlBbNAHhmx3GnDasH1vBw4fW9WSJHW84hWveOSmm24a04hSQ7WxxyxJUt9qXY95LKpqQa9rkCRpPNljliSpRQxmSZJaZEKfypYkabVG8XjTeDvyyCN3/NGPfvSMBx98cNqsWbP2POWUU+49+eSThz9RNCKDeR3tse0MFp72ql6XoVZZ69HeJE0CF1xwwe1r3mpknsqWJKlFDGZJklrEYJYkqUUMZkmSWsRgliSpRQxmSZJaxMelJEmT1h5n7zGuwz4unr94tc9F33rrrRu84Q1v2HHJkiUbTJkyhfnz5y/5y7/8y/vH8hn2mCVJGicbbLABp59++t233XbbdZdffvkNn/3sZ5+1aNGi6WM5hsEsSdI42WGHHZ488MADHwV45jOfObDTTjs9duedd45pCEiDWZKkLrjppps2vP766zc+6KCDHhnLfgazJEnjbNmyZVNe+9rX7nTaaafdtcUWWwyMZV+DWZKkcfTEE0/kVa961U7HHnvs0vnz5/9yrPsbzJIkjZOBgQGOO+64HXbZZZfHFyxY8Iu1OYaPS0mSJq01Pd403r71rW9t+rWvfW3LnXfe+bE5c+bsBvDBD37wnnnz5o162DmDWZKkcfLyl7/8kapapz8GPJUtSVKLGMySJLWIwSxJUosYzJIktYjBLElSixjMkiS1iI9LraPF9yxj9infWC+fdcf0E9bL52ji2GPH7XtdgkbhS6eu6HUJ427XG2/odQmjcsOcXcd12Mddb7xhtY9CPfroo9l///3nLF++PCtXrsyRRx754BlnnHHvWD7DYJYkaZxMnz69LrvssptmzJgx8MQTT2S//fb7nYsvvnjZS1/60l+N9hieypYkaZxMmTKFGTNmDAAsX748K1asSJKxHaMrlUmS1KdWrFjBnDlzdps1a9ZeBx100EOHHnroqHvLYDBLkjSupk2bxo033nj9nXfeec0VV1yxyeWXXz59LPsbzJIkdcFWW2218sADD3z4ggsumDGW/QxmSZLGyb333jvtgQcemArwyCOP5NJLL91s1113fXwsx+jqXdlJZgFnAC8EHgSWA39fVV/t5ueuoo6DgeVV9YMR1h8F7FZVp63PuiRJ3bWmx5vG21133bXBiSeeuOPKlSupqhx99NFLjz/++FEP+QhdDObmNrSvAWdX1QnNsh2Ao7r1matxMPAI8LRgTjKtqs4Hzl/fRUmSJpf999//sRtuuOH6dTlGN09lH0qnl/rpwQVV9bOq+qckU5N8JMnlSa5J8kfQCfNm+bVJFieZ1yw/OMl3knwpyc1JTkvyhiQ/abbbqdluZpIvN8e9PMmLk8wG3gqcnOSqJL+b5HNJ/iHJJcCHk5yY5BPNMWYl+WqSq5uvF3XxZyRJ0lN081T284ArRlj3ZmBZVe2XZCPg+0kuAvYB9gb2ArYCLk/y3WafvYBdgaXAbcBZVfWCJO8C3gm8G/gYcEZVXZZke+DCqto1yaeBR6rqowBJ3gzsAhxWVSuTnDikto8D36mq1ySZCmw6Hj8MSZJGY729+SvJPwMH0rnO/DNgzySva1bPAHZu1n+xqlYCv0jyHWA/4CHg8qr6eXOsnwIXNfsuBg5ppg8DdhvyMPdmSZ4xQknnNp8z3KHA/wJo1j/t2kCSk4CTAKZuNnPNjZckrS8DAwMDmTJlSvW6kJEMDAwEGBhpfTeD+TrgmMGZqnp7kq2AhcCdwDur6sKhOyR55WqO98SQ6YEh8wP8ph1TgAOq6rFhx13V8cb0wPdQVXUmcCbARlvv3Nr/+JLUh65dsmTJbjNnzlzWxnAeGBjIkiVLZgDXjrRNN4P528CHkrytqj7VLNu4+X4h8LYk366qJ5PsAtwDfBf4oyRnA1sALwHeB8wZ5WdeBLwD+AhAkr2r6irgYWCzUR7jYuBtwD82p7I3qaqHRrmvJKmHVqxY8Zb77rvvrPvuu2932vlI8ABw7YoVK94y0gZdC+aqqiSvBs5I8n5gCZ1e6p8C5wKzgSuau7eXAK8GvgocAFwNFPD+qrovyWiD+Y+Bf05yDZ22fZfOjV8XAOclOZrO9ejVeRdwZnMdeiWdkP7hKD9fktRD++677/305umfcZOq1vX0J5SNtt65tp7/j+vlsxz2UcM57OPE4LCPT5dkUVXNHadyJpU2dvMlSepbBrMkSS1iMEuS1CIGsyRJLWIwS5LUIt6VvY7mzp1bCxcu7HUZkjSheFf2yOwxS5LUIgazJEktYjBLktQiBrMkSS1iMEuS1CIGsyRJLWIwS5LUIgazJEktYjBLktQiBrMkSS0ybTQbJXkm8Jyh21fVFd0qSpKkfrXGYE7yt8CJwE+BwRdrF3Bo98qSJKk/jabH/Hpgp6pa3u1iJEnqd6O5xnwtsHmX65AkSYyux3wqcGWSa4EnBhdW1VFdq0qSpD41mmA+G/gwsBgY6G45kiT1t9EE8wNV9fGuVyJJkkYVzIuSnAqcz1NPZfu4lCRJ42w0wfz85vsLhyzzcSlJkrpgjcFcVYesj0IkSdLoXjCyEXAMMJunvvnrb7pXliRJ/Wk0p7K/DiwDFjHkGrMa914JC2b0ugpJY7THjtv3uoQJbfH8xb0uYdIaTTBvV1WHd70SSZI0qjd//SDJHl2vRJIkjdxjTrKYzt3X04A/SHIbnVPZAaqq9lw/JUqS1D9Wdyr7iPVWhSRJAlYTzFX1M4Akn6+q3x+6Lsnngd9f5Y6SJGmtjeYa8/OGziSZCuzbnXIkSepvIwZzkg8keRjYM8lDzdfDwP10HqGSJEnjbMRgrqpTq+oZwEeqarPm6xlVtWVVfWA91jiiJK9JUknmDFn2giSXJrklyRVJvjHSXeVJjkpyygjrHulW3ZIkjWR1d2XPqaobgXOT7DN8fUsGsTgeuAw4DliQZBbwJeCEqvoBQJIDgZ3oDFv5a0mmVdX5dAbnkCSpFVZ3V/Z7gJOA01exrueDWCTZFHgxcAidcF0AvAM4ezCUAarqsiH7fA5YSmdgjiuaR8LmVtU7kuwIfIHOz+Sb66kZkiQ9xeruyj4pyRTgL6rq++uxptF6NfDNqro5ydKmV/884Ow17LcLcFhVrUxy4pDlHwM+VVX/luTtqztAkpPo/NHC9jOytvVLkvQ0q70ru6oGgI+up1rG6njgnGb6nGb+KZL8OMkNST42ZPG5VbVyFcd7MfDFZvrzq/vgqjqzquZW1dyZGxvMkqTxM5p3ZV+U5BjgK1VV3S5oNJJsSedU+u5JCphK5/T62cA+NHeNV9X+SV7HU1+W8qvVHLoV7ZMk9a/RPMf8HuBcYPngI1NJHupyXWvyOuDfqmqHqppdVc8BbgcuAk5M8qIh2248ymN+n85NZABvGL9SJUkavTX2mJtHptrmeOC0Ycu+DJwAzAM+nGRbOs9cPwCMZuzodwFfSPKu5liSJK13Gc3Z6SRHAS9pZi+tqv/salUTyNxtptbCkzbtdRmSxsjxmNfNuo7HnGRRVc0dp3ImlTWeyk5yGp3e5PXN17uaZZIkaZyN5uavVwJ7N3dok+Rs4EpglW/MkiRJa280N38BbD5kekYX6pAkSYyux3wqcGWSS4DQudbcindlS5I02YzmruwvJrkU2I9OMP9pVd3X7cIkSepHawzmIQNY3N183ybJJsDPqmpF1yqbKLZ5PixY2OsqJI3Rut1TLHXPaE5lf5LO27SuodNj3r2Z3jLJW6vqoi7WJ0lSXxnNzV93AM9v3g29L52Rma4FDgP+vou1SZLUd0YTzHOq6rrBmaq6nk5Q39a9siRJ6k+jOZV9U5JP8ZuRnOYBNyfZCHiya5VJktSHRtNjPhG4FXg3cDJwW7PsSeCQLtUlSVJfGs3jUo8Bpzdfwz0y7hVJktTHRgzmJIsZeXziqqq9ulOSJEn9a3U95iNWsSzAdsCfdaccSZL624jBXFU/G5xOsjedsY5fD9yO4xVLktQVqzuVvQtwHHA88P+A/6AzfrM3fEmS1CWrO5V9I/A94MiquhUgycnrpSpJkvrU6h6XOga4D7gkyf9J8lI615glSVKXjBjMVfXVqpoHzAEupfMM86wkn0rysvVUnyRJfWWNLxipql9V1b9X1RF07si+Cjil24VJktSPRvPmr1+rqqVV9ZmqOrRbBUmS1M/GFMySJKm7DGZJklrEYJYkqUUMZkmSWsRgliSpRQxmSZJaZI3jMWv1Ft+zjNmnfKPXZUxYd0w/odclqE/tseP2vS5hQls8f3GvS5i07DFLktQiBrMkSS1iMEuS1CIGsyRJLWIwS5LUIgazJEktMqmDOclrklSSOc387CTXDttmQZL3NtOfS3J7kquSXJHkgF7ULUnqX5M6mIHjgcuA48awz/uqam86Y05/phtFSZI0kkkbzEk2BV4MvJmxBfOg7wK/Pa5FSZK0BpP5zV+vBr5ZVTcnWZpkH2ApsFOSq4Zs92zgo6vY/0jAV9tIktarSdtjpnMa+5xm+pxmHuCnVbX34Bfw6WH7faQJ7pPo9LafJslJSRYmWbjy0WXjX7kkqW9Nyh5zki2BQ4HdkxQwFSjgk6PY/X1Vdd7qNqiqM4EzATbaeudax3IlSfq1ydpjfh3wb1W1Q1XNrqrnALcD2/W4LkmSVmuyBvPxwFeHLfsy8Gc9qEWSpFGblKeyq+rgVSz7OPDxVSxfMGT6xG7WJUnSmkzWHrMkSROSwSxJUosYzJIktYjBLElSixjMkiS1yKS8K3t92mPbGSw87VW9LmMC881p6g3ft6u2sscsSVKLGMySJLWIwSxJUosYzJIktYjBLElSixjMkiS1iMEsSVKLGMySJLWIwSxJUosYzJIktYjBLElSixjMkiS1iMEsSVKLGMySJLWIwSxJUosYzJIktYjBLElSixjMkiS1iMEsSVKLGMySJLWIwSxJUotM63UBE969V8KCGb2uQtIY7bHj9r0uYUJbPH9xr0uYtOwxS5LUIgazJEktYjBLktQiBrMkSS1iMEuS1CIGsyRJLWIwS5LUIhMumJOsTHJVkuuSXJ3kPUmmDNvm60l+OGT+jCTvHjJ/YZKzhsyfnuQ9zfS0JA8kOXU9NEeSpKeYcMEMPFZVe1fV84DfA14J/PXgyiSbA/sAmyfZsVn8A+BFzfopwFbA84Yc80XA95vplwE3Aa9Pki62Q5Kkp5mIwfxrVXU/cBLwjiEhegxwAXAOcFyz7Ps0wUwnkK8FHk7yzCQbAbsCVzbrjwc+BtwJvLDrjZAkaYgJHcwAVXUbnXY8q1l0PPDF5uv4Zpt7gRVJtqcT0D8EfgwcAMwFrqmq5Ul+C3gp8J9D9x8uyUlJFiZZuOTR6lrbJEn9Z8IHcyMASWYBvw1cVlU30wnj3ZttBnvNg8H8wyHzP2i2OQK4pKoeBb4MvCbJ1OEfVlVnVtXcqpo7c2PPdkuSxs+ED+YkzwVWAvcD84BnArcnuQOYzW9OZw9eZ96DzqnsH9HpMQ+9vnw8cFiz7yJgS+CQ9dAMSZKACR7MSWYCnwY+UVVFJ1gPr6rZVTUb2JenXmc+AlhaVSuraimwOZ1w/mGSzYADge2H7P92RjidLUlSN0zEYP6twcelgP8BLgI+mGQ2sD2dnjAAVXU78FCS/YHFdO7G/tGQYy0GllXVA8BrgW9X1RND1n8dOKq5QUySpK6bcOMxV9XTrvk27gC2XcX2+wyZ3WzYuhOHTH8O+Nyw9UuBmWtVqCRJa2Ei9pglSZq0DGZJklrEYJYkqUUMZkmSWsRgliSpRSbcXdmts83zYcHCXlchaYwW97oAaQT2mCVJahGDWZKkFjGYJUlqEYNZkqQWMZglSWoRg1mSpBYxmCVJahGDWZKkFjGYJUlqEYNZkqQWSVX1uoYJLcnDwE29rqOHtgIe6HURPWT7+7f9/dx2WPf271BVM8ermMnEd2Wvu5uqam6vi+iVJAttv+3vdR290M9tB9vfTZ7KliSpRQxmSZJaxGBed2f2uoAes/39rZ/b389tB9vfNd78JUlSi9hjliSpRQxmSZJaxGBeB0kOT3JTkluTnNLreropyXOSXJLkhiTXJXlXs3yLJN9Kckvz/Zm9rrWbkkxNcmWS/2zm+6b9STZPcl6SG5t/Bwf0WftPbv7tX5vki0mmT+b2J/mXJPcnuXbIshHbm+QDze/Cm5K8vDdVTw4G81pKMhX4Z+AVwG7A8Ul2621VXbUC+JOq2hV4IfD2pr2nABdX1c7Axc38ZPYu4IYh8/3U/o8B36yqOcBedH4OfdH+JNsCfwzMrardganAcUzu9n8OOHzYslW2t/ldcBzwvGafTza/I7UWDOa19wLg1qq6raqWA+cAR/e4pq6pqp9X1RXN9MN0filvS6fNZzebnQ28uicFrgdJtgNeBZw1ZHFftD/JZsBLgM8CVNXyqvolfdL+xjTgt5JMAzYG7mUSt7+qvgssHbZ4pPYeDZxTVU9U1e3ArXR+R2otGMxrb1vgriHzdzfLJr0ks4HnAz8GZlXVz6ET3sCzelhat/0j8H5gYMiyfmn/c4ElwL82p/LPSrIJfdL+qroH+ChwJ/BzYFlVXUSftH+Ikdrbt78Pu8FgXntZxbJJ/+xZkk2BLwPvrqqHel3P+pLkCOD+qlrU61p6ZBqwD/Cpqno+8Csm12nb1WqupR4N7AhsA2yS5I29rapV+vL3YbcYzGvvbuA5Q+a3o3Nqa9JKsgGdUP73qvpKs/gXSbZu1m8N3N+r+rrsxcBRSe6gc9ni0CT/l/5p/93A3VX142b+PDpB3S/tPwy4vaqWVNWTwFeAF9E/7R80Unv77vdhNxnMa+9yYOckOybZkM6ND+f3uKauSRI61xdvqKp/GLLqfGB+Mz0f+Pr6rm19qKoPVNV2VTWbzn/rb1fVG+mf9t8H3JXkd5pFLwWup0/aT+cU9guTbNz8v/BSOvdZ9Ev7B43U3vOB45JslGRHYGfgJz2ob1LwzV/rIMkr6Vx3nAr8S1X9XW8r6p4kBwLfAxbzm2usf0bnOvOXgO3p/PI6tqqG3zAyqSQ5GHhvVR2RZEv6pP1J9qZz49uGwG3AH9D5475f2v9BYB6dJxSuBN4CbMokbX+SLwIH0xne8RfAXwNfY4T2Jvlz4E10fj7vrqr/Xv9VTw4GsyRJLeKpbEmSWsRgliSpRQxmSZJaxGCWJKlFDGZJklrEYJZWI8mzk5yT5KdJrk/yX0l26fJnXppk7hq2eXeSjYfM/1eSzcfhs+9IsjjJ1UkuSvLsdT3mWtSwd/MootSXDGZpBM2LJL4KXFpVO1XVbnSe3Z7V28oAeDedgRQAqKpXNoNKjIdDqmovYCGd9q7ROI8ktDdgMKtvGczSyA4BnqyqTw8uqKqrqup7SQ4eHJMZIMknkpzYTN+R5ENJfphkYZJ9klzY9Lrf2mwz4v5DJflUc4zrmhdckOSP6byv+ZIklwz5zK2SfDjJ/x6y/4Ikf9JMvy/J5UmuGTzWGnwX+O10xqD+yJB9/2hIGy5J8gVgcbPdR5se9zVJ3tlst2+S7yRZ1PwcBl/peGlT70+S3Jzkd5u36P0NMC/JVUnmjaJOaVKZ1usCpBbbHVjbQSvuqqoDkpxBZ1zbFwPTgeuAT69ux2H+vKqWNj3Si5PsWVUfT/IeOj3bB4Ztfw6dt9F9spl/PXB4kpfReU3iC+gMOHB+kpc0Q/uN5Ag6b3p7M53RlPZLshHw/SQXNdu8ANi9qm5P8jY6gzw8v6pWJNmieb/6PwFHV9WSJmj/js4bogCmVdULmlPXf11VhyX5KzrjHr9jDD8nadIwmKXuGHxv+mJg02YM64eTPD7Ga8GvT3ISnf9XtwZ2A64ZaeOqujLJs5JsA8wEHqyqO5te9svovEoSOq+S3JlOr3i4S5KsbD7nL+i8hnPPJK9r1s9o9l0O/KQZfxc6Az18uqpWNLUsTbI7nT9wvtW5MsBUOsMmDhocDGURMHsUPw9p0jOYpZFdB7xuhHUreOqloOnD1j/RfB8YMj04P20U+9MMBvBeYL+qejDJ51a13Sqc19T9bDo9aOj0kk+tqs+MYv+n9MSba+3vrKoLh9V3MJ3hH3+9iKcP9Rfguqo6YITPGvzZrMTfRxLgNWZpdb4NbJTkDwcXJNkvyUHAz4DdmtF0ZtAZbWgsRrP/ZnSCb1mSWcArhqx7GHjGCMc+h84IWK+jE9IAFwJvSmc8bZJsm+RZI+w/3IXA25rT0iTZJckmq9juIuCtSaY1220B3ATMTHJAs2yDJM9bw+etrm3SpGcwSyOozggvrwF+r7lx6zpgAXBvVd1FZ5Sda4B/5zeniEd77DXuX1VXN8uvA/4F+P6Q1WcC/z1489ew/a6jE2z3VNXPm2UXAV8AfphkMZ3AHm34nUVniMcrklwLfIZV927PojPi0DVJrgZOqKrldP5A+HCz7Co64xivziV0/mjx5i/1JUeXkiSpRewxS5LUIgazJEktYjBLktQiBrMkSS1iMEuS1CIGsyRJLWIwS5LUIv8fWVuM8DmdERIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "IvXZbvWKg9sF"
   },
   "source": [
    "##### Epsilon .05"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "id": "EhAzxPqohF_t"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, '_0.05_')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "XsPyJmtZhG-V",
    "outputId": "cc6deaf5-3f82-4f2c-e111-d0e17116147b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=10.582, p=0.032\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "D14lUo17hHEc",
    "outputId": "cb4ae057-c350-4518-8cdf-784a148e1f1d"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.04624</td>\n",
       "      <td>0.24454</td>\n",
       "      <td>0.74831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.04624</td>\n",
       "      <td>0.24454</td>\n",
       "      <td>0.74831</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.04624</td>\n",
       "      <td>0.04624</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.41335</td>\n",
       "      <td>0.04463</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.24454</td>\n",
       "      <td>0.24454</td>\n",
       "      <td>0.41335</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.15817</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.74831</td>\n",
       "      <td>0.74831</td>\n",
       "      <td>0.04463</td>\n",
       "      <td>0.15817</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 1.00000 0.04624    0.24454  0.74831\n",
       "AHP       1.00000 1.00000 0.04624    0.24454  0.74831\n",
       "DAWA      0.04624 0.04624 1.00000    0.41335  0.04463\n",
       "Geometric 0.24454 0.24454 0.41335    1.00000  0.15817\n",
       "Laplace   0.74831 0.74831 0.04463    0.15817  1.00000"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "QHGdzI2qhHHM",
    "outputId": "dfb34abb-e300-49a4-d4e3-e204981f809a"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcUElEQVR4nO3deZiedX3v8fcnCUsRCQIREYQghYPIJsSdCihVFMQFMYD2hKqletQK1trYNbZXNR6lrdYqcrCFeqq04oalFSyC+0LCloCyCGFTJJxoBBcwyff8cd+jD2MmmSTzzHPPPO/Xdc019/58f0OYz/zu7ZeqQpIkdcOMQRcgSZJ+xWCWJKlDDGZJkjrEYJYkqUMMZkmSOmTWoAuY6nbZZZeaO3fuoMuQpCll6dKl91XVnEHX0UUG8xaaO3cuS5YsGXQZkjSlJLl90DV0laeyJUnqEINZkqQOMZglSeoQrzFLkqaNpUuXPnrWrFnnAgfSzc7nOmD5mjVrXnP44Yffu74NDGZJ0rQxa9ascx/zmMc8Yc6cOT+cMWNG5waDWLduXVauXHnAPffccy5wwvq26eJfE5Ikba4D58yZ8+MuhjLAjBkzas6cOatpevTr32YS65Ekqd9mdDWUR7T1jZm/BrMkSR3iNWZJ0rQ1d+HFh0/k8VYsPm7pxra58MILd3jLW96y57p163jlK1953zve8Y57NuUzDOYttOzu1cxdePGgy5C0iVZse+qgS5jaFq0edAWdtGbNGs4888w9L7nkkpse//jH/+KQQw55woknnvijww8//OfjPYansiVJmiBXXHHFI/baa68HDzjggIe23XbbeulLX7rqwgsv3HFTjmEwS5I0Qe68886td99994dG5vfYY4+H7r777q035RgGsyRJE6Tq128IT7JJd4kbzJIkTZA999zzYT3ku+66a+vHPvaxv9iUYxjMkiRNkCOPPPInK1as2PY73/nO1j//+c/zyU9+cqcTTzzxR5tyDO/KliRNW+N5vGkibbXVVpx11ll3HHvssfutXbuWU0899b558+aN+45sMJglSZpQ8+fPXz1//vzNfp6sc6eykzwwAceYm2T5RNQjSdJk6lwwS5I0zKZEMCd5YZJvJrk6yX8n2bVdvijJR5J8IcnNSX5vPfvOTfLlJFe1X8/oWffWJMuSXJtkcbtsnySfS7K03W//yWupJGnYTZVrzF8BnlZVleQ1wFuBP2zXHQw8DXgEcHWS0e/HvBf47ar6eZJ9gY8B85I8H3gx8NSq+mmSndrtzwFeW1U3J3kq8AHg2b0HTHI6cDrAzB3mTHBTJUnDbKoE8x7AvyXZDdgauK1n3Weq6mfAz5JcDjwFuKZn/VbA+5McCqwF9muXHwP8c1X9FKCqViXZHngG8PEkI/tvM7qYqjqHJsDZZrd9Oz28mCRpapkqwfwPwN9W1UVJjgIW9awbHYyj588EfgAcQnPqfuS29axn2xnAj6rq0C2uWJKkzTBVgnk2cHc7vWDUuhcleSfNqeyjgIU0verefe+qqnVJFgAz2+WXAn+R5KMjp7LbXvNtSU6qqo+n6TYfXFXX9qthkqQ+WjR7Qod9ZNHqjT4XfdJJJ8297LLLZu+8885rbr755us39SO6ePPXdknu6vl6M00P+eNJvgzcN2r7bwEXA98A/rqqvjdq/QeABUm+QXMa+ycAVfU54CJgSZJrgLe0278CeHWSa4HrgRdNdAMlSdPXq171qvsuuuiimzd3/871mKtqrD8WPjPG8puq6vRRx1gBHNhO30xzg9iIt/VstxhYPGrf24BjN61qSZIaz3/+8x+48cYbN2lEqV5d7DFLkjS0Otdj3hRVtWjQNUiSNJHsMUuS1CEGsyRJHTKlT2VLkrRB43i8aaK98IUv3Psb3/jGI3/4wx/O2nXXXQ9euHDh984888zRTxSNyWDeQgftPpsli48bdBmSNtlmj8onbdBnP/vZ2za+1dg8lS1JUocYzJIkdYjBLElShxjMkiR1iMEsSVKHGMySJHWIj0tJkqatg84/aEKHfVy2YNkGn4u+5ZZbtnrFK16x98qVK7eaMWMGCxYsWPnnf/7n927KZ9hjliRpgmy11VacddZZd916663XX3nlld/+8Ic//OilS5duuynHMJglSZoge+211y+OOOKInwI86lGPWrfPPvv87I477tikISANZkmS+uDGG2/c+oYbbtjuyCOPfGBT9jOYJUmaYKtXr57x0pe+dJ/FixffudNOO63blH0NZkmSJtCDDz6Y4447bp+TTjpp1YIFC360qfsbzJIkTZB169Zx8skn77Xffvv9fNGiRT/YnGP4uJQkadra2ONNE+3zn//89p/+9Kd33nfffX+2//77HwDw9re//e758+ePezgzg1mSpAnyvOc974Gq2qI/BjyVLUlShxjMkiR1iMEsSZpO1q1bty6DLmJD2vrGfITKYJYkTSfLV65cObur4bxu3bqsXLlyNrB8rG28+UuSNG2sWbPmNffcc8+599xzz4F0s/O5Dli+Zs2a14y1gcEsSZo2Dj/88HuBEwZdx5YwmLfQsrtXM3fhxYMuQx2yYttTB12ChtRBe+85aZ+1bMGySfusYdPFbr4kSUPLYJYkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlD+hrMSXZN8tEktyZZmuTrSV7Sz88co46jkjxjA+tPSLJwMmuSJGl9+vYcc5IAnwbOr6pT22V7MZgHv48CHgC+NnpFkllVdRFw0WQXJUnSaP3sMT8beKiqzh5ZUFW3V9U/JJmZ5N1JrkxyXZLfhybM2+XLkyxLMr9dflSSLyb59yQ3JVmc5BVJvtVut0+73Zwkn2iPe2WSZyaZC7wWODPJNUl+K8l5Sf42yeXAu5KcluT97TF2TfKpJNe2X2P2tCVJmmj9fPPXE4Grxlj3amB1VT05yTbAV5NcChwGHAocAuwCXJnkS+0+hwBPAFYBtwLnVtVTkrwJeCNwBvBe4O+q6itJ9gQuqaonJDkbeKCq3gOQ5NXAfsAxVbU2yWk9tb0P+GJVvSTJTGD7ifhhSJI0HpP2Ss4k/wgcATwE3A4cnORl7erZwL7t+o9V1VrgB0m+CDwZ+DFwZVV9vz3Wd4FL232XAUe308cABzRn0QHYIckjxyjp4+3njPZs4H8CtOtXr6ctpwOnA8zcYc7GGy9J0jj1M5ivB04cmamq1yfZBVgC3AG8saou6d0hyQs2cLwHe6bX9cyv41ftmAE8vap+Nuq46zveT8bRhvWqqnOAcwC22W3f2tzjSJI0Wj+vMX8B2DbJ63qWbdd+vwR4XZKtAJLsl+QRwJeA+e016DnAs4BvbcJnXgq8YWQmyaHt5P3AWD3n0S4DXtfuPzPJDpvw+ZIkbZG+BXNVFfBi4MgktyX5FnA+8MfAucANwFVJlgMfoun1fgq4DriWJtjfWlX3bMLH/gEwr72h7Aaam74APgu8ZOTmr40c403A0UmWAUtprpVLkjQp0uSnNtc2u+1buy34+0GXoQ5x2EcNylQa9jHJ0qqaN0HlTCu++UuSpA4xmCVJ6hCDWZKkDjGYJUnqEINZkqQOmbQ3f01XB+0+myWLjxt0GeqUX3tZnDQptuw+aXWFPWZJkjrEYJYkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUMMZkmSOsRgliSpQ8b1Ss4kjwIe17t9VV3Vr6IkSRpWGw3mJH8NnAZ8F6h2cQHP7l9ZkiQNp/H0mF8O7FNVD/W7GEmSht14rjEvB3bscx2SJInx9ZjfCVydZDnw4MjCqjqhb1VJkjSkxhPM5wPvohnqc11/y5EkabiNJ5jvq6r39b0SSZI0rmBemuSdwEU8/FS2j0tJkjTBxhPMT2q/P61nmY9LSZLUBxsN5qo6ejIKkSRJ43vByDbAicBcHv7mr7/qX1mSJA2n8ZzK/gywGlhKzzVmNZbdvZq5Cy8edBnStLFi21MHXYLGY9HqQVcwbY0nmPeoqmP7XokkSRrXm7++luSgvlciSZLG7jEnWUZz9/Us4HeT3EpzKjtAVdXBk1OiJEnDY0Onso+ftCokSRKwgWCuqtsBknykqn6nd12SjwC/s94dJUnSZhvPNeYn9s4kmQkc3p9yJEkabmMGc5K3JbkfODjJj9uv+4F7aR6hkiRJE2zMYK6qd1bVI4F3V9UO7dcjq2rnqnrbJNY4piQvSVJJ9u9Z9pQkVyS5OclVSS4e667yJCckWTjGugf6VbckSWPZ0F3Z+1fVd4CPJzls9PqODGJxCvAV4GRgUZJdgX8HTq2qrwEkOQLYh2bYyl9KMquqLqIZnEOSpE7Y0F3ZbwZOB85az7qBD2KRZHvgmcDRNOG6CHgDcP5IKANU1Vd69jkPWEUzMMdV7SNh86rqDUn2Bj5K8zP53CQ1Q5Kkh9nQXdmnJ5kB/FlVfXUSaxqvFwOfq6qbkqxqe/VPBM7fyH77AcdU1dokp/Usfy/wwar6lySv39ABkpxO80cLM3eYs7n1S5L0azZ4V3ZVrQPeM0m1bKpTgAva6Qva+YdJ8s0k307y3p7FH6+qtes53jOBj7XTH9nQB1fVOVU1r6rmzdxu9maULknS+o3nXdmXJjkR+GRVVb8LGo8kO9OcSj8wSQEzaU6vnw8cRnvXeFU9NcnLePjLUn6ygUN3on2SpOE1nueY3wx8HHho5JGpJD/uc10b8zLgX6pqr6qaW1WPA24DLgVOS/KMnm23G+cxv0pzExnAKyauVEmSxm+jPeb2kamuOQVYPGrZJ4BTgfnAu5LsTvPM9X3AeMaOfhPw0SRvao8lSdKky3jOTic5AXhWO3tFVf1HX6uaQrbZbd/abcHfD7oMadpwPOYpYgvHY06ytKrmTVA108pGT2UnWUzTm7yh/XpTu0ySJE2w8dz89QLg0PYObZKcD1wNrPeNWZIkafON5+YvgB17pn0+SJKkPhlPj/mdwNVJLgdCc625E+/KliRpuhnPXdkfS3IF8GSaYP7jqrqn34VJkjSMNhrMPQNY3NV+f2ySRwC3V9WavlU2RRy0+2yWLD5u0GVI08iW3e0rTXXjOZX9AZq3aV1H02M+sJ3eOclrq+rSPtYnSdJQGc/NXyuAJ7Xvhj6cZmSm5cAxwP/uY22SJA2d8QTz/lV1/chMVd1AE9S39q8sSZKG03hOZd+Y5IP8aiSn+cBNSbYBftG3yiRJGkLj6TGfBtwCnAGcCdzaLvsFcHSf6pIkaSiN53GpnwFntV+jPTDhFUmSNMTGDOYkyxh7fOKqqkP6U5IkScNrQz3m49ezLMAewJ/0pxxJkobbmMFcVbePTCc5lGas45cDt+F4xZIk9cWGTmXvB5wMnAL8P+DfaMZv9oYvSZL6ZEOnsr8DfBl4YVXdApDkzEmpSpKkIbWhx6VOBO4BLk/yf5I8h+YasyRJ6pMxg7mqPlVV84H9gStonmHeNckHkzx3kuqTJGmobPQFI1X1k6r616o6nuaO7GuAhf0uTJKkYTSeN3/9UlWtqqoPVdWz+1WQJEnDbJOCWZIk9ZfBLElShxjMkiR1iMEsSVKHGMySJHWIwSxJUodsdDxmbdiyu1czd+HFgy5DmjZWbHvqoEvQeCxaPegKpi17zJIkdYjBLElShxjMkiR1iMEsSVKHGMySJHWIwSxJUodM62BO8pIklWT/dn5ukuWjtlmU5C3t9HlJbktyTZKrkjx9EHVLkobXtA5m4BTgK8DJm7DPH1XVoTRjTn+oH0VJkjSWaRvMSbYHngm8mk0L5hFfAn5zQouSJGkjpvObv14MfK6qbkqyKslhwCpgnyTX9Gz3GOA969n/hcCyvlcpSVKPadtjpjmNfUE7fUE7D/Ddqjp05As4e9R+726D+3Sa3vavSXJ6kiVJlqz9qa+lkyRNnGnZY06yM/Bs4MAkBcwECvjAOHb/o6q6cEMbVNU5wDkA2+y2b21huZIk/dJ07TG/DPiXqtqrquZW1eOA24A9BlyXJEkbNF2D+RTgU6OWfQL4kwHUIknSuE3LU9lVddR6lr0PeN96li/qmT6tn3VJkrQx07XHLEnSlGQwS5LUIQazJEkdYjBLktQhBrMkSR0yLe/KnkwH7T6bJYuPG3QZ0jTi2/Q03OwxS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhswZdwFS37O7VzF148aDLmFArtj110CVI6rpFqwddwbRlj1mSpA4xmCVJ6hCDWZKkDjGYJUnqEINZkqQOMZglSeoQg1mSpA6ZcsGcZG2Sa5Jcn+TaJG9OMmPUNp9J8vWe+b9LckbP/CVJzu2ZPyvJm9vpWUnuS/LOSWiOJEkPM+WCGfhZVR1aVU8Efht4AfCXIyuT7AgcBuyYZO928deAZ7TrZwC7AE/sOeYzgK+2088FbgReniR9bIckSb9mKgbzL1XVvcDpwBt6QvRE4LPABcDJ7bKv0gYzTSAvB+5P8qgk2wBPAK5u158CvBe4A3ha3xshSVKPKR3MAFV1K007Ht0uOgX4WPt1SrvN94A1SfakCeivA98Eng7MA66rqoeS/AbwHOA/evcfLcnpSZYkWbL2p76WTpI0caZ8MLcCkGRX4DeBr1TVTTRhfGC7zUiveSSYv94z/7V2m+OBy6vqp8AngJckmTn6w6rqnKqaV1XzZm43u4/NkiQNmykfzEkeD6wF7gXmA48CbkuyApjLr05nj1xnPojmVPY3aHrMvdeXTwGOafddCuwMHD0JzZAkCZjiwZxkDnA28P6qKppgPbaq5lbVXOBwHn6d+XhgVVWtrapVwI404fz1JDsARwB79uz/esY4nS1JUj9MxWD+jZHHpYD/Bi4F3p5kLrAnTU8YgKq6DfhxkqcCy2juxv5Gz7GWAaur6j7gpcAXqurBnvWfAU5obxCTJKnvptx4zFX1a9d8WyuA3dez/WE9szuMWndaz/R5wHmj1q8C5mxWoZIkbYap2GOWJGnaMpglSeoQg1mSpA4xmCVJ6hCDWZKkDplyd2V3zUG7z2bJ4uMGXcYE8zWjkjQo9pglSeoQg1mSpA4xmCVJ6hCDWZKkDjGYJUnqEINZkqQOMZglSeoQg1mSpA4xmCVJ6hCDWZKkDklVDbqGKS3J/cCNg65jgHYB7ht0EQNk+4e3/cPcdtjy9u9VVXMmqpjpxHdlb7kbq2reoIsYlCRLbL/tH3QdgzDMbQfb30+eypYkqUMMZkmSOsRg3nLnDLqAAbP9w22Y2z/MbQfb3zfe/CVJUofYY5YkqUMMZkmSOsRg3gJJjk1yY5JbkiwcdD39lORxSS5P8u0k1yd5U7t8pySfT3Jz+/1Rg661n5LMTHJ1kv9o54em/Ul2THJhku+0/w6ePmTtP7P9t788yceSbDud25/kn5Lcm2R5z7Ix25vkbe3vwhuTPG8wVU8PBvNmSjIT+Efg+cABwClJDhhsVX21BvjDqnoC8DTg9W17FwKXVdW+wGXt/HT2JuDbPfPD1P73Ap+rqv2BQ2h+DkPR/iS7A38AzKuqA4GZwMlM7/afBxw7atl629v+LjgZeGK7zwfa35HaDAbz5nsKcEtV3VpVDwEXAC8acE19U1Xfr6qr2un7aX4p707T5vPbzc4HXjyQAidBkj2A44BzexYPRfuT7AA8C/gwQFU9VFU/Ykja35oF/EaSWcB2wPeYxu2vqi8Bq0YtHqu9LwIuqKoHq+o24Baa35HaDAbz5tsduLNn/q522bSXZC7wJOCbwK5V9X1owht49ABL67e/B94KrOtZNiztfzywEvjn9lT+uUkewZC0v6ruBt4D3AF8H1hdVZcyJO3vMVZ7h/b3YT8YzJsv61k27Z89S7I98AngjKr68aDrmSxJjgfuraqlg65lQGYBhwEfrKonAT9hep223aD2WuqLgL2BxwKPSPLKwVbVKUP5+7BfDObNdxfwuJ75PWhObU1bSbaiCeV/rapPtot/kGS3dv1uwL2Dqq/PngmckGQFzWWLZyf5vwxP++8C7qqqb7bzF9IE9bC0/xjgtqpaWVW/AD4JPIPhaf+Isdo7dL8P+8lg3nxXAvsm2TvJ1jQ3Plw04Jr6Jklori9+u6r+tmfVRcCCdnoB8JnJrm0yVNXbqmqPqppL89/6C1X1Soan/fcAdyb5H+2i5wA3MCTtpzmF/bQk27X/LzyH5j6LYWn/iLHaexFwcpJtkuwN7At8awD1TQu++WsLJHkBzXXHmcA/VdXfDLai/klyBPBlYBm/usb6JzTXmf8d2JPml9dJVTX6hpFpJclRwFuq6vgkOzMk7U9yKM2Nb1sDtwK/S/PH/bC0/+3AfJonFK4GXgNszzRtf5KPAUfRDO/4A+AvgU8zRnuT/CnwKpqfzxlV9V+TX/X0YDBLktQhnsqWJKlDDGZJkjrEYJYkqUMMZkmSOsRgliSpQwxmaQOSPCbJBUm+m+SGJP+ZZL8+f+YVSeZtZJszkmzXM/+fSXacgM9ekWRZkmuTXJrkMVt6zM2o4dD2UURpKBnM0hjaF0l8CriiqvapqgNont3edbCVAXAGzUAKAFTVC9pBJSbC0VV1CLCEpr0bNcEjCR0KGMwaWgazNLajgV9U1dkjC6rqmqr6cpKjRsZkBkjy/iSntdMrkrwjydeTLElyWJJL2l73a9ttxty/V5IPtse4vn3BBUn+gOZ9zZcnubznM3dJ8q4k/6tn/0VJ/rCd/qMkVya5buRYG/El4DfTjEH97p59f7+nDZcn+SiwrN3uPW2P+7okb2y3OzzJF5MsbX8OI690vKKt91tJbkryW+1b9P4KmJ/kmiTzx1GnNK3MGnQBUocdCGzuoBV3VtXTk/wdzbi2zwS2Ba4Hzt7QjqP8aVWtanuklyU5uKrel+TNND3b+0ZtfwHN2+g+0M6/HDg2yXNpXpP4FJoBBy5K8qx2aL+xHE/zprdX04ym9OQk2wBfTXJpu81TgAOr6rYkr6MZ5OFJVbUmyU7t+9X/AXhRVa1sg/ZvaN4QBTCrqp7Snrr+y6o6Jslf0Ix7/IZN+DlJ04bBLPXHyHvTlwHbt2NY35/k55t4LfjlSU6n+X91N+AA4LqxNq6qq5M8OsljgTnAD6vqjraX/VyaV0lC8yrJfWl6xaNdnmRt+zl/RvMazoOTvKxdP7vd9yHgW+34u9AM9HB2Va1pa1mV5ECaP3A+31wZYCbNsIkjRgZDWQrMHcfPQ5r2DGZpbNcDLxtj3Roefilo21HrH2y/r+uZHpmfNY79aQcDeAvw5Kr6YZLz1rfdelzY1v0Ymh40NL3kd1bVh8ax/8N64u219jdW1SWj6juKZvjHXy7i14f6C3B9VT19jM8a+dmsxd9HEuA1ZmlDvgBsk+T3RhYkeXKSI4HbgQPa0XRm04w2tCnGs/8ONMG3OsmuwPN71t0PPHKMY19AMwLWy2hCGuAS4FVpxtMmye5JHj3G/qNdAryuPS1Nkv2SPGI9210KvDbJrHa7nYAbgTlJnt4u2yrJEzfyeRtqmzTtGczSGKoZ4eUlwG+3N25dDywCvldVd9KMsnMd8K/86hTxeI+90f2r6tp2+fXAPwFf7Vl9DvBfIzd/jdrveppgu7uqvt8uuxT4KPD1JMtoAnu84XcuzRCPVyVZDnyI9fduz6UZcei6JNcCp1bVQzR/ILyrXXYNzTjGG3I5zR8t3vyloeToUpIkdYg9ZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUMMZkmSOuT/AwDCBMFgMm+eAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LgSfpqHhhLEO"
   },
   "source": [
    "##### Epsilon .01"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "id": "aSXa3oQ4hNeE"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, '_0.01_')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "r9-WcU7AhOWc",
    "outputId": "2acc218f-0ea3-4d96-836a-9e874048833a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=16.000, p=0.003\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "1atm8mdWhOY7",
    "outputId": "28379ac2-8f1e-49dd-c75f-49a85b118ddc"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.00122</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.00122</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.00122</td>\n",
       "      <td>0.00122</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.00122</td>\n",
       "      <td>0.00122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.00122</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.00122</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 1.00000 0.00122    1.00000  1.00000\n",
       "AHP       1.00000 1.00000 0.00122    1.00000  1.00000\n",
       "DAWA      0.00122 0.00122 1.00000    0.00122  0.00122\n",
       "Geometric 1.00000 1.00000 0.00122    1.00000  1.00000\n",
       "Laplace   1.00000 1.00000 0.00122    1.00000  1.00000"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "MgDwvxxnhObO",
    "outputId": "152d5fe4-e042-4109-9c66-f2e0a8088d5e"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbHElEQVR4nO3de5SddX3v8feHhEsBCQoRMAijFg5yl0QUtQpIPSiIImiA2gNVS/WoBSy1se1qabsq8ahttVaRw2mhngotqBVLK7QIXhDEhFuCCigJAhqBE414xSTf88fzjG7GTDJJZs9+Mvv9WmvWPPf9/Q1hPvN7br9UFZIkqRu2GnQBkiTpFwxmSZI6xGCWJKlDDGZJkjrEYJYkqUNmDrqALd2uu+5aIyMjgy5DkrYoixcvfqSqZg+6ji4ymDfTyMgIixYtGnQZkrRFSXLfoGvoKk9lS5LUIQazJEkdYjBLktQhXmOWJE0bixcvfvLMmTMvAg6km53PtcDS1atXv2Hu3LkPrWsDg1mSNG3MnDnzot133/2Zs2fP/u5WW23VucEg1q5dm4cffnj/FStWXAScsK5tuvjXhCRJm+rA2bNnf7+LoQyw1VZb1ezZs1fR9OjXvc0U1iNJUr9t1dVQHtXWN27+GsySJHWI15glSdPWyIKr5k7m8ZYvPG7xhra54oordjr33HP3Wrt2La997Wsfeec737liYz7DYN5MSx5cxciCqwZdhiRNqeULjxt0CZ20evVqzjnnnL2uvvrqu5/+9Kf/7JBDDnnmSSed9L25c+f+ZKLH8FS2JEmT5Prrr99h7733/un+++//2HbbbVevetWrVl5xxRU7b8wxDGZJkibJ/fffv82cOXMeG53fc889H3vwwQe32ZhjGMySJE2Sql++ITzJRt0lbjBLkjRJ9tprr8f1kB944IFtnvKUp/xsY45hMEuSNEle9KIX/XD58uXbfe1rX9vmJz/5ST7+8Y8/6aSTTvrexhzDu7IlSdPWRB5vmkxbb701733ve7957LHH7rtmzRpOO+20R+bNmzfhO7LBYJYkaVLNnz9/1fz581dt6v6dO5Wd5AeTcIyRJEsnox5JkqZS54JZkqRhtkUEc5KXJ/lSkluT/FeS3drl5yX5SJLPJLknyW+vY9+RJJ9Pckv79byedW9PsiTJ7UkWtsuekeTTSRa3++03dS2VJA27LeUa8xeA51ZVJXkD8Hbg99p1BwPPBXYAbk0y9v2YDwG/XlU/SbIPcCkwL8lLgVcCz6mqHyV5Urv9hcAbq+qeJM8BPggc3XvAJGcCZwLM2Gn2JDdVkjTMtpRg3hP45yR7ANsAy3rWfbKqfgz8OMl1wOHAbT3rtwY+kORQYA2wb7v8GOAfqupHAFW1MsmOwPOAy5OM7r/t2GKq6kKaAGfbPfbp9PBikqQty5YSzH8L/FVVXZnkSOC8nnVjg3Hs/DnAd4BDaE7dj962nnVsuxXwvao6dLMrliRpE2wpwTwLeLCdPn3MulckOZ/mVPaRwAKaXnXvvg9U1dokpwMz2uXXAH+S5KOjp7LbXvOyJK+uqsvTdJsPrqrb+9UwSVIfnTdrUod95LxVG3wu+tWvfvXItddeO2uXXXZZfc8999y5sR/RxZu/tk/yQM/X22h6yJcn+TzwyJjtbwauAm4C/qKqvjVm/QeB05PcRHMa+4cAVfVp4EpgUZLbgHPb7X8DeH2S24E7gVdMdgMlSdPX6173ukeuvPLKezZ1/871mKtqvD8WPjnO8rur6swxx1gOHNhO30Nzg9iod/RstxBYOGbfZcCxG1e1JEmNl770pT+46667NmpEqV5d7DFLkjS0Otdj3hhVdd6ga5AkaTLZY5YkqUMMZkmSOmSLPpUtSdJ6TeDxpsn28pe//Gk33XTTE7773e/O3G233Q5esGDBt84555yxTxSNy2DeTAfNmcWihccNugxJUkd86lOfWrbhrcbnqWxJkjrEYJYkqUMMZknSdLJ27dq12fBmg9PWt3a89QazJGk6Wfrwww/P6mo4r127Ng8//PAsYOl423jzlyRp2li9evUbVqxYcdGKFSsOpJudz7XA0tWrV79hvA0MZknStDF37tyHgBMGXcfm6OJfE5IkDS2DWZKkDjGYJUnqEINZkqQOMZglSeoQg1mSpA4xmCVJ6hCDWZKkDjGYJUnqEINZkqQOMZglSeoQg1mSpA4xmCVJ6hCDWZKkDjGYJUnqEMdj3kxLHlzFyIKrBl2GJE2p5QuPG3QJ05Y9ZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUP6GsxJdkvy0ST3Jlmc5MYkJ/bzM8ep48gkz1vP+hOSLJjKmiRJWpe+PcecJMC/ApdU1Wntsr2BE/r1metxJPAD4ItjVySZWVVXAldOdVGSJI3Vzx7z0cBjVXXB6IKquq+q/jbJjCTvTvLlJHck+R1owrxdvjTJkiTz2+VHJvlskn9JcneShUl+I8nN7XbPaLebneRj7XG/nOT5SUaANwLnJLktya8luTjJXyW5DnhXkjOSfKA9xm5JPpHk9vZr3J62JEmTrZ9v/joAuGWcda8HVlXVs5NsC9yQ5BrgMOBQ4BBgV+DLST7X7nMI8ExgJXAvcFFVHZ7kLOCtwNnA+4C/rqovJNkLuLqqnpnkAuAHVfUegCSvB/YFjqmqNUnO6Knt/cBnq+rEJDOAHSfjhyFJ0kRM2Ss5k/wd8ALgMeA+4OAkJ7erZwH7tOsvrao1wHeSfBZ4NvB94MtV9e32WN8Armn3XQIc1U4fA+zfnEUHYKckTxinpMvbzxnraOB/ALTrV62jLWcCZwLM2Gn2hhsvSdIE9TOY7wROGp2pqjcn2RVYBHwTeGtVXd27Q5KXred4P+2ZXtszv5ZftGMr4Iiq+vGY467reD+cQBvWqaouBC4E2HaPfWpTjyNJ0lj9vMb8GWC7JG/qWbZ9+/1q4E1JtgZIsm+SHYDPAfPba9CzgRcCN2/EZ14DvGV0Jsmh7eSjwHg957GuBd7U7j8jyU4b8fmSJG2WvgVzVRXwSuBFSZYluRm4BPgD4CLgK8AtSZYCH6bp9X4CuAO4nSbY315VKzbiY38XmNfeUPYVmpu+AD4FnDh689cGjnEWcFSSJcBimmvlkiRNiTT5qU217R771B6n/82gy5CkKbW5wz4mWVxV8yapnGnFN39JktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIVP25q/p6qA5s1i0mXcnSpI0yh6zJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CETeiVnkicCT+3dvqpu6VdRkiQNqw0Gc5K/AM4AvgFUu7iAo/tXliRJw2kiPebXAM+oqsf6XYwkScNuIteYlwI797kOSZLExHrM5wO3JlkK/HR0YVWd0LeqJEkaUhMJ5kuAdwFLgLX9LUeSpOE2kWB+pKre3/dKJEnShIJ5cZLzgSt5/KlsH5eSJGmSTSSYn9V+f27PMh+XkiSpDzYYzFV11FQUIkmSJvaCkW2Bk4ARHv/mrz/vX1mSJA2niZzK/iSwClhMzzVmNZY8uIqRBVcNugxJmlLLFx436BKmrYkE855VdWzfK5EkSRN689cXkxzU90okSdL4PeYkS2juvp4J/FaSe2lOZQeoqjp4akqUJGl4rO9U9vFTVoUkSQLWE8xVdR9Ako9U1W/2rkvyEeA317mjJEnaZBO5xnxA70ySGcDc/pQjSdJwGzeYk7wjyaPAwUm+3349CjxE8wiVJEmaZOMGc1WdX1VPAN5dVTu1X0+oql2q6h1TWOO4kpyYpJLs17Ps8CTXJ7knyS1JrhrvrvIkJyRZMM66H/SrbkmSxrO+u7L3q6qvAZcnOWzs+o4MYnEq8AXgFOC8JLsB/wKcVlVfBEjyAuAZNMNW/lySmVV1Jc3gHJIkdcL67sp+G3Am8N51rBv4IBZJdgSeDxxFE67nAW8BLhkNZYCq+kLPPhcDK2kG5rilfSRsXlW9JcnTgI/S/Ew+PUXNkCTpcdZ3V/aZSbYC/riqbpjCmibqlcCnq+ruJCvbXv0BwCUb2G9f4JiqWpPkjJ7l7wM+VFX/mOTN6ztAkjNp/mhhxk6zN7V+SZJ+yXrvyq6qtcB7pqiWjXUqcFk7fVk7/zhJvpTkq0ne17P48qpas47jPR+4tJ3+yPo+uKourKp5VTVvxvazNqF0SZLWbSLvyr4myUnAx6uq+l3QRCTZheZU+oFJCphBc3r9EuAw2rvGq+o5SU7m8S9L+eF6Dt2J9kmShtdEnmN+G3A58NjoI1NJvt/nujbkZOAfq2rvqhqpqqcCy4BrgDOSPK9n2+0neMwbaG4iA/iNyStVkqSJ22CPuX1kqmtOBRaOWfYx4DRgPvCuJHNonrl+BJjI2NFnAR9NclZ7LEmSplwmcnY6yQnAC9vZ66vq3/pa1RZk2z32qT1O/5tBlyFJU2pzx2NOsriq5k1SOdPKBk9lJ1lI05v8Svt1VrtMkiRNsonc/PUy4ND2Dm2SXALcCqzzjVmSJGnTTeTmL4Cde6Z9PkiSpD6ZSI/5fODWJNcBobnW3Il3ZUuSNN1M5K7sS5NcDzybJpj/oKpW9LswSZKG0QaDuWcAiwfa709JsgNwX1Wt7ltlW4iD5sxi0WbenShJ0qiJnMr+IM3btO6g6TEf2E7vkuSNVXVNH+uTJGmoTOTmr+XAs9p3Q8+lGZlpKXAM8L/6WJskSUNnIsG8X1XdOTpTVV+hCep7+1eWJEnDaSKnsu9K8iF+MZLTfODuJNsCP+tbZZIkDaGJ9JjPAL4OnA2cA9zbLvsZcFSf6pIkaShN5HGpHwPvbb/G+sGkVyRJ0hAbN5iTLGH88Ymrqg7pT0mSJA2v9fWYj1/HsgB7An/Yn3IkSRpu4wZzVd03Op3kUJqxjl8DLMPxiiVJ6ov1ncreFzgFOBX4f8A/04zf7A1fkiT1yfpOZX8N+Dzw8qr6OkCSc6akKkmShtT6Hpc6CVgBXJfkfyd5Mc01ZkmS1CfjBnNVfaKq5gP7AdfTPMO8W5IPJXnJFNUnSdJQ2eALRqrqh1X1T1V1PM0d2bcBC/pdmCRJw2gib/76uapaWVUfrqqj+1WQJEnDbKOCWZIk9ZfBLElShxjMkiR1iMEsSVKHGMySJHWIwSxJUodscDxmrd+SB1cxsuCqQZchSVNq+cLjBl3CtGWPWZKkDjGYJUnqEINZkqQOMZglSeoQg1mSpA4xmCVJ6pBpHcxJTkxSSfZr50eSLB2zzXlJzm2nL06yLMltSW5JcsQg6pYkDa9pHczAqcAXgFM2Yp/fr6pDacac/nA/ipIkaTzTNpiT7Ag8H3g9GxfMoz4H/OqkFiVJ0gZM5zd/vRL4dFXdnWRlksOAlcAzktzWs93uwHvWsf/LgSV9r1KSpB7TtsdMcxr7snb6snYe4BtVdejoF3DBmP3e3Qb3mTS97V+S5Mwki5IsWvOjVZNfuSRpaE3LHnOSXYCjgQOTFDADKOCDE9j996vqivVtUFUXAhcCbLvHPrWZ5UqS9HPTtcd8MvCPVbV3VY1U1VOBZcCeA65LkqT1mq7BfCrwiTHLPgb84QBqkSRpwqblqeyqOnIdy94PvH8dy8/rmT6jn3VJkrQh07XHLEnSFslgliSpQwxmSZI6xGCWJKlDDGZJkjpkWt6VPZUOmjOLRQuPG3QZkqRpwh6zJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR0yc9AFbOmWPLiKkQVXDboMadpYvt1pgy5BE3HeqkFXMG3ZY5YkqUMMZkmSOsRgliSpQwxmSZI6xGCWJKlDDGZJkjrEYJYkqUO2uGBOsibJbUnuTHJ7krcl2WrMNp9McmPP/F8nObtn/uokF/XMvzfJ29rpmUkeSXL+FDRHkqTH2eKCGfhxVR1aVQcAvw68DPjT0ZVJdgYOA3ZO8rR28ReB57XrtwJ2BQ7oOebzgBva6ZcAdwGvSZI+tkOSpF+yJQbzz1XVQ8CZwFt6QvQk4FPAZcAp7bIbaIOZJpCXAo8meWKSbYFnAre2608F3gd8E3hu3xshSVKPLTqYAarqXpp2PLlddCpwaft1arvNt4DVSfaiCegbgS8BRwDzgDuq6rEkvwK8GPi33v3HSnJmkkVJFq35ka+lkyRNni0+mFsBSLIb8KvAF6rqbpowPrDdZrTXPBrMN/bMf7Hd5njguqr6EfAx4MQkM8Z+WFVdWFXzqmrejO1n9bFZkqRhs8UHc5KnA2uAh4D5wBOBZUmWAyP84nT26HXmg2hOZd9E02Puvb58KnBMu+9iYBfgqClohiRJwBYezElmAxcAH6iqognWY6tqpKpGgLk8/jrz8cDKqlpTVSuBnWnC+cYkOwEvAPbq2f/NjHM6W5KkftgSg/lXRh+XAv4LuAb4syQjwF40PWEAqmoZ8P0kzwGW0NyNfVPPsZYAq6rqEeBVwGeq6qc96z8JnNDeICZJUt9tceMxV9UvXfNtLQfmrGP7w3pmdxqz7oye6YuBi8esXwnM3qRCJUnaBFtij1mSpGnLYJYkqUMMZkmSOsRgliSpQwxmSZI6ZIu7K7trDpozi0ULjxt0GdI04mtuNdzsMUuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdkqoadA1btCSPAncNuo4B2hV4ZNBFDJDtH972D3PbYfPbv3dVzZ6sYqYT35W9+e6qqnmDLmJQkiyy/bZ/0HUMwjC3HWx/P3kqW5KkDjGYJUnqEIN581046AIGzPYPt2Fu/zC3HWx/33jzlyRJHWKPWZKkDjGYJUnqEIN5MyQ5NsldSb6eZMGg6+mnJE9Ncl2Srya5M8lZ7fInJfnPJPe035846Fr7KcmMJLcm+bd2fmjan2TnJFck+Vr77+CIIWv/Oe2//aVJLk2y3XRuf5K/T/JQkqU9y8Ztb5J3tL8L70ry3wdT9fRgMG+iJDOAvwNeCuwPnJpk/8FW1Vergd+rqmcCzwXe3LZ3AXBtVe0DXNvOT2dnAV/tmR+m9r8P+HRV7QccQvNzGIr2J5kD/C4wr6oOBGYApzC9238xcOyYZetsb/u74BTggHafD7a/I7UJDOZNdzjw9aq6t6oeAy4DXjHgmvqmqr5dVbe004/S/FKeQ9PmS9rNLgFeOZACp0CSPYHjgIt6Fg9F+5PsBLwQ+D8AVfVYVX2PIWl/aybwK0lmAtsD32Iat7+qPgesHLN4vPa+Arisqn5aVcuAr9P8jtQmMJg33Rzg/p75B9pl016SEeBZwJeA3arq29CEN/DkAZbWb38DvB1Y27NsWNr/dOBh4B/aU/kXJdmBIWl/VT0IvAf4JvBtYFVVXcOQtL/HeO0d2t+H/WAwb7qsY9m0f/YsyY7Ax4Czq+r7g65nqiQ5HnioqhYPupYBmQkcBnyoqp4F/JDpddp2vdprqa8AngY8BdghyWsHW1WnDOXvw34xmDfdA8BTe+b3pDm1NW0l2ZomlP+pqj7eLv5Okj3a9XsADw2qvj57PnBCkuU0ly2OTvJ/GZ72PwA8UFVfauevoAnqYWn/McCyqnq4qn4GfBx4HsPT/lHjtXfofh/2k8G86b4M7JPkaUm2obnx4coB19Q3SUJzffGrVfVXPauuBE5vp08HPjnVtU2FqnpHVe1ZVSM0/60/U1WvZXjavwK4P8l/axe9GPgKQ9J+mlPYz02yffv/wotp7rMYlvaPGq+9VwKnJNk2ydOAfYCbB1DftOCbvzZDkpfRXHecAfx9Vf3lYCvqnyQvAD4PLOEX11j/kOY6878Ae9H88np1VY29YWRaSXIkcG5VHZ9kF4ak/UkOpbnxbRvgXuC3aP64H5b2/xkwn+YJhVuBNwA7Mk3bn+RS4Eia4R2/A/wp8K+M094kfwS8jubnc3ZV/cfUVz09GMySJHWIp7IlSeoQg1mSpA4xmCVJ6hCDWZKkDjGYJUnqEINZWo8kuye5LMk3knwlyb8n2bfPn3l9knkb2ObsJNv3zP97kp0n4bOXJ1mS5PYk1yTZfXOPuQk1HNo+iigNJYNZGkf7IolPANdX1TOqan+aZ7d3G2xlAJxNM5ACAFX1snZQiclwVFUdAiyiae8GTfJIQocCBrOGlsEsje8o4GdVdcHogqq6rao+n+TI0TGZAZJ8IMkZ7fTyJO9McmOSRUkOS3J12+t+Y7vNuPv3SvKh9hh3ti+4IMnv0ryv+bok1/V85q5J3pXkf/bsf16S32unfz/Jl5PcMXqsDfgc8KtpxqB+d8++v9PThuuSfBRY0m73nrbHfUeSt7bbzU3y2SSL25/D6Csdr2/rvTnJ3Ul+rX2L3p8D85PclmT+BOqUppWZgy5A6rADgU0dtOL+qjoiyV/TjGv7fGA74E7ggvXtOMYfVdXKtkd6bZKDq+r9Sd5G07N9ZMz2l9G8je6D7fxrgGOTvITmNYmH0ww4cGWSF7ZD+43neJo3vb2eZjSlZyfZFrghyTXtNocDB1bVsiRvohnk4VlVtTrJk9r3q/8t8IqqergN2r+keUMUwMyqOrw9df2nVXVMkj+hGff4LRvxc5KmDYNZ6o/R96YvAXZsx7B+NMlPNvJa8GuSnEnz/+oewP7AHeNtXFW3JnlykqcAs4HvVtU32172S2heJQnNqyT3oekVj3VdkjXt5/wxzWs4D05ycrt+VrvvY8DN7fi70Az0cEFVrW5rWZnkQJo/cP6zuTLADJphE0eNDoayGBiZwM9DmvYMZml8dwInj7NuNY+/FLTdmPU/bb+v7ZkenZ85gf1pBwM4F3h2VX03ycXr2m4drmjr3p2mBw1NL/n8qvrwBPZ/XE+8vdb+1qq6ekx9R9IM//jzRfzyUH8B7qyqI8b5rNGfzRr8fSQBXmOW1uczwLZJfnt0QZJnJ3kRcB+wfzuaziya0YY2xkT234km+FYl2Q14ac+6R4EnjHPsy2hGwDqZJqQBrgZel2Y8bZLMSfLkcfYf62rgTe1paZLsm2SHdWx3DfDGJDPb7Z4E3AXMTnJEu2zrJAds4PPW1zZp2jOYpXFUM8LLicCvtzdu3QmcB3yrqu6nGWXnDuCf+MUp4okee4P7V9Xt7fI7gb8HbuhZfSHwH6M3f43Z706aYHuwqr7dLrsG+ChwY5IlNIE90fC7iGaIx1uSLAU+zLp7txfRjDh0R5LbgdOq6jGaPxDe1S67jWYc4/W5juaPFm/+0lBydClJkjrEHrMkSR1iMEuS1CEGsyRJHWIwS5LUIQazJEkdYjBLktQhBrMkSR3y/wF9xKHF/K29OwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ik23MDyMlcvQ"
   },
   "source": [
    "#### Different Tasks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G_02c-stlsPo"
   },
   "source": [
    "##### Distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "id": "yeMFtsl4ljug"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, 'distribution')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5HJcGI6dlwMY",
    "outputId": "eb5d2f6b-dd75-41ca-d9f4-d31823e0fc98"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=18.407, p=0.001\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "v4MlqIU7lxsY",
    "outputId": "37c8ddda-a597-4525-8c14-f3cbebefff18"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.95674</td>\n",
       "      <td>0.51598</td>\n",
       "      <td>0.74401</td>\n",
       "      <td>0.51598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.95674</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.51598</td>\n",
       "      <td>0.74401</td>\n",
       "      <td>0.51598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.51598</td>\n",
       "      <td>0.51598</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.74401</td>\n",
       "      <td>0.33890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.74401</td>\n",
       "      <td>0.74401</td>\n",
       "      <td>0.74401</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.51598</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.51598</td>\n",
       "      <td>0.51598</td>\n",
       "      <td>0.33890</td>\n",
       "      <td>0.51598</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.95674 0.51598    0.74401  0.51598\n",
       "AHP       0.95674 1.00000 0.51598    0.74401  0.51598\n",
       "DAWA      0.51598 0.51598 1.00000    0.74401  0.33890\n",
       "Geometric 0.74401 0.74401 0.74401    1.00000  0.51598\n",
       "Laplace   0.51598 0.51598 0.33890    0.51598  1.00000"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "baGIMWr-lzDI",
    "outputId": "41243577-a384-4a26-f254-d6875f655be6"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdx0lEQVR4nO3de5zWdZ338dd7GITUwAMjmgZDBAuISjKe3cBDpXlMNATbG7ZaqruDYtZSe4h2Hxu0ZZaVGre1WnfpKmli7gqtQqZpMSA6IKAmCGIk3NgoHsCZ+dx//H6TF+Mcrhmua67fXNf7+XjMg9/5+nwHmPd8f6evIgIzMzPLhqpSF2BmZmZvcjCbmZlliIPZzMwsQxzMZmZmGeJgNjMzy5DqUhfQ1w0ZMiRqa2tLXYaZWZ+yYsWK7RFRU+o6ssjBvJdqa2upr68vdRlmZn2KpGdLXUNW+VS2mZlZhjiYzczMMsTBbGZmliG+xmxmZmVjxYoVh1RXV98IjCebnc8WYHVTU9PHJ06c+EJ7GziYzcysbFRXV9946KGHjq2pqXmxqqoqc4NBtLS0aNu2beO2bt16I3B+e9tk8bcJMzOznhpfU1PzUhZDGaCqqipqamoaSXr07W/Ti/WYmZkVW1VWQ7lVWl+H+etgNjMzyxBfYzYzs7JVO+eeiYU83sb556zoapuFCxcOuuqqq4a1tLTwkY98ZPvXvva1rd35DAfzXmrY0kjtnHtKXYZZpzYOnF7qEsreUSOGlbqEgrhtXlNe241dt7bIlfRNTU1NzJ49e9jixYuffNe73vXGMcccM3bKlCl/njhx4uv5HsOnss3MzApk2bJl+w0fPnzXuHHjdg8cODAuuuiiHQsXLjygO8dwMJuZmRXI5s2b9zn88MN3t84fccQRu7ds2bJPd47hYDYzMyuQiLfeEC6pW3eJO5jNzMwKZNiwYXv0kJ977rl93vGOd7zRnWM4mM3MzApk0qRJr2zcuHHgunXr9nn99dd1xx13HDRlypQ/d+cYvivbzMzKVj6PNxVS//79ufrqqzedddZZo5ubm5k+ffr2urq6vO/IBgezmZlZQU2dOrVx6tSpjT3dP3OnsiXtLMAxaiWtLkQ9ZmZmvSlzwWxmZlbJ+kQwSzpP0u8kPSrpfyQNTZfPlfQTSfdLekrS37Wzb62k30hamX6dnLPui5IaJD0maX66bKSkeyWtSPcb03stNTOzStdXrjE/CJwYESHp48AXgc+n644GTgT2Ax6V1Pb9mC8A74uI1yWNAm4B6iSdDVwInBARr0o6KN1+AfDJiHhK0gnAdcDpuQeUNAuYBdBvUE2Bm2pmZpWsrwTzEcB/SjoM2AfYkLPuroh4DXhN0lLgeGBVzvr+wPckTQCagdHp8jOB/4iIVwEiYoek/YGTgdslte4/oG0xEbGAJMAZcNioTA8vZmZmfUtfCebvAt+KiEWSJgNzc9a1Dca287OBPwHHkJy6b71tXe1sWwX8OSIm7HXFZmZmPdBXgnkwsCWdntFm3QWS5pGcyp4MzCHpVefu+1xEtEiaAfRLly8B/lnSz1pPZae95g2SLomI25V0m4+OiMeK1TAzMyuiuYMLOuwjcxu7fC76kksuqb3vvvsGH3zwwU1PPfXUmu5+RBZv/tpX0nM5X1eS9JBvl/QbYHub7X8P3AM8AvxrRDzfZv11wAxJj5Ccxn4FICLuBRYB9ZJWAVel218GfEzSY8Aa4IJCN9DMzMrXRz/60e2LFi16qqf7Z67HHBEd/bJwVwfLn4yIWW2OsREYn04/RXKDWKsv5Ww3H5jfZt8NwFndq9rMzCxx9tln71y/fn23RpTKlcUes5mZWcXKXI+5OyJibqlrMDMzKyT3mM3MzDLEwWxmZpYhffpUtpmZWafyeLyp0M4777wRjzzyyNtffPHF6qFDhx49Z86c52fPnt32iaIOOZj30lGHD6Z+/jmlLsOsCz0egc7y1FDqAgql7ZsirNvuvvvuDV1v1TGfyjYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYgflzIzs7J11M1HFXTYx4YZDZ0+F/3000/3v+yyy0Zs27atf1VVFTNmzNj2T//0Ty905zPcYzYzMyuQ/v37c/XVVz/3zDPPrFm+fPnaH/7wh4esWLFiYHeO4WA2MzMrkOHDh79x6qmnvgpw4IEHtowcOfK1TZs2dWsISAezmZlZEaxfv36fJ554Yt9Jkybt7M5+DmYzM7MCa2xsrLroootGzp8/f/NBBx3U0p19HcxmZmYFtGvXLp1zzjkjL7nkkh0zZsz4c3f3dzCbmZkVSEtLC5deeunw0aNHvz537tw/9eQYflzKzMzKVlePNxXar371q/1/8YtfHDxq1KjXxowZMw7gq1/96papU6fmPcSbg9nMzKxAPvCBD+yMiL36ZcCnss3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeLHpfZSw5ZGaufcU+oyrEg2Dpxe6hLK3lEjhpW6hD7ttnlNJfncsevWluRzu2vtmLEFHfZx7Lq1nT4K9eqrr+qEE04Ys3v3bjU3N+u888578Zprrnm+O5/hYDYzMyuQgQMHxoMPPrh+8ODBLbt27dJxxx33V/fdd1/jGWec8Uq+x/CpbDMzswKpqqpi8ODBLQC7d+9WU1OTJHXvGEWpzMzMrEI1NTUxZsyYcUOHDj1m0qRJL51++ul595bBwWxmZlZQ1dXVrFu37olNmzY9vnLlyv2WL18+sDv7O5jNzMyKYMiQIc2nnnrqy3fffffg7uznYDYzMyuQ559/vnr79u39AHbu3Klly5YNGjt27OvdOUZR78qWNBS4BjgReBHYDfx7RNxZzM9tp47JwO6I+G0H688HxkXE/N6sy8zMiqurx5sKbfPmzf1nzpw5orm5mYjQBRdcsGPatGl5D/kIRQzm9Da0XwA3R8T0dNlw4PxifWYnJgM7gbcEs6TqiFgELOrtoszMrLyccMIJr61du/aJvTlGMU9ln07SS72hdUFEPBsR35XUT9I3JC2X9LikT0AS5uny1ZIaJE1Nl0+W9GtJt0l6UtJ8SZdJ+n263ch0uxpJP0+Pu1zSKZJqgU8CsyWtkvTXkm6S9C1JS4GvS5op6XvpMYZKulPSY+nXyUX8HpmZme2hmKeyjwRWdrDuY0BjRBwnaQDwkKQlwLHABOAYYAiwXNID6T7HAGOBHcAzwI0Rcbyky4HPAlcA3wGuiYgHJQ0DFkfEWEk3ADsj4psAkj4GjAbOjIhmSTNzarsW+HVEfEhSP2D/QnwzzMzM8tFrb/6S9H3gVJLrzM8CR0u6OF09GBiVrr8lIpqBP0n6NXAc8BKwPCL+mB7rD8CSdN8G4LR0+kxgXM7D3IMkvb2Dkm5PP6et04H/BZCuf8u1AUmzgFkA/QbVdN14MzPrLS0tLS2qqqqKUhfSkZaWFgEtHa0vZjCvAaa0zkTEpyUNAeqBTcBnI2Jx7g6SPtjJ8XblTLfkzLfwZjuqgJMi4rU2x23veN164DtXRCwAFgAMOGxUZv/yzcwq0Opt27aNq6mpacxiOLe0tGjbtm2DgdUdbVPMYL4f+JqkT0XE9emyfdM/FwOfknR/RLwhaTSwBXgA+ISkm4GDgPcCXwDG5PmZS4DPAN8AkDQhIlYBLwOD8jzGfcCngG+np7L3i4iX8tzXzMxKqKmp6eNbt269cevWrePJ5iPBLcDqpqamj3e0QdGCOSJC0oXANZK+CGwj6aX+PXA7UAusTO/e3gZcCNwJnAQ8BgTwxYjYKinfYP4c8H1Jj5O07QGSG7/uBhZKuoDkenRnLgcWpNehm0lC+uE8P9/MzEpo4sSJL1Cap38KRhGZ6+n3KQMOGxWHzfh2qcuwIvGwj8XnYR/3Tl8d9lHSioioK1A5ZSWL3XwzM7OK5WA2MzPLEAezmZlZhjiYzczMMsTBbGZmliG+K3sv1dXVRX19fanLMDPrU3xXdsfcYzYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGVOezkaQDgXfmbh8RK4tVlJmZWaXqMpgl/SswE/gD0Ppi7QBOL15ZZmZmlSmfHvOHgZERsbvYxZiZmVW6fK4xrwYOKHIdZmZmRn495nnAo5JWA7taF0bE+UWryszMrELlE8w3A18HGoCW4pZjZmZW2fIJ5u0RcW3RKzEzM7O8gnmFpHnAIvY8le3HpczMzAosn2B+T/rniTnL/LiUmZlZEXQZzBFxWm8UYmZmZvm9YGQAMAWoZc83f/1L8coyMzOrTPmcyr4LaARWkHON2RINWxqpnXNPqcuwItk4cHqpS7AiOWrEsFKXkEm3zWvKa7ux69YWuZLKlU8wHxERZxW9EjMzM8vrzV+/lXRU0SsxMzOzjnvMkhpI7r6uBv5W0jMkp7IFREQc3TslmpmZVY7OTmWf22tVmJmZGdBJMEfEswCSfhIRf5O7TtJPgL9pd0czMzPrsXyuMR+ZOyOpHzCxOOWYmZlVtg6DWdKXJL0MHC3ppfTrZeAFkkeozMzMrMA6DOaImBcRbwe+ERGD0q+3R8TBEfGlXqyxQ5I+JCkkjclZdrykZZKekrRS0j0d3VUu6XxJczpYt7NYdZuZmXWks7uyx0TEOuB2Sce2XZ+RQSymAQ8ClwJzJQ0FbgOmR8RvASSdCowkGbbyLyRVR8QiksE5zMzMMqGzu7KvBGYBV7ezruSDWEjaHzgFOI0kXOcCnwFubg1lgIh4MGefm4AdJANzrEwfCauLiM9IGgH8jOR7cm8vNcPMzGwPnd2VPUtSFfCPEfFQL9aUrwuBeyPiSUk70l79kcDNXew3GjgzIpolzcxZ/h3g+oj4saRPd3YASbNIfmmh36CantZvZmb2Fp3elR0RLcA3e6mW7poG3JpO35rO70HS7yStlfSdnMW3R0RzO8c7Bbglnf5JZx8cEQsioi4i6vrtO7gHpZuZmbUvn3dlL5E0BbgjIqLYBeVD0sEkp9LHSwqgH8np9ZuBY0nvGo+IEyRdzJ4vS3mlk0Nnon1mZla58nmO+UrgdmB36yNTkl4qcl1duRj4cUQMj4jaiHgnsAFYAsyUdHLOtvvmecyHSG4iA7iscKWamZnlr8sec/rIVNZMA+a3WfZzYDowFfi6pMNJnrneDuQzdvTlwM8kXZ4ey8zMrNflcyobSecD701nl0XEL4tXUtciYnI7y67NmZ3UwX4z28zfBNyUTm8ATspZ3Tb4zczMiq7LU9mS5pP0Jp9Ivy5Pl5mZmVmB5dNj/iAwIb1DG0k3A48C7b4xy8zMzHoun5u/AA7ImfbzQWZmZkWST495HvCopKWASK41Z+Jd2WZmZuUmn7uyb5G0DDiOJJj/PiK2FrswMzOzSqSu3hnS3gAWQCPwbEQ0FaWqPqSuri7q6+tLXYaZWZ8iaUVE1JW6jizK51T2dSRv03qcpMc8Pp0+WNInI2JJEeszMzOrKPnc/LUReE/6buiJJCMzrQbOBP69iLWZmZlVnHyCeUxErGmdiYgnSIL6meKVZWZmVpnyOZW9XtL1vDmS01TgSUkDgDeKVpmZmVkFyqfHPBN4GrgCmA08ky57AzitSHWZmZlVpHwel3oNuDr9amtnwSsyMzOrYB0Gs6QGOh6fOCLimOKUZGZmVrk66zGf284yAUcAXy5OOWZmZpWtw2COiGdbpyVNIBnr+MPABjxesZmZWVF0dip7NHApMA34f8B/krwpzDd8mZmZFUlnp7LXAb8BzouIpwEkze6VqszMzCpUZ49LTQG2Aksl/R9JZ5BcYzYzM7Mi6TCYI+LOiJgKjAGWkTzDPFTS9ZLe30v1mZmZVZQuXzASEa9ExE8j4lySO7JXAXOKXZiZmVklyufNX38RETsi4gcRcXqxCjIzM6tk3QpmMzMzKy4Hs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZllSJfjMVvnGrY0UjvnnlKXYWVm48DppS4hc44aMazUJXTqtnlNpS6hV41dt7bUJZQt95jNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhlS1sEs6UOSQtKYdL5W0uo228yVdFU6fZOkDZJWSVop6aRS1G1mZpWrrIMZmAY8CFzajX2+EBETSMac/kExijIzM+tI2QazpP2BU4CP0b1gbvUA8O6CFmVmZtaFcn7z14XAvRHxpKQdko4FdgAjJa3K2e5Q4Jvt7H8e0FD0Ks3MzHKUbY+Z5DT2ren0rek8wB8iYkLrF3BDm/2+kQb3LJLe9ltImiWpXlJ986uNha/czMwqVln2mCUdDJwOjJcUQD8ggOvy2P0LEbGwsw0iYgGwAGDAYaNiL8s1MzP7i3LtMV8M/DgihkdEbUS8E9gAHFHiuszMzDpVrsE8DbizzbKfA18uQS1mZmZ5K8tT2RExuZ1l1wLXtrN8bs70zGLWZWZm1pVy7TGbmZn1SQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhmiCL8fY2/U1dVFfX19qcswM+tTJK2IiLpS15FF7jGbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYZUl7qAvq5hSyO1c+4pdRmZs3Hg9FKXYBXqqBHDSl1Cr7ptXlNJPnfsurUl+dxK4B6zmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ0ueCWVKzpFWS1kh6TNKVkqrabHOXpIdz5q+RdEXO/GJJN+bMXy3pynS6WtJ2SfN6oTlmZmZ76HPBDLwWERMi4kjgfcAHga+0rpR0AHAscICkEeni3wInp+urgCHAkTnHPBl4KJ1+P7Ae+LAkFbEdZmZmb9EXg/kvIuIFYBbwmZwQnQLcDdwKXJoue4g0mEkCeTXwsqQDJQ0AxgKPpuunAd8BNgEnFr0RZmZmOfp0MANExDMk7TgkXTQNuCX9mpZu8zzQJGkYSUA/DPwOOAmoAx6PiN2S3gacAfwyd/+2JM2SVC+pvvnVxqK1zczMKk+fD+aUACQNBd4NPBgRT5KE8fh0m9Zec2swP5wz/9t0m3OBpRHxKvBz4EOS+rX9sIhYEBF1EVHXb9/BRWyWmZlVmj4fzJLeBTQDLwBTgQOBDZI2ArW8eTq79TrzUSSnsh8h6THnXl+eBpyZ7rsCOBg4rReaYWZmBvTxYJZUA9wAfC8igiRYz4qI2oioBSay53Xmc4EdEdEcETuAA0jC+WFJg4BTgWE5+3+aDk5nm5mZFUNfDOa3tT4uBfwPsAT4qqRaYBhJTxiAiNgAvCTpBKCB5G7sR3KO1QA0RsR24CLg/ojYlbP+LuD89AYxMzOzoutz4zFHxFuu+aY2Aoe3s/2xObOD2qybmTN9E3BTm/U7gJoeFWpmZtYDfbHHbGZmVrYczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyRMnjv9ZTdXV1UV9fX+oyzMz6FEkrIqKu1HVkkXvMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxC/knMvSXoZWF/qOkpoCLC91EWUkNtfue2v5LbD3rd/eETUFKqYclJd6gLKwPpKft+rpHq33+0vdR2lUMltB7e/mHwq28zMLEMczGZmZhniYN57C0pdQIm5/ZWtkttfyW0Ht79ofPOXmZlZhrjHbGZmliEOZjMzswxxMO8FSWdJWi/paUlzSl1PMUl6p6SlktZKWiPp8nT5QZJ+Jemp9M8DS11rMUnqJ+lRSb9M5yum/ZIOkLRQ0rr038FJFdb+2em//dWSbpE0sJzbL+lHkl6QtDpnWYftlfSl9GfhekkfKE3V5cHB3EOS+gHfB84GxgHTJI0rbVVF1QR8PiLGAicCn07bOwe4LyJGAfel8+XscmBtznwltf87wL0RMQY4huT7UBHtl3Q48DmgLiLGA/2ASynv9t8EnNVmWbvtTX8WXAocme5zXfoz0nrAwdxzxwNPR8QzEbEbuBW4oMQ1FU1E/DEiVqbTL5P8UD6cpM03p5vdDFxYkgJ7gaQjgHOAG3MWV0T7JQ0C3gv8ECAidkfEn6mQ9qeqgbdJqgb2BZ6njNsfEQ8AO9os7qi9FwC3RsSuiNgAPE3yM9J6wMHcc4cDm3Pmn0uXlT1JtcB7gN8BQyPij5CEN3BICUsrtm8DXwRacpZVSvvfBWwD/iM9lX+jpP2okPZHxBbgm8Am4I9AY0QsoULan6Oj9lbsz8NicDD3nNpZVvbPnknaH/g5cEVEvFTqenqLpHOBFyJiRalrKZFq4Fjg+oh4D/AK5XXatlPptdQLgBHAO4D9JH2ktFVlSkX+PCwWB3PPPQe8M2f+CJJTW2VLUn+SUP5pRNyRLv6TpMPS9YcBL5SqviI7BThf0kaSyxanS/q/VE77nwOei4jfpfMLSYK6Utp/JrAhIrZFxBvAHcDJVE77W3XU3or7eVhMDuaeWw6MkjRC0j4kNz4sKnFNRSNJJNcX10bEt3JWLQJmpNMzgLt6u7beEBFfiogjIqKW5O/6/oj4CJXT/q3AZkl/lS46A3iCCmk/ySnsEyXtm/5fOIPkPotKaX+rjtq7CLhU0gBJI4BRwO9LUF9Z8Ju/9oKkD5Jcd+wH/Cgi/q20FRWPpFOB3wANvHmN9csk15lvA4aR/PC6JCLa3jBSViRNBq6KiHMlHUyFtF/SBJIb3/YBngH+luSX+0pp/1eBqSRPKDwKfBzYnzJtv6RbgMkkwzv+CfgK8As6aK+kfwA+SvL9uSIi/rv3qy4PDmYzM7MM8alsMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7NZJyQdKulWSX+Q9ISk/5I0usifuUxSXRfbXCFp35z5/5J0QAE+e6OkBkmPSVoi6dC9PWYPapiQPopoVpEczGYdSF8kcSewLCJGRsQ4kme3h5a2MgCuIBlIAYCI+GA6qEQhnBYRxwD1JO3tUoFHEpoAOJitYjmYzTp2GvBGRNzQuiAiVkXEbyRNbh2TGUDS9yTNTKc3SvqapIcl1Us6VtLitNf9yXSbDvfPJen69Bhr0hdcIOlzJO9rXippac5nDpH0dUn/O2f/uZI+n05/QdJySY+3HqsLDwDvVjIG9Tdy9v1EThuWSvoZ0JBu9820x/24pM+m202U9GtJK9LvQ+srHZel9f5e0pOS/jp9i96/AFMlrZI0NY86zcpKdakLMMuw8UBPB63YHBEnSbqGZFzbU4CBwBrghs52bOMfImJH2iO9T9LREXGtpCtJerbb22x/K8nb6K5L5z8MnCXp/SSvSTyeZMCBRZLemw7t15FzSd709jGS0ZSOkzQAeEjSknSb44HxEbFB0qdIBnl4T0Q0SToofb/6d4ELImJbGrT/RvKGKIDqiDg+PXX9lYg4U9I/k4x7/JlufJ/MyoaD2aw4Wt+b3gDsn45h/bKk17t5LfjDkmaR/F89DBgHPN7RxhHxqKRDJL0DqAFejIhNaS/7/SSvkoTkVZKjSHrFbS2V1Jx+zj+SvIbzaEkXp+sHp/vuBn6fjr8LyUAPN0REU1rLDknjSX7B+VVyZYB+JMMmtmodDGUFUJvH98Os7DmYzTq2Bri4g3VN7HkpaGCb9bvSP1typlvnq/PYn3QwgKuA4yLiRUk3tbddOxamdR9K0oOGpJc8LyJ+kMf+e/TE02vtn42IxW3qm0wy/ONfFvHWof4ErImIkzr4rNbvTTP+eWQG+BqzWWfuBwZI+rvWBZKOkzQJeBYYl46mM5hktKHuyGf/QSTB1yhpKHB2zrqXgbd3cOxbSUbAupgkpAEWAx9VMp42kg6XdEgH+7e1GPhUeloaSaMl7dfOdkuAT0qqTrc7CFgP1Eg6KV3WX9KRXXxeZ20zK3sOZrMORDLCy4eA96U3bq0B5gLPR8RmklF2Hgd+ypuniPM9dpf7R8Rj6fI1wI+Ah3JWLwD+u/Xmrzb7rSEJti0R8cd02RLgZ8DDkhpIAjvf8LuRZIjHlZJWAz+g/d7tjSQjDj0u6TFgekTsJvkF4evpslUk4xh3ZinJLy2++csqkkeXMjMzyxD3mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzy5D/Dw/8vIv+Z7mkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Bhnn_LCBlzXL"
   },
   "source": [
    "##### Correlation\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "id": "IXQCsTkLl2iQ"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, 'correlation')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "BxapZiRzl2kQ",
    "outputId": "328102a1-c86e-4d1a-80ce-721b7330045a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=9.258, p=0.055\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "tqQ1B4Ucl2mg",
    "outputId": "9efebc9b-0f13-4b82-e849-7a7403389e00"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.87291</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.88748</td>\n",
       "      <td>0.87291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.88748</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.87291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>0.87291</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.87291 0.87291    0.87291  0.87291\n",
       "AHP       0.87291 1.00000 0.87291    0.87291  0.87291\n",
       "DAWA      0.87291 0.87291 1.00000    0.88748  0.87291\n",
       "Geometric 0.87291 0.87291 0.88748    1.00000  0.87291\n",
       "Laplace   0.87291 0.87291 0.87291    0.87291  1.00000"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "0ovVN0yFl2op",
    "outputId": "01e2db0e-6df9-4dc0-bec0-35af603fce71"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdt0lEQVR4nO3de5zWdZ338dd7GIQ8gAdGPAVjBAuISjKe3cBDpXlMNETbG7ZaqruDYtZSe4h2Hxu0ZZaVGre1WnfJKmli7gqtQqZpMSA6IKAmCGIk3NgoHsCZ+dx//H6TF+NcM9cMc831m+t6Px+PefA7X5/vAPOe7+/0VURgZmZm2VBV6gLMzMzsLQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhlSXeoC+rohQ4ZEbW1tqcswM+tTli9fvi0iakpdRxY5mPdQbW0t9fX1pS7DzKxPkfRcqWvIKp/KNjMzyxAHs5mZWYY4mM3MzDLE15jNzKxsLF++/ODq6uqbgXFks/PZAqxqamr6+IQJE15sbwMHs5mZlY3q6uqbDznkkDE1NTUvVVVVZW4wiJaWFm3dunXsli1bbgYuaG+bLP42YWZm1l3jampqXs5iKANUVVVFTU1NI0mPvv1terEeMzOzYqvKaii3SuvLm78OZjMzswzxNWYzMytbtbPundCTx9sw99zlnW2zYMGCQddcc82wlpYWPvKRj2z72te+tqUrn+Fg3kMNmxupnXVvqcswy5wNAy8vdQkdOvrIYaUuoUO3z2kqdQkdGrN2TalLyKSmpiZmzpw5bNGiRU+9613vevPYY48dM3ny5D9PmDDhjUKP4VPZZmZmPWTp0qX7DB8+fOfYsWN3DRw4MC6++OLtCxYs2L8rx3Awm5mZ9ZBNmzbtdfjhh+9qnT/iiCN2bd68ea+uHMPBbGZm1kMi3n5DuKQu3SXuYDYzM+shw4YN262H/Pzzz+912GGHvdmVYziYzczMesjEiRNf3bBhw8C1a9fu9cYbb+jOO+88cPLkyX/uyjF8V7aZmZWtQh5v6kn9+/fn2muv3Xj22WePam5u5vLLL99WV1dX8B3Z4GA2MzPrUVOmTGmcMmVKY3f3z9ypbEk7euAYtZJW9UQ9ZmZmvSlzwWxmZlbJ+kQwSzpf0u8kPSbpfyQNTZfPlvQTSQ9IelrS37Wzb62k30hakX6dkrPui5IaJD0uaW66bISk+yQtT/cb3XstNTOzStdXrjE/BJwUESHp48AXgc+n644BTgL2AR6T1Pb9mC8C74uINySNBG4D6iSdA1wEnBgRr0k6MN1+HvDJiHha0onADcAZuQeUNAOYAdBvUE0PN9XMzCpZXwnmI4D/lHQosBewPmfd3RHxOvC6pCXACcDKnPX9ge9JGg80A6PS5WcB/xERrwFExHZJ+wKnAHdIat1/QNtiImIeSYAz4NCRmR5ezMzM+pa+EszfBb4VEQslTQJm56xrG4xt52cCfwKOJTl133rbutrZtgr4c0SM3+OKzczMuqGvBPNgYHM6Pa3NugslzSE5lT0JmEXSq87d9/mIaJE0DeiXLl8M/LOkn7Weyk57zeslXRoRdyjpNh8TEY8Xq2FmZlZEswf36LCPzG7s9LnoSy+9tPb+++8ffNBBBzU9/fTTq7v6EVm8+WtvSc/nfF1N0kO+Q9JvgG1ttv89cC/wKPCvEfFCm/U3ANMkPUpyGvtVgIi4D1gI1EtaCVyTbn8F8DFJjwOrgQt7uoFmZla+PvrRj25buHDh093dP3M95ojI98vC3XmWPxURM9ocYwMwLp1+muQGsVZfytluLjC3zb7rgbO7VrWZmVninHPO2bFu3boujSiVK4s9ZjMzs4qVuR5zV0TE7FLXYGZm1pPcYzYzM8sQB7OZmVmG9OlT2WZmZh0q4PGmnnb++ecf+eijj+730ksvVQ8dOvSYWbNmvTBz5sy2TxTl5WDeQ0cfPpj6ueeWugyzDOr2qHe9oqHUBXSm7RsbrM+455571ne+VX4+lW1mZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxA/LmVmZmXr6FuP7tFhHxumNXT4XPQzzzzT/4orrjhy69at/auqqpg2bdrWf/qnf3qxK5/hHrOZmVkP6d+/P9dee+3zzz777Oply5at+eEPf3jw8uXLB3blGA5mMzOzHjJ8+PA3TzvttNcADjjggJYRI0a8vnHjxi4NAelgNjMzK4J169bt9eSTT+49ceLEHV3Zz8FsZmbWwxobG6suvvjiEXPnzt104IEHtnRlXwezmZlZD9q5c6fOPffcEZdeeun2adOm/bmr+zuYzczMekhLSwuXXXbZ8FGjRr0xe/bsP3XnGH5cyszMylZnjzf1tF/96lf7/uIXvzho5MiRr48ePXoswFe/+tXNU6ZMKXi4NQezmZlZD/nABz6wIyL26JcBn8o2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIH5faQw2bG6mddW+py7Ai2W/MrFKX0GfdPqep1CVYEY1Zu6bUJRRkzegxPTrs45i1azp8FOq1117TiSeeOHrXrl1qbm7W+eef/9J11133Qlc+w8FsZmbWQwYOHBgPPfTQusGDB7fs3LlTxx9//F/df//9jWeeeearhR7Dp7LNzMx6SFVVFYMHD24B2LVrl5qamiSpa8coSmVmZmYVqqmpidGjR48dOnTosRMnTnz5jDPOKLi3DA5mMzOzHlVdXc3atWuf3Lhx4xMrVqzYZ9myZQO7sr+D2czMrAiGDBnSfNppp71yzz33DO7Kfg5mMzOzHvLCCy9Ub9u2rR/Ajh07tHTp0kFjxox5oyvHKOpd2ZKGAtcBJwEvAbuAf4+Iu4r5ue3UMQnYFRG/zbP+AmBsRMztzbrMzKy4Onu8qadt2rSp//Tp049sbm4mInThhRdunzp1asFDPkIRgzm9De0XwK0RcXm6bDhwQbE+swOTgB3A24JZUnVELAQW9nZRZmZWXk488cTX16xZ8+SeHKOYp7LPIOml3tS6ICKei4jvSuon6RuSlkl6QtInIAnzdPkqSQ2SpqTLJ0n6taTbJT0laa6kKyT9Pt1uRLpdjaSfp8ddJulUSbXAJ4GZklZK+mtJt0j6lqQlwNclTZf0vfQYQyXdJenx9OuUIn6PzMzMdlPMU9lHASvyrPsY0BgRx0saADwsaTFwHDAeOBYYAiyT9GC6z7HAGGA78Cxwc0ScIOlK4LPAVcB3gOsi4iFJw4BFETFG0k3Ajoj4JoCkjwGjgLMiolnS9Jzargd+HREfktQP2LcnvhlmZmaF6LU3f0n6PnAayXXm54BjJF2Srh4MjEzX3xYRzcCfJP0aOB54GVgWEX9Mj/UHYHG6bwNwejp9FjA252HuQZL2y1PSHenntHUG8L8A0vVvuzYgaQYwA6DfoJrOG29mZr2lpaWlRVVVVVHqQvJpaWkR0JJvfTGDeTUwuXUmIj4taQhQD2wEPhsRi3J3kPTBDo63M2e6JWe+hbfaUQWcHBGvtzlue8fr0gPfuSJiHjAPYMChIzP7l29mVoFWbd26dWxNTU1jFsO5paVFW7duHQysyrdNMYP5AeBrkj4VETemy/ZO/1wEfErSAxHxpqRRwGbgQeATkm4FDgTeC3wBGF3gZy4GPgN8A0DS+IhYCbwCDCrwGPcDnwK+nZ7K3iciXi5wXzMzK6GmpqaPb9my5eYtW7aMI5uPBLcAq5qamj6eb4OiBXNEhKSLgOskfRHYStJL/XvgDqAWWJHevb0VuAi4CzgZeBwI4IsRsUVSocH8OeD7kp4gaduDJDd+3QMskHQhyfXojlwJzEuvQzeThPQjBX6+mZmV0IQJE16kNE//9BhFZK6n36cMOHRkHDrt26Uuw4rEwz52n4d9LG97OuyjpOURUddD5ZSVLHbzzczMKpaD2czMLEMczGZmZhniYDYzM8sQB7OZmVmG+K7sPVRXVxf19fWlLsPMrE/xXdn5ucdsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDKkuZCNJBwDvzN0+IlYUqygzM7NK1WkwS/pXYDrwB6D1xdoBnFG8sszMzCpTIT3mDwMjImJXsYsxMzOrdIVcY14F7F/kOszMzIzCesxzgMckrQJ2ti6MiAuKVpWZmVmFKiSYbwW+DjQALcUtx8zMrLIVEszbIuL6oldiZmZmBQXzcklzgIXsfirbj0uZmZn1sEKC+T3pnyflLPPjUmZmZkXQaTBHxOm9UYiZmZkV9oKRAcBkoJbd3/z1L8Ury8zMrDIVcir7bqARWE7ONWZLNGxupHbWvaUuwyrUfmNmlboEK5Lb5zSVuoQOjVm7ptQllK1CgvmIiDi76JWYmZlZQW/++q2ko4teiZmZmeXvMUtqILn7uhr4W0nPkpzKFhARcUzvlGhmZlY5OjqVfV6vVWFmZmZAB8EcEc8BSPpJRPxN7jpJPwH+pt0dzczMrNsKucZ8VO6MpH7AhOKUY2ZmVtnyBrOkL0l6BThG0svp1yvAiySPUJmZmVkPyxvMETEnIvYDvhERg9Kv/SLioIj4Ui/WmJekD0kKSaNzlp0gaamkpyWtkHRvvrvKJV0gqd0HQSXtKFbdZmZm+XR0V/boiFgL3CHpuLbrMzKIxVTgIeAyYLakocDtwOUR8VsASacBI0iGrfwLSdURsZBkcA4zM7NM6Oiu7KuBGcC17awr+SAWkvYFTgVOJwnX2cBngFtbQxkgIh7K2ecWYDvJwBwr0kfC6iLiM5KOBH5G8j25r5eaYWZmtpuO7sqeIakK+MeIeLgXayrURcB9EfGUpO1pr/4o4NZO9hsFnBURzZKm5yz/DnBjRPxY0qc7OoCkGSS/tNBvUE136zczM3ubDu/KjogW4Ju9VEtXTQXmp9Pz0/ndSPqdpDWSvpOz+I6IaG7neKcCt6XTP+nogyNiXkTURURdv70Hd6N0MzOz9hXyruzFkiYDd0ZEFLugQkg6iORU+jhJAfQjOb1+K3Ac6V3jEXGipEvY/WUpr3Zw6Ey0z8zMKlchzzFfDdwB7Gp9ZErSy0WuqzOXAD+OiOERURsR7wTWA4uB6ZJOydl27wKP+TDJTWQAV/RcqWZmZoXrtMecPjKVNVOBuW2W/Ry4HJgCfF3S4STPXG8DChk7+krgZ5KuTI9lZmbW6wo5lY2kC4D3prNLI+KXxSupcxExqZ1l1+fMTsyz3/Q287cAt6TT64GTc1a3DX4zM7Oi6/RUtqS5JL3JJ9OvK9NlZmZm1sMK6TF/EBif3qGNpFuBx4B235hlZmZm3VfIzV8A++dM+/kgMzOzIimkxzwHeEzSEkAk15oz8a5sMzOzclPIXdm3SVoKHE8SzH8fEVuKXZiZmVklUmfvDGlvAAugEXguIpqKUlUfUldXF/X19aUuw8ysT5G0PCLqSl1HFhVyKvsGkrdpPUHSYx6XTh8k6ZMRsbiI9ZmZmVWUQm7+2gC8J3039ASSkZlWAWcB/17E2szMzCpOIcE8OiJWt85ExJMkQf1s8coyMzOrTIWcyl4n6UbeGslpCvCUpAHAm0WrzMzMrAIV0mOeDjwDXAXMBJ5Nl70JnF6kuszMzCpSIY9LvQ5cm361taPHKzIzM6tgeYNZUgP5xyeOiDi2OCWZmZlVro56zOe1s0zAEcCXi1OOmZlZZcsbzBHxXOu0pPEkYx1/GFiPxys2MzMrio5OZY8CLgOmAv8P+E+SN4X5hi8zM7Mi6ehU9lrgN8D5EfEMgKSZvVKVmZlZherocanJwBZgiaT/I+lMkmvMZmZmViR5gzki7oqIKcBoYCnJM8xDJd0o6f29VJ+ZmVlF6fQFIxHxakT8NCLOI7kjeyUwq9iFmZmZVaJC3vz1FxGxPSJ+EBFnFKsgMzOzStalYDYzM7PicjCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYZ0Oh6zdaxhcyO1s+4tdRlWJBsGXl7qEvqso48cVuoSOnT7nKZSl9CnjVm7ptQllC33mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGVLWwSzpQ5JC0uh0vlbSqjbbzJZ0TTp9i6T1klZKWiHp5FLUbWZmlausgxmYCjwEXNaFfb4QEeNJxpz+QTGKMjMzy6dsg1nSvsCpwMfoWjC3ehB4d48WZWZm1olyfvPXRcB9EfGUpO2SjgO2AyMkrczZ7hDgm+3sfz7QUPQqzczMcpRtj5nkNPb8dHp+Og/wh4gY3/oF3NRmv2+kwT2DpLf9NpJmSKqXVN/8WmPPV25mZhWrLHvMkg4CzgDGSQqgHxDADQXs/oWIWNDRBhExD5gHMODQkbGH5ZqZmf1FufaYLwF+HBHDI6I2It4JrAeOKHFdZmZmHSrXYJ4K3NVm2c+BL5egFjMzs4KV5ansiJjUzrLrgevbWT47Z3p6MesyMzPrTLn2mM3MzPokB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDFGE34+xJ+rq6qK+vr7UZZiZ9SmSlkdEXanryCL3mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQ6pLXUBf17C5kdpZ95a6jLw2DLy81CWYZdKa+YeVuoQ+bczaNaUuoWy5x2xmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYb0uWCW1CxppaTVkh6XdLWkqjbb3C3pkZz56yRdlTO/SNLNOfPXSro6na6WtE3SnF5ojpmZ2W76XDADr0fE+Ig4Cngf8EHgK60rJe0PHAfsL+nIdPFvgVPS9VXAEOConGOeAjycTr8fWAd8WJKK2A4zM7O36YvB/BcR8SIwA/hMTohOBu4B5gOXpcseJg1mkkBeBbwi6QBJA4AxwGPp+qnAd4CNwElFb4SZmVmOPh3MABHxLEk7Dk4XTQVuS7+mptu8ADRJGkYS0I8AvwNOBuqAJyJil6R3AGcCv8zdvy1JMyTVS6pvfq2xaG0zM7PK0+eDOSUASUOBdwMPRcRTJGE8Lt2mtdfcGsyP5Mz/Nt3mPGBJRLwG/Bz4kKR+bT8sIuZFRF1E1PXbe3ARm2VmZpWmzwezpHcBzcCLwBTgAGC9pA1ALW+dzm69znw0yansR0l6zLnXl6cCZ6X7LgcOAk7vhWaYmZkBfTyYJdUANwHfi4ggCdazI6I2ImqBCex+nfk8YHtENEfEdmB/knB+RNIg4DRgWM7+nybP6WwzM7Ni6IvB/I7Wx6WA/wEWA1+VVAsMI+kJAxAR64GXJZ0INJDcjf1ozrEagMaI2AZcDDwQETtz1t8NXJDeIGZmZlZ0fW485oh42zXf1Abg8Ha2Py5ndlCbddNzpm8BbmmzfjtQ061CzczMuqEv9pjNzMzKloPZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYYoefzXuquuri7q6+tLXYaZWZ8iaXlE1JW6jixyj9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4ldy7iFJrwDrSl1HCQ0BtpW6iBJy+yu3/ZXcdtjz9g+PiJqeKqacVJe6gDKwrpLf9yqp3u13+0tdRylUctvB7S8mn8o2MzPLEAezmZlZhjiY99y8UhdQYm5/Zavk9ldy28HtLxrf/GVmZpYh7jGbmZlliIPZzMwsQxzMe0DS2ZLWSXpG0qxS11NMkt4paYmkNZJWS7oyXX6gpF9Jejr984BS11pMkvpJekzSL9P5imm/pP0lLZC0Nv13cHKFtX9m+m9/laTbJA0s5/ZL+pGkFyWtylmWt72SvpT+LFwn6QOlqbo8OJi7SVI/4PvAOcBYYKqksaWtqqiagM9HxBjgJODTaXtnAfdHxEjg/nS+nF0JrMmZr6T2fwe4LyJGA8eSfB8qov2SDgc+B9RFxDigH3AZ5d3+W4Cz2yxrt73pz4LLgKPSfW5If0ZaNziYu+8E4JmIeDYidgHzgQtLXFPRRMQfI2JFOv0KyQ/lw0nafGu62a3ARSUpsBdIOgI4F7g5Z3FFtF/SIOC9wA8BImJXRPyZCml/qhp4h6RqYG/gBcq4/RHxILC9zeJ87b0QmB8ROyNiPfAMyc9I6wYHc/cdDmzKmX8+XVb2JNUC7wF+BwyNiD9CEt7AwSUsrdi+DXwRaMlZVintfxewFfiP9FT+zZL2oULaHxGbgW8CG4E/Ao0RsZgKaX+OfO2t2J+HxeBg7j61s6zsnz2TtC/wc+CqiHi51PX0FknnAS9GxPJS11Ii1cBxwI0R8R7gVcrrtG2H0mupFwJHAocB+0j6SGmrypSK/HlYLA7m7nseeGfO/BEkp7bKlqT+JKH804i4M138J0mHpusPBV4sVX1FdipwgaQNJJctzpD0f6mc9j8PPB8Rv0vnF5AEdaW0/yxgfURsjYg3gTuBU6ic9rfK196K+3lYTA7m7lsGjJR0pKS9SG58WFjimopGkkiuL66JiG/lrFoITEunpwF393ZtvSEivhQRR0RELcnf9QMR8REqp/1bgE2S/ipddCbwJBXSfpJT2CdJ2jv9v3AmyX0WldL+VvnauxC4TNIASUcCI4Hfl6C+suA3f+0BSR8kue7YD/hRRPxbaSsqHkmnAb8BGnjrGuuXSa4z3w4MI/nhdWlEtL1hpKxImgRcExHnSTqICmm/pPEkN77tBTwL/C3JL/eV0v6vAlNInlB4DPg4sC9l2n5JtwGTSIZ3/BPwFeAX5GmvpH8APkry/bkqIv6796suDw5mMzOzDPGpbDMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezWQckHSJpvqQ/SHpS0n9JGlXkz1wqqa6Tba6StHfO/H9J2r8HPnuDpAZJj0taLOmQPT1mN2oYnz6KaFaRHMxmeaQvkrgLWBoRIyJiLMmz20NLWxkAV5EMpABARHwwHVSiJ5weEccC9STt7VQPjyQ0HnAwW8VyMJvldzrwZkTc1LogIlZGxG8kTWodkxlA0vckTU+nN0j6mqRHJNVLOk7SorTX/cl0m7z755J0Y3qM1ekLLpD0OZL3NS+RtCTnM4dI+rqk/52z/2xJn0+nvyBpmaQnWo/ViQeBdysZg/obOft+IqcNSyT9DGhIt/tm2uN+QtJn0+0mSPq1pOXp96H1lY5L03p/L+kpSX+dvkXvX4ApklZKmlJAnWZlpbrUBZhl2Digu4NWbIqIkyVdRzKu7anAQGA1cFNHO7bxDxGxPe2R3i/pmIi4XtLVJD3bbW22n0/yNrob0vkPA2dLej/JaxJPIBlwYKGk96ZD++VzHsmb3j5GMprS8ZIGAA9LWpxucwIwLiLWS/oUySAP74mIJkkHpu9X/y5wYURsTYP230jeEAVQHREnpKeuvxIRZ0n6Z5Jxjz/The+TWdlwMJsVR+t70xuAfdMxrF+R9EYXrwV/WNIMkv+rhwJjgSfybRwRj0k6WNJhQA3wUkRsTHvZ7yd5lSQkr5IcSdIrbmuJpOb0c/6R5DWcx0i6JF0/ON13F/D7dPxdSAZ6uCkimtJatksaR/ILzq+SKwP0Ixk2sVXrYCjLgdoCvh9mZc/BbJbfauCSPOua2P1S0MA263emf7bkTLfOVxewP+lgANcAx0fES5JuaW+7dixI6z6EpAcNSS95TkT8oID9d+uJp9faPxsRi9rUN4lk+Me/LOLtQ/0JWB0RJ+f5rNbvTTP+eWQG+BqzWUceAAZI+rvWBZKOlzQReA4Ym46mM5hktKGuKGT/QSTB1yhpKHBOzrpXgP3yHHs+yQhYl5CENMAi4KNKxtNG0uGSDs6zf1uLgE+lp6WRNErSPu1stxj4pKTqdLsDgXVAjaST02X9JR3Vyed11DazsudgNssjkhFePgS8L71xazUwG3ghIjaRjLLzBPBT3jpFXOixO90/Ih5Pl68GfgQ8nLN6HvDfrTd/tdlvNUmwbY6IP6bLFgM/Ax6R1EAS2IWG380kQzyukLQK+AHt925vJhlx6AlJjwOXR8Qukl8Qvp4uW0kyjnFHlpD80uKbv6wieXQpMzOzDHGP2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDPn/G464i9qO6z4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t31r0K1nl6VR"
   },
   "source": [
    "##### Clusters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "id": "utIyfwS7l73A"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, 'clusters')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9GpjIu2el75Q",
    "outputId": "448044ce-685b-427d-c620-fbe642d50a18"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=9.301, p=0.054\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "RUUxu7Fhl77g",
    "outputId": "93cf6171-1ada-433b-cf54-11ddaadecefc"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.97555</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.97555</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.97555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.92226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.92226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.97555</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>0.92226</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.97555 0.92226    0.92226  1.00000\n",
       "AHP       0.97555 1.00000 0.92226    0.92226  0.97555\n",
       "DAWA      0.92226 0.92226 1.00000    0.92226  0.92226\n",
       "Geometric 0.92226 0.92226 0.92226    1.00000  0.92226\n",
       "Laplace   1.00000 0.97555 0.92226    0.92226  1.00000"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "PGFWKkDCl79Q",
    "outputId": "b13690c3-5872-45d2-933b-aeb14f876d06"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdxElEQVR4nO3de5yWdZ3/8dd7GITUwAMjnoIhggVEJRnPbuCh0jwmGoLtwlZL9eugmLXUbhvtPjZoyywrNX7Wam3JKmli7gqtQqZpMSA6IKAmCGIk/DAUD+DMfH5/XNfkzTj3zD3D3HNfc9/v5+MxD67z/fkOMO/5XqevIgIzMzPLhqpSF2BmZmZvcjCbmZlliIPZzMwsQxzMZmZmGeJgNjMzy5DqUhfQ2w0aNChqa2tLXYaZWa+yfPnybRFRU+o6ssjBvJdqa2upr68vdRlmZr2KpGdLXUNW+VS2mZlZhjiYzczMMsTBbGZmliG+xmxmZmVj+fLlh1RXV98EjCWbnc9mYFVjY+PHxo8f/0JbGziYzcysbFRXV9906KGHjq6pqXmxqqoqc4NBNDc3a+vWrWO2bNlyE3BBW9tk8bcJMzOzrhpbU1PzUhZDGaCqqipqamp2kPTo296mB+sxMzMrtqqshnKLtL68+etgNjMzyxBfYzYzs7JVO+ue8d15vA1zz13e0TYLFiwYcPXVVw9pbm7mwx/+8Lavfe1rWzrzGQ7mvdSweQe1s+4pdRlWoTb0n1rqEnqto4cNKXUJ7bptTmOpS2jX6LVrSl1CJjU2NjJz5swhixYtevKd73znG8cee+zoSZMm/Xn8+PGvF3oMn8o2MzPrJkuXLt1v6NChu8aMGbO7f//+cfHFF29fsGDBAZ05hoPZzMysm2zatGmfI444YnfL/JFHHrl78+bN+3TmGA5mMzOzbhLx1hvCJXXqLnEHs5mZWTcZMmTIHj3k5557bp/DDz/8jc4cw8FsZmbWTSZMmPDKhg0b+q9du3af119/XXfcccdBkyZN+nNnjuG7ss3MrGwV8nhTd+rbty/XXHPNxrPPPntkU1MTU6dO3VZXV1fwHdngYDYzM+tWkydP3jF58uQdXd0/c6eyJe3shmPUSlrVHfWYmZn1pMwFs5mZWSXrFcEs6XxJv5P0qKT/lTQ4XT5b0k8k3S/pKUl/38a+tZJ+I2lF+nVKzrovSGqQ9Jikuemy4ZLulbQ83W9Uz7XUzMwqXW+5xvwgcFJEhKSPAV8APpeuOwY4CdgPeFRS6/djvgC8NyJelzQCuBWok3QOcBFwYkS8KumgdPt5wCci4ilJJwLXA2fkHlDSDGAGQJ8BNd3cVDMzq2S9JZiPBP5L0mHAPsD6nHV3RcRrwGuSlgAnACtz1vcFvidpHNAEjEyXnwX8R0S8ChAR2yXtD5wC3C6pZf9+rYuJiHkkAU6/w0ZkengxMzPrXXpLMH8X+FZELJQ0EZids651MLaenwn8CTiW5NR9y23ramPbKuDPETFurys2MzPrgt4SzAOBzen0tFbrLpQ0h+RU9kRgFkmvOnff5yKiWdI0oE+6fDHwz5J+1nIqO+01r5d0aUTcrqTbfExEPFashpmZWRHNHtitwz4ye0eHz0Vfeumltffdd9/Agw8+uPGpp55a3dmPyOLNX/tKei7n6yqSHvLtkn4DbGu1/e+Be4BHgH+NiOdbrb8emCbpEZLT2K8ARMS9wEKgXtJK4Op0+8uBj0p6DFgNXNjdDTQzs/L1kY98ZNvChQuf6ur+mesxR0S+XxbuyrP8yYiY0eoYG4Cx6fRTJDeItfhiznZzgbmt9l0PnN25qs3MzBLnnHPOznXr1nVqRKlcWewxm5mZVazM9Zg7IyJml7oGMzOz7uQes5mZWYY4mM3MzDKkV5/KNjMza1cBjzd1t/PPP3/YI4888vYXX3yxevDgwcfMmjXr+ZkzZ7Z+oigvB/NeOvqIgdTPPbfUZVjF6vLIchWvodQFdKT1Gxus17j77rvXd7xVfj6VbWZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLED8uZWZmZevoW47u1mEfG6Y1tPtc9NNPP9338ssvH7Z169a+VVVVTJs2beuXv/zlFzrzGe4xm5mZdZO+fftyzTXXPPfMM8+sXrZs2Zof/vCHhyxfvrx/Z47hYDYzM+smQ4cOfeO00057FeDAAw9sHj58+GsbN27s1BCQDmYzM7MiWLdu3T5PPPHEvhMmTNjZmf0czGZmZt1sx44dVRdffPHwuXPnbjrooIOaO7Ovg9nMzKwb7dq1S+eee+7wSy+9dPu0adP+3Nn9HcxmZmbdpLm5mcsuu2zoyJEjX589e/afunIMPy5lZmZlq6PHm7rbr371q/1/8YtfHDxixIjXRo0aNQbgq1/96ubJkycXPBScg9nMzKybvP/9798ZEXv1y4BPZZuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsSPS+2lhs07qJ11T6nLsCLZ0H9qqUvotY4eNqTUJfRqt81pLHUJ7Rq9dk2pSyjImlGju3XYx9Fr17T7KNSrr76qE088cdTu3bvV1NSk888//8Vrr732+c58hoPZzMysm/Tv3z8efPDBdQMHDmzetWuXjj/++L+67777dpx55pmvFHoMn8o2MzPrJlVVVQwcOLAZYPfu3WpsbJSkzh2jKJWZmZlVqMbGRkaNGjVm8ODBx06YMOGlM844o+DeMjiYzczMulV1dTVr1659YuPGjY+vWLFiv2XLlvXvzP4OZjMzsyIYNGhQ02mnnfby3XffPbAz+zmYzczMusnzzz9fvW3btj4AO3fu1NKlSweMHj369c4co6h3ZUsaDFwLnAS8COwG/j0i7izm57ZRx0Rgd0T8Ns/6C4AxETG3J+syM7Pi6ujxpu62adOmvtOnTx/W1NREROjCCy/cPmXKlIKHfIQiBnN6G9ovgFsiYmq6bChwQbE+sx0TgZ3AW4JZUnVELAQW9nRRZmZWXk488cTX1qxZ88TeHKOYp7LPIOml3tiyICKejYjvSuoj6RuSlkl6XNLHIQnzdPkqSQ2SJqfLJ0r6taTbJD0paa6kyyX9Pt1ueLpdjaSfp8ddJulUSbXAJ4CZklZK+mtJN0v6lqQlwNclTZf0vfQYgyXdKemx9OuUIn6PzMzM9lDMU9lHASvyrPsosCMijpfUD3hI0mLgOGAccCwwCFgm6YF0n2OB0cB24Bngpog4QdIVwGeAK4HvANdGxIOShgCLImK0pBuBnRHxTQBJHwVGAmdFRJOk6Tm1XQf8OiI+KKkPsH93fDPMzMwK0WNv/pL0feA0kuvMzwLHSLokXT0QGJGuvzUimoA/Sfo1cDzwErAsIv6YHusPwOJ03wbg9HT6LGBMzsPcAyS9PU9Jt6ef09oZwN8CpOvfcm1A0gxgBkCfATUdN97MzHpKc3Nzs6qqqqLUheTT3NwsoDnf+mIG82pgUstMRHxK0iCgHtgIfCYiFuXuIOkD7RxvV850c858M2+2owo4OSJea3Xcto7XqQe+c0XEPGAeQL/DRmT2L9/MrAKt2rp165iampodWQzn5uZmbd26dSCwKt82xQzm+4GvSfpkRNyQLts3/XMR8ElJ90fEG5JGApuBB4CPS7oFOAh4D/B5YFSBn7kY+DTwDQBJ4yJiJfAyMKDAY9wHfBL4dnoqe7+IeKnAfc3MrIQaGxs/tmXLlpu2bNkylmw+EtwMrGpsbPxYvg2KFswREZIuAq6V9AVgK0kv9R+A24FaYEV69/ZW4CLgTuBk4DEggC9ExBZJhQbzZ4HvS3qcpG0PkNz4dTewQNKFJNej23MFMC+9Dt1EEtIPF/j5ZmZWQuPHj3+B0jz9020Ukbmefq/S77ARcdi0b5e6DCsSD/vYdR72ce+U+7CPkpZHRF03lVNWstjNNzMzq1gOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4ruy91JdXV3U19eXugwzs17Fd2Xn5x6zmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMqS6kI0kHQi8I3f7iFhRrKLMzMwqVYfBLOlfgenAH4CWF2sHcEbxyjIzM6tMhfSYPwQMj4jdxS7GzMys0hVyjXkVcECR6zAzMzMK6zHPAR6VtArY1bIwIi4oWlVmZmYVqpBgvgX4OtAANBe3HDMzs8pWSDBvi4jril6JmZmZFRTMyyXNARay56lsPy5lZmbWzQoJ5nenf56Us8yPS5mZmRVBh8EcEaf3RCFmZmZW2AtG+gGTgFr2fPPXvxSvLDMzs8pUyKnsu4AdwHJyrjFbomHzDmpn3VPqMqxCbeg/tdQl9FpHDxtS6hLadducxlKX0K7Ra9eUuoSyVUgwHxkRZxe9EjMzMyvozV+/lXR00SsxMzOz/D1mSQ0kd19XA38n6RmSU9kCIiKO6ZkSzczMKkd7p7LP67EqzMzMDGgnmCPiWQBJP4mIv8ldJ+knwN+0uaOZmZl1WSHXmI/KnZHUBxhfnHLMzMwqW95glvRFSS8Dx0h6Kf16GXiB5BEqMzMz62Z5gzki5kTE24FvRMSA9OvtEXFwRHyxB2vMS9IHJYWkUTnLTpC0VNJTklZIuiffXeWSLpA0K8+6ncWq28zMLJ/27soeFRFrgdslHdd6fUYGsZgCPAhcBsyWNBi4DZgaEb8FkHQaMJxk2Mq/kFQdEQtJBucwMzPLhPbuyr4KmAFc08a6kg9iIWl/4FTgdJJwnQ18GrilJZQBIuLBnH1uBraTDMyxIn0krC4iPi1pGPAzku/JvT3UDDMzsz20d1f2DElVwD9FxEM9WFOhLgLujYgnJW1Pe/VHAbd0sN9I4KyIaJI0PWf5d4AbIuLHkj7V3gEkzSD5pYU+A2q6Wr+ZmdlbtHtXdkQ0A9/soVo6awowP52en87vQdLvJK2R9J2cxbdHRFMbxzsVuDWd/kl7HxwR8yKiLiLq+uw7sAulm5mZta2Qd2UvljQJuCMiotgFFULSwSSn0sdKCqAPyen1W4DjSO8aj4gTJV3Cni9LeaWdQ2eifWZmVrkKeY75KuB2YHfLI1OSXipyXR25BPhxRAyNiNqIeAewHlgMTJd0Ss62+xZ4zIdIbiIDuLz7SjUzMytchz3m9JGprJkCzG217OfAVGAy8HVJR5A8c70NKGTs6CuAn0m6Ij2WmZlZjyvkVDaSLgDek84ujYhfFq+kjkXExDaWXZczOyHPftNbzd8M3JxOrwdOzlndOvjNzMyKrsNT2ZLmkvQmn0i/rkiXmZmZWTcrpMf8AWBceoc2km4BHgXafGOWmZmZdV0hN38BHJAz7eeDzMzMiqSQHvMc4FFJSwCRXGvOxLuyzczMyk0hd2XfKmkpcDxJMP9DRGwpdmFmZmaVSB29M6StASyAHcCzEdFYlKp6kbq6uqivry91GWZmvYqk5RFRV+o6sqiQU9nXk7xN63GSHvPYdPpgSZ+IiMVFrM/MzKyiFHLz1wbg3em7oceTjMy0CjgL+Pci1mZmZlZxCgnmURGxumUmIp4gCepnileWmZlZZSrkVPY6STfw5khOk4EnJfUD3ihaZWZmZhWokB7zdOBp4EpgJvBMuuwN4PQi1WVmZlaRCnlc6jXgmvSrtZ3dXpGZmVkFyxvMkhrIPz5xRMSxxSnJzMyscrXXYz6vjWUCjgS+VJxyzMzMKlveYI6IZ1umJY0jGev4Q8B6PF6xmZlZUbR3KnskcBkwBfh/wH+RvCnMN3yZmZkVSXunstcCvwHOj4inASTN7JGqzMzMKlR7j0tNArYASyT9X0lnklxjNjMzsyLJG8wRcWdETAZGAUtJnmEeLOkGSe/rofrMzMwqSocvGImIVyLipxFxHskd2SuBWcUuzMzMrBIV8uavv4iI7RHxg4g4o1gFmZmZVbJOBbOZmZkVl4PZzMwsQxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDKkw/GYrX0Nm3dQO+ueUpdhljkb+k8tdQntWjP/8FKX0KuNXrum1CWULfeYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZUtbBLOmDkkLSqHS+VtKqVtvMlnR1On2zpPWSVkpaIenkUtRtZmaVq6yDGZgCPAhc1ol9Ph8R40jGnP5BMYoyMzPLp2yDWdL+wKnAR+lcMLd4AHhXtxZlZmbWgXJ+89dFwL0R8aSk7ZKOA7YDwyWtzNnuUOCbbex/PtBQ9CrNzMxylG2PmeQ09vx0en46D/CHiBjX8gXc2Gq/b6TBPYOkt/0WkmZIqpdU3/Tqju6v3MzMKlZZ9pglHQycAYyVFEAfIIDrC9j98xGxoL0NImIeMA+g32EjYi/LNTMz+4ty7TFfAvw4IoZGRG1EvANYDxxZ4rrMzMzaVa7BPAW4s9WynwNfKkEtZmZmBSvLU9kRMbGNZdcB17WxfHbO9PRi1mVmZtaRcu0xm5mZ9UoOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZogi/H2Nv1NXVRX19fanLMDPrVSQtj4i6UteRRe4xm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGVJe6gN6uYfMOamfdU+oyeq0N/aeWuoRe7ehhQ0pdQq9125zGUpfQq41eu6bUJZQt95jNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8uQXhfMkpokrZS0WtJjkq6SVNVqm7skPZwzf62kK3PmF0m6KWf+GklXpdPVkrZJmtMDzTEzM9tDrwtm4LWIGBcRRwHvBT4AfKVlpaQDgOOAAyQNSxf/FjglXV8FDAKOyjnmKcBD6fT7gHXAhySpiO0wMzN7i94YzH8RES8AM4BP54ToJOBuYD5wWbrsIdJgJgnkVcDLkg6U1A8YDTyarp8CfAfYCJxU9EaYmZnl6NXBDBARz5C045B00RTg1vRrSrrN80CjpCEkAf0w8DvgZKAOeDwidkt6G3Am8Mvc/VuTNENSvaT6pld3FK1tZmZWeXp9MKcEIGkw8C7gwYh4kiSMx6bbtPSaW4L54Zz536bbnAcsiYhXgZ8DH5TUp/WHRcS8iKiLiLo++w4sYrPMzKzS9PpglvROoAl4AZgMHAisl7QBqOXN09kt15mPJjmV/QhJjzn3+vIU4Kx03+XAwcDpPdAMMzMzoJcHs6Qa4EbgexERJMF6dkTURkQtMJ49rzOfB2yPiKaI2A4cQBLOD0saAJwGDMnZ/1PkOZ1tZmZWDL0xmN/W8rgU8L/AYuCrkmqBISQ9YQAiYj3wkqQTgQaSu7EfyTlWA7AjIrYBFwP3R8SunPV3ARekN4iZmZkVXa8bjzki3nLNN7UBOKKN7Y/LmR3Qat30nOmbgZtbrd8O1HSpUDMzsy7ojT1mMzOzsuVgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhSh7/ta6qq6uL+vr6UpdhZtarSFoeEXWlriOL3GM2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhviVnHtJ0svAulLXUUKDgG2lLqKE3P7KbX8ltx32vv1DI6Kmu4opJ9WlLqAMrKvk971Kqnf73f5S11EKldx2cPuLyaeyzczMMsTBbGZmliEO5r03r9QFlJjbX9kquf2V3HZw+4vGN3+ZmZlliHvMZmZmGeJgNjMzyxAH816QdLakdZKeljSr1PUUk6R3SFoiaY2k1ZKuSJcfJOlXkp5K/zyw1LUWk6Q+kh6V9Mt0vmLaL+kASQskrU3/HZxcYe2fmf7bXyXpVkn9y7n9kn4k6QVJq3KW5W2vpC+mPwvXSXp/aaouDw7mLpLUB/g+cA4wBpgiaUxpqyqqRuBzETEaOAn4VNreWcB9ETECuC+dL2dXAGty5iup/d8B7o2IUcCxJN+Himi/pCOAzwJ1ETEW6ANcRnm3/2bg7FbL2mxv+rPgMuCodJ/r05+R1gUO5q47AXg6Ip6JiN3AfODCEtdUNBHxx4hYkU6/TPJD+QiSNt+SbnYLcFFJCuwBko4EzgVuyllcEe2XNAB4D/BDgIjYHRF/pkLan6oG3iapGtgXeJ4ybn9EPABsb7U4X3svBOZHxK6IWA88TfIz0rrAwdx1RwCbcuafS5eVPUm1wLuB3wGDI+KPkIQ3cEgJSyu2bwNfAJpzllVK+98JbAX+Iz2Vf5Ok/aiQ9kfEZuCbwEbgj8COiFhMhbQ/R772VuzPw2JwMHed2lhW9s+eSdof+DlwZUS8VOp6eoqk84AXImJ5qWspkWrgOOCGiHg38Arlddq2Xem11AuBYcDhwH6SPlzaqjKlIn8eFouDueueA96RM38kyamtsiWpL0ko/zQi7kgX/0nSYen6w4AXSlVfkZ0KXCBpA8llizMk/SeV0/7ngOci4nfp/AKSoK6U9p8FrI+IrRHxBnAHcAqV0/4W+dpbcT8Pi8nB3HXLgBGShknah+TGh4UlrqloJInk+uKaiPhWzqqFwLR0ehpwV0/X1hMi4osRcWRE1JL8Xd8fER+mctq/Bdgk6a/SRWcCT1Ah7Sc5hX2SpH3T/wtnktxnUSntb5GvvQuByyT1kzQMGAH8vgT1lQW/+WsvSPoAyXXHPsCPIuLfSltR8Ug6DfgN0MCb11i/RHKd+TZgCMkPr0sjovUNI2VF0kTg6og4T9LBVEj7JY0jufFtH+AZ4O9IfrmvlPZ/FZhM8oTCo8DHgP0p0/ZLuhWYSDK845+ArwC/IE97Jf0j8BGS78+VEfE/PV91eXAwm5mZZYhPZZuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzdoh6VBJ8yX9QdITkv5b0sgif+ZSSXUdbHOlpH1z5v9b0gHd8NkbJDVIekzSYkmH7u0xu1DDuPRRRLOK5GA2yyN9kcSdwNKIGB4RY0ie3R5c2soAuJJkIAUAIuID6aAS3eH0iDgWqCdpb4e6eSShcYCD2SqWg9ksv9OBNyLixpYFEbEyIn4jaWLLmMwAkr4naXo6vUHS1yQ9LKle0nGSFqW97k+k2+TdP5ekG9JjrE5fcIGkz5K8r3mJpCU5nzlI0tcl/Z+c/WdL+lw6/XlJyyQ93nKsDjwAvEvJGNTfyNn34zltWCLpZ0BDut030x7345I+k243XtKvJS1Pvw8tr3Rcmtb7e0lPSvrr9C16/wJMlrRS0uQC6jQrK9WlLsAsw8YCXR20YlNEnCzpWpJxbU8F+gOrgRvb27GVf4yI7WmP9D5Jx0TEdZKuIunZbmu1/XySt9Fdn85/CDhb0vtIXpN4AsmAAwslvScd2i+f80je9PZRktGUjpfUD3hI0uJ0mxOAsRGxXtInSQZ5eHdENEo6KH2/+neBCyNiaxq0/0byhiiA6og4IT11/ZWIOEvSP5OMe/zpTnyfzMqGg9msOFrem94A7J+OYf2ypNc7eS34Q5JmkPxfPQwYAzyeb+OIeFTSIZIOB2qAFyNiY9rLfh/JqyQheZXkCJJecWtLJDWln/NPJK/hPEbSJen6gem+u4Hfp+PvQjLQw40R0ZjWsl3SWJJfcH6VXBmgD8mwiS1aBkNZDtQW8P0wK3sOZrP8VgOX5FnXyJ6Xgvq3Wr8r/bM5Z7plvrqA/UkHA7gaOD4iXpR0c1vbtWFBWvehJD1oSHrJcyLiBwXsv0dPPL3W/pmIWNSqvokkwz/+ZRFvHepPwOqIODnPZ7V8b5rwzyMzwNeYzdpzP9BP0t+3LJB0vKQJwLPAmHQ0nYEkow11RiH7DyAJvh2SBgPn5Kx7GXh7nmPPJxkB6xKSkAZYBHxEyXjaSDpC0iF59m9tEfDJ9LQ0kkZK2q+N7RYDn5BUnW53ELAOqJF0crqsr6SjOvi89tpmVvYczGZ5RDLCyweB96Y3bq0GZgPPR8QmklF2Hgd+ypuniAs9dof7R8Rj6fLVwI+Ah3JWzwP+p+Xmr1b7rSYJts0R8cd02WLgZ8DDkhpIArvQ8LuJZIjHFZJWAT+g7d7tTSQjDj0u6TFgakTsJvkF4evpspUk4xi3ZwnJLy2++csqkkeXMjMzyxD3mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGeJgNjMzy5D/Dz7Fu4v4a8E0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V_3admOanIR_"
   },
   "source": [
    "#### Different Bin Sizes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7uw6rgz8nK1P"
   },
   "source": [
    "##### 32 Bins\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "id": "Lc33TiNlnOig"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, '32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "wBTRL1sZnOkm",
    "outputId": "70043516-f71e-4ea7-cf83-437cb96d34b8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=19.889, p=0.001\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "mksRs5rQnOmv",
    "outputId": "26f7cb73-df9b-47f8-ae55-dd8f2764d268"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.67554</td>\n",
       "      <td>0.59541</td>\n",
       "      <td>0.59541</td>\n",
       "      <td>0.86797</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.67554</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.75756</td>\n",
       "      <td>0.67554</td>\n",
       "      <td>0.72155</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.59541</td>\n",
       "      <td>0.75756</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.78939</td>\n",
       "      <td>0.59541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.59541</td>\n",
       "      <td>0.67554</td>\n",
       "      <td>0.78939</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.59541</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.86797</td>\n",
       "      <td>0.72155</td>\n",
       "      <td>0.59541</td>\n",
       "      <td>0.59541</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.67554 0.59541    0.59541  0.86797\n",
       "AHP       0.67554 1.00000 0.75756    0.67554  0.72155\n",
       "DAWA      0.59541 0.75756 1.00000    0.78939  0.59541\n",
       "Geometric 0.59541 0.67554 0.78939    1.00000  0.59541\n",
       "Laplace   0.86797 0.72155 0.59541    0.59541  1.00000"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "urKSipNVnO4o",
    "outputId": "9ae743e0-2dd8-45d0-b13c-0076cb3bafea"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdz0lEQVR4nO3de5zWdZ338dd7GITUwAMjmgZDBAuISjKe3cBDpXlMNATbG7ZaqruDYtZSe4h2Hxu0ZZaVGre1WnfpKmli7gqtQqZpMSA6IKAmCGIk3NgoHsCZ+dx//H6TF+NcM9cM1zXXb+Z6Px+PefA7X5/vAPOe7+/0VURgZmZm2VBV7gLMzMzsTQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhlSXe4CershQ4ZEbW1tucswM+tVVqxYsT0iaspdRxY5mPdSbW0t9fX15S7DzKxXkfRsuWvIKp/KNjMzyxAHs5mZWYY4mM3MzDLE15jNzKzPWLFixSHV1dU3AuPJZuezBVjd1NT08YkTJ77Q3gYOZjMz6zOqq6tvPPTQQ8fW1NS8WFVVlbnBIFpaWrRt27ZxW7duvRE4v71tsvjbhJmZWXeNr6mpeSmLoQxQVVUVNTU1jSQ9+va36cF6zMzMSq0qq6HcKq0vb/46mM3MzDLE15jNzKzPqp1zz8RiHm/j/HNWdLbNwoULB1111VXDWlpa+MhHPrL9a1/72taufIaDeS81bGmkds495S7DSmTjwOnlLsFK5KgRw8pdQodum9dU7hI6NHbd2nKXkElNTU3Mnj172OLFi59817ve9cYxxxwzdsqUKX+eOHHi64Uew6eyzczMimTZsmX7DR8+fNe4ceN2Dxw4MC666KIdCxcuPKArx3Awm5mZFcnmzZv3Ofzww3e3zh9xxBG7t2zZsk9XjuFgNjMzK5KIt94QLqlLd4k7mM3MzIpk2LBhe/SQn3vuuX3e8Y53vNGVYziYzczMimTSpEmvbNy4ceC6dev2ef3113XHHXccNGXKlD935Ri+K9vMzPqsQh5vKqb+/ftz9dVXbzrrrLNGNzc3M3369O11dXUF35ENDmYzM7Oimjp1auPUqVMbu7t/5k5lS9pZhGPUSlpdjHrMzMx6UuaC2czMrJL1imCWdJ6k30l6VNL/SBqaLp8r6SeS7pf0lKS/a2ffWkm/kbQy/To5Z90XJTVIekzS/HTZSEn3SlqR7jem51pqZmaVrrdcY34QODEiQtLHgS8Cn0/XHQ2cCOwHPCqp7fsxXwDeFxGvSxoF3ALUSTobuBA4ISJelXRQuv0C4JMR8ZSkE4DrgNNzDyhpFjALoN+gmiI31czMKllvCeYjgP+UdBiwD7AhZ91dEfEa8JqkpcDxwKqc9f2B70maADQDo9PlZwL/ERGvAkTEDkn7AycDt0tq3X9A22IiYgFJgDPgsFGZHl7MzMx6l94SzN8FvhURiyRNBubmrGsbjG3nZwN/Ao4hOXXfetu62tm2CvhzREzY64rNzMy6obcE82BgSzo9o826CyTNIzmVPRmYQ9Krzt33uYhokTQD6JcuXwL8s6SftZ7KTnvNGyRdEhG3K+k2Hx0Rj5WqYWZmVkJzBxd12EfmNnb6XPQll1xSe9999w0++OCDm5566qk1Xf2ILN78ta+k53K+riTpId8u6TfA9jbb/x64B3gE+NeIeL7N+uuAGZIeITmN/QpARNwLLALqJa0Crkq3vwz4mKTHgDXABcVuoJmZ9V0f/ehHty9atOip7u6fuR5zROT7ZeGuPMufjIhZbY6xERifTj9FcoNYqy/lbDcfmN9m3w3AWV2r2szMLHH22WfvXL9+fZdGlMqVxR6zmZlZxcpcj7krImJuuWswMzMrJveYzczMMsTBbGZmliG9+lS2mZlZhwp4vKnYzjvvvBGPPPLI21988cXqoUOHHj1nzpznZ8+e3faJorwczHvpqMMHUz//nHKXYSXT7ZHbLOMayl1AZ9q+scF6jbvvvntD51vl51PZZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MM8eNSZmbWZx1181FFHfaxYUZDh89FP/300/0vu+yyEdu2betfVVXFjBkztv3TP/3TC135DPeYzczMiqR///5cffXVzz3zzDNrli9fvvaHP/zhIStWrBjYlWM4mM3MzIpk+PDhb5x66qmvAhx44IEtI0eOfG3Tpk1dGgLSwWxmZlYC69ev3+eJJ57Yd9KkSTu7sp+D2czMrMgaGxurLrroopHz58/ffNBBB7V0ZV8Hs5mZWRHt2rVL55xzzshLLrlkx4wZM/7c1f0dzGZmZkXS0tLCpZdeOnz06NGvz50790/dOYYflzIzsz6rs8ebiu1Xv/rV/r/4xS8OHjVq1GtjxowZB/DVr351y9SpUwseqs7BbGZmViQf+MAHdkbEXv0y4FPZZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MM8eNSe6lhSyO1c+4pdxm91saB08tdQq921Ihh5S6h17ptXlO5S+jVxq5bW+4SCrJ2zNiiDvs4dt3aDh+FevXVV3XCCSeM2b17t5qbm3Xeeee9eM011zzflc9wMJuZmRXJwIED48EHH1w/ePDgll27dum44477q/vuu6/xjDPOeKXQY/hUtpmZWZFUVVUxePDgFoDdu3erqalJkrp2jJJUZmZmVqGampoYM2bMuKFDhx4zadKkl04//fSCe8vgYDYzMyuq6upq1q1b98SmTZseX7ly5X7Lly8f2JX9HcxmZmYlMGTIkOZTTz315bvvvntwV/ZzMJuZmRXJ888/X719+/Z+ADt37tSyZcsGjR079vWuHKOkd2VLGgpcA5wIvAjsBv49Iu4s5ee2U8dkYHdE/DbP+vOBcRExvyfrMjOz0urs8aZi27x5c/+ZM2eOaG5uJiJ0wQUX7Jg2bVrBQz5CCYM5vQ3tF8DNETE9XTYcOL9Un9mBycBO4C3BLKk6IhYBi3q6KDMz61tOOOGE19auXfvE3hyjlKeyTyfppd7QuiAino2I70rqJ+kbkpZLelzSJyAJ83T5akkNkqamyydL+rWk2yQ9KWm+pMsk/T7dbmS6XY2kn6fHXS7pFEm1wCeB2ZJWSfprSTdJ+pakpcDXJc2U9L30GEMl3SnpsfTr5BJ+j8zMzPZQylPZRwIr86z7GNAYEcdJGgA8JGkJcCwwATgGGAIsl/RAus8xwFhgB/AMcGNEHC/pcuCzwBXAd4BrIuJBScOAxRExVtINwM6I+CaApI8Bo4EzI6JZ0syc2q4Ffh0RH5LUD9i/GN8MMzOzQvTYm78kfR84leQ687PA0ZIuTlcPBkal62+JiGbgT5J+DRwHvAQsj4g/psf6A7Ak3bcBOC2dPhMYl/Mw9yBJb89T0u3p57R1OvC/ANL1b7k2IGkWMAug36CazhtvZmY9paWlpUVVVVVR7kLyaWlpEdCSb30pg3kNMKV1JiI+LWkIUA9sAj4bEYtzd5D0wQ6OtytnuiVnvoU321EFnBQRr7U5bnvH69ID37kiYgGwAGDAYaMy+5dvZlaBVm/btm1cTU1NYxbDuaWlRdu2bRsMrM63TSmD+X7ga5I+FRHXp8v2Tf9cDHxK0v0R8Yak0cAW4AHgE5JuBg4C3gt8ARhT4GcuAT4DfANA0oSIWAW8DAwq8Bj3AZ8Cvp2eyt4vIl4qcF8zMyujpqamj2/duvXGrVu3jiebjwS3AKubmpo+nm+DkgVzRISkC4FrJH0R2EbSS/174HagFliZ3r29DbgQuBM4CXgMCOCLEbFVUqHB/Dng+5IeJ2nbAyQ3ft0NLJR0Acn16I5cDixIr0M3k4T0wwV+vpmZldHEiRNfoDxP/xSNIjLX0+9VBhw2Kg6b8e1yl9FredjHveNhH7vPwz7unb0d9lHSioioK1I5fUoWu/lmZmYVy8FsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEN8V/Zeqquri/r6+nKXYWbWq/iu7PzcYzYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGVBeykaQDgXfmbh8RK0tVlJmZWaXqNJgl/SswE/gD0Ppi7QBOL11ZZmZmlamQHvOHgZERsbvUxZiZmVW6Qq4xrwYOKHEdZmZmRmE95nnAo5JWA7taF0bE+SWryszMrEIVEsw3A18HGoCW0pZjZmZW2QoJ5u0RcW3JKzEzM7OCgnmFpHnAIvY8le3HpczMzIqskGB+T/rniTnL/LiUmZlZCXQazBFxWk8UYmZmZoW9YGQAMAWoZc83f/1L6coyMzOrTIWcyr4LaARWkHON2RINWxqpnXNPucuwXm7jwOnlLiEzjhoxrNwl9Fq3zWvqsc8au25tj31WpSkkmI+IiLNKXomZmZkV9Oav30o6quSVmJmZWf4es6QGkruvq4G/lfQMyalsARERR/dMiWZmZpWjo1PZ5/ZYFWZmZgZ0EMwR8SyApJ9ExN/krpP0E+Bv2t3RzMzMuq2Qa8xH5s5I6gdMLE05ZmZmlS1vMEv6kqSXgaMlvZR+vQy8QPIIlZmZmRVZ3mCOiHkR8XbgGxExKP16e0QcHBFf6sEa85L0IUkhaUzOsuMlLZP0lKSVku7Jd1e5pPMlzcmzbmep6jYzM8uno7uyx0TEOuB2Sce2XZ+RQSymAQ8ClwJzJQ0FbgOmR8RvASSdCowkGbbyLyRVR8QiksE5zMzMMqGju7KvBGYBV7ezruyDWEjaHzgFOI0kXOcCnwFubg1lgIh4MGefm4AdJANzrEwfCauLiM9IGgH8jOR7cm8PNcPMzGwPHd2VPUtSFfCPEfFQD9ZUqAuBeyPiSUk70l79kcDNnew3GjgzIpolzcxZ/h3g+oj4saRPd3QASbNIfmmh36Ca7tZvZmb2Fh3elR0RLcA3e6iWrpoG3JpO35rO70HS7yStlfSdnMW3R0RzO8c7Bbglnf5JRx8cEQsioi4i6vrtO7gbpZuZmbWvkHdlL5E0BbgjIqLUBRVC0sEkp9LHSwqgH8np9ZuBY0nvGo+IEyRdzJ4vS3mlg0Nnon1mZla5CnmO+UrgdmB36yNTkl4qcV2duRj4cUQMj4jaiHgnsAFYAsyUdHLOtvsWeMyHSG4iA7iseKWamZkVrtMec/rIVNZMA+a3WfZzYDowFfi6pMNJnrneDhQydvTlwM8kXZ4ey8zMrMcVciobSecD701nl0XEL0tXUuciYnI7y67NmZ2UZ7+ZbeZvAm5KpzcAJ+Wsbhv8ZmZmJdfpqWxJ80l6k0+kX5eny8zMzKzICukxfxCYkN6hjaSbgUeBdt+YZWZmZt1XyM1fAAfkTPv5IDMzsxIppMc8D3hU0lJAJNeaM/GubDMzs76mkLuyb5G0DDiOJJj/PiK2lrowMzOzSqTO3hnS3gAWQCPwbEQ0laSqXqSuri7q6+vLXYaZWa8iaUVE1JW7jiwq5FT2dSRv03qcpMc8Pp0+WNInI2JJCeszMzOrKIXc/LUReE/6buiJJCMzrQbOBP69hLWZmZlVnEKCeUxErGmdiYgnSIL6mdKVZWZmVpkKOZW9XtL1vDmS01TgSUkDgDdKVpmZmVkFKqTHPBN4GrgCmA08ky57AzitRHWZmZlVpEIel3oNuDr9amtn0SsyMzOrYHmDWVID+ccnjog4pjQlmZmZVa6OeszntrNMwBHAl0tTjpmZWWXLG8wR8WzrtKQJJGMdfxjYgMcrNjMzK4mOTmWPBi4FpgH/D/hPkjeF+YYvMzOzEunoVPY64DfAeRHxNICk2T1SlZmZWYXq6HGpKcBWYKmk/yPpDJJrzGZmZlYieYM5Iu6MiKnAGGAZyTPMQyVdL+n9PVSfmZlZRen0BSMR8UpE/DQiziW5I3sVMKfUhZmZmVWiQt789RcRsSMifhARp5eqIDMzs0rWpWA2MzOz0nIwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGdDoes3WsYUsjtXPuKXcZViIbB04vdwm91lEjhpW7hA7dNq+p3CX0amPXrS13CX2We8xmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswzp08Es6UOSQtKYdL5W0uo228yVdFU6fZOkDZJWSVop6aRy1G1mZpWrTwczMA14ELi0C/t8ISImkIw5/YNSFGVmZpZPnw1mSfsDpwAfo2vB3OoB4N1FLcrMzKwTffnNXxcC90bEk5J2SDoW2AGMlLQqZ7tDgW+2s/95QEPJqzQzM8vRZ3vMJKexb02nb03nAf4QERNav4Ab2uz3jTS4Z5H0tt9C0ixJ9ZLqm19tLH7lZmZWsfpkj1nSwcDpwHhJAfQDAriugN2/EBELO9ogIhYACwAGHDYq9rJcMzOzv+irPeaLgR9HxPCIqI2IdwIbgCPKXJeZmVmH+mowTwPubLPs58CXy1CLmZlZwfrkqeyImNzOsmuBa9tZPjdnemYp6zIzM+tMX+0xm5mZ9UoOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZogi/H2Nv1NXVRX19fbnLMDPrVSStiIi6cteRRe4xm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGVJe7gN6uYUsjtXPuKXcZRbVx4PRyl2AV7KgRw8pdQtHdNq+p3CUU3dh1a8tdQp/lHrOZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhnS64JZUrOkVZLWSHpM0pWSqtpsc5ekh3Pmr5F0Rc78Ykk35sxfLenKdLpa0nZJ83qgOWZmZnvodcEMvBYREyLiSOB9wAeBr7SulHQAcCxwgKQR6eLfAien66uAIcCROcc8GXgonX4/sB74sCSVsB1mZmZv0RuD+S8i4gVgFvCZnBCdAtwN3Apcmi57iDSYSQJ5NfCypAMlDQDGAo+m66cB3wE2ASeWvBFmZmY5enUwA0TEMyTtOCRdNA24Jf2alm7zPNAkaRhJQD8M/A44CagDHo+I3ZLeBpwB/DJ3/7YkzZJUL6m++dXGkrXNzMwqT68P5pQAJA0F3g08GBFPkoTx+HSb1l5zazA/nDP/23Sbc4GlEfEq8HPgQ5L6tf2wiFgQEXURUddv38ElbJaZmVWaXh/Mkt4FNAMvAFOBA4ENkjYCtbx5Orv1OvNRJKeyHyHpMedeX54GnJnuuwI4GDitB5phZmYG9PJgllQD3AB8LyKCJFjPiojaiKgFJrLndeZzgR0R0RwRO4ADSML5YUmDgFOBYTn7f5o8p7PNzMxKoTcG89taH5cC/gdYAnxVUi0wjKQnDEBEbABeknQC0EByN/YjOcdqABojYjtwEXB/ROzKWX8XcH56g5iZmVnJ9brxmCPiLdd8UxuBw9vZ/tic2UFt1s3Mmb4JuKnN+h1ATbcKNTMz64be2GM2MzPrsxzMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDJEyeO/1l11dXVRX19f7jLMzHoVSSsioq7cdWSRe8xmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEL+Scy9JehlYX+46ymgIsL3cRZSR21+57a/ktsPet394RNQUq5i+pLrcBfQB6yv5fa+S6t1+t7/cdZRDJbcd3P5S8qlsMzOzDHEwm5mZZYiDee8tKHcBZeb2V7ZKbn8ltx3c/pLxzV9mZmYZ4h6zmZlZhjiYzczMMsTBvBcknSVpvaSnJc0pdz2lJOmdkpZKWitpjaTL0+UHSfqVpKfSPw8sd62lJKmfpEcl/TKdr5j2SzpA0kJJ69J/BydVWPtnp//2V0u6RdLAvtx+ST+S9IKk1TnL8rZX0pfSn4XrJX2gPFX3DQ7mbpLUD/g+cDYwDpgmaVx5qyqpJuDzETEWOBH4dNreOcB9ETEKuC+d78suB9bmzFdS+78D3BsRY4BjSL4PFdF+SYcDnwPqImI80A+4lL7d/puAs9osa7e96c+CS4Ej032uS39GWjc4mLvveODpiHgmInYDtwIXlLmmkomIP0bEynT6ZZIfyoeTtPnmdLObgQvLUmAPkHQEcA5wY87iimi/pEHAe4EfAkTE7oj4MxXS/lQ18DZJ1cC+wPP04fZHxAPAjjaL87X3AuDWiNgVERuAp0l+Rlo3OJi773Bgc878c+myPk9SLfAe4HfA0Ij4IyThDRxSxtJK7dvAF4GWnGWV0v53AduA/0hP5d8oaT8qpP0RsQX4JrAJ+CPQGBFLqJD258jX3or9eVgKDubuUzvL+vyzZ5L2B34OXBERL5W7np4i6VzghYhYUe5ayqQaOBa4PiLeA7xC3zpt26H0WuoFwAjgHcB+kj5S3qoypSJ/HpaKg7n7ngPemTN/BMmprT5LUn+SUP5pRNyRLv6TpMPS9YcBL5SrvhI7BThf0kaSyxanS/q/VE77nwOei4jfpfMLSYK6Utp/JrAhIrZFxBvAHcDJVE77W+Vrb8X9PCwlB3P3LQdGSRohaR+SGx8WlbmmkpEkkuuLayPiWzmrFgEz0ukZwF09XVtPiIgvRcQREVFL8nd9f0R8hMpp/1Zgs6S/ShedATxBhbSf5BT2iZL2Tf8vnEFyn0WltL9VvvYuAi6VNEDSCGAU8Psy1Ncn+M1fe0HSB0muO/YDfhQR/1beikpH0qnAb4AG3rzG+mWS68y3AcNIfnhdEhFtbxjpUyRNBq6KiHMlHUyFtF/SBJIb3/YBngH+luSX+0pp/1eBqSRPKDwKfBzYnz7afkm3AJNJhnf8E/AV4Bfkaa+kfwA+SvL9uSIi/rvnq+4bHMxmZmYZ4lPZZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmazDkg6VNKtkv4g6QlJ/yVpdIk/c5mkuk62uULSvjnz/yXpgCJ89kZJDZIek7RE0qF7e8xu1DAhfRTRrCI5mM3ySF8kcSewLCJGRsQ4kme3h5a3MgCuIBlIAYCI+GA6qEQxnBYRxwD1JO3tVJFHEpoAOJitYjmYzfI7DXgjIm5oXRARqyLiN5Imt47JDCDpe5JmptMbJX1N0sOS6iUdK2lx2uv+ZLpN3v1zSbo+Pcaa9AUXSPocyfual0pamvOZQyR9XdL/ztl/rqTPp9NfkLRc0uOtx+rEA8C7lYxB/Y2cfT+R04alkn4GNKTbfTPtcT8u6bPpdhMl/VrSivT70PpKx2Vpvb+X9KSkv07fovcvwFRJqyRNLaBOsz6lutwFmGXYeKC7g1ZsjoiTJF1DMq7tKcBAYA1wQ0c7tvEPEbEj7ZHeJ+noiLhW0pUkPdvtbba/leRtdNel8x8GzpL0fpLXJB5PMuDAIknvTYf2y+dckje9fYxkNKXjJA0AHpK0JN3meGB8RGyQ9CmSQR7eExFNkg5K36/+XeCCiNiWBu2/kbwhCqA6Io5PT11/JSLOlPTPJOMef6YL3yezPsPBbFYare9NbwD2T8ewflnS6128FvxhSbNI/q8eBowDHs+3cUQ8KukQSe8AaoAXI2JT2st+P8mrJCF5leQokl5xW0slNaef848kr+E8WtLF6frB6b67gd+n4+9CMtDDDRHRlNayQ9J4kl9wfpVcGaAfybCJrVoHQ1kB1Bbw/TDr8xzMZvmtAS7Os66JPS8FDWyzflf6Z0vOdOt8dQH7kw4GcBVwXES8KOmm9rZrx8K07kNJetCQ9JLnRcQPCth/j554eq39sxGxuE19k0mGf/zLIt461J+ANRFxUp7Pav3eNOOfR2aArzGbdeR+YICkv2tdIOk4SZOAZ4Fx6Wg6g0lGG+qKQvYfRBJ8jZKGAmfnrHsZeHueY99KMgLWxSQhDbAY+KiS8bSRdLikQ/Ls39Zi4FPpaWkkjZa0XzvbLQE+Kak63e4gYD1QI+mkdFl/SUd28nkdtc2sz3Mwm+URyQgvHwLel964tQaYCzwfEZtJRtl5HPgpb54iLvTYne4fEY+ly9cAPwIeylm9APjv1pu/2uy3hiTYtkTEH9NlS4CfAQ9LaiAJ7ELD70aSIR5XSloN/ID2e7c3kow49Likx4DpEbGb5BeEr6fLVpGMY9yRpSS/tPjmL6tIHl3KzMwsQ9xjNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZlliIPZzMwsQ/4/2HG8i8AcR+AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4n4Z6pl6nUD3"
   },
   "source": [
    "##### 64 Bins"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "id": "wGfYRY4tnY8v"
   },
   "outputs": [],
   "source": [
    "filteredDf = getFilteredArray(answerDataFrame, '64')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "VthKZzL4nY_X",
    "outputId": "031f0171-235c-42ad-854b-54dbf703648b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Statistics=24.967, p=0.000\n"
     ]
    }
   ],
   "source": [
    "getFriedmanResult(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 206
    },
    "id": "Nk_OazWxnZBn",
    "outputId": "9495b936-8e8f-4f07-f0ce-928449c98840"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AGrid</th>\n",
       "      <th>AHP</th>\n",
       "      <th>DAWA</th>\n",
       "      <th>Geometric</th>\n",
       "      <th>Laplace</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>AGrid</th>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.51746</td>\n",
       "      <td>0.51746</td>\n",
       "      <td>0.69122</td>\n",
       "      <td>0.26833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>AHP</th>\n",
       "      <td>0.51746</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.31106</td>\n",
       "      <td>0.74526</td>\n",
       "      <td>0.43497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>DAWA</th>\n",
       "      <td>0.51746</td>\n",
       "      <td>0.31106</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.34890</td>\n",
       "      <td>0.05628</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Geometric</th>\n",
       "      <td>0.69122</td>\n",
       "      <td>0.74526</td>\n",
       "      <td>0.34890</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.34890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Laplace</th>\n",
       "      <td>0.26833</td>\n",
       "      <td>0.43497</td>\n",
       "      <td>0.05628</td>\n",
       "      <td>0.34890</td>\n",
       "      <td>1.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AGrid     AHP    DAWA  Geometric  Laplace\n",
       "AGrid     1.00000 0.51746 0.51746    0.69122  0.26833\n",
       "AHP       0.51746 1.00000 0.31106    0.74526  0.43497\n",
       "DAWA      0.51746 0.31106 1.00000    0.34890  0.05628\n",
       "Geometric 0.69122 0.74526 0.34890    1.00000  0.34890\n",
       "Laplace   0.26833 0.43497 0.05628    0.34890  1.00000"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "getConoverPostHoc(filteredDf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 279
    },
    "id": "plGSLVFVnZEI",
    "outputId": "6178bd25-85f2-4c0d-cdb8-a41b4b008cdc"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeYAAAEGCAYAAACw1zZNAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAdu0lEQVR4nO3de5zWdZ338dd7GITUwAMjmgZDBAuISjKe3cBDpXlMNATbG7ZaqruDYtZSe4h2Hxu0ZZaVGre1WnfpKmli7gqtQqZpMSA6IKAmCGIk3NgoHsCZ+dx//H6TF+NcM9cM1zXXb+Z6Px+PefA7X5/vAPOe7+/0VURgZmZm2VBV7gLMzMzsTQ5mMzOzDHEwm5mZZYiD2czMLEMczGZmZhlSXe4CershQ4ZEbW1tucswM+tVVqxYsT0iaspdRxY5mPdSbW0t9fX15S7DzKxXkfRsuWvIKp/KNjMzyxAHs5mZWYY4mM3MzDLE15jNzKzPWLFixSHV1dU3AuPJZuezBVjd1NT08YkTJ77Q3gYOZjMz6zOqq6tvPPTQQ8fW1NS8WFVVlbnBIFpaWrRt27ZxW7duvRE4v71tsvjbhJmZWXeNr6mpeSmLoQxQVVUVNTU1jSQ9+va36cF6zMzMSq0qq6HcKq0vb/46mM3MzDLE15jNzKzPqp1zz8RiHm/j/HNWdLbNwoULB1111VXDWlpa+MhHPrL9a1/72taufIaDeS81bGmkds495S7DrKJtHDi93CV06KgRw8pdQtE1zGgodwmZ1NTUxOzZs4ctXrz4yXe9611vHHPMMWOnTJny54kTJ75e6DF8KtvMzKxIli1btt/w4cN3jRs3bvfAgQPjoosu2rFw4cIDunIMB7OZmVmRbN68eZ/DDz98d+v8EUccsXvLli37dOUYDmYzM7MiiXjrDeGSunSXuIPZzMysSIYNG7ZHD/m5557b5x3veMcbXTmGg9nMzKxIJk2a9MrGjRsHrlu3bp/XX39dd9xxx0FTpkz5c1eO4buyzcyszyrk8aZi6t+/P1dfffWms846a3RzczPTp0/fXldXV/Ad2eBgNjMzK6qpU6c2Tp06tbG7+2fuVLaknUU4Rq2k1cWox8zMrCdlLpjNzMwqWa8IZknnSfqdpEcl/Y+koenyuZJ+Iul+SU9J+rt29q2V9BtJK9Ovk3PWfVFSg6THJM1Pl42UdK+kFel+Y3qupWZmVul6yzXmB4ETIyIkfRz4IvD5dN3RwInAfsCjktq+H/MF4H0R8bqkUcAtQJ2ks4ELgRMi4lVJB6XbLwA+GRFPSToBuA44PfeAkmYBswD6DaopclPNzKyS9ZZgPgL4T0mHAfsAG3LW3RURrwGvSVoKHA+sylnfH/iepAlAMzA6XX4m8B8R8SpAROyQtD9wMnC7pNb9B7QtJiIWkAQ4Aw4blenhxczMrHfpLcH8XeBbEbFI0mRgbs66tsHYdn428CfgGJJT9623raudbauAP0fEhL2u2MzMrBt6SzAPBrak0zParLtA0jySU9mTgTkkvercfZ+LiBZJM4B+6fIlwD9L+lnrqey017xB0iURcbuSbvPREfFYqRpmZmYlNHdwUYd9ZG5jp89FX3LJJbX33Xff4IMPPrjpqaeeWtPVj8jizV/7Snou5+tKkh7y7ZJ+A2xvs/3vgXuAR4B/jYjn26y/Dpgh6RGS09ivAETEvcAioF7SKuCqdPvLgI9JegxYA1xQ7AaamVnf9dGPfnT7okWLnuru/pnrMUdEvl8W7sqz/MmImNXmGBuB8en0UyQ3iLX6Us5284H5bfbdAJzVtarNzMwSZ5999s7169d3aUSpXFnsMZuZmVWszPWYuyIi5pa7BjMzs2Jyj9nMzCxDHMxmZmYZ0qtPZZuZmXWogMebiu28884b8cgjj7z9xRdfrB46dOjRc+bMeX727NltnyjKy8G8l446fDD1888pdxlmFa7bI+z1iIZyF2A96u67797Q+Vb5+VS2mZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxD/LiUmZn1WUfdfFRRh31smNHQ4XPRTz/9dP/LLrtsxLZt2/pXVVUxY8aMbf/0T//0Qlc+wz1mMzOzIunfvz9XX331c88888ya5cuXr/3hD394yIoVKwZ25RgOZjMzsyIZPnz4G6eeeuqrAAceeGDLyJEjX9u0aVOXhoB0MJuZmZXA+vXr93niiSf2nTRp0s6u7OdgNjMzK7LGxsaqiy66aOT8+fM3H3TQQS1d2dfBbGZmVkS7du3SOeecM/KSSy7ZMWPGjD93dX8Hs5mZWZG0tLRw6aWXDh89evTrc+fO/VN3juHHpczMrM/q7PGmYvvVr361/y9+8YuDR40a9dqYMWPGAXz1q1/dMnXq1IKHQHMwm5mZFckHPvCBnRGxV78M+FS2mZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxD/LjUXmrY0kjtnHvKXYZVqI0Dp5e7hEw4asSwcpfQq902r6nL+4xdt7YElRTf2jFjizrs49h1azt8FOrVV1/VCSecMGb37t1qbm7Weeed9+I111zzfFc+w8FsZmZWJAMHDowHH3xw/eDBg1t27dql44477q/uu+++xjPOOOOVQo/hU9lmZmZFUlVVxeDBg1sAdu/eraamJknq2jFKUpmZmVmFampqYsyYMeOGDh16zKRJk146/fTTC+4tg4PZzMysqKqrq1m3bt0TmzZtenzlypX7LV++fGBX9ncwm5mZlcCQIUOaTz311JfvvvvuwV3Zz8FsZmZWJM8//3z19u3b+wHs3LlTy5YtGzR27NjXu3KMkt6VLWkocA1wIvAisBv494i4s5Sf204dk4HdEfHbPOvPB8ZFxPyerMvMzEqrs8ebim3z5s39Z86cOaK5uZmI0AUXXLBj2rRpBQ/5CCUM5vQ2tF8AN0fE9HTZcOD8Un1mByYDO4G3BLOk6ohYBCzq6aLMzKxvOeGEE15bu3btE3tzjFKeyj6dpJd6Q+uCiHg2Ir4rqZ+kb0haLulxSZ+AJMzT5aslNUiami6fLOnXkm6T9KSk+ZIuk/T7dLuR6XY1kn6eHne5pFMk1QKfBGZLWiXpryXdJOlbkpYCX5c0U9L30mMMlXSnpMfSr5NL+D0yMzPbQylPZR8JrMyz7mNAY0QcJ2kA8JCkJcCxwATgGGAIsFzSA+k+xwBjgR3AM8CNEXG8pMuBzwJXAN8BromIByUNAxZHxFhJNwA7I+KbAJI+BowGzoyIZkkzc2q7Fvh1RHxIUj9g/2J8M8zMzArRY2/+kvR94FSS68zPAkdLujhdPRgYla6/JSKagT9J+jVwHPASsDwi/pge6w/AknTfBuC0dPpMYFzOw9yDJL09T0m3p5/T1unA/wJI17/l2oCkWcAsgH6DajpvvJmZ9ZSWlpYWVVVVRbkLyaelpUVAS771pQzmNcCU1pmI+LSkIUA9sAn4bEQszt1B0gc7ON6unOmWnPkW3mxHFXBSRLzW5rjtHa9LD3zniogFwAKAAYeNyuxfvplZBVq9bdu2cTU1NY1ZDOeWlhZt27ZtMLA63zalDOb7ga9J+lREXJ8u2zf9czHwKUn3R8QbkkYDW4AHgE9Iuhk4CHgv8AVgTIGfuQT4DPANAEkTImIV8DIwqMBj3Ad8Cvh2eip7v4h4qcB9zcysjJqamj6+devWG7du3TqebD4S3AKsbmpq+ni+DUoWzBERki4ErpH0RWAbSS/174HbgVpgZXr39jbgQuBO4CTgMSCAL0bEVkmFBvPngO9LepykbQ+Q3Ph1N7BQ0gUk16M7cjmwIL0O3UwS0g8X+PlmZlZGEydOfIHyPP1TNIrIXE+/Vxlw2Kg4bMa3y12GVSgP+5jwsI97pxzDPkpaERF1e3WQPiqL3XwzM7OK5WA2MzPLEAezmZlZhjiYzczMMsTBbGZmliG+K3sv1dXVRX19fbnLMDPrVXxXdn7uMZuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDqgvZSNKBwDtzt4+IlaUqyszMrFJ1GsyS/hWYCfwBaH2xdgCnl64sMzOzylRIj/nDwMiI2F3qYszMzCpdIdeYVwMHlLgOMzMzo7Ae8zzgUUmrgV2tCyPi/JJVZWZmVqEKCeabga8DDUBLacsxMzOrbIUE8/aIuLbklZiZmVlBwbxC0jxgEXueyvbjUmZmZkVWSDC/J/3zxJxlflzKzMysBDoN5og4rScKMTMzs8JeMDIAmALUsuebv/6ldGWZmZlVpkJOZd8FNAIryLnGbImGLY3Uzrmn3GVYiWwcOL3cJViJHDViWLlLyIzb5jV1eZ+x69aWoBKDwoL5iIg4q+SVmJmZWUFv/vqtpKNKXomZmZnl7zFLaiC5+7oa+FtJz5CcyhYQEXF0z5RoZmZWOTo6lX1uj1VhZmZmQAfBHBHPAkj6SUT8Te46ST8B/qbdHc3MzKzbCrnGfGTujKR+wMTSlGNmZlbZ8gazpC9Jehk4WtJL6dfLwAskj1CZmZlZkeUN5oiYFxFvB74REYPSr7dHxMER8aUerDEvSR+SFJLG5Cw7XtIySU9JWinpnnx3lUs6X9KcPOt2lqpuMzOzfDq6K3tMRKwDbpd0bNv1GRnEYhrwIHApMFfSUOA2YHpE/BZA0qnASJJhK/9CUnVELCIZnMPMzCwTOror+0pgFnB1O+vKPoiFpP2BU4DTSMJ1LvAZ4ObWUAaIiAdz9rkJ2EEyMMfK9JGwuoj4jKQRwM9Ivif39lAzzMzM9tDRXdmzJFUB/xgRD/VgTYW6ELg3Ip6UtCPt1R8J3NzJfqOBMyOiWdLMnOXfAa6PiB9L+nRHB5A0i+SXFvoNqulu/WZmZm/R4V3ZEdECfLOHaumqacCt6fSt6fweJP1O0lpJ38lZfHtENLdzvFOAW9Lpn3T0wRGxICLqIqKu376Du1G6mZlZ+wp5V/YSSVOAOyIiSl1QISQdTHIqfbykAPqRnF6/GTiW9K7xiDhB0sXs+bKUVzo4dCbaZ2ZmlauQ55ivBG4Hdrc+MiXppRLX1ZmLgR9HxPCIqI2IdwIbgCXATEkn52y7b4HHfIjkJjKAy4pXqpmZWeE67TGnj0xlzTRgfptlPwemA1OBr0s6nOSZ6+1AIWNHXw78TNLl6bHMzMx6XCGnspF0PvDedHZZRPyydCV1LiImt7Ps2pzZSXn2m9lm/ibgpnR6A3BSzuq2wW9mZlZynZ7KljSfpDf5RPp1ebrMzMzMiqyQHvMHgQnpHdpIuhl4FGj3jVlmZmbWfYXc/AVwQM60nw8yMzMrkUJ6zPOARyUtBURyrTkT78o2MzPrawq5K/sWScuA40iC+e8jYmupCzMzM6tE6uydIe0NYAE0As9GRFNJqupF6urqor6+vtxlmJn1KpJWRERduevIokJOZV9H8jatx0l6zOPT6YMlfTIilpSwPjMzs4pSyM1fG4H3pO+GnkgyMtNq4Ezg30tYm5mZWcUpJJjHRMSa1pmIeIIkqJ8pXVlmZmaVqZBT2eslXc+bIzlNBZ6UNAB4o2SVmZmZVaBCeswzgaeBK4DZwDPpsjeA00pUl5mZWUUq5HGp14Cr06+2dha9IjMzswqWN5glNZB/fOKIiGNKU5KZmVnl6qjHfG47ywQcAXy5NOWYmZlVtrzBHBHPtk5LmkAy1vGHgQ14vGIzM7OS6OhU9mjgUmAa8P+A/yR5U5hv+DIzMyuRjk5lrwN+A5wXEU8DSJrdI1WZmZlVqI4el5oCbAWWSvo/ks4gucZsZmZmJZI3mCPizoiYCowBlpE8wzxU0vWS3t9D9ZmZmVWUTl8wEhGvRMRPI+JckjuyVwFzSl2YmZlZJSrkzV9/ERE7IuIHEXF6qQoyMzOrZF0KZjMzMystB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZUin4zFbxxq2NFI7555yl2FWUhsHTi93CUV31IhhPfZZt81r6rHP6ilj160tdwl9lnvMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MM6dPBLOlDkkLSmHS+VtLqNtvMlXRVOn2TpA2SVklaKemkctRtZmaVq08HMzANeBC4tAv7fCEiJpCMOf2DUhRlZmaWT58NZkn7A6cAH6NrwdzqAeDdRS3KzMysE335zV8XAvdGxJOSdkg6FtgBjJS0Kme7Q4FvtrP/eUBDyas0MzPL0Wd7zCSnsW9Np29N5wH+EBETWr+AG9rs9400uGeR9LbfQtIsSfWS6ptfbSx+5WZmVrH6ZI9Z0sHA6cB4SQH0AwK4roDdvxARCzvaICIWAAsABhw2KvayXDMzs7/oqz3mi4EfR8TwiKiNiHcCG4AjylyXmZlZh/pqME8D7myz7OfAl8tQi5mZWcH65KnsiJjczrJrgWvbWT43Z3pmKesyMzPrTF/tMZuZmfVKDmYzM7MMcTCbmZlliIPZzMwsQxzMZmZmGaIIvx9jb9TV1UV9fX25yzAz61UkrYiIunLXkUXuMZuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswxxMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhlSXu4DermFLI7Vz7il3Gb3WxoHTy12C9QFHjRhW7hKK7rZ5TeUuoUNj160tdwl9lnvMZmZmGeJgNjMzyxAHs5mZWYY4mM3MzDLEwWxmZpYhDmYzM7MMcTCbmZllSK8LZknNklZJWiPpMUlXSqpqs81dkh7Omb9G0hU584sl3Zgzf7WkK9PpaknbJc3rgeaYmZntodcFM/BaREyIiCOB9wEfBL7SulLSAcCxwAGSRqSLfwucnK6vAoYAR+Yc82TgoXT6/cB64MOSVMJ2mJmZvUVvDOa/iIgXgFnAZ3JCdApwN3ArcGm67CHSYCYJ5NXAy5IOlDQAGAs8mq6fBnwH2AScWPJGmJmZ5ejVwQwQEc+QtOOQdNE04Jb0a1q6zfNAk6RhJAH9MPA74CSgDng8InZLehtwBvDL3P3bkjRLUr2k+uZXG0vWNjMzqzy9PphTApA0FHg38GBEPEkSxuPTbVp7za3B/HDO/G/Tbc4FlkbEq8DPgQ9J6tf2wyJiQUTURURdv30Hl7BZZmZWaXp9MEt6F9AMvABMBQ4ENkjaCNTy5uns1uvMR5Gcyn6EpMece315GnBmuu8K4GDgtB5ohpmZGdDLg1lSDXAD8L2ICJJgPSsiaiOiFpjInteZzwV2RERzROwADiAJ54clDQJOBYbl7P9p8pzONjMzK4XeGMxva31cCvgfYAnwVUm1wDCSnjAAEbEBeEnSCUADyd3Yj+QcqwFojIjtwEXA/RGxK2f9XcD56Q1iZmZmJdfrxmOOiLdc801tBA5vZ/tjc2YHtVk3M2f6JuCmNut3ADXdKtTMzKwbemOP2czMrM9yMJuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLECWP/1p31dXVRX19fbnLMDPrVSStiIi6cteRRe4xm5mZZYiD2czMLEMczGZmZhniYDYzM8sQB7OZmVmGOJjNzMwyxMFsZmaWIQ5mMzOzDHEwm5mZZYiD2czMLEP8Ss69JOllYH256yijIcD2chdRRm5/5ba/ktsOe9/+4RFRU6xi+pLqchfQB6yv5Pe9Sqp3+93+ctdRDpXcdnD7S8mnss3MzDLEwWxmZpYhDua9t6DcBZSZ21/ZKrn9ldx2cPtLxjd/mZmZZYh7zGZmZhniYDYzM8sQB/NekHSWpPWSnpY0p9z1lJKkd0paKmmtpDWSLk+XHyTpV5KeSv88sNy1lpKkfpIelfTLdL5i2i/pAEkLJa1L/x2cVGHtn53+218t6RZJA/ty+yX9SNILklbnLMvbXklfSn8Wrpf0gfJU3Tc4mLtJUj/g+8DZwDhgmqRx5a2qpJqAz0fEWOBE4NNpe+cA90XEKOC+dL4vuxxYmzNfSe3/DnBvRIwBjiH5PlRE+yUdDnwOqIuI8UA/4FL6dvtvAs5qs6zd9qY/Cy4Fjkz3uS79GWnd4GDuvuOBpyPimYjYDdwKXFDmmkomIv4YESvT6ZdJfigfTtLmm9PNbgYuLEuBPUDSEcA5wI05iyui/ZIGAe8FfggQEbsj4s9USPtT1cDbJFUD+wLP04fbHxEPADvaLM7X3guAWyNiV0RsAJ4m+Rlp3eBg7r7Dgc0588+ly/o8SbXAe4DfAUMj4o+QhDdwSBlLK7VvA18EWnKWVUr73wVsA/4jPZV/o6T9qJD2R8QW4JvAJuCPQGNELKFC2p8jX3sr9udhKTiYu0/tLOvzz55J2h/4OXBFRLxU7np6iqRzgRciYkW5aymTauBY4PqIeA/wCn3rtG2H0mupFwAjgHcA+0n6SHmrypSK/HlYKg7m7nsOeGfO/BEkp7b6LEn9SUL5pxFxR7r4T5IOS9cfBrxQrvpK7BTgfEkbSS5bnC7p/1I57X8OeC4ifpfOLyQJ6kpp/5nAhojYFhFvAHcAJ1M57W+Vr70V9/OwlBzM3bccGCVphKR9SG58WFTmmkpGkkiuL66NiG/lrFoEzEinZwB39XRtPSEivhQRR0RELcnf9f0R8REqp/1bgc2S/ipddAbwBBXSfpJT2CdK2jf9v3AGyX0WldL+Vvnauwi4VNIASSOAUcDvy1Bfn+A3f+0FSR8kue7YD/hRRPxbeSsqHUmnAr8BGnjzGuuXSa4z3wYMI/nhdUlEtL1hpE+RNBm4KiLOlXQwFdJ+SRNIbnzbB3gG+FuSX+4rpf1fBaaSPKHwKPBxYH/6aPsl3QJMJhne8U/AV4BfkKe9kv4B+CjJ9+eKiPjvnq+6b3Awm5mZZYhPZZuZmWWIg9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzTog6VBJt0r6g6QnJP2XpNEl/sxlkuo62eYKSfvmzP+XpAOK8NkbJTVIekzSEkmH7u0xu1HDhPRRRLOK5GA2yyN9kcSdwLKIGBkR40ie3R5a3soAuIJkIAUAIuKD6aASxXBaRBwD1JO0t1NFHkloAuBgtorlYDbL7zTgjYi4oXVBRKyKiN9Imtw6JjOApO9JmplOb5T0NUkPS6qXdKykxWmv+5PpNnn3zyXp+vQYa9IXXCDpcyTva14qaWnOZw6R9HVJ/ztn/7mSPp9Of0HSckmPtx6rEw8A71YyBvU3cvb9RE4blkr6GdCQbvfNtMf9uKTPpttNlPRrSSvS70PrKx2XpfX+XtKTkv46fYvevwBTJa2SNLWAOs36lOpyF2CWYeOB7g5asTkiTpJ0Dcm4tqcAA4E1wA0d7djGP0TEjrRHep+koyPiWklXkvRst7fZ/laSt9Fdl85/GDhL0vtJXpN4PMmAA4skvTcd2i+fc0ne9PYxktGUjpM0AHhI0pJ0m+OB8RGxQdKnSAZ5eE9ENEk6KH2/+neBCyJiWxq0/0byhiiA6og4Pj11/ZWIOFPSP5OMe/yZLnyfzPoMB7NZabS+N70B2D8dw/plSa938VrwhyXNIvm/ehgwDng838YR8aikQyS9A6gBXoyITWkv+/0kr5KE5FWSo0h6xW0tldScfs4/kryG82hJF6frB6f77gZ+n46/C8lADzdERFNayw5J40l+wflVcmWAfiTDJrZqHQxlBVBbwPfDrM9zMJvltwa4OM+6Jva8FDSwzfpd6Z8tOdOt89UF7E86GMBVwHER8aKkm9rbrh0L07oPJelBQ9JLnhcRPyhg/z164um19s9GxOI29U0mGf7xL4t461B/AtZExEl5Pqv1e9OMfx6ZAb7GbNaR+4EBkv6udYGk4yRNAp4FxqWj6QwmGW2oKwrZfxBJ8DVKGgqcnbPuZeDteY59K8kIWBeThDTAYuCjSsbTRtLhkg7Js39bi4FPpaelkTRa0n7tbLcE+KSk6nS7g4D1QI2kk9Jl/SUd2cnnddQ2sz7PwWyWRyQjvHwIeF9649YaYC7wfERsJhll53Hgp7x5irjQY3e6f0Q8li5fA/wIeChn9QLgv1tv/mqz3xqSYNsSEX9Mly0BfgY8LKmBJLALDb8bSYZ4XClpNfAD2u/d3kgy4tDjkh4DpkfEbpJfEL6eLltFMo5xR5aS/NLim7+sInl0KTMzswxxj9nMzCxDHMxmZmYZ4mA2MzPLEAezmZlZhjiYzczMMsTBbGZmliEOZjMzswz5/xAJuixmSPInAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "getVisual(filteredDf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LLht7SM9oWMe"
   },
   "source": [
    "#### Statistical Metrics vs Visual Utility"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "id": "zwecDN_bodg1"
   },
   "outputs": [],
   "source": [
    "import scipy.stats as stats"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XWQW85sr-gns"
   },
   "source": [
    "The code for generating the pilot plots and utility metrics csv can be found at [pilotPrivatePlotGenerationAndStatsErrors.ipynb](pilotPrivatePlotGenerationAndStatsErrors.ipynb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "id": "rZz2_zZ0NMZT"
   },
   "outputs": [],
   "source": [
    "dfMetrics = pd.read_csv('./statisticalUtilityMetrics.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 112
    },
    "id": "jfcNLnI40xsb",
    "outputId": "86dc591a-fb49-42ba-ebc7-baccefed7953"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>tau</th>\n",
       "      <th>pValue</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Random Query</th>\n",
       "      <td>-0.17027</td>\n",
       "      <td>0.00051</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>MSSIM</th>\n",
       "      <td>0.47452</td>\n",
       "      <td>0.00000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  tau  pValue\n",
       "Random Query -0.17027 0.00051\n",
       "MSSIM         0.47452 0.00000"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from re import M\n",
    "import numpy as np\n",
    "len(answerDataFrame['totalRating'])\n",
    "\n",
    "metrics = ['Random Query', 'MSSIM']\n",
    "\n",
    "associationDf = pd.DataFrame(columns = ['tau', 'pValue'])\n",
    "for m in metrics:\n",
    "  tau, p_value = stats.kendalltau(dfMetrics['totalRating'], dfMetrics[m])\n",
    "  row = pd.Series({'tau':tau,'pValue':p_value},name= m)\n",
    "  associationDf = associationDf.append(row)\n",
    "\n",
    "associationDf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Correlation Coefficient Comparison\n",
    "\n",
    "We use Fisher’s z-transformation to test all the different correlations for any difference. If the p-value is less than .05 for at least one of the comparisons we accept hypothesis 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Code copied from https://github.com/psinger/CorrelationStats/blob/master/corrstats.py\n",
    "\n",
    "from scipy.stats import t, norm\n",
    "from math import atanh, pow\n",
    "from numpy import tanh\n",
    "\n",
    "def independent_corr(xy, ab, n, n2 = None, twotailed=True, conf_level=0.95, method='fisher'):\n",
    "    \"\"\"\n",
    "    Calculates the statistic significance between two independent correlation coefficients\n",
    "    @param xy: correlation coefficient between x and y\n",
    "    @param xz: correlation coefficient between a and b\n",
    "    @param n: number of elements in xy\n",
    "    @param n2: number of elements in ab (if distinct from n)\n",
    "    @param twotailed: whether to calculate a one or two tailed test, only works for 'fisher' method\n",
    "    @param conf_level: confidence level, only works for 'zou' method\n",
    "    @param method: defines the method uses, 'fisher' or 'zou'\n",
    "    @return: z and p-val\n",
    "    \"\"\"\n",
    "\n",
    "    if method == 'fisher':\n",
    "        xy_z = 0.5 * np.log((1 + xy)/(1 - xy))\n",
    "        ab_z = 0.5 * np.log((1 + ab)/(1 - ab))\n",
    "        if n2 is None:\n",
    "            n2 = n\n",
    "\n",
    "        se_diff_r = np.sqrt(1/(n - 3) + 1/(n2 - 3))\n",
    "        diff = xy_z - ab_z\n",
    "        z = abs(diff / se_diff_r)\n",
    "        p = (1 - norm.cdf(z))\n",
    "        if twotailed:\n",
    "            p *= 2\n",
    "\n",
    "        return z, p\n",
    "    elif method == 'zou':\n",
    "        L1 = rz_ci(xy, n, conf_level=conf_level)[0]\n",
    "        U1 = rz_ci(xy, n, conf_level=conf_level)[1]\n",
    "        L2 = rz_ci(ab, n2, conf_level=conf_level)[0]\n",
    "        U2 = rz_ci(ab, n2, conf_level=conf_level)[1]\n",
    "        lower = xy - ab - pow((pow((xy - L1), 2) + pow((U2 - ab), 2)), 0.5)\n",
    "        upper = xy - ab + pow((pow((U1 - xy), 2) + pow((ab - L2), 2)), 0.5)\n",
    "        return lower, upper\n",
    "    else:\n",
    "        raise Exception('Wrong method!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(16.827402661799514, 0.0)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "independent_corr(associationDf['tau'][0], associationDf['tau'][1], 1200, 1200)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Preregistration.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}