141_AcceleratedPCA.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a7d76853",
   "metadata": {},
   "source": [
    "<div style=\"background-color:#000;\"><img src=\"pqn.png\"></img></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b12ac1e8",
   "metadata": {},
   "source": [
    "### Import libraries and set up"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a48f90b5",
   "metadata": {},
   "source": [
    "We start by importing necessary libraries for data analysis, visualization, and machine learning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43a5bd07",
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext cuml.accel\n",
    "import yfinance as yf\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.decomposition import PCA\n",
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be402e6f",
   "metadata": {},
   "source": [
    "These libraries provide tools for financial data retrieval, data manipulation, numerical computations, principal component analysis, and data visualization. We'll use them to analyze S&P 500 stock data and perform dimensionality reduction."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a6a0176",
   "metadata": {},
   "source": [
    "### Download S&P 500 data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1076410a",
   "metadata": {},
   "source": [
    "We fetch the list of S&P 500 companies and download their historical stock data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39c3fe50",
   "metadata": {},
   "outputs": [],
   "source": [
    "snp_symbols = pd.read_html(\"https://en.wikipedia.org/wiki/List_of_S%26P_500_companies\")[0].Symbol.tolist()\n",
    "symbols = [sym.replace(\".\", \"-\") for sym in snp_symbols]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0df2f9f",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = yf.download(symbols, start=\"2020-01-01\", end=\"2024-12-31\")\n",
    "portfolio_returns = data['Close'].pct_change().dropna()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbda1a67",
   "metadata": {},
   "source": [
    "We retrieve the list of S&P 500 company symbols from Wikipedia and adjust them for compatibility with Yahoo Finance. Then, we download historical stock data for these companies from 2020 to 2024. We calculate the daily returns of each stock based on closing prices."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76c2652b",
   "metadata": {},
   "source": [
    "### Perform PCA analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13b37003",
   "metadata": {},
   "source": [
    "We apply Principal Component Analysis to reduce the dimensionality of our portfolio returns data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4accc7ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "pca = PCA(n_components=5)\n",
    "pca.fit(portfolio_returns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83cfd3b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "pct = pca.explained_variance_ratio_\n",
    "pca_components = pca.components_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05caee80",
   "metadata": {},
   "source": [
    "We use PCA to identify the main factors driving our portfolio returns. We set the number of components to 5, which will give us the five most significant factors. The explained variance ratio tells us how much of the total variance each component accounts for."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db368870",
   "metadata": {},
   "source": [
    "### Visualize PCA results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8899179",
   "metadata": {},
   "source": [
    "We create bar and line plots to visualize the contribution of each principal component."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d47056d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "cum_pct = np.cumsum(pct)\n",
    "x = np.arange(1,len(pct)+1,1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fec3781a",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.subplot(1, 2, 1)\n",
    "plt.bar(x, pct * 100, align=\"center\")\n",
    "plt.title('Contribution (%)')\n",
    "plt.xlabel('Component')\n",
    "plt.xticks(x)\n",
    "plt.xlim([0, 6])\n",
    "plt.ylim([0, 100])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76877dda",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.subplot(1, 2, 2)\n",
    "plt.plot(x, cum_pct * 100, 'ro-')\n",
    "plt.title('Cumulative contribution (%)')\n",
    "plt.xlabel('Component')\n",
    "plt.xticks(x)\n",
    "plt.xlim([0, 6])\n",
    "plt.ylim([0, 100])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f429a95f",
   "metadata": {},
   "source": [
    "We create two subplots: a bar chart showing the individual contribution of each component, and a line plot showing the cumulative contribution. This helps us visualize how much variance each component explains and how many components we need to explain most of the variance in our data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fe94022",
   "metadata": {},
   "source": [
    "### Calculate factor returns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8896b016",
   "metadata": {},
   "source": [
    "We use the PCA components to calculate factor returns for our portfolio."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "052bbbf8",
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.asarray(portfolio_returns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abedda02",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_returns = X.dot(pca_components.T)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4dadaeda",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_returns = pd.DataFrame(\n",
    "    columns=[\"f1\", \"f2\", \"f3\", \"f4\", \"f5\"], \n",
    "    index=portfolio_returns.index,\n",
    "    data=factor_returns\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4cb5306d",
   "metadata": {},
   "outputs": [],
   "source": [
    "factor_returns.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93a2c3e9",
   "metadata": {},
   "source": [
    "We transform our portfolio returns using the PCA components to get factor returns. These factor returns represent the behavior of our portfolio in terms of the five most significant factors we identified. We store these in a DataFrame for easy analysis."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f1174015",
   "metadata": {},
   "source": [
    "### Visualize factor returns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35f93103",
   "metadata": {},
   "source": [
    "We create a 3D scatter plot to visualize the relationships between the first three factors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88adc03a",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(111, projection='3d')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c3d0e0e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "sc = ax.scatter(data[:, 0], data[:, 1], data[:, 2], c=data[:, 2], cmap='viridis')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69507573",
   "metadata": {},
   "outputs": [],
   "source": [
    "ax.set_xlabel('f1')\n",
    "ax.set_ylabel('f2')\n",
    "ax.set_zlabel('f3')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83981c8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.colorbar(sc, ax=ax, label='f3')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4aa811bc",
   "metadata": {},
   "source": [
    "We create a 3D scatter plot to visualize how the first three factors relate to each other. Each point represents a day in our dataset, plotted according to its values for the first three factors. The color of each point is based on the value of the third factor, giving us an additional dimension of information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "914e6f57",
   "metadata": {},
   "source": [
    "<a href=\"https://pyquantnews.com/\">PyQuant News</a> is where finance practitioners level up with Python for quant finance, algorithmic trading, and market data analysis. Looking to get started? Check out the fastest growing, top-selling course to <a href=\"https://gettingstartedwithpythonforquantfinance.com/\">get started with Python for quant finance</a>. For educational purposes. Not investment advise. Use at your own risk."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}