18.06-PS5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "#### 18.06 Spring 2015\n",
    "# Problem Set 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "\n",
    "# Running this cell will set up auto-grading\n",
    "# It might take a while to execute.\n",
    "\n",
    "# Please fill in the MIT id form that appears below. You only need to do this once.\n",
    "\n",
    "Pkg.rm(\"Homework\")\n",
    "Pkg.clone(\"git://github.com/shashi/Homework.jl.git\")\n",
    "\n",
    "using Homework\n",
    "\n",
    "Homework.show_mit_form()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Homework.progress()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 1* (3 points)*\n",
    "\n",
    "**Projection Matrix**\n",
    "\n",
    "On paper or using Julia, find the projection matrix onto  the plane $$x−y−2z=0$$ by first\n",
    "finding two independent vectors in this plane.  <br> \n",
    "Find the projection matrix P, and display it to three decimal places\n",
    "by typing round(P,3).  (You can compute P on paper and enter it, or compute P with Julia.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "attempts": 1,
    "collapsed": false,
    "explanation": null,
    "max_attempts": 1000,
    "max_score": 3,
    "precision": 3,
    "question": "1",
    "score": 0,
    "status": 1
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 2 *(2 points)*\n",
    "\n",
    "Construct a 3x3 matrix A with no zero entries whose columns are mutually perpendicular. The first column should be [1,1,1], the second column should be [1,2,-3], and\n",
    "the third column has bottom entry 1.\n",
    "\n",
    "Remember, the Julia syntax for constructing arrays by column is: `A=[[a,b,c][d,e,f][g,h,i]]` <br>\n",
    "The array should be labeled as a `3x3 Array{Int64,2}`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "2a"
   },
   "outputs": [],
   "source": [
    "A = [ [1, 1, 1] [1, 2, -3] [?, ?, 1] ]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is $A^TA$?  (Julia: `A' * A`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "2b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Problem 3 (3 points)\n",
    "\n",
    "Construct the projection matrix $P_1$ onto the line through the vector (1,0) <br>\n",
    "Express your answer as a 2x2 Array{Int64,2}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "3a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct the projection matrix $P_2$ onto the line through the vector (1,−1) <br>\n",
    "(Note: if you are computing in Julia, `dot(u,v)` may be more useful than `u'*v` for vectors, as\n",
    "the latter creates a matrix not a scalar.) <br>\n",
    "Express your answer as a 2x2 Array{Float64,2}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "3b",
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Is it true that $(P_1+P_2)^2=P_1+P_2$?  In other words, is $(P_1+P_2)$ a projection matrix? <br>\n",
    "Answer true or false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "3c"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 4\n",
    "### Problem 4a  *(1 point)*\n",
    "Suppose b equals 2 times the first column of A.<br> What is the projection of b onto the column space of A? <br> Type your answer in the form (double quote) (letter) (double quote) as in `\"x\"`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "4a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 4b  *(1 point)*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Is it always true that $P=I$ in this case?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "4b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 4c  *(1 point)*\n",
    "\n",
    "Suppose b=(0,2,4) and the columns of A are (0,1,2) and (1,2,0). <br> What is  p, the projection of b? (format: Int64 vector  e.g. [1,2,3,4] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "4c"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 4d  *(1 point)*\n",
    "Compute the projection matrix $P_1$. <br> Enter or return `int(21 * P1)` as a `3x3 Array{Int64,2}`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "4d"
   },
   "outputs": [],
   "source": [
    "A  = [ [0,1,2] [1,2,0] ]\n",
    "P₁ = ?\n",
    "int(21*P₁)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 5 *(3 pt)*\n",
    "\n",
    "What linear combination of $(1,2,−1)$ and $(1,0,1)$ is closest to $b=(2,1,1)$? <br>\n",
    "Enter the vector as integers, (e.g. [1,2,3])  or compute `int(P*b)`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "3",
    "question": "5a"
   },
   "outputs": [],
   "source": [
    "A=[ [1,2,-1] [1,0,1] ]\n",
    "P = ?\n",
    "b=[2,1,1]\n",
    "int(P*b)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 6\n",
    "\n",
    "### Problem 6a *(1 point)*\n",
    "Suppose L is a one-dimensional subspace (a line) in $R^3$. <br>\n",
    "Its orthogonal complement $L^⊥$ is the   <span style=\"border-bottom: 3px solid black; padding-left: 50px\">&nbsp;</span> perpendicular to L.  <br>\n",
    "(Format, write a familiar word from elementary geometry in small letters surrounded by double quotes as in `\"circle\"`)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "6a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 6b *(1 point)*\n",
    "Then  $(L^⊥)^⊥$  is a <span style=\"border-bottom: 3px solid black; padding-left: 50px\">&nbsp;</span>\n",
    "perpendicular to $L^⊥$. (geometry word again)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "6b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem 6c *(1 point)*\n",
    "In fact, $(L^⊥)^⊥$  is the same as  <span style=\"border-bottom: 3px solid black; padding-left: 50px\">&nbsp;</span> <br> (Format: letter in double quotes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "6c"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 7. Fitting Lines\n",
    "\n",
    "** 7a) (2 pts.) **\n",
    "\n",
    "Find the best line C+Dt to fit b = 7,7,21 at times t = −1,1,2.  <br>\n",
    "Format: [C, D] (ints)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "2",
    "question": "7a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 7b) (2 pts.)**\n",
    "\n",
    "Find the best line C+Dt to fit b = 4,2,−1,0,0 at times t =− 2,−1,0,1,2. <br>\n",
    "Format: [C, D] (ints) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "2",
    "question": "7b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 8. (2 pts.) Error Term\n",
    "\n",
    "Suppose that Ax=b has no solution. Let p be the projection of b on the column space of A. The length of b−Ax is minimized at x = $\\hat x$. <br>\n",
    "The error vector e = b−p is orthogonal to\n",
    "\n",
    "1. b <br>\n",
    "2. p <br>\n",
    "3. e <br>\n",
    "4. $\\hat x$ <br>\n",
    "5. A$\\hat x$\n",
    "\n",
    "Choose the answers that apply in order, example: [1,3,5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "2",
    "question": "8"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 9\n",
    "\n",
    "**Problem 9a.  (1 pt)** If A has three orthogonal columns each of length 4, what is A$^T$A?\n",
    "\n",
    "$1. \\hspace{.2in} \\left( \\begin{array}{cc} 2 & 0 & 0  \\\\ 0 & 2 & 0  \\\\ 0 & 0 & 2  \\end{array} \\right)$\n",
    "\n",
    "$2. \\hspace{.2in} \\left( \\begin{array}{cc} 4 & 0 & 0 \\\\ 0 & 4 & 0  \\\\ 0 & 0 & 4  \\end{array} \\right)$\n",
    "\n",
    "$3. \\hspace{.2in} \\left( \\begin{array}{cc} 16  & 0 & 0  \\\\ 0 & 16 & 0  \\\\ 0 & 0 & 16   \\end{array} \\right)$\n",
    "\n",
    "$4. \\hspace{.2in} \\left( \\begin{array}{cc} 4  & 4 & 4  \\\\ 4 & 4 & 4  \\\\ 4 & 4 & 4   \\end{array} \\right)$\n",
    "\n",
    "$5. \\hspace{.2in} \\left( \\begin{array}{cc} 16  & 16 & 16  \\\\ 16 & 16 & 16  \\\\ 16 & 16 & 16   \\end{array} \\right)$\n",
    "\n",
    "Format: one integer from 1, 2, 3, 4, 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "9a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**9b. (2 pts.)**  If A has three orthogonal columns of lengths 1,2,3, what is A$^T$A? <br>\n",
    "Format:<br> [1 2 3  <br>\n",
    "         4 5 6  <br>\n",
    "         7 8 9]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "2",
    "question": "9b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 10. (3 pts) (Gram-Schmidt process)\n",
    "\n",
    "(a) Probably with pencil and paper, find orthonormal vectors q$_1$,q$_2$,q$_3$ such that q$_1$,q$_2$ span the column space of \n",
    "$A=\\left( \\begin{array}\\\\1&1\\\\ 2 & -1 \\\\ -2 & 4 \\end{array} \\right)$ <br>\n",
    "using the Gram-Schmidt process on the columns of A. (Hint: to find $q_3$, add a column to A so that the three columns of A are linearly independent, and then apply the Gram-Schmidt process.) <br>\n",
    "(If you do qr with Julia you run the risk of not exercising skills for your exam, and it's possible\n",
    "you will compute $\\pm$q.)\n",
    "\n",
    "**10a(i) (1 pt.) q$_1$=?**\n",
    "\n",
    "Format:  `[ 5//7, -1//3, 2//9]`  (Vector of Rationals)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "10a.1"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 10a(ii) (1 pt.)** q$_2$=?  (Format as above)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "10a.2"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**10a(iii)** (1 pt.) what is q$_3$ if its first component is positive? (Format as above)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "10a.3"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**10(b) (1pt)** Which of the four fundamental subspaces contains q$_3$?\n",
    "\n",
    "1. Row Space <br>\n",
    "2. Column Space <br>\n",
    "3. Null Space <br>\n",
    "4. Left nullspace \n",
    "\n",
    "(Format: 1, 2, 3, or 4)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "10b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**10(c) (1 pt)** Solve Ax=(1,2,7) by least squares.\n",
    "\n",
    "Format: integer vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "10c"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 11. Gram Schmidt Process II\n",
    "\n",
    "**11a.** If A=QR then A$^T$A=R$^T$R= \n",
    "<span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span>\n",
    "(1)<span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span> triangular times <span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span>(2) <span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span>triangular. Please write the type of matrix in corresponding blanks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 11a.(i) (1 pt)** Write (1) as a lowercase string (example: \"symmetric\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "11a.1"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**11a.(ii) (1 pt)** Write (2) as a lowercase string (example: \"symmetric\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "11a.2"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**11b. (1 pt)** Gram-Schmidt on A corresponds to elimination on $A^TA$. The pivots for A$^T$A must be the squares of diagonal entries of R.<br> Let A=\n",
    "$\\left( \\begin{array} \\\\{-1} & 1 \\\\ 2 & 1 \\\\ 2 & 4 \\end{array}\\right)$\n",
    "so that A$^T$A=\n",
    "$\\left( \\begin{array} \\\\ 9  & 9 \\\\ 9 & 18 \\end{array}\\right)$ =\n",
    "$\\left( \\begin{array} \\\\ 1  & 0 \\\\ 1 & 1 \\end{array}\\right)$\n",
    "$\\left( \\begin{array} \\\\ 9  & 0 \\\\ 0 & 9 \\end{array}\\right)$\n",
    "$\\left( \\begin{array} \\\\ 1  & 1 \\\\ 0 & 1 \\end{array}\\right)$ <br>\n",
    " What is the R obtained from the Gram-Schmidt process on A?\n",
    " \n",
    " (Format: Array{Int64,2}, example [1 2;3 4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "11b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 12. (Orthogonal Matrix)\n",
    " \n",
    "**12a. (1 pt)**  Q$^{−1}$ is an orthogonal matrix when Q is an orthogonal matrix. <br>\n",
    "true or false ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "12a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**12b. (1 pt)**  If Q (3 by 2) has orthonormal columns then ||Qx|| always equals ||x||. <br>\n",
    "true or false?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "12b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Problem 13. (Fourier Basis)\n",
    "\n",
    "**13a. (1 pt)** By the trig identity \n",
    "$ 2 \\cos(jx) \\cos(kx) = \\cos((j+k)x) +\\cos((j−k)x),$ <br>\n",
    "we have that<br>\n",
    "$\\frac{1}{\\pi}∫_0^{2π}$ cos(jx)cos(kx)dx = <span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span>  when j ≠ k.\n",
    "\n",
    "Format: int"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "13a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** 13b. (1 pt)** And $\\frac{1}{\\pi}∫_0^{2π}$ cos(jx)cos(kx)dx = <span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span> \n",
    "when j =  k.\n",
    "\n",
    "Format: int\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "13b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Problem 14. (Legendre Polynomials)\n",
    "\n",
    "**(a) (3 pts)** Let $f(x)=1,g(x)=x,h(x)=x^2−1/3$. Compute (by hand):\n",
    "\n",
    "[  ∫$_{-1}^{1}$f(x)g(x)dx ,  ∫$_{-1}^{1}$g(x)h(x)dx,  ∫$_{-1}^{1}$h(x)f(x)dx]\n",
    "\n",
    "Format: 3-element array of int's ([1,3,5])\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "3",
    "question": "14a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(b) ( 1 pt )** Write 2x$^2$ as a combination of those orthogonal functions. Coefficients of f,g,h are typed in as\n",
    "[<span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span>, <span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span>, <span style=\"border-bottom: 2px solid black; padding-left: 75px\">&nbsp;</span>]\n",
    "\n",
    "Format: (Three Rationals: [5, -3//5, 12] )\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "question": "14b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(c) (2 pts)** Choose c so that the fourth polynomial x$^3$−cx is orthogonal to the first three. <br> (Here, two functions are said to be orthogonal when the integration of their product from x=−1 to x=1 is equal to 0.)\n",
    "\n",
    "Format: Rational such as 22//7 ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "2",
    "question": "14c"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Problem 15. (5 pts in total)  Linear Regression\n",
    "\n",
    "Where does projection and least squares approximation occur in the real world? The answer is anywhere there is data. All data comes with some measurement error. So when you look to fit your data to a curve in science and engineering, you are using the method of least squares approximation to get the best fit. The most basic example of least squares approximation also has another name: linear regression. The goal of linear regression is to fit your data with the best fit line. This is the common term used in statistical analysis.\n",
    "\n",
    "In this example, we'll analyze data to predict the thermal coefficient of expansion of steel given discrete temperatures T and the coefficient of thermal expansion α.\n",
    "\n",
    "If we assume that there are coefficients c and d such that α(T) = c+dT, find the matrix A in terms of the data so that\n",
    "A$\\left[ \\begin{matrix} c \\\\ d \\end{matrix} \\right]$=α.\n",
    "\n",
    "Use the method of least squares to find the equation. Find the error between the expected value given by your equation, and your actual data. Find the residual, which is the magnitude squared of that error vector.\n",
    "\n",
    "You might expect that to find the least squares approximation, you will have to use some code of the form: x = A'*A\\A'*α, but you don't have to! Julia already uses least squares approximation in their algorithm for the backslash operator \\. This means that x = A'*A\\A'*α and x = A\\α will give you the same vector output! (Except for roundoff error.)  Go ahead and check it out."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Data (do not edit, but do execute)\n",
    "T = [80 : -20: -300]; # (The Temperature numbers 80,60,40,...,-300)\n",
    "α = [6.47e-6; 6.36e-6; 6.24e-6; 6.12e-6; 6e-6; 5.86e-6; 5.43e-6; 5.28e-6; 5.09e-6;4.91e-6; 4.72e-6;4.52e-6; 4.3e-6; 4.08e-6; 3.83e-6; 3.58e-6; 3.33e-6; 3.07e-6; 2.76e-6; 2.45e-6];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(2 pts)** Construct the matrix A that is needed for the best fit line:\n",
    "(Things to try: What does ones(T) do?, what does [T T] do?)\n",
    "\n",
    "Format: 20x2 Array{Int64,2}:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "2",
    "question": "15a"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(1 pt) Use the method of least squares, \n",
    "also known as linear regression  to find the best fit line to the data.<br> (To type the α you can type  \\alpha[tab] )\n",
    "\n",
    "Format: 2-element Array{Float64,1}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "precision": 12,
    "question": "15b"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Not graded but interesting to see the comparison between the data and the line\n",
    "\n",
    "using PyPlot\n",
    "#Plot data and line, compare\n",
    "plot(T,α,\"o\");\n",
    "plot(T,A*x,\"r\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(1 pt)** The error is the difference between `A*x` and `α`.\n",
    "Find the error vector $e$\n",
    "\n",
    "Format: 20-element Array{Float64,1}."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": "1",
    "precision": 12,
    "question": "15c"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(1 pt)** The total error is the magnitude squared of the error vector.\n",
    "Find the total error.\n",
    "\n",
    "Format: a Floating point number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "max_attempts": 1000,
    "max_score": 1,
    "precision": 22,
    "question": "15d"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Homework.progress()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "homework": {
   "admins": [
    "mit.edelman@gmail.com"
   ],
   "course": "MIT-18.06-Spring-2015",
   "mode": "answering",
   "problemset": "pset5"
  },
  "kernelspec": {
   "display_name": "Julia 0.3.11",
   "language": "julia",
   "name": "julia-0.3"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "0.3.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}