{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"source": [
"#**Lab 4**\n",
"\n",
"**Name: (your name)**\n",
"\n"
],
"metadata": {
"id": "6VpZ01tyznHo"
}
},
{
"cell_type": "markdown",
"source": [
"Some remarks before you start:\n",
"* Import the Python packages below each time you start a session.\n",
"* Before submitting your report, remove all ouputs by clicking on **Edit🠦Clear all ouputs**. Then download your report by clicking on **File🠦Download🠦.ipynb**. This is the file you will submit on Canvas.\n",
"\n",
"In this lab, you will practice:\n",
"* graph functions of two variables and their level sets\n",
"* use Gradient Descent/Ascent method to find local min/max\n",
"\n",
"\\begin{array}{|c| c|}\n",
" \\hline\n",
" \\text{Problems} & \\text{Points}\\\\\n",
" \\hline\\hline\n",
" 1, 2, 3 & 5 \\\\\n",
" \\hline\n",
" 4, 5 & 6 \\\\\n",
" \\hline\n",
" \\text{Readability of your report} & 3 \\\\\n",
" \\hline\n",
" \\text{Total: 5} & \\text{Total: 30} \\\\\n",
" \\hline\n",
"\\end{array}\n",
"\n"
],
"metadata": {
"id": "3cVoL2qvzufn"
}
},
{
"cell_type": "markdown",
"source": [
"##**I. Import necessary Python packages**\n",
"Execute the following code each time you start a session."
],
"metadata": {
"id": "bQagzUIHzzb_"
}
},
{
"cell_type": "code",
"source": [
"from matplotlib.pyplot import*\n",
"from matplotlib.pyplot import contour as contourplot\n",
"from matplotlib.pyplot import scatter as scatterplot\n",
"from plotly.graph_objects import*\n",
"from autograd.numpy import*\n",
"from autograd import elementwise_grad as diff"
],
"metadata": {
"id": "sGBvQoGSzoih"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## **II. Graph a multivariable function and its level sets**\n",
"\n",
"To define a function on Python, you will use the keyword **def**. For example, the code below defines the function $$f(x,y)=\\cos(x)\\sin(y)$$"
],
"metadata": {
"id": "yGFzT_PXz431"
}
},
{
"cell_type": "code",
"source": [
"def f(x,y):\n",
" return cos(x)*sin(y)"
],
"metadata": {
"id": "kxIif3jP1xeP"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"To graph the function, you first need to generate an array of $x$-values and an array of $y$-values using the command **linspace**. For example,"
],
"metadata": {
"id": "QqnmIFJyDe1-"
}
},
{
"cell_type": "code",
"source": [
"x = linspace(-5, 5, 26) # generate 26 evenly-spaced points between -5 and 5\n",
"y = linspace(-5, 5, 26)"
],
"metadata": {
"id": "_fLg6iO5z5sG"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Next, you \"mesh\" the array of $x$-values and the array of $y$-values on the coordinate system. Then you create a matrix of $z$-values."
],
"metadata": {
"id": "XiBiRB6CcBvc"
}
},
{
"cell_type": "code",
"source": [
"X, Y = meshgrid(x, y)\n",
"Z = f(X,Y)"
],
"metadata": {
"id": "KhlTQZAycceB"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"There are two packages of Python that are commonly used for graphing a multivariable function, namely **plotly** and **matplotlib**. Plotly has interactive features that allow you to drag, rotate, scale a graph. Matplotlib's graphs are static but are more customizable. With **plotly**, you graph the function $f$ as follows. You can interact with the graph with your mouse."
],
"metadata": {
"id": "PZSxjPbZFM2L"
}
},
{
"cell_type": "code",
"source": [
"fig = Figure(data=[Surface(z=Z, x=X, y=Y)])\n",
"fig.update_layout(title='$f(x, y) = \\cos x\\sin y$')"
],
"metadata": {
"id": "ndThAdz3nzYV"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"With **matplotlib**, you graph the function $f$ as follows."
],
"metadata": {
"id": "FjIpWh3Ff14r"
}
},
{
"cell_type": "code",
"source": [
"fig = figure().add_subplot(projection='3d')\n",
"fig.plot_surface(X, Y, Z)\n",
"fig.view_init(elev=55, azim=40) # (optional) set viewpoint, where elev is elevation and azim is azimuth angle (in degree)."
],
"metadata": {
"id": "jxalM0SpgDwe"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Suppose you want to sketch a contour map of $c$-level curves, where $c$ ranges from $-2$ to $2$ with a step size of $0.2$. With **plotly**, you will run the following. *Note that you can use your mouse to zoom in any region (drag with left click) or zoom out (double click).*"
],
"metadata": {
"id": "4tQvmrM3ixZ1"
}
},
{
"cell_type": "code",
"source": [
"fig2 = Figure(data=Contour(z=Z,x=x,y=y,contours=dict(start=-2.0,end=2.0,size=0.2)))\n",
"fig2.update_layout(width=600,height=600)"
],
"metadata": {
"id": "mZA-iba7i-Tf"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"With **matplotlib**, you run the following:"
],
"metadata": {
"id": "NZUD96ggjaOW"
}
},
{
"cell_type": "code",
"source": [
"contourplot(X, Y, Z, levels=linspace(-2,2,21), cmap='viridis')\n",
"colorbar()"
],
"metadata": {
"id": "3Bti5ooH8P4i"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"###**Exercise 1**\n",
"A tumor in a patient's brain is detected by an MRI scan, which models density as a function of position:\n",
"\n",
"$$f(x,y)=e^{-2(x^2 + y^2)} + 0.3\\,e^{-(x^2 + y^2)}$$\n",
"\n",
"where $D(x,y)$ represents tissue density at point $(x,y)$.\n",
"\n",
"* Graph the density function to visualize the tumor shape.\n",
"* Find level curves for different densities and determine the tumor boundary.\n",
"* If the tumor boundary is determined to be where the tissue density is equal to $0.15$, find the area of this tumor."
],
"metadata": {
"id": "DIQOwXFQHl-b"
}
},
{
"cell_type": "markdown",
"source": [
"###**Exercise 2**\n",
"Meteorologists model temperature (in Celsius) as a function of latitude and longitude:\n",
"\n",
"$$T(x,y)=20 - 0.01x^2-0.02y^2+5\\sin(0.1x)\\cos(0.1y)$$\n",
"\n",
"where $x$ and $y$ represent position coordinates (kilometers from a reference point).\n",
"\n",
"* Graph this temperature distribution in 3D.\n",
"* Sketch level curves (isotherms) for specific temperatures $T=10^\\circ C,15^\\circ C,20^\\circ C$.\n",
"* Interpret how temperature changes in different regions."
],
"metadata": {
"id": "Go2IWVSOKeW_"
}
},
{
"cell_type": "markdown",
"source": [
"##**III. Gradient Descent/Ascent algorithm for optimization**\n",
"The *Gradient Descent method* is a simple algorithm to find a local minimum of a function $f$. You start with an initial guess $(x_0,y_0)$ and successively update it using the recursion\n",
"\n",
"$$(x_{n+1},y_{n+1})=(x_n,y_n)-\\alpha \\nabla f(x_n,y_n)$$\n",
"\n",
"This vector-equation is equivalent to a system of two equations:\n",
"\n",
"$$x_{n+1}=x_n-\\alpha f_x(x_n,y_n)$$\n",
"$$y_{n+1}=y_n-\\alpha f_y(x_n,y_n)$$\n",
"\n",
"The *Gradient Ascent method* is to find a local maximum. It uses the same recursion formulas as above, except that all minus signs are changed to plus signs. In practice, the common stopping condition is when the magnitude of the gradient vector $\\nabla f(x_n,y_n)$ is under a certain threshold\n",
"\n",
"$$|\\nabla f(x_n,y_n)|=\\sqrt{f_x(x_n,y_n)^2+f_y(x_n,y_n)^2}<\\epsilon$$\n",
"\n",
"or when $(x_n,y_n)$ falls out of a certain range of interest.\n",
"\n",
"The Python code to compute $(x_0,y_0)$, $(x_1,y_1)$, $(x_2,y_2)$,..., $(x_n,y_n)$ in the Gradient Descent method is as follows."
],
"metadata": {
"id": "Z_m2zyE8PBDJ"
}
},
{
"cell_type": "code",
"source": [
"def gradd(f,x0,y0,alpha,n):\n",
" dfdx = diff(f, argnum=0)\n",
" dfdy = diff(f, argnum=1)\n",
" x = float(x0)\n",
" y = float(y0)\n",
" print(0, x, y, f(x,y), sqrt(dfdx(x,y)**2+dfdy(x,y)**2))\n",
" points=[(x,y)]\n",
" for i in range(n):\n",
" xold = x\n",
" x = x - alpha*dfdx(x,y)\n",
" y = y - alpha*dfdy(xold,y)\n",
" points.append((x,y))\n",
" print(i+1, x, y, f(x,y), sqrt(dfdx(x,y)**2+dfdy(x,y)**2))\n",
" return points"
],
"metadata": {
"id": "0x6z6B0_XHkG"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"For illustration purposes, let us consider\n",
"\n",
"$$f(x,y)=\\cos(x)\\sin(y)$$"
],
"metadata": {
"id": "CRi7aHzEAtT9"
}
},
{
"cell_type": "code",
"source": [
"def f(x,y):\n",
" return cos(x)*sin(y)\n",
"\n",
"x = linspace(-5, 5, 26)\n",
"y = linspace(-5, 5, 26)\n",
"\n",
"X, Y = meshgrid(x, y)\n",
"Z = f(X,Y)"
],
"metadata": {
"id": "V1GrQDYn0QcB"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"With the initial guess $(x_0,y_0)=(1,2)$ and learning rate $\\alpha =0.1$, run the following to get a list of points $(x_0,y_0)$, $(x_1,y_1)$, $(x_2,y_2)$,..., $(x_5,y_5)$ in the Gradient Descent method."
],
"metadata": {
"id": "qjvqbmz10TIA"
}
},
{
"cell_type": "code",
"source": [
"points=gradd(f,1,2,0.1,5)"
],
"metadata": {
"id": "yxGHMwua0Bkw"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"With **plotly**, you can sketch the points on top of the contour map as follows:"
],
"metadata": {
"id": "5dWUil5E1hUC"
}
},
{
"cell_type": "code",
"source": [
"xn, yn = zip(*points)\n",
"fig = Figure()\n",
"fig = Figure(data=Contour(z=Z,x=x,y=y,contours_coloring='lines',contours=dict(start=-2.0,end=2.0,size=0.2)))\n",
"fig.add_trace(Scatter(\n",
" x=xn,\n",
" y=yn,\n",
" mode=\"markers+text\", # Markers with text labels\n",
" marker=dict(size=8, color=\"blue\"),\n",
" text=[f\"{i}\" for i in range(len(points))], # Labels\n",
" textposition=\"top center\"\n",
"))\n",
"fig.update_layout(width=600,height=600)\n",
"#fig.show()"
],
"metadata": {
"id": "D5OWhLyD6LAM"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"With **matplotlib**, you sketch the points on top of the contour map using the following code:"
],
"metadata": {
"id": "FxahX7vUA048"
}
},
{
"cell_type": "code",
"source": [
"x, y = zip(*points)\n",
"contourplot(X, Y, Z, levels=linspace(-2,2,11), cmap='viridis')\n",
"colorbar()\n",
"scatterplot(x=x,y=y,s=10) # s is the size of each dot\n",
"for i, (xn, yn) in enumerate(points):\n",
" text(xn, yn, str(i), fontsize=9, ha='right', va='bottom', color=\"red\")"
],
"metadata": {
"id": "7RGBMJL_A5YW"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"###**Exercise 3**\n",
"* Explain what each line of the function **gradd** defined above.\n",
"* Explain what goes wrong if you replace the three lines:\n",
"\n",
"```\n",
" xold = x\n",
" x = x - alpha*dfdx(x,y)\n",
" y = y - alpha*dfdy(xold,y)\n",
"```\n",
"by the two lines:\n",
"```\n",
" x = x - alpha*dfdx(x,y)\n",
" y = y - alpha*dfdy(x,y)\n",
"```"
],
"metadata": {
"id": "F_ezhh0EZjX_"
}
},
{
"cell_type": "markdown",
"source": [
"###**Exercise 4**\n",
"Let $f(x,y)=x^3 + xy^2 -6x +y^2$\n",
"* Graph the function on the square $(x,y)\\in[-3,3]\\times[-3,3]$.\n",
"* Sketch the contour map.\n",
"* Use the Gradient Descent/Ascent method to find the maximum and minimum values of $f$ in this square and find the values of $(x,y)$ where they are attained. Stop when either $(x_n,y_n)$ falls out of the square or $|\\nabla f(x_n,y_n)|< 0.01.$\n",
"* Visualize your sequence of $(x_n,y_n)$ as illustrated above."
],
"metadata": {
"id": "J9xAgbPfOvHc"
}
},
{
"cell_type": "markdown",
"source": [
"###**Exercise 5**\n",
"With the same function as in [Exercise 4](#grad),\n",
"* Use the Gradient Descent/Ascent method to find the maximum and minimum values of $f$ in the circle $x^2+y^2\\le 9$ and find the values of $(x,y)$ where they are attained. Stop when either $(x_n,y_n)$ falls out of the circle or $|\\nabla f(x_n,y_n)|< 0.01.$\n",
"* Visualize your sequence of $(x_n,y_n)$ as illustrated above.\n",
"* Illustrate an example of initial guess and learning rate such that the sequence eventually escapes the circle. Visualize this sequence."
],
"metadata": {
"id": "YuB9SU9c4S2A"
}
}
]
}