{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "# Tutorial 01 – Working with Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "In this tutorial we will take a look at how to represent, manipulate, store, and load data. We will use two main packages called `NumPy` and `pandas`. The actual representation of data is handled by `NumPy` and it uses n-dimensional array objects to stored the data. Part of the code is written in C, C++, and Fortran to make it fast. `pandas` is high-level abstraction that makes data manipulation like selecting a specific subset of data, grouping, and basic plotting simple. Mastering just these two packages almost qualifies you as a data scientist.\n",
    "\n",
    "While none of these two packages is really doing any machine learning you can go quite a long way with them. `NumPy` alone is perfect for any linear algebra like matrix multiplication and it is an essential requirement for other machine learning packages. `pandas` is great for exploring data and storing/loading data.\n",
    "\n",
    "Remember, nicely prepared data are the key to success in machine learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "## NumPy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "`NumPy` is typically imported with alias `np` to save some characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "You can create a simple array right from a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "alist = [1,2,3,4]\n",
    "array = np.array(alist)\n",
    "array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "They behave quite similarly in terms of indexing and slicing. Doing any operations with arrays is quite different though. They behave more like vectors (which they are after all). The key difference is that it is a proper array and it cannot be easily extended with new elements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "print(array[1])\n",
    "print(array[1::2])\n",
    "print(array * 2)\n",
    "print(array + 10)\n",
    "print(array + array)\n",
    "print(array * array)\n",
    "# array.append(10) this does not work!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Notice that the math operations are done element-wise. Multiplying `array` with it self results in vector of squared elements. If you would like to do some linear algebra you need to call a method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "array.dot(array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Now its time to go to higher dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "matrix = np.array([\n",
    "    [1,2,3],\n",
    "    [4,5,6],\n",
    "    [7,8,9]])\n",
    "matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Multidimensional arrays can be also index in each dimension. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "matrix[1:, :2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "You can even use array of booleans to take elements on indices with value True."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "matrix[[True, False, True]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Often you will need to create larger arrays with some specific values and you would not want to manually fill all the numbers. Luckily, there are handful of methods at your disposal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "print(np.zeros(10))\n",
    "print(np.full([2, 4], fill_value=[2, 1]))\n",
    "print(np.identity(10))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Now, every array has its shape, i.e., dimensions. They are always represented as tuples and each dimension is called axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "print(array.shape)\n",
    "print(np.identity(10).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Note that when creating an array you can supply any wild shape you want the new array to have. That's right, you can make a tensor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "np.ones(shape=(3, 5, 7))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Every array can be reshaped into any shape you like as long as the number of elements is equal to the product of array dimensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 1</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Reshape the vector $\\mathbf{a}$ into shape $(3,4,2)$, multiply it with matrix $\\mathbf{M}$ and than flatten it back to vector. The expected result is $[ 11,  53,  95, 137, 179, 221, 263, 305, 347, 389, 431, 473]$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "M = np.arange(10, 12)\n",
    "a = np.arange(24)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "# TODO: your code goes here..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "All these operations are incredibly fast. Below is a code that takes the matrix and normalizes the values, i.e., divides all values by maximal value in the matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "matrix = np.random.randint(0, 100, (1000, 1000))\n",
    "%timeit -n 5 matrix / matrix.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 2</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Write a function that does the same thing with list of lists and compare the times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "def normalize(list_matrix):\n",
    "    # TODO: your code goes here...\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "list_matrix = matrix.tolist()\n",
    "%timeit -n 5 normalize(list_matrix)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 3</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Later in the course we will use metrics to evaluate machine learning models. Write a function `rmse(x, y)` that computes one such metric called Root Mean Squared Error (RMSE) of two vectors $x$ and $y$ given by the following formula $$\\text{RMSE} = \\sqrt{\\frac{\\sum_{i=0}^{n}{(x_i-y_i)^2}}{n}}$$\n",
    "\n",
    "The expected result for the predefined vectors is 2.16794833886788."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "x = np.array([0, 4, 1, 0, 2, 0, 2, 2, 4, 2])\n",
    "y = np.array([1, 1, 2, 3, 1, 4, 1, 0, 3, 4])\n",
    "\n",
    "def rmse(x, y):\n",
    "    # TODO: your code goes here...\n",
    "    pass\n",
    "\n",
    "rmse(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 4</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Typical operation in linear algebra you might encounter is a matrix multiplication. Let's say we measure three quantities (e.g. width, height, and depth of real life objects). For each observation we have a vector of these measurements $(w, h, d)$. Now suppose we need to transform these measurements for further processing such that each observation is transformed into $(w+h, h-d)$. We want to transform these observations at once using transformation matrix $\\mathbf{M}$.\n",
    "1. Stack observation into a matrix of observations $\\mathbf{X}$ with each row corresponding to one of the observations, i.e. $\\mathbf{X} = \\begin{pmatrix}x_1 \\\\ x_2 \\\\ x_3 \\\\ x_4\\end{pmatrix}$.\n",
    "2. Write a transformation matrix $\\mathbf{M}$ such that $\\mathbf{X}\\mathbf{M} = \\mathbf{Y}$ where $\\mathbf{Y}$ has transformed observations as rows. In other words, the following equations needs to hold.\n",
    "$$\\begin{pmatrix}1 & 4 & 6 \\\\ 2 & 4 & 7 \\\\ 1 & 2 & 8 \\\\ 1 & 4 & 9\\end{pmatrix} \\times \\mathbf{M} = \\begin{pmatrix}5 & -2 \\\\ 6 & -3 \\\\ 3 & -6 \\\\ 5 & -5 \\end{pmatrix}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "x1 = np.array([1, 4, 6])\n",
    "x2 = np.array([2, 4, 7])\n",
    "x3 = np.array([1, 2, 8])\n",
    "x4 = np.array([1, 4, 9])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "# TODO: your code goes here..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 5</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Matrix multiplication is also basis for projections that helps to reduce dimensionality of the data. One such projection is known as Principal Component Analysis (PCA) that we will cover in depth later. For now, let's say we have observation $\\mathbf{x_1},\\ldots,\\mathbf{x_4}$ and we want to project them using PCA.\n",
    "1. Stack observation vectors into a matrix $\\mathbf{X}$.\n",
    "2. Center each column of $\\mathbf{X}$ by subtracting its mean. We lable the centered matrix $\\mathbf{X_c}$.\n",
    "3. Compute $\\mathbf{C_X}$ which is a covariation matrix of $\\mathbf{X^T}$, i.e., transposed matrix $\\mathbf{X}$.\n",
    "4. Computer projected observations $\\mathbf{Y}$ given by the formula\n",
    "$$\\mathbf{Y} = \\mathbf{X_c}\\mathbf{P}$$\n",
    "where $\\mathbf{P}$ is a matrix of eigenvectors of $\\mathbf{C_X}$.\n",
    "\n",
    "Expected result for the given observations is $$Y = \\begin{pmatrix}  \\phantom{-}0.61750836 & \\phantom{-}0.10599488 & -0.02394807 & 0. \\\\ -0.48025785 & -0.12818168 & -0.03235508 & 0. \\\\ \\phantom{-}0.22710374 & -0.20107790 & \\phantom{-}0.03341513 & 0. \\\\ -0.36435424 & \\phantom{-}0.22326470 & \\phantom{-}0.02288802 &  0. \\end{pmatrix}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "x1 = np.array([1, 1, 1, 1])\n",
    "x2 = np.array([1, 0.5, 0.9, 0])\n",
    "x3 = np.array([1, 1, 1, 0.5])\n",
    "x4 = np.array([1, 0.3, 1, 0.3])\n",
    "X = np.stack((x1,x2,x3,x4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "# TODO: your code goes here..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "## Pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "`Pandas` is usually imported with alias `pd` to save some characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "The main feature of `Pandas` is a data structure called `DataFrame`. You can think of it as a table with named columns and number rows. Let's create some data frames."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df = pd.DataFrame([[10, \"hey\", 0.3], [20, \"hi\", 0.1],[30, \"hello\", 0.16]], columns=[\"number\", \"greeting\", \"ratio\"], index=[\"first\", \"second\", \"third\"])\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "You can also specify data column-wise. Both styles produce the same result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "pd.DataFrame({\"number\": [10, 20, 30], \"greeting\": [\"hey\", \"hi\", \"hello\"], \"ratio\": [0.3, 0.1, 0.16]}, index=[\"first\", \"second\", \"third\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "You can select a sub-table both by names of columns/rows or by their indices. The data are actually saved as `Numpy` array so you can use the same indexing techniques."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df[\"number\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df[[\"number\", \"ratio\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df.loc[\"first\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df.loc[[\"first\", \"third\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df.loc[[\"first\", \"third\"], [\"greeting\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df.iloc[1:, :2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "You can also query the data frame and get rows that satisfy a defined condition. This can be done either by supplying binary index vector (the first example) or explicitly calling `query` method (the second example)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df[(df.number >= 20) & (df.greeting.str.contains('e'))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df.query('number > (ratio * 100)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "`Pandas` also implements all the basic statistics you might need to know about data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df['number'].median()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Apart from manipulating data in memory, it is also handy for loading and storing data on disk. It implements many standard file formats like `csv`, `hdf`, and `pickle`. Saving and loading a file is as easy as calling a single method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "df.to_csv('example.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Notice that the process of saving and loading the same file does not necessarily result in the identical data frame. In this example, the index of original data frame has become a column. In this very example we need to specify column that will be used as index. This, however, is not a universal solution, you might encounter many problems."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "pd.read_csv(\"example.csv\", index_col=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "You can even specify an URL of file you would like to read."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "weather = pd.read_csv(\"https://www.fi.muni.cz/~xcechak1/IB031/datasets/weather.csv\")\n",
    "weather"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 6</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Let's do some \"data science\"! Compute a mean temperature of days when golf was played. The expected result is 73.0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "# TODO: your code goes here..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 7</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Compute the most common weather outlook when golf was not played. The expected result is sunny."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "# TODO: your code goes here..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "It is really easy to modify and calculate with whole columns. You can also save the result as new column. Notice that you can access columns either like values in dictionaries but also as attributes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-warning\"><b>Exercise 8</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Compute the correlation between columns `windy` and `play`. The expected result is -0.258199."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "# TODO: your code goes here..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Until now, we just took the data as is and computed something on on it or its subset. Sometimes this is not enough and we need to rearrange the data a bit. There are three common operations: grouping, pivoting, and melting. Grouping is useful if you want to aggregate information base on value in a specified column(s). Pivot and melt operations reshapes the data frame. You might need these operations to reshape the data frame for easier plotting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "weather[['outlook', 'humidity']].groupby('outlook').mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "The function applied to the groups can be arbitrary, e.g., take the last row with the given value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "weather.groupby('outlook').apply(lambda group: group.iloc[-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "melted_df = weather.melt(id_vars='play', value_vars=['windy', 'outlook'])\n",
    "melted_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "Collapsed": "false"
   },
   "outputs": [],
   "source": [
    "weather[:5].pivot(index='windy', columns='temperature', values=['play'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "<div class=\"alert alert-block alert-danger\"><b>Exercise 9</b></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "Collapsed": "false"
   },
   "source": [
    "Download and load dataset `attempts.csv` from https://github.com/adaptive-learning/adaptive-learning-research/tree/master/data/robomission-2019-12. Read the instructions on the page and briefly read through data description.\n",
    "\n",
    "1. Find id of the student with the most solved problems. Expected result is 16364.\n",
    "2. Compute average success rate of problems. Expected results starts with 0.870189, 0.973117, 0.866077, 0.362319, 0.658576.\n",
    "3. Add 2 minutes to all time stamps in column `start`.\n",
    "4. Create a table with row for each student and column for each item. The value on row $i$ and column $j$ says whether student $j$ have ever solved problem $j$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}