# Wk 1

## Regression and classification

$$y\_i \in \mathbb{R}$$ -- regression task

* salary prediction
* movie rating prediction

$$y\_i$$ belongs to a finite set -- classification task

* object recognition
* topic classification

### Linear model for regression

$$a(x) = b + w\_1x\_1 + w\_2x\_2 + \dots + w\_dx\_d$$

* $$w\_1, \dots , w\_d$$-- coefficients (weights)
* $$b$$ -- bias
* $$d$$ + 1 parameters
* to make it simple: there is always a constant feature

Vector notation:

$$
a(x) = w^T x
$$

For a sample $$X$$:

$$
a(X) = Xw \\

X = \begin{pmatrix}x\_{11} \cdots x\_{1d} \ \vdots \ddots \vdots \ x\_{n1} \cdots x\_{nd}\end{pmatrix}
$$

### Loss function

How to measure model quality?

$$
\text{Mean squared error:} \\
L(w) = \frac{1}{n}\sum^n\_{i=1}(w^Tx\_i - y\_i)^2 \\
\= \frac{1}{n}\Vert Xw - y\Vert^2
$$

### Training a model

Fitting a model to training data:

$$
L(w) = \frac{1}{n} \Vert Xw - y \Vert^2 \rightarrow \underset{w}min
$$

Exact solution:

$$
w = (X^TX)^{-1} X^Ty
$$

**But inverting a matrix is hard for high-dimensional data!**

### Gradient Descent

![](/files/-LptsPK4YK72RdznITSs)

![](/files/-LptsXrnSbkaLCymRKov)

![](/files/-LpttDLDseZJbjYpCAJQ)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://stephanosterburg.gitbook.io/scrapbook/coding/advanced-machine-learning/wk-1.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.