Logistic Regression Gradient Descent

Recap:

$z=w^Tx+b$

$\hat{y}=a=\sigma{(z)}$ - prediction

$L(a,y)=-(y\log(a)+(1-y)log(1-a))$ - loss of one example

In logistic regression, we want to modify the parameters, W and B, in order to reduce this loss.

$\frac{dL(a,y)}{da} = -\frac{y}{a}+\frac{1-y}{1-a}$

in Python we denote it as $da$ .

$dz=\frac{dL}{dz}=\frac{dL(a,y)}{dz}= a-y$ $= \frac{dL}{da}*\frac{da}{dz}$

where

$\frac{da}{dz}=a(1-a)$

$\frac{dL}{dw_1}="dw_1"=x_1*dz$

$\frac{dL}{dw_2}="dw_2"=x_2*dz$

$\frac{dL}{db}="db"=dz=a-y$

If you want to do gradient descent with respect to just this one example, what you would do is the following;