Minors and cofactors

You know $\begin{vmatrix} a & b \\ c & d \end{vmatrix} = ad - bc$. Cover the row and column of $a$: only $d$ is left. Cover the row and column of $b$: only $c$ is left. So why does $b$ get a minus sign and $a$ a plus?