AMC 12 Mock Exam 1 Problem 18

Place $A=(0,0)$, $B=(4,0)$, $C=(4,4)$, $D=(0,4)$. By symmetry the center of $\omega_1$ lies on the perpendicular bisector $x=2$ of $\overline{AB}$, and tangency to $\overline{CD}$ (the line $y=4$) means its radius is $\rho=4-k$ if its center is $(2,k)$. Passing through $A$ gives $2^2+k^2=(4-k)^2$, so $4=16-8k$, $k=\frac32$, and $\rho=\frac52$. (It touches $\overline{CD}$ at its midpoint $(2,4)$.)

A circle of radius $r$ inside the square tangent to $\overline{BC}$ ($x=4$) and $\overline{CD}$ ($y=4$) has center $(4-r,4-r)$. External tangency to $\omega_1$ requires the distance between centers to equal $\frac52+r$: $$\left(2-r\right)^2+\left(\tfrac52-r\right)^2=\left(\tfrac52+r\right)^2 .$$ The right side minus the second term on the left is $10r$, so $(2-r)^2=10r$, i.e., $r^2-14r+4=0$, giving $r=7\pm3\sqrt5$. Only $r=7-3\sqrt5\approx0.29$ fits inside the square, so the radius is $\boxed{7-3\sqrt5}$.