2007 AMC 10B Problem 21

Let $s$ be the side of the square and $h$ the altitude from $B$ to $AC.$ Then $h=\dfrac{AB\cdot BC}{AC}=\dfrac{3\cdot 4}{5}=\dfrac{12}{5}.$ The small triangle above the square is similar to $\triangle ABC$ with the square’s top side as its base, giving $\dfrac{h-s}{h}=\dfrac{s}{AC},$ so $s=\dfrac{AC\cdot h}{AC+h}.$ Substituting, $s=\dfrac{5\cdot\frac{12}{5}}{5+\frac{12}{5}}=\dfrac{12}{\frac{37}{5}}=\dfrac{60}{37}.$ Thus, the correct answer is B.