Just to clarify, the oil doesn't have to be cold...like cold from the fridge. Room temp oil is fine
And, when the fries get brown..or golden...I usually gently stir them around a bit. You'll be able to tell if they are fried enough to move safely without breaking them up.