But the Scandal star says he wouldn’t want it any other way.
“Fatherhood is insane,” Foley, 42, told PEOPLE at the 46th NAACP Awards on Friday in Pasadena, California. “I’m lucky enough to have three healthy children. And I’m lucky enough to be woken up every day between 3 a.m. and 5 a.m.”
Surprising, the culprit isn’t 3-month-old son Konrad: It’s his big brother, Keller.
“My 2½-year-old, if he doesn’t ‘drop’ anything at 3 a.m., it’s a good night,” Foley said, laughing.