Josh Duhamel’s Mother’s Day Plans for Fergie Will Be ‘Low-Key’: ‘We’re Just Going to Be a Family’

05/07/2016 at 05:15 PM ET

Josh Duhamel and Fergie know how to enjoy a night out, but this Mother’s Day, it’s going to be a cozy night in.

“Ferg is right in the middle of shooting a video, so it’s going to be a pretty low-key Mother’s Day,” Duhamel, 43, told PEOPLE while attending the Kentucky Derby Saturday with G.H. Mumm champagne.

Between both of their schedules, the actor says he and his pop star wife have been “crazy busy” for the last couple of weeks and are looking forward to enjoying some quiet family time with son Axl, 2.

“I [asked her], what do you want to do? We got the whole day off,” said Duhamel, 43. “She said: ‘I just want to order some food in, and be with you and Axl.’ We’re just going to sit at home and just be a family.”

“It’s going to be nice just to have a very low-key, at home, Mother’s Day,” Duhamel said. “We’ll probably order in from someplace fancy, but that’s about as extravagant as we’ll get.”

Josh Duhamel and Fergie
JB Lacroix/WireImage

Want all the latest pregnancy and birth announcements, plus celebrity mom blogs? Click here to get those and more in the PEOPLE Babies newsletter.

Of Axl, who will turn 3 in August, the actor marveled at how special it’s been to watch his son grow up before his very eyes.

“It’s pretty amazing to watch a newborn baby turn into an almost-3-year-old,” he said. “It’s like, wow. You hear everybody talking about that but to see it — and everyone says it just keeps getting better and better, and it truly does. Every day he’s blossoming right in front of us. It’s really beautiful. He’s a happy kid, too. He’s a happy boy and that’s really what we want.”

My son thinks I can actually fly an F16. Let's just go with that for a while…

A photo posted by Josh Duhamel (@joshduhamel) on

And hopefully, Axl will be growing up with a little brother or sister by his side at some point.

“That’s the plan,” Duhamel said of expanding his family with Fergie. “It all depends on when we get a window of opportunity, you know. And it’ll happen. It’s definitely been our plan.”

Congrats to UND Hockey Team for making the Frozen Four!!!!

A photo posted by Josh Duhamel (@joshduhamel) on

In the meantime, Fergie, 41, is busy working on her upcoming album, while Duhamel is enjoying the big race in Louisville, Kentucky, for the first time in person.

“It’s crazy here,” he said. “It’s just endless, endless people everywhere you look, all dressed up to the nines, giant hats and flower dresses and dudes in suits.”

“I watch it every year on TV, and to see it in person and to see all the extravagant outfits and hats, all the pomp and circumstance that goes with it, is quite a sight to see,” he added.

And while he’ll “put a little money on a horse,” what he’s most excited about is all the people-watching: “It’s as much about people-watching as it is about watching the horses race! That’s my favorite pastime, people-watching.”

As for whether they might make it a family trip next year? Duhamel says he “would love to take [Fergie and Axl] to see this.”

“Ferg is jealous that I got to come and see it while she was working, but I told her next year we’ll come,” he said. “I think my boy would love it, too.”

— Aurelie Corinthios

Share this story:

Your reaction:

Add A Comment

PEOPLE.com reserves the right to remove comments at their discretion.

Showing 6 comments

Ilona on

Josh is the father and the mother in this familly !!! Fergie seems no maternal !!!

Lyn on

What is with the Josh D media blitz? He has a commercial for North Dakota Tourism. Big deal. He’s an example of just how boring North Dakota is, and yes I’ve been there. In other BIG news Katherine Heigl is promoting cat litter. WTH?

Gawd on

IIona, ok mother of the year. Fergie is a mother, more than Kimkow is a mother to her Petri dish accessories…grow up

Anonymous on

Josh is an awesome guy!

sam on

He looks like a hands-on father, not sure about Fergie, haven’t seen many pictures of her with her son.

West on

I agree with llona , Josh is the most interested parent . He loves being a dad . You can just see it . Fergi on the other hand seems like having a child is more like having an accessory. That is the impression I get at least .

From Our Partners

Sign up for our daily newsletter and other special offers.
    Choose your newsletters
Thank you for signing up! Your request may take up to one week to be processed.
    see all newsletters