John Stamos is a loyal man.

During a recent appearance on the Good Guys podcast, the actor, 62, gushed over his fellow Full House alum Lori Loughlin and their longtime friendship. While the pair played a couple, Aunt Becky and Uncle Jesse, on the popular ‘80s and ‘90s sitcom, Stamos explained that he and Loughlin, 61, were never lovers in the real world.

“A lot of people think, ‘Oh, you guys were romantic,’ or whatever,” Stamos said. “You know, I happened to say she was the one that got away in something — I think it was before I met [wife] Caitlin [McHugh] — and, you know, it was an interesting thing.”

“I wrote about in the book, there was a very small window of the two of us being single at the same time,” he continued, referring to his 2023 memoir.

Stamos likened a younger Loughlin to Sandy in Greece “before the leather,” while his ex-wife Rebecca Romijn “was Sandy after the leather and the, ‘Tell me about it, stud.’”

“There was a moment where I had the choice, I think, and I was more into the rebellious,” he admitted. “Lori was so sweet, and I loved working with her, but she was too nice for me.”

On Oct. 3, news broke that the When Hope Calls actress separated from husband of nearly 28 years Mossimo Giannulli, and Stamos shared that he and McHugh have been there to support Loughlin as she navigates the new chapter.

“It’s not a clear thing all the time with something like this,” Stamos noted. “She lives in my neighborhood, Lori, and so, you know, I was over there a lot and she came to our house a couple times and Caitlin’s been good about it.”

Podcast hosts Ben Soffer and Josh Peck then joked that “the biggest question” that fans submitted ahead of the episode was asking if Stamos would “ever leave your wife for” Loughlin.

“No f——- way!” Stamos exclaimed. “I’m not Mormon. Do you do that with your wife? Are you a swinger?”

The Born to Ride star further revealed that although his relationship with Loughlin was “always platonic,” they might have had one intimate moment off-camera when they were both single.

“I always thought that we made out on some ride at Disneyland — I’ve known her since the early ’80s. I remembered, I think she said it didn’t happen,” he quipped. So, I don’t know if it was something I made up in my mind. You know, some kind of fantasy.”

After that, both Stamos and Loughlin entered new relationships. He married Romijn in 1998 before they divorced in 2005, and he went on to marry McHugh in 2018. Meanwhile, Loughlin was married to Michael R. Burns from 1980 to 1996 and has been with Giannulli since 1997 until their separation earlier this month.

“I’m just heartbroken for her right now,” Stamos said. “She’s just devastated. For a girl who has lived her life really well, a good person, a good mother, a good wife — I know all this for a fact — to be thrown into now this separation and, you know, exploding, blowing up her family this way. I just hate to see her go through this. I really do.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

He also didn’t hold back on sharing his opinion on Giannulli, who, along with Loughlin, previously served jail time after pleading guilty to wire and mail fraud in the college admissions scandal.

“I will never talk to him again,” Stamos declared. “He’s a terrible narcissist, and I don’t think you ever get out of that unless you’re able to.”

“I pray for this guy,” he concluded. “I pray that he gets a hold of whatever hole that he’s trying to fill, however he’s trying to fill it. I pray that he realizes that what he had was everything that anyone could ever hope for and that he gets some help. I think this guy needs help.”

When reached by PEOPLE, Giannulli said he had “zero comments” about his split from Loughlin.