Mark Ronson is looking back at an earlier memory involving Michael Jackson.

In Ronson’s memoir Night People: How to Be a DJ in ’90s New York City, the music producer wrote about an early childhood memory with the King of Pop that has since changed following the sex abuse allegations surrounding the late Jackson.

Ronson, 50, reminisced about living blocks away from the Dakota apartments in New York City, which was “always full of fascinating characters.”

The late John Lennon‘s son Sean Lennon lived there. Jackson, who died in 2009 at 50, also stayed there during his Bad tour.

The Grammy-winning producer said that Jackson “wanted to goof around” during his time there and throw “soggies,” which were described as “tightly packed mounds of wet toilet paper,” out his window.

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

Ronson was “fixated on getting a hit song out of him” to share with his stepfather, Foreigner’s Mick Jones. “Michael, Michael, stand still for a sec and sing us a bass line!” he recalled shouting out to him.

The producer eventually got Jackson to hum a few bars of “Smooth Criminal.” The “Thriller” singer, however, seemed more interested in running down the hallway and “shining his laser pointer out the window,” and acted more like a kid than Ronson felt he did.

Ronson never told anyone at school about his interactions with Jackson (even at that age, he knew how odd it would be to name-drop the King of Pop), and wasn’t sure what was more bizarre: “hanging out with Michael Jackson and throwing soaked Charmin balls onto the street or that it felt normal.”

Years later, Ronson reflected on the friendship with Jackson following sexual abuse allegations. “Obviously, with allegations that came later, of course it made me re-examine that event many times,” he told The Sun in an interview published on Monday, Sept. 15.

“I wouldn’t say it’s a highlight of my childhood, but it was certainly one of the most memorable experiences.”

“And of course I put it back through a lens a hundred times. I was like, for whatever reason, there was nothing weird or untoward on that night,” added Ronson.

The sexual abuse allegations made against Jackson date back to the 1990s. They were highlighted in the two-part 2019 HBO documentary Leaving Neverland.  Wade Robson and James Safechuck alleged that Jackson sexually molested them both at separate times when they were children.

Jackson’s estate has spent years publicly defending the late singer against the abuse allegations.

Night People: How to Be a DJ in ’90s New York City is available for purchase wherever books are sold.

If you or someone you know has been a victim of sexual abuse, text “STRENGTH” to the Crisis Text Line at 741-741 to be connected to a certified crisis counselor.