Justin Bieber is back in his element. The 26-year-old singer gave not one but two unforgettable performances on Saturday Night Live on Oct. 17. Kicking off his first number, the Canadian star belted out the lyrics to his “Holy” track. He also brought out Chance the Rapper, who is featured on the single. For his set-up, Justin performed below a neon cross, which perfectly complemented the tune’s references about faith and Christianity. His follow-up performance was just as moving. Justin debuted his newest tune, “Lonely,” which he dropped on Friday, Oct. 16. The pop star skipped the typical SNL stage set and opted to croon backstage in a dressing room. After looking at his reflection in a mirror, he slowly got up and walked through the dimly lit halls of Studio 8H. This marked Justin’s fourth time onstage, however, it was unlike any other appearance. To celebrate the special occasion, the “Intentions” singer enjoyed a night out with his wife, Hailey Bieber, Chance the Rapper and his wife, Kirsten Corley. A source tells E! News the group went to the New York City hot spot Catch Steak after the long-running comedy sketch show wrapped up. “Justin and Hailey arrived at Catch Steak and went straight upstairs. Chance and his wife were there, too,” the source shares, adding that they partied all throughout the night until early in the morning. The source believes Justin and Hailey “flew straight out of town” after their hangout. Just last month, the couple celebrated their second wedding anniversary in sweet style. At the time, a source told E! News they had a “mellow and fun” hangout. “They had a relaxing morning and then decided to pick up sandwiches from Bay Cities Deli in Santa Monica and have a picnic at their house,” the source explained. “They loved relaxing and hanging out together just them two. They talked about their wedding day and vows and had an emotional moment thinking back on the last couple of years.” The source added, “They have both been busy and it was nice for them to relax together. They always have fun no matter what they are doing.” From their late-night out on Saturday, it’s clear they do have fun no matter the occasion.