Speaking on the Stick to Football podcast, Neville said: "I took the Valencia job purely because I knew the owner and I felt like I owed him something and I was loyal to him, he believed in me in terms of the business.
"I've been asked by Middlesbrough for an interview, Newcastle and Derby asked to speak to me, all in the previous three years I had been on Sky. I said no to every offer because it wasn't what I wanted to be."
Under Neville, Valencia won just three of 16 league games and suffered a humiliating 7-0 Copa del Rey defeat at Barcelona. He was dismissed with the club 14th in La Liga, just six points above the relegation zone.
Neville, 49, had previously served as assistant manager to Roy Hodgson with England, joined Sky Sports during the 2011/12 season when his playing career was curtailed by injury. And he said it was position with the broadcaster that could have paved the way for him to head to the north-east.
"I had started my businesses and my media career," he added. "So I didn't want to be a coach and I had said no to Newcastle, which came from someone at Sky who had a contact at Newcastle as they wanted to speak to me, which was the same with Middlesbrough. I'm not saying that would have got me the job but it could have."
Neville is currently a co-owner at League Two side Salford City but since leaving Valencia, has ruled out a return to management, saying the experience left him 'scarred'. And he also opened up on how he reluctantly took the role with the La Liga club, which came via his relationship with Peter Lim who had become a shareholder at Salford.
Recalling his intentions post-retirement, Neville said: "I didn't want to be a manager and I didn't want to be a coach, and I said no to Peter Lim (Valencia CF owner) the first time.
"But he said that he really wanted me to do it. I signed on a five-month contract, and I didn't last that time because it became impossible for Peter, for me and for everyone."
Largest Football Social Network