The real reason Robbie Williams and Ayda Field are ‘gutted’ to QUIT The X-Factor after just one series