Rory Stewart \'would feel comfortable voting with Labour\'