‘Dravidian parties were chief destroyers of temples’