Prime Minister Narendra Modi started meditating at the Vivekananda Rock Memorial in Kanniyakumari on Friday, where Swami Vivekananda once meditated.