By 9pm, Labor candidate John Ormond Kennedy was in front with a stunning 11.1 per cent swing.

Mr Pesutto, the Shadow Attorney-General, was obliged to watch the loss of an unloseable seat from the ABC TV panel, with nowhere to hide.

He delivered live what sounded like a concession speech while he trailed by just 2.5 per cent.

“You get a short time on the stage in politics… it’s a difficult road back. John Kennedy is a very decent man, and I’ve enjoyed being on the campaign trail with him,” Mr Pesutto told ABC.

A former school teacher born and raised in Sydney, Mr Kennedy received 38.9 per cent of the federal vote when he ran against federal treasurer Josh Frydenberg in Kooyong in 2013.

“Once upon a time we were regarded as a bunch of ratbags in these parts,” Mr Kennedy told The Age.

“Now Labor is becoming more and more acceptable in the eastern suburbs."

Mr Pesutto identified new high-density residential buildings as crucial in swaying the outcome.

“I never regarded Hawthorn as a safe seat. There’s a lot of people coming and going,” he said.

To take advantage of those new voters, Mr Kennedy says he attended all 11 days of pre-polling, crucial in a seat where approximately one quarter of residents voted early.

“Throughout the campaign I certainly thought people were really cynical about law and order. Labor seemed to be more substantial, and Andrews has done a good job,” he added.

Cautious optimism in Caulfield

Labor has refused to declare victory in the blue-ribbon seat of Caulfield, held by Shadow Energy Spokesperson David Southwick, despite holding a commanding 3.5 per cent lead with almost 50 per cent counted.

Never held by Labor in the seat's history, the Labor candidate Sorina Grasso has seen an almost 4 per cent primary swing towards her.

Addressing Labor's Caulfield rank-and-file, a largely Jewish demographic, Ms Grasso said "it's looking really, really good".

"It's been a wild ride," she said. "Thank you all for accepting and supporting me."

Ms Grasso reserved special thanks to the now-federal candidate for McNamara Josh Burns, who in 2014 as the candidate for Caulfield achieved a significant swing to Labor to bring it within the party's reach.

She was cautiously optimistic of snatching the seat, noting the need to wait until pre-poll votes had been counted.

Caulfield scrutineers said the count would be close and was likely to go late into the night as postal votes were counted.

Battle ahead in Brighton

Meanwhile, in Brighton, never held by Labor in the seat's history and held by retiring MP Louise Asher on the solid margin of 9.8 per cent, Labor candidate Declan Martin was startled to enjoy an 11 per cent primary swing with 40 per cent of the vote counted.

'This is a momentous moment for politics in Brighton," said Mr Martin, a 19-year-old urban planning student who joined the party two months ago.

He remained cautious and flagged the possibility of postal ballots breaking towards the Liberals.

"Obviously it's too early to call, but I'm incredibly proud of what's been achieved."

Liberal candidate James Newbury did not wish to comment when contacted by phone just before 10pm.

"I do not want to talk," he said. "Ring me tomorrow," he repeated several times.

"I'm not picking up the phone before 9am tomorrow."