St Helier House will be sold to a private developer.
Parishioners have narrowly voted through the £2.9 million pound deal for the former care home, despite opposition from the Housing Minister.
Senator Sam Mezec had urged ratepayers to block the bid amid concerns the site might be over-developed and the homes sold to foreign buy-to-let-investors.
Town Constable Simon Crowcroft called it 'paranoia'.
"People have assumed on social media that there is going to be tiny, luxury flats created on the site, and that is to completely ignore the fact that we have a planning policy, we have an application process, and I don't believe that Planning would allow something to be built on the site that would produce cramped accommodation."
Constable Crowcroft has said that some of the profit from the sale will be used to relocate Westmount Day Nursery as soon as possible.
The vote taken at last night's Parish Assembly was 48 in favour, 43 against of the offer from property firm Columbia Group.
But a late change to the proposal demands that the parish negotiates a betterment clause to restrict the new owner's future profits from the site.
St Helier House closed at the end of 2017 after it was found to have the same cladding as Grenfell Tower.

Channel Islands Christmas Lottery tickets on sale
Albert Pier improvement works underway
Swimmer found 300 metres offshore
Renewed push for whistleblowing legislation
Islands Unlimited explores inter-islands car ferry
More funded nursery hours 'wonderful for families'
Guernsey's Alex Scott wins first England cap
Ice-rink returns to Weighbridge this winter