Leading 2-1, the Spireites conceded deep into six minutes of added time when Michael Cheek scored his second goal of the game to end the Blues’ 100% start to the campaign.
Cheek gave the Ravens the lead in the first-half against the run of play from the penalty spot but George Carline struck an instant equaliser.
Manny Oyeleke deservedly put the Blues in front with less than 20 minutes remaining before Cheek pinched his second.
Rowe said: "You have got to kill the game off and if you don’t then you can be punished from the slightest mistake and obviously we have made a mistake for their second goal.
Chesterfield set to announce signing of new goalkeeper ahead of Aldershot Town clash
Latest National League transfers as Scunthorpe United, Southend United and York City all do business
'Active at both ends of market' - Chesterfield boss Paul Cook on latest transfer activity at Spireites
Chesterfield striker could be loaned out to improve fitness after long time out
Chesterfield attempt to get defender's ban overturned before Aldershot Town match
"I am shocked because I did not see that goal coming. It was a mistake, it happens, but we can't vilify the player (Calvin Miller) for diving in because we can deal with the cross better as well.
“It is unlucky, it happens, but what you can't do is let the adversity rot in. I am sure with the character we have got we won’t that won’t happen.”
Rowe felt Chesterfield should have been awarded a penalty when Jack Clarke went down under a challenge from visiting goalkeeper Mark Cousins in the first-half and said the decision for Bromley’s spot-kick was ‘harsh.’
He was also frustrated at the decision to play six minutes of added time.
"The referee has been here twice, that is two penalties he has given against us, and two potential penalties for us that he has not given, that is disappointing.
“We played six minutes added on. Normally when there is that amount added on there is a major incident that has happened and I don't think there was one.”
Miller was excellent throughout, including creating the goal for Oyeleke, but he was at fault for Bromley’s equaliser.
“For all the great stuff he did, that moment cost us,” Rowe said.
Despite the result it was a dominant performance from Town.
“I think that is the best we have played from home for quite some time,” Rowe added.
"Their keeper has pulled off two excellent saves but I think we weren't ruthless enough.”