Caldwell stepped in on a short-term basis last month following the departure of Chris Hogg, and leaves with the recruitment process to appoint Hogg's full-time successor nearing completion.
Current assistant Kevin Richardson will take the team for its upcoming fixtures ahead of a permanent appointment being announced shortly.
The club would like to thank Gary for his contribution and wishes him well for the future.