A driver offering 'special temple tour' for cheap is the gem-shop scam, just like in Bangkok. Walk away.
Use Grab or hired tour vehicles. Don't accept tour offers from drivers at hotels or tourist sites.
If you do want a guided temple tour, book through your hotel or Klook — pricier but legitimate.