Was in Sri Lanka on holiday last year, it's a fantastic country.
We toured for the first week with a car and driver, staying in hotels, so I couldn't comment on the likes of apartments etc in Colombo, but what I would say is that you should rely on a local to drive you rather than taking on the challenge yourself. Let's just say that the driving style is somewhat different to what we're used to!
Public transport is very cheap, we went on the train from our beach resort in Bentota to Galle on the south coast for about a pound each.
We never felt particularly unsafe anywhere, you just have to be a aware that when you're out and about everyone will want to take you to their uncle's shop or something for a few rupees.
As far as things to see, the ancient cities in the cultural triangle are fascinating, along with the many sacred Buddhist sites. The scenery is also fairly spectacular in the tea growing hill country, but remember to take a jacket or jumper, as it is significantly cooler up there.
Let me know if I can help any further.