Thats shocking. I'd probably have walked away as well. You've not done anything techincally wrong by taking is photograph but it doesn't matter whether or not he knows that.
If someone wants to be a pain in the ar*e then they'll be a pain in the ar*e!
You could have stood your ground, and you'd have been (legally) in the right, but I expect it would all end in tears. More passers by would get involved, then a PCSO or "proper copper" would probably get involved at some point and then massively over step their legal authority by asking you what your name is or why you're taking photos.
I think you took the sensible option - which is a very very sad reflection on our society.
[/rant]
