I had someone do this on my FR a long time ago. It didn't bother me, but then again, only happened once. I was more WTF than anything. I would find it rude as well if it happened multiple times!

I guess the old adage 'it doesn't hurt to ask' is sorta iffy here, huh? :}