Speaking from a Swedes perspective, i.e Swedish sign language.
Grammar can be very tricky to get your mind around. Then there are the tiny differences between signs in quick succession by fluent speakers that can really throw you off. (Think morse code at different speeds)
I remember a test I had for my college course.
A guy was telling us about a pillow fight he and his friends had.
If you missed the sign for 'pillow', many of us did, you had no idea what was going on. We all had very different interpretations of what he told us.