Simon opined that the fisherman was wearing the helmet rather than risk theft of it off his motorbike.
In remote locations, there may be reason for that, but in most city locations, theft of helmets stored on motorbikes does not seem to be a problem (at least yet). But I do notice people wearing them in places where they are in transit between locations and need to keep their helmet in hand. For example, if you are going to take a bus trip, you probably need to take your helmet with you so you are prepared for the inevitable motorbike rides at your destination. Rather than carry the helmets, many people deem it easier just to wear them.
I am always anxious to get the helmet off as soon as we stop the bike, since they are hot and uncomfortable without the wind going by when on the road. Many Vietnamese seem to have adapted to helmet use quite well, however, and maybe even forget to take them off.
Most Vietnamese (both male and female) have always worn hats when outside to provide a little protection from the sun. Now that helmet use is mandated, one doesn't carry around both a hat and a helmet, so one is left with the helmet to wear out in the sun rather than a more traditional hat. I carry around a crushable hat in my pack to cover this situation.