Leisure seems to have changed.
During previous eras, I don't know that there was much in the way of leisure, unless you had some royal something going for you. Even then, I have a feeling there were things you "had" to do. Fun was really not relaxing in the truest sense, you could ride a horse, but as a rule, it was to get from point A to point B, not because horse riding was "fun" per se.
Then we started to progress. We invented machines. This gave us time! Especially for women, there was sometimes a bit of time (depending on status) where a bit of something that wasn't a "have to" could be accomplished. Needlework, like cross stitch and embroidery became popular, especially when "recipe" or "receipt" books with patterns came out. Knitting was a popular one, as well.
World Wars 1 and 2 had people making things for the war effort, fun was also practical. You could knit, but you were knitting for the 'boys at the front'. Get togethers were often for war bond collection, or scrap metal drives. And because of rationing, you didn't just hop in a rig and go for a drive.
After the wars, the country came into prosperity. Cars, parties, what have you were more common. People could "do" things for fun.
Today, I have noticed puttering takes a lot of forms. Some of my friends bike, some do crafts, or garden, some cook, some play with angry or flappy birds, some read, electronically, or not, and some just seem to lay back and watch the world go by.
I think I shall endeavor to savor "goofing off" as much as I can. Who's with me?