Yes, Hungary is quite south of Europe. But it can get really cold in Winter. You don't want to ruin your trip needing to look for clothes when you are already there.

It is no better time to travel than during the Summer season, however, it still rains many places during the summer and it is important to pack warm clothes and not only your swimsuit.