honestly, picking up a game after a long break is sort of a separate little game to me..^^ i have a very bad memory, but when i read the vn, i kinda start piecing together the bits of lore/story characters drop in their dialogues, and its kinda fun! but maybe its too weird ww.. i do the same with manga btw!
i dunno, its just a lot of fun for me to encounter something and feel it pop up in my head again as a faded memory.. of course it doesnt replace the actual smooth 0 to 100 experience, but as long as it works!
so ultimately, i think it depends on how "vaguely" you remember the plot. if you spend 20 mins thinking "who the hell are all these people ive never heard these names in my life" when coming back, then it would probably be for the better to start from the beginning..! but as woodypei mentioned, you can try to read reviews online to see if jogging your memory helps!!:)