It has something to do with maintaining the timeline. My understanding is that the future affects the past just as the past affects the future. That the future dictates that we have attended this party, for us to have not attended would have caused a paradox.

Which isn't to say our host doesn't have another agenda for inviting us here.