I mean really what are the odds that you would do the exactly perfect thing to ensure the ideal future? A billion to one?

Good point! What if foreknowledge made it harder to engineer a desired outcome? Either by trying to achieve it and failing because you know what you have to do? Or you've gone around the bend due to pressure of what you have to do and that you are changing the already occurred future? Or a condition of the desired outcome being achieved is that you cannot have that prior knowledge?