Is the SSL configured for the Office365 end? You will need to have this resolved at both ends to allow the token to verify and authenticate - this could be the cause of the problem.
The AD FS token received is not valid until: 05/08/2013 13:30:27 - This could mean the token received from Microsoft isn't verified with the on-prem version and as such cannot communicate. You are federating so both ends must have their tokens in order to efficiently communicate.