Can my users have a single sign on experience when using Office 365? I.e. not have to enter a username and password for SharePoint when in the office?

Yes, but only on the Enterprise plans E1-3, by configuring ADFS integration.