That was one of the discussions coming from the Public Sector CIO Summit last month. Microsoft hosts the event, bringing together public sector employees from the federal, state and local governments.

Microsoft is investing $9.5 billion in research, with the aim to be more nimble for customers, said Curt Kolcun, the vice president for Microsoft's Public Sector Business.

The trends emerging in the commercial marketplace are the use of hybrids and pay-as-you-go models, Kolcun said.

The Office of Management and Budget's cloud-first policy directs agencies to move three technologies off their network and onto the Internet over the next 18 months.

"In this rapid push, because of the cost constraints, customers need to think about, What are my regulatory and statutory requirements about data sovereignty?" Kolcun said.

Cost will be a key factor in agencies' decision for a cloud provider, but so too will be the longevity of the service. Kolcun said Microsoft allows more options for "differentiation" opposed to an "all or nothing" choice.