Hybrid Cloud obstacles when prototyping with Power BI

It’s hard to believe in this day and age of a highly competitive Business Intelligence market that Microsoft didn’t simplify Power BI’s cloud infrastructure requirement to connect directly to an on-premise database.  I wish the process was more straight forward; however, lesson learned I wanted to share my experience.

Read more: Hybrid Cloud obstacles when prototyping with Power BI

Data Integration in the Cloud, History Repeats Itself

As quickly as organizations began moving critical functions to the cloud, software vendors responded with the ability to access their cloud data for the promise of analytics.  The Rest API made such data access to these cloud services super easy and the potential is amazing.  Despite these advances an irony exists.  The biggest challenge organizations experienced with analyzing disparate internal system data in prior decades has remained with today's cloud service data.   Getting the data is one thing: integrating and harmonizing the different data sources across multiple dimensions of business is another.  The tools and service providers are available to face this challenge.  Decisions on where to go amidst the myriad of ISVs and providers and how to do it without the risk of a failed project is overwhelming many who do analytics in the cloud.

Read more: Data Integration in the Cloud, History Repeats Itself

Analytics as a Service

Running an analytics practice can be highly variable in terms of human and capital resource demands. This is often compounded by the scarcity of highly skilled resources in the space (Statisticians, Data Architects, BI Architects, Data Modelers, Data Scientists, etc.).  The challenge escalates as data volumes grow into the “Big” range.  Traditionally the demand for such variable analytics skills and resources is acquired on a project-by-project basis according to capital spending and all of the budgeting and process surrounding capital.  The unfortunate truth of capital spending is that while it may save on taxes it greatly slows the pace of innovation and in the case of analytics an organization's access to information.

The cloud has taught us that subscribing to software and infrastructure as a variable expense can dramatically speed IT turnaround times to near real-time proportions.  The same too is possible with scarce skillsets in analytics.  Imagine an operational paradigm where analytics, data integration and system architecture expertise was available together with the technology those experts require on a part-time and scalable subscription basis: Analytics-as-a-Service (AaaS).  While you lose the tax savings from capital funded projects you may gain months, even years, in your access to information from closing the gaps created from capital budgeting and recruiting skillsets.

One of the great breakthroughs with AaaS comes from empowering data experts to be agile and solve problems without encumbering them with IT infrastructure project demands.  Consider the scenario of an analytic question that requires a new data source and thus an unknown amount of additional disk space and hours of computing time to answer.  A good AaaS subscription would provide you real-time access to the expertise to identify the exact capacity and technology gap to answer the question.  By leveraging the cloud your analytics provider would stand up the extra capacity and any firewall rules needed for answering that question that same day.  No special project or prolonged wait for expertise and infrastructure would be necessary.

Analytics as a Service  can take on many flavors.  Read our subsequent posts for a few examples.


Staff Augmentation

Before the cloud there was staff augmentation which remains and will likely continue to remain the most widely used form of AaaS.  By contracting consulting experts customers have been able to solve those tougher problems that don't fit into a cube, pivot table or BI report using their existing infrastructure. 

One technology evolution for staff augmentation that parallels the arrival of the cloud is the onset of web meeting software, fast internet and VPNs.  These technologies have opened up the potential for contractors to work remote.  Customers that trust this model and their contractors can potentially save on costs by engaging multiple skill sets from a larger consulting firm on a part-time basis.  The technology advancement allows the consulting firm to support multiple concurrent customers per resource without overhead incurred from putting people onsite.

Analytics Infrastructure Outsourcing, Trusted Partner

The trusted partner outsourcing model is where your analytics consulting firm will operate a cloud account for your organization on your behalf.  Your account would be with a major cloud platform vendor like AWS, Rackspace, Azure or Google Cloud.  Within this account you’d have a fixed monthly budget for infrastructure and software which your trusted partner would have full discretion to use as part of your subscription with them to solve analytic problems.  Your consulting partner may have other customers however your account’s infrastructure and technology would be isolated and managed independently from those other customers.  You may choose to limit security access to your cloud account to one or more dedicated consultants within your partner firm to prevent misuse.

Using the Redshift/Vertica example from our prior post, your account would likely include its own datawarehouse platform that operates out of your own budget.  However, if your consulting partner decides they need a Hadoop cluster temporarily to answer an analytics question, budget should be present to afford the temporary capacity increase.  The customer involvement in technology decisions to answer a question is purely optional and the discretion of the customer.