Blog Categories

Subscribe to RSS feed


Archive for the ‘Virtualization’ Category

Pervasive Data in Microsoft’s Cloud OS

As early as the beginning of this year, Microsoft began positioning their “Cloud OS” concept, based on the pillars of Windows Server and Microsoft (formerly Windows) Azure.  This perspective on the spectrum of Microsoft’s offerings casts Cloud OS as a giant next generation operating system, with all the scope and scalability that cloud computing offers.

Pervasive Data in Microsoft's Cloud OSComplementary to Windows Server and Microsoft Azure, additional supporting technologies provide services like identity management (Active Directory and Azure AD), built-in virtualization capability (Hyper-V), and consolidated management capability (System Center).  Basically, it’s a bundle of products that can be mostly seamlessly joined to provide a completely cloud-based computing environment.  Microsoft technologies are increasingly being tailored to help flesh out the Cloud OS story, and they demonstrate Microsoft’s pivot towards the “platforms and services” line.

But another critical part of the Cloud OS story is data, and that’s where SQL Server comes in.  SQL Server 2014 — running on-premises in your datacenter, in the cloud on Azure VMs, or on both — is your modern organization’s data backbone.  As part of the Cloud OS story, SQL Server 2014 is a bridge between on-premise and (Azure) cloud-based data assets.  Advanced integration with Microsoft Azure allows SQL Server to support next-generation Hybrid architectures for backups, DR, Replication and data storage/delivery.

Schedule automatic offsite backups.  Pin a frequently used Data Mart in-memory for maximum query performance.  Leverage your Big Data assets against SQL Server data with an Azure HDInsight cluster.  Refresh Power BI content from a Data Warehouse hosted on an Azure VM.  All from within the cost-effective environment of the Cloud.

So what does the Cloud OS story mean to users?  It means that what we’re seeing now in terms of shared experiences across our tablets, smartphones, and TVs is just the beginning.  As the era of Big Data continues to dawn, we will all be floating on a sea of data.  And the cloud is where that sea will reside.

The Cloud OS as a whole ultimately empowers consumers and users with computing capability over sea of data — wherever they are, wherever they want it.   In terms of data, this moves toward the larger goal of giving business the ability to identify, gather, and use data from an endless variety of internal and external sources to unlock business insights, and turn that information into action.

That’s the idea of pervasive data.  And in Microsoft’s Cloud OS story, it’s empowered by self-service BI from Office365 and SharePoint Online, using SQL Server and Azure technologies under the covers but all accessed through an interface as familiar and comfortable as Excel.    And it’s coming soon to a device near you…

Strengthen Company Culture with Yammer enhanced by HDInsight

In a world of broadband internet connections, online collaboration tools and the ability to work from almost anywhere – office culture can be difficult to sustain.  This especially holds true for people who live in large cities (where the commute can be problematic) or in harsh climates (like the never ending winter in Chicago this year).   Yammer can help by creating remote social interactions.

Strengthen Company Culture with Yammer enhanced by HDInsightYammer is an enterprise social network that aims to connect people in the office.  A few of its features are instant messaging, user profiles, a primary news-feed, interest groups, recommendations for people to follow, groups to join as well and a recent activity feed.  The interface is clean and well designed.  One of the great things is that once you start using Yammer it is really easy to continue.

There is one area where Yammer seems to fall short.  There is no clear way to bring people together who have common interests.  The other users and groups that are recommended to me by Yammer are made based on the groups I am a part of and people I follow.  It does not take into consideration any of the data in my user profile.

Perficient recently held a hack-a-thon where my team identified this short coming.  Social interaction via online collaboration tools wasn’t cutting it.  In an online culture how can we leverage all of our tools to help facilitate more meaningful social gatherings?  The answer was to use interest data that co-workers have provided through Yammer to generate meaningful recommendations.  A Yammer profile consists of many different “interest groups”.  It lists categories such as Expertise, Interests, Previous Company and Schools Attended.  All of these can be classified as conversation topics and can be used as a common social interest.

This is where HDInsight powered by Hadoop and Mahout can help.  Mahout can consume massive quantities of information and return logical connections represented within the data.  For additional reading about Hadoop and Mahout click here.

Using an HDInsight Hadoop cluster in coordination with the Mahout recommendation engine we could provide meaningful recommendations to users based on their individual interests.  This wouldn’t just recommend topics that a user might be interested in but also groups they could create or join with other users based on their mutual interests – similar to the recommendations Facebook suggests regarding people you may know, groups to join or pages you may like.

Creating these logical, online groups would “connect the dots” to uncover a similarity between people where it might otherwise remain hidden.  It could also help facilitate in-person group outings, social gatherings or simply more friends and comraderie in the office.  Through this you are creating a more meaningful environment aided by technology.

A thriving office culture can stand out in a world where telecommuting tends to be more convenient.  This may not convince everyone to come to the office. However, instead of viewing it as obligatory, implementing a solution like this can encourage more people to choose to commute to the office for the social comraderie.  All of this can be done for free through the Yammer API and a Windows Azure account.

Windows Azure: Retiring Windows Server 2008 and how to upgrade

Beginning on June 2, 2014 Windows Azure will be retiring Windows Server 2008.  This means that you will no longer be able to deploy a new Cloud Service or manage your existing services on virtual machines running Windows Server 2008.

Windows Azure: Retiring Windows Server and how to UpgradeWindows Azure currently supports four different GuestOS ‘versions’:

  • GuestOS 1.x – Windows Server 2008
  • GuestOS 2.x – Windows Server 2008 R2
  • GuestOS 3.x – Windows Server 2012
  • GuestOS 4.x – Windows Server 2012 R2

If your Cloud Service has not been upgraded and is still running on Windows Server 2008 you must upgrade the servers that power your service.  How do you do that?  Isn’t the point of a running a PaaS cloud service instead of using IaaS to handle the operating system and hardware for me?  The short answer is yes, but…

PaaS will take care of much of the hardware, IIS patches and OS patches for you but Azure will not do entire OS upgrades for your entire service unless you tell it to.  This happens because incompatibilities between cloud services and operating systems are likely to arise.  This would cause developers to try and fix code on the fly.  That is not only bad for up time but could also come with some very serious security holes.

Thankfully, living in a world where you have to manually upgrade the server OS for your service is in the past.  Azure makes it easy to upgrade the guest OS for your service.  You can even have your production service remain on Windows Server 2008 while upgrading your staging environment and deploying your service there.  This will allow developers to fix any outstanding bugs that are introduced with the operating system upgrade.

How do you upgrade your staging environment?  It is pretty straight forward.  From the cloud service dashboard select your staging environment and choose Configure.  At the bottom of the page find the operating system section.  You will see drop down menus for OS Family and OS Version.  Select proper OS Family (in this case anything but 1.x) and OS Version.  To always have the most up to date OS Version select automatic.  This ensures your cloud service will always be running on the latest Azure VM that is available.  If you do not want this select a static version of an OS.  This guarantees that your cloud service will remain running this OS until you upgrade it in the future.

When the service is cleared for production it is time to configure your production environment.  Upgrading your production environment can lead to some downtime for your service, but there is a way to avoid it.  Normally you will need to configure your staging and production environment independently but now you can swap your staging and production environments using the Swap option in the portal.  This will effectively swap your staging environment into production.  The change will happen within seconds and any downtime experienced will be minimal.

After the swap you can rebuild and configure the former production environment, which is now your staging environment to match that of your current production environment.

New Azure VMs improve SQL Server Data Warehousing in the cloud

While poking around in Azure, looking to set up a BI Demo VM , I noticed that Microsoft had added a few SQL Server-oriented images to their catalog.   VMs labeled “SQL Server… for Data Warehousing”!

There was one for SQL Server 2012 (SQL Server 2012 SP1 for Data Warehousing on WS 2012) and one for the current CTP2 version of SQL Server 2014  (SQL Server 2014 CTP2 Evaluation for Data Warehousing on WS 2012)!

My curiosity piqued, I ran (figuratively) to the Bings, and sure enough!  There it was:  confirmation, including some nice guidelines on configuration of VMs for DW purposes.

My favorite factoids:

  • Use an A6 VM for the SQL 2012 image, an A7 for 2014.   (This was well-timed for me because I was about to put the 2014 on an A6…)
  • Use page compression for Data Warehouses up to 400GB
  • Use one file per filegroup for best throughput (this prevents multilevel disk striping), and for Data Warehouses under 1GB you should need only one filegroup
  • However, you can look at using multiple file groups to store staging data separately from production, to separate low-latency data from high-latency, to run parallel data loads, and more.

Just be aware that the SQL Server 2014 Azure image will expire at the beginning of August, as that product moves along the path to RTM.


Windows Azure: PaaS changing the landscape of online gaming

Titanfall is a new blockbuster game for the Xbox One.  It is being published by Electronic Arts and is due to be released in March 2014.  Titanfall is a first person shooter that will have much of its AI hosting , physics calculations, online match making and multi-player dedicated servers hosted in Windows Azure.  This means several things:

  1. Azure’s IaaS provides dedicated servers for multi-player games providing near infinite bandwidth with low server pings and anti-cheat enabledWindows Azure: PaaS Changing the Landscape of Online Gaming
  2. Azure’s PaaS is being utilized to provide physics calculations and specialized AI to learn your style of play
  3. PaaS and dedicated servers auto scale to provide fast dynamic content to players around the world on a consistent scale

Multi-player infrastructure background

Traditionally multi-player games have been played using a client/server paradigm.  This paradigm generally involves a computer acting a dedicated server for the game.  This dedicated server accepts connections from a specific amount of players and handles communication between the clients/players.  The server normally does not perform any game relevant calculations but would act as a central repository where players send update information which would then be distributed and consumed by every client.

Recently the game development community has moved away from the dedicated server model due to operational cost and replaced it with a player-host model.  The player-host model essentially means that one player hosts the game and every other player connects to the host.  This new paradigm has several disadvantages to network multi-player gaming but was implemented to save costs on running dedicated servers as game hosts.  A few of the obvious disadvantages to the player-host model are:

  1. Inconsistent bandwidth and server lag of the player chosen to be the host
  2. No anti-cheat enabled on host
  3. Slower updates / increased lag due to server not being dedicated
  4. Local player receives faster updates than other players

How Azure fixes this

The dependence on a cloud infrastructure for a fast paced reactionary game is a significant leap of faith.  Video games generally run in a continuous loop created by the game engine to repeatedly update all of the game data (AI, particles, physics, player movement, event handling etc.) and then draw that data to the screen.  It takes a  substantial amount of CPU and GPU power to calculate and render all of the in-game objects at speeds necessary to achieve the target of 60 frames per second.

The developer of Titanfall, Respawn Entertainment, is utilizing Azure PaaS to handle several expensive calculations normally performed by the local host (console or PC).  These calculations are typically done on the local host so the player experiences minimal lag.  With these calculations off loaded to the cloud and not affecting any game play, it allows the developers to optimize the Xbox One hardware to handle more graphically intense environments.  This strategy could also extend the life of the Xbox One even further in the future.

Cloud computing services such as Azure have allowed dedicated servers to once again be economical.  With automatic server scaling and incredibly cheap virtual machine costs, the server cost and total hours of man power have been significantly reduced.  The more calculations that are performed in the cloud the more you can do with the hardware available.  Another way to look at this is, the more calculations you can do in the cloud significantly impacts the entry point for other hardware platforms.  If a developer is able to process 90% of intense calculations on an Azure compute cluster then the hardware needed to play the game can be anything from a tablet to a workstation.  This has the opportunity to increase the install base substantially.

Games are real time applications that depend on milliseconds and timing.  Azure is effectively performing calculations for a real time application and delivering results to multiple parties simultaneously.  If the Titanfall launch performs well, expect hundreds of future Xbox One games to utilize Windows Azure in making the cloud (and Azure) a dominant force in multi-player gaming for years to come.

Windows Server 2012 R2 Hyper-V – Overview of Generation 2 VM’s

With the release of Windows Server 2012 R2 comes many great new features, including a improved virtual machine named generation 2.

Generation 2 virtual machines provide quite a few enhancements across the spectrum of Hyper-V VM technology. Perhaps most notable is the removal of legacy emulated hardware. Removal of the legacy network adapter, IDE controller, floppy controller, serial controller (COM ports), and PCI bus, results in a more efficient VM. You should see faster boot times, and quicker installations from .iso. How does a VM boot without these integral components? Where necessary, they have been replaced with software based versions.

Other enhancements include:

  • Replaced BIOS with UEFI (Unified Extensible Firmware Interface)
    • Faster boot times
    • Support for boot volumes up to 64TB (Uses GPT instead of MBR)
  • Enhanced Security
    • Smaller attack surface
    • Secure Boot – Prevents unauthorized firmware, drivers and OS from running during boot.
  • Expansion of data and boot disks while VM is running. Nice!
  • Complete reliance on VHDX file format resulting in much better performance (VHD’s are no longer supported).
  • Enhanced Session Mode
    • This allows device redirection and the ability to control display configuration when connected via the Virtual Machine Connection tool.

Some things to keep in mind with generation 2 machines: Read the rest of this post »

Windows Azure: What is Platform as a Service?

What is Platform as a Service (PaaS)?  How does it differ from Infrastructure as a Service (Iaas)?

Let’s start with IaaS.  When “The Cloud” first became popular IaaS was the target.  The point of IaaS is to migrate a company data center into Windows Azure.  This involves converting whatever physical servers you have to Hyper-V virtual machines and upload the contents to Azure(or sending them via FedEx for Microsoft to upload).  While this process is cumbersome and time consuming it does work and has some great benefits.  The next step in this process is to create a secured link between your former data center and the Azure data center.  Doing this will allow your users connectivity to all of their pre-existing applications.  In fact, your users should not even notice that the data center has been moved.  Everything should operate exactly how it did prior to the migration.  Depending on the size of your infrastructure IaaS has the ability to save you time, space, money and will allow IT admins to breathe easy knowing they don’t have to be overly concerned with server hardware failure.  From that perspective, IaaS saves you money because you have zero server maintenance / replacement cost.  Another benefit of IaaS is that all of your existing third party software is guaranteed to work on IaaS.  If a business has an old Citrix application that is critical for the accounting team, then migrating to IaaS will guarantee that application still works as expected.  The problem with IaaS is that you are still running an entire infrastructure.  IT will still have to manage the servers, Active Directory, patches, and updates.  In short, you aren’t benefiting from a lot of the major benefits of the cloud.

PaaS aims to solve that.

PAASPaaS in Azure is synonymous with “Cloud Service”.  The target of PaaS is custom business applications that will replace your existing third party software  infrastructure.  These applications can be written in either .NET or Java.  In PaaS you essentially rent the hardware the application runs on.  Renting the hardware means you do not have administrative access to the virtual machines powering the application.  The permissions are set so that Remote Desktop to the virtual machine is enabled and users can administer IIS.

What is the advantage of renting hardware the application runs on?   Why wouldn’t I simply use IaaS to retain tighter control over the environment?

Azure manages the infrastructure powering the application so IT administrators no longer have to worry about it.  The infrastructure used to power a global application with millions of users generating massive amounts of traffic (like SnapChat)  is massive.  The work and time it takes to manage that infrastructure is several times larger.  With PaaS you are able to eliminate the management of the environment the app runs on.  It is also cheaper to run an application with PaaS than creating dozens upon dozens of virtual machines.  Scaling is much easier with PaaS.  It is possible to scale to a massive size very quickly(and automatically) with PaaS.  To do the same with IaaS, virtual machines must be created in geographically relevant locations.

PaaS provides most of the flexibility that IaaS provides.  By sacrificing a little control administrators and developers are able to automate most of the infrastructure maintenance that comes with growing web applications.  This not only saves money but allows developers to create new features without having to worry if the infrastructure can handle it.  Whether the application is of massive global scale like SnapChat or simply a small application which applies business logic to users; PaaS on Azure maximizes productivity by removing laborious infrastructure maintenance.

Windows Azure: Web Roles vs. Websites

Windows Azure has many different devices for publishing and consuming content.  Two of those which are often confused are web roles and web sites.  The two are very similar and share some common strengths, but given specific conditions one may fit your need better than the other.  For example, web roles and web sites will both support auto scaling, database support, blob storage access,  ASP.NET, NodeJS, Python and PHP.   Here is a brief overview of specific uses for each.

Web sites:

Web sites on Windows Azure operate very similar to those run by other web hosting companies but with added Azure benefits.  You can access the site you created via FTP or Git which means changes you make to the code are updated instantaneously upon submission.  If you need to use any of the templates that are provided  by Windows Azure then web sites are the way to go.  Azure has a slew of templates that you can install, configure and update from the gallery.  Everything that you are familiar with in traditional web sites applies to Windows Azure web sites.

Web roles are where the power of the Windows Azure platform begin to shine.  Here are some of my favorite advantages of the web role cloud service.

cloudWeb role:

A Web role is a cloud service.  Cloud services are really where Platform as a Service ideology really begins to work for you.  Web roles allow developers much more control over the environment.  They are created within virtual machines and allow developers direct access to IIS (or web server of their choosing).  Because web roles run in virtual machines they also give you the flexibility that virtual machines give you, without having to manage the virtual environment.  What does that mean?  With a virtual environment you have to stay up to date on patches for the operating system and web server along with any other maintenance the machine will need.  With a web role that is not necessary.  While you still have remote access the to virtual machine your web role is running on, you do not need to manage it.  The virtual environments and patches are managed by Azure.

Web roles can also be attached to one or many worker roles.  Think of a worker role as a console application that will run a computationally heavy process that may take a considerable amount of time.  These types of processes are not for immediate user consumption but to be calculated and stored for future access.  For example, generating recommendations for users in a way similar to Netflix or Amazon is computationally expensive.  To do that within a GUI front end like a web site or web role is not practical.  Unlike a web site, a web role can call a worker role to run a background process.  This worker role can run on a different virtual machine so as not to ruin the performance of your web role.  In addition you can set up Windows Azure Virtual Network so the web and worker roles are on the same subnet.  This allows the two to talk directly to one another on the inside of firewalls instead of having to venture out of the firewall and routed through the internet.

There are many more advantages to using Web Roles over Websites but the last few I want to touch of briefly are multiple staging environments, Content Delivery Network (CDN) connectivity, support for unsupported platforms, and running scripts with elevated privileges.  Websites do not support CDN’s and can’t run scripts that require administrative permissions.  So if a CDN is something your website requires or if you are migrating to Azure and have a few legacy cgi scripts that require admin rights websites will not work for you.

From a development standpoint multiple staging environments is a big win for web roles.  When a cloud service is set up, a staging and production environment are created for you.  There is no longer the fear of having working code in your test environment and crashing your production environment.  The environments are identical in Azure.  If it works in staging it will work in production.

Web Roles and Websites also scale out differently.  Websites only need to worry about scaling up your site in order to keep up with demand.  In a Web role the scaling is similar but what if your web application needs two worker roles for each web role to handle all of the computation.  Scaling with Web roles can facilitate this type of configuration.

I have just pointed out a few of the practical differences between websites and web roles.  Websites are perfectly suitable for publishing content fast, accessing databases and processing records.  They are great for content management systems and you can even use them for e-commerce .  If the cloud service you build will need multiple virtual machines in addition to a virtual network where information will be passed inside the firewall or any significant back end, then a web role / worker role setup is most likely for you.  It provides you the flexibility you need without having to manage the infrastructure you are creating.

Windows Azure for ITPros Webinar Followup


I gave a presentation about Windows Azure the other day, and got some great questions that I would like to address with this blog post.  Here they are in no particular order, with some answers I hope are useful to you:

 “We have multiple Operating Companies who may want to leverage the features of Windows Azure. Is it recommended to setup a separate Windows Azure tenant for each Operating Company or Unit?

This is an interesting operations question.  I have worked with customers with separate independent organizational structures.  Some of these firms converged, others diverged while some stayed put as is.  Each situation has its own set of operational challenges.  I am a big fan of simplicity, so I am naturally inclined to say use a single subscription if possible.  There are some efficiencies you gain by having a single subscription – billing simplicity, single point of administration, technology solution simplicity, intra versus inter network performance and so on.  From these broader advantages, you might start to recognize indirect benefits – sharing customized gold VM images across organizational departments/divisions, using the cloud as a model of security identity consolidation especially if this is something that is likely to happen with OnPremise Directory deployments later on, connecting resources and merging data together from these operating units est.

However there might be legal/regulatory/policy reasons for keeping individual subscriptions for each operating unit of the organization.  For example, you might have two operating units in different countries, each with data and assets that should be kept physically separate as much as possible, from a legal and regulatory perspective.  Check with the Legal/Policy department.  Another reason is billing.  If invoices are not handled by a single entity within the organization, it might be necessary to have separate subscriptions, so that you can bill each organization appropriately.  With single and multiple subscriptions, I think you should have at least one person that has administrative access to all subscriptions, and has an organization wide view of how Windows Azure is being utilized. 

 “What about HIPAA compliance?

Specific Windows Azure features [Virtual Machines, Cloud Services, Storage – Tables, Blobs, Queues, and Drives – and Networking] are covered by HIPAA BAA [Business Associate], an offering Microsoft provides to organizations that have Enterprise Agreements with them.  Not all Windows Azure features are covered [for example, Media services] but that might change in the future as Microsoft works to expand coverage to its growing portfolio of Windows Azure services.  If you are interested in Windows Azure and you belong to an organization that deals with PHI data, contact your Microsoft Account Manager to make sure that Windows Azure covers your specific needs.

Windows Azure meets other data protection and privacy laws: ISO/IEC 27001:2005 Audit and Certification, SOC 1 and SOC 2 SSAE 16/ISAE 3402 Attestation, Cloud Security Alliance Cloud Controls Matrix and Federal Risk and Authorization Management Program (FedRAMP).  For more information please review Microsoft’s Windows Azure Trust Center Compliance Page

 “Does it mean multiple customer shares one VM for the free and shared model?

Let’s start with some background to this question: the Windows Azure Web Sites feature is PaaS [Platform-as-a-Service] offering from Microsoft that currently comes in three flavors: Free [host up to ten sites], Shared and Standard.  Both Free and Shared modes share the same architecture, and this architecture does host multiple websites instances for various subscribers/Windows Azure customers using a Shared VM approach.  To get dedicated VMs for your applications, you would have to deploy your web site to the Windows Azure Web Sites Standard model.  Each model plays really well to different scenarios.  For example, it might make sense for your organization to use the free mode for your development environment, the Shared mode for QA and the dedicated mode for Production.

 “Are the Server Platforms supported in Private Cloud Hosting?

Again, some perspective with regards to this question: As of November 2013 Windows Azure Virtual Machines officially supports the following minimum platform versions – 64-bit versions of SQL Server 2008, SharePoint Server 2010, Team Foundation Server 2012, Project Server 2013, System Center 2012 SP1, HPC Pack 2012, BizTalk Server 2013, Dynamics GP 2013, Dynamics NAV 2013 and Forefront Identity Manager 2010 R2 SP1,.  That is not to say that you cannot install earlier versions of these platforms on Windows Azure VMs.  However, even though such workloads install successfully, they will not be supported by Microsoft.  Which might be okay if you need to spin up a Development environment, and don’t really require support from Microsoft.

This leads up to the original question, which is more about private clouds, and not public offerings like Windows Azure.  Microsoft uses their own virtualization platform to run Windows Azure (Hyper-V).  As such, if you are running a Microsoft Hyper-V Virtualization Platform Private Cloud solution, the platforms listed above are supported as well, at a minimum.  In fact, at the moment, OnPremise Private Cloud Hyper-V deployments supports even more server platforms than Windows Azure currently does.  If you are using VMware or open source products instead, you will need to check with your vendor to ensure that your workload will be supported if it is virtualized on their platform.

For more information, take a look at the following:  Hyper-V Supported Virtual Machines and Guest Operating Systems, Microsoft Server Software and Supported Virtualization Environments and Microsoft Server Software Support for Windows Azure Virtual Machines

How to: Setting up an HDInsight Hadoop cluster in Windows Azure

Edit: Part 3 using Mahout here

In my previous post I described the basics of HDInsight on Windows Azure and an example of what a Hadoop cluster can do for you.

Without further delay, lets build a cluster!  If you don’t already have a Windows Azure account go here and sign up (it’s free!!)


Login to your Azure portal and you will have a dashboard similar to this:


If HDInsight is not initially on the dashboard simply add it by going here and selecting “Try it now” under “Windows Azure HDInsight Preview”.  After installing HDInsight should now appear on your dashboard.

To create a cluster select HDInsight; upon clicking “Create an HDInsight Cluster” you will be presented with the following screen. Read the rest of this post »